Continuing on the subject of threading, a coworker asked me an interesting question the other day. She has an application that's basically a packet-logger. She connects up to some service, sends it a request, and if she gets anything back, she logs it to an XML file. To communicate with this service takes a while, so she has two threads going at once, each one doing both polling and logging. The constraint here is that the data has to be logged in real time. (Why real time? I don't know, maybe in case of a robot attack on the data center. You think a robot will give you an extra 5 ms to write to your log file?!) Because I like to act like I know what I'm talking about, she asked me for help.
If you have two threads trying to access the same physical resource at random times, something bad is going to happen. Sooner or later, both threads will try to write at the same time and something will blow up; all you can do is hope that this blown-up something is neither you nor your pants. How do you fix this brewing catastrophe?? You fix it with locks. By locking your resource, you give one thread exclusive access, and then you alert the other thread to go ahead (in .NET, you communicate with other threads using Monitor.Pulse and Monitor.WaitPulse).
The problem here is that she said it HAS to be real-time. If you're locking something, you're not going to be logging in real-time. If a thread has the file locked that the other thread wishes to access, there must be a wait. It may not be much of a wait, but depending on how long you're talking to that service and writing to the disk, it could be a while. So how would you get around it?
One idea I had was to divorce the logging part from the service communication part. What if you still had your two threads communicating with this slow, crappy service, and whenever they had something to log, they inserted it into a shared, thread-safe queue? Let's go further and say this is a smart queue where we've added some wizardry so that anytime an element is enqueued, we write it to the disk.
The problem is that even with the queue, if you want multiple threads to access it safely, you have to lock it. The good part is that you're locking an in-memory object, not a physical resource. I would assume this is faster, although I have no numbers to back that up. In the spirit of bloggers everywhere, I will just make up a number: this solution is 27.8% faster than locking the XML file.
It's not real-time, but it's thread-safe. You can have fast and dangerous, or you can have slightly less fast and much safer. I realize most programmers have to be clubbed over the head to renounce the first, but this might be a good situation to do so, simply because threads get weird.
The Austin .NET Users Group is hosting a Code Camp on March 4th and I'm scheduled to be there. No, not in a janitorial capacity, but as a speaker. I'll be talking about writing maintainable multi-threaded code from 10 AM - 11 AM, so if that sounds good and you live in Austin OR are a crazy billionaire who wants to fly in to hear me stammer through this thing, you are encouraged to attend.
For the presentation, I had to come up with an outline. Here's what I have so far. Let me know if I'm missing anything.
Introduction
- What is a thread?
- What does it mean to make use of multiple threads in the .NET environment?
- Advantages of threading
- UI is more responsive
- Long-running algorithms can be split into multiple threads, enhancing execution speed
- Disadvantages of threading
- It's hard to write, hard to maintain
- It's scary
- The disadvantages can be minimized through training and experience.
Multi-threading in Action
- Hello World Example
Introduction to Problem
- Implement a Windows Form that validates large prime numbers
- Display implementation in single-threaded mode: UI is unresponsive, algorithm takes a long time to complete
- The solution is to implement multi-threading
Threading 101
- What are the methods of implementing multi-threading in .NET?
- ThreadStart delegates
- ThreadPool class, which can be used directly or indirectly (call BeginInvoke on any delegate)
- How do you know which method to use?
- ThreadPool is simpler, but it's limited in resources
Concurrency Issues
- What are the scariest bugs that can arise from multi-threading? concurrency issues.
- Multiple threads accessing and manipulating the same data at roughly the same time.
- Show example.
- You can't assume that two operations can occur on a thread without another thread getting in the way with potentially disastrous results.
- To prevent this bugs, we need a way of locking our data on a per-thread basis.
- Multiple threads can read, but only one can write.
- How to implement?
Locking
- How do you implement locking in .NET?
- One way is with the Monitor object.
- Calling Monitor.Enter on an object or a variable will lock that object to any other threads.
- Unlock through Monitor.Exit
- Show example
- While Monitor is easy, there are some significant drawbacks
- What happens if Monitor.Enter and Monitor.Exit aren't always balanced?
- What happens if an exception surfaces and Monitor.Exit isn't in a finally block?
- Framework addresses all of these with the lock statement.
- If a variable is wrapped in a lock block, Monitor.Enter and Monitor.Exit called automatically.
- Show example.
What else can possibly go wrong?
- If we're not using Monitor.Enter and Exit, we're not performing volatile reads and writing, meaning it's possible that we're viewing or changing an old value in memory.
- May be too complicated for presentation.
- In our prime validator, one thread locks the list of primes while another locks the list of potentials.
- Infinite loop entered, all hope is lost.
- This is a deadlock, and it occurs when, in a group of threads, each one holds a monitor that the other wants.
Deadlocks
- Hard to diagnose
- Hard to debug
- Hard to fix
- How do you solve this issue?
- You solve it by eliminating all possibilities of deadlock.
- Always remove locks in the same order.
- If you must lock multiple items, don't lock the second until you already have the first.
- Deadlocks are harder with calls between objects.
- You don't know what the other class is locking, so how can you know what to unlock?
- Best solution is, within a lock, make very few calls outside of your class.
How Else Can We Prevent Deadlocks?
- One way to mitigate deadlocks is to communicate between threads
- Monitor has some methods for this, like Monitor.Wait and Monitor.Pulse
- Monitor.Wait releases the lock on an object and blocks the current thread until it reacquires the lock
- Monitor.Pulse notifies a thread in the waiting queue of a change in the locked object's state
- Show example of deadlock prevention using Pulse and Wait
- Some classic Win32 ways to do the same thing are exposed by the WaitHandle object and its subclasses (Mutex, Manual and AutoResetEvent)
- Mutex is more complicated because it contains a lot of info, like a count on times it's been acquired and the thread that currently owns it. It can be used across many processes, and is useful in determining if the application is running more than once.
Thread Affinity and Windows Forms
- Certain types can only be used in the threads on which the type was created. The best example is a control.
- What does this mean? You can't change or access anything in the user-interface from a different thread.
- The safe way to do this is through BeginInvoke, EndInvoke, Invoke, CreateGraphics, or InvokeRequired.
- By calling these, you can ask the UI thread to call a method for you.
- Invoke is synchronous, BeginInvoke is asynchronous. Both are used to invoke a method.
- You must specify a delegate here. To get a return value asynchronously, you need EndInvoke.
- Show example.
Unit Testing Threads
- How to test for locking? Show example.
- How to test for deadlock prevention? Show example.
Conclusions
- Multi-threading can be complicated, but there are a few rules of thumb.
- Lock and unlock any resources you access in a thread
- If multiple threads vie for the same resource, use Monitor.Wait and Monitor.Pulse to your advantage
- If you're accessing a UI element from another thread, stick to one of the Invoke methods.
- If all else fails, nod confidently and declare, "It's supposed to do that."
I realized something unpleasant last week: it's highly unlikely that David Hasselhoff will ever get another prime-time show. Shortly afterwards, I realized something else that was even more unpleasant: the design reviews we were doing at work just weren't helping.
Most people would agree that a design review is helpful. Not only does it allow the designer to bounce ideas off of a group and perhaps find some improvements in his original concepts, but it allows the group to learn something about a part of the program with which they may not be familiar. Both of those are good things. However, when you try to implement design reviews in real life, you may find that an actual review is far less helpful than one originally conceived. What's going on here?
Usually, a design review is done after you've designed something. This is bad for two reasons. First, how is the review supposed to be collaborative if the design is already done? The best time to collaborate on something is at the beginning when changes are cheap, not at the end when changes become costly.
Second, if you're anything like me (both impatient and brutally handsome), you code as you design. I can think about the design as much as I want, but the best way for me to see its limitations is to interact with it. By the time the design review rolls around, it's highly likely that not only is the design done, but the code is done also. Thus, any changes will effect the design and the code, which is probably already working. Why change something that already works? How many times can you change something for the sake of design purity?
I speak from experience here; our team originally set up our design reviews just as I previously described, and they weren't working as well as we wanted. As any engineer can tell you, a feedback loop is useless if the loop is too long. Taking that to heart, we shifted the feedback from the end of the design process to the beginning. Now, rather than having a lone developer slaving away in her office on a design (which was never mandated, but how it turned out often), whenever one of us is designing something, we have to get together with at least one other dev to discuss our design ideas. This other person isn't a road-block, just a check to make sure that we're not missing anything big. There has to be a consensus before you can proceed. This allows us to find the big mistakes at the beginning, not at the end.
Like any other sane individual, I'm a big believer that a process should help, not hurt. Before, we had a clear case of a bad process, one that kept us from correcting mistakes unless the mistakes were mind-boggingly large. Well, maybe the mistake wasn't huge right then, but after a few years of maintenance, it's easy to see how a minor design oddity could become the size of a brontosaurus, smashing cubicles and builds left and right. The situation is still not perfect, but it's a little better. That's enough for me.
I have plenty of stuff written up for the site, but then whenever I get home, I find myself distracted by something like Bear Baiting with the Stars. Anyway, let this serve as a note to self: if, in the future, you add a control programmatically to the form and it's not showing up even though it's Visible == true, it's probably because you didn't add it to the form's Controls collection. I do this once a week, like a regular lord of the dumb asses. The cycle stops now!