Unit testing multithreading code: Just Don’t Do It

Seriously, don’t try to write unit tests for multithreaded code. It will bite you in the ass later.

The reason unit testing code that runs on multiple threads is hard is the fact that you can’t control the order of execution of threads, nor the allocated time per thread – this is an OS decision.

Thus, in order to make sure the code submitted on the other thread executes and populates the expected data, you’ll need to block the main thread, where unit tests usually runs, for an amount of time large enough to be sure the other thread finishes the work.

Now, the tricky part is to find what that amount of time should be. Make it too short and you’ll see random failures of your unit tests, make it too long and you’ll increase the duration of your unit tests more and more. And theoretically there’s no upper limit for how long will need to wait for the other thread to finish, as this is out of our control (remember, the OS decides which thread to pick up next and how much time to allocate to it).

And worse, when a unit test like this starts failing on a CI machine but it doesn’t fail on your machine, who’s to blame: the CI machine for being too slow, or your code misbehaving in some conditions that happen only on the CI machine? This ambiguity can lead to lot of time wasted in trying to figure out what the hack happens with the tested code.

Conclusion: don’t try to write unit tests for code that executes parts of its work on a different thread. The reason is simple: robust unit tests need to have control over all inputs of the tested code, and the second thread is not something it can control (unless you mock the thread dispatching, but that’s another story).

Instead, push as much logic as you can into single-thread methods, and tests those methods instead. In the end, most of the bugs occur due to incorrect business logic.

3 comments on Unit testing multithreading code: Just Don’t Do It

  1. What can we do to make sure our multi-threaded code works fine ? How can some one tests it’s multi-threaded code to make sure 80%-90% that will work in production ?

    1. @Engineer, no one is stopping you from unit testing multithread code, however beware of the pitfalls presented in the article. My recommendation would be to push as much logic as you can in non-multithreaded functions/methods, and test those. This will result in only a small amount of multithreaded code, which is easier to review for flaws.

Leave a Reply

Your email address will not be published. Required fields are marked *