Unit testing is dead? Hardly.

Despite what His Majesty, David Heinemeier Hansson may have said, unit testing is by no means dead. And, in fact, system testing is no more a complete testing strategy than 100% test coverage with unit tests. Let me explain.

Test Driven Development (TDD) is a philosophy that asserts testing is so important that the tests should be written first, to emphasize the design of the code. The idea is that by writing a failing test, and then writing code that passes that test, you end up with an overall better architecture.

Of course, TDD has some flaws: namely it’s a “write to the tests” design strategy that, while potentially encouraging better design, can potentially harm your overall design by removing “big picture thinking.”

The problem is, DHH has basically said TDD is worthless, and system testing, not unit testing, is the way to go. And yet, this philosophy has just as many blind spots as TDD does.

Socorro used an “integration test” methodology to accomplish the majority of it’s testing. And while this worked well to avoid regressions in many cases, it also had many holes. One such hole that frustrated me was the complete lack of tests for the stored procedures in the database: I would write a stored procedure, and a middleware to access the results, but because the stored procedure created data on a cron cycle, there was no way to validate it automatically. You can guess which parts of my code often broke on staging.

Of course, system testing has a place alongside unit testing. System testing ensures a high-level assessment of the overall application, and ensures that we are getting the right answer at a high level. It makes sure that all our components come together and do their jobs. Unit tests won’t catch failed or missing database migrations, or Javascript bugs.

Unit testing, on the other hand, provides an opportunity for us to get nitty-gritty and to test specific avenues of our code for accuracy and effectiveness. Unlike a system test, a well-written failed unit test points right to the source of the problem, making it easier to correct. A system test will tell you that the system is broken, but you have to hunt for the cause.

By all means, we should be writing more, not fewer, tests. And we should be testing whole systems as well as specific units of code. Testing is not an either-or situation whereby we can only do one or the other.

more