The post I wrote recently on recycling tests in TDD got quite a few responses. I’m going to take this opportunity to respond to some of the points that got raised.
Do we really need to use the term “recycling”?
The TDD cycle as popularly taught includes the instruction to “write a failing test”. The point of my article was to observe that there are two ways to do that:
- write a new test that fails
- change an existing, passing test to make it fail
It’s this second approach that I’m calling “recycling”. Alistair Cockburn says that “it’s a mystery this should need a name” and it probably doesn’t. However, I’ve regularly seen novice TDD-ers get into a mess when making the current test pass causes other test(s) to fail. Their safety net is compromised and they have a few options, none of which seem very appealing:
- Roll back to last green
- Comment out the failing test(s)
- Modify the failing test(s) to make them pass again
Whichever way you go to get out, you’ll want to try to avoid painting yourself into a similar corner in future.
Why do tests that used to pass start failing?
Ron Jeffries suggests that this will only happen if the tests don’t “say something universally true about the problem and solution.” Several people (including George Dinwiddie and Sandro Mancuso) demonstrated that this problem can be solved by writing a series of tests that each say something “universally true.” However, to me, this seems like a similar approach to that recommended by Alistair Cockburn in his “Thinking Before Programming” post.
I’m a big fan of thinking before programming. In the courses that I deliver, I routinely prevent students from touching the keyboard until they’ve thought their way around the problem. But, it’s just not realistic to expect that any developer will be able to solve every problem in this way. And one of the great strengths of the TDD approach is that it encourages us to incrementally build a safety net of tests that anchor the solution.
This is where test recycling comes in. By acknowledging that the test we’ve just written doesn’t identify a universal truth, we can iterate on our understanding. By rewriting the test we are admitting that we are changing the SPECIFICATION. This lets us safely evolve the test from a low-fidelity, incorrect specification to a high-fidelity, correct specification while minimising the risk that we will cause other tests to fail in the process.
Would more, smaller tests provide better documentation?
One of the major benefits of a good set of unit tests is that it documents how the code is supposed to behave. A few people pointed out that, by recycling tests in the way that I demonstrated in my previous post, I was continuously rewriting the same test. I was replacing existing documentation with updated documentation, but the amount of documentation (the number of tests) remained unchanged. Won’t this lead to less readable test suites?
My experience has shown me that the number of tests in a test suite does not correlate with the quality of the test suite. I don’t want to pick a fight with my learned colleagues, but I think that a test suite that describes how the code behaves using examples is often easier to understand than one that asserts mathematical truths. The test suites written by Alistair, Ron, George, Sandro, Jon and others all exercise the functionality of the code and assert universal truths about the solution. But take a moment to look at the problem instructions on Cyber-Dojo:
The problem is described using 2 examples. What would you think if the tests looked like this (ignoring error handling)?
I find that test suites that document the behaviour of the code are easier to understand. They’re also less likely to be coupled to the implementation, which will be beneficial when we decide to move from ‘trays and bumpers’ to ‘squashed circles’ or something even more exciting!
There’s always room for another tool
And, like I said in my first post, this is just one tool (out of many) in the developer’s toolbox. Give it a try when you’re finding it difficult to keep your tests passing, and see if it helps you out of that corner.
Leave a Reply