Diamond recycling (and painting yourself into a corner)

The post I wrote recently on recycling tests in TDD got quite a few responses. I’m going to take this opportunity to respond to some of the points that got raised.

Do we really need to use the term “recycling”?

The TDD cycle as popularly taught includes the instruction to “write a failing test”. The point of my article was to observe that there are two ways to do that:

  1. write a new test that fails
  2. change an existing, passing test to make it fail

It’s this second approach that I’m calling “recycling”. Alistair Cockburn says that “it’s a mystery this should need a name” and it probably doesn’t. However, I’ve regularly seen novice TDD-ers get into a mess when making the current test pass causes other test(s) to fail. Their safety net is compromised and they have a few options, none of which seem very appealing:

  1. Roll back to last green
  2. Comment out the failing test(s)
  3. Modify the failing test(s) to make them pass again

Whichever way you go to get out, you’ll want to try to avoid painting yourself into a similar corner in future.

Why do tests that used to pass start failing?

Ron Jeffries suggests that this will only happen if the tests don’t “say something universally true about the problem and solution.” Several people (including George Dinwiddie and Sandro Mancuso) demonstrated that this problem can be solved by writing a series of tests that each say something “universally true.” However, to me, this seems like a similar approach to that recommended by Alistair Cockburn in his “Thinking Before Programming” post.

I’m a big fan of thinking before programming. In the courses that I deliver, I routinely prevent students from touching the keyboard until they’ve thought their way around the problem. But, it’s just not realistic to expect that any developer will be able to solve every problem in this way. And one of the great strengths of the TDD approach is that it encourages us to incrementally build a safety net of tests that anchor the solution.

This is where test recycling comes in. By acknowledging that the test we’ve just written doesn’t identify a universal truth, we can iterate on our understanding. By rewriting the test we are admitting that we are changing the SPECIFICATION. This lets us safely evolve the test from a low-fidelity, incorrect specification to a high-fidelity, correct specification while minimising the risk that we will cause other tests to fail in the process.

Would more, smaller tests provide better documentation?

One of the major benefits of a good set of unit tests is that it documents how the code is supposed to behave. A few people pointed out that, by recycling tests in the way that I demonstrated in my previous post, I was continuously rewriting the same test. I was replacing existing documentation with updated documentation, but the amount of documentation (the number of tests) remained unchanged. Won’t this lead to less readable test suites?

My experience has shown me that the number of tests in a test suite does not correlate with the quality of the test suite. I don’t want to pick a fight with my learned colleagues, but I think that a test suite that describes how the code behaves using examples is often easier to understand than one that asserts mathematical truths. The test suites written by Alistair, Ron, George, Sandro, Jon and others all exercise the functionality of the code and assert universal truths about the solution. But take a moment to look at the problem instructions on Cyber-Dojo:

Print Diamond instructions

The problem is described using 2 examples.  What would you think if the tests looked like this (ignoring error handling)?

Exmplar tests

I find that test suites that document the behaviour of the code are easier to understand. They’re also less likely to be coupled to the implementation, which will be beneficial when we decide to move from ‘trays and bumpers’ to ‘squashed circles’ or something even more exciting!

There’s always room for another tool

And, like I said in my first post, this is just one tool (out of many) in the developer’s toolbox. Give it a try when you’re finding it difficult to keep your tests passing, and see if it helps you out of that corner.


Posted

in

, , , ,

by

Comments

5 responses to “Diamond recycling (and painting yourself into a corner)”

  1. George Dinwiddie Avatar

    Seb, I think “recycling tests” is an excellent technique for getting yourself out of the corner when you’ve painted yourself there. Thinking ahead is a time-honored technique. I demonstrated the way that I currently find most helpful and convenient. None of these techniques are “the best way” because we start with different preconceptions about the problem and different skills for approaching it in different ways. Learning is always valuable, and learning different ways to work will be helpful for any software developer.

    The three tests that you present at the end of this post represent the result that our customer wants to see. As that, I tend to call them Acceptance Tests or Story Tests. I often start with such tests, but do not expect them to pass in one go. Instead, I break the behavior down into smaller Unit Tests, as I demonstrated in my post, and build the functionality baby-step by baby-step until the Acceptance Tests pass.

    I’m sure you’re aware of this process, but I wanted to call it out explicitly for your (and my) readers.

    – George

    1. Seb Rose Avatar

      Thanks, George. Test classification is a diverting pastime, but notoriously difficult. Acceptance tests, component tests, integration tests, unit tests: they’re all tests.

      For me, the important question is “what are they useful for?” Some are transient (and help me arrive at a solution) and others are more permanent (and remain to document the behaviour and protect from regressions). As you point out, “recycling” is just one technique, that is sometimes useful. I often refer people to Kent Beck’s quote from a StackOverflow answer:

      “I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence… I suspect this level of confidence is high compared to industry standards”

  2. […] Seb Rose: Diamond recycling (and painting yourself into a corner) […]

  3. […] nächste Kata ist hier wie folgt […]

  4. James Murphy Avatar
    James Murphy

    Having done the workshop at Lean Agile Scotland in 2017 I love the technique but struggle (a lot) with the name – I wasn’t the only one.

    Broadly I understand that we’re taking an existing test and, I suppose, reusing it hence recycle but to my mind there’s a strong implication that we’re not really changing the test – if we change it then its not the same test and the recycle bit seem a bit odd (we’re recycling the [Fact] attribute…).

    Interestingly Sal Freudenberg in her talk on 6 thinking modes posited a 7th, based on conversations at the event, of “Whittling” – the idea that we don’t know who to solve the whole of a problem but we know how to solve bits around the edges and so start shaping the solution – this seems to map quite nicely to the process described. We can’t shape the whole result but we know this end has to be pointy so we’ll make it a bit pointy and then from there we’ll whittle away (in this case make the shape of the test more complex) at another bit. So “whittling at tests” seems an nice possibility.

    As a separate aside having “theory” tests (in xunit parlance, though we were doing nunit) seemed affect the shape of the tests and the direction we progressed – I think we’d have got (were getting) further before we hit the “solve everything” step.

Leave a Reply to James Murphy Cancel reply

Your email address will not be published. Required fields are marked *