Principles of unit testing

I was paging through the current version of Emily Bache’s new LeanPub book “The Coding |Dojo Handbook” when I came across her chapter on Principles for Agile Automated Test Design. I read it with interest, because last Saturday I gave my first airing of a new presentation at DunDDD, “Bad test, good test”. In it I laid out the 6 principles of unit test design that I use, so I wanted to compare these to the 4 principles that Emily uses. Of course Emily’s principles are specifically tailored for agile automated tests, while mine are intended to be of general use for all writers of unit tests, but I hope that there is a general foundation of good-ness to be described.


Emily’s principles are:

  • Coverage – some
    test should fail if there’s a bug in the feature under test

  • Readability –
    the test should be easy to read and understand

  • Robustness – a
    failing test should always indicate a bug, and a bug should only
    cause a single test to fail

  • Speed – tests
    should run fast

I think there is a good overlap between Emily’s principles and the principles that I propose good unit tests should exhibit:

  • Understandable –
    the test should be easy to read and understand

  • Maintainable –
    the test suite should be easy to modify when the requirements change

  • Repeatable – the
    test should either always pass or always fail

  • Necessary –
    every test should test something that needs tested and isn’t tested

  • Granular – each
    test should validate a single piece of behaviour

  • Fast – tests
    should run fast

I think I can claim that at some level:

  • Emily.Readability
    == Seb.Understandable

  • Emily.Fast ==

  • Emily.Robustness Seb.Repeatable + Seb.Necessary

I don’t have any principle the aligns with Coverage. Perhaps this is an omission, but I wonder if this is a principle too far, since this is a property of the test suite rather than of a test.

Conversely, I don’t think that Emily has direct equivalents of Maintainable or Granular, though they could be seen as factors that contribute to Readability and Robustness.

Maintainability is a property of well factored code that adheres to pragmatic coding guidelines. Methods should be small and well named, code should be free of duplication, common functionality should be extracted into well named helper methods and so on. When requirements or implementations change it should be simple to fix up the test code. The last thing we need is having test code act as a drag on the evolution of the codebase.

Granularity derives directly from the Single Responsibility Principle and requires each test to validate a single behaviour. This doesn’t limit tests to a single assertion, but all assertions should be related to the behaviour that the test is intended to validate. This helps keep tests understandable and should ensure that you can easily tell broadly what the problem is simply by reading the name of the test.

Emily goes onto quote Roy Osherove’s 3 pillars of unit test, from his book “The Art of Unit Testing”, which are slightly different again.

Maybe we’re just demonstrating the computer programmers way of counting:

  • None –
    unprincipled hackery

  • One – test your

  • Many – pick a
    set of guidelines that works for you

Do you have any principles or heuristics for determining the good-ness of a unit test that you’d like to share? I, for one, would like to hear from you.






2 responses to “Principles of unit testing”

  1. Emily Bache Avatar

    Glad to hear you’re also thinking about good test design, I think we’re broadly in agreement! I do think you’ve misinterpreted my principles a little, I think they map like this:Emily.Readability == Seb.UnderstandableEmily.Fast == Seb.Fastso far we agree – and actually I think your principles describe agile tests just like mine do. If you’re not agile then you care much less about speed, since you run your tests much less often. You also care much less about maintainable since the requirements don’t change often. You also care less about repeatable because you’re willing to examine each test run and re-run failures manually, since the default was manual testing anyway… test automation in non-agile projects is pretty odd, I think.Emily.Coverage == Seb.NecessaryI guess I need to describe that better since you clearly misunderstood it. I want something to fail if there is a bug, and not have superfluous tests that never fail even if there is one.Emily.Robustness == Seb.Repeatable + Seb.Maintainable (ish)I think your description of Emily.Robustness is far too narrow, I define it much wider, to include isolation, lack of flickering and duplication, ability to handle (irrelevant) changes in the SUT… Seb.Maintainable == Emily.Readable + Emily.Robust + …Maintainable is important, and I think it’s nearly the same as readable and robust. If it’s not readable, you can’t easily change it, if it’s not robust, you’ll get fed up with maintaining it. I’m thinking it’s an emergent property, not something you can optimize for directly.I’m not sure you always need Seb.Granularity. It’s a good feature for unit tests, but isn’t always in larger scale automated system tests. My principles are as much about them as unit tests.Perhaps Roy will join the discussion and tell us how I’ve misinterpreted him too 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *