Principles of unit testing

I was paging through the current version of Emily Bache’s new LeanPub book “The Coding |Dojo Handbook” when I came across her chapter on Principles for Agile Automated Test Design. I read it with interest, because last Saturday I gave my first airing of a new presentation at DunDDD, “Bad test, good test”. In it I laid out the 6 principles of unit test design that I use, so I wanted to compare these to the 4 principles that Emily uses. Of course Emily’s principles are specifically tailored for agile automated tests, while mine are intended to be of general use for all writers of unit tests, but I hope that there is a general foundation of good-ness to be described.

Emily’s principles are:

• Coverage – some
  test should fail if there’s a bug in the feature under test

• Readability –
  the test should be easy to read and understand

• Robustness – a
  failing test should always indicate a bug, and a bug should only
  cause a single test to fail

• Speed – tests
  should run fast

I think there is a good overlap between Emily’s principles and the principles that I propose good unit tests should exhibit:

• Understandable –
  the test should be easy to read and understand

• Maintainable –
  the test suite should be easy to modify when the requirements change

• Repeatable – the
  test should either always pass or always fail

• Necessary –
  every test should test something that needs tested and isn’t tested
  elsewhere

• Granular – each
  test should validate a single piece of behaviour

• Fast – tests
  should run fast

I think I can claim that at some level:

• Emily.Readability
  == Seb.Understandable

• Emily.Fast ==
  Seb.Fast

• Emily.Robustness ≈ Seb.Repeatable + Seb.Necessary

I don’t have any principle the aligns with Coverage. Perhaps this is an omission, but I wonder if this is a principle too far, since this is a property of the test suite rather than of a test.

Conversely, I don’t think that Emily has direct equivalents of Maintainable or Granular, though they could be seen as factors that contribute to Readability and Robustness.

Maintainability is a property of well factored code that adheres to pragmatic coding guidelines. Methods should be small and well named, code should be free of duplication, common functionality should be extracted into well named helper methods and so on. When requirements or implementations change it should be simple to fix up the test code. The last thing we need is having test code act as a drag on the evolution of the codebase.

Granularity derives directly from the Single Responsibility Principle and requires each test to validate a single behaviour. This doesn’t limit tests to a single assertion, but all assertions should be related to the behaviour that the test is intended to validate. This helps keep tests understandable and should ensure that you can easily tell broadly what the problem is simply by reading the name of the test.

Emily goes onto quote Roy Osherove’s 3 pillars of unit test, from his book “The Art of Unit Testing”, which are slightly different again.

Maybe we’re just demonstrating the computer programmers way of counting:

• None –
  unprincipled hackery

• One – test your
  code

• Many – pick a
  set of guidelines that works for you

Do you have any principles or heuristics for determining the good-ness of a unit test that you’d like to share? I, for one, would like to hear from you.