unit testing

/Tag:unit testing

Unit tests are your specification

Recently a Schalk Cronjé forwarded me a tweet from Joshua Lewis about some unit tests he’d written.

. @ysb33r I’m interested in your opinion on how expressive these tests are as documentation https://t.co/YpB1A3snUV /@DeveloperUG

— Joshua Lewis (@joshilewis) December 3, 2015

I took a quick look and thought I may as well turn my comments into a blog post. You can see the full code on github.
Comment 1 – what a lot of member variables
Why would we use member variables in a test fixture? The fixture is recreated before each test, so it’s not to communicate between the tests (thankfully).

In this case it’s because there’s a lot of code in the setup() method (see comment 2) that initialises them, so that they can be used by the actual tests.

At least it’s well laid out, with comments and everything. If you like comments – and guess what – I don’t. And they are wrapped in a #region so we don’t even have to look at them, if our IDE understands it properly.
Comment 2 – what a big setup() you have
I admit it, I don’t like setup()s – they move important information out of the test, damaging locality of reference, and forcing me to either remember what setup happened (and my memory is not good) or to keep scrolling to the top of the page. Of course I could use a fancy split screen IDE and keep the setup() method in view too, but that just seems messy.

Why is there so much in this setup()? Is it all really necessary? For every test? Looking at the method, it’s hard to tell. I guess we’ll find out.
Comment 3 – AcceptConnectionForUnconnectedUsersWithNoPendingRequestsShouldSucceed
Ok, so I’m an apostate – I prefer test names to be snake_case. I just find it so much easier […]

By |December 5th, 2015|Agile, BDD, TDD, Unit testing|9 Comments

Making a meal of architectural alignment and the test-induced-design-damage fallacy

Starter
A few days ago Simon Brown posted a thoughtful piece called “Package by component and architecturally-aligned testing.” The first part of the post discusses the tensions between the common packaging approaches package-by-layer and package-by-feature. His conclusion, that neither is the right answer, is supported by a quote from Jason Gorman (that expresses the essence of thought over dogma):
The real skill is finding the right balance, and creating packages that make stuff easier to find but are as cohesive and loosely coupled as you can make them at the same time
Simon then introduces an approach that he calls package-by-component, where he describes a component as:
a combination of the business and data access logic related to a specific thing (e.g. domain concept, bounded context, etc)
By giving every component a public interface and package-protected implementation, any feature that needs to access data related to that component is forced to go through the public interface of the component that ‘owns’ the data. No direct access to the data access layer is allowed. This is a huge improvement over the frequent spaghetti-and-meatball approach to encapsulation of the data layer. I like this architectural approach. It makes things simpler and safer. But Simon draws another implication from it:
how we mock-out the data access code to create quick-running “unit tests”? The short answer is don’t bother, unless you really need to.
I tweeted that I couldn’t agree with this, and Simon responded:
This is a topic that polarises people and I’m still not sure why
Main course
I’m going to invoke the rule of 3 to try and lay out why I disagree with Simon.
Fast feedback
The main benefit of automated tests is that you get feedback quickly when something has gone wrong. The longer it takes to run the tests, the longer […]

By |March 19th, 2015|Agile, Practices|2 Comments

Entanglement (or there’s nothing new under the sun)

I’ve just read The Age of Entanglement : When Quantum Physics was Reborn by Louisa Gilder. It’s a tremendous book, looking at the interplay between great physicists over the whole of the 20th century. If you want to learn about quantum physics itself, this is probably not the book for you, but if a mix of science and history is your thing, then I can’t recommend this book enough. But that’s not why I’m writing this post.

As I read the book, there were two passages that jumped out at me because they were so relevant to experiences I have regularly. One was about testing and the other was about collaboration. Maybe I shouldn’t have been surprised, but there you have it – I was.

Experimental physicists understand that testing saves time. They sound a lot like developers who:
“want to slap it all together, turn it on, and see what happens.”
Inevitably:
“you can almost guarantee it’s not going to work right.”
Doesn’t this sound familiar? Their conclusion might sound familiar too:
“People always think you don’t have the time to test everything. The truth is you don’t not have the time. It’s actually a time-saving way of doing it.”

And then I found the description of a conference that sounded like a pre-cursor to the modern, open space ‘unconferences’ that have been springing up. An explicit acknowledgement that:
“the best part of any conference is always the conversation over coffee or beer, the chance meeting in the hall, the argument over dinner.”
This led directly to their decision to:
“organise their conference to be nothing but these events. No prepared talks, no schedule, no proceedings.”
I’ve heard of regular, private get-togethers like this that go on in the software community, where a selected group of invitees hole up […]

By |February 19th, 2015|Musings, Practices, Systems, Unit testing|0 Comments

Half a glass

Is this glass half empty or half full?

There’s normally more than one way to interpret a situation, but we often forget that the situation itself may be under our control.

I often find my clients have backed themselves into a corner by accepting an overly restrictive understanding of what they’re trying to achieve. They will tell me, for example, that this story is too big BUT that it can’t be split, because then it wouldn’t deliver value. Or, they might be saying that there’s no point writing unit tests BECAUSE the architecture doesn’t support it. Or even, it’s not worth fixing this flickering test, because it ALWAYS works when they run it locally.

I encourage them to rethink the situation. A slice through that story may not deliver the final solution the customer needs, but it will act as a small increment towards the solution. Minor refactorings are usually enough to make a section of the code independently testable. That flickering test is undermining any confidence there might be in the test suite – fix it or delete it.

Don’t try to put a spin on a half-full glass – rethink the glass.

Instead of asking if the glass is half-full or half-empty, use half a glass.

(It turns out that wittier minds than mine have already applied themselves to this proverb – here’s a bumper collection.)

By |December 15th, 2014|Agile, Musings, Practices|0 Comments

Diamond recycling (and painting yourself into a corner)

The post I wrote recently on recycling tests in TDD got quite a few responses. I’m going to take this opportunity to respond to some of the points that got raised.
Do we really need to use the term “recycling”?
The TDD cycle as popularly taught includes the instruction to “write a failing test”. The point of my article was to observe that there are two ways to do that:

write a new test that fails
change an existing, passing test to make it fail

It’s this second approach that I’m calling “recycling”. Alistair Cockburn says that “it’s a mystery this should need a name” and it probably doesn’t. However, I’ve regularly seen novice TDD-ers get into a mess when making the current test pass causes other test(s) to fail. Their safety net is compromised and they have a few options, none of which seem very appealing:

Roll back to last green
Comment out the failing test(s)
Modify the failing test(s) to make them pass again

Whichever way you go to get out, you’ll want to try to avoid painting yourself into a similar corner in future.
Why do tests that used to pass start failing?
Ron Jeffries suggests that this will only happen if the tests don’t “say something universally true about the problem and solution.” Several people (including George Dinwiddie and Sandro Mancuso) demonstrated that this problem can be solved by writing a series of tests that each say something “universally true.” However, to me, this seems like a similar approach to that recommended by Alistair Cockburn in his “Thinking Before Programming” post.

I’m a big fan of thinking before programming. In the courses that I deliver, I routinely prevent students from touching the keyboard until they’ve thought their way around the problem. But, it’s just not realistic to expect that […]

By |December 9th, 2014|Agile, BDD, Cyber-Dojo, TDD, Uncategorized|3 Comments

Using SpecFlow on Mono from the command line

SpecFlow is the open source port of Cucumber for folk developing under .NET. It has been compatible with Mono (the open source, cross platform implementation of the .NET framework) for several years, but most of the documentation talks about using it from within the MonoDevelop IDE. I wanted to offer SpecFlow as one of the options in Cyber-Dojo and, since the Cyber-Dojo IDE is your browser, I was looking for a way to make it all happen from the command line.

Cyber-Dojo already offers C#/NUnit as an option, so I used this as my starting point for making SpecFlow available. I came up with a list of tasks:

Install SpecFlow
Generate ‘code-behind’ each feature file
Include generated code in compilation

Install SpecFlow
I found an interesting website that had detailed instructions for installing SpecFlow on Mono. There don’t seem to be any handy ‘apt-get’ packages, so it is basically a process of downloading the binaries and installing them in the GAC, for example:

  gacutil -i TechTalk.SpecFlow.dll
Generate ‘code-behind’ each feature file
SpecFlow ships with a command line utility, specflow.exe. I tried following the instructions from the article that had helped me with the installation, but they didn’t work for me. I ended up invoking the utility directly:

  mono ./specflow.exe

Running specflow.exe like this lists the operations that the utility provides, from which I chose ‘generateall’, because (according to the documentation) it should do exactly what I want:
re-generate all outdated unit test classes based on the feature file.
Unfortunately it needs a Visual Studio project file (csproj) as input, and since we’re not using Visual Studio we don’t have one. This led me to insert a fourth item in my task list: “Create csproj file”
Create csproj file
I started with an MSDN article that describes creating a minimal […]

By |October 5th, 2014|BDD, Cyber-Dojo, Practices, TDD, Unit testing|1 Comment

To TDD or not to TDD? That is not the question.

Over the past few days a TDD debate has been raging (again) in the blog-o-sphere and on Twitter. A lot of big names have been making bold statements and setting out arguments, of both the carefully constructed and the rhetorically inflammatory variety. I’m not going to revisit those arguments – go read the relevant posts, which I have collected in a handy timeline at the end of this post.
Everyone is right
Instead of joining in the argument, I want to consider a conciliatory post by Cory House entitled “The TDD Divide: Everyone is right.” He proposes an explanation for these diametrically opposed views, based upon where you are in the software development eco-system:
Software “coaches” like Uncle Bob believe strongly in TDD and software craftsmanship because that’s their business. Software salespeople like Joel Spolsky, Jeff Atwood, and DHH believe in pragmatism and “good enough” because their goal isn’t perfection. It’s profit.
This is a helpful observation to make. We work in different contexts and these affect our behaviour and colour our perceptions. But I don’t believe this is the root cause of the disagreement. So what is?
How skilled are you?
In Japanese martial arts they follow an age old tradition known as Shu Ha Ri, which is a concept that describes the stages of learning to mastery. This roughly translates as “first learn, then detach, and finally transcend.” (I don’t want to overload you with Japanese philosophy, but if you are interested, please take a look at Endo Shihan’s short explanation)

This approach has been confirmed, and expanded on, in modern times by research conducted by Stuart and Hubert Dreyfus, which led to a paper published in 1980. There’s a lot of detail in their paper, but this diagram shows the main thrust […]

By |May 2nd, 2014|Agile, TDD, Unit testing|3 Comments

TDD at interviews

Allan Kelly posted an article on DZone this week predicting that TDD would be a required skill for developers by 2022. Vishal Biyani asked on Twitter about how one might test TDD skills, and I promised to blog about my experience of using Cyber-Dojo in interview situations.

Cyber-Dojo is a browser-based dojo environment developed by Jon Jagger that supports a lot of programming languages and xDD frameworks. It’s great for dojos because it has few of the productivity frills that we’ve come to depend on over the years – no syntax highlighting; no autocompletion; no suggested fixes. That means we have to think about what we’re doing, rather than relying on muscle memory.

As Jon eloquently puts it in the FAQ: “Listen. Stop trying to go faster, start trying to go slower. Don’t think about finishing, think about improving. Think about practising as a team. That’s what cyber-dojo is built for.”

That might be what cyber-dojo was built for, but it turns out that it’s also excellent as an interview environment. Your interviewee writes real code and has to diagnose with real compiler/runtime errors. They’ll have to use a browser to remind themselves of all the basic knowledge that has atrophied during years of nanny-IDE development. And, best of all, there’s no save, build or run functionality provided by cyber-dojo. There’s only a single button and it’s labelled nice and clear: TEST.

Use one of the katas whose instructions have been helpfully included with cyber-dojo, or roll one of your own, and see how your interviewee responds. Every time they press the TEST button, all the code they’ve written is sent over to the server to be built & run and the response is returned, along with a traffic light: green for “all tests passed”, red for […]

By |January 11th, 2014|Practices, TDD, Unit testing|1 Comment

When is a tester not a tester?

No, I’m not trawling through my xmas cracker jokes. I was looking through the programme for DevWeek 2014 and both my sessions are tagged as “Test”. This is following a pattern started at ScanDev last year and followed by several other conferences at home and abroad.

Why am I bothered? It’s not that I mind being associated with testing at all. I don’t think of testers as a lower form of life. I *love* testers. It’s for the same reason that Dan North and Chris Matts started using the “should” word instead of the “test” word all those years ago – developers think that the test track is not for them.

Both my sessions at DevWeek are about types of testing that developers should be doing routinely. “So long, and thanks for all the tests” explores what makes a test valuable and what practices developers should consider adopting. “Mutation testing – better code by making bugs” is an alternative to code meaningless coverage metrics that can help developers ensure they’re sticking to their definition of done.

Q. When is a tester not a tester?
A. When they’re a developer.

You’re right. It’s not funny. So, it’s ideal for a cracker.

By |January 9th, 2014|Practices, Unit testing|0 Comments

The Beer Belly testing anti-pattern

The ‘Testing Pyramid’ is often trotted out to illustrate a suggested distribution of tests. More small “unit” tests; less deep “end-to-end” tests. And various people have observed common anti-patterns, specifically the Ice Cream Cone, where there are lots of end-2-end tests and hardly any unit tests.

 

The anti-pattern that I see most often is the Beer Belly. This stems from a misunderstanding of what constitutes a unit test. It’s proven difficult to define exactly what a unit test is, but Mike Feathers described what ISN’T a unit test – notably tests that hit the network, the file system or the database. Most developers don’t seem to have taken this on board and write many tests that rely on some (or all) of these external components.

Combine this tendency to write integration tests instead of unit tests with a whole pile of manual test scripts and you get the classic beer belly topped with a big, cloudy head of woe.

And you know the best way to cure a hangover. Hair of the dog, anyone?

By |November 21st, 2013|Practices|0 Comments