specflow_logoSpecFlow is the open source port of Cucumber for folk developing under .NET. It has been compatible with Mono (the open source, cross platform implementation of the .NET framework) for several years, but most of the documentation talks about using it from within the MonoDevelop IDE. I wanted to offer SpecFlow as one of the options in Cyber-Dojo and, since the Cyber-Dojo IDE is your browser, I was looking for a way to make it all happen from the command line.

Cyber-Dojo already offers C#/NUnit as an option, so I used this as my starting point for making SpecFlow available. I came up with a list of tasks:

  1. Install SpecFlow
  2. Generate ‘code-behind’ each feature file
  3. Include generated code in compilation

Install SpecFlow

I found an interesting website that had detailed instructions for installing SpecFlow on Mono. There don’t seem to be any handy ‘apt-get’ packages, so it is basically a process of downloading the binaries and installing them in the GAC, for example:

  gacutil -i TechTalk.SpecFlow.dll

Generate ‘code-behind’ each feature file

SpecFlow ships with a command line utility, specflow.exe. I tried following the instructions from the article that had helped me with the installation, but they didn’t work for me. I ended up invoking the utility directly:

  mono ./specflow.exe

Running specflow.exe like this lists the operations that the utility provides, from which I chose ‘generateall’, because (according to the documentation) it should do exactly what I want:

re-generate all outdated unit test classes based on the feature file.

Unfortunately it needs a Visual Studio project file (csproj) as input, and since we’re not using Visual Studio we don’t have one. This led me to insert a fourth item in my task list: “Create csproj file”

Create csproj file

I started with an MSDN article that describes creating a minimal csproj file by hand. I then inspected an actual csproj file from a project that was using SpecFlow and, through guess work and experimentation, ended up with:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <None Include="Example.feature">

I created a simple feature file called Example.feature and ran specflow.exe with the csproj (shown above) as input. The output was a correctly generated C# file Example.feature.cs


Include generated code in compilation

I tried the existing compilation line from Cyber-Dojo:

  dmcs -t:library \
       -r:/usr/lib/cli/nunit.framework-2.6/nunit.framework.dll \
       -out:Example.dll *.cs

Unsurprisingly it responded with the error : Unknown symbol 'TechTalk': are you missing an assembly reference. Clearly I was, so I had a look around to see if I could find the relevant SpecFlow DLL. I couldn’t – it had successfully installed in the GAC, but I couldn’t work out how to tell dmcs where to find it.

I looked around and stumbled upon this documentation in the Mono project that described installing assemblies in the GAC. So, I reinstalled SpecFlow using a modified gacutils incantation:
  gacutil -i TechTalk.SpecFlow.dll -package SpecFlow

And rewrote the compilation command to be:

  dmcs -t:library -pkg:/usr/lib/mono/SpecFlow \
       -r:/usr/lib/cli/nunit.framework-2.6/nunit.framework.dll \
       -out:Example.dll *.cs

Still the same error: “are you missing an assembly reference”. I had another look at the file system and found the DLLs I expected in /usr/lib/mono/SpecFlow. I changed the compilation command to:

  dmcs -t:library -r:/usr/lib/mono/SpecFlow/TechTalk.SpecFlow.dll \ 
       -r:/usr/lib/cli/nunit.framework-2.6/nunit.framework.dll \ 
       -out:Example.dll *.cs

This worked. (If anyone can tell me a better way of installing and referencing the SpecFlow assemblies, I’d be very grateful).

Mucking around in Cyber-Dojo

So, now I had enough information to add a new Cyber-Dojo “language”. All the instructions for that are on the blog, so I won’t go into details. The main gotchas that got me were:

  • getting my head round Docker – especially WORKDIR
  • forgetting to add specflow.exe to the list of support_filenames in the language’s manifest file
  • forgetting a comma when I did add a file to a list in the manifest

I span up the Cyber-Dojo test instance and created a SpecFlow/C# kata. At this point the generated project contained Cyber-Dojo’s traditional failing unit test, and a feature file with a single scenario, but no step definitions, so I expected the unit test to fail and the scenario to error because of missing step definitions.

The unit test did failed, but the missing step definitions marked the scenario as inconclusive, which was interpreted by Cyber-Dojo as a pass. I could go one of two ways:

  • modify Cyber-Dojo to interpret inconclusive scenarios as failures
  • configure SpecFlow to treat missing step definitions as an error

The latter seemed easier, so I went back to the SpecFlow documentation.

Configuring SpecFlow

SpecFlow has a host of configuration options, but they need to be configured through App.config, another Microsoft file that I hadn’t needed up till now.

I went off and copied a minimal App.config and added it to the support_filenames list:

<?xml version="1.0" encoding="utf-8" ?>
    <section name="specFlow"
    <unitTestProvider name="NUnit" />
    <runtime missingOrPendingStepsOutcome="Error" />

I also added it to the csproj file, so that the specflow.exe utility knew about it:

  <None Include="App.config" />

The missing step definitions were still not being treated as errors. I went back to the internet and found this useful post from Charlie Poole:

Mono expects the config to have the name of the exe plus ‘.config’ – for example, for the xxx.exe, the config is xxx.exe.config. Visual Studio uses a convention of renaming the App.config file correctly and copying it to the output directory.

So, I renamed App.config to Example.dll.config and tried again – and everything worked as I wanted!


End game

I added a step definition file to the default project, checked in my branch and issued a pull request. That went live on October 4th 2014.

You can now practice writing SpecFlow features in Cyber-Dojo. If there are any missing step definitions the build will fail, but the output will contain a snippet for each missing step definition that you can paste into your step definition class.

Because NUnit is still part of the environment, you can write unit tests as well as features. This means that you can use SpecFlow features as an outer Red/Green/Refactor loop to drive development from business specifications AND use NUnit directly in an inner Red/Green/Refactor loop to TDD any complex business logic.

If you have any suggestions about how to improve the deployment process, I’d love to hear them.