Implementing Specification By Example

Recently I have had the pleasure to work with different teams and companies. The interesting part of seeing how different people work is that I learn a lot from their codebase.  At my current work we follow the concept of specification by example, however these specifications are often a pain point for our teams. I wanted to take the time to explore some successful ways of implementing specification by example.

Background

There are many terms that have been described for these sets of practices that I won't go into. However I think specification by example is the better term, as we tend to describe software via interactions (or examples). To get yourself up to speed I will recommend two great books on the topic:

1. Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing
2. Specification by Example: How Successful Teams Deliver the Right Software

Recommendations

Below will be a set of topics that I believe are important in having good specification by example. Please take them as a recommendation and not as a rule. I also encourage you to challenge these recommendations to find better ones.

Language

The biggest thing I find when writing these features is the language that is being used. We have to remember that the language that we use needs to reflect the domain we are trying to model. Often I encounter that these features are written through the eyes of the computer.

What we have to remember when writing these features is to focus on the WHAT not the HOW. A great friend of mine has done a fantastic job explaining how to do this. I urge you all to read it.

The language is really important. Try to get other people to read it and get them to explain what the example is trying to do.

Pick your battle

Follow the guidelines of the testing pyramid. Often I see people complain about their tests taking too long and being brittle. Remember that the UI is brittle at times, however in my view the end-to-end test is the most important one. If you can comfortably test a layer below it then go ahead. Use your judgement.

Informative Steps

Often I have seen that some information is missed out because there is no action for that step. This should be avoided. It does not matter if a step performs an action, if it helps with the intent write it in.

Behind the Scenes

Here is the big thing that I think people get wrong (of course this is my opinion). When writing features we seem to forget that we need to still apply all the great concepts we do for our main application. The biggest thing here is knowing what layer does what.

Usually when writing these features, people tend to dump all of their code into the steps file. This quickly becomes a nightmare to manage. Here are some guidelines I have used to make sure this becomes easy to follow:

1. The steps should only contain the builders and expectations.
2. Use the concept of a PageObject pattern. If you want a great library use dill.
3. Limit the use of the World. This just becomes a massive god class.
4. If you find yourself writing similar code, move it to a reusable module that all your projects can use.

Remember: Treat your test code the same way as your production code.

Creating Preconditions

Often we need to create a precondition (the given step in gherkin speak). What I have found is that people create a table where the key of that table is put through a mapping of the property of the model. This should be avoided. One of the worst things that you can do is create a translation layer. DDD has a concept of Ubiquitous Language, I suggest that you follow it.

Watch your Failures

One of the most underrated techniques I see is that people don't look at the failure message. How often have we seen expected true but got false. These errors don't help. Make sure you get a failing scenario first and read the message. Does it make any sense? Could you diagnose the issue?

Conclusion

These patterns have served me well in the past so I wanted to share them with you all. Feel free to challenge them so we can all learn to do things better. Of course context is king. So if you have any specific examples you would like discussed feel free to share.

System.Spec - Specifications By Example in C#

System.Spec

I am a big fan of TDD and when I first started doing it I was accustomed to using NUnit in C#. Then I started learning Ruby and fell in love in how they were doing TDD/BDD. The gem they use is called RSpec.

For a while I was jealous that we did not have something as cool in C#. There is NSpec, however for me if did not read that well.

So I decided to create System.Spec. We are currently using at the place I work at and we are really enjoying it. As we continue dogfooding this solution we will build more features into it.

An example specification looks like the following:

namespace YourCompany.Specs
{
    using NSubstitute;
    using System.Spec;

    public class ImportantSpecification : Specification
    {
        protected override void Define()
        {
            Describe("A very cool specification", () => {
                It("should do something that is good", () => {
                    var testInterface = Substitute.For<ITestInterface>();
                    testInterface.Received().TestMethod();
                });
            });
        }
    }
}

As you can see there is some C# ceremonies that we need to take into consideration, Ruby allows for a much cleaner syntax. It follows closely the design of Jasmine.

Moving Forward

The following is what I plan doing in the coming weeks/months:

1. Add a runner for ReSharper
2. Add a runner for VS 2010
3. Add a runner for Xamarin Studio (if possible)
4. Allow reordering of tests.

I urge you to look at the solution and if you have any thoughts or enhancements just yell out.