Classifying BDD Tools (Unit-Test-Driven vs. Acceptance Test Driven) and a bit of BDD history

BDD (Behavior Driven Developement) seems to gain momentum lately.

BDD is hard to grasp, because it is more than a development methodology (unlike TDD) and there are currently very different ideas what it really means. To make matters worse there are a lot of BDD tools around that pursue quite different concepts.
In this post I try to classify two main categories among those tools.

Important for the understanding is that BDD can be practiced at different levels. At the code/unit level and at the feature level. In both cases it is a good practice to use examples to specify the behavior of the units resp. features. However in both cases we have different stakeholders and intents with the BDD process.
Both approaches to BDD are not exclusive, they can be combined.
(see also "BDD for Acceptance Tests vs. BDD for Unit Tests (or: ATDD vs. TDD)" on the SpecFlow mailing list)

Currently I would differentiate two flavors of BDD tools:

1. Tools with a business readable output
2. Tools with a business readable input

With tools from the first category (business readable output) the support for the BDD process is typically focused on the developers. All artifacts involved are owned by the developers and are typically code. This is not a bad thing, as responsible and committed developers are often the main stakeholders in successful software projects. Other stakeholders are only addressed by the tools by providing understandable reports after developers have done their work.

Tools from the second category (business readable input) try to widen the focus of the BDD process by enabling more involvement by all other stakeholders: customers, business analysts, testers maybe even operations.
This involvement is possible upfront, meaning before the developers have done their work. This business readable input is an artifact commonly owned by all stakeholders. By having a commonly owned artifact, the process tries to ensure that all involved stakeholders build up a shared understanding which is crucial for successfully software projects (for more information I suggest reading Bridging the Communication Gap by Gojko Adzic).

Because this shared understanding is input (as opposed to output) and business readable, the process can even be taken further and make the artifact business writable. If this is achieved successfully then we have done a big step towards executable specifications.

In practice we often see tools of the first category (business readable input) used as a replacement/extension for TDD at a unit-testing level.
In this case the behavior of the unit under development is focused by trying to specify this behavior upfront in a less technical way. However this is usually not interesting for other stakeholders than developers, because the technical units under development are usually too fine grained and have no meaning to non technical persons. Also technical units are the primary focus of developers and they are the main stakeholders (think of testability, separation of concern, maintainability ...), other stakeholders should not be too involved here. So this is mainly a tool for developers and allows in some cases the mapping of technical units to overlying features.

The second kind of tools aim clearly at Acceptance Testing and Acceptance Test Driven Development. Here system features are clearly focused. Features are usually more coarse grained than the behaviors of single technical units. Features should always be understood and driven by business requirements.

It is certainly possible to write acceptance tests with the first category of tools, but the driving aspect (as in Acceptance Test Driven Developement) is certainly harder to achieve and less supported by these kind of tools.


Examples:
Easyb is a Groovy based tool of the first category. The following example (from this post) shows how behavior is specified at at unit-level (class-level):

scenario "Two amounts with the same currencies are added", {
  given "Two different amounts with the same currencies", {
    money1 = new Money(12, "CHF")
    money2 = new Money(14, "CHF")
    expected = new Money(26, "CHF")
  }
  when "Add given amounts" , {
    result = money1.add(money2)
  }
  then "New amount is sum of two given ones", {
    result.equals(expected).shouldBe true

  }
}

scenario "Two amounts with different currencies are added", {
  given "Two amounts with different currencies", {
    money1 = new Money(12, "CHF")
    money2 = new Money(14, "EURO")
  }
  when "Add given amounts", {
    add = {
      money1.add(money2)
    }
  }
  then "Operation should fail", {
    ensureThrows(IllegalArgumentException) {
      add()
    }
  }
}

Executing the above story gives you the follwowing readable report:

2 scenarios executed successfully.

  Story: money

    scenario Two amounts with the same currencies are added
      given Two different amounts with the same currencies
      when Add given amounts
      then New amount is sum of two given ones

    scenario Two amounts with different currencies are added
      given Two amounts with different currencies
      when Add given amounts
      then Operation should fail


(of course easyb can also generate a fancy html report)


Cucumber is a tool of the second category. The following example from cuke4duke shows how an executable feature is specified:

Feature: Book search
  In order to find books I might buy
  As a potential customer
  I want to search for books by different criterias
 
  Background:
    Given the following books
      | Author          | Title                                           | Year | Publisher      |
      | Martin Fowler   | Patterns of Enterprise Application Architecture | 2002 | Addison Wesley |
      | Eric Evans      | Domain Driven Design                            | 2003 | Addison Wesley |
      | Gerard Meszaros | xUnit Test Patterns                             | 2007 | Addison Wesley |
 
  Scenario: Search for title
    When I search for title 'Patterns'
    Then the result list should contain 2 books
 
  Scenario: Search for author
    When I search for author 'Fowler'
    Then the result list should contain 2 books


(notice the error in the second scenario!)

When executing this example (using cuke4duke) the whole EJB stack is exercised, including database-access. The result is the following report:

(of course Cucumber can generate a lot of different outputs)


History (as I could reconstruct it):
Tools from the first category evolved from TDD. Dan North introduced BDD in 2002, see his introductory blog post.
JBehave was one of the first BDD tools, at this point clearly part of the first of the above categories.
Then the BDD movement was mainly driven by the Ruby/Rails community. RBehave was introduced and then merged into RSpec as RSpec Story Runner.
RSpec then was as a tool that could satisfy both of the above catgories

RSpec StoryRunner was then dropped and Cucumber was created. Cucumber coined the current flavor of BDD tools with plain-text specifications. Cucumber also is aiming at extending its reach beyond Ruby/Rails (see cuke4duke, cuke4Nuke, gherkin).
Other first-generation tools then adapted and also support plain-text specifications (JBehave2, NBehave).

Beside all those explicit BDD tools there is FIT/FitNesse. FIT was also invented in 2002 by Ward Cunningham. Its goal was enabling collaboration and communication through automated accepatance testing. FitNesse took up the basics of FIT and provided an intuitive, wiki-based frontend/IDE.
While FIT seems to have practically died, FitNesse  is still thriving. FitNesse also recently added Parameterized Scenario Tables and there is GivWenZen. This makes it possible to use the plain-text GWT (Given-When-Then) syntax, that was made popular by the plain-text BDD tools.

I will list a more complete overview of BDD tools for the Java and .Net platforms in later posts.

follow me on twitter, I need some friends :-)

EntityFramework in Action: Great book, but the concerns remain

I did a review for Manning for the upcoming title "Microsoft Entity Framework in Action".

This is a thorough book about the EntityFramework v4. It not only discusses the EntityFramework but also gives some guidelines how to build applications with it. The book is written in a clear and well organized style that makes it an easy and enjoyable read.
What I liked in particular is the fact that the author is objective and also critical and does not blindly promote the EntityFramework as a silver bullet.

I can fully recommend this book to developers and architects that want to use the EF in a project or just want to learn more about the EF in order to be able to evaluate it.

The first chapter is probably the best introduction to the topic of object-relational mapping I have read so far.
Another highlight of the book are the chapters about Linq, which are also also valuable outside of the context of the EF.

But after reading the available chapters, I still have my reservations about the EntityFramework itself.
The book is very objective and does not hide the problems that still exist with the EntityFramework. The author also sometimes states his critical opinion about certain characteristics of the EntityFramework. I like this very much (better than hiding or ignoring the problems), but in the big picture this leads me to suggest for everybody to evaluate other ORMs before deciding for the EF.

Some critics have been addressed since the "Vote of no Confidence", but some still remain.
Here the main issues I still have with the EntityFramework after reading the book:

The tool-focused way of the mapping (using the designer is almost a must).

The "model first Approach" is still not really very smooth compared to other ORMs.

POCO support comes not out of the box, probably because of legacy support for EFv1. If you use the provided code-generation templates, the generated entities are not POCOs. You actually have to download special templates to have the EF generate real POCO entities.

The mapping mechanisms of the EF are overly complex, and I don't see the benefits over other ORMs (concepts like EntitySets and foreign-key vs. independent associations are just two examples that are not present in other ORMs (e.g.JPA), while I think those other ORMs have proven to be fully capable of solving the problem).

The verbose and redundant xml-mapping files (.csdl, .ssdl, .msl) remind me heavily of the J2EE days from about 5 years ago. As in J2EE 5 years ago, tooling is the proposed solution for accidental complexity. In the Java world of today this heavyweight J2EE approach is generally accepted as a failure.

Some mapping artifacts (.edmx, .ssdl, .msl) are still too big and monolithic, making them not suitable for source-control and team environments.

follow me on twitter, I need some friends :-)

Parkinson's Laws

I only recently stumbled over Parkinsons Laws, even though I think in the IT business we are constantly confronted with them:


Parkinson's Law:

Work expands so as to fill the time available for its completion.

Who has not seen self fulfilling estimations?


Parkinson's Law of Triviality:

Organisations give disproportionate weight to trivial issues.

(the time spent on any item of the agenda will be in inverse proportion to the sum involved.)

Think of this next time you are on a meeting discussing the pros and cons of using underscores as prefix for member variables ...

follow me on twitter, I need some friends :-)

.Net Annoyance: Define |DataDirectory| for Connection Strings

Update 20100326: For an acceptable workaround see at the bottom.

One thing  where I think common practice in .Net is behind Java is the idea of using an embedded database for integration tests.

Sql Server Express Edition has the nice feature to specify the DB-File in the connection string with AttachDbFilename.
This makes it easy to distribute your database with your project, which is a nice thing. The project template of ASP.Net MVC for instance uses this feature out of the box.

However the connection string usually contains this fragment:
AttachDbFilename=|DataDirectory|\BookShopDB.mdf
|DataDirectory| is a placeholder, that in the case of the ASP.Net MVC template refers to the directory App_Data. This way it is possible to specify a relative path for your DB-File.

This seems a nice feature, but only as long as you are only distributing your solution containing the application.
The problems start, when you would like to open a connection to the same database from outside of the actual application.This is a common scenario for integration tests, which live in another project. AttachDbFilename seems not to accept relative paths that point to a location higher in the directory structure … this is a major annoyance!

You can define the value for |DataDictionary| like this:
AppDomain.CurrentDomain.SetData("DataDirectory", @"C:\XYZ\App_Data\");
… but this does not solve the problem, since it still is not possible to use a relative path that point to a location higher in the directory structure.

The following results in a runtime exception:
AppDomain.CurrentDomain.SetData("DataDirectory", @"..\..\XYZ\App_Data");
Argh! Why make it so hard?

Does anybody have a solution for this problem?
Here the problem on Stackoverflow, and here an explanation on the Data Access blog.

Update 20100326: I settled for the following acceptable workaround:

In my App.config of my test-project I have the following section:

  <appSettings>
    <add key="DataDirectory" value="..\..\..\BookShop\App_Data\"/>
  </appSettings>

In the test-setup I then execute the following code:

var dataDirectory = ConfigurationManager.AppSettings["DataDirectory"];
var absoluteDataDirectory = Path.GetFullPath(dataDirectory);
AppDomain.CurrentDomain.SetData("DataDirectory", absoluteDataDirectory);

BDD Breakfast in Zürich

Next Tuesday (16.3.2010) my employer TechTalk is arranging a breakfast in the Technopark in Zürich.

Christian and I will hold a technical talk about Behavior Driven Development (BDD) and also present examples with SpecFlow.

The event starts at 8:00. Everybody is invited. Its free and there will be coffee and croissants. Please register here.

follow me on twitter, I need some friends :-)

Dead-End Heroes

Apart from the wonderful scottish accent there was one train of thought that particularly caught my interest in SE-Radio Episode 156: Kanban with David Anderson:

Estimation is a choice! Estimates lead to commitments that drive a dysfunction that is undesirable:

Estimates lead to setting an artificial target and forcing people to meet that. That tends to drive heroic behavior. [...] When people act heroically they stop improving [...] and they stop learning.

- David Anderson, SE Radio #156

follow me on twitter, I need some friends :-)