Traditional software releases, agile or not, usually contain the following steps:
In a perfect world, the specification enables developers to write a program with the right functionality. The software would be written with no bugs and be delivered to the customer after successful testing.
Unfortunately things go wrong.
- communication problems due to complexity of the problem
- misunderstandings about what the customer asked for
- implicit assumptions about desired behaviour – e.g edge cases, performance etc..
- environmental issues such as external systems not being ready, test data being unavailable or unrealistic
- we may not appreciate the size of the work until we are in the middle of it
The later we discover these problems, the costlier they are to solve.
Specifications, no matter how well they are documented and in how much detail, are too often lost in translation. Also, if developers and testers are not involved to suggest simpler alternatives – the solution can become unnecessarily expensive. Traditional system documentation is out-of-date almost as soon as it’s written, whereas agile acceptance tests/specifications by example when used as the basis of integration/unit tests become your system documentation and are up-to-date, or the tests fail.
As a feature is developed, changes very often arise. Customers may not require what they first thought they wanted. Often new features are invented by developers that may cover (in their minds) possible future requirements. So what usually happens is that the specification does not match what the program actually does. How then can we keep the spec in sync with the code?
Typically, the development-testing cycle is a bottleneck that can seriously delay a project. Testers are lumbered with testing an application they know little about. They have to pick away at original requirements that may not match what the application actually should do. This may mean repeating the same information in further documents used as test scripts.
BDD addresses some of the problems that TDD cannot solve on its own. It involves collaboration between developers, QA, business experts and the customer. The aim is to uncover incorrect assumptions and discover functional gaps before development starts.
It was coined by Dan North in 2003 and has more recently become quite a mainstream idea. Around that time, the term ‘Agile Acceptance Testing’ had pretty much the same goals. However, when people hear the word Testing, it can create the impression that it is something done after coding and has nothing to do with requirements. In fact BDD is all about requirements and so the word Testing can lead to some misunderstanding within this context. BDD prescribes a particular way of wording the expected behaviour through concrete examples or scenarios. This tends to work well in situations such as workflows but not in others such as state transitions or calculations. They may be better described in table format.
BDD allows us to deliver the right software collaboratively. Communication is at the core of this. BDD is definitely not a development methodology whereas TDD is. It is also not so important which tool is used to link specifications to code. What is more important is the focussed discussion about what is required. The tool we use for BDD is nowhere near as important as the process of getting the examples: the collaborative approach to build and refine them; and (hopefully) a common language evolving.
BDD also really feeds nicely into TDD, iteratively guiding it from the outside-in.
Each scenario can be automated against the existing code so that we have living documentation – i.e. Anything changes in either the specs or the code and the business knows about it.
Automating acceptance tests can be quite a headache as they typically are more end-to-end. They might involve some complex setup and configuration before running. In that respect, the BDD tools are not ideal for writing lots of regression tests with. The focus should be on identifying the requirements and endeavouring to prove important scenarios via test fixtures.