How ATDD Guides You in Collaboratively Creating User Centric Applications?

11 mins read
By Marjan Venema
Updated on January 31, 2024

Drive User and Customer Acceptance With ATDD

The software engineering landscape is riddle with jargon and initialisms. It seems particularly fond of initialisms ending with D-D, for instance. It can often be frustrating trying to keep up with so many concepts. In this post, we’ll talk about one of the many DD-ending initialisms in software: Acceptance Test-Driven Development.

Acceptance Test-Driven Development, or ATDD, is a software development methodology, often associated with agile methodologies, that fosters collaboration between developers, testers and business stakeholders, and in which test automation plays a major role.

As its name suggests, ATDD is related to TDD, or Test-Driven Development. It also has similarities with approaches such as SBE (Specification by Example) and BDD (Behavior-driven development.) Throughout the article, you’ll understand the similarities and differences between these approaches.

What is Acceptance Test Driven Development?

Let’s start by telling you what ATDD is not. It’s not a testing technique, despite what its name might suggest. Instead, acceptance test-driven development is a programming technique. Similarly to its “cousin” TDD, ATDD puts test automation front and center. When adopting it, teams will use automated test cases to drive the development of the production code.

Unlike TDD, which is generally thought of as an engineering practice, ATDD is a team effort: it brings different players in the software development process – testers, engineers, business stakeholders together to collaborate. The result of said collaboration are the requirements for the application, expressed in a format understandable by everyone, which are then turned into automated acceptance tests.

What is The Main Goal of Acceptance Test Driven Development?

ATDD seeks to foster collaboration that gives rise to a shared understanding of the system’s requirements, in the form of specifications written in plain English. The specifications are then turned into automated acceptance test cases. What is the benefit in doing that?

To understand why that’s useful, first think of unit testing. This is a form of testing which, by its nature, is very developer-centric. It helps engineers document their assumptions about their code in executable format. Unit testing solves the problem of “are we building the thing right?”

However, unit tests don’t solve the problem of “are we building the right thing?” For that, you’d have to resort to acceptance testing, that verifies the workings of the application against a set of acceptance criteria discovered through conversations with a domain expert.

In a nutshell, acceptance test-driven development solves the problem of the development team implementing features that don’t solve the customer’s needs.

An essential component of ATDD is automation: the specifications created from the discussions are turned into executable tests that ensure software engineers implement the features according to the requirements.

Brief History

In his seminal book “Test-driven development: By Example”, first published in 2000, Kent Beck briefly mentions ATDD—which he calls Application Test-Driven Development— but dismisses the idea.

However, the approach started to gain steam anyway, probably due to the success of tools like Fit and Fitnesse.

In 2006, Dan North introduced the concept of Behavior-driven development, which originally was merely meant to replace some of the TDD vocabulary, but ended up evolving into requirement analysis. BDD shares many similarities with ATDD and, in time, much of the vocabulary, tools and approaches of the former were adopted by the latter.

How Does it Work / Principles and Practices?

A General Overview of ATDD

Acceptance test-driven development has a similar workflow to that of TDD, since it consists of creating tests before writing the production code. Unlike TDD, which relies on unit tests, ATDD tests are acceptance tests. The tests are born from specifications that arise from discussions involving developers, testers and business stakeholders or clients.

Src: Mysoftwarequality & Gojko Adzic

After the creation of the tests, developers write code to make those tests pass, implementing the features according to the specifications.

That way, you not only get comprehensive specifications that everyone can understand and contribute to—regardless of coding skills—, but also you ensure the developers actually build what the customer needs.

Acceptance Test-driven Development Cycle

While there isn’t universal agreement on the phases of the acceptance test-driven development cycle, you’ll often see these 4 steps:discuss, distill, develop, and demo.

Discuss

During the planning meeting—common to virtually all methodologies/frameworks under the agile umbrella—testers and developers will discuss the tasks/user stories picked for implementation for the next iteration or sprint with the stakeholder/product owner/client in order to extract as much information as possible from them.

During the discussion phase, the team has the opportunity to clarify misunderstandings in the requirements, which can lead the team to prevent a bug from being introduced in the application, which is way cheaper than having to debug, diagnose and fix it afterwards.

Also, the “discuss” phase might lead the team to find out that what seemed like a simple user story is, in fact, much more complex. In such a circumstance, the team, along with the business stakeholder, might choose to split the story into two or more stories.

The output of this phase are acceptance tests in the form of plain English phrases or tables that can be understood by everyone in the organization.

Distill

The second phase consists of converting the tests produced in the previous step to a format compatible with whatever acceptance testing tool you happen to be using. Depending on the tool, those formats include:

tables
wiki pages
or even code in a DSL (domain-specific language.)

So, the output of this phase are the acceptance tests. However, they’re not quite ready for execution, as you’ll see.

Develop

By this point, the test produced in the previous step can’t yet really test the system under test. They are merely skeletons, at this stage; you need to wire them to actual test code that will exercise application code.

The details of this wiring up vary according to the actual tools being used. Suffice it to say that after the wiring up is complete, the tests will fail, since the features they’re testing do not yet exist.

Then, the developers implement the feature, according to the specifications gathered in the “discuss” phase and that were converted into tests in the “distill” phase.

Demo

After the developers implement the feature, they demo it to stakeholders. Having a meeting at the end of each cycle in which the team discusses the product and/or how to improve the process itself is a very common feature of many agile methodologies. So, the demo phase of ATDD fits nicely with that.

After the implementation is complete, testers can also perform some manual exploratory testing. Such tests can find defects, but also find room for improvements which can then be turned into further stories.

Acceptance Test-driven Development Example

We’ll now walk you through a brief example of acceptance test-driven development.

The User Story

For this example, suppose you’re working on an e-commerce website, and you need to implement a shopping cart. In agile methodologies, requirements are often expressed as user stories. A common template for user stories is as follows:

As a
I want
So that I can

So, the user story for the shopping cart could be as follows:

As a buyer
I want to be able to add items to my shopping cart
So I can purchase them

Discuss: Coming Up With Acceptance Criteria

During the “discuss” phase, developers and testers ask the domain expert/business stakeholder as many questions as possible, with the goal of finding out blindspots in the requirements, learning about the happy and sad paths, and uncovering edge cases.

Here are some examples of possible questions:

If I add another instance of the same product to the cart, should it increase the number or add it as a new item?
Is there a limit to how many units of the same product I can add to my shopping cart?
Is there a limit to how many items I can add to my cart?

Based on the results of the discussion, testers and developers might come up with the following table detailing acceptance tests scenarios:

_____________________________

Initial Scenario	Action	Expected Result
Empty cart	Add product “Book ‘Test-Driven Development By Example’”, which costs $29, to the cart	The cart should contain one item, and the total should be $29.
Cart with one product (“book test-driven development by example’” costing $29)	Remove the item from the cart	Empty cart. Total of 0.
Cart containing 3 units of the same product.	Add another unit of the same product.	Cart still with three items. Message is displayed saying clients can’t buy more than three items of the same product.

_____________________________

Distill: Turning The Criteria Into Automated Tests

In the distill phase, developers and testers convert the acceptance tests written in plain language into formats compatible with the automation tools they use.

Let’s now see what that would look like, for our example, using two different tools/formats.

Fitnesse

Fitnesse is one of the most popular tools for automated acceptance testing. It is unique among other testing tools in the peculiar format it adopts for expressing tests: tables in a Wiki.

Src: assertselenium.com

This is what a Fitnesse table for testing the scenario of “adding items to a cart” could look like:

The table above isn’t a mere table. Those are real tests that can be executed. However, an attempt to execute them results in the following error:

We still don’t have a fixture, that is, an actual class that “glues” our Fitnesse test to actual code.

Gherkin Language (Cucumber)

Let’s now see how we could represent the same acceptance criteria using Gherkin, which is the special language used by the tool Cucumber.

Using Gherkin, users can describe scenarios using a common template, known as given-when-then:

Given
When
Then

Here’s an example:

Feature: Shopping Cart

Buyers need to be able to add items to their carts

Scenario: Adding a product to an empty cart

Given my cart is empty
When I add a product called “Book TDD By Example” which costs $29 to my cart

Then my cart should contain that product

And the total should be $29

Trying to execute the test above yields a result similar to what you’ve seen with Fitnesse:

It says the scenario has missing steps, and then it goes further and gives us advice on how to actually fill in those blanks.

Develop: Wiring Up The Tests and Implementing The Feature

For the example above, this is how we could fill in the blanks:

Given(“my cart is empty”, () -> {
this.cart = new ShoppingCart();
});

When(“I add a product called {string} which costs ${int} to my cart”, (String string, Integer int1) -> {
this.product = new Product(string, int1);
this.cart.add(this.product);
});

Then(“my cart should contain that product”, () -> {
assertEquals(1, this.cart.getItemsQuantity());
assertEquals(this.product, this.cart.items[0]);
});

Then(“the total should be ${int}”, (Integer int1) -> {
assertEquals(int1, this.cart.total);
});

We’re using instance variables to hold an instance of a ShoppingCart class, which represents—you’ve guessed it—our shopping cart. Such a class could—and should—be developed by using the test-first approach (TDD.)

As you can see, Cucumber is smart enough to replace the product’s name and price with placeholders, which are then turned into variables. That way, we can parametrize those tests, reusing the same logic with a number of different input values.

Similarities and Differences

Let’s now discuss how ATDD is similar or different from other practices.

Acceptance Test-driven Development vs Test-Driven Development

Acceptance test-driven development is often compared to test-driven development. Obviously, they’re related, but how do the two of them compare?

TDD – test-driven development – is a methodology in which developers start development by writing a failing unit test and only then writing the production code to make the test pass.

By using TDD, developers aim to create clean and simple code, composed of loosely coupled modules, leading to higher maintainability. However, TDD is a developer-centric approach that relies on unit testing. As you’ve seen, unit testing can lead to the problem of correctly implementing the wrong features.

ATDD isn’t centered around the developer, but encourages collaboration between developers and other team members, even those without coding skills.

You can and should use TDD and ATDD together. At the start of each iteration, start by creating automated acceptance tests based on discussions with the domain expert.

When it’s time to actually implement the production code, use TDD to test-drive the development of said code. In other words: TDD can be leveraged as an “internal” cycle inside the overall ATDD phases.

Acceptance Test-driven Development vs Behavior Driven Development

BDD – Behavior-driven development – was originally introduced by Dan North as a replacement for TDD. He felt the word “test” was problematic, and searched to create new vocabulary centered around behaviors and scenarios. So, BDD originated at the unit test level.

Only when a friend pointed out that his approach sounded like analysis, he started applying the approach to requirements.

When compared to acceptance test-driven development, BDD has a stronger focus on creating a shared understanding of the system’s expected behavior. Concepts from Eric Evans’ Domain Driven Design book – such as ubiquitous language – were also very influential on the development of BDD.

ATDD on the other hand, was created from the start with a focus on acceptance testing and test automation.

Common Pitfalls

ATDD tools might be a source of complexity/learning curve. For instance, Fitnesse requires people to use its markup language. Although there are alternatives—such as creating the tests using an excel spreadsheet and then importing into Fitnesse—there’s no denying there’s some complexity involved.

To successfully implement not only ATDD but acceptance testing in general will require training and mentoring for people to be ready to perform what’s expected of their roles. You’ll also need effective championing or evangelizing to get everyone on board and ready to apply the approach.

Without all that, you’re likely to get only limited benefits.

Getting Started

How to get started with ATDD? Here are some practical steps.

Train everybody on the topic to get them on the same page. Start by ensuring everyone in the organization has access to training materials so they can be educated when it comes to ATDD.
Train everybody in the skills for their roles and responsibilities. Implementing ATDD requires very specific skills, including social ones—knowing how to discuss and collaborate with domain experts, for instance. Of course, technical skills, such as competence of testing tools are also required.
Get familiar with requirements in the format of user stories. User stories are surely one of the foundations of ATDD. It’s crucial that all team members are comfortable reading and also writing software requirements using this format. Later, you should expand this into also being competente in the Gherkin language.
During the initial meeting at the start of each iteration, start practicing extracting scenarios/acceptance criteria from the stakeholders and writing tests in plain English with them. It’s not required to use a specific format or language for this.
After getting better at the previous step, search for a tutorial or guide on a specific ATDD acceptance-testing framework. Adopt the framework and start implementing the acceptance tests according to the ATDD cycle.
Regularly evaluate the approach and adapt when needed.

Further Reading

Test Driven Development: By Example, by Kent Beck
ATDD by Example: A Practical Guide to Acceptance Test-Driven Development by Markus Gärtner
Driving Development with Tests: ATDD and TDD – Paper by Elisabeth Hendrickson
Behavior Driven Development (BDD): Creating Useful Software Customers Want

Deliver Software That Solves Your Users’ Problems

Unfortunately, it’s common for development teams to create software that doesn’t do what the customer actually asked for. Acceptance test-driven development seeks to solve this problem, by using test automation to capture user requirements in a format well-understood by everyone.

To be able to reap the benefits of ATDD, you must have a solid knowledge on its foundations. This includes topics like user stories, as you’ve seen.

If you’re not familiar with the concept of user stories—or even if you are, but want to brush up on your knowledge a little bit—take a look at our article about this popular technique.

Speed up your Agile planning and execution!

Signup for a FREE Trial of Nimble Agile