My friends and family are under attack in Ukraine.
Donate to protect them directly or help international organizations.

Getting Started With Acceptance Tests

Acceptance tests ensure that a feature works the way the stakeholder described it. This usually refers to the acceptance criteria of a user story. Once all the criteria have been satisfied, the work is considered complete.

For example, a credit card payment feature might be considered complete when the following scenarios are implemented:

Scenario: Buyer can purchase using a credit card
  Given I selected a product
  When I submit a valid credit card
  Then I should see a payment receipt
Scenario: Buyer cannot use an expired card
  Given I selected a product
  When I submit an expired credit card
  Then I should see an error message

I will use Behat and PHP for the examples, but this works similarly with any behavior-driven development (BDD) tool for any language. I used the same approach in C#, Java, PHP and Ruby.

How it Works

The BDD framework executes feature files. Feature files contain simple sentences readable by non-technical people:

Then I should see a payment receipt

For each sentence, the framework finds the associated method and executes it.

/**
 * @Then I should see a payment receipt
 */
public function iShouldSeeAPaymentReceipt()
{
}

The framework can also execute additional commands before and after a scenario.

/**
 * @BeforeScenario
 */
public function beforeScenario()
{
}

What Goes Into Which Step

There are no strict rules. This approach served me well over the years:

  1. Before Scenario: reset application state.
  2. Given: create new application state.
  3. When: execute the action.
  4. Then: verify the result.

To reset the application state, I typically reload the database from scratch along with a small subset of data without which the application cannot function at all.

To create a new state, I will execute a few actions. For example, I might create a new record through a form for the purpose of testing the deletion. Sometimes, going through the application to set its state is impractical. I then opt to change the state directly by executing database queries or creating files.

Executing the action can be done in many ways. When writing APIs, I usually interact with them by sending HTTP requests. With command-line apps, I execute the process. With web apps that have a lot of JavaScript, I do as much as I can through HTTP, but then I also have a few tests using a browser driver. Browser tests are the hardest to maintain and longest to execute, so I try to keep that to a minimum.

To verify the result, I may crawl the HTML output using a library, verify the JSON, or query the browser driver. Ideally, I don't look at the database content here, because that's not something that a user sees. There are some exceptions where the user is another system that does actually care about the database content.

Who Writes the Tests

If there are no tests yet, then the developer can write some tests to capture the current behavior. These can then be reviewed by a product owner and improved upon.

I like to collaborate with the product owners to write new acceptance tests. It helps to shape a shared understanding before we begin implementation.

Acceptance Versus Unit Tests

Acceptance tests cover use cases. Unit tests cover lines of code. Neither is a substitute for the other. They answer a different question and therefore we need both.

Acceptance tests tell the stakeholders whether the software behaves as they expect. Example: does it reject expired credit cards with a comprehensive error message? These expectations come from the stakeholders, not the developers.

Unit tests tell the developers whether the software's components are implemented correctly. Example: this method will throw an exception if provided a string that is not valid JSON. Some unit-tested code is not even reachable by the application under normal circumstances.

Learn about more topics