In DevOps for a build to be deployed into production, it needs to pass a series of checks. The objectives of these checks are to ensure that changes from new or revised code are consistent with the existing build. Acceptance testing is an excellent choice for promoting cooperation between development and operations teams because it provides a common framework.
Acceptance testing is concerned with assuring stakeholders the software will work as they require it. Acceptance testing is implemented on the basis of scenarios specified by software end users. Acceptance testing differs from unit testing. Whereas unit testing is concerned with ensuring pieces of code are functional acceptance testing is concerned with ensuring the end user needs are met.
A good acceptance test has several elements, which are highlighted below: –
• Easily maintainable – when we add or update code that leads to a test break, we need to be able to quickly identify and correct the problem.
• Repeatability – the results yielded by the test should be consistent and repeatable
• Relevance – the test should be able to accurately capture the end user’s needs
• Easily developed – because testing will involve writing many tests, the tests should be easy to write
• Privacy – the test should work independently from other tests and systems
One benefit of a well implemented test acceptance is transparency. This is because the features to be delivered as well as areas that function and those that do not function are known. Therefore, the operations team is aware of features deployed in production and the support effort required. Another benefit of an acceptance test is it acts as a communication vehicle. This is because acceptance testing involves test execution, result sharing and communication.
Before you can begin working on your acceptance tests, you first need to create an acceptance criteria. Best practice in acceptance testing involves using simple to understand examples to explain system functionality. Let us consider an example of an online job board to illustrate how to come up with an acceptance criteria.
On our job board example, we would express our requirements using the statements below:
• To quickly find a relevant job posting, a job seeker needs to be able to browse the jobs in each category, in each city and by each company.
• To reduce the amount of effort of searching for a job, a job seeker needs relevant job recommendations based on their CV
• To make navigation easy, we need to provide links than send users to different areas of the website
• Finally, we need to enable the end user to search for their preferred jobs
After identifying end user requirements, we need to come up with acceptance criteria. Examples of acceptance criteria are listed below.
• To quickly find a job, our end users need to be able to browse jobs by category
• Our end users need to be able to use keywords to list all the jobs they are interested in
• Display all the jobs in “Administrative Assistant” category
• Display all the jobs that can be found in “Chicago” at “United Health Group”
When we give examples, we present a clear view of the requirements that need to be met. Use of examples also guides the discussion on requirement scope and details.
After identifying our acceptance criteria, we need to provide more structure to them using the “given-when-then” approach. When giving the acceptance criteria structure emphasis is on the actions the user will perform and the outcomes expected from the actions. If we take one criterion from the list above, the structured criterion would be formulated as shown below.
Given that Sammy is searching for an entry accountant job, browsing jobs in the “Entry Level” category using “accountant” keyword should list all jobs related to accounting.
When giving structure to our acceptance criteria, we are only concerned with the value provided to end users without being concerned with feature implementation.
After coming up with our acceptance criteria the next step is automating them. In the automation test, we incorporate acceptance tests into the build process that is already automated. Two tools available for automated acceptance testing are Jbehave and Thucydides. Thucydides is an open source framework that simplifies definition, implementation and reporting about acceptance criteria. Implementation of Thucydides testing can be done using Junit, easyb or JBehave frameworks. In this post, our discussion will be limited to JBehave.
JBehave is an open source solution for automating acceptance criteria using test scenarios expressed in the “given-when-then” structure. We considered an example of writing a test scenario earlier using this approach. All the scenarios you write will be placed in a .story file. At the beginning of your .story file you can include a narrative to provide background information about the tests and objectives. To implement your stories you write classes and methods using a JVM language such as Java or Scala. Another programming language that can be used to implement stories is Groovy.
A method implementing the story we considered earlier is shown below.
public class SearchSteps {
@Given("Sammy wants to search for $a job")
public void sammy_searching_for_a_job(String gift) {
// test code
}
@When("When she looks for jobs in the $category category containing $keyword in $region")
public void looking_for_an_ad(String category, String keyword, String region){
// more test code
}
}
Thucydides and JBehave work together to support you in acceptance testing. To use them in your project, you first need to include Thucydides JBehave plugin in your project. If you are using Maven, you just need to include the dependencies in your pom.xml as shown in the snippet below.
<dependency>
<groupId>net.thucydides</groupId>
<artifactId>thucydides-core</artifactId>
<version>0.9.275</version>
</dependency>
<dependency>
<groupId>net.thucydides</groupId>
<artifactId>thucydides-jbehave-plugin</artifactId>
<version>0.9.275</version>
</dependency>
In this post, we introduced acceptance testing and explained its role and benefits in a project. We highlighted important elements of a good acceptance testing criteria. We used an example of a job search website to illustrate how to come up with requirements and acceptance test criteria. Finally, we discussed how Thucydides and JBehave are used in automating acceptance testing.
