If you have ever dealt with application testing, you have surely come across the Behavior-Driven Development approach. However, the foundations of this methodology aren’t often fully understood and because of this, sometimes it’s used incorrectly.

In this article we will try to present the main assumptions of BDD while keeping in mind its pros and cons and will try to answer the question posed in the title. In other words, we will check whether BDD is really an ultimate solution for all testing problems.

But let us start at the very beginning by recalling what levels of testing can be distinguished.

Levels of application testing

The basic definition of test levels is the so-called V model. A test level is an organized and managed group of tests with a strictly assigned responsibility in the context of the application and system.

Graphical representation of the V model.

From the model above, we can learn that there are at least four different levels of testing. So where should Behavior-Driven Development be applied?

Let’s start again with the definition itself.

What is Behavior-Driven Development?

It is an approach, a process aimed at ensuring high quality of our product by defining specific test cases describing real end-user (or other actor) behaviors and their final effects.

This approach can be reduced to 3 basic stages:

• • Discovery – defining business requirements for the new functionality, discussing detailed behaviors at 3 amigos meetings (more about that later),

• Formulation – documentation of specific requirements in the form of specific scenarios that will ultimately be automated,
• Automation – implementation of test cases and their automation. 

Overall, BDD derives from the methodologies of Test-Driven Development and Domain-Driven Design.

From the former, it adopts the approach of defining test cases before or during the implementation of new functionality, thereby ensuring that the product meets all business requirements.

In practice, this comes down to organizing so-called “3 amigos” meetings involving the product owner (representing the business), the tester, and the developer, during which new functionality is discussed in detail from the product user’s perspective. In the case of a less obvious or more complex topic, the presence of a domain expert may also be advisable.

From the latter methodology, BDD borrows the concept of a ubiquitous language, meaning a shared language for business stakeholders, testers, and developers alike. At the same time, it provides a communication bridge that streamlines collaboration between them.

This is implemented using the language offered by Gherkin, which defines a set of keywords and rules used to create test scenarios.

It may therefore seem, in line with common opinion, that BDD is an excellent methodology that should be applied in every project. Let us then try to verify that view. To begin with, let us focus on the disadvantages of this approach.

Problems with Behavior-Driven Development

Additional workload

Properly defined scenarios can cover all the behaviors and functionalities of an application. However, this comes with additional effort, which can sometimes be disproportionately high.

For BDD to work effectively, it is also necessary to ensure that each time a specific new feature satisfies all expected behaviors. In turn, this involves additional meetings dedicated to every new feature.

Buzzword-Driven Architecture

Since the IT world and the technologies used within it are largely driven by current trends and “fashion,” BDD is often applied in ways that go against its intended purpose. This, of course, stems from a lack of understanding of where and how it should actually be used.

Large projects

In large-scale projects, the effort required to implement all possible test cases can be significant. Difficulties also arise when implementing complex application flows, and executing all scenarios may take many hours.

Gherkin

Despite the benefits provided by Gherkin, it enforces a certain standardized structure for scenarios (including principles such as the Single Responsibility Principle). This is, of course, another advantage, but there are cases where implementing a test scenario while adhering strictly to these constraints may not be optimal (more on this later).

As shown in the overview above, the BDD approach may not always be a lightweight solution. However, most of the mentioned drawbacks result from misunderstandings about what BDD actually is. Let us therefore re-evaluate our initial definition.

What, then, is BDD, and how should this approach be applied in practice?

BDD is a set of rules and principles aimed at producing a high-quality product by defining the expected behavior of an application from the business perspective.

Test stories are described using scenarios that outline the expected behavior of the product from the point of view of the system’s user.

With this in mind, BDD is not necessarily well-suited for implementing system tests.

System tests should still be performed at the code level (with direct access to the current state of the application and fast execution times). BDD, on the other hand, should primarily be applied to acceptance testing (User Acceptance Tests), where the actor has the appropriate domain knowledge and is able to execute complex scenarios, even manually if necessary.

As mentioned earlier, specific scenarios are defined using a standardized syntax. To better understand the strengths and weaknesses of this approach, let us briefly discuss the syntax itself.

Gherkin

Test scenarios are defined using dedicated keywords provided and then parsed by the Gherkin parser, which enables linking the scenario description with the underlying code. In this way, the language serves as a ubiquitous language, understandable to all parties involved in the software development process.

Składnia

We distinguish three types of steps that make up a typical scenario:

• Context (preconditions) – bringing the application to a specific initial state from which the scenario can be executed,
• Events – an action performed by the user,
• Outcomes – verification of the behavior or result after the action has been performed.

The scenario above contains a number of keywords and steps that are interpreted by the Cucumber tool and mapped to their corresponding implementations. Each of the mentioned keywords serves a strictly defined purpose:

• Feature – defines the context being tested. For example, it may describe expected behaviors related to a login page, a homepage, or more specific functionality. Below a brief definition of the scope of a given Feature file, there should be a description from the user’s perspective explaining what behavior is being verified and what the expected outcome is (in a general sense within that context).
• Scenario / Scenario Outline – marks the beginning of a specific scenario along with its short description. The difference between these keywords is that Scenario is used to define a single scenario, while Scenario Outline allows a parameterized definition that is executed for all listed parameter sets.
• Given – defines a step that performs some initial setup, bringing the application into a starting state from which the scenario can be executed.
• When – specifies the action performed by the user, which constitutes the core element of the scenario.
• Then – steps used to verify the behavior, result, or functionality of the scenario. For example, after clicking a button, it may verify whether an expected element appears on the screen.
• And – represents the same type of step as the preceding keyword, used to avoid repeating the same keyword in each line and to make the syntax read more naturally in English.

A more detailed description of the syntax can be found in the Cucumber documentation.

Problems with Gherkin

Gherkin enables a descriptive way of defining test cases by enforcing a specific syntax that is understandable both to business stakeholders and technical team members. Despite its clear advantages, this approach also has its drawbacks.

Single responsibility principle

When defining a specific Scenario, the correct order of steps must be maintained: Given ➝ When ➝ Then. Only this sequence is allowed, and steps that break this order should not be used interchangeably.

This is, of course, a consequence of maintaining a consistent syntax understood by Gherkin. In practice, however, we may encounter scenarios where this limitation results in additional overhead, particularly in terms of test execution time. In such cases, test scenarios should be limited to those that are critical from a business or end-user perspective.

The Single Responsibility Principle further states that individual steps should be responsible for only a single action. In other words, the word and should not appear within a step definition itself. For example, “Given I am logged in and I am on a Home page” violates SRP. Instead, this definition should be split into two separate steps.

Imperative approach

Another common issue is the incorrect definition of the scenarios themselves. To ensure the greatest and longest compatibility of a given scenario with the application, its definition should be independent of the application’s implementation.

A declarative approach should therefore be used. Unlike the imperative approach, we should focus on describing what behavior is expected in a given context rather than how the user performs each action.

For example, when testing the login functionality, we should not define every individual action performed by the user, but rather describe what outcome is expected.

Example of an incorrectly written scenario (imperative style):

• Given: I opened the login page
• When: I enter the username in the first text field
• And: I enter the password in the second text field
• And: I click the Login button
• Then: I see the personalized homepage

Example of a correctly written scenario (declarative style):

• Given: I opened the login page
• When: I log in with valid credentials
• Then: I see the personalized homepage

Summary

Taking all aspects of BDD into account, it can be concluded that this approach should be used only for acceptance testing. It should not be applied to system testing, because in that context, the advantages of BDD effectively become a burden and may eventually work against us.

An additional argument in favor of using BDD only for acceptance tests is the fact that, when tests fail, the root causes can be difficult to identify.

The Behavior-Driven Development approach is excellent for gathering requirements related to an application’s business logic and for validating the final implementation. Unfortunately, this methodology is often used in ways that do not align with its intended purpose, which tends to obscure its actual strengths.

To paraphrase the timeless phrase “it depends,” this methodology should only be adopted after a thorough analysis of the product itself and the business expectations, while taking into account all the aspects discussed in this article.

Useful links:

1. Cucumber documentation, https://cucumber.io/docs/bdd/>
2. Gherkin syntax, https://cucumber.io/docs/gherkin/
3. BDD guide, https://inviqa.com/blog/bdd-guide
4. How to write good Gherkin, https://medium.com/@wladislavk/how-to-write-good-gherkin-514aea53948c
5. BDD 4 rules writing good Gherkin, https://techbeacon.com/app-dev-testing/better-behavior-driven-development-4-rules-writing-good-gherkin
6. Introducing BDD, http://dannorth.net/introducing-bdd/
7. BDD 101, https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/