BDD Code Review Checklist

Automation Guild Online Conference Top ad

BDD Gone Bad

I started in a new position several months back, and have been working on helping get a Behavior Driven Development process going. Because there are so many scrum teams involved in this new, company- wide initiative, one of the areas in our process that needs help is reviewing each team's BDD statements.

The biggest roadblock we've experienced is that people are treating BDD as an automation framework rather than as a way of having discussions to insure we're developing what the customer wants. A side effect of this is that some of the early BDDs running in the continuous integration environment are suffering from all kinds of random, flaky, unreliable behavior.

Fortunately, we're about to start a brand new project, and in order to try and get things right the first time around, I'm trying to find ways to make the new BDD process better. One method would be to create a high- level BDD code checklist that I'll try to have everyone use before their BDDs are accepted in the CI environment. This is what I have so far:


The BDD Code Review Check List

  • Scenario has run multiple times locally, in a pass status
  • Feature typically contains less than 20 scenarios. If more, you might want to make sure that your feature is not too broad.
  • Scenario Names and Steps make sense
  • Readability of G/W/T
  • Not all tests are GUI tests (if API/web service is available).
  • Scenarios should test critical boundary cases. For example: analyze each Assert() to understand all possible values.
  • All scenarios have been discussed with all stakeholders –BDD is all about conversations.
  • Limited use of technical and UI terms
  • Describes behavior — not technical implementation (declarative style not imperative style used)
  • Each scenario can be executed independently of any other scenario.
  • None of the scenarios contain environment-specific, hardcoded test data (unless environment setup scripts are also provided as part of setup).
  • All verification steps are in the Then steps — not the Given or When.
  • None of the scenarios should be overly long. Typically, each should have less than 15 steps (not considering step multiplication by example tables). Rule of thumb is that an entire scenario can be viewed in the IDE without scrolling.
  • Should this even be a BDD test? Or would a lower-level test (Like a unit test) be better?

Agree/Disagree

Most of the ideas found in the checklist were gleaned from my experience with previous projects, Google searches, The Cucumber Book, and advice from the BDD master consultants (Lance and Wes) at SolutionsIq.

Let me know what you think. Do you agree/disagree with the above, and if so, what do you feel should be on a BDD code review checklist?

5 comments
sb - January 15, 2014

Thanks for your time and effort for this.

We have one banking project coming in.
For this, how is BDD useful ?

What is the REAL benefit you see from your past experience.
Has it –
1. Reduced automation effort?
2. Improved quality for business functionality?

3. Did it really connect Banking domain guys with QA?
4. Who should write them and who should run them?
5. What is the starting point to start this whole process.

What is the time it took for you to had this implemented.

Before I suggest this in any QA meeting, I need to give answers to the questions above.

Best regards

Reply
    Joe Colantonio - January 17, 2014

    Hi – a full response to this would take a couple of post :) So the biggest benefit of BDD is that it improves communication between Users,Product Owners, Dev and Testers. I work for a big company so results have been different from team to team. In general I’ve found that teams that actually design their BDD at a high level are telling me that they have found the benefits very helpful. Teams that are just writing BDD at a low level, treating it like an automation framework are not doing as well. Yes I think it has improved software quality because before we are even writing a line of code we are having discussions to make sure we understand what needs to be developed. Typically the Product owners with QA engineers along with input for the whole team write the actual BDD statements. Developers write the code for the BDD’s step definitions.

    Reply
Nagesh - March 6, 2014

Hello

I have an issue.@facets
The grids in the application are recognized as Winlist.the items in the list are returned as junk values (chineese characters)…If you have seen this issue ..and have a solution ,please get back to me
Thanks in advance.

Reply
Mark Johnson - August 1, 2014

Hi Joe, I really like this checklist. I have a question about one of the points (all verification steps are in the THEN clause). When you go to automate, don’t you need to automate testing the Given clause as well? SO that…”Given a patient has an exam within the past 30 days” is perfectly fine for discussion purposes…for automation, do you need to “Test” this or “Create” this?
Thanks, Mark

Reply
    Joe Colantonio - August 2, 2014

    I would say that the Given would run the code to get your application into a particular state before running the rest of the test. So maybe you have a method named getpatientWithExam would do all the work needed to get a patient with an exam within 30days setup and ready before running your When/Then statements.

    Reply
Click here to add a comment

Leave a comment: