The #1 Killer of Selenium Script Performance and Reliability

Automation Guild Online Conference Top ad

The #1 Killer of Selenium Script Performance and Reliability

Do your scripts suffer from the following automation test flaky symptoms?

  • Test randomly fail
  • Works on your machine but not on another machine
  • Test take a long time to run
  • False positives/negatives results
  • Unreliable test run in CI have you contemplating a new career as a sheep herder

If so, your test might suffer from a common under-diagnosed automation disease known as Stinky Synchronization Syndrome.

One of the biggest killers of test automation script reliability is the failure to use proper synchronization/wait points in Selenium. Read the rest of this post to learn how to treat this world wide epidemic.

Two Types of Waits

In Selenium there are two types of synchronization/wait methods that can be used:

  • Explicit Waits ßPreferred Method
  • Implicit Waits

Why use Synchronization Points?


In test automation, this concept is often referred to as synchronization. Adding synchronization points in your script facilitates your test when you need to wait for a response from an application.

This is important, because when testing a web application, server response times can vary for any number of reasons. So your script may pass or fail randomly depending on how fast the response occurs.

To help make your scripts more reliable, you should get into the habit of using synchronization points!

For example, some applications may use AJAX to populate data. More often than not, an Element not found error will occur when running a Selenium test against these types of applications. This is due to issues with the element not being loaded into the DOM because it's in the process of being rendered in your browser. One way to work around this issue is to use an Explicit Wait to wait until the element appears before continuing to the next test step.

Don't use Thread.Sleep!

Using Thread.Sleep is one of the biggest causes of slow running tests that I've seen. Because it's so easy to use, it's tempting to use it in order to quickly create a test script. However, it will cause big performance issues later on — especially when you start running multiple tests in a continuous integration environment. DO NOT USE Thread.Sleep unless you literally have to wait X amount of minutes for an event to occur before your test script proceeds to the next step.

As a matter of fact, I think Thread.Sleep is such an assiduous method that in my test environment I'm going to try to see if CI's code coverage tools can flag all instances of its use.

The problem with Thread.Sleep is that it will wait the specified time, irrespective of the object state. So if you have a sleep set for five seconds, and the object you're waiting for appears in one second, it will still wait the full five seconds before continuing. These little pauses add up quickly, so please do not use.

Explicit Wait

This is the preferred method to use for synchronization. Using Explicit waits allows you to tell Selenium to wait only until a certain condition is met. Once the condition is met, the tests will proceedto the next step.

So, if you set a wait for 20 seconds, and the object appears in two seconds that means the execution processed to the next step after two seconds rather than waiting the full 20 seconds. To illustrate, let's take a look at a real example:

public boolean wait_for_logo(){
 try{
      WebDriverWait wait = new WebDriverWait(getDriver(),10);
      wait.until(ExpectedConditions.visibility(joeLogo));
      return true; 
    }catch(org.openqa.selenium.NoSuchElementException e){
      return false;
    } 
}

This waits up to ten seconds for the expected logo image to appear before throwing a TimeoutException; or if it finds the element, will return it in zero to ten seconds. WebDriverWait, by default, calls the ExpectedCondition every 500 milliseconds until it returns successfully. A Boolean value (true or false) is returned for this ExpectedCondition type.

The ExpectedCondition class gives you a bunch of cool predefined conditions — like waiting for elements to be present/not present, visible/not visible, clickable/non-clickable and much more:


Just Because a Test Passes doesn't mean it's working

A good thing to remember, when creating an automated test, is that just because your test passed X number of times doesn't mean it's really working. You should run your test multiple times in a row to verify that does not randomly fail due to timing issues. Always try to use synchronization points in places in your script that you know may be problematic.

Implicit Waits

The other method for synchronization is using Implicit Waits. Basically, an implicit wait tells WebDriver to poll the DOM for a certain amount of time while trying to find an element or elements if they are not immediately available. The default setting is zero. Once set, the implicit wait is set for the life of the WebDriver object instance. For example:


In the above code, if an element cannot be found then the findElement would take a full 10 seconds before throwing an Exception.

This really isn't optimal since it will apply every time WebDriver looks up an element, which can really slow down the tests in other places. Also using Implicit waits makes it harder later on to fix synchronization issues. The more refined and recommended approach to use is Explicit Waits.

Conclusion

In summary, DON'T USE THREAD.SLEEP! Use Explicit waits instead. Want to hear more? Check out my podcast TestTalks episode 6: The Top 6 Test Automation Best Practices

3 comments
Dev - May 9, 2014

Hi Joe, Nice article about Synchronization point. I need small clarification on wait statements used in QTP and Selenium.
Is the implementation of Wait statement in selenium is different from Wait statement used in QTP ? In QTP wait will wait for full amount of time unlike wait statement in selenium. Please clarify me.

Reply
    Joe Colantonio - May 13, 2014

    It depends on what type of wait statement you use in Selenium. The equivalent of QTP wait in Selenium would be Thread.Sleep(n). That would what the full amount of time since its a hard wait. Just like in QTP you want to avoid using hard waits

    Reply
mutha - May 9, 2014

Hi Joe, I am a beginner in qtp. I am facing issues like false positives or negative results in my test reports. can you please suggest me how to overcome these issues in qtp.

Reply
Click here to add a comment

Leave a comment: