Ultimate Automation Testing Tools Guide (Over 40 Tools Listed)

Automation Testing Tools

Ultimate Automation Testing Tools (Over 40 Tools Listed)

Finding the right testing tool from all the available automation testing tools is tough. I’ve interviewed over 200 testers and developers on my TestTalks podcast. Based on this treasure trove of test automation awesomeness, I’ve taken the top functional automation testing tools mentioned by our experts and listed them below.

This list includes both free software testing tools and vendor automated testing tools but many of the paid tools offer a free trial version of their solutions. Also, this automation tools list is in no particular order (placement does not indicate ranking)

How to Select an Automated Test Tool?

I don’t think there is one tool that all teams should use for software testing. What you decide to choose should be based on your team’s unique needs.

I always recommend regardless of what automation testing tools are selected to do a two week proof of concept (POC) and make sure the solution actually fits in with your team’s development workflow. For more details on this and other automation best practices check out my Ultimate Test Automation and Best Practices Guide.

Selenium WebDriver

OK–this is a no-brainer. As of now, Selenium is the leading automated regression testing option for browser-based test automation. Selenium is not a standalone-testing tool. It’s an API that allows you to programmatically write tests using most programming languages to interact with a browser the way a real user would.

Cucumber

Cucumber was not designed to be an automation test tool; it’s a tool that helps conversation and removes ambiguity from requirements before your team invests time in writing code.

However, it does have a unique benefit; once you’ve got test scenarios created, you can generate what they can “executable specifications,” using whatever automation library is supported by the programming language that you use.

Although Cucumber’s main benefit is helping communication and collaboration, many teams use it simply as an automated testing tools framework. If you do decide to use Cucumber I highly recommend that you read The Cucumber Book: Behaviour-Driven Development for Testers and Developers.

Serenity

If you need an automated test tool Java-based framework that integrates with behavior-driven development (BDD) tools such as Cucumber and JBehave, and lets you keep your test scenarios at a high level while accommodating lower-level implementation details in your reports, Serenity BDD (formerly Thucydides) might be the tool for you. It’s designed to make writing automated acceptance and regression tests easier.

Serenity acts as a wrapper on top of Selenium WebDriver and BDD tools. It makes writing BDD and Selenium tests easier because it abstracts away much of the boilerplate code you sometimes need to write to get started. Serenity also offers plenty of built-in functionality, such as handling WebDriver management, managing state between steps, taking screenshots, running tests in parallel, facilitating Jira integration and more—all without having to write a single line of code.

Serenity creates highly detailed reports. Out of the box, it creates living documentation you can use not only to view your Selenium BDD test results but also as documentation for your application.

It also has an extra annotation method called @Step. When used correctly, @Step gives you an extra level of abstraction that makes your tests more reusable and maintainable.

Serenity takes care of many things you would typically have to code from scratch if you were creating your own BDD framework. It’s one of the better open-source frameworks available, although I might be a bit biased; it’s the tool I use for my day-to-day test automation efforts.

John Fergusan Smart also wrote an excellent book on BDD in Action that includes tips and tricks for using Serenity and other automated testing tools.

Unified Functional Testing – Quick Test Professional

Unified Functional Testing (UFT) is a test automation tool that was initially developed by Mercury Interactive and was named Quick Test Professional (QTP). It was later acquired by HP and rebranded as Unified Functional Testing. Micro Focus recently purchased it, so Lord knows what the marketing geniuses there will end up calling it.

Before Selenium became the defacto browser-based automation solution, QTP/UFT was one of the most popular automation testing tools on the market.

It’s still is one of the top automation tools for folks who need a tool that automates more than just web browsers. Not only does it have web testing functionality, but it also has support for other diverse technologies like WPF, NET, Oracle, PeopleSoft, SAP, Siebel, Terminal Emulators and more. Using a tool like UFT allows you to automate complete end-to-end testing workflows that might have interactions with non-web-based applications.

LeanFT

Lean Functional Testing (LeanFT) essentially combines the best of the vendor-based and open-source worlds by morphing Selenium with some essential functionality currently found in UFT.

HP built LeanFT from the ground up to address the growing modern automation demands of continuous testing, mobile a, d DevOps. What’s especially cool is that LeanFT fits into your developer’s ecosystem, so your developers can still use their favorite IDEs, programming languages and unit testing frameworks–not to mention the added benefit of access to HP Web/Windows-based technologies that Selenium doesn’t provide.

PhantomJS

PhantomJs

PhantomJS is an open source browser that runs headlessly. It gives you the ability to create fast scripts using its JavaScript API to automate a bunch of cool things including the ability to run functional tests using frameworks like Jasmine and WebDriver as well as do screen capturing and network monitoring.

We’ll have to wait and see what happens to Phantom after the release of headless Chrome, but as of this post, it’s still very much alive.

Automation Anywhere

Automation Anywhere has a marketing twist when describing its automated test tool solution. They explain that their automation uses “software robots” to automate scriptless end-to-end tests. The three main areas AA highlights for their product is:

  • RAP – automate every rule-based process
  • Cognitive – self-learning automation
  • Analytics – real-time, operational intelligence with enterprise-wide business

CodedUI

CodedUI is Microsoft technology for test automation for the Windows platform.

You can automate WPF, Windows Presentation Foundation add-ins, Windows Forms applications, Windows 32 apps, mobile applications and of course, browser-based automation in IE and Chrome and Firefox with a Selenium plugin.

CodedUI is an API you can program against using Visual Studio and languages like Visual Basic and C#.

TestComplete

SmartBear’s TestComplete platform helps you create repeatable and accurate automated tests across multiple devices, platforms and environments quickly and easily.

The latest version of TestComplete is even better due to its support of Selenium WebDriver and unit testing. Best of all, you can choose from script-free, drag and drop functionality or JavaScript, Python, VBScript, JScript, DelphiScript, C++Script or C#Script as a scripting language.

Sikuli

SikuliX can help automate anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. Sikuli uses image recognition powered by OpenCV to identify and control GUI components.

AutoIt

AutoIt v3 is a freeware, BASIC-like scripting language designed for automating the Windows GUI and general scripting. While not the most robust tool from all the other automated testing tools but many teams integrate AutoIt with Selenium to work around non-browser windows that appear in an automated test workflow.

Applitools

While most automated testing tools on this listed were created for functional testing Applitools was developed from the ground up, specifically for visual validation. Visual Software Testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, Visual Testing focuses on proving the layout and appearance of each visual element of the UI and the UI as a whole.

Protractor

Protractor is a layer that sits on top of Selenium used for end-to-end testing for Angular.

It runs Selenium in the background, but it provides all sorts of really cool hooks into your AngularJs application. As a result, you can start grabbing your data bindings, and you can begin using some Angular goodness like ng-repeats, etc.

You can also grab those elements with the Angular semantics and you can add to them, read from them, and verify that your page has the correct data.

If you’re looking for automated testing tools that were made specifically for AngularJS application you should check this out.

Watir

Watir is an open-source Ruby library for automating tests. Watir interacts with a browser the same way people do: clicking links, filling out forms and validating text.

PowerShell

PowerShell is a Microsoft product. Anytime you want to use the command line or automate anything inside of the Windows ecosystem (or pretty much any Microsoft product) you’ll want to go with PowerShell. While not usually considered in a list of automated testing tools I still consider this a viable option for DevOps based automation. There is even a BDD framework for it called Pester that some teams love.

QASymphony qTest

qTest Scenario is a JIRA plugin that was designed to help you quickly manage your BDD/TDD end-to-end workflows, including creating and executing tests as well as acting as a centralized repository for test results, test defects management and resolution. They developed qTest Scenario with some essential team benefits in mind, like:

  • Helping your teams work faster
  • Increasing productivity and collaboration
  • Helping you focus more on your end users
  • Helping you scale your test-first efforts across your organization

White Framework

White Framework .Net WPF
Are you a QTP/UFT engineer who has wished there was a way to automate .NET and WPF applications using Visual Studio and C# rather than QuickTest Professional or Unified Functional Testing?

Do you need to automate a thick client application like WPF, Silverlight, Win32 or WinForms? If so, then White may be the automation framework for you.

White is an open-source library from Test Stack. Like CodedUI, it has a more narrow scope; it only targets desktop applications, not web applications. Both CodedUI and White are based on UI Automation Library for Microsoft.

There are not many automation testing tools that handle .NET applications so this is a valuable tool for folks that have to test such technologies. It’s also easy to get started using White.

Appium

Appium is a free, open-source, cross-platform mobile UI automation framework. It allows you to write UI tests for your mobile apps, Android iOS and some other OSs. It also lets you create test scripts using the Selenium JSON writer you know and love from web automation.

Appium is becoming an industry standard for mobile testing, much as Selenium WebDriver became the standard for browser-based automation a few years ago.

Robot Framework

If you want to use Python for test automation, you can’t go wrong with the Robot Framework. This mature product, created for testers, uses a keyword-driven approach to make tests readable and easy to create. It also includes many test libraries and other tools you can use. The Robot Framework is Python-based, but you can also use Jython (Java) or IronPython (.NET).

While the Selenium WebDriver library may be the most used external test library, Robot Framework can test things other than websites including FTP, MongoDB, Android and Appium. In addition to all this open-source awesomeness, it has a lot of APIs that help make it as extensible as possible.

The keyword approach used by Robot Framework is excellent for testers who are already familiar with other vendor-based, keyword-driven test tools, making the transition to open source much more comfortable for them.

If your team is mostly made up of testers, Robot Framework is an excellent option for your automation framework.

CA Agile Designer

The CA Agile Requirements Designer automation tool takes a unique approach to test automation. Rather than focusing on the code, it creates automated tests automatically using model-based requirements. Using a model-based approach is cool because it can auto-generate and update test cases whenever a change is made to the model.

GraphWalker

If a model-based approach for test automation sounds like a solution that would work for your team, but you have a tight budget, check out GraphWalker.

GraphWalker is an open source Model-based testing tool for test automation. It’s designed to make it easy to design your tests using graphs.

Ranorex

Ranorex test studio is a functional test automation tool that is driven by the user interface. It helps you automate anything when it comes to web applications, mobile applications or desktop-based applications.

HipTest

HipTest is a new breed of test management tool and BDD Framework. Many teams that use BDD have a hard time managing all their manual, exploratory and automated tests in one place.

To address this issue, HipTest has developed a single platform to help with those BDD testing efforts. It’s a simple platform in the Cloud that supports real-time collaboration between product owner, testers, and developers.

TestPlant

TestPlant is one of the only test automation tools on this list that has strong support for Apple’s platform. Because of its unique, image-based recognition abilities, it’s known for its ability to test hard-to-automate applications; especially those with object recognition issues. Unfortunately, however, anyone who has done image-based, functional test automation understands how difficult these types of tests can be to maintain, and some customers have noted that as an issue.

TestArchitect

TestArchitect is a codeless, keyword-driven automation framework that makes it easy for teams to create and manage large test suites.

When I asked Hung Q. Nguyen the co-founder of LogiGear why he created Test Architect he mentioned that creating a test is one thing. But tests change and they change very rapidly. How do you rapidly change the test and maintaining it and updating it so that you can scale? That was the thinking behind the design of Test Architect.

Galen Framework

If you need to focus your automation efforts on user experience design (UX) or layout testing, Galen Framework might be a perfect fit for your needs.

Created specifically for UX testing, Galen has a special syntax and rules you can use to describe and verify your application’s layout. It also lets you specify your browser size, then run tests against your software to check layout specifications. Galen tests also generate detailed HTML reports with screenshots, and it includes a visual image comparison with a cool heat map feature.

You can write tests in a simple text file using the Galen syntax, JavaScript, or Java.

Tricentis

Tricentis is self-billed as “the continuous testing company.” They have a strong focus on supporting Agile testing and CI processes. Their solution Tosca boasts a scriptless technology that is used for functional test automation

Espresso

Espresso is a native mobile testing framework for Android testing.

Testim

Testim leverages machine learning to speed up the authoring, execution and–most importantly–the maintenance of automated tests. Their goal is to help you to start trusting your tests.

Testim focuses on reducing your flaky tests and test maintenance, which they see as one of the most significant challenges for most organizations.

Oren Rubin, co-founder of Testim, mentioned in a recent TestTalks interview that the firm’s primary goal is to help liberate test automation from the exclusive realm of developers and make it simple enough for anyone on the team to create.

Cypress.io

Compared to most of the other tools on this list, Cypress is a more developer-centric test automation framework that focuses on making test-driven development (TDD) a reality for developers. One of its design principles was to be able to package and bundle everything together to make the entire end-to-end testing experience pleasant and simple.

It has a different architecture than Selenium; while Selenium WebDriver runs remotely outside the browser, Cypress runs inside of it.

This approach delivers more consistent results since it’s able to understand everything that happens inside and outside the browser. It also gives you native access to every object without requiring you to deal with object serialization or over-the-wire protocols. You’re pulling your application into Cypress.

As a result, it can synchronously notify you of every single thing that happens inside the browser so that you have native access to every distributed object model (DOM) element.

Cypress also makes it easy to simply drop a debugger into your application, which in turn makes it easier to use the developer tools while you’re developing. Of all the automation testing tools listed here, Cypress is the one that I hear most developers prefer to use.

WebDriver.IO

WebDriver.IO is another browser-based automation testing tools module but this one is for Node.JS that makes it possible to write super easy Selenium tests in your favorite BDD/TDD test framework that will either run locally or in the Cloud using Sauce Labs, BrowserStack or TestingBot.

I like how Kevin Lamping in TestTalks Episode 179 describes the chief benefit of using WebDriverIO as being like the SeleniumJS binding on steroids!

WebDriverIO is agnostic about the test framework you use so that you can leverage tools like Cucumber, Jasmine, and Mocha+Chai.

If you’re a QA engineer who’s just getting started, WebDriver.IO might be the way to go because–in my opinion–JavaScript with WebDriverIO is a lot easier to pick up than Java.

Mabl

Mabl is one of the newer automation testing tools on the market that focuses on the hot trend of using AI and machine learning to help folks with their testing efforts. In a nutshell, to get started you merely point Mabl to your application. It will then crawl your app’s screens and begin to run default tests that are common for most applications.

Along the way, it also uses machine learning algorithms to improve test execution and defect detection. Using the Mabl trainer chrome extension, you can also train Mabl to learn journeys in your application which describe the functionality Mabl will test.

Jest

There are tons of JavaScript automation frameworks out there, but Jest seems to be the most developer friendly. It’s used by Facebook to test all JavaScript code including React applications. It also is advertised as having zero configuration setup time, so it’s easy to get started with.

Katalon Studio

Katalon Studio is an open-source solution that includes many features you would typically only find in a paid vendor tool like Record and Playback, reporting, Object Spy, Object Map and more. You can use it to create automated tests for Web, API and mobile apps. It even runs on both Windows and Mac.

Progress

For those of you who may not be familiar with Progress, they recently acquired Telerik test studio, which is the home of the favorite free debugging tool Fiddler. I also know a few test engineers who use Progress’s Test Studio as a front end for their Selenium test automation efforts.

Strengths of Progress are its integration with Visual Studio and its supported languages.

Sahi

Developed for testing web applications, Sahi offers both an open source and a pro version. It works as a proxy server you can use within a browser. From the Sahi dashboard, you can launch the browser you want to test.

The Sahi controller guides you through the recording and capturing of the interaction you want to perform against the application. When you hover over any element in the Sahi browser, for instance, it shows you all of the actions you can perform against that element.

You can also use the controller to play back your script. Sahi’s Record and Playback functionality make it easy to get started with test automation.

Gauge

Gauge is one of the newest open-source test automation tools for Mac, Windows and Linux. It was developed by the folks at ThoughtWorks–the same company that created Selenium.

Practitioners of TDD and BDD will appreciate Gauge’s focus on creating living and executable documentation.

Gauge automation tests, called “specs,” are written in a simple markdown language syntax using Java, Ruby, and C# within developer IDEs such as Eclipse and Visual Studio. You can also extend Gauge’s functionality with its support for plugins.

The 2017 Online Automation Guild conference had an awesome hands-on session on  cross-bowser testing with Gauge and Selenium WebDriver that you can still get recordings for.

RedwoodHQ

RedwoodHQ takes a slightly different approach than the other tools on this list. It creates a website interface that lets multiple testers work together and run tests from one Web-accessible location.

You can write tests in Java/Groovy, Python and C# to test Web applications with Selenium, APIs or databases using their web IDE, which also includes built-in Git integration.

You can leverage RedwoodHQ’s existing action keywords to start creating and modifying tests quickly. To create a test script, simply find the action you want to perform, drag it into your test case, and enter the parameter values it expects.

The built-in IDE lets you create and modify test cases and actions as well as execute tests. You can also run your tests against predefined machines in parallel and see a running history of all your previous tests runs.

RedwoodHQ is tester friendly and has many built-in features you can use to create complete test automation and test management solutions for your teams.

WinAppDriver

Windows Application Driver (WinAppDriver), developed by Yousef Durr at Microsoft, is the Windows implementation of Appium to automate Windows applications. This is really cool because it shows the next evolution of Appium to expand beyond just mobile applications and more toward automating all the things the Appium team is calling the StarDriver vision.

IBM Rational Functional Tester

Like most companies, IBM’s test portfolio has grown with the acquisition of tools like Rational and Green Hat. Much of the strength of Rational Functional Tester comes from its support of numerous technologies like Siebel, PowerBuilder, Java and can run on Windows, Mac, and mobile platforms.

Test.AI 

Test.AI is billed as a tool that will add an AI brain to Selenium and Appium. It was created by Jason Arbon, co-author of How Google Tests Software and the founder of Appdiff. Tests are defined in a simple format similar to the BDD syntax of Cucumber, so it requires no code and no need to mess with element identifiers.

The AI identifies screens and elements dynamically in any app and automatically drives your application to execute test cases.

Worksoft

Some time only developer ERP applications. In those cases, a tool like Worksoft might be the perfect choice for you. Worksoft is well known for its ERP business end-to-end solutions.

Oracle Application Testing Suite (OATS) 

Does your team do a lot of Oracle-based software development? Need a tool to help test your regression and performance test your Oracle applications or databases?

Oracle Application Testing suite contains a bunch of integrated products to help with Oracle Functional Testing for regression testing of web applications and Oracle Load Testing for performance testing of your Oracle packaged solutions.

TestingWhiz

TestingWhiz motto is Code Less, Test More. They have solutions for Web Test Automation, Mobile Test Automation, Automated Database Testing, Mobile Test Automation and more.

Automation Testing Tools

What About Non-functional Automation Testing Tools?

For other non-functional automation testing tools check out:

How to Stay Up To Speed with the Latest in Automation Testing?

AutomationGuildLogo

If you feel overwhelmed with staying up to date with all the latest automation testing tools and best practices check out my annual online conference dedicated 100% to just automation testing — Automation Guild.

20comments
jacob samuel - March 28, 2018

– > Which is the best tool to automate mobile apps with BDD?
-> Is there a way I can convert my POM scripts to BDD? (weird question but is there an option?)

Reply
    Joe Colantonio - March 28, 2018

    Hi Jacob not sure I understand your use case. Is this to create BDD test that validates the correct versions of dependences in your POM file?

    Reply
Robert Linton - March 28, 2018

what about SILK Test ? not a player ?

thanks,

Reply
    Joe Colantonio - March 28, 2018

    Hi Robert! Hmmm, is Silk Test still being used by anyone? :) No one has mentioned it on TestTalks and its a little confusing because MicroFocus now owns both Silk and UFT so not sure what direction they are going in.

    Reply
J - March 28, 2018

Check out Usetrace.

Reply
Toyer Mamoojee - March 28, 2018

Great list Joe. I have probably tried using about 80% of these Automation tools in some capacity at some point. Hopefully i can try out the remaining 20% or so in future. I see other popular tools which is used alot nowadays missing from this list. Nightwatch.js and TestCafe

Reply
Guljeet Nagpaul - March 28, 2018

Check out accelQ y’all.
Visual Application map, AI based Automated test design, Autonomic Natural language automation engine. web, API any service backend validations all in the same flow !

Reply
Jan Jaderny - March 29, 2018

What about TestCafé?

I’ve come across it in some dev’s article that listed it as the best tool for Javascript end-to-end testing – should be a competitor to Cypress.io – and seems to have similar number of stars on Github :)

Reply
    Joe Colantonio - April 24, 2018

    Hi Jan – I only included tools on this list that past TestTalks guests have mentioned. That being said I recently interviewed someone at TestCafe and will be releasing their episode soon.

    Reply
Nathan - March 31, 2018

Hi Joe,

Have you heard of TestCafe? I’ve used it and it is awesome!
https://devexpress.github.io/testcafe/

Reply
Nathan - April 1, 2018

Hey Joe,

Have you heard of TestCafe?
It’s a pretty great contender.

https://devexpress.github.io/testcafe/

Reply
    Joe Colantonio - April 10, 2018

    I just interview the CTO of DevExpress the folks that make TestCafe and will update this post with more info soon.

    Reply
Kelvin Reid - April 3, 2018

Thank you for the list of automation tools, but I was expecting XCTest to be on this list. What are your thoughts about it?

Reply
    Joe Colantonio - April 24, 2018

    Good point – I missed it. I will add soon

    Reply
Frank Sullivan - April 9, 2018

Hi Joe,

How about Appium Studio as an option for app testing on Android and IOS devices?

Thanks!

Reply
    Joe Colantonio - April 24, 2018

    Hi Frank! This list is based on tools that past TEstTalks guest have recommended. I haven’t had a guest talk about Appium Studio before. I’ve also never heard of it till now. I would have to find someone to interview about it then add it to the list :)

    Reply
Colin Hammond - April 19, 2018

A great list, but missing automated requirements testing.

Forgive the self promotion, but you should consider adding ScopeMaster.com

Reply
    Joe Colantonio - April 24, 2018

    Colin this list is based on tool guest on TestTalks have recommended only. No one has ever mentioned an automated req testing tool before

    Reply
LarryG - April 20, 2018

Hi Joe,

I see Nightwatch.js on the list but that framework is not really being maintained, do you know what the future for it is.
I know there are a lot of projects out there thats still using it, hence the reason for my question

LarryG

Reply
    Joe Colantonio - April 24, 2018

    HI, LarryG I have not heard or seen anywhere that Nightwatch.js is being deprecated. I will let you know if I hear anything officially. Looking at the GitHib repository it has been updated in the past few months

    Reply
Click here to add a comment

Leave a comment: