We had a great opening keynote at the Automation Guild Conference by Adam Carmi. He is the co-founder and CTO of Applitools, which is a cloud service provider for automated visual testing. Adam’s session was about Advanced Test Automation Techniques for Responsive Apps and Sites.
A question that came up more than once was “Are there any FREE open-source visual validation tools options? “ (FYI–for those of you that missed it, you can still register to get all the Automation Guild Conferences sessions and recorded Q&A videos [Click here to get them now])
Based on this question, I decided to compile a list of 21 free tools you can use for your visual validation testing efforts.
If you don’t know what visual validation testing is, check out my video on Visual Validation Testing using Selenium & Java – Mini Course.
FREE Visual Validation Tools
I came across this list in one of Adam Carmi’s other presentations, but I couldn’t find a post with link and info about each tool. That’s what I’ve done here for you.
Except for FBSnapshotTestCase, as far as I can tell all tools only work for website and not Mobile apps. That’s another reason why I like using a paid solution like Applitools that works with everything.
Although all the tools share the same workflow, they come in two flavors:
- Code/Script Based
- Configuration Based
What are the Top Visual Validation Tools for Testers?
Most of you are probably wondering what the top automaton frameworks for Visual Validation testing actually are. Here’s what I found. I ranked the frameworks in order of popularity based on GitHub star numbers.
Also, the list is broken up into the two main approaches used for visual validation testing: code based and configuration based.
Code/Script Based Visual Validation Tools:
1. PhantomCSS (UJs CapserJs)
PhantomCSS takes screenshots captured by CasperJS and compares them to baseline images using Resemble.js to test for rgb pixel differences. PhantomCSS then generates image diffs to help you find the cause.
¤ 4170 Github Stars
2. FBSnapshotTestCase (Uses XCTest)
FBSnapshotTestCase uses a “snapshot test case” that takes a configured UIView or CALayer and uses the renderInContext: method to get an image snapshot of its contents. It compares this snapshot to a “reference image” stored in your source code repository and fails the test if the two images don’t match. Check out Writing your first FBsnapshot testcase from TestTalks alumni Andy Obusek.
¤2,222 Github stars
3. Gemini (Uses JS DSL)
¤784 Github stars
4. Needle (Uses Python)
Needle is a tool for testing visuals with Selenium and nose.
It checks that visuals (CSS/fonts/images/SVG/etc.) render correctly by taking screenshots of portions of a website and comparing them against known good screenshots. It also provides tools for testing calculated CSS values and the position of HTML elements. See example in the Visual Regression Testing post.
¤463 Github stars
5. Rspec Page Regression (Uses Capibara)
Rspec-page-regression uses PhantomJS to headlessly render web page snapshots, by virtue of the Poltergeist driver for Capybara. You can also use the Selenium driver to test against real browsers.
¤64 Github stars
Pix-Diff is a lightweight Protractor plugin for image comparisons on (mobile/desktop) browsers and hybrid app screens or screen regions. 24 Github stars
7. Selenium Visual Diff (Uses Java and WD)
With Selenium Visual Diff the goal is a better integration of screenshots taking in maven executed selenium2 functional tests, storing and versioning of screenshots to get a report of visual differences between two application versions.
¤17 Github stars
8. Vizregress (Uses .NET and WD)
Vizregress is used to spot visual regression in your web application by comparing screen-shots against an approved set of screen-shots. This was created as a research project where they used TDD (Test Driven Development) to support visual regression testing. They also used SpecFlow as their BDD (Behavior Driven Development) tool to perform browser based tests.
¤5 Github stars
9. VisualCeption (Uses PHP and CodeCeption)
VisualCeption uses a combination of the “make a screenshot” feature in webdriver, imagick and jquery to compare visual elements on a website.
¤2 Github stars
10. Specter (Uses a JS DSL)
¤1 Github star
Configured Based Visual Validation Tools:
BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time. A good place to get started with BackstopJs would be css-tricks article on automating CSS regression tests.
¤1919 Github stars
2. dpxdtMake –Depicted
DpxdtMake makes continuous deployment safe by comparing before and after webpage screenshots for each release. Depicted shows when any visual, perceptual differences are found. This is the ultimate, automated end-to-end test. See it in example on Make your Deployment QA Safer
¤1331 Github stars
¤394 Github stars
4. Grunt Photobox
Grunt Photobox is a plugin to prevent your project of broken layout via screenshot photo sessions of your site.
¤277 Github stars
VIFF helps you to find visual differences between web pages in different environments (developing, staging, production) and browsers.
¤183 Github stars
GreenOnion helps solve the issue of having to catch regression bugs and solve the testing of HTML and CSS issue of having to be such a back and forth between designers and devs. It also help you to follow TDD and BDD against the presentation layer? GreenOnion helps you get the same results on testing front-end styling that you’ve enjoyed in your unit and integration tests up to now.
GreenOnion records ‘skins’, which are snapshots of the current state of a view (or any page that a browser can navigate to). The first time that it is run on a view, it saves the skin; the second time that it is run, it will compare the first skin to the fresh (new) skin. You can either compare with a diff image that is automatically created, or by an outputted percentage of difference.
¤168 Github stars
Kobold is a visual regression testing framework, comparing screenshots from multiple builds.
¤138 Github stars
The original Wraith is a screenshot comparison tool, created by developers at BBC News, for headless browser url-based comparison of responsive web sites.
Wraith-Selenium uses either PhantomJS or SlimerJS to create screen-shots of different environments and then creates a diff of the two images, the affected areas are highlighted in blue.
¤83 Github stars
9. CSS Visual Test
CSS Visual Test helps you to test your front end components visually. Currently supports components built with react and commonjs / browserify.
¤10 Github stars
10. Snap And Compare
SnapAndCompare is an easy to use screenshot comparison tool used for CSS Visual Regression Testing built using NodeJS.
¤8 Github stars
Grunt-Vigo is a simple visual regression testing tool using phantomJS 2
For more Visual Validation Tool Testing Awesomeness
Check out Adam Carmi and Peter Kim’s Automation Guild sessions on Using Visual Validation Testing.
Sorry you missed the LIVE event. But due to demand I decided to keep registration open. So you can still get all pre-recorded sessions and recorded Q&A. These recording will be be available forever so register now to get access now.