The Top 7 iOS Automation Tools for Mobile Testing

The Top 7 iOS Automation Tools for Mobile Testing

iOs Mobile Apps

I’m pulling together some of the notes around iOS mobile testing that I’ve gleaned from research and interviews with TestTalks guests around mobile testing. Also picking an iOS testing framework is just the beginning so make sure to check out my post on how you can take the guessing out of you mobile testing coverage efforts

Here are 7 iOS tools that seemed to be mentioned most (and that many people are using) that you need to know about for mobile automation testing. For the video version check [click here]




1. XCTest

For unit testing, you can use the framework called XCTest. XCTest is provided by Apple and has pretty good integration with the IDE for writing your tests, running your tests and even doing TDD workflow.

If you performed any unit testing prior to iOS 6, you may have heard of a tool called OCUnit. OCUnit has now depreciated, and ever since Xcode 5, the recommended testing framework has been XCTest.

2. UI Automation

For your more typical functional tests (or black-box tests), in which you’re going to write code that simulates an end user navigating your app, there is UI Automation. UI Automation is provided by Apple and is the Apple-sanctioned way of performing iOS functional testing.

To use UI Automation you need to use Instruments, one of the tools that comes with Xcode.

The only problem with UI Automation is that tests are written in JavaScript, where iOS code itself is written in Objective-C or Swift, which differs significantly from JavaScript.

UI Automation itself is pretty lightly documented. If you have problems using it, you’re kind of left up to trial and error. What’s even worse is that there’s is not much of a community around it that you can consult for support.

Also — when newer versions of XCODE are released Apple rarely, if ever, issues any new updates to UI Automation.

That’s kind of the state of testing in terms of what Apple provides.

If the Apple tools don’t meet your needs there are also a couple of pretty popular third party ways to test your iOS code that might actually work better for you.


3. OCmock

I’m aware that many teams love and heavily use OCmock. It’s like your typical mock object framework, and it’s really good for stubbing out and making mock objects of stuff, like code that leverages in a remote API, like a web service.

You can also use it for code that relies on a UI being available, like UI objects that you can’t easily instantiate yourself through unit tests.

4. KIF (Keep It Functional)

A good alternative to UI Automation is a tool called KIF.

KIF stands for “Keep it Functional.” I like to refer to it as UI Automation ++.

It’s third party, and it’s open source.

You write tests much in the same way as UI Automation, where you’re simulating a user, tapping through the app.

Like UI Automation, it relies on accessibility labels; you define your application, which has the side benefit of making your app accessible to people who may be visually impaired that are being prompted through audio queues and how to use your app.

Basically UI Automation and KIF both rely on you defining accessibility labels on your UI elements in terms of how they navigate the app.

A couple of specific benefits KIF has over UI Automation is that you actually are writing the test in Objective-C which is now supporting Swift. There’s no learning curve of swapping languages between Objective-C and JavaScript, like you had to do with UI Automation.

Additionally, the way KIF tests are executed in the same type of target that your unit tests are executed in, which is not the case for UI Automation.

In order to execute UI Automation tests, you have to use the other tool by Apple called Instruments (it’s also their profiler tool). It’s actually a great tool, but kind of like UI Automation that is tacked onto it.

As always your goal should always be to make your tests repeatable and quickly executable. By integrating with the same way that XCTest runs unit tests, KIF makes those kind of GUI- based tests much faster to execute.


5. FBSnapshotTestCase

FBSnapshotTestCase was created and is maintained by Facebook. Its basic function is that it allows you to verify your UI code by doing visual diffs between saved screenshots.

You can do it at a more granular level than the full screenshot of the device. You can pick a little UI view in what you subclass when you see a little widget on the screen; anything from a piece of text to something more complicated. You can also isolate that snapshot test case to that UI view subclass.

I definitely see some usefulness in that beyond the normal kind of simulated click and play you might get with UI Automation.


6. FRANK – BDD for iOS

If you want to do end-to-end testing in iOS and wish you could use BDD and Cucumber, no worries –there’s a tool called Frank that will allow you to create acceptance tests and requirements using Cucumber.

Frank gives you the ability to run your BDD tests against your iOS application since it acts like an adaptor that connects your app to Cucumber.

The Cucumber recipes book has a nice step-by-step recipe(28) called Test iOS Using Frank that is really helpful.

7. APPIUM


Appium is a free, open-source, cross-platform mobile UI automation framework. It allows you to write UI tests for your mobile apps, for Android iOS, and some other OSs. It also lets you write it using the Selenium JSON writer you know and love from web automation. Appium is becoming an industry standard for mobile testing, much as Selenium WebDriver become the industry standard for browser-based automation a few years ago. Check out my Appium Tutorial for Beginners for more info on Appium.

Mobile Automation Awesomeness for Your Earbuds

I highly recommend that you not only learn more about these tools, but heed the advice of engineers who have successfully used these tools to automate their own projects in the real world. Check out:

13comments
Automate the Planet - August 31, 2015

[…] Day Unit Testing) Advanced Smoke Testing with PhantomJS – Eli Weinstock-Herman(LessThanDot) The Top 6 iOS Automation Tools for Mobile Testing – Joe Colantonio What to Expect When You’re Automating Testing – Joe Colantonio […]

Reply
Linkesh Kanna Velu - September 7, 2015

How about using Appium for automating Native, Web as well as Hybrid iOS applications?

Reply
    Joe Colantonio - September 7, 2015

    Good point – I don’t know how I forgot about Appium. I think I was I trying to focus on the tools that most people don’t know about

    Reply
Denys - January 14, 2016

I heared that UI Automation will be deprecated in favor to XCTest.
Interesting – they mean? Does it mean that UI Automation will be useless at all or what?.. If yes – then Appium could lose the ability of UI automation for iOS.
What do You think about it?

Reply
    Joe Colantonio - February 17, 2016

    Spoke with the creator of Appium Dan Cuellar on my TestTalks Podcast and he mentioned that Appium 1.5 includes an iOS implementation which does not use UI automation Javascript but rather uses the new Swift XC UI tests library. So we are future proof going forward, so we will continue to work with versions of Xcode that do continue to support UI automation Javascript.

    Reply
Michael - January 21, 2016

Good list. I’ve recently used Testmunk to test an app, and I was surprised by how good they were, since you never see stuff about them. They’re worth checking out.

Reply
    Joe Colantonio - January 23, 2016

    Thanks Michael – I’ll check Testmunk out

    Reply
Donald - March 24, 2016

Joe,

What will be the best open source image based automation tool for iPad applications, application is hybrid . We have to automate a lot of scenarios in very less time, if you can suggest a good framework too that will be awesome. Currently we are using HP insight for same.

Reply
    Joe Colantonio - March 29, 2016

    There are lots of option have you tried something like Frank or Calabash?

    Reply
chris l - August 22, 2016

Hey, eggPlant is pretty neat. I use the same test scripts across devices and browsers! I can reuse test scripts for iOS and Android, and then work against a network and load capability that is pretty robust. Check it out if you have not heard of it!

Reply
Jeevan - April 10, 2017

Out of 7 tools which one would you suggest interms of portability, fast, maintenance, compatibility, community support. In order of most preferred on top and so on …

Reply
Amy - August 18, 2017

What are the tools to test the performance of application, like there is AT&T ARO .

Reply
Click here to add a comment

Leave a comment: