I just listened to an excellent presentation from CAST 2014 with Richard Bradshaw the Friendly Tester on automated testing. I think this is a “must listen to” presentation. Here is my quick overview:
Automation is not testing
Automation will never replace a real tester. That is why it’s ridiculous for anyone to say that 100% of tests should be automated. If all we relied on test automation alone, we’d be in some serious trouble.
Automated tests can’t think, they can only check things that they’re told to check. Therefore, automation is for checking — not testing.
Furthermore, fewer testers should not be the goal of automation. Richard Bradshaw goes into detail regarding the reasons why you cannot (or should not) want to rely on automated testing alone to test an application.
What is Automation?
So what is test automation? The definition Richard offers up is that Test Automation is “any use of hardware or software tools to support testing.” Notice it is to support testing —
not replace it.
If you flip test automation around to automation in testing, it can really change your perspective. Automation is just a tool. It is mindless, can’t think and can only do what you tell it to do.
We (testers), as humans, need to provide the actual thinking. Automation supports testing but it is not testing.
My main concern
I’m all about automation, but my main concern is that teams who are focusing all their efforts on automated testing will undoubtedly miss quality issues that can only be found by human testers.
Our goal as test engineers is to produce a quality product, not automated tests. A co-worker of mine, Mark Johnson, has an interesting take on this. He compares automation to remodeling kitchens (not a surprise to anyone who knows Mark—he’s a pretty handy guy).
“When planning a kitchen remodel, there will be a number of known components: appliances, counters, cabinets.
Automation is good at ‘checking’ characteristics of these components to verify that they are complete and that they work as intended. Some components are more complex than others (e.g. a refrigerator compared to a section of countertop).
Remodeling experts can design kitchens using these components, and are able to rely on the fact that a sink cabinet will be a certain height and width, and what strength and/or size can be expected to successfully hold up a sink and counter.
They can use those components to design various kitchens, making decisions and adjustments in designs that will not be automated (exploratory testing).
There may be certain known configurations that can be automated (validation level automation workflows), but these are small in number.
Not every engineer needs to become a programmer; their expertise should be leveraged, not sublimated.”
Check out Richard Bradshaw’s full video presentation on YouTube: Test Automation != Less Testers || Faster Testing || More Time For ET – YouTube
What do you think?
Do you agree or disagree with the Friendly Tester? Leave a comment and let me know your thoughts. And thanks, Richard, for starting what I think is a must-have conversation within the testing community.