Breathe In, Breathe Out
We all know that performance testing can be frustrating. So many things can go wrong, and many of them are hard to find. No worries. Take a deep breath. Breathe in and Breathe out. Become one with your favorite performance test tool and follow these four steps to experience performance nirvana.
- Run & Measure it
- Confirm it
- Examine it
- Tune it
“Do not dwell in the past, do not dream of the future, concentrate the mind on the present moment” – Buddha
Run & Measure It
Fire off your tests and make sure you're getting repeatable results. NEVER go by the results of just one test. Going by the results of one test is like watching just one quarter of a football game and thinking you know who won the game. Bad idea. You want to run at least three sets of the same exact test without changes before comparing results.
After running the same exact test many times, take a look at the results and ensure that each run had approximately the same results. If they are not roughly the same, stop and find the cause. This is key, because if your results have more than a 5% difference between them and you make a tuning change, you wouldn't be able to determine whether change helped or not – since your baseline is unpredictable to start with. For my first round of testing, I usually start with brief, 15 minute, one & three user tests.[smartads]
After the test runs are finished, you'll want to look at the results and confirm there's nothing funky going on. That means that no errors are appearing in the test tool or in any log files, and all of your performance counters are working with no odd behavior occurring. Some examples of odd performance behavior could be:
- High CPU
- Large processor queue lengths
- Low throughput
- High Heap utilization
- High Response Time
- Counters that are trending upward or downward in multiple runs of the same exact test
Gather all your test results and consolidate them into one location such as Excel or a test tools reporting system. Then check:
- Are you meeting your performance goals?
- Are there any glaring issues that need to be reported?
- Any weird behavior showing up?
- Find predictable patterns in your info
Did you find a performance issue? If so, it's time to make a change and ensure that it improves performance. Before you start changing code and switching settings, confirm that it's not a hardware issue. Also — make sure that you only change one thing at a time. If you start changing multiple elements, you'll have no way of knowing which tuning adjustment actually helped to improve performance.
Repeat the 4 steps as needed, and rest in the peace of knowing. You've now found performance bliss.
I Am Awake!
When the Buddha began to wander about India shortly after his enlightenment, he encountered several men who recognized him to be a very extraordinary being.
They asked him, “Are you a god?”
“No,” he replied.
“Are you a reincarnation of god?”
“No,” he replied.
“Are you a wizard, then?”
“Well, are you a man?”
“Well, are you a testing guru?”
“So what are you?” they asked, being very perplexed.
“I am awake.”