Rainbow Testing

What are you smoking?

Some smart aleck in your organization has suggested “Smoke Testing” you’re wondering just what they have been smoking! Now you’ve been tasked with putting together a “Smoke Test” and explaining to your team just what it is.

You’re in luck. This will be easier than lighting a pipe in a high wind.

First you need to know the purpose of the activity, the rest will follow.

A smoke test is just entrance criteria. It’s the “You must be this tall to ride this ride” of the testing cycle. You’re a test lead or a test manager, or just a tester who is not willing to waste their time on un-testable code. Your purpose is to find the most efficient series of tests to determine the fitness of the code. This means going long and staying high. (Is that where the smoking comes from? No.) Don’t bother with the nit-picky stuff, it has to pass the smoke test first.

That’s “why” now you may be looking for “how.”

There are simple methods for selecting the cases for your Smoke Test suite. First look to the Happy Path. Remember: “If the application can’t walk the Happy Path, it can’t walk at all.” 

The next place to look is in the weight or priority applied to your test cases. If your requirements were ranked and the test cases traced to them were ranked you should look at the high level test cases; the test cases linked to the critical requirements. If more than 10% of your test cases are of the highest rank or priority and you need for your smoke test to be streamlined, look for overlaps in functionality, duplications, or equivalents. Can you eliminate down to one case for each area or function? Remember, your smoke test isn’t the last word, it’s the “go” word.

(Streamlined, streams of smoke, is that where we got the word? Not quite.)

Is there a shortcut to finding the showstoppers? Do you have access to insight on the code or on the formatting of messages? Sometimes the fastest way to find the problem is to look into the code itself. If you can shortcut the process by putting some white box testing up front, don’t hesitate to move this to the head of the line. Don’t wait for the testers who only have blind access through the interface to stumble on the problem if you can see it faster by having a developer provide you with access to the internal message formatting. (Getting smoke in your eyes, is that it? No, that's not it either.)

Here are three examples of the usefulness of smoke testing:

You’re a test manager with a team of 7 testers that are already working on various projects. You know that it takes a certain amount of time for them to switch gears and that’s time lost. The code you are getting from the development team on this brand new project isn’t always ready for test. Sometimes you can’t even log in to the new version without it crashing. You’re not willing to “throw it over the wall” to the whole team if you can’t log in and process the most basic transactions. You put together a “Happy Path” through the code to touch the most basic functionality, without visiting the exception cases. If it fails the Happy Path, you can push it back to the development team without breaking the concentration of your testers.

You are a single tester with a detailed 1,000 test case suite ready to go. Each section of the suite plumbs the depths of the functionality of the new development. Running the test cases in order you could take 3 – 4 hours to detect a show stopper bug in a specific module, depending on which section you ran first. But running the 1^st test case from each section will cover the breadth of the development and uncover any show stoppers in the 1^st hour of testing.

Your testing involves white box and black box testing. The black box functional testing will be performed through the user interface by a user group who will be rattled by catastrophic failures. The white box team can see into the code and examine the formatting of the messages that will shape the user group’s experience of the application. If the message formatting isn’t right, the user interface won’t function correctly. It is simple to run a few messages through and examine their formatting to find the errors that are guaranteed to cause failure. This will save you time and energy.

Don’t confuse the purpose of smoke testing with your detailed testing. This is a quick high level exercise that is designed to detect a leak that could turn into a disaster. And by the way, that’s where we got the term: from plumbing!

Efficient and cost effective, smoke testing has become a world-wide standard for finding leaks in sewer and plumbing systems. By forcing smoke-filled air through a sewer or plumbing system, leaks can be quickly detected as smoke escapes through problem areas.

Now you’re ready to smoke!

Where does "smoke" testing come from

{ 05:10, 12 June 2007 } { Posted by mike }

And there was me thinking it came from hardware engineering:

Plug it in and turn on the power. If it starts to smoke it's not worth testing!

Mike

PS: Could you turn ont he RSS feed for your blog. It's a good one to read, but I don't always think to start opening a browser and find it.

RSS feed

{ 01:17, 12 June 2007 } { Posted by prainbow }

Hey Mike,

Thanks for your comments, much appreciated. I do have RSS on, try this:

http://www.sqablogs.com/rss.php?w=rainbowtesting

Let me know how that works out....

RSS Feed

{ 04:39, 15 June 2007 } { Posted by mike }

Thanks. That works great.

I hadn't found a link on the page and google reader couldn't work one out but it works fine with the link you gave.

Getting ready to start my own blog soon

Mike

RSS feed

{ 10:12, 15 June 2007 } { Posted by prainbow }

Mike,

I want to thank you for bringing this to my attention. It took a little figuring but I now have the link on the right hand side of the blog.

When you get yours started, investigate the support guide page on tags. I found the info on RSS feed URL threre.

Good luck!

Paulie