Answer: A subset of all defined/planned test cases that cover the main functionality of a component or system, to ascertain that the most crucial functions of a program work, but not bothering with finer details. A daily build and smoke test is among industry best practices. [ISTQB]
When you want to quickly assess the state of a software build, you often turn to a Smoke Test (sometimes called a Build Verification Test). Essentially, you are trying to do something that is relatively quick and inexpensive, that will give you a general feeling for the software. You aren't looking for all the bugs. You aren't trying all the scenarios. You just want a general idea if the software works or not.
Often, the goal of this Smoke Test is to decide if more in-depth, more expensive testing is worthwhile, or if the software is too broken to be bothered.
Usually, you exercise only the basic paths of the software, and avoid all the unusual conditions. And usually, you don't test deeply at all - you just skim the surface.
In her book Effective Software Testing: 50 Specific Ways To Improve Your Testing, Elfriede Dustin says that a Smoke Test is "a condensed version of a regression test suite". And many companies do just that - extract a portion of their overall regression test suite, and use it repeatedly as a Smoke Test.
Manual Smoke Testing Sometimes, a manual Smoke Test just involves "playing with the software for a while". For an experienced user/tester of the software this may make sense. Such a tester may have a good sense of what to try, and just how deep to go.
But often, a written script tells the manual Smoke Tester what to test and how.
Either way, the manual Smoke Test should be brief and to the point. If the software is in generally good shape, the manual Smoke Test should pass. If not, the Smoke Test should fail.
Automated Smoke Testing I see this as one of the basic uses for my Test Automation Suite. I want to be able to quickly run a test after the build, so I can assess whether or not my team should bother to dig in and spend their time testing it. If the system-under-test passes the Smoke Test, we can proceed. If not, we reject the build until it is fixed.
If the builds occur overnight, I like to be able to schedule this Smoke Test so that it runs after the build and so that the results are ready and waiting for me when I get in the next morning. Sometimes, this allows me to run a larger overnight test and still have the results ready for the morning.
When I start to automate the testing of a system, the Smoke Test is usually the first automation I create.
In their excellent book How We Test Software At Microsoft, Alan Page, Ken Johnston, and Bj Rollison list these attributes of an automated Smoke Test (Build Verification Test):
Automate Everything - run it on every build, the same every time.
Test a Little - verify basic functionality.
Test Fast - should execute quickly
Fail Perfectly - fail only when the build isn't suitable for further testing
Test Broadly, Not Deeply - focus on primary usage scenarios
Debuggable and Maintainable - easy to fix, and keep up to date
Trustworthy - you must be able to trust these tests
Critical - requires time and careful thought to get it right
< I just noticed that my free site-visit counter is defective and rolls over on some strange boundary between 20K and 21K>
After reading this you may agree that we need to call on all manufacturers of numeric New Year's Eve year party eyeglasses to conduct product testing and make product alterations as indicated by such tests. Remember all the Year 2000 cussing, fussing, mussing and dooming and glooming? Well, unlike the actual results which were much milder than the predicted results, the New Year’s Eve celebration of the arrival of the Year 2010 will likely be catastrophic unless product design testing of numeric year party eyeglasses occurs, and – design changes are made accordingly.
Picture a million revelers in New York City’s Times Square on December 31, 2009. How many more millions will be assembled in a similar fashion around the world? How many of them will be wearing those goofy-looking numeric year party ungreen plastic eyeglasses – 2010? Did you notice something? “2009” glasses are natural for the eyes since they do not obscure one’s vision; the zeroes aligned perfectly with the vision-giving apertures. “2010” with the “1” over the left eye is going to impair depth perception. That sounds low-risk – correct? Think again. Imagine millions of shoulder-to-shoulder, hip-to-hip revelers worldwide already or soon to be impaired by giddiness, alcohol and d.r.u.g.s. Toss in impaired depth perception. We have a looming major problem of a magnitude not previously seen.
All hell will break loose with a series of small incidents that would be otherwise insignificant if one was sporting 2009 glasses.Here is how major mayhem and mass rioting will shake the very core of this planet Earth, causing worldwide tremors and sending seismographs into frenzied activity. Some newly engaged couple already under the depth-perception-robbing effects of hemp will attempt to kiss each other, with someone partially “lodged” between the kisser and the kissee.The kiss will not land on the fiancée, but instead land on the upper lip of that someone running block betwixt the original kisser and the kissee. Instant jealously and misunderstanding will drive the ensuing actions. A punch from the dominant and protective fiancée will fly, miss the intended face and land on an unintended face. Faster than a security hole in Windows can be exploited, the jealously, misunderstanding, and punch will go plural and spread. Similarly and concurrently elsewhere in these masses of humanity, other activities will trigger spawning mayhem. Other kisses will land at the business ends of cold-driven runny noses belonging to unfamiliar persons. Grabs for the usual body parts of familiar people will result in grabs of familiar parts on unfamiliar people. Males will be overheard asking the right question of the wrong females, "Did a mobile plastic surgeon just attend to you?" - OR - the wrong question of the right female where the response is a slap and escalation to feed the all-out brawl in progress.
Left-to-right, right-to-left eye-darting speed users will strobe themselves into seizures and nausea. The stroboscopic effect will act as an emetic and the resulting ejecta will find a partially bared bosom; triggering yet another rapidly spreading fight. Cocaine users armed with razor blades will miss the handheld mirrors and slice/dice someone’s digits.Human males wanting to unload processed liquids will miss the ground and decorate someone’s party stockings, pant legs, or ankle bracelet.More fighting erupts. Police on foot and horseback will respond. Some of those police and some of those horses will be wearing 2010 glasses. One can easily imagine the impact of horses with impaired depth perception. Some revelers will instantly acquire hoof-in-mouth disease. Other revelers may be handed some barn biscuits courtesy of the horses’ backside exits. They will launch these into the crowd and feed the Santa Ana winds-driven fire of hysteria. The perception-impaired police officers will miss their handcuff targets. Trump and O’Donnell will be accidentally cuffed together.OMG! The never-one-so-large largest riot is now underway. The fighting is unstoppable. People try to flee. It will be like a soccer game in Europe or a Who concert in Cincinnati! Hockey fans will feel at home. Those party-glasses equipped revelers on rooftops will misjudge the edges of their perches, soar to new unheights and then tattoo the road surfaces below. Unfortunately, Jerry Springer’s camera crews will not be on hand to film what would be his crowning achievement - millions of people with impaired depth-perception, fighting with temporarily defective targeting systems.
You can help prevent this looming tragedy of worldwide proportions. Please implore of the manufacturers of theseto consider alternate designs.
For ABC Television and their Rocking Eve, please send me a set of earplugs and glasses for the year 2111 if you are going to air the Pussycat Dolls again.
Hey, anyone know how to get this thing working again?
Apparently, starting around midnight last night, all first-generation 30GB Model Zunes - every one - restarted themselves and locked up at the boot screen.
Just take a look at some of the Headlines:
30GB Zunes Failing En Masse
30GB Zune apocalypse arrives as devices enter digital coma
Zune 30s all freezing up at once. Ack! Aliens!
Zune plagued by massive freeze
Did the Y2K failure come to Zune 30 devices 9 years later?
Microsoft Zunes spontaneously dying all over the place
Microsoft Zunes Hit By Rash of Lock-Up Bugs
Microsoft Zune 30GB users reporting freezing problems
Zunesday
30GB Zunes Everywhere Are Frozen. Z2K9?
Worldwide Zune suicide?
30GB Zunes Fail Simultaneously Everywhere
30GB Zunes Killing Themselves In Droves
Microsoft's Latest Global Problem
30GB Zunes hibernating for 2009?
Z-Day Hits Zune 30s
Some Zunes Expire Along with 2008
Z2K for Zune music players?
Think some Boundary Value Analysis might be in order?
At this time, Microsoft's Zune Support page says only:
zune service status
Status:
Customers with 30gb Zune devices may experience issues when booting their Zune hardware. We’re aware of the problem and are working to correct it. Sorry for the inconvenience, and thanks for your patience!
When I checked at 2:00 PM Eastern time, Microsoft's Zune Support page now says:
zune service status
Status:
Customers with 30gb Zune devices may experience issues when booting their Zune hardware. We’re aware of the problem and are working to correct it. The Zune Social might be slow or inaccessible. Sorry for the inconvenience, and thanks for your patience!
So apparently it was a leap year problem.
Today 5:43 PM
Official response for Zune 30 Freezing Issue (Zune 30gb stuck at reboot screen)
Early this morning we were alerted by our customers that there was a widespread issue affecting our 2006 model Zune 30GB devices (a large number of which are still actively being used). The technical team jumped on the problem immediately and isolated the issue: a bug in the internal clock driver related to the way the device handles a leap year. The issue should be resolved over the next 24 hours as the time change moves to January 1, 2009. We expect the internal clock on the Zune 30GB devices will automatically reset tomorrow (noon, GMT). By tomorrow you should allow the battery to fully run out of power before the unit can restart successfully then simply ensure that your device is recharged, then turn it back on. If you’re a Zune Pass subscriber, you may need to sync your device with your PC to refresh the rights to the subscription content you have downloaded to your device.
Customers can continue to stay informed via the support page on zune.net (zune.net/support).
We know this has been a big inconvenience to our customers and we are sorry for that, and want to thank them for their patience.
Q: Why is this issue isolated to the Zune 30 device? It is a bug in a driver for a part that is only used in the Zune 30 device.
Q: What fixes or patches are you putting in place to resolve this situation? This situation should remedy itself over the next 24 hours as the time flips to January 1st.
Q: What’s the timeline on a fix? The issue Zune 30GB customers are experiencing today will self resolve as time changes to January 1.
Q: Why did this occur at precisely 12:01 a.m. on December 31, 2008? There is a bug in the internal clock driver causing the 30GB device to improperly handle the last day of a leap year.
Q: What is Zune doing to fix this issue? The issue should resolve itself.
Q: Are you sure that this won’t happen to all 80, 120 or other flash devices? This issue is related to a part that is only used in Zune 30 devices.
Q: How many 30GB Zune devices are affected? All 30GB devices are potentially affected.
Q: Will you update the firmware before the next leap year (2012)? Yes.
-------------------------------------------------------------------------------- Matt Akers Zune Product Team
Now who could possibly have thought to test for leap year issues? After all, that "leap year" thing is a fairly new invention, right?
Perhaps they should have tested more.
An update.
Supposedly, this is the code which failed:
year = ORIGINYEAR; /* = 1980 */
while (days > 365) { if (IsLeapYear(year)) { if (days > 366) { days -= 366; year += 1; } } else { days -= 365; year += 1; } }
In How We Test Software at Microsoft, Alan Page, Ken Johnston, and Bj Rollison provide a terrific mix of insight into Microsoft, along with in-depth explanations of practical test processes.
From the introduction:
"This book is for anyone who is interested in the role of test at Microsoft or for those who want to know more about how Microsoft approaches testing. This book isn't a replacement for any of the numerous other great texts on software testing. Instead, it describes how Microsoft applies a number of testing techniques and methods of evaluation to improve our software."
I would also add that this book is for anyone who wants to learn some extremely useful, real-world approaches to both typical and complex testing situations.
Contents:
Software Engineering at Microsoft
Software Test Engineers at Microsoft
Engineering Life Cycles
A Practical Approach to Test Case Design
Functional Testing Techniques
Structural Testing Techniques
Analyzing Risk with Code Complexity
Model-Based Testing
Managing Bugs and Test Cases
Test Automation
Non-functional Testing
Other Tools
Customer Feedback Systems
Testing Software Plus Services
Solving Tomorrow's Problems Today
Building the Future
While not all of the solutions will apply to everyone (unless you happen to work at a company with over 9,000 testers), everyone will learn something. The excellent explanations of Equivalence Class Partitioning and Boundary Value Analysis are among the best I have ever read.
This is a very good book - one I highly recommend to all current and would-be testers.
Have a Testing or QA-related book you particularly like? Email Me
A terrific season by Matt Cassell, and a good season by the team. Unfortunately, not quite good enough, given the high standards in New England. Next year...
Who knew that simply texting the word "reboot" would actually cause a phone to reboot?
Apparently, words texted right after a reboot were interpreted and executed as commands with root priveleges by Android, rather than simply being sent.
A bit of a security flaw, no?
"I was in the middle of a text conversation with my girl when she asked why I hadn't responded. I had just rebooted my phone and the first thing I typed was a response to her text which simply stated "Reboot" - which, to my surprise, rebooted my phone."
"Google hurried to repair the problem, which causes the phone to interpret any text entered just after the phone was turned on as a command."
"Linux and Unix users are advised to use their systems with "root" privileges reserved only for administrators, but Android was actually giving anybody that privilege."
"The Android bug has to rate as one of the great software bloopers of all time."
"We tried really hard to secure Android. This is definitely a big bug."
Perhaps they should have tested more?
Here are a few helpful suggestions for words to use while testing the fix for this particular bug:
Restart
Shut Down
Ctrl Alt Del
$1,000,000
Travel Back In Time
Melt
ET Phone Home
Nuclear Winter
Delete Traces Of All The Calls I Just Made To Those 900 Numbers
iPhone
Sarah Palin President (no wait, this one simply isn't worth the risk)
After three straight seasons of reaching the MLS Cup, and six straight seasons of reaching the Eastern Conference finals, the New England Revolution bowed out this year in the first playoff series.
Lots of injuries this year eventually become too much to overcome.
The good folks at Simtec have released a new version of one of the tools in my toolbox - HttpWatch.
The biggest news is that HttpWatch now supports Firefox as well as Internet Explorer!
Here's the complete list of new features:
HttpWatch now supports Firefox 2.0 and 3.0 on Windows as well as IE 6, 7 and 8 Beta 2
Added a Properties window that can opened or closed from the View menu. It displays information about the log including browser type, time zone and operating system version
A comment can be assigned to a log file using the Properties window or through the automation interface
The values shown in the Summary window's Timings tab can now be accessed through the automation interface.
Started times can now be displayed as local time, GMT or as an offset from the start of the log
The Timings tab in the Summary window now provides minimum, maximum, average and total for each type of timing (e.g. DNS Lookup, Connect, etc)
Visual Studio style tabs are displayed in HttpWatch Studio for each log file making it easier to switch between files
The automation interface has been updated to support Firefox, timing summaries and log properties
I like it because it's very simple to use, but also very powerful. And I find some of the results fascinating.
Among the features provided by StatCounter is a Keyword Analysis report.
This report can show you the searh terms people entered into various search engines that ended up as a click through to your site. Here's a small sample:
Num
Perc.
Search Term
2
22.22%
define:load testing
2
22.22%
qa interview questions
1
11.11%
qa department business plan
1
11.11%
define:beta testing
1
11.11%
board games tock
1
11.11%
wintast version 3.5
1
11.11%
is yucart good for your
9
100.00%
Usually it's fairly obvious what the seearcher was looking for, but occasionally, I see a real puzzler.
I'll list some here as I encounter them, with a guess as to how the relevant search engine happened to serve up a link to All Things Quality in the results.
how to crash a browser
I assume the searcher was trying to find ways to crash browser.
And they learned about crashing Google Chrome the easy way.
While the searcher may have been looking for "yogurt", apparently this term brought the searcher to my post about some Spam that arrives in my email spam filter. One of the "from" names was "Yucart Churchgoing".
Do you suffer from Testile Dysfunction? Are you no longer invited to Requirements or Design Review meetings because you ask important questions – questions where the responses are typically, “No customer will ever do that?” – Or – “Please confine yourself to testing and let us handle the design”? Are you depressed because every software build exposes R&D test neglect? These issues can lead to a serious condition known as Testile Dysfunction (TD), a condition in which you lose your enthusiasm for testing and all related test engineering activities. TD reduces your level of readiness when you are ultimately called upon to execute tests.
Could you be ready for that moment when that moment is right? When R&D hurls the next application build over the wall-of-distrust and all the developers run for shelter – could you be ready?
Testalis® might be right for you.
Testalis® can restore your enthusiasm for your role. Only you can decide if Testalis® is right for you. Before taking Testalis®, ask your test engineering lead if you should proceed with functional and regression test activity and be sure to tell your test engineering lead about any angst you may have with R&D and all soured relationships with R&D. Don't take Testalis® if you take Tiagra® as the combination can cause a sudden, rapid rise in defect-discovery output and clog your organization’s defect-handling process.
Testalis® for daily use remains in your body for as long as you take it. Tell your test engineering lead about all medications, especially if you are going to attend any formal requirements or design review meetings, so your test engineering lead can be aware of potential review meeting conflicts.
Do drink alcohol in excess with Testalis®, as this will increase your funness, increase the illegibilty of your defect reports so they are rejected and thus bring you back into defect-discovery norms. Testalis® does not protect against intimately transmitted diseases, if engaging in these types of activities during testing. The most common side effect with Testalis® is too rapid of output of defect reports. Business and R&D complaints accompanied this anomalous behavior.
As with any TD tablet, in the rare event of testing enthusiasm lasting more than 10 hours, seek out and imbibe multiples of the nearest alcohol beverages of choice in order to exude an increased level of funness, thus offsetting the ability to construct a coherent defect report.
In rare instances, test engineers taking prescription Testile Dysfunction tablets (including Testalis®) reported a sudden interest in stepping down their careers to developing software. It's not possible to determine if these events are related directly to the Testile Dysfunction tablets or to other factors. If you have a sudden interest in developing software for a living, stop taking any ED tablet, including Testalis® and call your shrink right away.
In my company, we've recently started a Game Night.
We take turns bringing in games. After hours, we grab a conference room, and play for an hour or so. It's been lots of fun so far, and helps create a nice bit of teamwork.
The games
need to be able to accomodate around 5-10 players at a time (so checkers, chess are out)
need to be quick to learn
need to be accessible for teammates who are not native English speakers (so complex word or idiom-based games are out)
can't take more than 1-2 hours to play
Here are some we have played so far...
Foppen
"A trick-taking card game where the object is to empty your hand. The twist is the player who plays the lowest card in a trick sits out the next round."
"Sharpen your pencils - and your intuition - for this quick playing "think-in-sync" party game. Draw a category card and in 45 seconds, list as many related words that come to mind. When the timer runs out, roll the die - if it lands on HIT, pick a word that you think everyone wrote; if it lands on MISS, pick one that only you wrote. Choose wisely and score big points. The player with the highest score wins. Hit or Miss - the can't MISS game that will turn your next party into a HIT!"
"There are 104 cards numbered from 1 to 104. Every card has at least 1 small flag on it, which will score against you. The deck is shuffled and players are dealt 10 cards each. 4 more are dealt up on the table to form the start of 4 rows.
When each player has chosen a card from their hand, these are revealed and put on the ends of the rows according to simple rules.
As the rows get longer, a row with 5 cards in it is full. If your card is to be the 6th, you pick up the 5 cards in the row, and your 6th card goes to the front to restart the row. The cards you pick up do not go into your hand, but sit in front of you to score against you at the end of the round. Play rounds until someone hits 74 (the minimum speed of a hurricane) and the lowest score wins."
"The deck contains 6 kinds of cards. On each turn a player will take a stack of 3 cards (1 hidden and 2 visible) and keep one hidden and one visible card. The second visible card will be given to another player. At the end of the round, the player with the largest stack of cards in each color will not score points for those cards, while all other players will score points for those cards."
"The aim of the game is for each player to earn the most money from capturing famous outlaws. 2 to 4 players take up the roles of Sheriffs who are hot on the outlaws’ tails, trying to capture the outlaws with most rewards on their heads."
"A very simple railway game. Each player has a set of 5 cities strung across the US that need to be connected by rail. Players place either 1 or 2 rails each turn. The player who can make the best use of the other players' networks is generally victorious."
"As card games go, this one is quite revolutionary. Perhaps its oddest feature is that you cannot rearrange your hand, as you need to play the cards in the order that you draw them. The cards are colorful depictions of beans in various descriptive poses, and the object is to make coins by planting fields (sets) of these beans and then harvesting them. To help players match their cards up, the game features extensive trading and deal making."
"Tock is a board game, similar to Ludo or Sorry!, in which players race their four tokens around the game board from start to finish—the objective being to be the first to take all of one's tokens "home". Like Sorry!, it is played with cards rather than dice."
"The UNO classic card game goes revolutionary! When a spin card is played, someone must spin the wheel. Will luck be on your side? Will players get to discard cards, be forced to pick up more cards, or even exchange hands? In a single turn, everything can change! Players can come from behind and suddenly take the lead. It's fast-paced fun that'll make your head spin! Includes one UNO Spin? wheel and 112 UNO Spin cards and instructions."
• January 4, 2009 - QA Q and A - Smoke Test
QA
Q&A
When you want to quickly assess the state of a software build, you often turn to a Smoke Test (sometimes called a Build Verification Test). Essentially, you are trying to do something that is relatively quick and inexpensive, that will give you a general feeling for the software. You aren't looking for all the bugs. You aren't trying all the scenarios. You just want a general idea if the software works or not.
Often, the goal of this Smoke Test is to decide if more in-depth, more expensive testing is worthwhile, or if the software is too broken to be bothered.
Usually, you exercise only the basic paths of the software, and avoid all the unusual conditions. And usually, you don't test deeply at all - you just skim the surface.
In her book Effective Software Testing: 50 Specific Ways To Improve Your Testing, Elfriede Dustin says that a Smoke Test is "a condensed version of a regression test suite". And many companies do just that - extract a portion of their overall regression test suite, and use it repeatedly as a Smoke Test.
Manual Smoke Testing
Sometimes, a manual Smoke Test just involves "playing with the software for a while". For an experienced user/tester of the software this may make sense. Such a tester may have a good sense of what to try, and just how deep to go.
But often, a written script tells the manual Smoke Tester what to test and how.
Either way, the manual Smoke Test should be brief and to the point. If the software is in generally good shape, the manual Smoke Test should pass. If not, the Smoke Test should fail.
Automated Smoke Testing
I see this as one of the basic uses for my Test Automation Suite. I want to be able to quickly run a test after the build, so I can assess whether or not my team should bother to dig in and spend their time testing it. If the system-under-test passes the Smoke Test, we can proceed. If not, we reject the build until it is fixed.
If the builds occur overnight, I like to be able to schedule this Smoke Test so that it runs after the build and so that the results are ready and waiting for me when I get in the next morning. Sometimes, this allows me to run a larger overnight test and still have the results ready for the morning.
When I start to automate the testing of a system, the Smoke Test is usually the first automation I create.
In their excellent book How We Test Software At Microsoft, Alan Page, Ken Johnston, and Bj Rollison list these attributes of an automated Smoke Test (Build Verification Test):
For other QA and Testing terms, see: http://www.sqablogs.com/jstrazzere/46/A+Glossary+of+Testing+Terms.html
Peridocally, I'll pick a QA or Testing term and try to explain it in a bit more detail. If you have a term that you'd like explained - Email Me