Untitled
Software Testing is NOT "Breaking Things"
For some odd reason, I really don't like it when software testers say "I enjoy breaking things".
copyright © 2005 by Martin Hoffmann and Fred Mellender
When you test and find a bug, you haven't broken anything - it was already broken! If anything, the developer who wrote the code broke it.
And now that you have found a breakage, your job has just begun. You need to dig in much further:
- Under what conditions does this break occur? Under what conditions does it not occur?
- What steps are required to reproduce this break? And can you express those steps in simple terms so that developers and other testers can see it for themselves?
- Can you gather related symptoms, logs, images, etc - to help make fixing this break simpler?
- How long might it take to test a fix for this break?
- Is this break indicative of a more general problem? How will we know?
- Does the presence of this break, or a fix for this break, mean we should re-execute some of our tests? If so, which ones?
- What risks does this break expose?
- When did this break get introduced?
- Was it possible to find this break sooner? If so, why didn't we already find it?
- Should we modify our testing processes to find breaks like this more effectively?
If you enjoy breaking things, perhaps demolition is a good profession for you.
But if you enjoy planning, conducting, and analyzing the results from controlled experiments designed to find existing (or potential) breakages, then software testing might be right for you.
One Answer to the Question About the Ratio of Testers to Developers
Often I hear questions like "What is the best ratio of Testers to Developers?" or "What is the industry standard ratio of Testers to Developers?"
As I have mentioned before, those questions really have no answer. The appropriate ratio depends totally on context - the industry, the company, the software, the projects, the budget, the role of the testers, etc, etc.
But, for those who really crave a ratio, and don't care about context, the current issue (December 2006) of Better Software Magazine provides an answer.
Hundreds of their readers answered a survey about their employment situation.
In the results, they present several charts - one of which is the "Ratio of Testers to Developers".
While precise numbers are not given, their chart appears to show the following:
- about 5% report a 1:1 ratio
- about 45% report from a 1:2 to a 1:4 ratio
- just over 30% report from a 1:5 to a 1:10 ratio
- about 10% report a ratio of 1:10 or more
- just a few percent report 2:1, 3:1, 4:1 or 5:1 ratios
You should consider signing up for a free subscription at http://www.stickyminds.com/BetterSoftware/magazine.asp. Good stuff free!
Perhaps They Should Have Tested More - Moneris Solutions Corp.
Annually, the holiday season brings us shopping, visits from far-flung relatives, overeating, and reports of software failure.
I laughed when I read the response of the Senior VP of Marketing - "we would like to reassure them that we have identified the problem as a software problem". That's supposed to be reassuring?
Fortunately their system is "now up and running in a highly reliable fashion" - presumably as opposed to the largely unreliable fashion prior to this timely failure.
| Times Colonist; CanWest News Service; Canadian Press |
Saturday, November 25, 2006
Many shoppers on the Island and across the country couldn't pay for their purchases yesterday afternoon after a debit and credit card system failed.
A software glitch at payment processor Moneris Solutions Corp. was blamed.
Card holders were left scrambling to find ways to pay for their purchases.
"Some people were definitely inconvenienced," said Alex Mutrie, a clerk at the Petro Canada station on Douglas Street.
"They pumped their gas, tried to use their debit and had no way to pay," said Mutrie. "We took their ID, something they'll come back for. A couple of people were really angry."
The outage began around 1 p.m. Pacific time and lasted until about 3:45 p.m., said Royal Bank spokeswoman Beja Rodeck.
Moneris is jointly owned by the Royal Bank and Bank of Montreal, but the problems appeared to affect whatever credit or debit card was used in a Moneris point-of-sale terminal.
"This is a highly unusual incident. Our system has been running without incident for years," said Brian Green, Moneris senior vice-president of marketing.
"It kind of screws up your whole day," said Brianna Cameron, who was walking around Mayfair Shopping Centre with a friend.
"The stores told us we couldn't use our debit. So we went to get a drink at the food court and their debit wasn't working either," said Thais Robson, a Grade 9 student at Reynolds Secondary School.
Christmas shopping glitch was in the cards
Tough time for Royal Bank
TORONTO -- A software glitch at Moneris Solutions Corp. prevented some merchants across the country from completing credit and debit transactions for about 2 1/2 hours yesterday until the problem was fixed.
Brian Green, senior vice-president of marketing for Moneris, said the system went down about 4 p.m. Eastern time and was fixed by about 6:30 p.m. ET.
The problem was traced to a software application.
"We were able to isolate that software and essentially pull it out and thereby restore service fully," Green said. "This is a highly unusual incident. Our system has been running without incident for years."
Moneris is Canada's largest processor of debit and credit card transactions. It processes more than 2.3 billion payment transactions a year. Green said the problems cropped up across the country.
Moneris is jointly owned by Royal Bank of Canada (TSX:RY) and Bank of Montreal (TSX:BMO) but the problems yesterday affected whatever credit card or bank card was used in a Moneris point-of-sale terminal.
"We deeply regret the inconvenience and frustration that we caused our customers and their customers," Green said.
"However, we would like to reassure them that we have identified the problem as a software problem, certainly not a capacity or volume problem, and our system is now up and running in a highly reliable fashion."
Moneris Restores Service After ‘Glitch’ Cuts off POS Traffic in Canada
(November 27, 2006) While American consumers were flocking to the stores on Friday and whipping out their credit and debit cards for payment, their Canadian counterparts were forced to find cash, their checkbooks, or to come back later because the network of the nation’s largest merchant acquirer went down for two and a half hours.
Moneris Solutions Corp. reports that a software problem in its main processing switch that began about 4 p.m. Eastern time left its merchants unable to process any credit or debit card transactions until about 6:30 p.m. A spokesperson for Toronto-based Moneris, which has 300,000 merchant locations, did not have details Monday morning about the technical nature of the problem. The problem, however, did not arise from heavy volume or insufficient capacity, Moneris reports. Nor does there appear to be evidence of outside tampering. “All indications point to an internal glitch,” the spokesperson says.
In a news release late Friday, Moneris said that when it became aware of the problem it immediately started a diagnostic and restoration process and concurrently set in motion a process to move to its back-up system. The restoration process was successful and the back-up system conversion was not implemented. During the outage, calls flooded into Moneris’s customer-service center, creating a backlog that caused some merchants to receive a busy signal.
Besides Visa and MasterCard credit card sales, the glitch affected Interac PIN-based point-of-sale transactions and American Express Co. transactions in Canada, the spokesperson says. The problem did not affect Moneris’s U.S. affiliate, Moneris Solutions Inc., which is based in suburban Chicago.
The spokesperson says that until Friday’s incident, Moneris’s system had operated virtually flawlessly for years. Network uptime exceeds 99.9%, according to the release. “It was a minor headache, and certainly frustrating for the merchants and customers,” he says.
Moneris is a joint venture of RBC Financial Group and BMO Financial Group, parent companies of the Royal Bank of Canada and Bank of Montreal, respectively. It processes more than 2.3 billion transactions annually.
|
Interac glitch slows holiday sales | ||||||
By Peter Rusland News Leader Nov 29 2006
Local fallout from Friday’s nationwide computer software glitch is still being tallied by hundreds of Cowichan shoppers whose debit and credit cards were refused for use in shops throughout the Valley.
“It was hit and miss; some cards worked and others didn’t. It was a hodgepodge mess and it’s happening again today,” said Bruce’s Grocery manager Loren Halloran.
Staffer Jason Battie noted customers were able to get cash from the store’s ATM machine while regular shoppers charged groceries to their Bruces’ account.
“They seemed to be taking it fine considering the situation.”
Duncan Safeway’s first assistant manager Darren Bognar said his store’s customers were also understanding during Friday’s 1 to 3:30 p.m. downage that affected businesses using payment processors from Toronto-based Moneris Solutions Corp.
“The customers were really good about the inconvenience,” said Bognar who wasn’t on duty Friday.
“I’m sure it was cash only and some people got really good deals for waiting,” he said. “We tried to take care of our customers.
“We haven’t had any problems with our system for that long a period before outside of individual machines that had nothing to do with Moneris.”
Moneris spokesman Matthew Cramm says the Canada-wide blackout believed caused by wonky software is still being probed by the firm owned by the Royal Bank and the Bank of Montreal.
“It caused the network to go down so they took that software off line and restarted the network,” he told the News Leader.
“As far as I know the new network has worked. It was as if your Internet crashed.”
Cramm calls the problem “extremely rare.”
“Moneris’ network has been running incident-free for years. It was software and had nothing to do with (purchasing) volume.
“Their network is designed to handle even more capacity and was built to grow over time. It was just one of those things but it was frustrating for merchants and customers.”
Most bank machines were unaffected.
Canada’s half-dozen other payment processors for bank debit cards, plus Visa, MasterCard and American Express charge cards were unaffected, he notes.
Customers should contact their local bank to discuss problems or call Moneris’ merchant line at 1-866-319-7450. |
A New Tool in My Tester's Tool Box - WinTask
The good folks at TaskWare have released a new version of one of my favorite tools - WinTask.
I use WinTask for pretty much all my test automation these days. It can be used with Windows-based and browser-based applications, has a very easy-to-use recording capability, as well as a robust scripting language. It provides most of the capabilities of a full-featured test automation toolset, at just a fraction of the cost.
With the new Version 3.3, WinTask now supports Internet Explorer 7.
From their web site:
Automate any combination of tasks, whether Web or Windows applications... Transform your PC into a lean, mean automation machine with WinTask!
Version 3.3 is the latest and greatest.
You can see some of my WinTask utility scripts starting here: http://www.sqablogs.com/jstrazzere/WinTask/
See all the tools in my tool box at
http://www.sqablogs.com/jstrazzere/55/Tester%26%2339%3Bs+Tool+Box.html
Perhaps They Should Have Tested More - Excelsior Software
Teachers' Input of Grades Crashes System
By Daniel de Vise
Washington Post Staff Writer
Saturday, November 4, 2006; B03
There are probably some Montgomery County students who would prefer that their first-quarter grades never saw the light of day. For a few hours this week, it almost appeared that their prayers would be answered.
A new computerized grading system in 52 middle and high schools seized up Wednesday, overwhelmed as thousands of teachers simultaneously typed in final grades for the marking period. It was the first real test of a new electronic grade book that frees teachers from the tedium of marking grades in ovals with No. 2 pencils and feeding them into Scantron machines.
Officials eventually shut down the system and fixed a glitch that had caused the networking equivalent of a rush-hour pileup on the Beltway.
At a union meeting Wednesday night, frustrated teachers logged what might be the first-ever no-confidence vote in an educational software program.
"They had spent hours in front of their computers, trying to enter their data, and it wouldn't go through," said Tom Israel, executive director of the Montgomery County Education Association, which represents teachers.
The Pinnacle electronic grade book, piloted in four schools last year, is scheduled for countywide use in secondary schools next year. A timesaver for teachers, it also offers parents a chance to monitor their children's progress from week to week on the Edline Internet site.
School system officials said the brief system failure would not delay Thursday's scheduled release of old-fashioned, hard-copy report cards to students.
A New Tool in My Tester's Tool Box - WinMerge
Ok, so it's not a new tool. But rather a new version of an old tool.
I use WinMerge all the time for comparing folders full of files, and viewing the differences.
Many times, my automated scripts will compare text (perhaps from a screen full of text) against a baseline file. If they are the same, the script just continues. If they are different, I note that in my log file and write the text to a file in the results folder for later analysis. I use WinMerge for this file-comparison-analysis.
From their web site:
WinMerge is an Open Source visual text file differencing and merging tool for Win32 platforms. It is highly useful for determing what has changed between project versions, and then merging changes between versions.
Version 2.6.0 is the latest and greatest.
See all the tools in my tool box at
http://www.sqablogs.com/jstrazzere/55/Tester%26%2339%3Bs+Tool+Box.html
Perhaps They Should Have Tested More - Hart InterCivic
Virginia Ballot Glitch Chops Names
By Associated Press
October 24, 2006, 1:35 PM EDT
ALEXANDRIA, Va. -- U.S. Senate candidate James H. "Jim" Webb has lost his last name on electronic ballots in three Virginia cities where election computers can't cope with long names.
The glitch in Alexandria, Falls Church and Charlottesville also affects other candidates with long names, officials said.
Webb, a Democrat, appears with his full name on the ballot page where voters make their choices. The error -- referring to him only as James H. "Jim" -- shows up on a summary page, where voters are supposed to review their selections.
Election officials emphasized that the problem shouldn't cause votes to be cast incorrectly, though it might cause some confusion.
The mistake stems from the ballots' larger type size, election officials said.
It affects only the three jurisdictions that use balloting machines manufactured by Hart InterCivic of Austin, Texas.
"We're not happy about it," Webb spokeswoman Kristian Denny Todd told The Washington Post, adding that the campaign learned about the problem one week ago. "I don't think it can be remedied by Election Day. Obviously, that's a concern."
Every candidate on Alexandria's summary page has been affected in some way. Even if their full names appear, as is the case with Webb's Republican opponent, incumbent Sen. George F. Allen, their party affiliations have been cut off.
Jean Jensen, secretary of the Virginia State Board of Elections, pledged to have the issue fixed by the 2007 statewide elections.
"If I have to personally get on a plane and bring Hart InterCivic people here myself, it'll be corrected," Jensen said.
Hart InterCivic officials said Monday they intend to correct the problem by next fall. Jensen said Hart InterCivic already has written a software upgrade and recently applied for state certification to apply the fix, but the installation process can be time-consuming because of security measures.
In the meantime, Jensen said, the three affected jurisdictions have started educating voters and will place notices in each polling booth to explain the summary page problem.
Copyright 2006 Newsday Inc.
Things To Look For When Testing Your Site with IE7
Soon, many of us will need to test our web applications and web sites for Internet Explorer 7 compatibility.
As I have time, I'll list some of the problematic and potentially problematic issues that I encounter here. If you find others, send me an email!
Address Bar in all windows
In IE7, an Address Bar will be included in all windows. This is intended as a security feature - a way to help users avoid clicking on problematic popups and links within them.
But many sites use simple popup windows with specific dimensions as dialog boxes. Often, they haven't allocated space for the Address Bar. Thus, with IE7, many of these popups will now have scrollbars where before they did not.
Status Bar scripting
In IE7, scripts can no longer change the Status Bar text when the user chooses the default security settings. If your web app writes text to the Status Bar, you will either need to instruct your uses to customize their security settings (by setting "Allow status bar updates via script" to Enable) or change your web app to write this text somewhere else.
Script Access to the Clipboard
In IE7, if a script attempts to access the clipboard, the user will be prompted for permission. Accessing the clipboard is probably not something you would usually want in your web app. (see: http://www.sqablogs.com/jstrazzere/545/Web+Pages+Can+See+Your+Clipboard.html)
ClearType Text
By default, IE7 renders text using ClearType. For the most part, this is probably good. But, you'll still want to check it and see how it looks with your web app - using both LCD and CRT monitors.
(see also: http://blogs.msdn.com/ie/archive/2006/02/03/524367.aspx)
Some Tools Don't (Yet) Support IE7
Not specifically a problem with your web site, but some support tools you may use might not yet be IE7-ready themselves.
I use WinTask a lot, for example. The current version of WinTask (Version 3.2) does not work with IE7. The folks at Taskware tell me they are working on the next version, and will have it out soon.
It won't stop us from supporting IE7, but it might make things harder.
AJAX issues
It turns out that IE7 has a new feature “Native XMLHttp Support”. The new feature is enabled by default. This may or may not cause problems with web applications built using AJAX.
see: http://www.techtoolblog.com/archives/ie-7-native-xmlhttprequest-not-so-good
Adobe Reader 6 doesn't work well with IE7
In order to display PDFs in IE7, I had to upgrade the version of Adobe Reader I was using from version 6 to version 7.
HTTP Response Codes
|
1xx Codes |
Informational |
|
100 - Continue |
An interim response telling the browser the initial part of its request has been received and not rejected by the server. A final response code should be sent when the remainder of the material has been sent. |
|
101 - Switching Protocols |
The browser may wish to change protocols it's using. If such a request is sent and approved by the server this response is given. |
|
2xx Codes |
Success |
|
200 - OK |
The request was successful and information was returned. This is, by far, the most common code returned on the web. |
|
201 - Created |
If a POST command is issued by a browser (usually in processing a form) then the 201 code is returned if the resource requested to be created was actually created. If there is a delay in creating the resource the response should be 202, but may be 201 and contain a description of when it will be created. |
|
202 - Accepted |
If a request for processing was sent and accepted but not acted upon and the delay in acting is unknown, then this code should be sent instead of 201. Note that 202 does not commit to processing the request; it only says the request was accepted. A pointer to some status monitor for the task is often included with this response so users can check back later. |
|
203 - Non-Authoritative Information |
Usually the preliminary information sent from a server to a browser comes directly from the server. If it does not, then this code might also be sent to indicate that information did not come from a known source. |
|
204 - No New Content |
The request was accepted and filled but no new information is being sent back. The browser receiving this response should not change its screen display (although new, and changed, private header information may be sent). |
|
205 - Reset Content |
When you fill in a form and send the data, the server may send this code telling the browser that the data was received and the action carried out so the browser should now clear the form (or reset the display in some manner). |
|
206 - Partial Content |
This code indicates the server has only filled part of a specific type of request. |
|
3xx |
Redirection |
|
300 - Multiple Choice |
You should not see 300 standing alone; it serves as a template for the following specific codes. |
|
301 - Moved Permanently |
As the name implies, the addressed resource has moved and all future requests for that resource should be made to a new URL. Sometimes there is an automatic transfer to the new location. |
|
302 - Moved Temporarily |
The addresses resource has moved, but future requests should continue to come to the original URL. Sometimes there is an automatic transfer to the new location. |
|
303 - See Other |
The response to your browser's request can be found elsewhere. Automatic redirection may take place to the new location. |
|
304 - Not Modified |
In order to save bandwidth your browser may make a conditional request for resources. The conditional request contains an "If-Modified-Since" field and if the resource has not changed since that date the server will simply return the 304 code and the browser will use its cached copy of the resource. |
|
305 - Use Proxy |
This is notice that a specific proxy server must be used to access the resource. The URL of the proxy should be provided. |
|
4xx |
Error - Client Side |
|
400 - Bad Request |
The server did not understand the request. This is usually cured by resending the request. |
|
401 - Unauthorized |
The request requires some form of authentication (e.g., userid and/or password) but did not contain it. Usually, this code results in a box popping up in your browser asking you for the required information. Once you supply it the request is sent again. |
|
402 - Payment Required |
Reserved for future use. [Who says the web is not moving toward being a commercial medium!] |
|
403 - Forbidden |
This is a sort of catch-all refusal. If the server understood the request but, for whatever reason, refuses to fill it, a code 403 will often be returned. The server may or may not explain why it is sending a 403 response and there is not much you can do about it. |
|
404 - Not Found |
If you happen to mistype a URL or enter an old one that no longer exists this is the error you will likely see. The condition may be temporary or permanent but this information is rarely provided. Sometimes code 403 is sent in place of 404. |
|
405 - Method Not Allowed |
Your browser has requested a resource using a procedure not allowed to obtain that resource. The response should contain allowed procedures. |
|
406 - Not Acceptable |
Your browser said only certain response types will be accepted and the server says the content requested does not fit those response types. (This is one way content monitoring can be implemented.) |
|
407 - Proxy Authentication Required |
This code is similar to 401, except that the browser must first authenticate itself. |
|
408 - Request Timeout |
Your browser waited too long and the server timed out. A new request must be sent. |
|
409 - Conflict |
If a site allows users to change resources and two users attempt to change the same resource there is a conflict. In this, and other such situations, the server may return the 409 code and should also return information necessary to help the user (or browser) resolve the conflict. |
|
410 - Gone |
Code 410 is more specific than 404 when a resource can't be found. If the server knows, for a fact, that the resource is no longer available and no forwarding address is known, then 410 should be returned. If the server does not have specific information about the resource, then 404 is returned. |
|
411 - Length Required |
For some processes a server needs to know exactly how long the content is. If the browser does not supply the proper length code 411 may result. |
|
412 - Precondition Failed |
A browser can put conditions on a request. If the server evaluates those conditions and comes up with a false answer, the 412 code may be returned. |
|
413 - Request Entity Too Large |
If your browser makes a request that is longer than the server can process code 413 may be returned. Additionally, the server may even close the connection to prevent the request from being resubmitted (this does not mean a phone connection will hang up; just that the browser's link to the site may be terminated and have to be started over again). |
|
414 - Request-URI Too Long |
You will likely not see this one as it is rare. But, if the resource address you've sent to the browser is too long this code will result. One of the reasons this code exists is to give the server a response when the server is under attack by someone trying to exploit fixed-length buffers by causing them to overflow. |
|
415 - Unsupported Media Type |
If your browser makes a request using the wrong format, this code may result. |
|
5xx |
Error - Server Side |
|
500 - Internal Server Error |
An unexpected condition prevented the server from filling the request. |
|
501 - Not Implemented |
The server is not designed (or does not have the software) to fill the request. |
|
502 - Bad Gateway |
When a server acts as a go-between it may receive an invalid request. This code is returned when that happens. |
|
503 - Service Unavailable |
This code is returned when the server cannot respond due to temporary overloading or maintenance. Some users, for example, have limited accounts which can only handle so many requests per day or bytes send per period of time. When the limits are exceeded a 503 code may be returned |
|
504 - Gateway Timeout |
A gateway or proxy server timed out without responding. |
|
505 - HTTP Version Not Supported |
The browser has requested a specific transfer protocol version that is not supported by the server. The server should return what protocols are supported. |
Do what you love: Gaming on the ground floor
Since many people seem interested in testing games, here's an interesting article from The News Tribune in Tacoma, WA.
Do what you love: Gaming on the ground floor
BILL HUTCHENS; The News Tribune
While the depression in your sofa gets deeper and deeper, you might consider putting some of those video-gaming talents to work for you.
The Northwest, particularly the Puget Sound area, is a hotbed for game development. Dozens of gaming companies are stationed here, from big dogs like Nintendo and Microsoft in Redmond to PopCap, king of bite-size or “casual” games, in downtown Seattle.
And where there are game developers, there’s a need for game testers. Gamer or no, just about anyone can test games.
Those who have considered getting into the business of making games have heard the warning: “There’s a huge difference between playing games and making games.” The latter involves all kinds of mind-bogglingly complex math and science. Just thinking about the stuff that goes on behind the wizard’s curtain can spoil the fun of playing.
But game testers, these are the folks who really do play games either professionally or just for the thrill of being able to say “I played it first” or “I helped make that blockbuster game what it is.”
Below you’ll find a cross-sectional look at game testing in our area. There are many more game-testing opportunities than are listed here, but these descriptions should give an idea of the scope of positions available.
You want to test video games? Here are some places to start
Microsoft
Division: Microsoft Games Studios
Title: Official software tester
Foot in the door: “Official software tester” sounds pretty fancy-schmancy, but that’s the title Microsoft gives its volunteer game testers. That’s right: volunteer. Once every two months you can have a shot at being called in to test video and computer games.
Some compensation: OK, so you won’t make any money. You will get a nice gift, most likely a game or game-related goody to take home with you. And talk about an entry-level position. Work hard, intelligently document bugs and glitches, and who knows? You might work your way in – and up to the top. Be aware that you’ll need a lot of that “mind-bogglingly complex math and science” to truly succeed.
Be prepared: Never heard of a nondisclosure agreement? Better get ready to sign a hefty one. Like most high-tech companies, Microsoft is protective of its intellectual property. You’ll have to sign an NDA in order to test, and that means ixnay on the aggingbray. Take this line from their online FAQ: “You can say that you were at Microsoft and that you participated in a playtest, but you must agree not to talk about any specifics, such as: game title, graphics, features, etc., in order to participate. We appreciate your understanding in this matter.” Keep in mind, too, that the NDA lasts “indefinitely,” according to Microsoft.
Open enrollment: The program is open to gamers and nongamers of all ages. Microsoft wants input from young and old, experienced and inexperienced players. Looking at a test product from a nongamer’s perspective could reveal something a hard-core gamer might not notice – and vice versa.
Remember: It’s not entirely fun and games. You’re mostly likely going to play a product that isn’t finished. It’ll probably be buggy, and you’ll have to have patience while you describe and document every glitch you find.
Online enrollment form: Complete the online enrollment form at www.microsoft.com/playtest/playform.htm, and you could be on your way.
GameInstinct Inc.
Location: Redmond
The middle man: No matter how wide a game company’s net might be, it might not haul in the testers needed, the feedback wanted. Some game makers are looking to ready-made armies of game testers. That’s where companies like GameInstinct come in.
Throwing people at problems: GameInstinct amasses quality testers and then contracts with big game companies for temporary testing projects. So, you might find yourself working behind closed doors in the top-secret development lab of your favorite games company – but you’ll still represent and answer to GameInstinct. Like their clients, they’ll want the best of the best, gamers who can demonstrate flawless or near flawless game-play ability.
Roll ’em: Some game companies want to see video of their game being played from opening scene to closing credits. Sometimes the task falls to talented game testers.
Got what it takes? For more information, go online to www.gameinstinct.com.
Salary: GameInstinct charges its clients based on the size and scope of a given testing project. But compensation for most part-time game testers around the country is in the $9- to $12-an-hour range. It’s not much, but remember that you are basically getting paid to play video games and write about any problems you come across.
See also: www.vmcgamelabs.com. VCM has a game testing facility in Redmond and was chosen by Microsoft for several Xbox 360 testing projects.
PopCap Games
Location: Downtown Seattle
In-house only: Shawn Conard leads four salaried full-time game testers in PopCap’s Quality Assurance department. Their testing includes meticulously documenting every blip in PopCap’s notoriously addictive “casual” games such as the insanely popular and simple puzzle games Zuma and Bejeweled. These guys get paid not just to play games, but to try and break them.
No middle man: Unlike the quality-control processes in other companies, which often require multiple levels of scrutiny (and sometimes translation from one language to another and then back again) before any changes can be OK’d, conveying issues at PopCap often is as easy as throwing a squishy toy at a developer in a nearby cubicle. Once you have their attention, explain the issue.
Paying dues: All of the guys in PopCap’s QA department have spent many hours with temp agencies such as 1, 2, 3 Test, playing and testing games for many big game companies but never fully belonging to those big game companies. Hard work and perseverance brought them to their sweet gigs at PopCap.
Words of wisdom: “It’s not just about playing games or testing games and finding problems,” Conard said. “It’s about how well you communicate those problems to the development team.”
No vacancies: Sorry, team’s full at the moment. But they do sometimes call in a group of outside testers, a throwback to the good ol’ days when PopCap’s “A team” of game evaluators included the mothers of employees. The current team is a stable group without many openings, either, Conard said. But keep checking www.popcap.com for news.
Unknown
Job title: Console game tester
Reposting this message elsewhere is: NOT okay.
Look familiar? We can’t reprint this particular Oct. 2 posting from Craigslist.org. Just know that it’s a Seattle/Eastside company, and they need testers IMMEDIATELY.
’Tis the season: In all seriousness, this is the time of year when you ought to be checking classified ads for notices about game-testing opportunities like this one, especially if you’re looking for a fun way to make a little extra holiday cash and have some fun at the same time. The only drawback is, the companies seldom refer to themselves by name and never tell you up front the name of the game(s) you’ll be testing.
READ UP
These books offer information about breaking into the gaming industry.
- “Game Testing All in One” by Charles P. Schultz, Robert Bryant and Tim Langdell. This book can be a bit technical since it goes into not just testing for bugs and glitches but also covers methods of fixing them. If you’re technical-minded, this could be just the stepping stone you need to move from game testing into higher levels of quality assurance and possibly on to game development.
- “Game Plan: The Insider’s Guide to Breaking in and Succeeding in the Computer and Video Game Business” by Alan Gershenfeld, Mark Loparco and Cecilia Barajas. More of a manual on how to tap into the education needed for a variety of gaming careers, “Game Plan” also covers the all-important foot-in-the-door aspect of the industry.
Bill Hutchens, The News Tribune Bill hutchens: 253-597-8460
bill.hutchens@thenewstribune.com
Internet Explorer 7 was released today!
IE7 was released today, with Automatic Update distribution coming soon.
I'll post are some links you may want to check out as you test your web app in this new browser, and add more as I encounter them:
http://www.microsoft.com/windows/ie/default.mspx
http://www.microsoft.com/windows/ie/ie7/about/quickreference.mspx
http://www.microsoft.com/windows/ie/support/default.mspx
http://www.microsoft.com/downloads/details.aspx?familyid=D13EE10D-2718-47F1-AA86-1E32D526383D&displaylang=en
http://msdn.microsoft.com/ie/releasenotes/default.aspx
http://www.ieaddons.com/default.aspx?cid=home&scid=0
http://www.softpedia.com/get/Internet/Browsers/Internet-Explorer-7.shtml
http://www.winsupersite.com/reviews/ie7.asp
An Interesting Interview with a Game Tester
From the Washington Post, here's an interesting interview with a Game Tester.
Article published Oct 15, 2006
Game tester spends hours playing the same levels
By MIKE MUSGROVE
The Washington Post
Timothy McCracken, an amiable guy who works for Firaxis Games in Hunt Valley, Md., has what is either a gamer’s dream job or the classic “be careful what you wish for” sort of gig.
He’s a bug hunter. All day long, he plays computer games that are under development, constantly on the prowl for glitches. His business card identifies him as the company’s “chief deficiency inspector.”
Is it fun? Yes. Well, yes, sort of. It’s also a bit maddening.
“Imagine playing the first level of Mario nonstop for four hours,” he told me during a visit to Firaxis. “You have to jump on that block 400 times to see if it breaks on 401.”
McCracken doesn’t play a game so much as he works through every possible scenario that a player might try. When he’s done, he’ll do the same thing on a computer with a slightly different configuration.
And so on.
In the company’s podcast, McCracken has, jokingly, compared the job to the scene in “A Clockwork Orange” where the main character is strapped to a chair with his eyes forced open and made to watch an endless stream of movies.
Last month, McCracken was still testing the company’s latest title, a simulation game called “Sid Meier’s Railroads,” which was scheduled to hit the shelves for the holiday season.
That’s when most games are sold, especially the family-friendly sort of titles in which Firaxis specializes.
It’s just a fact of life that all games and other software have bugs. It’s McCracken’s responsibility to limit those bugs to as few as possible and hope that the remaining glitches will be the ones that few people ever experience or notice.
Some game companies have spotty reputations for putting out buggy titles; Firaxis doesn’t.
During the course of testing and polishing the railroad simulator over the last year, the team has identified about 1,800 bugs. The three-person Firaxis team also had the help of 150 fans who got early versions of the game and submitted their feedback.
In one early bug, trains would take off and launch into space. In another, smoke blew backward.
Bug hunting and killing is a process that continues until just before a master disk containing the final code is printed and shipped to the manufacturer. As the programmers fix software problems, a new internal release of the game is posted on the company’s servers nearly every hour.
The job of finding problems, as it turns out, doesn’t make you popular inside a game company. Colleagues have been known to close their office doors and shutters when McCracken comes around. They know he’s there because he has found a glitch that needs to be addressed. Tempers flare. Toy weapons – toys and video games are everywhere at Firaxis – have been brandished.
It goes with the territory, he says with a shrug.
“We make the programmers’ life a living hell,” he said. “We’re destroying their baby. Our goal is to be the worst critic they ever had.” If the bug testers had their way and spent the time to make it perfect, a game might not ever get released, he admits.
The office where McCracken and his bug-testing team works – better known as “The Pit” – is littered with games and computer equipment.
Over in a corner, a few computer systems are running the “Railroads” game 24 hours a day – if there is a crash or if the automated programs notice any strange behavior, the bug guys and the game’s programming team get an e-mail alert.
On this day, McCracken is wearing shorts and a pirate-themed T-shirt from a goth festival, his hair tied back in a long ponytail. Once an English major at Towson State University, he clearly loves his job, though he hopes to eventually get out of “quality assurance” and into game design.
It could happen, especially at this company.
Even though the industry is starting to see an influx of recent college grads armed with video-game-design degrees, “QA” is still a traditional entry point into the business. That’s how the president and the head producer at Firaxis started their careers.
McCracken is looking forward to the holiday game releases – because that’s when the games and systems come out that he and his fellow bug testers are looking forward to playing on their own: “Neverwinter Nights,” “Bully” and “Guitar Hero 2,” among others.
“We don’t have lives,” he said.
Perhaps They Should Have Tested More - ISoft
ISoft's latest glitch is 'pretty scary' for hospital· Software group has another dissatisfied customer
· North Staffordshire trust fears shortfall of up to £16m
Simon Bowers
Monday October 16, 2006
The Guardian
It is the latest blow in a series of troubled episodes surrounding recently installed iSoft software and could be the source of some embarr***ment when the financially stretched company holds its annual shareholder meeting in Manchester tomorrow.
North Staffordshire, which is struggling to claw back debts from last year of £15m and is shedding 1,000 staff, is having problems getting its new IT system to generate basic information on patient treatments in order to send bills out to the primary care trusts. It said the problem could leave the trust short by £4.5m to £16.2m by April. "The sums look pretty scary," said its finance director, Mark Mansfield.
At the end of last week hospital staff received an email from the IT department saying: "Due to technical difficulties, trust departments/wards will be unable to view results via iCM until further notice. We are working closely ... to resolve this issue as a matter of priority."
ISoft's North Staffordshire installations do not form part of the NHS's £6.2bn National Programme for IT (NPfIT), the largest non-military project of its kind in the world. However, after lengthy delays in the development of iSoft's next-generation Lorenzo software - the cornerstone of the NPfIT, earmarked for 60% of NHS trusts in England - the old iCM and iPM are being rolled out as stopgap measures. IPM has been used widely for about 10 years, mainly in primary care trusts. But its track record in acute hospitals has been mixed.
This summer Sheffield Teaching Hospitals, a foundation trust, pulled out of NPfIT, forgoing central government funding for an iSoft system that included iPM.
Sheffield's decision followed visits to two other acute hospital trusts using iPM: Scarborough and University Hospital Birmingham. A review of the visits, presented to the trust board, concluded that "the iSoft iPM patient administration system product as currently presented is 'not fit for purpose' and can't be deployed without implications for operational services and clinical/financial risk".
Scarborough and University Hospital Birmingham have remained supportive of iPM, iSoft and the NPfIT. Sheffield has said its decision to opt out of the NPfIT was nothing to do with visits to these sites.
At Plymouth - one of the first trusts to install iPM and iCM and the blueprint for the NPfIT rollout - iSoft systems are believed to be in use on only two wards years after they were installed at the trust.
Mike Brereton, trust chairman at North Staffordshire, told the Guardian good progress had been made in fixing the trust's billing crisis. Some £450,000 of July income the trust had previously described as unrecoverable had since been recovered in full. "We are continuing to work with iSoft on issues surrounding billing for the remaining three quarters of the year," he said. "Significant progress has been made."
Asked if the trust was satisfied with iSoft as its IT supplier, Mr Brereton said: "It is too early to say."
Some Interesting Bug Reports from Mozilla
Saw this linked on Reddit and thought I should share it with my QA friends.
Now, I'll remember Bug 95849 in my next Bug Triage Meeting.
Bug 95849 - Lack of Sex is interfering with my ability to triage bugs
https://bugzilla.mozilla.org/show_bug.cgi?format=multiple&id=95849
Here's another good one.
Bug 60455 - Mozilla doesn't add time to the day
https://bugzilla.mozilla.org/show_bug.cgi?format=multiple&id=60455
And
Bug 51015 - Microsoft should use Mozilla
https://bugzilla.mozilla.org/show_bug.cgi?format=multiple&id=51015
And this one
Bug 360981 - Keep (and expand upon) the Eudora sense of humor
https://bugzilla.mozilla.org/show_bug.cgi?id=360981
Perhaps They Should Have Tested More - Linden Lab's Second Life
Oops... virtual nudity!
(Imagine the bug report for that one...)
Oct 11, 2006
Sun Microsystems hosts virtual news conference on Second Life
By RACHEL KONRAD
AP Technology Writer
SAN FRANCISCO— Sun Microsystems Inc. spared the stodgy PowerPoint slides when it announced its new gaming strategy.
Instead, 60 journalists, analysts and product developers from around the world sent their virtual proxies - known as avatars - to a simulated world on the Internet. The event, hosted by the avatar of Sun Chief Researcher John Gage and held on an island in the online game "Second Life," was billed as the first news conference by a Fortune 500 company in the game.
"Second Life" is a subscription-based 3-D fantasy world devoted to capitalism - a 21st century version of Monopoly that generates real money for successful players. More than 885,000 people have avatars who interact with one another in the virtual world.
"We've been trapped inside the text world for so long," Gage said. "It's time for us all to get more Second Lifey."
Santa Clara-based Sun, which develops hardware and software for corporate networks and for gaming servers, hopes its "Second Life" outpost will become a destination for 4 million people worldwide who help write Sun's open-source code. No more than 22,000 can make it to Sun's annual physical gathering in San Francisco.
"We'll have bean bag chairs, and it will be a great place for people to try out code," Gage's avatar said on an outdoor stage flanked by billowing trees and ocean. "We want it to be just like your local neighborhood."
Brands such as Toyota Motor Corp.'s Scion, Intel Corp., CNet Networks Inc., Advance Publications Inc.'s Wired magazine, Adidas AG and American Apparel Inc. have already been building "Second Life" outposts. In August, former Virginia Gov. Mark Warner became the first real-world politician to host a "Second Life" town hall meeting.
"What corporate presence within 'Second Life' allows for is a different type of immersion in the product," said Donald Jones, Georgetown University graduate student writing his thesis on "Second Life." "It provides the corporation with an opportunity to seem like they're cutting edge. It helps them sell their image and their lifestyle within cyberspace."
Sun's virtual news conference Tuesday wasn't entirely glitch-free. The avatar of Philip Rosedale, Linden Lab's founder and CEO, briefly appeared on stage naked because of a software bug.
A New Tool in My Tester's Tool Box - JR Screen Ruler
JR Screen Ruler is a nice little free tool you can use to check the alignment and size of GUI objects during your UI testing.
From the web site:
Free virtual ruler for your computer screen
This Screen Ruler is a great tool that allows you to accurately measure anything on your screen. This is especially useful when measuring graphics, web page browser sizes or whatever. Screen Ruler can display Pixels, Inches, Picas or Centimeters.
http://www.spadixbd.com/freetools/jruler.htm
The web site also include a Pro version with even more features!
See all the tools in my tool box at
http://www.sqablogs.com/jstrazzere/55/Tester%26%2339%3Bs+Tool+Box.html
Fall in New England
My wife and I took a trip to Stockbridge, Massachusetts yesterday.
It was a beautiful fall day in New England. Trees changing color, blue skies, sunny.
We walked around Stockbridge center for a while and had lunch.
Saw Alice's Restaurant ("You can get anything you want... at Alice's Restuarant.") and the Red Lion Inn.
Went to the Norman Rockwell museum. Around back they have his workshop. He had a nice view.
(The view from behind Norman Rockwell's studio. October 7th, 2006.)
Fall in New England...
with the one you love...
life doesn't get any better than that!
A Week inside the mind of the Patriots
NFL teams are meticulous in their preparation; and the Patriots under Bill Belichick even more so. I think seeing the details of their preparation is fascinating.
A week inside the mind of the Patriots
Bill Belichick offers a rare glimpse into his team's exhaustive game preparation
FOXBOROUGH -- When Bill Belichick puts on his headset and starts pacing the sideline today, 10 days of preparation will begin to unfold in front of him.
What exactly happens in those 10 days?
There is film review. Team meetings. Individual meetings with players. More film review. Coaching meetings. Practices. Media obligations. Some more film review.
With the Patriots set to take on the Dolphins today, Belichick offered an inside glimpse of the game-planning process, through the lens of the head coach.
``It really starts on Thursday of the previous week, where we have the offensive and defensive quality-control coaches break down the films of Miami's last three games," Belichick said. ``For the rest of the staff, we'll take a look at those games on Friday afternoon, Friday night, Saturday, and even Sunday morning, just to get familiar with them."
Last week was a balancing act for Belichick and the team's assistants. They had Sunday's game with the Bengals at the forefront of their mind, but they were also laying the groundwork for the Dolphins. The headstart was important, although for some coaches, it could lead to mixing up game plans.
``One thing I do on Sunday morning, so I can get the other game out of my mind, is go back over the current game," Belichick said. ``Even though we've already spent as many hours preparing for the game, you want to make sure you're still sharp and you don't forget anything."
After the Patriots beat the Bengals, 38-13, last Sunday, the team boarded a plane and returned to Gillette Stadium around midnight. About six hours later, Belichick was back in his office, preparing for a full workday on Monday.
The game with the Bengals had been loaded into the team's computer system.
``At that point, I'm going through the game again, making notes, thinking about plays that I want to show the team," Belichick said. ``Maybe a situation comes up that you want to talk about, so that everybody understands what's happening, and a lot of times, those situations go both ways.
``A good example from last week was early in the game, when Kevin Faulk caught an angle route and gained around 20 yards, but the ball squirted out at the end. That was a chance to review the down-by-contact rule with the team. So it's things like that, or sometimes it's pulling out plays we look vulnerable on, or areas we'd like to improve, like kickoff returns."
The rest of the coaches were doing the same thing, specific to the positions they coach. At 10:30 a.m., the entire staff reviewed the game, while also setting an agenda for the rest of the day.
There was a full team meeting at 12:30 in which the game film was reviewed, with Belichick at the controls. After that meeting, the squad had a special teams segment, then broke up into offensive and defensive meetings, which lasted until about 3 p.m.
Belichick held a press conference at 2:45 p.m., and made his regular radio appearance on WEEI at 5:30 p.m.
As that was unfolding, the Dolphins' fourth game of the season -- a loss to the Houston Texans on Sunday -- had been broken down by the quality-control staff. Now all of the team's coaches had the Dolphins games in the computer system.
``By late Monday afternoon, everyone is getting their own information together, their scouting report together, and each coach has a different segment they are working on," Belichick said.
``That's when we start formulating game-plan ideas and personnel thoughts."
Once Belichick signed off from his radio interview at 6 p.m., he was almost ready to turn the page from the Bengals game, except for one final responsibility.
``I usually take a little time and make some final notes on the game for when we might play them the next time," he said. ``It's stuff you pick up along the way, after watching the film, or talking to the players, they tell you things going on that maybe you can't see on the field -- that the quarterback did this, or a lineman was calling this on that play. By that point, Monday night, things have settled, you've seen the tape a second time, and it's a chance to reorganize your thoughts."
Belichick's notes are usually entered in a
Before cashing in for the night, Belichick watched some film on the Dolphins, focusing on specific elements like running plays, or play-action passes, or big plays on both sides of the ball.
He made sure to wrap up fairly early, as he was short on sleep from the night before.
Tuesday is a day to formulate the game plan, and the office lights are usually on by 6 a.m.
Belichick started the day by looking at more Dolphins film, beginning with their defense. At around 11 a.m., he turned his attention to the Miami offense. After that, the focus was on individual Dolphins players.
``You're trying to have a good feel for each player and how they are being used, what kind of skill set that player has, and how that player matches up," he said. ``At the same time you're thinking about your own team -- who you could put on him, who might be able to handle him, who might have some trouble with him and might need help."
The full staff met around noon that day, and ideas for the game plan were discussed. Personnel was also a big part of the meeting, as the staff attempted to get a handle on which players might be available for action.
Belichick said the game plan was put onto paper by the end of the day.
``By Tuesday night, everyone is pretty well along on the plan; we know what we want to do, and then it's about preparing for different meetings with the players," Belichick said. ``We'll want to show film the next day -- here is how this play we're running will look against them, or here is how they would handle this certain formation, or this is where they line up when we do this.
``A lot of this is preparation for players, and at the same time, it's a review for each of us as to what our game plan is.
``We also talk about any adjustments to rules that we want players to follow. Like a team with two tight ends in the game, the rule is always to go to the guy on the line, but if [the Giants'] Jeremy Shockey isn't on the line, we still want to go to him. Since that rule changed from what it normally is, those are the type of things we're discussing."
Coaches are usually out of the office by 11 p.m. at the latest.
Belichick and the coaches returned to the office by 6 a.m. on Wednesday, and they prepared for a full team meeting at 8. When the players arrived, Belichick showed film of the Dolphins -- offense, defense, and special teams.
``It's basically, `This is what the team does well, this is what we'll have to try to stop,' " Belichick said. ``And we'll show them examples of the team in those phases of the game, both visually and verbally, all the way through the week as we're installing the game plan.
``You try to find examples of how our game plan would apply, like earlier in the year, when Cleveland played Buffalo [in the preseason]. They're similar to what we do."
The day broke down as follows:
Coaches again worked deep into the night, preparing a similar schedule for the next day, but with different points of emphasis.
Thursday, Friday
Third down, blitz pickup, the team's own blitzes, short-yardage plays, and special teams were on the Thursday menu. Then Thursday night, the coaches were preparing for Friday, which would include red-area, two-minute, and special situation plays (e.g. Hail Mary, onside kicks).
``All the way along, you're adding and deleting things," said Belichick, who in a normal week would start to sprinkle in some preparation on the next week's opponent on Friday (the Patriots have a bye after the Dolphins game). ``There are things that you thought might be a good idea, but they turn out to be too much trouble. Maybe [Tom] Brady isn't comfortable with a pass, so you throw it out. Maybe a linebacker isn't comfortable with a read, so you get rid of that. Then there might be one or two things added in.
``When we put in the game plan on Wednesday, it's probably only 60 percent in, and that's where Thursday and Friday come in. When you get to Friday, you're really starting to think about specifics, like what you'll call on second and 1, or second and 15."
Along the way, Belichick met with Brady specifically, gauging his comfort level with the plan. He also spent time with the front seven on defense.
Saturday
The full team always meets on Saturday morning, and Belichick often turns back the clock.
``We'll usually go back to what we talked about on Wednesday," he said. ``You're telling them, `OK, we went over a lot of stuff this week, but let's get back to where we started, and the things we need to do to win.' Last week, it was tackle Rudi Johnson on inside runs, don't give up the deep ball, and try to get Carson Palmer off the spot in the pocket.
``You're going over how you're going to start the game, the first 5-10 calls, maybe not in the exact order because it will depend on down and distance, but you're getting it down and you want to start thinking about them. There might be 70 plays on a game plan and this is your starting point. So Saturday is really bringing the whole game into a focus, whereas the last few days had been situational-oriented."
The squad also reviewed special teams and walked through every kicking situation. That night, the team gathered and watched film, going over all the calls they'll have in the game.
Then comes game day.
Belichick usually has a radio interview with color commentator Gino Cappelletti before the contest, and meets with the officials an hour and a half before kickoff.
``Any last-minute game-time decisions are made, and those come from the trainers, the strength coach, or the players, any combination," Belichick said. ``Those can come down to the wire."
Then the ball is in the air, with 10 days of preparation unfolding before another sellout crowd. 
Best and Worst Technical Interview Questions
Recently, Esther Schindler visited SQAForums.com. She's writing an an article for DevSource.com, and as she occasionally does, came by to ask a question and gather ideas.
This time she asked us "What are the best and worst technical interview questions you have heard?" Here are my answers.
Worst Interview Question:
Any brain teasers.
This interviewing fad started a while ago, and got popularized by Microsoft, I believe. Now everyone thinks it's clever to ask "why are manhole covers round" or "how would you test this pencil" or other assorted puzzles.
I always do my best to answer truthfully and without sarcasm. And, while I may not always enjoy them, I'm reasonably good at brain teasers.
But, I usually follow up with a question of my own like "Have you found that people who are good at answering these brain teasers actually turn out to be better employees than those who aren't good at it?"
I have yet to find a potential employer who could honestly answer "Yes" to that question. Usually, they just mumble something about "we just wanted to get a sense of your thought process" and move on.
Sometimes I have to conclude that they just aren't very good at interviewing. I put that on the "potential problem" side of the mental checklist I always keep about prospective employers.
Best Interview Question:
"As a QA Manager - what keeps you awake at night?"
I found it to be a really good question, and led to some really deep discussion about what was important to this company.
I learned a lot about them, they learned a lot about me, we found out that we thought alike.
And yes, I did get hired.
And here's Esther's complete article:
http://www.devsource.com/c/a/Techniques/The-Best-and-Worst-Tech-Interview-Questions/
(It was pretty good, but she consistently spelled "Massachusetts" incorrectly. Don't they have editors for that sort of thing?)
A New Tool in My Tester's Tool Box - TimeSnapper Classic
A nice new tool to take periodic "snapshots" - TimeSnapper.
From the website:
What is it?
TimeSnapper is an Automatic Screenshot Journal
It runs in the background of your computer, taking screenshots of your desktop every few seconds all week long.
TimeSnapper lets you play back your week just like a movie. You can play it at any speed you like, and jump in at any time you like.
When it's time to fill out that dreaded timesheet, TimeSnapper is a savior. No need to tear your hair out trying to remember where all the time went.
The "Classic" version is free. For even more features, check out the "Professional" version!
See all the tools in my tool box at
http://www.sqablogs.com/jstrazzere/55/Tester%26%2339%3Bs+Tool+Box.html
<- Last Page :: Next Page ->