You know, sometimes you get tripped up by being too narrow minded and believing what you see or hear is the truth.  Here is a little story that happened to me this week that I thought I would share – maybe others will learn the lesson, maybe I will next time!

We have had a problem in live running with one user (out of many thousands) who was complaining that on one particular screen the system kept freezing.  The user has complained a number of times and no-one could reproduce the problem.  His machine was swapped out, his mouse changed, keyboard changed all to no avail, he continued to report it was freezing. The help desk had repeatedly spoken to the guy to determine what he was doing and he seemed to be doing everything correctly.  We, in the test team tried, and failed to recreate the problem.  In the end I got one of my team to write an automated script that repeatedly went into the screen, doing something (varying combinations of input) and coming out.  We ran that script for hours and it never froze.  I told the tester to give up, it wasn’t reproducible.  Then the customer started to get letters from the user complaining that the system was unusable and, not surprisingly the customer put pressure on us.  Exasperated, I looked at all of the calls that this user had made to the helpdesk and he had made a lot of calls.  On examining each of the calls to do with freezing screens they all looked to be the same and no extra information, then on one call he mentioned a blank screen appearing.  My test expert in this area immediately said “I know what he is doing”, ran off to her test machine and recreated the problem immediately.  The problem was not that the screen was freezing, the screen was very much alive which is what had fooled us all.  The cause was that the user, instead of clicking on a hyperlink to select an item was clicking and dragging the item to another part of the screen (frames are wonderful!) and the browser had gone berserk.  Depending on what he clicked and where he dragged it to there were different results, a blank screen, a screen with hyperlinks on that wouldn’t work or the wrong screen.  My tester had raised a bug report on this over 2 years ago and it had been rejected as “will never happen in live” and the customer had agreed with that assessment and I had gone along with the decision.

Morals of the story. 

Don’t assume you know what a user means when recording a problem.

Don’t rely on a tool to recreate an intermittent problem. 

If the tool doesn’t show up the problem, don’t assume it doesn’t exist.

Bad decisions may come back to haunt you!