My wife has an interest in “New age” beliefs, some of which she believes in, some of which she is sceptical about.  Me?  I’m always sceptical, however, I try to be as open-minded as possible and see what I can gain from any beliefs.  Last night, we watched a DVD called “Law of Attraction” which, if I boil down 1hr 50 minutes into a single sentence, is “if you want something, focus on it until you get it”.  OK, I think I was taught that when I about 5 years old, but hey, maybe some people need a reminder.  

One of the analogies used interested me from a testing perspective (yes, I am getting to testing eventually!).  The analogy was of a Satellite Navigation system in a car.  You tell the Sat Nav where you want to go (your goal), it works out the best route for you and then tells you how to get there.  The premise in the DVD is that we all have messages being given to us by non-physical means to guide us to our goal and all we need to do is listen to these messages and we will get there and achieve our goal. Personally, I found this hard to stomach, but maybe that is just me; there are too many variables in life for this to make any sort of sense and do I believe that I have this non-visible, non-physical “presence” sat with me all the time trying to guide me as to what to do?  No, I do not.

Getting back to having a goal, the Sat Nav and testing.  I got to thinking; do we have one overriding goal when we are testing? Wouldn’t it be great if there was a system where we can plug in that goal and it will tell us how to get there?  Call it the Test Sat Nav (TSN)

I got to thinking that probably we do have a goal and that goal is to complete the testing successfully.  The problem with that statement is what does “successfully” mean?  I would guess that every Test Manager has their own, slightly different, interpretation of what “successfully” means and every Project Manager has their own interpretation that will be different from the Test Manager and every customer will have their own interpretation that will be different from the Project Manager and the Test Manager.  So if we can’t agree a common meaning for the goal, how can we ever achieve it? And, therefore, how could we ever design a TSN that would enable us to plug in that goal and come up with the means of getting there?

So, maybe the goal is wrong, maybe it needs to be better defined, indeed if I put into the Sat Nav that I want to go to Birmingham, Birmingham is a big place and the chance of getting exactly where I want is not high. So maybe my goal is to meet the exit criteria?  Possibly better in terms of being able to tell the TSN, it might be easier.  But the problem with that is that we all do things during testing that are not directly related to the exit criteria.   Imagine putting No 10. Broad Street, Birmingham into my Sat Nav and deciding halfway up the motorway that I need to use the men’s room in the service centre?  That isn’t catered for in my instructions to the Sat Nav and it gets confused when I turn off the road it told me to go on and it isn’t part of my exit criteria.  In testing we go in a different direction than planned because we feel we have to, something needs a different focus, priorities change.  If on my road to Birmingham, the water company has decided to dig up the road and I get diverted round different streets, the Sat Nav is continually trying to get me back to where it thinks I should be, but I can’t because the road is blocked off.  In testing we get diverted by what happens to us, bugs being found that cause us to stop testing, changes that happen to the project and we have to take a different route.  The exit criteria haven’t changed, but the route there has. 

By this time, I am now wondering if we have a goal in testing at all that we can quantify, plan for and know we have met.  My idea of the TSN looks dead before it has started, if we don’t know where we are going, how can we plan to get there?  

Then, the “Eureka” moment occurred.  The totally unrealistic goal we have is that everything is tested 100% and there are no bugs of any kind in the delivered software. We, therefore, cannot get to that goal, it is an impossible goal; all we can do is travel towards that goal and at some point decide we have gone far enough.  So, testing does not have a destination that can be measured in the way of knowing you have got there, definitively.  All you know is that you have got sufficiently close to the destination and you do not need to go any further and you know that because of the measurements and metrics that you have been taking and, in the final analysis, by using your judgement. 

Having decided that, the TSN is easy, all I need is a program that will predict the changes that will happen along the way, the diversions that will happen and be able to predict the judgement call.  In fact, all I need is another New age belief, that of psychic prediction – I’ll get my wife to sort this out for me.