My wife has an interest in “New age” beliefs, some of which she believes in, some of which she is sceptical about.  Me?  I’m always sceptical, however, I try to be as open-minded as possible and see what I can gain from any beliefs.  Last night, we watched a DVD called “Law of Attraction” which, if I boil down 1hr 50 minutes into a single sentence, is “if you want something, focus on it until you get it”.  OK, I think I was taught that when I about 5 years old, but hey, maybe some people need a reminder.  

One of the analogies used interested me from a testing perspective (yes, I am getting to testing eventually!).  The analogy was of a Satellite Navigation system in a car.  You tell the Sat Nav where you want to go (your goal), it works out the best route for you and then tells you how to get there.  The premise in the DVD is that we all have messages being given to us by non-physical means to guide us to our goal and all we need to do is listen to these messages and we will get there and achieve our goal. Personally, I found this hard to stomach, but maybe that is just me; there are too many variables in life for this to make any sort of sense and do I believe that I have this non-visible, non-physical “presence” sat with me all the time trying to guide me as to what to do?  No, I do not.

Getting back to having a goal, the Sat Nav and testing.  I got to thinking; do we have one overriding goal when we are testing? Wouldn’t it be great if there was a system where we can plug in that goal and it will tell us how to get there?  Call it the Test Sat Nav (TSN)

I got to thinking that probably we do have a goal and that goal is to complete the testing successfully.  The problem with that statement is what does “successfully” mean?  I would guess that every Test Manager has their own, slightly different, interpretation of what “successfully” means and every Project Manager has their own interpretation that will be different from the Test Manager and every customer will have their own interpretation that will be different from the Project Manager and the Test Manager.  So if we can’t agree a common meaning for the goal, how can we ever achieve it? And, therefore, how could we ever design a TSN that would enable us to plug in that goal and come up with the means of getting there?

So, maybe the goal is wrong, maybe it needs to be better defined, indeed if I put into the Sat Nav that I want to go to Birmingham, Birmingham is a big place and the chance of getting exactly where I want is not high. So maybe my goal is to meet the exit criteria?  Possibly better in terms of being able to tell the TSN, it might be easier.  But the problem with that is that we all do things during testing that are not directly related to the exit criteria.   Imagine putting No 10. Broad Street, Birmingham into my Sat Nav and deciding halfway up the motorway that I need to use the men’s room in the service centre?  That isn’t catered for in my instructions to the Sat Nav and it gets confused when I turn off the road it told me to go on and it isn’t part of my exit criteria.  In testing we go in a different direction than planned because we feel we have to, something needs a different focus, priorities change.  If on my road to Birmingham, the water company has decided to dig up the road and I get diverted round different streets, the Sat Nav is continually trying to get me back to where it thinks I should be, but I can’t because the road is blocked off.  In testing we get diverted by what happens to us, bugs being found that cause us to stop testing, changes that happen to the project and we have to take a different route.  The exit criteria haven’t changed, but the route there has. 

By this time, I am now wondering if we have a goal in testing at all that we can quantify, plan for and know we have met.  My idea of the TSN looks dead before it has started, if we don’t know where we are going, how can we plan to get there?  

Then, the “Eureka” moment occurred.  The totally unrealistic goal we have is that everything is tested 100% and there are no bugs of any kind in the delivered software. We, therefore, cannot get to that goal, it is an impossible goal; all we can do is travel towards that goal and at some point decide we have gone far enough.  So, testing does not have a destination that can be measured in the way of knowing you have got there, definitively.  All you know is that you have got sufficiently close to the destination and you do not need to go any further and you know that because of the measurements and metrics that you have been taking and, in the final analysis, by using your judgement. 

Having decided that, the TSN is easy, all I need is a program that will predict the changes that will happen along the way, the diversions that will happen and be able to predict the judgement call.  In fact, all I need is another New age belief, that of psychic prediction – I’ll get my wife to sort this out for me.

I went to an interview last week.  I was looking forward to the interview as I had heard good things about the company and they knew what good testing was all about.  It looked like a good opportunity and the project sounded interesting.

I liked the interviewer and I answered all the questions as honestly as I could.  Most of the questions were situational, of the format “what would you do if….?” Which makes you think about what you would do based on your own experience.

At the end of the interview, the interviewer told me that I would not be getting the position.  I asked why and was told that I was “too nice” for the role.  I know a few people who would disagree with any sentence that had my name and the word “nice” in it, however, the interviewer explained that the role required someone who, in the interviewer’s words, was a “b*****d” due to the nature of the project and the personalities involved and I didn’t have the personality required.  It became clear that the project had an aggressive character; there was a lot of political manoeuvring; conflict was the norm.  A female friend of mine calls such projects TDD.  No, not “Test Driven Development” but “Testosterone Driven Development”.

I left the interview somewhat disappointed and deflated.  One the way home, I reflected on the answers I had given in the interview and tried to work out what answers the interviewer was looking for.  I came up with some answers that maybe would have fit the bill and worked out that, had I given them, they would not have been a true reflection of the way I like to work.  I finally had to agree with the interviewer, I was not the right person for the position and I would not have done a good job in it.  Clearly, the interviewer was a good judge of character!

However, I was still non-plussed by the “too nice” tag.  I have been called a few things as a Test Manager, but “too nice” has never been one of them!  So, I examined the answers I had given and compared them with the ones that the interviewer wanted and I came to the conclusion (which I knew already) that I like to work in a co-operative environment where everyone is working as a team towards a common goal.

A co-operative environment does not mean that you are always nice to each other, hugs and kisses around the meeting room are not going to help the project, and I have my fair share of violent disagreements and arguments but they are almost always concerned with the project, not personalities.  I firmly believe that the co-operative environment gets the job done with better quality, lower cost and quicker than in a non-co-operative environment.  If you are all pulling together with the same aim and following the same agenda, things work better.

But, what do you do if you find yourself in an aggressive environment?  It has happened to me as I am sure it has happened to a number of us.  My first reaction is to be co-operative, even in that environment, and it can work, I have proved it.  I have one particular example where I turned an aggressive environment into a co-operative one.  OK, it took me some months and a lot of hard work, but I got there.  If trying the co-operative approach does not work, then you may have to get aggressive yourself, but if it is not in your nature you will find it difficult – I find it difficult and can only keep it up for a short period of time.

So, going back to this position I didn’t get.  A part of me wonders if I could have turned it around and made it into a co-operative environment.  I will never know.  The interviewer would not take the risk with me and in the interviewer’s place I wouldn’t have taken the risk with me either, the interviewer made exactly the right decision, in my opinion.

I’ll stick with believing in the co-operative environment, but I doubt I will ever hear anyone call me “too nice” again!

I have found a way of making my millions.  I have a problem with moles in my garden; they dig up my flower beds and leave hills in my lawn.  I don’t like to kill the little creatures as all they are doing is what is natural to them and surely I can put up with a bit of inconvenience, it is not worth an animal’s life to stop that is it?  But, I would like them to stop, so I want to deter them. 

I have tried sonic devices that moles don’t like the sound of, I have tried scent related deterrents, but the following morning, there is another one or two mole hills in my lawn.

I have discovered a method of removing moles from my garden that is as effective as all the proprietary methods and is considerably cheaper.  My method is to put posters around my garden with slogans saying “Moles keep out”, “Moles go home”, “This is a mole free garden”.  Believe me when I tell you that this is as effective as anything you can buy, i.e. it is not effective at all – just like anything else you buy!!  I am sure if I market this “solution” as well as all the other products I have spent a fortune on that haven’t worked, I could make millions.

Slogans are pointless, they do nothing to improve any situation.  Around our office, the Account Manager had posters put up saying “Are you the key to the next release’s success” and “Do it right first time every time” and my all time hated poster “Are you at peace with your piece of the next release?”  What on earth does anyone think these will do for the project? 

I find it incredible that any reasonably intelligent human can think that a slogan on a poster will make a difference to another reasonably intelligent human being.  It may work in a factory where the work is mind-blowingly boring and repetitive, I don’t know, but it will not work in a creative environment like an IT shop.  I tackled the Project Director about these posters and he stated that they were intended to keep people focussed.  Oh, that is alright then, so people would not be focussed if the posters were not there?  Of course they would, all the posters have done is give some amusement and a lot of irritation, can’t you high-ups see that?

I tore down the two posters put into my area and put them in the recycling, see, I knew I would find a use for them!  It is my job to motivate people and keep them focussed and I do not need posters to do it for me.  I didn’t see any lack of focus afterwards.

By the way, if you want to see some posters that are the exact opposite, go to www.despair.com, they made me laugh.

Moles, however, are much less intelligent beings, they are sure to be affected by my posters and they will be deterred from my garden.  Please email me with your order.  I can accept cash, cheques and I am just about to set up a PayPal account.  Only £2.00 per poster or 5 for £8.00 – you won’t regret it..

A few years ago, I went into a large multi-national bank to perform a consultancy task to review their testing practices.  As a consultant, you get one of three types of reactions when you enter the building:

1. A Groan – This is often because they have had bad experiences with consultants before and think this one will be no different.  I don’t mind a groan, this often tells me that they have decent practices already and you won’t find much to worry about.  This type of assignment is generally praising what they have done and recommending improvements in some areas.
2. A Hurrah – This is the juicy assignment.  A hurrah usually means that the workers know that what they are doing is insufficient and are hoping that the consultant will finally tell the bosses that they need to get their act together.  This assignment often means that there are no (or very bad) practices and you are starting from ground zero.
3. No reaction – These are the worrying and most hateful assignments.  This often means that someone high up has called in a consultant with no buy-in from the staff and they are not going to co-operate with a single thing you do.  You have an up-hill battle to even find a desk or get told where the toilets are.  Your best bet is to do what you can, write the report and get out.

Anyway, back to this bank.  I got the “Hurrah” reaction when I walked in.  I soon found out why.  In the section I had been called into, there was a development team of about 80 and 2 testers, yes, count them, 1, 2.  These 2 testers were from one of the high street branches and had been called in a couple of years previously to give the system a quick once over before it went live.  2 years later, the system was still a mess.  Some releases had gone to live and some of it even worked, but the system had some chronic problems.

I started doing the usual things for gathering information and it quickly became clear that the main problem was not only the testing (surprise!) but the Project Managers.  One Project Manager, during my discussion with him stated “We incentivise our staff with a bonus which gets more the fewer bugs are found in the code”.  I asked him how the tester was incentivised and he replied with a puzzled look on his face “The same, of course”.  So, the testers got a greater bonus the fewer bugs they found!

The testers had been given no training in testing, they were both intelligent people but they were not IT people, they were business people.  I asked them for their test plans and test scripts and it came as no surprise to me that I got a 1 page test plan and 2 pages of test script.  

The reaction to the presentation I gave to the senior bosses was extremely positive.  You could almost hear the sighs of relief around the room that the problems had been identified and that an action plan had been proposed.  This came as a surprise to me as I was expecting some hostility (I had pulled no punches).  It turned out that Director had not been with the bank for very long, which I knew, and he had been haranguing the senior managers about the lack of quality and they just didn’t know what to do about it – now they had a way forward.

That 4 week assignment to review their testing practices turned into a 4 year stint with them as they asked me to put the action plan in place.  And a very happy 4 years they were for me.  And I kept on the two business people, trained them and they became very effective business testers.

The lessons I learnt from this experience were:

·        Just because it is a large company, it doesn’t mean they have good practices

·        Senior IT managers do not always understand how to improve quality, they need telling.

·        The situation of lack of testing in an organisation is an opportunity, not a reason to be depressed.

·        “Shooting from the hip” sometimes works!

Should testers execute their own test cases?

Ask this question and you will get a barrage of differing views all with their reasons for a “yes” or “no” answer.  It seems that this is an area that testers and test managers just can’t agree on.  

So maybe there is something we can agree on, or at least debate, without having the poles apart yes/no extreme to the question.

Can we agree that test cases should be created?  No, we can’t. James Bach would argue that test cases have no place in doing testing.  If I read him correctly, he believes that formally written test cases have limited value and can actually be counter-productive to good testing and their only use is if there are legal reasons to create them (apologies, James, if I have mis-represented your view and you ever read this).  I have a lot of sympathy with that view and like to reduce (note, not eliminate) the number of test cases that my team writes to a minimum level necessary.  Key phrase there is “minimum level necessary”.  I believe there are occasions when the most effective way of testing something is by writing a test case.

Ok, so we can’t agree on whether we should write test cases, hmm, where to go from here?

Perhaps we can agree that tests need to be designed?  If we take the definition of “designed” as meaning that the tests are thought about before actually executing, then I think we can agree.  Even if you are doing Exploratory testing you are designing tests, by jottings on pads of paper, frameworks in your head or bright ideas that just pop-up as you learn more about the system you are testing.  Ad hoc tests are not designed, but I would argue that Ad hoc testing isn’t testing, it is playing around and has no place in the professional tester’s thinking.

So, if we can agree that tests need to be designed, then what is the implementation of that design?  

Before we determine that, maybe we need to look at what is the purpose of the test.  The widest view of the purpose of any test is to find a bug.  OK, we can argue about risk, control, confidence level, etc. but at the end of the day we are trying to find bugs.  But, it isn’t quite as simple as that.  From a project perspective, yes you are trying to find bugs, but there are supplementary purposes which are valuable to the test team, the project and the company.  Some examples:

·        Teaching a tester how to test.  The only way I know of teaching someone how to test is to get them to test.  Training courses are all very well, but they only give pointers on how to test.

·        Teaching a tester about the product.  It would be nice if every tester knew everything about a product before they started to test it.  Some testers may, but some (maybe all) will not.

·        Improving a tester’s skill set.  This may be in terms of getting the tester to use a previously unknown/unused technique; it may be in terms of getting them to lead a test effort.

The implementation of the test design, therefore, should be dependent on the purposes of the test. 

But we still do not have a full answer on how to implement the test design, because now we should consider what is the approach taken to testing?  By this I mean are you a test case shop, an Exploratory shop, unstructured or a mixture of approaches.  Your implementation of the test design will depend on your approach and your approach may change depending on the product you are testing this week.

When you have implemented your test design, then you need to execute the implementation.  Part (a small part) of me argues that if your design is good, then your method of execution is irrelevant, because you have done all the hard work of thinking and the rest is mechanical.  The rest (and more dominant part) of me argues that all you have done is laid a framework, no matter how good your test design is, the system under test is going to surprise you and you will not have covered everything you find you need to cover.

If that is true, that your test design implementation is only a framework, how do you determine who executes the test?  That decision should be made on who is the most appropriate person to execute the test, based, again, on what is the purpose of the test and the method of test design implementation.

No doubt in twenty years time people will still be arguing about whether the test case author should also execute the test.  Me?  I’ll stick with saying that “it depends” and take each situation on its merits.

I turned 50 last year.  No big deal, no big party, I didn’t want one.  After all I don’t feel I am getting old or even middle aged, in many respects I still feel the same way I did when I was in my 20s.  There are signs, however, of getting a bit older.  I was reminded of these signs over the holiday weekend just gone in the UK when I was out shopping with my wife:

·        I look at some of the young people and think “how can they wear those clothes?”

·        I hear some of the popular music and think “but it all sounds like noise to me”

·        I reminisce with my wife and friends and we talk about things that we did when we were “young” and realise that we saw things that happened well before current teenagers were born

·        I used to think I knew it all, as I get older the more I learn the more I realise there is to learn

·        Conversations often end with the phrase “which was a lot of money in those days”

Maybe I am turning into my Dad, they say we do eventually.

I was thinking more about this today as I drove to work listening to a Jimi Hendrix CD and realised he has been dead for 36 years.  My first major row with my parents was over him, they wouldn’t let me go to see him in concert because he took banned substances and they said I was too young to go to Woodstock (they were probably right, but what a gig that was and how envious I was of the people I knew who did go!).

What has all this rubbish got to do with Software Testing, you may well ask!  I was thinking, whilst listening to “Voodoo Chile”, about my early days in Software Testing and there are signs of getting older:

·        I see a question on a forum from a newbie and think “how can anyone not know that?”

·        I hear someone talk about how they found a bug and think “but that is how we have always found that type of bug”

·        I reminisce with colleagues and we talk about how software development used to be and realise that some of the computers we used were built well before some of the current new testers were born

·        I went into Software testing thinking I knew it all, as I get older the more I learn the more I realise there is to learn

One of the signs of getting older is less tolerance for the less experienced, the less knowledgeable, the “young”.  It is something I see on forums that the experienced people can be terse with the inexperienced.  Maybe we “oldies” should just step back and think “That was me 25 years ago, I needed help with the basics and I didn’t know it”.  I try to be patient with inexperienced people and know that I don’t always succeed, but that is my problem, not theirs.

I know I have got fed up with answering standard questions on forums and haven’t done it for some time, maybe I should start again, with more patience, after all, they will be paying for my pension in a few years time!

And, lastly, I remember my first Test department annual budget which was £150k, which was a lot of money in those days.

I, and others, have written a couple of blog entries on how activities outside of software testing can help understand testing (I refer to my entries on Dancing and Gardening).  Here is a story that has happened to me over the last couple of weeks that show the reverse, i.e. where testing skills have helped in my non-work life.

My wife is very interested in alternative therapies, herbal remedies, Reiki, crystal healing and the like.  This in turn has made her interested in the more mystical aspects that seem to go along with these alternative therapies, such as reading and understanding Auras, Angels and so on.  Their overriding philosophy is one of openness and honesty in all dealings – remember this for later!

Because my wife has this interest, I have also now got a (mild) interest although I am much more sceptical than her.  It is interesting, though, how a lot of the written works in this area have a direct correlation with some of the management books I have read, particularly on managing people, motivation and intuition.

However, I digress.  There is a company local to us that has a shop selling things like crystals, incense sticks, books, etc.  They also run a number of courses on the types of healing and how to become more in tune with yourself.  My wife is a frequent visitor to the shop and knows the owners quite well.  

A couple of weeks ago the company sent out an email asking for an investor/business partner and were offering a third of the business in return for the investment as well as asking for the investor to participate in the management of the business.  My wife was interested, so we asked for information.  They told us that they wanted £x for the third of the business and that the investor could expect to make about 6% return.  I thought £x was a bit high, so after some thought, we asked the following, basic, questions:

·                What was the investment to be used for

·                How was the business value of £3x calculated

·                What was the percentage return on investment for the previous year.

I also told them that depending on their answers I would then want my accountant to get involved.

All went quiet for almost a week.

We then got an email saying that they had decided to put the business up for sale.  The price?  £x/2.  So they had valued the business for an investor at £3x, but were prepared to sell it for a sixth of that value.  It smelled very fishy.  We suspect that the £x investment was to clear debts and the business is in trouble.  So much for openness and honesty, it feels like they were trying to con us.

So, by asking simple questions that you would expect a tester to ask (or, indeed any potential investor) we avoided a potentially damaging financial loss.

The lesson from this story is that even though someone would appear to have the highest values, you still cannot fully trust them.  This can be true in Testing.  Even the most honest appearing Project Manager can try to hide things from you to get what they want.

Bottom line:  If it looks like a duck, walks like a duck and quacks like a duck, you still need to do a DNA test to show it really is a duck!

I like IDMTDT testing, it can be a rich source of errors and can show up data integrity faults and usability problems that you might not otherwise find.  The big advantage is that you can then see how the system handles CRUD activities in unusual combinations.

Oh, sorry, IDMTDT stands for “I didn’t mean to do that” and is a set of tests that try to recover from when the user has done something stupid and wants to undo somewhere down the line in the Business process. 

A simple example:

User creates a new record for a new customer

User enters name/address/Post code

User then creates new accounts record for new customer

User puts credit limit on new customer account

User enters sales representatives name

User activates new customer.

At this point, User finds that the Post code is mistyped.  How do you recover?  If the system allows an edit of the customer field, then it is easy, but lets say, for the sake of the example, that there is no edit function for the Post code, it maybe a primary key in the database that is uneditable.  Let us also assume that deletion of customer records is not allowed as the system requirement is to keep all records for audit purposes and the design assumes that the user does not make this type of mistake. 

The recovery from this type of problem in a real world situation can be a nightmare (I have seen this type of scenario in a number of systems).  The testing can, however, be quite fun as you have a number of points at which you can try to recover the situation; after the new record created, after the account set up, after the sales rep entered, after activation.  Each of these scenarios will have a different set of tests and may involve exercising different functionality.

Planning and scripting such testing tends to be difficult as it is often the type of scenario not covered well in requirements or business scenarios (because people don’t make mistakes, do they?), so the best technique I have found is to include it in Exploratory tests.

It is a type of testing that can be forgotten or is done haphazardly.  It also, as a general guideline, should be done once the software is relatively stable as you are not trying to find functionality bugs, but business flow bugs.

Introverts can’t be managers.

I have heard this statement from staff during objectives reviews and career counselling sessions.  I like to ask the question “Do you think I am an introvert or an extrovert?” and invariably the answer comes back that they think I am an extrovert.  Wrong! I have taken a number of personality tests over the years and I am a very strong introvert.  I look at all the attributes of an introvert and I fit them perfectly, yet I have been in managerial positions for 20 years and I like to think I am reasonably successful at it.  So why is there this impression that introverts can’t be managers?

I have no evidence for the following, it is merely my opinion based on my observations of people and my own self-analysis. 

Let us look at the main properties of an introvert:

The natural inclination of an introvert is to hide away in a corner on their own and get on with whatever they have to do; they keep ideas in their heads without vocalising them; they think about things quietly; they are difficult for others to know well and only let a few people in close to them.  

The key phrase in the above sentence is “natural inclination”.  This is not a bad thing (says the introvert!) as that is what you are most comfortable with and the way that you are most effective.  However, that is not too good for a manager, especially a test manager, who spends a lot of their day communicating with people.  

How do you know if you are introvert or an extrovert?  I would suggest that you take the Myers-Briggs test as I have found it to be the most reliable measure (see http://www.myersbriggs.org/) and get yourself tested.

So how does an introvert become an effective manager?  I can only go from my own experience, my own way of dealing with being an introvert and from the advice and guidance I have given to other introverts. 

The key message is to break out of your comfort zone.  Force yourself to take on attributes of an extrovert.  Note, I am not saying that you should try to change your personality to be an extrovert as that is not good for you personally, but in your professional life “act” like an extrovert.  A Myers-Briggs consultant called me a “learned extrovert” which means that I learned how to behave like an extrovert when the situation demanded it.  

This learned extrovert behaviour is difficult for us introverts and takes time and a lot of practice and you never stop learning.  As an example, one of the worst nightmares for an introvert is giving a presentation, especially to people you don’t know.  I had to give a presentation to 150 people, most of whom I didn’t know.  I was highly stressed before the presentation and dreaded it, but I pushed myself to perform and I got great reviews for the presentation.  I have given training courses in the past, still do the odd one, for people I have never met and this is highly stressful as you are performing for 1, 2, 3 or 4 days.  Again, you have to push yourself to do it.  It gets easier the more you do it, but it never stops being difficult.  One of the training courses I give is on how to do presentations and I inevitably get the introverts saying “but I don’t like doing it and you make it look so easy” Yes, I do, but I am quaking inside, it is a matter of managing your nerves and not giving in to them.

You don’t have to give presentations or training courses to “get over” being an introvert, just push yourself to speak out more when with others.  You will feel uncomfortable, that is natural for you, but force yourself.

So far, I have concentrated on the negative aspects of being an introverted manager, but there are positives as well.  We are generally better at managing other introverts as we understand them and can empathise.  We think things through before opening our mouths and therefore, hopefully, have a more coherent conversation than the loud mouthed extrovert who thinks with their mouth open.  This can be a useful trait when you are trying to persuade.

An important point.  If you are introvert, do not try to suppress your introverted nature, you can make yourself ill (seriously, it can bring on illness).  So make time for yourself, make sure you take a lunch break on your own sometimes, make sure you do have think-time.  Personally, I find that meditation is very effective for me although some people find it useless.

Finally, then, a message for all introverts, you can be an effective manager.  If that is what you would like to do, go for it.

Following my entry entitled “Are you positive about being negative”, Joe Strazzere asked what do I call the tests that we execute when we run out of time.  This is a good question and one that I should have answered in the original entry as Joe has correctly pointed out that I now don’t have a name for them.  

One of the reasons for not putting a tag on this type of testing is that it can cloud the thinking when determining what tests to run.  “Happy path”, for example, would indicate a certain type of test.

If you need some type of “tag”, the best I can come up with is MAT, which stands for “Most Appropriate Tests”.  Let me explain what I mean.

Whenever you are short of time for some tests you have to determine what tests you will run to make the best use of the time available. The thought processes go along the lines of:

• What are the risk areas of the system
• What are the areas that have spawned the most bugs recently
• What are the most business critical areas
• What are the areas that would be most embarrassing if they failed
• What has changed recently and therefore could have de-stabilised the system

So, the upshot is that I can’t give these tests a generic term, the answer is “it depends”, you should run your MAT suite!

Software Development can be a lonely business.  There is the developer sat at his/her desk churning out code day after day with only the compiler to talk to.  The only light relief that a developer has is when the tester finds a bug and needs to talk about it.  The joy of interaction with something that doesn’t just say “Syntax Error” is something to behold.  Watch the developer’s face light up at the chance of human interaction, at the opportunity to exercise vocal chords rusty through lack of use, a conversation!

As testers we should support these poor, lonely creatures.  Please give as much time as you can to this worthy cause.

Are you positive about being negative?

I don’t believe there is a distinction between Positive testing and Negative testing.  I don’t believe that there is such a test as a “Positive” or “Negative” test.

There, I have said it and the heavens haven’t opened up and I haven’t been hit by a thunderbolt.

You often hear the question “What is negative testing?” and you get various answers.

Some definitions I have read/heard:

1. Those test that are designed to return an error, using invalid input
2. Tests that are designed to cause the system to crash. 
3. Tests that are designed to make the system do something “off spec"

With reference to definition 1, so if I have an integer field and I put in an alpha character I expect to get an error, yes?  So what is negative about that test?  I have made a positive input and expect a positive response, i.e. an error message. 

With reference to definition 2, so, if I set up corrupt data I expect the system to crash, yes?  Again what is negative about that?  I have made a positive decision to try to force the system into an error condition to see how it handles it.  Where is the negativity?

With reference to definition 3, so, I am doing something to the system that might make it do something unusual.  Is this negative?  I have designed tests that try to force the system to do something it shouldn’t.  This is the closest that I can come to saying it is a “negative” test, but even here I am making a positive decision.

If one of the primary reasons we test is to find bugs, then we should be designing tests that are targeted at finding bugs.  Any means at the tester’s disposal should be used to find those bugs, use whatever techniques you have in your toolbox, whatever tools you have and you will execute “just tests”, not positive, not negative.

What does it matter?  My concern with classifying tests as positive or negative is that it is an artificial classification that clouds tester thinking.  There are no positive or negative tests, only tests that exercise the system looking for bugs.  I don’t understand the reason why testers want to distinguish between positive and negative, what benefit does it give?

I have heard people say “we are running out of time, only run your positive tests”.  But which are those?  I have a set of tests, I can’t differentiate them according to positive or negative.  What people usually mean by that statement is “Only run those tests that exercise the system the way the user will normally use it”.  OK, I can identify those tests, they are sometimes called “Happy path” tests, and we always create those, but they are a small percentage of the tests we prepare, I estimate less than 10% of tests fall into that category.  Are they positive tests?  By some definitions, yes, but what if one (or more) fails? Does that make it a negative test?  Again, does it matter?  A failure is a failure whatever classification is used for the test.

I don’t want a tester to go off and run a set of “positive” tests, I want him/her to execute tests that exercise the system in such a way that have a reasonable degree of probability of finding bugs.  That should be by exercising the system in the way the user will normally do it and doing other things as well.

I guess that I am not going to suddenly stop all the testers in the world from using “Positive” and “Negative” tags for tests, all I can do is try to stop my own testers!

Computers are more important than human life?

Way back in the early 1980s I worked on a large mainframe computer in London.  This beast ran at an amazing speed (for then) and generated a significant amount of heat.  The computer room was the size of a football pitch and contain latest state of the art hard disk drives, tape drives, printers, etc.

The computers were cooled using halon, so there were massive tanks of the stuff circulating round the hardware trying to keep it cool.  The management’s main concern was a fire as there was no offsite backup of the data and so they installed halon gas extinguishers into the ceiling of the computer room which would go off in the event of smoke being detected.  They calculated that the data was safe for about 10 seconds in the event of fire, so set the halon to go off 7 seconds after detecting the fire.  

For those of you who don’t know, halon has the property of removing oxygen so the fire would go out.  One tiny detail was overlooked – humans need oxygen to live.  If you were at the far end of the computer room, i.e. furthest away from the door, we calculated we could not run to the door in less than 20 seconds, even at panic speed.  Given the efficiency of halon, we reckoned we would be dead in less than 15 seconds.  But, it got worse, 10 seconds after the fire alarm went off the lead lined exit door was locked automatically to prevent any fire spreading, so even if you were half way down the room you wouldn’t get out in time.

Where were the system consoles positioned?  You’ve guessed it, right at the far end of the computer room.

Of course the management were very concerned when we pointed this out to them, they immediately sprung into action and moved the system consoles to just near the door – 2 years after we notified them of the problem.

RSS Feed - now on page.  Should work, at least it does for me when I put it into my reader.  That will be quite useful as now I will be notified that I have updated my blog......

I think I need a lie down.

Why do you like being in Testing?

This is a question I am asked fairly often by people who both know me and new acquaintances.  Sometimes the question is couched in the “so, you can’t program so you test?”.  As an ex-developer, I can scotch that one!  My answers are various, along these lines:

• I enjoy seeing the bigger picture that testing gives you.
• I like the challenge that testing gives you
• I like the variety that you get in testing
• I like finding problems
• I have a questioning mind
• I get a kick out of seeing better quality products than there would have been had I not tested it
• There is so much to learn in the field of testing
• Etc, etc.

If I analyse these answers, they are quite bland, they don’t really get to the heart of the reasons and they don’t really say what I feel.  Anyone who has done any consulting or psychology will tell you that there is a technique that you can use with people to “bottom out” an answer, that is, you find out the fundamental reason for why someone does something.  

I decided I needed to bottom out why I like testing, just for my own benefit.  After some careful thought and analysis I came up with “I like being in Testing because I like being around testers”.  In other words, it is a people thing, I get a good feeling by being around people who have a testing mindset who are thought provoking who are questioning and who make me laugh (as most testers do!).  My natural instincts are to be with these type of people and always has been, well before I got into testing.  I, like a lot of testers, fell into testing by accident and felt at home once I was there because of the people.

I have yet to answer the question honestly in public and say “Because I like testers”, because the questioner probably wouldn’t understand the answer.

As a Test Manager you get asked all sorts of questions and usually they start with words like “How do I …”, “When should I….” or “What should I…..”.  Occasionally, you get the “Why do I/we….”.  It is these questions that I like.  The How, When or What questions are usually fairly easy to answer, you follow your experience, process, knowledge to answer them.  The Why question is much harder and, therefore, much more interesting because it is these questions that make you think “Yes, why do we…”.  The situation on any project that you have been on for any length of time is that you start to accept the situation you are in, you have done things the same way for some time and it works and where it doesn’t work you try to change it, but you are generally in a comfort zone where everything is familiar and almost routine. 

A friend of mine years ago made an interesting statement.  We had been on a project for a couple of years and we were being well paid for a challenging, but relatively routine piece of work.  He said “We are in a fur-lined rut”.  That phrase made me think and ask myself “Why am I staying here?”.  Within a few months I had left the company because my friend was right, I wasn’t moving forward and I wasn’t in a position within that company to change it (long story!).  It is the “Why” question that should keep being asked and if you get the answer “I don’t know” or “Because we have always done it that way” or “If it ain’t broke don’t fix it” then it is time to get things changed and improved.  This is the essence of improvement, be it personal improvement, process improvement or any other type of improvement.  

The answer to the “Why” question may be that there is a good reason why and we don’t need to change, but the very asking of the question makes you go back and re-evaluate why.  If you come up with the answer that you don’t need to change, that is OK, but then you have made a conscious decision not to change for good reasons, not because you are maintaining the status quo.

What made me think of writing this entry was that recently one of my team asked me a “Why” question that made me do some thinking.  It was a good question and a hard one to answer.  As it happens, I am not changing anything as a result of the question, but it made me think hard and that is a good thing. 

Sometimes managers are not good at listening to the “Why” questions with an open mind, especially if the question comes from someone who is less experienced than them.  Sometimes team members are not very good at asking the “Why” question as they think their manager knows better than them or it makes them look stupid or it will seem that they are criticising their manager’s judgement.  

Managers should encourage the “Why” question and should take time to ask themselves the “Why” question.  It is often the only way to get us all to think rather than do.

Maybe we should have a "Why" question day on a regular basis?

Here are the tasks I have to perform in my “typical” working day:

• Read approximately 100 emails, most of which I have been cc’ed on or forwarded on with an “FYI” attached.  Most of which I really didn’t need the information, but you have to read it just in case there is a snippet that you really do need
• Respond to approximately 15 of the 100 I read.
• Send on approximately 15 of these emails to interested parties (with FYI on the front, naturally!)
• Write approximately a 15 emails.
• Attend, on average, 2 hours of meetings.
• Be interrupted umpteen times by all manner of people
• Analyse the previous days bug stats
• Check progress against plan
• Answer approximately 5 phone calls
• Make a number (can’t quantify) of decisions varying from the trivial to the critical
• Assist team members who need help
• Liaise with the customer at least 5 times

You can see that I don’t do much in the way of productive work.  I am primarily interrupt driven and I need to do parallel processing.  

Why am I telling you this?  Not to get sympathy, nobody gives sympathy to a manager!  I attended a talk by Scott Barber, who I have a great deal of respect for, and part of the thrust of his talk was that managers don’t listen to Performance testers and don’t understand what they are saying, i.e. we managers are not good at attending to the needs of non-functional, technical, testers.  

To some extent, I agree with him, we are not good at it.  I would generalise the point even further, though, in that it is not just Performance testers, it is all testers that are in our team.  

My rebuttal to Scott in an email I sent to him went along these lines.  I have a lot of people and things that are vying for my attention.  I prioritise my time by classifying each item requiring my attention by Importance and Urgency.  If something is high Importance and high Urgency, then it gets my immediate attention, conversely low Importance and low Urgency are unlikely to get any of my time.  I believe it is the responsibility of any of my team to assist me in establishing the Importance and Urgency of what they want from me.  In my experience, technical testers are the worst at this.  I am reasonably technically minded, but I can be (and sometimes am) blinded by the language that these guys use.  So if I am told by a Performance tester, for example, that the Pareto curve on the disk accesses on the EMC disk for physical buffering is causing them concern, I might start to glaze over.  If they, however, tell me that the Service Level Agreement will not be met if we don’t improve the performance of the database writes, then I am very interested.  

I am not asking for things to be dumbed down for me, I just want to have the headline as to what the problem is.  I can then delve down, if necessary, and understand what the low level reasons for the problem are and get it explained to me.  If I am given the low level information first, I can’t work out for myself the level of Importance and Urgency.

So, help us poor managers, give us a clue as to why we should be interested in your problem/issue/concern.  If you don’t we won’t be any the wiser, we may dismiss what you are saying when we shouldn’t and we will come across as not caring or as stupid.  We have a lot of demands on our time and I don’t mind being managed by my team and influenced as to what I should spend my next few minutes on, in fact I welcome it.  

I often try to compare activities outside of the testing world, or indeed software development, with how we behave within testing – see my blog entry on Agile Gardening.  This blog entry is in a similar (slightly whimsical) vein.  I find it useful to do the comparison as testing is such an abstract thing with nothing you can touch, smell or feel at the end of it as it is primarily a set of thought processes.  You might say that the test scripts, the screen dumps, the output files, the bug reports etc. are physical outputs of the testing process and they are, but they are not a record of the behaviours of testing.  It is this behaviour that makes us good/bad/indifferent testers.  Maybe I can explain what I mean with this comparison.

I like dancing, I have been Ballroom and Latin American dancing for some years now and I am slowly getting better at it.  I am told that I am one of the better dancers in my club.  I wasn’t always so.  When I started dancing classes I had two left, flat, feet, I shuffled round the floor like an arthritic robot (can robots get arthritis?).  I watched other dancers glide round the floor with apparent ease, looking graceful, masterful and in control and I wanted to be like them.