Sep. 4, 2008 - I have a little bone to pick...

Do I really have time for this?

Wow, it’s been a while since I last posted an entry - is that how blogging works or am I supposed to do it every day?

I’ve been so busy over the past few weeks and entering a blog is never far from my mind, but the issue is time.

Now for something that really gets up my skirt!

I tend to brows over a few blogs, usually by stumbling upon them as I’m doing my ‘research’ on the web. I can fairly say that I am more than capable of weeding out the ‘cut n’ paste’ jobs fairly quickly!  I mean, why or why do soooooo many bloggers go on and on about different kinds of testing? 

I am NOT going to list them here, because my guess is, if you are reading this then you are already a tester by profession – even if you are at the entry level, you should already be familiar with these.  If you know nothing about testing, then I apologies, my blog is not where you want to be, here are 3 links to get you started, try here or here, or even here. 

There are many bloggers out there, who want to come across as ‘testing experts’, so they think the best way is to have multiple blog pages that just regurgitate the same old material over and over (you’ll see that Wikipedia, is their best friend).  

Now, I’m all for knowledge sharing and self learning but having multiple pages on your blog, about types of software testing (commonly titled ‘Introduction to Software Testing), that you obviously copied from somebody else’s blog, who probably copied that same information from someone else, does not make you an expert! 

Wed 20 Aug 2008 - Bug emotions - correction

I had a problem posting my last entry, it appears that copy and paste of a Word table doesn't work too well here.  It looked fine on preview, but not once it was in there. 

My only excuses are that it worked on my machine and I ran out of time to test it - Aaaargh, I am going back to my developer days!

Now re-posted, not as a table. 

Tue 19 Aug 2008 - Bugs with emotions........

In my last entry, Joe commented on me assigning emotions to bugs.  That got me thinking that sometimes we testers do assign characteristics to bugs which are, after all, inanimate.  However, just for fun, I tried to come up with the type of emotions that bugs might have if they could feel.  I am sure that others could improve on these and/or add more.

Acceptance             A bug that accepts it will be found.  Typically a bug that is obvious as soon as you login (or even one that prevents you from logging in!

Affection                 A bug that loves other bugs.  Some bugs are gregarious and like to live together, these are generally found all in one place

Alertness                A bug that is alert to being found.  It shifts its position so that just as you think you have it nailed down, it moves somewhere else.  Memory leaks are good examples.

Ambivalence          A bug that doesn’t care whether it is found.  Typically a bug that you come across by accident

Anger                      A bug that hits you whenever you get near it, e.g. the blue screen of death

Angst                      See Anxiety

Annoyance             A bug that gets cross when it is found and causes another bug to be created once fixed

Anticipation            A bug that expects to be found and looks forward to it.  Typically a bug that keeps showing up throughout the application

Anxiety                   A bug that worries about being found.  These bugs hide in the corner and have to be winkled out with specially designed tests

Apathy                    See Ambivalence

Awe                         A bug that is amazed you have found it.  It pops up with unusual error messages like “Error: This should never happen!”

Boredom                 A bug that just gets fed up with being there.  These bugs usually go away of their own volition in the next release

Calmness                A bug that can’t get you excited.  Typically, a cosmetic bug, such as a mis-spelling

Compassion            A bug that cares about you.  This type of bug gives you an indication of what the problem is with a sensible error message.

Confusion               A bug that is not sure whether it exists or not.  Typically this type of bug can cause endless arguments with developers

Contempt                A bug that really thinks testers are lowly creatures.  Usually manifests itself with incomprehensible techno-speak in the error message

Contentment          A bug that is happy with its lot, it likes being there.  Usually these bugs are difficult to eradicate as they just like being there.

Curiosity                 A bug that looks for different parts of the application to get into to see what sort of damage it can do there.  This type of bug is often due to a continual bad coding practice throughout the application

Depression             A bug that throws itself at you suicidally.  Normally found early on in the testing cycle

Desire                     A bug that finds testers incredibly attractive.  Every tester on the project keeps finding this bug, usually finds its way into the bug tracking system multiple times.

Disappointment      A bug that is disappointed you didn’t find it.  Usually shows itself in Live

Disgust                   A bug that finds testers disgusting.  Generally gives a supercilious error message

Doubt                      A bug that doubts its cause.  Could be one of a number of different reasons why this bug exists

Ecstasy                   A bug that is high on illegal substances.

Embarrassment      A bug that is embarrassed to be found.  Generally has error messages like “Sorry, you can’t do that”

Empathy                 A bug that feels for the tester.  Typically shows itself after you have had your morning coffee and never last thing at night

Emptiness               A bug that has no content.  This type of bug is where functionality is missing that should be there.

Enthusiasm             A bug that has to show itself spectacularly, typically a system crash

Envy                       A bug that wishes it could be like other bugs.  Typically this is a bug that looks serious, but in reality is only a minor problem

Fanaticism              A bug that goes round telling other bugs how to create mayhem.  Typically a comms bug that goes round multiple systems.

Fear                        A bug that fears being found.  A particularly difficult bug to find and has to be tempted out into the open

Frustration             A bug that wishes it could be a working system Tries hard to work, so only shows itself intermittently

Gratification           A bug not found until system in Live.  This is a happy bug has defeated all attempts to find it and has finally achieved its aim of thwarting you

Gratitude                A bug that thanks you for finding it.  When it is found it shows you the position of other bugs, so you get multiple bugs shown on one screen

Grief                       A bug that is inconsolable at the death of one of its fellow bugs and gives up trying to hide.  Found when a previous bug, now fixed, was hiding this bug

Guilt                        A bug that regrets being there.  These bugs are keen to atone for their sins and are, therefore, easily fixed

Happiness              Found during “Happy path” testing.

Hatred                    A bug that hates you is one that you think you have found, but keeps coming back to haunt you in every release

Hope                       A bug that hopes not to be found.  This type of bug just sits there and waits.  It may be an easy bug to find or a difficult one, but is always obvious

Hopelessness         See Despair

Hostility                  A bug that shouts at you.  Usually shows itself with an error message ALL IN CAPS

Humiliation             A bug that is so ashamed at being found that this part of the system never displays any bugs ever again

Hysteria                 A bug that shouts and screams all over the application, causing all around to react with panic.  Usually a shows as a system crash

Inspiration              A bug that is as a result of some really clever coding brought about through an inspired developer….who got it wrong.

Jealousy                 A bug that shows the symptoms of another type of bug which is superior to it

Kindness                A bug that is kind to the developer by being easy to fix

Loneliness              A bug that is a hermit.  This type of bug is isolated from the rest of the application in an area that has no other bugs

Love                       Another gregarious bug.  Not only does this bug congregate with other bugs, it actively hangs on to them, so you find two, three or more all doing the same thing

Lust                        This is a bug that spawns other bugs.  A prolific breeder which means both tester and developer are forever chasing its children

Nostalgia                Bugs aren’t what they used to be.  This bug bemoans the fact that it hasn’t been found by Grace Hopper

Panic                       This bug causes all the parts of the system to go into meltdown.  Typically this is a tight loop that consumes all of the CPU

Patience                  This bug just takes its time.  Often seen as a Performance bug

Pride                       A bug that thinks it is the best bug in the whole system. Often surfaces before a fall-over

Repentance            A bug that feels bad about existing and once fixed, fixes a number of other bugs as a side effect

Resentment            A bug that does not like to be found, such that once fixed it re-appears again and again

Shyness                  A bug that is really hard to find and only surfaces in rare circumstances

Suffering                 A bug that causes the entire system to suffer.  Often found as a result of security testing

Surprise                  A bug that does not expect to exist.  Typically a simple bug that should have been found in unit test

Wonder                   A bug that is amazed to exist and cannot understand how a developer could possibly have coded it that way

Aug. 8, 2008 - Getting to know PageTest (Online Version)

I spent most of today giving valuable time to AOL’s PageTest, it’s extremely easy to use and the demo presentation by Dave Arts really did prove invaluable to me when analysing my report.  There are 2 versions of PageTest that users can choose from, a Download version and an Online Version.  I’ve been using the Online Version, and found 1 bug in the results page.

Testers, eh! Who’d have them?  We are ALWAYS on duty.

I’m sure I’m not the only one who’s seen this bug and reported it.  It could be just a server issue, but that's not for me to sort out.

Step 3 of PageTest gives the option to choose from 1 to 10 the amount of times you want to run your test. The higher the number, the longer it will take to run your test. The test result page shows the results broken down per ‘Run’, and users are able to click on each of the results to see more detailed report for the ‘Waterfall’ and ‘Optimization Checklist’ charts. However clicking on ‘Run 10’ for both the ‘Waterfall’ and ‘Optimization Checklist’ brings a ‘Not Found’ bug.

Come on AOL, you can do better.

PageTest also provides Test History – A clickable log of ALL previous tests and their results, the earliest recorded test is 2008/02/02, and it was for AOL, so, I’m guessing this was when PageTest was made open-source.  Once opened, it defaults to the last item on the list (test results are listed in chronological order with the most recent entry being the last on the list). Though this is a useful log, personally I feel that it should be made clear that everyone will be able to access your test results

So, note to me: Do not use PageTest online version for a site in development that you do not want the general public to have access to.

Naturally, when ‘nosing’ around the test history to see what others are up to and how performance compare, I couldn’t help myself and found the same ‘Run 10’ bug, on any website test results.

My final thoughts on PageTest online version, its fast (even when you run 10 tests), breakdown details given of test results are really useful to the less technical amongst us, useful to have a permanent url that can be circulated to interested parties and raw data provided for test results is great for those of us that love figures and making them into fancy charts.

Fri 8 Aug 2008 - Old Dogs....New tricks.

My dog, Rusty, is getting old, he will be 14 years old next month and we have had him since he was 12 weeks old.  He is a terrier cross with all the bad traits of the terrier, but all of the good ones too. He has always been “good” at killing; rats, mice, rabbits have all fallen victim to him over the years.  He is now blind in one eye, mainly deaf and gets out of breath after playing for more than about 10 minutes, but he still enjoys life and otherwise is very healthy for his age.  Recently, we have had a rabbit visiting our garden every day and Rusty tried a couple of times to catch it, but the rabbit was too quick for an aged dog.  After the first couple of failures, Rusty gave up chasing the rabbit.  The rabbit was eating my plants and despite putting down animal deterrent chemicals, sharp sticks, cages round valued plants, the **** thing kept eating the plants.  It started to become more and more blasé about me, my wife or Rusty coming into the garden when it was there, lolloping off slowly rather than fleeing in panic.  Rusty kept eyeing it up, but didn’t chase it.  Last night, Rusty got it – the rabbit got too close, was too slow and too complacent.  Part of me was delighted to get rid of the pesky thing, part of me was saddened at the death of such a marvellous creature.  I got to thinking, however, did Rusty play the game of letting the rabbit get more and more complacent with the thought in his mind that sooner or later he would get it? Or, did he just see the opportunity and go for it?  He is an intelligent dog.  Yes, I know everyone thinks their dog/child/pet is the best/most intelligent/etc. but he really is intelligent, so maybe he did play the patient, long term game.  

What has this got to do with testing?  As often happens, my mind turned to the similarity between life and testing.  Bug hunting is not just about going full blast as fast as you can hunting down those pesky bugs, sometimes that fails, sometimes we have to play the long term game.  If we are suspicious of an area in the system, we may have to be patient, try a different strategy.  Intermittent bugs are one area where this strategy is best employed.  Patience is key to finding an intermittent bug, determining what are the conditions that cause the bug to appear, trying different strategies, minutely changing some parameters to see what might cause the bug.  Finally, to complete the analogy, the bug becomes too complacent at not being found and we are able to catch it and kill it.  Strangely, there is delight at finding the pesky thing, but some sadness over the death of such a marvellous creature.

Real life and testing sometimes are too close together! Or maybe I need to get a life and stop thinking about testing….

Aug. 7, 2008 - I stumbled upon PageTest - web performance measuring tool by AOL

Wed 16 Jul 2008 - Why is it so difficult to find good testers?

Who would have believed that it would be so difficult to recruit decent test automation staff?

I have now been looking for someone to beef up my automation team for over 3 months and only just found someone who is worth hiring.  I have not been looking for someone out of the ordinary; at least I don’t think so.  I have been looking for someone who is experienced in test automation, minimum of 3 years, has written automation frameworks and who has a reasonable idea of testing principles.  QTP experience was desirable, but the key attribute was ability to automate.  Salary would not be too much of a problem for the right person.

I have had over 60 CVs to go through, about 20 telephone interviews and face to face interviews.

Firstly, the CVs.  Many people putting themselves forward for automation roles have only used capture/replay.  This surprised me as I had the, obviously mistaken, belief that the days of doing only capture/replay had gone years ago; it appears not and the old myth of capture/replay being the way to automate seems to be alive and well.  And some of the salary requests were outrageous.  Many CVs were from software testing consultancies or outsourcing companies and the level of knowledge displayed by these people was woeful which only went to strengthen my opinion that software testing consultancies and outsourcing companies mainly body-shop and are only interested in placing people not skills.  But, I digress.  Many CVs came from people who had just arrived in the UK from India, which has given me the impression that there is quite a few migrants coming this way; which was interesting but most of the skills were not up to the mark and the CVs were, generally, quite badly written – although, having said that, the person I am going to hire has only just come to the UK from India.

Secondly, the interviews.  I find telephone interviews useful to weed out the obvious no-hopers and people who do not tell the whole truth on their CVs.  Being an interviewee over the phone is as difficult as interviewing over the phone, but the number of people who just could not communicate over the phone was staggering.  The ones I got in for face to face interviews faired not much better.  I don’t much like tests in interviews, but for an automation role, a very brief test was given, to write a short VBscript to solve a common problem (finding one item from one array in another array, finding the maximum of three numbers) and I was amazed at how many either could not do it or got it hopelessly wrong – one candidate even wrote the “VBscript” in “C” and that was wrong!  There were some that had blagged the telephone interview that soon came unstuck.  Most had (or said they had) the ISEB foundation certificate and could not answer simple questions that ISEB poses.

Thirdly, the recruitment agencies.  I was explicit in my requirements, I even provided some sample questions with expected answers for them to ask candidates before I even got to see the CV.  However, the agencies persisted in sending totally unsuitable CVs and people who could not answer the sample questions – not one single agency has been good that we have used.

But the thing that really makes me think about this whole exercise is that there are a lot of people out there who don’t know how to do a testing job and/or an automation job.  They have been working, most for over 3 years, doing a testing job.  The skill level is, therefore, low.  No wonder that some people think anyone can do testing.  It saddens me and I wish I knew how we testing professionals could change that.

10 July 2008 - WAHOOO!!!!!!

24 June 2008 - Someone Help me!!!!

12 June 2008 - Still Waiting for backup......

6 June 2008 - What will I be?

23 May 2008 - Project Managers, Friend? or Foe?

23 May 2008 - Casual office attire

21 May 2008 - Sorry!!

12 May 2008 - Junior Senior?

Fri 9 May 2008 - The Test Sat Nav

My wife has an interest in “New age” beliefs, some of which she believes in, some of which she is sceptical about.  Me?  I’m always sceptical, however, I try to be as open-minded as possible and see what I can gain from any beliefs.  Last night, we watched a DVD called “Law of Attraction” which, if I boil down 1hr 50 minutes into a single sentence, is “if you want something, focus on it until you get it”.  OK, I think I was taught that when I about 5 years old, but hey, maybe some people need a reminder.  

One of the analogies used interested me from a testing perspective (yes, I am getting to testing eventually!).  The analogy was of a Satellite Navigation system in a car.  You tell the Sat Nav where you want to go (your goal), it works out the best route for you and then tells you how to get there.  The premise in the DVD is that we all have messages being given to us by non-physical means to guide us to our goal and all we need to do is listen to these messages and we will get there and achieve our goal. Personally, I found this hard to stomach, but maybe that is just me; there are too many variables in life for this to make any sort of sense and do I believe that I have this non-visible, non-physical “presence” sat with me all the time trying to guide me as to what to do?  No, I do not.

Getting back to having a goal, the Sat Nav and testing.  I got to thinking; do we have one overriding goal when we are testing? Wouldn’t it be great if there was a system where we can plug in that goal and it will tell us how to get there?  Call it the Test Sat Nav (TSN)

I got to thinking that probably we do have a goal and that goal is to complete the testing successfully.  The problem with that statement is what does “successfully” mean?  I would guess that every Test Manager has their own, slightly different, interpretation of what “successfully” means and every Project Manager has their own interpretation that will be different from the Test Manager and every customer will have their own interpretation that will be different from the Project Manager and the Test Manager.  So if we can’t agree a common meaning for the goal, how can we ever achieve it? And, therefore, how could we ever design a TSN that would enable us to plug in that goal and come up with the means of getting there?

So, maybe the goal is wrong, maybe it needs to be better defined, indeed if I put into the Sat Nav that I want to go to Birmingham, Birmingham is a big place and the chance of getting exactly where I want is not high. So maybe my goal is to meet the exit criteria?  Possibly better in terms of being able to tell the TSN, it might be easier.  But the problem with that is that we all do things during testing that are not directly related to the exit criteria.   Imagine putting No 10. Broad Street, Birmingham into my Sat Nav and deciding halfway up the motorway that I need to use the men’s room in the service centre?  That isn’t catered for in my instructions to the Sat Nav and it gets confused when I turn off the road it told me to go on and it isn’t part of my exit criteria.  In testing we go in a different direction than planned because we feel we have to, something needs a different focus, priorities change.  If on my road to Birmingham, the water company has decided to dig up the road and I get diverted round different streets, the Sat Nav is continually trying to get me back to where it thinks I should be, but I can’t because the road is blocked off.  In testing we get diverted by what happens to us, bugs being found that cause us to stop testing, changes that happen to the project and we have to take a different route.  The exit criteria haven’t changed, but the route there has. 

By this time, I am now wondering if we have a goal in testing at all that we can quantify, plan for and know we have met.  My idea of the TSN looks dead before it has started, if we don’t know where we are going, how can we plan to get there?  

Then, the “Eureka” moment occurred.  The totally unrealistic goal we have is that everything is tested 100% and there are no bugs of any kind in the delivered software. We, therefore, cannot get to that goal, it is an impossible goal; all we can do is travel towards that goal and at some point decide we have gone far enough.  So, testing does not have a destination that can be measured in the way of knowing you have got there, definitively.  All you know is that you have got sufficiently close to the destination and you do not need to go any further and you know that because of the measurements and metrics that you have been taking and, in the final analysis, by using your judgement. 

Having decided that, the TSN is easy, all I need is a program that will predict the changes that will happen along the way, the diversions that will happen and be able to predict the judgement call.  In fact, all I need is another New age belief, that of psychic prediction – I’ll get my wife to sort this out for me.

8 May 2008 - Tester going Solo!!!

7 May 2008 - Just when you thought it was safe.....

7 May 2008 - It's a Big Galaxy out there!!

Mon 20 Aug 2007 - Am I "too nice"?

I went to an interview last week.  I was looking forward to the interview as I had heard good things about the company and they knew what good testing was all about.  It looked like a good opportunity and the project sounded interesting.

I liked the interviewer and I answered all the questions as honestly as I could.  Most of the questions were situational, of the format “what would you do if….?” Which makes you think about what you would do based on your own experience.

At the end of the interview, the interviewer told me that I would not be getting the position.  I asked why and was told that I was “too nice” for the role.  I know a few people who would disagree with any sentence that had my name and the word “nice” in it, however, the interviewer explained that the role required someone who, in the interviewer’s words, was a “b*****d” due to the nature of the project and the personalities involved and I didn’t have the personality required.  It became clear that the project had an aggressive character; there was a lot of political manoeuvring; conflict was the norm.  A female friend of mine calls such projects TDD.  No, not “Test Driven Development” but “Testosterone Driven Development”.

I left the interview somewhat disappointed and deflated.  One the way home, I reflected on the answers I had given in the interview and tried to work out what answers the interviewer was looking for.  I came up with some answers that maybe would have fit the bill and worked out that, had I given them, they would not have been a true reflection of the way I like to work.  I finally had to agree with the interviewer, I was not the right person for the position and I would not have done a good job in it.  Clearly, the interviewer was a good judge of character!

However, I was still non-plussed by the “too nice” tag.  I have been called a few things as a Test Manager, but “too nice” has never been one of them!  So, I examined the answers I had given and compared them with the ones that the interviewer wanted and I came to the conclusion (which I knew already) that I like to work in a co-operative environment where everyone is working as a team towards a common goal.

A co-operative environment does not mean that you are always nice to each other, hugs and kisses around the meeting room are not going to help the project, and I have my fair share of violent disagreements and arguments but they are almost always concerned with the project, not personalities.  I firmly believe that the co-operative environment gets the job done with better quality, lower cost and quicker than in a non-co-operative environment.  If you are all pulling together with the same aim and following the same agenda, things work better.

But, what do you do if you find yourself in an aggressive environment?  It has happened to me as I am sure it has happened to a number of us.  My first reaction is to be co-operative, even in that environment, and it can work, I have proved it.  I have one particular example where I turned an aggressive environment into a co-operative one.  OK, it took me some months and a lot of hard work, but I got there.  If trying the co-operative approach does not work, then you may have to get aggressive yourself, but if it is not in your nature you will find it difficult – I find it difficult and can only keep it up for a short period of time.

So, going back to this position I didn’t get.  A part of me wonders if I could have turned it around and made it into a co-operative environment.  I will never know.  The interviewer would not take the risk with me and in the interviewer’s place I wouldn’t have taken the risk with me either, the interviewer made exactly the right decision, in my opinion.

I’ll stick with believing in the co-operative environment, but I doubt I will ever hear anyone call me “too nice” again!