Selling Exploratory Testing

One of the difficulties I expected to have when introducing Exploratory Testing was having to sell Exploratory Testing to the customer and to my own management.  What I didn’t expect was having to sell it to my own testers.  Some of them had used ET before, some had not.  Those that had needed no persuading, they already knew the benefits that ET gave.  Some of those that had not used it before were very sceptical and could not see how you could test something without scripting it first.  

This surprised me, as testers, by their very nature, explore the system under test whether they are scripting or not (see James Bach’s articles on this on www.satisfice.com).  I tried to determine why there was this resistance to a way of testing.  The answers I came up with are, inevitably, based on a very small sample of testers (c40) and are, therefore, just my view on what I saw.  [Perhaps someone will want to pay me for a larger study J]

1) The way a lot of us have been trained in IT has been to document first, then do.  Just “doing” is considered to be unprofessional and cavalier, akin to hacking.  So just testing can have an uncomfortable feel to someone trained that way.  

2) There is the attitude that if you can’t see what I have done, then you can’t measure how good I am.  This attitude stems from the CYA attitude that comes from bitter experience that if something goes wrong, you had better have some evidence to show that you did your job.

3) In my own shop, we had the customer crawling all over us in the early days, distrust was rife, so everything had to be specified minutely and thoroughly reviewed by the customer.  It was drummed into the test team that documentation was everything.  This legacy meant that changing the way of working was difficult purely because it was a change and some people don’t like change.

4) Related to point 3, there is a fear of the unknown.  Testers who had never done ET before were nervous of failing when doing something new.  Even though they were experienced testers, it was an approach that they had to learn.

How did I get round this resistance?  

Myself and one of my Test Team Leaders who was experience in ET did a number of courses to the teams on what ET is, what the benefits are and how we were going to implement it on our project.  As part of the course, the team had to do some ET.  This ET was on a piece of the system that everyone knew, as it is core to the system and supposedly very stable.  Each of the courses found problems in this part of the system that we didn’t know were there.  None of the problems were serious, none would have caused us to stop shipping, but it the made the point that ET can find problems that had not been found.

We also put in place Session Based Testing (as defined by Jonathan Bach) and this helped, because the testers could see that there is documentation associated with ET and that the documentation was useful, even if it is totally different than what they were used to.

Now ET is a major weapon in our armoury and you wonder how we ever got along without doing it in the past.