There are a number of ways of measuring the processes in a test organisation, CMMI, TPI being just two. It is all very well having these methods and any way of measuring is useful, but how do you go about determining where your problems are and how to resolve them?  TPI is great and it does provide some useful pointers, but what are you going to do in your shop?

A method I have found useful is a SWOT analysis.  For  anyone who does not know , SWOT stands for Strengths, Weaknesses, Opportunities, Threats. 

Strengths are those things you as a test team are good at.

Weaknesses are those things that you as a test team are not good at.

Opportunities are those things that are outside of the test team that if happened would be good for you

Threats are those things that are outside of the test team that if happened would be bad for you.

I have run this exercise a number of times in order to improve the processes that we use in test teams and it has proved quite successful.

You run the SWOT analysis as a workshop, or series of workshops and you will invite some of your key team members to participate.

The last time I ran the exercise at the end of 2005, I split the workshops into the following subjects:

• Test Preparation
• Test Execution
• Interfaces and communication
• Bug process
• Test Management
• Releases to Testing

The first set of workshops identified the SWOT for each of these subjects and then the second set of workshops identified the actions that we could take, who would take the actions and when they should be done by.

The actions were then monitored over a period to ensure that they got done.

It is difficult to measure the effect of each of the individual actions, but following the workshops, I can see that we are getting more effective and better at what we do.  The exercise will be done again soon, so that we should get a new set of actions to continually improve.

An example of the output follows:

Test Process under examination:  Bug Process

Strength:  Good Level of detail in bug reports

Weakness: Analysis of No Fault Founds take too long

Opportunities: Fix fail rate is improving

Threat: Bug reports from Help Desk have insufficient detail making them difficult to retest

Actions: Educate Help Desk into better bug reporting.  Analyse fix fail rate to improve further.  Work closer with developer on No Fault Founds,.

On the last exercise, we identified 66 action points that would make us more effective and/or efficient.  Most of these actions have now been completed and we will soon be ready to go again.