One thing I have been interested in over the years is the question of determing when testing is done. When enough testing has been completed and the product is ready to be shipped. In the past I have used completion metrics such as Defect Resolution Rate (how fast are you fixing things and when are you fixing more than your finding) and Test Execution Status to aid in this task. Test Execution Status can be the amount of coverage of the requirements (test cases covering requirements) and their pass/fail rate along with execution rate (tests planned and executed) of the tests. This show the traditional growth curves when plotted. It give you an idea of how much you have done. Which helps to determine where you are at in the cycle.
An interesting thing I have read about in relation to the Agile methods (SCRUM in particular) is the 'Burn Down Rate' which is used to show how much is left to do for 'stories' (requirments/use cases) over a specified time period. It is a downward sloping curve that can used to say you have so much left until the cycle is complete. Considering this is more in line with what we get asked (how much testing is left to do, and when will it be done by) by management it may be a more effective way of also showing how much testing is left. Instead of stories you plug in the test cases for those stories, and you plot the trend of how fast you are 'burning down' the list of things to do. This way you could say to management we have X percent of test cases left to execute and we look to have them done by Y date. This is what they want to hear.
I have not done this on a project yet, but am looking forward to trying it out. After reading this if you have implemented it please respond back with your experiences. If not, then maybe this will inspire you to change your perception of test execution status reporting & monitoring, and to see if you can 'burn down' some of your testing.
As you can see from the WebLog title I am a bit sarcastic and cynical about this thing we call Software Testing. Over my years of experience in Software Development and Testing I have seen some very very Dilbert things happen.
Hopefully this Blog will be a good place for you to learn from some of the things I have experienced and allow you to be more effective in your efforts in Software Testing.
• June 5, 2006 - Burn it down
One thing I have been interested in over the years is the question of determing when testing is done. When enough testing has been completed and the product is ready to be shipped. In the past I have used completion metrics such as Defect Resolution Rate (how fast are you fixing things and when are you fixing more than your finding) and Test Execution Status to aid in this task. Test Execution Status can be the amount of coverage of the requirements (test cases covering requirements) and their pass/fail rate along with execution rate (tests planned and executed) of the tests. This show the traditional growth curves when plotted. It give you an idea of how much you have done. Which helps to determine where you are at in the cycle.
An interesting thing I have read about in relation to the Agile methods (SCRUM in particular) is the 'Burn Down Rate' which is used to show how much is left to do for 'stories' (requirments/use cases) over a specified time period. It is a downward sloping curve that can used to say you have so much left until the cycle is complete. Considering this is more in line with what we get asked (how much testing is left to do, and when will it be done by) by management it may be a more effective way of also showing how much testing is left. Instead of stories you plug in the test cases for those stories, and you plot the trend of how fast you are 'burning down' the list of things to do. This way you could say to management we have X percent of test cases left to execute and we look to have them done by Y date. This is what they want to hear.
I have not done this on a project yet, but am looking forward to trying it out. After reading this if you have implemented it please respond back with your experiences. If not, then maybe this will inspire you to change your perception of test execution status reporting & monitoring, and to see if you can 'burn down' some of your testing.