Software Quality Assurance and not only 

The roles of QA Analysts

• Find defects
• This is the classic objective of testing. A test is running in order to trigger failures that expose defects.
• Generally, we look for defects in all interesting parts of the product
  

• Maximize bug count
• The distinction between this and “find defects” is that total number of bugs is more important than coverage.
• We might focus on only a few high risk features, if this is the way to find the most bugs in the available time.
  

• Block premature product releases
• This analyst stops premature shipment by finding bugs so serious that no one would ship the product until they are fixed.
• For every release decision meeting, the tester’s goal is to have new show stopper bugs.
  

• Help managers make ship / no-ship decisions
• Managers are typically concerned with risk in the field.
• They want to know about coverage, and how important the known problems are.
• Problems that appear significant on paper but will not lead to customer dissatisfaction are probably not relevant to the ship decision.
  

• Minimize technical support costs
• Working in conjunction with a technical support or help desk group, the test team identifies the issues that lead to calls for support.
• These are often peripherally related to the product under test
• For example, getting the product to work with a specific printer or to import data successfully from a third party database might prevent more calls than a data-corrupting crash.
  

• Assess conformance to specification
• Any claim made in the specification is checked.
• Program characteristics not addressed in the specification are not checked (as part of this objective).
  

• Conform to regulations
• If a regulation specifies a certain type of coverage, the test group creates the appropriate tests.
• If the regulation specifies a style for the specifications or other documentation, the test group should check the style.
• In general, the test group is focusing on anything covered by regulation and (in the context of this objective) nothing that is not covered by regulation.
  

• Minimize safety-related lawsuit risk
• Any error that could lead to an accident or injury is of primary interest.
• Errors that lead to loss of time or data or corrupt data, but that don’t carry a risk of injury or damage to physical things will be out of scope.
  

• Find safe scenarios for use of the product (find ways to get it to work, in spite of the bugs)
• Sometimes, all that you’re looking for is one way to do a task that will consistently work.
• To do one set of instructions that someone else can follow, which will reliably deliver the benefit they are supposed to lead to.
• In this case, the tester is not looking for bugs. He is trying out, empirically refining and documenting, a way to do a task.
  

• Assess quality
• This is a tricky objective because quality is multi-dimensional.
• The nature of quality depends on the nature of the product.
• For example, a computer game that is rock solid but not entertaining is a lousy game.
• To assess quality – to measure and report back on the level of quality
• We probably need a clear definition of the most important quality criteria for a product, and then we need a theory that relates test results to the definition.
• For example, reliability is not just about the number of bugs in the product.
• It is often defined as being about the number of reliability-related failures that can be expected in a period of time or a period of use.
• Reliability-related – In measuring reliability, an organization might not care, for example, about misspellings in error messages.
• To make this prediction, you need a mathematically and empirically sound model that links test results to reliability.
• Testing involves gathering the data needed by the model.
• This might involve extensive work in areas of the product believed to be stable as well as some work in weaker areas.
• Imagine a reliability model based on counting bugs found (perhaps weighted by some type of severity) per N lines of code or per K hours of testing.
• Finding the bugs is important.
• Eliminating duplicates is important.
  

• Verify correctness of the product
• It is impossible to do this by testing.
• You can prove that the product is not correct or you can demonstrate that you didn’t find any errors in a given period of time using a given testing strategy.
• However, you can’t test exhaustively, and the product might fail under conditions that you did not test.