Performance reviews are important tools in assessing the productivity and value offered to a company by its employees. They require the use of metrics that can be measured, documented, and compared to get an accurate idea of what an employee/team’s contributions are truly worth. Testers and QA teams, much like all other employees, should also be subject to the scanner to ensure that they are contributing adequately to the development efforts. As such, choosing the right metrics is paramount in ensuring that the QA team’s efforts are being duly recognized and measured.
Assessment of QA team performance
There are many models and approaches available to assess QA team’s performance, one way could be using the following metrics:
1. Defect Leakage:
This metric measures the share of defects that were not identified during System Testing (ST) but were recognized as part of User Acceptance Testing (UAT) or production environment. In other words, this is a measure of defects found after the deployment of the software. The formula for calculating defect leakage is:
DLP = (total defects found in UAT)/(total defects found in ST + total defects found in UAT)
2. Defect Rejection Ratio:
Using this metric, we get an understanding of how many defects identified were “worth resolving” by the developer. It is the ratio of total defects rejected by the developer to the total defects identified by tester. DRR is calculated as below:
DRR = (total rejected defects)/(total defects)
3. Test Execution Cycle Time:
This is the time required to execute the test cases for a standard release. One can observe whether this time is gradually increasing/ decreasing across multiple releases. A decrease in the test cycle time means that QA is finding new ways to verify the same functionality in less time.
4. Test Execution Productivity:
TEP is a measure of the amount of testing achieved per unit time. It is the ratio of the total number of test cases executed to the total effort applied. Total effort is always calculated in terms of hours. Its formula is:
TEP = (number of test cases executed)/(total effort applied for execution of tests)
5. Error Discovery Rate:
Error Discovery Rate (EDR) measures the coverage of test cases employed. It measures the number of defects identified per test case. Higher the number of identified defects, higher is the coverage of the test case. One issue with this metric is that it fails in cases where code being tested is perfect in its current form.
EDR = (number of defects identified)/single test case
There is no single straightforward way of measuring a QA team’s success/effectiveness. Different metrics present different insights about the team’s successes and shortcomings. The ideal measure of success is, thus, an overall consideration of information provided by each of the metrics, and undeniably, a good customer feedback.