tldr: Test metrics measure the health of your testing and the quality of what you ship. A few are genuinely useful; many are vanity numbers that look productive and drive bad behavior. The skill is tracking the handful that change a decision and ignoring the rest.
Metrics worth tracking
Useful metrics tie back to a decision someone makes. The ones that earn their place:
- Escaped defects: bugs found in production, not testing. The clearest signal of how well testing is working.
- Defect density: defects per area or per change, to find where quality is weakest.
- Test coverage of critical paths: whether the flows that matter are tested, not raw line coverage. See test coverage metrics.
- Mean time to detect: how fast a regression is caught after it lands.
- Flaky test rate: the share of tests that fail nondeterministically, because flakiness erodes trust in the whole suite.
Metrics that mislead
Some popular numbers actively cause harm:
- Raw test count: more tests is not better testing. It rewards volume over value.
- Line coverage as a target: chasing 100% produces tests that execute code without asserting anything meaningful.
- Bugs found by tester: turns QA into a numbers game and punishes the person who prevents bugs early.
Goodhart's law applies. The moment a metric becomes a target, people optimize the number instead of the outcome it was meant to represent.
What to track
Pick a small set tied to decisions: escaped defects to judge effectiveness, flaky rate to judge trust, critical-path coverage to judge risk. Three honest metrics beat a dashboard of twenty.
Watch trends, not absolutes. Escaped defects rising over three releases is a real signal. A single number with no baseline tells you nothing.
Tying metrics to release decisions
Metrics should feed the go or no-go call. If critical-path coverage has gaps or escaped defects are climbing, that is information for the release decision, not a report filed and forgotten.
Continuous, maintained coverage keeps these numbers meaningful, because a rotting suite produces flaky failures that drown the real signal. See regression testing and test maintenance for what keeps the data trustworthy.
FAQs
Which test metrics matter most?
Escaped defects, defect density, critical-path coverage, time to detect, and flaky test rate. Each ties to a real decision about quality or risk.
Why is line coverage a bad target?
Chasing a coverage percentage produces tests that run code without meaningfully asserting behavior. Coverage of critical paths matters more than a raw number.
What is the single best quality metric?
Escaped defects: bugs that reach production. It directly measures whether testing is catching what it should before release.
Why avoid "bugs found per tester"?
It rewards volume and punishes prevention. A tester who stops bugs early looks worse on the metric while doing the more valuable work.