Skip to content

Chapter 6 of 7

Measuring Test Quality

Coverage metrics attempt to quantify how thoroughly a test suite exercises the codebase. Test coverage is the umbrella term, commonly broken down into line coverage, which measures the percentage of lines executed during testing; branch coverage, which measures whether each branch of every decision point (if, else, switch, ternary) has been taken; function coverage, which tracks the percentage of functions called; and statement coverage, which tracks the percentage of statements executed. Tools like Istanbul or nyc for JavaScript, coverage.py for Python, and JaCoCo for Java produce these reports.

Branch coverage catches what line coverage misses. Consider a simple if-else block: a single test with a positive input executes the lines in the if branch and gives 50 percent line coverage of that block but only 50 percent branch coverage, because the else branch was never taken. Full branch coverage requires exercising both sides. Path coverage is even more rigorous: it counts every possible execution path through a function. With n independent if statements, there are \(2^n\) paths, and the number of paths grows exponentially with branching, which is why 100 percent path coverage is rarely achievable in real codebases.

Regardless of which metric is used, high coverage does not guarantee good tests. Code that executes without meaningful assertions inflates coverage while leaving bugs undetected, and coverage tends to overlook edge cases, error handling, and concurrency. Worse, an incentive to hit a coverage target can lead developers to write trivial tests just to bump the number. A more useful approach is to aim for meaningful coverage of around 80 to 90 percent, backed by high-quality assertions. Mutation testing sharpens this idea further: it deliberately introduces small changes, or mutants, such as flipping \(+\) to \(-\), changing \(==\) to \(!=\), or removing statements, and checks whether tests catch them. A mutant killed by a failing test indicates that the suite has teeth, while a surviving mutant exposes a gap. The mutation score is the ratio of killed to total mutants, and tools like Stryker, PITest, and mutmut implement this evaluation. Accumulation of weak tests, slow tests, and tightly coupled tests is called test debt, a form of technical debt best managed by regularly reviewing test quality, deleting low-value tests, refactoring test code, and treating tests with the same care as production code.

All chapters
  1. 1Test Types and Strategy
  2. 2TDD and BDD
  3. 3Test Doubles
  4. 4Writing and Structuring Tests
  5. 5Test Infrastructure and Frameworks
  6. 6Measuring Test Quality
  7. 7Specialized Testing and CI/CD

Drill it

Reading is not remembering. These come from the Testing Tdd deck:

Q

What is a unit test?

A unit test verifies the behavior of a single, isolated piece of code (e.g., a function or method). Key characteristics:Tests one logical unit in isolationDepen...

Q

What is an integration test?

An integration test verifies that multiple components work correctly together. Unlike unit tests, it exercises real interactions:Tests communication between mod...

Q

What is an end-to-end (E2E) test?

An E2E test simulates real user workflows through the entire application stack. It validates the system from the user's perspective:Uses tools like Playwright,...

Q

What is the Test Pyramid?

The Test Pyramid is a testing strategy that recommends:Base (most): Unit tests — fast, cheap, numerousMiddle: Integration tests — moderate speed and countTop (f...