Line coverage answers "did the tests execute this code?" — not "would they notice if it broke?" A suite can hit 95% coverage with assertions so weak that deleting half the logic passes green. Mutation testing measures the thing you actually care about: it plants small bugs (mutants) into your code, reruns the tests, and reports which bugs went undetected. The mutation score — killed mutants over total — is the most honest automated measure of test-suite strength available.
A mutation tool parses your code and applies mutation operators — systematic single-point changes drawn from real bug patterns: > becomes >=, + becomes -, a boolean flips, a conditional block empties, a return value is replaced, a method call is removed. Each mutant is your program with exactly one such change. The suite runs against each:
A survived mutant on code you care about means a missing or under-asserting test — frequently a test that exercises the path but asserts too little.
STRONGER/ALL groups exist when you want more aggression.mutmut run), results cached across runs, mutmut show <id> prints each surviving diff. (cosmic-ray exists for distributed runs; mutpy is abandoned.)All report per-file, per-mutant detail; all cost real compute — the suite runs once per mutant (mitigated by test targeting), which shapes how you deploy them.
Some mutants change syntax without changing behavior — mutating an optimization that only affects speed, altering a value that is always overwritten. These equivalent mutants are unkillable in principle, and detecting them is undecidable in general, so no tool eliminates them. Practically they run 5–15% of survivors: treat the mutation score as a directional metric, not a purity test, and never mandate 100%. Review survivors; mark genuinely equivalent ones ignored (all three major tools support exclusions/annotations) so the signal stays clean.
The compute cost is the other adoption barrier, and the answer to both problems is the same: don't run everything, everywhere, always.
Whole-codebase mutation runs belong in a weekly or nightly job at most. The pattern that makes mutation testing a daily tool is changed-code-only: mutate only the lines touched by the current diff and run only the tests covering them. The tooling support differs by ecosystem: Stryker's --incremental flag diffs against a stored report and re-tests only changed mutants; mutmut caches per-mutant results across runs and skips unchanged code automatically; open-source pitest ships incremental analysis (withHistory, reusing prior results keyed on class/test hashes), while true git-diff PR scoping on the JVM comes from the commercial Arcmutate git extension. Runtimes drop from hours to minutes, and the feedback lands where it is most useful — on the PR, about the new code, while the author still has context. A reasonable policy: PRs report (and optionally gate on) mutation score for changed lines; the full-project score is tracked as a trend, not a gate.
Survivors are not equally important. Triage order that works:
>=/> confusions in pricing, limits, permissions are exactly the off-by-one bugs that ship. Fix by adding boundary-value assertions.A cluster of survivors in one class is a code smell as much as a test smell — logic too tangled to assert on is often logic too tangled, full stop.
Keep line coverage as the cheap, always-on floor (it finds untested code instantly) and use mutation score as the periodic depth-check (it finds badly tested code). The pair costs little: coverage on every run, changed-code mutation on every PR, full mutation nightly. Teams adopting this combination consistently report the same discovery: the scary number isn't the coverage they lack, it's the assertions they lack in tests they already have.