Property-Based Testing

Example-based tests check the inputs you thought of; the bugs live in the inputs you didn't. Property-based testing (PBT) inverts the contract: you state something that must hold for all valid inputs, and the framework generates hundreds of them — adversarially biased toward the nasty ones — hunting for a counterexample. Born as QuickCheck in Haskell, the technique is mainstream now via Hypothesis (Python), jqwik (JVM), and fast-check (JavaScript/TypeScript). It routinely finds boundary bugs — empty inputs, Unicode astral-plane strings, integer edges, NaN — that years of example tests miss.

Properties worth writing

The skill of PBT is finding properties, and a small taxonomy covers most real code:

Round-trips — decode(encode(x)) == x for any serializer, parser, or codec pair. The single highest-yield property in existence; if you have custom serialization anywhere, start here.
Invariants — outputs always satisfy a structural rule: a sort's output is ordered and a permutation of its input; a balancing tree stays balanced after any insert.
Oracles — the optimized implementation agrees with a slow-but-obviously-correct one: fast_search(xs, k) == naive_search(xs, k). Ideal when replacing or optimizing existing logic.
Metamorphic relations — when you can't state the right answer, relate answers across inputs: adding an item never decreases a total; querying with a stricter filter returns a subset.
Idempotence and commutativity — normalize(normalize(x)) == normalize(x); merge order doesn't matter.

Hypothesis in Python

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_sort_is_idempotent_permutation(xs):
    out = my_sort(xs)
    assert out == sorted(out)                 # ordered
    assert sorted(out) == sorted(xs)          # same multiset

@given(st.dictionaries(st.text(), st.integers() | st.text() | st.none()))
def test_config_roundtrip(d):
    assert parse_config(render_config(d)) == d

Strategies compose: st.builds(Order, item=st.sampled_from(SKUS), qty=st.integers(1, 999)) generates domain objects; .filter() and .map() refine; st.data() supports dependent draws. Hypothesis persists a local database of past failures and replays them first — a found bug stays found. The @example(...) decorator pins regression cases explicitly.

jqwik and fast-check

The same model translates directly. jqwik rides JUnit 5:

@Property
void roundTrip(@ForAll("orders") Order o) {
    assertThat(Order.parse(o.serialize())).isEqualTo(o);
}
@Provide Arbitrary<Order> orders() {
    return Combinators.combine(Arbitraries.strings().alpha(),
                               Arbitraries.integers().between(1, 999))
                      .as(Order::new);
}

fast-check mirrors it in TypeScript: fc.assert(fc.property(fc.array(fc.integer()), xs => ...)), with first-class async property support for testing promise-based code. Kotlin's kotest and Rust's proptest complete the mainstream coverage — the concepts transfer verbatim.

Shrinking: from random noise to minimal counterexample

When a property fails on a 400-element list, the framework does not hand you that monster — it shrinks, systematically simplifying the input while the failure persists, and reports a minimal case ([0, -1], the empty string, a single NaN). Shrinking is the difference between PBT being usable and being a noise generator, and it is why integrated shrinking (Hypothesis, jqwik, fast-check all have it) matters: the minimal counterexample usually states the bug almost verbatim.

Stateful and model-based testing

Beyond pure functions, PBT generates operation sequences against stateful systems, checking behavior against a simple in-memory model: Hypothesis's RuleBasedStateMachine and fast-check's commands API generate interleavings of put/get/delete/reopen against your cache, store, or state machine, comparing to a dict-backed model. This is a lightweight cousin of full model-checking and it excels at finding sequence-dependent bugs — the delete-then-reinsert-loses-TTL class — that no hand-written scenario covers.

Fitting PBT into a real suite

Placement — PBT complements example tests: keep examples as readable specification anchors and regression pins; add properties where input spaces are large (parsers, math, serialization, collections, anything with invariants).
Budget — default case counts (100/property) keep suites fast; a nightly job can raise max_examples 10–100x for deeper search without taxing PR feedback.
Determinism in CI — random generation means a property can fail on a case no previous run saw. That is the feature, not flakiness — but configure CI to print the reproduction seed/blob (all three frameworks support fixed seeds and failure replay) so any red run is exactly reproducible. Commit Hypothesis's failure database (or its CI cache) to make found bugs permanent regressions. See Managing Flaky Tests for the broader discipline.
Generator honesty — an over-constrained generator silently tests a toy subspace. Occasionally assert distribution statistics (jqwik's Statistics, Hypothesis's event()) to confirm the generator actually reaches the regions you care about.