Hypothesis property tests: compare generated cases with an independent contract

Operation contract

The quantity aggregation test compares a streaming accumulator with built-in sum over bounded integer lists. A second property tries values outside the accepted element type/range and requires rejection. The settings use deterministic generation and no example database so the teaching run owns no persistent state. Assertions check behavior rather than a handpicked output alone.

Failure and ownership boundary

Passing generated cases is not a proof over every possible input. A strategy that never produces a relevant boundary cannot find that boundary’s failure. Avoid assuming a counterexample’s exact text or order stays fixed across versions. Pytest fixtures: isolated files and parameterized rejection tests and Python window exercise: verify a fixed-width maximum against direct slices complement generated properties.

Tested environment

Dependency check: this program was executed on CPython 3.14.6 with hypothesis==6.168.3. Install these versions in a separate virtual environment. The download includes the recorded environment snapshot; no third-party package is part of the website runtime.

Working program

from hypothesis import given, settings, strategies as st

def aggregate_quantities(quantities):
    if len(quantities) > 32:
        raise ValueError("batch budget rejected")
    total = 0
    for quantity in quantities:
        if type(quantity) is not int or not 0 <= quantity <= 20:
            raise ValueError("quantity rejected")
        total += quantity
    return total

@settings(max_examples=100, derandomize=True, database=None, deadline=None)
@given(st.lists(st.integers(min_value=0, max_value=20), max_size=32))
def accepted_property(quantities):
    assert aggregate_quantities(quantities) == sum(quantities)

@settings(max_examples=60, derandomize=True, database=None, deadline=None)
@given(st.one_of(st.booleans(), st.integers(max_value=-1), st.integers(min_value=21), st.text(max_size=8)))
def rejected_property(quantity):
    try:
        aggregate_quantities([quantity])
    except ValueError:
        return
    raise AssertionError("rejected field became accepted")

accepted_property()
rejected_property()
print("accepted and rejected properties passed")

Output

accepted and rejected properties passed

Costs and limits

Work depends on generated example count, input bounds and any shrinking after a failure. Disabling a deadline avoids noisy wall-clock failures in this fixture; it is not an application timeout policy. Both strategies impose small list/text bounds.

Common Mistakes

• A generated test can miss inputs excluded by its strategy.
• Use an independent answer or invariant instead of repeating the implementation.

Connected lessons

Pytest fixtures: isolated files and parameterized rejection tests, Python window exercise: verify a fixed-width maximum against direct slices, Python aggregation exercise: validate records before returning group totals.

Check this related boundary

Python testing: check a reconciliation relation without copying the implementation.