Pandas groupby(dropna=False): keep a missing category visible

Operation contract

Three receipt amounts include one with an absent region. A default groupby would leave that amount out of the regional totals; the fixture uses dropna=False, then prints the named region and the missing-key total separately. The accounted sum equals the raw sum, so no amount disappears in the report.

Failure and ownership boundary

A missing region is not automatically a valid business category. It may need rejection or separate remediation before publication. Pandas nullable integers: missing amounts are not zero, Pandas groupby: retain missing keys and define all-null totals and Python reconciliation exercise: reject duplicate IDs before comparing ledgers help define that decision.

Tested environment

Dependency check: this program was executed on CPython 3.14.6 with pandas==3.0.6, numpy==2.5.3. Install these versions in a separate virtual environment. The download includes the recorded environment snapshot; no third-party package is part of the website runtime.

Working program

import pandas as pd

receipts = pd.DataFrame({"region": ["north", None, "north"], "minor": [125, 75, 50]})
totals = receipts.groupby("region", dropna=False)["minor"].sum()
print("north:", int(totals.loc["north"]))
print("missing:", int(totals[totals.index.isna()].iloc[0]))
print("accounted:", int(totals.sum()) == int(receipts["minor"].sum()))

Output

north: 175
missing: 75
accounted: True

Costs and limits

Grouping n rows requires work and memory proportional to the input and number of groups; exact implementation costs depend on dtype and key distribution. The equality check is useful but does not prove each row was classified correctly.

Common Mistakes

• Default groupby can hide missing-key amounts from a grouped report.
• Keeping a missing group is a visibility policy, not proof the data is valid.

Connected lessons

Pandas nullable integers: missing amounts are not zero, Pandas groupby: retain missing keys and define all-null totals, Python reconciliation exercise: reject duplicate IDs before comparing ledgers.