Python process pools: importable workers and serialization costs

Operation contract

The receipt checksum is defined at module scope so spawned workers can import it. The main guard creates a two-worker pool using an explicit spawn context, avoiding accidental pool creation during worker import. map returns results in input order even if tasks finish in another order. The fixture submits only three bounded integers and uses no external service.

Failure and ownership boundary

Pool setup and data transfer can cost more than the arithmetic here. A closure or open connection is not an appropriate worker payload. Waiting with a timeout also does not terminate arbitrary already-running work; executor shutdown can still wait for it. Python thread pools: collect results and observe worker failures and Python subprocess: argument vectors, exit codes and bounded fixtures need different shutdown decisions.

Working program

from concurrent.futures import ProcessPoolExecutor
from multiprocessing import get_context

def receipt_checksum(receipt_id):
    if type(receipt_id) is not int or not 1 <= receipt_id <= 1000:
        raise ValueError("receipt ID out of range")
    return receipt_id * receipt_id % 97

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=2, mp_context=get_context("spawn")) as pool:
        results = list(pool.map(receipt_checksum, [41, 42, 43]))
    print(results)

Output

[32, 18, 6]

Costs and limits

For n fixed-size jobs, serialization and collection grow with n, plus process startup cost. This fixture is not a CPU speed comparison and does not bound an arbitrary worker’s memory or execution time.

Common Mistakes

• Keep spawned worker definitions importable and pool startup behind the main guard.
• A future wait timeout is not a guarantee that running work has stopped.

Connected lessons

Python modules: separate import-time definitions from program execution, Python thread pools: collect results and observe worker failures, Python subprocess: argument vectors, exit codes and bounded fixtures.