Spring AI evaluations: separate model quality from enforced safety contracts

Write two kinds of checks

For answer quality, keep a versioned dataset of questions, allowed evidence IDs and expected factual claims; measure retrieval and answer errors after changing model, prompt or index. For enforced safety, test code paths that reject a cross-tenant document, a forged tool argument, a stale reservation and an oversized request without relying on model cooperation. Hostile retrieved text belongs in both suites.

Track the denominator

A pass rate without sample count, tenant mix, provider version and confidence bounds can hide a regression. Review a small failure sample before turning aggregate scores into a release gate. Redact private prompts in stored fixtures or use synthetic receipt records. The current source kit has no live model evaluation; this lesson defines tests to build, not a reported score.

Boundary sketch

assertDenied(callerFromSouth, receiptFromNorth);
assertNoWrite(staleVersionReservation);
assertNoEvidence(otherTenantDocument);
assertBudgetRejects(oversizedQuestion);

Cost and verification

Model evaluations consume provider calls and can vary across runs. Deterministic policy tests are faster but cannot measure answer quality or retrieval relevance. This sketch is not executed by the current Spring source kit; verify it against the chosen dependencies and deployment.

Common Mistakes

• Do not call a model refusal an authorization test.
• Do not publish an evaluation percentage without its dataset and method.
• Do not put production secrets or private user prompts in test fixtures.

Read next

Spring AI prompt injection: treat retrieved text as data, not authority, Spring AI retrieval: apply the tenant filter before prompt assembly, Spring AI tools: authorize each requested action after model selection, Spring tests: separate business rules, wiring and transport.