LLM Evaluation: From Scores to Release Decisions

Begin with a specific decision: release a prompt revision, switch a model, change retrieval, permit a tool action or hold a deployment for investigation. Define the failure that would change your answer. An unsupported instruction in a customer-facing response, an incorrect tool argument and a slow but correct answer may require different tests and different owners.

Write a short evaluation contract. State the application version, intended users, permitted behavior, unacceptable outcomes, dataset scope, reviewer and release authority. Include a rule for ambiguous results. A team that cannot agree what a failure means should not expect a model judge or a composite score to settle the disagreement automatically.

For a document assistant, separate whether relevant material was retrieved, whether the answer relied on that material and whether the final response met the user's task. For a tool-using agent, inspect the action and arguments as well as the explanatory text. The correct-looking final answer can conceal an unauthorized or incorrect intermediate action.

Prefer deterministic checks when the required property is explicit: valid structure, an allowed tool name, a required field or an exact identifier. Use a human rubric when the judgment requires context that is not captured by a simple rule. Model-based scoring can help organize that work, but it still needs a defined rubric and calibration against examples your team has reviewed.

The Ragas metric documentation organizes metrics around different evaluation concerns and required inputs; MLflow documents evaluation workflows and scorers. Those primary references are useful implementation starting points. They do not select your business threshold or establish that one available metric is suitable for every task.

Keep measurement and interpretation distinct. A result may show that an answer resembles a reference without establishing that the reference is correct. A judge may reward a fluent explanation without validating a tool action. Before adopting a score, create counterexamples that should receive different judgments and inspect whether the method actually distinguishes them.

Give every baseline and candidate observation the same case identity and scoring rubric. Divide conclusive pairs into four groups: both meet the rubric, only the candidate meets it, only the baseline meets it, and neither meets it. Keep unexecuted, failed and inconclusive observations outside those four groups. A missing response is not a negative label, and a baseline-only result cannot establish an improvement.

Consider an illustrative six-case review: two cases improve, one regresses, two remain acceptable and one candidate execution fails. Calling that a net improvement loses the most important questions. Was the regression an unauthorized action? Was the failed execution caused by the changed dependency? Record those cases before discussing an aggregate. Do not turn this small, deliberately chosen example into a population error rate or a statistical significance claim.

Keep repeated runs attached to their case rather than counting every repetition as an independent user. If one case alternates between acceptable and unacceptable outputs, retain the variability and investigate its cause. Compare the same application boundary: changing retrieval data, tool permissions and model version together prevents a reviewer from attributing an outcome to a prompt edit alone. The working comparison below records one summarized observation per side; keep individual repeat traces in your own evidence system.

Run the baseline and candidate on the same controlled inputs and record their configurations. Review newly failing cases, newly passing cases and cases where the judge and a human disagree. Show latency and cost separately from behavioral results so the team can discuss the trade-off explicitly rather than hiding it in an unexplained weighted score.

Write the release decision as a brief evidence memo: what changed, which groups improved, which regressions appeared, which exceptions were accepted and who owns follow-up. If the candidate is held, preserve the unresolved cases and the next experiment. This makes the evaluation useful even when it does not produce a release.

After deployment, use monitoring to collect new failure candidates, then review and sanitize them before adding them to the regression dataset. Online observations and a controlled experiment answer different questions. A useful toolchain preserves the relationship between them without treating production traffic as a complete test set or a benchmark score as proof of safe operation.