Evals are the product

Every LLM project hits the point where the prompt stops being the artifact you care about and the eval suite takes its place. Rewriting a prompt takes five minutes. Knowing whether the rewrite was better takes far longer, and staring harder at the outputs does not close that gap. The evals are the only thing standing between 'feels better on the three examples I tried' and 'is better across the cases that matter.'

Most teams write their evals once, watch them go green, and never look again. Unit tests survive that treatment. Evals do not, because LLM behavior drifts with every model bump, prompt tweak, and retrieval change, and a stale eval set stops measuring the thing you actually ship. I have had better luck treating the set as a living spec. Every bug report becomes a case. Every regression becomes a case. The suite grows toward the real distribution of how the system gets used, which is the only distribution worth scoring against.

Graders are where this gets hard. Exact-match works for a narrow slice of tasks. Everything else leaves you two options. A rubric that an LLM applies, or a human reading the outputs. LLM-as-judge scales, and it also inherits the same biases as the system under test, which is a real problem rather than a footnote. So I anchor the judge against a small human-labeled set first. If it disagrees with people there, it has no business scoring thousands of cases.

The payoff is speed. Once the suite reflects reality you can refactor a pipeline hard, because the evals tell you within minutes what you broke. That is the whole advantage. A team that can answer 'did that change help?' with evidence will outrun a team trading prompt tricks, and it will not be close.

Here is the rule I hold to. Write the eval before the feature feels done, not after it breaks in production.