Testing a Model or Prompt Change Before Users See It: Golden Sets, Graders, and Shadow Runs

A model swap or a prompt rewrite carries a quieter risk than most incidents: not that the new version is worse, but that nobody measured it before users did. LLM regression testing and prompt change verification exist to make that measurement routine. The working set is three artifacts and five steps. The artifacts: a golden set of real inputs with known-good answers, graders that score new outputs, and a shadow run that puts the change beside live traffic before it replaces anything. The steps: define success criteria, build the set, choose the grading method, run the shadow, then apply thresholds decided in advance. This is the first piece in the site's change-verification layer; the migration playbook covers what to do when a model you depend on goes away, and this one covers proving a change before you adopt it.

LLM regression testing: define success criteria before anything else

The official evaluation guidance for Claude starts with a sentence worth framing: building a successful LLM application starts with clearly defining your success criteria and then designing evaluations to measure them, and that cycle is central to prompt engineering. The same guidance adds that most use cases need multidimensional evaluation along several success criteria, which is the part teams skip when they reduce "did the change break anything" to a single number.

Useful criteria are specific, measurable, achievable, and relevant. "Good performance" is none of those; "accurate sentiment classification on the labels we defined" is. For an API integration, four categories cover most needs:

• Task fidelity. Does the output actually answer the user's request, including on rare or difficult inputs?
• Format and structure. Can downstream code parse it, and does it respect the contract your system depends on?
• Safety and refusal behavior. Does the model refuse what should be refused, and not refuse what should not be?
• Cost and stability. Usage levels and error rates act as proxies here, which is the same guardrail role the cost guide gives verification in its order of operations.

Write each criterion as a sentence a reviewer could decide on. If you cannot decide whether a criterion passed by reading its definition, no downstream score will be comparable.

Golden set evaluation: sample from production, organize by edges

A golden set is a frozen collection of inputs paired with expected outputs, and its first rule is origin: sample from real production traffic, do not invent the inputs. Invented examples test the model against your imagination, which is precisely the part of the problem that is already solved.

The second rule is organization by edge categories, not by convenience. The official example evals organize their cases around the inputs that historically break systems, including sarcasm, mixed sentiment, misleading titles, abrupt topic shifts, reliance on context from much earlier in a conversation, and sensitive information such as protected health information. Steal that taxonomy and adapt it: every category in your set should correspond to a way your application has been confused before, or could be.

On size, keep it qualitative and strict in both directions: small enough that the whole set runs on every meaningful change, including inside CI, and broad enough that every edge category you identified is represented. A set that takes too long gets skipped; a set that covers only the happy path gives false confidence. Version the set and the grading logic together, and review changes to either the way you review code, because the set is now part of your system's contract.

LLM graders, code-based checks, and choosing the judge

Grading is where the process becomes mechanical, and the official guidance orders the methods by reliability: code-based grading first, because it is the fastest, most reliable, and most scalable option; human review when judgment demands it, at real cost; and LLM-based grading when the question is nuanced enough that rules cannot express it. In practice, working down that order gives you:

1. Code-based checks. Exact match against the golden answer (output == golden_answer, typically after normalizing whitespace and case) and string-match checks (key_phrase in output) cover a surprising share of cases, and cost nothing to run. Machine-checkable output formats make this category bigger, which is the broader lesson of making outputs structurally verifiable.
2. LLM-based grading. For tone, coherence, or qualitative standards, a model judge follows a rubric and scores the output. Two documented tips matter: write detailed, clear rubrics, and generally use a different model to evaluate than the model that generated the output, since self-evaluation carries its own bias.
3. Human review. Most flexible and high quality, but slow and expensive; the guidance is to avoid it if possible. Reserve it for calibrating the other two methods.

On the OpenAI side, evals package this as a describable test: an eval is configured with data_source_config, the schema for the test data, and testing_criteria, the graders that determine whether the model output is correct; runs take test inputs and produce results you analyze and iterate on, in a loop the documentation compares to behavior-driven development. The same page states the thesis of this article directly: writing evals to understand how applications perform against expectations, especially when upgrading or trying new models, is an essential component of building reliable applications. Graders are the scoring side: they compare reference answers against model-generated answers and can award partial credit rather than a strict pass or fail, with documented types including string checks, text similarity, and code-based checks, and they can be nested and combined. Both pages carry transition notes about the evolving eval tooling: treat the mechanics as living documentation, verify details against the official pages at the time you build, and check the deprecation notes for what applies to your account today. The evals page states the timeline plainly: OpenAI is deprecating the Evals platform; existing evals content remains available during the transition window; Evals will become read-only for existing users on October 31, 2026; and the platform is scheduled to shut down on November 30, 2026. Graders are being deprecated as part of the evals and fine-tuning workflows they support. For the current timelines, see the official deprecations page: https://developers.openai.com/api/docs/deprecations#2026-06-03-evals-platform. The practical case follows directly: the golden set and the grading logic are your own assets, so version them, keep them portable, and treat the hosted features around them as replaceable.

Shadow traffic testing: run the change beside production

The last artifact before a switch is the shadow run: copy a slice of real traffic to the new model or prompt, record everything, return nothing to users. Because the shadow path never answers a user, it is the only way to see how a change behaves on today's actual inputs rather than yesterday's golden set, at zero user risk.

Compare the same surfaces you would monitor in production, on both paths:

• format compliance, measured as how often output parses and validates;
• refusal rates, split by category;
• the distribution of error responses, using the status-code families from standard triage;
• rate-limit and rejection behavior under real concurrency;
• usage levels and cache-hit behavior, so cost surprises surface before they bill;
• mechanism-level behavior: tool-calling patterns such as which tools get called with which arguments, and whether streams terminate cleanly; the tool calling guide covers why these surfaces merit their own regression checks.

Two disciplines keep the shadow run honest. First, the shadow path must not consume production quota or add latency to the user-facing path; if it does, you have built an incident, not a test. Second, shadow records must be tagged as such and kept out of production metrics and dashboards, because a metric that silently mixes shadow and real traffic corrupts every comparison you will want later. The tool-calling comparison, for instance, exists because a change can pass every text check and still alter how tools are invoked.

Thresholds and the rollback switch

Decide the switch criteria before the first run, in writing, or you will end up tuning the standard to whatever result showed up. Thresholds should be qualitative and specific at once:

• key use cases must show zero regressions against the golden set;
• every failure on an edge case must have an explanation, not a shrug;
• cost and usage indicators must stay inside the budget envelope you set for the change.

The mirror requirement is a rollback switch: the old model or prompt must be one configuration change away from being restored, and you should actually rehearse restoring it once, on purpose, before you need it under pressure. Then write the conclusion down next to the sample set, the same way model selection decisions record their reasoning in the constraint-based selection guide, because the person running the next change will read your notes before they read the docs.

Common failure patterns

• Only the happy path is covered. Fix: organize the golden set around the edge categories above and review coverage per category.
• The model grades itself. Fix: use a different model for evaluation, or prefer code-based checks wherever the answer is decidable by rule.
• An average improves, so the change ships. Fix: keep key cases on a zero-regression requirement, evaluated separately from aggregate scores.
• Mechanism surfaces are ignored. Fix: put tool-calling behavior and stream termination explicitly on the regression list, not just text quality.
• Shadow traffic leaks into production metrics. Fix: tag it, isolate it, and keep the dashboards clean.
• No threshold until after results. Fix: write the switch criteria first; they are part of the experiment design, not its conclusion.
• Nothing is recorded. Fix: the sample set, grading logic, run results, and decision go into version control together.

The verification checklist

1. Write success criteria first, multidimensional and decidable, before designing any test.
2. Sample the golden set from real inputs; organize it by edge categories; keep it runnable on every change.
3. Order your grading methods by reliability: code-based checks first, LLM judges for nuance with clear rubrics and a separate evaluator model, humans for calibration.
4. Shadow-run the change against real traffic; compare format, refusals, errors, limits, usage and cache behavior, and mechanism-level behavior such as tool calls and stream endings.
5. Tag shadow data separately and keep it out of production metrics and quotas.
6. Set thresholds before the run and keep a rehearsed one-step rollback.
7. Record the set, the grading logic, the results, and the decision together, so the next change starts from evidence.

None of this is exotic machinery, and that is the point: the difference between a change you chose and an incident you caused is usually a set of cases, a scoring method, and one run beside production. When your calls go through a compatible endpoint, the endpoint-level specifics live in the platform's own documentation.