Ten to fifty examples: the golden set to build before you change a prompt

Every prompt change is a small bet that the agent will get better and nothing else will get worse. Plenty of teams place that bet without checking.

The check is cheap. It's a short list of examples where you've already decided the right answer. Phase 2 of our AI Transformation Playbook, Gated Agents, calls it the eval set, and it's the smallest artifact in the phase. It's also the easiest one to skip.

What the playbook asks for

Play 6 of Phase 2 is one line long:

Eval set of golden examples. Ten to fifty "this is correct / this is wrong / this must never ship" cases. Run them before every prompt or tool change.

The same section lists the failure mode it prevents. It's called No golden set, and what it looks like is "Every change is vibes." The repair is blunt: "Freeze changes until eval exists."

That repair sounds harsh until you picture the alternative. Without a fixed set of examples, every change gets judged by whoever happens to try it that afternoon, on whatever inputs they think of. Sometimes that works. You just can't tell when it hasn't.

Why ten to fifty is enough to start

Teams put this off because they assume an eval needs hundreds of cases. Anthropic's engineering team makes the opposite case in Demystifying evals for AI agents (January 2026): "20-50 simple tasks drawn from real failures is a great start." Their reasoning is that early on, each change tends to have a clear, noticeable effect, so a small sample is enough to see it. More mature agents need bigger and harder sets later.

They add a warning that matches the playbook's freeze: evals get harder to build the longer you wait. Early on, requirements turn into test cases naturally. Wait, and you end up reverse-engineering what "correct" meant from a system that's already live.

The three kinds of example

The playbook's wording gives you three buckets. Each one does a different job.

This is correct. Real inputs paired with output you'd send as-is. Say your agent drafts replies to support tickets. A correct case is a refund question answered accurately, in your tone, with the right policy cited. These tell you the agent can still do the job.

This is wrong. Output that's plausible but not acceptable: the right policy with the wrong amount, or a reply that answers a question the customer didn't ask. These teach the set to catch near-misses, the errors a tired reviewer is most likely to wave through.

This must never ship. The cases where the answer isn't "needs an edit" but "stop." Promising something you can't offer. Sharing one customer's details with another. Anything your quality bar says never goes external. Keep these few and unambiguous. A change that breaks one of them doesn't go out, whatever else it improved.

Where the examples come from

You don't have to invent them. If you ran Phase 1, you already have a head start: the playbook asks each function to "Capture three 'good / better / don't ship' examples," and the phase ends with an example library. Those are your first golden cases.

After that, Anthropic's advice is to start with what you already check by hand before each release, and then go to your bug tracker and support queue. Real failures make the best cases, because they're proven to happen.

A few rules for writing them, also from Anthropic's post:

• Make each case unambiguous. A good task is one where two domain experts would reach the same pass/fail verdict on their own. If your own team would argue about it, rewrite it.
• Include a reference answer. A known good output proves the case can be passed and that your grading works.
• Test both directions. Include cases where the agent should act and cases where it shouldn't. A set that only checks one side pushes the agent toward doing that thing everywhere.

Run it before every change

The playbook says "before every prompt or tool change," and both halves matter. Swapping a tool, a model, or a retrieval source can shift behavior as much as rewriting the prompt.

Treat the set as a regression test. Anthropic describes regression evals as the ones that should hold a nearly 100% pass rate: they ask whether the agent still handles everything it used to. A drop means something broke.

Two practical notes from the same post:

• Run each case more than once. Model output varies between runs. Their example: an agent with a 75% per-trial success rate passes all three of three trials only about 42% of the time. One clean run doesn't prove much.
• Grade the result, not the path. Checking for an exact sequence of steps makes tests brittle. Check whether the output is right.

When a case fails, that's the gate doing its job. Don't ship the change, and don't quietly edit the case to make it pass. If the case itself turns out to be wrong, fix it in the open with a note on why. The playbook's failure drill includes "how you fix the golden examples," and that's part of the same habit. Every real failure in production should become a new case.

The step teams are skipping

There's some evidence this is getting less attention. KPMG's Q3 2026 AI Pulse, a survey of 314 US C-suite and business leaders at companies with $1 billion or more in revenue (fielded July 24 to August 25, 2026), found that the share of organizations building controls into their agents alongside monitoring and evaluation procedures fell to 30%, from 43% two quarters earlier. KPMG says many are relying on guardrails already in place. It's self-reported, and it measures controls broadly, not golden sets specifically. But it's the wrong direction for a step this cheap.

What to do this week

Pick the one agent or workflow you're most likely to change next. Pull ten real inputs from the last month. Write the correct output for six, one plausible-but-wrong output for three, and one must-never-ship case. Have two people grade them independently and rewrite the ones they disagree on. Then make a rule: no prompt or tool change goes out until it passes.

That's your golden set. It will grow toward fifty as real failures show up. The playbook's 90-day picture expects that, with the set "growing" during the pilot.

Get the playbook

The full Phase 2 section, with all seven plays, the exit artifacts, and the failure-modes table, is in the 231 Labs AI Transformation Playbook. It's free at 231labs.com/playbook.