Writing acceptance criteria for AI coding agents

Ask a coding agent to "add password reset" and you'll get password reset. Probably. Maybe without the expiry check, maybe with an email template that says "TODO", maybe with a second, slightly different reset flow next to the one it built yesterday. The agent didn't do anything wrong: it stopped when the request looked satisfied, and the request didn't say what satisfied means.

Acceptance criteria fix that. They're the short list of things that must be true for a piece of work to count as done. For people they're good practice; for agents they're the difference between work you can trust and work you have to re-check line by line.

Why agents need them more than people do

A person who's unsure what "done" means asks. An agent usually doesn't; it fills the gap with a reasonable guess and keeps going. Three things make that worse:

• No memory between sessions. Tomorrow's session doesn't know what today's assumed.
• Confidence. Agents report success fluently. Without criteria, "done" is a feeling.
• Scope drift. Without a boundary, agents add things nobody asked for, or leave out the part that was hard.

Criteria give the agent a stopping condition and give you a checklist for review.

What good criteria look like

Good acceptance criteria are:

• Observable. Someone (or a test) can check them without reading the agent's mind. "An expired link shows a 'request a new link' page", not "handle expiry properly".
• Specific about behaviour, not implementation. Say what must happen, not which function to write. Leave the agent room to follow the codebase's patterns.
• Few. Three to five per story. If you need ten, the story is too big.
• Including the edge that matters. The empty state, the error, the permission check: whichever one would bite in production.

Before and after

A vague story:

Add password reset.

The same story with criteria:

Password reset by email

- "Forgot password?" on the sign-in page asks for an email and always
  shows the same confirmation, whether or not the account exists
- The email contains a single-use link that expires after 30 minutes
- Opening an expired or used link shows "This link expired" with a
  button to request a new one
- Setting a new password signs the person in and ends their other sessions
- Works at 360px wide

The second version is barely longer, and every line is something you can check in a minute. It also quietly answers questions the agent would otherwise guess at: account enumeration, expiry time, what happens to other sessions.

Sizing stories for agents

The best story size for an agent is one session: something it can build, test and hand back without the context window filling up or you having to step in. In practice:

• Too big: "Build authentication." Several sessions, many decisions, and progress that's impossible to judge midway.
• Right: "Email and password sign-up with a verification link." One clear outcome, a handful of criteria.
• Too small: "Add the email field." Fine on its own, but a feature made of twenty of these is all overhead.

A feature is then a few right-sized stories in a sensible order: data model, then API, then UI, then the email, then the polish.

Patterns that work

Given / when / then, when behaviour is subtle

Given a story assigned to another agent,
when an agent calls start_issue on it,
then the call is refused and the story is unchanged.

Use it for permissions, state machines and anything with a "but only if". For simple UI it's overkill.

Include the verification

If the project has tests, make running them part of done: "unit and e2e tests pass" as a criterion, or once in your rules file as a working agreement so you don't repeat it on every story.

Name the non-goals

"Out of scope: social sign-in" in the story description stops an agent from deciding it would be nice to add.

Point at the files

A line like "likely files: src/server/auth.ts, src/app/login" helps the agent start in the right place and helps you spot two stories that would collide.

Patterns that backfire

• Implementation as criteria. "Create a ResetTokenService class" locks in a design and ignores what the codebase already has.
• Unverifiable adjectives. "Fast", "clean", "user-friendly". Replace with a number or a behaviour ("loads in under 1 second on a cold start", "the error says what to do next").
• Criteria that are really new stories. "Also add an admin page to see resets" is a story, not a criterion.
• Everything in one story. If the criteria span the database, three pages and an email provider, split it.

Examples by kind of work

UI work

Link list: search and empty state

- Typing in the search box filters links by title or URL as you type
- No matches shows "No links match 'x'" with a button to clear the search
- A brand-new account sees "Create your first link" instead of an empty table
- Keyboard: / focuses search, Esc clears it
- Works at 360px with no horizontal scroll

UI criteria should cover the states people forget: empty, no results, error, loading, small screens and keyboard use. Those are exactly the states an agent skips when nobody mentions them.

API work

DELETE /api/links/:id

- Deletes the link and its click history; returns 204
- Another workspace's link returns 404, not 403 (don't reveal it exists)
- A read-only token gets 403 with a message saying why
- Tests cover all three cases

Bug fixes

For a bug, the first criterion is the reproduction and the second is the regression test:

Fix: double-clicking "Save" creates two links

- Reproduce: double-click Save on the new-link form creates two links
- After the fix, a double-click creates exactly one link
- A test fails before the fix and passes after
- Other forms using the same submit pattern are checked and listed

Asking for the test that fails first stops the agent from "fixing" a symptom it never actually reproduced.

Refactors

Refactors need criteria too, mostly about what must not change: "all existing tests pass unchanged", "no public API changes", "the bundle doesn't grow". Without them, a refactor quietly turns into a rewrite.

Making agents use the criteria

Writing criteria helps only if the agent reads them at the start and checks them at the end. Three ways to make that happen:

1. Put them where the agent reads them. In the story itself, not in a chat message from last week.
2. Ask for them to be ticked one by one. An agent that has to mark each criterion as passing tends to actually check each one. In Klio, start_issue returns the story's criteria and the agent calls check_criterion as each really passes, then complete_issue to send it for review.
3. Keep "done" a human decision. The agent moves work to review; you check the criteria and approve. It's quick precisely because the criteria are a checklist.

Criteria and tests

Acceptance criteria and automated tests answer the same question from two sides: criteria say what must be true in words a person can check, tests prove it in code the machine can re-run. For agent work you want both, and the criteria are a good source for the tests.

A practical habit: for each criterion, ask "could a test check this?" If yes, ask the agent to write that test as part of the story. "An expired link shows the request-a-new-link page" becomes an end-to-end test that opens an expired link and looks for the button. "Another workspace's link returns 404" becomes an API test. Criteria that can't reasonably be automated ("the error message says what to do next") stay as things you check by eye in review.

This pays off twice. The agent has an objective way to know a criterion passes, instead of judging its own work. And the next agent that touches the same code, maybe weeks later and with no memory of this story, gets a failing test the moment it breaks the behaviour, rather than a bug report from a user.

One caution: an agent that writes both the code and the test can write a test that passes for the wrong reason. Ask for the test to fail first (before the change, or with the change reverted) whenever the behaviour is subtle. For bug fixes, always.

Letting the agent draft them

You don't have to write every criterion yourself. A good workflow is: you describe the feature in a sentence or two, the agent proposes stories with criteria, you edit and approve, then it saves the plan. You spend your time on judgement (the edge cases, the order, what's out of scope) instead of typing.

Plan "password reset by email" as a feature: 3-5 stories, each finishable
in one session, each with 3-5 checkable acceptance criteria. Include the
security edge cases. Show me before saving.

In Klio, the agent calls brief_feature to read the project's stack and existing features first, so the stories fit the code you already have, then create_feature once you've approved them.

A quick checklist

Before handing a story to an agent:

1. Can it be finished in one session?
2. Are there three to five criteria, each checkable by looking or by a test?
3. Is the risky edge case covered (errors, permissions, empty states, mobile)?
4. Are non-goals stated if there's an obvious tempting extra?
5. Does it say how to verify (tests, or the project's working agreement)?