> The full index of these docs is at [llms.txt](https://u22a8.ai/llms.txt).

# Write a criterion

State a criterion A8 judges well, with one judgment per criterion, the situation that decides it and the practices that keep verdicts steady.

A **criterion** is what is being judged, stated in plain words. A8 reads the whole request: the conversation, the schema and the parameters. It finds the criterion, the subject and any standard wherever the request puts them. This page gives the practices that get a criterion judged well, and the clearest layout.

## 1 Principles

- **One judgment per criterion.** A criterion asks one thing. Two judgments are two criteria, each in a schema property of its own.
- **Name the situation that decides the answer.** “Does the reply promise a refund?” names what anyone reading the reply would see. A8 judges a criterion best when people reading the subject agree on its answer.
- **Give the verdict a shape.** A schema says what comes back: a boolean, one of an `enum`, or a number on a declared scale.
- **Send the standard with the subject.** A reference answer, a source or the question asked gives A8 something to judge the subject against.
- **Set the minimum accuracy in the model’s tag.** `a8:95` asks for 95. A8 reads words about certainty in the prompt as part of the criterion. See [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md).
- **Keep the wording fixed once you teach it.** A8 identifies a teach by its exact `input` and `format`, so the same words reach the same examples. See [Teaching](https://u22a8.ai/docs/a8-teaching.md).

## 2 Weak and strong, side by side

| Weak                                              | Strong                                                                                             | What changed                         |
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------ |
| Is this reply good?                               | Does the reply answer the question the customer asked?                                             | Names the situation that decides it  |
| Rate the reply’s accuracy and tone.               | Two criteria: “Does the reply answer the question?” and a `tone` property with its own description | States one judgment per criterion    |
| Is the reply correct? The refund went out Monday. | “Does the output match the reference?”, with `reference` beside the subject                        | Sends the standard with the subject  |
| Answer only if you are 95% sure. Is this spam?    | “Is the message spam?”, with the model set to `a8:95`                                              | Sets the minimum accuracy in the tag |
| Score the reply from 1 to 5 for politeness.       | “How polite is the reply?”, with an integer property from `1` to `5`                               | Gives the verdict a shape            |

*Table 1.* Criteria before and after: each strong version states one judgment, gives the verdict a shape and sets the minimum accuracy in the model’s tag.

## 3 A clear layout

A8 reads a prompt in any layout. The layout below keeps each part easy to find for the people who read your requests too, and it is the layout A8 reads as written with reasoning off:

- **The criterion in the system message.** One question, in plain words.
- **The subject in the user message.** Follow-ups go in later user messages, after A8’s earlier responses. See [API overview](https://u22a8.ai/docs/api-overview.md).
- **The shape in the response format.** A JSON Schema whose properties are the verdict’s fields.
- **The minimum accuracy in the model’s tag.** `model="a8:95"` asks for 95, in every interface.

### 3.1 The subject and its standards

Where a judgment relates the subject to something else, send the subject as a JSON object: `output`, plus any of `reference`, `context` and `question`. The keys are on [API overview](https://u22a8.ai/docs/api-overview.md#subject-context):

```python
import json

subject = {
    "output": "Your refund went out on Monday.",
    "context": ["Refund issued on Monday to the original card."],
}
messages = [
    {"role": "system", "content": "Is the output supported by the context?"},
    {"role": "user", "content": json.dumps(subject)},
]
```

A criterion taught against a standard is judged against that standard. A request that leaves it out abstains, and the abstention names the standard in `missing`.

### 3.2 The verdict’s shape

Each schema property is one field of the verdict. A `boolean` or an `enum` is a label, and a `number` or `integer` is a score on its declared scale. See [API overview](https://u22a8.ai/docs/api-overview.md#verdict-shape) for how A8 reads each property type.

### 3.3 Several criteria in one request

A property with a `description` carries its own criterion, and A8 judges it separately. The request’s criterion applies to the other properties. Each field answers or abstains on its own. See [Check one subject against several criteria](https://u22a8.ai/docs/check-one-subject-against-several-criteria.md).

## 4 Prompts written for other judges

A prompt that puts the criterion and the subject in one message, the way many eval frameworks write their graders, works as it is. A8 finds the criterion, the subject and the format the prompt asks for, such as `GRADE: C` or a trailing `Y` or `N`, and answers in that format. A8 reads the same request the same way every time, so it gets the same verdict.

A request that is small talk, a greeting, a statement or several questions at once comes back as an abstention of kind `no_question`. Send each question as a request of its own. Each tool’s page shows how to point an existing grader at A8: [OpenAI SDK](https://u22a8.ai/docs/openai-sdk.md), [Anthropic SDK](https://u22a8.ai/docs/anthropic-sdk.md), [promptfoo](https://u22a8.ai/docs/promptfoo.md), [Inspect](https://u22a8.ai/docs/inspect.md) and [DeepEval](https://u22a8.ai/docs/deepeval.md).

## 5 Common fixes

- **Every request abstains.** The criterion is untaught, or its evidence backs a lower level than the request asks for. Read `answerable_at` and `earned` on the abstention, then [teach](https://u22a8.ai/docs/a8-teaching.md) examples, or send the reference the criterion was taught against.
- **Two phrasings of one criterion behave differently.** Each phrasing is its own request with its own examples. Pick one wording and keep it.
- **A threshold in the prompt changes the verdict.** The words are part of the criterion. Move the threshold to the model’s tag.

## Next steps

- [Teaching](https://u22a8.ai/docs/a8-teaching.md): How to teach and retract through MCP, when a teach lands in your organization’s model and what it changes.
- [Check one subject against several criteria](https://u22a8.ai/docs/check-one-subject-against-several-criteria.md): Judge one subject on several criteria in one request, with each criterion in a schema property and each field answering on its own.
- [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md): How a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.