Model
Accuracy and abstention
How a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.
Applies to a8-1. Last reviewed 2026-10-22.
This page as Markdown: https://u22a8.ai/docs/a8-abstention.md. The index of every docs page: https://u22a8.ai/llms.txt.
The minimum accuracy is the accuracy a criterion’s evidence must show before A8 returns a verdict: an integer from 0 to 99, set on each request. An abstention is the declined verdict A8 returns where the evidence falls short of the minimum accuracy. Cases below the minimum accuracy come back as abstentions, so the rate promised for the answered verdicts holds.
Responses report the level a verdict was served at as min_accuracy, and name it in the tag of their model.
§1Minimum accuracy
The minimum accuracy is one integer from 0 to 99, 90 by default. At 95, at least 95% of the verdicts A8 returns at that level are correct. You set it in the model’s tag, the same in every interface, so the value you set is the value that applies, however the prompt is worded.
§1.1The model’s tag
The tag follows the model after a colon: a8:95 asks for 95, and a pinned identifier carries it before its horizon, as a8-1-1:95@20260715t140322z.
- The tag is a whole number from
0to99. Anything else, such asa8:100ora8:9.5, returns400namingmodel. - By default, the minimum accuracy is 90, in every interface and over MCP.
- The response’s
modelnames the level that answered,a8-1-1:90@20260715t140322zfor a request at the default, so a pinned identifier copied from a response holds the level still too. - Effort and
top_pleave the minimum accuracy alone. Effort is each interface’s reasoning switch, andtop_pis ignored liketemperature.
The tag travels in model, which every SDK and framework passes through. The identifier’s other parts are on Model identifiers.
§1.2The two ranges
| Value | Meaning |
|---|---|
1 to 99 | Answer where at least that share of answers is correct |
0 | Answer wherever a reading exists, with the verdict alone and its interval left out |
Minimum accuracy is one scale. At 0 it requires 0% accuracy, so every request with a reading is answered.
§2What the promise means
At min_accuracy: N, at least N% of the verdicts A8 answers are correct, on traffic that resembles the material the level was earned on. The promise is a rate over the answered set: across the verdicts returned at level 90, at least 90% are right. Coverage is the share of requests answered: the price of the level, measured on its own.
What counts as correct depends on the verdict’s shape:
| Verdict | Correct means |
|---|---|
| A label: a category, or a yes or a no | The label matches. At level 90, at least 90% of returned labels are the right one |
| A score: a number on the schema’s declared scale | The returned interval contains the true value. At level 90, at least 90% of intervals do |
“Correct” is defined by the verdict’s shape: a label matches, or an interval contains the true value.
A8 returns a verdict when the evidence behind its criterion meets the minimum accuracy. The trade is coverage for accuracy: a higher minimum accuracy raises the share of answers that are correct and lowers the share of requests that are answered. The hard cases arrive as abstentions, and the answers carry a rate you can measure against the promise.
§3Earned levels
Each accuracy level is earned separately, per criterion, on examples held out of teaching. A brand-new criterion abstains at every level above 0, and the levels a criterion has earned form an uneven set. Evidence accumulates in whole samples, so achievable rates move in steps: a criterion’s earned levels can be 70, 90 and 95.
A request lands on the nearest earned level at or above the minimum accuracy it asks for, and abstains when every earned level is below it. Every answer reports the level it was served at, so a request for 91 can be answered at 95:
{
"min_accuracy": 95, // the level this verdict is promised at
"choices": [{"message": {"content": "{\"refund\": true}"}}]
}
Asking for more than a criterion has earned is a useful probe: the requests that abstain at 95 and answer at 90 measure how much evidence that criterion still needs.
§4An abstention
An abstention is a successful response with status 200. In place of a verdict, a top-level abstention says which check the evidence failed and at what level the same request is answered. In Chat Completions it looks like this:
{
"choices": [
{
"message": {
"content": null,
"refusal": "Too close to call at min_accuracy 95; answered at 80."
}
}
],
"abstention": {
"kind": "too_close",
"asked": 95,
"answerable_at": 80,
"earned": 97,
"missing": null
}
}
Recorded from a8-1-1@20260715t140322z on 2026-10-22.
| Field | Meaning |
|---|---|
kind | The check the evidence failed: one of the five kinds below |
asked | The min_accuracy the request asked for |
answerable_at | The highest min_accuracy at which this same request is answered. 0 means only a minimum accuracy of 0 answers it, and null means every level abstains |
earned | The highest level the criterion has earned, or null until it earns one |
missing | The standards the criterion was taught against and the request left out: reference, context or question. null when the request sent them all |
The fields of an abstention. Send the request again with the model tagged at answerable_at, as a8:80, and it returns a verdict.
The refusal text sits where each API puts an answer the model declined to give. The text is for people to read; branch your code on kind. Each interface page shows its own form.
With reasoning on, A8 also filters some requests whose text is harmful, such as hate speech, violence or self-harm. A filtered request sits in the slot each API keeps for content its filters stop, apart from abstentions, and is the same at every level. Each interface page shows its form.
When a schema asks for several fields, each is judged on its own criterion, so some fields answer while others abstain. The abstaining fields are null in the verdict, and their abstentions arrive keyed by field:
{
"abstention": {
"fields": {
"tone": {
"kind": "level_not_earned",
"asked": 90,
"answerable_at": 70,
"earned": 70,
"missing": null
}
}
}
}
Recorded from a8-1-1@20260715t140322z on 2026-10-22.
§4.1The five kinds
kind | What happened | What clears it |
|---|---|---|
no_evidence | A criterion that scores is untaught and the request sends the subject alone, or the criterion was taught against a standard the request left out | Send the standards named in missing. When missing is null, teach the criterion, or send a reference or context in the subject |
level_not_earned | The criterion’s evidence backs a lower level than the one asked for. An untaught criterion that picks a label answers at 0 alone | Teach it more, or ask at answerable_at |
unfamiliar_subject | The subject sits outside the material the criterion has seen, and the promise covers that material | Teach examples like it |
too_close | The criterion has earned the level and the subject is familiar. This case sits close to the line | Ask at answerable_at, or send the case to a person |
no_question | The request holds something other than one question to judge: small talk, a greeting, a statement, several questions at once, a recall in a verdict format | State one thing to judge. This kind is the same at every level |
Five things produce an abstention, and each points at a different remedy.
The first three concern the criterion’s evidence and shrink with teaching. The fourth concerns the subject and persists at some rate at every level above 0. The fifth concerns the request itself.
§5Auditing the promise
Every answer reports its served level, so you can audit the promise. Run subjects whose right answer you already know, and group the answers by the level they were served at. Within each group, the share that came out right should clear that group’s number.
- Abstentions leave the denominator. The accuracy rate is defined over the answered set.
- Track coverage beside accuracy. The share of requests answered is what a level costs, and it is the number to watch when raising one.
- Group by the served level. It sits above the asked level whenever a request resolved upward.
- On the Decisions API, cut at a threshold. Each answer’s number is the strictest level it clears, so the answers at or above
0.95together hold the promise of 95. See Decisions. - Small counts move. A 90% promise can show 87% on 30 answers and still hold. Compare against an interval.
A group that misses its number over enough answers to mean something is a defect in the promise. Report it to [email protected]. Testing has a script that scores this way, and so do the configs for promptfoo and Inspect.
§6Edge cases
| Case | Behaviour |
|---|---|
A minimum accuracy of 0 | A8 answers wherever a reading exists, with the verdict alone. The promise starts at 1 |
| A brand-new criterion | Abstains at every level above 0 until teaching earns one |
| A score near the boundary | Answers: a score honours the level by the width of its interval, and the width states how far the evidence constrains the value |
| A criterion whose traffic has shifted | Levels earned on the old material are withdrawn, and every level above 0 abstains until teaching on the new material earns them again |
Four situations with behaviour of their own.
The last is the promise’s one standing condition: incoming requests resemble the material the level was earned on. When you point A8 at a different population, such as a new domain, another language or output from a different system, the subjects under judgment differ from the ones the criterion’s evidence describes. Known limits: a8-1 lists this and the other behaviours with their remedies.
§7Related concepts
- Teaching: how a criterion earns its levels
- Testing: measuring accuracy and coverage on your own questions
- Precedents: the taught examples behind a verdict
- Horizons: the version a level was earned on