Using A8

Check one subject against several criteria

Judge one subject on several criteria in one request, with each criterion in a schema property and each field answering on its own.

One request, several verdicts. Put each criterion in a schema property of its own, and each field answers or abstains on its own evidence.

§1Motivation

A subject is often judged on several questions at once: which queue a ticket belongs in, whether it is urgent and how polite it is. Each is its own criterion with its own taught examples and its own earned levels. Sending them in one request keeps the subject in one call, and keeps each judgment apart: a field whose evidence falls short abstains, and the others answer.

§2Example

A support desk reads each ticket for its queue, its urgency and its tone.

§2.1Step 1: one property per criterion

A property with a description carries its own criterion. The system message is the criterion for the remaining properties, here queue:

Python

import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.u22a8.ai/eval/v1", api_key=os.environ["U22A8_API_KEY"]
)

schema = {
    "type": "object",
    "properties": {
        "queue": {"enum": ["billing", "shipping", "account"]},
        "urgent": {
            "type": "boolean",
            "description": "Does the customer need an answer today?",
        },
        "tone": {
            "type": "integer",
            "minimum": 1,
            "maximum": 5,
            "description": "How polite is the message?",
        },
    },
}

resp = client.chat.completions.create(
    model="a8",
    messages=[
        {
            "role": "system",
            "content": "Which queue does the ticket belong in?",
        },
        {"role": "user", "content": "It arrived crushed. I want a refund."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "ticket", "schema": schema},
    },
)

§2.2Step 2: read each field and its abstention

A field that abstains is null in the verdict, and its abstention arrives keyed by field:

JSON

{
  "choices": [{
    "message": {
      "content": "{\"queue\": \"billing\", \"urgent\": true, \"tone\": null}"
    }
  }],
  "abstention": {
    "fields": {
      "tone": {
        "kind": "level_not_earned",
        "asked": 90,
        "answerable_at": 70,
        "earned": 70,
        "missing": null
      }
    }
  }
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.

Python

content = resp.choices[0].message.content  # null when every field abstains
verdict = json.loads(content) if content else {}
declined = (resp.model_extra.get("abstention") or {}).get("fields", {})

for field in schema["properties"]:
    if field in declined:
        review(field, declined[field]["kind"])  # your own function
    else:
        act(field, verdict[field])  # your own function

§3How the settings were chosen

One minimum accuracy for the request. The model’s tag sets one value, and every field is held to it. To hold a field to a different minimum accuracy, send it in a request of its own.
One criterion per property. A property that asks two things is two criteria. Split it, so each earns its levels on its own examples. See Write a criterion.
Teach each criterion on its own. A teach covers one criterion, so a correction to tone is a request whose system message is the description, “How polite is the message?”, and whose format holds tone with its type and range alone. See Teaching.

Next steps

Write a criterionState a criterion A8 judges well, with one judgment per criterion, the situation that decides it and the practices that keep verdicts steady.Accuracy and abstentionHow a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.TeachingHow to teach and retract through MCP, when a teach lands in your organization’s model and what it changes.