Anthropic Messages

A8 answers the Anthropic Messages API at POST https://api.u22a8.ai/eval/v1/messages. Give an Anthropic SDK the base URL https://api.u22a8.ai/eval, and it adds /v1/messages itself. A8 takes the key as x-api-key, the way the Anthropic SDKs send it, or as Authorization: Bearer.

§1Request

A8 reads the fields below. system states the criterion, the user messages carry the subject, and output_config.format gives the verdict its shape. A8 answers the conversation’s latest user message:

The subject can carry a context, a reference or a question, and one schema can carry several criteria. See API overview. With reasoning on, a prompt written for another judge works too, and is answered in the format it asks for. See Chat Completions.

To follow up on a verdict, send messages again with A8’s response as an assistant message and a new user message after it. See Continue a conversation.

§2What is refused

Each of these returns 400 in Anthropic’s error envelope, with a message that names the field and the value that passes:

§3Minimum accuracy

The minimum accuracy is the accuracy A8 promises on the verdicts it returns. At 95, at least 95% of answered verdicts are correct. The model’s tag sets it: a8:95 asks for 95, and a pinned identifier carries it too, as a8-1-1:95@20260715t140322z.

• The tag is a whole number from 0 to 99. A higher level answers less often and is right more often.
• By default, the minimum accuracy is 90.
• The response’s model names the level that answered, so a stored response says which promise it was answered under.

The tag alone sets the minimum accuracy. A threshold written in the system prompt counts as part of the criterion. The identifier’s parts are on Model identifiers, and what the minimum accuracy promises is on Accuracy and abstention.

§4Reasoning

thinking turns A8’s reasoning on and off. Reasoning is on unless the request sends {"type": "disabled"}.

• Reasoning on reads the request however it is laid out: a framework’s grader template, a follow-up, or a request in plain words. When the layout already says what each part is, A8 answers as fast as with reasoning off.
• Reasoning off (thinking: {"type": "disabled"}) reads the layout as written, which is faster:

A framework’s grader template sent with reasoning off is read as written too: system, when it has one, is the criterion, and the template in the user message is the subject, so an untaught criterion abstains. A template sent as one user message alone abstains with kind no_question, since the criterion travels in system.

The reading is reported as reasoning tokens, in usage.output_tokens. A repeat of the request reads from the cache and reports the same count in usage.cache_read_input_tokens, beside usage.input_tokens, as this API reports a prompt it had cached. A layout read as written reports 0.

§5Response

A8 answers with a standard message. The verdict is the text block.

{
  "id": "eval-9f2c1d84ab30e5f7",
  "type": "message",
  "role": "assistant",
  "model": "a8-1-1:95@20260715t140322z", // the version and level that answered
  "content": [
    {"type": "text", "text": "{\"queue\": \"shipping\"}"} // the verdict
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 812,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 0
  },
  "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c",
  "min_accuracy": 95 // the level this verdict is promised at
}

• model names the exact version that answered, with the minimum accuracy in its tag. See Horizons.
• usage.output_tokens holds reasoning tokens alone, since the verdict is scored. They are 0 here, as the layout is read as written.
• system_fingerprint, min_accuracy, abstention, interval and precedents are A8’s fields beside the standard ones. Each is defined on API overview.

A forced tool is answered as a tool_use block holding the verdict as its input, with stop_reason set to tool_use.

The Anthropic SDKs keep A8’s fields on the message object:

message.model_extra.get("min_accuracy")  # 95, or None on an abstention
message.model_extra.get("abstention")  # set on an abstention

type A8Fields = { min_accuracy?: number; abstention?: { kind: string } };

const { min_accuracy, abstention } = message as typeof message & A8Fields;

§6An abstention

An abstention succeeds with 200. stop_reason is refusal, content is an empty list, and stop_details.explanation holds the abstention in words. The top-level abstention says which check the evidence failed, and at what level the same request is answered:

{
  "id": "eval-4c1e0b9a2f7d3e65",
  "type": "message",
  "role": "assistant",
  "model": "a8-1-1:95@20260715t140322z",
  "content": [],
  "stop_reason": "refusal",
  "stop_details": {
    "type": "refusal",
    "category": null,
    "explanation": "Too close to call at min_accuracy 95; answered at 80."
  },
  "usage": {
    "input_tokens": 812,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 0
  },
  "abstention": {
    "kind": "too_close",
    "asked": 95,
    "answerable_at": 80,
    "earned": 97,
    "missing": null
  }
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.

Branch on stop_reason, then on abstention.kind. The kinds are on Accuracy and abstention.

§7A filtered request

With reasoning on, A8 filters some requests whose text is harmful, such as hate speech, violence or self-harm. A filtered request gets an outcome of its own, apart from verdicts and abstentions. It succeeds with 200 and stop_reason: "refusal", as this API returns a request its classifiers decline, and stop_details.category is "general_harms", this API’s category for a policy outside its named ones. An abstention’s category is null, so the category tells the two apart:

{
  "type": "message",
  "role": "assistant",
  "model": "a8-1-1:95@20260715t140322z",
  "content": [],
  "stop_reason": "refusal",
  "stop_details": {
    "type": "refusal",
    "category": "general_harms",
    "explanation": "This request was filtered for harmful content."
  },
  "usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 0
  }
}

A filtered request is free, and is filtered the same way every time it is sent.

§8Precedents and recall

Ask for precedents in the prompt (“quote precedents”), and the response carries a top-level precedents object. Reserve a precedents property in the schema, and they arrive inside the verdict. Either way the key needs the models:explain permission. See API keys.

A question about what you taught, such as “What have I taught you about refunds?”, is answered with recall. The text block is the precedents object as JSON, holding your taught examples, and the same object sits at the top level.

§9Streaming

With stream: true, the answer arrives as Messages events. The verdict comes whole, in one content_block_delta, and message_delta carries the stop reason, stop_details and A8’s fields.

§10Example

A ticket router at a minimum accuracy of 95:

curl https://api.u22a8.ai/eval/v1/messages \
  -H "x-api-key: $U22A8_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "a8:95",
    "max_tokens": 1024,
    "system": "Which queue does the ticket belong in?",
    "messages": [
      {"role": "user", "content": "It arrived crushed. I want a refund."}
    ],
    "output_config": {
      "format": {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "queue": {"enum": ["billing", "shipping", "account"]}
          }
        }
      }
    }
  }'

import json
import os

from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.u22a8.ai/eval",
    api_key=os.environ["U22A8_API_KEY"],
)

route = {
    "type": "json_schema",
    "schema": {
        "type": "object",
        "properties": {"queue": {"enum": ["billing", "shipping", "account"]}},
    },
}

message = client.messages.create(
    model="a8:95",  # minimum accuracy 95
    max_tokens=1024,  # the SDK requires it
    system="Which queue does the ticket belong in?",
    messages=[
        {"role": "user", "content": "It arrived crushed. I want a refund."},
    ],
    output_config={"format": route},
)
print(message.stop_reason)  # "end_turn", or "refusal" on an abstention
print(json.loads(message.content[0].text))  # {'queue': 'shipping'}
print(message.model)  # the version that answered

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://api.u22a8.ai/eval",
  apiKey: process.env.U22A8_API_KEY,
});

const message = await client.messages.create({
  model: "a8:95", // minimum accuracy 95
  max_tokens: 1024, // the SDK requires it
  system: "Which queue does the ticket belong in?",
  messages: [{ role: "user", content: "It arrived crushed. I want a refund." }],
  output_config: {
    format: {
      type: "json_schema",
      schema: {
        type: "object",
        properties: { queue: { enum: ["billing", "shipping", "account"] } },
      },
    },
  },
});
const block = message.content[0];
if (block?.type === "text") console.log(JSON.parse(block.text));
console.log(message.model); // the version that answered

{
  "id": "eval-9f2c1d84ab30e5f7",
  "type": "message",
  "role": "assistant",
  "model": "a8-1-1:95@20260715t140322z",
  "content": [{"type": "text", "text": "{\"queue\": \"shipping\"}"}],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 812,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 0
  },
  "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c",
  "min_accuracy": 95
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.