> The full index of these docs is at [llms.txt](https://u22a8.ai/llms.txt). # Chat Completions Call A8 through Chat Completions, the fields it reads, minimum accuracy and where the verdict, an abstention and precedents land. A8 answers the OpenAI Chat Completions API at `POST https://api.u22a8.ai/eval/v1/chat/completions`. Any client that takes a custom base URL calls it this way: the OpenAI SDKs, LangChain, LlamaIndex, the Vercel AI SDK, Pydantic AI and LiteLLM among them. ## 1 Request A8 reads the fields below. The system message states the criterion, the user messages carry the subject, and the schema gives the verdict its shape. A8 answers the conversation’s latest user message: | Field | Type | How A8 reads it | Default | | ---------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | | `model` | string | `a8` for your organization’s latest version, a pinned identifier, or `a8-1-1@reference`. Its tag sets the minimum accuracy: `a8:95` | Required | | `messages` | array | The conversation: system and developer messages state the criterion, user messages carry the subject and its follow-ups, and assistant messages are A8’s earlier responses | Required | | `response_format` | object | The verdict’s shape, as a `json_schema` | A label or a number | | `tools`, `tool_choice` | array, object | A forced function call: its parameters are the verdict’s shape | None | | `reasoning_effort` | string | Reasoning: `none` reads the layout as written, and any other value reads the request however it is laid out | Reasoning on | | `n` | integer | The number of choices, `1` to `128`; A8 computes each on its own | `1` | | `stream` | boolean | Streams the verdict as `chat.completion.chunk` events | `false` | *Table 1.* The fields A8 reads. Every other field is accepted and ignored, `temperature`, `top_p`, `max_tokens` and `seed` among them, so a body written for another provider works as it is. The subject can carry a context, a reference or a question beside the output it holds, and one schema can carry several criteria. Both work the same way in every interface. See [API overview](https://u22a8.ai/docs/api-overview.md). To follow up on a verdict, send the conversation again with A8’s response as an assistant message and a new user message after it. See [Continue a conversation](https://u22a8.ai/docs/api-overview.md#continue-a-conversation). ### 1.1 Prompts written for other judges Eval frameworks often put the criterion and the subject in one message and ask for a format such as `GRADE: C` or a trailing `Y`/`N`. With reasoning on, A8 answers that prompt too: it finds the criterion, the subject and the format, and answers in the format the prompt asks for. A request that is only conversation, such as small talk, comes back as an abstention of kind `no_question`. ## 2 What is refused Each of these returns `400`, with an error that names the field and the value that passes: | Request | Status | Send | | ------------------------------------------------------------------------------------------ | ------ | ------------------------------------------------------------------ | | `min_accuracy` in the body | `400` | The minimum accuracy as the model’s tag, as `a8:95` | | A `model` tag outside `0` to `99` | `400` | A whole number from `0` to `99` | | `expected` in the body | `400` | The answer as a teach, over [MCP](https://u22a8.ai/docs/a8-mcp.md) | | A `reasoning_effort` outside `none`, `minimal`, `low`, `medium`, `high`, `xhigh` and `max` | `400` | One of those values | | An `n` outside `1` to `128` | `400` | An integer from `1` to `128` | | A `response_format` of type `json_object` | `400` | `json_schema`, which names the fields to answer in | | A `tool_choice` naming a tool outside `tools` | `400` | The name of a tool in `tools` | *Table 2.* Refused fields and values. Error bodies are on [Errors](https://u22a8.ai/docs/errors.md). ## 3 Minimum accuracy The minimum accuracy is the accuracy A8 promises on the verdicts it returns. At 95, at least 95% of answered verdicts are correct. The model’s tag sets it: `a8:95` asks for 95, and a pinned identifier carries it too, as `a8-1-1:95@20260715t140322z`. - **The tag is a whole number from `0` to `99`.** A higher level answers less often and is right more often. - **By default, the minimum accuracy is 90.** - **The response’s `model` names the level that answered,** so a stored response says which promise it was answered under. The tag alone sets the minimum accuracy. “Only answer if you are sure” in a prompt counts as part of the criterion. The identifier’s parts are on [Model identifiers](https://u22a8.ai/docs/a8-model-identifiers.md), and what the minimum accuracy promises, with the levels a criterion earns, is on [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md). ## 4 Reasoning `reasoning_effort` turns A8’s reasoning on and off. Reasoning is on unless the request sends `none`. - **Reasoning on** reads the request however it is laid out: a framework’s grader template, a follow-up, or a request in plain words. When the layout already says what each part is, A8 answers as fast as with reasoning off. - **Reasoning off** (`reasoning_effort: "none"`) reads the layout as written, which is faster: | Part | Where it goes | | ---------------------------- | ---------------------------------------------------------------------------- | | Criterion | Every system and developer message, in order, as written | | Subject | The latest user message, whole | | Reference, context, question | Keys of a subject object: a user message holding a JSON object with `output` | | Verdict shape | `response_format`, or a forced tool. Without either, a label or a number | | Labels | The schema’s `enum` | | Precedents | A `precedents` property reserved in the schema | *Table 3.* The layout with reasoning off. A8 judges the latest user message alone, and earlier turns are history. A framework’s grader template sent with reasoning off is read as written too: its system message, when it has one, is the criterion, and the template in the user message is the subject, so an untaught criterion abstains. A template sent as one user message alone abstains with kind `no_question`, since the criterion travels in a system message. The reading is reported as reasoning tokens, in `usage.completion_tokens_details.reasoning_tokens` and inside `usage.completion_tokens`. A repeat of the request reads from the cache and reports the same count as cached input, in `usage.prompt_tokens_details.cached_tokens` and inside `usage.prompt_tokens`, as this API reports a prompt it had cached. A layout read as written reports `0`. ## 5 Response A8 answers with a standard `chat.completion`. The verdict is the message content: a label, a number or JSON that matches the schema. ```jsonc { "id": "eval-9f2c1d84ab30e5f7", "object": "chat.completion", "model": "a8-1-1:95@20260715t140322z", // the version and level that answered "choices": [ { "index": 0, "message": { "role": "assistant", "content": "{\"queue\": \"shipping\"}", // the verdict "refusal": null }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 812, "completion_tokens": 0, "total_tokens": 812, "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0} }, "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c", "min_accuracy": 95 // the level this verdict is promised at } ``` - **`model`** names the exact version that answered, with the minimum accuracy in its tag. Send it back to get the same verdict. See [Horizons](https://u22a8.ai/docs/a8-horizons.md). - **`usage.completion_tokens`** holds reasoning tokens alone, since the verdict is scored. They are `0` here, as the layout is read as written. - **With `n` above `1`,** A8 computes each choice on its own, and `usage.prompt_tokens` counts the input once per choice. The same request gets the same verdict, so every choice holds it: comparing them checks determinism. - **`min_accuracy`**, `abstention`, `interval` and `precedents` are A8’s fields beside the standard ones. Standard clients carry them through. Each is defined on [API overview](https://u22a8.ai/docs/api-overview.md). A forced tool call is answered as that tool call, with the verdict as its `arguments` and `finish_reason` set to `tool_calls`. The OpenAI SDKs keep A8’s fields on the response object: **Python** ```python resp.model_extra.get("min_accuracy") # 95, or None on an abstention resp.model_extra.get("abstention") # set on an abstention ``` **TypeScript** ```typescript type A8Fields = { min_accuracy?: number; abstention?: { kind: string } }; const { min_accuracy, abstention } = resp as typeof resp & A8Fields; ``` ## 6 An abstention An abstention succeeds with `200`. The message carries a `refusal` and its `content` is `null`, so a typed client reads it as it reads any refusal. The top-level `abstention` says which check the evidence failed, and at what level the same request is answered: ```json { "object": "chat.completion", "model": "a8-1-1:95@20260715t140322z", "choices": [ { "index": 0, "message": { "role": "assistant", "content": null, "refusal": "Too close to call at min_accuracy 95; answered at 80." }, "finish_reason": "stop" } ], "abstention": { "kind": "too_close", "asked": 95, "answerable_at": 80, "earned": 97, "missing": null } } ``` Recorded from `a8-1-1@20260715t140322z` on 2026-10-22. The refusal text is for people to read. Branch on `abstention.kind`. The kinds, and what clears each, are on [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md). ## 7 A filtered request With reasoning on, A8 filters some requests whose text is harmful, such as hate speech, violence or self-harm. A filtered request gets an outcome of its own, apart from verdicts and abstentions. It succeeds with `200`, as this API reports content its filters omit: `finish_reason` is `"content_filter"`, and `content` and `refusal` are both `null`: ```json { "object": "chat.completion", "model": "a8-1-1:95@20260715t140322z", "choices": [ { "index": 0, "message": {"role": "assistant", "content": null, "refusal": null}, "finish_reason": "content_filter" } ], "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0} } } ``` A filtered request is free, and the same request is filtered every time it is sent. With `n` above `1`, every choice is filtered. ## 8 Precedents and recall Ask for [precedents](https://u22a8.ai/docs/a8-precedents.md) in the prompt (“quote precedents”), and the response carries a top-level `precedents` object: the taught examples behind the verdict. Reserve a `precedents` property in your schema, and they arrive inside the verdict, where a typed client reads them with the rest. Either way the key needs the `models:explain` permission. See [API keys](https://u22a8.ai/docs/api-keys.md). A question about what you taught is answered with recall, your taught examples: ```json { "model": "a8", "messages": [ {"role": "user", "content": "What have I taught you about refunds?"} ] } ``` The message content is the `precedents` object as JSON, and the same object sits at the top level of the response. ## 9 Streaming With `stream: true`, the answer arrives as `chat.completion.chunk` events. The verdict comes whole, in one delta, and each choice’s finish chunk carries A8’s fields. With `stream_options.include_usage`, a usage chunk follows them. With `n` above `1`, each choice arrives under its own `index`. ## 10 Example A ticket router at a minimum accuracy of 95. The Python and TypeScript samples read the verdict into a typed object: **curl** ```bash curl https://api.u22a8.ai/eval/v1/chat/completions \ -H "Authorization: Bearer $U22A8_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "a8:95", "messages": [ {"role": "system", "content": "Which queue does the ticket belong in?"}, {"role": "user", "content": "It arrived crushed. I want a refund."} ], "response_format": { "type": "json_schema", "json_schema": { "name": "route", "schema": { "type": "object", "properties": { "queue": {"enum": ["billing", "shipping", "account"]} } } } } }' ``` **Python** ```python import os from typing import Literal from openai import OpenAI from pydantic import BaseModel client = OpenAI( base_url="https://api.u22a8.ai/eval/v1", api_key=os.environ["U22A8_API_KEY"], ) class Route(BaseModel): queue: Literal["billing", "shipping", "account"] resp = client.chat.completions.parse( model="a8:95", # minimum accuracy 95 messages=[ {"role": "system", "content": "Which queue does the ticket belong in?"}, {"role": "user", "content": "It arrived crushed. I want a refund."}, ], response_format=Route, ) message = resp.choices[0].message print(message.parsed) # Route(queue='shipping'), or None on an abstention print(message.refusal) # None, or the abstention in words print(resp.model) # the version that answered ``` **TypeScript** ```typescript import OpenAI from "openai"; import { zodResponseFormat } from "openai/helpers/zod"; import { z } from "zod"; const client = new OpenAI({ baseURL: "https://api.u22a8.ai/eval/v1", apiKey: process.env.U22A8_API_KEY, }); const Route = z.object({ queue: z.enum(["billing", "shipping", "account"]) }); const resp = await client.chat.completions.parse({ model: "a8:95", // minimum accuracy 95 messages: [ { role: "system", content: "Which queue does the ticket belong in?" }, { role: "user", content: "It arrived crushed. I want a refund." }, ], response_format: zodResponseFormat(Route, "route"), }); const message = resp.choices[0].message; console.log(message.parsed); // { queue: "shipping" }, or null on an abstention console.log(resp.model); // the version that answered ``` ```json { "id": "eval-9f2c1d84ab30e5f7", "object": "chat.completion", "created": 0, "model": "a8-1-1:95@20260715t140322z", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "{\"queue\": \"shipping\"}", "refusal": null }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 812, "completion_tokens": 0, "total_tokens": 812, "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0} }, "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c", "min_accuracy": 95 } ``` Recorded from `a8-1-1@20260715t140322z` on 2026-10-22. The minimum accuracy travels in `model`, which every SDK and framework passes through, so it is set wherever the model is. ## Next steps - [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md): How a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention. - [API overview](https://u22a8.ai/docs/api-overview.md): What every request to A8 shares, from the base URL and authentication to the request parts and the response fields. - [Errors](https://u22a8.ai/docs/errors.md): Every status A8 returns for a failed request, with its body, its cause and its fix, and how each SDK raises it.