Responses

A8 answers the OpenAI Responses API at POST https://api.u22a8.ai/eval/v1/responses. The OpenAI SDKs reach it with the same base URL as Chat Completions, and the same request returns the same verdict through either.

§1Request

A8 reads the fields below. instructions states the criterion, input carries the subject, and text.format gives the verdict its shape. A8 answers the latest user item of input:

The subject can carry a context, a reference or a question, and one schema can carry several criteria. See API overview. With reasoning on, a prompt written for another judge works too, and is answered in the format it asks for. See Chat Completions.

§1.1Continue from a stored response

To follow up on a verdict, pass its id as previous_response_id and put the new user item in input. instructions applies to the request that carries it, so send the criterion with each request:

route = {
    "type": "json_schema",
    "name": "route",
    "schema": {
        "type": "object",
        "properties": {"queue": {"enum": ["billing", "shipping", "account"]}},
    },
}

first = client.responses.create(
    model="a8-1-1@20260715t140322z",
    instructions="Which queue does the ticket belong in?",
    input="It arrived crushed. I want a refund.",
    text={"format": route},
)
second = client.responses.create(
    model="a8-1-1@20260715t140322z",
    instructions="Which queue does the ticket belong in?",
    previous_response_id=first.id,
    input="The customer adds: the box was fine. I was charged twice.",
    text={"format": route},
)
print(second.output_text)  # {"queue": "billing"}

Recorded from a8-1-1@20260715t140322z on 2026-10-26.

A request with previous_response_id reads the same conversation as one that sends the earlier items in input itself, and gets the same verdict. A response is stored for 30 days, in your organization. See Continue a conversation.

A response’s id follows from the request: the same conversation, schema and minimum accuracy against the same version get the same id through every interface. Sending a request again stores it for 30 days from then, and fetching it returns the response as first returned.

A stored response that quotes your taught examples, as precedents or a recall, is read with the permission that asked for them: fetching it, listing its input items and continuing from it need models:explain too. So does a response that continues from one, since its input items hold the earlier output.

§1.2Fetch a stored response

GET https://api.u22a8.ai/eval/v1/responses/{id} returns a stored response as it was first returned, verdict, model and min_accuracy included. It needs the models:infer permission:

stored = client.responses.retrieve(first.id)
print(stored.output_text)  # {"queue": "shipping"}
print(stored.model)  # the version that answered

Fetching a response returns 404 with the code response_not_found when it was deleted, stored more than 30 days ago or sent with store: false. See Errors.

§1.3List a stored response’s input items

GET https://api.u22a8.ai/eval/v1/responses/{id}/input_items lists the input items a stored response read, those from previous_response_id included. It needs the models:infer permission, and pages as the Responses API does:

• order is desc, newest item first, by default, or asc.
• limit takes 1 to 100 items per page, 20 by default.
• after takes an item’s id, and the page starts after it. has_more says whether another page follows.

for item in client.responses.input_items.list(second.id):
    print(item.role)  # user, assistant, user

§1.4Delete a stored response

DELETE https://api.u22a8.ai/eval/v1/responses/{id} removes a stored response before its 30 days end. It needs the models:infer permission:

client.responses.delete(first.id)

{"id": "eval-9f2c1d84ab30e5f7", "object": "response", "deleted": true}

A deleted response is gone from your organization’s store: fetching it returns 404, and continuing from it with previous_response_id returns 400. Verdicts already returned stay as they are.

§2What is refused

Each of these returns 400, with an error that names the field and the value that passes:

§3Minimum accuracy

The minimum accuracy is the accuracy A8 promises on the verdicts it returns. At 95, at least 95% of answered verdicts are correct. The model’s tag sets it: a8:95 asks for 95, and a pinned identifier carries it too, as a8-1-1:95@20260715t140322z.

• The tag is a whole number from 0 to 99. A higher level answers less often and is right more often.
• By default, the minimum accuracy is 90.
• The response’s model names the level that answered, and so does a stored response.

The tag alone sets the minimum accuracy. A threshold written in the instructions counts as part of the criterion. The identifier’s parts are on Model identifiers, and what the minimum accuracy promises is on Accuracy and abstention.

§4Reasoning

reasoning.effort turns A8’s reasoning on and off. Reasoning is on unless the request sends none.

• Reasoning on reads the request however it is laid out: a framework’s grader template, a follow-up, or a request in plain words. When the layout already says what each part is, A8 answers as fast as with reasoning off.
• Reasoning off (reasoning: {"effort": "none"}) reads the layout as written, which is faster:

A framework’s grader template sent with reasoning off is read as written too: instructions or a system item, when it has one, is the criterion, and the template in the user item is the subject, so an untaught criterion abstains. A template sent as one user item alone abstains with kind no_question, since the criterion travels in instructions.

The reading is reported as reasoning tokens, in usage.output_tokens_details.reasoning_tokens and inside usage.output_tokens. A repeat of the request reads from the cache and reports the same count as cached input, in usage.input_tokens_details.cached_tokens and inside usage.input_tokens, as this API reports a prompt it had cached. A layout read as written reports 0.

§5Response

A8 answers with a standard response object. The verdict is the output_text part, so the SDKs’ output_text and output_parsed read it as they read any model’s answer.

{
  "id": "eval-9f2c1d84ab30e5f7",
  "object": "response",
  "status": "completed",
  "model": "a8-1-1:95@20260715t140322z", // the version and level that answered
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "{\"queue\": \"shipping\"}" // the verdict
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 812,
    "input_tokens_details": {"cached_tokens": 0},
    "output_tokens": 0,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 812
  },
  "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c",
  "min_accuracy": 95 // the level this verdict is promised at
}

• model names the exact version that answered, with the minimum accuracy in its tag. See Horizons.
• usage.output_tokens holds reasoning tokens alone, since the verdict is scored. They are 0 here, as the layout is read as written.
• system_fingerprint, min_accuracy, abstention, interval and precedents are A8’s fields beside the standard ones. Each is defined on API overview.

A forced tool call is answered as a function_call output item, with the verdict as its arguments.

The OpenAI SDKs keep A8’s fields on the response object:

resp.model_extra.get("min_accuracy")  # 95, or None on an abstention
resp.model_extra.get("abstention")  # set on an abstention

type A8Fields = { min_accuracy?: number; abstention?: { kind: string } };

const { min_accuracy, abstention } = resp as typeof resp & A8Fields;

§6An abstention

An abstention succeeds with 200, and the response is completed. The message holds a refusal part where an answer holds output_text. The top-level abstention says which check the evidence failed, and at what level the same request is answered:

{
  "object": "response",
  "status": "completed",
  "model": "a8-1-1:95@20260715t140322z",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "refusal",
          "refusal": "Too close to call at min_accuracy 95; answered at 80."
        }
      ]
    }
  ],
  "abstention": {
    "kind": "too_close",
    "asked": 95,
    "answerable_at": 80,
    "earned": 97,
    "missing": null
  }
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.

On an abstention, output_parsed is None in Python and null in TypeScript. Branch on abstention.kind. The kinds are on Accuracy and abstention.

§7A filtered request

With reasoning on, A8 filters some requests whose text is harmful, such as hate speech, violence or self-harm. A filtered request gets an outcome of its own, apart from verdicts and abstentions. It succeeds with 200, as this API reports a response its filters stopped: status is "incomplete", incomplete_details.reason is "content_filter", and output is empty:

{
  "object": "response",
  "status": "incomplete",
  "incomplete_details": {"reason": "content_filter"},
  "model": "a8-1-1:95@20260715t140322z",
  "output": [],
  "usage": {
    "input_tokens": 0,
    "input_tokens_details": {"cached_tokens": 0},
    "output_tokens": 0,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 0
  }
}

A filtered request is free, and the same request is filtered every time it is sent. Streamed, it ends with response.incomplete.

§8Precedents and recall

Ask for precedents in the prompt (“quote precedents”), and the response carries a top-level precedents object. Reserve a precedents property in text.format, and they arrive inside the verdict. Either way the key needs the models:explain permission. See API keys.

A question about what you taught, such as “What have I taught you about refunds?”, is answered with recall. The output_text part is the precedents object as JSON, holding your taught examples, and the same object sits at the top level.

§9Streaming

With stream: true, the answer arrives as Responses events. The verdict comes whole, in one response.output_text.delta, or on an abstention in one response.refusal.delta and response.refusal.done, and response.completed carries the full object with A8’s fields. A filtered request ends with response.incomplete, its output empty.

§10Example

A ticket router at a minimum accuracy of 95:

curl https://api.u22a8.ai/eval/v1/responses \
  -H "Authorization: Bearer $U22A8_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "a8:95",
    "instructions": "Which queue does the ticket belong in?",
    "input": "It arrived crushed. I want a refund.",
    "text": {
      "format": {
        "type": "json_schema",
        "name": "route",
        "schema": {
          "type": "object",
          "properties": {
            "queue": {"enum": ["billing", "shipping", "account"]}
          }
        }
      }
    }
  }'

import os
from typing import Literal

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.u22a8.ai/eval/v1",
    api_key=os.environ["U22A8_API_KEY"],
)


class Route(BaseModel):
    queue: Literal["billing", "shipping", "account"]


resp = client.responses.parse(
    model="a8:95",  # minimum accuracy 95
    instructions="Which queue does the ticket belong in?",
    input="It arrived crushed. I want a refund.",
    text_format=Route,
)
print(resp.output_parsed)  # Route(queue='shipping'), or None on an abstention
print(resp.model)  # the version that answered

import OpenAI from "openai";
import { zodTextFormat } from "openai/helpers/zod";
import { z } from "zod";

const client = new OpenAI({
  baseURL: "https://api.u22a8.ai/eval/v1",
  apiKey: process.env.U22A8_API_KEY,
});

const Route = z.object({ queue: z.enum(["billing", "shipping", "account"]) });

const resp = await client.responses.parse({
  model: "a8:95", // minimum accuracy 95
  instructions: "Which queue does the ticket belong in?",
  input: "It arrived crushed. I want a refund.",
  text: { format: zodTextFormat(Route, "route") },
});
console.log(resp.output_parsed); // { queue: "shipping" }, or null
console.log(resp.model); // the version that answered

{
  "id": "eval-9f2c1d84ab30e5f7",
  "object": "response",
  "created_at": 0,
  "status": "completed",
  "model": "a8-1-1:95@20260715t140322z",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [{"type": "output_text", "text": "{\"queue\": \"shipping\"}"}]
    }
  ],
  "usage": {
    "input_tokens": 812,
    "input_tokens_details": {"cached_tokens": 0},
    "output_tokens": 0,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 812
  },
  "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c",
  "min_accuracy": 95
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.