> The full index of these docs is at [llms.txt](https://u22a8.ai/llms.txt).

# A8 with the OpenAI SDK

Judge with A8 from the OpenAI SDK, set the minimum accuracy in the model's tag, read an abstention, and move a judge prompt you run today.

The OpenAI SDK calls A8 through Chat Completions at `https://api.u22a8.ai/eval/v1`. This page builds a judge on it, then moves a judge prompt that runs on another model.

## 1 Before you start

Create an API key with the `models:infer` permission, as [API keys](https://u22a8.ai/docs/api-keys.md) shows, and set it as `U22A8_API_KEY`.

**Python**

```bash
pip install openai pydantic
export U22A8_API_KEY="<your key>"
```

**TypeScript**

```bash
npm install openai zod
export U22A8_API_KEY="<your key>"
```

## 2 Judge one subject

Point the client at `https://api.u22a8.ai/eval/v1`. The system message states the criterion, the user message holds the subject, and the response format gives the verdict its shape:

**Python**

```python
import os

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.u22a8.ai/eval/v1",
    api_key=os.environ["U22A8_API_KEY"],
)


class Verdict(BaseModel):
    refund: bool


resp = client.chat.completions.parse(
    model="a8-1-1:95@20260715t140322z",  # pinned, at a minimum accuracy of 95
    messages=[
        {"role": "system", "content": "Does the reply promise a refund?"},
        {"role": "user", "content": "We will refund you in full by Friday."},
    ],
    response_format=Verdict,
)
message = resp.choices[0].message
print(message.parsed)  # Verdict(refund=True), or None on an abstention
```

**TypeScript**

```typescript
import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";

const client = new OpenAI({
  baseURL: "https://api.u22a8.ai/eval/v1",
  apiKey: process.env.U22A8_API_KEY,
});

const Verdict = z.object({ refund: z.boolean() });

const resp = await client.chat.completions.parse({
  model: "a8-1-1:95@20260715t140322z", // pinned, at a minimum accuracy of 95
  messages: [
    { role: "system", content: "Does the reply promise a refund?" },
    { role: "user", content: "We will refund you in full by Friday." },
  ],
  response_format: zodResponseFormat(Verdict, "verdict"),
});
const message = resp.choices[0].message;
console.log(message.parsed); // { refund: true }, or null on an abstention
```

```json
{"refund": true}
```

Recorded from `a8-1-1@20260715t140322z` on 2026-10-26.

A pinned identifier keeps a test on one version, and its tag holds the minimum accuracy. `a8` follows your organization’s latest version. See [Model identifiers](https://u22a8.ai/docs/a8-model-identifiers.md).

## 3 Set the minimum accuracy

The model's tag sets the minimum accuracy: `model="a8:95"` asks for 95, and a pinned identifier takes the same tag, as in `a8-1-1:95@20260715t140322z`. A whole number from `0` to `99` passes. The bare `a8` asks for 90. What the minimum accuracy promises is on [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md).

`reasoning_effort` is the reasoning switch. `"none"` reads the request as it is laid out: the system message is the criterion and the latest user message the subject. Every other value, and the default, read the request however it is written. See [Chat Completions](https://u22a8.ai/docs/a8-chat-completions.md).

## 4 Read an abstention

An abstention returns `200`. `message.refusal` holds the abstention in words and `message.parsed` is `None`. The SDK keeps A8’s `abstention` object in `resp.model_extra`:

```python
if message.refusal:
    abstention = resp.model_extra["abstention"]
    print(abstention["kind"])  # "too_close"
    print(abstention["answerable_at"])  # 80: the level that answers it
else:
    print(message.parsed.refund)
```

In a test run, count an abstention as unanswered: it lowers coverage and leaves accuracy as it is. See [Testing](https://u22a8.ai/docs/a8-testing.md).

## 5 Move a judge prompt

A judge prompt that runs on another model through the OpenAI SDK moves in six changes:

1. Set `base_url` to `https://api.u22a8.ai/eval/v1` and `api_key` to your A8 key.
2. Set `model` to a pinned identifier, with the minimum accuracy as its tag, such as `a8-1-1:95@20260715t140322z`.
3. Replace `response_format={"type": "json_object"}` with a Pydantic model, or with a `json_schema` format that names the fields to answer in.
4. Move a threshold written in the prompt, such as “only answer if you are sure”, to the model's tag. A8 reads the prompt’s words as part of the criterion.
5. Remove `reasoning_effort="none"` if the old configuration sends it. With reasoning off, A8 reads the layout as written, and a prompt that holds the criterion and the subject in one message needs reasoning on.
6. Check `message.refusal` before you read the verdict.

The prompt itself stays. A8 reads a prompt that holds the criterion and the subject in one message as it is, and answers in the format the prompt asks for:

```python
GRADER = """You are grading a support reply.
Question: Does the reply promise a refund?
Reply: {reply}
Answer with Y or N."""

resp = client.chat.completions.create(
    model="a8-1-1:95@20260715t140322z",
    messages=[{"role": "user", "content": GRADER.format(reply=reply)}],
)
print(resp.choices[0].message.content)  # "Y"
```

## 6 Errors on a moved judge

| Change left out                        | Status | `code`                  | Message                                                  |
| -------------------------------------- | ------ | ----------------------- | -------------------------------------------------------- |
| The key is still the old provider’s    | `401`  | `invalid_api_key`       | The API key is invalid or revoked.                       |
| The old model name                     | `400`  | `unknown_model`         | Use a8, or an identifier from GET /models.               |
| `response_format` set to `json_object` | `400`  | `unsupported_value`     | Use json\_schema, which names the fields to answer in.   |
| A minimum accuracy tag outside 0 to 99 | `400`  | `unsupported_value`     | Tag the model with a level from 0 to 99, as a8:95.       |
| `min_accuracy` in the body             | `400`  | `unsupported_parameter` | Set the minimum accuracy with the model's tag, as a8:95. |

*Table 1.* The error A8 returns for each change left out. Every status and its body is on [Errors](https://u22a8.ai/docs/errors.md).

## Next steps

- [A8 through Chat Completions](https://u22a8.ai/docs/a8-chat-completions.md): Call A8 through Chat Completions, the fields it reads, minimum accuracy and where the verdict, an abstention and precedents land.
- [Accuracy and abstention](https://u22a8.ai/docs/a8-abstention.md): How a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.
- [Testing A8](https://u22a8.ai/docs/a8-testing.md): Test what A8 learned the way you would test a student, with new questions, known answers, accuracy and coverage, on a pinned version.