Get started

Quickstart: a first verdict

Get a key, call A8 from an SDK, from curl or from your assistant, and read the verdict it returns.

This page takes you from a new key to a first verdict. Choose the route that fits how you work: an SDK or curl in Call A8 from code, or an AI assistant that runs MCP servers in Call A8 from your assistant.

§1Before you start

An account. Sign up to create your organization.
Access to A8. In Console → A8, say what you will use it for and select Request access. We write to you when A8 is on for your organization. Until then, a request returns 403 with code model_not_enabled.
For code: Python 3.10 or later, or Node.js 20 or later, with the openai package. Any client that takes a custom base URL works the same way.
For an assistant: an MCP host that connects to remote servers, such as Claude, ChatGPT, Claude Code or Cursor.

§2Call A8 from code

§2.1Create an API key

In Console → API keys, select Create key and grant it models:infer. Copy the secret: the console shows it once. Then set it in your shell:

Shell

export U22A8_API_KEY="<your key>"

See API keys for permissions.

§2.2Install the client

curl ships with macOS, Linux and Windows 10 or later.

Shell

$ pip install openai

Shell

$ npm install openai

§2.3Send a criterion and a subject

The system message is the criterion: what is being judged. The user message is the subject: the thing under judgment. The schema is the shape of the verdict. A follow-up user message after A8’s response continues the conversation, and A8 answers it in turn.

Shell

curl https://api.u22a8.ai/eval/v1/chat/completions \
  -H "Authorization: Bearer $U22A8_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "a8",
    "messages": [
      {"role": "system", "content": "Does the reply promise a refund?"},
      {"role": "user", "content": "We will refund you in full by Friday."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "verdict",
        "schema": {
          "type": "object",
          "properties": {"refund": {"type": "boolean"}}
        }
      }
    }
  }'

Python

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.u22a8.ai/eval/v1",
    api_key=os.environ["U22A8_API_KEY"],
)

resp = client.chat.completions.create(
    model="a8",
    messages=[
        {"role": "system", "content": "Does the reply promise a refund?"},
        {"role": "user", "content": "We will refund you in full by Friday."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "verdict",
            "schema": {
                "type": "object",
                "properties": {"refund": {"type": "boolean"}},
            },
        },
    },
)
print(resp.choices[0].message.content)  # the verdict
print(resp.model)  # the version that answered

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.u22a8.ai/eval/v1",
  apiKey: process.env.U22A8_API_KEY,
});

const resp = await client.chat.completions.create({
  model: "a8",
  messages: [
    { role: "system", content: "Does the reply promise a refund?" },
    { role: "user", content: "We will refund you in full by Friday." },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "verdict",
      schema: {
        type: "object",
        properties: { refund: { type: "boolean" } },
      },
    },
  },
});
console.log(resp.choices[0].message.content); // the verdict
console.log(resp.model); // the version that answered

§2.4Read the verdict

The response is a standard chat.completion object. The verdict is the message content, as JSON that matches your schema:

JSON

{
  "id": "eval-9f2c1d84ab30e5f7",
  "object": "chat.completion",
  "created": 0,
  "model": "a8-1-1:90@20260715t140322z",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"refund\": true}",
        "refusal": null
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 0,
    "total_tokens": 24,
    "prompt_tokens_details": {"cached_tokens": 0},
    "completion_tokens_details": {"reasoning_tokens": 0}
  },
  "system_fingerprint": "fit-1a2b3c4d5e6f7890.9e2a1b3c",
  "min_accuracy": 90
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.

Three fields carry what you need:

choices[0].message.content is the verdict.
model names the exact version that answered, with the minimum accuracy it answered at as its tag. Send it back as model to get the same verdict again. See Horizons.
min_accuracy is the level A8 promises the verdict at. The request named the bare model, so the minimum accuracy was the default and A8 answered at 90, or at the next level up it has earned: at least that share of the verdicts it returns at the level is correct. min_accuracy names the level.

A case whose evidence falls short of the minimum accuracy comes back as an abstention: content is null, and refusal and a top-level abstention object give the level at which A8 answers the same request. See Accuracy and abstention.

§3Call A8 from your assistant

A8 runs as an MCP server, so an assistant can judge, teach and retract for you from inside a conversation. Judging needs models:infer. Teaching and retracting need models:train, and asking what you taught needs models:explain.

§3.1Connect the server

Add a remote MCP server at https://api.u22a8.ai/mcp. Hosts that support sign-in take you through it, and the server acts with your role’s permissions in the organization you pick. Hosts that take a header send your API key as Authorization: Bearer, and the server acts with the key’s permissions.

In Claude, open Settings → Connectors, select Add custom connector, and enter https://api.u22a8.ai/mcp. Claude takes you through sign-in.

Shell

$ claude mcp add --transport http a8 https://api.u22a8.ai/mcp \
    --header "Authorization: Bearer $U22A8_API_KEY"

To sign in instead, add the server with the URL alone, then run /mcp in Claude Code.

Add the server to ~/.cursor/mcp.json. Cursor reads the key from your environment:

JSON

{
  "mcpServers": {
    "a8": {
      "url": "https://api.u22a8.ai/mcp",
      "headers": {"Authorization": "Bearer ${env:U22A8_API_KEY}"}
    }
  }
}

§3.2Ask for a verdict

Ask in plain words:

“Ask A8 whether this reply promises a refund: We will refund you in full by Friday.”

Your assistant calls A8’s judge tool and reports the verdict:

JSON

{
  "verdict": {"refund": true},
  "model": "a8-1-1:90@20260715t140322z",
  "min_accuracy": 90
}

Recorded from a8-1-1@20260715t140322z on 2026-10-22.

See MCP for the tools, their schemas and the permission each needs.

§4Check that it worked

From code: the response has status 200, model names a version such as a8-1-1@20260715t140322z, and content is JSON that matches your schema.
From your assistant: the host lists A8’s judge, teach and retract tools, and the judge result carries a verdict and a model.

A 401 means the key is missing, invalid or revoked. A 403 names the permission the key lacks, or has code model_not_enabled while A8 is off for your organization. See Errors for every status code.

Next steps

A8 model cardA8 on one page: model IDs, pricing, capabilities, endpoints, features, rate limits and how it compares.Write a criterionState a criterion A8 judges well, with one judgment per criterion, the situation that decides it and the practices that keep verdicts steady.Accuracy and abstentionHow a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.