Documentation
Two ways to run an eval. A8 (Touchstone) is a general eval model: state any criterion and it returns a verdict, over an OpenAI-compatible endpoint. The Metrics API serves the catalog of named metrics — criteria already trained and benchmarked — over our own REST surface.
Both return a measured score rather than generated text, and the same input always returns the same score.
§1Quickstart
A first verdict from A8, in three steps. The full request and response contract is on the A8 reference.
§1.1Get a key
Issue one in the console. It needs the eval:run ability.
§1.2Point a client at the base URL
Any OpenAI-compatible client works. Only the base URL changes.
# pip install openai
from openai import OpenAI
client = OpenAI(
base_url="https://api.u22a8.ai/eval/v1",
api_key="<your key>",
)
§1.3State a criterion, send a subject
The system message is the criterion — what is being judged. The last user message is the subject — the thing under judgment.
resp = client.chat.completions.create(
model="a8",
messages=[
{"role": "system", "content": "Is the reply free of hedging?"},
{"role": "user", "content": "It may possibly work, but I could be wrong."},
],
)
print(resp.choices[0].message.content) # the verdict
print(resp.model) # the snapshot that answered
§2Where to go next
A8 Touchstone
What A8 is, the shape of a call, and where to go next.
Read first →
Metrics API
Named criteria, already trained and benchmarked, scored over REST.
Browse the concepts →