Research

Benchmarks, methodology, and the raw artifacts behind them. We publish work here when the comparisons are reproducible end-to-end and the failure modes are stateable.

Benchmarks May 5, 2026

qed-bench: benchmarking metrics against task-appropriate baselines

We trained metrics on four content-judgment tasks — holistic essay quality, SMS spam, AI-vs-human authorship, and LLM authorship attribution — and compared each one to its task-appropriate baseline: trained human raters, gold labels, or an eight-model LLM-as-judge panel. Notebooks, models, and per-judge artifacts at github.com/u22a8/qed-bench.
Read →

We also run The AI Oversight Study, on how organizations check and trust the automated systems they run, at oversight.study.

← back to the landing

qed-bench: benchmarking metrics against task-appropriate baselines