The catalog

Benchmarks and datasets, ready to evaluate.

Our benchmarks show where frontier models fail. Our datasets help them improve. Every dataset starts as a sample of 5 to 100 items, with a datasheet and model results where the task can be scored.

Benchmarks

Where models fail today.

Live

Citizen-Bench

Can AI answer citizens' legal questions? The same 77 real questions, asked in Algeria, France and the US. Every model gives its weakest answers in Algeria.

Explore Citizen-Bench Leaderboard Samples

Average score, same 77 questions

Algeria34%
France63%
US58%

More benchmarks

New benchmarks are on the way. Each one tests a skill that current models get wrong.

Suggest one

Is there a task AI keeps failing for you? Tell us and we may build the benchmark.

Contact us
Datasets

Sample datasets you can evaluate before you order.

Showcase20 samples

CAD computer-use recordings

A professional solves a design brief in open-source CAD tools. Screen video, action log and the finished file.

Request samples
Evaluation100 samples

CAD files with design history

Original parametric parts with their brief and step history, plus how frontier models do on the same brief.

Request samples
Showcase50 clips

Egocentric work video

First-person video of skilled manual work in real workshops. Anonymised and described step by step.

Request samples
Showcase5 samples

Real company data

Anonymised codebases and project archives from real companies, each with a qualification report.

Request samples
Evaluation50 samples

Open-source desktop tool use

Tasks in LibreOffice, GIMP, Inkscape and QGIS, each with an automatic checker and agent pass rates.

Request samples
Evaluation20 samples

Professional tasks with file deliverables

Briefs, source files, expert deliverables and scoring rubrics in architecture, engineering and accounting.

Request samples
Showcase10 samples

Day-in-the-life workflows

Full working sessions where a professional reads several sources, then does the work.

Request samples
Custom

Something else?

We build to specification. Describe the skill and the format, and we'll scope a sample.

Describe your need

Exclusive and non-exclusive licences available.

Every sample includes

Enough to make a decision.

A datasheet

Source, collection method, schema, licence and known limits.

Model results

Pass rates for named frontier models, where the task can be scored.

Deliverable volume

How many items we can produce, and how fast.