TextCode
Multi-Turn Tool-Calling SFT
Multi-turn tool/function-calling training data across 8 task categories.
SFTTaxonomy
We design and deliver the goldens, rubrics, verifiers, and human-annotated datasets that AI teams use to train, evaluate, and benchmark their models — across text, code, image, video, and audio.
Benchmarked across frontier models · Five modalities · Reproducible pipelines
golden pass rate — by design, while frontier models fail ≥25% of mandatory criteria
We calibrate difficulty so benchmarks stay discriminating, not saturated — real signal you can act on.
modalities covered
reproducible benchmarking
grounded rubric verifiers
structured, drop-in deliverables
From SFT goldens and reproducible benchmarks to image/video annotation, transcription, and multimodal evaluation — the full data stack, across every modality.
Our goldens, taxonomies, and verifiers are calibrated so frontier models fail ≥25% of mandatory criteria while human goldens pass 100%.
passes mandatory criteria
fails ≥25% of mandatory criteria
// calibrated per project · evidence-grounded · reproducible
Every deliverable runs the same pipeline — from scoping to structured-JSON delivery.
We align on the capability under test, the deliverable schema, and what 'good' means before any data is produced.
Senior authors write the reference goldens — ideal responses and trajectories — that everything else is measured against.
We author rubric verifiers with criterion, justification, and evidence — calibrated to the difficulty target.
Contributors execute the task under clear guidelines; every submission is traceable and evidence-grounded.
Quality leads review submissions in layers — catching noise, calibration drift, and spec violations before delivery.
Where applicable, we run candidate models in Docker with pytest-based scoring for fully reproducible results.
Final artifacts ship as structured JSON with patches, logs, and taxonomies — ready to drop into your pipeline.
A slice of the capability case studies we deliver — across SFT, RLHF, RLVR, agent trajectories, and multimodal evaluation.
Two ways in — whether you have work to ship or want to contribute to it.