Rubrics and graded answers for RL
For verifiable tasks, where a script can check the final answer, we write the reference answer. For open-ended tasks, an expert writes a rubric for that task alone: what a full-credit answer must include and how many points each part earns.
We can also grade your model's responses against the rubric.
What you can order
- Rubrics written for your tasks
- Answers graded against a rubric
- Tasks with reference answers
Who grows it
Experts in each task's field write the rubrics and do the grading.
What to tell us on the order slip
- The tasks, or a few examples of the kind you have in mind.
- Whether answers can be checked automatically or need a rubric.
- Pass or fail, or points per criterion.
- How many tasks and graded answers, and your deadline.
If you will train against a model-based grader, order graded answers along with the rubrics, and check its scores against the expert's on the same responses before training.
How to get a free sample
- Send us an order slip.
- Agree the spec with a farmer.
- Experts grow a free pilot batch to your spec.
- Run it through your evals, then order by the crate.

