Preference data for RLHF and DPO
You send the responses, usually sampled from your own model. Our judges compare them two at a time or rank a full set, and write a rationale for each judgment when you ask for one.
A medical prompt is judged by a clinician, and a contract question by a lawyer.
What you can order
- Preference pairs (A vs. B)
- Rankings of several responses
Who grows it
Judges are matched to the field of each prompt.
What to tell us on the order slip
- Where the responses come from, and how many there are per prompt.
- Pairs or full rankings, and whether each judgment needs a written reason.
- Your criteria, and whether you have a rater guideline we should work from.
- How many comparisons you need, and by when.
Ask for written reasons on the pilot batch. On close calls they show where your guideline leaves room for judgment, and we settle those points with you before a larger order.
How to get a free sample
- Send us an order slip.
- Agree the spec with a farmer.
- Experts grow a free pilot batch to your spec.
- Run it through your evals, then order by the crate.

