# Expert human data for frontier AI.

> Verified engineers, scientists, doctors and Indic-language specialists, ready in days.

HTML version: https://www.anthroly.com/expert-data

## At a glance

- 500 verified experts on the bench
- 100 hold a PhD
- 10 days to a staffed pilot team
- 100 experts deployed within a week of scale-up

## Services

### RL environments and tasks

Engineers build tasks an agent has to actually complete: a bug to fix in a real repository, a ticket to close in a simulated enterprise system. Every task ships with an automatic grader, so the reward is computed, not guessed. Experts then attack the grader to find the shortcuts a model could take to score without doing the work.

- Repository tasks with fail-to-pass and pass-to-pass tests
- Enterprise workflows with tools, state and a checkable end result
- An automatic grader per task, plus a reward-hacking review
- Held-out task splits kept separate from training sets

### Expert evals and benchmarks

A benchmark is only useful if it is hard. Specialists write prompts from their own field, write a rubric of small, checkable criteria for each one, and run them against current frontier models. Prompts the models already pass are cut. What remains measures the gap you care about.

- Prompts written from first-hand professional experience
- Atomic rubrics: each criterion checkable on its own
- Model-failure verification before a prompt is accepted
- Private, never-published sets to avoid contamination

### SFT data

Demonstrations of what a great answer looks like, written by the people who give those answers for a living. Single-turn responses, multi-turn conversations with follow-ups and corrections, and step-by-step reasoning where the path matters as much as the result.

- Ideal responses written from scratch, never model drafts
- Multi-turn conversations with realistic follow-ups
- Worked reasoning for math, science and code
- Style and policy guides applied consistently

### RLHF

Experts compare model outputs side by side, pick the better one and explain why. Where a single preference is too blunt, they grade outputs against a rubric so your reward model learns which parts of an answer were right, not just which answer won.

- Pairwise and ranked preferences with written rationales
- Rubric-based grading of individual outputs
- Error tagging: factual, reasoning, safety, instruction-following
- Calibration rounds so raters judge the same way

### Indic expert data

Most expert data is written in English. We build it natively in Indian languages: clinical questions written by doctors who practise in those languages, legal reasoning from lawyers who argue in them, and STEM problems from teachers and researchers who think in them. Written in the language, not translated into it.

- Medical, legal and STEM tasks authored natively
- Evals and rubrics in Indian languages, including code-mixed text
- SFT and preference data reviewed by native-speaking specialists
- Cultural and regional context checks, not just grammar

## Domains

- **Software and coding:** Software engineers, SREs, data and ML engineers. Repository tasks, code review, debugging, terminal and tool use.
- **Math and science:** Mathematicians, physicists, chemists, biologists, many at PhD level. Proof-style problems, research-grade questions, worked solutions.
- **Medicine:** Licensed doctors and clinical specialists. Clinical reasoning, diagnosis cases, guideline-grounded answers.
- **Indian languages:** Native-speaking doctors, lawyers, engineers and teachers. Medicine, law and STEM written natively in Indian languages.

## Vetting

- **Identity check:** Government ID verified for every expert before they see a single task.
- **Credential check:** Degrees, licences and work history confirmed against the claims made in the application.
- **Paid skills test:** A real task in their field, scored by a senior reviewer. We pay for the test, so strong people take it seriously.
- **Project calibration:** Before live work, experts complete gold-answer tasks for your project and only move on once they match the standard.

## Quality controls

- **Gold-answer checks:** Tasks with known correct answers are mixed into live work. Experts never know which ones they are.
- **Two layers of review:** Every item is checked by a second expert, then sampled by a senior reviewer before delivery.
- **Agreement scores:** We measure how often experts agree with each other and with the gold answers, and act on the gaps.
- **No AI tools, strictly:** Experts may not use AI tools to produce or edit their work. Breaking the rule ends the engagement.
- **Weekly quality reports:** Every week you get the numbers: accuracy, agreement, rework and what we changed because of them.

## Security

- **NDA and IP assignment:** Every expert signs both before starting. Everything they produce belongs to you.
- **Managed accounts:** Work happens in accounts we issue and control, never in personal ones.
- **Two-factor login:** Required on every account that can touch your project.
- **Least-access permissions:** Experts see only the tasks assigned to them, and access is removed when the work ends.
- **No public file sharing:** No public links, no personal drives, no attachments outside the project workspace.

## How an engagement runs

- **01 Pilot** (1–2 weeks · free or low cost): A small expert team works on a slice of your real task. You see the data, the review notes and the quality numbers before committing to anything.
- **02 Scoped team** (Agreed volume and quality bar): We agree the guidelines, the acceptance criteria and the weekly volume, then staff a dedicated team matched to the domain.
- **03 Scale-up** (Up to 100 experts in a week): Once quality holds, we add vetted experts from the bench. Same calibration, same review layers, same weekly report.

## FAQ

### What does a pilot cost?

Pilots run for one to two weeks and are free or low cost, depending on the size of the task. The point is for you to judge the data before you commit to a scoped team.

### How fast can you staff a project?

We can put a vetted pilot team in place within 10 days. Once a project scales, we can deploy up to 100 experts from our bench of 500 within a week.

### How do you vet experts?

Every expert passes an identity check, a credential check and a paid skills test scored by a senior reviewer. Before live work on your project, they also complete gold-answer calibration tasks.

### How do you stop experts using AI tools?

It is a strict rule in every expert agreement, and breaking it ends the engagement. We combine it with gold-answer checks and two layers of review, which catch work that does not match an expert’s own standard.

### Who owns the data?

You do. Every expert signs an NDA and an IP assignment before they start, so everything produced on your project belongs to you.

### Which Indian languages do you cover?

Tell us the languages and domains you need. We match native-speaking specialists in medicine, law and STEM and confirm coverage before the pilot begins.

### How do I join as an expert?

Email sharatth@anthroly.com with your field, highest degree, languages and a link to your CV or LinkedIn. If there is a fit, we send a paid skills test.

## Get started

- Request a pilot: https://cal.com/sarath-kavuru-0uizws/15min
- Join as an expert: sharatth@anthroly.com