Careers

Summary

CaML is hiring an ML researcher to run our training work: synthetic document mid-training, RL post-training on top of it, and the evals that tell us whether the values we put in are still there afterwards. We want someone who has run LLM training experiments before, follows alignment research closely, and will tell us when we’re wrong.

What CaML does

CaML is an AI safety nonprofit. We study how values get into models during training and whether they survive the training that comes after: how to instill a specific set of values through mid-training data and RL, how to measure whether it worked, and how to keep it from being trained back out.

So far we’ve built the TAC benchmark (NeurIPS 2026 AI4Good workshop), the Manager Coercion Bench, HarvestBench in UK AISI’s Inspect evals, and the leaderboard at compassionbench.com. Our current open question is whether values instilled in mid-training survive later RL, and when they don’t. We’re a small California nonprofit.

The work

• RL post-training on open-weight models, mostly around the 12B scale. RLHF, RLAIF, DPO and its variants, GRPO, RLVR.
• Mid-training and continued pretraining data: synthetic document generation at scale, matched control conditions, mixing ratios, and measuring what the data did to the model.
• Evals that catch the difference between a model that says the right thing and a model whose reasoning holds up under adversarial prompting and distribution shift.
• Studying how values degrade through training: reward hacking, sycophancy, specification gaming, and whether an instilled value generalises.
• Writing it up for arXiv and LessWrong.

You’d own the training side of the research agenda in a small team with no management layers.

How we work

We want to be told when we’re wrong, by the people who work here. That includes being wrong about things the EA and alignment communities mostly agree on. A lot of that agreement rests on a few influential posts that nobody has tested, and we’d rather find out than assume.

In practice this means reporting the negative result, saying “I don’t know” when that’s the truth, flagging when one of our own evals is broken even if it hurts a paper we’ve already published, and disagreeing with well-known people on the merits.

Requirements

We check these against the application before reading anything else.

• You’ve done your own research that involved training LLMs: RL or preference optimisation, fine-tuning, or mid-training on synthetic data, with results written up as a paper, a post or a program project like MATS or ASTRA.
• You can read a recent post-training paper and say specifically what you think is wrong with it or what you’d try next.
• You follow alignment research. You read LessWrong, and nothing in this ad needed explaining. Ideally you’ve posted there, or you argue about this work on Twitter and people in the field reply.
• You write clearly about technical work, in English.

Things that help but aren’t required:

• A LessWrong post, a paper, or a Twitter thread that people in the field read and responded to.
• Someone we know in alignment research who can vouch for you.
• MATS with any mentor, ASTRA or FIG.

We’re not hiring a pure engineer. If you’d rather be handed a spec than argue about what to run, this is the wrong role.

How to apply

The application form takes about 15 minutes. It asks three short questions, and we read the answers before the CV.

1. Your training experience. LLM training you’ve done in your own research, including at MATS, ASTRA or a similar program, and what you found. Two or three sentences.
2. A result you’d build on. A post-training or alignment result from the last twelve months that changed what you’d work on, and what you’d do next with it. Two or three sentences.
3. Something you wrote. A link to a LessWrong post, a paper, or a Twitter thread that you think is your best work on alignment or training.

Then link your CV, and add your LessWrong username, Twitter handle and Google Scholar link if you have them. None of those is required on its own, but an application with none of them will struggle against the requirements above.

If you’re a good fit, we’ll reach out within two weeks. After that: a 45-minute technical conversation about your answers and how you’d approach our research problem, a paid work trial of one to two weeks on a real problem, then an offer. Under a month end to end if you’re available.

Pay and terms

$200,000 to $300,000 depending on experience, full-time. Fully remote, so no visa is needed and we don’t sponsor one. You need some overlap with California working hours for calls, but you don’t need to work California hours. We’ll bring you to the Bay Area for onsites as needed.

Apply now →