Senior Research Scientist, Model Evaluation

In this role you will advance the state-of-the-art in LLM evaluation by building ambitious benchmarks and scalable evaluation tools. You will collaborate with cross-functional teams to translate model feedback into trustworthy assessments that reflect real capabilities. Your work will shape how we measure and guide progress as models scale. This is a hands-on, rigorous role at the cutting edge of AI evaluation with meaningful impact on product quality and safety.

• Open and inclusive culture
• Health and dental benefits
• Remote-flexible with global offices
• Parental leave top-up
• Stipends for meals and co-working
• Six weeks of vacation

• Create ambitious new evaluation benchmarks to push model capabilities
• Collaborate with cross-functional teams to translate feedback into reliable evaluations
• Conduct research to advance LLM evaluation methods, including training judges and refining data synthesis pipelines
• Build scalable, reusable tools to analyze model performance

• Strong software engineering skills
• Experience reviewing complex data and LLM outputs to ensure high data quality
• Passion for rigorous measurement and alignment with real capabilities
• Proactive prototyping to demonstrate LLM boundaries

• Cross-functional collaboration
• Analytical rigor
• Curiosity
• LLM evaluation methods
• Data synthesis pipelines
• Tooling for model performance analysis