Application Guide

How to Apply for Research Lead, Evaluations and Benchmarks

at Alice

🏢 About Alice

Alice is a trust, safety, and security AI company focused on measuring and mitigating frontier AI risks. Working here means being at the forefront of AI safety, developing benchmarks that inform labs, policymakers, and the public about emerging harms. The company's remote-first, research-driven culture appeals to those who want to shape the future of safe AI.

About This Role

As Research Lead for Evaluations and Benchmarks, you'll own the end-to-end process of shipping safety benchmarks every 2-3 weeks, from ideation to release. You'll set the quality bar, manage freelancers and researchers, and collaborate with the CTO to align benchmarks with emerging risks. This role is pivotal in ensuring that AI safety evaluations are rigorous, novel, and reproducible across labs.

💡 A Day in the Life

A typical day might involve reviewing draft evaluations from researchers, coordinating with freelancers on data collection, and meeting with the CTO to refine the quarterly roadmap. You'll also spend time reading recent AI safety papers and touching base with lab contacts to stay ahead of emerging risks.

🚀 Application Tools

Generate Cover Letter

AI-powered, tailored for Alice

Interview Prep

Practice questions for this role

🎯 Who Alice Is Looking For

  • Has hands-on experience designing and shipping AI safety benchmarks or evaluations, ideally in a fast-paced research environment.
  • Possesses strong project management skills, with a track record of coordinating freelancers and researchers to meet tight deadlines.
  • Deeply understands the AI safety ecosystem, including key risks, evaluation methodologies, and reproducibility challenges.
  • Can balance technical depth with strategic vision, contributing to quarterly roadmaps and adapting to emerging needs.

📝 Tips for Applying to Alice

1

Highlight specific benchmarks or evaluations you've shipped, emphasizing novelty, clarity, and reproducibility.

2

Demonstrate your ability to manage multiple stakeholders by describing how you've directed freelancers or kept researchers on schedule.

3

Show familiarity with Alice's existing work—mention their benchmarks or publications and suggest how you could extend them.

4

Quantify your impact: e.g., 'shipped X benchmarks in Y months' or 'reduced evaluation turnaround by Z%.'

5

In your cover letter, propose a potential benchmark idea that addresses an emerging AI safety risk, showcasing your proactive thinking.

✉️ What to Emphasize in Your Cover Letter

['Your experience shipping benchmarks rapidly without sacrificing quality, with concrete examples.', 'Your ability to lead and coordinate distributed teams, including freelancers and researchers, to meet aggressive release cycles.', 'Your strategic contributions to research roadmaps, ideally in collaboration with C-level executives.', 'Your up-to-date knowledge of the AI safety landscape, including recent developments and key lab contacts.']

Generate Cover Letter →

🔍 Research Before Applying

To stand out, make sure you've researched:

  • → Review Alice's website and any published benchmarks to understand their methodology and focus areas.
  • → Read recent papers or blog posts from Alice's team to grasp their perspective on AI safety and evaluation.
  • → Identify key players in the AI safety ecosystem (e.g., labs, researchers) and understand how Alice collaborates or competes with them.
  • → Look into the CTO's background and public statements to align your interview answers with their vision.
Visit Alice's Website →

💬 Prepare for These Interview Topics

Based on this role, you may be asked about:

1 How would you design a benchmark to measure a specific frontier AI risk (e.g., deception, bias) in 2-3 weeks?
2 Describe a time you managed a research project under a tight deadline. How did you ensure quality and reproducibility?
3 What emerging AI safety risks do you think are underexplored, and how would you prioritize them in a quarterly roadmap?
4 How do you maintain relationships with researchers across labs while ensuring your benchmarks are unbiased and rigorous?
5 Walk us through your process for reviewing an evaluation for novelty, clarity, and reproducibility.
Practice Interview Questions →

⚠️ Common Mistakes to Avoid

  • Focusing only on technical skills without demonstrating project management or leadership experience.
  • Ignoring the fast-paced nature of the role—failing to show how you can ship benchmarks every 2-3 weeks.
  • Being vague about AI safety knowledge; avoid generic statements without referencing specific risks or evaluations.

📅 Application Timeline

This position is open until filled. However, we recommend applying as soon as possible as roles at mission-driven organizations tend to fill quickly.

Typical hiring timeline:

1

Application Review

1-2 weeks

2

Initial Screening

Phone call or written assessment

3

Interviews

1-2 rounds, usually virtual

✓

Offer

Congratulations!

Ready to Apply?

Good luck with your application to Alice!

← Back to Job Listing Apply Now →