How to apply for Evaluators, Systematic Reviews

About Elicit

Elicit is a public benefit company building an AI research assistant for researchers. Its system is designed for systematicity and transparency, with supervision of process rather than outcomes. This role suits people who care about research quality and want to shape how AI supports evidence synthesis.

About the role

You will evaluate systematic reviews written with Elicit's help in life sciences and social science. Your evaluations feed back into improving the product's output. You choose the review topics, which means you can work on questions you know well and analyse them in depth.

A typical day

You pick a review topic in your field, run it through Elicit, and read the output closely against the underlying literature. You write up an evaluation of its quality, noting where the process was sound and where it failed. You submit evaluations on your own schedule as a contractor.

Who Elicit is looking for

• Has hands-on experience conducting research in life sciences or social science, including familiarity with systematic review methods
• Can read a review and judge whether its search, screening, and synthesis are sound, not just whether the prose reads well
• Is comfortable writing detailed evaluations that explain why a review is strong or weak
• Can work independently on contract terms and manage their own topic selection and output

Tips for this application

• Name the specific systematic review guidelines or frameworks you have used (for example PRISMA or Cochrane) and describe a review you contributed to
• Explain how you would judge the quality of a review produced with AI assistance, including what you would check first
• Propose two or three review topics in life sciences or social science that you could evaluate in depth, and say why you know those areas
• Since the role is contract and remote, state your availability, time zone, and expected weekly output in the application
• Look at Elicit's product before applying and refer to a specific feature or workflow you would evaluate, rather than speaking about AI research tools in general

What to cover in your cover letter

Your direct experience with systematic reviews in life sciences or social science. How you assess review quality and what criteria you apply. Why evaluating AI-assisted reviews specifically interests you. A concrete example of a topic you would choose and how you would analyse it.

Research before applying

• Use Elicit on a research question in your field and note where the output is strong or weak
• Read Elicit's description of its product as a scalable ML system with supervision of process, not outcomes, and consider what that means for evaluation work
• Check what a public benefit company is and how it differs from a standard startup
• Find published systematic reviews in your field and compare their methods sections to see what good evaluation criteria look like

Likely interview topics

Based on the job description, expect questions about:

• Walk through a systematic review you have conducted or evaluated. What decisions did you make and why?
• How would you evaluate a review generated with Elicit's help? What would you check first, and what would make you flag it as low quality?
• How do you handle a topic where the evidence is mixed or the literature is thin?
• What is your process for keeping evaluations consistent across many reviews?
• Which life science or social science topics would you choose to work on, and what makes them suitable for this kind of evaluation?

Common mistakes to avoid

• Applying without having used Elicit or read how its review workflow operates
• Describing general research experience without mentioning systematic reviews or evaluation criteria
• Treating the role as generic AI data annotation rather than expert review of scientific literature