How to apply for Research Scientist

About AI Verification and Evaluation Research Institute

The AI Verification and Evaluation Research Institute (AVERI) is a nonprofit dedicated to ensuring that the most powerful AI systems are rigorously audited for safety and security by independent third parties. Unlike many AI safety organizations, AVERI focuses on hands-on pilot audits with AI companies, bridging the gap between theoretical safety research and real-world implementation. Working here means shaping the emerging field of frontier AI auditing and having a direct impact on how AI companies demonstrate safety.

About the role

As a Research Scientist at AVERI, you will design and run evaluations of frontier AI systems' capabilities, risks, robustness, and alignment-relevant behavior. You will develop audit methods and protocols, build reproducible experimental pipelines, and produce audit reports that translate technical findings into actionable recommendations for developers and policymakers. This role is impactful because you will directly influence how AI companies are held accountable and help establish auditing standards for the industry.

A typical day

A typical day might involve designing a new evaluation protocol for a frontier LLM's robustness, coding the experimental pipeline, and analyzing results. You might also meet with an AI company's safety team to discuss audit findings, draft a section of an audit report, or brainstorm with policy experts on how to frame recommendations. Collaboration is key, so you'll frequently interact with engineers, researchers, and external partners to refine methods and ensure reproducibility.

Who AI Verification and Evaluation Research Institute is looking for

• PhD in AI/ML or related field plus 3+ years of independent research experience beyond the PhD, preferably in AI safety, evaluations, robustness, or security.
• Proven track record of designing and executing rigorous empirical studies with large language models in safety-relevant contexts, such as red-teaming, bias evaluation, or robustness testing.
• Experience building reproducible experimental pipelines and tools for evaluating AI systems, with strong software engineering skills in Python and ML frameworks.
• Ability to communicate complex technical findings to diverse stakeholders, including AI developers, policymakers, and standards bodies, and to work collaboratively in cross-functional teams.

Tips for this application

• Highlight specific examples of AI safety or evaluation research you have conducted, including details on experimental design, metrics, and outcomes, and how they relate to auditing frontier models.
• Demonstrate familiarity with AVERI's pilot audit approach by referencing their public reports or blog posts, and explain how your experience aligns with their mission of third-party auditing.
• Show your ability to build reproducible pipelines by linking to open-source code or detailed methodology sections from past projects, emphasizing rigor and scalability.
• Address the nonprofit context: explain why you are motivated to work in a mission-driven organization and how you can contribute to shaping auditing standards, not just publishing papers.
• Tailor your resume to include keywords from the job description, such as 'audit methods', 'alignment-relevant behavior', 'reproducible experimental pipelines', and 'actionable recommendations'.

What to cover in your cover letter

['Your hands-on experience evaluating large language models in safety-critical contexts, with concrete examples of how your findings influenced AI development or policy.', 'Your ability to design and implement audit protocols from scratch, including any experience collaborating with external partners or standards bodies.', "Your commitment to AVERI's nonprofit mission and understanding of the unique challenges in third-party auditing, such as balancing transparency with proprietary concerns.", 'Your track record of translating technical results into actionable recommendations for non-technical audiences, such as policymakers or industry practitioners.']

Research before applying

• Read AVERI's published pilot audit reports and blog posts to understand their methodology, focus areas, and findings.
• Explore the backgrounds of AVERI's team members, including their research interests and prior affiliations, to identify potential collaboration areas.
• Familiarize yourself with existing AI auditing frameworks and standards, such as those from NIST, ISO, or the EU AI Act, to understand the regulatory landscape AVERI operates in.
• Investigate recent frontier AI safety evaluations from organizations like ARC, METR, or Apollo Research to see how AVERI's approach differs or complements them.

Likely interview topics

Based on the job description, expect questions about:

• How would you design an audit to evaluate a frontier LLM's robustness to adversarial attacks, and what metrics would you use?
• Describe a time when your evaluation of an AI system revealed a safety risk. How did you communicate it and what was the outcome?
• What are the key challenges in creating reproducible evaluation pipelines for large language models, and how would you address them?
• How do you stay current with the latest AI safety research, and which recent papers or developments have influenced your approach?
• Why do you want to work at a nonprofit like AVERI, and how do you see your role contributing to the broader AI auditing ecosystem?

Common mistakes to avoid

• Focusing only on academic publications without demonstrating how your research translates into practical audits or evaluations of real-world AI systems.
• Neglecting to mention experience with large language models or safety-relevant contexts, which is a core requirement for this role.
• Submitting a generic cover letter that doesn't address AVERI's nonprofit mission or the specific challenges of third-party auditing, suggesting a lack of genuine interest.