How to apply for AI Safety Argumentation Platform Research Engineer

About Future of Life Institute

The Future of Life Institute is a nonprofit focused on reducing global catastrophic and existential risks from advanced AI. CARMA, its Center for AI Risk Management & Alignment, builds tools and frameworks to help society manage transformative AI risks. This is a chance to work on AI safety infrastructure rather than at a typical tech company.

About the role

You will build and operate an evidentiary infrastructure for AI risk arguments. The system combines ontologies, knowledge graphs, defeasible argumentation frameworks, and LLM-assisted population pipelines under expert curation. Your work feeds structured argument content into communications flows for policymakers, technical audiences, journalists, and the public.

A typical day

You might spend the morning refining an ontology for AI risk claims, then work on the LLM pipeline that extracts arguments from papers. In the afternoon, you could review structured arguments with domain experts and adjust the defeasible framework. You may also prepare argument content for a communications flow aimed at policymakers or journalists.

Who Future of Life Institute is looking for

• Has hands-on experience with defeasible argumentation frameworks or formal argumentation theory.
• Can design and maintain ontologies and knowledge graphs for complex, contested domains.
• Has built LLM-assisted data population pipelines with expert curation or human-in-the-loop review.
• Understands AI risk arguments well enough to formalize them and spot weak or motivated reasoning.

Tips for this application

• Show concrete work with argumentation frameworks or knowledge graphs. Link to code, papers, or systems you built.
• Explain how you would turn fragmented AI risk literature into structured, machine-readable arguments.
• Address the LLM-assisted population pipeline directly. Describe how you handle quality control and expert curation.
• Mention any experience communicating technical arguments to policymakers, journalists, or non-technical audiences.
• Note that the role is remote but requires occasional travel. Confirm you can meet that requirement.

What to cover in your cover letter

Describe a specific system you built that combined formal methods with LLMs or knowledge graphs. Explain how you would structure AI risk arguments so they are both formally tractable and persuasive. Show you understand why bad arguments for AI risk are bad, not just that they are bad. State how you would work with expert curators to keep the pipeline accurate.

Research before applying

• Read CARMA's published work on AI risk management and alignment. Look for their stance on AGI and ASI risks.
• Study the Future of Life Institute's existing projects and policy positions to understand their framing.
• Learn the basics of defeasible argumentation frameworks and ontology design if you are not already familiar.
• Find any public talks or papers by CARMA or FLI staff about argumentation, knowledge graphs, or AI risk communication.

Likely interview topics

Based on the job description, expect questions about:

• How would you represent a defeasible argument about AGI risk in a knowledge graph?
• What is your approach to LLM-assisted population of an argumentation system while maintaining expert curation?
• How do you decide which AI risk arguments are well-formed versus motivated?
• Describe a time you built infrastructure that had to serve both technical and non-technical users.
• How would you measure whether the system is actually helping stakeholders understand AI risk arguments?

Common mistakes to avoid

• Treating this like a standard software engineering role. The job is about formal argumentation and AI risk, not generic app development.
• Ignoring the expert curation requirement. The pipeline is not fully automated, and you should not propose replacing experts with LLMs.
• Being vague about AI risk. You need to show you can engage with specific arguments about AGI and ASI catastrophic risks, not just say you care about safety.