How to apply for Research Engineer

About Neo Research

Neo Research is a Singapore-based research lab focused on ensuring the safe deployment of frontier AI at scale. Unlike many AI labs, they specialize in safety evaluations for loss-of-control and harmful manipulation risks, making their work critical for responsible AI development. Working here means contributing to cutting-edge safety research in a remote, collaborative environment.

About the role

As a Research Engineer, you will implement and run safety evaluations on frontier AI models, focusing on loss-of-control and harmful manipulation risks. You'll build and maintain agent scaffolds, tool integrations, and evaluation infrastructure, ensuring reproducibility across all experiments. This role is impactful because your work directly helps prevent catastrophic AI failures and informs safety practices at the highest levels.

A typical day

A typical day might start with a stand-up to discuss ongoing evaluations, then you'll write code to integrate a new tool into an agent scaffold or debug a logging issue. After lunch, you'll run a batch of evaluations on a new model release, analyze the results, and document findings for the research team. The day ends with a review of reproducibility checks and planning next steps.

Who Neo Research is looking for

• Has hands-on experience implementing evaluations on large language models or other AI systems, using frameworks like LangChain, AutoGPT, or custom agent scaffolds.
• Proficient in Python, with strong skills in building agent tool integrations (e.g., APIs, code executors) and managing environments (Docker, conda, experiment tracking).
• Demonstrates meticulous attention to reproducibility, including logging, sampling strategies (temperature, top-p), and result analysis with statistical rigor.
• Can translate vague research questions (e.g., 'Can this model manipulate a user?') into concrete, testable evaluation protocols with clear metrics.

Tips for this application

• Tailor your resume to highlight specific evaluation projects you've built, not just research you've participated in. Use metrics (e.g., 'Built agent scaffold that reduced evaluation time by 40%').
• In your cover letter, explicitly mention your experience with reproducibility practices (e.g., version control for prompts, seed locking, environment snapshots).
• Showcase any work with agent scaffolds, particularly if you've integrated tools like web search, code execution, or file manipulation. Link to GitHub repositories.
• Research Neo Research's published work on safety evaluations (e.g., their blog or papers) and reference specific risk categories (loss-of-control, manipulation) in your application.
• Prepare a brief portfolio or write-up of a past evaluation you've run, including challenges faced and how you ensured rigor. Attach it as a supplement.

What to cover in your cover letter

["Emphasize your ability to build robust evaluation infrastructure, not just run one-off tests. Mention tools you've used (e.g., MLflow, Weights & Biases, Docker).", 'Describe your experience with agent safety evaluations, especially for loss-of-control or manipulation scenarios. Provide a concrete example.', "Highlight your collaborative skills with researchers: how you've turned informal research questions into structured experiments with clear success criteria.", "Express genuine interest in AI safety and alignment, and connect your past work to Neo Research's mission of safe deployment at scale."]

Research before applying

• Read Neo Research's published work on safety evaluations, especially any blog posts or papers about loss-of-control risks or manipulation.
• Understand the company's stance on frontier AI safety and how they differ from other labs (e.g., Anthropic, OpenAI). Check their website and interviews.
• Review the latest research on agent scaffolds and tool use in LLMs, such as from the Toolformer or WebGPT papers, to discuss current best practices.
• Look into common failure modes in agent evaluations (e.g., reward hacking, specification gaming) and be ready to discuss how you'd mitigate them.

Likely interview topics

Based on the job description, expect questions about:

• Design an evaluation protocol for a frontier model to test if it can manipulate a user into revealing sensitive information. Walk through your approach.
• How do you ensure reproducibility when running evaluations across different environments? Discuss logging, versioning, and environment management.
• Describe a time you had to debug an agent scaffold failure. What tools did you use and how did you fix it?
• Given a research question like 'Does the model exhibit sycophancy?', how would you operationalize it into a rigorous evaluation with metrics?
• How do you handle stochasticity in model outputs? Discuss sampling strategies, number of runs, and statistical analysis.

Common mistakes to avoid

• Don't focus only on model training or fine-tuning experience; this role is about evaluation and infrastructure, not building models.
• Avoid vague claims like 'I care about AI safety' without concrete examples of safety-related work or projects.
• Don't neglect reproducibility: failing to mention logging, version control, or environment management will signal inexperience with rigorous evaluations.