AI Evaluation & Agent Systems Engineer

Build evaluation software for AI models and agents, designing tests and measuring behavioral changes.
- Develop evaluation harnesses for model behavior, tool use, and multi-step agent tasks.
- Record inputs, configurations, tool calls, and observations in structured traces.
- Build replay and regression tests comparing system behavior before and after changes.
- Requires experience with evaluation software, model integrations, or agent systems.
- Requires understanding of experiment design, nondeterminism, and measurement limitations.

Syntony supports frontier AI organizations, public institutions, and private organizations with safety research, governance design, adversarial evaluation, and risk advisory.