How to apply for Senior Software Engineer (AI Inference & Runtime Platform)

About AZX, PBC.

AZX is a company focused on AI transformation for critical industries, with a specialization in physics-informed ML and enterprise AI solutions for climate and sustainability challenges. They have worked with category leaders like CBRE, LevelTen Energy, Flexe, and utilities, and are backed by investors focused on AI, climate, and energy. The company bootstrapped profitably for its first year and aims to create a long-term place to work for those passionate about AI and positive impact.

About the role

This senior software engineer role owns the inference control plane and agent-sandboxing platform at AZX. You will manage open-weight models and custom task-model zoos across managed GPU clouds and customer-managed Kubernetes clusters, focusing on scale-to-zero economics, cold-start discipline, and per-token cost accounting. You will also build hardware-isolated microVMs for running untrusted, agent-generated code.

A typical day

You might start by reviewing inference metrics and cost dashboards, then work on optimizing cold-start times for a new model. Later, you could collaborate with the team to deploy a Kubernetes operator update or test a new microVM isolation feature. You will also troubleshoot any issues with per-token accounting or client disconnects.

Who AZX, PBC. is looking for

• Experience operating inference systems for open-weight models, including cold-start optimization and per-token cost accounting.
• Strong Kubernetes skills: operators, autoscaling, node lifecycle management, and multi-cluster deployments.
• Familiarity with hardware-isolated microVMs (e.g., Firecracker) for sandboxing untrusted code.
• Ability to work across managed GPU clouds and customer-managed Kubernetes environments.

Tips for this application

• Highlight specific projects where you managed inference cost accounting, especially handling client disconnects mid-stream.
• Describe your experience with Kubernetes operators and autoscaling in production, including node lifecycle management.
• Mention any work with microVM technologies (e.g., Firecracker, Kata Containers) for sandboxing.
• Explain how you have optimized cold-start times for model inference in a cloud or Kubernetes setting.
• Reference AZX's clients (CBRE, LevelTen Energy, Flexe) and connect your work to climate and sustainability challenges.

What to cover in your cover letter

['Your hands-on experience with inference control planes and model hosting across GPU clouds and Kubernetes.', 'Specific examples of implementing scale-to-zero economics and accurate per-token cost accounting.', 'Your work with hardware-isolated microVMs for running untrusted code, and how it relates to agent sandboxing.', 'Why you want to apply your skills to climate and sustainability challenges at AZX.']

Research before applying

• AZX's clients and projects in real estate, energy, logistics, and utilities to understand their AI applications.
• The company's focus on physics-informed ML and how it differs from standard ML approaches.
• Recent developments in open-weight models and inference optimization techniques.
• Kubernetes operators and microVM technologies like Firecracker for sandboxing.

Likely interview topics

Based on the job description, expect questions about:

• How to design a per-token cost accounting system that remains correct when a client disconnects mid-stream.
• Strategies for reducing cold-start latency in serverless GPU inference.
• Kubernetes operator patterns for managing model deployments and autoscaling.
• Security and isolation trade-offs in microVM-based sandboxing for agent-generated code.
• Approaches to multi-cloud and customer-managed Kubernetes deployments for AI workloads.

Common mistakes to avoid

• Focusing only on model training and ignoring inference and runtime concerns.
• Not demonstrating experience with cost accounting or scale-to-zero economics in cloud environments.
• Overlooking the security aspects of running untrusted code in sandboxes.