Member of Technical Staff, Model Training (infra & platform)

The work

Lead the distributed infrastructure that enables large AI experiments and efficient specialized model deployment. Make architectural decisions across training, GPU compute, inference, simulation and agent execution while owning demanding engineering components.

• Design and scale shared systems for supervised fine tuning, reinforcement learning and on policy distillation across language, multimodal and robotics workloads.
• Coordinate GPU workers, rollout creation, teacher inference, reward calculation and simulated environments in distributed pipelines.
• Optimize training and inference infrastructure for GPU use, memory, communication, storage and execution efficiency.
• Develop agent runtimes, tool interfaces, environment connections, model registries and serving APIs.
• Build experiment tracking, data and model version control, checkpoints, recovery and monitoring to support repeatable research and dependable operation.
• Find infrastructure bottlenecks and shorten experiment cycles as workloads grow in complexity.
• Define interfaces, architectural conventions and engineering standards for collaboration among research, infrastructure and applied teams.
• Lead initiatives from design to production, mentor other engineers and retain direct responsibility for difficult technical work.

Experience and skills

• Strong computer science grounding in distributed systems, concurrency, networking and performance.
• Extensive experience creating and operating scalable backend services or distributed computing infrastructure.
• Proficiency in Python and a systems language such as C++, Rust or Go.
• Practical experience with GPU ML infrastructure, distributed model training, inference or large compute orchestration.
• Understanding of compute efficiency, memory use, scheduling, communication costs and fault tolerance.
• Experience building reliable APIs and infrastructure for complex research or engineering workflows.
• Evidence of architectural ownership, cross team leadership and delivering systems relied on by other engineers.
• Comfort with evolving requirements and rapid research experimentation.
• Useful additional tools include PyTorch Distributed, DeepSpeed, Ray, Megatron-LM, vLLM and TensorRT-LLM.
• RL infrastructure, rollout coordination and distributed agent execution experience are welcome.
• CUDA, GPU kernels, NCCL, Kubernetes, containers, cluster scheduling and cloud GPU experience are valuable.
• Simulation, robotics infrastructure, large agent evaluation systems or open source infrastructure contributions are further advantages.

Working arrangements

This full time remote position has a national base salary range of $200,000 to $320,000. Employees are eligible for equity, while performance bonuses depend on the role and applicable policies.

Benefits include paid time off, reimbursement of up to 100% of health insurance premiums and a 401(k) with an employer match.