About the role
You will bring new models and inference features to cloud platforms. You will build and operate validation and deployment infrastructure, identify cross-platform inference gaps, improve release speed and reliability, and use observability data to resolve production performance and cost regressions.
Responsibilities
- Bring up inference for new model architectures and ship them to cloud platforms
- Integrate new inference features for cloud platforms
- Identify and fix cross-platform inference differences at their source
- Design, build, and own CI/CD infrastructure for inference servers and load balancers
- Improve validation speed, parallelism, and cost efficiency without reducing reliability
- Analyze observability data and remediate performance bottlenecks, cost anomalies, and regressions
Requirements
- Strong interest in large language model serving
- Significant software engineering experience with high-performance large-scale distributed systems
- Experience building automation or test infrastructure that improved release velocity or reliability
- Experience building or operating services on AWS, GCP, or Azure
- Kubernetes, Infrastructure as Code, or container orchestration exposure
- Experience collaborating across functions
- Ability to learn new technologies, hardware platforms, and provider ecosystems
- End-to-end ownership of technical problems
Benefits
- Optional equity donation matching
- Generous vacation
- Parental leave
- Flexible working hours
Requirements
- Strong interest in large language model serving
- Significant software engineering experience with high-performance large-scale distributed systems
- Experience building automation or test infrastructure that improved release velocity or reliability
- Experience building or operating services on AWS, GCP, or Azure
- Kubernetes, Infrastructure as Code, or container orchestration exposure
- Experience collaborating across functions