About the role
You will get new model architectures running on the hardware first, in simulation. You will implement operators, numerics and memory layouts bit-exact against reference implementations, extend simulator coverage across the inference path, evaluate new architectures, bring up customer models and help automate that process.
Responsibilities
- Implement operators, numerics and memory layouts for new model architectures bit-exact against a reference
- Extend simulator coverage across prefill, decode, embeddings, attention variants, mixture-of-experts routing and quantized formats
- Maintain reference comparisons and regression coverage
- Evaluate new architectures for compute, memory and data movement needs
- Bring customer models up in simulation and own turnaround time
- Partner on automating new-model bring-up end to end
Requirements
- 4+ years as an ML systems or inference engineer
- Strong understanding of transformer internals including attention variants, MoE, embeddings, KV cache and quantization (INT8, FP8)
- Experience implementing or porting model inference in PyTorch, llama.cpp, vLLM, TensorRT-LLM or a custom runtime
- Experience debugging numerical mismatches down to the bit
- Ability to turn architecture papers into verified implementations
- Strong Rust skills or expert C/C++ with willingness to use Rust, and strong Python skills
- Excellent communication skills
Benefits
- Fully company-paid medical, dental and vision insurance for you and dependents
- Company-paid life and disability coverage
- Supplemental hospital, critical illness and accident coverage available
- Unlimited paid time off
- 13 paid company holidays
- Remote-first culture with company-provided computer and home office setup
- Equity
- 401(k) with company matching from day one
Requirements
- 4+ years as an ML systems or inference engineer
- Strong understanding of transformer internals including attention variants, MoE, embeddings, KV cache and quantization (INT8, FP8)
- Experience implementing or porting model inference in PyTorch, llama.cpp, vLLM, TensorRT-LLM or a custom runtime
- Experience debugging numerical mismatches down to the bit