About the role
You will accelerate and optimize AI inference pipelines for video and language models. You will implement attention optimization, quantization, GPU parallelism, high-performance CUDA and NCCL workloads, and production deployments. You will contribute to code reviews, technical discussions, mentoring, and, optionally, training-efficiency improvements.
Responsibilities
- Lead and implement inference acceleration techniques, including attention optimization and quantization
- Engineer and optimize tensor, sequence, and pipeline parallelism for distributed inference
- Develop and optimize high-performance kernels and distributed workloads using CUDA and NCCL
- Deploy video-generation and large language models into production
- Contribute to model training speed, stability, and resource-utilization improvements
- Drive code reviews, technical discussions, and mentoring on inference and GPU programming
Requirements
- 5+ years of engineering experience
- Inference acceleration and model deployment experience at scale
- Inference optimization expertise, including quantization, attention acceleration, and deep learning compiler stacks
- GPU programming knowledge, including CUDA and NCCL
- Distributed inference parallelism experience, including SP, TP, and PP
- Familiarity with video-generation models and large language models
- Cross-disciplinary communication skills
Benefits
- Equity
- Comprehensive health benefits
- Monthly stipends
- Company retreats
Requirements
- 5+ years of engineering experience
- Inference acceleration and model deployment experience at scale
- Inference optimization expertise, including quantization, attention acceleration, and deep learning compiler stacks
- GPU programming knowledge, including CUDA and NCCL
- Distributed inference parallelism experience, including SP, TP, and PP
- Familiarity with video-generation models and large language models