About the role
Nebius is hiring a Senior Machine Learning Engineer on the Applied AI team to own model and endpoint optimization from artifacts to production deployment. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and inference engine tuning to improve latency, throughput, memory efficiency, GPU utilization, and token cost.
You will collaborate with kernel and platform engineers to diagnose issues, evaluate configurations, and ensure safe production rollouts
Own optimization projects for model families, endpoints, or backends. Compare inference engines and recommend serving configurations per workload. Diagnose model-quality and performance regressions in production. Improve LLM/VLM endpoint latency, throughput, memory efficiency, GPU utilization, and cost per token. Deploy, configure, benchmark, and extend engines like vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo. Develop model-compression workflows including quantization and distillation. Implement speculative decoding, KV-cache optimization, and disaggregated prefill/decode serving. Strong programming skills in Python and PyTorch. Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference. Practical knowledge of modern inference stacks (e.g., vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo). Understanding transformer bottlenecks: KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving. Ability to quantify trade-offs among latency, throughput, quality, utilization, and cost. Experience designing distributed inference architectures (request routing, scheduling, PDD, multi-node inference, or expert parallelism). Strong communication and collaboration across research, kernel, infrastructure, product, and customer teams. Competitive compensation Career growth Learning opportunities Flexibility Ownership Collaborative culture Impactful AI projects International environment