In short
Senior ML engineer who owns the AI evaluation and benchmarking baseline: constructing eval sets aligned to the golden dataset, automating the accuracy measurement harness, integrating MLflow for experiment tracking, and delivering KPI observability reporting. Core stack is Python and MLflow.
About the role
Role Summary
Owns the AI Evaluation and Benchmarking Baseline: eval set construction aligned to the golden dataset, and accuracy measurement harness automation.
Key responsibilities
- Construct the evaluation set aligned to the golden dataset
- Automate the accuracy measurement harness
- Integrate MLflow for experiment tracking
- Own KPI observability reporting
Required skills & experience
- 5+ years ML engineering experience with an evaluation / observability focus
- Hands-on experience with MLflow or an equivalent experiment-tracking tool
- Benchmarking framework design experience, including KPI reporting for stakeholder review
- Strong Python skills
Preferred qualifications
- Experience resequencing an eval milestone around a late-arriving dataset dependency
- KPI observability reporting experience for AI systems specifically
Education
B.Tech/MCA in Computer Science, AI/ML, or a related field, or equivalent practical experience.
Requirements
- 5+ years ML engineering experience with an evaluation / observability focus
- Hands-on experience with MLflow or an equivalent experiment-tracking tool
- Benchmarking framework design experience, including KPI reporting for stakeholder review
- Strong Python skills
- B.Tech/MCA in Computer Science, AI/ML, or a related field, or equivalent practical experience
Nice to have
- Experience resequencing an eval milestone around a late-arriving dataset dependency