About the role
Hailing from Mitratech, this role seeks a Senior Software Engineer specializing in Generative AI and large language models to architect and deliver production-grade AI systems for complex business needs. You will work on multi-agent orchestration, RAG pipelines, and rigorous evaluation frameworks in a fast-paced, globally distributed team.
The position emphasizes cloud-native AI infrastructure, CI/CD for models and agents, and robust data quality practices, with collaboration across engineering,
Design, build, and operate multi-agent workflows and tool-enabled agents, with orchestration logic, state management, safety guardrails, and fallback strategies. Architect and maintain end-to-end RAG systems, covering ingestion, chunking, embeddings, vector retrieval, reranking, and answer synthesis. Evaluate and integrate LLMs and GenAI services across cost, performance, and privacy dimensions; select a mix of managed and in‑house models. Develop, version, and optimise prompting strategies; implement automated prompt testing and regression tracking. Define and own evaluation frameworks for generative outputs, including automated metrics and human evaluation protocols. Apply classical NLP techniques and assess when to use them alongside LLMs. Build and operate scalable, secure AI infrastructure on AWS (Bedrock, SageMaker, Lambda, OpenSearch) following well-architected principles. Own the full deployment lifecycle: CI/CD for models and agents, testing strategies, observability, rollback procedures. Ensure data quality through rigorous validation and augmentation, and source datasets for training and evaluation. Agent Orchestration: Production experience designing multi-agent systems with tool use, memory/state management, and fault-tolerant routing. RAG & Retrieval: End-to-end pipelines: chunking, embedding models, vector databases, retrieval tuning, and answer synthesis. Evaluations: Pipelines for generative AI, automated scoring, human evaluation design, drift monitoring. LLMs & GenAI: Foundation models, Bedrock/OpenAI/Anthropic; fine-tuning and prompt engineering at scale. Traditional NLP: NER, text classification, intent detection, topic modelling; apply with LLMs when suitable. AWS Bedrock: Bedrock APIs, Knowledge Bases, Guardrails; Bedrock Model Evaluation is a plus. AWS Ecosystem: SageMaker, Lambda, ECS/EKS, S3, OpenSearch, IAM, CloudWatch, VPC networking. MLOps & CI/CD: Model registries, CI/CD for ML, feature stores, canary deployments, monitoring, rollback. IaC: Terraform or AWS CDK for reproducible infra. Python: Production-grade Python, packaging, pytest, type hints, async, ML pipelines. ML Frameworks: Familiar with traditional ML frameworks and fine‑tuning workflows. Experiment Tracking: Langfuse, Arize, Langsmith, or equivalent for metrics and artefacts.