About the role
Senior AI Engineer
Company: Mallow Technologies
Job Location: Karur, Tamil Nadu, India
Employment Type: Full-time
Experience Required: 5+ years
Salary: 18L24L a year
About the Role
Role OverviewWe're looking for a Senior AI Engineer to design, build, and ship production-grade GenAI systems for our clients.
You'll own solutions end to end:
RAG systems, chatbots, and AI agents that are reliable, observable, and cost-efficient in production.
This role is about applied LLM engineering, not training models from scratch.
Key Responsibilities
Design and build RAG pipelines end to end, covering document ingestion, chunking strategies, embeddings, hybrid search, re-ranking, and retrieval evaluation
Build conversational AI and chatbots with multi-turn memory, grounding, fallback handling, and human-handoff flows
Develop AI agents and multi-agent workflows using frameworks such as Lang
Graph, Lang
Chain, OpenAI Agents SDK, or CrewAI, including tool calling, planning, and state management
Build and integrate MCP (Model Context Protocol) servers and clients to connect LLMs with client systems, APIs, and data sources
Apply context engineering to decide what goes into the context window (retrieval, memory, tool outputs, summarisation, compaction) for accuracy and efficiency
Optimise prompts systematically through versioning, A/B testing, and automated approaches (e.g., DSPy), not trial and error
Manage tokenomics:
track and reduce token usage and cost through prompt caching, model routing, batching, and right-sizing models per task
Evaluate, optimise, and monitor AI system performance for accuracy, reliability, latency, and cost, using tracing and observability tools like Lang
Smith, Langfuse, or Arize Phoenix; after deployment, catch issues such as data drift, prompt or model regressions, and degraded accuracy, and ship fixes
Build evaluation frameworks with golden datasets, LLM-as-judge, regression evals, and online/offline metrics, so that quality is measured before and after release
Implement guardrails for prompt-injection defence, PII handling, output validation, and hallucination controls
Collaborate with product managers, backend engineers, frontend developers, and clients to identify opportunities for AI-driven solutions, applying emerging AI technologies, frameworks, and best practices to real business problems; mentor junior AI engineers
Fine-tune open-weight models (e.g., Llama, Mistral, Qwen, Gemma) using LoRA/QLoRA, SFT, or DPO when prompting and RAG aren't enough, and decide when fine-tuning is actually justified (an added advantage)
Required Skills & Qualifications5+ years of software engineering experience, with 2+ years building and shipping LLM-based applications to production
Strong Python skills (async, APIs, testing); FastAPI or similar
Hands-on experience with LLM provider APIs (OpenAI, Anthropic, Gemini, Bedrock / Azure OpenAI), including tool calling and structured outputs
Proven experience building RAG systems, and an understanding of why they fail (retrieval quality, chunking, context limits)Experience building AI agents with Lang
Graph or an equivalent framework, and hands-on experience building MCP (Model Context Protocol) servers and clients
Working knowledge of vector databases (pgvector, Qdrant, Pinecone, Weaviate, or similar) and embedding models
Practical experience with LLM evaluation and observability tooling
Clear understanding of LLM cost and latency trade-offs and how to optimise them
Experience deploying AI services on a cloud platform (AWS, GCP, or Azure) with Docker and CI/CD
Good to Have
Hands-on fine-tuning experience (Hugging Face, Unsloth, Axolotl) and model serving (vLLM, Ollama, TGI)Experience with the A2A (Agent-to-Agent) protocol
Familiarity with prompt optimisation frameworks such as DSPyMultimodal experience (vision + language, document parsing)Exposure to AI security and governance practices (red-teaming, OWASP Top 10 for LLMs)Client-facing delivery experience in a services or consulting setup
Open-source contributions in the GenAI space
What We Offer
Opportunity to work on cutting-edge AI projects with chance to shape a growing AI practice: its patterns, tooling, and standardsA collaborative, engineering-driven culture
Competitive compensation and growth opportunities .