About the role
Job Description: Principal AI Engineer – Agentic AI / LLM
Location: Bank Street, Canary Wharf, London, UK
Work Model: Onsite / Hybrid – to be confirmed
Employment Type: Full-Time
Experience: 8–14 Years (10+ years preferred)
Working Hours: UK business hours with EST overlap as required
Compensation: Competitive UK market salary (GBP £)
About the Role
We are seeking an experienced Principal AI Engineer to lead the architecture, development, and deployment of enterprise-scale Agentic AI and LLM solutions for a leading financial services organization.
This is a hands-on technical leadership position focused on building production-ready autonomous AI agents, intelligent workflows, and enterprise-grade RAG applications.
The ideal candidate will have strong software engineering fundamentals, deep Python expertise, and proven experience delivering production-grade Agentic AI systems using frameworks such as LangGraph, Google ADK, CrewAI, or equivalent.
You will own AI systems end to end, including data pipelines, backend services, agent orchestration, retrieval systems, evaluation frameworks, and cloud infrastructure, ensuring scalability, reliability, security, and governance within a regulated financial services environment.
Key Responsibilities
- Design, architect, and develop enterprise-grade Agentic AI systems and autonomous AI workflows.
- Build multi-agent architectures using LangGraph, Google ADK, CrewAI, Claude Agent SDK, or similar frameworks.
- Develop production-ready RAG pipelines, including document chunking, embeddings, vector search, hybrid retrieval, reranking, and response validation.
- Build MCP servers and secure tool-calling interfaces to integrate LLMs with enterprise applications and data sources.
- Develop scalable backend services, APIs, microservices, and distributed systems using Python.
- Implement AI guardrails, validation mechanisms, policy checks, self-correction loops, and deterministic fallbacks.
- Optimize LLM applications for latency, token consumption, context management, and infrastructure costs.
- Build evaluation frameworks, automated testing, monitoring, and observability for production AI applications.
- Collaborate with business stakeholders, architects, data scientists, and engineering teams to translate business requirements into technical solutions.
- Provide technical leadership, architecture guidance, and mentorship to engineering teams.
Required Technical Skills
Core Software Engineering
- 8–14 years of software engineering experience, with 10+ years preferred.
- Strong hands-on Python development experience.
- Experience with Go, Rust, Java, or C/C++ is highly desirable.
- Strong understanding of data structures, algorithms, APIs, microservices, and system design.
- Experience building production-grade distributed systems and data pipelines.
Agentic AI & LLM
- 2+ years of hands-on LLM engineering experience.
- Proven experience designing and deploying Agentic AI systems into production.
- Strong experience with LangGraph, Google ADK, CrewAI, Claude Agent SDK, or equivalent.
- Experience implementing MCP (Model Context Protocol) servers and tool calling.
- Strong knowledge of LLM orchestration, prompt engineering, and multi-agent workflows.
RAG & Vector Databases
- Hands-on experience implementing RAG architectures from first principles.
- Strong understanding of document chunking, embeddings, semantic search, and hybrid retrieval.
- Experience with vector databases such as Pinecone, Milvus, Weaviate, FAISS, or equivalent.
- Knowledge of reranking, retrieval optimization, and response validation.
LLM Optimization & Evaluation
- Token optimization and context-window management.
- Model routing, caching, batching, streaming, and parallel tool execution.
- Experience developing LLM evaluation frameworks and regression testing.
- Familiarity with LangSmith, Langfuse, or similar observability platforms.
Cloud & DevOps
- Hands-on experience with AWS, including ECS/EKS, Lambda, S3, DynamoDB, Redshift, and Step Functions.
- Azure or GCP experience is also valuable.
- CI/CD pipelines, Docker, Kubernetes, and infrastructure as code.
- Terraform or CloudFormation experience.
Preferred Qualifications
- Previous experience in banking, financial services, or other regulated enterprise environments.
- Experience working with enterprise security, data governance, and compliance requirements.
- Familiarity with Amazon Bedrock, Google Vertex AI, or Azure AI.
- Experience delivering customer-facing enterprise AI implementations.
- Strong communication and stakeholder management skills.
Ideal Candidate Profile
We are looking for someone who:
- Has 10+ years of strong software engineering experience.
- Is highly hands-on with Python and enterprise system architecture.
- Has built at least 2 production-grade Agentic AI applications.
- Has strong expertise in LangGraph, RAG, MCP, LLMs, chunking, and embeddings.
- Understands enterprise-scale AI deployment, governance, and optimization.
- Can independently lead technical architecture and implementation.