About the role
About the Role
Dailoqa is looking for an experienced Senior QA Engineer
– Agentic AI Testing & Evaluation to own the quality, testing, and
evaluation strategy for our Broccoli Platform. This role will be
responsible for establishing robust testing frameworks for AI agents,
multi-agent workflows, orchestration layers, tools, and enterprise-grade AI
applications.
The ideal candidate will have deep expertise in test
automation, quality engineering, AI/ML validation, security testing, and
performance engineering, with a strong understanding of Large Language
Models (LLMs), Agentic AI systems, and modern DevOps practices.
You will work closely with Capability Leads, AI
Architects, Product Teams, and Platform Engineers to ensure that Broccoli
delivers reliable, secure, scalable, and high-performing AI solutions.
Requirements
Required Skills & Qualifications
Technical Expertise
- 7–10
years of experience in Quality Assurance, Test Automation, or Quality
Engineering.
- Strong
experience with automation frameworks using Python, Java, or
JavaScript/TypeScript.
- Hands-on
experience with:
- Selenium,
Playwright, Cypress, or Robot Framework
- API
testing tools such as Postman, Rest Assured, or Karate
- CI/CD
platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab
- Experience
with cloud platforms such as Azure, AWS, or Google Cloud.
Agentic AI & LLM Testing
- Understanding
of:
- Large
Language Models (LLMs)
- Retrieval-Augmented
Generation (RAG)
- AI
Agents and Agent Orchestration Frameworks
- Prompt
Engineering and Evaluation
- Experience
with AI evaluation platforms and frameworks is preferred.
- Knowledge
of hallucination detection, safety validation, guardrails, and model
benchmarking.
Performance & Security Testing
- Experience
with performance testing tools such as JMeter, k6, or Locust.
- Knowledge
of application security testing and threat modeling.
- Familiarity
with AI security testing, adversarial testing, and governance controls.
Soft Skills
- Strong
analytical and problem-solving skills.
- Excellent
stakeholder management and communication abilities.
- Experience
working in cross-functional Agile teams.
- Ability
to lead quality initiatives in fast-paced, innovation-driven environments.
Preferred Qualifications
- Experience
testing AI-powered enterprise applications.
- Exposure
to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic
Kernel, or similar agent frameworks.
- Certifications
such as:
- ISTQB
Advanced Test Analyst/Test Manager
- Certified
Agile Tester
- Azure
AI Engineer Associate
- Azure
DevOps Engineer Expert
- Experience
working with BFSI, enterprise transformation, or regulated industry
environments.