About the role
Job Description: Azure AI Solution Architect
Location: Newark, NJ
Experience: 8–14 Years
Role: Azure AI Solution Architect
1. Role Summary
We are looking for an experienced Azure AI Solution Architect to design and deliver Generative AI and Agentic AI solutions, taking AI initiatives from proof of concept to production.
The ideal candidate will have strong hands-on experience building LLM-powered applications and architecting secure, scalable, cost-efficient, and enterprise-ready AI solutions on Microsoft Azure.
The role involves designing Retrieval-Augmented Generation (RAG) applications, enterprise copilots, and multi-agent systems capable of reasoning, using tools, and executing complex business workflows.
The individual will be responsible for defining the technical architecture, identifying AI use cases, driving implementation strategies, and collaborating with business stakeholders, data scientists, and engineering teams to deliver production-grade AI solutions.
2. Key Responsibilities
AI Architecture & Solution Design
- Design high-level and low-level architectures for Generative AI and Agentic AI solutions on Azure.
- Align architectures with the Azure Well-Architected Framework and Microsoft's AI reference architectures.
- Translate complex business requirements into scalable and practical AI solutions.
Agentic AI & Generative AI Development
- Design and implement single-agent and multi-agent systems capable of planning, tool/API calling, and executing multi-step business workflows.
- Architect RAG pipelines, enterprise copilots, and document intelligence solutions.
- Define chunking, indexing, vector search, prompt engineering, and model selection strategies.
- Implement orchestration patterns, memory management, human-in-the-loop checkpoints, and failure-handling mechanisms.
Use-Case Discovery & Business Consulting
- Collaborate with business stakeholders to identify and prioritize high-value AI use cases.
- Evaluate technical feasibility, data readiness, and business impact.
- Develop AI implementation roadmaps with measurable success criteria.
- Advise stakeholders on AI adoption strategies, technical trade-offs, and associated risks.
Responsible AI, Security & Governance
- Implement AI security controls, content safety measures, and prompt-injection defenses.
- Ensure data privacy, access control, and auditability across AI applications.
- Establish Azure governance and policy controls to meet enterprise and regulatory requirements.
Evaluation, Monitoring & LLMOps
- Define evaluation frameworks to measure AI quality, accuracy, groundedness, latency, and cost.
- Implement monitoring, tracing, and observability for production AI applications.
- Drive continuous improvement and optimization of AI solutions.
Production Deployment & Integration
- Lead AI solutions from proof of concept and pilot stages through enterprise-scale production deployment.
- Integrate AI applications with enterprise systems, data platforms, and existing applications.
- Collaborate with DevOps and engineering teams to establish automated CI/CD pipelines.
Performance & Cost Optimization
- Optimize token consumption, model routing, caching, and resource utilization.
- Drive capacity planning and cost optimization across Azure AI workloads.
- Evaluate and manage deployment options, including Provisioned Throughput Units (PTU) and pay-as-you-go models.
Technical Leadership & Collaboration
- Provide technical direction and guidance to developers, data scientists, ML engineers, and DevOps teams.
- Mentor engineering teams and ensure implementation aligns with the defined architecture.
- Support solution design discussions, technical estimations, and pre-sales activities.
3. Required Skills & Experience
Professional Experience
- 8–14 years of experience in software engineering, data engineering, or cloud architecture.
- Minimum 2 years of hands-on experience designing and delivering Generative AI solutions.
- Experience taking at least one Generative AI solution from concept or pilot to production.
Azure AI Platform
- Strong hands-on expertise in:
- Azure OpenAI
- Microsoft Foundry (formerly Azure AI Foundry)
- Azure AI Search
- Azure AI Document Intelligence
- Azure AI Content Safety
Agentic AI & Frameworks
- Practical experience building AI agents using frameworks such as:
- Microsoft Agent Framework
- Semantic Kernel
- AutoGen
- Foundry Agent Service
- LangGraph
- Strong understanding of tool calling, multi-agent orchestration, and agentic workflows.
- Familiarity with integration standards such as Model Context Protocol (MCP).
Generative AI & LLM Engineering
- Strong understanding of RAG architecture and implementation.
- Experience with embeddings, vector databases, and semantic search.
- Expertise in prompt engineering, context management, and LLM selection.
- Experience working with commercial and open-weight Large Language Models (LLMs).
Evaluation & Observability
- Experience evaluating LLM output quality, accuracy, and safety.
- Knowledge of monitoring, tracing, and observability for production AI applications.
Azure Cloud Infrastructure
- Strong understanding of Azure PaaS, serverless, and container services, including:
- Azure Functions
- Azure App Service
- Azure Kubernetes Service (AKS)
- Azure Container Apps
- Knowledge of virtual networking, private endpoints, Microsoft Entra ID, and Azure Key Vault.
Programming & Automation
- Proficiency in Python.
- Experience with C# or TypeScript is an added advantage.
- Infrastructure as Code (IaC) experience using Terraform or Bicep.
- Experience implementing CI/CD pipelines using Azure DevOps or GitHub Actions.
AI Security & Responsible AI
- Understanding of AI-specific security risks, including prompt injection, data leakage, and hallucinations.
- Knowledge of responsible AI practices and implementation of relevant security controls.
4. Preferred Qualifications
- Experience with Microsoft data and analytics technologies, including Microsoft Fabric, Azure Databricks, Cosmos DB, and Azure SQL.
- Familiarity with Microsoft 365 Copilot extensibility and Copilot Studio.
- Experience with model fine-tuning, model adaptation, or deploying open-weight models on Azure.
- Knowledge of AI regulatory frameworks such as the EU AI Act and NIST AI Risk Management Framework.
- Previous consulting or client-facing solution delivery experience.
- Experience working with regulated industries such as financial services, healthcare, life sciences, or publishing.
5. Education & Certifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Data Science, or a related discipline.
- Microsoft Certified: Azure Solutions Architect Expert (AZ-305) preferred.
- Microsoft Certified: Azure AI Engineer Associate or its current equivalent preferred.
6. Leadership & Communication Skills
- Demonstrated ability to lead technical teams and mentor engineers.
- Strong stakeholder management and communication skills.
- Ability to explain complex AI architectures, technical risks, and trade-offs to business and non-technical stakeholders.
- Strong problem-solving, analytical, and solution-design capabilities.