Senior LLMOps Engineer

Overview

We are looking for an experienced Senior LLMOps Engineer to design, implement, and maintain production-grade large language model (LLM) pipelines, deployment architectures, and monitoring systems across enterprise environments. The Senior LLMOps Engineer will play a critical role in operationalizing generative AI capabilities, ensuring that LLM-based applications are scalable, secure, reliable, and compliant with emerging AI risk and governance frameworks. This role spans model deployment, orchestration, evaluation, optimization, and Retrieval-Augmented Generation (RAG) pipelines.

Responsibilities

• Architect, build, and maintain scalable LLM and Retrieval-Augmented Generation (RAG) pipelines, including model hosting, inference optimization, retrieval layers, and context management frameworks.
• Lead the design and implementation of secure GenAI infrastructure across cloud environments, ensuring reliability, performance, scalability, and cost efficiency.
• Build and manage automated evaluation systems that assess LLM output quality, safety, latency, and adherence to AI governance requirements.
• Develop CI/CD workflows tailored for LLM- and GenAI-based applications, including dataset versioning, model lineage, and automated testing of prompt and model behaviors.
• Collaborate with AI Product Engineers and Data Scientists to productionize LLM-based prototypes into enterprise-grade, maintainable systems.
• Integrate vector databases, model gateways, content filters, and guardrail frameworks into end-to-end LLM and RAG solutions.
• Implement observability and monitoring solutions that track performance metrics, hallucination rates, cost profiles, and user interaction patterns.
• Lead troubleshooting and root-cause analysis for issues related to LLM deployment, inference performance, retrieval pipelines, or overall pipeline reliability.
• Develop and maintain Python-based services, integrations, and automation supporting LLM and RAG pipelines.
• Stay current with emerging LLM architectures, inference optimizations, fine-tuning techniques, and relevant MLSecOps patterns.
• Ensure compliance with data privacy, ethical AI, security, and AI governance frameworks throughout pipeline design and operations.
• Mentor junior engineers and contribute to AI engineering best practices, tooling, and reusable infrastructure patterns.
• You will contribute to the growth of our AI & Data Exploitation Practice!

Qualifications

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor’s degree and 10+ years of total experience.
• 5+ years of experience in software engineering, data engineering, MLOps, or cloud engineering, with 2+ years focused specifically on LLM or GenAI operations.
• Hands-on experience designing, building, and maintaining Retrieval-Augmented Generation (RAG) pipelines.
• Fluency in Python.
• Strong experience deploying models using frameworks such as Hugging Face Transformers, vLLM, TensorRT-LLM, or similar.
• Experience with operational tooling and frameworks such as FastAPI, PyTorch, LangChain, LlamaIndex, and vector databases such as FAISS, Milvus, Pinecone, or similar.
• Advanced knowledge of cloud platforms such as AWS, Azure, or GCP, including model hosting, distributed compute, and secure networking patterns.
• Hands-on experience building CI/CD pipelines, automated testing frameworks, and environment provisioning for AI/ML workloads.
• Experience with Docker, Kubernetes, and infrastructure-as-code tools such as Terraform or CloudFormation.
• Familiarity with MLSecOps, AI governance, model hardening, prompt injection defenses, and content safety monitoring.
• Strong understanding of logging, observability, and performance profiling for high-throughput LLM inference systems.
• Excellent written and verbal communication skills, with the ability to explain trade-offs and architectural decisions to technical and non-technical stakeholders.
• Demonstrated ability to balance long-term platform thinking with hands-on operations and rapid problem solving.
• Experience working in Agile development environments and using modern project management tools.