AI Engineer (Agentic AI / AWS Bedrock)

Title

AI Engineer (Agentic AI / AWS Bedrock)

Location

Los Angeles, CA - locals only

Work Arrangement

Hybrid - 2 days onsite in LA, CA

Duration

6+ month contract-to-hire

Experience Range

6-10 years

Our client is seeking an AI Engineer with 6-10 years of experience to design, develop, and deploy intelligent AI agents using Claude Agent SDK and Amazon Bedrock, building production-ready agentic systems with comprehensive observability and evaluation frameworks.

Responsibilities & Qualifications

• Design, develop, and deploy AI agents using Claude Agent SDK and Python; build intelligent, autonomous workflows capable of executing complex tasks and interacting with external tools, APIs, and knowledge bases.
• Develop agent orchestration workflows on Amazon Bedrock AgentCore Runtime with error handling, retry mechanisms, and guardrails; integrate agents with enterprise applications, data sources, and foundation models.
• Implement comprehensive observability and monitoring using Langfuse for LLM and agent tracing, capturing prompts, responses, tool calls, execution metrics, and production performance analytics.
• Design and implement continuous evaluation frameworks for production AI agent traffic using LLM-as-a-Judge, RAGAS, DeepEval, and custom metrics to detect regressions and ensure quality benchmarks.
• Build scalable RAG pipelines and vector database solutions (OpenSearch, Pinecone, FAISS, pgvector) for document ingestion, chunking, embedding generation, and knowledge base retrieval optimization.
• Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing and data transformation at scale.
• Build and maintain CI/CD pipelines using GitHub Actions, automating agent code builds, testing, deployment, and integrated evaluation workflows for Amazon Bedrock AgentCore Runtime.
• Implement AWS security best practices including IAM policies, access controls, authentication, CloudWatch monitoring, and cost optimization for production agent systems.

Requirements

• 6-10 years of professional software engineering or AI/ML experience with demonstrated expertise in Python and LLM-based systems.
• Advanced proficiency with Claude Agent SDK, agent orchestration, tool calling, and agentic AI frameworks (LangChain, LangGraph); proven experience deploying AI agents in production environments.
• Deep expertise with Amazon Bedrock, Bedrock AgentCore Runtime, and AWS services (Step Functions, Textract, Comprehend, OpenSearch, IAM, CloudWatch).
• Hands-on experience with LLM observability and evaluation tools: Langfuse, LLM-as-a-Judge frameworks, RAGAS, DeepEval, and custom evaluation metrics.
• Strong proficiency with vector databases and RAG implementation (Pinecone, OpenSearch, FAISS, pgvector); experience with embeddings, semantic search, and retrieval optimization.
• Expertise in CI/CD and DevOps: GitHub Actions, Git, Docker, Terraform, Infrastructure as Code, and automated deployment pipelines.
• Proficiency in prompt engineering, Model Context Protocol (MCP), AI governance principles, and AWS architecture best practices for secure, scalable systems.
• W2 employment only; must be available for hybrid work arrangement.