Sr. Data Engineer (GenAI & AWS) Remote/Hybrid with Travel

Sr. Data Engineer (GenAI & AWS) Remote/Hybrid with Travel

The key is time zone coverage and so we can expand our search across the West Coast - remote/hybrid but necessarily located in the western time zones. You must be comfortable to travel to the client site for workshops/meetings.

• Senior Data Engineer (GenAI / AWS)

Overall experience

12-20+ years Cloud Platform

AWS Primary Language

Python

Key Responsibilities

• Design and develop data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend.
• Build automated document-processing workflows for extracting, transforming, and processing structured and unstructured data.
• Implement document chunking, embeddings, vector indexing, and semantic search.
• Develop and maintain vector databases and knowledge bases to support Retrieval-Augmented Generation (RAG).
• Integrate knowledge bases with AI agents for contextual information retrieval.
• Build AI agents using Claude Agent SDK and deploy them on Amazon Bedrock AgentCore Runtime.
• Develop CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments.
• Optimize data pipelines for scalability, reliability, security, and performance.

Must-have vs. good-to-have skills

Must-have

Good-to-have

Python, SQL

TypeScript

AWS Step Functions, Lambda, S3

AWS Glue

Amazon Textract and Comprehend

Document AI / OCR

Vector databases and embeddings

OpenSearch, Pinecone, pgvector

Document chunking and indexing

Advanced retrieval and reranking

RAG and knowledge-base integration

LangChain / LlamaIndex

Amazon Bedrock

Claude Agent SDK

GitHub Actions CI/CD

Terraform / AWS CDK

AWS IAM and API integrations

AgentCore Runtime

Important

If this individual must personally build and deploy the Claude agents, move Claude Agent SDK and AgentCore Runtime into the must-have category.