Sr. Data Engineer

Sr. Data Engineer

Remote

Our customer is a growing health benefits company. The company is hiring a Sr Data Engineer to build the data foundation behind its cost prediction and pricing models. This is the beginning of an overarching technology modernization for a rapidly growing organization. The ideal ideal candidate has experience designing and building data pipelines and models in AWS for claims data.

Responsibilities

• Design, build, and maintain fully serverless data pipelines on AWS (Lambda, Glue, Athena, Step Functions, S3) supporting analytics, reporting, and AI use cases
• Build versioned data models and feature stores with clear semantics, reproducible backfills, and safeguards against leakage or inconsistent definitions
• Develop batch and event-driven ingestion pipelines (Kafka, Kinesis, or similar) for near-real-time scoring and decisioning
• Implement automated data quality controls, monitoring, and lineage, including schema change, reconciliation, and data maturity checks
• Partner with AI and Analytics teams on MLOps: orchestrating training and scoring workflows and capturing model and data versions to close the learning loop
• Optimize cost and performance through Athena query tuning, columnar formats (Parquet, Iceberg), and efficient S3 partitioning
• Enforce engineering best practices, including code review, CI/CD for data pipelines, and Infrastructure as Code with AWS CDK

Requirements

• 5+ years of data engineering with ownership of production pipelines and analytics platforms; hands-on ML/AI support strongly preferred
• Advanced Python and SQL, plus production experience with AWS serverless data services; PySpark strongly preferred
• Infrastructure as Code with AWS CDK (TypeScript)
• Experience with streaming or event-driven pipelines (Kafka preferred) and open table formats such as Iceberg
• Data governance and security for sensitive, regulated data (PHI, financial); healthcare claims or insurance experience strongly preferred