Senior AI Data Engineer

Senior AI Data Engineer

Job Summary

We are seeking a Senior AI Data Engineer to design and operate the data foundation that our AI systems depend on. This role owns the movement, modeling, and quality of data from source systems through the warehouse and into the retrieval and feature layers that power LLM pipelines, agentic workflows, and analytical products.

The ideal candidate is a rigorous software engineer first and a data specialist second: someone who models a warehouse deliberately, writes production Python that other engineers can extend, and treats pipelines as versioned, tested, observable software rather than scripts.

This role partners closely with the AI/ML Data Scientist, who owns model behavior and retrieval strategy.

The boundary

you own the pipeline, the schema, and the guarantees; they own the algorithm, the prompt, and the evaluation.

Required Qualifications

5-10+ years in software engineering or data engineering, with substantial time in production data platform work.

Data warehousing

demonstrable command of Kimball dimensional modeling - not just familiarity with the vocabulary, but the judgment to choose a grain, resolve a many-to-many relationship, and know when to denormalize. Working knowledge of alternative approaches (Data Vault, One Big Table, Inman) and the tradeoffs against Kimball.

SQL

expert-level - window functions, CTEs, query plan reading, and performance tuning on a columnar warehouse.

Python

expert-level, production-grade - typing, packaging, dependency management, testing.

Design

SOLID and domain-driven design applied in real systems, with examples you can walk through.

Orchestration

Airflow, Prefect, Dagster, or equivalent, in production.

Cloud

expert-level on AWS, Azure, or GCP - storage, compute, IAM, networking, and cost management.

Platform

containerization, Kubernetes (EKS/AKS/GKE), and CI/CD.

Experience with lakehouse table formats (Iceberg, Delta Lake, Hudi) and their maintenance characteristics: compaction, snapshot expiry, schema and partition evolution.

Preferred Qualifications

Experience building the data layer beneath production RAG systems, including hybrid search infrastructure and index freshness guarantees.

Streaming systems

Kafka, Kinesis, Flink, or Spark Structured Streaming.

dbt or an equivalent transformation and testing framework.

Data quality tooling (Great Expectations, Soda, or similar) and catalog/lineage platforms.

Familiarity with the model-facing side of the stack - MLflow, Weights & Biases, feature stores - sufficient to collaborate credibly with data scientists.

Working knowledge of a second language

TypeScript, Java, Go, Scala, or Rust.

Experience with AI security, governance, and compliance frameworks.

Open-source contributions to data or AI infrastructure projects.