Data Engineer, Healthcare Clinical Data, Databricks & Snowflake (26-02054) -

Data Engineer, Healthcare Clinical Data, Databricks & Snowflake

Remote

Mandatory areas – Senior Data Engineer (Healthcare Clinical Data): 8+ years

Must Have skills

Senior Data Engineer (Healthcare Clinical Data) with expertise in:

• Databricks
• Snowflake
• PySpark/Spark
• Healthcare Clinical Data
• Healthcare Data Standards (HL7, FHIR, ICD-10, CPT, etc.)

This is a domain-heavy role, where healthcare knowledge is just as important as technical skills.

(Onshore) Role Summary

• We are looking for an experienced Data Engineer with strong healthcare clinical data knowledge and hands-on expertise in Databricks and Snowflake. This person will work closely with client stakeholders, the Data Architect, data science teams and offshore engineers to build and maintain scalable clinical data pipelines and datasets that support predictive analytics. The role will also coordinate day-to-day engineering activities with the offshore team and help ensure smooth delivery. Key Responsibilities
• Work directly with client stakeholders to understand clinical data requirements, clarify source-to-target mappings and translate requirements into scalable data engineering solutions.
• Develop and maintain data pipelines using Databricks, Snowflake, Spark/PySpark and SQL.
• Build and manage data layers within a Lakehouse/Medallion architecture, including Bronze, Silver and Gold datasets using Delta Lake and related data platforms.
• Design and implement data ingestion, transformation and integration between Databricks, Snowflake and upstream/downstream healthcare systems.
• Ingest, transform and standardize clinical data from EHR/EMR and other healthcare sources, including encounters, diagnoses, procedures, labs and medications.
• Work with healthcare data standards such as HL7, FHIR, CCD, ICD-10, CPT, LOINC, SNOMED and RxNorm to prepare standardized and analytics-ready datasets.
• Build high-quality data foundations for predictive analytics use cases such as readmission risk, risk stratification and care gaps.
• Partner with Data Architects and data science teams to prepare curated datasets and features required for machine learning and predictive models.
• Implement data quality checks, reconciliation, exception handling, monitoring and logging to ensure reliable data pipelines.
• Apply data governance, lineage, access controls and HIPAA/PHI security requirements across Databricks, Snowflake and the broader data ecosystem.
• Optimize Spark, SQL, Databricks and Snowflake workloads for performance, scalability and cost.
• Coordinate with offshore engineers on requirements, development, testing and issue resolution, while providing regular delivery updates to client and project stakeholders.
• Troubleshoot pipeline and data issues, identify root causes and support timely resolution across development and production environments.

Required Skills & Experience

• 6+ years of experience in data engineering, with strong exposure to healthcare and clinical data.
• Good understanding of healthcare clinical data and standards including HL7, FHIR, CCD, ICD-10, CPT, LOINC, SNOMED and RxNorm.
• Strong hands-on experience with Databricks, Spark/PySpark, SQL, Delta Lake, Unity Catalog and Databricks Workflows.
• Strong working experience with Snowflake, including data loading, transformation, data modeling, performance optimization and security/access controls.
• Experience building ETL/ELT pipelines and implementing Lakehouse and Medallion architectures.
• Experience integrating and moving data across Databricks, Snowflake and cloud-based data platforms.
• Experience preparing data for predictive analytics, advanced analytics or machine learning use cases.
• Good working knowledge of Azure or AWS cloud environments.
• Understanding of healthcare data security, HIPAA and PHI handling requirements.
• Strong SQL skills with experience in data transformation, performance optimization and data quality frameworks.
• Good communication and stakeholder management skills, with experience working in an onshore-offshore delivery model.