
Azure Data Engineer - Lead
Artech · Stamford, CT · Hybrid
Strong hiring record · median 11 days open
- First seen
- 17 November 2026
- Reposted
- Not since September 2026
All reasons
In its favour
- Posted today
- Still listed on the employer's own board
- Publishes a salary range
Job Title
Azure Data Engineer - Lead
Location
Stamford, CT- Hybrid (Minimum 2 days a week at Onsite)
Duration
6 months (Extendable)
Pay Range
$50-55/Hour on W2/C2C (All Inclusive)
Request ID
116323-1
Job Description
Experience Required
8-10 Years
Primary skills
Databricks, Pyspark/Python
Role Description-
PySpark, Azure Data Factory, Azure Databricks, SQL, ADLS Gen2
Roles & Responsibilities
- Lead the Azure Data Engineering team and provide day-to-day technical direction, delivery oversight, and issue resolution.
- Design, develop, orchestrate, and maintain scalable ADF pipelines for batch ingestion, data integration, transformation, and downstream processing.
- Develop complex and reusable data-transformation frameworks using PySpark and Azure Databricks notebooks.
- Design metadata-driven ingestion, transformation, validation, reconciliation, logging, and restart/recovery components.
- Build and maintain solutions across ADLS Gen2, Azure Databricks, SQL databases, storage accounts, Azure Key Vault, and upstream/downstream enterprise systems.
- Migrate RDBMS and traditional ETL workloads to modern Data Lake, ELT, and cloud-native processing patterns.
- Optimize PySpark workloads for partitioning, caching, shuffle reduction, file sizing, scalability, and efficient processing of large datasets.
- Implement data-quality rules, control totals, exception handling, auditability, monitoring, alerting, and operational support mechanisms.
- Support data lake and SQL Server control/governance frameworks, including standards, templates, naming conventions, and directory structures.
- Analyze defects, incidents, change requests, and enhancement needs; perform root-cause analysis and implement sustainable fixes.
- Convert functional and business requirements into technical designs, estimates, implementation plans, and deployment activities.
- Perform code reviews and enforce engineering standards, security practices, source-control discipline, CI/CD, and documentation quality.
- Coordinate with onsite architects, business analysts, QA teams, application teams, production support teams, and customer stakeholders.
- Own offshore deliverables, dependencies, risks, status reporting, release readiness, and post-production stabilization.
- Mentor team members, build technical capability, and ensure high-quality, timely completion of deliverables.
Insurance / Reinsurance Domain Expectations
The candidate should have practical experience working with Insurance and/or Reinsurance data, preferably across several of the following areas:
- Policy, contract, coverage, and exposure data
- Premium, billing, claims, and loss transactions
- Broker, client, insured, and ceding company / cedant information
- Treaty and facultative reinsurance concepts
- Reinsurance premiums, recoveries, claims, and settlements
- Catastrophe, event, and loss-related datasets
- Financial, accounting, bordereaux, and reconciliation feeds
- Regulatory, management, and downstream analytical reporting
Generic Managerial Skills, If any
Key Words to search in Resume
Primary / Must-Have
PySpark, Python, Azure Data Factory, ADF, Azure Databricks, Databricks, ADLS Gen2, Azure Data Lake Storage, SQL, SQL Server, Azure Data Engineer, Data Engineering, ETL, ELT, Data Pipeline, Data Lake
Lead / Seniority
Technical Lead, Tech Lead, Data Engineering Lead, Azure Data Lead,
PySpark
How many years of hands-on experience do you have with PySpark/Apache Spark? Briefly describe your experience developing and optimizing large-scale data transformations.
Azure Data Factory
How many years of hands-on experience do you have with Azure Data Factory (ADF)? Have you designed complex, reusable or metadata-driven ingestion/ETL/ELT pipelines?
Azure Databricks
How many years of hands-on experience do you have with Azure Databricks? Have you designed and developed reusable Databricks notebooks/frameworks?
ADLS Gen2
Do you have hands-on experience with ADLS Gen2 / Azure Data Lake, including data organization, ingestion, processing and integration?
SQL
How many years of hands-on experience do you have with SQL/SQL Server, including complex queries, performance optimization and data reconciliation?
Performance Optimization
Have you optimized PySpark/Spark workloads involving partitioning, caching, shuffle reduction, file sizing or large-volume processing? Please provide a brief example.
Data Engineering Frameworks
Have you implemented metadata-driven ingestion, data-quality validation, reconciliation, logging, auditing, exception handling, and restart/recovery frameworks?
Azure Ecosystem
Do you have hands-on experience with Azure Key Vault, Storage Accounts, Azure DevOps/Git and CI/CD for data-engineering solutions?
Insurance/Reinsurance Domain
Do you have project experience in Insurance and/or Reinsurance? If yes, specify experience with Policy, Premium, Claims, Coverage, Exposure, Treaty, Facultative, Cedant, Bordereaux, Recoveries, Catastrophe/Loss, or Settlement data.
Technical Leadership
Have you worked as a Technical Lead/Data Engineering Lead, including code reviews, technical design, estimation, mentoring, issue resolution and delivery ownership?
Migration Experience
Have you worked on migrating RDBMS/traditional ETL workloads to Azure Data Lake, Databricks, or cloud-native ELT architectures?
Stakeholder Management
Have you coordinated with architects, business analysts, QA, application/support teams and customer stakeholders for end-to-end delivery?
Skills
- Azure
- Databricks
- PySpark
- Python
- Azure Data Factory
- Azure Databricks
- SQL
- Azure Key Vault
- ETL
- ELT
- SQL Server
- Azure Data Lake Storage
- Spark
- Azure DevOps
- Git
More jobs at Artech
All 420- Python AI EngineerArtech · Jersey City, NJ · USD 65–70/hrtoday
- Technical Lead – OTSM MigrationArtech · Remote · USD 55–58/hrtoday
- QA AnalystArtech · Ottawa, ON · USD 67/hrtoday
- ReactJS DeveloperArtech · Phoenix, AZ · USD 45–48/hrtoday
- Sr Prin Cybersecurity SpecialistArtech · Virtual, MN · USD 48–70/hrtoday
Similar roles
- Data EngineerToast · Remote, US · USD 106,000–170,000/yrtoday
- Senior Nursing Data AnalystMayo Clinic's · Phoenix, AZ, United States · USD 90,604–136,011/yrtoday
- Senior Battery Data EngineerRedwood Materials · San Francisco, California, United Statestoday
- Senior Data Engineering ManagerRedwood Materials · McCarran, NVtoday
- Data Engineer - SeniorAccenture Federal Services · Chantilly, VAtoday
- Head of Engineering, AI & Data PlatformOrion Placement · Washington D.C., United States · USD 210,000–260,000/yrtoday