
Azure Data Engineer
Diverse Lynx · Cary
Strong hiring record · median 12 days open
- First seen
- 16 November 2026
- Reposted
- Not since September 2026
All reasons
In its favour
- Posted today
- Still listed on the employer's own board
- Publishes a salary range
Role
Azure Data Engineer
Location
Cary, NC
Rate range
$60-$65/hr. on C2C
Key Responsibilities
- Collect, store, process and analyze large datasets to build and implement extract, transfer, load (ETL) processes.
- Lead migration of data pipelines from Synapse to Databricks.
- Architect and implement batch + real-time data pipelines using PySpark, Databricks, and Event Hubs.
- Establish data engineering best practices and mentor junior engineers.
- Drive CI/CD automation with AzDO and Git.
- Ensure data quality, governance, and compliance (Collibra).
- Collaborate with business/tech teams; leverage insurance domain knowledge where applicable.
- Develop reusable frameworks to reduce the development effort involved thereby ensuring cost savings for the projects.
- Develop quality code with thought through performance Eligible to workimizations in place right at the development stage.
- Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.
- Work with team spread across the globe in driving the delivery of projects and recommend development and performance improvements.
Essential Business Experience and Technical Skills
- Building and Implementing data ingestion and curation process developed using Big data tools such as Spark (Scala/python/Java), Hive, HDFS, Sqoop, Kafka, Kerberos, Impala etc. and CDP 7.x
- Ingesting huge volumes data from various platforms for Analytics needs and writing high-performance, reliable, and maintainable ETL code Strong
- Strong analytic skills related to working with unstructured datasets
- Strong experience in building/designing Data warehouses, data stores for analytics consumption On prem and Cloud (real time as well as batch use cases)
- Ability to interact with business analysts and functional analysts in getting the requirements and implementing the ETL solutions.
Required
- 7+ years in data engineering, 3+ in a lead/senior role.
- Strong expertise in Azure Databricks, Synapse, PySpark.
- Proven experience with real-time streaming (Event Hubs, Structured Streaming, Kafka).
- Deep knowledge of Azure ecosystem (Data Lake, Data Factory, Key Vault).
- Strong SQL, performance tuning, and CI/CD pipeline skills.
- Excellent leadership and communication skills
- Building and Implementing data ingestion and curation process developed using Big data tools such as Spark(Scala/python/Java), Data bricks, Delta Lake, Hive, HDFS, Sqoop, Kafka, Kerberos, Impala etc.
Hands on expertise in
Implementing analytical data stores in Azure platform using ADLS/ADF/Data Bricks and Cosmos DB and CDP 7x
- Ingesting huge volumes of data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code Strong
- SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
- Proficiency and extensive Experience with Spark & Scala, Python and performance tuning is a MUST
- Hive database management and Performance tuning is a MUST. (Partitioning / Bucketing)
- Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
- Strong analytic skills related to working with unstructured datasets
- Performance tuning and problem-solving skills is a must.
- Code versioning experience using Bitbucket/AzDo. Working knowledge of AzDo pipelines would be a big plus.
- Monitoring performance and advising any necessary infrastructure changes.
- Strong experience in building designing Data warehouses, data stores for analytics consumption. (real time as well as batch use cases)
- Eagerness to learn new technologies on the fly and ship to production
- Expert in technical program delivery across cross-functional / LOB teams
- Expert in driving delivery through collaboration in highly complex, matrixed environment
- Possesses strong leadership and negotiation skills
- Excellent communication skills, both written and verbal
- Ability to interact with senior leadership teams in IT and business Preferred
Preferred
- Collibra, insurance industry experience, Azure/Databricks certifications.
- Expertise in Python and experience writing Azure functions using Python/Node.js
- Experience using Event Hub for data integrations.
Hands on expertise in
Implementing analytical data stores in Azure platform using ADLS/Azure data factory /Data Bricks and Cosmos DB (mongo/graph API)
- Experience ingesting using Azure data factory, Complex ETL using Data Bricks.
- Eagerness to learn new technologies on the fly and ship to production.
Role Descriptions
Azure Data Engineer
Essential Skills
Azure Data Engineer
Skills
Microsoft Azure
Experience Required
8-10
Skills
More jobs at Diverse Lynx
All 406- Java Technical LeadDiverse Lynx · SAN DIEGO, CA · USD 15–50/hrtoday
- Network Lead /ArchitectDiverse Lynx · Irvine, CA · USD 55/hrtoday
- QA SREDiverse Lynx · Plano, TXtoday
- Vulnerability Management LeadDiverse Lynx · Remote · USD 55/hrtoday
- Senior ServiceNow AI DeveloperDiverse Lynx · Remote · USD 70/hrtoday
Similar roles
- Senior Data EngineerEmployer not named · Houston, XX, United States · EUR 45,000–65,000/yrtoday
- Data AnalystNeuronetics, Inc. / Greenbrook Mental Wellness Centers · Malvern, PAtoday
- Data EngineerDiverse Lynx · Las Vegas, NV · USD 55–60/hrtoday
- Data ArchitectDiverse Lynx · Hartford, CT · USD 60/hrtoday
- Tech Lead Manager, Claude Code Data ScientistAnthropic · New York City, NY · San Francisco, CA | New York City, NY · Seattle, WA · USD 460,000/yrtoday