Azure Data Engineer
Diverse Lynx · Cary, NC · Onsite
Strong hiring record · median 4 days open
- First seen
- 14 November 2026
- Reposted
- Not since September 2026
- Last checked
- Today · still on the employer's board
All reasons
In its favour
- Posted 2 days ago
- Still listed on the employer's own board
- Publishes a salary range
Title
Azure Data Engineer
Location
Cary, NC -Face to face - Onsite
Duration
6-12+ Months contract
Rate $60-65/hr on c2c with VMS charges
Note- Face to Face Interview
Job Description
-
- Collect, store, process and analyze large datasets to build and implement extract, transfer, load (ETL) processes.
- Lead migration of data pipelines from Synapse to Databricks.
- Architect and implement batch + real-time data pipelines using PySpark, Databricks, and Event Hubs.
- Establish data engineering best practices and mentor junior engineers.
- Drive CI/CD automation with AzDO and Git.
- Ensure data quality, governance, and compliance (Collibra).
- Collaborate with business/tech teams; leverage insurance domain knowledge where applicable.
- Develop reusable frameworks to reduce the development effort involved thereby ensuring cost savings for the projects.
- Develop quality code with thought through performance Eligible to workimizations in place right at the development stage.
- Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.
- Work with team spread across the globe in driving the delivery of projects and recommend development and performance improvements.
Essential Business Experience and Technical Skills
- Building and Implementing data ingestion and curation process developed using Big data tools such as Spark (Scala/python/Java), Hive, HDFS, Sqoop, Kafka, Kerberos, Impala etc. and CDP 7.x
- Ingesting huge volumes data from various platforms for Analytics needs and writing high-performance, reliable, and maintainable ETL code Strong
- Strong analytic skills related to working with unstructured datasets
- Strong experience in building/designing Data warehouses, data stores for analytics consumption On prem and Cloud (real time as well as batch use cases)
- Ability to interact with business analysts and functional analysts in getting the requirements and implementing the ETL solutions.
Required
- 7+ years in data engineering, 3+ in a lead/senior role.
- Strong expertise in Azure Databricks, Synapse, PySpark.
- Proven experience with real-time streaming (Event Hubs, Structured Streaming, Kafka).
- Deep knowledge of Azure ecosystem (Data Lake, Data Factory, Key Vault).
- Strong SQL, performance tuning, and CI/CD pipeline skills.
- Excellent leadership and communication skills
- Building and Implementing data ingestion and curation process developed using Big data tools such as Spark(Scala/python/Java), Data bricks, Delta Lake, Hive, HDFS, Sqoop, Kafka, Kerberos, Impala etc.
Hands on expertise in
Implementing analytical data stores in Azure platform using ADLS/ADF/Data Bricks and Cosmos DB and CDP 7x
- Ingesting huge volumes of data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code Strong
- SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
- Proficiency and extensive Experience with Spark & Scala, Python and performance tuning is a MUST
- Hive database management and Performance tuning is a MUST. (Partitioning / Bucketing)
- Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
- Strong analytic skills related to working with unstructured datasets
- Performance tuning and problem-solving skills is a must.
- Code versioning experience using Bitbucket/AzDo. Working knowledge of AzDo pipelines would be a big plus.
- Monitoring performance and advising any necessary infrastructure changes.
- Strong experience in building designing Data warehouses, data stores for analytics consumption. (real time as well as batch use cases)
- Eagerness to learn new technologies on the fly and ship to production
- Expert in technical program delivery across cross-functional / LOB teams
- Expert in driving delivery through collaboration in highly complex, matrixed environment
- Possesses strong leadership and negotiation skills
- Excellent communication skills, both written and verbal
- Ability to interact with senior leadership teams in IT and business Preferred
Preferred
- Collibra, insurance industry experience, Azure/Databricks certifications.
- Expertise in Python and experience writing Azure functions using Python/Node.js
- Experience using Event Hub for data integrations.
Hands on expertise in
Implementing analytical data stores in Azure platform using ADLS/Azure data factory /Data Bricks and Cosmos DB (mongo/graph API)
- Experience ingesting using Azure data factory, Complex ETL using Data Bricks.
- Eagerness to learn new technologies on the fly and ship to production.
Skills
More jobs at Diverse Lynx
All 126- FDE (Forward Deployed Engineer / Solution Architect)Diverse Lynx · Orlando, FL (Hybrid 3 days onsite).2d ago
- .NET Full Stack DeveloperDiverse Lynx · Coraopolis, Pennsylvania · USD 13–50/hr2d ago
- Adobe Experience Manager (AEM) Lead DeveloperDiverse Lynx · Remote, CA · USD 60–65/hr2d ago
- Agentic AI EngineerDiverse Lynx · Chandler, AZ(3 days onsite) · USD 60–65/hr2d ago
- Senior Full Stack Developer(Python/Angular)Diverse Lynx · NYC · USD 20–100/hr2d ago
Similar roles
- Senior Database Administrator - Production SupportJobgether · US · USD 116,450–157,550/yrtoday
- Power BI Developer / Data Visualization AnalystJobgether · UStoday
- Data Engineering Solution LeadJobgether · UStoday
- Staff Data Platform Engineer, Knowledge PlatformAirwallex · US - Seattletoday
- Data Platform Principal Architect, Seattle/San FranciscoAirwallex · US - Seattletoday
- Senior Battery Data EngineerRedwood Materials · San Francisco, California, United Statestoday