Azure Data Engineer

Job Title

Azure Data Engineer

Location

Houston, TX 77079 (Onsite)

Duration

06 Months

Interview Process (Is face to face required?) Yes (2nd round)

Must Have Skills

• Azure Data Factory Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries.
• Azure Databricks, Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, Cluster Configuration, and Optimization.

SQL

Expert-level SQL (CTEs, Window Functions, Query Tuning, and working with large datasets).

Nice to Have Skills

• Agile.
• Infrastructure as Code (IaC) exposure is a plus.

Detailed Job Description

Experience

5–8 years in Data Engineering, with at least 3 years on Azure.

Azure Data Factory

Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries.

Azure Databricks

Databricks Notebooks, Jobs/Workflows, PySpark, Scala, Delta Lake, Cluster Configuration, and Optimization.

Programming

Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest).

SQL

Expert-level SQL (CTEs, Window Functions, Query Tuning, and working with large datasets).

Data Storage Formats

ADLS Gen2, Delta Lake, Parquet, Partitioning, Z-Ordering, OPTIMIZE, and VACUUM.

Ops

CI/CD, Git, Branching Strategies, and CI/CD for ADF/Databricks Environment Promotion.

• Infrastructure as Code (IaC) exposure is a plus.

Minimum years of experience

8-10 years

Top 3 responsibilities

Coding and Development

UI testing

Stakeholder management