Lead Data Engineer

Job Title

Lead Data Engineer

Work Location & Reporting Address

Duluth, GA 30097 (Onsite--client's office on Day 1)

Contract duration

6 Months

Must Have Skills

⦁Strong hands-on experience in Google BigQuery, Microsoft Fabric/Azure, SQL, and PySpark

⦁Solid understanding of

1.BigQuery architecture (partitioning, clustering, query optimization)

2.Microsoft Fabric components (Lakehouse, OneLake, Data Factory pipelines, Notebooks)

3.Azure data services (Data Factory, Synapse, Data Lake Storage)

4.PySpark-based transformation and data processing frameworks

5.ETL/ELT and data integration concepts across GCP and Azure

Nice to Have Skills

⦁Experience with workflow orchestration tools such as Cloud Composer (Apache Airflow) or Azure Data Factory

⦁Familiarity with

1.Power BI or other reporting and BI tools

2.CI/CD and version control (Git, Azure DevOps or GitHub)

3.Data modeling, data quality checks, and performance tuning

4.Data governance and security concepts (IAM, access control)

⦁Experience migrating or integrating data between GCP and Azure environments

⦁Retail or enterprise client domain experience (a plus for the client)

⦁Strong communication skills, with the ability to work with client tech leads and onsite teams

Detailed Job Description

⦁Data Engineer to support the development of a cloud analytics pipeline spanning Google BigQuery and Microsoft Fabric/Azure.

⦁The engineer will be responsible for preparing and validating data in BigQuery, moving and mirroring data into Microsoft Fabric, developing Fabric-based data pipelines and Lakehouse data models, and performing detailed source-to-target reconciliation.

⦁The ideal candidate should be strong in SQL and PySpark, understand dimensional data modeling, and be comfortable troubleshooting large-volume customer datasets across GCP and Microsoft Fabric.

⦁The candidate will need to be able to work with Microsoft and come up with solutions

Minimum Years of Experience

⦁6 years

Certifications Needed

⦁None