Azure Data Engineer

APN Consulting, Inc. is a progressive IT staffing and services company offering innovative business solutions to improve client business outcomes. We focus on high impact technology solutions in ServiceNow, Fullstack, Cloud & Data, and AI / ML. Due to our globally expanding service offerings we are seeking top-talent to join our teams and grow with us.

Job Description

Job Title- Azure Data Engineer

Experience- 6-8 Year

Location- Bangalore (Hybrid)

• Build and maintain Lakehouses, Warehouses and SQL analytics endpoints; implement medallion layers with Delta tables, V-Order, OPTIMIZE/VACUUM and partitioning strategies.
• Develop ingestion and orchestration with Data Factory pipelines, Copy jobs, Dataflows Gen2 and Notebooks (PySpark, Spark SQL); configure OneLake shortcuts and Mirroring for source systems.
• Implement real-time scenarios with Eventstream, Eventhouse and KQL databases, Real-Time dashboards and Activator alerts.
• Build Power BI semantic models (Direct Lake, Import, composite), DAX measures, incremental refresh and RLS/OLS.
• Tune Spark environments (starter pools, custom pools, libraries, high-concurrency sessions) and SQL query performance; monitor capacity consumption with the Fabric Capacity Metrics app and Monitoring hub.
• Execute migrations from Synapse, ADF, SSIS and Databricks: inventory, conversion, reconciliation testing and cutover.
• Configure Fabric Git integration with Azure DevOps Repos or GitHub; enforce branching, pull-request policies and code-review standards for notebooks, pipelines and semantic models (TMDL/PBIP formats).
• Build CI/CD pipelines in Azure DevOps (YAML) or GitHub Actions using Fabric REST APIs, fabric-cicd, deployment pipelines and parameter files for environment promotion.
• Write and maintain Terraform (azurerm, azapi, Microsoft Fabric provider) or Bicep modules for capacities, workspaces, connections, networking, Key Vault and role assignments.
• Implement security gates in pipelines: SAST (CodeQL, SonarQube), secret scanning, IaC scanning (Checkov, Trivy), dependency scanning, unit tests for notebooks (pytest, Great Expectations / data-quality checks) and approval gates.
• Use OIDC / workload identity federation and managed identities; store secrets in Azure Key Vault; eliminate hard-coded credentials.

We are committed to fostering a diverse, inclusive, and equitable workplace where individuals from all backgrounds feel valued and empowered to contribute their unique perspectives. We strongly encourage applications from candidates of all genders, races, ethnicities, abilities, and experiences to join our team and help us build a culture of belonging.