Data Engineer 1

Pay Rate Range

$70.00 - $75.35/hr. on W2

About the Role

• We are looking for a Cloud & Data Engineer to help build, optimize, and operate our Voice of Customer (VoC) AI Platform.
• The platform ingests structured and unstructured data from multiple enterprise sources with varying degrees of signal-to-noise ratio and is primarily built on Microsoft Azure, with data processing and ETL/ELT workloads powered by Azure Databricks.
• This is a hands-on role spanning data engineering, cloud infrastructure, and platform reliability.

Key Responsibilities

Data Engineering

• Build and maintain ETL/ELT pipelines for structured and unstructured data sources.
• Develop data-processing workloads using Azure Databricks, Python/PySpark, and SQL.
• Optimize Databricks jobs, compute, and data layouts for performance and cost.
• Work with Delta Lake and Unity Catalog for data storage, governance, and access control.
• Implement data quality checks, monitoring, error handling, and recovery mechanisms.

Cloud & Platform Engineering

• Build, operate, and optimize Azure services including App Service, Functions, Blob/ADLS storage, Azure SQL DB for Postgres, Key Vault, Container Registry and Azure Monitor.
• Experience with cloud AI services including vector search and embedding pipelines.
• Improve platform performance, scalability, reliability, security, and cost efficiency.
• Implement secure identity, and access management using Azure managed identities, Key Vault, secrets managed via scopes.
• Support development, staging, and production environments.

Automation & Operations

• Improve platform observability through logging, monitoring, dashboards, and alerting.
• Automate infrastructure and deployments using Infrastructure as Code (IaC) and CI/CD.
• Troubleshoot issues across cloud services, storage, databases, networking, and data pipelines.
• Support production incidents and root-cause analysis.
• Collaborate with application, AI/ML, and data engineering teams.

Skills

Required Qualifications

• 3–5 years of experience in data engineering, cloud engineering, or a related role.
• Hands-on experience with Microsoft Azure, Databricks, APIs, and object storage.
• Strong skills in Python/PySpark, SQL.
• Experience building and operating production data ETL/ELT pipelines.
• Experience with relational and semi-structured/unstructured data, database technologies.
• Experience with Spark, Delta Lake, and lakehouse architectures.
• Working knowledge of Databricks Unity Catalog and data-access controls.
• Understanding of cloud security concepts such as RBAC, managed identities, service principals, and Key Vault.
• Experience with Git, CI/CD, monitoring, troubleshooting, and performance optimization.

Preferred Qualifications

• Experience with Terraform or ARM templates.
• Experience with Azure DevOps or GitHub Actions.
• Familiarity with Azure networking, including VNets and private endpoints.
• Experience with Azure Data Factory, Databricks Workflows, or Airflow.
• Familiarity with data platforms supporting AI/ML, NLP, analytics, or GenAI applications.

Education

• No formal requirements if skills are met.