Data Engineer

Title

Data Engineer

Location

Gurgaon

Exp

3-6 Yrs

Job Description

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Integrate data from multiple internal and external data sources.
• Develop data transformation and processing workflows.
• Build and optimize SQL queries and data models.
• Ensure data quality, consistency, availability, and reliability.
• Collaborate with Data Scientists and ML Engineers to provide high-quality datasets.
• Develop and maintain data ingestion and processing workflows.
• Troubleshoot and optimize data pipelines and processing jobs.
• Contribute to data architecture and cloud-based data solutions.
• Implement appropriate data security, governance and quality practices.
• Document data pipelines, processes and technical solutions.

Required Technical Skills

• Strong proficiency in Python.
• Strong proficiency in SQL.
• Hands-on experience with ETL/ELT pipelines.
• Good understanding of relational and non-relational databases.
• Experience with data warehousing and data modeling.
• Knowledge of Spark/PySpark.
• Experience with workflow orchestration tools such as Airflow or equivalent.
• Exposure to AWS, Azure or GCP.
• Understanding of distributed data processing and scalable data architectures.

Preferred Skills

• Cloud data platforms.
• Databricks or equivalent platforms.
• Kafka or other data-streaming technologies.
• CI/CD and DevOps practices.
• Experience working with large-scale datasets.