Data Engineer

Title

Data Engineer

Location

Chennai

Exp

6+ Yrs

Job Description

Key Responsibilities-

Design, develop, and maintain scalable data pipelines for processing large volumes of structured and unstructured data.

Develop and optimize ETL/ELT workflows for data extraction, transformation, and loading.

Integrate data from multiple sources, including databases, APIs, and external systems.

Develop data models and maintain data quality, consistency, and integrity.

Optimize SQL queries, database performance, and data processing workflows.

Implement data validation, error handling, and monitoring to ensure reliable data pipelines.

Troubleshoot data pipeline failures and resolve performance-related issues.

Collaborate with data analysts, data scientists, and other engineering teams to support business requirements.

Maintain technical documentation for data pipelines, data models, and data integration processes.

Follow data security, governance, and best practices for data management.

Required Skills and Qualifications

6+ years of experience in Data Engineering or a related field.

Strong programming skills in Python or another relevant programming language.

Strong proficiency in SQL and relational databases.

Hands-on experience with ETL/ELT processes and data pipeline development.

Experience with data modeling, data warehousing, and data integration.

Knowledge of data quality, validation, and performance optimization techniques.

Experience working with large datasets and distributed data processing.

Strong analytical, problem-solving, and troubleshooting skills.

Good communication and collaboration skills.

Preferred Skills-

Experience with cloud platforms such as AWS, Azure, or GCP.

Knowledge of Apache Spark, PySpark, or similar big data technologies.

Experience with orchestration tools such as Apache Airflow.

Familiarity with cloud data warehouses and modern data platforms.

Understanding of CI/CD, Git, and automated deployment practices.