AWS Data Engineer

Job Role

AWS Data Engineer

Salary

$130k to $145k per annum

Role Summary –

Design, develop and support AWS-based data pipelines for enterprise data onboarding, transformation, and consumption. The engineer will work hands-on with Python/PySpark and AWS-native data services to create reliable, scalable, and analytics/AI-ready datasets.

Key Responsibilities -

Data Pipeline Development

• Develop ingestion and transformation pipelines using AWS Glue, EMR, S3, Python and PySpark.
• Ingest data from legacy and enterprise source systems into AWS.
• Develop transformations required to create standardized, consumption-ready datasets.
• Build and maintain ETL/ELT workflows supporting enterprise data domains.

AWS Data Engineering

• Work with S3 for data storage and processing.
• Develop workloads using Glue and EMR.
• Develop and optimize analytical datasets and workloads involving Redshift and Athena.
• Support Iceberg-based data structures on S3 where required.
• Work with other AWS data services such as DynamoDB when appropriate to the solution.

Data Quality & Reliability

• Implement data validation, reconciliation, and quality checks within pipelines.
• Identify and resolve data-processing failures and data-quality issues.
• Support performance optimization of PySpark/SQL workloads and data pipelines.
• Participate in troubleshooting and production support.

Development & Delivery

• Develop clean, maintainable, and reusable Python/PySpark code.
• Perform unit and integration testing.
• Participate in peer code reviews.
• Document data flows, mappings, transformations, and technical designs.
• Work with the Data Engineering Lead and client stakeholders to understand source data and transformation requirements.