AWS Data Engineer

AWS DATA ENGINEER – CONTRACT

Job Title

AWS Data Engineer

Location

[JHB/Hybrid] ***MUST BE LOCAL IN SOUTH AFRICA

Job Type

[Full-Time /Contract]

Job Description

We are seeking a skilled AWS Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with specific expertise in AWS technologies. You will be responsible for designing, developing, and maintaining our data infrastructure, ensuring efficient data processing and storage solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using AWS services.
• Write efficient and maintainable Python and PySpark code for data processing.
• Implement and manage ETL processes using AWS Glue.
• Develop and optimize SQL queries for data extraction and transformation.
• Utilize AWS Athena for querying and analyzing large datasets.
• Work with AWS Redshift for data warehousing solutions.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Ensure data quality and integrity across various data sources.
• Monitor and troubleshoot data pipeline issues to ensure smooth operations.
• Stay updated with the latest industry trends and best practices in data engineering and AWS technologies.

Required Qualifications

• Must be strong in MDM knowledge: Experience with building or integrating MDM systems — e.g., designing golden records, matching/merging, data mastering, data stewardship workflows
• Python: Proficient in writing Python code for data processing and automation.
• Spark: Strong understanding of Apache Spark and experience with PySpark.
• AWS Data Stack: Glue, S3, Lambda, Redshift, Athena, Step Functions
• SQL: Advanced skills in SQL, with the ability to write complex queries.
• AWS Athena: Experience with AWS Athena for querying large datasets.
• Redshift: Familiarity with AWS Redshift, including data warehousing concepts and usage.
• Data Warehousing: Solid understanding of data warehousing principles and best practices.

Preferred Qualifications

• Experience with other AWS services such as S3, Lambda, and CloudFormation.
• Knowledge of data modeling and schema design.
• Familiarity with data governance and security best practices.
• Experience with version control systems like Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.

Education and Experience

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 4+ years of experience in data engineering or a related role.