Data Engineer

Key Responsibilities

• Develop and maintain data pipelines and ETL/ELT/analytics engineering workflows to support advanced search and retrieval capabilities.
• Collaborate with data teams to understand requirements and automate deployment and monitoring.
• Optimize data storage and troubleshoot issues to enhance performance.
• Work closely with data engineers and relevant parties to ensure sprints are planned out properly and completed timely.
• Perform BAU monitoring, investigations and resolutions.
• Assist with data governance and data management initiatives.
• Production and application support from a day-to-day basis.

Skillset Requirements

• Possess a degree in Computer Science/Information Technology or related fields.
• At least 5 years of hands-on experience in software or data engineering with proficiency in Python.
• Strong experience in unit and integration testing.
• Familiarity with DevOps practices and Agile methodologies.
• Experience with AWS and Kubernetes (K8s).
• Familiarity with data platforms such as Snowflake, Databricks, Apache Spark, Apache Hive, open table formats (Delta Lake, Apache Iceberg), and vector databases.
• Experience with orchestration tools such as Apache Airflow, Dagster, Prefect, and Temporal.
• Familiarity with GitHub workflows and Datadog.
• Good written and verbal communication skills.
• Agile, fast learner and able to adapt to changes.