Senior Data Engineer (Pyspark, Databricks)

Senior Data Engineer (PySpark)

About the Role

We are looking for a Senior Data Engineer with 5+ years of experience to join our Data Engineering team. The ideal candidate should have strong hands-on experience with PySpark, Spark, SQL, and data pipelines, along with the ability to work with large-scale datasets.

You will play a key role in building and optimizing data pipelines while improving their reliability, efficiency, scalability, and performance. This is also an opportunity to work on AI-ready datasets and modern data platforms, with exposure to technologies such as Databricks and Spark.

Key Responsibilities

• Build and maintain end-to-end data pipelines.
• Develop and implement best practices for data modeling and pipeline development.
• Create AI-ready analytical datasets with appropriate structure, metadata, documentation, and business context.
• Solve complex data pipeline challenges using PySpark and SQL.
• Work with stakeholders to understand requirements and incorporate business logic into data pipelines.
• Work with large-scale datasets and optimize data processing for performance and reliability.
• Contribute to the development and improvement of data engineering standards and processes.
• Gain hands-on exposure to Databricks, Spark, AI technologies, and modern ETL tools.
• Collaborate closely with engineering and business stakeholders to deliver high-quality data solutions.

What We're Looking For

• 5+ years of experience in Data Engineering or a related technical role.
• Strong hands-on experience with PySpark, Spark, and SQL.
• Proven experience building and maintaining data pipelines.
• Experience working with large-scale datasets.
• Strong understanding of data transformations, modeling, and pipeline architecture.
• Hands-on experience with Databricks, Delta Lake, or similar technologies.
• Ability to understand business requirements and translate them into effective data solutions.
• Strong problem-solving and analytical skills.
• Excellent verbal and written communication skills.
• Self-motivated, collaborative, and comfortable taking ownership.
• Willingness and ability to learn new technologies quickly.

Nice to Have

• Apache Airflow
• dbt
• Snowflake
• Modern ETL/ELT tools
• AI/ML data pipelines

Education

Bachelor’s or Master’s degree in Computer Science

Culture of Belonging

At SSI, we are committed to fostering a culture of belonging where everyone feels valued, respected, and empowered to contribute and grow.