Hiring Data Engineer

Role

Data Engineer

Duration

12 months

Hybrid

3 days a week in office

Max BR

52 CAD/ hr.

An experienced Data Engineer with strong expertise in Big Data technologies to design, develop, and support enterprise-scale data platforms. The ideal candidate should possess hands-on experience in PySpark, Apache Spark, Kafka, Hadoop ecosystem components, and Apache NiFi, with a strong understanding of data ingestion, transformation, and real-time processing frameworks.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark, Spark, Hadoop, and Apache NiFi.
• Build and maintain batch and real-time data processing solutions.
• Develop and support Kafka-based streaming applications and event-driven architectures.
• Create and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
• Develop complex SQL queries for data extraction, transformation, validation, and troubleshooting.
• Implement data ingestion solutions from databases, APIs, files, and streaming sources.
• Monitor, troubleshoot, and enhance the performance of Spark jobs and data pipelines.
• Collaborate with architects, business analysts, and development teams to deliver high-quality data solutions.
• Support platform upgrades, deployments, testing, certification, and production releases.
• Ensure data quality, governance, security, and operational excellence across data platforms.

Mandatory Skills

• PySpark
• Apache Spark (Spark SQL, DataFrames)
• Apache Kafka
• Hadoop Ecosystem (HDFS, Hive, YARN)
• Apache NiFi
• SQL
• Python

Preferred Skills

• Spark Streaming
• Airflow / Oozie
• Hive
• Scala
• Jenkins, Bitbucket, Git
• JIRA, Confluence
• Cloud Platforms (GCP/AWS/Azure)
• Data Warehousing concepts and Dimensional Modeling

Custom Fields

Name

Child

Value

IS-BFSI-Canada-1-Group2

Name

Assets

Value

None

Name

Parent

Value

IS-BFSI-Canada-1-Parent

Name

Requisition ID

Value

11003737

Name

Type of Asset

Value

None

Name

Business Vertical

Value

CL-BFSI Americas 2

Name

Work Assignment Location

Value

Hybrid

Name

Work Location CDF

Value

~Toronto 24th Floor~TCCI Canada Corporate~