Databricks Data Engineer
SGF Global · Mexico City, DF
Strong hiring record · median 6 days open
- First seen
- 13 November 2026
- Reposted
- Not since September 2026
All reasons
In its favour
- Posted 2 days ago
- Still listed on the employer's own board
Databricks Data Engineer
Key Responsibilities
- Design and implement end-to-end data pipelines using Azure Databricks including batch and streaming workflows
- Build and maintain Delta Lake architecture including bronze, silver, and gold layers
- Develop scalable ETL/ELT pipelines using Apache Spark (PySpark / Scala)
- Optimize Databricks jobs for performance, cost, and reliability — including cluster tuning, caching, and query optimization
- Implement Delta Live Tables (DLT) for declarative pipeline development
- Build and maintain Unity Catalog for data governance, lineage, and access control
- Collaborate with data architects to design data models (medallion architecture, star/snowflake schemas)
- Integrate Databricks with cloud services: Azure Data Factory, Azure Synapse, S3, ADLS Gen2
- Implement CI/CD pipelines for Databricks notebooks and jobs using Azure DevOps / GitHub Actions
- Ensure data quality, monitoring, and alerting across data pipelines
- Contribute to client workshops, technical design sessions, and solution architecture reviews
- Nice to have if knowledge in Snowflake, IICS, Stonebranch
- Should be good with YAML files work - metadata/config driven pipelines for platform requires handling data ingestion and moving data from Bronze to Silver using configuration files (YAML),
Required Skills & Experience
Core Technical Skills
- 3–8 years of experience in data engineering, data warehousing, or big data solutions
- Hands-on expertise with Databricks (Databricks Certified Associate/Professional preferred)
- Strong proficiency in PySpark and/or Scala for large-scale data processing
- Experience with Delta Lake — ACID transactions, schema evolution, time travel, merge operations
- Proficiency in SQL (SparkSQL, HiveQL) and Python
- Experience with cloud platforms: Microsoft Azure
- Experience with orchestration tools: Apache Airflow, Azure Data Factory, Databricks Workflows
Data Architecture
- Strong understanding of medallion architecture (Bronze / Silver / Gold)
- Experience designing data lakehouses, data warehouses, or data mesh architectures
- Familiarity with data modeling concepts — dimensional modeling, data vault
- Knowledge of data governance principles and cataloging tools (Unity Catalog, Purview, Collibra)
DevOps & Collaboration
- Experience with Git and version control best practices
- Familiarity with CI/CD tools (Azure DevOps, GitHub Actions, Jenkins)
- Experience working in Agile/Scrum delivery environments
- Strong communication skills — ability to explain technical concepts to non-technical stakeholders
Skills
- Databricks
- Azure Databricks
- Delta Lake
- ETL
- ELT
- Spark
- PySpark
- Scala
- Delta Live Tables
- Unity Catalog
- Snowflake
- Azure Data Factory
- Synapse
- S3
- Azure DevOps
- GitHub Actions
- YAML
- SQL
- Python
- Azure
- Airflow
- Git
- Jenkins
More jobs at SGF Global
All 14- Power BI EngineerSGF Global · Mexico City2d ago
- Hyatt Hotels - Senior Manual QASGF Global · Brazil · Mexico · Colombia +22d ago
- Software Engineer III BRLSGF Global · Sao Paulo, SP3d ago
- Kontoor Brands - Systems EngineerSGF Global · Mexico · Brazil · Colombia +25d ago
- Salesforce Developer Contractor: 6-9 years (Advanced)SGF Global · Dallas, TX5d ago
Similar roles
- Sr Data EngineerSequoia Connect · Ciudad de México, CDMX, Mexicotoday
- Data Engineer, Marketing SciencePower Digital · Mexicotoday
- AWS Data EngineerSequoia Connect · Ciudad de México, CDMX, Mexicoyesterday
- Senior Data Analyst - GCP/ BigQuerySequoia Connect · Ciudad de México, CDMX, Mexico2d ago
- Senior Analytics EngineerMotive · Mexico City2d ago
- Data Scientist MexicoLinkedIn Job Wrapping · Mexico2d ago