
Lead Data Engineer, Databricks
Jobgether · US · Remote
- Internet & E-commerce
- 11–50 employees
- Brussels
Strong hiring record · median 5 days open
- First seen
- 16 November 2026
- Reposted
- Not since September 2026
- Last checked
- Today · still on the employer's board
All reasons
In its favour
- Posted today
- Still listed on the employer's own board
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Data Engineer, Databricks based in the United States.
The Lead Data Engineer will drive the delivery of production-grade data engineering solutions for enterprise customers, taking ownership of technical workstreams from design through production handover. This role combines hands-on development with technical leadership, building scalable batch and streaming pipelines using Databricks, PySpark, SQL, and Delta Lake. You will establish engineering standards, guide implementation decisions, and ensure solutions meet performance, quality, and cost objectives. Working directly with customers and cross-functional technical teams, you will translate complex requirements into reliable data platforms. You will also mentor engineers, collaborate with data architects, and communicate technical trade-offs to stakeholders. This is a fully remote opportunity for an experienced engineer who enjoys combining technical excellence, customer engagement, and team leadership.
Accountabilities
The Lead Data Engineer will serve as the technical owner of engineering workstreams across customer engagements, ensuring that data solutions are designed, implemented, optimized, and delivered to production.
Key responsibilities include
- Lead data engineering activities across customer engagements, from initial design and implementation through production handover.
- Build, maintain, and optimize batch and streaming data pipelines using PySpark, SQL, and Delta Lake on Databricks.
- Implement Medallion architecture across cloud environments and establish consistent data layer conventions and engineering patterns.
- Take ownership of platform performance and cost efficiency by benchmarking Spark jobs, tuning SQL queries, right-sizing compute resources, and evaluating technical trade-offs.
- Implement data quality and validation processes as code, including Lakeflow expectations and automated testing within CI pipelines.
- Establish and maintain engineering standards, ensuring the reliability, scalability, and quality of delivered solutions.
- Mentor junior and mid-level engineers through code reviews, collaborative development, technical guidance, and design feedback.
- Partner with Data Architects to validate architectural decisions and identify potential design limitations during implementation.
- Communicate technical decisions, delivery progress, and implementation trade-offs clearly to customers and non-technical stakeholders.
- Work closely with customer teams and relevant platform specialists to ensure successful delivery of enterprise data solutions.
Requirements
The ideal candidate is a highly experienced data engineer with a strong background in Databricks and cloud-based data platforms, combined with proven technical leadership and customer-facing communication skills.
Required qualifications include
- At least 7 years of professional data engineering experience, including experience leading projects or engineering teams.
- Strong proficiency in Python, SQL, Apache Spark, and Delta Lake.
- Hands-on production experience building both batch and streaming data pipelines at enterprise scale.
- Practical experience with at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform (GCP), including its data services.
- Experience with data modeling, Git, and CI/CD practices for data engineering workloads.
- Demonstrated ability to optimize Spark performance, improve pipeline efficiency, and manage platform costs.
- Strong understanding of scalable data architecture and modern data engineering practices.
- Excellent client communication skills, including the ability to explain complex technical concepts and trade-offs to non-technical stakeholders.
- Proven ability to guide engineers, provide constructive feedback, and take ownership of technical delivery.
Nice to Have
- Experience in healthcare or life sciences environments.
- Databricks Certified Data Engineer Associate or Professional certification.
- Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles.
- Familiarity with dbt, Airflow, or equivalent orchestration and data transformation tools.
- Experience migrating legacy ETL systems to modern cloud-based data platforms.
- Previous consulting or client-facing engineering experience.
Benefits
- Unlimited paid time off.
- Medical, dental, and vision coverage.
- Retirement savings plan, including 401(k) and IRA options.
- Basic and voluntary life insurance.
- Accidental Death and Dismemberment (AD&D) coverage.
- Maternity and paternity leave.
- Short-term and long-term disability coverage.
- Training and professional development opportunities.
- Dedicated certification roadmap for technical employees.
- In-house training delivered by Databricks-approved trainers.
- Dedicated Employee Success Manager focused on supporting the employee experience.
How Jobgether works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice
By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
More jobs at Jobgether
All 1,514Similar roles
- Power BI Developer / Data Visualization AnalystJobgether · UStoday
- Data Engineering Solution LeadJobgether · UStoday
- Staff Data Platform Engineer, Knowledge PlatformAirwallex · US - Seattletoday
- Data Platform Principal Architect, Seattle/San FranciscoAirwallex · US - Seattletoday
- Senior Battery Data EngineerRedwood Materials · San Francisco, California, United Statestoday
- Sales Analytics EngineerBaseten · San Franciscotoday