Databricks Data Engineer Architect

Databricks Data Engineer Architect – Onshore

Remote work

6 months + contract

Position Overview

We are seeking an experienced Databricks Data Engineer Architect to design, architect, and implement scalable, secure, and high-performance data solutions using Databricks. The ideal candidate will have extensive hands-on experience with Databricks, a strong background in data architecture, and proven experience migrating data from Salesforce and NetSuite into the Databricks platform.

This role requires a strong technical leader who can translate business requirements into robust data architectures, drive data migration initiatives, and establish best practices for data engineering, integration, and analytics.

Key Responsibilities

• Design and architect end-to-end data engineering solutions using Databricks and cloud-based data platforms.
• Lead the architecture, design, and implementation of scalable data pipelines for batch and incremental data processing.
• Architect and execute data migration solutions from Salesforce and NetSuite to Databricks, ensuring data accuracy, completeness, consistency, and integrity.
• Integrate data from Salesforce CRM, NetSuite ERP, and other enterprise source systems into a centralized lakehouse platform.
• Develop and optimize ETL/ELT pipelines using PySpark, SQL, Python, and Databricks workflows.
• Implement the Databricks Lakehouse architecture, including Delta Lake, data modeling, data transformation, and storage optimization.
• Establish data ingestion, transformation, orchestration, monitoring, and error-handling frameworks.
• Define and implement data governance, security, access controls, data quality, and metadata management standards.
• Optimize data pipelines and Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Collaborate with business stakeholders, enterprise architects, data analysts, and engineering teams to understand requirements and deliver technical solutions.
• Create architecture diagrams, technical design documents, data mapping specifications, and migration strategies.
• Provide technical leadership, conduct code reviews, and mentor data engineering teams.
• Ensure adherence to enterprise architecture standards, development best practices, and data compliance requirements.

Required Qualifications

• Extensive hands-on experience with Databricks data engineering and solution architecture.
• Proven experience architecting and delivering end-to-end enterprise data solutions using Databricks.
• Demonstrated experience migrating data from Salesforce and NetSuite to Databricks.
• Strong expertise in Python, PySpark, SQL, and ETL/ELT development.
• In-depth knowledge of Delta Lake, Lakehouse architecture, and scalable data processing.
• Experience integrating SaaS and enterprise applications with modern cloud data platforms.
• Strong understanding of data modeling, data warehousing, dimensional modeling, and data lake design.
• Experience designing incremental data loads, change data capture (CDC), and data reconciliation processes.
• Hands-on experience with workflow orchestration, pipeline monitoring, troubleshooting, and performance tuning.
• Knowledge of cloud platforms such as Microsoft Azure, AWS, or Google Cloud, preferably aligned with the organization's Databricks environment.
• Experience with data security, governance, access management, and production deployment practices.
• Excellent communication, problem-solving, stakeholder management, and technical documentation skills.

Preferred Qualifications

• Databricks certifications in data engineering or solution architecture.
• Experience with Unity Catalog, Delta Live Tables/Lakeflow, and Databricks Workflows.
• Familiarity with Salesforce APIs, Bulk API, and enterprise data extraction patterns.
• Familiarity with NetSuite SuiteTalk APIs, SuiteAnalytics, or SuiteQL for data extraction and integration.
• Experience with CI/CD, Git, automated testing, and infrastructure-as-code practices.
• Experience implementing enterprise data quality frameworks and production-grade data observability.

Role Requirements

• Position: Databricks Data Engineer Architect
• Work Arrangement: Onshore
• Experience Level: Senior / Lead / Architect
• Primary Focus: Databricks architecture, data engineering, and Salesforce/NetSuite data migration

Ideal Candidate Profile

The ideal candidate is a hands-on Databricks architect who has successfully designed and delivered production-grade data platforms, with direct experience migrating and integrating Salesforce and NetSuite data into Databricks. The candidate should be capable of owning the solution architecture from requirements gathering and technical design through implementation, optimization, and production support.