Application Architect III

Job Summary

We are seeking a highly experienced Senior Databricks Engineer / Technical SME to join our Data Engineering team and drive enterprise data modernization initiatives. The ideal candidate will have strong hands-on experience with the Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Delta Sharing, data governance, and CI/CD automation.

The candidate will work closely with Application Leads, Data Engineers, Architects, and business stakeholders to design, implement, optimize, and govern scalable data solutions. This role will focus on Databricks platform engineering, Oracle-to-Databricks migration, performance tuning, cost optimization, security, and analytics enablement.

The ideal candidate should be a technical expert capable of guiding engineering teams, establishing best practices, and delivering enterprise-grade data engineering solutions.

Required Skills

Databricks & Lakehouse Platform

Strong hands-on experience with Databricks, Delta Lake, Lakehouse Architecture, Databricks Workflows, and scalable data engineering solutions.

Unity Catalog & Data Governance

Expertise in Unity Catalog, RBAC, data access controls, security, data governance, lineage, and Delta Sharing.

Performance Tuning & Cost Optimization

Proven experience optimizing Databricks workloads, Spark performance, cluster configurations, scalability, and platform costs.

Oracle to Databricks Migration

Experience in data warehouse modernization, Oracle migration, data transformation, and migrating enterprise data workloads to Databricks.

CI/CD & DevOps Automation

Experience implementing CI/CD pipelines using Git, Azure DevOps, GitHub, Jenkins, and Infrastructure as Code tools such as Terraform.

Databricks SQL & Analytics Enablement

Experience building semantic data models and business layers using Databricks SQL, AI/BI Dashboards, Genie, and Power BI.

Key Responsibilities

• Design, develop, and implement scalable enterprise data solutions using the Databricks Lakehouse Platform.
• Provide technical guidance and implementation best practices to application and data engineering teams.
• Optimize Databricks environments for performance, scalability, reliability, and cost efficiency.
• Implement and manage Unity Catalog, data governance, security, and data lineage.
• Design and implement RBAC and secure data access mechanisms.
• Implement Delta Sharing for secure internal and external data distribution.
• Lead and support Oracle-to-Databricks migration and modernization initiatives.
• Establish CI/CD pipelines, deployment automation, and DevOps best practices.
• Build and manage semantic data models and business layers for enterprise analytics.
• Enable integration with Databricks SQL, AI/BI Dashboards, Genie, and Power BI.
• Develop reusable frameworks, engineering standards, and implementation patterns.
• Mentor engineering teams and promote Databricks development best practices.

Additional Requirements

• 7+ years of overall IT/Data Engineering experience.
• Minimum 5+ years of Data Engineering experience.
• 3+ years of hands-on Databricks implementation experience.
• Strong understanding of modern data architectures and enterprise data warehousing.
• Experience with Azure, AWS, or GCP cloud platforms.
• Excellent stakeholder management, communication, and problem-solving skills.
• Ability to work independently as an Individual Contributor and Technical SME.

Good to Have / Preferred Skills

• Oracle-to-Databricks migration and modernization experience.
• Informatica PowerCenter / Informatica IDMC.
• Databricks Genie and AI/BI Dashboards.
• Databricks SQL and Power BI integration.
• Databricks Auto Loader, Delta Live Tables (DLT), and Lakeflow.
• MLflow and Databricks Workflows.
• Terraform and Infrastructure as Code.
• Databricks certifications.