Databricks Data Lake Architect

Our Client, an IT Services and Consultant company, is looking for a Databricks Data Lake Architect for their Plano, TX/Hybrid location.

Responsibilities

• Design robust lakehouse architectures that integrate Databricks Delta Lake with enterprise data sources to enable scalable analytics and generative AI solutions that advance strategic business outcomes.
• Architect Spark and Pyspark based data pipelines optimized for performance, reliability, and cost efficiency to support complex analytical workloads and downstream applications.
• Develop Python services and reusable components that operationalize GenAI basics and data processing logic, ensuring maintainable and testable code across projects.
• Define Databricks Workflows orchestration patterns that coordinate Spark jobs, Delta Lake operations, and data quality checks to deliver predictable and resilient data products. -Configure Databricks Jobs and Lakeflow Jobs to manage end to end batch and near real time processing while monitoring execution health and optimizing resource utilization.
• Establish Databricks SQL models, queries, and shared datasets that provide consistent and governed access to curated information for analytics and reporting teams.
• Implement Databricks Unity Catalog structures including catalogs, schemas, and access controls that enforce enterprise data governance and regulatory compliance requirements.
• Collaborate with data engineers, data scientists, and product stakeholders in a hybrid work setting to translate analytical needs into technical designs and implementation plans.
• Review solution designs and code across Spark, Python, and Databricks assets to ensure adherence to architectural standards, security practices, and performance benchmarks.
• Document end to end architecture including data lineage, dependency mapping, and operational procedures to support long term maintainability and knowledge sharing.
• Advise business partners on leveraging GenAI capabilities safely and effectively using governed data in Databricks to create measurable impact for customers and society.
• Drive continuous improvement initiatives by analyzing platform metrics, identifying bottlenecks in workflows, and implementing enhancements that increase reliability and throughput.
• Guide adoption of best practices for hybrid collaboration, ensuring that design discussions, reviews, and knowledge sessions are inclusive and accessible to distributed teams.

Requirements

• Demonstrate deep expertise in Spark, Pyspark, and SparkSQL with extensive experience designing high volume data pipelines and performance tuned transformations in production environments.
• Show advanced proficiency in Python three dot x for data engineering and analytics use cases including modular coding standards, automated testing, and robust error handling.
• Apply strong hands on experience with Databricks Workflows, Jobs, and Lakeflow Jobs to build orchestrated processes that meet strict service level objectives and audit needs.
• Exhibit solid knowledge of Databricks SQL and Delta Lake for building reliable lakehouse models, incremental ingestion patterns, and optimized query structures.
• Use practical experience with Databricks Unity Catalog to design secure data governance frameworks that support role based access and consistent metadata management.
• Bring working understanding of GenAI basics including prompt driven patterns, data preparation constraints, and responsible usage aligned with organizational guidelines.
• Leverage prior background of twelve to fifteen years in data architecture or related fields to handle complex stakeholder environments and large scale platform evolution.
• Communicate clearly in hybrid work settings, using structured documentation and concise technical summaries to align cross functional teams on architectural decisions.

Why Should You Apply?

• Health Benefits
• Referral Program
• Excellent growth and advancement opportunities