Databricks Architect

Job Description: Databricks Architect

Job Details

• Location: Louisville, KY
• Work Location: 101 E Main Street, Louisville, KY 40202 / Remote
• Duration: 6 months
• Experience Required: 8–10 years
• Base Salary Range: Minimum and maximum salary to be specified
• Role: Google Gen AI Cloud | Databricks Architect

Must-Have Technical/Functional Skills

• Expertise in designing, delivering, and implementing Google Cloud Platform (GCP) infrastructure and solutions.
• Experience architecting and implementing large-scale migration pipelines from on-premises systems to Azure Data Lake Storage (ADLS).
• Strong experience in designing and developing data pipelines using Azure Data Factory (ADF).
• Strong knowledge of GCP AI/ML services and infrastructure.
• Proficiency in AI/ML frameworks such as TensorFlow or PyTorch.
• Strong technical skills in Python, data modeling, SQL, and APIs.
• Hands-on experience with Terraform for Infrastructure as Code (IaC) and automation.
• Proven experience delivering AI/ML solutions in cloud environments.
• Experience performing data analytics and generating insights for stakeholders.
• Experience ingesting data from multiple sources using Azure Data Factory.
• Ability to handle high-volume unstructured data with reliability, performance, and cost efficiency.
• Experience implementing data quality checks, logging, and traceability to meet audit and compliance requirements.
• Hands-on experience transforming data using Azure Databricks and Delta Lake.

Roles and Responsibilities

Architecture and Design

• Design enterprise-grade AI platforms, including Retrieval-Augmented Generation (RAG) and agentic AI solutions.
• Develop scalable, secure, and high-performance cloud architectures.

GCP Implementation

• Develop and deploy AI models using Vertex AI and Gemini.
• Ensure model security, scalability, and performance.

Pipeline Development

• Build and manage data pipelines and model lifecycle workflows.
• Implement CI/CD, MLOps, and LLMOps practices.
• Utilize Cloud Build and Vertex AI Pipelines for automation and deployment.

Cloud Infrastructure

• Leverage Google Kubernetes Engine (GKE), Cloud Run, and BigQuery for robust application deployment and data processing.
• Design and manage scalable cloud infrastructure.

AI Strategy and Governance

• Implement AI safety features and governance frameworks.
• Monitor model performance and reliability.
• Optimize infrastructure and model costs.

Generic / Managerial Skills

• Experience working in Agile development projects.
• Strong collaboration and communication skills.
• Ability to work with cross-functional teams and business stakeholders.

Essential Skills

• Google Gen AI Cloud
• Google Cloud Platform (GCP)
• Databricks Architecture
• Generative AI and Agentic AI
• Vertex AI and Gemini
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure Databricks and Delta Lake
• AI/ML and MLOps/LLMOps

Desirable Skills

• Additional cloud architecture and AI platform implementation experience.
• Experience with enterprise-scale data migration and governance.
• Knowledge of AI security, compliance, and cost optimization.

Keywords

• Google Gen AI Cloud
• GCP
• Databricks Architect
• Generative AI
• Agentic AI
• RAG
• Vertex AI
• Gemini
• TensorFlow
• PyTorch
• Python
• Terraform
• Azure Data Factory
• Azure Data Lake Storage
• Azure Databricks
• Delta Lake
• BigQuery
• GKE
• Cloud Run
• MLOps
• LLMOps
• CI/CD
• Data Engineering
• AI/ML Solutions