Databricks Administrator / Databricks Platform Engineer

Company Name

VARITE India Private Limited

About The Client

An Indian multinational information technology (IT) consulting company headquartered in Noida, The company has offices in 52 countries and over 225,944 employees. The Client is a global IT services and consulting company that offers a wide range of services and products across various industries like IT Infrastructure Services, Cybersecurity Services, Cloud Services, Big Data and Analytics, Internet of Things (IoT) Solutions, Semiconductor Services and Enterprise Software Products.

About The Job

• We are looking for an experienced Databricks Administrator / Platform Engineer to administer, govern, optimize, and support Databricks environments across Development, Test, and Production.
• The role will focus on platform administration, Unity Catalog governance, security and access management, cluster and workload optimization, operational reliability, automation, cost governance, and L3 production support.
• The ideal candidate should have strong hands-on experience with Databricks, Apache Spark, SQL, Python, Azure/ADLS, Unity Catalog, RBAC, platform monitoring, and automation.

Essential Job Functions

Databricks Platform Administration

• Administer and manage Databricks workspaces across Development, Test, and Production environments.
• Manage Databricks objects, including notebooks, workflows, Delta tables, dashboards, and repositories.
• Manage platform maintenance activities and Databricks Runtime upgrades.
• Monitor platform health, availability, resource utilization, and operational KPIs.
• Maintain platform documentation, SOPs, governance standards, and knowledge articles.

Unity Catalog & Data Governance

• Administer Unity Catalog, including catalogs, schemas, tables, views, external locations, and storage credentials.
• Configure and govern data access policies, permissions, and security controls.
• Support audit, compliance, security governance, and vulnerability remediation activities.
• Ensure appropriate access controls and governance standards are consistently applied across environments.

Identity & Access Management

• Implement and maintain Role-Based Access Control (RBAC).
• Manage user provisioning, groups, permissions, and service principal access.
• Collaborate with Security and Infrastructure teams to maintain appropriate platform access and security controls.

Cluster & Compute Management

• Configure, manage, and optimize interactive and job clusters.
• Define and enforce cluster policies, auto-scaling, and compute governance standards.
• Monitor cluster utilization and resource consumption.
• Perform cluster right-sizing and workload optimization to improve performance and control costs.
• Forecast compute capacity, platform growth, and future resource requirements.

Performance Optimization & Troubleshooting

• Troubleshoot Spark cluster issues, failed jobs, driver/executor errors, and performance bottlenecks.
• Perform Spark and SQL workload tuning to improve performance and cost efficiency.
• Conduct Root Cause Analysis (RCA) for recurring platform and workload issues.
• Implement preventive and corrective measures to improve platform stability.

Jobs, Workflows & Operations

• Monitor and support Databricks Jobs, Workflows, scheduled pipelines, and production workloads.
• Provide L3 operational support for Databricks platform-related incidents and service issues.
• Participate in Incident, Problem, and Change Management processes.
• Develop operational dashboards, monitoring reports, and platform KPIs.

Integration & Automation

• Support Databricks integration with ADLS, Azure services, and external data platforms.
• Automate administrative and operational activities using:
• Python
• SQL
• REST APIs
• Databricks Workflows
• Identify opportunities to reduce manual operational effort through automation.

Continuous Improvement

• Drive platform modernization, automation, reliability, and continuous improvement initiatives.
• Collaborate closely with Data Engineering, Security, Infrastructure, and Operations teams.
• Identify opportunities to improve platform performance, governance, operational efficiency, and cost optimization.

Qualifications

Required Technical Skills

• Strong hands-on experience in Databricks platform administration
• Strong knowledge of Unity Catalog and data governance
• Experience with RBAC, identity and access management, groups, and service principals
• Strong understanding of Apache Spark architecture and troubleshooting
• Experience managing Databricks clusters, cluster policies, auto-scaling, and compute resources
• Strong knowledge of Spark and SQL performance tuning
• Experience supporting Databricks Jobs and Workflows
• Experience with Delta tables / Delta Lake
• Working knowledge of Python and SQL
• Experience with REST API-based automation
• Experience integrating Databricks with Azure and ADLS
• Knowledge of platform monitoring, capacity planning, and cost optimization
• Experience in Incident, Problem, Change, and RCA processes
• Knowledge of security, compliance, audit, and vulnerability remediation practices

Preferred Skills

• Experience managing enterprise-scale Databricks environments.
• Strong understanding of production platform governance and operational controls.
• Experience developing platform monitoring and KPI dashboards.
• Experience driving automation and platform modernization initiatives.
• Strong troubleshooting and analytical skills.
• Ability to work effectively with Data Engineering, Infrastructure, Security, and application teams.

How to Apply

Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India – VARITE.

Unlock Rewards

Refer Candidates and Earn.

If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.

Experience Level Bonus Referral

0-2 years

INR 5,000

2-6 years

INR 7,500

6+ years

INR 10,000

About VARITE

VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.

Equal Opportunity Employer

VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.