
Senior Engineer – Data Engineering (Databricks) - Contractor
Jobgether · India · Remote
- Internet & E-commerce
- 11–50 employees
- Brussels
Strong hiring record · median 5 days open
- First seen
- 16 November 2026
- Reposted
- Not since September 2026
- Last checked
- Today · still on the employer's board
All reasons
In its favour
- Posted today
- Still listed on the employer's own board
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Engineer – Data Engineering (Databricks) – Contractor based in India.
As a Senior Data Engineer, you will play a key role in designing and delivering modern, scalable data platforms using Databricks and cloud-based lakehouse architectures. You will develop advanced data pipelines and distributed data processing solutions that support complex enterprise data and analytics requirements. Working with technologies such as Delta Lake, Unity Catalog, Apache Spark, and Delta Live Tables, you will help organizations transform how they manage, process, and leverage data. You will collaborate with solution architects and cross-functional engineering teams to design innovative solutions, optimize data workflows, and ensure high standards of data quality and reliability. You will also contribute to technical innovation by developing reusable solution accelerators and exploring emerging data engineering and AI technologies. This is an opportunity to work on challenging enterprise projects within a collaborative, technology-driven consulting environment.
Accountabilities
- Design, develop, and implement scalable data engineering solutions using the Databricks platform and modern cloud-based data lakehouse architectures.
- Build and maintain advanced data pipelines leveraging Databricks technologies, including Delta Lake, Delta Live Tables (Lakeflow Declarative Pipelines), Unity Catalog, and Delta Uniform (Iceberg).
- Design and optimize robust, YAML configuration-driven data pipelines supporting batch, micro-batch, and real-time streaming ingestion and processing.
- Implement efficient data ingestion and transformation workflows using tools and technologies such as Databricks Auto Loader and Apache Spark.
- Develop complex job scheduling and workflow orchestration solutions using Databricks Jobs and Workflows or external orchestration platforms such as Apache Airflow and dbt.
- Collaborate with solution architects to design and implement components of enterprise data platform architectures aligned with client requirements.
- Participate in technical design sessions, break down complex development requirements into manageable tasks, and deliver high-quality solutions within agreed timelines.
- Develop and implement data quality, pipeline auditing, monitoring, and observability frameworks using native Databricks capabilities and complementary tools.
- Optimize data processing performance, pipeline reliability, scalability, and resource utilization across distributed computing environments.
- Work with big data technologies and cloud data warehouses to integrate and manage large-scale enterprise datasets.
- Implement DevOps and CI/CD practices using Databricks Asset Bundles, including automated testing, validation, and deployment processes.
- Contribute to infrastructure automation and platform management initiatives using Infrastructure as Code (IaC) tools.
- Explore and integrate emerging technologies in data engineering, AI, and machine learning to improve platform capabilities and operational efficiency.
- Participate in Databricks Center of Excellence (CoE) initiatives, contributing reusable frameworks, solution accelerators, technical standards, and engineering best practices.
- Stay up to date with new Databricks platform features, emerging industry trends, and modern data engineering methodologies.
- Share technical knowledge, collaborate with fellow engineers, and contribute to a culture of innovation and continuous learning.
Requirements
- At least 5 years of professional experience in data engineering, including a minimum of 2 years of hands-on experience designing and developing data pipelines using Databricks.
- Strong practical expertise with Databricks platform capabilities, particularly Delta Lake, Delta Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative Pipelines), and Unity Catalog.
- Proven experience designing and implementing scalable, YAML configuration-driven data pipelines for batch, micro-batch, and streaming data processing.
- Strong understanding of modern data ingestion patterns and practical experience with tools such as Databricks Auto Loader.
- Experience developing complex data orchestration and workflow management solutions using Databricks Jobs and Workflows, Apache Airflow, dbt, or comparable tools.
- Knowledge of data quality management, pipeline auditing, monitoring, and observability frameworks, with exposure to tools such as Great Expectations, Collibra, and dbt.
- Strong experience with big data processing technologies, particularly Apache Spark, Presto, and Amazon EMR.
- Familiarity with cloud data warehousing platforms such as Amazon Redshift, Snowflake, and Google BigQuery.
- Advanced SQL skills, including stored procedures, query optimization, and complex data transformations.
- Strong programming skills in Python, PySpark, Java, or other object-oriented programming languages.
- Experience implementing DevOps and CI/CD pipelines, particularly using Databricks Asset Bundles, automated testing, and deployment validation.
- Familiarity with Infrastructure as Code (IaC) practices and tools such as Terraform is a strong advantage.
- Exposure to NoSQL databases and container management technologies.
- Understanding of AI and machine learning technologies, including MLflow, prompt engineering, and modern AI agentic workflows.
- Databricks Data Engineer Associate and/or Professional certifications are highly desirable.
- Demonstrated experience delivering multiple Databricks-based data engineering projects in enterprise environments.
- Strong analytical and problem-solving abilities, with the capacity to break down complex technical challenges into effective solutions.
- Excellent communication and collaboration skills, with the ability to work effectively with solution architects, developers, and cross-functional stakeholders.
- Self-motivated and proactive, with the ability to work independently while contributing to broader engineering initiatives.
- Curious mindset and enthusiasm for exploring new technologies, sharing knowledge, and continuously improving technical expertise.
- Ability to manage multiple development priorities and deliver reliable, high-quality solutions with minimal supervision.
Benefits
- Competitive contractor compensation aligned with technical expertise and experience.
- Fully remote working arrangement within India.
- Initial contract duration of 6 months or longer.
- Opportunity to work on large-scale enterprise data engineering and cloud transformation projects.
- Exposure to advanced Databricks technologies and modern lakehouse architectures.
- Opportunity to collaborate with experienced solution architects and multidisciplinary engineering teams.
- Participation in a dedicated Databricks Center of Excellence.
- Opportunities to develop reusable data engineering frameworks and innovative technical solutions.
- Exposure to leading cloud platforms, distributed processing technologies, and modern AI/ML tools.
- Continuous professional development through hands-on experience with emerging technologies and complex technical challenges.
- Opportunity to contribute to impactful data and analytics initiatives for leading global organizations.
How Jobgether works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice
By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
Skills
- Databricks
- Delta Lake
- Unity Catalog
- Spark
- Delta Live Tables
- Apache Iceberg
- YAML
- Airflow
- dbt
- Machine Learning
- Great Expectations
- Presto
- EMR
- Redshift
- Snowflake
- BigQuery
- SQL
- Python
- PySpark
- Java
- Terraform
- MLflow
- Prompt Engineering
- GDPR
More jobs at Jobgether
All 1,514Similar roles
- Clinical Data EngineerJobgether · Indiatoday
- Data ScientistNielsenIQ · Pune, MH, Indiatoday
- Senior Database AdministratorEnsono · Bengaluru, India · Chennai, India · Hyderabad, India +1today
- Lead Analytics Consultant (Power BI)Jobgether · Indiatoday
- Workday Data AnalystJobgether · Indiatoday
- Lead Data Architect (Fabric)Jobgether · Indiatoday