
Technical Integration Lead Databricks
Diverse Lynx · Indianapolis, IN
Strong hiring record · median 15 days open
- First seen
- 19 November 2026
- Reposted
- Not since September 2026
All reasons
In its favour
- Posted yesterday
- Still listed on the employer's own board
- Publishes a salary range
Job Title
Technical Integration Lead Databricks Data Products
Location
Indianapolis, IN
Duration
6+ months Contract
Skills
Databricks
Hourly Rate
$56-$65/hour
Required Skills & Qualifications
Job Description
Integration Lead – Databricks Data Products & Enterprise CRM Integration
Role Type Technical Integration Lead (Hands-On) Industry Life Sciences / Pharmaceutical
Primary Platform Databricks Data Products, ETL, Veeva Vault CRM Client Commercial CRM for Life Sciences
Delivery Model Onsite – Offshore Delivery Context Multi-Vendor (Dual System Integrator)
Experience 7+ years IT, 5+ years on ETL, 3+ years on Databricks Location Onsite
Role Overview
We are seeking an experienced, hands-on Integration Lead to design, build, and deliver integrations that consume Master Data products published in Databricks and feed them into an enterprise commercial CRM platform, as part of a large-scale CRM modernization program.
Master Data sources (customer, account, and product master) are exposed as governed data products in Databricks – curated Delta tables registered in Unity Catalog. This role owns the consumption of those data products and their reliable, validated delivery into the target CRM and enterprise ecosystem, while re-platforming existing integrations onto modern, cloud-native patterns.
This is a working, hands-on leadership role. The person leads the integration workstream and personally performs the coding, code review, and deployment work – not oversight alone. Every skill listed is expected to be backed by direct, hands-on delivery experience.
Key Responsibilities
- Hands-On Integration Delivery
- Personally perform coding, code review, and deployment activities, and any other assigned work related to this transformation – leading the team does not replace individual hands-on contribution.
- Must have hands-on experience across all required skills; the person is expected to build, not only direct others who build.
Own the end-to-end integration lifecycle
design specification, build, unit testing, deployment across dev / sandbox / UAT / prod, runbooks, and knowledge transfer.
- Produce design specifications, deployment steps, and rollback approaches, and deliver them hands-on.
- Databricks Data-Product Consumption & Integration
- Consume Databricks data products (Delta Lake tables via Unity Catalog) as the source of master data, respecting the data-product contract – schema, refresh cadence, quality guarantees, and access model.
- Move master data from Databricks into the CRM using the appropriate pattern – Apache Iceberg interoperability, Spark / PySpark-based movement, or API-based delivery – selecting the right mechanism per integration by volume, latency, and directionality.
- Build transformation and mapping logic (PySpark / Spark SQL) to reconcile the data-product schema with the target CRM data model, including handling master-data change (deltas, data-change requests, reference data).
- Re-platform existing master-data integrations onto the Databricks-data-product and modern integration patterns, ensuring parity, reconciliation, and no data loss during transition.
- Implement observability, data-quality checks, error handling, data-gap detection, and reprocessing across the Databricks-to-CRM flows.
- Client Engagement & Meetings
- Participate in and lead all relevant client meetings – design reviews, working sessions, governance forums, and status discussions – representing the integration workstream directly with the client.
- Communicate technical decisions, risks, and dependencies clearly to client stakeholders and delivery teams.
- Multi-Vendor (Dual-SI) Collaboration
- Collaborate with the second System Integrator (SI) partner responsible for Consultantly testing and validating this team’s build.
- Review the second SI’s test findings, triage them, and resolve the defects – ensuring fast, clean cross-vendor root-cause and closure rather than back-and-forth.
- Maintain clean build-to-test handoffs and shared traceability so defects are resolved efficiently across vendors.
- Patterns, Standards & Data Partnership
- Define reusable integration patterns and interface contracts for master-data flows that other build and assurance teams can work to.
- Partner with data-product owners and data-engineering teams to align on data-product schemas, versioning, refresh frequency, access, and change management.
- Ensure data integrity, security, privacy, and, where applicable, regulated-industry compliance across connected systems.
Must-Have Technical Skills
All of the following require direct, hands-on experience
- Databricks – hands-on with Delta Lake, Unity Catalog, and consuming / publishing data products; lakehouse / medallion architecture.
- AI – hands-on experience applying AI in a data, integration, or software-development context.
- Data services on AWS – hands-on experience building data and transformation services / pipelines on AWS.
- API – hands-on REST API design, build, and implementation at scale.
- PySpark – strong, hands-on PySpark and Spark SQL for data movement and transformation.
- Master Data / MDM – hands-on experience integrating customer, account, and product master data into downstream systems, including schema mapping, reconciliation, and master-data change handling.
- Demonstrated ability to lead a build team while remaining hands-on – doing the build, review, and deployment work directly.
Other Skills
- Informatica – IICS and ETL (PowerCenter); ability to understand and re-platform existing Informatica integrations.
- Veeva CRM and Veeva Vault CRM integration knowledge – Vault REST API, Direct Data API, Spark Messaging, Vault Loader, VQL, and the Vault data model.
- Experience remediating or migrating enterprise CRM platforms.
- Regulated-industry (pharma / life sciences) experience; GxP / CSV / 21 CFR Part 11 familiarity.
- Experience in a multi-vendor / dual-SI delivery model; global, multi-geography delivery with data-residency awareness.
- Databricks certification.
Leadership & Delivery Competencies
- A true practitioner who personally builds – owns code, reviews, and deployments, not only direction.
- Strong client-facing communication; able to lead client meetings and represent the integration workstream credibly.
- Collaborative across data-engineering, platform, and the second SI partner – resolves cross-vendor test findings without friction.
- Strong analytical thinking, root-cause discipline, and problem-solving under delivery pressure.
- Willing and able to be on-site in Indianapolis and travel for key meetings.
Role Criticality
This role is critical because the Integration Lead is the hands-on owner of the integrations that bridge governed Databricks & other data products into the enterprise CRM. Accuracy in these flows is non-negotiable – an error in master data reaches the field and business operations quickly. The person in this role must combine deep, hands-on Databricks, AWS, and PySpark engineering with the ability to lead client meetings, resolve cross-vendor test findings, and deliver reliably in a regulated, multi-vendor program
Role Descriptions
We are seeking an experienced| hands-on Integration Lead to design| build| and deliver integrations that consume Master Data products published in Databricks and feed them into an enterprise commercial CRM platform| as part of a large-scale CRM modernization program.
Master Data sources (customer| account| and product master) are exposed as governed data products in Databricks curated Delta tables registered in Unity Catalog.
This role owns the consumption of those data products and their reliable| validated delivery into the target CRM and enterprise ecosystem| while re-platforming existing integrations onto modern| cloud-native patterns.
This is a working| hands-on leadership role. The person leads the integration workstream and personally performs the coding| code review| and deployment work not oversight alone. Every skill listed is expected to be backed by direct| hands-on delivery experience.
Skills
- Databricks
- ETL
- HashiCorp Vault
- Unity Catalog
- Delta Lake
- Apache Iceberg
- Spark
- PySpark
- SQL
- AWS
- Informatica
More jobs at Diverse Lynx
All 445- Sr Appian developerDiverse Lynx · Louisville, KY · USD 8–60/hrtoday
- OpenShift CP4 AdminDiverse Lynx · Remotetoday
- Senior DevOps EngineerDiverse Lynx · Remote · USD 110,000/yrtoday
- Senior Apache Spark Technical Lead - Scala, PythonDiverse Lynx · Basking Ridge, NJ · USD 50–53/hrtoday
- Mulesoft DeveloperDiverse Lynx · Louisville, KY-Day 1 · USD 45–50/hrtoday
Similar roles
- Senior Technical Architect (Assoc. Dir.), Integrations & Data Readiness Solutions (Informatica, Databricks, Snowflake)Lantheus Brand · Bedford, MA, United States · USD 139,000–232,000/yryesterday
- USA_EngineerVarite · Indianapolis, IN · USD 68–72/hryesterday
- Associate Principal Engineer - Integration Solutions Architect (Hands-on)Nagarro · Jersey City, NJ, United States · USD 150,000–170,000/yr20d ago
- Software developer with data engineering experienceDiverse Lynx · chicago · USD 80–85/hr5d ago
- Principal Data & Analytics EngineerSynergenX Health Holdings · Houston, TX · USD 150,000/yr9d ago
- Databricks Platform Engineer (Terraform / AWS / GitLab CI/CD)Compu-Vision Consulting · NJtoday