Databricks Engineer

Data Engineer, Azure Databricks (Corporate Vertical Developer)

We are seeking a corporate vertical developer who will operate across multiple lines of business within corporate, including RESO, HCG, Compliance, Facilities, and Finance. This role focuses on building scalable data solutions, driving data integration, and enabling analytics across the firm. The ideal candidate brings strong data engineering experience with Azure and Databricks, and works effectively across teams to deliver shared data platforms and pipelines.

Location

NYC area, on-site 3 days a week

Responsibilities include, but are not limited to

• Design, build, and maintain scalable data pipelines using Azure and Databricks
• Develop and optimize ETL and ELT workflows for structured and unstructured data
• Collaborate with business and technical teams across corporate functions to define data requirements and deliver solutions
• Implement data models and data architectures to support analytics and reporting use cases
• Integrate enterprise systems such as ServiceNow, Dataverse, and Workday into centralized data platforms
• Ensure data quality, governance, and security standards are met across all solutions
• Monitor and troubleshoot data workflows, ensuring reliability and performance
• Contribute to the evolution of enterprise data platforms and recommend improvements based on emerging technologies

Required Skills

• Strong experience as a Data Engineer with focus on Azure cloud services
• Hands-on experience with Databricks, including Spark and distributed data processing
• Strong SQL skills and experience working with large-scale data systems
• Experience building data pipelines using Azure Data Factory or similar tools
• Experience with Azure SQL Database and performance optimization
• Proficiency in Python or Scala for data engineering workloads
• Experience with C# and PowerShell for automation and integration tasks
• Experience developing and deploying Azure Functions for data processing and system integration
• Experience integrating APIs, event-driven systems, and enterprise platforms
• Experience designing scalable and secure data architectures
• 5+ years of experience in data engineering or related field
• Strong communication skills and ability to work across multiple business domains

Beneficial Skills

• Experience with Azure DevOps and CI/CD pipelines
• Experience with Delta Lake and modern data lakehouse architectures
• Familiarity with data governance and data catalog tools
• Experience working in Agile or Scrum environments
• Microsoft or Databricks certifications