
Senior Consultant - Technical - Data
Value Creed · Hyderabad · Onsite
- Energy & Utilities
- 51–200 employees
- Dallas, Texas
- First seen
- 18 November 2026
- Reposted
- Not since September 2026
- Last checked
- Today · still on the employer's board
All reasons
In its favour
- Posted today
- Still listed on the employer's own board
We are seeking an experienced Technical Data Consultant with strong end-to-end expertise in architecting, building, and optimizing cloud-native data platforms and analytical ecosystems. The ideal candidate will possess deep hands-on proficiency in Azure Databricks, Apache Spark, PySpark, Snowflake, and Azure Data Factory (ADF), alongside modern orchestration and workflow automation frameworks such as Apache Airflow, Databricks Workflows, and Microsoft Fabric Data Pipelines.
You will lead the design of scalable data Lakehouse and medallion architectures (Bronze, Silver, Gold), enterprise data warehouses, and dimensional data models while driving enterprise integration across REST APIs, GraphQL, and event-driven systems. A critical part of the role involves partnering with business stakeholders to deliver actionable business intelligence via Power BI, Microsoft Power BI Service, and Tableau ensuring enterprise-grade security, observability, CI/CD automation, and high performance throughout.
Job Responsibilities
- Data Engineering & Pipeline Optimization: Design, develop, and optimize high-throughput data pipelines using Azure Databricks, Apache Spark, PySpark, SQL, Snowflake, and Azure Data Factory (ADF) to ingest and transform large volumes of structured and unstructured data.
- Lakehouse & Data Architecture: Architect scalable cloud data platforms leveraging dimensional modelling, enterprise data warehousing, data lakehouse principles, and medallion architecture (Bronze, Silver, Gold) tailored to energy and commodity trading clients.
- Orchestration & Workflow Automation: Implement reliable end-to-end data pipeline orchestration, scheduling, and workflow automation using Apache Airflow, Azure Data Factory, Databricks Workflows, and Microsoft Fabric Data Pipelines.
- Enterprise System Integration: Design and oversee seamless integrations across enterprise systems, trading platforms, and applications using REST APIs, GraphQL, event-driven architectures, and message-based integration patterns.
- Business Intelligence & Semantic Modelling: Develop enterprise dashboards, analytical reports, and semantic models using Power BI, Microsoft Power BI Service, and Tableau leveraging DAX, Power Query, and advanced calculations for executive consumption.
- DevOps, CI/CD & Containerization: Establish automated deployment pipelines and data platform management practices using Azure DevOps, GitHub, and CI/CD workflows, alongside containerized workloads with Docker and Kubernetes.
- Security, Governance & Observability: Implement identity and access security using Microsoft Entra ID and maintain proactive platform monitoring, logging, and observability using Azure Monitor, Log Analytics, and Application Insights.
- Technical Leadership & Mentorship: Provide technical leadership across client engagements, mentor data engineers, conduct code and architecture reviews, and enforce engineering best practices.
- Client Advisory & Problem Solving: Collaborate closely with business and technical stakeholders to translate complex domain requirements such as market volatility, risk management, and regulatory compliance into future-proof data solutions.
Required Skills & Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related technical discipline.
- A total of 5–8 years of hands-on experience in Data Engineering, including a minimum of 2 years in a Technical Lead or Senior Leadership role, as demonstrated by the candidate’s professional work history.
- Strong, proven experience with Azure Databricks, Apache Spark, PySpark, Snowflake, and Azure Data Factory (ADF), alongside deep proficiency in Python and SQL.
- Strong understanding of data modelling, dimensional modelling, data lakehouse architecture, medallion architecture (Bronze, Silver, Gold), pipeline optimization, and enterprise data warehousing.
- Hands-on experience implementing data pipelines, orchestration, and workflow automation using Apache Airflow, Azure Data Factory, Databricks Workflows, and Microsoft Fabric Data Pipelines.
- Experience integrating enterprise systems and applications using REST APIs, GraphQL, event-driven architectures, and message-based integration.
- Experience developing enterprise dashboards, semantic models, and reporting solutions using Power BI, Microsoft Power BI Service, and Tableau (including DAX and Power Query).
- Practical experience with Azure DevOps, GitHub, and CI/CD pipelines for automated deployment and data platform lifecycle management.
- Working knowledge of containerization and orchestration technologies such as Docker and Kubernetes.
- Knowledge of Microsoft Entra ID, Azure Monitor, Log Analytics, and Application Insights for enterprise security, monitoring, and observability.
- Exceptional analytical, problem-solving, communication, and stakeholder management skills.
- Preferred / Optional Domain Advantage: Experience with energy and commodity trading systems (ETRM/CTRM), including integration with trading, risk management, market data, and financial systems, would be a distinct advantage
Skills
- Azure Databricks
- Spark
- PySpark
- Snowflake
- Azure Data Factory
- Airflow
- Databricks
- Microsoft Fabric
- GraphQL
- Power BI
- Tableau
- SQL
- DAX
- Azure DevOps
- GitHub
- Docker
- Kubernetes
- Microsoft Entra ID
- Azure Monitor
- Azure Log Analytics
- Azure Application Insights
- Python
More jobs at Value Creed
All 8- Senior Database AdministratorValue Creed · Hyderabad14d ago
- Consultant/Senior Consultant - Functional - SalesforceValue Creed · Hyderabad15d ago
- ETRM - Technical ArchitectValue Creed · Hyderabad2mo ago
- Senior Developer - AIValue Creed · Hyderabad2mo ago
- Consultant/Senior Consultant - Technical - SalesforceValue Creed · Hyderabad2mo ago
Similar roles
- DA Senior Data EngineerLabcorp · Bengaluru, KA, Indiatoday
- Data Scientist IIIMX Build Technologies India Private · Chennai, Tamil Nadu, Indiatoday
- Senior GCP Data Engineer (R-19655)Dun & Bradstreet · Hyderabad - Indiatoday
- Senior Manager, Data and Analytics ServicesLabcorp · Bengaluru, KA, Indiatoday
- Associate Data AnalystLabcorp · Bengaluru, KA, Indiatoday
- Senior Database AdministratorEnsono · Bengaluru, India · Chennai, India · Hyderabad, India +1today