Data & BI Engineer

Data & BI Engineer

Position Summary

Looking for a highly technical and business-focused Data & BI Engineer to manage, develop, and enhance the organization's enterprise data, reporting, analytics, and automation capabilities.

This role is responsible for owning the end-to-end data lifecycle, from extracting information from source systems and APIs to processing and storing data in MySQL, developing Power BI semantic models and dashboards, and delivering actionable insights to business stakeholders.

The Data & BI Engineer will support multiple business functions, including Finance, Human Resources, Talent Acquisition, Marketing, Operations, Learning & Development, and Shared Services. The position combines data engineering, database development, business intelligence, Python automation, and analytics consulting within client's Microsoft Azure and Microsoft 365 environment.

The successful candidate will work directly with business stakeholders to understand operational needs, define meaningful KPIs, automate manual processes, improve data quality, and build scalable solutions that support informed decision-making.

This is an end-to-end engineering role, not simply a report development position. The ideal candidate will own the journey from source-system integration through data processing, database management, analytics development, application support, production operations, and business insight. The role also includes supporting Linux-based application environments, automated data processing workloads, and business-critical analytics platforms.

Key Responsibilities

Data Engineering & Integration

• Design, develop, and maintain automated ETL/ELT processes to extract, transform, validate, and deliver data from multiple source systems.
• Build and maintain integrations between business applications, APIs, databases, and enterprise reporting platforms.
• Develop scalable and reusable data pipelines using Python, SQL, and appropriate integration technologies.
• Extract data from REST APIs and other supported interfaces, handling authentication, pagination, rate limits, and errors as required.
• Monitor data processing workflows and troubleshoot pipeline failures, integration issues, and data discrepancies.
• Implement logging, error handling, validation, and reconciliation procedures to improve data reliability.
• Maintain data quality, consistency, and traceability from source systems through final reporting outputs.

Power BI Development & Business Intelligence

• Develop and maintain Power BI datasets, semantic models, reports, dashboards, scorecards, and executive reporting solutions.
• Create and optimize DAX measures, calculated columns, Power Query transformations, relationships, and data models.
• Translate business requirements into meaningful KPIs, reporting frameworks, and actionable insights.
• Optimize report performance, model efficiency, data refresh processes, and user experience.
• Manage Power BI Service workspaces, scheduled refreshes, gateway connectivity, and deployment practices as applicable.
• Implement appropriate reporting standards, access controls, row-level security, and governance practices.
• Support self-service analytics while maintaining consistent business definitions, metrics, and reporting standards.
• Investigate and resolve reporting errors, refresh failures, performance issues, and discrepancies between source data and reported results.

MySQL Database Development & Management

• Design, develop, and maintain MySQL database structures supporting enterprise reporting, analytics, and operational data processing.
• Write and optimize complex SQL queries, joins, views, stored procedures, and other database logic as appropriate.
• Develop efficient data models, indexes, and database structures to support reliable and scalable reporting.
• Monitor query performance and troubleshoot database bottlenecks, slow queries, and data integrity issues.
• Maintain data accuracy, consistency, and integrity across reporting databases and integrated systems.
• Support database changes, data migrations, schema enhancements, and reporting platform improvements.
• Document database structures, data relationships, dependencies, and business rules.
• Coordinate with IT on database hosting, connectivity, backup and recovery, access controls, and infrastructure administration as applicable.

Python Automation & Process Improvement

• Develop Python scripts and reusable utilities for data extraction, transformation, validation, reporting, and operational automation.
• Automate repetitive manual processes and reduce reliance on spreadsheet-based or manual reporting workflows.
• Build and maintain API integrations and automated data processing routines.
• Implement appropriate logging, exception handling, retry mechanisms, and error notifications.
• Support scheduled and event-driven automation processes where applicable.
• Identify opportunities to improve operational efficiency, reduce processing time, and eliminate redundant activities.
• Maintain readable, reusable, and well-documented code that can be supported and enhanced over time.

Analytics Consulting & Business Partnership

• Work directly with department leaders and business users to understand reporting needs, operational challenges, and business objectives.
• Gather requirements and translate them into technical specifications, data models, dashboards, and automated workflows.
• Define and document KPIs, calculation methods, reporting rules, and measurement frameworks.
• Analyze operational and business data to identify trends, performance gaps, opportunities, and potential improvements.
• Validate results with stakeholders to ensure reporting outputs accurately reflect business processes.
• Recommend data-driven improvements to workflows, reporting practices, and business operations.
• Support management reporting, strategic planning, and decision-making through timely and reliable analytics.
• Manage competing priorities and communicate project status, technical issues, risks, and delivery timelines.

Microsoft Azure, Microsoft 365 & Platform Support

• Support data integration, reporting, and automation workloads operating within client's Microsoft Azure and Microsoft 365 environment.
• Understand connectivity between source systems, MySQL databases, Power BI, and cloud or on-premises infrastructure.
• Support Power BI Service configuration, gateway connectivity, refresh monitoring, and workspace administration as required.
• Develop or maintain business workflows using Power Automate, SharePoint, Excel Power Query, and related Microsoft 365 tools.
• Support scheduled jobs, platform monitoring, troubleshooting, and operational reliability.
• Work with IT and security teams on identity and access management, permissions, secure connectivity, and compliance requirements.
• Follow established practices for credential protection, version control, testing, deployment, and change management.
• Document system architecture, data flows, configurations, dependencies, and operational procedures.
• Administer and support Linux-based application servers hosting reporting, analytics, integration, and automation workloads.
• Monitor platform health, troubleshoot production issues, review logs, and ensure operational reliability of reporting and analytics solutions.
• Support application deployments, scheduled jobs, configuration changes, and environment maintenance activities across development, QA, and production environments.

Required Technical Skills Advanced Proficiency MySQL & SQL Development

• Advanced SQL querying and complex joins.
• Database design and relational data modeling.
• Query optimization, indexing, and performance tuning.
• Views, stored procedures, and database logic.
• Data integrity, validation, and troubleshooting.

Power BI & Business Intelligence

• DAX and advanced measure development.
• Data modeling and relationship management.
• Power Query and M transformations.
• Semantic model and dataset development.
• Dashboard, scorecard, and executive report development.
• Performance optimization and troubleshooting.
• Power BI Service, refresh management, and workspace administration.

Python & Data Engineering

• Python scripting and data processing.
• API integrations and data extraction.
• ETL/ELT development and transformation workflows.
• Automation scripting and reusable utilities.
• Error handling, logging, validation, and debugging.
• Scheduling and operational support for automated processes.

Intermediate to Advanced Proficiency Microsoft Technology Stack

• Microsoft Azure services relevant to data and reporting workloads.
• Power BI Service and gateway connectivity.
• Power Automate and workflow automation.
• SharePoint and Microsoft 365 integrations.
• Excel, including Power Query.
• Cloud and on-premises data connectivity.
• Identity, access, and security concepts.

Data Engineering & Governance

• ETL/ELT design and implementation.
• Data warehousing and reporting architecture.
• Data quality management and reconciliation.
• Data governance, lineage, and documentation.
• API integration patterns and troubleshooting.
• Scalable data processing and pipeline monitoring.

Development & Infrastructure Practices

• Git or equivalent source control.
• Development, testing, and production deployment practices.
• Scheduled job management and monitoring.
• Linux administration and command-line troubleshooting.
• Cloud infrastructure fundamentals, with Azure preferred.
• Production troubleshooting, recovery, and change management.
• Application monitoring and operational support.
• Incident troubleshooting and root cause analysis.

Soft Skills & Professional Competencies

• Strong analytical thinking and problem-solving skills.
• Excellent written and spoken English communication.
• Ability to communicate technical concepts to nontechnical stakeholders.
• Strong documentation and knowledge-sharing habits.
• Effective requirements gathering and business analysis skills.
• Ability to work independently and manage multiple priorities.
• Strong attention to detail and commitment to data accuracy.
• Collaborative approach when working with business, IT, and security teams.
• Proactive mindset focused on continuous improvement, automation, and operational efficiency.

Experience Requirements Required

• Five or more years of professional experience in data analytics, business intelligence development, data engineering, database development, or a related field.
• Demonstrated hands-on experience developing and maintaining Power BI reports, dashboards, semantic models, and DAX measures.
• Strong practical experience with MySQL and advanced SQL development.
• Experience developing Python scripts for data processing, integration, or automation.
• Demonstrated experience extracting, transforming, validating, and integrating data from multiple sources.
• Experience supporting production reporting solutions and troubleshooting data quality, performance, and refresh issues.
• Experience gathering requirements and translating business needs into technical solutions.
• Experience supporting multiple business functions and working directly with stakeholders.
• Ability to independently deliver technical solutions from initial requirements through implementation and ongoing support.

Preferred

• Experience supporting enterprise-wide reporting and analytics platforms.
• Experience working with Microsoft Azure and Microsoft 365 environments.
• Experience integrating business applications through APIs.
• Experience automating workflows using Power Automate or similar tools.
• Experience with cloud and on-premises data connectivity.
• Experience with data warehouse design, dimensional modeling, and data governance.
• Experience implementing version control, deployment practices, and production monitoring.
• Experience supporting executive reporting and cross-functional business performance measurement.