Data Engineer & Insights Analyst

Job Summary

The Commonwealth of Pennsylvania is seeking a skilled professional to join its analytics team to support the Pennsylvania Longitudinal Data System (PALDS). The role involves transforming complex data into reliable, actionable information to enhance program delivery, operational improvements, and decision-making within state government agencies. This position requires an understanding of policy-relevant research questions, evaluating data sources, and developing analytics products to provide actionable insights.

Responsibilities

• Create and maintain optimal data pipeline architecture.
• Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Build and maintain data orchestration workflows using tools like Apache Airflow and AWS Step Functions.
• Assemble large, complex data sets meeting business requirements.
• Engineer analytical data environments for advanced statistical analysis, predictive modeling, and evidence-building activities.
• Partner with data scientists and analysts to ensure data assets support reproducible research and modeling.
• Identify, design, and implement internal process improvements.
• Build infrastructure for optimal data extraction, transformation, and loading from various sources using SQL and 'big data' technologies.
• Develop dashboards, reports, analytical models, and decision-support products.
• Ensure compliance with data governance standards, privacy requirements, and security controls.

Required Qualifications

• Advanced SQL knowledge and proficiency with relational databases.
• Experience building and optimizing 'big data' pipelines, architectures, and data sets.
• Advanced proficiency in designing dashboards, reports, and analytical models.
• Experience with student-level higher education data and establishing linkages to other sources.
• Strong analytic skills for working with unstructured datasets.
• Experience in data transformation, data structures, and workload management.
• Proficiency with cloud-based data services such as AWS.
• Proficiency in scripting languages like Python, R, Java, C++, Scala.
• Experience with version control systems like Git and CI/CD pipelines.
• Knowledge of data modeling techniques and data governance frameworks.
• Experience with big data tools such as Hadoop, Spark, Kafka.
• Experience with data pipeline and workflow management tools.
• Experience with data integration services from vendors like Informatica, MuleSoft, Talend, and TIBCO.

Location

Role is remote - however final interview is onsite in Harrisburg PA. Harrisburg, the capital city of Pennsylvania, offers a rich history and vibrant culture. It is a hub for government and business activities in the state, providing a dynamic environment for professionals.