Lead AWS Data Engineer

Job Title

Lead AWS Data Engineer

Location

Miami, Florida or Irving, Texas (Hybrid)

Duration

Contract to Hire

A Career that Empowers You to Build Your Future

The Lead Data Engineer is responsible for overseeing the data engineering team and driving the strategic direction of data initiatives. The role involves managing projects, ensuring the scalability of data solutions, and collaborating with stakeholders to meet organizational goals.

Your Responsibilities on the Team

• Lead and manage the data engineering team, including project planning, resource allocation, and performance management.
• Develop and enforce data engineering standards, processes, and best practices, ensuring consistency and quality across all data projects.
• Collaborate with stakeholders, including business leaders, data scientists, and IT teams, to understand and prioritize data-related business needs and translate them into actionable technical requirements.
• Ensure data solutions are scalable, secure, and aligned with the long-term goals of the organization, considering both current and future data needs.
• Drive the adoption of new technologies, tools, and methodologies that improve the efficiency, reliability, and scalability of data engineering practices.
• Oversee the development and maintenance of comprehensive data documentation, including data architecture diagrams, process flows, and technical specifications.
• Manage the team's technical debt, ensuring that legacy systems are modernized and that new solutions are built with maintainability and scalability in mind.
• Provide technical leadership in the resolution of complex data issues, guiding the team in troubleshooting and problem-solving efforts

Knowledge

• Extensive experience with data pipeline orchestration tools (e.g., Apache Airflow, Luigi) and data integration platforms (e.g., Apache NiFi, Glue).
• Strong leadership and project management skills, with experience leading cross-functional teams and managing multiple projects simultaneously.
• Expertise in cloud data platforms (AWS) and big data ecosystems, with a focus on optimizing cost, performance, and scalability.
• Proficient in advanced data modeling, including the design of enterprise data warehouses, data lakes, and data marts.
• In-depth knowledge of data governance practices, including data cataloging, data lineage, and data quality management.
• Experience with enterprise data management frameworks, such as Master Data Management (MDM) and metadata management.
• Advanced skills in programming and scripting languages (e.g., Python, Java, Scala) for developing and automating data workflows.
• Experience with real-time data processing and event-driven architectures, leveraging tools such as Apache Kafka, Spark Streaming, and AWS Lambda.

Your Toolbox

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 7-10 years of experience in data engineering, with significant leadership and project management experience.
• AWS, Snowflake, Azure
• AWS Glue, AWS AppFlow, Prefect, dbt, S3, Qlik
• SQL, Python, Java
• Exceptional ability to influence and engage senior leaders with data-driven insights.
• Expert in resolving complex, high-stakes problems with innovative solutions.
• Leads with a collaborative spirit, driving team success through shared goals.
• Sets the standard for integrity within the organization, fostering a culture of trust and ethical behavior.
• Demonstrates unwavering resilience in the face of adversity, guiding the team through challenges.
• Acts as a change agent, leading transformative initiatives that drive business success.
• Leads and coordinates efforts across multiple departments, ensuring alignment with organizational goals.
• Provides visionary leadership, inspiring and developing future leaders within the organization.
• Establishes a culture of accountability, ensuring that all team members meet their commitments and deliver high-quality work.