Senior Data Engineer

Overview

We are looking for a seasoned Senior Data Engineer to work with our team and our clients to develop enterprise grade data platforms, services, and pipelines in an AI-enabled software development environment. We are looking for more than just a "Senior Data Engineer". We are seeking a technologist with excellent communication and customer service skills and a passion for data and problem solving.

Responsibilities

• Lead and architect migration of data environments with performance and reliability
• Assess and understand the ETL jobs, workflows, BI tools, and reports
• Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products
• Experience in crafting database / data warehouse solutions in cloud (Preferably AWS. Alternatively Azure, GCP).
• Key must have skill sets – Python, AWS
• Support an Agile software development lifecycle
• You will contribute to the growth of our Data Exploitation Practice!

Qualifications

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor's Degree and 8+ years of experience.
• 5-7 years industry experience coding commercial software and a passion for solving complex problems.
• 5-7 years direct experience in Data Engineering with experience in tools such as:
• Big data tools: Hadoop, Spark, Kafka, etc.
• Relational SQL and NoSQL databases, including Postgres and Cassandra.
• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• AWS cloud services: EC2, EMR, RDS, Redshift (or Azure equivalents)
• Data streaming systems: Storm, Spark-Streaming, etc.
• Search tools: Solr, Lucene, Elasticsearch
• Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Experience with Databricks.
• Advanced working SQL knowledge and experience working with relational databases, query authoring and optimization (SQL) as well as working familiarity with a variety of databases.
• Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Experience manipulating, processing, and extracting value from large, disconnected datasets.
• Experience manipulating structured and unstructured data for analysis.
• Experience constructing complex queries to analyze results using databases or in a data processing development environment.
• Experience with data modeling tools and process.
• Experience architecting data systems (transactional and warehouses).
• Experience aggregating results and/or compiling information for reporting from multiple datasets.
• Experience working in an Agile environment.
• Experience supporting project teams of developers and data scientists who build web-based interfaces, dashboards, reports, and analytics/machine learning models.
• Databricks experience preferred.