Data Engineer

Data Engineer

Location

Charlotte, NC

Experience

10+ Years

Work Arrangement

Day 1 Onsite (Local)

Position IDs

114068-20-1 and 114068-21-1

In-Person is mandatory

Key Responsibilities

• Optimize data pipelines that process data from Amazon Kinesis to Oracle, improving exception handling for schema and data type mismatches and implementing data quality alerts.
• Develop services to read and write data to NetApp Object Storage using S3 APIs.
• Create analytical datasets in Parquet and Apache Iceberg formats.
• Develop housekeeping scripts for log cleanup, archiving, compression, and object storage file management.

Required Skills

• Strong expertise in Python, Java, and Data Engineering.
• Hands-on experience building and deploying data pipelines using Apache Spark, Apache Flink, and distributed data processing frameworks.
• Experience with real-time streaming architectures, REST APIs, and database connectors.
• Proficiency in relational databases, particularly Oracle.
• Experience with Amazon S3 APIs for data storage and integration.

Preferred Skills

• Experience with Apache Iceberg.