JobHabor
Menu

AWS PySpark Databricks Developer

Diverse Lynx · NYC, NY · Remote

Strong hiring record · median 5 days open

100100

Hiring signal

Actively hiring

Open 5 days · Never reposted · How we score

First seen
12 November 2026
Reposted
Not since September 2026
All reasons

In its favour

  • Posted 6 days ago
  • Still listed on the employer's own board
  • Someone has already applied through it

Did you apply?

AWS PySpark Databricks Developer at Diverse Lynx. We’ll keep it in your applied jobs, so you can see it later and be warned before applying to the same employer twice.

Bill Rate

MAX CONFIRMED-$65/HR.

Location

REMOTE-EST HOURS

Duration

6 months

Role

Databricks Engineer

Role Descriptions

Terradata to Databricks migration

Essential Skills

Terradata to Databricks migration

Desirable Skills

Keyword

Skills

Digital

Python~Digital

Databricks~Digital

PySpark

Experience Required

8 + Years

CANDIDATE MUST HAVE HANDSON EXP IN DATABRICKS, PYTHON AND PYSPARK

Job Title AWS PySpark Databricks Developer

Relevant Experience

(in Yrs) 5-6 years+

Technical/Functional Skills

  • 5–6 years of total experience in data engineering or big data development.
  • 2–3 years hands-on experience with Databricks and Apache Spark.
  • Proficient in AWS cloud services (S3, Glue, Lambda, EMR, Redshift, CloudWatch, IAM).

FastAPI, GKE, Cloud Run, REST APIs, and Terraform following enterprise DevOps, CI/CD, IA

  • Strong programming skills in PySpark, Python, and Eligible to workionally Scala.
  • Solid understanding of data lakes, lakehouses, and Delta Lake concepts.
  • Experience in SQL development and performance tuning.
  • Familiarity with Airflow, dbt, or similar orchestration tools is a plus.
  • Experience in CI/CD tools like Jenkins, GitHub Actions, or Code Pipeline.
  • Knowledge of data security, governance, and compliance frameworks.

ETL workflows using GCP Dataflow, Snowflake, and BigQuery

Experience Required 5-6 years+

Roles & Responsibilities

  • Develop and maintain scalable data pipelines using Apache Spark on Databricks.
  • Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions.
  • Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.
  • Design and implement data models, schemas, and Lakehouse architecture in Databricks.
  • Eligible to workimize and tune Spark jobs for performance and cost-efficiency.
  • Integrate data from multiple structured and unstructured data sources.
  • Monitor and manage data workflows, ensuring data quality, consistency, and security.
  • Follow best practices in CI/CD, code versioning (Git), and DevOps practices for data applications.
  • Write clean, reusable, well-documented code using Python / PySpark / Scala.

Skills

More jobs at Diverse Lynx

All 135

Similar roles