JobHabor
Menu

Senior Infrastructure Engineer – Terraform & Databricks

Apptad · San Francisco, CA, United States · Hybrid

Weak hiring record · median 66 days open

100100

Hiring signal

Actively hiring

Posted today · Never reposted · Checked today · How we score

First seen
18 November 2026
Reposted
Not since September 2026
Last checked
Today · still on the employer's board
All reasons
  • 55% of this employer's openings are over 60 days old

In its favour

  • Posted today
  • Still listed on the employer's own board

Did you apply?

Senior Infrastructure Engineer – Terraform & Databricks at Apptad. We’ll keep it in your applied jobs, so you can see it later and be warned before applying to the same employer twice.

Senior Infrastructure Engineer – Terraform & Databricks

San Francisco, CA

Need candidates who can work onsite from Day 1 (3 days a week : Tuesday, Wednesday and Thursday – 3 days work from office) – NO REMOTE

Highly preferred local candidates

Experience: 09+ years

About the role

Building an enterprise Databricks Lakehouse on AWS, and all of it is managed as code. In this role you'll design, build and run the Terraform/Terragrunt foundation behind our Databricks workspaces, Unity Catalog, compute and security controls.

You'll work closely with Cloud Infrastructure/SRE, Security Engineering and Data Engineering. Your goal is to take our platform from sandbox to production-ready multi-environment setup that is secure, well governed and cost-efficient.

What you will do

  • Design, build and maintain reusable Terraform modules for Databricks account-level (MWS) and workspace-level resources. This covers workspaces, Unity Catalog (metastores, catalogs, schemas, volumes, grants), cluster policies, instance pools, SQL warehouses, jobs, DLT pipelines and service principals.
  • Use Terragrunt to manage environments consistently across sandbox, nonprod, preprod and prod, with minimal configuration drift.
  • Provision and secure the supporting AWS infrastructure: customer-managed VPCs, PrivateLink/VPC endpoints, security groups, NACLs, IAM roles and policies, S3, KMS customer-managed keys (CMK) and Secrets Manager.
  • Lead security hardening. That means fixing findings from security architecture reviews and cloud security posture management (CSPM) tools such as Wiz, enforcing PrivateLink-only access, IP access lists, CMK encryption and secret/OAuth token rotation.
  • Set up identify and access controls, including Okta SSO and SCIM sync, role-based (RBAC) and tag-based (ABAC) access in Unity Catalog, and least-privilege service principals.
  • Build and own CI/CD for infrastructure. This includes GitHub Actions plan/apply workflows with approvals, pre-commit hooks, Terratest, and security scanning with Checkov or tfsec.
  • Set up monitoring and alerting with CloudWatch, Grafana and Splunk log forwarding. That covers cluster and job health, token expiry and cost anomalies.
  • Drive FinOps practices: cluster policy guardrails, spot usage, moving work-loads from all-purpose to job clusters, enforcing tagging, and analysing DBU usage.
  • Enable integrations with Dagster, dbt Cloud and BI tools such as Tableau, Power BI and Looker via SQL warehouses.
  • Write and maintain architecture docs, ADRs, runbooks and module references, and enforce naming and tagging standards.
  • Databricks – Hands-on build data pipelines using Pyspark
  • Used Agentic IDE to build data pipelines
  • Understands Medallion architecture
  • Understands ML flow – Model inferencing
  • Have domain expertise or understanding of Credit decisioning or Risk or Servicing (collections)

Skills Required

  • 7+ years in infrastructure, platform or DevOps engineering with atleast 3 years of hands-on Terraform in production. Terragrunt experience is strongly preferred.
  • 2+ years running Databricks on AWS, including workspace provisioning, Unity Catalog, cluster policies and the Databricks Terraform provider.
  • Deep AWS knowledge; VPC networking, PrivateLink, IAM, KMS, S3, Secrets Manager and CloudWatch.
  • Experience with CI/CD for Infrastructure as code (GitHub Actions, Jenkins or similar) and with GitOps workflows.
  • Strong grasp of cloud security and compliance: encryption, network isolation, least privilege and audit logging. Experience in regulated environments such as financial services is a plus.
  • Scripting skills in Python and/or Bash. Go is a plus for Terratest.
  • Comfortable with Git, Code reviews and writing clear technical documentation.

Nice to have

  • Databricks certifications (Data Engineer Associate/Professional or Platform Administrator), or HashiCorp Terraform Associate.
  • Experience with Dela Live Tables, Vector Search, Model Serving or MLops Infrastructure.
  • Familiarilty with Dagster, Dbt, Spark and Delta Lake.
  • Experience with Wiz, Checkov, tfsec or similar policy-as-code and CSPM tools.
  • Background in FinOps or cloud cost optimization.
  • Experience with SOC 2 or similar compliance frameworks

Skills

More jobs at Apptad

All 278

Similar roles