Staff Engineer (Cloudera CDP & Hadoop Administration)

Requirements

• Experience : 5.5+ years
• Hands-on Tier-2 production experience in Cloudera CDP and Hadoop administration.
• Strong experience installing, configuring, maintaining, and troubleshooting Cloudera CDP environments.
• Experience with Cloudera CDP 7.1.9 or later and Cloudera Data Services, including versions 1.5.4 and 1.5.5.
• Strong understanding of Hadoop ecosystem components, particularly Spark, Hive, and YARN.
• Proven experience troubleshooting Spark, Hive, and YARN jobs in production environments.
• Experience optimizing Cloudera cluster performance, resource utilization, and workload execution.
• Strong understanding of CDP security configuration and implementation of security and regulatory requirements.
• Hands-on experience with automation using Ansible, Shell scripting, Terraform, Groovy, and Python.
• Experience building and managing automated CI/CD pipelines and workflows using GitHub Actions, Jenkins, ArgoCD, or equivalent tools.
• Good understanding of GitOps practices and source control using Git.
• Experience with containerization technologies such as Podman, Docker, Kubernetes, and OpenShift/OCP.
• Knowledge of Infrastructure as Code (IaC) concepts and automated infrastructure provisioning.
• Experience implementing observability and monitoring for data platforms and production environments.
• Experience automating security, compliance, regulatory checks, and patching activities.
• Strong Linux administration and basic networking knowledge.
• Experience working in Agile engineering and DevOps-oriented environments.
• Strong troubleshooting, analytical, and problem-solving skills with the ability to support critical production data platforms.
• Ability to collaborate effectively with data engineers, infrastructure teams, security teams, and other technical stakeholders.

Responsibilities

• Administer, install, configure, maintain, and support Cloudera CDP and Hadoop environments in Tier-2 production environments.
• Perform installation, configuration, upgrades, patching, and ongoing maintenance of the latest supported Cloudera CDP versions, including CDP 7.1.9 or later.
• Configure and implement security requirements across Cloudera CDP environments, including automated security and regulatory compliance checks.
• Configure and maintain Cloudera Data Services, including versions 1.5.4, 1.5.5, and later supported releases.
• Troubleshoot and resolve production issues related to Spark, Hive, and YARN jobs.
• Monitor and optimize cluster health, resource utilization, capacity, and overall CDP performance.
• Work closely with data engineering teams to troubleshoot and optimize Spark job performance and resource utilization.
• Implement automation for cluster administration, operational processes, security checks, patching, and data/analytics workloads.
• Develop automation scripts and infrastructure workflows using Shell, Ansible, Terraform, Groovy, and Python.
• Design and maintain CI/CD pipelines and automated workflows for data projects using tools such as GitHub Actions, Jenkins, and ArgoCD.
• Implement GitOps-based deployment and operational practices for data and analytics platforms where applicable.
• Support containerized Cloudera Data Services using technologies such as Podman, Docker, Kubernetes, and OpenShift Container Platform (OCP).
• Automate infrastructure provisioning and configuration using Infrastructure as Code (IaC) practices.
• Implement and maintain observability solutions for Cloudera clusters, data services, jobs, and infrastructure components.
• Perform automated patching and support lifecycle management activities across the platform.
• Collaborate with engineering, data, infrastructure, security, and operations teams to resolve production issues and improve platform reliability.
• Maintain source code, automation scripts, configurations, and deployment artifacts using Git-based source control.
• Follow Agile engineering practices and contribute to continuous improvement, operational automation, and platform standardization.

Bachelor’s or master’s degree in computer science, Information Technology, or a related field.