Hadoop Administrator

Job Title

Senior Software Engineer

Experience

8 – 11 years

Location

SFO/ Houston

Primary Skills

Set up production Hadoop clusters with optimum configurations

Drive automation of Hadoop deployments, cluster expansion and maintenance operations

Manage Hadoop cluster, monitoring alerts and notification

Deployment of upgrades, updates and patches

Provide 24x7 tier-3 troubleshooting and break-fix support for production services

Diagnosis of installation & configuration issues

Diagnosis of cluster management issues

Diagnosis of performance issues

Job scheduling, monitoring, debugging and troubleshooting

Monitoring and management of the cluster in all respects, notably availability, performance and security

File system management and monitoring

Data transfer between Hadoop and other data stores (incl. relational database)

Ingesting additional data sources into Hadoop in either streaming or batch mode

Commissioning and decommissioning worker nodes

Configuring the cluster to be rack-aware

Set up High Availability/Disaster Recovery environment

Debug/Troubleshoot environment failures/downtime

Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users

Set up security configurations in Hadoop

Performance tuning of Hadoop clusters and Hadoop Map Reduce routines

Manage and review Hadoop log files

Engineering Graduate, MCA, Masters/Post Graduates (preferably IT/ CS)