Hadoop Administrator Loc: Sacramento, CA
100,000 - 200,000
Job Description:
- Install, configure, and upgrade Hadoop components in the Horton works HDP 2.x platform.
- Troubleshoot cluster issues, and support developers running Map Reduce or Tez jobs in Pig and Hive.
- Proactively optimize and tune cluster configuration for performance.Organize, document, and improve process around data inventory management for 100+ TB of data.
- Monitor Hadoop cluster with Ambari, Nagios, and GangliaManage cluster resources and multiple concurrent workloads to maintain availability and SLAs.
- Write Linux Shell scripts for automating tasksCreate scripted workflows and work with Pig/Hive developers to develop automated production ETL processesImplement cluster backups and manage production data deployments between Data Centers.
- Implement and maintain security policies in the Hadoop & Linux environmentResearch latest developments in the Hadoop open source platform and recommend solutions and improvements.
- Evaluate tools and technologies to improve cluster performance and ETL processing.Install, configure, and administrate Linux cloud based servers running in Amazon Web Services.
- Create documentation on cluster architecture, configuration, and best practices.
- Work with Hortonworks support team to investigate and resolve tickets for cluster or developer issues.
- Opportunity to contribute in Hadoop architecture, Hive ETL development, and data QA, depending on candidate background.
- BS in Computer Science or related field, or equivalent experience.
- Must have strong hands on experience with Hadoop administration using Ambari and other tools.
- Must be intimately familiar with the many configuration options available to Hadoop admins, and their impact on cluster performance and stability.
- Strong Linux administration and shell scripting experience.
- Understanding of HDFS, Distributed Hadoop 2.x architecture, Yarn, and Map Reduce concepts.
- Development experience with Pig, Hive, or Spark a huge plus.
- Experience with real-time streaming workflows using Kafka, Spark, or Storm a plus.
- Development experience with Java, Python, or other general programming languages a plus.
- Experience with AWS or other cloud service providers is a plus.
- Responsible team member able to take ownership to independently drive projects and resolve issues without task micro management.
- Ability to quickly learn and a passion to experiment and stay current with latest technology trends and best practices.
- Possess strong communication and collaboration skills.