Hadoop Admin responsibilities

Hadoop Admin Responsibilities:

• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Aligning with the system engineering team to propose and deploy new hardwares and software environments required for Hadoop and to expand existing environments.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users.
• Cluster maintenance as well as creation and removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise, Dell Open Manage and other tools.
• Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee with high data quality and availability.
• Performance tuning of Hadoop clusters and Hadoop MapReduce routine.
• Monitor Hadoop cluster and their connectivity and security
• Manage and review Hadoop log files.
• File system management and monitoring.
• HDFS support and maintenance.
• Point of Contact for Vendor escalation
• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades when required.
• Screen Hadoop cluster job performance and capacity planning

DBA Responsibilities Performed by Hadoop Administrator:

• Disk space management.
• Data modelling, design & implementation based on recognized standards.
• Software installation and configuration.
• Database connectivity and security.
• Performance monitoring and tuning.
• Database backup and recovery.
• Automate manual tasks.
• Software patches and upgrades.

DWH Development Responsibilities Performed by Hadoop Administrator:

• Accomplish data restatements.
• Perform primary key execution.
• Load large data volumes in a timely manner.
• Ensure Referential integrity.