Apache Hadoop 2.x Includes the following Modules:

• Hadoop Common: The common utilities that support the other Hadoop modules.
• Hadoop Distributed File System (HDFS): A distributed file system that provides high-throughput access to application data.
• Hadoop YARN: A framework for job scheduling and cluster resource management.
• Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

1. Download Latest Apache Hadoop Stable Release:

   Apache Hadoop Stable tar.gz

   Apache Hadoop.

2. Double-Click on Archive to Extract

3. Open Terminal Window
   (Press “Enter” to Execute Commands)

   

4. Relocate Apache Hadoop Directory

   rm $HOME/Downloads/hadoop*tar.gz

   sudo mv $HOME/Downloads/hadoop* /usr/local/hadoop

   sudo mkdir /usr/local/hadoop/tmp

   sudo chown -R root:wheel /usr/local/hadoop*

5. Check if Java JDK 7+ is Installed

   java --version

   How-to Install Required Oracle JDK 7+ on MacOS X:

   Install Oracle JDK 7+ for Mac

   Setup Oracle Jdk 7+ on Mac

6. Set JAVA_HOME in Hadoop Env File

   sudo su

   If Got “User is Not in Sudoers file” then Look: Solution

   mkdir /usr/local/hadoop/conf

   nano /usr/local/hadoop/conf/hadoop-env.sh

   Append:

   export JAVA_HOME=$(/usr/libexec/java_home)
   

   Ctrl+x to Save & Exit 🙂

7. Configuration for Pseudo-Distributed mode

   nano /usr/local/hadoop/conf/core-site.xml

   The Content Should Look Like:

    
    
    
    
    hadoop.tmp.dir
    /usr/local/hadoop/tmp
    
    
    fs.default.name
    hdfs://localhost:8020
    
    
   

   Next:

   nano /usr/local/hadoop/conf/hdfs-site.xml

   The Content Should Look Like:

    
    
    
    
    dfs.replication
    1
    
    
    
    dfs.name.dir
    /usr/local/hadoop/cache/hadoop/dfs/name
    
    
   

   Last:

   nano /usr/local/hadoop/conf/mapred-site.xml

   The Content Should Look Like:

    
    
    
    
    mapred.job.tracker
    localhost:8021
    
    
   

8. SetUp Local Path & Environment

   exit

   cd $HOME

   nano .profile

   Inserts:

   HADOOP_HOME=/usr/local/hadoop
   export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
   

   The JAVA_HOME is Set Following Oracle Java JDK6+ Installation Version…

   Then Load New Setup:

   source $HOME/.profile

9. SetUp Needed Local SSH Connection
   Enable SSH Connection:

   System Preferences >> Sharing

   
   To Enable SSH Login without Pass:

   ssh-keygen -t rsa

   Press enter for each line…

   cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

   chmod og-wx ~/.ssh/authorized_keys

   Testing Connection:

   ssh 127.0.0.1

   exit

10. Formatting HDFS

    cd /usr/local/hadoop

    sudo su

    bin/hadoop namenode -format

    

11. Starting Up Hadoop Database

    sbin/start-all.sh

12. Apache Hadoop Database Quick-Start Guide:

    Hadoop MapReduce Quick-Start

    The Linked Guide Contains a Step-by-Step Jump Start Guide for Apache Hadoop Database.

Eclipse Hadoop 2.X Integration with Free Plugin:

Hadoop 2.X Eclipse Plugin SetUp
SetUp Eclipse Hadoop Plugin.