These instructions are for CentOS

1. Install java (JDK 6.0 or later)

For multi node cluster, install java on all nodes.

2. This step is required only if you need to setup multi node cluster
Modify hosts file on each node to add ip and host name of other nodes that will be
part your cluster

3. From the home directory, generate ssh key on each node using below command
ssh-keygen -t rsa (Same for all the nodes) or

4. Copy public key to all other nodes in cluster, i.e. from N1 copy it to N2, N3 and N1,
then from N2 copy it to N1, N3 and N2, and continue this of all nodes in cluster

5. Download and install rpm "cloudera-cdh-4-0.x86_64.rpm "

6. Install job tracker (install this on N1 if multi node cluster)

"hadoop-0.20-mapreduce-jobtracker"

7. Install name node (install this on N2 if multi node cluster)

"hadoop-hdfs-namenode "

8. Install data node and task tracker (install this on N3 if multi node cluster)
"hadoop-0.20-mapreduce-tasktracker and hadoop-hdfs-datanode"

9. Update core-site.xml to add below:

<property>
<name>fs.defaultFS</name>
<value>hdfs://namenode-host.company.com/</value>
</property>

<property>
<name>fs.trash.interval</name>
<value>1440</value>
</property>

10. Update hdfs-site.xml to add below:

<property>
<name>dfs.permissions.superusergroup</name>
<value>hadoop</value>
</property>

<property>
<name>dfs.namenode.name.dir</name>
<value>/data/1/dfs/nn,/nfsmount/dfs/nn</value>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>/data/1/dfs/dn,/data/2/dfs/dn,/data/3/dfs/dn</value>
</property>

11. This step is required only if you need setup multi node cluster
Copy core and hdfs site xmls to all nodes in cluster

12. setup directories

Create dir : /data/1/dfs/nn /nfsmount/dfs/nn - on Name node
chown -R hdfs:hdfs /data/1/dfs/nn /nfsmount/dfs/nn - on Name
node
chmod 700 /data/1/dfs/nn /nfsmount/dfs/nn - on Name node

Create dir : /data/1/dfs/dn /data/2/dfs/dn /data/3/dfs/dn

/data/4/dfs/dn - on Data node(s)
chown -R hdfs:hdfs /data/1/dfs/dn /data/2/dfs/dn
/data/3/dfs/dn /data/4/dfs/dn - on Data node(s)

13. hdfs hadoop namenode -format

14. Update mapred-site.xml to add below:

<property>
<name>mapred.job.tracker</name>
<value>jobtracker-host.company.com:8021</value>
</property>

<property>
<name>mapred.local.dir</name>
<value>/data/1/mapred/local,/data/2/mapred/local,/data/3/mapre
d/local</value>
</property>
15. Setup maperd properties
Create dir: /data/1/mapred/local /data/2/mapred/local
/data/3/mapred/local /data/4/mapred/local
chown -R mapred:hadoop /data/1/mapred/local
/data/2/mapred/local /data/3/mapred/local
/data/4/mapred/local

16. Create necessary directories in hdfs

hadoop fs -mkdir /tmp
hadoop fs -chmod -R 1777 /tmp
hadoop fs -mkdir -p /var/lib/hadoop-
hdfs/cache/mapred/mapred/staging
hadoop fs -chmod 1777 /var/lib/hadoop-
hdfs/cache/mapred/mapred/staging
hadoop fs -chown -R mapred /var/lib/hadoop-hdfs/cache/mapred
hadoop fs -mkdir /tmp/mapred/system
hadoop fs -chown mapred:hadoop /tmp/mapred/system

17. Start/Restart all components: NN, TS, JT