Scribd
Upload
8 views3 pages

Ap21110010351 4

Uploaded by

varumalla43
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
8 views3 pages

Ap21110010351 4

Uploaded by

varumalla43
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

CLOUD COMPUTING LAB 318L

ASSIGNMENT IV

T.Naga Abhiram
AP21110010404
Explain each step with commands and proper screenshots on how to install and
configure Hadoop on your virtual machine.

• First install Java in the VM. [sudo apt install openjdk-8-jdk]

• Run this command and at the last paste the following commands. [sudo nano
.bashrc]

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$PATH:/usr/lib/jvm/java-11-openjdk-amd64/bin
export HADOOP_HOME=~/hadoop-3.3.6/
export PATH=$PATH:$HADOOP_HOME/bin export
PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export
HADOOP_STREAMING=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-strea
ming-3.3.6.jar
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
export PDSH_RCMD_TYPE=ssh

Because of you are overwriting a file, you have to save it by ‘ctrl + o’. Then ‘ctrl+x’.

• Now run ‘ssh’ command. [sudo apt-get install ssh].


• Now, download the Hadoop tar file in the VM.
• To extract all file after downloading the tar file of the hadoop run the command
tar –zxvf ~/Downloads/hadoop-3.3.4.tar.gz in terminal of vm.
• To fix the path for java in hadoop environment we have the following
commands.

• Run [sudo nano hadoop-env.sh] and set JAVA_HOME =


/usr/lib/jvm/java-11-openjdk-amd64
• Run [sudo nano core-site.xml] and paste the following inside configuration.
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value> </property>
<property>
<name>hadoop.proxyuser.dataflair.groups</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.dataflair.hosts</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.server.hosts</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.server.groups</name> <value>*</value>
</property>

• Run [sudo nano hdfs-site.xml] and paste the following inside configuration.
<property>
<name>dfs.replication</name>
<value>1</value>
</property>

• Run [sudo nano mapred-site.xml] and paste the following inside configuration.
<property>
<name>mapreduce.framework.name</name> <value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_M
APRED_HOME/share/hadoop/mapreduce
/lib/*</value>
</property>

• Run [sudo nano yarn-site.xml] and paste the following inside configuration.
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,H
ADOOP_CONF_DIR,CLASSPATH_P
REP END_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</val
ue>
</property>

• For ssh, run [ssh localhost].


• Then [ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub >>
~/.ssh/authorized_keys ]
• Run [chmod 0600 ~/.ssh/authorized_keys]
• Now change the format of namenode, by running
[hadoop-3.3.4/bin/hdfs namenode –format]
• Again format the file system. [export PDSH_RCMD_TYPE=ssh]

• Now to start, run [start-all.sh]

To check whether the hadoop is working correctly or


not,open browser:localhost:9870.

You might also like