Scribd
Upload
5 views2 pages

Dsbda 2

The document outlines a series of commands executed in a Hadoop environment, including creating directories, changing permissions, and compiling Java classes for a MapReduce job. It details the process of converting an access log to CSV format, setting up the classpath, and running a Hadoop job to process the data. The final steps involve uploading input data to HDFS and executing the MapReduce job to generate output.

Uploaded by

piyushb960
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views2 pages

Dsbda 2

The document outlines a series of commands executed in a Hadoop environment, including creating directories, changing permissions, and compiling Java classes for a MapReduce job. It details the process of converting an access log to CSV format, setting up the classpath, and running a Hadoop job to process the data. The final steps involve uploading input data to HDFS and executing the MapReduce job to generate output.

Uploaded by

piyushb960
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Pract 2

commands-
su - hduser
51 sudo mkdir analyzelogs
52 ls
53 sudo chmod -R 777 analyzelogs/
54 cd
55 ls
60 pwd
61 sudo chown -R hduser analyzelogs/
71 sudo cp /home/it/Desktop/count_logged_users/* ~/analyzelogs/
72 cd analyzelogs/
74 ls
75 pwd

convert accesslog.txt to .csv

rows 2, separated by other -

77 ls -ltr
78 ls -al
80 sudo chmod +r *.*
81 export CLASSPATH="$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-
core-2.9.0.jar:$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-common-
2.9.0.jar:$HADOOP_HOME/share/hadoop/common/hadoop-common-
2.9.0.jar:~/analyzelogs/SalesCountry/*:$HADOOP_HOME/lib/*"
82 javac -d . SalesMapper.java SalesCountryReducer.java SalesCountryDriver.java
87 sudo gedit Manifest.txt
88 jar -cfm analyzelogs.jar Manifest.txt SalesCountry/*.class
93 start-all.sh
95 sudo mkdir ~/input2000
98 sudo cp access_log_short.csv ~/input2000/
99 $HADOOP_HOME/bin/hdfs dfs -put ~/input2000 /
100 $HADOOP_HOME/bin/hadoop jar analyzelogs.jar /input2000 /output2000
101 $HADOOP_HOME/bin/hdfs dfs -cat /output2000/part-00000
output-

You might also like