Scribd
Upload
95 views8 pages

Industrial Training Report

The document discusses the process of installing Hadoop on an Ubuntu Linux machine. It provides the step-by-step commands needed to install Java, copy the Hadoop files, configure Hadoop settings, and finally start the Hadoop processes. Key components configured include the HDFS, MapReduce and installing Hadoop files from a tar file.

Uploaded by

chetanruparel07aws
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
95 views8 pages

Industrial Training Report

The document discusses the process of installing Hadoop on an Ubuntu Linux machine. It provides the step-by-step commands needed to install Java, copy the Hadoop files, configure Hadoop settings, and finally start the Hadoop processes. Key components configured include the HDFS, MapReduce and installing Hadoop files from a tar file.

Uploaded by

chetanruparel07aws
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 8

INDUSTRIAL TRAINING – I

Partial Fulfilment of
B. Tech – Computer Science and Engineering
In
Industrial Training with I-Medita Learning Solutions Pvt. Ltd
From 22.06.2016 to 06.07.2016
By
CHETAN RUPAREL
Reg. No. : RA1411003010397
Report Submitted to : Mrs. S.P.Kanmani
(Class-In-Charge)
Department of Computer Science and Engineering
SRM University, School of Computing, Faculty of Engineering and
Technology, Kattankulathur – 603203, Kancheepuram Dist., Chennai,
Tamil Nadu
ACKNOWLEDGEMENT
I would like to take this opportunity to thank the Director of E&T, Dr. Muthamizchelvan and
Head of Department of Computer Science and Engineering, Dr. B. Amutha , for making the
Industrial Training a part of the curriculum and possible for the students.

I would like to express my special thanks of gratitude to my teacher Mr. S. Sundar Pandian
and Mr. Venkata Sudhann as well as ibnc and I-Medita team who gave me the golden
opportunity to do this wonderful project on the topic, Big Data Hadoop and Cloud
Computing , which also helped me in doing a lot of Research and I came to know about so
many new things I am really thankful to them.
About The Company
I-Medita, is an ISO 9001:2008 certified Training, development and placement Company registered by Ministry of
Corporate affairs as an Educational Service Provider. I-Medita is also a Cisco Registered Partner.

We help in providing industry oriented skilled training to students and professionals to kick start their career in various
domains. Our efforts are to keep momentum with the Industry technological demands and diversifying universe of
knowledge.
Our AIM:
• Holistic development of Individuals.
• Imparting appropriate skills that enable students to be employable.
I-Medita is a Global firm offering various Educational training services across the globe. We provide Technical Hand on
Training to Colleges, Universities and we also run Training Program at our Pune Main Office.
In last Two-Year of Existence, we have trained more than 16000+ students from various colleges at Pan India Level through
our Unique Hands-on workshops and we are also trusted by more than 200 Colleges for various kinds of Training in their
campuses.
TRAINING DETAILS
Company: I-Medita Learning Solutions Pvt. Ltd.
Branch: Chennai
Technology: Big Data, Hadoop and Cloud Computing
Time Duration: 15 days
Time Period: 22 June 2016 to 06 July 2016
Trainers: Mr. Sundar Pandian and Mr. Venkata Sudyan
TABLE OF CONTENTS
S. No
Title
Page No:
1.
Big Data

1.1 Introduction to Big Data


6

1.2 Installtion on ubuntu 14.04 OS


6

1.3 Map Reduce Programming in Java


10

1.4 Cassandra
15

1.5 MongoDB
17

1.6 Spark
18

1.7 Hive
19

1.8 Pig
23
2.
Cloud Computing using Amazon Web Services

2.1 Virtuaization

24

2.2 Deploying Apache on server


25

2.3 Deploying Web Application


25

2.4 Creating an AMI


26

2.5 Deploying an EBS across different Availability Zones


26
2.6 Deploying an Elastic Load Balancer
26

2.7 Alerts for deployed servers


26

2.8 Auto Scaling


26

2.9 RDS
26

2.10 User Creation with limited access


27

2.11 Using Route 53 to connect the servers with a domain name


27
3.
Bibiliographic References
28
BIG DATA-HADOOP
• Introduction
Data sets are growing rapidly in part because they are increasingly gathered by cheap and
numerous information-sensing mobile devices, aerial (remote sensing), software logs,
cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor
networks. The world's technological per-capita capacity to store information has roughly
doubled every 40 months since the 1980s; as of 2012, every day 2.5 exabytes (2.5×1018) of
data is created .
There is a need to manage all these data efficiently with higher speeds of operation.
Big data is a term for data sets that are so large or complex that traditional data processing
applications are inadequate .
Big data usually includes data sets with sizes beyond the ability of commonly used software
tools to capture, curate, manage, and process data within a tolerable elapsed time. Big data
"size" is a constantly moving target, as of 2012 ranging from a few dozen terabytes to many
petabytes of data. Big data requires a set of techniques and technologies with new forms of
integration to reveal insights from datasets that are diverse, complex, and of a massive scale.
This is the first draft of the presntation

• Installation

The steps which I used for installation on hadoop on ubuntu linux 14.04 are:-
STEP:
1.GO TO VMWARE WORKSTATION.
2.OPEN UPUNDU LINUX MACHINE
3.DRAG AND PLACE THE TERMINAL FROM FILE MENU.
4.GO TO TERMINAL AND TYPE THE FOLLOWING COMMANDS

sudo apt-get update

sudo apt-get install openssh-server


ssh-keygen
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh localhost

sudo apt-get install openjdk-7-jdk

copy hadoop 1.2.1 jar file to home folder of linux

tar -xvzf hadoop-1.2.1-bin.tar.gz

sudo cp -r hadoop-1.2.1 /usr/local/hadoop

sudo nano $HOME/.bashrc

add the following in bashrc file at the end

export HADOOP_PREFIX=/usr/local/hadoop
export PATH=$PATH:$HADOOP_PREFIX/bin
save it

exec bash

sudo vi /usr/local/hadoop/conf/core-site.xml

type inside <configuration>

<property>
<name>fs.default.name</name>
<value>hdfs://192.168.68.128:10001</value>
</property>

<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop/tmp</value
</property>
save it and exit

sudo vi /usr/local/hadoop/conf/hadoop-env.sh

uncomment the below line and change as

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64
export HADOOP_OPTS=-Djava.net.preferIPv4Stack=true

save and exit

sudo vi /usr/local/hadoop/conf/mapred-site.xml

type inside <configuration>

<property>
<name>mapred.job.tracker</name>
<value>192.168.68.128:10002</value>
</property>

save it and exit

sudo mkdir /usr/local/hadoop/tmp

sudo chown pandian /usr/local/hadoop/tmp

sudo chown pandian /usr/local/hadoop

hadoop namenode -format

jps
start-all.sh

jps

You might also like