Ex.

No : 1
Install Virtualbox/VMware Workstation
Date :

AIM:
Find procedure to Install Virtualbox/VMware Workstation with different flavours of linux or
windows OS on top of windows7 or 8.

This experiment is to be performed through portal.

PROCEDURE :
Step 1- Download Link
Link for downloading the software is https://www.vmware.com/products/workstation-pro/workstation-
pro-evaluation.html. Download the software for windows. Good thing is that there is no signup
process. Click and download begins. Software is around 541 MB.

Step 2- Download the installer file

It should probably be in the download folder by default, if you have not changed the settings in your
browser. File name should be something like VMware-workstation-full-15.5.1-15018445.exe. This file
name can change depending on the version of the software currently available for download. But for
now, till the next version is available, they will all be VMware Workstation 15 Pro.

Step 3- Locate the downloaded installer file

For demonstration purpose, I have placed the downloaded installer on my desktop. Find the installer on
your system and double click to launch the application.

Step 4- User Access Control (UAC) Warning

Step 5- VMware Workstation Setup wizard

Now you will see VMware Workstation setup wizard dialog box. Click next to continue.

Step 6- End User Licence Agreement

This time you should see End User Licence Agreement dialog box. Check “I accept the terms in the Li-
cence Agreement” box and press next to continue.

Step 7- Custom Setup options

Select the folder in which you would like to install the application. There is no harm in leaving the de-
faults as it is. Also select Enhanced Keyboard Driver check box.

Step 8- User Experience Settings

Next you are asked to select “Check for Updates” and “Help improve VMware Workstation Pro”. Do as
you wish. I normally leave it to defaults that is unchecked.
Step 9- Application Shortcuts preference
Next step is to select the place you want the shortcut icons to be placed on your system to launch the ap-
plication. Please select both the options, desktop and start menu and click next.

Step 10- Installation begins

Step 11- Launch VMware Workstation

After the installation completes, you should see VMware Workstation icon on the desktop. Double click
on it to launch the application.

Step 12- Licence Key

If you see the dialog box asking for licence key, click on trial or enter the licence key. Then what you
have is the VMware Workstation 15 Pro running on your windows 10 desktop. If don’t have the licence.

Step 13- At some point if you decide to buy

At some point of time if you decide to buy the Licence key, you can enter the Licence key by going
to Help->Enter a Licence Key
You can enter the 25 character licence key in the dialog box shown below and click OK. Now you have
the licence version of the software.
OUTPUT:

Step 1:
Download the VMware from their official website.

Step 2:
Select the desired version.(here it is 14.0)
Step 3:
Select the version.

Step 4:
Click Download Now.
Step 5:
Double Click on the downloaded file. Click Install.
Step 6:
Click Finish.

Step 7:
Select “Use VMware workstation 14 player for free for non-commercial use”. Click Continue.
Step 8:
Click Remind Me Later.

Step 9:
VMware will be launched.
RESULT :

Thus the procedure to Install Virtualbox/VMware Workstation with different flavours of

linux or windows OS on top of windows7 or 8 is implemented.
 Install a C compiler in the virtual machine created using
Ex. No : 2
virtual box and execute Simple Programs
Date :

AIM :
To install a C compiler in the virtual machine created using virtual box and
execute Simple Programs.

PROCEDURE :

Install VirtualBox :
step-1: Visit http://www.virtualbox.org/wiki/downloads.
step-2: Download ,Virtual Box platform packages for your OS.
step-3: Open the Installation Package by double clicking.
step-4: Click continue and finish installing Virtual Box.
step-5: When finished close the window.
Install Linux using Virtual Box :
step-1: Run VirtualBox by double-clicking the icon.
step-2: Click “New” button on the top left corner.
step-3: Choose the amount of memory to allocate (I suggest choosing between 5I2 MB to
I024 MB).
step-4: Click Continue or Next.
step-5: Choose create a new virtual hard disk.
step-6: Click Continue or Next.
step-7: Choose a VDI.
step-8: Click Continue or Next.
step-9: Choose “Dynamically Allocated” click continue.
step-10: This way, the size of your Virtual Hard Disk will grow as you use.
step-11: Click the folder icon and choose the ubuntu iso file you downloaded.
step-12: Select the size of the Virtual Disk (I recommend choosing 8 GB) and click con-
tinue.
step-13: Click create.
Running Linux :
step-1: Choose Ubuntu from left column and click Start and continue.
step-2: Click the folder icon and choose the
ubuntu iso file you
step-3: Check “Download updates” and click Forward.
step-4: Choose “Erase disk and install Ubuntu” and click Forward.
step-5: Click “Install Now” and wait. Maybe grab a snack.When finished, click Restart n
and press Enter.
C Programming on Linux :
step-1: Open Terminal (Applications-Accessories-Terminal).
step-2: Open gedit by typing “gedit &” on terminal.
step-3: Type the following on gedit:

#include<stdio.h>
Main(){
printf(“hello world”) ;
}
step-4: Save this file as “helloworld.c”.
step-5: Type “ls” on Terminal to see all files under current folder.
Confirm that “helloworld.c” is in the current directory. If not, type cd
DIRECTORY_PATH to go
to the directory that has “helloworld.c”.
step-6: Type “gcc helloworld.c” to compile, and type “ls” to confirm that a new executable
file “a.out” is created.
step-7: Type “./a.out” on Terminal to run the program
step-8: If you see “Hello World” on the next line, you just successfully ran your first C
program!
step-9: Try other codes from “A Shotgun Introduction to C” on professor Edwards’s
webpage.You can
also find many C programing guides online.
OUTPUT :

Creating a New Virtual Machine

Step 1:
Click Create a New Virtual Machine.

Step 2:
Select the desired Operating System's image file.(here it is Ubuntu)
Click Open.

Step 3:
Click Next.
Step 4:
Give your Name, Username and Password. Click Next.
Step 5:
Give your Virtual Machine a Name and select the location. Click Next.

Step 6:
Click Next.
Step 7:
Click Finish.

Running a C Program is Ubuntu

Step 1:
Launch the Virtual Machine. Right Click on the screen and select Open in Terminal.
Step 2:
Run the command “sudo apt install build-essential” to ensure every Pre-requisite is
installed.
Give password if required.
Step 3:
Create a C file using the command “touch hello.c”.
Step 4:
Double Click on the C file. Write an Hello World Program and save it.

Step 5:
Open the terminal and type this command “gcc hello.c -o test”. This will create a test
file.
Step 6:
To run the program run this “./test” command in terminal. Output will be displayed.

RESULT :

Thus a C compiler in the virtual machine created using virtual box is

installed and Simple Programs is executed.
 Install Google App Engine. Create hello world app and other
Ex. No : 3
simple web applications using python/java.
Date :

AIM :
To install Google App Engine. Create hello world app and other simple web
applications using python/java.

This document describes the installation of the Google App Engine Software Development
Kit (SDK) on a Microsoft Windows and running a simple “hello world” application.The App
Engine SDK allows you to run Google App Engine Applications on your local computer. It
simulates the run-‐time environment of the Google App Engine infrastructure.

PROCEDURE :

Download and Install:

step-1: You can download the Google App Engine SDK by going
to:http://code.google.com/appengine/downloads.html and down-
load the
appropriate install package.

step-2: Download the Windows installer – the simplest thing

is to download
it to you Desktop or another folder that you remem-
ber.

step-3: Double Click on the GoogleApplicationEngine in-

staller.

step-4: Click through the installation wizard, and it should

install the App
Engine. If you do not have Python 2.5, it will install
Python 2.5.

step-5: Once the install is complete you can discard the downloaded installer.

step-6: Make a folder for your Google App Engine applications. I am going to
make the Folder on my Desktop called “apps” – the
path to this
folder is:
C:\Documents and Settings\csev\Desktop\app
step-7: And then make a sub-‐folder in within apps called “ae-01--
trivial” – the path
To this folder would be:
C:\ Documents a nd Settings \csev\Desktop\apps\ae-01-trivial

step-8: Using a text editor such as JEdit (www.jedit.org), create a file called app.yaml
in the
ae-01-trivial folder with the following contents:
a
p
p
l
i
c
a
t
i
o
n
:

a
e
-
0
1
-
t
r
i
v
i
a
l

v
e
r
s
i
o
 n
:

1
r
u
n
t
i
m
e
:

p
y
t
h
o
n

a
p
i
_
v
e
r
s
i
o
n
:

handlers:
url: /.*
script: index.py

step-9: Then create a file in the ae-01-trivial folder called index.py with three lines in it:

print
'Con-
tent-
Type:
text/
plain'
print '
'
print 'Hello there Chuck'
OUTPUT :
Python Installation
Step 1:
Download Python from python.org and install it in your PC.(python 2.5 is used here)

Step 2:
 Select “Install to all users” and click Next.

Step 3:
Select the directory for the Python files and click Next.

Step 4:
Click Finish.
Google App Engine Installation
Step 1:
Download and install the Google App Engine.
Please make sure that Python is installed properly in your PC because Python is a Pre-
requisite.
Step 2:
Check “I accept the terms in the License Agreement” and click Next.

Step 3:
Select the Path and click Next.
Step 4:
Click Install.

Step 5:
Click Finish.
Notepad++ Installation
Step 1:
Download and Install Notepad++.

Step 2:
Click Next.
Step 3:
Read the terms and conditions. Click I Agree.

Step 4:
Click Install.
Step 5:
Click Finish.

Running a Python program using Google App Engine

Step 1:
Open Notepad++ and create hello.py file.
In hello.py write the following code.
Step 2:
Create an app.yaml file and write the following code.

Step 3:
Launch Google App Engine. Select File->Create New Application.
Step 4:
Select the correct folder(here it is apps->app-01). Click OK.

Step 5:
Set the Port to 8080 and click Create.
Step 6:
Click on Run.
Step 7:
Go to http://localhost:8080/ in desired browser. Output will be displayed.
RESULT :
Thus the Google App Engine is installed and hello world app and
other simple web applications is created using python/java.
Ex. No : 4 Use GAE launcher to launch the web applications.
Date :

AIM :
To use GAE launcher to launch the web applications.

PROCEDURE :

step-1: Then start the GoogleAppEngineLauncher program that can be found under
Applications. Use the File -> Add Existing Application command and navi-
gate into the
apps directory and select the ae-01-trivial folder.

step-2: Once you have added the application, select it so that you can control
the application using
the launcher.

step-3: Watching the Log::You can watch the internal log of the actions that
the web server is
performing when you are interacting with your application in the browser.
Select your
application in the Launcher and press the Logs button to bring up a log win-
dow.

step-4: Dealing With Errors:With two files to edit, there are two general categories
of errors that
you may encounter. If you make a mistake on the app.yaml file, the App En-
gine will not start
and your launcher will show a yellow icon near your application.

step-5: In this instance – the mistake is mis-‐indenting the last line in the app.yaml (line
8).If you make
a syntax error in the index.py file, a Python trace back error will appear in
your browser.

step-6: When you make a mistake in the app.yaml file – you must the fix the mistake
and attempt to
start the application again. If you make a mistake in a file like index.py, you
can simply fix the
file and press refresh in your browser – there is no need to restart the server.
 step-7: Shutting Down the Server:To shut down the server, use the Launcher, select
your
application and press the Stop button.

OUTPUT :

Step-1:Open the Google App engine Launcher.

step-2:Check out for the errors over the program.

Step-3:Set path for the document.

Step-4:Enter the code Log Console.

Step-6:Shut down the console.
 RESULT :
Thus the GAE launcher is used to launch the web applicatons.

AIM :
To simulate a cloud scenario using CloudSim and run a scheduling
algorithm that is not present in CloudSim.

Ex. No : 5 Simulate a cloud scenario using CloudSim and run a sched-

Date : uling algorithm that is not present in CloudSim.

PROCEDURE :
step-1: Download CloudSim installable files from https://code.google.com/p/cloudsim/
downloads/list
and unzip
Open Eclipse

step-2: Create a new Java Project: File -> New

step-3: Import an unpacked CloudSim project into the new Java Project
step-4: The first step is to initialise the CloudSim package by initialising the
CloudSim library, as follows:
CloudSim.init(num_user, calendar, trace_flag)

step-5: Data centres are the resource providers in CloudSim; hence,

creation of data
centres is a second step. To create Datacenter, you need the
DatacenterCharacteristics object that stores the properties of a
data centre
Such as architecture, OS, list of machines, allocation policy
that covers the
time or spaceshared, the time zone and its price:
Datacenter datacenter9883 = new Datacenter(name, characteristics,
new
VmAllocationPolicySimple(hostList),

step-6: The third step is to create a broker:

DatacenterBroker broker = createBroker();

step-7: The fourth step is to create one virtual machine unique ID of

the VM,
userId ID of the VM’s owner, mips, number Of Pes amount
of CPUs,
amount of RAM, amount of bandwidth, amount of storage,
virtual
machine monitor, and cloudletScheduler policy for cloudlets:
Vm vm = new Vm(vmid, brokerId, mips,
pesNumber, ram, bw, size, vmm, new
CloudletSchedulerTimeShared())

step-8: Submit the VM list to the broker:

broker.submitVmList(vmlist)

step-9: Create a cloudlet with length, file size, output size, and utilisation model:
Cloudlet cloudlet = new Cloudlet(id, length, pesNumber, fileSize, output-
Size, utilizationModel, utilizationMode

step-10: Submit the cloudlet list to the broker:

broker.submitCloudletList(cloudletList)
 step-11: Start the simulation:

Clou
dSim
start-
Sim-
ula-
tion(
)
Sample Output from the Existing Example:
Start-
ing
CloudS
imEx-
am-
ple1...
Initial-
ising...
Start-
ing
Cloud
Sim
version
3.0
Data-
center_
0 is
start-
ing...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>null
B
r
o
k
e
r

i
s

s
t
a
r
t
i
n
g
.
.
 .

E
n
t
i
t
i
e
s

s
t
a
r
t
e
d
.
: Broker: Cloud Resource List received
with 1 resource(s) 0.0: Broker: Try-
ing to Create VM #0 in Datacenter_0
: Broker: VM #0 has been created in Datacenter #2, Host #0
0.1: Broker: Sending cloudlet 0 to VM #0
400.1: Broker: Cloudlet 0 received
: Broker: All Cloudlets exe-
cuted. Finishing... 400.1:
Broker: Destroying VM #0
Broker is
shutting
down...
Simula-
tion: No
more fu-
ture events
CloudInformationService: Notify all CloudSim en-
tities for shutting down. Datacenter_0 is shutting
down...

Bro
ker
is
shut
ting
dow
n...
Sim
ula-
tion
com
plet
ed.
Simulation completed.

========== OUTPUT ==========

Cloudlet ID STATUS Data center ID VM ID Time
Start Time Finish Time 0 SUCCESS 2
0 400 0.1 400.1
*****Data-
center: Data-
center_0****
* User id

Debt
3 35.6

CloudSimExample1 finished!
OUTPUT :
Step 1:
Go to http://cloudbus.org/cloudsim/ and click on download – https://github.com/Cloudslab/cloudsim/releases

Step 2:
Now download the cloudsim-3.0.3.zip file and after download extract the file.
Step 3:
Now open NetBeans IDE and select File-> New Project.

Step 4:
Click on Java -> Java application then click next.
Step 5:
Give project name as Cloudsim and click finish.

Step 6:
Now right click on libraries option in the left side and select Add jar /folder.
Step 7:
From the extracted cloudsim folder click on the jars folder and select cloudsim-3.0.3.jar and
click open.

Step 8:
In the cloudsim-3.0.3.zip folder copy the folder named examples and paste it in src of your
project folder
Step 9:
Once done the example folders appear in NetBeans, now click on CloudExample1.java

Step 10:
Right click on the code and click run file

Step 11:
The Sample Output from the Existing Example is obtained.

RESULT :
Thus a cloud scenario is simulated using CloudSim and a scheduling algorithm
that is not present in CloudSim successfully.
 Find a procedure to transfer the files from one virtual ma-
Ex. No : 6 chine to another virtual machine.
Date :

AIM :
To find a procedure to transfer the files from one virtual machine to another virtual
machine.

PROCEDURE:

step-1: One can copy few (or more) lines with copy & paste mechanism.
For this you need to share clipboard between host OS and guest
OS, installing Guest Addition on both the virtual machines
(probably setting bidirectional and restarting them). You copy from
guest OS in the clipboard that is shared with the host OS.
Then you paste from the host OS to the second guest OS.

step-2: You can enable drag and drop too with the same method
(Click on the machine, settings, general, advanced, drag and drop:
set to bidirectional )

step-3: You can have common Shared Folders on both virtual

machines and use one of the directory shared as buffer to
copy.
Installing Guest Additions you have the possibility to set Shared
Folders too. As you put a file in a shared folder from host OS or
from guest OS, is immediately visible to the other. (Keep in mind
that can arise some problems for date/time of the files when there
are different clock settings on the different virtual machines).
If you use the same folder shared on more machines you can
exchange files directly copying them in this folder.

step-4: You can use usual method to copy files between 2 different
computer with client-server application. (e.g. scp with sshd active
for linux, winscp... you can get some info about SSH servers e.g.
here)
You need an active server (sshd) on the receiving machine and a
client on the sending machine. Of course you need to have the au-
thorization setted (via password or, better, via an automatic au-
thentication method).
Note: many Linux/Ubuntu distribution install sshd by default:
you can see if it is running with pgrep sshd from a shell. You can
install with sudo apt-get install openssh-server.

step-5: You can mount part of the file system of a virtual ma-
chine via NFS or SSHFS on the other, or you can share file
and directory with Samba.
 You may find interesting the article Sharing files between
guest and host without VirtualBox shared folders with
detailed step by step instructions.

step-6: Each virtual machine is an instance of a program owned by an

user in the hosting operative system and should undergo the restric-
tions of the user in the hosting OS.

step-7: Enable Drag & drop to be cosy with the Windows machines and
the Shared folders or to be cosy with Linux.

step-8: When you will need to be fast with Linux you will feel the need of ssh-key-
gen andto Generate once SSH Keys to copy files on/from a remote machine without writ-
ing password anymore. In this way it functions bash auto-completion remotely.
OUTPUT :

Step-1:
Install the VMware
Step-2:
Set up the VMware.
Step-3:
Setup the Windows XP in the VMware.
Step-4:
Setup the VMware tool.
Step-5:
Open the system.
Step-6:
Set the VMware in the OS for file transfer.

Step-7:
Select the file to be transferred.
Step-8:
Connect to the VM

Step-9:
Check the properties of the LAN.
Step-10:
Setup the TCP/IP in the LAN properties and also get the IP address.

Step-11:
Now enter the file name correctly which must be transferred .
Step-12:
Click yes.
The file is transferred.
 RESULT :

Thus the files from one virtual machine is transferred to another virtual machine.

Ex. No : 7 Find a procedure to launch virtual machine using trystack

(Online Openstack Demo Version)
Date :

AIM :
To find a procedure to launch virtual machine using trystack (Online Openstack
Demo Version).

PROCEDURE :

OpenStack is an open-source software cloud computing plat-

form. OpenStack is primarily used for deploying an infrastruc-
ture as a service (IaaS) solution like Amazon Web Service
(AWS). In other words, you can make your own AWS by using
OpenStack. If you want to try out OpenStack, TryStack is the
easiest and free way to do it.
OpenStack Compute Dashboard

Step 1: Create Network

Network? Yes, the network in here is our own local network. So, your
instances will be not mixed up with the others. You can imagine this as
your own LAN (Local Area Network) in the cloud.

1. Go to Network > Networks and then click Create Network.

2. In Network tab, fill Network Name for example internal and then click Next.
3. In Subnet tab,
1. Fill Network Address with appropriate CIDR, for example
192.168.1.0/24. Use private network CIDR block as the best
practice.
2. Select IP Version with appropriate IP version, in this case IPv4.

3. Click Next.
In Subnet Details tab, fill DNS Name Servers with 8.8.8.8 (Google DNS) and then click
Create.

Step 2:Create Instance

1. Go to Compute > Instances and then click Launch Instance.

2. In Details tab,
1. Fill Instance Name, for example Ubuntu 1.
2. Select Flavor, for example m1.medium.
3. Fill Instance Count with 1.
4. Select Instance Boot Source with Boot from Image.
5. Select Image Name with Ubuntu 14.04 amd64 (243.7 MB) if you want install
 Ubuntu
14.04 in your virtual machine.
3. In Access & Security tab,
1. Click [+] button of Key Pair to import key pair. This key pair is a
public and private key that we will use to connect to the instance
from our machine.
2. In Import Key Pair dialog,
1. Fill Key Pair Name with your machine name (for example Edward-Key).
2. Fill Public Key with your SSH public key (usually is in
~/.ssh/id_rsa.pub). See description in Import Key Pair di-
alog box for more information. If you are using Win-
dows, you can use Puttygen to generate key pair.
3. Click Import key pair.
3. In Security Groups, mark/check default.
4. In Networking tab,
1. In Selected Networks, select network that have been created in Step 1, for example
internal.
5. Click Launch.
6. If you want to create multiple instances, you can repeat step 1-5. I
created one more instance with instance name Ubuntu 2.

Step 3: Create Router

I guess you already know what router is. In the step 1, we created our
network, but it is isolated. It doesn’t connect to the internet. To make our
network has an internet connection, we need a router that running as the
gateway to the internet.
1. Go to Network > Routers and then click Create Router.
2. Fill Router Name for example router1 and then click Create router.
3. Click on your router name link, for example router1, Router Details page.
4. Click Set Gateway button in upper right:
1. Select External networks with external.
2. Then OK.
5. Click Add Interface button.
1. Select Subnet with the network that you have been created in Step 1.
2. Click Add interface.
6. Go to Network > Network Topology. You will see the network topol-
ogy. In the example, there are two network, i.e. external and internal,
those are bridged by a router. There are
instances those are joined to internal network.

Step 4: Configure floating IP address

1. Go to Compute > Instance.
2. In one of your instances, click More > Associate Floating IP.
3. In IP Address, click Plus [+].
4. Select Pool to external and then click Allocate IP.
5. Click Associate.
6. Now you will get a public IP, e.g. 8.21.28.120, for your instance.
 Step 5: Configure Access & Security

OpenStack has a feature like a firewall. It can whitelist/blacklist your

in/out connection. It is called Security Group.
1. Go to Compute > Access & Security and then open Security Groups tab.
2. In default row, click Manage Rules.
3. Click Add Rule, choose ALL ICMP rule to enable ping into your instance, and then
click Add.
4. Click Add Rule, choose HTTP rule to open HTTP port (port 80), and then click Add.
5. Click Add Rule, choose SSH rule to open SSH port (port 22), and then click Add.
6. You can open other ports by creating new rules.

Step 6: SSH to Your Instance

Now, you can SSH your instances to the floating IP address that you got
in the step 4. If you are using Ubuntu image, the SSH user will be
ubuntu.
 RESULT :
Thus the virtual machine using trystack (Online Openstack Demo Version) is
launched successfully.

Ex. No : 8 Install Hadoop single node cluster and run simple applica-
Date : tions like wordcount.

AIM :
To install Hadoop single node cluster and run simple applica-
tions like wordcount.

PROCEDURE :
1. Install hadoop.
2. Start all services using the command.

hduser@prince-VirtualBox:/usr/local/hadoop/bin$ jps
3242 Jps

hduser@prince-VirtualBox:/usr/local/hadoop/bin$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
16/09/15 15:38:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hduser-namenode-
prince-VirtualBox.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hduser-datanode-
prince-VirtualBox.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hduser-sec-
ondarynamenode-prince-VirtualBox.out
16/09/15 15:39:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
starting yarn daemons
startingresourcemanager, logging to /usr/local/hadoop/logs/yarn-hduser-resourcemanager-
prince-VirtualBox.out
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-hduser-nodemanager-
prince-VirtualBox.out

hduser@prince-VirtualBox:/usr/local/hadoop/bin$ jps
16098 NameNode
16214 DataNode
16761 NodeManager
16636 ResourceManager
16429 SecondaryNameNode
19231 Jps

PROGRAM CODING:

hduser@prince-VirtualBox:/usr/local/hadoop/bin$ nano wordcount7.java

importjava.io.IOException;
importjava.util.StringTokenizer;
importorg.apache.hadoop.conf.Configuration;
importorg.apache.hadoop.fs.Path;
importorg.apache.hadoop.io.IntWritable;
importorg.apache.hadoop.io.Text;
importorg.apache.hadoop.mapreduce.Job;
importorg.apache.hadoop.mapreduce.Mapper;
importorg.apache.hadoop.mapreduce.Reducer;
importorg.apache.hadoop.mapreduce.lib.input.FileInputFormat;
importorg.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class wordcount7 {

public static class TokenizerMapper

extends Mapper<Object, Text, Text, IntWritable>{

private final static IntWritable one = new IntWritable(1);

private Text word = new Text();

public void map(Object key, Text value, Context context

) throws IOException, InterruptedException {
StringTokenizeritr = new StringTokenizer(value.toString());
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
context.write(word, one);
}
}
}

public static class IntSumReducer

extends Reducer<Text,IntWritable,Text,IntWritable> {
privateIntWritable result = new IntWritable();

public void reduce(Text key, Iterable<IntWritable> values,

Context context
) throws IOException, InterruptedException {
int sum = 0;
for (IntWritableval : values) {
sum += val.get();
}
result.set(sum);
context.write(key, result);
}
}

public static void main(String[] args) throws Exception {

Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "word count");
job.setJarByClass(wordcount7.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}

TO COMPILE:
hduser@prince-VirtualBox:/usr/local/hadoop/bin$ hadoopcom.sun.tools.javac.Main word-
count7.java

TO CREATE A JAR FILE:

hduser@prince-VirtualBox:/usr/local/hadoop/bin$ jar cf wc2.jar wordcount7*.java

TO CREATE A DIRECTORY IN HDFS:

hduser@prince-VirtualBox:/usr/local/hadoop/bin$hadoopdfs -mkdir /deepika

TO LOAD INPUT FILE:

hduser@prince-VirtualBox:/usr/local/hadoop/bin$hdfs -put /home/prince/Downloads/
wc.txt /deepika/wc1

TO EXECUTE:
hduser@prince-VirtualBox:/usr/local/hadoop/bin$ hadoop jar wc2.jar wordcount7 /
deepika/wc1.txt /deepika/out2

16/09/16 14:34:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for

your platform... using builtin-java classes where applicable
16/09/16 14:34:17 INFO Configuration.deprecation: session.id is deprecated. Instead, use df-
s.metrics.session-id
16/09/16 14:34:17 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=Job-
Tracker, sessionId=
16/09/16 14:34:17 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing
not performed. Implement the Tool interface and execute your application with ToolRunner
to remedy this.
16/09/16 14:34:17 INFO input.FileInputFormat: Total input paths to process : 1
16/09/16 14:34:17 INFO mapreduce.JobSubmitter: number of splits:1
16/09/16 14:34:18 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_lo-
cal364071501_0001
16/09/16 14:34:18 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
16/09/16 14:34:18 INFO mapreduce.Job: Running job: job_local364071501_0001
16/09/16 14:34:18 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/09/16 14:34:19 INFO mapred.LocalJobRunner: OutputCommitter is org.a-
pache.hadoop.mapreduce.lib.output.FileOutputCommitter
16/09/16 14:34:19 INFO mapred.LocalJobRunner: Waiting for map tasks
16/09/16 14:34:19 INFO mapred.LocalJobRunner: Starting task:
attempt_local364071501_0001_m_000000_0

16/09/16 14:34:19 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]

16/09/16 14:34:19 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/
deepika/wc1:0+712
16/09/16 14:34:19 INFO mapreduce.Job: Job job_local364071501_0001 running in uber-
mode : false
16/09/16 14:34:23 INFO mapreduce.Job: map 0% reduce 0%
16/09/16 14:34:24 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
16/09/16 14:34:24 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
16/09/16 14:34:24 INFO mapred.MapTask: soft limit at 83886080
16/09/16 14:34:24 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
16/09/16 14:34:24 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
16/09/16 14:34:24 INFO mapred.MapTask: Map output collector class = org.a-
pache.hadoop.mapred.MapTask$MapOutputBuffer
16/09/16 14:34:26 INFO mapred.LocalJobRunner:
16/09/16 14:34:26 INFO mapred.MapTask: Starting flush of map output
16/09/16 14:34:26 INFO mapred.MapTask: Spilling map output
16/09/16 14:34:26 INFO mapred.MapTask: bufstart = 0; bufend = 1079; bufvoid =
104857600
16/09/16 14:34:26 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend =
26214032(104856128); length = 365/6553600
16/09/16 14:34:26 INFO mapred.MapTask: Finished spill 0
16/09/16 14:34:26 INFO mapred.Task: Task:attempt_local364071501_0001_m_000000_0 is
done. And is in the process of committing
16/09/16 14:34:26 INFO mapred.LocalJobRunner: map
16/09/16 14:34:26 INFO mapred.Task: Task 'attempt_local364071501_0001_m_000000_0'
done.
16/09/16 14:34:26 INFO mapred.LocalJobRunner: Finishing task:
attempt_local364071501_0001_m_000000_0
16/09/16 14:34:26 INFO mapred.LocalJobRunner: map task executor complete.
16/09/16 14:34:26 INFO mapred.LocalJobRunner: Waiting for reduce tasks
16/09/16 14:34:26 INFO mapred.LocalJobRunner: Starting task:
attempt_local364071501_0001_r_000000_0
16/09/16 14:34:26 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
16/09/16 14:34:26 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.a-
pache.hadoop.mapreduce.task.reduce.Shuffle@2ee9ab75
16/09/16 14:34:26 INFO mapreduce.Job: map 100% reduce 0%
16/09/16 14:34:26 INFO reduce.MergeManagerImpl: MergerManager:
memoryLimit=363285696, maxSingleShuffleLimit=90821424, mergeThreshold=239768576,
ioSortFactor=10, memToMemMergeOutputsThreshold=10
16/09/16 14:34:26 INFO reduce.EventFetcher: attempt_local364071501_0001_r_000000_0
Thread started: EventFetcher for fetching Map Completion Events
16/09/16 14:34:26 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map
attempt_local364071501_0001_m_000000_0 decomp: 1014 len: 1018 to MEMORY
16/09/16 14:34:27 INFO reduce.InMemoryMapOutput: Read 1014 bytes from map-output
for attempt_local364071501_0001_m_000000_0
16/09/16 14:34:27 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of
size: 1014, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->1014
16/09/16 14:34:27 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning
16/09/16 14:34:27 INFO mapred.LocalJobRunner: 1 / 1 copied.
16/09/16 14:34:27 INFO reduce.MergeManagerImpl: finalMerge called with 1 in-memory
map-outputs and 0 on-disk map-outputs

16/09/16 14:34:27 INFO mapred.Merger: Merging 1 sorted segments

16/09/16 14:34:27 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left
of total size: 991 bytes
16/09/16 14:34:27 INFO reduce.MergeManagerImpl: Merged 1 segments, 1014 bytes to disk
to satisfy

reduce memory limit

16/09/16 14:34:27 INFO reduce.MergeManagerImpl: Merging 1 files, 1018 bytes from disk
16/09/16 14:34:27 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from
memory into reduce
16/09/16 14:34:27 INFO mapred.Merger: Merging 1 sorted segments
16/09/16 14:34:27 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left
of total size: 991 bytes
16/09/16 14:34:27 INFO mapred.LocalJobRunner: 1 / 1 copied.
16/09/16 14:34:27 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead,
use mapreduce.job.skiprecords
16/09/16 14:34:30 INFO mapred.Task: Task:attempt_local364071501_0001_r_000000_0 is
done. And is in the process of committing
16/09/16 14:34:30 INFO mapred.LocalJobRunner: 1 / 1 copied.
16/09/16 14:34:30 INFO mapred.Task: Task attempt_local364071501_0001_r_000000_0 is
allowed to commit now
16/09/16 14:34:30 INFO output.FileOutputCommitter: Saved output of task 'attempt_lo-
cal364071501_0001_r_000000_0' to hdfs://localhost:54310/deepika/out2/_temporary/0/
task_local364071501_0001_r_000000
16/09/16 14:34:30 INFO mapred.LocalJobRunner: reduce > reduce
16/09/16 14:34:30 INFO mapred.Task: Task 'attempt_local364071501_0001_r_000000_0'
done.
16/09/16 14:34:30 INFO mapred.LocalJobRunner: Finishing task:
attempt_local364071501_0001_r_000000_0
16/09/16 14:34:30 INFO mapred.LocalJobRunner: reduce task executor complete.
16/09/16 14:34:30 INFO mapreduce.Job: map 100% reduce 100%
16/09/16 14:34:31 INFO mapreduce.Job: Job job_local364071501_0001 completed success-
fully
16/09/16 14:34:31 INFO mapreduce.Job: Counters: 38
File System Counters
FILE: Number of bytes read=8552
FILE: Number of bytes written=507858
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=1424
HDFS: Number of bytes written=724
HDFS: Number of read operations=13
HDFS: Number of large read operations=0
HDFS: Number of write operations=4
Map-Reduce Framework
Map input records=10
Map output records=92
Map output bytes=1079
Map output materialized bytes=1018
Input split bytes=99
Combine input records=92
Combine output records=72

Reduce input groups=72

Reduce shuffle bytes=1018
Reduce input records=72
Reduce output records=72
Spilled Records=144
Shuffled Maps =1

Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=111
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=242360320
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=712
File Output Format Counters
Bytes Written=724

INPUT FILE:
wc1.txt

STEPS:
1. Open an editor and type WordCount program and save as WordCount.java
2. Set the path as export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
3. To compile the program, bin/hadoopcom.sun.tools.javac.Main WordCount.java
4. Create a jar file, jar cf wc.jar WordCount*.class
5. Create input files input.txt,input1.txt and input2.txt and create a directory in hdfs, /mit/
wordcount/input
6. Move these i/p files to hdfs system, bin/hadoopfs –put /opt/hadoop-2.7.0/input.txt /mit/
wordcount/input/input.txt repeat this step for other two i/p files.
7. To execute, bin/hadoop jar wc.jar WordCount /mit/wordcount/input /mit/wordcount/out-
put.
8. The mapreduce result will be available in the output directory.

OUTPUT:
/mit/wordcount/input 2
/mit/wordcount/input/input.txt 1
/mit/wordcount/output. 1
/opt/hadoop-2.7.0/input.txt 1
1. 1
2. 1
3. 1
4. 1
5. 1
6. 1
7. 1
8. 1
Create 2
HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar 1
Move 1
Open 1

STEPS: 1
Set 1
The 1
To 2
WordCount 2
WordCount*.class 1
WordCount.java 2
a2
an 1
and 4
as 2
available 1
be 1
bin/hadoop 3
cf 1
com.sun.tools.javac.Main 1
compile 1
create 1
directory 1
directory. 1
editor 1
execute, 1
export 1
file, 1
files 2
files. 1
for 1
fs 1
hdfs 1
hdfs, 1
i/p 2
in 2
input 1
input.txt,input1.txt 1
input2.txt 1
jar 3
mapreduce 1
other 1
output 1
path 1
program 1
program, 1
repeat 1
result 1
save 1
step 1
system, 1
the 3
these 1
this 1

to 1
two 1
type 1
wc.jar 2
will 1
–put 1
RESULT :
Thus hadoop single node cluster is installed and simple wordcount appli-
cation is executed.