CC Labs All
CC Labs All
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand private cloud deployment
2. Understand Hadoop distribution and need for it
3. Install and configure Cloudera private cloud
Theory:
Apache Hadoop is a collection of open-source software utilities that facilitates using a
network of many computers to solve problems involving massive amounts of data and
computation. It provides a software framework for distributed storage and processing of big
data using the MapReduce programming model. Cloudera, Inc. is a US-based software
company that provides a software platform for data engineering, data warehousing, machine
learning and analytics that runs in the cloud or on premises. Cloudera started as a hybrid
open-source Apache Hadoop distribution, CDH (Cloudera Distribution Including Apache
Hadoop),that targeted enterprise-class deployments of that technology.
Hadoop is a software framework for distributed processing of large datasets across large
clusters of computers
Large datasets Terabytes or petabytes of data
Large clusters hundreds or thousands of nodes
Hadoop is open-source implementation for Google MapReduce. Hadoop is based on a
simple programming model called MapReduce. Hadoop is based on a simple data model,
any data will fit.
1
Cloud Computing 2021-22
Map Reduce
2
Cloud Computing 2021-22
3
Cloud Computing 2021-22
Highlights of Cloudera
1. Control costs and manage resources
– Auto-scale
– Auto-suspend
2. Easy provisioning and support for multiple types of workloads
3. Consistent security and data governance across applications and datasets
4. Enables enterprise IT staff to quickly respond to business demands
For Cloudera
3. Start VirtualBox.
4
Cloud Computing 2021-22
5. Click the Folder icon.
8. Click Import.
5
Cloud Computing 2021-22
9. The virtual machine image will be imported. This can take several minutes.
10. Launch Cloudera VM. When the importing is finished, the quickstart-vm-5.4.2-0 VM
will appear on the left in the VirtualBox window. Select it and click the Start button to
launch the VM.
11. Cloudera VM booting. It will take several minutes for the Virtual Machine to start. The
booting process takes a long time since many Hadoop tools are started.
6
Cloud Computing 2021-22
12. The Cloudera VM desktop. Once the booting process is complete, the desktop will
appear with a browser.
Workaround Procedure:
7
Cloud Computing 2021-22
Download Location: https://downloads.cloudera.com/demo_vm/docker/cloudera-
quickstart-vm-5.12.0-0-beta-docker.tar.gz
8
Cloud Computing 2021-22
Questions:
1. How is private cloud different from public cloud in Cloudera?
9
Cloud Computing 2021-22
3. Why managing resources at platform level is better than at application level? How is
it done in Cloudera?
10
Cloud Computing 2021-22
4. What are containers in cloud? How does it help in Cloudera?
11
Cloud Computing 2021-22
Aim: Create datacenters and run simulation examples after installing CloudSim
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand how to create datacenters
2. Understand cloudlets and basic concepts of cloud simulation
3. Install and configure CloudSim from CLOUDS lab and run all examples
Theory:
Cloud computing is the leading technology for delivering reliable, secure, fault-tolerant,
sustainable, and scalable computational services, which are presented as Software,
Infrastructure, or Platform as services (SaaS, IaaS, PaaS). Moreover, these services may be
offered in private data centers (private clouds), may be commercially offered for clients
(public clouds), or yet it is possible that both public and private clouds are combined in
hybrid clouds. These already wide ecosystem of cloud architectures, along with the
increasing demand for energy-efficient IT technologies, demand timely, repeatable, and
controllable methodologies for evaluation of algorithms, applications, and policies before
actual development of cloud products. Because utilization of real testbeds limits the
experiments to the scale of the testbed and makes the reproduction of results an extremely
difficult undertaking, alternative approaches for testing and experimentation leverage
development of new Cloud technologies. A suitable alternative is the utilization of
simulations tools, which open the possibility of evaluating the hypothesis prior to software
development in an environment where one can reproduce tests. Specifically in the case of
Cloud computing, where access to the infrastructure incurs payments in real currency,
simulation-based approaches offer significant benefits, as it allows Cloud customers to test
their services in repeatable and controllable environment free of cost, and to tune the
performance bottlenecks before deploying on real Clouds. At the provider side, simulation
environments allow evaluation of different kinds of resource leasing scenarios under varying
load and pricing distributions. Such studies could aid the providers in optimizing the
resource access cost with focus on improving profits. In the absence of such simulation
platforms, Cloud customers and providers have to rely either on theoretical and imprecise
evaluations, or on try-and-error approaches that lead to inefficient service performance and
revenue generation. Therefore, we use CloudSim from CLOUDS lab [1]
For self-paced learning, there is CloudSim tutorials available [2]
1
Cloud Computing 2021-22
Reference
[1] http://www.cloudbus.org/cloudsim/
[2] https://www.cloudsimtutorials.online/
Procedure:
The three important steps along with the systematic guide shown below for reference.
Step 1 – Setting up development environment
Step 2- Use eclipse with CloudSim
Step 3 – Run sample examples using CloudSim
Let us start
Step 1 – Setting up development environment
Development environment requires the following (Eclipse Based)
a) Java Development Kit
b) Eclipse for Java
c) CloudSim Tool kit
Step a) If you have JDK installed on your machine skip these steps and goto to Step b)
Else follow the steps as shown below.
To install and run JDK go to the below link and select a package based on your OS
Java SE - Downloads | Oracle Technology Network | Oracle
2
Cloud Computing 2021-22
After download is complete, Run the installation package to show the below figure
Step b) If you have Eclipse for Java installed on your machine skip these steps and goto to
Step c)
3
Cloud Computing 2021-22
To install and run eclipse go to the following download link
Eclipse Downloads | The Eclipse Foundation
Then select Eclipse for Java after clicking Download packages as shown in the figure below
After download is complete, extract the files and check if you see eclipse.exe
Step c)
To install and run CloudSim package go to following download link
http://code.google.com/p/cloudsim/downloads
Download only the cloudsim-3.0.tar.gz file from the link
4
Cloud Computing 2021-22
Once the download is complete use WinZip software to unzip the file so that you can see
cloudsim-3.0 folder in the tar.gz folder. The extraction of package will take time.
5
Cloud Computing 2021-22
Select workspace and Launch
6
Cloud Computing 2021-22
Untick the Use Default Location
Browse and target it to where you have extracted CloudSim3.0
Click on Next
7
Cloud Computing 2021-22
8
Cloud Computing 2021-22
9
Cloud Computing 2021-22
It may show some minor errors but proceed to get the following output
10
Cloud Computing 2021-22
This is a simple example showing how to create a datacenter with one host and run one cloudlet on it.
http://www.cloudbus.org/cloudsim/examples.html
Screenshots :
https://docs.google.com/document/d/1tfQg28SdudBgjmwUbjBwO0cXakkJRl8o2h8KkFvw
AOw/edit?usp=sharing
Reading – read all basic and network examples and about DVFS
Questions
1. What are cloudlets? What is the need to use them?
11
Cloud Computing 2021-22
12
Cloud Computing 2021-22
4. Assess the need of container in cloud? Choose the best container? Justify
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
Theory:
Load Balancing is a very important topic in Cloud Computing as it used to distribute total
workload of multiple network links to achieve maximum throughput, minimize response
time and avoid overloading. There are mainly three algorithms used to distribute the load
keeping in mind the performance, time and cost.
Round Robin: This algorithm works in the round robin fashion where each node is allotted
a time slot in which they have to perform their task. This algorithm does not consider the
load on different machines. The complexity of this algorithm is less compared to the other
two algorithms.
Equally spread current execution load: This algorithm focuses on servers with less load
to allot tasks. The load balancer looks over the job queue frequently for new jobs and then
allots them to the list of free virtual servers. The jobs are allotted based on lightly loaded
servers and heavily loaded servers are avoided. The balancer also maintains the list of tasks
allotted to virtual servers, which helps them to identify that which virtual machines are free
and need to be allotted with new jobs.
Throttled Load balancing: This algorithm finds the appropriate virtual machine for
assigning a particular job. The job manager maintains the list of virtual machines to which it
allots specific tasks. If the job is well suited for a particular machine then that job is assigned
to that machine. If no virtual machines are available to accept jobs then the job manager
waits for the client request and takes the job in queue for fast processing.
About Cloud Analyst
Cloud Analyst is a tool developed at the University of Melbourne whose goal is to support
evaluation of social networks tools according to geographic distribution of users and data
centers. In this tool, communities of users and data centers supporting the social networks
are characterized and, based on their location; parameters such as user experience while
1
Cloud Computing 2021-22
using the social network application and load on the data center are obtained/logged.
CloudAnalyst is used to model and analyse real world problems through case studies of
social networking applications deployed on the cloud.
Procedure:
http://www.cloudbus.org/cloudsim/CloudAnalyst.zip
Step 2: Extract from ZIP file to get the following folder and structure
2
Cloud Computing 2021-22
Step 3: There are two methods to run the CloudAnalyst, one is to run from the command
line as shown below or just double click the run.bat file
Step 4: The CloudAnalyst opens and we can see the following as shown below.
Before getting into details, let us try to understand the tabs seen on the left side of the
display pane.
Configure Simulation – This tab to enter the configuration and set all the parameters.
3
Cloud Computing 2021-22
Define Internet Characteristics – This tab is to set the internet parameters needed for the
simulation. (Won’t change these in our case)
Run Simulation – This tab is to run the simulation and display the results. The results can
also be exported as excel file. (run simulation then always exit and reconfigure again)
There is an option to Save Configuration. Given a suitable name to the configuration file and
save it with .sim extension
Below one can see the Main configuration window with options to change the Simulation
Duration, Add and remove User Databases, Set the Service Broker Policy, Add New and
Remove Applications Deployed.
Second Tab is Data Center Configuration where one can add and remove Data Centers
based on the regions.
4
Cloud Computing 2021-22
Third tab is the Advanced configuration Tab to set basic parameters alongwith Load
Balancing policy across VM’s in a single Data Center.
After the configurations done in Configure Simulation then click Done before running the
configuration using the Run Configuration tab on the left pane
5
Cloud Computing 2021-22
As shown in above figure click on Add New to create a new Data Centre.
6
Cloud Computing 2021-22
We have added a new Data Center and the details will be as shown in figure above. Now
click on DC1 to get the details of the physical hardware configuration.
7
Cloud Computing 2021-22
Here the Physical Hardware configuration for the DC can be seen and changes made as
needed.
We can also copy the same DC configurations to avoid configuring same setting many times
for ease of use
We can see the Load Balancing Policy and on clicking we get three standard Algorithms to
8
Cloud Computing 2021-22
which we will set and get the desired output for different configurations
User Base: it models group of users and generates traffic as per setting
9
Cloud Computing 2021-22
We can set and change the properties of each server as needed and VMs can be allocated to
fulfill requests for different User Bases for different Data centers
10
Cloud Computing 2021-22
The saved file can be opened and click on Load Configuration as shown below
11
Cloud Computing 2021-22
12
Cloud Computing 2021-22
After closing this window we can see the Main window which shows the complete statistics
based on the Userbases, Datacenters and Regions Selected
If you run simulation again it will give the SIMULATION Error. In case of SIMULATION
ERROR shown on Clicking on this window again just click on Exit on Left side Menu Bar
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
Scenarios to be completed:
R0 – North America, R1 – South America, R2 – Europe, R3 – Asia, R4 – Africa, R5 - Australia
Exercise 1 (Graph plot needed)
02 Datacenters in R0 and R1 and Userbases in R3, R4 and R5. Plot the graphs as shown
above
Round Robin
15
Cloud Computing 2021-22
Equally Throttled
16
Cloud Computing 2021-22
Throttled
17
Cloud Computing 2021-22
18
Cloud Computing 2021-22
19
Cloud Computing 2021-22
Round Robin
20
Cloud Computing 2021-22
21
Cloud Computing 2021-22
Throttled
22
Cloud Computing 2021-22
Scenario 1
Avg Min Time: - DC at R2 and R5
Load Balancing -: Round robin
23
Cloud Computing 2021-22
24
Cloud Computing 2021-22
Scenario 2
Least Min Time DC at R4 and R5
25
Cloud Computing 2021-22
Scenario 3
DC at R2 and R5
26
Cloud Computing 2021-22
28
Cloud Computing 2021-22
29
Cloud Computing 2021-22
Experiment 4: NextCloud
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand Apache2 and installation in Ubuntu
2. Understand NextCloud open source cloud system.
3. Configure and make use of NextCloud
Theory:
Nextcloud, a fork of ownCloud, is a file sharing server that permits you to store your personal
content, like documents and pictures, in a centralized location, much like Dropbox. The
difference with Nextcloud is that all of its features are open-source. It also returns the control
and security of your sensitive data back to you, thus eliminating the use of a third-party cloud
hosting service.
Procedure:
Step 1 – Installing apache2 in Ubuntu
Enter root to install apache 2 and run the install command as shown below
1
Cloud Computing 2021-22
Once completed then type exit command and exit the root then run the update command to
update once again
The above command will show localhost by default. Add few more domains to the list of
trusted domains as shown below
Then run the get command again to check if the trusted domains are added correctly
This command will generate the keys self-signed certificate. Update the rules so as to
enable access
2
Cloud Computing 2021-22
Enter the Username and password you had setup earlier to get to main dashboard
1. Login Page
3
Cloud Computing 2021-22
2. Main Dashboard
4
Cloud Computing 2021-22
5
Cloud Computing 2021-22
3. Set Status
6
Cloud Computing 2021-22
7
Cloud Computing 2021-22
8
Cloud Computing 2021-22
9
Cloud Computing 2021-22
10
Cloud Computing 2021-22
Questions:
1. List the advantages and limitations of Nextcloud. Include security features as well
11
Cloud Computing 2021-22
2. If a large file needs to be uploaded to NextCloud what is the limit on size of the file?
Can it be extended?
12
Cloud Computing 2021-22
13
Cloud Computing 2021-22
Experiment 5: GO Language
Learning Outcomes:
After completion of this experiment, student should be able to
1. Install and use GO language
2. Write and compile programs in GO
3. Understand how to use GO language
Theory:
Go language was developed by Google in year 2007. It is statically typed language with syntax
similar to C. It is an open source project to make programmers more productive. Go is
expressive, concise, clean, and efficient. Its concurrency mechanisms make it easy to write
programs that get the most out of multicore and networked machines, while its novel type
system enables flexible and modular program construction. A fast, statically typed, compiled
language feels like a dynamically typed, interpreted language.
Features of GO Language
Support for environment adopting patterns similar to dynamic languages. For
example, type inference (x := 0 is valid declaration of a variable x of type int)
Compilation time is fast.
Inbuilt concurrency support: lightweight processes (via go routines), channels, select
statement.
Go programs are simple, concise, and safe.
Support for Interfaces and Type embedding.
Production of statically linked native binaries without external dependencies.
Go language supports Garbage collection and Type safety. We can build go program using go
build command. It will generate an executable file.
To run Go programs we need a text editor and Go compiler or go to the following link
https://play.golang.org then run the code. Go program is written into one or more text files
with the extension ".go". e.g. hello.go. One can use "vi", "vim" or any other text editor to write
your Go program into a file.
1
Cloud Computing 2021-22
Example 1
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, playground")
}
Example 2
package main
import (
"fmt"
)
func main() {
var i int
i = 42
fmt.Println(i)
}
Example 3
package main
import (
"fmt"
)
func main() {
var i float32 = 27
j := 99
fmt.Printf("%v, %T\n", i ,i)
fmt.Printf("%v, %T\n", j ,j)
}
2
Cloud Computing 2021-22
Example 4 – Global Variable, Package Variable and Block variable
package main
import (
"fmt"
)
var i int = 27
func main() {
var i int = 42
fmt.Println(i)
}
package main
import (
"fmt"
)
func main() {
var theHTTP string = “https://google.com”
fmt.Println(i)
}
package main
import (
"fmt"
)
func main() {
var i int = 42
fmt.Printf("%v,%T\n",i,i)
var j float32
j= float32(i)
fmt.Printf("%v,%T\n",j,j)
3
Cloud Computing 2021-22
}
Convert from Float to int and add decimal to i and check what happens
Also check if we keep j = i and run the code
package main
import (
"fmt"
"strconv"
)
func main() {
var i int = 42
fmt.Printf("%v,%T\n",i,i)
var j string
j= strconv.Itoa(i)
fmt.Printf("%v,%T\n",j,j)
Primitives
Example 1- Boolean types - True or False
package main
import (
"fmt"
)
func main() {
var n bool = true
fmt.Printf("%v,%T\n",n,n)
}
package main
import (
4
Cloud Computing 2021-22
"fmt"
)
func main() {
n := 1 == 1
m := 1 == 2
fmt.Printf("%v,%T\n",n,n)
fmt.Printf("%v,%T",m,m)
}
package main
import (
"fmt"
)
func main() {
a := 10
b := 3
fmt.Println(a + b)
fmt.Println(a - b)
fmt.Println(a * b)
fmt.Println(a / b)
fmt.Println(a % b)
}
References
https://golang.org/
https://gobyexample.com/
http://www.golang-book.com/
https://miek.nl/go/
https://www.cosmiclearn.com/go/
https://gophercises.com/
https://www.tutorialspoint.com/go/
5
Cloud Computing 2021-22
package main
import (
"fmt"
"sort"
)
// Main function
func main() {
Emp := []struct {
ID int
designation string
age int
dob string
doj string
salary int
}{
6
Cloud Computing 2021-22
{001, "CEO", 35, "07-04-1991", "05-03-
2007", 410000},
{002, "CFO", 34, "06-02-1934", "21-11-
2010", 130000},
{003, "CMO", 35, "28-06-1986", "03-02-
2011",150000},
{004, "COO", 31, "07-01-1989", "08-05-
2009", 250000},
{005, "Product Manager", 35, "16-03-1986",
"19-06-2012", 300000},
{006, "HR", 44, "15-12-1979", "03-07-2005",
90000},
{007, "Analyst", 40, "04-07-1982", "12-11-
2009", 120000},
{021, "Consultant", 25, "26-04-1986", "03-
09-2013", 85000},
{022, "Director", 41, "06-04-1990", "08-05-
2015", 150000},
{010, "VP", 52, "16-03-1986", "19-06-2012",
300000},
{011, "Senior VP", 23, "15-12-1979", "03-
07-2005", 90000},
{012, "Sales exec", 38, "04-07-1982", "12-
11-2009", 120000},
{013, "Developer", 30, "26-04-1986", "03-
09-2013", 85000},
{014, "Senior Consultant", 29, "06-04-
1990", "08-05-2015", 150000},
7
Cloud Computing 2021-22
{015, "CSO", 45, "16-03-1986", "19-06-
2012", 300000},
}
fmt.Println(Emp)
// Sorting Emp by their ID using Slice()
function
sort.Slice(Emp, func(p, q int) bool {
return Emp[p].ID < Emp[q].ID
})
fmt.Println()
fmt.Println("Sort Employees according to their"
+ "ID:")
fmt.Println(Emp)
fmt.Println()
fmt.Println("Sort Emp according to their" +"
Age:")
fmt.Println(Emp)
8
Cloud Computing 2021-22
// Sorting Emp by their date of birth using
Slice() function
sort.Slice(Emp, func(p, q int) bool {
return Emp[p].dob < Emp[q].dob
})
fmt.Println()
fmt.Println("Sort Emp according to the their
Date of Birth:")
fmt.Println(Emp)
Emp[5].salary = 92300
fmt.Println(Emp[5])
Emp[8].salary = 92300
fmt.Println(Emp[8])
Emp[4].salary = 92300
fmt.Println(Emp[4])
fmt.Println(Emp)
fmt.Println()
10
Cloud Computing 2021-22
Output :
11
Cloud Computing 2021-22
Product Manager 35 16-03-1986 19-06-2012 300000} {13 CSO 45 16-03-1986 19-06-
2012 300000} {2 CFO 34 06-02-1934 21-11-2010 130000}]
For Employee
{17 Consultant 25 26-04-1986 03-09-2013 85000}
Retirement Age is
2057
For Employee
{9 Senior VP 23 15-12-1979 03-07-2005 90000}
Retirement Age is
2059
For Employee
{6 HR 44 15-12-1979 03-07-2005 90000}
Retirement Age is
2038
For Employee
{7 Analyst 40 04-07-1982 12-11-2009 92300}
Retirement Age is
2042
For Employee
{10 Sales exec 38 04-07-1982 12-11-2009 92300}
Retirement Age is
2044
12
Cloud Computing 2021-22
For Employee
{2 CFO 34 06-02-1934 21-11-2010 130000}
Retirement Age is
2048
For Employee
{12 Senior Consultant 29 06-04-1990 08-05-2015 150000}
Retirement Age is
2053
For Employee
{18 Director 41 06-04-1990 08-05-2015 92300}
Retirement Age is
2041
For Employee
{3 CMO 35 28-06-1986 03-02-2011 150000}
Retirement Age is
2047
For Employee
{4 COO 31 07-01-1989 08-05-2009 250000}
Retirement Age is
2051
For Employee
{8 VP 52 16-03-1986 19-06-2012 300000}
Retirement Age is
2030
For Employee
{5 Product Manager 35 16-03-1986 19-06-2012 300000}
Retirement Age is
2047
For Employee
{13 CSO 45 16-03-1986 19-06-2012 300000}
Retirement Age is
2037
For Employee
{1 CEO 35 07-04-1991 05-03-2007 410000}
Retirement Age is
2047
Program exited.
Questions
1) Does GO language have exceptions
2) How does Goroutines help us in Cloud
3) List the advantages and limitations of GO language
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
15
Cloud Computing 2021-22
Theory:
GreenCloud provides the features for detailed fne-grained modelling of the energy consumed
by the data centre IT equipment like servers, communication switches and communication
links. It helps researchers to investigate, observe, interact and measure the cloud’s
performance based on multiple parameters. Most of the code of GreenCloud is written in C++.
TCL is also included in the library of GreenCloud. GreenCloud is an extension of the network
simulator ns-2 that is widely used for creating and executing network scenarios. It provides
the simulation environment that enables energy-aware cloud computing data centres.
GreenCloud mainly focuses on the communications within a cloud. Here, all of the processes
related to communication are simulated at the packet level.
Procedure:
GreenCloud can be installed in both non-VM as well as VM based machine
If you are building GreenCloud on a non-VM machine, here are the basic instructions:
1. Download GreenCloud
2. Unpack the downloaded software. It comes already integrated into NS-2 source
code.
3. Navigate to the extracted directory.
4. Run ./install.sh to do a full installation (it should work on any Debian- based system
with a 3.2+ kernel, i.e. Ubuntu 12.x and higher).
5. Execute the simulation script by running ./run
6. View the dashboard by opening show-dashboard.html
For a VM based machine Unzip and extract and run the GreenCloud Simulator file.
Results and Graph
Output results can be accessed as raw values in trace files (./traces/) or in the form of a
1
Cloud Computing 2021-22
dashboard generated from the raw values. The first section of the dashboard presents the
summary for simulation identified by the time it was executed. The summary includes a text
overview and a pie chart which presents the total energy consumed by different data center
IT systems.
2
Cloud Computing 2021-22
The Data Center Network section describes link loads and queue sizes of data center
networking links.
3
Cloud Computing 2021-22
The Uplink and Downlink statistics are grouped in the respective subsections. The plots
against time present the status of single links selected in setup_params.tcl.
4
Cloud Computing 2021-22
This last section presents the Energy Consumption statistic as the total consumption per
server and switch.
Scenarios to be completed:
Exercise 1 – Complete the table based on variation in number of users (other parameters
remains same)
Switch Energy Switch Energy Switch Energy Server
Users (core) (agg) (access) Energy Total Energy
10 51.1 102.1 9.0 509.1 671.3
15 51.1 102.1 9.0 510.2 672.4
20 51.1 102.1 9.0 510.8 673
5
Cloud Computing 2021-22
Exercise 2 - Complete the table based on variation in number of servers (other parameters
remains same. Assume 05 users this time)
Switch Energy Switch Energy Switch Energy Server
Servers (core) (agg) (access) Energy Total Energy
100 51.1 102.1 6.1 346.2 505.5
144 51.1 102.1 9.0 498.2 660.4
188 51.1 102.1 12.0 649.2 814.4
[Plot the graph of Exercise 1 and Exercise 2 and write your observations here]
6
Cloud Computing 2021-22
7
Cloud Computing 2021-22
8
Cloud Computing 2021-22
User variation
Users : 10
9
Cloud Computing 2021-22
10
Cloud Computing 2021-22
11
Cloud Computing 2021-22
Users : 15
12
Cloud Computing 2021-22
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
Users : 20
15
Cloud Computing 2021-22
16
Cloud Computing 2021-22
17
Cloud Computing 2021-22
Server variations
Servers : 100
18
Cloud Computing 2021-22
19
Cloud Computing 2021-22
Servers :144
20
Cloud Computing 2021-22
21
Cloud Computing 2021-22
22
Cloud Computing 2021-22
Servers : 188
23
Cloud Computing 2021-22
24
Cloud Computing 2021-22
25
Cloud Computing 2021-22
26
Cloud Computing 2021-22
Experiment 7: Oracle VM
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand Oracle VM Virtual Box template
2. Understand IAAS model of Cloud
3. Configure and make use of server pool and storage in Oracle VM
Theory:
Oracle VM VirtualBox is a free and widely used desktop virtualization tool. It is installed on
an x86 operating system (OS) and is available on all the major x86 operating systems, for
example, Windows XP, Windows 7, many Linux distributions, Apple Mac OS, Oracle Solaris
10, and Oracle Solaris 11. Therefore, the native operating system on the laptop, desktop,
server machine used for this lab can be any of these operating systems.
Pre-requisite:
The minimal configuration needed for your laptop, desktop, or server is as follows:
Minimum 8 GB of memory
Intel or AMD x86 quad-thread processor (quad-core or dual-core dual-thread)
15 GB of disk space to store the files needed to start the lab
Downloads:
Oracle VM VirtualBox template for Oracle VM Manager 3.1.1 build 365 (2.75 GB)
Oracle VM VirtualBox template for Oracle VM Server 3.1.1 build 365 (249 MB)
1
Cloud Computing 2021-22
$ su -
# rpm -ivh jre-7u4-linux-x64.rpm
Procedure:
2
Cloud Computing 2021-22
1. In Oracle VM VirtualBox, select File-> Import Appliance and then select the file
OracleVMServer3.1.1build365.ova.
2. In the Appliance Import Settings screen, do the following:
a. Double-click OracleVMServer3.1.1 and change the name to HOL9558_ovm-srv.
b. Change CPU to 3
c. If you have 8 GB of RAM, change RAM to 2048 MB.
d. Click Import.
3. Accept the license agreement.
4. Wait for the import to complete (about two minutes).
5. Modify the storage settings:
a. Select the VM HOL9558_vm-srv and click Settings.
b. Go to Storage and remove the disk2 entry under SATA Controller by clicking icon, as
shown in Figure below.
6. Add a new 100 GB hard drive (we will use this drive later as a repository) by selecting
SATA Controller and clicking icon “+”. Select VDI as the file type, choose Dynamically
allocated, specify the name repo, and specify a size of 100 GB.
3
Cloud Computing 2021-22
7. Modify the network settings, as shown in Figure 3:
a. Go to Network, and click the Adapter 1 tab.
b. From the Attached to list, select Host-only Adapter.
c. Expand the Advanced section.
d. For Promiscuous Mode, select Allow All. Make sure you enable the promiscuous mode
on the network interface. If you do not, your Oracle VM virtual machine will not be
accessible from your host machine.
4
Cloud Computing 2021-22
5
Cloud Computing 2021-22
To gain access to the mouse, press the rightmost CTRL key on your keyboard and
minimize the ovm-srv window.
Important: On your physical machine's native OS, open a new terminal window by
selecting Applications-> System Tools-> Terminal and run ping 192.168.56.2 to
check that the Oracle VM VirtualBox virtual machine is OK.
6
Cloud Computing 2021-22
1. In Oracle VM VirtualBox, select File-> Import Appliance and then select the file
OracleVMManager3.1.1build365.ova.
2. In the Appliance Import Settings screen, do the following:
a. Double-click OracleVMManager3.1.1build365 and change the name to
HOL9558_ovm-mgr.
b. Click Import.
Note: There is no need to change the RAM (4096 MB) or the CPU (1 vcpu) settings.
3. Accept the license agreement.
4. Wait for the import to complete (about three minutes).
5. Modify the network settings:
a. Select the VM ovm-mgr and click Settings.
b. Go to Network and click the Adapter 1 tab.
c. From the Attached to list, select Host-only Adapter.
Note: There is no need to set promiscuous mode to "Allow all" for the network interface
because no VM will run on the manager.
7
Cloud Computing 2021-22
2. Look at the VM console and wait for the message New Unix Passwd.
8
Cloud Computing 2021-22
4. Wait for the VM to be ready (when the GNOME desktop is displayed), which takes about
two minutes.
9
Cloud Computing 2021-22
5. To gain access to the mouse, press the rightmost CTRL key on your keyboard and
minimize the ovm-mgr window.
10
Cloud Computing 2021-22
11
Cloud Computing 2021-22
12
Cloud Computing 2021-22
d. Leave all other parameters as the default values by clicking Next-> Next-> Next-> Finish.
4. Create VNICs.
The VNIC Manager creates VNICs, which can be used by virtual machines as network cards.
You create virtual network interfaces by defining a range of MAC addresses to use for each
VNIC.
13
Cloud Computing 2021-22
The virtual machines running on a server can be "live migrated" to another server in the
same pool. Of course, you need to have a shared storage system (NFS, iSCSI, FC)
between all the servers of the pool. Such a pool is called clustered.
Here, for simplicity, we will create a non-clustered server pool with a single server and
use a local HDD (hard disk drive) (actually a file on the laptop that is seen by Oracle VM
VirtualBox as an HDD).
14
Cloud Computing 2021-22
Note: The Virtual IP Address of the pool is assigned to one server in the pool, called the master
server. Several actions are executed by this master server. In a non-clustered pool with just
one server, the server is also the master server. In a clustered pool with at least two servers,
one of them is the master server. In case of failure on the master server, another server will
take the master server pool and get this IP address. If you want to create a clustered server
pool (not needed here), you need to have dedicated storage (NFS file system, or iSCSI LUN, or
FC LUN) of at least 12 GB.
f. Click Next.
g. In the next window, click icon to add all servers (here there is only one) to the pool, and
click Finish.
You should now see the pool and our Oracle VM Server, ovm-srv, in it.
15
Cloud Computing 2021-22
My PC was lagging while taking screenshots so I physically took photos of some steps
16
Cloud Computing 2021-22
6. Create the storage repository.
Before creating VMs hosted on the server pool, we need to create at least one storage
repository to store not only the VMs' files but also ISO files, templates, and so on.
f. Click Finish.
g. Wait a few seconds for the repository to be created.
Note: For non-NFS storage (which is our case here), an OCFS2 (Oracle Cluster File
System) is created on the repository.
h. You should now see the new repository. Click icon to expand it.
17
Cloud Computing 2021-22
18
Cloud Computing 2021-22
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand Hadoop and HDFS
2. Understand concept of MapReduce
3. Configure and make use of Hadoop and MapReduce
Theory:
Hadoop is a software framework for distributed processing of large datasets across large
clusters of computers
Large datasets Terabytes or petabytes of data
Large clusters hundreds or thousands of nodes
Hadoop is open-source implementation for Google MapReduce. Hadoop is based on a
simple programming model called MapReduce. Hadoop is based on a simple data model,
any data will fit.
Map Reduce
1
Cloud Computing 2021-22
2
Cloud Computing 2021-22
Word Count
3
Cloud Computing 2021-22
Procedure:
For Cloudera
3. Start VirtualBox.
4
Cloud Computing 2021-22
5. Click the Folder icon.
8. Click Import.
5
Cloud Computing 2021-22
9. The virtual machine image will be imported. This can take several minutes.
10. Launch Cloudera VM. When the importing is finished, the quickstart-vm-5.4.2-0 VM
will appear on the left in the VirtualBox window. Select it and click the Start button to
launch the VM.
11. Cloudera VM booting. It will take several minutes for the Virtual Machine to start. The
booting process takes a long time since many Hadoop tools are started.
6
Cloud Computing 2021-22
12. The Cloudera VM desktop. Once the booting process is complete, the desktop will
appear with a browser.
1. Open a browser. Open the browser by click on the browser icon on the top left of the
screen.
2. Download the Shakespeare. We are going to download a text file to copy into HDFS.
Enter the following link in the
browser: http://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt
7
Cloud Computing 2021-22
2. Open a terminal shell. Open a terminal shell by clicking on the square black box on the
top right of the screen to open the command line CLI
3. Copy file to HDFS. Run hadoop fs –copyFromLocal words.txt to copy the text file to HDFS.
4. Verify file was copied to HDFS. Run hadoop fs –ls to verify the file was copied to HDFS.
8
Cloud Computing 2021-22
5. Copy a file within HDFS. You can make a copy of a file in HDFS. Run hadoop fs -cp
words.txt words2.txt to make a copy of words.txt called words2.txt
6. Copy a file from HDFS. We can also copy a file from HDFS to the local file system.
Run hadoop fs -copyToLocal words2.txt . to copy words2.txt to the local directory.
Let's run ls to see that the file was copied to see that words2.txt is there.
7. Delete a file in HDFS. Let's the delete words2.txt in HDFS. Run hadoop fs -rm words2.txt
Run hadoop fs -ls to see that the file has been deleted.
8. See example MapReduce programs. Hadoop comes with several example MapReduce
applications. You can see a list of them by running hadoop jar/urs/share/doc/hadoop-0.20-
mapreduce/examples/hadoop-examples.jar. We are interested in running WordCount.
9
Cloud Computing 2021-22
The output says that WordCount takes the name of one or more input files and the name of
the output directory. Note that these files are in HDFS, not the local file system.
9. Verify input file exists. In the previous steps we downloaded the complete works of
Shakespeare and copied them into HDFS. Let's make sure this file is still in HDFS so we can
run WordCount on it. Run hadoop fs -ls
10. See WordCount command line arguments. We can learn how to run WordCount by
examining its command-line arguments. Run hadoop jar /usr/jars/hadoop-examples.jar
wordcount.
10
Cloud Computing 2021-22
11. Run WordCount. Run WordCount for words.txt: hadoop jar /usr/jars/hadoop-
examples.jar wordcount words.txt out
As WordCount executes, the Hadoop prints the progress in terms of Map and Reduce. When
the WordCount is complete, both will say 100%.
12. See WordCount output directory. Once WordCount is finished, let's verify the output
was created. First, let's see that the output directory, out, was created in HDFS by
running hadoop fs –ls
We can see there are now two items in HDFS: words.txt is the text file that we previously
created, and out is the directory created by WordCount.
13. Look inside output directory. The directory created by WordCount contains several
files. Look inside the directory by running hadoop –fs ls out
The file part-r-00000 contains the results from WordCount. The file _SUCCESS means
WordCount executed successfully.
14. Copy WordCount results to local file system. Copy part-r-00000 to the local file
system by running hadoop fs –copyToLocal out/part-r-00000 local.txt
11
Cloud Computing 2021-22
15. View the WordCount results. View the contents of the results: more local.txt
Each line of the results file shows the number of occurrences for a word in the input file.
For example, Accuse appears four times in the input, but Accusing appears only once.
12
Cloud Computing 2021-22
Screenshots
Opening the file using cloudera vm
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
15
Cloud Computing 2021-22
Final Wordcount output
Questions:
1. What are the advantages and disadvantages of Map Reduce
2. Discuss few real applications where Map Reduce is used.
3. Write in few lines on
a) Distributed cache in MapReduce
b) Heartbeat in HDFS
c) Combiner
d) Partitioner
16
Cloud Computing 2021-22
17
Cloud Computing 2021-22
18
Cloud Computing 2021-22
19
Cloud Computing 2021-22
Experiment 9: Openstack
Learning Outcomes:
After completion of this experiment, student should be able to
1. Install and use Openstack using Devstack
2. Create Instance and check its working
3. Understand how to use Openstack
Theory:
Open Stack, also called the open source cloud operating system, is a cloud operating
system that controls large pools of compute, storage, and networking resources throughout
a datacentre, all managed through a dashboard that gives administrators control while
empowering their users to provision resources through a web interface.
Users primarily deploy the project as an infrastructure as a service (IaaS) solution. The
technology consists of a series of interrelated projects that control pools of processing,
1
Cloud Computing 2021-22
storage, and networking resources throughout a data centre. Users can manage their cloud
environment through a web-based dashboard, command-line tools, or a Restful API.
OpenStack Compute Provision’s and manage large networks of virtual machines. The
OpenStack cloud operating system enables enterprises and service providers to offer on-
demand computing resources, by provisioning and managing large networks of virtual
machines. Compute resources are accessible via APIs for developers building cloud
applications and viaweb interfaces for administrators and users. The compute architecture is
designed to scale horizontally on standard hardware, enabling the cloud economics companies
have come to expect.
OpenStack is designed to provide flexibility as you design your cloud, with no proprietary
hardware or software requirements and the ability to integrate with legacy systems and third
party technologies. It is designed to manage and automate pools of compute resources and can
work with widely available virtualization technologies, as well as bare metal and high-
performance computing (HPC) configurations. Administrators often deploy OpenStack
Compute using one of multiple supported hypervisors in a virtualized environment. KVM and
XenServer are popular choices for hypervisor technology and recommended for most use
cases. Linux container technology such as LXC is also supported for scenarios where users wish
to minimize virtualization overhead and achieve greater efficiency and performance. In addition
to different hypervisors, OpenStack supports ARM and alternative hardware architectures.
An example of some of the most popular cases of Openstack include service providers offering an
IaaS compute platform or services higher up the stack, IT departments acting as cloud service
providers for business units and project teams, Processing big data with tools like Hadoop, Scaling
compute up and down to meet demand for web resources and applications, High- performance
computing (HPC) environments processing diverse and intensive workloads
Procedure:
To install Openstack using CENT OS follow the steps below
1. Download the CENT OS file from the MS Teams folder
2. Open the file in Virtual Box
3. The CENT OS opens and command line is shown
4. When username and password is prompted for enter “openstack” for both
5. Run the following command in the terminal – sudo dhclient enp0s3
6. Open the browser on your main PC and goto browser and goto http://localhost:8888/
7. You can view the openstack login window and then use admin/openstack to enter and work
2
Cloud Computing 2021-22
Step 7: Create volumes by following step below and then click on Create Volume
3
Cloud Computing 2021-22
4
Cloud Computing 2021-22
Goto Network Topology Goto Graph Click on Toggle Labels to see screen as below
5
Cloud Computing 2021-22
Step 10: Goto Compute InstancesCreate a new Instance
Click on Launch Instance button
Enter Instance Name as “webserver” then click on Next
After Clicking on Next, In Networks Select private-net, Skip Ports and Groups tabs and goto Key Pair
and Create and save Key Pair as shown below
6
Cloud Computing 2021-22
After Key Pair is generated then click on Next till it gets disabled and then Launch Instance
After some time it will show that the instance is ready and running
7
Cloud Computing 2021-22
Questions
1) List the key components of OpenStack
2) What are the data privacy concerns in OpenStack? How to remediate them?
3) What are the favourable circumstances of utilizing OpenStack?
8
Cloud Computing 2021-22
Conclusion: In this lab we learnt about the OpenStack operating system. We then downloaded
OpenStack using CentOS and followed various steps such as creating a volume, a network, a
router and an interface and finally created an instance and were able to successfully run it.
9
Cloud Computing 2021-22
Learning Outcomes:
After completion of this experiment, student should be able to
1. Understand Eucalyptus software and its significance
2. Understand Node Controller and Front End.
3. Configure and understand use of Cloud-in-a-box
Theory:
Procedure:
Download the two setups as shown below
1. VMWare Workstation for Windows/Linux
https://www.vmware.com/in/products/workstation-pro/workstation-pro-
evaluation.html
2. Faststart-3.4.1.iso File
https://drive.google.com/file/d/1UION-home5vZWmjJWXXAq17k7Gg6DXjz/view
Create a new virtual machine and follow the steps shown below
1
Cloud Computing 2021-22
Browse and select the Image file downloaded earlier and move on to next step
2
Cloud Computing 2021-22
3
Cloud Computing 2021-22
4
Cloud Computing 2021-22
Power on the virtual machine and go ahead with installation of Eucalyptus and select
Cloud-in-a box
5
Cloud Computing 2021-22
After clicking Next The default language should be selected as English and specifically US
English the move on to next screen. Select “Yes, discard any data” Option and proceed
6
Cloud Computing 2021-22
7
Cloud Computing 2021-22
Setup a simple password and ensure that it is more than six characters.
8
Cloud Computing 2021-22
9
Cloud Computing 2021-22
On the last step select first choice of Use All Space and finish the setup
10
Cloud Computing 2021-22
11
Cloud Computing 2021-22
12
Cloud Computing 2021-22
13
Cloud Computing 2021-22
14
Cloud Computing 2021-22
15
Cloud Computing 2021-22
16
Cloud Computing 2021-22
Create user by filling up details as shown below
17
Cloud Computing 2021-22
Keep note of the detail as these will be used to login to user and management consoles
18
Cloud Computing 2021-22
19
Cloud Computing 2021-22
20
Cloud Computing 2021-22
Launch Instance
21
Cloud Computing 2021-22
22
Cloud Computing 2021-22
23
Cloud Computing 2021-22
Goto system tools and open terminal as shown below
24
Cloud Computing 2021-22
25
Cloud Computing 2021-22
26
Cloud Computing 2021-22
Final screenshots
27
Cloud Computing 2021-22
Questions
1) Discuss the advantages and limitations of Eucalyptus cloud.
28
Cloud Computing 2021-22
29
Cloud Computing 2021-22
30