Mcad22e3 Cloud Computing Notes
Mcad22e3 Cloud Computing Notes
APPLICATIONS
MCAD22E3
CLOUD COMPUTING
Second Semester
Semester – III
DIRECTORATEOFDISTANCEEDUCATION
SRMInstituteofScienceandTechnology,
Potheri, Chengalpattu District 603203, Tamil Nadu, India.
Phone:044–27417040/41
Website:www.srmist.edu.in/Email:[email protected]
SRMISTDDEMCASelfInstructionalMaterial
CourseWriter(s):Dr.R. Jayashree
DIRECTORATEOFDISTANCEEDUCATION
SRMInstituteofScienceandTechnology,
Potheri, Chengalpattu District 603203, Tamil Nadu,IndiaPhone: 044 –
27417040 / 41
Website:www.srmist.edu.in/Email:[email protected]
SRMISTDDEMCASelfInstructionalMaterial
MCAD22E3
CLOUD COMPUTING
SEMESTER – III
INSTRUCTIONAL OBJECTIVES
Student Outcomes
At the end of this course the learner is expected:
1. To understand the concepts of Cloud Computing and Learn about various
a
public cloud services
2. To explore about Web Services and Service Oriented Architecture. a
3. To learn about Cloud Management Products,Cloud Storage and Cloud
e l k
Security.
SRMISTDDEMCASelfInstructionalMaterial
HDFS MapReduce - Google App Engine (GAE) - Programming Environment for
GAE - Architecture of GFS - Case Studies: Openstack, Heroku and Docker
Containers - Amazon EC2 – AWS - Microsoft Azure - Google Compute Engine
TEXT BOOKS
1. Andrew S. Tanenbaum, Maarten Van Steen, “Distributed Systems - Principles and Paradigms”, Second Edition,
Pearson, 2006.
2. Buyya R., Broberg J., Goscinski A., “Cloud Computing: Principles and Paradigm”, John Wiley& Sons, 2011.
REFEENCE:
1. Kai Hwang, Geoffrey C Fox, Jack G Dongarra, "Distributed and Cloud Computing, From Parallel Processing to
the Internet of Things", Morgan Kaufmann Publishers, 2012.
2. Mukesh Singhal, "Advanced Concepts In Operating Systems", McGraw Hill Series in Computer Science, 1994.
3.John W.Rittinghouse, James F.Ransome, "Cloud Computing: Implementation
"Management, and Security", CRC Press, 2010
SRMISTDDEMCASelfInstructionalMaterial
CONTENTS
Module 1
Introduction to Distributed Systems – Characteristics - Issues in Distributed
Systems - Distributed System Model - Request/Reply Protocols – RMI
Module 2
Logical Clocks and Casual Ordering of Events –CAP Theorem - Election Algorithm
- Distributed Mutual Exclusion - Distributed Deadlock Detection Algorithms
Module 3
Introduction to Cloud Computing - Evolution of Cloud Computing - Cloud
Characteristics- Elasticity in Cloud - On-demand Provisioning - NIST Cloud
Computing Reference Architecture - Architectural Design Challenges
Module 4
Deployment Models: Public, Private and Hybrid Clouds - Service Models: IaaS-
PaaS – SaaS - Benefits of Cloud Computing - Disadvantages of cloud computing
Module 5
Introduction to Web Service and Service Oriented Architecture - SOAP and REST
– Basics of Virtualization - Full and Para Virtualization - Implementation Levels
of Virtualization
Module 6
Tools and Mechanisms - Virtualization of CPU - Memory – I/O Devices - Desktop
Virtualization - Server Virtualization
Module 7
Resource Provisioning and Methods - Cloud Management Products - Cloud
Storage – Provisioning Cloud Storage - Managed and Unmanaged Cloud Storage
Module 8
Cloud Security Overview - Cloud Security Challenges - Architecture Design –
Virtual Machine Security - Data Security
Module 9
HDFS MapReduce - Google App Engine (GAE) - Google App- Programming
Environment for GAE
Module 10
Case Studies: Openstack, Heroku and Docker Containers - Amazon EC2 – AWS -
Microsoft Azure - Google Compute Engine
SRMISTDDEMCASelfInstructionalMaterial
MCAD22E3CLOUD COMPUTING
MODULEI
1.2. Characteristics
1.6. RMI
1.1. Introduction to Distributed Systems
Distributed Systems
A distributed system contains multiple nodes that are physically separate but linked
together using the network. All the nodes in this system communicate with each other
and handle processes in tandem. Each of these nodes contains a small part of the
distributed operating system software.
A diagram to better explain the distributed system is −
The nodes in the distributed systems can be arranged in the form of client/server
systems or peer to peer systems. Details about these are as follows –
a) Client/Server Systems
In client server systems, the client requests a resource and the server provides that
resource. A server may serve multiple clients at the same time while a client is in
contact with only one server. Both the client and server usually communicate via a
computer network and so they are a part of distributed systems.
b) Peer to Peer Systems
The peer to peer systems contains nodes that are equal participants in data sharing. All
the tasks are equally divided between all the nodes. The nodes interact with each other
as required as share resources. This is done with the help of a network.
Advantages of Distributed Systems
● All the nodes in the distributed system are connected to each other. So nodes
can easily share data with other nodes.
● More nodes can easily be added to the distributed system i.e. it can be scaled
as required.
● Failure of one node does not lead to the failure of the entire distributed system.
Other nodes can still communicate with each other.
● Resources like printers can be shared with multiple nodes rather than being
restricted to just one.
●
1.2. Characteristics
Resource sharing means that the existing resources in a distributed system can be
accessed or remotely accessed across multiple computers in the system.
Hardware resources are shared for reductions in cost and convenience. Data is shared
for consistency and exchange of information.
● Heterogeneity
● Openness
● Concurrency
Concurrency is a property of a system representing the fact that multiple activities are
executed at the same time. The concurrent execution of activities takes place in
different components running on multiple machines as part of a distributed system. In
addition, these activities may perform some kind of interactions among them.
Concurrency reduces the latency and increases the throughput of the distributed
system.
● Scalability
● Fault Tolerance
In a distributed system hardware, software, network anything can fail. The system
must be designed in such a way that it is available all the time even after something
has failed.
● Transparency
The banker’s algorithm is a resource allocation and deadlock avoidance algorithm that
tests for safety by simulating the allocation for predetermined maximum possible
amounts of all resources, then makes an “s-state” check to test for possible activities,
before deciding whether allocation should be allowed to continue.
In other words, the bank would never allocate its money in such a way that it can no
longer satisfy the needs of all its customers. The bank would try to be in safe state
always.
Following Data structures are used to implement the Banker’s Algorithm:
Let ‘n’ be the number of processes in the system and ‘m’ be the number of resources
types.
Available :
● It is a 1-d array of size ‘m’ indicating the number of available resources of each
type.
● Available[ j ] = k means there are ‘k’ instances of resource type Rj
Max :
● It is a 2-d array of size ‘n*m’ that defines the maximum demand of each process
in a system.
● Max[i, j ] = k means process Pi may request at most ‘k’ instances of resource
type Rj.
Allocation :
● It is a 2-d array of size ‘n*m’ that defines the number of resources of each type
currently allocated to each process.
● Allocation[i, j ] = k means process Pi is currently allocated ‘k’ instances of
resource type Rj
Need :
● It is a 2-d array of size ‘n*m’ that indicates the remaining resource need of each
process.
● Need [ i, j ] = k means process Pi currently need ‘k’ instances of resource type Rj
for its execution.
Allocationi specifies the resources currently allocated to process P i and Needi specifies
the additional resources that process Pi may still request to complete its task.
Banker’s algorithm consists of Safety algorithm and Resource request algorithm
b)Safety Algorithm
The algorithm for finding out whether or not a system is in a safe state can be
described as follows:
1) Let Work and Finish be vectors of length ‘m’ and ‘n’ respectively.
Initialize: Work = Available
Finish[i] = false; for i=1, 2, 3, 4….n
2) Find an i such that both
a) Finish[i] = false
b) Needi <= Work
if no such i exists goto step (4)
3) Work = Work + Allocation[i]
Finish[i] = true
goto step (2)
4) if Finish [i] = true for all i
then the system is in a safe state
c)Resource-Request Algorithm
Let Requesti be the request array for process Pi. Requesti [j] = k means process
Pi wants k instances of resource type R j. When a request for resources is made by
process Pi, the following actions are taken:
d)Example:
Considering a system with five processes P 0 through P4 and three resources of type A,
B, C. Resource type A has 10 instances, B has 5 instances and type C has 7 instances.
Suppose at time t0 following snapshot of the system has been taken:
Question1. What will be the content of the Need matrix?
Question2. Is the system in a safe state? If Yes, then what is the safe sequence?
We must determine whether this new system state is safe. To do so, we again execute
Safety algorithm on the above data structures.
Hence the new system state is safe, so we can immediately grant the request for
process P1 .
// Available Resources
avail = new int[] { 3, 3, 2 };
}
void isSafe()
{
int count=0;
//visited array to find the already allocated process
booleanvisited[] = new boolean[n];
for (int i = 0;i< n; i++)
{
visited[i] = false;
}
//work array to store the copy of available resources
int work[] = new int[m];
for (int i = 0;i< m; i++)
{
work[i] = avail[i];
}
while (count<n)
{
boolean flag = false;
for (int i = 0;i< n; i++)
{
if (visited[i] == false)
{
int j;
for (j = 0;j< m; j++)
{
if (need[i][j] > work[j])
break;
}
if (j == m)
{
safeSequence[count++]=i;
visited[i]=true;
flag=true;
void calculateNeed()
{
for (int i = 0;i< n; i++)
{
for (int j = 0;j< m; j++)
{
need[i][j] = max[i][j]-alloc[i][j];
}
}
}
gfg.initializeValues();
//Calculate the Need Matrix
gfg.calculateNeed();
f)Python3
# Banker's Algorithm
# Driver code:
if __name__=="__main__":
# Allocation Matrix
alloc = [[0, 1, 0 ],[ 2, 0, 0 ],
[3, 0, 2 ],[2, 1, 1] ,[ 0, 0, 2]]
# MAX Matrix
max = [[7, 5, 3 ],[3, 2, 2 ],
[ 9, 0, 2 ],[2, 2, 2],[4, 3, 3]]
avail = [3, 3, 2] # Available Resources
f = [0]*n
ans = [0]*n
ind = 0
for k in range(n):
f[k] = 0
need = [[ 0 for i in range(m)]fori in range(n)]
for i in range(n):
for j in range(m):
need[i][j] = max[i][j] - alloc[i][j]
y=0
for k in range(5):
for i in range(n):
if (f[i] == 0):
flag = 0
for j in range(m):
if (need[i][j] > avail[j]):
flag = 1
break
if (flag == 0):
ans[ind] = i
ind += 1
for y in range(m):
avail[y] += alloc[i][y]
f[i] = 1
1. Architectural Models
2. Interaction Models
3. Fault Models
1. Architectural Models
a) Client-server model
The system is structured as a set of processes, called servers, that offer services to the
users, called clients.
A server can itself request services from other servers; thus, in this new relation,
the server itself acts like a client.
b) Peer-to-peer
● A large number of data objects are shared; any individual computer holds only
a small part of the application database.
● Processing and communication loads for access to objects are distributed
across many computers and access links.
● This is the most general and flexible model.
Main features:
Important consequences:
1. In a synchronous distributed system there is a notion of global physical time
(with a known relative precision depending on the drift rate).
2. Only synchronous distributed systems have a predictable behavior in terms of
timing. Only such systems can be used for hard real-time applications.
3. In a synchronous distributed system it is possible and safe to use timeouts in
order to detect failures of a process or communication link.
4. It is difficult and costly to implement synchronous distributed systems.
Important consequences:
● Failures can occur both in processes and communication channels. The reason
can be both software and hardware faults.
● Fault models are needed in order to build systems with predictable behavior in
case of faults (systems which are fault tolerant).
● such a system will function according to the predictions, only as long as the
real faults behave as defined by the “fault model”.
Request/Reply Communication
a)Synchronous Messaging
In a synchronous call, a client sends a request to a server, which performs the
requested action while the client waits. The server then sends the reply to the
client, which receives the reply. This is known as Synchronous Request/Reply
Communication.
b) Asynchronous Messaging
In an asynchronous call, the client does not wait for a service request it has
submitted to finish before undertaking other tasks. Instead, after issuing a request,
the client performs additional tasks (which may include issuing more requests).
When a reply to the first request is available, the client retrieves it.This is known as
Asynchronous Request/Reply Communication.
1.6. RMI
RMI stands for Remote Method Invocation. It is a mechanism that allows an object
residing in one system (JVM) to access/invoke an object running on another JVM.
RMI is used to build distributed applications; it provides remote communication
between Java programs. It is provided in the package java.rmi.
● Inside the server program, a remote object is created and reference of that
object is made available for the client (using the registry).
● The client program requests the remote objects on the server and tries to
invoke its methods.
The following diagram shows the architecture of an RMI application.
Let us now discuss the components of this architecture.
Whenever a client invokes a method that accepts parameters on a remote object, the
parameters are bundled into a message before being sent over the network. These
parameters may be of primitive type or objects. In case of primitive type, the
parameters are put together and a header is attached to it. In case the parameters are
objects, then they are serialized. This process is known as marshalling.
At the server side, the packed parameters are unbundled and then the required
method is invoked. This process is known as unmarshalling.
d)RMI Registry
RMI registry is a namespace on which all server objects are placed. Each time the
server creates an object, it registers this object with the RMIregistry
(using bind() or reBind() methods). These are registered using a unique name
known as bind name.
To invoke a remote object, the client needs a reference of that object. At that time,
the client fetches the object from the registry using its bind name
(using lookup() method).
The following illustration explains the entire process –
e)Goals of RMI
● Minimize the difference between working with local and remote objects.
MCAD22E3CLOUD COMPUTING
MODULE2
a) Example
If we go outside then we have made a full plan that at which place we have to go first,
second and so on. We don’t go to second place at first and then the first place. We
always maintain the procedure or an organization that is planned before. In a similar
way, we should do the operations on our PCs one by one in an organized way.
Suppose, we have more than 10 PCs in a distributed system and every PC is doing it’s
own work but then how we make them work together. There comes a solution to this
i.e. LOGICAL CLOCK.
Method-1:
● This means that if one PC has a time 2:00 pm then every PC should have the
same time which is quite not possible. Not every clock can sync at one time.
Then we can’t follow this method.
Method-2:
●
Another approach is to assign Timestamps to events.
● Taking the example into consideration, this means if we assign the first place
as 1, second place as 2, third place as 3 and so on. Then we always know that
the first place will always come first and then so on. Similarly, If we give each
PC their individual number than it will be organized in a way that 1st PC will
complete its process first and then second and so on. But Timestamps will
only work as long as they obey causality.
b)Causality
● Taking single PC only if 2 events A and B are occurring one by one then TS(A) <
TS(B). If A has timestamp of 1, then B should have timestamp more than 1, then
only happen before relationship occurs.
● Taking 2 PCs and event A in P1 (PC.1) and event B in P2 (PC.2) then also the
condition will be TS(A) < TS(B). Taking example- suppose you are sending
message to someone at 2:00:00 pm, and the other person is receiving it at 2:00:02
pm.Then it’s obvious that TS(sender) < TS(receiver).
● Concurrent Event
This means that not every process occurs one by one, some processes are made to
happen simultaneously i.e., A || B.
d)Causal ordering
Causal ordering is a vital tool for thinking about distributed systems. Once you
understand it, many other concepts become much simpler.
(i)The fundamental property of distributed systems:
Messages sent between machines may arrive zero or more times at any point after
they are sent
This is the sole reason that building distributed systems is hard.
For example, because of this property it is impossible for two computers
communicating over a network to agree on the exact time. You can send me a
message saying "it is now 10:00:00" but I don't know how long it took for that
message to arrive. We can send messages back and forth all day but we will never
know for sure that we are synchronized.
If we can't agree on the time then we can't always agree on what order things happen
in. Suppose I say "my user logged on at 10:00:00" and you say "my user logged on at
10:00:01". Maybe mine was first or maybe my clock is just fast relative to yours. The
only way to know for sure is if something connects those two events.
For example, if my user logged on and then sent your user an email and if you
received that email before your user logged on then we know for sure that mine was
first.
This concept is called causal ordering and is written like this:
A -> B (event A is causally ordered before event B)
Let's define it a little more formally. We model the world as follows: We have a
number of machines on which we observe a series of events. These events are either
specific to one machine (eg user input) or are communications between machines. We
define the causal ordering of these events by three rules:
If A and B happen on the same machine and A happens before B then A -> B
If I send you some message M and you receive it then (send M) -> (recv M)
(ii)Clocks
Lamport clocks and Vector clocks are data-structures which efficiently approximate
the causal ordering and so can be used by programs to reason about causality.
If A -> B then LC_A < LC_B
11. Consistency
When mutable state is distributed over multiple machines each machine can receive
update events at different times and in different orders.
If the final state is dependent on the order of updates then the system must choose a
single serialisation of the events, imposing a global total order.
A distributed system is consistent exactly when the outside world can never observe
two different serialisations.
The first choice risks violating consistency if some other machine makes the same
choice with a different set of events.
The second violates availability by waiting for every other machine that could
possibly have received a conflicting event before performing the requested action.
There is no need for an actual network partition to happen - the trade-off between
availability and consistency exists whenever communication between components is
not instant.
Ordering requires waiting
Even your hardware cannot escape this law. It provides the illusion of synchronous
access to memory at the cost of availabilty. If you want to write fast parallel programs
then you need to understand the messaging model used by the underlying hardware.
2.3. Election Algorithm
a)Distributed Algorithm:
Each processor has its own memory and they communicate via communication
networks.
b)Election Algorithms:
Election algorithm assumes that every active process in the system has a unique
priority number.
The process with highest priority will be chosen as a new coordinator. Hence, when a
coordinator fails, this algorithm elects that active process which has highest priority
number. Then this number is send to every active process in the distributed system.
We have two election algorithms for two different configurations of distributed
system.
Example:
We start with 6 processes, all directly connected to each other. Process 6 is the
leader,
as it has the highest number.
Process 6 fails.
Process 3 notices that Process 6 does not respond. So it starts an
election, notifying those processes with ids greater than 3.
Both Process 4 and Process 5 respond, telling Process 3 that they'll take over from
here.
Process 4 sends election messages to both Process 5 and Process 6.
Algorithm –
1. If process P1 detects a coordinator failure, it creates new active list which is
empty initially. It sends election message to its neighbour on right and adds
number 1 to its active list.
2. If process P2 receives message elect from processes on left, it responds in 3 ways:
● (I) If message received does not contain 1 in active list then P1 adds 2 to its
active list and forwards the message.
● (II) If this is the first election message it has received or sent, P1 creates new
active list with numbers 1 and 2. It then sends election message 1 followed by
2.
● (III) If Process P1 receives its own election message 1 then active list for P1
now contains numbers of all the active processes in the system. Now Process
P1 detects highest priority number from list and elects it as the new
coordinator.
Example:
We start with 6 processes, connected in a logical ring. Process 6 is the leader, as it has
the highest number.
Process 6 fails
Process 3 notices that Process 6 does not respond. So it starts an election, sending a
message containing its id to the next node in the ring.
Process 5 passes the message on, adding its own id to the message.
Process 0 passes the message on, adding its own id to the message.
Process 1 passes the message on, adding its own id to the message.
Process 4 passes the message on, adding its own id to the message.
When Process 3 receives the message back, it knows the message has gone around the
ring, as its own id is in the list. Picking the highest id in the list, it starts the
coordinator message "5 is the leader" around the ring.
Process 5 passes on the coordinator message.
import java.io.*;
import java.util.Scanner;
class Anele{
static int n;
static int pro[] = new int[100];
static int sta[] = new int[100];
static int co;
int i,j,k,l,m;
for(i=0;i<n;i++)
{
System.out.println("For process "+(i+1)+":");
System.out.println("Status:");
sta[i]=in.nextInt();
System.out.println("Priority");
pro[i] = in.nextInt();
}
elect(ele);
System.out.println("Final coordinator is "+co);
}
Output:
Enter the number of process
7
For process 1:
Status:
1
Priority
1
For process 2:
Status:
1
Priority
2
For process 3:
Status:
1
Priority
3
For process 4:
Status:
1
Priority
4
For process 5:
Status:
1
Priority
5
For process 6:
Status:
1
Priority
6
For process 7:
Status:
0
Priority
7
Which process will initiate election?
4
Election message is sent from 4 to 5
Election message is sent from 5 to 6
Election message is sent from 6 to 7
Election message is sent from 5 to 7
Election message is sent from 4 to 6
Election message is sent from 6 to 7
Election message is sent from 4 to 7
Final coordinator is 6
import java.util.Scanner;
class Process{
public int id;
public boolean active;
}
public class Ring{
int noOfProcesses;
Process[] processes;
Scanner sc;
public Ring(){
sc=new Scanner(System.in);
}
public void initialiseRing(){
System.out.println(“Enter no of processes”);
noOfProcesses=sc.nextInt();
processes = new Process[noOfProcesses];
for(int i=0;i<processes.length;i++){
processes[i]= new Process(i);
}
}
}
}
return maxIdIndex;
}
public void performElection(){
while(true){
if(processes[next].active){
System.out.println(“Process “+processes[prev].id+” pass
Election(“+processes[prev].id+”) to”+processes[next].id);
prev=next;
}
next = (next+1)%noOfProcesses;
if(next == initiatorProcesss){
break;
}
}
prev = coordinator;
next =(prev+1)%noOfProcesses;
while(true){
if(processes[next].active)
{
System.out.println(“Process “+ processes[prev].id +” pass
Coordinator(“+coordinator+ “) message to process “+processes[next].id );
prev = next;
}
next = (next+1) % noOfProcesses;
if(next == coordinator)
{
System.out.println(“End Of Election “);
break;
}
}
Output:
C:\Users\Garry\Desktop\CLIX\Bully>java Bully
Enter No of Processes
5
Process no 4 fails
Process 0Passes Election(0) message to process 1
Process 0Passes Election(0) message to process 2
Process 0Passes Election(0) message to process 3
Process 1Passes Ok(1) message to process 0
Process 2Passes Ok(2) message to process 0
Process 3Passes Ok(3) message to process 0
Process 1Passes Election(1) message to process 2
Process 1Passes Election(1) message to process 3
Process 2Passes Ok(2) message to process 1
Process 3Passes Ok(3) message to process 1
Process 2Passes Election(2) message to process 3
Process 3Passes Ok(3) message to process 2
Finally Process 3 Becomes Coordinator
Process 3Passes Coordinator(3) message to process 2
Process 3Passes Coordinator(3) message to process 1
Process 3Passes Coordinator(3) message to process 0
End of Election
2.4. Distributed Mutual Exclusion
Mutual exclusion in distributed system
In single computer system, memory and other resources are shared between different
processes. The status of shared resources and the status of users is easily available in
the shared memory so with the help of shared variable (For example: Semaphores)
mutual exclusion problem can be easily solved.
In Distributed systems, we neither have shared memory nor a common physical clock
and there for we cannot solve mutual exclusion problem using shared variables. To
eliminate the mutual exclusion problem in distributed system approach based on
message passing is used.
A site in distributed system does not have complete information of state of the system
due to lack of shared memory and a common physical clock.
● No Deadlock:
Two or more site should not endlessly wait for any message that will never arrive.
● No Starvation:
Every site who wants to execute critical section should get an opportunity to
execute it in finite time. Any site should not wait indefinitely to execute critical
section while other site are repeatedly executing critical section
● Fairness:
Each site should get a fair chance to execute critical section. Any request to
execute critical section must be executed in the order they are made i.e Critical
section execution requests should be executed in the order of their arrival in the
system.
● Fault Tolerance:
As we know shared variables or a local kernel can not be used to implement mutual
exclusion in distributed systems. Message passing is a way to implement mutual
exclusion. Below are the three approaches based on message passing to implement
mutual exclusion in distributed systems:
● If a site possesses the unique token, it is allowed to enter its critical section
● This approach uses sequence number to order requests for the critical section.
● Each requests for critical section contains a sequence number. This sequence
number is used to distinguish old and current requests.
● This approach insures Mutual exclusion as the token is unique
Example:
● Suzuki-Kasami’s Broadcast Algorithm
● A site communicates with other sites in order to determine which sites should
execute critical section next. This requires exchange of two or more successive
round of messages among sites.
● This approach use timestamps instead of sequence number to order requests
for the critical section.
● When ever a site make request for critical section, it gets a timestamp.
Timestamp is also used to resolve any conflict between critical section
requests.
● All algorithm which follows non-token based approach maintains a logical
clock. Logical clocks get updated according to Lamport’s scheme
Example:
● Lamport's algorithm, Ricart–Agrawala algorithm
● Instead of requesting permission to execute the critical section from all other
sites, Each site requests only a subset of sites which is called a quorum.
● Any two subsets of sites or Quorum contains a common site.
● This common site is responsible to ensure mutual exclusion
In a distributed system deadlock can neither be prevented nor avoided as the system is
so vast that it is impossible to do so. Therefore, only deadlock detection can be
implemented. The techniques of deadlock detection in the distributed system require
the following:
● Progress –
The method should be able to detect all the deadlocks in the system.
● Safety –
There are three approaches to detect deadlocks in distributed systems. They are as
follows:
I. Centralized approach –
MCAD22E3CLOUD COMPUTING
MODULE3
A set of new technologies has come along that, along with the need for more
efficient and affordable computing, has enabled an on-demand system to
develop.
Clouds can come in many different types, and the services and applications that
run on clouds may or may not be delivered by a cloud service provider. These
different types and levels of cloud services mean that it is important to define
what type of cloud computing system you are working with.
The Internet offers abstraction, runs using the same set of protocols and
standards, and uses the same applications and operating systems. These same
characteristics are found in an intranet, an internal version of the Internet.
Cloud computing is an abstraction based on the notion of pooling physical
resources and presenting them as a virtual resource. It is a new model for
provisioning resources, for staging applications, and for platform-independent
user access to services.
Examples:
The cloud computing has changed the nature of commercial system
deployment, consider these three examples:
Google: In the last decade, Google has built a worldwide network of datacenters
to service its search engine. In doing so Google has captured a substantial portion
of the world’s advertising revenue. That revenue has enabled Google to offer free
software to users based on that infrastructure and has changed the market for
user-facing software. This is the classic Software as a Service case.
Cloud computing is all about renting computing services. This idea first came in the
1950s. In making cloud computing what it is today, five technologies played a vital
role. These are distributed systems and its peripherals, virtualization, web 2.0, service
orientation, and utility computing.
● Distributed Systems:
● Mainframe computing:
Mainframes which first came into existence in 1951 are highly powerful and
reliable computing machines. These are responsible for handling large data such
as massive input-output operations. Even today these are used for bulk
processing tasks such as online transactions etc. These systems have almost no
downtime with high fault tolerance. After distributed computing, these increased
the processing capabilities of the system. But these were very expensive. To
reduce this cost, cluster computing came as an alternative to mainframe
technology.
● Cluster computing:
● Grid computing:
In 1990s, the concept of grid computing was introduced. It means that different
systems were placed at entirely different geographical locations and these all
were connected via the internet. These systems belonged to different
organizations and thus the grid consisted of heterogeneous nodes. Although it
solved some problems but new problems emerged as the distance between the
nodes increased. The main problem which was encountered was the low
availability of high bandwidth connectivity and with it other network associated
issues. Thus. cloud computing is often referred to as “Successor of grid
computing”.
● Virtualization:
It was introduced nearly 40 years back. It refers to the process of creating a
virtual layer over the hardware which allows the user to run multiple instances
simultaneously on the hardware. It is a key technology used in cloud computing.
It is the base on which major cloud computing services such as Amazon EC2,
VMware vCloud, etc work on. Hardware virtualization is still one of the most
common types of virtualization.
● Web 2.0:
It is the interface through which the cloud computing services interact with the
clients. It is because of Web 2.0 that we have interactive and dynamic web pages.
It also increases flexibility among web pages. Popular examples of web 2.0
include Google Maps, Facebook, Twitter, etc. Needless to say, social media is
possible because of this technology only. In gained major popularity in 2004.
● Service orientation:
● Utility computing:
Paradigm shift
When you choose a cloud service provider, you are renting or leasing part of an
enormous infra-structure of datacenters, computers, storage, and networking
capacity.
For example, there are some 20 datacenters in Amazon Web Service’s cloud
and Google’s cloud includes perhaps some 35 datacenters worldwide.
Amazon.com’s infrastructure was built to support elastic demand so the system
could accommodate peak traffic on a busy shopping day such as “Black Monday”,
which is the Monday after Thanksgiving in the United States when Internet
Christmas sales traditionally start. Because much of the capacity was idle,
Amazon.com first opened its network to partners and then as Amazon Web
Services to customers.
As these various datacenters grew in size, businesses have developed their
datacenters as “green-field” projects. Datacenters have been sited to do the
following:
● Collaboration applications
● Cloud backup
● Business applications
The last five years have seen a proliferation of services and productivity
applications delivered on-line as cloud computing applications.
For example, many people have used ChannelAdvisor.com for their auction
listings and sales management. That site recently expanded its service to include
a CRM connector to Salesforce.com. One of the largest call center operations
companies is a cloud-based service, Liveops.com.
The cost advantages of cloud computing have enabled new software vendors to
create productivity applications that they can make available to people at a
much smaller cost.
For example, if a customer needs to utilize additional servers for the duration of
a project, they can do so and then drop back to the previous level after the
project is completed.
ODC make computing resources such as storage capacity, computational speed
and software applications available to users as needed for specific temporary
projects, known or unexpected workloads, routine work, or long-term
technological and computing requirements.
FIGURE 1.1
The NIST cloud model doesn’t address a number of intermediary services such
as transaction or service brokers, provisioning, integration, and
interoperability services that form the basis for many cloud computing
discussions.
The Cloud Cube Model
The Open Group maintains an association called the Jericho Forum whose main
focus is how to protect cloud networks. The group has an interesting model that
attempts to categorize a cloud network based on four dimensional factors.
The type of cloud networks you use dramatically changes the notion of where the
boundary between the client’s network and the cloud begins and ends.
The four dimensions of the Cloud Cube Model are shown in Figure 1.2 and listed
here:
Physical location of the data:
Internal (I) / External (E) determines your organization’sboundaries.
Ownership:
Proprietary (P) / Open (O) is a measure of not only the technology
ownership,but of interoperability, ease of data transfer, and degree of vendor
application lock-in.
Security boundary:
Perimeterised (Per) / De-perimiterised (D-p) is a measure of whetherthe
operation is inside or outside the security boundary or network firewall.
Cloud computing is used for enabling global access to mutual pools of resources such
as services, apps, data, servers, and computer networks. It is done on either a third-
party server located in a data center or a privately owned cloud. This makes data-
accessing contrivances more reliable and efficient, with nominal administration effort.
Because cloud technology depends on the allocation of resources to attain consistency
and economy of scale, similar to a utility, it is also fairly cost-effective, making it the
choice for many small businesses and firms.
But there are also many challenges involved in cloud computing, and if you’re not
prepared to deal with them, you won’t realize the benefits. Here are six common
challenges you must consider before implementing cloud computing technology.
1. Cost
Cloud computing itself is affordable, but tuning the platform according to the
company’s needs can be expensive. Furthermore, the expense of transferring the data
to public clouds can prove to be a problem for short-lived and small-scale projects.
Companies can save some money on system maintenance, management, and
acquisitions. But they also have to invest in additional bandwidth, and the absence of
routine control in an infinitely scalable computing platform can increase costs.
3. Downtime
Downtime is a significant shortcoming of cloud technology. No seller can promise a
platform that is free of possible downtime. Cloud technology makes small companies
reliant on their connectivity, so companies with an untrustworthy internet connection
probably want to think twice before adopting cloud computing.
4. Password Security
Industrious password supervision plays a vital role in cloud security. However, the
more people you have accessing your cloud account, the less secure it is. Anybody
aware of your passwords will be able to access the information you store there.
Businesses should employ multi-factor authentication and make sure that passwords
are protected and altered regularly, particularly when staff members leave. Access
rights related to passwords and usernames should only be allocated to those who
require them.
5. Data privacy
Sensitive and personal information that is kept in the cloud should be defined as being
for internal use only, not to be shared with third parties. Businesses must have a plan
to securely and efficiently manage the data they gather.
6. Vendor lock-in
Entering a cloud computing agreement is easier than leaving it. “Vendor lock-in”
happens when altering providers is either excessively expensive or just not possible. It
could be that the service is nonstandard or that there is no viable vendor substitute.
It comes down to buyer carefulness. Guarantee the services you involve are typical
and transportable to other providers, and above all, understand the requirements.
Cloud computing is a good solution for many businesses, but it’s important to know
what you’re getting into. Having plans to address these six prominent challenges first
will help ensure a successful experience.
MCAD22E3CLOUD COMPUTING
MODULE4
Cloud Types
There are two distinct sets of models:
Deployment models: This refers to the location and management of the
cloud’s infrastructure.
Service models: This consists of the particular types of services that you can
access on a cloud computing platform.
Deployment models
A deployment model defines the purpose of the cloud and the nature of how the
cloud is located.
4.2. Public, Private and Hybrid Clouds
The NIST definition for the four deployment models is as follows:
Public cloud:
The public cloud infrastructure is available for public use alternatively for large
industry group and is owned by an organization selling cloud services.
Private cloud: The private cloud infrastructure is operated for the exclusive
use of an organization. The cloud may be managed by that organization or a
third party. Private clouds may be either on- or off-premises.
Community cloud: A community cloud is one where the cloud has been
organized to serve a common function or purpose.
It may be for one organization or for several organizations, but they share
common concerns such as their mission, policies, security, regulatory
compliance needs, and so on. A community cloud may be managed by the
constituent organization(s) or by a third party.
Figure 1.3 shows the different locations that clouds can come in.
The United States Government, under the auspices of the General Services
Administrator (GSA),launched a cloud computing portal called Apps.gov, as
shown in Figure 1.4, with the purpose ofproviding cloud services to federal
agencies. Described under the “U.S. Federal Cloud ComputingInitiative
(http://www.scribd.com/doc/17914883/US-Federal-Cloud-Computing-
nitiative-FQ-GSA), the goal of the initiative is to make large portions of the
federal government’s apparatus available under a cloud computing model. This
is a good example of a community cloud deployment, with the government being
the community.
Apps.gov is the U.S. government’s cloud computing system for its various
agencies.
Apps.gov is also making available connections to free media services from its
cloud, such as Twitter and YouTube. An example of this connection in practice
is the YouTube channel created by the White House for citizens’ outreach. You
can find the White House channel at http:// www.youtube.com/whitehouse
and the general U.S. Government YouTube channel at
http://www.youtube.com/usgovernment. You can see YouTube in action when
you visit WhiteHouse.gov and click the video link that usually appears on that
home page.
Service models
On-demand self-service:
A client can provision computer resources without the need for interaction with
cloud service provider personnel.
Resource pooling:
A cloud service provider creates resources that are pooled together in a system
that supports multi-tenant usage.
Physical and virtual systems are dynamically allocated or reallocated as
needed. Intrinsic in this concept of pooling is the idea of abstraction that hides
the location of resources such as virtual machines, processing, memory,
storage, and network bandwidth and connectivity.
Rapid elasticity:
Resources can be rapidly and elastically provisioned.
The system can add resources by either scaling up systems (more powerful
computers) or scaling out systems (more computers of the same kind), and
scaling may be automatic or manual. From the standpoint of the client, cloud
computing resources should look limit-less and can be purchased at any time
and in any quantity.
Measured service:
The use of cloud system resources is measured, audited, and reportedto the
customer based on a metered system.
A client can be charged based on a known metric such as amount of storage used,
number of transactions, network I/O (Input/Output) or bandwidth, amount of
processing power used, and so forth. A client is charged based on the level of
services provided.
Additional advantages:
Lower costs:
Because cloud networks operate at higher efficiencies and with greater
utilization, significant cost reductions are often encountered.
Ease of utilization:
Depending upon the type of service being offered, you may find thatyou do not
require hardware or software licenses to implement your service.
Quality of Service:
The Quality of Service (QoS) is something that you can obtain undercontract
from your vendor.
Reliability:
The scale of cloud computing networks and their ability to provide load
balancing and failover makes them highly reliable, often much more reliable
than what you can achieve in a single organization.
Outsourced IT management:
A cloud computing deployment lets someone else manageyour computing
infrastructure while you manage your business. In most instances, you achieve
considerable reductions in IT staffing costs.
When you use an application or service in the cloud, you are using something
that isn’t necessarily as customizable as you might want.
All cloud computing applications suffer from the inherent latency that is
intrinsic in their WAN connectivity.
All the requests you use in HTTP: PUTs, GETs, and so on are requests to a
service provider. The service provider then sends a response. Although it may
seem that you are carrying on a conversation between client and provider, there
is an architectural disconnect between the two. That lack of state allows
messages to travel over different routes and for data to arrive out of sequence,
and many other characteristics allow the communication to succeed even when
the medium is faulty. Therefore, to impose transactional coherency upon the
system, additional overhead in the form of service brokers, transaction
managers, and other middleware must be added to the system. This can
introduce a very large performance hit into some applications.
If you had to pick a single area of concern in cloud computing, that area would
undoubtedly be privacy and security. When your data travels over and rests on
systems that are no longer under your control, you have increased risk due to
the interception and malfeasance of others.
The process for creating new files is the same for all file types. Watch the video
below to learn more.
3. The Rename dialog box will appear. Type a name for your file, then
click OK.
4. Your file will be renamed. You can access the file at any time from
your Google Drive, where it will be saved automatically. Simply
double-click to open the file again.
You may notice that there is no Save button for your files. This is because Google
Drive uses autosave, which automatically and immediately saves your files as
you edit them.
How to run a Python script in the cloud?
Below is a quick step-by-step guide to starting a cloud computer also called a cloud
instance.
2. Log in to AWS
4. (Optional) In the top-right corner select the region that is closest to you.
7. Click Review and Launch and on the next page click Launch
8. You can select Create a new key pair in the pop-up window and give it any
name you want.
10. Save the .pem file somewhere you’ll remember. This is your AWS key and we
will need it later
After a short while you should see that a new instance is up and running. You can see
this on the AWS browser page under Instances.
2. You first need to change the access permissions of your AWS key file:
$ chmod 400 my-key-pair.pem
3. We now ssh into the cloud computer. This will create a secure connection
between our laptop and the cloud
$ ssh -i /path/my-key-pair.pem ec2-user@public_dns_name
If all went well, you are now remotely logged in to your cloud computer.
Your instance should come with Python 2.7 pre-installed. I will install Python 3
below. To see the Python-related packages available for install on your cloud
computer type:
$ yum list | grep python
yum is a package manager for Linux and grep is a search command applied to the
output of yum (this is what the | sign achieves). grep looks for anything with “python”
in the output of yum.
To install Python 3:
$ sudo yum install python35
The sudo prefix ensures that the above command is run with administrator privileges,
which is necessary to install stuff.
To install Python packages with pip (Python package manager), I also needed to run:
$ sudo yum install python35-devel$ sudo yum install gcc
It is best practice to set up a Python virtual environment and install any new packages
there instead of installing packages in the Python root.
You can choose a name other then “venv” for your virtual environment.
To exit the vi editor, first press Esc to exit insert mode and then press :x. This will also
save the file.
If you would like to upload existing code from your machine, use the secure copy
command below.
$ scp -ri ~/documents/path/yourkey.pem ~/documents/path/example/ ec2-user@ec2–
23–21–11–38.eu-central-1.compute.amazonaws.com:/home/ec2-user
Replace the above two files paths with the path to your AWS key and the folder
containing the Python code. You can find out the last parameter that is needed above
by right-clicking on the instance you have just started and selecting Connect.
The option -r is for recursive copy as we are copying a complete folder and option -
i stands for identity_file to provide the path to your AWS key.
If you are only uploading a single file, the -r option is not necessary.
To keep Python running even after you disconnect from the cloud instance we
install tmux.
$ sudo yum install tmux
Start tmux
$ tmux
Remember to change the above parameters to the ones you are using.
On the AWS browser page under Instances, right-click the instance you want to
terminate, select Instance State and select Terminate. This is also when billing stops.
Lab 2 :Explore public cloud services like Amazon, Google, Sales Force, Digital
Ocean etc
With the public cloud raging the market, virtual hardware is shared by many
companies. The multi-employer environment makes it easy to differentiate
infrastructure costs for multiple users. Due to cost benefits and payment model, the
public cloud is suitable for small and medium-sized businesses. In general, sensitive,
community-oriented web applications that can receive unexpected traffic can increase
the cloud mass very well.
While defining the cloud strategy for their business, enterprises can choose between a
public cloud, a private cloud or a hybrid cloud for efficient scaling. The choice
depends on several factors such as the type of business application, the costs involved,
the technical specifications, and other business requirements. In this blog, we will
take a closer look at the social cloud and its benefits for businesses.
The public cloud is the most popular computer model. In this case, cloud service
providers use the Internet and make services such as infrastructure, storage, servers
etc. available for businesses. Third-party providers own and use shared physical
hardware and provide that to companies according to their needs. Amazon Elastic
Compute Cloud (EC2), IBM Blue Cloud, Google App Engine, Sun Cloud, Microsoft
Azure are some of the most popular cloud providers in the community.
Let us get a close look into the top 3 cloud vendors that are high in demand with the
IT sector.
● Popularly known as AWS, Amazon Web Services is the leading cloud service
provider with a 33% market share.
Microsoft Azure:
● Back in 2017, Gartner called Azure a top leader in the Cloud Infrastructure as
a service space.
● Globally, 90% of Fortune 500 companies use Microsoft Azure to run their
business.
● Using the deeply integrated Azure cloud services, businesses can quickly
build, deploy and manage simple and complex systems with ease.
● Azure supports multiple programming languages, frameworks, operating
systems, information, and devices, allowing businesses to use the tools and
technologies they rely on.
● With precise looks, low cost, attractive features and flexible computer
options, GCP is an attractive option for both AWS and Azure. It uses complete
encryption for all data and communication channels including traffic between
data centres.
● Some of the areas where Google Cloud competes fiercely with AWS include
model setting and payment, privacy and traffic security, cost-effectiveness,
and machine learning.
● While all three cloud providers offer up to 75% discounts on a one- to three-
year commitment, Google Cloud additionally offers up to 30% continuous
user discount on each model that works for more than 25% per month.
● Google Cloud offers several in-house APIs related to computer viewing, native
language processing and translation. Machine learning engineers can create
models based on the Cloud Machine Learning Engine open-source
TensorFlow open-source learning.
Conclusion:
Focusing your IT team on projects that can bring in more revenue rather than working
furiously to manage on-premises systems is a predominant priority to most IT
companies. With finite resources, companies are looking to adopt cloud models that
can cater to their multiple IT requirements. Cloud-native technologies empower
organizations to build and run scalable SRP-based microservers-based applications in
modern, dynamic environments.
MCAD22E3CLOUD COMPUTING
MODULE5
SOA is an architectural style for building software applications that use services
available in a network such as the web. It promotes loose coupling between software
components so that they can be reused. Applications in SOA are built based on
services. A service is an implementation of a well-defined business functionality, and
such services can then be consumed by clients in different applications or business
processes.
SOA allows for the reuse of existing assets where new services can be created from
an existing IT infrastructure of systems. In other words, it enables businesses to
leverage existing investments by allowing them to reuse existing applications, and
promises interoperability between heterogeneous applications and technologies. SOA
provides a level of flexibility that wasn't possible before in the sense that:
Sun's Java Web Services Developer Pack 1.5 (Java WSDP 1.5) and Java 2 Platform,
Enterprise Edition (J2EE) 1.4 can be used to develop state-of-the-art web services to
implement SOA. The J2EE 1.4 platform enables you to build and deploy web services
in your IT infrastructure on the application server platform. It provides the tools you
need to quickly build, test, and deploy web services and clients that interoperate with
other web services and clients running on Java-based or non-Java-based platforms. In
addition, it enables businesses to expose their existing J2EE applications as web
services. Servlets and Enterprise JavaBeans components (EJBs) can be exposed as
web services that can be accessed by Java-based or non-Java-based web service
clients. J2EE applications can act as web service clients themselves, and they can
communicate with other web services, regardless of how they are implemented.
API Description
Note: JAX-RPC 1.1 and SAAJ 1.2 include support for the Web Services
Interoperability (WS-I) and the Web Services Interoperability Basic Profile (WSI-
BP), currently being developed by http://www.ws-i.org, which provides a set of
guidelines on how to develop interoperable web services.
With the APIs described in Table 1, you can focus on high-level programming tasks,
rather than low-level issues of XML and web services. In other words, you can start
developing and using Java WSDP 1.5 and J2EE 1.4 web services without knowing
much about XML and web services standards. You only need to deal with Java
semantics, such as method invocation and data types. The dirty work is done behind
the scenes, as discussed further in the next section.
Figure 2 illustrates how the JAXR and JAX-RPC APIs play a role in publishing,
discovering, and using web services and thus realizing SOA.
Once a web service is discovered, the client makes a request to a web service. The
web service processes the request and sends the response back to the client. To get a
feeling for what happens behind the scenes, consider Figure 2, which shows how a
Java client communicates with a Java web service in the J2EE 1.4 platform. Note that
J2EE applications can use web services published by other providers, regardless of
how they are implemented. In the case of non-Java-based clients and services, the
figure would change slightly, however.
As mentioned earlier, all the details between the request and the response happen
behind the scenes. You only deal with typical Java programming language semantics,
such as Java method calls, Java data types, and so forth. You needn't worry about
mapping Java to XML and vice-versa, or constructing SOAP messages. All this low-
level work is done behind the scenes, allowing you to focus on the high-level issues.
Figure 3: A Java Client Calling a J2EE Web
Note: J2EE 1.4 and Java WSDP 1.5 support both RPC-based and document-oriented
web services. In other words, once a service is discovered, the client can invoke
remote procedure calls on the methods offered by the service, or send an XML
document to the web service to be processed
Interoperability
Interoperability is the most important principle of SOA. This can be realized through
the use of web services, as one of the key benefits of web services is interoperability,
which allows different distributed web services to run on a variety of software
platforms and hardware architectures. The Java programming language is already a
champion when it comes to platform independence, and consequently the J2EE 1.4
and Java WSDP 1.5 platforms represent the ideal platforms for developing portable
and interoperable web services.
Interoperability and portability start with the standard specifications themselves. The
J2EE 1.4 and Java WSDP 1.5 platforms include the technologies that support SOAP,
WSDL, UDDI, and ebXML. This core set of specifications -- which are used to
describe, publish, enable discovery, and invoke web services -- are based on XML
and XML Schema. If you have been keeping up with these core specifications, you
know it's difficult to determine which products support which levels (or versions) of
the specifications. This task becomes harder when you want to ensure that your web
services are interoperable.
Since the J2EE 1.4 and Java WSDP 1.5 platforms adhere to the WS-I Basic Profile
1.0, they ensure not only that applications are portable across J2EE implementations,
but also that web services are interoperable with any web service implemented on any
other platform that conforms to WS-I standards such as .Net.
Challenges in Moving to SOA
SOA is usually realized through web services. Web services specifications may add to
the confusion of how to best utilize SOA to solve business problems. In order for a
smooth transition to SOA, managers and developers in organizations should known
that:
● SOA is an architectural style that has been around for years. Web services
are the preferred way to realize SOA.
In addition, Sun's Java BluePrints provide developers with guidelines, patterns, and
sample applications. Java BluePrints has a book on Designing Web Services with
J2EE 1.4, which is the authoritative guide to the best practices for designing and
integrating enterprise-level web services using J2EE 1.4. It provides the guidelines,
patterns, and real-world examples architects and developers need in order to shorten
the learning curve and start building robust, scalable, and portable solutions.
JSR 208 Java Business Integration (JBI), is a specification for a standard that
describes plug-in technology for system software that enables a service-oriented
architecture for building integration server software. JBI adopts SOA to maximize the
decoupling between components, and create well-defined interoperation semantics
founded on standards-based messaging. JSR 208 describes the service provider
interfaces (SPIs) that service engines and bindings plug into, as well as the normalized
message service that they use to communicate with each other. It is important to note
that JSR 208 doesn't define the engines or tools themselves. JSR 208 has the
following business advantages:
● Service engines could be implemented in any language as long as they support the
SPI definition implemented by JSR 208 compliant systems.
● New engines can be added to the container by plugging them into the standard SPI
and defining the messages they will use to interact with the rest of the system.
● ISVs that specialize in one of these components could be able to plug special-
purpose engines into industry-standard integration solutions.
● Open interfaces will enable free and open competition around the implementation
of these engines. This means that customers will be free to choose the best solution
available, and their integration code can be migrated between implementations.
A JSR 208 example architecture is shown in Figure 4.
As you can see, JBI provides an environment in which plug-in components reside.
Interaction between the plug-in components is by means of message-based service
invocation. Services produced and consumed by plug-in components are modeled
using WSDL (version 2.0). A normalized message consists of two parts: the abstract
XML message, and message metadata (or message context data), which allows for
association of extra information with a particular message as it is processed by plug-in
and system components.
Project Shasta
Sun's Project Shasta, which is based on the JSR 208 architecture, aims to build a next-
generation integration solution. This project will be implemented on Sun's J2EE
application server and leverage J2EE services such as Java Message Service (JMS),
J2EE Connector Architecture (JCA), failover, and high availability. It will feature
many of the emerging standards in the web services (such as web service notification,
coordination, and transaction management) and integration space. The project will be
focused on web services and using them to enable the creation of service-oriented
architectures. Figure 5 depicts what a fully implemented product could look like.
Web services let developers reuse existing information assets by providing developers
with standard ways to access middle-tier and back-end services and integrate them
with other applications.
Since web services represent gateways to existing back-end servers, strong support for
back-end integration is required. This is where the J2EE platform comes into play.
The J2EE platform provides industry-standard APIs (such as the J2EE Connector
Architecture, the JDBC API, Java Message Service (JMS), among others) for
accessing legacy information systems. J2EE 1.4 (which supports web services)
provides an excellent mechanism to integrate legacy EIS and expose their
functionality as interoperable web services, thereby making legacy data available on
heterogeneous platform environments.
Cloud-based API data models have not only enhanced the cloud experience, but also
provided a way for developers and administrators to integrate workloads into the
cloud using those APIs. For most enterprises, APIs let share information across
various on-premises and cloud-based applications. They also play an important role to
integrate platform workloads more seamlessly. As cloud adoption continues to grow,
there is more demand for integration points between applications inside and outside of
the cloud environment. Rise of multicloud strategy along with need for enhancement
in cross cloud capabilities have increased the dependency on cloud API environment.
But which approach is better and what support do you get in your cloud environment?
SOAP in a nutshell
SOAP (short for Simple Object Access Protocol), the older approach, had
industrywide support ranging from product companies such as IBM and Microsoft to
service implementers. It also came with a comprehensive yet complex set of
standards. Microsoft team who designed SOAP made it to be extremely flexible—to
be able to communicate over private networks, across the internet and emails. It was
supported by several standards as well. Initial version of SOAP was part of a
specification that contained Universal Description, Discovery, and Integration
(UDDI) and Web Services Description Language (WSDL) as well.
SOAP essentially provides the envelope for sending the web services messages. The
architecture itself is designed to help the performance of various operations between
software programs. Communication between programs usually happens via XML
based requests and HTTP based responses. HTTP is mostly used protocol of
communication, but other protocols may be used as well.
[ Also on InfoWorld: How to make the most of the Google Cloud free tier ]
A SOAP message contains some mandatory parts such as ENVELOPE, HEADER, BODY,
and FAULT. The ENVELOPE object defines the start and end of XML message
request, HEADER contains any header elements to be processed by the server, and
the BODY contains the remaining XML object that constitutes the
request. FAULT object is used any error handling.
REST
REST is developer-friendly because its simpler style makes it easier to implement and
use than SOAP. REST is less verbose, and less volume of data is sent when
communicating between two endpoints.
While SOAP is like using an envelope that contains lots of processing information
inside it, REST can be considered a postcard that has an URI as destination address, is
lightweight, and can be cached. REST is data-driven and is primary used to access a
resource (URI) for certain data; SOAP is a protocol that is function-driven. REST
provides flexibility in choosing data format (plain text, HTML, XML, or JSON) while
SOAP only uses XML.
1. Full Virtualization :
Full Virtualization was introduced by IBM in the year 1966. It is the first software
solution of server virtualization and uses binary translation and direct approach
technique. In full virtualization, guest OS is completely isolated by the virtual
machine from the virtualization layer and hardware. Microsoft and Parallels
systems are examples of full virtualization.
2. Paravirtualization :
S.No
. Full Virtualization Paravirtualization
To do this, you will need a hypervisor. Now, what is the role of the hypervisor? It is a
bridge between the hardware and the virtual operating system, which allows smooth
functioning.
For the basic emulation, an interpreter is needed, which interprets the source code and
then converts it into a hardware format that can be read. This then allows processing.
This is one of the five implementation levels of virtualization in cloud computing.
Multiple users will not be able to use the same hardware and also use multiple
virtualization instances at the very same time. This is mostly used in the cloud-based
infrastructure.
When there are several users, and no one wants to share the hardware, then this is
where the virtualization level is used. Every user will get his virtual environment
using a virtual hardware resource that is dedicated. In this way, there is no question of
any conflict.
It lets the high-level language programs compiled to be used in the application level
of the virtual machine run seamlessly.
MCAD22E3CLOUD COMPUTING
MODULE6
6.3. Memory
With CPU Virtualization, all the virtual machines act as physical machines and
distribute their hosting resources like having various virtual processors. Sharing of
physical resources takes place to each virtual machine when all hosting services get
the request. Finally, the virtual machines get a share of the single CPU allocated to
them, being a single-processor acting as a dual-processor.
This CPU Virtualization is software-based where with the help of it, application code
gets executed on the processor and the privileged code gets translated first, and that
translated code gets executed directly on the processor. This translation is purely
known as Binary Translation (BT). The code that gets translated is very large in size
and also slow at the same time on execution. The guest programs that are based on
privileged coding runs very smooth and fast. The code programs or the applications
that are based on privileged code components that are significant such as system calls,
run at a slower rate in the virtual environment.
There is hardware that gets assistance to support CPU Virtualization from certain
processors. Here, the guest user uses a different version of code and mode of
execution known as a guest mode. The guest code mainly runs on guest mode. The
best part in hardware-assisted CPU Virtualization is that there is no requirement for
translation while using it for hardware assistance. For this, the system calls runs faster
than expected. Workloads that require the updation of page tables get a chance of
exiting from guest mode to root mode that eventually slows down the program’s
performance and efficiency.
Despite having specific software behavior of the CPU model, the virtual machine still
helps in detecting the processor model on which the system runs. The processor
model is different based on the CPU and the wide variety of features it offers, whereas
the applications that produce the output generally utilize such features. In such cases,
vMotion cannot be used to migrate the virtual machines that are running on feature-
rich processors. Enhanced vMotion Compatibility easily handles this feature.
CPU Virtualization adds the amount of overhead based on the workloads and
virtualization used. Any application depends mainly on the CPU power waiting for
the instructions to get executed first. Such applications require the use of CPU
Virtualization that gets the command or executions that are needed to be executed
first. This overhead takes the overall processing time and results in an overall
degradation in performance and CPU virtualisation execution.
That means a two-stage mapping process should be maintained by the guest OS and
the VMM, respectively: virtual memory to physical memory and physical memory to
machine memory. Furthermore, MMU virtualization should be supported, which is
transparent to the guest OS. The guest OS continues to control the mapping of virtual
addresses to the physical memory addresses of VMs. But the guest OS cannot directly
access the actual machine memory. The VMM is responsible for mapping the guest
physical memory to the actual machine memory. Figure 3.12 shows the two-level
memory mapping procedure.
Since each page table of the guest OSes has a separate page table in the VMM
corresponding to it, the VMM page table is called the shadow page table. Nested page
tables add another layer of indirection to virtual memory. The MMU already handles
virtual-to-physical translations as defined by the OS. Then the physical memory
addresses are translated to machine addresses using another set of page tables defined
by the hypervisor. Since modern operating systems maintain a set of page tables for
every process, the shadow page tables will get flooded. Consequently, the perfor-
mance overhead and cost of memory will be very high.
Since the efficiency of the software shadow page table technique was too low, Intel
developed a hardware-based EPT technique to improve it, as illustrated in Figure
3.13. In addition, Intel offers a Virtual Processor ID (VPID) to improve use of the
TLB. Therefore, the performance of memory virtualization is greatly improved. In
Figure 3.13, the page tables of the guest OS and EPT are all four-level.
When a virtual address needs to be translated, the CPU will first look for the L4
page table pointed to by Guest CR3. Since the address in Guest CR3 is a physical
address in the guest OS, the CPU needs to convert the Guest CR3 GPA to the host
physical address (HPA) using EPT. In this procedure, the CPU will check the EPT
TLB to see if the translation is there. If there is no required translation in the EPT
TLB, the CPU will look for it in the EPT. If the CPU cannot find the translation in the
EPT, an EPT violation exception will be raised.
When the GPA of the L4 page table is obtained, the CPU will calculate the GPA of
the L3 page table by using the GVA and the content of the L4 page table. If the entry
corresponding to the GVA in the L4
page table is a page fault, the CPU will generate a page fault interrupt and will let the
guest OS kernel handle the interrupt. When the PGA of the L3 page table is obtained,
the CPU will look for the EPT to get the HPA of the L3 page table, as described
earlier. To get the HPA corresponding to a GVA, the CPU needs to look for the EPT
five times, and each time, the memory needs to be accessed four times. There-fore,
there are 20 memory accesses in the worst case, which is still very slow. To overcome
this short-coming, Intel increased the size of the EPT TLB to decrease the number of
memory accesses.
6.4. I/O Virtualization
I/O virtualization involves managing the routing of I/O requests between virtual
devices and the shared physical hardware. At the time of this writing, there are three
ways to implement I/O virtualization: full device emulation, para-virtualization, and
direct I/O. Full device emulation is the first approach for I/O virtualization. Generally,
this approach emulates well-known, real-world devices.
A single hardware device can be shared by multiple VMs that run concurrently.
However, software emulation runs much slower than the hardware it emulates
[10,15]. The para-virtualization method of I/O virtualization is typically used in Xen.
It is also known as the split driver model consisting of a frontend driver and a backend
driver. The frontend driver is running in Domain U and the backend dri-ver is running
in Domain 0. They interact with each other via a block of shared memory. The
frontend driver manages the I/O requests of the guest OSes and the backend driver is
responsible for managing the real I/O devices and multiplexing the I/O data of
different VMs. Although para-I/O-virtualization achieves better device performance
than full device emulation, it comes with a higher CPU overhead.
Direct I/O virtualization lets the VM access devices directly. It can achieve close-
to-native performance without high CPU costs. However, current direct I/O
virtualization implementations focus on networking for mainframes. There are a lot of
challenges for commodity hardware devices. For example, when a physical device is
reclaimed (required by workload migration) for later reassign-ment, it may have been
set to an arbitrary state (e.g., DMA to some arbitrary memory locations) that can
function incorrectly or even crash the whole system. Since software-based I/O
virtualization requires a very high overhead of device emulation, hardware-assisted
I/O virtualization is critical. Intel VT-d supports the remapping of I/O DMA transfers
and device-generated interrupts. The architecture of VT-d provides the flexibility to
support multiple usage models that may run unmodified, special-purpose,
or “virtualization-aware” guest OSes.
Another way to help I/O virtualization is via self-virtualized I/O (SV-IO) [47]. The
key idea of SV-IO is to harness the rich resources of a multicore processor. All tasks
associated with virtualizing an I/O device are encapsulated in SV-IO. It provides
virtual devices and an associated access API to VMs and a management API to the
VMM. SV-IO defines one virtual interface (VIF) for every kind of virtua-lized I/O
device, such as virtual network interfaces, virtual block devices (disk), virtual camera
devices, and others. The guest OS interacts with the VIFs via VIF device drivers.
Each VIF consists of two mes-sage queues. One is for outgoing messages to the
devices and the other is for incoming messages from the devices. In addition, each
VIF has a unique ID for identifying it in SV-IO.
Since the user devices is basically a display, keyboard, and mouse, a lost or stolen
device presents a reduced risk to the organization. All user data and programs exist in
the desktop virtualization server, not on client devices.
By having each physical server divided into multiple virtual servers, server
virtualization allows each virtual server to act as a unique physical device. Each
virtual server can run its own applications and operating system. This process
increases the utilization of resources by making each virtual server act as a physical
server and increases the capacity of each physical machine.
Lab Exercise 1:
Specifically, I'm going to walk through the creation of a simple Python Flask app
that provides a RESTful web service. The service will provide an endpoint to:
But before I jump in, keep a couple of things to keep in mind. First, I will not be
bothering with any sort of frontend display functionality, so you don't need to worry
about HTML or CSS. Second, my organization follows Flask's own suggested
organization. I am going to skip the single module pattern and go straight to
the Packages and Blueprints models.
There is a large range of Flask tutorials. On one hand, there are tutorials that explain
how to build small, simple apps (where the entire app fits in a single file). On the
other hand, there are tutorials that explain how to build much larger, complicated
apps. This tutorial fills a sweet spot in the middle and demonstrates a structure that is
simple, but which can immediately accommodate increasingly complex requirements.
Project structure
The structure of the project that I'm going to build, which comes from Explore Flask,
is shown below:
Threatstack-to-s3
├── app
│ ├── __init__.py
│ ├── models
│ │ ├── __init__.py
│ │ ├── s3.py
│ │ └── threatstack.py
│ └── views
│ ├── __init__.py
│ └── s3.py
├── gunicorn.conf.py
├── requirements.osx.txt
├── requirements.txt
└── threatstack-to-s3.py
Top-level files
I'll start the discussion with the top-level files that are useful to me as I build the
service:
Gunicorn.conf.py: This is a configuration file for the Gunicorn WSGI HTTP server
that will serve up this app. While the application can run and accept connections on its
own, Gunicorn is more efficient at handling multiple connections and allowing the
app to scale with load.
The app package is my application package. The logic for the application is
underneath this directory. As I mentioned earlier, I have chosen to break the app into
a collection of smaller modules rather than use a single, monolithic module file.
● app
● app.views.s3
● app.models.threatstack
● app.models.s3
Note: app.views and app.models do not provide anything and their __init__.py files
are empty.
App module
The app module has the job of creating the Flask application. It exports a single
function, create_app(), that will create a Flask application object and configure it.
Currently it initializes application blueprints that correspond to my application views.
Eventually, create_app() will do other things such as initialize logging, but I'm
skipping that now for clarity and simplicity.
App/__init__.py
def _initialize_blueprints(application):
'''
Register Flask blueprints
'''
from app.views.s3 import s3
application.register_blueprint(s3, url_prefix='/api/v1/s3')
def create_app():
'''
Create an app by initializing components.
'''
application = Flask(__name__)
_initialize_blueprints(application)
# Do it!
return application
Copy
Threatstack-to-s3.py
#!/usr/bin/env python
from app import create_app
if __name__ == '__main__':
# Entry point when run via Python interpreter.
print("== Running in debug mode ==")
application.run(host='localhost', port=8080, debug=True)
Copy
Views and Flask blueprints
Before discussing the remaining three modules, I'll talk about what views and Flask
blueprints and then dive into the app.views.s3 module.
Views: Views are what the application consumer sees. There's no front end to this
application, but there is a public API endpoint. Think of a view as what can and
should be exposed to the person or thing (e.g., the consumer) who is using this
application. The best practice is to keep views as simple as possible. If an endpoint's
job is to take data in and copy it to S3, make it perform that function, but hide the
details of how that was done in the application models. Views should mostly represent
the actions a consumer wants to see happen, while the details (which consumers
shouldn't care about) live in the application models (described later).
Flask Blueprints: Earlier I said that I am going to use a Packages and Blueprints
layout instead of a single module application. Blueprints contain a portion of my API
endpoint structure. This lets me logically group related portions of my API. In my
case, each view module is its own blueprint.
Learn more
Explore Flask is a book about best practices and patterns for developing web
applications with Flask.
App.views.s3 module
The threatstack-to-s3 service takes Threat Stack webhook HTTP requests in and
stores a copy of the alert data in S3. This is where I store the set of API endpoints that
allow someone to do this. If you look back at app/__init__.py, you will see that I
have rooted the set of endpoints at /api/v1/s3.
From app/__init__.py:
● API: To note that this is an API and I should not expect a front end. Maybe one
day I'll add a front end. Probably not, but I find this useful mentally and as a
sign to others
● V1: This is version 1 of the API. If I need to make breaking changes to
accommodate new requirements, I can add a v2 so that two APIs exist as I
migrate all consumers over to the new version
● S3: This is the service I'm connecting to and manipulating. I have some freedom
here to name this portion of the path whatever I want, but I like to keep it
descriptive. If the service was relaying data to HipChat, for example, I could
name this portion of the path hipchat
Remember: When building APIs, URL paths should represent nouns and HTTP
request methods should represent verbs.
App/views/s3.py
'''
API to archive alerts from Threat Stack to S3
'''
s3 = Blueprint('s3', __name__)
@s3.route('/alert', methods=['POST'])
def put_alert():
'''
Archive Threat Stack alerts to S3.
'''
webhook_data = request.get_json()
for alert in webhook_data.get('alerts'):
alert_full = threatstack_model.get_alert_by_id(alert.get('id'))
s3_model.put_webhook_data(alert)
s3_model.put_alert_data(alert_full)
status_code = 200
success = True
response = {'success': success}
I'm a fan of typing brevity and consistency. I could have done this the following way
to import the model modules:
import app.models.s3
import app.models.threatstack
Copy
app.models.s3.put_webhook_alert(alert)
Copy
However, this would break when I create the s3 Blueprint object a few lines later
because I'd overwrite the s3 model module.
For these reasons, importing the model modules and renaming them slightly is just
easier.
Now I'll walk through the app endpoint and function associated with it.
@s3.route('/alert', methods=['POST'])
def put_alert():
'''
Archive Threat Stack alerts to S3.
'''
Copy
The first line is called a decorator. I'm adding a route to the s3 Blueprint
called /alert (which expands to /api/v1/s3/alert) that when an HTTP POST request is
made to it will cause put_alert() to be called.
webhook_data = request.get_json()
for alert in webhook_data.get('alerts'):
alert_full = threatstack_model.get_alert_by_id(alert.get('id'))
s3_model.put_webhook_data(alert)
s3_model.put_alert_data(alert_full)
Copy
Once that's done, I return a simple JSON doc back, indicating the success or failure of
the transaction. (Note: There's no error handling in place, so of course I've hardcoded
the success response and HTTP status code. I'll change that when error handling is
added at a later date.)
status_code = 200
success = True
response = {'success': success}
At this point, I've satisfied my request and done what the consumer requested. Notice
that I haven't included any code demonstrating how I fulfilled the request. What did I
have to do to get the alert's detail? What actions did I perform to store the alert? How
are the alerts stored and named in S3? The consumer doesn't really care about those
details. This is a good way to think about organizing your code in your own service:
What the consumer needs to know about should live in your view. The details the
consumer doesn't need to know should live in your model, which I am about to cover.
Before discussing the remaining modules, I'll talk about models, which are how to
talk to the services I'm using, such as Threat Stack and S3.
Models
Models describe "things," and these "things" are what I want to perform actions on.
Typically, when you search for help on Flask models, blogs and documentation like to
use databases in their examples. While what I'm doing right now isn't far off, I'm just
storing data in an object store instead of a database. It's not the only sort of thing I
might do in the future with the data received from Threat Stack.
App.models.threatstack Module
'''
Communicate with Threat Stack
'''
import os
import requests
THREATSTACK_BASE_URL = os.environ.get('THREATSTACK_BASE_URL
', 'https://app.threatstack.com/api/v1')
THREATSTACK_API_KEY = os.environ.get('THREATSTACK_API_KEY')
def get_alert_by_id(alert_id):
'''
Retrieve an alert from Threat Stack by alert ID.
'''
alerts_url = '{}/alerts/{}'.format(THREATSTACK_BASE_URL, alert_id)
resp = requests.get(
alerts_url,
headers={'Authorization': THREATSTACK_API_KEY}
)
return resp.json()
Copy
I don't want to keep the Threat Stack API in my code. This is just good clean
code/security living. I'm going to get the API key from my environment for now
because it's a quick and simple solution. At some point, I should centralize all
configuration in a single file instead of hiding it here, so the code and setup are a little
cleaner. That's a job for another time, and for now the setup is documented
in README.md.
def get_alert_by_id(alert_id):
'''
Retrieve an alert from Threat Stack by alert ID.
'''
alerts_url = '{}/alerts/{}'.format(THREATSTACK_BASE_URL, alert_id)
resp = requests.get(
alerts_url,
headers={'Authorization': THREATSTACK_API_KEY}
)
return resp.json()
Copy
The get_alert_by_id() function takes an alert ID, queries the Threat Stack platform
for the alert data, and returns that data. I'm using the Python requests module to make
an HTTP GET request to the Threat Stack API endpoint that returns alert info for the
given alert.
App.models.s3 Module
'''
Manipulate objects in AWS S3.
'''
import boto3
import json
import os
import time
TS_AWS_S3_BUCKET = os.environ.get('TS_AWS_S3_BUCKET')
TS_AWS_S3_PREFIX = os.environ.get('TS_AWS_S3_PREFIX', None)
def put_webhook_data(alert):
'''
Put alert webhook data in S3 bucket.
'''
alert_time = time.gmtime(alert.get('created_at')/1000)
alert_time_path = time.strftime('%Y/%m/%d/%H/%M', alert_time)
alert_key = '/'.join([alert_time_path, alert.get('id')])
if TS_AWS_S3_PREFIX:
alert_key = '/'.join([TS_AWS_S3_PREFIX, alert_key])
s3_client = boto3.client('s3')
s3_client.put_object(
Body=json.dumps(alert),
Bucket=TS_AWS_S3_BUCKET,
Key=alert_key
)
return None
def put_alert_data(alert):
'''
Put alert data in S3.
'''
alert_id = alert.get('id')
alert_key = '/'.join(['alerts',
alert_id[0:2],
alert_id[2:4],
alert_id
])
if TS_AWS_S3_PREFIX:
alert_key = '/'.join([TS_AWS_S3_PREFIX, alert_key])
s3_client = boto3.client('s3')
s3_client.put_object(
Body=json.dumps(alert),
Bucket=TS_AWS_S3_BUCKET,
Key=alert_key
)
return None
Copy
TS_AWS_S3_BUCKET = os.environ.get('TS_AWS_S3_BUCKET')
TS_AWS_S3_PREFIX = os.environ.get('TS_AWS_S3_PREFIX', None)
Copy
Again, there's no config file for this app, but I need to set an S3 bucket name and
optional prefix. I should fix this eventually—the setup is documented in
the README.md, which is good enough for now.
def put_webhook_data(alert):
'''
Put alert webhook data in S3 bucket.
'''
alert_time = time.gmtime(alert.get('created_at')/1000)
alert_time_path = time.strftime('%Y/%m/%d/%H/%M', alert_time)
alert_key = '/'.join(['webhooks', alert_time_path, alert.get('id')])
if TS_AWS_S3_PREFIX:
alert_key = '/'.join([TS_AWS_S3_PREFIX, alert_key])
s3_client = boto3.client('s3')
s3_client.put_object(
Body=json.dumps(alert),
Bucket=TS_AWS_S3_BUCKET,
Key=alert_key
)
return None
Copy
I start by getting the alert time. Threat Stack has sent the alert time in milliseconds
since the Unix epoch, and that needs to be converted into seconds, which is how
Python handles time. I take that time and parse it into a string that will be the
directory path. I then join the top-level directory where I store webhook data, the
time-based path, and finally the alert ID to form the path to the webhook data in S3.
Boto 3 is the primary module in Python for working with AWS resources. I initialize
a boto3 client object so I can talk to S3 and put the object there.
The s3_client.put_object() is fairly straightforward with
its Bucket and Key arguments, which are the name of the S3 bucket and the path to
the S3 object I want to store. The Body argument is my alert converted back to a
string.
Lab Exercise :2
In this lab, you will learn how to install the Oracle Solaris 11.2 Image for Oracle
VMVirtualBox—the easiest way to get up and running with Oracle Solaris
11.2.
Prerequisites
This lab is the first is a series of labs for Oracle Solaris 11. All of the labs in the series
have these prerequisites in common:
● Memory: 2 GB of RAM
Before starting the lab, ensure you have installed the following:
● Download and install Oracle VM VirtualBox; see Installation Details for notes on
installing on any of the above operating systems.
● Download and install the Oracle VM VirtualBox Extension Pack and follow the
instructions at Installing VirtualBox and extension packs.
Also, you must enable hardware virtualization support in the BIOS. Oracle Solaris
depends on those capabilities.
Download the template (that is, the virtual machine [VM]) called Oracle Solaris 11.2
Oracle VM Template for Oracle VM VirtualBox.
Browse to the location where you downloaded the Oracle Solaris 11.2 VM and select
it. Notice that Figure 1 mentions the OVF format, but the downloaded file is a .ova
file, which is the entire archive (including OVF.xml). Click Continue.
● 3. Before importing the VM, check the memory settings.
Scroll down to check how much memory is allocated to the image. Oracle Solaris
11.2 (or later) requires a minimum of 2 GB of memory.
Figure 3. Checking the amount of allocated memory
In this exercise, we will run Oracle Solaris 11 for the first time—getting a basic
understanding of what's there:
● 1.Select the Oracle Solaris 11.2 VM and click the green arrow labeled Start.
Figure 4. Starting the VM
On first boot, the System Configuration Tool runs, prompting you to enter system
information providing an explanation of what is to follow. Note that during
installation, you have to actively switch between the VM and your host OS. Oracle
VM VirtualBox will open a window to explain this. After the VM boots, the
environments are integrated, so when you move the mouse pointer over the VM, any
input will be directed to the VM, and when you move the mouse pointer outside the
VM, subsequent input will go to the host OS.
Figure 5. First screen of the System Configuration Tool
● 4. The third screen will prompt for networking settings. Choose Automatic.
● 5. Next will be three screens to set the time:
● 6. Then set the date.
● 7. And select the keyboard.
● 8. Set the password. Make sure you enter user account information as well as the root
password. You will log in through the user account.
Figure 6. Screen for entering user account and password information
● 9. Next, there will be two screens to enable the Oracle Configuration Manager, that is,
'phone home' capability. Unless you enter your My Oracle Support credentials (e-mail
address and password) this data gathering facility will not be activated. (No specific
user information is collected and Oracle treats the data collected as a customer's
private information.)
● 10. Finally, you will be presented with a summary page:
o 11. Press F2 to apply the specified configuration, and then Oracle Solaris will
complete the configuration/boot process.
o 12.Log in to Oracle Solaris using the user account you set up in Step 8 above.
Figure 9. Opening a terminal window
Figure 11. Listing data about all the pools and subpools
sudo cat /etc/sudoers
Doing this will give you privileges to run as the root user for five minutes. The demo
account user attributes are in /etc/user_attr. If you look at the contents of the file, you
will notice that when the demo user was created, it was given the role
of type=root. Hence, it can operate with root privileges.
o 13. After logging in, you should see the blank background of the user's desktop. Bring
up a terminal window by clicking the icon that looks like a computer screen (located
on the left side of the top bar of the Oracle VM VirtualBox window).
▪ 14. To get started, investigate the Oracle VM VirtualBox package by running the
command pkginfo -l SUNWvboxguest in the terminal window.
▪ 15. Next, enter the following command:
▪ 16. If you regularly use sudo(1), you can type in a command such as the following and
enter the password of the demo account.
Exercise 4: Take a Snapshot
There are two ways to take a snapshot of your environment. The first is the traditional
mechanism for VMs—capture all the information of that machine so that you can start
it up from that saved state later. This includes a snapshot of the local file system.
To take a snapshot, from the VirtualBox menu, select Machine > Take
Snapshot. Give the snapshot a name and optional description:
Figure 12. Creating a snapshot
The other approach is using the capability of ZFS. There are two ways to do this: one
for system administrators and one for users.
Lab Exercise: 3
● Web services in cloud computing can be searched for over the network and
can also be invoked accordingly.
● When invoked, the web service would be able to provide the functionality to
the client, which invokes that web service.
Web Service Characteristics
In order for a web service to be fully functional, there are certain components that
need to be in place. These components need to be present irrespective of whatever
development language is used for programming the web service.
Let's look at these components in more detail.
SOAP (Simple Object Access Protocol)
SOAP is known as a transport-independent messaging protocol. SOAP is based on
transferring XML data as SOAP Messages. Each message has something which is
known as an XML document. Only the structure of the XML document follows a
specific pattern, but not the content. The best part of Web services and SOAP is that
its all sent via HTTP, which is the standard web protocol.
Here is what a SOAP message consists of
● Each SOAP document needs to have a root element known as the <Envelope>
element. The root element is the first element in an XML document.
● The "envelope" is in turn divided into 2 parts. The first is the header, and the
next is the body.
● The header contains the routing data which is basically the information which
tells the XML document to which client it needs to be sent to.
● The body will contain the actual message.
The diagram below shows a simple example of the communication via SOAP.
S
OAP Protocol
We will discuss SOAP in detail
WSDL (Web services description language)
A web service cannot be used if it cannot be found. The client invoking the web
service should know where the web service actually resides.
Secondly, the client application needs to know what the web service actually does, so
that it can invoke the right web service. This is done with the help of the WSDL,
known as the Web services description language. The WSDL file is again an XML-
based file which basically tells the client application what the web service does. By
using the WSDL document, the client application would be able to understand where
the web service is located and how it can be utilized.
Web Service Example
A Web services example of a WSDL file is given below.
<definitions>
<message name="TutorialRequest">
<part name="TutorialID" type="xsd:string"/>
</message>
<message name="TutorialResponse">
<part name="TutorialName" type="xsd:string"/>
</message>
<portType name="Tutorial_PortType">
<operation name="Tutorial">
<input message="tns:TutorialRequest"/>
<output message="tns:TutorialResponse"/>
</operation>
</portType>
<binding name="Tutorial_Binding" type="tns:Tutorial_PortType">
<soap:binding style="rpc"
transport="http://schemas.xmlsoap.org/soap/http"/>
<operation name="Tutorial">
<soap:operationsoapAction="Tutorial"/>
<input>
<soap:body
encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
namespace="urn:examples:Tutorialservice"
use="encoded"/>
</input>
<output>
<soap:body
encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
namespace="urn:examples:Tutorialservice"
use="encoded"/>
</output>
</operation>
</binding>
</definitions>
The important aspects to note about the above WSDL declaration examples of web
services are as follows:
1. Provider - The provider creates the web service and makes it available to
client application who want to use it.
2. Requestor - A requestor is nothing but the client application that needs to
contact a web service. The client application can be a .Net, Java, or any other
language based application which looks for some sort of functionality via a
web service.
3. Broker - The broker is nothing but the application which provides access to
the UDDI. The UDDI, as discussed in the earlier topic enables the client
application to locate the web service.
The diagram below showcases how the Service provider, the Service requestor and
Service registry interact with each other.
Web Services
Architecture
1. Publish - A provider informs the broker (service registry) about the existence
of the web service by using the broker's publish interface to make the service
accessible to clients
2. Find - The requestor consults the broker to locate a published web service
3. Bind - With the information it gained from the broker(service registry) about
the web service, the requestor is able to bind, or invoke, the web service.
MCAD22E3CLOUD COMPUTING
MODULE7
Cloud Provisioning
Cloud provisioning is the allocation of a cloud provider's resources and services to a
customer.
Cloud provisioning is a key feature of the cloud computing model, relating to how a
customer procures cloud services and resources from a cloud provider. The growing
catalog of cloud services that customers can provision includes infrastructure as a
service (IaaS), software as a service (SaaS) and platform as a service (PaaS) in public
or private cloud environments.
With advanced provisioning, the customer signs a formal contract of service with the
cloud provider. The provider then prepares the agreed-upon resources or services for
the customer and delivers them. The customer is charged a flat fee or is billed on a
monthly basis.
With user self-provisioning, also called cloud self-service, the customer buys
resources from the cloud provider through a web interface or portal. This usually
involves creating a user account and paying for resources with a credit card. Those
resources are then quickly spun up and made available for use -- within hours, if not
minutes. Examples of this type of cloud provisioning include an employee purchasing
cloud-based productivity applications via the Microsoft 365 suite or G Suite.
Clou
d provisioning can be conducted in one of three processes: advanced, dynamic and
user self-provisioning.
Why cloud provisioning matters
Cloud provisioning offers organizations numerous benefits that aren't available with
traditional provisioning approaches.
Organizations can also benefit from cloud provisioning's speed. For example, an
organization's developers can quickly spin up an array of workloads on demand,
removing the need for an IT administrator who provisions and manages compute
resources.
Another benefit of cloud provisioning is the potential cost savings. While traditional
on-premises technology can exact large upfront investments from an organization,
many cloud providers allow customers to pay for only what they consume. However,
the attractive economics presented by cloud services can present its own challenges,
which organizations should address in a cloud management strategy.
Resource and service dependencies. Applications and workloads in the cloud often
tap into basic cloud infrastructure resources, such as compute, networking and
storage. Beyond those, public cloud providers' big selling point is in higher-level
ancillary services, such as serverless functions, machine learning and big data
capabilities. However, those services may carry dependencies that might not be
obvious, which can lead to unexpected overuse and surprise costs.
Cost controls. Beyond provisioning policies, automated monitoring and alerts about
usage and pricing thresholds are essential. Be aware that these might not be real-time
warnings; in fact, an alert about an approaching budget overrun for a cloud service
could arrive hours or days after the fact.
Cloud provisioning tools and software
Organizations can manually provision whatever resources and services they need, but
public cloud providers offer tools to provision multiple resources and services:
● AWS CloudFormation
Alternatively, third-party tools for cloud resource provisioning include the following:
● CloudBolt
● Morpheus Data
● Scalr
While they all have features that differentiate them from each other, they’re also
focused on one key concept: providing information about cloud computing systems. If
your needs run into provisioning, the choices become more distinct than choosing
“agent vs. agentless” or “SNMP vs. WBEM.”
The main cloud infrastructure management products offer similar core features:
When it comes to meeting those three criteria, there are a few vendors that offer
pervasive approaches in handling provisioning and managing metrics in hybrid
environments: RightScale, Kaavo, Zeus, Scalr and Morph. There are also options
offered by cloud vendors themselves that meet the second and third criteria, such as
CloudWatch from Amazon Web Services.
The large companies known for their traditional data center monitoring applications
have been slow to hit the cloud market, and what products they do have are rehashes
of existing applications that do little in the way of providing more than reporting and
alerting tools. CA is on an acquisition spree to fix this and just acquired 3Tera, a
cloud provisioning player.
An example of the confusion in the industry is IBM’s Tivoli product page for cloud
computing. You’ll notice that clicking the Getting Started tab results in a 404 error.
Nice work, IBM.
Meanwhile, HP’s OpenView (now called Operations Manager) can manage cloud-
based servers, but only insofar as it can manage any other server. BMC is working on
a cloud management tool, but doesn’t have anything beyond its normal products out at
the moment.
In place of these behemoths, secondary players making splashes on the market are
offering monitoring-focused applications from companies like Scout, UpTime
Systems, Cloudkick, NetIQ and ScienceLogic. There is also the “app formerly known
as” Hyperic, now owned by VMware through the acquisition of SpringSource.
In truth, we could rival John Steinbeck and Robert Jordan in word count when it
comes to writing about all the products in this field, though within a year or two it
should be a much smaller space as acquisitions occur, companies fail and the market
sorts itself out. There’s a lot on the way in cloud computing, not the least of which is
specifications. Right now the cloud is the Wild West: vast, underpopulated, and
lacking order except for a few spots of light.
RightScale
RightScale is the big boy on the block right now. Like many vendors in the nascent
market, they offer a free edition with limitations on features and capacity, designed to
introduce you to the product (and maybe get you hooked, ala K.C. Gillette’s famous
business model at the turn of the 20th century). RightScale’s product is broken down
into four components:
A fifth feature states that the “Readily Extensible Platform supports programmatic
access to the functionality of the RightScale Platform.” In looking at the product,
these features aren’t really separate from one another, but make a nice, integrated
offering.
RightScale’s management environment is the main interface users will have with the
software. It is designed to walk a user through the initial process of migrating to the
cloud using their templates and library. The management environment is then used for
(surprise!) managing that environment, namely continuing builds and ensuring
resource availability. This is where the automation engine comes into play: being able
to quickly provision and put into operation additional capacity, or remove that excess
capacity, as needed. Lastly, there is the Multi-Cloud Engine, supporting Amazon,
GoGrid, Eucalyptus and Rackspace.
Kaavo
Kaavo plays in a very similar space to RightScale. The product is typically used for:
One great feature in IMOD is its multi-cloud, single system tool. For instance, you
can create a database backend in Rackspace while putting your presentation servers
on Amazon. Supporting Amazon and Rackspace in the public space and Eucalyptus in
the private space is a strong selling point, though it should be noted that most cloud
management can support Eucalyptus if it can also support Amazon, as
Eucalyptus mimics Amazon EC2 very closely.
Zeus
Zeus was famous for its rock-solid Web server, one that didn’t have a lot of market
share but did have a lot of fanatical fans and top-tier customers. With Apache, and to
a lesser-extent, IIS, dominating that market, not to mention the glut of load balancers
out there, Zeus took its expertise in the application server space and came up with
the Application Delivery Controller piece of the Zeus Traffic Controller. It uses
traditional load balancing tools to test availability and then spontaneously generate or
destroy additional instances in the cloud, providing on-the-fly provisioning. Zeus
currently supports this on the Rackspace and, to a lesser extent, Amazon platforms.
Scalr
Scalr is a young project hosted on Google Code and Scalr.net that creates dynamic
clusters, similar to Kaavo and RightScale, on the Amazon platform. It supports
triggered upsizing and downsizing based on traffic demands, snapshots (which can be
shared, incidentally, a very cool feature), and the custom building of images for each
server or server-type, also similar to RightScale. Being a new release, Scalr does not
support the wide number of platforms, operating systems, applications, and databases
that the largest competitors do, sticking to the traditional expanded-LAMP
architecture (LAMP plus Ruby, Tomcat, etc.) that comprises many content systems.
Morph
While not a true management platform, the MSP-minded Morph products offers
similar functionality in its own private space. Morph CloudServer is a newer product
on the market, filling the management and provisioning space as an appliance. It is
aimed at the enterprise seeking to deploy a private cloud. Its top-tier product, the
Morph CloudServer is based on the IBM BladeCenter, and supports hundreds of
virtual machines.
Under the core is an Ubuntu Linux operating system and the Eucalyptus cloud
computing platform. Aimed at the managed service provider market, Morph allows
for the creation of private clouds and the dynamic provisioning within those closed
clouds. While still up-and-coming, Morph has made quite a splash and bears
watching, particularly because of its open-source roots and participation in open-
cloud organizations.
CloudWatch
Amazon’s CloudWatch works on Amazon’s platform only, which limits its overall
usefulness as it cannot be a hybrid cloud management tool. Since Amazon’s Elastic
Compute Cloud (EC2) is the biggest platform out there (though Rackspace claims it is
closing that gap quickly), it still bears mentioning.
Emerging Standards
Cloud standards have eveolved but are different from U.S. and European standards
organizations and includes the Open Data Center Alliance, the Distributed
Management Task Force (DMTF), standards consortium OMG, storage and
networking standards group SNIA, and the European telecommunications and
network standards group, ETSI.
All of the organizations are doing their own work with end users and vendors to
establish cloud standards, which are then discussed among the organizations.
Technologies like cloud computing and virtualization are rapidly being adopted by
enterprise IT managers to better deliver services to their customers, lower IT costs and
improve operational efficiencies.
From a provider’s standpoint, cloud provisioning can include the supply and
assignment of required cloud resources to the customer. For example, the creation of
virtual machines, the allocation of storage capacity and/or granting access to cloud
software.
Managed hosting providers take away the worry of keeping your cloud
environment running efficiently, as well as monitoring your platform for cyber
threats and any potential security issues.
Hyve’s managed cloud
Hyve takes on the management of your cloud environment, helping with all
stages of the process including designing and planning your cloud
infrastructure, migration, and ongoing monitoring and management. We
proactively monitor your cloud and manage it up to the application layer,
saving you time and money.
● Reliability
On our high-availability VMware cloud, we guarantee 99.999% uptime SLA as
well as 100% power and network uptime SLA. We also offer a 20-minute
hardware replacement SLA for any faulty hardware.
● Scalability
Sudden traffic spikes without the correct scalability can cause a system to fail.
With a managed cloud you can rest assured that your hosting provider will be
monitoring traffic 24/7/365, and will scale out your cloud platform as and when it
is required.
● Control costs
Having a managed cloud provider removes the need for an in-house IT team.
Hiring and training staff is expensive, so a managed cloud is a great way to
reduce expenses. If you use an unmanaged cloud without the correct expertise in
place to manage it, costs can easily spin out of control, leaving you with a large
bill at the end of the month.
● Reduce risk
Using a managed provider helps to reduce risks to your business and prepares
you for any potential pitfalls. Managed providers will be experts in all areas of
cloud hosting and cybersecurity, and will be able to assist with any custom or
complex solutions.
● 24/7 support
With a managed hosting provider you will have a team of expert engineers
dedicated to your business. They will proactively monitor your cloud platform
and be available to assist you 24/7/365.
Managed v unmanaged
Managing a cloud platform whether that is Public or Private Cloud can seem
relatively simple, but for a cloud platform with a variety of services and
applications to run, it can be surprisingly complex to manage.
Although an unmanaged cloud often seems like the cheaper option, issues are
likely to occur along the way that need to be resolved by an expert. Having a
dedicated team of in-house IT experts working 24/7 is costly, and without this,
your cloud platform could be at risk. Managed providers become an extension
of your business and provide you with all you need for your business to
succeed.
Some customers also choose to retain some in-house engineers to do frontline
support or work on specific projects, but our technical support team fill any IT
skills gaps.
Managed Public Cloud
However, if you prefer the setup of a public cloud, Hyve can offer
a management layer for AWS, Google and Azure clouds. Our expert
engineers can work with your business to create a deployment or migration
plan and add the management layer on top, helping you optimise your public
cloud platform without needing an in-house IT team.
MCAD22E3CLOUD COMPUTING
MODULE8
Reduced costs: One of the benefits of utilizing cloud storage and security is that it
eliminates the need to invest in dedicated hardware. Not only does this reduce capital
expenditure, but it also reduces administrative overheads. Where once IT teams were
firefighting security issues reactively, cloud security delivers proactive security
features that offer protection 24/7 with little or no human intervention.
Reliability: Cloud computing services offer the ultimate in dependability. With the
right cloud security measures in place, users can safely access data and applications
within the cloud no matter where they are or what device they are using.
More and more organizations are realizing the many business benefits of moving their
systems to the cloud. Cloud computing allows organizations to operate at scale,
reduce technology costs and use agile systems that give them the competitive edge.
However, it is essential that organizations have complete confidence in their cloud
computing security and that all data, systems and applications are protected from data
theft, leakage, corruption and deletion.
All cloud models are susceptible to threats. IT departments are naturally cautious
about moving mission-critical systems to the cloud and it is essential the right security
provisions are in place, whether you are running a native cloud, hybrid or on-premise
environment. Cloud security offers all the functionality of traditional IT security, and
allows businesses to harness the many advantages of cloud computing while
remaining secure and also ensure that data privacy and compliance requirements are
met.
Secure Data in the Cloud
Cloud data security becomes increasingly important as we move our devices, data
centers, business processes, and more to the cloud. Ensuring quality cloud data
security is achieved through comprehensive security policies, an organizational
culture of security, and cloud security solutions.
Selecting the right cloud security solution for your business is imperative if you want
to get the best from the cloud and ensure your organization is protected from
unauthorized access, data breaches and other threats. Forcepoint Cloud Access
Security Broker (CASB) is a complete cloud security solution that protects cloud
apps and cloud data, prevents compromised accounts and allows you to set security
policies on a per-device basis.
Today’s businesses want it all: secure data and applications accessible anywhere from
any device. It’s possible with cloud technology, but there are inherent cloud
computing security challenges to making it a reality.
What can enterprise businesses do to reap the benefits of cloud technology while
ensuring a secure environment for sensitive information? Recognizing those
challenges is the first step to finding solutions that work. The next step is choosing the
right tools and vendors to mitigate those cloud security challenges.
In our technology driven world, security in the cloud is an issue that should be
discussed from the board level all the way down to new employees. The CDNetworks
blog recently discussed “what is cloud security” and explained some of its benefits.
Now that we understand what cloud security is, let’s take a look at some of the key
challenges that may be faced and why you want to prevent unauthorized access at all
costs.
As more and more businesses and operations move to the cloud, cloud providers are
becoming a bigger target for malicious attacks. Distributed denial of service (DDoS)
attacks are more common than ever before. Verisign reported IT services, cloud
platforms (PaaS) and SaaS was the most frequently targeted industry during the first
quarter of 2015.
Complementing cloud services with DDoS protection is no longer just good idea for
the enterprise; it’s a necessity. Websites and web-based applications are core
components of 21st century business and require state-of-the-art cybersecurity.
CHALLENGE 2: DATA BREACHES
Known data breaches in the U.S. hit a record-high of 738 in 2014, according to the
Identity Theft Research Center, and hacking was (by far) the number one cause.
That’s an incredible statistic and only emphasizes the growing challenge to secure
sensitive data.
Traditionally, IT professionals have had great control over the network infrastructure
and physical hardware (firewalls, etc.) securing proprietary data. In the cloud (in all
scenarios including private cloud, public cloud, and hybrid cloud situations), some of
those security controls are relinquished to a trusted partner meaning cloud
infrastructure can increase security risks. Choosing the right vendor, with a strong
record of implementing strong security measures, is vital to overcoming this
challenge.
When business critical information is moved into the cloud, it’s understandable to be
concerned with its security. Losing cloud data, either through accidental deletion and
human error, malicious tampering including the installation of malware (i.e. DDoS),
or an act of nature that brings down a cloud service provider, could be disastrous for
an enterprise business. Often a DDoS attack is only a diversion for a greater threat,
such as an attempt to steal or delete data.
To face this challenge, it’s imperative to ensure there is a disaster recovery process in
place, as well as an integrated system to mitigate malicious cyberattacks. In addition,
protecting every network layer, including the application layer (layer 7), should be
built-in to a cloud security solution.
One of the great benefits of the cloud is it can be accessed from anywhere and from
any device. But, what if the interfaces and particularly the application programming
interfaces (APIs) users interact with aren’t secure? Hackers can find and gain access
to these types of vulnerabilities and exploit authentication via APIs if given enough
time.
A behavioral web application firewall examines HTTP requests to a website to ensure
it is legitimate traffic. This always-on device helps protect web applications and APIS
from security breaches within cloud environments and data centers that are not on-
premises.
Cloud computing security issues and challenges are not insurmountable. With the
right cloud service provider (CSP), technology, and forethought, enterprises can
leverage the benefits of cloud technology.
CDNetworks’ cloud security solution integrates web performance with the latest in
cloud security technology. With 160 points of presence, websites and cloud
applications are accelerated on a global scale and, with our cloud security, our clients’
cloud-based assets are protected with 24/7 end to end security, including DDoS
mitigation at the network and application levels.
Virtualized Security
What is virtualized security?
Virtualized security, or security virtualization, refers to security solutions that are
software-based and designed to work within a virtualized IT environment. This differs
from traditional, hardware-based network security, which is static and runs on devices
such as traditional firewalls, routers, and switches.
For example, an enterprise can insert security controls (such as encryption) between
the application layer and the underlying infrastructure, or use strategies such as micro-
segmentation to reduce the potential attack surface.
It’s important to note, however, that many of these risks are already present in a
virtualized environment, whether security services are virtualized or not.
Following enterprise security best practices (such as spinning down virtual machines
when they are no longer needed and using automation to keep security policies up to
date) can help mitigate such risks.
Traditional security also relies heavily upon port and protocol filtering, an approach
that’s ineffective in a virtualized environment where addresses and ports are assigned
dynamically. In such an environment, traditional hardware-based security is not
enough; a cloud-based network requires virtualized security that can move around the
network along with workloads and applications.
Application security describes security measures at the application level that aim to
prevent data or code within the app from being stolen or hijacked. It encompasses the
security considerations that happen during application development and design, but it
also involves systems and approaches to protect apps after they get deployed.
Application security may include hardware, software, and procedures that identify or
minimize security vulnerabilities. A router that prevents anyone from viewing a
computer’s IP address from the Internet is a form of hardware application security.
But security measures at the application level are also typically built into the software,
such as an application firewall that strictly defines what activities are allowed and
prohibited. Procedures can entail things like an application security routine that
includes protocols such as regular testing.
Business challenges
The business value of data has never been greater than it is today. The loss of trade
secrets or intellectual property (IP) can impact future innovations and profitability.
So, trustworthiness is increasingly important to consumers, with a full 75% reporting
that they will not purchase from companies they don’t trust to protect their data.
More on data security
Encryption
Using an algorithm to transform normal text characters into an unreadable format,
encryption keys scramble data so that only authorized users can read it. File and
database encryption solutions serve as a final line of defense for sensitive volumes by
obscuring their contents through encryption or tokenization. Most solutions also
include security key management capabilities.
DataErasure
More secure than standard data wiping, data erasure uses software to completely
overwrite data on any storage device. It verifies that the data is unrecoverable.
DataMasking
By masking data, organizations can allow teams to develop applications or train
people using real data. It masks personally identifiable information (PII) where
necessary so that development can occur in environments that are compliant.
DataResiliency
Resiliency is determined by how well a data center is able to endure or recover any
type of failure – from hardware problems to power shortages and other disruptive
events.
Data security tools and technologies should address the growing challenges inherent
in securing today’s complex, distributed, hybrid, and/or multicloud computing
environments. These include understanding where data resides, keeping track of who
has access to it, and blocking high-risk activities and potentially dangerous file
movements. Comprehensive data protection solutions that enable enterprises to adopt
a centralized approach to monitoring and policy enforcement can simplify the task.
Backups
Maintaining usable, thoroughly tested backup copies of all critical data is a core
component of any robust data security strategy. In addition, all backups should be
subject to the same physical and logical security controls that govern access to the
primary databases and core systems.
Employee education
Training employees in the importance of good security practices and password
hygiene and teaching them to recognize social engineering attacks transforms them
into a “human firewall” that can play a critical role in safeguarding your data.
Network and endpoint security monitoring and controls
Implementing a comprehensive suite of threat management, detection, and response
tools and platforms across your on-premises environment and cloud platforms can
mitigate risks and reduce the probability of a breach.
Lab Exercise 1:
Use security tools like ACUNETIX, ETTERCAP to scan web
applications on the cloud.
ACUNETIX
This information gathering tool scans web applications on the cloud and
lists possible vulnerabilities that might be present in the given web
application. Most of the scanning is focused on finding SQL injection and
cross site scripting Vulnerabilities. It has both free and paid versions, with
paid versions including added functionalities. After scanning, it generates a
detailed report describing vulnerabilities along with the suitable action that
can be taken to remedy the loophole.
This tool can be used for scanning cloud applications. Beware: there is
always a chance of false positives. Any security flaw, if discovered through
scanning, should be verified. The latest version of this software, Acunetix
WVS version 8, has a report template for checking compliance with ISO
27001, and can also scan for HTTP denial of service attacks.
Several others tools are also available in this suite, including esside-ng,
wesside-ng and tkiptun-ng. Aircrack-ng can be used on both command line
interfaces and on graphical interfaces. In GUI, it is named Gerix Wi-Fi
Cracker, which is a freely available network security tool licensed to GNU.
In the latest version of Cain, the ‘sniffer’ feature allows for analyzing
encrypted protocols such as SSH-1 and HTTPS. This tool can be utilized for
ARP cache poisoning, enabling sniffing of switched LAN devices, thereby
performing Man in the Middle (MITM) attacks. Further functionalities have
been added in the latest version, including authentication monitors for
routing protocols, brute-force for most of the popular algorithms and
cryptanalysis attacks.
ETTERCAP
Ettercap is a free and open source tool for network security, designed for
analyzing computer network protocols and detecting MITM attacks. It is
usually accompanied with Cain. This tool can be used for pen testing cloud
networks and verifying leakage of information to an unauthorized third
party. It has four methods of functionality:
Vulnerability: An Overview
Vulnerability is a prominent factor of risk. ISO 27005 defines risk as “the potential
that a given threat will exploit vulnerabilities of an asset or group of assets and
thereby cause harm to the organization,” measuring it in terms of both the likelihood
of an event and its consequence. The Open Group’s risk taxonomy offers a useful
overview of risk factors (see Figure 1).
The Open Group’s taxonomy uses the same two top-level risk factors as ISO 27005:
the likelihood of a harmful event (here, loss event frequency) and its consequence
(here, probable loss magnitude).1 The probable loss magnitude’s subfactors (on the
right in Figure 1) influence a harmful event’s ultimate cost. The loss event frequency
subfactors (on the left) are a bit more complicated. A loss event occurs when a threat
agent (such as a hacker) successfully exploits a vulnerability. The frequency with
which this happens depends on two factors:
● The frequency with which threat agents try to exploit a vulnerability. This frequency
is determined by both the agents’ motivation (What can they gain with an attack?
How much effort does it take? What is the risk for the attackers?) and how much
access (“contact”) the agents have to the attack targets.
● The difference between the threat agents’ attack capabilities and the system’s strength
to resist the attack.
Defining Vulnerability
According to the Open Group’s risk taxonomy, “Vulnerability is the probability that
an asset will be unable to resist the actions of a threat agent. Vulnerability exists when
there is a difference between the force being applied by the threat agent, and an
object’s ability to resist that force.”
We can also describe computer vulnerability - that is, security-related bugs that you
close with vendor-provided patches - as a weakening or removal of a certain
resistance strength. A buffer-overflow vulnerability, for example, weakens the
system’s resistance to arbitrary code execution. Whether attackers can exploit this
vulnerability depends on their capabilities.
We’ll now examine how cloud computing influences the risk factors in Figure 1,
starting with the right-hand side of the risk factor tree.
From a cloud customer perspective, the right-hand side dealing with probable
magnitude of future loss isn’t changed at all by cloud computing: the consequences
and ultimate cost of, say, a confidentiality breach, is exactly the same regardless of
whether the data breach occurred within a cloud or a conventional IT infrastructure.
For a cloud service provider, things look somewhat different: because cloud
computing systems were previously separated on the same infrastructure, a loss event
could entail a considerably larger impact. But this fact is easily grasped and
incorporated into a risk assessment: no conceptual work for adapting impact analysis
to cloud computing seems necessary.
So, we must search for changes on Figure 1’s left-hand side - the loss event
frequency. Cloud computing could change the probability of a harmful event’s
occurrence. As we show later, cloud computing causes significant changes in the
vulnerability factor. Of course, moving to a cloud infrastructure might change the
attackers’ access level and motivation, as well as the effort and risk - a fact that must
be considered as future work. But, for supporting a cloud-specific risk assessment, it
seems most profitable to start by examining the exact nature of cloud-specific
vulnerabilities.
Cloud-Specific Vulnerabilities
Based on the abstract view of cloud computing we presented earlier, we can now
move toward a definition of what constitutes a cloud-specific vulnerability. A
vulnerability is cloud specific if it
Core-Technology Vulnerabilities
Second, Web application technologies must overcome the problem that, by design, the
HTTP protocol is a stateless protocol, whereas Web applications require some notion
of session state. Many techniques implement session handling and - as any security
professional knowledgeable in Web application security will testify - many session
handling implementations are vulnerable to session riding and session hijacking.
Whether session riding/hijacking vulnerabilities are intrinsic to Web application
technologies or are “only” prevalent in many current implementations is arguable; in
any case, such vulnerabilities are certainly relevant for cloud computing.
Following are examples of vulnerabilities with root causes in one or more of these
characteristics:
● Data recovery vulnerability. The cloud characteristics of pooling and elasticity entail
that resources allocated to one user will be reallocated to a different user at a later
time. For memory or storage resources, it might therefore be possible to recover data
written by a previous user.
● Metering and billing evasion. The cloud characteristic of measured service means that
any cloud service has a metering capability at an abstraction level appropriate to the
service type (such as storage, processing, and active user accounts). Metering data is
used to optimize service delivery as well as billing. Relevant vulnerabilities include
metering and billing data manipulation and billing evasion.
Here, we treat three examples of such control challenges. First, virtualized networks
offer insufficient network-based controls. Given the nature of cloud services, the
administrative access to IaaS network infrastructure and the ability to tailor network
infrastructure are typically limited; hence, standard controls such as IP-based network
zoning can’t be applied. Also, standard techniques such as network-based
vulnerability scanning are usually forbidden by IaaS providers because, for example,
friendly scans can’t be distinguished from attacker activity. Finally, technologies such
as virtualization mean that network traffic occurs on both real and virtual networks,
such as when two virtual machine environments (VMEs) hosted on the same server
communicate. Such issues constitute a control challenge because tried and tested
network-level security controls might not work in a given cloud environment.
Finally, security metrics aren’t adapted to cloud infrastructures. Currently, there are
no standardized cloud-specific security metrics that cloud customers can use to
monitor the security status of their cloud resources. Until such standard security
metrics are developed and implemented, controls for security assessment, audit, and
accountability are more difficult and costly, and might even be impossible to employ.
● SQL injection, in which the input contains SQL code that’s erroneously executed in
the database back end;
● command injection, in which the input contains commands that are erroneously
executed via the OS; and
● cross-site scripting, in which the input contains JavaScript code that’s erroneously
executed by a victim’s browser.
In addition, many widely used authentication mechanisms are weak. For example,
usernames and passwords for authentication are weak due to
● insecure user behavior (choosing weak passwords, reusing passwords, and so on), and
Cloud service models are commonly divided into SaaS, PaaS, and IaaS, and each
model influences the vulnerabilities exhibited by a given cloud infrastructure. It’s
helpful to add more structure to the service model stacks: Figure 2 shows a cloud
reference architecture that makes the most important security-relevant cloud
components explicit and provides an abstract overview of cloud computing for
security issue analysis.
Figure 2. The cloud reference architecture. We map cloud-specific vulnerabilities
to components of this reference architecture, which gives us an overview of
which vulnerabilities might be relevant for a given cloud service.
The reference architecture is based on work carried out at the University of California,
Los Angeles, and IBM. It inherits the layered approach in that layers can encompass
one or more service components. Here, we use “service” in the broad sense of
providing something that might be both material (such as shelter, power, and
hardware) and immaterial (such as a runtime environment). For two layers, the cloud
software environment and the cloud software infrastructure, the model makes the
layers’ three main service components - computation, storage, and communication -
explicit.
Top layer services also can be implemented on layers further down the stack, in effect
skipping intermediate layers. For example, a cloud Web application can be
implemented and operated in the traditional way - that is, running on top of a standard
OS without using dedicated cloud software infrastructure and environment
components. Layering and compositionality imply that the transition from providing
some service or function in-house to sourcing the service or function can take place
between any of the model’s layers.
● Supporting (IT) infrastructure. These are facilities and services common to any IT
service, cloud or otherwise. We include them in the architecture because we want to
provide the complete picture; a full treatment of IT security must account for a cloud
service’s non-cloud-specific components.
● Cloud-specific infrastructure. These components constitute the heart of a cloud
service; cloud-specific vulnerabilities and corresponding controls are typically
mapped to these components.
● Cloud service consumer. Again, we include the cloud service customer in the
reference architecture because it’s relevant to an all-encompassing security treatment.
Also, we make explicit the network that separates the cloud service consumer from
the cloud infrastructure; the fact that access to cloud resources is carried out via a
(usually untrusted) network is one of cloud computing’s main characteristics.
Using the cloud reference architecture’s structure, we can now run through the
architecture’s components and give examples of each component’s cloud-specific
vulnerabilities.
The cloud software infrastructure layer provides an abstraction level for basic IT
resources that are offered as services to higher layers: computational resources
(usually VMEs), storage, and (network) communication. These services can be used
individually, as is typically the case with storage services, but they’re often bundled
such that servers are delivered with certain network connectivity and (often) access to
storage. This bundle, with or without storage, is usually referred to as IaaS.
The cloud software environment layer provides services at the application platform
level:
● a development and runtime environment for services and applications written in one
or more supported languages;
Vulnerabilities in both the infrastructure and environment layers are usually specific
to one of the three resource types provided by these two layers. However, cross-tenant
access vulnerabilities are relevant for all three resource types. The virtual machine
escape vulnerability we described earlier is a prime example. We used it to
demonstrate a vulnerability that’s intrinsic to the core virtualization technology, but it
can also be seen as having its root cause in the essential characteristic of resource
pooling: whenever resources are pooled, unauthorized access across resources
becomes an issue.
Hence, for PaaS, where the technology to separate different tenants (and tenant
services) isn’t necessarily based on virtualization (although that will be increasingly
true), cross-tenant access vulnerabilities play an important role as well. Similarly,
cloud storage is prone to cross-tenant storage access, and cloud communication - in
the form of virtual networking - is prone to cross-tenant network access.
Computational Resources
Data leakage by virtual machine replication is a vulnerability that’s also rooted in the
use of cloning for providing on-demand service. Cloning leads to data leakage
problems regarding machine secrets: certain elements of an OS - such as host keys
and cryptographic salt values - are meant to be private to a single host. Cloning can
violate this privacy assumption. Again, the emerging marketplace for virtual machine
images, as in Amazon EC2, leads to a related problem: users can provide template
images for other users by turning a running image into a template. Depending on how
the image was used before creating a template from it, it could contain data that the
user doesn’t wish to make public.
There are also control challenges here, including those related to cryptography use.
Cryptographic vulnerabilities due to weak random number generation might exist if
the abstraction layer between the hardware and OS kernel introduced by virtualization
is problematic for generating random numbers within a VME. Such generation
requires an entropy source on the hardware level. Virtualization might have flawed
mechanisms for tapping that entropy source, or having several VMEs on the same
host might exhaust the available entropy, leading to weak random number generation.
As we noted earlier, this abstraction layer also complicates the use of advanced
security controls, such as hardware security modules, possibly leading to poor key
management procedures.
Storage
Communication
Virtualized networking also presents a control challenge: again, in cloud services, the
administrative access to IaaS network infrastructure and the possibility for tailoring
network infrastructure are usually limited. Also, using technologies such as
virtualization leads to a situation where network traffic occurs not only on “real”
networks but also within virtualized networks (such as for communication between
two VMEs hosted on the same server); most implementations of virtual networking
offer limited possibilities for integrating network-based security. All in all, this
constitutes a control challenge of insufficient network-based controls because tried-
and-tested network-level security controls might not work in a given cloud
environment.
A Web application uses browser technology as the front end for user interaction. With
the increased uptake of browser-based computing technologies such as JavaScript,
Java, Flash, and Silverlight, a Web cloud application falls into two parts:
In the future, developers will increasingly use technologies such as Google Gears to
permit offline usage of a Web application’s browser component for use cases that
don’t require constant access to remote data. We’ve already described two typical
vulnerabilities for Web application technologies: session riding and hijacking
vulnerabilities and injection vulnerabilities.
Other Web-application-specific vulnerabilities concern the browser’s front-end
component. Among them are client-side data manipulation vulnerabilities, in which
users attack Web applications by manipulating data sent from their application
component to the server’s application component. In other words, the input received
by the server component isn’t the “expected” input sent by the client-side component,
but altered or completely user-generated input. Furthermore, Web applications also
rely on browser mechanisms for isolating third-party content embedded in the
application (such as advertisements, mashup components, and so on). Browser
isolation vulnerabilities might thus allow third-party content to manipulate the Web
application.
It might seem obvious that all layers of the cloud infrastructure offer services, but for
examining cloud infrastructure security, it’s worthwhile to explicitly think about all of
the infrastructure’s service and application programming interfaces. Most services are
likely Web services, which share many vulnerabilities with Web applications. Indeed,
the Web application layer might be realized completely by one or more Web services
such that the application URL would only give the user a browser component. Thus
the supporting services and API functions share many vulnerabilities with the Web
applications layer.
Management Access
NIST’s definition of cloud computing states that one of cloud services’ central
characteristics is that they can be rapidly provisioned and released with minimal
management effort or service provider interaction. Consequently, a common element
of each cloud service is a management interface - which leads directly to the
vulnerability concerning unauthorized access to the management interface.
Furthermore, because management access is often realized using a Web application or
service, it often shares the vulnerabilities of the Web application layer and
services/API component.
All cloud services (and each cloud service’s management interface) require
mechanisms for identity management, authentication, authorization, and auditing
(IAAA). To a certain extent, parts of these mechanisms might be factored out as a
stand-alone IAAA service to be used by other services. Two IAAA elements that must
be part of each service implementation are execution of adequate authorization checks
(which, of course, use authentication and/or authorization information received from
an IAA service) and cloud infrastructure auditing.
Most vulnerabilities associated with the IAAA component must be regarded as cloud-
specific because they’re prevalent in state-of-the-art cloud offerings. Earlier, we gave
the example of weak user authentication mechanisms; other examples include
● Denial of service by account lockout. One often-used security control - especially for
authentication with username and password - is to lock out accounts that have
received several unsuccessful authentication attempts in quick succession. Attackers
can use such attempts to launch DoS attacks against a user.
Provider
Vulnerabilities that are relevant for all cloud computing components typically concern
the provider - or rather users’ inability to control cloud infrastructure as they do their
own infrastructure. Among the control challenges are insufficient security audit
possibilities, and the fact that certification schemes and security metrics aren’t
adopted to cloud computing. Further, standard security controls regarding audit,
certification, and continuous security monitoring can’t be implemented effectively.
MCAD22E3CLOUD COMPUTING
MODULE9
Big data is a collection of a large data set. It has three main properties: volume,
velocity, and variety. Hadoop is software that allows storing and managing big data.
It is an open source framework written in Java. Moreover, it supports distributed
processing of large data sets across clusters of computers. HDFS and MapReduce are
two modules in Hadoop architecture.
What is HDFS
HDFS stands for Hadoop Distributed File System. It is a distributed file system of
Hadoop to run on large clusters reliably and efficiently. Also, it is based on the
Google File System (GFS). Moreover, it also has a list of commands to interact with
the file system.
Furthermore, the HDFS works according to the master, slave architecture. The master
node or name node manages the file system metadata while the slave nodes or the
data notes store actual data.
Figure 1: HDFS Architecture
Besides, a file in an HDFS namespace is split into several blocks. Data nodes stores
these blocks. And, the name node maps the blocks to the data nodes, which handle the
reading and writing operations with the file system. Furthermore, they perform tasks
such as block creation, deletion etc. as instructed by the name node.
What is MapReduce
MapReduce is a software framework that allows writing applications to process big
data simultaneously on large clusters of commodity hardware. This framework
consists of a single master job tracker and one slave task tracker per cluster node. The
master performs resource management, scheduling jobs on slaves, monitoring and re-
executing the failed tasks. On the other hand, the slave task tracker executes the tasks
instructed by the master and sends the tasks status information back to the mater
constantly.
Figure 2: MapReduce Overview
Also, there are two tasks associated with MapReduce. They are the map task and the
reduce task. The map task takes input data and divides them into tuples of key, value
pairs while the Reduce task takes the output from a map task as input and connects
those data tuples into smaller tuples. Furthermore, the map task is performed before
the reduce task.
Main Functionality
Another difference between HDFS and MapReduce is that the HDFS provides high-
performance access to data across highly scalable Hadoop clusters while MapReduce
performs the processing of big data.
9.2. Google App Engine (GAE)
App Engine
Google App Engine (often referred to as GAE or simply App Engine, and also used
by the acronym GAE/J) is a platform as a service (PaaS) cloud computing platform
for developing and hosting web applications in Google-managed data centers.
Applications are sandboxed and run across multiple servers. App Engine offers
automatic scaling for web applications—as the number of requests increases for an
application, App Engine automatically allocates more resources for the web
application to handle the additional demand.
Google App Engine is free up to a certain level of consumed resources. Fees are
charged for additional storage, bandwidth, or instance hours required by the
application. It was first released as a preview version in April 2008, and came out of
preview in September 2011.
Currently, the supported programming languages are Python, Java (and, by extension,
other JVM languages such as Groovy, JRuby, Scala, Clojure, Jython and PHP via a
special version of Quercus), and Go. Google has said that it plans to support more
languages in the future, and that the Google App Engine has been written to be
language independent.
Python web frameworks that run on Google App Engine include GAE framework,
Django, CherryPy, Pyramid, Flask, web2py and webapp2, as well as a custom
Google-written webapp framework and several others designed specifically for the
platform that emerged since the release. Any Python framework that supports the
WSGI using the CGI adapter can be used to create an application; the framework can
be uploaded with the developed application. Third-party libraries written in pure
Python may also be uploaded.
Google App Engine supports many Java standards and frameworks. Core to this is the
servlet 2.5 technology using the open-source Jetty Web Server, along with
accompanying technologies such as JSP. JavaServer Faces operates with some
workarounds. Though the datastore used may be unfamiliar to programmers, it is
easily accessed and supported with JPA. JDO and other methods of reading and
writing data are also provided. The Spring Framework works with GAE, however the
Spring Security module (if used) requires workarounds. Apache Struts 1 is supported,
and Struts 2 runs with workarounds.
The Django web framework and applications running on it can be used on App
Engine with modification. Django-nonrel aims to allow Django to work with non-
relational databases and the project includes support for App Engine.
Applications developed for the Grails web application framework may be modified
and deployed to Google App Engine with very little effort using the App Engine
Plugin.
Google App engine logo
Google Apps for business is free for 30 days, $5 USD per user account and month
thereafter or $50 per year. Google Apps for Education is free and offers the same
amount of storage as free Gmail accounts. Google Apps for Education combines
features from the Standard and Premier editions.
In addition to shared apps (calendar, docs, etc.), there is Google Apps Marketplace, an
App “store” for Google Apps users. It contains various apps, both free and for a fee,
which can be installed to customize the Google Apps experience for the user.
Google Apps is available in a number of distinct editions. Each edition has a limit on
the number of users that may be active at any given time. Google Apps launched with
a default user allotment of 200 users, which was shortly changed to 100 users. In
addition, users could request to have their user limit increased through a manual
process taking (at least) 1–2 weeks for approval. In January 2009, the cap was
changed so that all new accounts would receive only 50 users as opposed to 100, and
could not request more without payment. This was confirmed as relating to the launch
of the Google Apps commercial reseller program. Existing Standard Edition users
before January 2009 kept their old allocation, in addition to the ability their “request”
more users, though these limit requests are now commonly answered with suggestions
to “upgrade your subscription”. In 2011, the limit on the free Google Apps product
was further reduced to 10 users, effective for new users.
The subscription level of a Google Apps edition is billed based on the total number of
available users in the Apps account, and the edition features apply to all users
accounts in that subscription. It is not possible to purchase upgrades for a subset of
users: to increase the user limit, subscriptions must be purchased for all accounts. For
example, an upgrade from a “Standard” limit of 50 users to allow up to 60 users
would involve paying for 60 users, whether they are used or not.
● Free
● Brandable name and logos in the control panel, i.e. @yourdomain.com
● Same storage space as regular gmail.com accounts (over 10,300 MB as of
October 15th, 2012)
● Text ads standard (can be turned off in each account)
● Limited to 10 users within same domain.
● Email attachments cannot be larger than 25 megabytes.
● Limited to sending email to 500 external recipients per day per email account.
● Free for K-12 schools, colleges, and universities with up to 30,000 users
● No ads for faculty, staff, or students
● Google may serve ads to accounts not associated with enrolled students, staff
or volunteers
● Storage space 25 GB as of June 24, 2011
The Google File System capitalized on the strength of off-the-shelf servers while
minimizing hardware weaknesses. GFS is also known as GoogleFS.
The GFS node cluster is a single master with multiple chunk servers that are
continuously accessed by different client systems. Chunk servers store data as Linux
files on local disks. Stored data is divided into large chunks (64 MB), which are
replicated in the network a minimum of three times. The large chunk size reduces
network overhead.
The largest GFS clusters have more than 1,000 nodes with 300 TB disk storage
capacities. This can be accessed by hundreds of clients on a continuous basis.
MCAD22E3CLOUD COMPUTING
MODULE10
10.4. AWS
First login into your AWS account. Once you are directed to the management
console. From the left click on “Services” and from the listed options click
on EC2.
Afterward, you will be redirected to the EC2 console. Here is the image
attached to refer to.
This was all about introducing you about Amazon EC2 or Amazon Elastic
Compute Cloud. If you wish to learn about creating an EC2 instance, follow
the linked article. And if you are also another free tier account user make sure
you delete all the instances or services you have used before logging out of
from your AWS account.
10.4. AWS
Introduction to Amazon Web Services
Amazon Web Services (AWS), a subsidiary of Amazon.com, has invested
billions of dollars in IT resources distributed across the globe. These resources
are shared among all the AWS account holders across the globe. These account
themselves are entirely isolated from each other. AWS provides on-demand IT
resources to its account holders on a pay-as-you-go pricing model with no
upfront cost. Enterprises use AWS to reduce capital expenditure of building
their own private IT infrastructure (which can be expensive depending upon the
enterprise’s size and nature). All the maintenance cost is also bared by the
AWS that saves a fortune for the enterprises.
Each region is divided into at least two availability zones that are physically
isolated from each other, which provides business continuity for the
infrastructure as in a distributed system. If one zone fails to function, the
infrastructure in other availability zones remains operational. The largest region
North Virginia (US-East), has six availability zones. These availability zones
are connected by high-speed fiber-optic networking.
There are over 100 edge locations distributed all over the globe that are used
for the CloudFront content delivery network. Cloudfront can cache frequently
used content such as images and videos at edge locations and distribute it to
edge locations across the globe for high-speed delivery for end-users. It also
protects from DDOS attacks.
What is Azure?
Azure is Microsoft’s cloud platform, just like Google has it’s Google Cloud
and Amazon has it’s Amazon Web Service or AWS.000. Generally, it is a
platform through which we can use Microsoft’s resource. For example, to set
up a huge server, we will require huge investment, effort, physical space and so
on. In such situations, Microsoft Azure comes to our rescue. It will provide us
with virtual machines, fast processing of data, analytical and monitoring tools
and so on to make our work simpler. The pricing of Azure is also simpler and
cost-effective. Popularly termed as “Pay As You Go”, which means how much
you use, pay only for that.
Azure History
Microsoft unveiled Windows Azure in early October 2008 but it went to live
after February 2010. Later in 2014, Microsoft changed its name from Windows
Azure to Microsoft Azure. Azure provided a service platform for .NET
services, SQL Services, and many Live Services. Many people were still very
skeptical about “the cloud”. As an industry, we were entering a brave new
world with many possibilities. Microsoft Azure is getting bigger and better in
coming days. More tools and more functionalities are getting added. It has two
releases as of now. It’s famous version Micorosft Azure v1 and
later Microsoft Azure v2. Microsoft Azure v1 was more like JSON script
driven then the new version v2, which has interactive UI for simplification and
easy learning. Microsoft Azure v2 is still in the preview version.
● Less Operational Cost: Azure has low operational cost because it runs on
its own servers whose only job is to make the cloud functional and bug-free,
it’s usually a whole lot more reliable than your own, on-location server.
● Easy Back Up and Recovery options: Azure keep backups of all your
valuable data. In disaster situations, you can recover all your data in a single
click without your business getting affected. Cloud-based backup and
recovery solutions save time, avoid large up-front investment and roll up
third-party expertise as part of the deal.
● Work from anywhere: Azure gives you the freedom to work from
anywhere and everywhere. It just requires a network connection and
credentials. And with most serious Azure cloud services offering mobile
apps, you’re not restricted to which device you’ve got to hand.
● Increased collaboration: With Azure, teams can access, edit and share
documents anytime, from anywhere. They can work and achieve future
goals hand in hand. Another advantage of the Azure is that it preserves
records of activity and data. Timestamps are one example of the Azure’s
record keeping. Timestamps improve team collaboration by establishing
transparency and increasing accountability.
Microsoft Azure Services
2. Networking: With Azure you can use variety of networking tools, like the
Virtual Network, which can connect to on-premise data centers; Load
Balancer; Application Gateway; VPN Gateway; Azure DNS for domain
hosting, Content Delivery Network, Traffic Manager, ExpressRoute
dedicated private network fiber connections; and Network Watcher
monitoring and diagnostics
3. Storage: Includes Blob, Queue, File and Disk Storage, as well as a Data
Lake Store, Backup and Site Recovery, among others.
7. Data + Analytics: Azure has some big data tools like HDInsight for
Hadoop Spark, R Server, HBase and Storm clusters
9. Internet of Things: Includes IoT Hub and IoT Edge services that can be
combined with a variety of machine learning, analytics, and
communications services.
Before we begin learning about Google Cloud Platform, we will talk about
what is Cloud Computing. Basically it is using someone else’s computer over
the internet. Example- GCP, AWS, IBM Cloud, etc. Some interesting features
of cloud computing are as follows:
● You get computing resources on-demand and self-service. The customer
has to use a simple User Interface and they get the computing power,
storage requirements, and network you need, without human intervention.
● You can access these cloud resources over the internet from anywhere on
the globe.
● The provider of these resources has a huge collection of these resources and
allocates them to customers out of that collection.
● The resources are elastic. If you need more resources you can get more,
rapidly. If you need less, you can scale down back.
● The customers pay only for what they use or reserve. If they stop using
resources, they stop paying.
All the services listed above are provided by Google hence the name Google
Cloud Platform (GCP). Apart from these, there are so many other services
provided by GCP and also many concepts related to it that we are going to
discuss in this article.
Regions and zones:
Let’s start at the finest grain level (i.e. the smallest or first step in the
hierarchy), the Zone. A zone is an area where Google Cloud Platform
Resources like virtual machines or storage is deployed.
For example, when you launch a virtual machine in GCP using Compute
Engine, it runs in a zone you specify (suppose Europe-west2-a). Although
people consider a zone as being sort of a GCP Data Center, that’s not strictly
accurate because a zone doesn’t always correspond to one physical building.
You can still visualize the zone that way, though.
Zones are grouped into regions which are independent geographic areas and
much larger than zones (for example- all zones shown above are grouped into a
single region Europe-west2) and you can choose what regions you want your
GCP resources to be placed in. All the zones within a neighborhood have fast
network connectivity among them. Locations within regions usually have trip
network latencies of under five milliseconds.
Pricing
Google was the primary major Cloud provider to bill by the second instead of
rounding up to greater units of your time for its virtual machines as a service
offering. This may not sound like a big deal, but charges for rounding up can
really add up for customers who are creating and running lots of virtual
machines. Per second billing is obtainable for a virtual machine use through
Compute Engine and for several other services too.
Normally, you choose a virtual machine type from a typical set of those values,
but Compute Engine also offers custom virtual machine types, in order that
you’ll fine-tune the sizes of the virtual machines you use. That way,
you’ll tailor your pricing for your workloads.
Open API’s
Some people are afraid to bring their workloads to the cloud because they’re
afraid they’ll get locked into a specific vendor. But in many ways, Google
gives customers the power to run their applications elsewhere, if Google
becomes not the simplest provider for his or her needs. Here are some samples
of how Google helps its customers avoid feeling locked in. GCP services
are compatible with open source products. For example, take Cloud
Bigtable, a database that uses the interface of the open-source database Apache
HBase, which provides customers the advantage of code portability. Another
example, Cloud Dataproc provides the open-source big data environment
Hadoop, as a managed service, etc.
● GCP allows you to choose between computing, storage, big data, machine
learning, and application services for your web, mobile, analytics, and,
back-end solutions.
● It’s global and it is cost-effective.
● It’s open-source friendly.
● It’s designed for security.
Advantages of GCP
Disadvantages of GCP
1. The support fee is sort of hefty: Around 150 USD per month for the
foremost basic service (Silver class).
2. Downloading data from Google Cloud Storage is expensive. 0, 12 USD
per GB.
3. Google Cloud Platform web interface is somewhat confusing. Sometimes
I am lost while browsing around the menus.
4. Prices in both Microsoft Azure (around 0.018 USD per GB/month) or
Backblaze B2 (about 0.005 USD per GB/month) are less than Google Cloud
Storage.
5. It has a high pricing schema, almost like AWS S3, so it’s easy to urge
unexpected costs (e.g. number of requests, transfers, etc.).
Lab Exercise 1 :
In this Openstack tutorial, I will take you through the best way to use devstack for
Openstack download and Openstack Deployment on RedHat/CentOS 7 using
Devstack. OpenStack is a free and open-source software platform for cloud
computing, mostly deployed as infrastructure-as-a-service, whereby virtual servers
and other resources are made available to customers. In this openstack tutorial I will
use Devstack tool but there is another way through packstack tool which you can use
for openstack download and openstack deployment.
You can even use this openstack tutorial for Openstack download and setup openstack
on laptop using Devstack. But before going through the steps to setup openstack on
laptop you need to understand the difference between how to configure openstack in a
Server and how to configure openstack in laptop. Steps on how to configure
openstack can also be done through open stackpack stack centos.
Packstack vs Devstack
a)Packstack is mostly suitable for Red Hat Distribution Linux like CentOS and
Fedora. It basically uses puppet modules to deploy various part of Openstack
Components through ssh.
b)Devstack is a script written to create an environment with Openstack minimal setup
which can be used to setup Openstack on laptop as well.
Visit OpenstackPackstack CentOS for openstack installation through Packstack on
CentOS 7.
Step 1: Prerequisites
a) Openstack minimal setup requires minimum memory of 4GB
should be available in your system.
b) Make sure latest version of python and pip is installed in your
system.
c) Install git using yum install git
* base: centos.excellmedia.net
* epel: mirrors.aliyun.com
* extras: centos.excellmedia.net
* updates: centos.excellmedia.net
kubernetes 433/433
Resolving Dependencies
Devstack perform lot of changes in your system hence it does not perform installation
through root user. Instead you need to create an user with sudo access to perform the
installation. In our example, I have created stackuser.
stackuser:x:1000:1000::/home/stackuser:/bin/bash
Switch user to stackuser and start the git cloning. Here we are using rocky version of
Devstack. You can choose any version as per your requirement.
[root@localhost~]#su - stackuser
remote: Total 44484 (delta 30), reused 17 (delta 0), pack-reused 44430
[[local|localrc]]
ADMIN_PASSWORD=test@123
DATABASE_PASSWORD=\$ADMIN_PASSWORD
RABBIT_PASSWORD=\$ADMIN_PASSWORD
SERVICE_PASSWORD=\$ADMIN_PASSWORD
HOST_IP=192.168.0.104
RECLONE=yes
[stackuser@localhostdevstack]$ ./stack.sh
Openstack Installation and configuration usually takes around 20-25 mins depends on
your network bandwidth.
Output:-
URL: http://192.168.0.104/dashboard
User: admin
Pass: test@123
Lab Exercise 2:
HOW TO CREATE NEW LINUX VM IN OPENSTACK
DASHBOARD (HORIZON)?
Select Instance Boot Source (eg. "Image"), and choose desired image (eg. "Ubuntu
16.04 LTS") by clicking on arrow.
If you do not need to have the system disk bigger than the size defined in a
chosen flavor, we recommend setting "Create New Volume" feature to "No"
state.
Choose Flavor (eg. eo1.xsmall).
Choose or generate SSH keypair for your VM. Next, launch your instance by clicking
on blue button.
You will see "Instances" menu with your newly created VM.
This article will demonstrate that how to launch the first Openstack instance.
In the previous articles ,we have setup the openstack software and gone
through the openstack dashboard functionalities. In an order to launch the
openstack instances, first we need to create the network security group, rules
& key pairs to access the instances from other network. In the security rule , I
will make the port 22 and ping protocol to allow in the firewall. Note that once
you have download the key pair , there is no way to download it again due to
security reason. Let’s create the first openstack instance.
3. Click on “Create Security group”. Enter the name and description for the
security group.
Create Security Group
4. Once the group has been created successfully, Click on “Manage Rules” .
Once you have added those rules to the security group, it will look like below.
2. Navigate to security & access. Click the tab called “Key Pairs” and click
on “Create key Pair” .
Key Pairs – Openstack
3. Enter the Key pair name. (Keep Some meaning full name). Click on
“Create key Pair”
At this point , we have created the new security group and key pair. The
security group will allows “ssh” &ping from anywhere.
● Availability Zone – nova . (Need to select your compute node). In our case
control node & compute nodes are same.
● Instance Name – Enter the desired instance name
● Flavour – Select the available flavouraccording to your need. (See the details
in right side)
● Instance Count – Enter the instance Count
● Boot Source – Select boot from pre-defined image.
● Image Name – select “cirros” since its very small Linux foot print for testing
openstack.
4. Click on Access & security tab for the instance. From the drop down box ,
select the key pair “UAPAIR” which we have created earlier. Also select the
security group which we have created. Click “Launch” to launch the new
instance.
Select the security group & Key Pair
5. Here you can see that instance has been launched. It will take few minutes
to boot the instance depends on the image size which we have selected.
6. Once the instance is completely up , you can see the screen like below.
Ope
nstack Instance is up
In the IP address tab , you can get the private IP address for the instance.
Using this IP , You should be able to access the instance.
7. If you would like to see the instance console , click the instance name and
select the console tab. You should be able to access the instance here as well
by double clicking the console bar.
Ins
tance Console
In Openstack’s kilo branch, console may not load properly if you didn’t add
the below parameter in the local.conf file during the installation.
“enable_service n-cauth”
8. You can also check the log to know the instance is booted or not . (If
console is not working due to above mentioned issue).
[email protected]'s password:
$ sudosu -
# ifconfig -a
collisions:0 txqueuelen:1000
collisions:0 txqueuelen:0
# route
192.168.204.0 * 255.255.255.0 U 0 0 0
eth0