Distributed and Cloud Computing

K. Hwang, G. Fox and J. Dongarra

Chapter 3: Virtual Machines and Virtualization

of Clusters and datacenters

Adapted from Kai Hwang

University of Southern California
March 30, 2012

Copyright © 2012, Elsevier Inc. All rights reserved. 1 1-1

 IMPLEMENTATION LEVELS OF VIRTUALIZATION
 Virtualization is a computer architecture technology by
which multiple virtual machines (VMs) are multiplexed in
the same hardware machine.
 The purpose of a VM is to enhance resource sharing by
many users and improve computer performance in terms
of resource utilization and application flexibility.
 3.Hardware resources (CPU, memory , I/O devices, etc.)
or software resources (operating system and software
libraries) can be virtualized in various functional layers.
 4.This virtualization technology has been revitalized as the
demand for distributed and cloud computing increased
sharply in recent years.

2
IMPLEMENTATION LEVELS OF VIRTUALIZATION

The idea is to separate the hardware from the software to

yield better system efficiency.
For example, computer users gained access to much
enlarged memory space when the concept of Virtual
memory was introduced.
Similarly, virtualization techniques can be applied to
enhance the use of compute engines, networks, and
storage.

3
 Virtualization for Datacenter Automation
to serve millions of clients, simultaneously

 Server Consolidation in Virtualized Datacenter

 Virtual Storage Provisioning and Deprovisioning

 Cloud Operating Systems for Virtual Datacenters

 Trust Management in virtualized Datacenters

Copyright © 2012, Elsevier Inc. All rights reserved. 4 1-4

Difference between Traditional Computer
and Virtual machines

(Courtesy of VMWare, 2008)

Copyright © 2012, Elsevier Inc. All rights reserved. 5 1-5

 Virtual Machine, Guest Operating System,
and VMM (Virtual Machine Monitor) :

The Virtualization layer is the middleware between the

underlying hardware and virtual machines represented in the
system, also known as virtual machine monitor (VMM) or
hypervisor.

Copyright © 2012, Elsevier Inc. All rights reserved. 6 1-6

Guest OS
 A guest OS is the software installed on either a
virtual machine (VM) or partitioned disk that
describes an operating system that is different
than the host operating system.
 Virtualization technology allows a computer to
run more than a single OS at the same time.

7
 (Courtesy of VMWare, 2008)

Copyright © 2012, Elsevier Inc. All rights reserved. 8 1-8

Virtualization Ranging from Hardware to
Applications in Five Abstraction Levels

9
Virtualization at ISA (Instruction Set Architecture) level:
Emulating a given ISA by the ISA of the host machine.
•e.g, MIPS binary code can run on an x-86-based host machine with the
help of ISA emulation.
• Typical systems: Bochs, Crusoe, Quemu, BIRD, Dynamo
Advantage:
• It can run a large amount of legacy binary codes written for
various processors on any given new hardware host machines
• best application flexibility
Shortcoming & limitation:
• One source instruction may require tens or hundreds of native
target instructions to perform its function, which is relatively slow.
• V-ISA requires adding a processor-specific software translation
layer in the complier.
Copyright © 2012, Elsevier Inc. All rights reserved. 10 1 - 10
Virtualization at Hardware Abstraction level:
Virtualization is performed right on top of the hardware.
•It generates virtual hardware environments for VMs, and manages the
underlying hardware through virtualization.
•Typical systems: VMware, Virtual PC, Denali, Xen

Advantage:
•Has higher performance and good application isolation
Shortcoming & limitation:
•Very expensive to implement (complexity)

Copyright © 2012, Elsevier Inc. All rights reserved. 11 1 - 11

Virtualization at Operating System (OS) level:
It is an abstraction layer between traditional OS and user applications.
•This virtualization creates isolated containers on a single physical server
and the OS-instance to utilize the hardware and software in datacenters.
•Typical systems: Jail / Virtual Environment / Ensim's VPS / FVM

Advantage:
•Has minimal starup/shutdown cost, low resource requirement, and high
scalability; synchronize VM and host state changes.

Shortcoming & limitation:

•All VMs at the operating system level must have the same kind of guest
OS
•Poor application flexibility and isolation.

Copyright © 2012, Elsevier Inc. All rights reserved. 12 1 - 12

Virtualization at OS Level

Copyright © 2012, Elsevier Inc. All rights reserved. 13 1 - 13

Virtualization for Linux and Windows NT
Platforms

Copyright © 2012, Elsevier Inc. All rights reserved. 14 1 - 14

Copyright © 2012, Elsevier Inc. All rights reserved. 15 1 - 15
Advantages of OS Extension for Virtualization

1. VMs at OS level has minimum startup/shutdown costs

2. OS-level VM can easily synchronize with its

environment

Disadvantage of OS Extension for Virtualization

All VMs in the same OS container must have the same or
similar guest OS, which restrict application flexibility of
different VMs on the same physical machine.

Copyright © 2012, Elsevier Inc. All rights reserved. 16 1 - 16

Library Support level:

It creates execution environments for running alien

programs on a platform rather than creating VM to run the
entire operating system.
•It is done by API call interception and remapping.
•Typical systems: Wine, WAB, LxRun , VisualMainWin

Advantage:
•It has very low implementation effort

Shortcoming & limitation:

•poor application flexibility and isolation

Copyright © 2012, Elsevier Inc. All rights reserved. 17 1 - 17

Virtualization with Middleware/Library Support

Copyright © 2012, Elsevier Inc. All rights reserved. 18 1 - 18

The vCUDA for Virtualization of GPGPU

Copyright © 2012, Elsevier Inc. All rights reserved. 19 1 - 19

User-Application level:
It virtualizes an application as a virtual machine.
•This layer sits as an application program on top of an
operating system and exports an abstraction of a VM that
can run programs written and compiled to a particular
abstract machine definition.
•Typical systems: JVM , NET CLI , Panot

Advantage:
•has the best application isolation

Shortcoming & limitation:

•low performance, low application flexibility and high
implementation complexity.

Copyright © 2012, Elsevier Inc. All rights reserved. 20 1 - 20

More Xs mean higher merit

Copyright © 2012, Elsevier Inc. All rights reserved. 21 1 - 21

Virtualization Structures/Tools and Mechanism
Hypervisor
A hypervisor is a hardware virtualization technique allowing multiple
operating systems, called guests to run on a host machine. This is also
called the Virtual Machine Monitor (VMM).

Type 1: bare metal hypervisor

•sits on the bare metal computer hardware like the CPU, memory, etc.
•All guest operating systems are a layer above the hypervisor.
•The original CP/CMS hypervisor developed by IBM was of this kind.

Type 2: hosted hypervisor

•Run over a host operating system.
•Hypervisor is the second layer over the hardware.
•Guest operating systems run a layer over the hypervisor.
•The OS is usually unaware of the virtualization
Copyright © 2012, Elsevier Inc. All rights reserved. 22 1 - 22
Major VMM and Hypervisor Providers

Copyright © 2012, Elsevier Inc. All rights reserved. 23 1 - 23

The XEN Architecture (1)

Copyright © 2012, Elsevier Inc. All rights reserved. 24 1 - 24

The XEN Architecture (2)

Copyright © 2012, Elsevier Inc. All rights reserved. 25 1 - 25

The XEN Architecture (3)

Copyright © 2012, Elsevier Inc. All rights reserved. 26 1 - 26

 Full Virtualization vs. Para-Virtualization
Full virtualization
•Does not need to modify guest OS, and critical instructions are
emulated by software through the use of binary translation.
•VMware Workstation applies full virtualization, which uses binary
translation to automatically modify x86 software on-the-fly to replace
critical instructions.
•Advantage: no need to modify OS.
•Disadvantage: binary translation slows down the performance.
Para virtualization
•Reduces the overhead, but cost of maintaining a paravirtualized OS is
high.
•The improvement depends on the workload.
•Para virtualization must modify guest OS, non-virtualizable instructions
are replaced by hypercalls that communicate directly with the hypervisor
or VMM.
•Para virtualization is supported by Xen, Denali and VMware ESX.
Copyright © 2012, Elsevier Inc. All rights reserved. 27 1 - 27
Full Virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 28 1 - 28

 Binary
Translation
of Guest OS
Requests
using a
VMM:

Copyright © 2012, Elsevier Inc. All rights reserved. 29 1 - 29

Para- Virtualization with Compiler Support.

The KVM builds offers kernel-based VM on the Linux

platform, based on para-virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 30 1 - 30

Copyright © 2012, Elsevier Inc. All rights reserved. 31 1 - 31
VMWare ESX Server for Para-Virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 32 1 - 32

VIRTUALIZATION OF CPU, MEMORY, AND
I/O DEVICES
To support virtualization, processors such as the
x86 employ a special running mode and instructions, known
as hardware-assisted virtualization. In this way, the VMM
and guest OS run in different modes and all sensitive
instructions of the guest OS and its applications are trapped in
the VMM. To save processor states, mode switching is
completed by hardware.
 Hardware Support for Virtualization
 CPU Virtualization
 Memory Virtualization
 I/O Virtualization
 Virtualization in Multi-Core Processors
33
Hardware Support for Virtualization
 Modern operating systems and processors permit multiple
processes to run simultaneously. If there is no protection
mechanism in a processor, all instructions from different
processes will access the hardware directly and cause a
system crash.
 Therefore, all processors have at least two modes, user mode
and supervisor mode, to ensure controlled access of critical
hardware.
 Instructions running in supervisor mode are called privileged
instructions. Other instructions are unprivileged instructions.

34
35
CPU Virtualization
 A VM is a duplicate of an existing computer system in which
a majority of the VM instructions are executed on the host
processor in native mode.
 The critical instructions are divided into three categories:
privileged instructions,
controlsensitive instructions, and
behavior-sensitive instructions.
 Privileged instructions execute in a privileged mode and will
be trapped if executed outside this mode.
 Control-sensitive instructions attempt to change the
configuration of resources used.
 Behavior-sensitive instructions have different behaviors
depending on the configuration of resources, including the
load and store operations over the virtual memory.
36
37
 Memory Virtualization
Virtual memory virtualization is similar to the virtual
memory support provided by modern operating systems. In a
traditional execution environment, the operating system
maintains mappings of virtual memory to machine memory
using page tables, which is a one-stage mapping from virtual
memory to machine memory.
 All modern x86 CPUs include a memory management unit
(MMU) and a translation lookaside buffer (TLB) to optimize
virtual memory performance.
 However, in a virtual execution environment, virtual memory
virtualization involves sharing the physical system memory in
RAM and dynamically allocating it to the physical memory of
the VMs.
 That means a two-stage mapping process should be maintained
by the guest OS and the VMM, respectively: virtual memory to
physical memory and physical memory to machine memory.
38
39
40
I/O Virtualization
 I/O virtualization involves managing the routing of
I/O requests between virtual devices and the shared
physical hardware. At the time of this writing, there
are three ways to implement I/O virtualization:
 full device emulation,
 para-virtualization, and
 Direct I/O.

41
42
43
Conclusions on CPU, Memory
and I/O Virtualization :
 CPU virtualization demands hardware-assisted traps of
sensitive instructions by the VMM

 Memory virtualization demands special hardware support

(shadow page tables by VMWare or extended page table by
Intel) to help translate virtual address into physical address
and machine memory in two stages.

 I/O virtualization is the most difficult one to realize due to

the complexity if I/O service routines and the emulation
needed between the guest OS and host OS.

Copyright © 2012, Elsevier Inc. All rights reserved. 44 1 - 44

 Multi-Core Virtualization:
VCPU vs. traditional CPU

Figure 3.16 Four VCPUs are exposed to the software, only three cores are actually
present. VCPUs V0, V1, and V3 have been transparently migrated, while VCPU V2
has been transparently suspended. (Courtesy of Wells, et al., “Dynamic
Heterogeneity and the Need for Multicore Virtualization”, ACM SIGOPS Operating
Systems Review, ACM Press, 2009 [68] )

Copyright © 2012, Elsevier Inc. All rights reserved. 45 1 - 45

 Virtual Cores vs. Physical Processor Cores

Physical cores Virtual cores

The actual physical cores present in There can be more virtual cores
the processor. visible to a single OS than there are
physical cores.
More burden on the software to write Design of software becomes easier
applications which can run directly on as the hardware assists the software
the cores. in dynamic resource utilization.
Hardware provides no assistance to Hardware provides assistance to the
the software and is hence simpler. software and is hence more
complex.
Poor resource management. Better resource management.
The lowest level of system software The lowest level of system software
has to be modified. need not be modified.

Copyright © 2012, Elsevier Inc. All rights reserved. 46 1 - 46

 (Courtesy of Marty and Hill, 2007)

Copyright © 2012, Elsevier Inc. All rights reserved. 47 1 - 47

Virtual Clusters in Many Cores
Space Sharing of VMs -- Virtual Hierarchy

(Courtesy of Marty and Hill, 2007)

Copyright © 2012, Elsevier Inc. All rights reserved. 48 1 - 48

VIRTUAL CLUSTERS AND RESOURCE MANAGEMENT
 Physical versus Virtual Clusters
Fast Deployment and Effective Scheduling
High-Performance Virtual Storage
 Live VM Migration Steps and Performance Effects
 Migration of Memory, Files, and Network Resources
Memory Migration
File System Migration
Network Migration
Live Migration of VM Using Xen
 Dynamic Deployment of Virtual Clusters

49
 Virtual Cluster Characteristics
 The virtual cluster nodes can be either physical or virtual machines. Multiple VMs
running with different OSs can be deployed on the same physical node.

 A VM runs with a guest OS, which is often different from the host OS, that manages the
resources in the physical machine, where the VM is implemented.

 The purpose of using VMs is to consolidate multiple functionalities on the same server.
This will greatly enhance the server utilization and application flexibility.

 VMs can be colonized (replicated) in multiple servers for the purpose of promoting
distributed parallelism, fault tolerance, and disaster recovery.

 The size (number of nodes) of a virtual cluster can grow or shrink dynamically, similarly to
the way an overlay network varies in size in a P2P network.

 The failure of any physical nodes may disable some VMs installed on the failing nodes.
But the failure of VMs will not pull down the host system.

Copyright © 2012, Elsevier Inc. All rights reserved. 50 1 - 50

Virtual Clusters vs. Physical Clusters

Copyright © 2012, Elsevier Inc. All rights reserved. 51 1 - 51

 The provisioning of VMs to a virtual cluster is done dynamically to
have the following interesting properties:
• The virtual cluster nodes can be either physical or virtual machines.
Multiple VMs running with different OSes can be deployed on the
same physical node.
• A VM runs with a guest OS, which is often different from the host OS,
that manages the resources in the physical machine, where the VM
is implemented.
• The purpose of using VMs is to consolidate multiple functionalities on
the same server. This will greatly enhance server utilization and
application flexibility.
• VMs can be colonized (replicated) in multiple servers for the purpose of
promoting distributed parallelism, fault tolerance, and disaster
recovery.
• The size (number of nodes) of a virtual cluster can grow or shrink
dynamically, similar to the way an overlay network varies in size in a
peer-to-peer (P2P) network.
• The failure of any physical nodes may disable some VMs installed on
the failing nodes. But the failure of VMs will not pull down the host
system.
52
Copyright © 2012, Elsevier Inc. All rights reserved. 53 1 - 53
Live Migration of Virtual Machines

Copyright © 2012, Elsevier Inc. All rights reserved. 54 1 - 54

Copyright © 2012, Elsevier Inc. All rights reserved. 55 1 - 55
Copyright © 2012, Elsevier Inc. All rights reserved. 56 1 - 56
Virtual Cluster Projects

Copyright © 2012, Elsevier Inc. All rights reserved. 57 1 - 57

Copyright © 2012, Elsevier Inc. All rights reserved. 58 1 - 58
Cluster-on-Demand (COD Project)
at Duke University

Copyright © 2012, Elsevier Inc. All rights reserved. 59 1 - 59

Copyright © 2012, Elsevier Inc. All rights reserved. 60 1 - 60
VIOLIN Project at Purdue University

Copyright © 2012, Elsevier Inc. All rights reserved. 61 1 - 61

Copyright © 2012, Elsevier Inc. All rights reserved. 62 1 - 62
VIRTUALIZATION FOR DATA-CENTER AUTOMATION
 Server Consolidation in Data Centers
 Virtual Storage Management
 Cloud OS for Virtualized Data Centers
 Trust Management in Virtualized Data Centers

63
Parallax for VM Storage Management

Copyright © 2012, Elsevier Inc. All rights reserved. 64 1 - 64

Cloud OS for Building Private Clouds

Copyright © 2012, Elsevier Inc. All rights reserved. 65 1 - 65

Eucalyptus : An Open-Source OS for Setting
Up and Managing Private Clouds

Copyright © 2012, Elsevier Inc. All rights reserved. 66 1 - 66

Copyright © 2012, Elsevier Inc. All rights reserved. 67 1 - 67
Copyright © 2012, Elsevier Inc. All rights reserved. 68 1 - 68
Trusted Zones for VM Insulation
Insulate Anti-malware
Federate infrastructure from
Identity Malware, Trojans Cybercrime
identities with
federation and cybercriminalsintelligence
public clouds
Strong
APP APP
Tenant #2 authentication
OS OS

Virtual Infrastructure Insulate

Control and information
Virtual isolate VM in
network the virtual from other Data loss
tenants prevention
security infrastructure APP APP
Tenant #1
OS OS

Virtual Infrastructure Insulate Encryption &

Access Segregate and information key mgmt
Mgmt control user from cloud
providers’
access employees Tokenization
Cloud Provider
Security Info. &
Event Mgmt Physical Infrastructure GRC
Enable end to end view of security events
and compliance across infrastructures
(Courtesy of L. Nick, EMC 2008)

Copyright © 2012, Elsevier Inc. All rights reserved. 69 1 - 69