Virtualization

<<<……………>>>
Motivation
 Three fundamental abstractions are necessary to describe the
operation of a computing systems:
(1) interpreters/processors, (2) memory, (3) communications
links
 As the scale of a system and the size of its users grows, it
becomes very challenging to manage its recourses
 Resource management issues:
 provision for peak demands  overprovisioning
 heterogeneity of hardware and software
 machine failures
 Virtualization is a basic enabler of Cloud Computing; it
simplifies the management of physical resources for
the three abstractions

 For example, the state of a virtual machine (VM) running under a

virtual machine monitor (VMM) can de saved and migrated to another
server to balance the load
 For example, virtualization allows users to operate in environments
2
they are familiar with, rather than forcing them to specific ones
 Motivation (cont’d)

 “Virtualization, in computing, refers to the act of

creating a virtual (rather than actual) version of
something, including but not limited to a virtual
computer hardware platform, operating system (OS),
storage device, or computer network resources.”

 Virtualization abstracts the underlying resources;

simplifies their use; isolates users from one another;
and supports replication which increases the elasticity
of a system

3
 Motivation (cont’d)

 Cloud resource virtualization is important for:

 Performance isolation
 as we can dynamically assign and account for
resources across different applications
 System security:
 as it allows isolation of services running on the
same hardware
 Performance and reliability:
 as it allows applications to migrate from one
platform to another
 The development and management of services
offered by a provider
4
History of Virtualization
 1960’s, IBM: CP/CMS control program: a virtual machine
operating system for the IBM System/360 Model 67
 2000, IBM: z-series with 64-bit virtual address spaces and
backward compatible with the System/360
 1974: Popek and Golberg from UCLA published “Formal
Requirements for Virtualizable Third Generation Architectures”
where they listed the conditions a computer architecture should
satisfy to support virtualization efficiently. The popular x86
architecture that originated in the 1970s did not support these
requirements for decades.
 1990’s, Stanford researchers, VMware: Researchers
developed a new hypervisor and founded VMware, the biggest
virtualization company of today’s. First virtualization solution
was is 1999 for x86.
 Today many virtualization solutions: Xen from Cambridge, KVM,
Hyper-V, …
 IBM was the first to produce and sell virtualization for the 5
mainframe. But, VMware popularised virtualization for the
Virtualization
 Virtualization simulates the interface to a
physical object by:

 Multiplexing: creates multiple virtual

objects from one instance of a physical
object. Many virtual objects to one
physical. Example - a processor is
multiplexed among a number of processes
or threads.
 Aggregation: creates one virtual object
from multiple physical objects. One virtual
object to many physical objects. Example -
a number of physical disks are aggregated 6
Virtualization
 Emulation: constructs a virtual
object of a certain type from a
different type of a physical object.
Example - a physical disk emulates a
Random Access Memory (RAM).
 Multiplexing and emulation.
Examples - virtual memory with
paging multiplexes real memory and
disk; a virtual address emulates a
real address.
7
Layering and Virtualization
 Layering – a common approach to manage system
complexity:
 Simplifies the description of the subsystems;
each subsystem is abstracted through its
interfaces with the other subsystems
 Minimizes the interactions among the subsystems
of a complex system
 With layering we are able to design, implement,
and modify the individual subsystems
independently

 Layering in a computer system:

 Hardware
 Software -


Layering and Interfaces
A1 Applications

API

Libraries A2

ABI
System calls

Operating System A3

ISA
System ISA User ISA
Hardware

Application Programming Interface (API), Application Binary Interface

(ABI), and Instruction Set Architecture (ISA). An application uses library
functions (A1), makes system calls (A2), and executes machine
instructions (A3)
Interfaces
 Instruction Set Architecture (ISA) – at the boundary between
hardware and software.

 Application Binary Interface (ABI) – allows the ensemble

consisting of the application and the library modules to access
the hardware; the ABI does not include privileged system
instructions, instead it invokes system calls.

 Application Program Interface (API) - defines the set of

instructions the hardware was designed to execute and gives the
application access to the ISA; it includes high-level language
(HLL) library calls which often invoke system calls

10
Code portability

 Binaries created by a compiler for a specific ISA and a specific

operating systems are not portable

 It is possible, though, to compile a HLL program for a virtual

machine (VM) environment where portable code is produced
and distributed and then converted by binary translators to the
ISA of the host system

 A dynamic binary translation converts blocks of guest

instructions from the portable code to the host instruction and
leads to a significant performance improvement, as such
blocks are cached and reused

11
HLL Language Translations
HLL code

Compiler front-end Compiler

Intermediate Portable
code code

Compiler back-end VM loader

Object code VM image

VM compiler/ VM compiler/
Loader
interpreter interpreter

Memory Memory Memory

image image ISA-1 image ISA-2

12
 Virtual Machine Monitor (VMM /
Hypervisor)
 A virtual machine monitor (VMM/hypervisor) partitions
the resources of computer system into one or more virtual
machines (VMs). Allows several operating systems to run
concurrently on a single hardware platform
 A VM is an execution environment that runs an OS
 VM – an isolated environment that appears to be a whole
computer, but actually only has access to a portion of the
computer resources
 A VMM allows:
 Multiple services to share
the same platform
 Live migration - the movement
of a server from one platform to another
 System modification while maintaining
backward compatibility with the original system
 Enforces isolation among the systems, thus security 13
 A guest operating system is an OS that runs in a VM under
VMM Virtualizes the CPU and the
Memory
 A VMM (also hypervisor):
 Traps the privileged instructions executed by a guest OS
and enforces the correctness and safety of the operation
 Traps interrupts and dispatches them to the individual guest
operating systems
 Controls the virtual memory management
 Maintains a shadow page table for each guest OS and
replicates any modification made by the guest OS in its own
shadow page table. This shadow page table points to the
actual page frame and it is used by the Memory
Management Unit (MMU) for dynamic address translation.
 Monitors the system performance and takes corrective
actions to avoid performance degradation. For example,
the VMM may swap out a VM to avoid thrashing.

14
Type 1 and 2 Hypervisors
Type 1 Hypervisor Type 2 Hypervisor

 Taxonomy of VMMs:
1. Type 1 Hypervisor (bare metal, native): supports multiple virtual
machines and runs directly on the hardware (e.g., VMware ESX ,
Xen, Denali)
2. Type 2 Hypervisor (hosted) VM - runs under a host operating
system (e.g., user-mode Linux)
15
Examples of Hypervisors

16
Performance and Security Isolation
 The run-time behavior of an application is affected by other
applications running concurrently on the same platform and
competing for CPU cycles, cache, main memory, disk and
network access. Thus, it is difficult to predict the completion
time!

 Performance isolation - a critical condition for QoS guarantees

in shared computing environments

 A VMM is a much simpler and better specified system than a

traditional operating system. Example - Xen has
approximately 60,000 lines of code; Denali has only about half:
30,000

 The security vulnerability of VMMs is considerably reduced as

the systems expose a much smaller number of privileged
functions. For example, Xen VMM has 28 hypercalls while Linux 17
Conditions for Efficient Virtualization:

 Conditions for efficient virtualization:

1.A program running under the VMM
should exhibit a behavior essentially
identical to that demonstrated when
running on an equivalent machine
directly.
2.The VMM should be in complete
control of the virtualized resources.
3.A statistically significant fraction of
machine instructions must be
executed without the intervention of 18
Dual-Mode Operation
 Dual-mode operation allows OS to protect itself and
other system components
 User mode and kernel mode
 Mode bit provided by hardware
 Ability to distinguish when system is running user or kernel
code
 Some instructions are privileged, only executable in
kernel mode
 System call changes mode to kernel, return resets it to
user

19
User-mode vs Kernel-mode

 Kernel-code (in particular, interrupt

handlers) runs in kernel mode
 the hardware allows all machine
instructions to be executed and
allows unrestricted access to
memory and I/O ports
 Everything else runs in user mode
 The OS relies very heavily on this
hardware-enforced protection
mechanism
20
 Challenges of x86 CPU
Virtualization
 Four layers of privilege execution  rings
 User applications run in ring 3
 OS runs in ring 0
 In which ring should the VMM run?
 In ring 0, then, same privileges as an OS  wrong
 In rings 1,2,3, then OS has higher privileges  wrong
 Move the OS to ring 1 and the VMM in ring 0  OK
 Three classes of machine instructions:
1. privileged instructions can be executed
in kernel mode. When attempted to be
executed in user mode, they cause a trap
and so executed in kernel mode.
2. nonprivileged instructions the ones that can be executed in
user mode
3. sensitive instructions can be executed in either kernel or user
but they behave differently. Sensitive instructions require special
21
precautions at execution time.
Techniques for Virtualizing CPU on
x86
1. Full virtualization with binary translation
2. OS-assisted Virtualization or Para-virtualization
3. Hardware assisted virtualization

22
 Techniques for Virtualizing CPU on
x86
Full virtualization – a guest OS can run unchanged under the
VMM as if it was running directly on the hardware platform. Each
VM runs an exact copy of the actual hardware.
 Binary translation rewrites parts of the code on the fly to replace
sensitive but not privileged instructions with safe code to emulate the
original instruction
 “The hypervisor translates all operating system instructions on the fly
and caches the results for future use, while user level instructions run
unmodified at native speed.”
 Examples: VMware, Microsoft Virtual Server
 Advantages:
 No hardware assistance,
 No modifications of the guest OS
 Isolation, Security
 Disadvantages:
 Speed of execution

23
 Techniques for Virtualizing CPU on
x86
Para-virtualization – “involves modifying the OS kernel to
replace non- virtualizable instructions with hypercalls that
communicate directly with the virtualization layer hypervisor. The
hypervisor also provides hypercall interfaces for other critical
kernel operations such as memory management, interrupt
handling and time keeping. “ (from VMware paper)

 Advantage: faster execution, lower virtualization overhead

 Disadvantage: poor portability
 Examples: Xen, Denali

24
Full Virtualization and
Paravirtualization

Guest OS Guest OS
Hardware Hardware
abstraction abstraction
layer layer

Hypervisor Hypervisor

Hardware Hardware

(a) Full virtualization (b) Paravirtualization

25
 Techniques for Virtualizing CPU on
x86
 Hardware Assisted Virtualization – “a new CPU execution
mode feature that allows the VMM to run in a new root mode
below ring 0. As depicted in Figure 7, privileged and sensitive
calls are set to automatically trap to the hypervisor, removing
the need for either binary translation or paravirtualization“
(from VMware paper)

 Advantage: even faster execution

 Examples: Intel VT-x, Xen 3.x

26
VT-x, a Major Architectural
Enhancement
 In 2005 Intel released two Pentium 4 models supporting VT-x.
 VT-x supports two modes of operations (Figure (a)):
1. VMX root - for VMM operations.
2. VMX non-root - support a VM.
 And a new data structure called the Virtual Machine Control
Structure including host-state and guest-state areas (Figure (b)).
 VM entry - the processor state is loaded from the guest-state of
the VM scheduled to run; then the control is transferred from VMM
to the VM.
 VM exit - saves the processor state in the guest-state area of the
running VM; then it loads the processor state from the host-state
area, finally transfers control to the VMM.
Virtual-machine control structure
VM entry
host-state
VMX root VMX non-root
guest-state
VM exit

(a) (b)
27
Xen - a VMM based on
Paravirtualization
 The goal of the Cambridge group - design a VMM capable of
scaling to about 100 VMs running standard applications and
services without any modifications to the Application Binary
Interface (ABI). (2003, Computing Laboratory, Cambridge University)
 Linux, Minix, NetBSD, FreeBSD and others can operate as
paravirtualized Xen guest OS running on x86, x86-64, Itanium,
and ARM architectures.
 Xen domain - ensemble of address spaces hosting a guest OS
and applications running under the guest OS. Runs on a virtual
CPU.
 Dom0 - dedicated to execution of Xen control functions and
privileged instructions.
 DomU - a user domain.
 Applications make system calls using hypercalls processed by
Xen; privileged instructions issued by a guest OS are
paravirtualized and must be validated by Xen.
28
 Xen

Management
OS Application Application Application

Guest OS Guest OS Guest OS

Xen-aware
device drivers
Xen-aware Xen-aware Xen-aware
device drivers device drivers device drivers

Xen
Domain0 control Virtual x86 Virtual physical Virtual block
interface Virtual network
CPU memory devices

X86 hardware

29
Strategies for virtual memory management, CPU
multiplexing, and I/O devices

30
Linux Containers
 A Linux Container is a Linux process (or processes) that is a
virtual environment with its own process network space.
(lightweight process virtualization)
 Containers share portions of the host kernel
 Containers use:
 Namespaces: per-process isolation of OS resources (filesystem, network
and user ids)
 Cgroups: resource management and accounting per process
 Examples for using containers:
 https://www.dotcloud.com/
 https://www.heroku.com/

31
Xen I/O
I/O channel
Driver domain Guest domain
Bridge
Xen zero-copy Backend Frontend

semantics for data Network

interface
transfer using I/O rings.
Event channel
(a) The communication XEN
between a guest
NIC
domain and the (a)

driver domain over

an I/O and an event Request queue
channel; NIC is the Producer Request
Consumer Request
Network Interface (private pointer in Xen) (shared pointer updated
by the guest OS)
Controller.
(b)The circular ring of
buffers.
Outstanding Unused
descriptors descriptors

Consumer Response
Producer Response
(private pointer maintained by
(shared pointer updated
by Xen)
Response queue the guest OS) 32
(b)
Xen Network Architecture
The original architecture The
optimised architecture

Driver domain Guest domain Driver domain Guest domain

Bridge Bridge

Offload
I/O
Driver I/O
channel channel High Level
NIC Backend Virtual NIC Backend Virtual
Driver Interface Interface Driver Interface
Interface

Physical Xen VMM Physical Xen VMM

NIC NIC

(a) (b)
33
The Darker Side of Virtualization

 In a layered structure, a defense mechanism at some layer can

be disabled by malware running at a layer below it.
 It is feasible to insert a rogue VMM, a Virtual-Machine Based
Rootkit (VMBR) between the physical hardware and an
operating system.
 Rootkit - malware with a privileged access to a system.
 The VMBR can enable a separate malicious OS to run
surreptitiously and make this malicious OS invisible to the
guest OS and to the application running under it.
 Under the protection of the VMBR, the malicious OS could:
 observe the data, the events, or the state of the target
system.
 run services, such as spam relays or distributed denial-of-
service attacks.
 interfere with the application. 34
The Darker Side of Virtualization (con’t)

Application

Application
Malicious Guest OS
OS
Operating
Malicious system (OS)
OS Virtual machine monitor

Virtual machine based rootkit Virtual machine based rootkit

Hardware Hardware

(a) (b)
The insertion of a Virtual-Machine Based Rootkit (VMBR) as the
lowest layer of the software stack running on the physical
hardware; (a) below an operating system; (b) below a legitimate
virtual machine monitor. The VMBR enables a malicious OS to
run surreptitiously and makes it invisible to the genuine or the
35
guest OS and to the application.