Resource Management and Monitoring

Module 10

© 2011 VMware Inc. All rights reserved

 You Are Here

Course Introduction Data Protection

Introduction to Virtualization Access & Authentication Control

Virtual Machines Resource Management and Monitoring

VMware vCenter Server High Availability

Configure and Manage Virtual Networks Scalability

Configure and Manage Virtual Storage Patch Management

Managing Virtual Machines Installing vSphere Components

VMware vSphere: Install, Configure, Manage – Revision A 10-2

© 2011 VMware Inc. All rights reserved

 Importance

Although the VMkernel works proactively to avoid resource

contention, maximizing performance requires both analysis and
ongoing monitoring.

VMware vSphere: Install, Configure, Manage – Revision A 10-3

© 2011 VMware Inc. All rights reserved

 Module Lessons

Lesson 1: Virtual CPU and Memory Concepts

Lesson 2: Resource Controls
Lesson 3: Resource Pools
Lesson 4: Monitoring Resource Usage
Lesson 5: Using Alarms

VMware vSphere: Install, Configure, Manage – Revision A 10-4

© 2011 VMware Inc. All rights reserved

 Lesson 1:
Virtual CPU and Memory Concepts

VMware vSphere: Install, Configure, Manage – Revision A 10-5

© 2011 VMware Inc. All rights reserved

 Learner Objectives

After this lesson, you should be able to do the following:

Discuss CPU and memory concepts in a virtualized environment.

Describe what over commitment of a resource means.

Identify additional technologies that improve memory utilization.

Describe how virtual SMP works and how hyperthreading is used by
the VMkernel.

VMware vSphere: Install, Configure, Manage – Revision A 10-6

© 2011 VMware Inc. All rights reserved

 Memory Virtualization Basics

virtual machine
There are 3 layers of memory
in VMware vSphere®.

Guest OS virtual memory is application
guest OS
presented to applications virtual memory
by the operating system.

Guest OS physical memory operating guest OS
is presented to the virtual physical memory
system
machine by the VMkernel.

Host physical memory
managed by the VMkernel VMware®
provides a contiguous, ESXi host
ESXi™ host
addressable memory space physical memory
that will be used by the
virtual machine.

VMware vSphere: Install, Configure, Manage – Revision A 10-7

© 2011 VMware Inc. All rights reserved

 Virtual Machine Memory Overcommitment

Allow RAM overcommitment allocated memory = 512 MB

A virtual machine swap file (.vswp) is reserved memory = 256 MB
created when a virtual machine’s
maximum RAM allocation exceeds On On On Off
its minimum RAM allocation
Virtual machines power on only if:

Minimum memory available, that is 256MB 256MB 256MB 256MB

overhead memory

Swap file size equals the difference
between allocated and reserved
memory
VM 1 VM 2 VM 3
256 MB 256 MB 256 MB
.vswp .vswp .vswp

VMware vSphere: Install, Configure, Manage – Revision A 10-8

© 2011 VMware Inc. All rights reserved

 Memory Reclamation Techniques

Economize use of physical memory pages

Transparent Page Sharing allows pages with identical contents to
be stored only once
Deallocate memory from one virtual machine for another

Ballooning mechanism, active when memory is scarce, forces virtual
machines to use their own paging areas
Memory compression

Attempts to reclaim some memory performance when memory
contention is high
Page virtual machine memory out to disk

Use of VMkernel swap space is the last resort, performs poorly

VMware vSphere: Install, Configure, Manage – Revision A 10-9

© 2011 VMware Inc. All rights reserved

 Memory Compression

Memory pages are compressed

to 2KB and stored in a per-VM
compression cache.

Memory pages that are 2K
candidates for swap to disk Guest OS
A 4K B
physical memory
are targeted for compression. 2K

Decompressing a
compressed page in memory
is faster than performing disk
I/O operations.

Compression only takes
place when there is
contention for physical
memory resources. = memory compression cache

VMware vSphere: Install, Configure, Manage – Revision A 10-10

© 2011 VMware Inc. All rights reserved

 Virtual SMP

VMware vSphere: Install, Configure, Manage – Revision A 10-11

© 2011 VMware Inc. All rights reserved

 Hyperthreading

Hyperthreading enables a core

to execute two threads, or sets
of instructions, at the same time.
To enable hyperthreading:
1. Verify that system supports
hyperthreading.
2. Enable hyperthreading in the
system BIOS.
3. Ensure that hyperthreading for
the VMware ESX®/ESXi host
is turned on.

VMware vSphere: Install, Configure, Manage – Revision A 10-12

© 2011 VMware Inc. All rights reserved

 CPU Load Balancing

VMware vSphere: Install, Configure, Manage – Revision A 10-13

© 2011 VMware Inc. All rights reserved

 Review of Learner Objectives

You should be able to do the following:

Discuss CPU and memory concepts in a virtualized environment.

Describe what over commitment of a resource means.

Identify additional technologies that improve memory utilization.

Describe how virtual SMP works and how hyperthreading is used by
the VMkernel.

VMware vSphere: Install, Configure, Manage – Revision A 10-14

© 2011 VMware Inc. All rights reserved

 Lesson 2:
Resource Controls

VMware vSphere: Install, Configure, Manage – Revision A 10-15

© 2011 VMware Inc. All rights reserved

 Learner Objectives

After this lesson, you should be able to do the following:

Describe the resources that can be optimized on virtual machines.

Assign share values for CPU, memory, and disk resources.

Establish CPU, memory, and disk reservations and limits.

Describe how virtual machines compete for resources.

VMware vSphere: Install, Configure, Manage – Revision A 10-16

© 2011 VMware Inc. All rights reserved

 Resource Contention

Since virtual machines simultaneously use the resources of a

physical server, they should know how to respond when virtual
machines are competing for resources.
For proper resource management, vSphere has mechanisms to do
the following:
• Enable less, more, or an equal amount of access to a defined resource
• Prevent a virtual machine from consuming large amounts of a resource
• Allow a virtual machine, whose performance is not adequate or requires a
certain amount of a resource to run properly, to have a defined amount of
resource

VMware vSphere: Install, Configure, Manage – Revision A 10-17

© 2011 VMware Inc. All rights reserved

 Shares, Limits, and Reservations

available capacity limit

Shares are used

to compete in
this range.

reservation

0 MHz/MB

A virtual machine will

power on only if its reservation
can be guaranteed.

VMware vSphere: Install, Configure, Manage – Revision A 10-18

© 2011 VMware Inc. All rights reserved

 How Virtual Machines Compete for Resources

Number of shares 1000 1000 1000

VM A VM B VM C

1000 3000 1000

Change number of
shares VM A VM B VM C

1000 3000 1000 1000

Power on virtual
machine VM A VM B VM C VM D

1000 3000 1000

Power off virtual
machine VM A VM B VM D

VMware vSphere: Install, Configure, Manage – Revision A 10-19

© 2011 VMware Inc. All rights reserved

 Systems for Optimizing Virtual Machine Resource Use

Configured by
Adjustable by
Managed by VMkernel virtual machine
administrator
creator
• Hyperthreading • VMware • Limit
CPU vSphere Virtual
• Load balancing • Reservation
cycles Symmetric
• NUMA Multiprocessing • Share allocation
• Transparent page
sharing
• Limit
• vmmemctl • Available
• Reservation
RAM • Memory compression memory
• Share allocation
• VMkernel swap files
for virtual machines
Disk
• Virtual machine
bandwidth • Multipathing
file location

Network • NIC teaming • Traffic shaping

bandwidth

VMware vSphere: Install, Configure, Manage – Revision A 10-20

© 2011 VMware Inc. All rights reserved

 Review of Learner Objectives

You should be able to do the following:

Describe the resources that can be optimized on virtual machines.

Assign share values for CPU, memory, and disk resources.

Establish CPU, memory, and disk reservations and limits.

Describe how virtual machines compete for resources.

VMware vSphere: Install, Configure, Manage – Revision A 10-21

© 2011 VMware Inc. All rights reserved

 Lesson 3:
Resource Pools

VMware vSphere: Install, Configure, Manage – Revision A 10-22

© 2011 VMware Inc. All rights reserved

 Learner Objectives

After this lesson, you should be able to do the following:

Describe resource allocation settings for CPU and memory.

Create a resource pool.

Set resource pool attributes.

Describe expandable reservations.

VMware vSphere: Install, Configure, Manage – Revision A 10-23

© 2011 VMware Inc. All rights reserved

 What Is a Resource Pool?

A resource pool is a
logical abstraction for
hierarchically managing
CPU and memory
resources.
It is used on standalone
hosts or clusters enabled
root
for vSphere Distributed resource
Resource Scheduler resource
pool
(DRS). pools

It provides resources for

virtual machines and child
pools.

VMware vSphere: Install, Configure, Manage – Revision A 10-24

© 2011 VMware Inc. All rights reserved

 Why Use Resource Pools?

Using resource pools can result in these benefits:

Flexible hierarchical organization

Isolation between pools and sharing within pools

Access control and delegation

Separation of resources from hardware

Management of sets of virtual machines running a multitier service

VMware vSphere: Install, Configure, Manage – Revision A 10-25

© 2011 VMware Inc. All rights reserved

 Resource Pool Attributes

Resource pool attributes:

Shares:
• Low, Normal, High, Custom

Reservations, in MHz and MB

Limits (in MHz and MB):
• Unlimited access, by default (up to
maximum amount of resource
accessible)

Expandable reservation?
• Yes – Virtual machines and subpools
can draw from this pool’s parent.
• No – Virtual machines and subpools
can draw only from this pool, even if
its parent has free resources.

VMware vSphere: Install, Configure, Manage – Revision A 10-26

© 2011 VMware Inc. All rights reserved

 Resource Pool Scenario

Company X’s IT department has two internal customers:

The finance department supplies two-thirds of the budget.

The engineering department supplies one-third of the budget.
Each internal customer has both production and test/dev virtual
machines.
We must cap the resource consumption of the test/dev virtual
machines.

VMware vSphere: Install, Configure, Manage – Revision A 10-27

© 2011 VMware Inc. All rights reserved

 Resource Pool Example

VMware vSphere: Install, Configure, Manage – Revision A 10-28

© 2011 VMware Inc. All rights reserved

 Resource Pools Example: CPU Shares

VMware vSphere: Install, Configure, Manage – Revision A 10-29

© 2011 VMware Inc. All rights reserved

 Resource Pools Example: CPU Contention

VMware vSphere: Install, Configure, Manage – Revision A 10-30

© 2011 VMware Inc. All rights reserved

 Expandable Reservation

Root resource pool Borrowing resources occurs

total CPU: 10,200MHz recursively from the ancestors of the
total memory: 3,000MB current resource pool.

Expandable Reservation option
Retail pool must be enabled.
reservation: 3,000MHz
This option offers more flexibility but
expandable reservation: Yes less protection.
eCommerce Apps eCommerce Web Expanded reservations are not
pool pool released until the virtual machine
reservation: reservation:
1,200MHz 1,000MHz
that caused the expansion is shut
expandable? Yes expandable? No
down or its reservation is reduced.

An expandable reservation might allow a rogue

application to claim all unreserved capacity.

VMware vSphere: Install, Configure, Manage – Revision A 10-31

© 2011 VMware Inc. All rights reserved

 Example of Expandable Reservation (1)

Root resource pool

eCommerce resource pools reserve
Total CPU: 10,200MHz
Total memory: 3,000 MB
2,200MHz of the 3,000MHz that the
Retail pool has reserved.
Retail pool
Power on virtual machines in the
reservation: 3,000MHz eCommerce Web pool.
expandable reservation: No
With Expandable Reservation
eCommerce Apps eCommerce Web disabled on the eCommerce Web
pool pool pool, VM3 cannot be started with a
reservation: reservation:
1,200MHz 1,000MHz reservation of 500MHz.
expandable? Yes expandable? No
Lower the virtual machine
reservation.
VM1
R=400
VM2
R=300

Enable Expandable Reservation.
VM3

Increase the eCommerce Web
R=500
pool’s reservation.

VMware vSphere: Install, Configure, Manage – Revision A 10-32

© 2011 VMware Inc. All rights reserved

 Example of Expandable Reservation (2)

Root resource pool

Total CPU: 10,000MHz Enable Expandable Reservation on
Total memory: 3,000MB
**200MHz used by Retail**
the eCommerce Web pool.

Retail pool The system considers the

reservation: 3,000MHz resources available in the child
expandable reservation: Yes resource pool and its direct parent
**full reservation used**
resource pool.
eCommerce Apps eCommerce Web
pool pool
The virtual machine’s reservation is
reservation: reservation: charged against the reservation for
1,200MHz 1,000MHz eCommerce Web.
expandable? Yes expandable? Yes
eCommerce Web’s reservation is
VM4 VM5 VM1 VM2 charged against the reservation for
R=500 R=500 R=400 R=300 Retail.
VM6 VM7 VM3
R=500 R=500 R=500

VMware vSphere: Install, Configure, Manage – Revision A 10-33

© 2011 VMware Inc. All rights reserved

 Admission Control for CPU and Memory Reservations

Create a new subpool Increase a pool’s

Power on a virtual machine.
with its own reservation. reservation.

Succeed Yes Can this pool

satisfy reservation?

No

No Expandable
Fail
reservation?

Yes – Go to parent pool.

VMware vSphere: Install, Configure, Manage – Revision A 10-34

© 2011 VMware Inc. All rights reserved

 Resource Pool Summary Tab

Click the resource pool’s Summary tab in

the Hosts and Clusters inventory view.

VMware vSphere: Install, Configure, Manage – Revision A 10-35

© 2011 VMware Inc. All rights reserved

 Resource Allocation Tab

Click the resource pool’s Resource

Allocation tab.

VMware vSphere: Install, Configure, Manage – Revision A 10-36

© 2011 VMware Inc. All rights reserved

 Scheduling Changes to Resource Settings

Schedule a task to change the

resource settings of a
resource pool or a virtual
machine.

VMware vSphere: Install, Configure, Manage – Revision A 10-37

© 2011 VMware Inc. All rights reserved

 Lab 15

In this lab, you will create and use resource pools on an ESXi host.
1. Create CPU contention.
2. Create a resource pool named Fin-Test.
3. Create a resource pool named Fin-Prod.
4. Verify resource pool functionality.

VMware vSphere: Install, Configure, Manage – Revision A 10-38

© 2011 VMware Inc. All rights reserved

 Review of Learner Objectives

You should be able to do the following:

Describe resource allocation settings for CPU and memory.

Create a resource pool.

Set resource pool attributes.

Describe expandable reservations.

VMware vSphere: Install, Configure, Manage – Revision A 10-39

© 2011 VMware Inc. All rights reserved

 Lesson 4:
Monitoring Resource Usage

VMware vSphere: Install, Configure, Manage – Revision A 10-40

© 2011 VMware Inc. All rights reserved

 Learner Objectives

After this lesson, you should be able to do the following:

Monitor a virtual machine’s resource usage:
• CPU
• Memory
• Disk
• Network bandwidth

VMware vSphere: Install, Configure, Manage – Revision A 10-41

© 2011 VMware Inc. All rights reserved

 Performance-Tuning Methodology

Assess performance.

Use appropriate monitoring
tools.

Record a numerical
benchmark before changes.
Identify the limiting resource. Do not make casual changes
to production systems.
Make more resources
available.

Allocate more.

Reduce competition.

Log your changes.
Benchmark again.

VMware vSphere: Install, Configure, Manage – Revision A 10-42

© 2011 VMware Inc. All rights reserved

 Guest Operating System Monitoring Tools

Iometer
Task Manager

VMware vSphere: Install, Configure, Manage – Revision A 10-43

© 2011 VMware Inc. All rights reserved

 Using Perfmon to Monitor Virtual Machine Resources

The Perfmon DLL in VMware Tools

provides virtual machine processor and
memory objects to access host statistics
inside a virtual machine.

VMware vSphere: Install, Configure, Manage – Revision A 10-44

© 2011 VMware Inc. All rights reserved

 VMware vCenter Server Performance Charts

The Performance tab

displays two kinds of
charts for hosts and
virtual machines:

Overview charts:
• Display the most
common metrics for an
object

Advanced charts:
• Display data counters not
shown in the overview
charts

VMware vSphere: Install, Configure, Manage – Revision A 10-45

© 2011 VMware Inc. All rights reserved

 Interpreting Data from the Tools

VMware vCenter Server™

monitoring tools and guest
operating system monitoring
tools provide different points of
view.

Task Manager in
guest operating system

CPU Usage
chart for host

VMware vSphere: Install, Configure, Manage – Revision A 10-46

© 2011 VMware Inc. All rights reserved

 Is a Virtual Machine CPU-Constrained?

Check the virtual

machine’s CPU
usage.

If CPU usage is continuously

high, the virtual machine is
constrained by CPU.
But the host might have enough
CPU for other virtual machines to
run.

VMware vSphere: Install, Configure, Manage – Revision A 10-47

© 2011 VMware Inc. All rights reserved

 Are Virtual Machines CPU-Constrained?

CPU Ready graph of several virtual machines

Task Manager of several operating systems

Multiple virtual machines are constrained by CPU if:

There is high CPU use in the guest operating system

There are relatively high CPU ready values for the virtual machines

VMware vSphere: Install, Configure, Manage – Revision A 10-48

© 2011 VMware Inc. All rights reserved

 Is a Virtual Machine Memory-Constrained?

Check the virtual machine’s ballooning activity:

If ballooning activity is high, this might not be a problem if all virtual machines
have sufficient memory.

If ballooning activity is high and the guest operating system is swapping, then
the virtual machine is constrained for memory.
VMware vSphere: Install, Configure, Manage – Revision A 10-49

© 2011 VMware Inc. All rights reserved

 Is the Host Memory-Constrained?

If there is active host-level swapping, then host memory is

overcommitted.

VMware vSphere: Install, Configure, Manage – Revision A 10-50

© 2011 VMware Inc. All rights reserved

 Monitoring Active Memory of a Virtual Machine

Monitor for increases in active memory on the host:

Host active memory refers to active physical memory used by virtual
machines and the VMkernel.

If amount of active memory is high, this could lead to virtual machines
that are memory-constrained.

VMware vSphere: Install, Configure, Manage – Revision A 10-51

© 2011 VMware Inc. All rights reserved

 Are Virtual Machines Disk-Constrained?

Disk-intensive applications can saturate the storage or the path.

If you suspect that a virtual machine is constrained by disk access:

Measure the throughput and latency between the virtual machine and
storage.

Use the advanced performance charts to monitor:
• Read rate and write rate
• Read latency and write latency

Inventory object Chart option Storage type

Host Datastore FC, iSCSI, NFS
Host Storage adapter FC
Host Storage path FC, iSCSI
Virtual machine Datastore FC, iSCSI, NFS
Virtual machine Virtual disk FC, iSCSI, NFS

VMware vSphere: Install, Configure, Manage – Revision A 10-52

© 2011 VMware Inc. All rights reserved

 Monitoring Disk Latency

To determine disk performance problems, monitor two disk latency

data counters:

Kernel command latency:
• The average time spent in the VMkernel per SCSI command.
• High numbers (greater than 2–3ms) represent either an overworked array
or an overworked host.

Physical device command latency:
• The average time the physical device takes to complete a SCSI command.
• High numbers (greater than 15–20ms) represent a slow or overworked
array.

VMware vSphere: Install, Configure, Manage – Revision A 10-53

© 2011 VMware Inc. All rights reserved

 Are Virtual Machines Network-Constrained?

Network-intensive applications often bottleneck on path segments

outside the ESX/ESXi host:

Example: WAN links between server and client
If you suspect that a virtual machine is constrained by the network:

Confirm that VMware Tools is installed.
• Enhanced network drivers are available.

Measure the effective bandwidth between the virtual machine and its
peer system.

Check for dropped receive packets and dropped transmit packets.

VMware vSphere: Install, Configure, Manage – Revision A 10-54

© 2011 VMware Inc. All rights reserved

 Lab 16

In this lab, you will see how CPU workload is reflected by system
monitoring tools.
1. Use vCenter Server to monitor CPU utilization.
2. Undo changes made to your virtual machines.

VMware vSphere: Install, Configure, Manage – Revision A 10-55

© 2011 VMware Inc. All rights reserved

 Review of Learner Objectives

You should be able to do the following:

Monitor a virtual machine’s resource usage:
• CPU
• Memory
• Disk
• Network bandwidth

VMware vSphere: Install, Configure, Manage – Revision A 10-56

© 2011 VMware Inc. All rights reserved

 Lesson 5:
Using Alarms

VMware vSphere: Install, Configure, Manage – Revision A 10-57

© 2011 VMware Inc. All rights reserved

 Learner Objectives

After this lesson, you should be able to do the following:

Create alarms with condition-based triggers.

Create alarms with event-based triggers.

View and acknowledge triggered alarms.

VMware vSphere: Install, Configure, Manage – Revision A 10-58

© 2011 VMware Inc. All rights reserved

 What Is an Alarm?

An alarm is a notification that

occurs in response to selected
events or conditions that occur
with an object in the inventory.
Default alarms exist for various
inventory objects: Default
datacenter

Many default alarms for hosts and alarms
virtual machines (partial list)
You can create custom alarms for
a wide range of inventory objects:

Virtual machines, hosts, clusters,
datacenters, datastores,
networks, distributed switches,
and distributed port groups

VMware vSphere: Install, Configure, Manage – Revision A 10-59

© 2011 VMware Inc. All rights reserved

 Alarm Settings

To create an alarm, right-click the

inventory object and select Alarm > Alarm types for:
Add Alarm.
Virtual machines
Hosts
Clusters
Datacenters
Datastores
Networks
Distributed switches
Distributed virtual port groups

VMware vSphere: Install, Configure, Manage – Revision A 10-60

© 2011 VMware Inc. All rights reserved

 Alarm Triggers

An alarm requires a trigger. Types of triggers:

Condition or state trigger – Monitors the current condition or state.
Example:
• A virtual machine’s current snapshot is above 2GB in size.
• A host is using 90 percent of its total memory.
• A datastore has been disconnected from all hosts.

Event – Monitors events. Example:
• The health of a host’s hardware has changed.
• A license has expired in the datacenter.
• A host has left the vNetwork distributed switch.

VMware vSphere: Install, Configure, Manage – Revision A 10-61

© 2011 VMware Inc. All rights reserved

 Configuring Condition Triggers

Condition triggers for a virtual machine

VMware vSphere: Install, Configure, Manage – Revision A 10-62

© 2011 VMware Inc. All rights reserved

 Configuring Event Triggers

Event trigger for a host

VMware vSphere: Install, Configure, Manage – Revision A 10-63

© 2011 VMware Inc. All rights reserved

 Configuring Reporting Options

Use the Reporting pane to avoid needless re-alarms.

Avoid
small
fluctuations.

Avoid
repeats.

VMware vSphere: Install, Configure, Manage – Revision A 10-64

© 2011 VMware Inc. All rights reserved

 Configuring Actions

Every alarm type has these actions:

Send a notification email, send a notification trap, or run a command.
Virtual machine alarms and host alarms have more actions.

VMware vSphere: Install, Configure, Manage – Revision A 10-65

© 2011 VMware Inc. All rights reserved

 Configuring vCenter Server Notifications

In the menu bar, select Administration > vCenter Server Settings.

Select Mail to set

SMTP parameters.

Select SNMP to
specify trap
destinations.

VMware vSphere: Install, Configure, Manage – Revision A 10-66

© 2011 VMware Inc. All rights reserved

 Viewing and Acknowledging Triggered Alarms

The Acknowledge Alarm feature is used to

track when triggered alarms are addressed.

VMware vSphere: Install, Configure, Manage – Revision A 10-67

© 2011 VMware Inc. All rights reserved

 Lab 17

In this lab, you will demonstrate the vCenter Server alarm feature.
1. Create a virtual machine alarm that monitors for a condition.
2. Create a virtual machine alarm that monitors for an event.
3. Trigger virtual machine alarms and acknowledge them.
4. Disable virtual machine alarms.

VMware vSphere: Install, Configure, Manage – Revision A 10-68

© 2011 VMware Inc. All rights reserved

 Review of Learner Objectives

You should be able to do the following:

Create alarms with condition-based triggers.

Create alarms with event-based triggers.

View and acknowledge triggered alarms.

VMware vSphere: Install, Configure, Manage – Revision A 10-69

© 2011 VMware Inc. All rights reserved

 Key Points

The VMkernel has built-in mechanisms (such as CPU load balancing

and transparent page sharing) for managing the CPU and memory
allocation on an ESX/ESXi host.

The Performance tab allows you to monitor a host or virtual
machine’s performance in real time or over a period of time.

Use alarms to monitor your vCenter Server inventory. Alarms notify
you when selected events or conditions have occurred.

VMware vSphere: Install, Configure, Manage – Revision A 10-70

© 2011 VMware Inc. All rights reserved