B CLI VMware VM-FEX UCSM Configuration Guide
B CLI VMware VM-FEX UCSM Configuration Guide
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version
of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks can be found at http://cisco.com/
go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco
and any other company. (1101R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown
for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
Preface vii
Audience vii
Conventions vii
Related Documentation viii
Obtaining Documentation and Submitting a Service Request ix
Introduction 1
Overview of Virtualization 1
Overview of Cisco Virtual Machine Fabric Extender 1
Virtualization with a Virtual Interface Card Adapter 2
Virtualization with Network Interface Cards and Converged Network Adapters 2
VM-FEX for VMware Components and Requirements 2
Hypervisor Host 3
Cisco UCS Manager 4
VMware vCenter 5
Modes of Operation 6
Standard Mode 6
High-Performance Mode 7
Configuring VM-FEX for VMware 7
Configuring a Service Profile with VM-FEX 9
Modifying the VMwarePassThrough Ethernet Adapter Policy 9
Configuring Dynamic vNIC Connection Policies 10
Dynamic vNIC Connection Policy 10
Creating a Dynamic vNIC Connection Policy 11
Deleting a Dynamic vNIC Connection Policy 12
Viewing Dynamic vNIC Properties in a VM 12
Configuring the VM Lifecycle Policy 13
VM Lifecycle Policy 13
Configuring the VM Lifecycle Policy 14
Audience
This guide is intended primarily for data center administrators with responsibilities and expertise in one or
more of the following:
• Server administration
• Storage administration
• Network administration
• Network security
Conventions
This document uses the following conventions:
Convention Indication
bold font Commands, keywords, GUI elements, and user-entered text
appear in bold font.
italic font Document titles, new or emphasized terms, and arguments for
which you supply values are in italic font.
Convention Indication
{x | y | z} Required alternative keywords are grouped in braces and
separated by vertical bars.
courier font Terminal sessions and information that the system displays
appear in courier font.
Tip Means the following information will help you solve a problem.
Caution Means reader be careful. In this situation, you might perform an action that could result in equipment
damage or loss of data.
Timesaver Means the described action saves time. You can save time by performing the action described in the
paragraph.
Warning Means reader be warned. In this situation, you might perform an action that could result in bodily injury.
Related Documentation
The following documentation is available for Cisco Virtual Machine Fabric Extender (VM-FEX) for VMware:
• Roadmap that lists all documentation for Cisco Unified Computing System (Cisco UCS) at the following
URL: http://www.cisco.com/go/unifiedcomputing/b-series-doc
• The Unify Virtual and Physical Networking with Cisco Virtual Interface Card White Paper
• The Cisco Unified Computing System with Cisco VM-FEX and VMware VMDirectPath2 Deployment
Guide
Overview of Virtualization
Virtualization allows the creation of multiple virtual machines (VMs) to run in isolation, side by side on the
same physical machine.
Each virtual machine has its own set of virtual hardware (RAM, CPU, NIC) upon which an operating system
and fully configured applications are loaded. The operating system sees a consistent, normalized set of hardware
regardless of the actual physical hardware components.
In a virtual machine, both hardware and software are encapsulated in a single file for rapid copying,
provisioning, and moving between physical servers. You can move a virtual machine, within seconds, from
one physical server to another for zero-downtime maintenance and continuous workload consolidation.
The virtual hardware makes it possible for many servers, each running in an independent virtual machine, to
run on a single physical server. The advantages of virtualization include better use of computing resources,
greater server density, and seamless server migration.
Component Interconnect Express (PCIe) device and a virtual port on a switch. This solution allows precise
rate limiting and quality of service (QoS) guarantees on the VM interface.
VM-FEX works with a Cisco virtual interface card (VIC) adapter to replace software-based switching on the
server with external hardware-based switching in the fabric interconnect.
The following figure shows these three main components and the methods by which they are connected.
These components must be configured correctly for VM-FEX for VMware to work.
Hypervisor Host
The hypervisor host has these requirements:
• You must install a Cisco UCS M81KR Virtual Interface Card in the server you intend to use as the
hypervisor host. For more information about installing a Cisco UCS M81KR Virtual Interface Card, see
the Cisco UCS 5108 Server Chassis Hardware Installation Guide.
• You must install the correct version of VMware ESX or ESXi software on the Cisco UCS Manager host.
For VM-FEX in standard mode, you must install VMware ESX version 4.0, Update 1 software or later
versions. For VM-FEX in standard mode and high-performance mode, you must install VMware ESXi
5.0 software or later versions. For information about upgrading these VMware software versions, see
Upgrading a Host to ESX or ESXi 4.0, Update 3 or to ESX or ESXi 4.1, Update 1 and the Compatible
Cisco VEM Software Bundle, page 18 or Upgrading a Host to ESXi 5.0 and the Compatible Cisco
VEM Software Bundle, page 21.
• You must install the correct version of the Cisco VEM software bundle on the hypervisor host. The
Cisco VEM software bundle that you install depends on the VMware ESX or ESXi version you have
installed on the hypervisor host. For information about the compatible versions of VMware ESX software
and Cisco VEM software bundles, click the Cisco Virtual Machine Fabric Extender (VM-FEX)
Downloads link on the Cisco UCS Manager launch page or see the Hardware and Software
Interoperability Matrix for B Series Servers.
Note The VEM software bundle is also a component of another product: the Cisco Nexus
1000V switch. Do not be concerned if you see references to this product during the
installation of the VEM bundle. This reference is cosmetic only and does not affect the
outcome of the installation and implementation of VM-FEX.
Important You cannot change an extension key that is being used by a DVS or an external virtualization manager.
If you want to use a custom extension key, we recommend that you create and register the custom key
before you create the DVS in Cisco UCS Manager to avoid any possibility of having to delete and recreate
the associated DVS.
Extension Key
Cisco UCS and VMware vCenter must be connected for management integration and network communication
with the host. To accomplish this connectivity, Cisco UCS provides an extension key that represents the Cisco
UCS identity. The extension key must be registered with the external virtualization manager before the Cisco
UCS instance can be acknowledged and management and network connectivity can be established.
SSL Certificate
Cisco UCS Manager generates a default, self-signed SSL certificate to support communication with a VMware
vCenter. You can also create your own custom certificate to communicate with multiple VMware vCenters.
When you create a custom certificate, Cisco UCS Manager recreates the extension files to include the new
certificate. If you subsequently delete the custom certificate, Cisco UCS Manager recreates the extension files
to include the default, self-signed SSL certificate.
To create a custom certificate, you must obtain and copy an external certificate into Cisco UCS and then create
a certificate for VM-FEX that uses the certificate you copied into Cisco UCS.
Port Profiles
Port profiles contain the properties and settings that you can use to configure virtual interfaces in Cisco UCS
for VM-FEX. The port profiles are created and administered in Cisco UCS Manager. After a port profile is
created, assigned to, and actively used by one or more DVSes, any changes made to the networking properties
of the port profile in Cisco UCS Manager are immediately applied to those DVSes.
In VMware vCenter, a port profile is represented as a port group. Cisco UCS Manager pushes the port profile
names to VMware vCenter, which displays the names as port groups. None of the specific networking properties
or settings in the port profile are visible in VMware vCenter. You must configure at least one port profile
client for a port profile if you want Cisco UCS Manager to push the port profile to VMware vCenter.
VMware vCenter
You need VMware vCenter (vCenter Server and vSphere Client) for VM-FEX for VMware. The VMware
vCenter must meet the following requirements:
• The Windows-based machine that you install VMware vCenter on must have network connectivity to
the Cisco UCS management port and to the uplink Ethernet port(s) being used by the ESX host. The
management port connectivity is used for management plane integration between VMware vCenter and
Cisco UCS Manager; the uplink Ethernet port connectivity is used for communication between VMware
vCenter and the ESX host.
• A VMware vCenter extension key provided by Cisco UCS Manager must be registered with VMware
vCenter before VMware vCenter acknowledges the Cisco UCS instance.
In addition, you must configure VMware vCenter with the following parameters:
• A datacenter.
• A distributed virtual switch (DVS).
• ESX hosts added to the DVS and configured to migrate to pass-through switching PTS/DVS.
• Virtual machines (VMs) required for the VMs on the server.
• (For VMware vMotion) Hosts with common shared storage (datastore) that are properly configured for
vMotion.
• (For VM-FEX in high-performance mode) All guest memory on the VMs must be reserved.
• (For VM-FEX in high-performance mode) The port profiles and VMwarePassThrough Ethernet adapter
policy that you have previously configured in Cisco UCS Manager must be specified.
For information about how to configure these required components in VMware vCenter, see the VMware
product documentation.
Modes of Operation
VM-FEX ports can operate in standard mode or high-performance mode.
Standard Mode
In standard mode, traffic to and from a virtual machine passes through the distributed virtual switch (DVS)
and the hypervisor.
The following figure shows the traffic paths taken by VM traffic on a Cisco UCS server with a VIC adapter
that has VM-FEX ports in standard mode.
High-Performance Mode
In high-performance mode, traffic to and from a virtual machine (VM) bypasses the DVS and hypervisor.
Traffic travels directly between VMs and the virtual interface card (VIC) adapter.
The benefits of high-performance mode are as follows:
• Increases I/O performance and throughput.
• Decreases I/O latency.
• Improves CPU utilization for virualized I/O-intensive applications.
With VMware, high-performance mode also supports vMotion. During vMotion, the hypervisor reconfigures
links in high-performance mode to be in standard mode, transitions the link to the new hypervisor, and then
reconfigures the link to be in high-performance mode. The following figure shows how VM-FEX operates
in high-performance mode with vMotion.
Step 2 Configure the installation of the You must configure the VMware ESX host and install the Cisco
Cisco VEM bundle on the VEM software bundle and a VMware vCenter for VM-FEX. For
hypervisor host. more information, see Installing the Cisco VEM Software Bundle,
page 17 and the VMware documentation.
Step 3 Connect Cisco UCS Manager to You must connect Cisco UCS Manager with VMware vCenter
VMware vCenter. to manage the distributed virtual switch (DVS) in Cisco UCS
Manager. For more information, see Connecting Cisco UCS
Manager to VMware vCenter, page 27.
Step 4 In Cisco UCS Manager, define You must create a distributed virtual switch (DVS) to use in place
a distributed virtual switch. of the VMware vSwitch. For more information, see Configuring
Distributed Virtual Switches in Cisco UCS, page 33.
Step 5 In Cisco UCS Manager, define You must create a port profile to define the properties and settings
a port profile and (optionally) used to configure the virtual interfaces in the DVS. Optionally,
create a port profile client. you can also create a port profile client that defines the DVSes
to which port profiles are assigned. For more information, see
Configuring Port Profiles, page 41.
Procedure
Step 2 UCS-A /org # scope eth-policy Enters organization Ethernet policy mode for the
VMWarePassThru system-provided adapter policy.
The following example shows you how to modify a system-provided Ethernet adapter policy and commit the
transaction:
UCS-A# scope org /
UCS-A /org* # scope eth-policy VMWarePassThru
UCS-A /org/eth-policy* # set descr "This is an Ethernet adapter policy example."
UCS-A /org/eth-policy* # commit-buffer
UCS-A /org/eth-policy #
Note If you migrate a server that is configured with dynamic vNICs using VMotion, the dynamic interface used
by the vNICs fails and Cisco UCS Manager notifies you of that failure.
When the server comes back up, Cisco UCS Manager assigns new dynamic vNICs to the server. If you
are monitoring traffic on the dynamic vNIC, you must reconfigure the monitoring source.
Procedure
Step 2 UCS-A /org # create Creates the specified vNIC connection policy and enters
dynamic-vnic-conn-policy organization vNIC connection policy mode.
policy-name This name can be between 1 and 16 alphanumeric characters.
You cannot use spaces or any special characters other than -
(hyphen), _ (underscore), : (colon), and . (period), and you
cannot change this name after the object has been saved.
Step 4 UCS-A Specifies the Ethernet adapter policy to use for this policy.
/org/dynamic-vnic-conn-policy # The adapter policy must already exist.
set adapter-policy policy-name
Step 5 UCS-A Specifies the number of dynamic vNICs to use for this policy.
/org/dynamic-vnic-conn-policy #
set dynamic-eth
{dynamic-eth-num | off}
Step 6 UCS-A Dynamic vNICs are always protected in Cisco UCS, but this
/org/dynamic-vnic-conn-policy # command allows you to select a preferred fabric, if any. You
set protection {protected | can choose one of the following options:
protected-pref-a |
• protected—Cisco UCS uses whichever fabric is
protected-pref-b}
available.
• protected-pref-a—Cisco UCS attempts to use fabric A,
but fails over to fabric B if necessary.
• protected-pref-b—Cisco UCS attempts to use fabric B,
but fails over to fabric A if necessary.
The following example shows how to create a dynamic vNIC connection policy named MyDynVnicConnPolicy
that uses the system-provided VMWarePassThru Ethernet adapter policy for 12 dynamic vNICs and commit
the transaction:
UCS-A# scope org /
UCS-A /org # create dynamic-vnic-conn-policy MyDynVnicConnPolicy
UCS-A /org/dynamic-vnic-conn-policy* # set adapter-policy VMWarePassThru
UCS-A /org/dynamic-vnic-conn-policy* # set desc "Dynamic vNIC for Eth policy"
UCS-A /org/dynamic-vnic-conn-policy* # set dynamic-eth 12
UCS-A /org/dynamic-vnic-conn-policy* # commit-buffer
UCS-A /org/dynamic-vnic-conn-policy #
Procedure
Step 2 UCS-A /org # delete Deletes the specified vNIC connection policy.
dynamic-vnic-conn-policy policy-name
Step 3 UCS-A /org # commit-buffer Commits the transaction.
The following example shows how to delete the dynamic vNIC connection policy named
MyDynVnicConnPolicy and commit the transaction:
UCS-A# scope org /
UCS-A /org # delete dynamic-vnic-conn-policy MyDynVnicConnPolicy
UCS-A /org* # commit-buffer
UCS-A /org #
Procedure
The following example shows how to display the properties of a dynamic vNIC in a VM:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # show virtual-machine
Virtual Machine:
UUID: 420a00c8-934b-4ae3-6af5-2ce9b8bd0f44
Service Profile: org-root/ls-PTS-ch6-7
Server: sys/chassis-6/blade-7
Status: Online
.
.
.
UCS-A /system/vm-mgmt/vmware # scope virtual-machine 420a08b0-cda7-9e0a-424f-01ec8653eea0
UCS-A /system/vm-mgmt/vmware/virtual-machine # show vnic detail
vNIC:
Name: 4479
Status: Online
MAC Address: 00:50:56:8A:07:B5
Profile Name: VM-mgmt
Virtual Adapter: sys/chassis-1/blade-1/adapter-1/host-eth-9
Vnic Dn: org-root/ls-PTS-ch1-1/ether-dynamic-prot-009
Current Task:
UCS-A /system/vm-mgmt/vmware/virtual-machine #
Note The VM database displayed by Cisco UCS Manager is for information and monitoring only. You cannot
manage VMs through Cisco UCS Manager. If you delete a VM from the Cisco UCS Manager database,
the VM is not deleted from the server or from vCenter.
Step 5 Switch-A Specifies the period of time (minutes, one day, one
/system/vm-mgmt/vm-life-cycle-policy hour, or five days) that an offline dynamic vNIC is
# set vnicretention {minutes | 1-day | retained in the database. If a dynamic vNIC remains
1-hour | 5-days} offline after that period, it is deleted from the database.
The minutes variable can be from 1 to 7200. The
default is 15 minutes.
The following example shows how to configure a one-day VM retention period and a one-hour vNIC retention
period and commit the transaction:
Switch-A# scope system
Switch-A /system # scope vm-mgmt
Switch-A /system/vm-mgmt # scope vm-life-cycle-policy
Switch-A /system/vm-mgmt/vm-mgmt/vm-life-cycle-policy # set vmretention 1-day
Switch-A /system/vm-mgmt/vm-mgmt/vm-life-cycle-policy* # set vnicretention 1-hour
Switch-A /system/vm-mgmt/vm-mgmt/vm-life-cycle-policy* # commit-buffer
Switch-A /system/vm-mgmt/vm-mgmt/vm-life-cycle-policy #
Note Cisco UCS Manager pushes BIOS configuration changes through a BIOS policy or default BIOS settings
to the Cisco Integrated Management Controller (CIMC) buffer. These changes remain in the buffer and
do not take effect until the server is rebooted.
We recommend that you verify the support for BIOS settings in the server that you want to configure.
Some settings, such as Mirroring Mode and Sparing Mode for RAS Memory, are not supported by all
Cisco UCS servers.
We recommend that you name this BIOS policy as VMwarePassThru so that you can identify it as being used
for VM-FEX in high-performance mode.
You must enable these specific parameters in the following BIOS settings:
• Processor—Enable Virtual Technology (VT) and Direct Cache Access.
Note You must enable VT if you intend to run 64-bit VMs on the ESX/ESXi host. An
ESX/ESXi host will not run 64-bit VMs unless VT is enabled.
Upgrading a Host to ESX or ESXi 4.0, Update 3 or to ESX or ESXi 4.1, Update 1 and the Compatible
Cisco VEM Software Bundle, page 18 or Upgrading a Host to ESXi 5.0 and the Compatible Cisco
VEM Software Bundle, page 21.
• You must install the correct version of the Cisco VEM software bundle on the hypervisor host. The
Cisco VEM software bundle that you install depends on the VMware ESX or ESXi version you have
installed on the hypervisor host. For information about the compatible versions of VMware ESX software
and Cisco VEM software bundles, click the Cisco Virtual Machine Fabric Extender (VM-FEX)
Downloads link on the Cisco UCS Manager launch page or see the Hardware and Software
Interoperability Matrix for B Series Servers.
Note The VEM software bundle is also a component of another product: the Cisco Nexus
1000V switch. Do not be concerned if you see references to this product during the
installation of the VEM bundle. This reference is cosmetic only and does not affect the
outcome of the installation and implementation of VM-FEX.
• You must obtain the Cisco VEM software bundle. To download the bundle, click the Cisco Virtual
Machine Fabric Extender (VM-FEX) Downloads link on the Cisco UCS Manager launch page.
• You must be aware of any installation caveats. For information, see the Release Notes for Cisco UCS
Manager.
For information about configuring VUM, see the VMware product documentation.
Note You can also perform this procedure using VMware VUM. For more information, see Installing or
Upgrading the Cisco VEM Software Bundle Using VUM, page 18 and the VMware documentation.
Procedure
Step 1 Power off the VMs or migrate all VMs to another host.
Step 2 Migrate the vmk0 management interface to the vSwitch.
Step 3 Remove the host from pass-through switching (PTS) mode.
Step 4 If you are upgrading from ESX or ESXi 4.0, Update 1 to ESX or ESXi 4.0, Update 3, uninstall the VIB that
enables pass-through switching (PTS). Otherwise, continue to step 6.
a) Enter the following command to identify the Cisco VEM VIB:
esxupdate query --vib-view | grep cross_cisco | grep installed
The output appears showing the VIB package to be removed.
b) Enter the following command to remove the VIB.
esxupdate -b VIB_package remove
Example:
[root@serialport -]# esxupdate query --vib-view | grep cross_cisco | grep installed
[root@serialport -]# cross_cisco-vem-v100-esx_4.0.4.1.1.31-1.11.11 installed
2010-03-11T10:02:29.076573-05:00
[root@serialport -]# esxupdate -b cross_cisco-vem-v100-esx_4.0.4.1.1.31-1.11.11 remove
Step 5 If you are upgrading a host from ESX or ESXi 4.0, Update 1 to ESX or ESXi 4.0, Update 3, copy the ESX
or ESXi Patch 04 (ESX400-201001001.zip) and the ESX or ESXi Patch 05 (ESX400-201003001.zip) to the
/tmp directory and install them, if you have not already installed them. Otherwise, continue to step 6.
vihostupdate --install --bundle "[path to VMware Update offline patch]" --server [vsphere host IP address]"
Example:
[root@serialport -]# vihostupdate --install --bundle "/tmp/Patch04/ESX400-201002001.zip"
--server 10.10.10.1
[root@serialport -]# vihostupdate --install --bundle
"/tmp/Patch05/BuiltByVMware/ESX/20100330/ESX400-201003001.zip" --server 10.10.10.1
Step 6 Go to the directory where the ESX or ESXi host software and Cisco VEM software bundle were copied.
Example:
[root@serialport -]# cd tmp
[root@serialport tmp]#
Step 7 Put the host in maintenance mode.
Step 8 Enter the vihostupdate command:
vihostupdate --install --bundle "[path to VMware Update offline bundle]", "[path to Cisco updated VEM
offline bundle] --server [vsphere host IP address]"
Example:
This example shows how to upgrade to ESX 4.0, Update 3 using the vihostupdate command from a client:
Example:
This example shows how to upgrade to ESX 4.1, Update 1 using the vihostupdate command from a client:
Example:
This example shows how to verify that the correct Cisco VEM software bundle was installed:
This example shows how to verify that ESX 4.0, Update 3 was installed:
Procedure
Step 1 Upgrade the Cisco UCS to release 1.4 and the ESX or ESXi host to one of the following releases:
• ◦ Release 4.0, Update 3 with Cisco VEM software bundle v130-4.2.1.1.4.0.0-1.20.1.vib (or
v129-4.2.1.1.4.0.0-1.20.1.vib if VUM was used)
◦ Release 4.1, Update 1 with Cisco VEM software bundle v130-4.2.1.1.4.0.0-2.0.1.vib (or
v129-4.2.1.1.4.0.0-2.0.1.vib if VUM was used)
For more information, see Upgrading a Host to ESX or ESXi 4.0, Update 3 or to ESX or ESXi 4.1, Update
1 and the Compatible Cisco VEM Software Bundle, page 18.
Step 3 Because VM-FEX for VMware has Cisco VEMs that require custom .vib files, create a custom ESX installable
ISO file that includes the Cisco VEM software bundle. This custom ESX installable ISO file is installed as
part of the ESXi 5.0 host upgrade.
For more information, see the "Upgrading Hosts That Have Third-Party Custom VIBs" section in the VMware
vSphere Upgrade document for vSphere 5.0.
Step 4 Upgrade the host to ESXi 5.0 and install the custom ESX installable ISO file.
For more information, see the VMware vSphere Upgrade document for vSphere 5.0.
Note You can also upgrade the host to ESXi 5.0 using VUM. For more information, see Installing or
Upgrading the Cisco VEM Software Bundle Using VUM, page 18 and the VMware documentation.
Note You can also perform this procedure using VMware VUM. For more information, see Installing or
Upgrading the Cisco VEM Software Bundle Using VUM, page 18 and the VMware documentation.
Procedure
Step 1 Go to the directory where the ESX or ESXi host software and Cisco VEM software bundle were copied.
Example:
[root@serialport -]# cd tmp
[root@serialport tmp]#
Step 2 If the host is at ESXi Release 5, proceed to step 3. If the host is at ESX or ESXi Release 4.0 or 4.1, do one of
the following :
• If you are performing this installation or upgrade remotely from a client, put the host in maintenance
mode and enter the vihostupdate command:
vihostupdate --install --bundle [path to desired offline Cisco VEM software bundle] --server [vsphere
host IP address]
Example:
The following examples shows how to install or upgrade a Cisco VEM software bundle remotely.
• If you are performing this installation or upgrade locally on the ESX or ESXi host, log into the each
host separately and, from the /tmp directory, use the esxupdate command to install the VEM software.
This command loads the software manually onto the host, loads the kernel modules, and starts the VEM
Agent on the running system.
esxupdate --bundle [VMware offline update bundle] update
Example:
The following example shows how to install or upgrade a Cisco VEM software bundle locally on a host.
Unpacking cross_cisco-vem-v132-esx_4.2.1.1.4.1.0-1.20.4.vib
############################################################################################
[100%]
Running [/usr/sbin/vmkmod-install.sh]...
ok.
Example:
The following example shows how to perform this installation or upgrade remotely from a client.
• If you are performing this installation or upgrade locally on a host, log into the each host separately and
from the /tmp directory, enter the following command:
esxcli software vib install -v /vib-file
Example:
The following example shows how to perform this installation or upgrade locally on a host.
Example:
The following example shows how to verify that the installation or upgrade of the Cisco VEM software bundle
was successful on an ESX 4.1 host:
• You are logged in to the ESX or ESXi host remotely by using secure shell (SSH).
Procedure
Step 1 If the host is at ESXi release 5.0, proceed to step 2. If the host is at ESX or ESXi release 4.0 or 4.1, uninstall
the Cisco VEM software bundle by using the esxupdate remove -b VEM_name command.
Example:
The following example shows how to remove the Cisco VEM software bundle from a host that is at ESX or
ESXi release 4.1:
Running [/usr/sbin/vmkmod-install.sh]...
ok.
[root@serialport ~]# esxupdate query --v |grep cisco
cross_cisco-vem-v132-esx_4.2.1.1.4.1.0-2.0.4 uninstalled
2011-09-05T09:04:51.096141-07:00
[root@serialport ~]#
Step 2 For an ESXi 5.0 host, uninstall the Cisco VEM software bundle by using the esxcli software vib remove
-n --maintenance-mode [Cisco VEM bundle software name] command.
Example:
The following example shows how to remove the Cisco VEM software bundle from a host that is at ESXi
release 5.0:
Example:
The following example shows how to verify that the Cisco VEM software bundle was successfully uninstalled
from an ESX or ESXi release 4.1 host:
Example:
~ # esxcli software vib get |grep cisco
~ #
Step 5 Reboot the system.
Procedure
What to Do Next
Export the vCenter extension file or files from Cisco UCS Manager.
Procedure
What to Do Next
Register the vCenter extension file or files in VMware vCenter.
Procedure
Step 2 UCS-A(local-mgmt)# copy Copies the certificate from its source location to its destination
from-filesystem: [ from-path ] location. For the from-filesystem: argument, use one of the
filename to-filesystem: [ to-path ] following syntax:
filename
• ftp:// username@server-ip-addr
• scp:// username@server-ip-addr
• sftp:// username@server-ip-addr
• tftp:// server-ip-addr : port-num
The following shows how to use FTP to copy a certificate (certificate.txt) to the temp folder in the workspace:
UCS-A # connect local-mgmt
Cisco UCS 6100 Series Fabric Interconnect
workspace:/temp/certificate.txt
UCS-A(local-mgmt)#
What to Do Next
Create a certificate for VM-FEX.
Procedure
Procedure
Step 3 UCS-A /system/vm-mgmt # scope vmware Enters system VM management VMware mode.
Step 5 UCS-A /system/vm-mgmt /vmware/cert-store Creates the specified certificate for VM-FEX
# create certificate certificate-name for VMware and enters system VM
management VMware certificate store
certificate mode.
The following example shows how to create a certificate for VM-FEX for VMware and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope cert-store
UCS-A /system/vm-mgmt/vmware/cert-store # create certificate VnLinkCertificate
UCS-A /system/vm-mgmt/vmware/cert-store/certificate* # set location workspace path /temp
certfile certificate.txt
UCS-A /system/vm-mgmt/vmware/cert-store/certificate* # commit-buffer
UCS-A /system/vm-mgmt/vmware/cert-store/certificate #
Step 5 UCS-A /system/vm-mgmt /vmware/cert-store Deletes the specified certificate for VM-FEX
# delete certificate certificate-name for VMware.
The following example shows how to delete a certificate for VM-FEX for VMware and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope cert-store
UCS-A /system/vm-mgmt/vmware/cert-store # delete certificate VnLinkCertificate
UCS-A /system/vm-mgmt/vmware/cert-store* # commit-buffer
UCS-A /system/vm-mgmt/vmware/cert-store #
Procedure
Step 9 UCS-A /system/vm-mgmt Creates the specified datacenter folder and enters system
/vmware/vcenter/data-center # create VM management VMware vCenter datacenter folder
folder folder-name mode.
The following example shows how to create a vCenter, a datacenter with the exact same name as the datacenter
in VMware vCenter, a DVS in the datacenter folder named Engineering, and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # create vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter* # set hostname 192.168.10.10
UCS-A /system/vm-mgmt/vmware/vcenter* # set description "vCenter running on my laptop"
UCS-A /system/vm-mgmt/vmware/vcenter* # create data-center MyDatacenter
UCS-A /system/vm-mgmt/vmware/vcenter/data-center* # create folder Engineering
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder* # create distributed-virtual-switch
LabSwitch
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder/distributed-virtual-switch* # enable
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder/distributed-virtual-switch* #
commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder/distributed-virtual-switch #
The following example shows how to add a vCenter folder named Lab 5 to the vCenter named MyVcenter
and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter* # create folder Lab5
UCS-A /system/vm-mgmt/vmware/vcenter/folder* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter/folder #
Procedure
Step 3 UCS-A /system/vm-mgmt # scope vmware Enters system VM management VMware mode.
The following example shows how to add a datacenter named SQA-Datacenter to the vCenter named MyVcenter
and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter* # create data-center SQA-Datacenter
UCS-A /system/vm-mgmt/vmware/vcenter/data-center* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter/data-center #
Step 3 UCS-A /system/vm-mgmt # scope vmware Enters system VM management VMware mode.
The following example shows how to add a datacenter folder named SoftwareQA to the datacenter named
MyDatacenter and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter* # scope data-center MyDatacenter
UCS-A /system/vm-mgmt/vmware/vcenter/data-center* # create folder SoftwareQA
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter/data-center/folder #
The following example shows how to delete the vCenter folder named Lab5 from the vCenter named MyVcenter
and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter # delete folder Lab5
UCS-A /system/vm-mgmt/vmware/vcenter* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter #
Step 3 UCS-A /system/vm-mgmt # scope vmware Enters system VM management VMware mode.
The following example shows how to delete the datacenter named SQA-Datacenter from the vCenter named
MyVcenter and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter* # delete data-center SQA-Datacenter
UCS-A /system/vm-mgmt/vmware/vcenter* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter #
The following example shows how to delete the datacenter folder named SoftwareQA from the datacenter
named MyDatacenter and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope vcenter MyVcenter
UCS-A /system/vm-mgmt/vmware/vcenter # scope data-center MyDatacenter
UCS-A /system/vm-mgmt/vmware/vcenter/data-center # delete folder SoftwareQA
UCS-A /system/vm-mgmt/vmware/vcenter/data-center* # commit-buffer
UCS-A /system/vm-mgmt/vmware/vcenter/data-center #
Port Profiles
Port profiles contain the properties and settings that you can use to configure virtual interfaces in Cisco UCS
for VM-FEX. The port profiles are created and administered in Cisco UCS Manager. After a port profile is
created, assigned to, and actively used by one or more DVSes, any changes made to the networking properties
of the port profile in Cisco UCS Manager are immediately applied to those DVSes.
In VMware vCenter, a port profile is represented as a port group. Cisco UCS Manager pushes the port profile
names to VMware vCenter, which displays the names as port groups. None of the specific networking properties
or settings in the port profile are visible in VMware vCenter. You must configure at least one port profile
client for a port profile if you want Cisco UCS Manager to push the port profile to VMware vCenter.
Step 5 UCS-A /system/vm-mgmt/vmware/profile-set # Creates the specified port profile and enters
create port-profile profile-name system VM management VMware profile set
port profile mode.
The following example shows how to create and configure a port profile named MyProfile and commit the
transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope profile-set
UCS-A /system/vm-mgmt/vmware/profile-set # create port-profile MyProfile
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set descr "This is my port profile"
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set host-nwio-perf high-performance
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set max-ports 24
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set nw-control-policy ncp5
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set pin-group PinGroup54
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # set qos-policy QosPolicy34
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # commit-buffer
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile #
What to Do Next
Add a port profile client to the port profile.
Procedure
The following example shows how to delete the port profile named MyProfile and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope profile-set
UCS-A /system/vm-mgmt/vmware/profile-set # delete port-profile MyProfile
UCS-A /system/vm-mgmt/vmware/profile-set* # commit-buffer
UCS-A /system/vm-mgmt/vmware/profile-set #
The following example shows how to add the VLAN named accounting to the port profile named MyProfile,
set the VLAN as non-native, and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware# scope profile-set
The following example shows how to delete the VLAN named accounting from the port profile named
MyProfile and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware# scope profile-set
UCS-A /system/vm-mgmt/vmware/profile-set # scope port-profile MyProfile
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile # delete vlan accounting
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # commit-buffer
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile #
The following example shows how to create a port profile client named OtherClient that applies the port
profile named MyProfile to all DVSes in the datacenter named OtherDatacenter and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope profile-set
UCS-A /system/vm-mgmt/vmware/profile-set # scope port-profile MyProfile
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile* # create client MyClient
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile/client* # set descr "This is my other
client for my port profile"
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile/client* # set data-center
OtherDatacenter
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile/client* # commit-buffer
UCS-A /system/vm-mgmt/vmware/profile-set/port-profile/client #
Procedure
The following example shows how to delete the port profile client named OtherClient from the port profile
named MyProfile and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
If the DVS cannot be successfully deleted from VMware vCenter, the DVS remains in the pending deletion
list until the pending deletion is deleted in Cisco UCS Manager or the properties for that pending deletion are
changed in a way that allows the DVS to be successfully deleted from VMware vCenter. When you delete a
pending deletion, the DVS is deleted from Cisco UCS Manager but is not deleted from VMware vCenter. If
the DVS remains in VMware vCenter, you must delete the DVS manually.
You can view the pending deletion list, delete a pending deletion, or change the properties for a pending
deletion in Cisco UCS Manager. For example, you can correct the VMware vCenter IP address for a pending
deletion so that Cisco UCS Manager can successfully initiate a connection and delete the DVS from VMware
vCenter. You cannot cancel the deletion of a DVS from Cisco UCS Manager.
Step 4 UCS-A /system/vm-mgmt /vmware # show Displays the list of pending deletions.
pending-deletion
The following example shows how to display the list of pending deletions:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # show pending-deletion
Pending Deletion:
Id Host Distributed Virtual Switch
---------- -------------------- --------------------------
1169232 192.168.10.10 LabDVS
1176508 192.168.100.20 OpsDVS
1176508 192.168.1.30 MyDVS
1176508 192.168.1.40 OtherDVS
The following example show how to display the properties for a specific pending deletion by using its assigned
number, in this case, number 1169232:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope pending-deletion 1169232
UCS-A /system/vm-mgmt/vmware/pending-deletion # show detail
Pending Deletion:
Id: 1169232
vCenter: vCenterLab
Host: 192.168.10.10
Data Center Folder:
Data center: Lab
Folder: LabFolder
Distributed Virtual Switch: LabDVS
Extension key: Cisco-UCSM-b32cc112-83bb-11de-acc_7
Certificate:
Current Task: external VM manager deletion from local fabric
(FSM-STAGE:sam:dme:ExtvmmSwitchDelTaskRemoveProvider:RemoveLocal)
Procedure
Step 5 UCS-A /system/vm-mgmt Changes the specified property for the pending
/vmware/pending-deletion # set {certificate deletion.
certificate-name | data-center data-center-name Note If you use a hostname rather than an
|data-center-folder folder-name | folder IP address, you must configure a DNS
folder-name | host {hostname | ip-addr}} server in Cisco UCS Manager.
Step 6 UCS-A /system/vm-mgmt Commits the transaction to the system
/vmware/pending-deletion # commit-buffer configuration.
The following example shows how to change the host IP address to 192.168.10.20 for a specific pending
deletion, in this case, number 1169232 and commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # scope pending-deletion 1169232
UCS-A /system/vm-mgmt/vmware/pending-deletion # set host 192.168.10.20
Procedure
Step 4 UCS-A /system/vm-mgmt /vmware # delete Deletes the specified pending deletion.
pending-deletion deletion-id
Step 5 UCS-A /system/vm-mgmt /vmware # Commits the transaction to the system
commit-buffer configuration.
The following example shows how to delete a specific pending deletion, in this case, number 1169232 and
commit the transaction:
UCS-A# scope system
UCS-A /system # scope vm-mgmt
UCS-A /system/vm-mgmt # scope vmware
UCS-A /system/vm-mgmt/vmware # delete pending-deletion 1169232
UCS-A /system/vm-mgmt/vmware* # commit-buffer
UCS-A /system/vm-mgmt/vmware #
A dynamic vNIC
viewing properties 12
adapters dynamic vNIC connection policies
NIC 2 deleting 12
VIC 2 dynamic vNIC connection policy
virtualization 2 about 10
creating 11
B
BIOS
E
creating policy 14 ESX 5.0
upgrade procedure 21
ESX software 3
C ESX/ESXi hosts
upgrading with Cisco VEM software bundle using CLI 18
certificate Ethernet adapter policies
VM-FEX 29 modifying 9
VM-FEX for VMware 30, 31 exporting
Cisco UCS extension files 28
Cisco M81KR VIC 3 extension files
Cisco UCS Manager 2, 4 exporting 28
components 4 modifying key 27
Cisco VEM software bundle extension key 2, 4
uninstalling 24 definition 4
Cisco VM-FEX 1 external virtualization manager 2
clients, port profiles
deleting 47
components 2
converged network adapters
H
virtualization 2 high-performance mode 7
hosts
upgrading to ESXi 5.0 with Cisco VEM software bundle 21
D hypervisor host 2, 5
VMware ESX host
deletion tasks definition 5
about 49
distributed virtual switches 4, 33
definition 4
DVS 4, 34
I
definition 4 IEEE 802.1Qbh 1
M software (continued)
virtual interface cards
modes VEM bundle installation on 17
high-performance 7 standard mode 6
standard 6 switches
modifying extension key 27 distributed virtual switches 33
DVSes 33
N
NIC adapters
T
virtualization 2 traffic
standard mode 6
P
Palo adapter
V
extension files VEM
exporting 28 uninstalling 24
modifying key 27 VEM bundle 3
pending deletions VEM software bundle
about 49 prerequisites 17
policies upgrading on ESX/ESXi hosts using CLI 18
BIOS 14 upgrading with ESXi 5.0 21
dynamic vNIC connection VEM software bundle installation
about 10 using VUM 18
creating 11 VEM software installation
VM lifecycle 13, 14 introduction 17
VM-FEX for VMware 4 VIC adapters
port profile virtualization 2
definition 4 virtual machines 1
port profile client virtualization 1, 2, 6, 13, 14, 27, 28, 29, 30, 31, 34, 49
definition 4 about 1
port profiles components
about 41 Cisco UCS Manager 2
clients 41, 47 extension key 2
deleting 47 external virtualization manager 2
creating 42 hypervisor host 2
deleting 43 converged network adapters 2
Port profiles DVS 34
VLANs NIC adapters 2
adding 44 Palo adapter
deleting 45 extension file 28
profiles extension key 27
port 41 VIC adapter 2
VM lifecycle policy 13, 14
VM-FEX 1, 29, 34
S about 1
copying certificate 29
servers VM-FEX for VMware
BIOS policy 14 creating a certificate 30
software deleting certificate 31
VEM bundle installation 17