10 Vsphere Networking PDF
10 Vsphere Networking PDF
10 Vsphere Networking PDF
Update 2
11 APR 2018
VMware vSphere 6.7
VMware ESXi 6.7
vCenter Server 6.7
vSphere Networking
You can find the most up-to-date technical documentation on the VMware website at:
https://docs.vmware.com/
If you have comments about this documentation, submit your feedback to
[email protected]
VMware, Inc.
3401 Hillview Ave.
Palo Alto, CA 94304
www.vmware.com
Copyright © 2009–2019 VMware, Inc. All rights reserved. Copyright and trademark information.
VMware, Inc. 2
Contents
VMware, Inc. 3
vSphere Networking
VMware, Inc. 4
vSphere Networking
8 Networking Policies 91
Applying Networking Policies on a vSphere Standard or Distributed Switch 92
Configure Overriding Networking Policies on Port Level 93
Teaming and Failover Policy 94
Load Balancing Algorithms Available for Virtual Switches 96
Configure NIC Teaming, Failover, and Load Balancing on a vSphere Standard Switch or
Standard Port Group 100
Configure NIC Teaming, Failover, and Load Balancing on a Distributed Port Group or
Distributed Port 102
VLAN Policy 104
Configure VLAN Tagging on a Distributed Port Group or Distributed Port 105
Configure VLAN Tagging on an Uplink Port Group or Uplink Port 106
Security Policy 106
Configure the Security Policy for a vSphere Standard Switch or Standard Port Group 107
Configure the Security Policy for a Distributed Port Group or Distributed Port 108
Traffic Shaping Policy 109
Configure Traffic Shaping for a vSphere Standard Switch or Standard Port Group 110
Edit the Traffic Shaping Policy on a Distributed Port Group or Distributed Port 111
Resource Allocation Policy 112
Edit the Resource Allocation Policy on a Distributed Port Group 113
Monitoring Policy 113
Enable or Disable NetFlow Monitoring on a Distributed Port Group or Distributed Port 113
Traffic Filtering and Marking Policy 114
Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group 114
Traffic Filtering and Marking on a Distributed Port or Uplink Port 122
VMware, Inc. 5
vSphere Networking
VMware, Inc. 6
vSphere Networking
VMware, Inc. 7
vSphere Networking
VMware, Inc. 8
vSphere Networking
VMware, Inc. 9
vSphere Networking
Virtual Machines on the Same Distributed Port Group and on Different Hosts Cannot Communicate
with Each Other 268
Attempt to Power On a Migrated vApp Fails Because the Associated Protocol Profile Is Missing 269
Networking Configuration Operation Is Rolled Back and a Host Is Disconnected from
vCenter Server 270
VMware, Inc. 10
About vSphere Networking
®
vSphere Networking provides information about configuring networking for VMware vSphere , including
how to create vSphere distributed switches and vSphere standard switches.
vSphere Networking also provides information on monitoring networks, managing network resources, and
networking best practices.
Intended Audience
The information presented is written for experienced Windows or Linux system administrators who are
familiar with network configuration and virtual machine technology.
Tasks for which the workflow differs significantly between the vSphere Client and the vSphere Web Client
have duplicate procedures that provide steps according to the respective client interface. The procedures
that relate to the vSphere Web Client, contain vSphere Web Client in the title.
Note In vSphere 6.7 Update 1, almost all of the vSphere Web Client functionality is implemented in the
vSphere Client. For an up-to-date list of any remaining unsupported functionality, see Functionality
Updates for the vSphere Client.
VMware, Inc. 11
Introduction to vSphere
Networking 1
Get to know the basic concepts of vSphere networking and how to set up and configure a network in a
vSphere environment.
Physical Network A network of physical machines that are connected so that they can send
data to and receive data from each other. VMware ESXi runs on a physical
machine.
Virtual Network A network of virtual machines running on a physical machine that are
connected logically to each other so that they can send data to and receive
data from each other. Virtual machines can be connected to the virtual
networks that you create when you add a network.
Opaque Network An opaque network is a network created and managed by a separate entity
outside of vSphere. For example, logical networks that are created and
®
managed by VMware NSX appear in vCenter Server as opaque networks
of the type nsx.LogicalSwitch. You can choose an opaque network as the
backing for a VM network adapter. To manage an opaque network, use the
management tools associated with the opaque network, such as VMware
®
NSX Manager or the VMware NSX API management tools.
Physical Ethernet A physical ethernet switch manages network traffic between machines on
Switch the physical network. A switch has multiple ports, each of which can be
connected to a single machine or another switch on the network. Each port
can be configured to behave in certain ways depending on the needs of the
VMware, Inc. 12
vSphere Networking
machine connected to it. The switch learns which hosts are connected to
which of its ports and uses that information to forward traffic to the correct
physical machines. Switches are the core of a physical network. Multiple
switches can be connected together to form larger networks.
vSphere Standard It works much like a physical Ethernet switch. It detects which virtual
Switch machines are logically connected to each of its virtual ports and uses that
information to forward traffic to the correct virtual machines. A vSphere
standard switch can be connected to physical switches by using physical
Ethernet adapters, also referred to as uplink adapters, to join virtual
networks with physical networks. This type of connection is similar to
connecting physical switches together to create a larger network. Even
though a vSphere standard switch works much like a physical switch, it
does not have some of the advanced functionality of a physical switch.
Standard Port Group Network services connect to standard switches through port groups. Port
groups define how a connection is made through the switch to the network.
Typically, a single standard switch is associated with one or more port
groups. A port group specifies port configuration options such as bandwidth
limitations and VLAN tagging policies for each member port.
vSphere Distributed A vSphere distributed switch acts as a single switch across all associated
Switch hosts in a data center to provide centralized provisioning, administration,
and monitoring of virtual networks. You configure a vSphere distributed
switch on the vCenter Server system and the configuration is propagated to
all hosts that are associated with the switch. This lets virtual machines
maintain consistent network configuration as they migrate across multiple
hosts.
Host Proxy Switch A hidden standard switch that resides on every host that is associated with
a vSphere distributed switch. The host proxy switch replicates the
networking configuration set on the vSphere distributed switch to the
particular host.
Distributed Port A port on a vSphere distributed switch that connects to a host’s VMkernel
or to a virtual machine’s network adapter.
Distributed Port Group A port group associated with a vSphere distributed switch that specifies
port configuration options for each member port. Distributed port groups
define how a connection is made through the vSphere distributed switch to
the network.
NIC Teaming NIC teaming occurs when multiple uplink adapters are associated with a
single switch to form a team. A team can either share the load of traffic
between physical and virtual networks among some or all of its members,
or provide passive failover in the event of a hardware failure or a network
outage.
VMware, Inc. 13
vSphere Networking
VMkernel TCP/IP The VMkernel networking layer provides connectivity to hosts and handles
Networking Layer the standard infrastructure traffic of vSphere vMotion, IP storage, Fault
Tolerance, and vSAN.
IP Storage Any form of storage that uses TCP/IP network communication as its
foundation. iSCSI and NFS can be used as virtual machine datastores and
for direct mounting of .ISO files, which are presented as CD-ROMs to
virtual machines.
TCP Segmentation TCP Segmentation Offload, TSO, allows a TCP/IP stack to emit large
Offload frames (up to 64KB) even though the maximum transmission unit (MTU) of
the interface is smaller. The network adapter then separates the large
frame into MTU-sized frames and prepends an adjusted copy of the initial
TCP/IP headers.
n Connecting VMkernel services (such as NFS, iSCSI, or vMotion) to the physical network.
The ESXi Dump Collector in ESXi supports both vSphere Standard and Distributed Switches. The ESXi
Dump Collector can also use any active uplink adapter from the team of the port group that handles the
VMkernel adapter for the collector.
Changes to the IP address for the ESXi Dump Collector interface are automatically updated if the IP
addresses for the configured VMkernel adapter changes. The ESXi Dump Collector also adjusts its
default gateway if the gateway configuration of the VMkernel adapter changes.
If you try to delete the VMkernel network adapter used by the ESXi Dump Collector, the operation fails
and a warning message appears. To delete the VMkernel network adapter, disable dump collection and
delete the adapter.
VMware, Inc. 14
vSphere Networking
There is no authentication or encryption in the file transfer session from a crashed host to the ESXi Dump
Collector. You should configure the ESXi Dump Collector on a separate VLAN when possible to isolate
the ESXi core dump from regular network traffic.
For information about installing and configuring the ESXi Dump Collector, see the vCenter Server
Installation and Setup documentation.
VMware, Inc. 15
Setting Up Networking with
vSphere Standard Switches 2
vSphere standard switches handle network traffic at the host level in a vSphere deployment.
VMware, Inc. 16
vSphere Networking
Port
Test groups Test
Management vMotion environment Production Production environment vMotion Management
Physical Switch
A vSphere Standard Switch is very similar to a physical Ethernet switch. Virtual machine network
adapters and physical NICs on the host use the logical ports on the switch as each adapter uses one
port. Each logical port on the standard switch is a member of a single port group. For information about
maximum allowed ports and port groups, see the Configuration Maximums documentation.
For example, you can create Production and Test environment port groups as virtual machine networks
on the hosts that share the same broadcast domain on the physical network.
A VLAN ID, which restricts port group traffic to a logical Ethernet segment within the physical network, is
optional. For port groups to receive the traffic that the same host sees, but from more than one VLAN, the
VLAN ID must be set to VGT (VLAN 4095).
VMware, Inc. 17
vSphere Networking
Procedure
4 Select a connection type for which you want to use the new standard switch and click Next.
Option Description
VMkernel Network Adapter Create a new VMkernel adapter to handle host management traffic, vMotion,
network storage, fault tolerance, or vSAN traffic.
Physical Network Adapter Add physical network adapters to an existing or a new standard switch.
Virtual Machine Port Group for a Create a new port group for virtual machine networking.
Standard Switch
c From the Failover order group drop-down menu, select from the Active or Standby failover lists.
For higher throughput and to provide redundancy, configure at least two physical network
adapters in the Active list.
d Click OK.
VMware, Inc. 18
vSphere Networking
7 If you create the new standard switch with a VMkernel adapter or virtual machine port group, enter
connection settings for the adapter or the port group.
Option Description
VMkernel adapter a Enter a label that indicates the traffic type for the VMkernel adapter, for
example vMotion.
b Set a VLAN ID to identify the VLAN that the network traffic of the VMkernel
adapter will use.
c Select IPv4, Ipv6 or both.
d Select a TCP/IP stack. After you set a TCP/IP stack for the VMkernel adapter,
you cannot change it later. If you select the vMotion or the Provisioning
TCP/IP stack, you will be able to use only this stack to handle vMotion or
Provisioning traffic on the host.
e If you use the default TCP/IP stack, select from the available services.
f Configure IPv4 and IPv6 settings.
Virtual machine port group a Enter a network Label or the port group, or accept the generated label.
b Set the VLAN ID to configure VLAN handling in the port group.
What to do next
n You might need to change the teaming and failover policy of the new standard switch. For example, if
the host is connected to an Etherchannel on the physical switch, you must configure the vSphere
Standard Switch with Rout based on IP hash as a load balancing algorithm. See Teaming and
Failover Policy for more information.
n If you create the new standard switch with a port group for virtual machine networking, connect virtual
machines to the port group.
The Add Networking wizard in the vSphere Web Client guides you through the process to create a
virtual network to which virtual machines can connect, including creating a vSphere Standard Switch and
configuring settings for a network label.
When you set up virtual machine networks, consider whether you want to migrate the virtual machines in
the network between hosts. If so, be sure that both hosts are in the same broadcast domain—that is, the
same Layer 2 subnet.
ESXi does not support virtual machine migration between hosts in different broadcast domains because
the migrated virtual machine might require systems and resources that it would no longer have access to
in the new network. Even if your network configuration is set up as a high-availability environment or
includes intelligent switches that can resolve the virtual machine’s needs across different networks, you
might experience lag times as the Address Resolution Protocol (ARP) table updates and resumes
network traffic for the virtual machines.
VMware, Inc. 19
vSphere Networking
Virtual machines reach physical networks through uplink adapters. A vSphere Standard Switch can
transfer data to external networks only when one or more network adapters are attached to it. When two
or more adapters are attached to a single standard switch, they are transparently teamed.
Procedure
3 In Select connection type, select Virtual Machine Port Group for a Standard Switch and click
Next.
4 In Select target device, select an existing standard switch or create a new standard switch.
5 If the new port group is for an existing standard switch, navigate to the switch.
a Click Browse.
6 (Optional) Оn the Create a Standard Switch page, assign physical network adapters to the standard
switch.
If you create a standard switch without physical network adapters, all traffic on that switch is confined
to that switch. No other hosts on the physical network or virtual machines on other standard switches
can send or receive traffic over this standard switch. You might create a standard switch without
physical network adapters if you want a group of virtual machines to be able to communicate with
each other, but not with other hosts or with virtual machines outside the group.
c Use the Failover order group drop-down menu to assign the adapter to Active adapters,
Standby adapters, or Unused adapters, and click OK.
d (Optional) Use the up and down arrows in the Assigned adapters list to change the position of
the adapter if needed.
e Click Next.
VMware, Inc. 20
vSphere Networking
7 On the Connection settings page, identify traffic through the ports of the group.
a Type a Network label for the port group, or accept the generated label.
The VLAN ID also reflects the VLAN tagging mode in the port group.
External Switch Tagging (EST) 0 The virtual switch does not pass traffic associated with a VLAN.
Virtual Switch Tagging (VST) From 1 to 4094 The virtual switch tags traffic with the entered tag.
Virtual Guest Tagging (VGT) 4095 Virtual machines handle VLANs. The virtual switch passes traffic
from any VLAN.
c Click Next.
8 Review the port group settings in the Ready to complete page, and click Finish.
Procedure
4 In the topology diagram of the switch, click the name of the port group.
5 Under the topology diagram title, click the Edit settings icon .
6 On the Properties page, rename the port group in the Network label text field.
External Switch Tagging (EST) 0 The virtual switch does not pass traffic associated with a VLAN.
Virtual Switch Tagging (VST) From 1 to 4094 The virtual switch tags traffic with the entered tag.
Virtual Guest Tagging (VGT) 4095 Virtual machines handle VLANs. The virtual switch passes traffic from
any VLAN.
8 On the Security page, override the switch settings for protection against MAC address impersonation
and for running virtual machines in promiscuous mode.
9 On the Traffic shaping page, override at the port group level the size of average and peak bandwidth
and of bursts.
VMware, Inc. 21
vSphere Networking
10 On the Teaming and failover page, override the teaming and failover settings inherited from the
standard switch.
You can configure traffic distribution and rerouting between the physical adapters associated with the
port group. You can also change the order in which host physical adapters are used upon failure.
11 Click OK.
Prerequisites
Verify that there are no powered-on virtual machines connected to the port group that you want to
remove.
Procedure
4 From the topology diagram of the switch, select the port group that you want to remove by clicking its
label.
5 From the toolbar in the switch topology, click the Remove selected port group action icon .
VMware, Inc. 22
vSphere Networking
Procedure
3 Select a standard switch from the table and click Edit settings.
You can enable jumbo frames by setting an MTU value greater than 1500. You cannot set an MTU
size greater than 9000 bytes.
5 Click OK.
If the physical adapter supports SR-IOV, you can enable it and configure the number of virtual functions to
use for virtual machine networking.
Procedure
The physical network adapters of the host appear in a table that contains details for each physical
network adapter.
3 Select the physical network adapter from the list and click the Edit adapter settings icon.
4 Select speed and duplex mode of the physical network adapter from the drop-down menu.
5 Click OK.
NIC teaming combines multiple network connections to increase throughput and provide redundancy
should a link fail. To create a team, you associate multiple physical adapters to a single vSphere Standard
Switch.
Procedure
3 Select the standard switch you want to add a physical adapter to.
VMware, Inc. 23
vSphere Networking
4 Click the Manage the physical network adapters connected to the selected switch icon.
The failover group determines the role of the adapter for exchanging data with the external
network, that is, active, standby or unused. By default, the adapters are added as active to the
standard switch.
c Click OK
The selected adapters appear in the selected failover group list under the Assigned Adapters list.
6 (Optional) Use the up and down arrows to change the position of an adapter in the failover groups.
The topology diagram of a standard switch provides a visual representation of the adapters and port
groups connected to the switch.
From the diagram you can edit the settings of a selected port group and of a selected adapter.
Procedure
The diagram appears under the list of virtual switches on the host.
VMware, Inc. 24
vSphere Networking
Figure 2‑2. Topology Diagram of a Standard Switch That Connects the VMkernel and Virtual
Machines to the Network
VMware, Inc. 25
Setting Up Networking with
vSphere Distributed Switches 3
With vSphere distributed switches you can set up and configure networking in a vSphere environment.
VMware, Inc. 26
vSphere Networking
vCenter Server
Management plane
Data plane
Host 1 Host 2
Virtual network
Physical network
Physical Switch
A network switch in vSphere consists of two logical sections that are the data plane and the management
plane. The data plane implements the package switching, filtering, tagging, and so on. The management
plane is the control structure that you use to configure the data plane functionality. A vSphere Standard
Switch contains both data and management planes, and you configure and maintain each standard
switch individually.
A vSphere Distributed Switch separates the data plane and the management plane. The management
functionality of the distributed switch resides on the vCenter Server system that lets you administer the
networking configuration of your environment on a data center level. The data plane remains locally on
every host that is associated with the distributed switch. The data plane section of the distributed switch is
called a host proxy switch. The networking configuration that you create on vCenter Server (the
management plane) is automatically pushed down to all host proxy switches (the data plane).
VMware, Inc. 27
vSphere Networking
The vSphere Distributed Switch introduces two abstractions that you use to create consistent networking
configuration for physical NICs, virtual machines, and VMkernel services.
Uplink port group An uplink port group or dvuplink port group is defined during the creation of
the distributed switch and can have one or more uplinks. An uplink is a
template that you use to configure physical connections of hosts as well as
failover and load balancing policies. You map physical NICs of hosts to
uplinks on the distributed switch. At the host level, each physical NIC is
connected to an uplink port with a particular ID. You set failover and load
balancing policies over uplinks and the policies are automatically
propagated to the host proxy switches, or the data plane. In this way you
can apply consistent failover and load balancing configuration for the
physical NICs of all hosts that are associated with the distributed switch.
Distributed port group Distributed port groups provide network connectivity to virtual machines
and accommodate VMkernel traffic. You identify each distributed port group
by using a network label, which must be unique to the current data center.
You configure NIC teaming, failover, load balancing, VLAN, security, traffic
shaping , and other policies on distributed port groups. The virtual ports that
are connected to a distributed port group share the same properties that
are configured to the distributed port group. As with uplink port groups, the
configuration that you set on distributed port groups on vCenter Server (the
management plane) is automatically propagated to all hosts on the
distributed switch through their host proxy switches (the data plane). In this
way you can configure a group of virtual machines to share the same
networking configuration by associating the virtual machines to the same
distributed port group.
For example, suppose that you create a vSphere Distributed Switch on your data center and associate
two hosts with it. You configure three uplinks to the uplink port group and connect a physical NIC from
each host to an uplink. In this way, each uplink has two physical NICs from each host mapped to it, for
example Uplink 1 is configured with vmnic0 from Host 1 and Host 2. Next you create the Production and
the VMkernel network distributed port groups for virtual machine networking and VMkernel services.
Respectively, a representation of the Production and the VMkernel network port groups is also created on
Host 1 and Host 2. All policies that you set to the Production and the VMkernel network port groups are
propagated to their representations on Host 1 and Host 2.
To ensure efficient use of host resources, the number of distributed ports of proxy switches is dynamically
scaled up and down. A proxy switch on such a host can expand up to the maximum number of ports
supported on the host. The port limit is determined based on the maximum number of virtual machines
that the host can handle.
VMware, Inc. 28
vSphere Networking
Figure 3‑2. NIC Teaming and Port Allocation on a vSphere Distributed Switch
vCenter Server
Host 1 Host 2 Host 1 Host 2
Distributed
0 1 2 3 4 port groups
VM network VMkernel network
For example, suppose that you create the VM network and the VMkernel network distributed port groups,
respectively with 3 and 2 distributed ports. The distributed switch allocates ports with IDs from 0 to 4 in
the order that you create the distributed port groups. Next, you associate Host 1 and Host 2 with the
distributed switch. The distributed switch allocates ports for every physical NIC on the hosts, as the
numbering of the ports continues from 5 in the order that you add the hosts. To provide network
connectivity on each host, you map vmnic0 to Uplink 1, vmnic1 to Uplink 2, and vmnic2 to Uplink 3.
To provide connectivity to virtual machines and to accommodate VMkernel traffic, you configure teaming
and failover to the VM network and to the VMkernel network port groups. Uplink 1 and Uplink 2 handle
the traffic for the VM network port group, and Uplink 3 handles the traffic for the VMkernel network port
group.
VMware, Inc. 29
vSphere Networking
Host 1
VM network VMkernel
network
0 1 3
Host Proxy
Switch
5 6 7
Uplink port group
Physical Switch
On the host side, the packet flow from virtual machines and VMkernel services passes through particular
ports to reach the physical network. For example, a packet sent from VM1 on Host 1 first reaches port 0
on the VM network distributed port group. Because Uplink 1 and Uplink 2 handle the traffic for the VM
network port group, the packet can continue from uplink port 5 or uplink port 6 . If the packet goes through
uplink port 5, it continues to vmnic0, and if the packet goes to uplink port 6, it continues to vmnic1.
Procedure
2 In the navigator, right-click the data center and select Distributed Switch > New Distributed Switch.
3 On the Name and location page, type a name for the new distributed switch, or accept the generated
name, and click Next.
VMware, Inc. 30
vSphere Networking
4 On the Select version page, select a distributed switch version and click Next.
Option Description
Distributed Switch: 6.5.0 Compatible with ESXi 6.5 and later. Features released with later vSphere
distributed switch versions are not supported.
Distributed Switch: 6.0.0 Compatible with ESXi 6.0 and later. Features released with later vSphere
distributed switch versions are not supported.
Uplink ports connect the distributed switch to physical NICs on associated hosts. The number of
uplink ports is the maximum number of allowed physical connections to the distributed switch per
host.
By using Network I/O Control you can prioritize the access to network resources for certain types
of infrastructure and workload traffic according to the requirements of your deployment. Network
I/O Control continuously monitors the I/O load over the network and dynamically allocates
available resources.
c Select the Create a default port group check box to create a new distributed port group with
default settings for this switch.
d (Optional) To create a default distributed port group, type the port group name in the Port group
name, or accept the generated name.
If your system has custom port group requirements, create distributed port groups that meet
those requirements after you add the distributed switch.
e Click Next.
6 On the Ready to complete page, review the settings you selected and click Finish.
A distributed switch is created in the data center. You can view the features supported on the distributed
switch as well as other details by navigating to the new distributed switch and clicking the Summary tab.
What to do next
Add hosts to the distributed switch and configure their network adapters on the switch.
VMware, Inc. 31
vSphere Networking
The upgrade of a distributed switch causes the hosts and virtual machines attached to the switch to
experience a brief downtime. For more information, see KB 52621.
Note To be able to restore the connectivity of the virtual machines and VMkernel adapters if the upgrade
fails, back up the configuration of the distributed switch.
If the upgrade is not successful, to recreate the switch with its port groups and connected hosts, you can
import the switch configuration file. See Export vSphere Distributed Switch Configurations and Import a
vSphere Distributed Switch Configuration.
Prerequisites
Procedure
2 Right-click the distributed switch and select Upgrade > Upgrade Distributed Switch.
3 Select the vSphere Distributed Switch version that you want to upgrade the switch to and click Next.
Option Description
Version 6.5.0 Compatible with ESXi version 6.5 and later. Features released with later vSphere
Distributed Switch versions are not supported.
Version 6.0.0 Compatible with ESXi version 6.0 and later. Features released with later vSphere
Distributed Switch versions are not supported.
Some ESXi instances that are connected to the distributed switch might be incompatible with the
selected target version. Upgrade or remove the incompatible hosts, or select another upgrade version
for the distributed switch.
Caution After you upgrade the vSphere Distributed Switch, you cannot revert it to an earlier version.
You also cannot add ESXi hosts that are running an earlier version than the new version of the
switch.
VMware, Inc. 32
vSphere Networking
Procedure
3 Click Edit.
Option Description
Number of uplinks Select the number of uplink ports for the distributed switch.
Click Edit uplink names to change the names of the uplinks.
Number of ports The number of ports for this distributed switch. This cannot be edited.
Network I/O Control Use the drop-down menu to enable or disable Network I/O control.
Option Description
MTU (Bytes) Maximum MTU size for the vSphere Distributed Switch. To enable jumbo frames,
set a value greater than 1500 bytes.
Multicast filtering mode n Basic. The distributed switch forwards traffic that is related to a multicast
group based on a MAC address generated from the last 23 bits of the IPv4
address of the group.
n IGMP/MLD snooping. The distributed switch forwards multicast traffic to
virtual machines according to the IPv4 and IPv6 addresses of subscribed
multicast groups by using membership messages defined by the Internet
Group Management Protocol (IGMP ) and Multicast Listener Discovery
protocol.
Discovery Protocol a Select Cisco Discovery Protocol, Link Layer Discovery Protocol, or (disabled)
from the Type drop-down menu.
b Set Operation to Listen, Advertise, or Both.
For information about Discovery Protocol, see Switch Discovery Protocol.
Administrator Contact Type the name and other details of the administrator for the distributed switch.
6 Click OK.
VMware, Inc. 33
vSphere Networking
VMware, Inc. 34
vSphere Networking
n Create distributed port groups for VMkernel services. For example, create distributed port groups for
management network, vMotion, and Fault Tolerance.
n Configure enough uplinks on the distributed switch for all physical NICs that you want to connect to
the switch. For example, if the hosts that you want to connect to the distributed switch have eight
physical NICs each, configure eight uplinks on the distributed switch.
n Make sure that the configuration of the distributed switch is prepared for services with specific
networking requirements. For example, iSCSI has specific requirements for the teaming and failover
configuration of the distributed port group where you connect the iSCSI VMkernel adapter.
You can use the Add and Manage Hosts wizard in the vSphere Web Client to add multiple hosts at a
time.
If some hosts on a distributed switch are associated to other switches in your data center, you can
migrate network adapters to or from the distributed switch.
If you migrate virtual machine network adapters or VMkernel adapters, make sure that the destination
distributed port groups have at least one active uplink, and the uplink is connected to a physical NIC on
the hosts. Another approach is to migrate physical NICs, virtual network adapters, and VMkernel adapters
simultaneously.
If you migrate physical NICs, leave at least one active NIC that handles the traffic of port groups. For
example, if vmnic0 and vmnic1 handle the traffic of the VM Network port group, migrate vmnic0 and leave
vmnic1 connected to the group.
n To add hosts to a different distributed switch, you can use the Add and Manage Hosts wizard to
migrate the network adapters on the hosts to the new switch all together. You can then remove the
hosts safely from their current distributed switch.
VMware, Inc. 35
vSphere Networking
n To migrate host networking to standard switches, you must migrate the network adapters in stages.
For example, remove physical NICs on the hosts from the distributed switch by leaving one physical
NIC on every host connected to the switch to keep the network connectivity up. Next, attach the
physical NICs to the standard switches and migrate VMkernel adapters and virtual machine network
adapters to the switches. Lastly, migrate the physical NIC that you left connected to the distributed
switch to the standard switches.
Prerequisites
n Verify that enough uplinks are available on the distributed switch to assign to the physical NICs that
you want to connect to the switch.
n Verify that there is at least one distributed port group on the distributed switch.
n Verify that the distributed port group have active uplinks configured in its teaming and failover policy.
If you migrate or create VMkernel adapters for iSCSI, verify that the teaming and failover policy of the
target distributed port group meets the requirements for iSCSI:
n Verify that only one uplink is active, the standby list is empty, and the rest of the uplinks are unused.
n Verify that only one physical NIC per host is assigned to the active uplink.
Procedure
3 On the Select task page, select Add hosts, and click Next.
4 On the Select hosts page, click New hosts, select from the hosts in your data center, click OK, and
then click Next.
5 On the Select network adapter tasks page, select the tasks for configuring network adapters to the
distributed switch and click Next.
6 On the Manage physical network adapters page, configure physical NICs on the distributed switch.
If you select physical NICs that are already connected to other switches, they are migrated to the
current distributed switch.
VMware, Inc. 36
vSphere Networking
For consistent network configuration, you can connect one and the same physical NIC on every host
to the same uplink on the distributed switch.
For example, if you are adding two hosts connect vmnic1 on of each host to Uplink1 on the
distributed switch.
7 Click Next.
Option Description
No impact iSCSI will continue its normal function after the new networking configuration is
applied.
Important impact The normal function of iSCSI might be disrupted if the new networking
configuration is applied.
Critical impact The normal function of iSCSI will be interrupted if the new networking
configuration is applied.
a If the impact on iSCSI is important or critical, click iSCSI entry and review the reasons that are
displayed in the Analysis details pane.
b After you troubleshoot the impact on iSCSI, proceed with your networking configuration.
10 Click Next.
a To connect all network adapters of a virtual machine to a distributed port group, select the virtual
machine, or select an individual network adapter to connect only that adapter.
c Select a distributed port group from the list and click OK.
What to do next
Having hosts associated with the distributed switch, you can manage physical NICs, VMkernel adapters,
and virtual machine network adapters.
VMware, Inc. 37
vSphere Networking
For consistent networking configuration throughout all hosts, you can assign the same physical NIC on
every host to the same uplink on the distributed switch. For example, you can assign vmnic1 from hosts
ESXi A and ESXi B to Uplink 1.
Procedure
4 In Select hosts, click Attached hosts and select from the hosts that are associated with the
distributed switch.
5 Click Next.
6 In Select network adapter tasks, select Manage physical adapters and click Next.
7 In Manage physical network adapters, select a physical NIC from the On other switches/unclaimed
list.
If you select physical NICs that are already assigned to other switches, they are migrated to the
current distributed switch.
10 Click Next.
Option Description
No impact iSCSI will continue its normal function after the new networking configuration is
applied.
Important impact The normal function of iSCSI might be disrupted if the new networking
configuration is applied.
Critical impact The normal function of iSCSI will be interrupted if the new networking
configuration is applied.
a If the impact on iSCSI is important or critical, click iSCSI entry and review the reasons that are
displayed in the Analysis details pane.
b After you troubleshoot the impact on iSCSI, proceed with your networking configuration.
VMware, Inc. 38
vSphere Networking
Procedure
4 In Select hosts, click Attached hosts and select from the hosts that are associated with the
distributed switch.
5 Click Next.
6 In Select network adapter tasks, select Manage VMkernel adapters and click Next.
7 In Manage VMkernel network adapters, select the adapter and click Assign port group.
9 Click Next.
Option Description
No impact iSCSI will continue its normal function after the new networking configuration is
applied.
Important impact The normal function of iSCSI might be disrupted if the new networking
configuration is applied.
Critical impact The normal function of iSCSI will be interrupted if the new networking
configuration is applied.
a If the impact on iSCSI is important or critical, click iSCSI entry and review the reasons that are
displayed in the Analysis details pane.
b After you troubleshoot the impact on iSCSI, proceed with your networking configuration.
You should dedicate one distributed port group for each VMkernel adapter. One VMkernel adapter should
handle only one traffic type.
Procedure
VMware, Inc. 39
vSphere Networking
4 In Select hosts, click Attached hosts and select from the hosts that are associated with the
distributed switch.
5 Click Next.
6 In Select network adapter tasks, select Manage VMkernel adapters and click Next.
8 In Select target device, select a distributed port group, and click Next.
9 On the Port properties page, configure the settings for the VMkernel adapter.
Option Description
Network label The network label is inherited from the label of the distributed port group.
Note The IPv6 option does not appear on hosts that do not have IPv6 enabled.
TCP/IP stack Select a TCP/IP stack from the list. Once you set a TCP/IP stack for the VMkernel
adapter, you cannot change it later. If you select the vMotion or the Provisioning
TCP/IP stack, you will be able to use only these stacks to handle vMotion or
Provisioning traffic on the host. All VMkernel adapters for vMotion on the default
TCP/IP stack are disabled for future vMotion sessions. If you set the Provisioning
TCP/IP stack, VMkernel adapters on the default TCP/IP stack are disabled for
operations that include Provisioning traffic, such as virtual machine cold
migration, cloning, and snapshot migration.
Enable services You can enable services for the default TCP/IP stack on the host. Select from the
available services:
n vMotion traffic. Enables the VMkernel adapter to advertise itself to another
host as the network connection where vMotion traffic is sent. The migration
with vMotion to the selected host is not possible if the vMotion service is not
enabled for any VMkernel adapter on the default TCP/IP stack, or there are
no adapters using the vMotion TCP/IP stack.
n Provisioning traffic. Handles the data transferred for virtual machine cold
migration, cloning, and snapshot migration.
n Fault Tolerance traffic. Enables Fault Tolerance logging on the host. You
can use only one VMkernel adapter for FT traffic per host.
n Management traffic. Enables the management traffic for the host and
vCenter Server. Typically, hosts have such a VMkernel adapter created when
the ESXi software is installed. You can create another VMkernel adapter for
management traffic on the host to provide redundancy.
n vSphere Replication traffic. Handles the outgoing replication data that is
sent from the source ESXi host to the vSphere Replication server.
n vSphere Replication NFC traffic. Handles the incoming replication data on
the target replication site.
n vSAN. Enables thevSAN traffic on the host. Every host that is part of a vSAN
cluster must have such a VMkernel adapter.
VMware, Inc. 40
vSphere Networking
10 If you selected the vMotion TCP/IP or the Provisioning stack, click OK in the warning dialog that
appears.
If a live migration is already initiated, it completes successfully even after the involved VMkernel
adapters on the default TCP/IP stack are disabled for vMotion. Same refers to operations that include
VMkernel adapters on the default TCP/IP stack that are set for the Provisioning traffic.
11 (Optional) On the IPv4 settings page, select an option for obtaining IP addresses.
Option Description
Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network.
Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
The VMkernel Default Gateway and DNS server addresses for IPv4 are obtained
from the selected TCP/IP stack.
Select the Override default gateway for this adapter check box and enter a
gateway address, if you want to specify a different gateway for the VMkernel
adapter.
12 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses.
Option Description
Obtain IPv6 addresses automatically Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the
through DHCP network.
Obtain IPv6 addresses automatically Use router advertisement to obtain IPv6 addresses.
through Router Advertisement In ESXi 6.5 and later router advertisement is enabled by default and supports the
M and O flags in accordance with RFC 4861.
Static IPv6 addresses a Click Add IPv6 address to add a new IPv6 address.
b Enter the IPv6 address and subnet prefix length, and click OK.
c To change the VMkernel default gateway, click Override default gateway for
this adapter.
The VMkernel Default Gateway address for IPv6 is obtained from the selected
TCP/IP stack.
13 Review your settings selections on the Ready to complete page and click Finish.
Prerequisites
Verify that at least one distributed port group intended for virtual machine networking exists on the
distributed switch.
VMware, Inc. 41
vSphere Networking
Procedure
4 In Select hosts, click Attached hosts and select from the hosts that are associated with the
distributed switch.
5 Click Next.
6 In Select network adapter tasks, select Migrate virtual machine networking and click Next.
a To connect all network adapters of a virtual machine to a distributed port group, select the virtual
machine, or select an individual network adapter to connect only that adapter.
c Select a distributed port group from the list and click OK.
Procedure
5 At the bottom of the dialog box, select Configure identical networking settings on multiple hosts
and click Next.
8 On the Manage physical network adapters and Manage VMkernel network adapters pages, make the
configuration changes that you need on the template host, and click Apply to all for all other hosts.
VMware, Inc. 42
vSphere Networking
On the Manage physical network adapters page of the wizard, assign a physical NIC to an uplink on the
template host and then click Apply to all to create the same configuration on the other host.
Figure 3‑4. Applying Physical NICs Configuration on a vSphere Distributed Switch by Using
a Template Host
On the Manage VMkernel network adapters page, assign a VMkernel adapter to a port group and click
Apply to all to apply the same configuration to the other host.
After you click the Apply to all button, the destination VMkernel adapter has both the Modified and the
Reassigned qualifiers. The Modified qualifier appears, because when you click the Apply to all button,
vCenter Server copies the configuration specifications of the template VMKernel adapter to the
destination VMkernel adapter even if the configurations of the template and destination adapters are
identical. As a result, the destination adapters are always modified.
VMware, Inc. 43
vSphere Networking
Prerequisites
n Verify that physical NICs on the target hosts are migrated to a different switch.
n Verify that VMkernel adapters on the hosts are migrated to a different switch.
n Verify that virtual machine network adapters are migrated to a different switch.
For details about migrating network adapters to different switches, see Tasks for Managing Host
Networking on a vSphere Distributed Switch
Procedure
5 Click Finish.
VMware, Inc. 44
vSphere Networking
For details about setting up VMkernel networking on host proxy switches, see Create a VMkernel Adapter
on a vSphere Distributed Switch.
To migrate virtual machine network adapters or VMkernel adapters, make sure that the destination
distributed port groups have at least one active uplink, and the uplink is connected to a physical NIC on
this host. Alternatively, migrate physical NICs, virtual network adapters, and VMkernel adapters at once.
To migrate physical NICs, make sure that the source port groups on the standard switch have at least one
physical NIC to handle their traffic. For example, if you migrate a physical NIC that is assigned to a port
group for virtual machine networking, make sure that the port group is connected to at least one physical
NIC. Otherwise the virtual machines on same VLAN on the standard switch will have connectivity
between each other but not to the external network.
Procedure
3 Select the destination distributed switch and click Migrate physical or virtual network adapters to
this distributed switch.
4 Select the tasks for migrating network adapters and click Next.
a From the On other switches/unclaimed list, select a physical NIC and click Assign uplink.
c Click Next.
You should connect one VMkernel adapter to one distributed port group at a time.
c Click Next.
VMware, Inc. 45
vSphere Networking
7 Review the services that are affected from the new networking configuration.
a If there is an important or serious impact reported on a service, click the service and review the
analysis details.
For example, an important impact on iSCSI might be reported as a result from an incorrect
teaming and failover configuration on the distributed port group where you migrate the iSCSI
VMkernel adapter. You must leave one active uplink on the teaming and failover order of the
distributed port group, leave the standby list empty, and move the rest of the uplinks to unused.
a Select a virtual machine or a virtual machine network adapter and click Assign port group.
If you select a virtual machine, you migrate all network adapters on the virtual machine. If you
select a network adapter, you migrate only this network adapter.
b Select a distributed port group from the list and click OK.
c Click Next.
9 On the Ready to complete page, review the new networking configuration and click Finish.
For details about creating VMkernel adapters on a vSphere distributed switch, see Create a VMkernel
Adapter on a vSphere Distributed Switch.
Prerequisites
Verify that the destination standard switch has at least one physical NIC.
Procedure
5 On the Select VMkernel network adapter page, select the virtual network adapter to migrate to the
standard switch from the list.
6 On the Configure settings page, edit the Network label and VLAN ID for the network adapter.
7 On the Ready to complete page, review the migration details and click Finish.
VMware, Inc. 46
vSphere Networking
Procedure
4 Click the Manage the physical network adapters connected to the selected switch icon.
5 Select a free uplink from the list and click Add adapter.
Procedure
4 Click the Manage the physical network adapters connected to the selected switch icon.
6 Click OK.
What to do next
When you remove physical NICs from active virtual machines, you might see the NICs you removed
reported in the vSphere Web Client. See Removing NICs from Active Virtual Machines.
The vSphere Web Client might report that the NIC has been removed, but you continue to see it attached
to the virtual machine.
VMware, Inc. 47
vSphere Networking
You might also still see the NIC attached to the virtual machine if the guest operating system of the virtual
machine does not support hot removal of NICs.
Procedure
2 Right-click the distributed switch and select Distributed port group > New distributed port group.
3 On the Select name and location page, enter the name of the new distributed port group, or accept
the generated name, and click Next.
4 On the Configure settings page, set the general properties for the new distributed port group and click
Next.
Setting Description
Port binding Select when ports are assigned to virtual machines connected to this distributed
port group.
n Static binding: Assign a port to a virtual machine when the virtual machine
connects to the distributed port group.
n Dynamic binding: Assign a port to a virtual machine the first time the virtual
machine powers on after it is connected to the distributed port group.
Dynamic binding has been deprecated since ESXi 5.0.
n Ephemeral - no binding: No port binding. You can assign a virtual machine
to a distributed port group with ephemeral port binding also when connected
to the host.
Port allocation n Elastic: The default number of ports is eight. When all ports are assigned, a
new set of eight ports is created. This is the default.
n Fixed: The default number of ports is set to eight. No additional ports are
created when all ports are assigned.
Number of ports Enter the number of ports on the distributed port group.
VMware, Inc. 48
vSphere Networking
Setting Description
Network resource pool Use the drop-down menu to assign the new distributed port group to a user-
defined network resource pool. If you have not created a network resource pool,
this menu is empty.
VLAN Use the VLAN type drop-down menu to select VLAN options:
n None: Do not use VLAN.
n VLAN: In the VLAN ID text box, enter a number between 1 and 4094.
n VLAN trunking: Enter a VLAN trunk range.
n Private VLAN: Select a private VLAN entry. If you did not create any private
VLANs, this menu is empty.
Advanced To customize the policy configurations for the new distributed port group, select
this check box.
5 (Optional) On the Security page, edit the security exceptions and click Next.
Setting Description
Promiscuous mode n Reject. Placing an adapter in promiscuous mode from the guest operating
system does not result in receiving frames for other virtual machines.
n Accept. If an adapter is placed in promiscuous mode from the guest
operating system, the switch allows the guest adapter to receive all frames
passed on the switch in compliance with the active VLAN policy for the port
where the adapter is connected.
Firewalls, port scanners, intrusion detection systems, and so on, need to run
in promiscuous mode.
MAC address changes n Reject. If you set this option to Reject and the guest OS changes the MAC
address of the adapter to a value different from the address in the .vmx
configuration file, the switch drops all inbound frames to the virtual machine
adapter.
If the guest OS changes the MAC address back, the virtual machine receives
frames again.
n Accept. If the guest OS changes the MAC address of a network adapter, the
adapter receives frames to its new address.
Forged transmits n Reject. The switch drops any outbound frame with a source MAC address
that is different from the one in the .vmx configuration file.
n Accept. The switch does not perform filtering and permits all outbound
frames.
6 (Optional) On the Traffic shaping page, enable or disable Ingress or Egress traffic shaping and click
Next.
Setting Description
Status If you enable either Ingress traffic shaping or Egress traffic shaping, you are
setting limits on the amount of networking bandwidth allocated for each virtual
adapter associated with this particular port group. If you disable the policy,
services have a free, clear connection to the physical network by default.
Average bandwidth Establishes the number of bits per second to allow across a port, averaged over
time. This is the allowed average load.
VMware, Inc. 49
vSphere Networking
Setting Description
Peak bandwidth The maximum number of bits per second to allow across a port when it is sending
and receiving a burst of traffic. This tops the bandwidth used by a port whenever it
is using its burst bonus.
Burst size The maximum number of bytes to allow in a burst. If this parameter is set, a port
might gain a burst bonus when it does not use all its allocated bandwidth.
Whenever the port needs more bandwidth than specified by Average bandwidth,
it might temporarily transmit data at a higher speed if a burst bonus is available.
This parameter tops the number of bytes that might be accumulated in the burst
bonus and thus transferred at a higher speed.
7 (Optional) On the Teaming and failover page, edit the settings and click Next.
Setting Description
Note IP-based teaming requires that the physical switch be configured with
EtherChannel. For all other options, disable EtherChannel.
Network failure detection Specify the method to use for failover detection.
n Link status only. Relies solely on the link status that the network adapter
provides. This option detects failures, such as cable pulls and physical switch
power failures, but not configuration errors, such as a physical switch port
being blocked by spanning tree or that is misconfigured to the wrong VLAN or
cable pulls on the other side of a physical switch.
n Beacon probing. Sends out and listens for beacon probes on all NICs in the
team and uses this information, in addition to link status, to determine link
failure. This detects many of the failures previously mentioned that are not
detected by link status alone.
Notify switches Select Yes or No to notify switches in case of failover. If you select Yes, whenever
a virtual NIC is connected to the distributed switch or whenever that virtual NIC’s
traffic might be routed over a different physical NIC in the team because of a
failover event, a notification is sent out over the network to update the lookup
tables on physical switches. In almost all cases, this process is desirable for the
lowest latency of failover occurrences and migrations with vMotion.
Note Do not use this option when the virtual machines using the port group are
using Microsoft Network Load Balancing in unicast mode. No such issue exists
with NLB running in multicast mode.
VMware, Inc. 50
vSphere Networking
Setting Description
Failback Select Yes or No to disable or enable failback.
This option determines how a physical adapter is returned to active duty after
recovering from a failure. If failback is set to Yes (default), the adapter is returned
to active duty immediately upon recovery, displacing the standby adapter that
took over its slot, if any. If failback is set to No, a failed adapter is left inactive
even after recovery until another currently active adapter fails, requiring its
replacement.
Failover order Specify how to distribute the workload for uplinks. To use some uplinks but
reserve others for emergencies if the uplinks in use fail, set this condition by
moving them into different groups:
n Active uplinks. Continue to use the uplink when the network adapter
connectivity is up and active.
n Standby uplinks . Use this uplink if one of the active adapters' connectivity is
down.
n Unused uplinks . Do not use this uplink.
8 (Optional) On the Monitoring page, enable or disable NetFlow and click Next.
Setting Description
Enabled NetFlow is enabled on the distributed port group. NetFlow settings can be
configured at the vSphere Distributed Switch level.
Selecting Yes shuts down all ports in the port group. This action might disrupt the normal network
operations of the hosts or virtual machines using the ports.
10 (Optional) On the Edit additional settings page, add a description of the port group and set any policy
overrides per port and click Next.
11 On the Ready to complete page, review your settings and click Finish.
Procedure
VMware, Inc. 51
vSphere Networking
Option Description
Name The name of distributed port group. You can edit the name in the text field.
Port binding Choose when ports are assigned to virtual machines connected to this distributed
port group.
n Static binding: Assign a port to a virtual machine when the virtual machine
connects to the distributed port group.
n Dynamic binding: Assign a port to a virtual machine the first time the virtual
machine powers on after it is connected to the distributed port group.
Dynamic binding has been deprecated since ESXi 5.0.
n Ephemeral: No port binding. You can also assign a virtual machine to a
distributed port group with ephemeral port binding when connected to the
host.
Port allocation n Elastic: The default number of ports is set to eight. When all ports are
assigned, a new set of eight ports is created. This is the default.
n Fixed: The default number of ports is set to eight. No additional ports are
created when all ports are assigned.
Number of ports Enter the number of ports on the distributed port group.
Network resource pool Use the drop-down menu to assign the new distributed port group to a user-
defined network resource pool. If you have not created a network resource pool,
this menu is empty.
Description Enter any information about the distributed port group in the description field.
4 Click OK.
Procedure
Option Description
Configure reset at disconnect From the drop-down menu, enable or disable reset at disconnect.
When a distributed port is disconnected from a virtual machine, the configuration
of the distributed port is reset to the distributed port group setting. Any per-port
overrides are discarded.
Override port policies Select the distributed port group policies to be overridden on a per-port level.
VMware, Inc. 52
vSphere Networking
4 (Optional) Use the policy pages to set overrides for each port policy.
5 Click OK.
Prerequisites
n Verify that all virtual machines connected to the corresponding labeled network are migrated to a
different labeled network.
n Verify that all VMkernel adapters connected to the distributed port group are migrated to a different
port group, or are deleted.
Procedure
Default distributed port configuration is determined by the distributed port group settings, but some
settings for individual distributed ports can be overridden.
Procedure
3 Click the Ports tab and select a port from the list.
VMware, Inc. 53
vSphere Networking
The ports table for the distributed port group displays runtime statistics for each distributed port.
The State column displays the current state for each distributed port.
Option Description
Procedure
3 Click the Ports tab, and select a distributed port from the table.
Information about the distributed port appears at the bottom of the screen.
5 On the Properties page and policy pages, edit information about the distributed port and click OK.
You can allow overrides at the port level by changing the Advanced settings of the distributed port
group. See Configure Overriding Networking Policies on Port Level.
Connect virtual machines to vSphere distributed switches by connecting their associated virtual network
adapters to distributed port groups. You can do this either for an individual virtual machine by modifying
the virtual machine’s network adapter configuration, or for a group of virtual machines by migrating virtual
machines from an existing virtual network to a vSphere distributed switch.
VMware, Inc. 54
vSphere Networking
Procedure
2 Right-click the data center in the navigator and select Migrate VMs to Another Network.
n Select Specific network and use the Browse button to select a specific source network.
n Select No network to migrate all virtual machine network adapters that are not connected to any
other network.
5 Select virtual machines from the list to migrate from the source network to the destination network
and click Next.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
2 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
3 Click Edit.
4 Expand the Network adapter section and select Show more networks from the Network adapter
drop-down menu.
5 In the Select Network dialog box, select a distributed port group and click OK.
6 Click OK.
VMware, Inc. 55
vSphere Networking
You can examine the components, arranged in port groups, whose traffic is handled by the switch, and
the connections between them. The diagram displays information about the physical adapter that
connects the virtual adapters to the external network.
You can view the components that are running on the entire distributed switch and on each host
participating in it.
Watch the video about the operations that you can perform from the topology diagram of vSphere
Distributed Switch.
Diagram Filters
You can use diagram filters to limit the information displayed in topology diagrams. The default filter limits
the topology diagram to display 32 port groups, 32 hosts, and 1024 virtual machines.
You can change the scope of the diagram by using no filters or by applying custom filters. By using a
custom filter, you can view information only about a set of virtual machines, a set of port groups on certain
hosts, or a port. You can create filters from the central topology diagram of the distributed switch.
Procedure
VMware, Inc. 56
vSphere Networking
By default the diagram shows up to 32 distributed port groups, 32 hosts, and 1024 virtual machines.
Figure 3‑6. Topology Diagram of a Distributed Switch That Handles VMkernel and Virtual
Machine Networking
What to do next
You can perform the following common tasks in the topology of the distributed switch:
n Use filters to view the networking components only for selected port groups on certain hosts, for
selected virtual machines, or for a port.
n Locate, configure and migrate virtual machine networking components across host and port groups
by using the Migrate Virtual Machine Networking wizard.
n Detect the virtual machine adapters that have no network assigned and move them to the selected
port group by using the Migrate Virtual Machine Networking wizard.
VMware, Inc. 57
vSphere Networking
n Handle networking components on multiple hosts by using the Add and Manage Hosts wizard.
n View the physical NIC or NIC team that carries the traffic related to a selected virtual machine adapter
or VMkernel adapter.
In this way you can also view the host on which a selected VMkernel adapter resides. Select the
adapter, trace the route to the associated physical NIC, and view the IP address or domain name next
to the NIC.
n Determine the VLAN mode and ID for a port group. For information about VLAN modes, see VLAN
Configuration.
Procedure
The topology of the host proxy switch appears under the list.
VMware, Inc. 58
Setting Up VMkernel
Networking 4
You set up VMkernel adapters to provide network connectivity to hosts and to accommodate system
traffic of vMotion, IP storage, Fault Tolerance logging, vSAN, and so on.
n VMkernel Networking Layer
The VMkernel networking layer provides connectivity to hosts and handles the standard system
traffic of vSphere vMotion, IP storage, Fault Tolerance, vSAN, and others. You can also create
VMkernel adapters on the source and target vSphere Replication hosts to isolate the replication data
traffic.
VMware, Inc. 59
vSphere Networking
vMotion TCP/IP stack Supports the traffic for live migration of virtual machines. Use the vMotion
TCP/IP to provide better isolation for the vMotion traffic. After you create a
VMkernel adapter on the vMotion TCP/IP stack, you can use only this stack
for vMotion on this host. The VMkernel adapters on the default TCP/IP
stack are disabled for the vMotion service. If a live migration uses the
default TCP/IP stack while you configure VMkernel adapters with the
vMotion TCP/IP stack, the migration completes successfully. However, the
involved VMkernel adapters on the default TCP/IP stack are disabled for
future vMotion sessions.
Provisioning TCP/IP Supports the traffic for virtual machine cold migration, cloning, and
stack snapshot migration. You can use the provisioning TCP/IP to handle
Network File Copy (NFC) traffic during long-distance vMotion. NFC
provides a file-specific FTP service for vSphere. ESXi uses NFC for
copying and moving data between datastores. VMkernel adapters
VMware, Inc. 60
vSphere Networking
configured with the provisioning TCP/IP stack handle the traffic from cloning
the virtual disks of the migrated virtual machines in long-distance vMotion.
By using the provisioning TCP/IP stack, you can isolate the traffic from the
cloning operations on a separate gateway. After you configure a VMkernel
adapter with the provisioning TCP/IP stack, all adapters on the default
TCP/IP stack are disabled for the Provisioning traffic.
Custom TCP/IP stacks You can add custom TCP/IP stacks at the VMkernel level to handle
networking traffic of custom applications.
Management traffic Carries the configuration and management communication for ESXi hosts,
vCenter Server, and host-to-host High Availability traffic. By default, when
you install the ESXi software, a vSphere Standard switch is created on the
host together with a VMkernel adapter for management traffic. To provide
redundancy, you can connect two or more physical NICs to a VMkernel
adapter for management traffic.
vMotion traffic Accommodates vMotion. A VMkernel adapter for vMotion is required both
on the source and the target hosts. Configure The VMkernel adapters for
vMotion to handle only the vMotion traffic. For better performance, you can
configure multiple NIC vMotion. To have multi-NIC vMotion, you can
dedicate two or more port groups to the vMotion traffic, respectively every
port group must have a vMotion VMkernel adapter associated with it. Then
you can connect one or more physical NICs to every port group. In this way,
multiple physical NICs are used for vMotion, which results in greater
bandwidth .
Note vMotion network traffic is not encrypted. You should provision secure
private networks for use by vMotion only.
Provisioning traffic Handles the data that is transferred for virtual machine cold migration,
cloning, and snapshot migration.
VMware, Inc. 61
vSphere Networking
IP storage traffic and Handles the connection for storage types that use standard TCP/IP
discovery networks and depend on the VMkernel networking. Such storage types are
software iSCSI, dependent hardware iSCSI, and NFS. If you have two or
more physical NICs for iSCSI, you can configure iSCSI multipathing. ESXi
hosts support NFS 3 and 4.1. To configure a software Fibre Channel over
Ethernet (FCoE) adapter, you must have a dedicated VMkernel adapter.
Software FCoE passes configuration information though the Data Center
Bridging Exchange (DCBX) protocol by using the Cisco Discovery Protocol
(CDP )VMkernel module.
Fault Tolerance traffic Handles the data that the primary fault tolerant virtual machine sends to the
secondary fault tolerant virtual machine over the VMkernel networking
layer. A separate VMkernel adapter for Fault Tolerance logging is required
on every host that is part of a vSphere HA cluster.
vSphere Replication Handles the outgoing replication data that the source ESXi host transfers to
traffic the vSphere Replication server. Dedicate a VMkernel adapter on the source
site to isolate the outgoing replication traffic.
vSphere Replication Handles the incoming replication data on the target replication site.
NFC traffic
vSAN traffic Every host that participates in a vSAN cluster must have a VMkernel
adapter to handle the vSAN traffic.
Procedure
3 To view information about all VMkernel adapters on the host, select VMkernel adapters.
4 Select an adapter from the VMkernel adapters list to view its settings.
Tab Description
All Displays all configuration information about the VMkernel adapter. This information includes port and NIC
settings, IPv4 and IPv6 settings, traffic shaping, teaming and failover, and security policies.
Properties Displays the port properties and NIC settings of the VMkernel adapter. The port properties include the port
group (network label) to which the adapter is associated, the VLAN ID, and the enabled services. The NIC
settings include MAC address and the configured MTU size.
VMware, Inc. 62
vSphere Networking
Tab Description
IP Settings Displays all IPv4 and IPv6 settings for the VMkernel adapter. IPv6 information is not displayed if IPv6 has not
been enabled on the host.
Policies Displays the configured traffic shaping, teaming and failover, and security policies that apply for the port group
to which the VMkernel adapter is connected.
Procedure
4 On the Select connection type page, select VMkernel Network Adapter and click Next.
5 On the Select target device page, select either an existing standard switch or select New standard
switch.
6 (Optional) On the Create a Standard Switch page, assign physical NICs to the switch.
You can create the standard switch without physical NICs and configure them later. During the time
that no physical NICs are attached to the host, the host does not have network connectivity to the
other hosts on the physical network. The virtual machines on the host are able to communicate with
each other.
a Click Add adapters and select as many physical NICs as you need.
b Use the up and down arrows to configure the active and standby NICs.
7 On the Port properties page, configure the settings for the VMkernel adapter.
Option Description
Network label Type a value for this label to indicate the traffic type for the VMkernel adapter, for
example Management traffic or vMotion.
VLAN ID Set a VLAN ID to identify the VLAN that the network traffic of the VMkernel
adapter will use.
Note The IPv6 option does not appear on hosts that do not have IPv6 enabled.
VMware, Inc. 63
vSphere Networking
Option Description
TCP/IP stack Select a TCP/IP stack from the list. After you set a TCP/IP stack for the VMkernel
adapter, you cannot change it later. If you select the vMotion or the Provisioning
TCP/IP stack, you will be able to use only this stack to handle vMotion or
Provisioning traffic on the host. All VMkernel adapters for vMotion on the default
TCP/IP stack are disabled for future vMotion sessions. If you use the Provisioning
TCP/IP stack, VMkernel adapters on the default TCP/IP stack are disabled for
operations that include the Provisioning traffic, such as virtual machine cold
migration, cloning, and snapshot migration.
Enable services You can enable services for the default TCP/IP stack on the host. Select from the
available services:
n vMotion traffic. Enables the VMkernel adapter to advertise itself to another
host as the network connection where vMotion traffic is sent. The migration
with vMotion to the selected host is not possible if the vMotion service is not
enabled for any VMkernel adapter on the default TCP/IP stack, or if no
adapters are using the vMotion TCP/IP stack.
n Provisioning traffic. Handles the data transferred for virtual machine cold
migration, cloning, and snapshot migration.
n Fault Tolerance traffic. Enables Fault Tolerance logging on the host. You
can use only one VMkernel adapter for FT traffic per host.
n Management traffic. Enables the management traffic for the host and
vCenter Server. Typically, hosts have such a VMkernel adapter created when
the ESXi software was installed. You can create another VMkernel adapter for
management traffic on the host to provide redundancy.
n vSphere Replication traffic.Handles the outgoing replication data that is
sent from the sourceESXi host to the vSphere Replication server.
n vSphere Replication NFC traffic. Handles the incoming replication data on
the target replication site.
n vSAN. Enables the vSAN traffic on the host. Every host that is part from a
vSAN cluster must have such a VMkernel adapter.
8 If you selected the vMotion TCP/IP or the Provisioning stack, click OK in the warning dialog that
appears.
If a live migration is already initiated, it completes successfully even after the involved VMkernel
adapters on the default TCP/IP stack are disabled for vMotion. Same refers to operations that include
VMkernel adapters on the default TCP/IP stack that are set for the Provisioning traffic.
9 (Optional) On the IPv4 settings page, select an option for obtaining IP addresses.
Option Description
Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network.
Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
The VMkernel Default Gateway and DNS server addresses for IPv4 are obtained
from the selected TCP/IP stack.
Select the Override default gateway for this adapter check box and enter a
gateway address, if you want to specify a different gateway for the VMkernel
adapter.
VMware, Inc. 64
vSphere Networking
10 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses.
Option Description
Obtain IPv6 addresses automatically Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the
through DHCP network.
Obtain IPv6 addresses automatically Use router advertisement to obtain IPv6 addresses.
through Router Advertisement In ESXi 6.5 and later router advertisement is enabled by default and supports the
M and O flags in accordance with RFC 4861.
Static IPv6 addresses a Click Add IPv6 address to add a new IPv6 address.
b Enter the IPv6 address and subnet prefix length, and click OK.
c To change the VMkernel default gateway, click Override default gateway for
this adapter.
The VMkernel Default Gateway address for IPv6 is obtained from the selected
TCP/IP stack.
11 Review your settings selections on the Ready to complete page and click Finish.
You should dedicate a single distributed port group per VMkernel adapter. For better isolation, you should
configure one VMkernel adapter with one traffic type.
Procedure
4 On the Select connection type page, select VMkernel Network Adapter and click Next.
5 From the Select an existing network option, select a distributed port group and click Next.
6 On the Port properties page, configure the settings for the VMkernel adapter.
Option Description
Network label The network label is inherited from the label of the distributed port group.
Note The IPv6 option does not appear on hosts that do not have IPv6 enabled.
VMware, Inc. 65
vSphere Networking
Option Description
TCP/IP stack Select a TCP/IP stack from the list. Once you set a TCP/IP stack for the VMkernel
adapter, you cannot change it later. If you select the vMotion or the Provisioning
TCP/IP stack, you will be able to use only these stacks to handle vMotion or
Provisioning traffic on the host. All VMkernel adapters for vMotion on the default
TCP/IP stack are disabled for future vMotion sessions. If you set the Provisioning
TCP/IP stack, VMkernel adapters on the default TCP/IP stack are disabled for
operations that include Provisioning traffic, such as virtual machine cold
migration, cloning, and snapshot migration.
Enable services You can enable services for the default TCP/IP stack on the host. Select from the
available services:
n vMotion traffic. Enables the VMkernel adapter to advertise itself to another
host as the network connection where vMotion traffic is sent. The migration
with vMotion to the selected host is not possible if the vMotion service is not
enabled for any VMkernel adapter on the default TCP/IP stack, or there are
no adapters using the vMotion TCP/IP stack.
n Provisioning traffic. Handles the data transferred for virtual machine cold
migration, cloning, and snapshot migration.
n Fault Tolerance traffic. Enables Fault Tolerance logging on the host. You
can use only one VMkernel adapter for FT traffic per host.
n Management traffic. Enables the management traffic for the host and
vCenter Server. Typically, hosts have such a VMkernel adapter created when
the ESXi software is installed. You can create another VMkernel adapter for
management traffic on the host to provide redundancy.
n vSphere Replication traffic. Handles the outgoing replication data that is
sent from the source ESXi host to the vSphere Replication server.
n vSphere Replication NFC traffic. Handles the incoming replication data on
the target replication site.
n vSAN. Enables thevSAN traffic on the host. Every host that is part of a vSAN
cluster must have such a VMkernel adapter.
7 If you selected the vMotion TCP/IP or the Provisioning stack, click OK in the warning dialog that
appears.
If a live migration is already initiated, it completes successfully even after the involved VMkernel
adapters on the default TCP/IP stack are disabled for vMotion. Same refers to operations that include
VMkernel adapters on the default TCP/IP stack that are set for the Provisioning traffic.
8 (Optional) On the IPv4 settings page, select an option for obtaining IP addresses.
Option Description
Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network.
Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
The VMkernel Default Gateway and DNS server addresses for IPv4 are obtained
from the selected TCP/IP stack.
Select the Override default gateway for this adapter check box and enter a
gateway address, if you want to specify a different gateway for the VMkernel
adapter.
VMware, Inc. 66
vSphere Networking
9 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses.
Option Description
Obtain IPv6 addresses automatically Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the
through DHCP network.
Obtain IPv6 addresses automatically Use router advertisement to obtain IPv6 addresses.
through Router Advertisement In ESXi 6.5 and later router advertisement is enabled by default and supports the
M and O flags in accordance with RFC 4861.
Static IPv6 addresses a Click Add IPv6 address to add a new IPv6 address.
b Enter the IPv6 address and subnet prefix length, and click OK.
c To change the VMkernel default gateway, click Override default gateway for
this adapter.
The VMkernel Default Gateway address for IPv6 is obtained from the selected
TCP/IP stack.
10 Review your settings selections on the Ready to complete page and click Finish.
Procedure
3 Select the VMkernel adapter that resides on the target distributed or standard switch and click Edit.
4 On the Port properties page, select the services that you want to enable.
vMotion traffic Enables the VMkernel adapter to advertise itself to another host as the network
connection where vMotion traffic is sent. If this property is not enabled for any
VMkernel adapter, migration with vMotion to the selected host is not possible.
Provisioning traffic Handles the data transferred for virtual machine cold migration, cloning, and
snapshot migration.
Fault Tolerance traffic Enables Fault Tolerance logging on the host. You can use only one VMkernel
adapter for FT traffic per host.
Management traffic Enables the management traffic for the host and vCenter Server. Typically, hosts
have such a VMkernel adapter created when the ESXi software was installed.
You can have an additional VMkernel adapter for management traffic on the host
to provide redundancy.
vSphere Replication traffic Handles the outgoing replication data that is sent from the sourceESXi host to the
vSphere Replication server.
vSphere Replication NFC traffic Handles the incoming replication data on the target replication site.
vSAN Enables vSAN traffic on the host. Every host that is part from a vSAN cluster must
have such a VMkernel adapter.
VMware, Inc. 67
vSphere Networking
5 On the NIC settings page, set the MTU for the network adapter.
6 With IPv4 enabled, in the IPv4 settings section, select the method by which IP addresses are
obtained.
Option Description
Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network.
Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
The VMkernel Default Gateway and DNS server addresses for IPv4 are obtained
from the selected TCP/IP stack.
Select the Override default gateway for this adapter check box and enter a
gateway address, if you want to specify a different gateway for the VMkernel
adapter.
7 With IPv6 enabled, in the IPv6 settings select an option for obtaining IPv6 addresses.
Note The IPv6 option does not appear on hosts that do not have IPv6 enabled.
Option Description
Obtain IPv6 addresses automatically Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the
through DHCP network.
Obtain IPv6 addresses automatically Use router advertisement to obtain IPv6 addresses.
through Router Advertisement In ESXi 6.5 and later router advertisement is enabled by default and supports the
M and O flags in accordance with RFC 4861.
Static IPv6 addresses a Click Add IPv6 address to add a new IPv6 address.
b Enter the IPv6 address and subnet prefix length, and click OK.
c To change the VMkernel default gateway, click Override default gateway for
this adapter.
The VMkernel Default Gateway address for IPv6 is obtained from the selected
TCP/IP stack.
On the IPv6 settings page, click Advanced settings to remove IPv6 addresses. If router advertisement
is enabled, removed addresses from this origin might reappear. Removal of DHCP addresses on the
VMkernel adapter is not supported. These addresses are removed only when the DHCP option is
turned off.
8 On the Analyze impact page, verify that the changes made to the VMKernel adapter will not disrupt
other operations.
9 Click OK.
Each TCP/IP stack on a host can have only one default gateway. This default gateway is part of the
routing table and all services that operate on the TCP/IP stack use it.
VMware, Inc. 68
vSphere Networking
For example, the VMkernel adapters vmk0 and vmk1 can be configured on a host.
n vmk0 is used for management traffic on the 10.162.10.0/24 subnet, with default gateway 10.162.10.1
If you set 172.16.1.1 as the default gateway for vmk1, vMotion uses vmk1 as its egress interface with the
gateway 172.16.1.1. The 172.16.1.1 gateway is a part of the vmk1 configuration and is not in the routing
table. Only the services that specify vmk1 as an egress interface use this gateway. This provides
additional Layer 3 connectivity options for services that need multiple gateways.
You can use the vSphere Web Client or an ESXCLI command to configure the default gateway of a
VMkernel adapter.
See Create a VMkernel Adapter on a vSphere Standard Switch, Create a VMkernel Adapter on a Host
Associated with a vSphere Distributed Switch, and Configure the VMkernel Adapter Gateway by Using
esxcli Commands.
Procedure
Option Description
IPv4
esxcli network ip interface ipv4 set –i vmknic -t static –g
IPv4 gateway -I IPv4 address -N mask
IPv6
Important You must turn off DHCPv6 or Router Advertisement before you can
set the IPv6 vmknic gateway.
VMware, Inc. 69
vSphere Networking
Where vmknic is the name of the VMkernel adapter, gateway is the IP address of the gateway, IP
address is the address of the VMkernel adapter, and mask is the network mask.
Procedure
If no custom TCP/IP stacks are configured on the host, you view the default, vMotion, and
Provisioning TCP/IP stacks on the host.
DNS and routing details about the selected TCP/IP stack appear below the TCP/IP Stacks table. You can
view the IPv4 and IPv6 routing tables, and the DNS and routing configuration for the stack.
Note The IPv6 routing table is only visible if IPv6 is enabled on the host.
The Advanced tab contains information about the configured congestion control algorithm and the
maximum number of allowed connections to the stack.
Note You can change the DNS and default gateway configuration of the default TCP/IP stack only.
Changing the DNS and default gateway configuration of custom TCP/IP stacks is not supported.
Procedure
VMware, Inc. 70
vSphere Networking
3 Select a stack from the table, click Edit and make the appropriate changes.
Page Option
Note Removing the default gateway might cause the client to lose connectivity with the host.
Advanced Edit the maximum number of connections and the congestion control algorithm of the stack
What to do next
You can add static routes to additional gateways by using CLI commands. For more information, see
http://kb.vmware.com/kb/2001426
Procedure
The custom TCP/IP stack is created on the host. You can assign VMkernel adapters to the stack.
VMware, Inc. 71
vSphere Networking
Procedure
3 Select a VMkernel adapter from the list, and click the Remove selected network adapter icon.
5 If you use software iSCSI adapters with port binding, review the impact on their networking
configuration.
Option Description
No impact iSCSI will continue its normal function after the new networking configuration is
applied.
Important impact The normal function of iSCSI might be disrupted if the new networking
configuration is applied.
Critical impact The normal function of iSCSI will be interrupted if the new networking
configuration is applied.
a If the impact on iSCSI is important or critical, click iSCSI entry and review the reasons that are
displayed in the Analysis details pane.
b Cancel the removal of the VMkernel adapter until you fix the reasons for any critical or important
impact on a service, or, if there are no impacted services, close the Analyze Impact dialog box.
6 Click OK.
VMware, Inc. 72
LACP Support on a vSphere
Distributed Switch 5
With LACP support on a vSphere Distributed Switch, you can connect ESXi hosts to physical switches by
using dynamic link aggregation. You can create multiple link aggregation groups (LAGs) on a distributed
switch to aggregate the bandwidth of physical NICs on ESXi hosts that are connected to LACP port
channels.
VMware, Inc. 73
vSphere Networking
vCenter Server
Physical Switch
When you create a LAG on a distributed switch, a LAG object is also created on the proxy switch of every
host that is connected to the distributed switch. For example, if you create LAG1 with two ports, LAG1
with the same number of ports is created on every host that is connected to the distributed switch.
VMware, Inc. 74
vSphere Networking
On a host proxy switch, you can connect one physical NIC to only one LAG port. On the distributed
switch, one LAG port can have multiple physical NICs from different hosts connected to it. The physical
NICs on a host that you connect to the LAG ports must be connected to links that participate in an LACP
port channel on the physical switch.
You can create up to 64 LAGs on a distributed switch. A host can support up to 32 LAGs. However, the
number of LAGs that you can actually use depends on the capabilities of the underlying physical
environment and the topology of the virtual network. For example, if the physical switch supports up to
four ports in an LACP port channel, you can connect up to four physical NICs per host to a LAG.
n The number of ports in the LACP port channel must be equal to the number of physical NICs that you
want to group on the host. For example, if you want to aggregate the bandwidth of two physical NICs
on a host, you must create an LACP port channel with two ports on the physical switch. The LAG on
the distributed switch must be configured with at least two ports.
n The hashing algorithm of the LACP port channel on the physical switch must be the same as the
hashing algorithm that is configured to the LAG on the distributed switch.
n All physical NICs that you want to connect to the LACP port channel must be configured with the
same speed and duplex.
n Configure a Link Aggregation Group to Handle the Traffic for Distributed Port Groups
VMware, Inc. 75
vSphere Networking
Table 5‑1. LACP Teaming and failover configuration of distributed port groups
Failover Order Uplinks Description
Active A single LAG You can only use one active LAG or multiple standalone uplinks to
handle the traffic of distributed port groups . You cannot configure
multiple active LAGs or mix active LAGs and standalone uplinks.
Standby Empty Having an active LAG and standby uplinks and the reverse is not
supported. Having a LAG and another standby LAG is not
supported.
Unused All standalone uplinks and other Because only one LAG must be active and the Standby list must
LAGs if any be empty, you must set all standalone uplinks and other LAGs to
unused.
Newly created LAGs do not have physical NICs assigned to their ports and are unused in the teaming
and failover order of distributed port groups. To handle the network traffic of distributed port groups by
using a LAG, you must migrate the traffic from standalone uplinks to the LAG.
Prerequisites
n Verify that for every host where you want to use LACP, a separate LACP port channel exists on the
physical switch. See Chapter 5 LACP Support on a vSphere Distributed Switch.
n Verify that the vSphere Distributed Switch where you configure the LAG is version 6.0 or later.
Procedure
1 Create a Link Aggregation Group
To migrate the network traffic of distributed port groups to a link aggregation group (LAG), you create
a new LAG on the distributed switch.
2 Set a Link Aggregating Group as Standby in the Teaming and Failover Order of Distributed Port
Groups
The new link aggregation group (LAG) by default is unused in the teaming and failover order of
distributed port groups. Because only one LAG or only standalone uplinks can be active for
distributed port groups, you must create an intermediate teaming and failover configuration, where
the LAG is standby. This configuration lets you migrate physical NICs to the LAG ports by keeping
the network connectivity up.
VMware, Inc. 76
vSphere Networking
4 Set the Link Aggregation Group as Active in the Teaming and Failover Order of the Distributed Port
Group
You migrated physical NICs to the ports of the link aggregation group (LAG). Set the LAG as active
and move all standalone uplinks as unused in the teaming and failover order of the distributed port
groups.
Procedure
Set the same number of ports to the LAG as the number of ports in the LACP port channel on the
physical switch. A LAG port has the same function as an uplink on the distributed switch. All LAG
ports form a NIC team in the context of the LAG.
Option Description
Active All LAG ports are in an Active negotiating mode. The LAG ports initiate
negotiations with the LACP port channel on the physical switch by sending LACP
packets.
Passive The LAG ports are in Passive negotiating mode. They respond to LACP packets
they receive but do not initiate LACP negotiation.
If the LACP-enabled ports on the physical switch are in Active negotiating mode, you can set the LAG
ports in Passive mode and the reverse.
7 Select a load balancing mode from the hashing algorithms that LACP defines.
Note The hashing algorithm must be the same as the hashing algorithm set to the LACP port
channel on the physical switch.
8 Set the VLAN and the NetFlow policies for the LAG.
This option is active when overriding the VLAN and NetFlow policies per individual uplink ports is
enabled on the uplink port group. If you set the VLAN and NetFlow policies to the LAG, they override
the policies set on the uplink port group level.
9 Click OK.
VMware, Inc. 77
vSphere Networking
The new LAG is unused in the teaming and failover order of distributed port groups. No physical NICs are
assigned to the LAG ports.
As with standalone uplinks, the LAG has a representation on every host that is associated with the
distributed switch. For example, if you create LAG1 with two ports on the distributed switch, a LAG1 with
two ports is created on every host that is associated with the distributed switch.
What to do next
Set the LAG as standby in the teaming and failover configuration of distributed port groups. In this way,
you create an intermediate configuration that lets you migrate the network traffic to the LAG without losing
network connectivity.
Procedure
2 From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups.
4 Select the port groups where you want to use the LAG.
5 In Failover order, select the LAG and use the up arrow to move it to the Standby uplinks list.
6 Click Next, review the message that informs you about the usage of the intermediate teaming and
failover configuration, and click OK.
What to do next
Prerequisites
n Verify that either all LAG ports or the corresponding LACP-enabled ports on the physical switch are in
active LACP negotiating mode.
VMware, Inc. 78
vSphere Networking
n Verify that the physical NICs that you want to assign to the LAG ports have the same speed and are
configured at full duplex.
Procedure
1 In the vSphere Web Client, navigate to the distributed switch where the LAG resides.
4 Select the host whose physical NICs you want to assign to the LAG ports and click Next.
5 On the Select network adapter tasks page, select Manage physical adapters and click Next.
6 On the Manage physical network adapters page, select a NIC and click Assign an uplink.
8 Repeat Step 6 and Step 7 for all physical NICs that you want to assign to the LAG ports.
Example: Configure Two Physical NICs to a LAG in the Add and Manage
Hosts Wizard
For example, if you have a LAG with two ports, you configure a physical NIC to each LAG port in the Add
and Manage Hosts wizard.
What to do next
Set the LAG as active and all standalone uplinks to unused in the teaming and failover order of distributed
port groups.
Procedure
2 From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups.
4 Select the port groups where you set the LAG as standby and click Next.
5 In Failover order, use the up and down arrows to move the LAG in the Active list, all standalone
uplinks in the Unused list, and leave the Standby list empty.
VMware, Inc. 79
vSphere Networking
You safely migrated network traffic from standalone uplinks to a LAG for distributed port groups and
created a valid LACP teaming and failover configuration for the groups.
Procedure
4 In the Name text box, type a new name for the LAG.
5 Change the number of ports for the LAG if you want to add more physical NICs to it.
The new NICs must be connected to ports that are part of an LACP port channel on the physical
switch.
If all ports on the physical LACP port channel are in Active LACP mode, you can change the LACP
mode of the LAG to Passive and the reverse.
You can select from the load balancing algorithms that LACP defines.
VMware, Inc. 80
vSphere Networking
This option is active when the option for overriding the VLAN and NetFlow policies for individual ports
is enabled on the uplink port group. If you change the VLAN and NetFlow policies for the LAG, they
override the policies set at the uplink port group level.
9 Click OK.
n The LACP is not supported with software iSCSI port binding. iSCSI multipathing over LAG is
supported, if port binding is not used.
n The LACP support does not work with the ESXi dump collector.
n The LACP control packets (LACPDU) do not get mirrored when port mirroring is enabled.
n The teaming and failover health check does not work for LAG ports. LACP checks the connectivity of
the LAG ports.
n The enhanced LACP support works correctly when only one LAG handles the traffic per distributed
port or port group.
VMware, Inc. 81
Backing Up and Restoring
Networking Configurations 6
vSphere enables you to backup and restore the configuration of a vSphere Distributed Switch , distributed
and uplink port groups in cases of invalid changes or a transfer to another deployment.
You can import or export a configuration of a distributed switch including its port groups. For information
about exporting, importing, and restoring a port group configuration, see Export, Import, and Restore
vSphere Distributed Port Group Configurations.
Note You can use a saved configuration file to restore policies and hosts associations on the distributed
switch. You cannot restore the connection of physical NICs to uplink ports or ports of link aggregation
groups.
Procedure
2 Right-click the distributed switch and select Settings > Export Configuration.
3 Choose to export the distributed switch configuration, or export the distributed switch configuration
and all port groups.
VMware, Inc. 82
vSphere Networking
5 Click OK.
What to do next
n Create a copy of the exported distributed switch in a vSphere environment. See Import a vSphere
Distributed Switch Configuration.
n Overwrite the settings on an existing distributed switch. See Restore a vSphere Distributed Switch
Configuration.
You can also export, import, and restore only port group configurations. See Export, Import, and Restore
vSphere Distributed Port Group Configurations.
The configuration file contains the networking settings of the switch. By using it you can also replicate the
switch in other virtual environments.
Note You can use a saved configuration file to replicate the switch instance, its host associations, and
policies. You cannot replicate the connection of physical NICs to uplink ports or ports on link aggregation
groups.
Procedure
2 Right-click the data center and select Distributed Switch > Import Distributed Switch.
4 To assign the keys from the configuration file to the switch and its port groups, select the Preserve
original distributed switch and port group identifiers check box and click Next.
You can use the Preserve original distributed switch and port group identifiers option in the
following cases:
All port groups are recreated and the hosts that have been connected to the switch are added again.
A new distributed switch is created with settings from the configuration file. If you have included
distributed port group information in the configuration file, the port groups are also created.
VMware, Inc. 83
vSphere Networking
Note You can use a saved configuration file to restore policies and hosts associations on the distributed
switch. You cannot restore the connection of physical NICs to uplink ports or ports of link aggregation
groups.
Procedure
2 Right-click the distributed switch in the navigator and select Settings > Restore Configuration.
4 Select Restore distributed switch and all port groups or Restore distributed switch only and
click Next
Restoring a distributed switch will overwrite the current settings of the distributed switch and its port
groups. It will not delete existing port groups that are not part of the configuration file.
6 Click Finish.
The distributed switch configuration has been restored to the settings in the configuration file.
You can export port group information at the same time you export distributed switch configurations. See
Backing Up and Restoring a vSphere Distributed Switch Configuration.
Procedure
VMware, Inc. 84
vSphere Networking
4 Click OK.
You now have a configuration file that contains all the settings for the selected distributed port group. You
can use this file to create multiple copies of this configuration on an existing deployment, or overwrite
settings of existing distributed port groups to conform to the selected settings.
What to do next
You can use the exported configuration file to do the following tasks:
n To create a copy of the exported distributed port group, see Import a vSphere Distributed Port Group
Configuration.
n To overwrite settings on an existing distributed port group, see Restore a vSphere Distributed Port
Group Configuration.
If an existing port group has the same name as the imported port group, the new port group name has a
number appended in parentheses. The settings from the imported configuration are applied to the new
port group and the settings of the original port group remain unchanged.
Procedure
2 Right-click the distributed switch and select Distributed Port Group > Import Distributed Port
Group.
3 Browse to the location of your saved configuration file and click Next.
5 Click Finish.
Procedure
VMware, Inc. 85
vSphere Networking
u Restore to previous configuration to roll your port group configuration back one step. You
cannot restore the port group configuration completely if you have performed more than one step.
u Restore configuration from a file lets you restore the port group configuration from an exported
backup file. You can also use a distributed switch backup file as long as it contains configuration
information for the port group.
The restore operation overwrites the current settings of the distributed port group with the settings
from the backup. If you are restoring the port group configuration from a switch backup file, the
restore operation does not delete existing port groups that are not a part of the file.
5 Click Finish.
VMware, Inc. 86
Rollback and Recovery of the
Management Network 7
You can prevent and recover from misconfiguration of the management network by using the rollback and
recovery support of the vSphere Distributed Switch and vSphere Standard Switch.
Rollback is available for use on both standard and distributed switches. To fix invalid configuration of the
management network, you can connect directly to a host to fix the issues through the DCUI.
In vSphere networking rollback is enabled by default. However, you can enable or disable rollbacks at the
vCenter Server level.
n Updating teaming and failover policies or traffic shaping policies of a standard port group that
contains the management VMkernel network adapter.
n Updating the VLAN of a standard port group that contains the management VMkernel network
adapter.
n Increasing the MTU of management VMkernel network adapter and its switch to values not supported
by the physical infrastructure.
VMware, Inc. 87
vSphere Networking
n Removing the management VMkernel network adapter from a standard or distributed switch.
n Removing a physical NIC of a standard or distributed switch containing the management VMkernel
network adapter.
n Migrating the management VMkernel adapter from vSphere standard to distributed switch.
If a network disconnects for any of these reasons, the task fails and the host reverts to the last valid
configuration.
n Changing the following settings in the distributed port group of the management VMkernel network
adapter:
n VLAN
n Traffic shaping
n Blocking all ports in the distributed port group containing the management VMkernel network adapter.
n Overriding the policies on at the level of the distributed port for the management VMkernel network
adapter.
If a configuration becomes invalid because of any of the changes, one or more hosts might become out of
synchronization with the distributed switch.
If you know where the conflicting configuration setting is located, you can manually correct the setting.
For example, if you have migrated a management VMkernel network adapter to a new VLAN, the VLAN
might not be actually trunked on the physical switch. When you correct the physical switch configuration,
the next distributed switch-to-host synchronization will resolve the configuration problem.
If you are not sure where the problem exists, you can restore the state of the distributed switch or
distributed port group to an earlier configuration. See Restore a vSphere Distributed Port Group
Configuration.
Procedure
VMware, Inc. 88
vSphere Networking
3 Click Edit.
If the key is not present, you can add it and set the value to false.
5 Click OK.
Procedure
1 On the host machine of vCenter Server, navigate to the directory that contains the configuration file:
<config>
<vpxd>
<network>
<rollback>false</rollback>
</network>
</vpxd>
</config>
If networking rollback is disabled, misconfiguring the port group for the management network on the
distributed switch leads to loss of connection between vCenter Server and the hosts that are added to the
switch. You have to use the DCUI to connect each host individually.
VMware, Inc. 89
vSphere Networking
If the uplinks that you use to restore the management network are also used by VMkernel adapters that
handle other types of traffic (vMotion, Fault Tolerance, and so on), the adapters loose network
connectivity after the restore.
For more information about accessing and using the DCUI, see the vSphere Security documentation.
Note Recovery of the management connection on a distributed switch is not supported on stateless
ESXi instances.
Prerequisites
Verify that the management network is configured on a port group on the distributed switch.
Procedure
3 Configure the uplinks and optionally the VLAN for the management network.
The DCUI creates a local ephemeral port and applies the values you provided for the VLAN and uplinks.
The DCUI moves the VMkernel adapter for the management network to the new local port to restore
connectivity to vCenter Server.
What to do next
After the connection of the host to vCenter Server is restored, correct the configuration of the distributed
port group and re-add the VMkernel adapter to the group.
VMware, Inc. 90
Networking Policies 8
Policies set at the standard switch or distributed port group level apply to all of the port groups on the
standard switch or to ports in the distributed port group. The exceptions are the configuration options that
are overridden at the standard port group or distributed port level.
Watch the video about applying networking policies on vSphere standard and distributed switches.
n VLAN Policy
VLAN policies determine how VLANs function across your network environment.
n Security Policy
Networking security policy provides protection of traffic against MAC address impersonation and
unwanted port scanning
VMware, Inc. 91
vSphere Networking
n Monitoring Policy
The monitoring policy enables or disables NetFlow monitoring on a distributed port or port group.
vSphere Standard Switch Entire switch When you apply policies on the entire standard
switch, the policies are propagated to all standard
port groups on the switch.
Standard port group You can apply different policies on individual port
groups by overriding the policies that are inherited
from the switch.
vSphere Distributed Switch Distributed port group When you apply policies on a distributed port group,
the policies are propagated to all ports in the group.
Uplink port group You can apply policies at uplink port group level, and
the are policies are propagated to all ports in the
group.
VMware, Inc. 92
vSphere Networking
Table 8‑2. Policies Available for a vSphere Standard Switch and vSphere Distributed Switch
Standard Distributed
Policy Switch Switch Description
Teaming and Yes Yes Lets you configure the physical NICs that handle the network traffic for
failover a standard switch, standard port group, distributed port group, or
distributed port. You arrange the physical NICs in a failover order and
apply different load balancing policies over them.
Security Yes Yes Provides protection of traffic against MAC address impersonation and
unwanted port scanning. The networking security policy is
implemented in Layer 2 of the networking protocol stack.
Traffic shaping Yes Yes Lets you restrict the network bandwidth that is available to ports, but
also to allow bursts of traffic to flow through at higher speeds. ESXi
shapes outbound network traffic on standard switches and inbound
and outbound traffic on distributed switches.
VLAN Yes Yes Lets you configure the VLAN tagging for a standard or distributed
switch. You can configure External Switch Tagging(EST), Virtual
Switch Tagging (VST), and Virtual Guest Tagging (VGT).
Monitoring No Yes Enables and disables NetFlow monitoring on a distributed port or port
group.
Traffic filtering and No Yes Lest you protect the virtual network from unwanted traffic and security
marking attacks or apply a QoS tag to a certain traffic type.
Resources No Yes Lets you associate a distributed port or port group with a user-defined
allocation network resource pool. In this way, you can better control the
bandwidth that is available to the port or port group. You can use the
resource allocation policy with vSphere Network I/O Control version 2
and 3.
Port blocking No Yes Lets you selectively block ports from sending and receiving data.
Procedure
VMware, Inc. 93
vSphere Networking
Option Description
Configure reset at disconnect From the drop-down menu, enable or disable reset at disconnect.
When a distributed port is disconnected from a virtual machine, the configuration
of the distributed port is reset to the distributed port group setting. Any per-port
overrides are discarded.
Override port policies Select the distributed port group policies to be overridden on a per-port level.
4 (Optional) Use the policy pages to set overrides for each port policy.
5 Click OK.
Note All ports on the physical switch in the same team must be in the same Layer 2 broadcast domain.
For more information about each load balancing algorithm, see Load Balancing Algorithms Available for
Virtual Switches.
VMware, Inc. 94
vSphere Networking
Link status only Relies only on the link status that the network adapter provides. Detects
failures, such as removed cables and physical switch power failures.
However, link status does not detect the following configuration errors:
Beacon probing Sends out and listens for Ethernet broadcast frames, or beacon probes,
that physical NICs send to detect link failure in all physical NICs in a team.
ESXi hosts send beacon packets every second. Beacon probing is most
useful to detect failures in the closest physical switch to the ESXi host,
where the failure does not cause a link-down event for the host.
Use beacon probing with three or more NICs in a team because ESXi can
detect failures of a single adapter. If only two NICs are assigned and one of
them loses connectivity, the switch cannot determine which NIC needs to
be taken out of service because both do not receive beacons and as a
result all packets sent to both uplinks. Using at least three NICs in such a
team allows for n-2 failures where n is the number of NICs in the team
before reaching an ambiguous situation.
Failback Policy
By default, a failback policy is enabled on a NIC team. If a failed physical NIC returns online, the virtual
switch sets the NIC back to active by replacing the standby NIC that took over its slot.
If the physical NIC that stands first in the failover order experiences intermittent failures, the failback
policy might lead to frequent changes in the NIC that is used. The physical switch sees frequent changes
in MAC addresses, and the physical switch port might not accept traffic immediately when an adapter
becomes online. To minimize such delays, you might consider changing the following settings on the
physical switch:
n Disable Spanning Tree Protocol (STP) on physical NICs that are connected to ESXi hosts .
n For Cisco based networks, enable PortFast mode for access interfaces or PortfFast trunk mode for
trunk interfaces. This might save about 30 seconds during the initialization of the physical switch port.
VMware, Inc. 95
vSphere Networking
Route Based on Originating Virtual Portis the default load balancing method on the vSphere Standard
Switch and vSphere Distributed Switch.
VMware, Inc. 96
vSphere Networking
Each virtual machine running on an ESXi host has an associated virtual port ID on the virtual switch. To
calculate an uplink for a virtual machine, the virtual switch uses the virtual machine port ID and the
number of uplinks in the NIC team. After the virtual switch selects an uplink for a virtual machine, it always
forwards traffic through the same uplink for this virtual machine as long as the machine runs on the same
port. The virtual switch calculates uplinks for virtual machines only once, unless uplinks are added or
removed from the NIC team.
The port ID of a virtual machine is fixed while the virtual machine runs on the same host. If you migrate,
power off, or delete the virtual machine, its port ID on the virtual switch becomes free. The virtual switch
stops sending traffic to this port, which reduces the overall traffic for its associated uplink. If a virtual
machine is powered on or migrated, it might appear on a different port and use the uplink, which is
associated with the new port.
Disadvantages n The virtual switch is not aware of the traffic load on the
uplinks and it does not load balance the traffic to uplinks
that are less used.
n The bandwidth that is available to a virtual machine is
limited to the speed of the uplink that is associated with the
relevant port ID, unless the virtual machine has more than
one virtual NIC.
VMware, Inc. 97
vSphere Networking
To calculate an uplink for a virtual machine, the virtual switch takes the last octet of both source and
destination IP addresses in the packet, puts them through a XOR operation, and then runs the result
through another calculation based on the number of uplinks in the NIC team. The result is a number
between 0 and the number of uplinks in the team minus one. For example if a NIC team has four uplinks,
the result is a number between 0 and 3 as each number is associated with a NIC in the team. For non-IP
packets, the virtual switch takes two 32-bit binary values from the frame or packet from where the IP
address would be located.
Any virtual machine can use any uplink in the NIC team depending on the source and destination IP
address. In this way, each virtual machine can use the bandwidth of any uplink in the team. If a virtual
machine runs in an environment with a large number of independent virtual machines, the IP hash
algorithm can provide an even spread of the traffic between the NICs in the team. When a virtual machine
communicates with multiple destination IP addresses, the virtual switch can generate a different hash for
each destination IP. In this way, packets can use different uplinks on the virtual switch that results in
higher potential throughput.
However, if your environment has a small number of IP addresses, the virtual switch might consistently
pass the traffic through one uplink in the team. For example, if you have a database server that is
accessed by one application server, the virtual switch always calculates the same uplink, because only
one source-destination pair exists.
VMware, Inc. 98
vSphere Networking
To ensure that IP hash load balancing works correctly, you must have an Etherchannel configured on the
physical switch. An Etherchannel bonds multiple network adapters into a single logical link. When ports
are bound into an Etherchannel, every time the physical switch receives a packet from the same virtual
machine MAC address on different ports, the switch updates its content addressable memory (CAM) table
correctly.
For example, if the physical switch receives packets on ports 01 and 02 from MAC address A, the switch
makes a 01-A and a 02-A entry in its CAM table. As a result, the physical switch distributes the incoming
traffic to the correct ports. Without an Etherchannel, the physical switch first makes a record that a packet
from MAC address A is received on port 01, then updates the same record that a packet from MAC
address A is received on port 02. Hence, the physical switch forwards incoming traffic only on port 02,
and might result in packets not reaching their destination and overloading the corresponding uplink.
n ESXi hosts support IP hash teaming on a single physical switch or stacked switches.
n ESXi hosts support only 802.3ad link aggregation in Static mode . You can only use a static
Etherchannel with vSphere Standard Switches. LACP is not supported. If you enable IP hash load
balancing without 802.3ad link aggregation and the reverse, you might experience networking
disruptions.
n You must use Link Status Only as network failure detection with IP hash load balancing.
n You must set all uplinks from the team in the Active failover list . The Standby and Unused lists must
be empty.
n The number of ports in the Etherchannel must be same as the number of uplinks in the team.
Considerations Description
VMware, Inc. 99
vSphere Networking
The distributed switch calculates uplinks for virtual machines by taking their port ID and the number of
uplinks in the NIC team. The distributed switch tests the uplinks every 30 seconds, and if their load
exceeds 75 percent of usage, the port ID of the virtual machine with the highest I/O is moved to a different
uplink.
Configure NIC teaming, failover, and load balancing depending on the network configuration on the
physical switch and the topology of the standard switch. See Teaming and Failover Policy and Load
Balancing Algorithms Available for Virtual Switches for more information.
If you configure the teaming and failover policy on a standard switch, the policy is propagated to all port
groups in the switch. If you configure the policy on a standard port group, it overrides the policy inherited
from the switch.
Procedure
3 Navigate to the Teaming and Failover policy for the standard switch, or standard port group.
Option Action
Standard port group a Select the switch where the port group resides.
b From the switch topology diagram, select the standard port group and click
Edit settings.
c Select Teaming and failover.
d Select Override next to the policies that you want to override.
4 From the Load balancing drop-down menu, specify how the virtual switch load balances the
outgoing traffic between the physical NICs in a team.
Option Description
Route based on the originating virtual Select an uplink based on the virtual port IDs on the switch. After the virtual
port switch selects an uplink for a virtual machine or a VMkernel adapter, it always
forwards traffic through the same uplink for this virtual machine or VMkernel
adapter.
Route based on IP hash Select an uplink based on a hash of the source and destination IP addresses of
each packet. For non-IP packets, the switch uses the data at those fields to
compute the hash .
IP-based teaming requires that the physical switch is configured with
EtherChannel.
Route based on source MAC hash Select an uplink based on a hash of the source Ethernet.
Use explicit failover order From the list of active adapters, always use the highest order uplink that passes
failover detection criteria. No actual load balancing is performed with this option.
5 From the Network failure detection drop-down menu, select the method that the virtual switch uses
for failover detection.
Option Description
Link status only Relies only on the link status that the network adapter provides. This option
detects failures such as removed cables and physical switch power failures.
Beacon probing Sends out and listens for beacon probes on all NICs in the team, and uses this
information, in addition to link status, to determine link failure.ESXi sends beacon
packets every second.
The NICs must be in an active/active or active/standby configuration because the
NICs in an unused state do not participate in beacon probing.
6 From the Notify switches drop-down menu, select whether the standard or distributed switch notifies
the physical switch in case of a failover.
Note Set this option to No if a connected virtual machine is using Microsoft Network Load Balancing
in unicast mode. No issues exist with Network Load Balancing running in multicast mode.
7 From the Failback drop-down menu, select whether a physical adapter is returned to active status
after recovering from a failure.
If failback is set to Yes, the default selection, the adapter is returned to active duty immediately upon
recovery, displacing the standby adapter that took over its slot, if any.
If failback is set to No for a standard port, a failed adapter is left inactive after recovery until another
currently active adapter fails and must be replaced.
8 Specify how the uplinks in a team are used when a failover occurs by configuring the Failover Order
list.
If you want to use some uplinks but reserve others for emergencies in case the uplinks in use fail, use
the up and down arrow keys to move uplinks into different groups.
Option Description
Active adapters Continue to use the uplink if the network adapter connectivity is up and active.
Standby adapters Use this uplink if one of the active physical adapter is down.
9 Click OK.
Configure NIC teaming, failover, and load balancing according with the network configuration on the
physical switch and the topology of the distributed switch. See Teaming and Failover Policy and Load
Balancing Algorithms Available for Virtual Switches for more information.
If you configure the teaming and failover policy for a distributed port group, the policy is propagated to all
ports in the group. If you configure the policy for a distributed port, it overrides the policy inherited from
the group.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
2 Navigate the Teaming and Failover policy on the distributed port group or port.
Option Action
Distributed port group a From the Actions menu, select Distributed Port Group > Manage
Distributed Port Groups.
b Select Teaming and failover.
c Select the port group and click Next.
Distributed port a On the Networks tab, click Distributed Port Groups and double-click a
distributed port group.
b On the Ports tab, select a port and click Edit distributed port settings.
c Select Teaming and failover.
d Select Override next to the properties that you want to override.
3 From the Load balancing drop-down menu, specify how the virtual switch load balances the
outgoing traffic between the physical NICs in a team.
Option Description
Route based on the originating virtual Select an uplink based on the virtual port IDs on the switch. After the virtual
port switch selects an uplink for a virtual machine or a VMkernel adapter, it always
forwards traffic through the same uplink for this virtual machine or VMkernel
adapter.
Route based on IP hash Select an uplink based on a hash of the source and destination IP addresses of
each packet. For non-IP packets, the switch uses the data at those fields to
compute the hash .
IP-based teaming requires that the physical switch is configured with
EtherChannel.
Route based on source MAC hash Select an uplink based on a hash of the source Ethernet.
Route based on physical NIC load Available for distributed port groups or distributed ports. Select an uplink based
on the current load of the physical network adapters connected to the port group
or port. If an uplink remains busy at 75 percent or higher for 30 seconds, the host
proxy switch moves a part of the virtual machine traffic to a physical adapter that
has free capacity.
Use explicit failover order From the list of active adapters, always use the highest order uplink that passes
failover detection criteria. No actual load balancing is performed with this option.
4 From the Network failure detection drop-down menu, select the method that the virtual switch uses
for failover detection.
Option Description
Link status only Relies only on the link status that the network adapter provides. This option
detects failures such as removed cables and physical switch power failures.
Beacon probing Sends out and listens for beacon probes on all NICs in the team, and uses this
information, in addition to link status, to determine link failure.ESXi sends beacon
packets every second.
The NICs must be in an active/active or active/standby configuration because the
NICs in an unused state do not participate in beacon probing.
5 From the Notify switches drop-down menu, select whether the standard or distributed switch notifies
the physical switch in case of a failover.
Note Set this option to No if a connected virtual machine is using Microsoft Network Load Balancing
in unicast mode. No issues exist with Network Load Balancing running in multicast mode.
6 From the Failback drop-down menu, select whether a physical adapter is returned to active status
after recovering from a failure.
If failback is set to Yes, the default selection, the adapter is returned to active duty immediately upon
recovery, displacing the standby adapter that took over its slot, if any.
If failback is set to No for a distributed port, a failed adapter is left inactive after recovery only if the
associated virtual machine is running. When the Failback option is No and a virtual machine is
powered off, if all active physical adapters fail and then one of them recovers, the virtual NIC is
connected to the recovered adapter instead of to a standby one after the virtual machine is powered
on. Powering a virtual machine off and then on leads to reconnecting the virtual NIC to a distributed
port. The distributed switch considers the port as newly added, and assigns it the default uplink port,
that is, the active uplink adapter.
7 Specify how the uplinks in a team are used when a failover occurs by configuring the Failover Order
list.
If you want to use some uplinks but reserve others for emergencies in case the uplinks in use fail, use
the up and down arrow keys to move uplinks into different groups.
Option Description
Active adapters Continue to use the uplink if the network adapter connectivity is up and active.
Standby adapters Use this uplink if one of the active physical adapter is down.
VLAN Policy
VLAN policies determine how VLANs function across your network environment.
A virtual local area network (VLAN) is a group of hosts with a common set of requirements, which
communicate as if they were attached to the same broadcast domain, regardless of their physical
location. A VLAN has the same attributes as a physical local area network (LAN), but it allows for end
stations to be grouped together even if not on the same network switch.
The scope of VLAN policies can be distributed port groups and ports, and uplink port groups and ports.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
2 Navigate to the VLAN policy on the distributed port group or distributed port.
Option Action
Distributed port group a From the Actions menu, select Distributed Port Group > Manage
Distributed Port Groups.
b Select VLAN and click Next.
c Select the port group and click Next.
Distributed port a On the Networks tab, click Distributed Port Groups and double-click a
distributed port group.
b On the Ports tab, select a port and click the Edit distributed port settings
icon.
c Select VLAN.
d Select Override next to the properties to override.
3 From the VLAN type drop-down menu, select the type of VLAN traffic filtering and marking, and click
Next.
Option Description
VLAN Trunking Pass VLAN traffic with ID within the VLAN trunk range to guest operating
system. You can set multiple ranges and individual VLANs by using a comma-
separated list. For example: 1702-1705, 1848-1849.
Use this option for Virtual Guest Tagging.
Private VLAN Associate the traffic with a private VLAN created on the distributed switch.
Use the VLAN policy at the uplink port level to propagate a trunk range of VLAN IDs to the physical
network adapters for traffic filtering. The physical network adapters drop the packets from the other
VLANs if the adapters support filtering by VLAN. Setting a trunk range improves networking performance
because physical network adapters filter traffic instead of the uplink ports in the group.
If you have a physical network adapter that does not support VLAN filtering, the VLANs still might not be
blocked. In this case, configure VLAN filtering on a distributed port group or on a distributed port.
For information about VLAN filtering support, see the technical documentation from the adapter vendors.
Prerequisites
To override the VLAN policy at the port level, enable the port-level overrides. See Configure Overriding
Networking Policies on Port Level.
Procedure
Option Action
Uplink port group a Right-click an uplink port group in the list and select Edit Settings.
b Click VLAN.
4 Type a VLAN trunk range value to propagate to the physical network adapters.
For trunking of several ranges and individual VLANs, separate the entries with commas.
5 Click OK.
Security Policy
Networking security policy provides protection of traffic against MAC address impersonation and
unwanted port scanning
The security policy of a standard or distributed switch is implemented in Layer 2 (Data Link Layer) of the
network protocol stack. The three elements of the security policy are promiscuous mode, MAC address
changes, and forged transmits. See the vSphere Security documentation for information about potential
networking threats.
Procedure
3 Navigate to the Security policy for the standard switch or port group.
Option Action
Standard port group a Select the standard switch where the port group resides.
b In the topology diagram, select a standard port group.
c Click Edit settings.
d Select Security and select Override next to the options to override.
4 Reject or accept promiscuous mode activation or MAC address changes in the guest operating
system of the virtual machines attached to the standard switch or port group.
Option Description
Promiscuous mode n Reject. The VM network adapter receives only frames that are addressed to
the virtual machine.
n Accept.The virtual switch forwards all frames to the virtual machine in
compliance with the active VLAN policy for the port to which the VM network
adapter is connected.
MAC address changes n Reject. If the guest OS changes the effective MAC address of the virtual
machine to a value that is different from the MAC address of the VM network
adapter (set in the .vmx configuration file), the switch drops all inbound
frames to the adapter.
If the guest OS changes the effective MAC address of the virtual machine
back to the MAC address of the VM network adapter, the virtual machine
receives frames again.
n Accept. If the guest OS changes the effective MAC address of the virtual
machine to a value that is different from the MAC address of the VM network
adapter, the switch allows frames to the new address to pass.
Forged transmits n Reject. The switch drops any outbound frame from a virtual machine adapter
with a source MAC address that is different from the one in the .vmx
configuration file.
n Accept. The switch does not perform filtering, and permits all outbound
frames.
5 Click OK.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
2 Navigate to the Security policy for the distributed port group or port.
Option Action
Distributed port group a From the Actions menu, select Distributed Port Group > Manage
Distributed Port Groups.
b Select Security.
c Select the port group and click Next.
Distributed port a On the Networks tab, click Distributed Port Groups and double-click a
distributed port group .
b On the Ports tab, select a port and click the Edit distributed port settings
icon.
c Select Security.
d Select Override next to the properties to override.
3 Reject or accept promiscuous mode activation or MAC address changes in the guest operating
system of the virtual machines attached to the distributed port group or port.
Option Description
Promiscuous mode n Reject. The VM network adapter receives only frames that are addressed to
the virtual machine.
n Accept.The virtual switch forwards all frames to the virtual machine in
compliance with the active VLAN policy for the port to which the VM network
adapter is connected.
MAC address changes n Reject. If the guest OS changes the effective MAC address of the virtual
machine to a value that is different from the MAC address of the VM network
adapter (set in the .vmx configuration file), the switch drops all inbound
frames to the adapter.
If the guest OS changes the effective MAC address of the virtual machine
back to the MAC address of the VM network adapter, the virtual machine
receives frames again.
n Accept. If the guest OS changes the effective MAC address of the virtual
machine to a value that is different from the MAC address of the VM network
adapter, the switch allows frames to the new address to pass.
Forged transmits n Reject. The switch drops any outbound frame from a virtual machine adapter
with a source MAC address that is different from the one in the .vmx
configuration file.
n Accept. The switch does not perform filtering, and permits all outbound
frames.
ESXi shapes outbound network traffic on standard switches and inbound and outbound traffic on
distributed switches. Traffic shaping restricts the network bandwidth available on a port, but can also be
configured to allow bursts of traffic to flow through at higher speeds.
Average Bandwidth Establishes the number of bits per second to allow across a port, averaged
over time. This number is the allowed average load.
Peak Bandwidth Maximum number of bits per second to allow across a port when it is
sending or receiving a burst of traffic. This number limits the bandwidth that
a port uses when it is using its burst bonus.
Burst Size Maximum number of bytes to allow in a burst. If this parameter is set, a port
might gain a burst bonus if it does not use all its allocated bandwidth. When
the port needs more bandwidth than specified by the average bandwidth, it
might be allowed to temporarily transmit data at a higher speed if a burst
bonus is available. This parameter limits the number of bytes that have
accumulated in the burst bonus and transfers traffic at a higher speed.
The traffic shaping policies that you set at switch or port group level are applied at each individual port
that participates in the switch or port group. For example, if you set an average bandwidth of 100000
Kbps on a standard port group, 100000 Kbps averaged over time can pass through each port that is
associated with the standard port group.
Procedure
3 Navigate to the traffic shaping policy on the standard switch or port group.
Option Action
Standard port group a Select the standard switch where the port group resides.
b In the topology diagram, select a standard port group.
c Click Edit settings.
d Select Traffic shaping and select Override next to the options to override.
Option Description
Status Enables setting limits on the amount of networking bandwidth allocated for each
port that is associated with the standard switch or port group.
Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over
time (the allowed average load).
Peak Bandwidth The maximum number of bits per second to allow across a port when it is sending
a burst of traffic. This setting tops the bandwidth used by a port whenever it is
using its burst bonus. This parameter can never be smaller than the average
bandwidth.
Burst Size The maximum number of bytes to allow in a burst. If this parameter is set, a port
might gain a burst bonus when it does not use all its allocated bandwidth.
Whenever the port needs more bandwidth than the average bandwidth specifies,
the port can temporarily transmit data at a higher speed if a burst bonus is
available. This parameter tops the number of bytes that can accumulate in the
burst bonus and can be transferred at a higher speed.
5 For each traffic shaping policy (Average Bandwidth, Peak Bandwidth, and Burst Size), enter a
bandwidth value.
6 Click OK.
The traffic shaping policies that you set at distributed port group level are applied on each individual port
that participates in the port group. For example, if you set an average bandwidth of 100000 Kbps on a
distributed port group, 100000 Kbps averaged over time can pass through each port that is associated
with the distributed port group.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
2 Navigate to the Traffic Shaping policy for the distributed port group or port.
Option Action
Distributed port group a From the Actions menu, select Distributed Port Group > Manage
Distributed Port Groups.
b Select Traffic shaping.
c Select the port group and click Next.
Distributed port a On the Networks tab, click Distributed Port Groups and double-click a
distributed port group .
b On the Ports tab, select a port and click the Edit distributed port settings
icon.
c Select Traffic shaping.
d Select Override next to the properties to override.
Note The traffic is classified to ingress and egress according to the traffic direction in the switch, not
in the host.
Option Description
Status Enable either Ingress traffic shaping or Egress traffic shaping by using the
Status drop-down menus.
Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over
time, that is, the allowed average load.
Peak Bandwidth The maximum number of bits per second to allow across a port when it is
sending/sending or receiving a burst of traffic. This parameter tops the bandwidth
used by a port whenever it is using its burst bonus.
Burst Size The maximum number of bytes to allow in a burst. If this parameter is set, a port
might gain a burst bonus when it does not use all its allocated bandwidth.
Whenever the port needs more bandwidth than the average bandwidth specifies,
the port can temporarily transmit data at a higher speed if a burst bonus is
available. This parameter tops the number of bytes that might accumulate in the
burst bonus and be transferred at a higher speed.
For information about creating and configuring network resource pools, see Chapter 11 vSphere Network
I/O Control.
Prerequisites
n Enable Network I/O Control on the distributed switch. See Enable Network I/O Control on a vSphere
Distributed Switch.
n Create and configure network resource pools. See Create a Network Resource Pool.
Procedure
2 Right-click the distributed switch in the navigator and select Distributed Port Groups > Manage
Distributed Port Groups.
5 Add or remove the distributed port group from the network resource pool and click Next.
n To add the distributed port group, select a user-defined resource pool from the Network
resource pool drop-down menu.
n To remove the distributed port group, select default from the Network resource pool drop-down
menu.
6 Review your settings in the Ready to complete section and click Finish.
Monitoring Policy
The monitoring policy enables or disables NetFlow monitoring on a distributed port or port group.
NetFlow settings are configured at the vSphere distributed switch level. See Configure the NetFlow
Settings of a vSphere Distributed Switch.
You configure the NetFlow settings on the vSphere Distributed Switch. See Configure the NetFlow
Settings of a vSphere Distributed Switch
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
2 Navigate to the monitoring policy for the distributed port group or distributed port.
Option Action
Distributed port group a From the Actions menu, select Distributed Port Group > Manage
Distributed Port Groups.
b Select Monitoring.
c Select the port group and click Next.
Distributed port a On the Networks tab, click Distributed Port Groups and double-click a
distributed port group .
b On the Ports tab, select a port and click the Edit distributed port settings
icon.
c Select Monitoring.
d Select Override next to the properties to override.
3 From the NetFlow drop-down menu, enable or disable NetFlow and click Next.
The traffic filtering and marking policy represents an ordered set of network traffic rules for security and
for QoS tagging of the data flow through the ports of a distributed switch. In general, a rule consists of a
qualifier for traffic, and of an action for restricting or prioritizing the matching traffic.
The vSphere distributed switch applies rules on traffic at different places in the data stream. The
distributed switch applies traffic filter rules on the data path between the virtual machine network adapter
and distributed port, or between the uplink port and physical network adapter for rules on uplinks.
n Enable Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group
Enable the traffic filtering and marking policy on a port group if you want to configure traffic security
and marking on all virtual machine network adapters or uplink adapters that are participating in the
group.
n Working with Network Traffic Rules on a Distributed Port Group or Uplink Port Group
Define traffic rules in a distributed port group or uplink port group to introduce a policy for processing
traffic related to virtual machines or to physical adapters. You can filter specific traffic or describe its
QoS demands.
n Disable Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group
Let traffic flow to virtual machines or physical adapters without additional control related to security
or QoS by disabling the traffic filtering and marking policy.
Note You can disable the traffic filtering and marking policy on a particular port to avoid processing the
traffic flowing through the port. See Disable Traffic Filtering and Marking on a Distributed Port or Uplink
Port.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
5 Click OK.
What to do next
Set up traffic marking or filtering on the data that is flowing through the ports of the distributed port group
or through the uplink port group. See Mark Traffic on a Distributed Port Group or Uplink Port Group and
Filter Traffic on a Distributed Port Group or Uplink Port Group.
Priority tagging is a mechanism to mark traffic that has higher QoS demands. In this way, the network can
recognize different classes of traffic. The network devices can handle the traffic from each class
according to its priority and requirements.
You can also re-tag traffic to either raise or lower the importance of the flow. By using a low QoS tag, you
can restrict data tagged in a guest operating system.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
5 Click New to create a new rule, or select a rule and click Edit to edit it.
6 In the network traffic rule dialog box, select the Tag option from the Action drop-down menu.
7 Set the priority tag for the traffic within the scope of the rule.
Option Description
CoS value Mark the traffic matching the rule with a CoS priority tag in network Layer 2.
Select Update CoS tag and type a value from 0 to 7.
DSCP value Mark the traffic associated with the rule with a DSCP tag in network Layer 3.
Select Update DSCP value and type a value from 0 to 63.
To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed
switch examines the direction of the traffic, and properties like source and destination, VLAN, next
level protocol, infrastructure traffic type, and so on.
a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress,
or both so that the rule recognizes it as matching.
The direction also influences how you are going to identify the traffic source and destination.
b By using qualifiers for system data type, Layer 2 packet attributes, and Layer 3 packet attributes
set the properties that packets must have to match the rule.
A qualifier represents a set of matching criteria related to a networking layer. You can match
traffic to system data type, Layer 2 traffic properties, and Layer 3 traffic properties. You can use
the qualifier for a specific networking layer or can combine qualifiers to match packets more
precisely.
n Use the system traffic qualifier to match packets to the type of virtual infrastructure data that
is flowing through the ports of the group . For example, you can select NFS for data transfers
to network storage.
n Use the MAC traffic qualifier to match packets by MAC address, VLAN ID, and next level
protocol.
Locating traffic with a VLAN ID on a distributed port group works with Virtual Guest Tagging
(VGT). To match traffic to VLAN ID if Virtual Switch Tagging (VST) is active, use a rule on an
uplink port group or uplink port.
n Use the IP traffic qualifier to match packets by IP version, IP address, and next level protocol
and port.
Voice over IP (VoIP) flows have special requirements for QoS in terms of low loss and delay. The traffic
related to the Session Initiation Protocol (SIP) for VoIP usually has a DSCP tag equal to 26, which stands
for Assured Forwarding Class 3 with Low Drop Probability (AF31).
For example, to mark outgoing SIP UDP packets to a subnet 192.168.2.0/24, you can use the following
rule:
Action Tag
DSCP value 26
Protocol UDP
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
5 Click New to create a new rule, or select a rule and click Edit to edit it.
6 In the network traffic rule dialog box, use the Action options to let traffic pass through the ports of the
distributed port group or uplink port group, or to restrict it.
To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed
switch examines the direction of the traffic, and properties like source and destination, VLAN, next
level protocol, infrastructure traffic type, and so on.
a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress,
or both so that the rule recognizes it as matching.
The direction also influences how you are going to identify the traffic source and destination.
b By using qualifiers for system data type, Layer 2 packet attributes, and Layer 3 packet attributes
set the properties that packets must have to match the rule.
A qualifier represents a set of matching criteria related to a networking layer. You can match
traffic to system data type, Layer 2 traffic properties, and Layer 3 traffic properties. You can use
the qualifier for a specific networking layer or can combine qualifiers to match packets more
precisely.
n Use the system traffic qualifier to match packets to the type of virtual infrastructure data that
is flowing through the ports of the group . For example, you can select NFS for data transfers
to network storage.
n Use the MAC traffic qualifier to match packets by MAC address, VLAN ID, and next level
protocol.
Locating traffic with a VLAN ID on a distributed port group works with Virtual Guest Tagging
(VGT). To match traffic to VLAN ID if Virtual Switch Tagging (VST) is active, use a rule on an
uplink port group or uplink port.
n Use the IP traffic qualifier to match packets by IP version, IP address, and next level protocol
and port.
Note You can override the rules of the policy for traffic filtering and marking at port level. See Working
with Network Traffic Rules on a Distributed Port or Uplink Port.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
5 Examine Action to see if the rule filters traffic (Allow or Deny) or marks traffic (Tag) with special QoS
demands.
6 From the upper list, select the rule for which you want to view the criteria for locating traffic.
The traffic qualifying parameters of the rule appear in the Traffic Qualifiers list.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
5 Click New to create a new rule, or select a rule and click Edit to edit it.
What to do next
Name the network traffic rule, and deny, allow, or tag the target traffic.
The vSphere distributed switch applies network traffic rules in a strict order. If a packet already satisfies a
rule, the packet might not be passed to the next rule in the policy.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
5 Select a rule and use the arrow buttons to change its priority.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu.
6 Click OK.
Note You can enable and set up the traffic filtering and marking policy on a particular port. See Enable
Traffic Filtering and Marking on a Distributed Port or Uplink Port.
Procedure
1 Locate a distributed port group or an uplink port group in the vSphere Web Client.
b Click Distributed Port Groups to see the list of distributed port groups, or click Uplink Port
Groups to see the list of uplink port groups.
5 Click OK.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 Select the Override check box, and from the Status drop-down menu, select Enabled.
6 Click OK.
What to do next
Set up traffic filtering or marking for the data flowing through the distributed port or through the uplink port.
See Mark Traffic on a Distributed Port or Uplink Port and Filter Traffic on a Distributed Port or Uplink Port.
Priority tagging is a mechanism to mark traffic that has higher QoS demands. In this way, the network can
recognize different classes of traffic. The network devices can handle the traffic from each class
according to its priority and requirements.
You can also re-tag traffic to either raise or lower the importance of the flow. By using a low QoS tag, you
can restrict data tagged in a guest operating system.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
4 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
5 Click New to create a new rule, or select a rule and click Edit to edit it.
You can change a rule inherited from the distributed port group or uplink port group. In this way, the
rule becomes unique within the scope of the port.
6 In the network traffic rule dialog box, select the Tag option from the Action drop-down menu.
7 Set the priority tag for the traffic within the scope of the rule.
Option Description
CoS value Mark the traffic matching the rule with a CoS priority tag in network Layer 2.
Select Update CoS tag and type a value from 0 to 7.
DSCP value Mark the traffic associated with the rule with a DSCP tag in network Layer 3.
Select Update DSCP value and type a value from 0 to 63.
To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed
switch examines the direction of the traffic, and properties like source and destination, VLAN, next
level protocol, infrastructure traffic type, and so on.
a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress,
or both so that the rule recognizes it as matching.
The direction also influences how you are going to identify the traffic source and destination.
b By using qualifiers for system data type, Layer 2 packet attributes, and Layer 3 packet attributes
set the properties that packets must have to match the rule.
A qualifier represents a set of matching criteria related to a networking layer. You can match
traffic to system data type, Layer 2 traffic properties, and Layer 3 traffic properties. You can use
the qualifier for a specific networking layer or can combine qualifiers to match packets more
precisely.
n Use the system traffic qualifier to match packets to the type of virtual infrastructure data that
is flowing through the ports of the group . For example, you can select NFS for data transfers
to network storage.
n Use the MAC traffic qualifier to match packets by MAC address, VLAN ID, and next level
protocol.
Locating traffic with a VLAN ID on a distributed port group works with Virtual Guest Tagging
(VGT). To match traffic to VLAN ID if Virtual Switch Tagging (VST) is active, use a rule on an
uplink port group or uplink port.
n Use the IP traffic qualifier to match packets by IP version, IP address, and next level protocol
and port.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
4 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
5 Click New to create a new rule, or select a rule and click Edit to edit it.
You can change a rule inherited from the distributed port group or uplink port group. In this way, the
rule becomes unique within the scope of the port.
6 In the network traffic rule dialog box, select the Allow action to let traffic pass through the distributed
port or uplink port, or the Drop action to restrict it.
To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed
switch examines the direction of the traffic, and properties like source and destination, VLAN, next
level protocol, infrastructure traffic type, and so on.
a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress,
or both so that the rule recognizes it as matching.
The direction also influences how you are going to identify the traffic source and destination.
b By using qualifiers for system data type, Layer 2 packet attributes, and Layer 3 packet attributes
set the properties that packets must have to match the rule.
A qualifier represents a set of matching criteria related to a networking layer. You can match
traffic to system data type, Layer 2 traffic properties, and Layer 3 traffic properties. You can use
the qualifier for a specific networking layer or can combine qualifiers to match packets more
precisely.
n Use the system traffic qualifier to match packets to the type of virtual infrastructure data that
is flowing through the ports of the group . For example, you can select NFS for data transfers
to network storage.
n Use the MAC traffic qualifier to match packets by MAC address, VLAN ID, and next level
protocol.
Locating traffic with a VLAN ID on a distributed port group works with Virtual Guest Tagging
(VGT). To match traffic to VLAN ID if Virtual Switch Tagging (VST) is active, use a rule on an
uplink port group or uplink port.
n Use the IP traffic qualifier to match packets by IP version, IP address, and next level protocol
and port.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
6 Examine Action to see if the rule filters traffic (Allow or Deny) or marks traffic (Tag) with special QoS
demands.
7 From the upper list, select the rule for which you want to view the criteria for locating traffic.
The traffic qualifying parameters of the rule appear in the Traffic Qualifiers list.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
6 Click New to create a new rule, or select a rule and click Edit to edit it.
You can change a rule inherited from the distributed port group or uplink port group. In this way, the
rule becomes unique within the scope of the port.
What to do next
Name the network traffic rule, and deny, allow, or tag the target traffic.
The vSphere distributed switch applies network traffic rules in a strict order. If a packet already satisfies a
rule, the packet might not be passed to the next rule in the policy.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
6 Select a rule and use the arrow buttons to change its priority.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status
drop-down menu, select Enabled.
7 Click OK.
Prerequisites
To override a policy on distributed port level, enable the port-level override option for this policy. See
Configure Overriding Networking Policies on Port Level.
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
5 Click Override, and from the Status drop-down menu, select Disabled.
6 Click OK.
To match the traffic in the scope of the rule more precisely, you can combine criteria for system data type,
Layer 2 header, and Layer 3 header.
You can select the type of traffic through the ports of the group that carries system data, that is, traffic for
® ®
management from vCenter Server , storage, VMware vSphere vMotion , and vSphere Fault Tolerance.
You can mark or filter only a specific traffic type, or for all system data traffic except for a infrastructure
feature. For example, you can mark with a QoS value or filter the traffic for management from
vCenter Server , storage and vMotion, but not the traffic carrying the Fault Tolerance data.
Protocol Type
The Protocol type attribute of the MAC traffic qualifier corresponds to the EtherType field in Ethernet
frames. EtherType represents the type of next level protocol that is going to consume the payload of the
frame.
You can select a protocol from the drop-down menu or type its hexadecimal number. For example, to
capture traffic for the Link Layer Discovery Protocol (LLDP) protocol, type 88CC.
VLAN ID
You can use the VLAN ID attribute of the MAC traffic qualifier to mark or filter traffic in a particular VLAN.
Note The VLAN ID qualifier on a distributed port group works with Virtual Guest Tagging (VGT).
If a flow is tagged with a VLAN ID through Virtual Switch Tagging (VST), it cannot be located by using this
ID in a rule on a distributed port group or distributed port. The reason is that the distributed switch checks
the rule conditions, including the VLAN ID, after the switch has already untagged the traffic. In this case,
to match traffic by VLAN ID successfully, you must use a rule on an uplink port group or uplink port.
Source Address
By using the Source Address group of attributes, you can match packets by the source MAC address or
network.
You can use a comparison operator to mark or filter packets that have or do not have the specified source
address or network.
Table 8‑6. Patterns for Filtering or Marking Traffic by MAC Source Address
Parameters to Match Traffic Source
Address Comparison Operator Networking Argument Format
MAC network matches or does not match Type the lowest address in the network
and a wildcard mask. Set zeroes at the
positions of the network bits, and ones
for the host part.
For example, for a MAC network with prefix 05:50:56 that is 23 bits long, set the address as
00:50:56:00:00:00 and mask as 00:00:01:ff:ff:ff.
Destination Address
By using the Destination Address group of attributes, you can match packets to their destination address.
The MAC destination address options have the same format as those for the source address.
Comparison Operators
To match traffic in a MAC qualifier more closely to your needs, you can use affirmative comparison or
negation. You can use operators such that all packets except the ones with certain attributes fall in the
scope of a rule.
IP Traffic Qualifier
By using the IP traffic qualifier in a rule, you can define criteria for matching traffic to the Layer 3 (Network
Layer) properties such as IP version, IP address, next level protocol, and port.
Protocol
The Protocol attribute of the IP traffic qualifier represents the next level protocol consuming the payload
of the packet. You can select a protocol from the drop-down menu or type its decimal number according
to RFC 1700.
For the TCP and UDP protocols, you can also match traffic by source and destination ports.
Source Port
By using the Source port attribute, you can match TCP or UDP packets by the source port. Consider the
traffic direction when matching traffic to a source port.
Destination Port
By using the Destination port attribute, you can match TCP or UDP packets by the destination port.
Consider the traffic direction when matching traffic to a destination port.
Source Address
By using the Source Address attribute, you can match packets by source address or subnet. Consider the
traffic direction when matching traffic to a source address or network.
IP subnet matches or does not match Type the lowest address in the subnet
and the bit length of the subnet prefix.
Destination Address
Use the Destination Address to match packets by IP address, subnet, or IP version. The destination
address has the same format as the one for the source.
Comparison Operators
To match traffic in an IP qualifier more closely to your needs, you can use affirmative comparison or
negation. You can define that all packets fall in the scope of a rule except packets with certain attributes.
Prerequisites
Procedure
2 Right-click the distributed switch in the object navigator and select Distributed Port Group >
Manage Distributed Port Groups.
3 On the Select port group policies page, select the check box next to the policy categories to modify
and click Next.
Option Description
Security Set MAC address changes, forged transmits, and promiscuous mode for the
selected port groups.
Traffic shaping Set the average bandwidth, peak bandwidth, and burst size for inbound and
outbound traffic on the selected port groups.
VLAN Configure how the selected port groups connect to physical VLANs.
Teaming and failover Set load balancing, failover detection, switch notification, and failover order for the
selected port groups.
Resource allocation Set network resource pool association for the selected port groups.
Traffic filtering and marking Configure policy for filtering (allow or drop) and for marking certain types of traffic
through the ports of selected port groups.
4 On the Select port groups page, select the distributed port group(s) to edit and click Next.
5 (Optional) On the Security page, use the drop-down menus to edit the security exceptions and click
Next.
Option Description
Promiscuous mode n Reject. Placing a guest adapter in promiscuous mode has no effect on which
frames are received by the adapter.
n Accept. Placing a guest adapter in promiscuous mode causes it to detect all
frames passed on the vSphere Distributed Switch that are allowed under the
VLAN policy for the port group that the adapter is connected to.
MAC address changes n Reject. If set to Reject and the guest operating system changes the MAC
address of the adapter to anything other than what is in the .vmx
configuration file, all inbound frames are dropped.
If the Guest OS changes the MAC address back to match the MAC address
in the .vmx configuration file, inbound frames are passed again.
n Accept. Changing the MAC address from the Guest OS has the intended
effect. Frames to the new MAC address are received.
Forged transmits n Reject. Any outbound frame with a source MAC address that is different from
the one currently set on the adapter are dropped.
n Accept. No filtering is performed and all outbound frames are passed.
6 (Optional) On the Traffic shaping page, use the drop-down menus to enable or disable Ingress or
Egress traffic shaping and click Next.
Option Description
Status If you enable either Ingress traffic shaping or Egress traffic shaping, you are
setting limits on the amount of networking bandwidth allocated for each VMkernel
adapter or virtual network adapter associated with this port group. If you disable
the policy, services have a free, clear connection to the physical network by
default.
Average bandwidth Establishes the number of bits per second to allow across a port, averaged over
time, that is, the allowed average load.
Peak bandwidth The maximum number of bits per second to allow across a port when it is sending
or receiving a burst of traffic. This maximum number tops the bandwidth used by
a port whenever it is using its burst bonus.
Burst size The maximum number of bytes to allow in a burst. If this parameter is set, a port
might gain a burst bonus when it does not use all its allocated bandwidth.
Whenever the port needs more bandwidth than specified by Average bandwidth,
it might be allowed to transmit data at a higher speed if a burst bonus is available.
This parameter tops the number of bytes that can be accumulated in the burst
bonus and transferred at a higher speed.
7 (Optional) On the VLAN page, use the drop-down menus to edit the VLAN policy and click Next.
Option Description
8 (Optional) On the Teaming and failover page, use the drop-down menus to edit the settings and click
Next.
Option Description
Load balancing IP-based teaming requires that the physical switch be configured with ether
channel. For all other options, ether channel should be disabled. Select how to
choose an uplink.
n Route based on the originating virtual port. Choose an uplink based on
the virtual port where the traffic entered the distributed switch.
n Route based on IP hash. Choose an uplink based on a hash of the source
and destination IP addresses of each packet. For non-IP packets, whatever is
at those offsets is used to compute the hash.
n Route based on source MAC hash. Choose an uplink based on a hash of
the source Ethernet.
n Route based on physical NIC load. Choose an uplink based on the current
loads of physical NICs.
n Use explicit failover order. Always use the highest order uplink, from the list
of Active adapters, which passes failover detection criteria.
Network failure detection Select the method to use for failover detection.
n Link status only. Relies solely on the link status that the network adapter
provides. This option detects failures, such as cable pulls and physical switch
power failures, but not configuration errors, such as a physical switch port
being blocked by spanning tree or that is misconfigured to the wrong VLAN or
cable pulls on the other side of a physical switch.
n Beacon probing. Sends out and listens for beacon probes on all NICs in the
team and uses this information, in addition to link status, to determine link
failure. Do not use beacon probing with IP-hash load balancing.
Notify switches Select Yes or No to notify switches in the case of failover. Do not use this option
when the virtual machines using the port group are using Microsoft Network Load
Balancing in unicast mode.
If you select Yes, whenever a virtual NIC is connected to the distributed switch or
whenever that virtual NIC’s traffic is routed over a different physical NIC in the
team because of a failover event, a notification is sent out over the network to
update the lookup tables on physical switches. Use this process for the lowest
latency of failover occurrences and migrations with vMotion.
Failover order Select how to distribute the work load for uplinks. To use some uplinks but
reserve others in case the uplinks in use fail, set this condition by moving them
into different groups.
n Active uplinks. Continue to use the uplink when the network adapter
connectivity is up and active.
n Standby uplinks . Use this uplink if one of the active adapter’s connectivity is
down. When using IP-hash load balancing, do not configure standby uplinks.
n Unused uplinks . Do not use this uplink.
9 (Optional) On the Resource allocation page, use the Network resource pool drop-down menu to
add or remove resource allocations and click Next.
10 (Optional) On the Monitoring page, use the drop-menu to enable or disable NetFlow and click Next.
Option Description
Enabled NetFlow is enabled on the distributed port group. You can configure NetFlow
settings at the vSphere Distributed Switch level.
11 (Optional) On the Traffic filtering and marking page, enable or disable traffic filtering and marking from
Status drop-down menu, configure traffic rules for filtering or marking specific data flows, and click
Next.
You can set the following attributes of a rule determining the target traffic and the action on it:
Option Description
Traffic direction Set whether the rule is for incoming, outgoing or incoming and outgoing traffic.
The direction also influences how you are going to identify the traffic source and
destination.
System traffic qualifier Indicate that the rule scopes over system traffic and set the type of infrastructure
protocol to apply the rule on. For example, mark with a priority tag the traffic for
management from vCenter Server.
Option Description
MAC qualifier Qualify the traffic for the rule by Layer 2 header.
n Protocol type. Set the next level protocol (IPv4, IPv6, etc.) consuming the
payload.
You can select a protocol from the drop-down menu or type its hexadecimal
number
For example, to locate traffic for the Link Layer Discovery Protocol (LLDP)
protocol, type 88CC.
n VLAN ID. Locate traffic by VLAN.
The VLAN ID qualifier on a distributed port group works with Virtual Guest
Tagging (VGT).
If you have a flow tagged with a VLAN ID through Virtual Switch Tagging
(VST), you cannot locate the flow by this ID in a distributed port group rule.
The reason is that the distributed switch checks the rule conditions, including
the VLAN ID, after the switch has already untagged the traffic. To match
successfully traffic to a VLAN ID, use a rule for an uplink port group or uplink
port.
n Source Address. Set a single MAC address or a MAC network to match
packets by source address.
For a MAC network you enter the lowest address in the network and a
wildcard mask. The mask contains zeroes at the positions of the network bits,
and ones for the host part.
For example, for a MAC network with prefix 05:50:56 that is 23 bits long, set
the address as 00:50:56:00:00:00 and the mask as 00:00:01:ff:ff:ff.
n Destination Address. Set a single MAC address or a MAC network to match
packets by destination address. The MAC destination address supports the
same format as the source address.
You can select a protocol from the drop-down menu or type its decimal
number according to RFC 1700, Assigned Numbers.
For TCP and UDP protocol, you can also set source and destination port.
n Source port. Match TCP or UDP packets to a source port. Consider the
direction of the traffic that is within the scope of the rule when determining the
source port to match packets to.
n Destination port. Match TCP or UDP packets by the source port. Consider
the direction of the traffic that is within the scope of the rule when determining
the destination port to match packets to.
n Source Address. Set the IP version, a single IP address or a subnet to
match packets by source address.
For a subnet you enter the lowest address and the bit length of the prefix.
n Destination Address. Set the IP version, a single IP address or a subnet to
match packets by source address. The IP destination address supports the
same format as the source address.
12 (Optional) On the Miscellaneous page, select Yes or No from the drop-down menu and click Next.
SelectYes to shut down all ports in the port group. This shutdown might disrupt the normal network
operations of the hosts or virtual machines using the ports.
13 Review your settings on the Ready to complete page and click Finish.
Blocking the ports of a distributed port group might disrupt the normal network operations of the hosts or
virtual machines using the ports.
Procedure
2 Right-click the distributed switch in the object navigator and select Distributed Port Group >
Manage Distributed Port Groups.
4 Select one or more distributed port group to configure and click Next.
5 From the Block all ports drop-down menu, enable or disable port blocking, and click Next.
Blocking the flow through a port might disrupt the normal network operations on the host or virtual
machine using the port.
Prerequisites
Enable the port-level overrides. See Configure Overriding Networking Policies on Port Level
Procedure
1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port.
n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups,
double-click a distributed port group from the list, and click the Ports tab.
n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups,
double-click an uplink port group from the list, and click the Ports tab.
4 In the Miscellaneous section, select the Override check box, and from the drop-down menu enable
or disable port blocking.
5 Click OK.
n Private VLANs
VLAN Configuration
Virtual LANs (VLANs) enable a single physical LAN segment to be further isolated so that groups of ports
are isolated from one another as if they were on physically different segments.
Watch the video about the benefits and main principles in introducing VLANs in a vSphere environment.
EST 0 The physical switch performs the VLAN tagging. The host network adapters are
connected to access ports on the physical switch.
VST Between 1 and 4094 The virtual switch performs the VLAN tagging before the packets leave the host.
The host network adapters must be connected to trunk ports on the physical switch.
VGT n 4095 for standard The virtual machine performs the VLAN tagging. The virtual switch preserves the
switch VLAN tags when it forwards the packets between the virtual machine networking
n Range of and stack and external switch. The host network adapters must be connected to trunk
individual VLANs for ports on the physical switch.
distributed switch The vSphere Distributed Switch supports a modification of VGT. For security
reasons, you can configure a distributed switch to pass only packets that belong to
particular VLANs.
Note For VGT you must have an 802.1Q VLAN trunking driver installed on the
guest operating system of the virtual machine.
Watch the video that explains the modes of VLAN tagging in virtual switches.
Private VLANs
Private VLANs are used to solve VLAN ID limitations by adding a further segmentation of the logical
broadcast domain into multiple smaller broadcast subdomains.
A private VLAN is identified by its primary VLAN ID. A primary VLAN ID can have multiple secondary
VLAN IDs associated with it. Primary VLANs are Promiscuous, so that ports on a private VLAN can
communicate with ports configured as the primary VLAN. Ports on a secondary VLAN can be either
Isolated, communicating only with promiscuous ports, or Community, communicating with both
promiscuous ports and other ports on the same secondary VLAN.
To use private VLANs between a host and the rest of the physical network, the physical switch connected
to the host needs to be private VLAN-capable and configured with the VLAN IDs being used by ESXi for
the private VLAN functionality. For physical switches using dynamic MAC+VLAN ID based learning, all
corresponding private VLAN IDs must be first entered into the switch's VLAN database.
Procedure
3 Click Edit.
4 To add a primary VLAN, under Primary VLAN ID click Add and enter the ID of a primary VLAN.
5 Click the plus sign (+) in front of the primary VLAN ID to add it to the list.
The primary private VLAN also appears under Secondary Private VLAN ID.
6 To add a secondary VLAN, in the right pane click Add and enter the ID of the VLAN.
7 Click the plus sign (+) in front of the secondary VLAN ID to add it to the list.
8 From the drop-down menu in the Secondary VLAN type column, select either Isolated or
Community.
9 Click OK.
What to do next
Configure a distributed port group or port to associate traffic with the private VLAN. See Configure VLAN
Tagging on a Distributed Port Group or Distributed Port.
When you remove a primary private VLAN, you also remove the associated secondary private VLANs.
Prerequisites
Verify that no port groups are configured to use the primary VLAN and its associated secondary VLANs.
Procedure
3 Click Edit.
7 Click OK.
Prerequisites
Verify that no port groups are configured to use the secondary VLAN.
Procedure
3 Click Edit.
6 Under the secondary VLAN ID list, click Remove and click OK.
n DirectPath I/O
n Jumbo Frames
DirectPath I/O
DirectPath I/O allows virtual machine access to physical PCI functions on platforms with an I/O Memory
Management Unit.
The following features are unavailable for virtual machines configured with DirectPath:
n Fault tolerance
n High availability
n DRS (limited availability. The virtual machine can be part of a cluster, but cannot migrate across
hosts)
n Snapshots
Caution If your ESXi host is configured to boot from a USB device or an SD card attached to a USB
channel, make sure that you do not enable DirectPath I/O passthrough for the USB controller. Passing
through a USB controller on an ESXi host that boots from a USB device or SD card might put the host in
a state where its configuration cannot be persisted.
Procedure
3 To enable DirectPath I/O passthrough for a PCI network device on the host, click Edit.
Icon Description
orange icon The state of the device has changed, and you must reboot the host before you can use the device.
4 Select the network device to be used for passthrough and click OK.
The selected PCI device appears in the table. Device information is displayed at the bottom of the
screen.
5 Reboot the host to make the PCI network device available for use.
When using passthrough devices with a Linux kernel version 2.6.20 or earlier, avoid MSI and MSI-X
modes because these modes have significant performance impact.
Prerequisites
Verify that a passthrough networking device is configured on the host of the virtual machine. See Enable
Passthrough for a Network Device on a Host.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
6 From the New device drop-down menu select PCI Device and click Add.
7 From the New PCI device drop-down menu select the passthrough device to use, and click OK.
Adding a DirectPath I/O device to a virtual machine sets memory reservation to the memory size of the
virtual machine.
Overview of SR-IOV
SR-IOV is a specification that allows a single Peripheral Component Interconnect Express (PCIe) physical
device under a single root port to appear as multiple separate physical devices to the hypervisor or the
guest operating system.
SR-IOV uses physical functions (PFs) and virtual functions (VFs) to manage global functions for the SR-
IOV devices. PFs are full PCIe functions that are capable of configuring and managing the SR-IOV
functionality. It is possible to configure or control PCIe devices using PFs, and the PF has full ability to
move data in and out of the device. VFs are lightweight PCIe functions that support data flowing but have
a restricted set of configuration resources.
The number of virtual functions provided to the hypervisor or the guest operating system depends on the
device. SR-IOV enabled PCIe devices require appropriate BIOS and hardware support, as well as SR-
IOV support in the guest operating system driver or hypervisor instance. See SR-IOV Support.
In vSphere, though a virtual switch (standard switch or distributed switch) does not handle the network
traffic of an SR-IOV enabled virtual machine connected to the switch, you can control the assigned virtual
functions by using switch configuration policies at port group or port level.
SR-IOV Support
vSphere supports SR-IOV in an environment with specific configuration only. Some features of vSphere
are not functional when SR-IOV is enabled.
Supported Configurations
To use SR-IOV in vSphere, your environment must meet several configuration requirements.
To verify that your physical hosts and NICs are compatible with ESXi releases, see the VMware
Compatibility Guide.
Availability of Features
The following features are not available for virtual machines configured with SR-IOV:
n vSphere vMotion
n Storage vMotion
n vShield
n NetFlow
n vSphere DRS
n vSphere DPM
Note Attempts to enable or configure unsupported features with SR-IOV in the vSphere Web Client
result in unexpected behavior in your environment.
Supported NICs
All NICs must have drivers and firmware that support SR-IOV. Some NICs might require SR-IOV to be
enabled on the firmware. The following NICs are supported for virtual machines configured with SR-IOV:
n Products based on the Intel 82599ES 10 Gigabit Ethernet Controller Family (Niantic)
In a host that runs virtual machine traffic on top of SR-IOV physical adapters, virtual machine adapters
directly contact the virtual functions to communicate data. However, the ability to configure networks is
based on the active policies for the port holding the virtual machines.
On an ESXi host without SR-IOV, the virtual switch sends external network traffic through its ports on the
host from or to the physical adapter for the relevant port group. The virtual switch also applies the
networking policies on managed packets.
Figure 10‑1. Data and Configuration Paths in the SR-IOV Support of vSphere
VM VM VM VM
VF driver VF driver
PF driver PF driver
VMware ESXi
1 2 5
IOMMU
VF VF PF PF
Physical network adapter with SR-IOV Physical network adapter without SR-IOV
Port association
Data path
Control path
3 The PF driver checks the configuration request with the virtual switch (standard switch or host proxy
switch of a distributed switch).
4 The virtual switch verifies the configuration request against the policy on the port with which the VF
enabled virtual machine adapter is associated.
5 The PF driver configures the VF if the new settings are in compliance with the port policy of the virtual
machine adapter.
For example, if the VF driver tries to modify the MAC address, the address remains the same if MAC
address change is not allowed in the security policy for the port group or port. The guest operating
system might show that the change is successful but a log message indicates that the operation has
failed. As a result, the guest operating system and the virtual device save different MAC addresses.
The network interface in the guest operating system might not be able to acquire an IP address and
communicate. In this case, you have to reset the interface in the guest operating system to get the
latest MAC address from the virtual device and acquire an IP address.
n VFs do not implement rate control in vSphere. Every VF can potentially use the entire bandwidth of a
physical link.
n When a VF device is configured as a passthrough device on a virtual machine, the standby and
hibernate functions for the virtual machine are not supported.
n The maximum number of VFs that you can create and the maximum number of VFs that you can use
for passthrough are different. The maximum number of VFs that you can instantiate depends on the
NIC capability and on the hardware configuration of the host. However, due to the limited number of
interrupt vectors available for passthrough devices, only a limited number of all instantiated VFs can
be used on an ESXi host.
The total number of interrupt vectors on each ESXi host can scale up to 4096 in the case of 32 CPUs.
When the host boots, devices on the host such as storage controllers, physical network adapters, and
USB controllers consume a subset of the 4096 vectors. If these devices require more than 1024
vectors, the maximum number of potentially supported VFs is reduced.
n The number of VFs that is supported on an Intel NIC might be different from the number that is
supported on an Emulex NIC. See the technical documentation from the NIC vendor.
n If you have Intel and Emulex NICs present with SR-IOV enabled, the number of VFs available for the
Intel NICs depends on how many VFs are configured for the Emulex NIC, and the reverse. You can
use the following formula to estimate the maximum number of VFs for use if all 3072 interrupt vectors
are available for passthrough:
3X + 2Y < 3072
where X is the number of Intel VFs, and Y is the number of Emulex VFs.
This number might be smaller if other types of devices on the host use more than 1024 interrupt
vectors from the total of 4096 vectors on the host.
n vSphere SR-IOV supports up to 1024 VFs on supported Intel and Emulex NICs.
n If a supported Intel NIC loses connection, all VFs from the physical NIC stop communication
completely, including that between VFs.
n If a supported Emulex NIC loses connection, all VFs stop communication with the external
environment, but communication between VFs still works
n VF drivers offer many different features, such as IPv6 support, TSO, and LRO checksum. See the
technical documentation of the NIC vendor for more details.
SR-IOV is beneficial in workloads with very high packet rates or very low latency requirements. Like
DirectPath I/O, SR-IOV is not compatible with certain core virtualization features, such as vMotion. SR-
IOV does, however, allow for a single physical device to be shared amongst multiple guests.
With DirectPath I/O you can map only one physical function to one virtual machine. SR-IOV lets you
share a single physical device, allowing multiple virtual machines to connect directly to the physical
function.
Prerequisites
Verify that the configuration of your environment supports SR-IOV. See SR-IOV Support.
Procedure
The traffic passes from an SR-IOV passthrough adapter to the physical adapter in compliance with the
active policy on the associated port on the standard or distributed switch.
To examine which virtual function is assigned to an SR-IOV passthrough network adapter, on the
Summary tab for the virtual machine expand the VM Hardware panel and check the properties of the
adapter.
The topology diagram of the switch marks virtual machine adapters that use virtual functions with the
icon.
What to do next
Set up the traffic passing through the virtual functions attached to the virtual machine by using the
networking policies on the switch, port group, and port. See Networking Options for the Traffic Related to
an SR-IOV Enabled Virtual Machine.
Procedure
You can look at the SR-IOV property to see whether a physical adapter supports SR-IOV.
5 In the Number of virtual functions text box, type the number of virtual functions that you want to
configure for the adapter.
6 Click OK.
The virtual functions become active on the NIC port represented by the physical adapter entry. They
appear in the PCI Devices list in the Settings tab for the host.
You can use the esxcli network sriovnic vCLI commands to examine the configuration of virtual
functions on the host.
What to do next
Associate a virtual machine with a virtual function through an SR-IOV passthrough network adapter.
Prerequisites
n Verify that the passthrough networking devices for the virtual functions are active in the PCI Devices
list on the Settings tab for the host.
n Verify that the virtual machine compatibility is ESXi 5.5 and later.
n Verify that Red Hat Enterprise Linux 6 and later or Windows has been selected as the guest operating
system when the virtual machine was created.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
5 From the New device drop-down menu, select Network and click Add.
6 Expand the New Network section and connect the virtual machine to a port group.
The virtual NIC does not use this port group for data traffic. The port group is used to extract the
networking properties, for example VLAN tagging, to apply on the data traffic.
8 From the Physical function drop-down menu, select the physical adapter to back the passthrough
virtual machine adapter.
9 To allow changes in the MTU of packets from the guest operating system, use the Guest OS MTU
Change drop-down menu.
10 Expand the Memory section, select Reserve all guest memory (All locked) and click OK.
I/O memory management unit (IOMMU) must reach all virtual machine memory so that the
passthrough device can access the memory by using direct memory access (DMA).
When you power on the virtual machine, the ESXi host selects a free virtual function from the physical
adapter and maps it to the SR-IOV passthrough adapter. The host validates all properties of the virtual
machine adapter and the underlying virtual function against the settings of the port group to which the
virtual machine belongs.
Table 10‑2. Networking Options for a Virtual Machine Adapter That Uses a VF
Networking Option Description
MTU size Change the size of the MTU, for example, to enable jumbo
frames.
Security policy for VF traffic n If the guest operating system changes the initially set MAC
address of a virtual machine network adapter that uses a
VF, accept or drop incoming frames for the new address by
setting the MAC address changes option.
n Enable global promiscuous mode for virtual machine
network adapters, including adapters that use VFs.
VLAN tagging mode Configure VLAN tagging in the standard or distributed switch,
that is, enable VLAN Switch Tagging (VST) mode, or let the
tagged traffic reach the virtual machines that are associated
with VFs, that is, enable Virtual Guest Tagging (VGT).
The PF of an SR-IOV physical adapter controls the VFs that virtual machines use, and can carry the
traffic flowing through the standard or distributed switch that handles the networking of these SR-IOV
enabled virtual machines.
The SR-IOV physical adapter works in different modes depending on whether it backs the traffic of the
switch.
Mixed Mode
The physical adapter provides virtual functions to virtual machines attached to the switch and directly
handles traffic from non SR-IOV virtual machines on the switch.
You can check whether an SR-IOV physical adapter is in mixed mode in the topology diagram of the
switch. An SR-IOV physical adapter in mixed mode appears with the icon in the list of physical
adapters for a standard switch or in the list of uplink group adapters for a distributed switch.
To verify whether the physical adapter is in SR-IOV only mode, examine the topology diagram of the
switch. In this mode, the physical adapter is in a separate list called External SR-IOV Adapters and
appears with the icon.
For information about running ESXi by using Auto Deploy with host profiles, see the vCenter Server
Installation and Setup documentation.
You can also enable SR-IOV virtual functions on the host by using the esxcli system module
parameters set vCLI command on the NIC driver parameter for virtual functions in accordance with the
driver documentation. For more information about using vCLI commands, see vSphere Command-Line
Interface Documentation.
Prerequisites
n Verify that the configuration of your environment supports SR-IOV. See SR-IOV Support.
n Create a host profile based on the SR-IOV capable host. See the vSphere Host Profiles
documentation.
Procedure
1 From the vSphere Web Client Home page, click Host Profiles.
2 Select the host profile from the list and click the Configure tab.
3 Click Edit Host Profile and expand the General System Settings node.
4 Expand Kernel Module Parameter and select the parameter of the physical function driver for
creating virtual functions.
For example, the parameter for the physical function driver of an Intel physical NIC is max_vfs.
5 In the Value text box, type a comma-separated list of valid virtual function numbers.
Each list entry indicates the number of virtual functions that you want to configure for each physical
function. A value of 0 ensures that SR-IOV is not enabled for that physical function.
For example, if you have a dual port, set the value to x,y where x or y is the number of virtual
functions you want to enable for a single port.
If the target number of virtual functions on a single host is 30, you might have two dual port cards set
to 0,10,10,10.
Note The number of virtual functions supported and available for configuration depends on your
system configuration.
6 Click Finish.
The virtual functions appear in the PCI Devices list on the Settings tab for the host.
What to do next
Associate a virtual function with a virtual machine adapter by using the SR-IOV passthrough network
adapter type. See Assign a Virtual Function as SR-IOV Passthrough Adapter to a Virtual Machine.
You can create SR-IOV virtual functions on the host by manipulating the NIC driver parameter for virtual
functions in accordance with the driver documentation.
Prerequisites
Install the vCLI package, deploy the vSphere Management Assistant (vMA) virtual machine, or use the
ESXi Shell. See Getting Started with vSphere Command-Line Interfaces.
Procedure
1 To create virtual functions by setting the parameter for virtual functions of the NIC driver, run the
esxcli system module parameters set command at the command prompt.
Where driver is the name of the NIC driver, and vf_param is the driver-specific parameter for creating
the virtual function.
You can use a comma-separated list to set values for the vf_param parameter, where each entry
indicates the number of virtual functions for a port. A value of 0 ensures that SR-IOV is not enabled
for that physical function.
If you have two dual port NICs, you can set the value to w,x,y,z, where w,x,y, and z is the number of
virtual functions you want to enable for a single port. For example, to create 30 virtual functions
distributed on two dual port Intel cards by using the ixgbe driver, run the following command for the
ixgbe driver and the max_vfs parameter:
What to do next
Associate a virtual function with a virtual machine adapter by using the SR-IOV passthrough network
adapter type. See Assign a Virtual Function as SR-IOV Passthrough Adapter to a Virtual Machine.
Problem
On an ESXi host, one or more virtual machines that use SR-IOV virtual functions (VFs) for networking fail
to power on if the total number of assigned virtual functions is close to the maximum number of VFs
specified in the vSphere Configuration Maximums guide.
The virtual machine log file vmware.log contains the following message about the VF:
The VMkernel log file vmkernel.log contains the following messages about the VF assigned to the
virtual machine:
Cause
The number of allocatable interrupt vectors scales up with the number of physical CPUs on an ESXi host.
An ESXi host that has 32 CPUs can provide a total of 4096 interrupt vectors. When the host boots,
devices on the host such as storage controllers, physical network adapters, and USB controllers consume
a subset of the 4096 vectors. If these devices require more than 1024 vectors, the maximum number of
potentially supported VFs is reduced.
When a virtual machine powers on and the guest operating system VF driver starts, interrupt vectors are
consumed. If the required number of interrupt vectors is not available, the guest operating system shuts
down unexpectedly without any error messages.
No rule presently exists to determine the number of interrupt vectors consumed or available on a host.
This number depends on the hardware configuration of the host.
Solution
u To be able to power on the virtual machines, reduce the total number of VFs assigned to virtual
machines on the host.
For example, change the SR-IOV network adapter of a virtual machine to an adapter that is
connected to a vSphere Standard Switch or vSphere Distributed Switch.
Overview of RDMA
RDMA allows direct memory access from the memory of one computer to the memory of another
computer without involving the operating system or CPU . The transfer of memory is offloaded to the
RDMA-capable Host Channel Adapters (HCA) . A PVRDMA network adapter provides remote direct
memory access in a virtual environment.
The PVRDMA device automatically selects the method of communication between the virtual machines .
For virtual machines that run on the same ESXi host with or without a physical RDMA device, the data
transfer is a memcpy between the two virtual machines . The physical RDMA hardware is not used in this
case .
For virtual machines that reside on different ESXi hosts and that have a physical RDMA connection, the
physical RDMA devices must be uplinks on the distributed switch. In this case, the communication
between the virtual machines by way of PVRDMA uses the underlying physical RDMA devices.
For two virtual machines that run on different ESXi hosts, when at least one of the hosts does not have a
physical RDMA device, the communication falls back to a TCP-based channel and the performance is
reduced.
PVRDMA Support
vSphere 6.5 and later supports PVRDMA only in environments with specific configuration.
Supported Configurations
To use PVRDMA in vSphere 6.5, your environment must meet several configuration requirements.
Host Channel Adapter (HCA) n Must be compatible with the ESXi release.
To verify that your physical hosts and HCAs are compatible with ESXi releases, see the VMware
Compatibility Guide.
Note Attempts to enable or configure unsupported features with PVRDMA in the vSphere Web Client
might result in unexpected behavior in your environment.
Prerequisites
Verify that your ESXi host meets the requirements for PVRDMA. See PVRDMA Support.
Procedure
5 Enter the value of the VMkernel adapter that you want to use, for example vmk0, and click OK .
Procedure
5 Scroll to the pvrdma rule and select the check box next to it.
6 Click OK.
Prerequisites
n Verify that the host on which the virtual machine is running is configured for RDMA. See Configure an
ESXi Host for PVRDMA.
n Verify that the virtual machine uses virtual hardware version 13.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
5 From the New device drop-down menu, select Network and click Add.
6 Expand the New Network section and connect the virtual machine to a distributed port group.
8 Expand the Memory section, select Reserve all guest memory (All locked), and click OK .
RDMA over Converged Ethernet (RoCE) is a network protocol that uses RDMA to provide faster data
transfer for network-intensive applications. RoCE allows direct memory transfer between hosts without
involving the hosts' CPUs.
There are two versions of the RoCE protocol. RoCE v1 operates at the link network layer (layer 2). RoCE
v2 operates at the Internet network layer (layer 3) . Both RoCE v1 and RoCE v2 require a lossless
network configuration. RoCE v1 requires a lossless layer 2 network, and RoCE v2 requires that both layer
2 and layer 3 are configured for lossless operation.
Note Do not team RoCE NICs, if you intend to use RDMA on those NICs.
For vendor-specific configuration information, refer to the official documentation of the respective device
or switch vendor.
Jumbo Frames
Jumbo frames let ESXi hosts send larger frames out onto the physical network. The network must support
jumbo frames end-to-end that includes physical network adapters, physical switches, and storage
devices.
Before enabling jumbo frames, check with your hardware vendor to ensure that your physical network
adapter supports jumbo frames.
You can enable jumbo frames on a vSphere distributed switch or vSphere standard switch by changing
the maximum transmission unit (MTU) to a value greater than 1500 bytes. 9000 bytes is the maximum
frame size that you can configure.
Important When you change the MTU size of a vSphere Distributed Switch, the physical NICs that are
assigned as uplinks are brought down and up again. This causes a short network outage of between 5 to
10 milliseconds for virtual machines or services that are using the uplinks.
Procedure
3 Click Edit.
4 Click Advanced and set the MTU property to a value greater than 1500 bytes.
You cannot set the MTU size to a value greater than 9000 bytes.
5 Click OK.
Procedure
3 Select a standard switch from the virtual switch table and click Edit settings.
4 In the Properties section, set the MTU property to a value greater than 1500 bytes.
5 Click OK.
Procedure
5 Click Edit.
6 Select NIC settings and set the MTU property to a value greater than 1500.
7 Click OK.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
2 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
3 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
4 Expand the Network adapter section. Record the network settings and MAC address that the
network adapter is using.
5 Click Remove to remove the network adapter from the virtual machine.
6 From the New device drop-down menu, select Network and click Add.
7 From the Adapter Type drop-down menu, select VMXNET 2 (Enhanced) or VMXNET 3.
8 Set the network settings to the ones recorded for the old network adapter.
9 Set the MAC Address to Manual, and type the MAC address that the old network adapter was using.
10 Click OK.
What to do next
n Check that the enhanced VMXNET adapter is connected to a standard switch or to a distributed
switch with jumbo frames enabled.
n Inside the guest operating system, configure the network adapter to allow jumbo frames. See the
documentation of your guest operating system.
n Configure all physical switches and any physical or virtual machines to which this virtual machine
connects to support jumbo frames.
TSO on the transmission path of physical network adapters, and VMkernel and virtual machine network
adapters improves the performance of ESXi hosts by reducing the overhead of the CPU for TCP/IP
network operations. When TSO is enabled, the network adapter divides larger data chunks into TCP
segments instead of the CPU. The VMkernel and the guest operating system can use more CPU cycles
to run applications.
To benefit from the performance improvement that TSO provides, enable TSO along the data path on an
ESXi host including physical network adapters, VMkernel and guest operating system. By default, TSO is
enabled in the VMkernel of the ESXi host , and in the VMXNET 2 and VMXNET 3 virtual machine
adapters.
For information about the location of TCP packet segmentation in the data path, see VMware Knowledge
Base article Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a
VMware environment.
Procedure
u Run these esxcli network nic software set console commands to enable or disable the
software simulation of TSO in the VMkernel.
where X in vmnicX represents the number of the NIC ports on the host.
Procedure
u Run this esxcli network nic software get console command to determine whether TSO is
enabled on the physical network adapters on a host.
By default, a host uses hardware TSO if its physical adapters support it.
Procedure
4 Edit the value of the Net.UseHwTSO parameter for IPv4 and of Net.UseHwTSO6 for IPv6.
6 To reload the driver module of the physical adapter, run the esxcli system module set console
command in the ESXi Shell on the host.
a To disable the driver, run the esxcli system module set command with the --enabled false
option.
b To enable the driver, run the esxcli system module set command with the --enabled true
option.
If a physical adapter does not support hardware TSO, the VMkernel segments large TCP packets coming
from the guest operating system and sends them to the adapter.
Procedure
Net.UseHwTSO shows the TSO state for IPv4, and Net.UseHwTSO6 for IPv6. TSO is enabled if the
property is set to 1.
Prerequisites
n Verify that the network adapter on the Linux virtual machine is VMXNET2 or VMXNET3.
Procedure
u In a terminal window on the Linux guest operating system, to enable or disable TSO, run the ethtool
command with the -K and tso options.
where Y in ethY is the sequence number of the NIC in the virtual machine.
Prerequisites
n Verify that ESXi supports the Windows guest operating system. See the VMware Compatibility Guide
documentation.
n Verify that the network adapter on the Windows virtual machine is VMXNET2 or VMXNET3.
Procedure
1 In the Network and Sharing Center on the Windows control panel, click the name of the network
adapter.
3 Click Properties, and beneath the network adapter type, click Configure.
4 On the Advanced tab, set the Large Send Offload V2 (IPv4) and Large Send Offload V2 (IPv6)
properties to Enabled or Disabled.
5 Click OK.
LRO reassembles incoming network packets into larger buffers and transfers the resulting larger but
fewer packets to the network stack of the host or virtual machine. The CPU has to process fewer packets
than when LRO is disabled, which reduces its utilization for networking especially in the case of
connections that have high bandwidth.
To benefit from the performance improvement of LRO, enable LRO along the data path on an ESXi host
including VMkernel and guest operating system. By default, LRO is enabled in the VMkernel and in the
VMXNET3 virtual machine adapters.
For information about the location of TCP packet aggregation in the data path, see VMware Knowledge
Base article Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a
VMware environment.
Procedure
vSphere supports software LRO for both IPv4 and IPv6 packets.
Prerequisites
Procedure
Prerequisites
Procedure
4 Examine the value of the LRO parameters for VMXNET2 and VMXNET3.
n For hardware LRO, examine the Net.Vmxnet3HwLRO parameter. If it is equal to 1, hardware LRO
is enabled.
n For software LRO, examine the Net.Vmxnet3SwLRO parameter. If it is equal to 1, hardware LRO
is enabled.
Procedure
4 Enter a value between 1 and 65535 for the Net.VmxnetLROMaxLength parameter to set the LRO
buffer size in bytes.
Procedure
n To enable LRO for the VMkernel network adapters on the host, set Net.TcpipDefLROEnabled to
1.
n To disable software LRO for the VMkernel network adapters on the host, set
Net.TcpipDefLROEnabled to 0.
Procedure
4 Enter a value between 1 and 65535 for the Net.TcpipDefLROMaxLength parameter to set the LRO
buffer size in bytes.
Prerequisites
Procedure
u In a terminal window on the Linux guest operating system, run the ethtool command with the -K
and lro options.
where Y in ethY is the sequence number of the NIC in the virtual machine.
where Y in ethY is the sequence number of the NIC in the virtual machine.
On Windows, the LRO technology is also referred to as Receive Side Coalescing (RSC).
Prerequisites
n Verify that the virtual machine runs Windows Server 2012 and later or Windows 8 and later.
n Verify that the virtual machine compatibility is ESXi 6.0 and later.
n Verify that the version of the VMXNET3 driver installed on the guest operating system is 1.6.6.0 and
later.
n Verify that LRO is enabled globally on a virtual machine that runs Windows Server 2012 and later or
Windows 8 and later. See Enable LRO Globally on a Windows Virtual Machine.
Procedure
1 In the Network and Sharing Center of the guest operating system's Control Panel, click the name of
the network adapter.
2 Click Properties, and under the VMXNET3 network adapter type, click Configure.
3 On the Advanced tab, set both Recv Segment Coalescing (IPv4) and Recv Segment Coalescing
(IPv6) to Enabled or Disabled.
4 Click OK.
Procedure
1 To verify whether LRO is disabled globally on a Windows Windows 8 and later or Windows Server
2012 guest OS, run the netsh int tcp show global command at the command prompt.
The command displays the status of the global TCP parameters that are set on the Windows 8.x OS.
If LRO is globally disabled on the Windows 8 and later or Windows Server 2012 machine, the
Receive Segment Coalescing State property appears as disabled.
2 To enable LRO globally on the Windows OS, run the netsh int tcp set global command at the
command prompt:
What to do next
Enable LRO for the VMXNET3 adapter on the Windows 8 and later or Windows Server 2012 virtual
machine. See Enable or Disable LRO on a VMXNET3 Adapter on a Windows Virtual Machine.
The NetQueue balancer in ESXi uses load balancing algorithms to effectively utilize Rx queues in the
physical NICs by managing vNIC and VMkernel adapter filters.
You can enable or disable different types of Rx queues. For more information, see the esxcli network
nic queue loadbalancer set command in the vSphere Command-Line Interface Reference
documentation.
Prerequisites
Procedure
2 Use the esxcli module parameters set command to configure the NIC driver to use NetQueue.
For example, on a dual-port Emulex NIC run this ESXCLI commands to configure the driver with 8
receive queues.
Prerequisites
Familiarize yourself with the information on configuring NIC drivers in Getting Started with vSphere
Command-Line Interfaces.
Procedure
1 In the VMware vSphere CLI, use the following command depending on the host version:
2 To disable NetQueue on the NIC driver, use the esxcli module parameters set command.
For example, on a dual-port Emulex NIC, run this ESXCLI commands to configure the driver with 1
receive queues.
Version 3 of the Network I/O Control feature offers improved network resource reservation and allocation
across the entire switch.
The two traffic categories have different nature. System traffic is strictly associated with an ESXi host. The
network traffic routes change when you migrate a virtual machine across the environment. To provide
network resources to a virtual machine regardless of its host, in Network I/O Control you can configure
resource allocation for virtual machines that is valid in the scope of the entire distributed switch.
Availability of Features
SR-IOV is not available for virtual machines configured to use Network I/O Control version 3.
Procedure
4 Click OK.
When enabled, the model that Network I/O Control uses to handle bandwidth allocation for system traffic
and virtual machine traffic is based on the Network I/O Control version that is active on the distributed
switch. See About vSphere Network I/O Control Version 3.
You can use Network I/O Control on a distributed switch to configure bandwidth allocation for the traffic
that is related to the main vSphere features:
n Management
n Fault Tolerance
n NFS
n vSAN
n vMotion
n vSphere Replication
n Virtual machine
vCenter Server propagates the allocation from the distributed switch to each physical adapter on the
hosts that are connected to the switch.
For example, on a distributed switch that is connected to ESXi hosts with 10 GbE network adapters, you
might configure reservation to guarantee 1 Gbps for management through vCenter Server, 1 Gbps for
vSphere Fault Tolerance, 1 Gbps for vSphere vMotion traffic, and 0.5 Gbps for virtual machine traffic.
Network I/O Control allocates the requested bandwidth on each physical network adapter. You can
reserve no more than 75 percent of the bandwidth of a physical network adapter, that is, no more than 7.5
Gbps.
You might leave more capacity unreserved to let the host allocate bandwidth dynamically according to
shares, limits, and use, and to reserve only bandwidth that is enough for the operation of a system
feature.
To enable bandwidth allocation for virtual machines by using Network I/O Control, configure the virtual
machine system traffic. The bandwidth reservation for virtual machine traffic is also used in admission
control. When you power on a virtual machine, admission control verifies that enough bandwidth is
available.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
Procedure
You see the bandwidth allocation for the types of system traffic.
4 Select the traffic type according to the vSphere feature that you want to provision and click Edit.
5 From the Shares drop-down menu, edit the share of the traffic in the overall flow through a physical
adapter.
Network I/O Control applies the configured shares when a physical adapter is saturated.
You can select an option to set a pre-defined value, or select Custom and type a number from 1 to
100 to set another share.
6 In the Reservation text box, enter a value for the minimum bandwidth that must be available for the
traffic type.
The total reservation for system traffic must not exceed 75% of the bandwidth supported by the
physical adapter with the lowest capacity of all adapters connected to the distributed switch.
7 In the Limit text box, set the maximum bandwidth that system traffic of the selected type can use.
vCenter Server propagates the allocation from the distributed switch to the host physical adapters that are
connected to the switch.
For example, if the virtual machine system traffic has 0.5 Gbps reserved on each 10 GbE uplink on a
distributed switch that has 10 uplinks, then the total aggregated bandwidth available for VM reservation
on this switch is 5 Gbps. Each network resource pool can reserve a quota of this 5 Gbps capacity.
The bandwidth quota that is dedicated to a network resource pool is shared among the distributed port
groups associated with the pool. A virtual machine receives bandwidth from the pool through the
distributed port group the VM is connected to.
By default, distributed port groups on the switch are assigned to a network resource pool, called default,
whose quota is not configured.
Figure 11‑1. Bandwidth Aggregation for Network Resource Pools Across the Uplinks of a
vSphere Distributed Switch
VM VM VM VM VM
ESXi Host ESXi Host ESXi Host ESXi Host ESXi Host
Bandwidth
reservation for
VM system
traffic: 0.5 Gbps
vmnic0 vmnic1 vmnic0 vmnic1 vmnic0 vmnic1 vmnic0 vmnic1 vmnic0 vmnic1
10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps
The total bandwidth reservation of the virtual machines on a host cannot exceed the reserved bandwidth
that is configured for the virtual machine system traffic.
The actual limit and reservation also depends on the traffic shaping policy for the distributed port group
the adapter is connected to. For example, if a VM network adapter requires a limit of 200 Mbps and the
average bandwidth configured in the traffic shaping policy is 100 Mbps, then the effective limit becomes
100Mbps.
Figure 11‑2. Configuration for Bandwidth Allocation for Individual Virtual Machines
VM VM
Tenant A
VM 50 2 Gbps 0.2 Gbps
Tenant B
VM 50 2 Gbps 0.3 Gbps
vmnic0
10 Gbps
Shares The relative priority, from 1 to 100, of the traffic through this VM
network adapter against the capacity of the physical adapter
that is carrying the VM traffic to the network.
n A physical adapter on the host can supply the minimum bandwidth to the VM network adapters in
accordance with the teaming policy and reservation.
n The reservation for a VM network adapter is less than the free quota in the network resource pool.
If you change the reservation for a network adapter of a running virtual machine, Network I/O Control
verifies again whether the associated network resource pool can accommodate the new reservation. If
the pool does not have enough unclaimed quota, the change is not applied.
To use admission control in vSphere Distributed Switch, perform the following tasks:
n Configure bandwidth allocation for the virtual machine system traffic on the distributed switch.
n Configure a network resource pool with a reservation quota from the bandwidth configured for virtual
machine system traffic.
n Associate the network resource pool with the distributed port group that connects the virtual machines
to the switch.
n Configure the bandwidth requirements of a virtual machine connected to the port group.
vSphere DRS migrates a virtual machine to another host to satisfy the bandwidth reservation of the virtual
machine in these situations:
n The reservation is changed to a value that the initial host can no longer satisfy.
n A physical adapter that carries traffic from the virtual machine is offline.
n Configure bandwidth allocation for the virtual machine system traffic on the distributed switch.
n Configure the bandwidth requirements of a virtual machine that is connected to the distributed switch.
For more information about resource management according to bandwidth demands of virtual machines,
see the vSphere Resource Management documentation.
n Configure the bandwidth requirements of a virtual machine that is connected to the distributed switch.
For more information about vSphere HA provides failover based on the bandwidth demands of virtual
machines, see the vSphere Availability documentation.
A network resource pool provides a reservation quota to virtual machines. The quota represents a portion
of the bandwidth that is reserved for virtual machine system traffic on the physical adapters connected to
the distributed switch. You can set aside bandwidth from the quota for the virtual machines that are
associated with the pool. The reservation from the network adapters of powered on VMs that are
associated with the pool must not exceed the quota of the pool. See About Allocating Bandwidth for
Virtual Machines.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
n Verify that the virtual machine system traffic has a configured bandwidth reservation. See Configure
Bandwidth Allocation for System Traffic.
Procedure
5 (Optional) Type a name and a description for the network resource pool.
6 Enter a value for Reservation quota, in Mbps, from the free bandwidth that is reserved for the virtual
machine system traffic.
The maximum quota that you can assign to the pool is determined according to the following formula:
max reservation quota = aggregated reservation for vm system traffic - quotas of the other
resource pools
where
n quotas of the other pools = the sum of the reservation quotas of the other network resource
pools
7 Click OK.
What to do next
Add one or more distributed port groups to the network resource pool so that you can allocate bandwidth
to individual virtual machines from the quota of the pool. See Add a Distributed Port Group to a Network
Resource Pool.
To assign a network resource pool to several distributed port groups at once, you can use the Resource
allocation policy in the Manage Distributed Port Groups wizard. See Manage Policies for Multiple Port
Groups on a vSphere Distributed Switch.
Network I/O Control allocates bandwidth to the virtual machines associated with the distributed port group
according to the model implemented in the Network I/O Control version that is active on the distributed
switch. See About vSphere Network I/O Control Version 3.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
Procedure
2 Select the distributed port group and click Edit distributed port group settings.
4 From the Network resource pool drop-down menu, select the network resource pool and click OK.
If the distributed switch does not contain network resource pools, you see only the (default) option in
the drop-down menu.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
n Verify that the virtual machine system traffic has a configured bandwidth reservation. See Configure
Bandwidth Allocation for System Traffic.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
2 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
3 Click Edit.
5 If you want to configure bandwidth allocation for a new VM network adapter, from the New device
drop-down menu select Network and click Add.
A New Network section displays options for bandwidth allocation and other network adapter settings.
6 If the VM network adapter is not connected to the distributed port group, select the port group from
the drop-down menu next to the Network adapter X or New Network label.
The Shares, Reservation, and Limit settings appear for the VM network adapter.
7 From the Shares drop-down menu, set the relative priority of the traffic from this virtual machine as
shares from the capacity of the connected physical adapter.
Network I/O Control applies the configured shares when a physical adapter is saturated.
You can select an option to set a pre-defined value, or select Custom and type a number from 1 to
100 to set another share.
8 In the Reservation text box, reserve a minimum bandwidth that must be available to the VM network
adapter when the virtual machine is powered on.
If you provision bandwidth by using a network resource pool, the reservation from the network
adapters of powered on VMs that are associated with the pool must not exceed the quota of the pool.
If vSphere DRS is enabled, to power on the virtual machine, make sure that the reservation from all
VM network adapters on the host does not exceed the bandwidth reserved for virtual machine system
traffic on the host physical adapters.
9 In the Limit text box, set a limit on the bandwidth that the VM network adapter can consume.
10 Click OK.
Network
I/O Control allocates the bandwidth that you reserved for the network adapter of the virtual machine out of
the reservation quota of the network resource pool.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
n Verify that the virtual machine system traffic has a configured bandwidth reservation. See Configure
Bandwidth Allocation for System Traffic.
n Verify that the virtual machines are associated with a specific network resource pool through the
connected distributed port groups. See Add a Distributed Port Group to a Network Resource Pool.
Procedure
A list of the VM network adapters that are connected to the selected network resource pool appears.
6 Select the VM network adapters whose settings you want to configure and click Edit.
7 From the Shares drop-down menu, set the relative priority of traffic from these virtual machines in the
scope of the physical adapters that carry the traffic.
Network I/O Control applies the configured shares when a physical adapter is saturated.
8 In the Reservation text box, reserve a minimum bandwidth that must be available to each VM
network adapter when the virtual machines are powered on.
If you provision bandwidth by using a network resource pool, the reservation from the network
adapters of powered on VMs that are associated with the pool must not exceed the quota of the pool.
9 In the Limit text box, set a limit on the bandwidth that each VM network adapter can consume.
10 Click OK.
Prerequisites
n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed
Switch.
n Verify that the virtual machine system traffic has a configured bandwidth reservation. See Configure
Bandwidth Allocation for System Traffic.
Procedure
4 Select a network resource pool from the list and click Edit.
5 In the Reservation quota text box, enter the bandwidth quota for virtual machines from the
aggregation of free bandwidth that is reserved for virtual machine system traffic on all physical
adapters on the switch.
6 Click OK.
Procedure
2 Select the distributed port group and click Edit distributed port group settings.
3 In the Edit Settings dialog box for the port group, click General.
4 From the Network resource pool drop-down menu, select (default) and click OK.
The distributed port group becomes associated with the default VM network resource pool.
Prerequisites
Uncouple the network resource pool from all associated distributed port groups. See Remove a
Distributed Port Group from a Network Resource Pool.
Procedure
For example, if the bandwidth allocation on a vSphere Distributed Switch is tailored on top of 10 GbE
NICs, you might not be able to add a 1GbE NIC to the switch because it cannot meet the higher allocation
requirements configured on the 10GbE NICs.
Prerequisites
Procedure
2 On the Configure tab, expand System and select Advanced System Settings .
3 Set the physical adapters that you need to function outside the scope of Network I/O Control as a
comma-separated list to the Net.IOControlPnicOptOut parameter.
Each network adapter manufacturer is assigned a unique three-byte prefix called an Organizationally
Unique Identifier (OUI), which it can use to generate unique MAC addresses.
VMware supports several address allocation mechanisms, each of them with a separate OUI:
n Generated for legacy virtual machines, but no longer used with ESXi
If you reconfigure the network adapter of a powered off virtual machine, for example by changing the
automatic MAC address allocation type, or setting a static MAC address, vCenter Server resolves any
MAC address conflict before the adapter reconfiguration takes effect.
The following schemes of MAC address generation are available in vCenter Server:
n Prefix-based allocation
n Range-based allocation
After the MAC address is generated, it does not change unless the virtual machine's MAC address
conflicts with that of another registered virtual machine. The MAC address is saved in the configuration
file of the virtual machine.
Note If you use invalid prefix- or range-based allocation values, an error is logged in the vpxd.log file.
vCenter Server does not allocate MAC addresses when provisioning a virtual machine.
When a virtual machine is powered on again, it might acquire a different MAC address. The change might
be caused by an address conflict with another virtual machine. While this virtual machine has been
powered off, its MAC address has been assigned to another virtual machine that has been powered on.
If you reconfigure the network adapter of a powered off virtual machine, for example, by changing the
automatic MAC address allocation type or setting a static MAC address, vCenter Server resolves MAC
address conflicts before the adapter reconfiguration takes effect.
For information about resolving MAC address conflicts, see the vSphere Troubleshooting documentation.
VMware OUI allocation is the default MAC address assignment model for virtual machines. The allocation
works with up to 64 vCenter Server instances, and each vCenter Server can assign up to 64000 unique
MAC addresses. The VMware OUI allocation scheme is suitable for small scale deployments.
The addresses created through the VMware OUI allocation are in the range 00:50:56:80:YY:ZZ -
00:50:56:BF:YY:ZZ.
Prefix-based MAC address allocation overcomes the limits of the default VMware allocation to provide
unique addresses in larger scale deployments. Introducing an LAA prefix leads to a very large MAC
address space (2 to the power of 46) instead of an universally unique address OUI which can give only
16 million MAC addresses.
Verify that the prefixes that you provide for different vCenter Server instances in the same network are
unique. vCenter Server relies on the prefixes to avoid MAC address duplication issues. See the vSphere
Troubleshooting documentation.
You specify one or more ranges using a starting and ending MAC addresses, for example,
(02:50:68:00:00:02, 02:50:68:00:00:FF). MAC addresses are generated only from within the
specified range.
You can specify multiple ranges of LAA, and vCenter Server tracks the number of used addresses for
each range. vCenter Server allocates MAC addresses from the first range that still has addresses
available. vCenter Server checks for MAC address conflicts within its ranges.
When using range-based allocation, you must provide different instances of vCenter Server with ranges
that do not overlap. vCenter Server does not detect ranges that might be in conflict with other
vCenter Server instances. See the vSphere Troubleshooting documentation for more information about
resolving issues with duplicate MAC addresses.
If you are changing from one type of allocation to another, for example changing from the VMware OUI
allocation to a range-based allocation, use the vSphere Web Client. However, when a schema is prefix-
based or range-based and you want to change to a different allocation schema, you must edit the
vpxd.cfd file manually and restart vCenter Server.
Change the allocation scheme from the default VMware OUI to range- or to prefixed-based allocation by
using the Advanced Settings available for the vCenter Server instance in the vSphere Web Client.
To switch from range- or prefixed-based allocation back to VMware OUI allocation, or between range- and
prefixed-based allocation, edit the vpxd.cfg file manually. See Set or Change Allocation Type.
Procedure
3 Click Edit.
config.vpxd.macAllocScheme.prefixScheme.prefix 005026
config.vpxd.macAllocScheme.prefixScheme.prefixLength 23
prefix and prefixLength determine the range of MAC address prefixes that newly added
vNICs have. prefix is the starting OUI of MAC addresses related to the vCenter Server
instance, and prefixLength determines the length of the prefix in bits.
For example, the settings from the table result in VM NIC MAC addresses starting with 00:50:26
or 00:50:27.
config.vpxd.macAllocScheme.rangeScheme.range[X].begin 005067000000
config.vpxd.macAllocScheme.rangeScheme.range[X].end 005067ffffff
X in range[X] stands for the range sequence number. For example, 0 in range[0] represents
the allocation settings of the first range for MAC address allocation.
5 Click OK.
Prerequisites
Decide on an allocation type before changing the vpxd.cfg file. For information on allocation types, see
MAC Address Assignment from vCenter Server
Procedure
1 On the host machine of vCenter Server, navigate to the directory that contains the configuration file:
3 Decide on an allocation type to use and enter the corresponding XML code in the file to configure the
allocation type.
<vpxd>
<macAllocScheme>
<VMwareOUI>true</VMwareOUI>
</macAllocScheme>
</vpxd>
u Prefix-based allocation
<vpxd>
<macAllocScheme>
<prefixScheme>
<prefix>005026</prefix>
<prefixLength>23</prefixLength>
</prefixScheme>
</macAllocScheme>
</vpxd>
u Range-based allocation
<vpxd>
<macAllocScheme>
<rangeScheme>
<range id="0">
<begin>005067000001</begin>
<end>005067000001</end>
</range>
</rangeScheme>
</macAllocScheme>
</vpxd>
The ESXi host generates the MAC address for a virtual machine adapter in one of the following cases:
n The virtual machine configuration file does not contain the MAC address and information about the
MAC address allocation type.
If you import a virtual machine with a host-generated MAC address from one vCenter Server to another,
select the I Copied It option when you power on the virtual machine to regenerate the address and avoid
potential conflicts in the target vCenter Server or between the vCenter Server systems.
The following cases show when you might set a static MAC address:
n Virtual machine adapters on different physical hosts share the same subnet and are assigned the
same MAC address, causing a conflict.
n Ensure that a virtual machine adapter always has the same MAC address.
By default, VMware uses the Organizationally Unique Identifier (OUI) 00:50:56 for manually generated
addresses, but all unique manually generated addresses are supported.
Note Make sure that no other non-VMware devices use addresses assigned to VMware components.
For example, you might have physical servers in the same subnet, which use 11:11:11:11:11:11,
22:22:22:22:22:22 as static MAC addresses. The physical servers do not belong to the vCenter Server
inventory, and vCenter Server is not able to check for address collision.
If you decide to use the VMware OUI, part of the range is reserved for use by vCenter Server, host
physical NICs, virtual NICs, and for future use.
You can set a static MAC address that contains the VMware OUI prefix in compliance with the following
format:
00:50:56:XX:YY:ZZ
where XX is a valid hexadecimal number between 00 and 3F, and YY and ZZ are valid hexadecimal
numbers between 00 and FF. To avoid conflict with MAC addresses that are generated by vCenter Server
or are assigned to VMkernel adapters for infrastructure traffic, the value for XX must not be greater than
3F.
00:50:56:3F:FF:FF
To avoid conflicts between the generated MAC addresses and the manually assigned ones, select a
unique value for XX:YY:ZZ from your hard-coded addresses.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware.
4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
Procedure
a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
b Click Virtual Machines and double-click the virtual machine from the list.
3 On the Configure tab of the virtual machine, expand Settings and select VM Options.
4 Click Edit and expand Advanced from the VM Options tab within the dialog box displaying the
settings.
Parameter Value
ethernetX.addressType static
ethernetX.address MAC_address_of_the_virtual_NIC
X next to ethernet stands for the sequence number of the virtual NIC in the virtual machine.
For example, 0 in ethernet0 represents the settings of the first virtual NIC device added to the virtual
machine.
7 Click OK.
IPv6 is designated by the Internet Engineering Task Force (IETF) as the successor to IPv4 providing the
following benefits:
n Increased address length. The increased address space resolves the problem of address exhaustion
and eliminates the need for network address translation. IPv6 uses 128-bit addresses compared with
the 32-bit addresses used by IPv4.
n vSphere DPM over Intelligent Platform Management Interface (IPMI) and Hewlett-Packard Integrated
Lights-Out (iLO). vSphere 6.5 supports only Wake-On-LAN (WOL) to bring a host out of standby
mode.
n vSAN
n Authentication Proxy
n vSphere Management Assistant and vSphere Command-Line Interface connected to Active Directory.
Use LDAP to connect the vSphere Management Assistant or the vSphere Command-Line Interface to
the Active Directory database.
Configuring one or more IPv6 addresses is also possible when you customize the guest operating system
of a virtual machine.
To deploy vCenter Server in a pure IPv6 environment, you must use FQDNs only.
If you plan to deploy vCenter Server and ESXi hosts in an IPv6 network, you must perform additional
steps.
n Enable IPv6 on a vSphere Installation
If you have a greenfield deployment of vSphere 6.5 in an IPv6 network, configure ESXi and
vCenter Server for pure IPv6 management connection by configuring IPv6 on the deployment nodes
and connecting them.
Prerequisites
n Verify that the IPv6 addresses for vCenter Server, the ESXi hosts and an external database, if used,
are mapped to fully qualified domain names (FQDNs) on the DNS server.
n Verify that the network infrastructure provides IPv6 connectivity for the ESXi hosts, vCenter Server
and external database if used.
n Verify that you have version 6.5 of vCenter Server installed with FQDN that is mapped to an IPv6
address. See the vCenter Server Installation and Setup documentation.
n Verify that the hosts have ESXi 6.5 installed. See the vCenter Server Installation and Setup
documentation.
Procedure
1 In the Direct Console User Interface (DCUI), configure each ESXi host as a pure IPv6 node.
b From the Configure Management Network menu, select IPv6 Configuration and press Enter.
Automatic address assignment using 1 Select the Use dynamic IPv6 address and network configuration
DHCPv6 option and select Use DHCPv6.
2 Press Enter to save the changes.
Static address assignment 1 Select the Set static IPv6 address and network configuration option
and enter the IPv6 address of the host and the default gateway.
2 Press Enter to save the changes.
d From the Configure Management Network menu, select IPv4 Configuration and press Enter.
e Select Disable IPv4 configuration for management network and press Enter.
Prerequisites
n Verify that the network infrastructure provides IPv6 connectivity for the ESXi hosts, vCenter Server
and external database if used.
n Verify that the IPv6 addresses for vCenter Server, the ESXi hosts and an external database, if used,
are mapped to fully qualified domain names (FQDNs) on the DNS server.
n Verify that you have version 6.x of vCenter Server installed or upgraded. See the vCenter Server
Installation and Setup and vCenter Server Upgrade documentation.
n Verify that all ESXi hosts are upgraded to version 6.x. See the VMware ESXi Upgrade
documentation.
Procedure
1 In the vSphere Web Client, disconnect the hosts from vCenter Server.
Static address assignment 1 Open an SSH connection and log in to the ESXi host.
2 Set a static IPv6 address for the management network vmk0 by running
the following command:
3 Set the default gateway for the management network vmk0 by running the
following command:
Automatic address assignment using 1 Open an SSH connection and log in to the ESXi host.
DHCPv6 2 Enable DHCPv6 for the management network vmk0 by running the
following command:
4 If vCenter Server uses an external database, configure the database as an IPv6 node.
Procedure
3 Click Edit.
4 From the IPv6 support drop-down menu, enable or disable IPv6 support.
5 Click OK.
What to do next
Configure the IPv6 settings of VMkernel adapters on the host, for example, of the management network.
See Set Up IPv6 on an ESXi Host.
Prerequisites
Verify that IPv6 is enabled on the ESXi host. See Enable or Disable IPv6 Support on a Host.
Procedure
3 Select the VMkernel adapter on the target distributed or standard switch and click Edit.
Obtain IPv6 address automatically Receive an IPv6 address for the VMkernel adapter from a DHCPv6 server.
through DHCP
Obtain IPv6 address automatically Receive an IPv6 address for the VMkernel adapter from a router through Router
through Router Advertisement Advertisement.
Static IPv6 addresses Set one or more addresses. For each address entry, enter the IPv6 address of
the adapter, subnet prefix length and IPv6 address of the default gateway.
You can select several assignment options according the configuration of your network.
6 (Optional) From the Advanced Settings section of the IPv6 settings page, remove certain IPv6
addresses that are assigned through Router Advertisement.
You might delete certain IPv6 addresses that the host obtained through Router Advertisement to stop
the communication on these addresses. You might delete all automatically assigned address to
enforce the configured static addresses on the VMkernel.
Procedure
1 On the vSphere Web Client main page, hover over the Home icon, click Home, and select System
Configuration.
Option Description
Obtain IPv6 settings automatically Assigns IPv6 addresses to the appliance automatically from the network by using
through DHCP DHCP.
Obtain IPv6 settings automatically Assigns IPv6 addresses to the appliance automatically from the network by using
through Router Advertisement router advertisement.
Static IPv6 addresses Uses static IPv6 addresses that you set up manually.
1 Click the Add icon.
2 Enter the IPv6 address and the subnet prefix length.
3 Click OK.
4 (Optional) Edit the default gateway.
You can configure the appliance to obtain the IPv6 settings automatically through both DHCP and
router advertisement. You can assign static a IPv6 address at the same time.
7 (Optional) To remove IPv6 addresses that are assigned automatically through Router Advertisement,
click Remove Addresses and delete the addresses.
You might want to delete certain IPv6 addresses that the vCenter Server Appliance obtained through
Router Advertisement to stop the communication on these addresses and to enforce the configured
static addresses.
What to do next
Connect the ESXi hosts to vCenter Server over IPv6 by using their FQDNs.
Procedure
u In the Network and Sharing Center folder of Windows control panel, configure the IPv6 address
settings of the host for the Local Area Connection.
What to do next
Connect the ESXi hosts to vCenter Server over IPv6 by using their FQDNs.
PacketCapture is a lightweight tcpdump utility that captures and stores only the minimum amount of data
that is needed to diagnose the network problem. PacketCapture is integrated in the rhttpproxy service of
ESXi and vCenter Server Appliance. You start and stop PacketCapture by editing the rhttpproxy service
XML configuration file.
Procedure
a Open an SSH connection and log in to the ESXi host or vCenter Server Appliance.
ESXi /etc/vmware/rhttpproxy/config.xml
<config>
<packetCapture>
<enabled>true</enabled>
<directory>/directory_path</directory>
The directory in which pcap and pcap.gz files are stored. The directory must
exist and be accessible.
<maxDataInPcapFile>52428800</maxDataInPcapFile>
The amount of captured data in bytes that each pcap and pcap.gz file can
store before rolling over to the next file. The minimum size is 5 MB on
vCenter Server Appliance and 2.5MB on ESXi.
<maxPcapFilesCount>5</maxPcapFilesCount>
The number of pcap or pcap.gz files to rotate. The minimum number is 2.
a Open an SSH connection and log in to the ESXi host or vCenter Server Appliance.
<config>
<packetCapture>
<enabled>false</enabled>
The pcap or pcap.gz files are stored in the following default directories.
ESXi /var/run/log
What to do next
Copy the pcap and pcap.gz files to a system that runs a network analyzer tool, such as Wireshark, and
examine the packet details.
Before you analyze the pcap and pcap.gz captured from an ESXi host, use the TraceWrangler utility to
fix the frame size metadata. For more information, see https://kb.vmware.com/kb/52843
In vSphere you can monitor packets on a host by using the pktcap-uw console utility. You can use the
utility without additional installation on an ESXi host. pktcap-uw provides many points in the host network
stack at which you can monitor traffic.
For detailed analysis of captured packets, you can save packet content from the pktcap-uw utility to files
in PCAP or PCAPNG format and open them in Wireshark. You can also troubleshoot dropped packets
and trace a packet's path in the network stack.
Note The pktcap-uw utility is not fully supported for backward compatibility across vSphere releases.
The options of the utility might change in the future.
Note Certain options of the pktcap-uw utility are designed for VMware internal use only and you should
use them only under the supervision of VMware Technical Support. These options are not described in
the vSphere Networking guide.
For details about the capture points of the pktcap-uw utility, see Capture Points of the
pktcap-uw Utility.
filter_options Filter captured packets according to source or destination address, VLAN ID, VXLAN ID,
Layer 3 protocol, and TCP port. See pktcap-uw Options for Filtering Packets.
output_control_options Save the contents of a packet to a file, capture only a number of packets, and capture a
number of bytes at the beginning of packets, and so on. See pktcap-uw Options for Output
Control.
The vertical bars | represent alternative values, and the curly brackets {} used with vertical bars specify a
list of choices for an argument or option.
output_control_options Save the contents of a packet to a file and trace only a number
of packets. See pktcap-uw Options for Output Control.
Table 14‑3. Options for Output Control That Are Supported by the pktcap-uw Utility
Option Description
{-o | --outfile} pcap_file Save captured or traced packets to a file in packet capture
(PCAP) format. Use this option to examine packets in a visual
analyzer tool such as Wireshark.
-P | --ng Save packet content in the PCAPNG file format. Use this
option together with the -o or --outfile option.
Table 14‑3. Options for Output Control That Are Supported by the pktcap-uw Utility
(Continued)
Option Description
{-s | --snaplen} snapshot_length Capture only the first snapshot_length bytes from each packet.
If traffic on the host is intensive, use this option to reduce the
load on the CPU and storage.
To limit the size of captured contents, set a value greater than
24.
To capture the complete packet, set this option to 0.
The vertical bars | represent alternative values, and the curly brackets {} used with vertical bars specify a
list of choices for an argument or option.
Filter Options
The filter options for pktcap-uw are valid when you capture and trace packets. For information about the
command syntax of the pktcap-uw utility, see pktcap-uw Command Syntax for Capturing Packets and
pktcap-uw Command Syntax for Tracing Packets.
--srcmac mac_address Capture or trace packets that have a specific source MAC
address. Use colons to separate the octets in it.
--dstmac mac_address Capture or trace packets that have a specific destination MAC
address. Use colons to separate the octets in it.
--ethtype 0xEthertype Capture or trace packets at Layer 2 according to the next level
protocol that consumes packet payload.
EtherType corresponds to the EtherType field in Ethernet
frames . It represents the type of next level protocol that
consumes the payload of the frame.
For example, to monitor traffic for the Link Layer Discovery
Protocol (LLDP) protocol, type --ethtype 0x88CC.
--srcip IP_addess|IP_address/subnet_range Capture or trace packets that have a specific source IPv4
address or subnet.
--dstip IP_addess|IP_address/subnet_range Capture or trace packets that have a specific destination IPv4
address or subnet.
--proto 0xIP_protocol_number Capture or trace packets at Layer 3 according to the next level
protocol that consumes the payload.
For example, to monitor traffic for the UDP protocol, type
--proto 0x11.
--srcport source_port Capture or trace packets according to their source TCP port.
You can specify a certain capture point in the data path between a virtual switch and a physical adapter,
or determine a capture point by traffic direction with regard to the switch and proximity to the packet
source or destination. For information about supported capture points, see Capture Points of the pktcap-
uw Utility.
Procedure
1 (Optional) Find the name of the physical adapter that you want to monitor in the host adapter list.
n In the vSphere Web Client, on the Configure tab for the host, expand Networking and select
Physical adapters.
n In the ESXi Shell to the host, to view a list of the physical adapters and examine their state, run
the following ESXCLI command:
Each physical adapter is represented as vmnicX. X is the number that ESXi assigned to the physical
adapter port.
2 In the ESXi Shell to the host, run the pktcap-uw command with the --uplink vmnicX argument and
with options to monitor packets at a particular point, filter captured packets and save the result to a
file.
where the square brackets [] enclose the options of the pktcap-uw --uplink vmnicX command
and the vertical bars | represent alternative values.
If you run the pktcap-uw --uplink vmnicX command without options, you obtain the content of
packets that are incoming to the standard or distributed switch in the console output at the point
where they are switched.
a Use the --capture option to check packets at another capture point or the --dir option at
another traffic direction.
--capture UplinkRcv Monitor packets immediately after they are received in the network stack from
the physical adapter.
b Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
c Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
3 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
Example: Capture Packets That Are Received at vmnic0 from an IP Address 192.168.25.113
To capture the first 60 packets from a source system that is assigned the IP address 192.168.25.113 at
vmnic0 and save them to a file called vmnic0_rcv_srcip.pcap, run the following pktcap-uw command:
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
You can specify a certain capture point in the data path between a virtual switch and a virtual machine
adapter. You can also determine a capture point by traffic direction with regard to the switch and proximity
to the packet source or destination. For information about supported capture points, see Capture Points
of the pktcap-uw Utility.
Prerequisites
Procedure
1 On the host, learn the port ID of the virtual machine adapter by using the esxtop utility.
a In the ESXi Shell to the host, to start the utility, run esxtop.
c In the USED-BY column, locate the virtual machine adapter, and write down the PORT-ID value
for it.
The USED-BY field contains the name of the virtual machine and the port to which the virtual
machine adapter is connected.
port_ID is the ID that the esxtop utility displays for the virtual machine adapter in the PORT-ID
column.
3 In the ESXi Shell, run the pktcap-uw command with the --switchport port_ID argument and with
options to monitor packets at a particular point, filter captured packets and save the result to a file.
where the square brackets [] enclose the options of the pktcap-uw --switchport port_ID
command and the vertical bars | represent alternative values.
If you run the pktcap-uw --switchport port_ID command without options, you obtain the content
of packets that are incoming to the standard or distributed switch in the console output at the point
when they are switched.
a To check packets at another capture point or direction in the path between the guest operating
system and the virtual switch, use the --capture option or combine the values of the --dir and
--stage options.
--capture Vmxnet3Rx Monitor packets when they arrive to the virtual machine.
--dir 1 --stage 0 Monitor packets immediately after they leave the virtual switch.
--dir 1 Monitor packets immediately before they enter the virtual machine.
--dir 0 --stage 1 Monitor packets immediately after they enter the virtual switch.
b Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
c Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
4 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
Example: Capture Packets That Are Received at a Virtual Machine from an IP Address
192.168.25.113
To capture the first 60 packets from a source that is assigned the IP address 192.168.25.113 when they
arrive at a virtual machine adapter with port ID 33554481 and save them to a file called
vmxnet3_rcv_srcip.pcap, run the following pktcap-uw command:
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
You can capture packets at a certain capture point in the flow between a virtual switch and a VMkernel
adapter. You can also determine a capture point by traffic direction with regard to the switch and proximity
to the packet source or destination. For information about supported capture points, see Capture Points
of the pktcap-uw Utility.
Procedure
1 (Optional) Find the name of the VMkernel adapter that you want to monitor in the VMkernel adapter
list.
n In the vSphere Web Client, expand Networking on the Configure tab for the host and select
VMkernel adapters.
n In the ESXi Shell to the host, to view a list of the physical adapters, run the following console
command:
Each VMkernel adapter is represented as vmkX, where X is the sequence number that ESXi assigned
to the adapter.
2 In the ESXi Shell to the host, run the pktcap-uw command with the --vmk vmkX argument and with
options to monitor packets at a particular point, filter captured packets and save the result to a file.
pktcap-uw --vmk vmkX [--capture capture_point|--dir 0|1 --stage 0|1] [filter_options] [--outfile
pcap_file_path [--ng]] [--count number_of_packets]
where the square brackets [] enclose the options of the pktcap-uw --vmk vmkX command and the
vertical bars | represent alternative values.
You can replace the --vmk vmkX option with --switchport vmkernel_adapter_port_ID, where
vmkernel_adapter_port_ID is the PORT-ID value that the network panel of the esxtop utility displays
for the adapter.
If you run the pktcap-uw --vmk vmkX command without options, you obtain the content of packets
that are leaving the VMkernel adapter.
a To check transmitted or received packets at a specific place and direction, use the --capture
option, or combine the values of the --dir and --stage options.
--dir 1 --stage 0 Monitor packets immediately after they leave the virtual switch.
--dir 1 Monitor packets immediately before they enter the VMkernel adapter.
--dir 0 --stage 1 Monitor packets immediately before they enter the virtual switch.
b Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
c Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
3 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
A packet might be dropped at a point in the network stream for many reasons, for example, a firewall rule,
filtering in an IOChain and DVfilter, VLAN mismatch, physical adapter malfunction, checksum failure, and
so on. You can use the pktcap-uw utility to examine where packets are dropped and the reason for the
drop.
Procedure
1 In the ESXi Shell to the host, run the pktcap-uw --capture Drop command with options to monitor
packets at a particular point, filter captured packets and save the result to a file.
where the square brackets [] enclose the options of the pktcap-uw --capture Drop command and
the vertical bars | represent alternative values.
a Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
b Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
Note You can see the reason and the place where a packet is dropped only when you capture
packets to the console output. The pktcap-uw utility saves only the content of packets to a .pcap
or .pcapng file.
2 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
Besides the contents of dropped packets, the output of the pktcap-uw utility displays the reason for the
drop and the function in the network stack that handled the packet last.
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
DVFilters are agents that reside in the stream between a virtual machine adapter and a virtual switch.
They intercept packets to protect virtual machines from security attacks and unwanted traffic.
Procedure
1 (Optional) To find the name of the DVFilter that you want to monitor, in the ESXi Shell, run the
summarize-dvfilter command.
The output of the command contains the fast-path and slow-path agents of the DVFilters that are
deployed on the host.
2 Run the pktcap-uw utility with the --dvfilter dvfilter_name argument and with options to
monitor packets at a particular point, filter captured packets and save the result to a file.
where the square brackets [] enclose optional items of the pktcap-uw --dvFilter vmnicX
command and the vertical bars | represent alternative values.
a Use the --capture option to monitor packets before or after the DVFilter intercepts them.
b Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
c Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
3 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
A capture point in the pktcap-uw utility represents a place in the path between a virtual switch on one
side and a physical adapter, VMkernel adapter or a virtual machine adapter on the other.
You can use certain capture points in combination with an adapter option. For example, you use the
UplinkRcv point when you capture uplink traffic. You can address other points standalone. For example,
use the Drop point to inspect all dropped packets.
Note Certain capture points of the pktcap-uw utility are designed for VMware internal use only and you
should use them only under the supervision of VMware Technical Support. These capture points are not
described in the vSphere Networking guide.
To examine a packet state or content at a capture point, add the --capturecapture_point option to the
pktcap-uw utility.
For traffic that is related to a physical, VMkernel or VMXNET3 adapter, by combining the --dir and
--stage options you can auto-select and switch between capture points to examine how a packet
changes before and after a point.
The pktcap-uw --uplink vmnicX command supports capture points for functions that handle traffic at
a specific place and direction in the path between the physical adapter and the virtual switch.
UplinkRcv The function that receives packets from the physical adapter.
PortInput The function that passes a list of packets from UplinkRcv to a port on the virtual switch.
PortOutput The function that passes a list of packets from a port on the virtual switch to the UplinkSnd point.
The pktcap-uw --switchport vmxnet3_port_ID command supports capture points for functions that
handle traffic packets at a specific place and direction in the path between a VMXNET3 adapter and a
virtual switch.
VnicRx The function in the virtual machine NIC backend that receives packets from the virtual switch.
VnicTx The function in the virtual machine NIC backend that sends packets from the virtual machine to the virtual
switch.
PortOutput The function that passes a list of packets from a port on the virtual switch to Vmxnet3Rx.
PortInput The function that passes a list of packets from Vmxnet3Tx to a port on the virtual switch. Default capture point
for traffic related to a VMXNET3 adapter.
PortOutput The function that passes a list of packets from a port on the virtual switch to the VMkernel adapter.
PortInput The function that passes a list of packets from the VMkernel adapter to a port on the virtual switch. Default
capture point for traffic related to a VMkernel adapter.
The pktcap-uw --dvfilter divfilter_name command requires a capture point that indicates
whether to capture packets when they enter the DVFilter or when they leave it.
Certain capture points are mapped directly to the network stack rather than to a physical, VMkernel or
VMXNET3 adapter.
Drop Captures dropped packets and shows the place where drops occur.
TcpipDispatch Capture packets at the function that dispatches traffic to the TCP/IP stack of the VMkernel from the virtual
switch, and the reverse.
VdrRxLeaf Capture packets at the receive leaf I/O chain of a dynamic router in VMware NSX. Use this capture point
together with the --lifID option.
VdrRxTerminal Capture packets at the receive terminal I/O chain of a dynamic router in VMware NSX. Use this capture point
together with the --lifID option.
VdrTxLeaf Capture packets at the transmit leaf I/O chain of a dynamic router in VMware NSX. Use this capture point
together with the --lifID option.
VdrTxTerminal Capture packets at the transmit terminal I/O chain of a dynamic router in VMware NSX. Use this capture point
together with the --lifID option.
For information about dynamic routers, see the VMware NSX documentation.
For information about the capture points of the pktcap-uw utility, see Capture Points of the pktcap-uw
Utility.
Procedure
u In the ESXi Shell to the host, run the pktcap-uw -A command to view all capture points that the
pktcap-uw utility supports.
The pktcap-uw utility shows the path of packets together with timestamps that note the time when a
packet is handled by a networking function on ESXi. The utility reports the path of a packet immediately
before it is released from the stack.
To view the full path information for a packet, you must print the result from the pktcap-uw utility in the
console output or save it to a PCAPNG file.
Procedure
1 In the ESXi Shell to the host, run the pktcap-uw --trace command with options to filter traced
packets, save the result to a file and limit the number of traced packets.
where the square brackets [] enclose optional items of the pktcap-uw --trace command and the
vertical bars | represent alternative values.
a Use a filter_options to filter packets according to source and destination address, VLAN ID,
VXLAN ID, Layer 3 protocol, and TCP port.
For example, to monitor packets from a source system that has IP address 192.168.25.113, use
the --srcip 192.168.25.113 filter option.
b Use options to save the contents of each packet or the contents of a limited number of packets to
a .pcap or .pcapng file.
n To save packets to a .pcapng file, use the --ng and --outfile options.
You can open the file in a network analyzer tool such as Wireshark.
By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
Note A .pcap file contains only the contents of traced packets. To collect packet paths besides
packet content, save the output to a .pcapng file.
2 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop
capturing or tracing packets.
What to do next
If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a
graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
Procedure
3 Type the Collector IP address and Collector port of the NetFlow collector.
4 Set an Observation Domain ID that identifies the information related to the switch.
5 To see the information from the distributed switch in the NetFlow collector under a single network
device instead of under a separate device for each host on the switch, type an IPv4 address in the
Switch IP address text box.
6 (Optional) In the Active flow export timeout and Idle flow export timeout text boxes, set the time,
in seconds, to wait before sending information after the flow is initiated.
7 (Optional) To change the portion of data that the switch collects, configure Sampling Rate.
The sampling rate represents the number of packets that NetFlow drops after every collected packet.
A sampling rate of x instructs NetFlow to drop packets in a collected packets:dropped packets ratio
1:x. If the rate is 0, NetFlow samples every packet, that is, collect one packet and drop none. If the
rate is 1, NetFlow samples a packet and drops the next one, and so on.
8 (Optional) To collect data on network activity between virtual machines on the same host, enable
Process internal flows only.
Collect internal flows only if NetFlow is enabled on the physical network device to avoid sending
duplicate information from the distributed switch and the physical network device.
9 Click OK.
What to do next
Enable NetFlow reporting for traffic from virtual machines connected to a distributed port group or a port.
See Enable or Disable NetFlow Monitoring on a Distributed Port Group or Distributed Port.
Port mirroring is used on a switch to send a copy of packets seen on one switch port (or an entire VLAN)
to a monitoring connection on another switch port. Port mirroring is used to analyze and debug data or
diagnose errors on a network.
vMotion
vMotion functions differently depending on which vSphere port mirroring session type you select. During
vMotion, a mirroring path could be temporarily invalid, but it is restored when vMotion completes.
Distributed Port Mirroring Non-uplink distributed port Yes Port mirroring between
source and destination distributed ports can only be
local. If the source and
destination are on different
hosts due to vMotion,
mirroring between them will
not work. However, if the
source and destination move
to the same host, port
mirroring works.
Remote Mirroring Source Non-uplink distributed port Yes When a source distributed
source port is moved from host A to
host B, the original mirroring
path from the source port to
A's uplink is removed on A,
and a new mirroring path from
the source port to B's uplink is
created on B. Which uplink is
used is determined by the
uplink name specified in
session.
IP destination No
When TSO is enabled on a vNIC, the vNIC might send a large packet to a distributed switch. When LRO
is enabled on a vNIC, small packets sent to it might be merged into a large packet.
TSO LRO Packets from the source vNIC might be large packets, and whether they are split is determined
by whether their sizes are larger than the destination vNIC LRO limitation.
TSO Any destination Packets from the source vNIC might be large packets, and they are split to standard packets at
the destination vNIC.
Any source LRO Packets from the source vNIC are standard packets, and they might be merged into larger
packets at the destination vNIC.
Prerequisites
Verify that the vSphere Distributed Switch is version 5.0.0 and later.
Procedure
Procedure
Option Description
Distributed Port Mirroring Mirror packets from a number of distributed ports to other distributed ports on the
same host . If the source and the destination are on different hosts, this session
type does not function.
Remote Mirroring Source Mirror packets from a number of distributed ports to specific uplink ports on the
corresponding host.
Remote Mirroring Destination Mirror packets from a number of VLANs to distributed ports.
Encapsulated Remote Mirroring (L3) Mirror packets from a number of distributed ports to the IP addresses of a remote
Source agent . The virtual machine’s traffic is mirrored to a remote physical destination
through an IP tunnel .
Distributed Port Mirroring (legacy) Mirror packets from a number of distributed ports to a number of distributed ports
and/or uplink ports on the corresponding host .
5 Click Next.
Procedure
1 Set the session properties. Different options are available for configuration depending on which
session type you selected.
Option Description
Name You can enter a unique name for the port mirroring session, or accept the
automatically generated session name.
Status Use the drop down menu to enable or disable the session.
Normal I/O on destination ports Use the drop-down menu to allow or disallow normal I/O on destination ports.
This property is only available for uplink and distributed port destinations.
If you disallow this option, mirrored traffic will be allowed out on destination ports,
but no traffic will be allowed in.
Mirrored packet length (Bytes) Use the check box to enable mirrored packet length in bytes. This puts a limit on
the size of mirrored frames. If this option is selected, all mirrored frames are
truncated to the specified length.
Sampling rate Select the rate at which packets are sampled. This is enabled by default for all
port mirroring sessions except legacy sessions.
Description You have the option to enter a description of the port mirroring session
configuration.
2 Click Next.
You can create a port mirroring session without setting the source and destinations. When a source and
destination are not set, a port mirroring session is created without the mirroring path. This allows you to
create a port mirroring session with the correct properties set. Once the properties are set, you can edit
the port mirroring session to add the source and destination information.
Procedure
1 Select the source of the traffic to be mirrored and the traffic direction.
Depending on the type of port mirroring session you selected, different options are available for
configuration.
Option Description
Add existing ports from a list Click Select distributed ports. A dialog box displays a list of existing ports.
Select the check box next to the distributed port and click OK. You can choose
more than one distributed port.
Add existing ports by port number Click Add distributed ports, enter the port number and click OK.
Set the traffic direction After adding ports, select the port in the list and click the ingress, egress, or
ingress/egress button. Your choice appears in the Traffic Direction column.
Specify the source VLAN If you selected a Remote Mirroring Destination sessions type, you must specify
the source VLAN. ClickAdd to add a VLAN ID. Edit the ID by using the up and
down arrows, or clicking in the field and entering the VLAN ID manually.
2 Click Next.
You can create a port mirroring session without setting the source and destinations. When a source and
destination are not set, a port mirroring session is created without the mirroring path. This allows you to
create a port mirroring session with the correct properties set. Once the properties are set, you can edit
the port mirroring session to add the source and destination information.
Port mirroring is checked against the VLAN forwarding policy. If the VLAN of the original frames is not
equal to or trunked by the destination port, the frames are not mirrored.
Procedure
Depending on which type of session you chose, different options are available.
Option Description
Select a destination distributed port Click Select distributed ports to select ports from a list, or click Add distributed
ports to add ports by port number. You can add more than one distributed port.
Select an uplink Select an available uplink from the list and click Add to add the uplink to the port
mirroring session. You can select more than one uplink.
Select ports or uplinks Click Select distributed ports to select ports from a list, or click Add distributed
ports to add ports by port number. You can add more than one distributed port.
Click Add uplinks to add uplinks as the destination. Select uplinks from the list
and click OK.
Specify IP address Click Add. A new list entry is created. Select the entry and either click Edit to
enter the IP address, or click directly in the IP Address field and type the IP
address. A warning appears if the IP address is invalid.
2 Click Next.
3 Review the information that you entered for the port mirroring session on the Ready to complete
page.
5 Click Finish.
The new port mirroring session appears in the Port Mirroring section of the Settings tab.
Procedure
3 Select a port mirroring session from the list to display more detailed information at the bottom of the
screen. Use the tabs to review configuration details.
5 (Optional) Click Edit to edit the details for the selected port mirroring session.
Procedure
3 Select a port mirroring session from the list and click Edit.
Depending on the type of port mirroring session being edited, different options are available for
configuration.
Option Description
Name You can enter a unique name for the port mirroring session, or accept the
automatically generated session name.
Normal I/O on destination ports Use the drop-down menu to allow or disallow normal I/O on destination ports.
This property is only available for uplink and distributed port destinations.
If you do not select this option, mirrored traffic will be allowed out on destination
ports, but no traffic will be allowed in.
Encapsulation VLAN ID Enter a valid VLAN ID in the field. This information is required for Remote
Mirroring Source port mirroring sessions.
Mark the check box next to Preserve original VLAN to create a VLAN ID that
encapsulates all frames at the destination ports. If the original frames have a
VLAN and Preserve original VLAN is not selected, the encapsulation VLAN
replaces the original VLAN.
Mirrored packet length (Bytes) Use the check box to enable mirrored packet length in bytes. This puts a limit on
the size of mirrored frames. If this option is selected, all mirrored frames are
truncated to the specified length.
Description You have the option to enter a description of the port mirroring session
configuration.
5 On the Sources page, edit sources for the port mirroring session.
Depending on the type of port mirroring session being edited, different options are available for
configuration.
Option Description
Add existing ports from a list Click the Select distributed ports… button. A dialog opens with a list of existing
ports. Select the check box next to the distributed port and click OK. You can
choose more than one distributed port.
Add existing ports by port number Click the Add distributed ports… button, enter the port number and click OK.
Set the traffic direction After adding ports, select the port in the list and click the ingress, egress, or
ingress/egress button. Your choice is displayed in the Traffic Direction column.
Specify the source VLAN If you selected a Remote Mirroring Destination sessions type, you must specify
the source VLAN. Click the Add button to add a VLAN ID. Edit the ID by either
using the up and down arrows, or clicking in the field and entering the VLAN ID
manually.
6 In the Destinations section, edit the destinations for the port mirroring session.
Depending on the type of port mirroring session being edited, different options are available for
configuration.
Option Description
Select a destination distributed port Click the Select distributed ports… button to select ports from a list, or click the
Add distributed ports… button to add ports by port number. You can add more
than one distributed port.
Select a uplinks Select an available uplink from the list and click Add > to add the uplink to the
port mirroring session. You can select more than one uplink.
Select ports or uplinks Click the Select distributed ports… button to select ports from a list, or click the
Add distributed ports… button to add ports by port number. You can add more
than one distributed port.
Click theAdd uplinks... button to add uplinks as the destination. Select uplinks
from the list and click OK.
Specify IP address Click the Add button. A new list entry is created. Select the entry and either click
the Edit button to enter the IP address, or click directly into the IP Address field
and enter the IP address. A warning dialog opens if the IP address is invalid.
7 Click OK.
vSphere runs regular health checks to examine certain settings on the distributed and physical switches
to identify common errors in the networking configuration. The default interval between two health checks
is 1 minute.
Important Depending on the options that you select, vSphere Distributed Switch Health Check can
generate a significant number of MAC addresses for testing teaming policy, MTU size, VLAN
configuration, resulting in extra network traffic. For more information, see
https://kb.vmware.com/s/article/2034795. After you disable vSphere Distributed Switch Health Check, the
generated MAC addresses age out of your physical network environment according to your network
policy.
Required Configuration on
Configuration Error Health Check the Distributed Switch
The VLAN trunk ranges configured on the Checks whether the VLAN settings on the At least two active physical
distributed switch do not match the trunk distributed switch match the trunk port NICs
ranges on the physical switch. configuration on the connected physical switch
ports.
The MTU settings on the physical network Checks whether the physical access switch port At least two active physical
adapters, distributed switch, and physical MTU jumbo frame setting based on per VLAN NICs
switch ports do not match. matches the vSphere distributed switch MTU
setting.
The teaming policy configured on the port Checks whether the connected access ports of At least two active physical
groups does not match the policy on the the physical switch that participate in an NICs and two hosts
physical switch port-channel. EtherChannel are paired with distributed ports
whose teaming policy is IP hash.
Health check is limited to only the access switch port to which the distributed switch uplink connects.
Procedure
Option Description
VLAN and MTU Reports the status of distributed uplink ports and VLAN ranges.
Teaming and Failover Checks for any configuration mismatch between theESXi host and the physical
switch used in the teaming policy.
4 Click OK.
What to do next
When you change the configuration of a vSphere Distributed Switch, you can view information about the
change in the Monitor tab in the vSphere Web Client. See View vSphere Distributed Switch Health
Status.
Prerequisites
Verify that health check for VLAN and MTU, and for teaming policy is enabled on the vSphere Distributed
Switch. See Enable or Disable vSphere Distributed Switch Health Check.
Procedure
3 In the Health Status Details section, examine the overall, VLAN, MTU and teaming health of the hosts
connected to the switch.
vSphere 5.0 and later supports Cisco Discovery Protocol (CDP) and Link Layer Discovery Protocol
(LLDP). CDP is available for vSphere standard switches and vSphere distributed switches connected to
Cisco physical switches. LLDP is available for vSphere distributed switches version 5.0.0 and later.
When CDP or LLDP is enabled for a particular vSphere distributed switch or vSphere standard switch,
you can view properties of the peer physical switch such as device ID, software version, and timeout from
the vSphere Web Client.
Procedure
4 In the Discovery Protocol section, select Cisco Discovery Protocol from the Type drop-down menu.
5 From the Operation drop-down menu, select the operational mode of the ESXi hosts connected to
the switch.
Option Description
Listen ESXi detects and displays information about the associated Cisco switch port, but
information about the vSphere Distributed Switch is not available to the Cisco
switch administrator.
Advertise ESXi makes information about the vSphere Distributed Switch available to the
Cisco switch administrator, but does not detect and display information about the
Cisco switch.
Both ESXi detects and displays information about the associated Cisco switch and
makes information about the vSphere Distributed Switch available to the Cisco
switch administrator.
6 Click OK.
Procedure
4 In the Discovery Protocol section, select Link Layer Discovery Protocol from the Type drop-down
menu.
5 From the Operation drop-down menu, select the operational mode of the ESXi hosts connected to
the switch.
Operation Description
Listen ESXi detects and displays information about the associated physical switch port,
but information about the vSphere Distributed Switch is not available to the switch
administrator.
Advertise ESXi makes information about the vSphere Distributed Switch available to the
switch administrator, but does not detect and display information about the
physical switch.
Both ESXi detects and displays information about the associated physical switch and
makes information about the vSphere Distributed Switch available to the switch
administrator.
6 Click OK.
Procedure
3 Select a physical adapter from the list to view its detailed information.
According to the enabled switch discovery protocol, the properties of the switch appear under the CDP or
LLDP tab. If the information is available in the network, under Peer device capability you can examine the
system capabilities of the switch.
From the diagram you can view the settings of a selected port group and of a selected adapter.
Prerequisites
The topology diagram of an N-VDS provides a visual representation of the adapters and port groups
connected to the switch.
Procedure
The diagram appears under the list of virtual switches on the host.
What to do next
You can use the topology diagram to examine whether a virtual machine or VMkernel adapter is
connected to the external network and to identify the physical adapter that carries the data.
Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
To configure the networking settings of virtual machines by using from network protocol profiles, perform
the following operations:
n Create network profiles at the level of a data center or a vSphere distributed switch.
n Associate a protocol profile with the port group of a vApp virtual machine.
n Enable the transient or static IP allocation policy from the settings of the vApp or from the vApp
options of a virtual machine.
Note If you move a vApp or a virtual machine that retrieves its network settings from a protocol profile to
another data center, to power it on you must assign a protocol profile to the connected port group on the
destination data center.
Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
Note If you move a vApp or a virtual machine that retrieves its network settings from a protocol profile to
another data center, to power on the vApp or virtual machine you must assign a protocol profile to the
connected port group on the destination data center.
Procedure
1 Navigate to a data center that is associated with the vApp and click the Configure tab.
Procedure
3 Click Next.
You can configure network protocol profile ranges for IPv4, IPv6, or both. vCenter Server uses these
ranges to dynamically allocate IP addresses to virtual machines when a vApp is set up to use transient IP
allocation.
Procedure
2 Select DHCP Present to indicate that the DHCP server is available on this network.
5 If you enable IP Pools, enter a comma-separated list of host address ranges in the IP pool range
field.
A range consists of an IP address, a pound sign (#), and a number indicating the length of the range.
The gateway and the ranges must be within the subnet. The ranges that you enter in the IP pool
range field cannot include the gateway address.
For example, 10.20.60.4#10, 10.20.61.0#2 indicates that the IPv4 addresses can range from
10.20.60.4 to 10.20.60.13 and 10.20.61.0 to 10.20.61.1.
6 Click Next.
You can configure network protocol profile ranges for IPv4, IPv6, or both.vCenter Server uses these
ranges to dynamically allocate IP addresses to virtual machines when a vApp is set up to use transient IP
allocation.
Procedure
2 Select DHCP Present to indicate that the DHCP server is available on this network.
5 If you enable IP Pools, enter a comma-separated list of host address ranges in the IP pool range
field.
A range consists of an IP address, a pound sign (#), and a number indicating the length of the range.
For example, assume that you specify the following IP pool range:
fe80:0:0:0:2bff:fe59:5a:2b#10,fe80:0:0:0:2bff:fe59:5f:b1#2
fe80:0:0:0:2bff:fe59:5a:2b - fe80:0:0:0:2bff:fe59:5a:34
and
fe80:0:0:0:2bff:fe59:5f:b1 - fe80:0:0:0:2bff:fe59:5f:b2
The gateway and the ranges must be within the subnet. The ranges that you enter in the IP pool
range field cannot include the gateway address.
6 Click Next.
Procedure
The search paths are specified as a list of DNS domains separated by commas, semi-colons, or
spaces.
4 Enter the server name and port number for the proxy server.
The server name can optionally include a colon and a port number.
5 Click Next.
u Review the settings and click Finish to complete adding the network protocol profile.
You can associate a port group of a standard switch or a distributed port group of a distributed switch with
a network protocol profile by using the settings of the group.
Procedure
1 Navigate to a distributed port group of a vSphere Distributed Switch or to a port group of a vSphere
Standard Switch in the Networking view of the vSphere Web Client.
The port groups of standard switches are under the data center. The vSphere Web Client displays
distributed port groups under the parent distributed switch object.
2 On the Configure tab, expand More and click Network Protocol Profiles.
3 Click Associate a network protocol profile with the selected network button in the upper right
corner.
4 On the Set association type page of the Associate Network Protocol Profile wizard, select Use an
existing network protocol profile and click Next.
If the existing network protocol profiles do not contain settings suitable for the vApp virtual machines
in the port group, you must create a new profile.
6 Examine the association and settings of the network protocol profile, and click Finish.
Prerequisites
Verify that the virtual machine is connected to a port group that is associated with the network protocol
profile.
Procedure
1 In the vSphere Web Client, navigate to the virtual machine or the vApp.
2 Open the settings of the vApp or the vApp Options tab of the virtual machine.
n Right-click a virtual machine, select Edit settings, and in the Edit Settings dialog box, click the
vApp Options tab.
4 Under Authoring, expand IP allocation and set the IP allocation scheme to OVF environment.
5 Under Deployment, expand IP allocation and set IP allocation to Transient - IP Pool or Static - IP
Pool.
Both the Static - IP Pool and Transient - IP Pool options allocate an IP address from the range in
the network protocol profile that is associated with the port group. If you select Static - IP Pool, the IP
address is assigned the first time the virtual machine or vApp is powered. The assigned IP address
persists across restarts. If you select Transient - IP Pool, an IP address is assigned every time the
virtual machine or vApp is powered on.
6 Click OK.
When the virtual machine is powered on, the adapters connected to the port group receive IP addresses
from the range in the protocol profile. When the virtual machine is powered off, the IP addresses are
released.
The switch does not interpret the IGMP messages that a virtual machine sends to join or leave a group.
The switch sends them directly to the local multicast router, which then interprets them to join the virtual
machine to or remove it from the group.
n A virtual machine might receive packets from groups that it is not subscribed for because the switch
forwards packets according to the destination MAC address of a multicast group, which can be
potentially mapped up to 32 IP multicast groups.
n A virtual machine that is subscribed for traffic from more than 32 multicast MAC addresses receives
packets that it is not subscribed for because of a limitation in the forwarding model.
n The switch does not filter packets according to source address as defined in IGMP version 3.
Multicast Snooping
In multicast snooping mode, a vSphere Distributed Switch provides IGMP and MLD snooping according
to RFC 4541. The switch dispatches multicast traffic more precisely by using IP addresses. This mode
supports IGMPv1, IGMPv2, and IGMPv3 for IPv4 multicast group addresses, and MLDv1 and MLDv2 for
IPv6 multicast group addresses.
The switch dynamically detects the membership of a virtual machine. When a virtual machine sends a
packet which contains IGMP or MLD membership information through a switch port, the switch creates a
record about the destination IP address of the group, and in the case of IGMPv3, about a source IP
address that the virtual machine prefers to receive traffic from. If a virtual machine does not renew its
membership to a group within a certain period of time, the switch removes the entry for the group from the
lookup records.
In multicast snooping mode of a distributed switch, a virtual machine can receive multicast traffic on a
single switch port from up to 256 groups and 10 sources.
Use multicast snooping if virtualized workloads on the switch subscribe to more than 32 multicast groups
or must receive traffic from specific source nodes. For information about the multicast filtering modes of
vSphere Distributed Switch, see Multicast Filtering Modes.
Prerequisites
Procedure
3 In the dialog box that displays the settings of the switch, click Advanced.
4 From the Multicast filtering mode drop-down menu, select IGMP/MLD snooping, and click OK.
Multicast snooping becomes active on hosts running ESXi 6.0 and later.
The default time interval for sending snooping queries is 125 seconds.
Procedure
2 On the Configure tab, expand System and select Advanced System Settings.
4 Click Edit and enter a new value in seconds for the setting.
Procedure
2 On the Configure tab, expand System and select Advanced System Settings.
4 Click Edit and enter a new value between 1 and 32 for the setting.
5 Click OK.
Every stateless ESXi boot is like a first boot. The ESXi host boots with networking connectivity to
vCenter Server through the built‐in standard switch. If the host profile specifies distributed switch
membership, vCenter Server joins the ESXi host to VMware distributed switches.
When planning the network setup for stateless ESXi hosts, you should keep the configuration as generic
as possible and avoid host‐specific items. Currently the design has no hooks to reconfigure physical
switches when deploying a new host. Any such requirement would need special handling.
To set up stateless deployment, one ESXi host must be installed in the standard fashion. Then find and
record the following network-related information to save in the host profile:
n vSphere standard switch instances and settings (port groups, uplinks, MTU, and so forth)
n vNIC information:
n Address information (IPv4 or IPv6, static or DHCP, gateway)
n Port groups and distributed port groups assigned to the physical network adapter (vmknic)
n If there are distributed switches, record VLAN, physical NICs bound to the vmknic, and if
Etherchannel is configured
The recorded information is used as a template for the host profile. Once the host profile virtual switch
information has been extracted and placed in the host profile, you have the opportunity to change any of
the information. Modifications are offered for both standard and distributed switches in these sections:
uplink selection policy, based on either vmnic name or device number, and auto discovery based on
VLAN ID. The (possibly modified) information is stored by the stateless boot infrastructure and applied to
a stateless ESXi host on its next boot. During network initialization, a generic network plug‐in interprets
the recorded host profile setting and does the following:
n Loads appropriate physical NIC drivers.
n Creates all standard switch instances, along with port groups. It selects uplinks based on policy. If the
policy is based on the VLAN ID, there is a probing process to gather relevant information.
n For VMkernel network adapters connected to the standard switch, it creates VMkernel network
adapters and connects them to port groups.
n For each VMkernel network adapter connected to a distributed switch, it creates a temporary
standard switch (as needed) with uplinks bound to the VMkernel network adapter. It creates a
temporary port group with VLAN and teaming policies based on recorded information. Specifically, IP‐
hash is used if Etherchannel was used in the distributed switch.
n Configures all VMkernel network adapter settings (assigns address, gateway, MTU, and so forth).
Basic connectivity is functioning, and the networking setup is complete if there is no distributed switch
present.
If there is a distributed switch present, the system stays in maintenance mode until distributed switch
remediation is complete. No virtual machines are started at this time. Because distributed switches
requires vCenter Server, the boot process continues until vCenter Server connectivity is established, and
vCenter Server notices that the host should be part of a distributed switch. It issues a distributed switch
host join, creating a distributed switch proxy standard switch on the host, selects appropriate uplinks, and
migrates the vmknic from the standard switch to the distributed switch. When this operation is complete, it
deletes the temporary standard switch and port groups.
At the end of the remediation process, the ESXi host is taken out of maintenance mode, and HA or DRS
can start virtual machines on the host.
In the absence of a host profile, a temporary standard switch is created with “default networking” logic,
which creates a management network switch (with no VLAN tag) whose uplink corresponds to the PXE
booting vNIC. A vmknic is created on the management network port group with the same MAC address
as the PXE booting vNIC. This logic was previously used for PXE booting. If there is a host profile, but the
networking host profile is disabled or fatally incomplete, vCenter Server falls back to default networking so
that the ESXi host can be managed remotely. This triggers a compliance failure, so vCenter Server then
initiates recovery actions.
n Isolate from one another the networks for host management, vSphere vMotion, vSphere FT, and so
on, to improve security and performance.
n Dedicate a separate physical NIC to a group of virtual machines, or use Network I/O Control and
traffic shaping to guarantee bandwidth to the virtual machines. This separation also enables
distributing a portion of the total networking workload across multiple CPUs. The isolated virtual
machines can then better handle application traffic, for example, from a Web client.
n To physically separate network services and to dedicate a particular set of NICs to a specific network
service, create a vSphere Standard Switch or vSphere Distributed Switch for each service. If this is
not possible, separate network services on a single switch by attaching them to port groups with
different VLAN IDs. In either case, verify with your network administrator that the networks or VLANs
you choose are isolated from the rest of your environment and that no routers connect them.
n Keep the vSphere vMotion connection on a separate network. When migration with vMotion occurs,
the contents of the guest operating system’s memory is transmitted over the network. You can do this
either by using VLANs to segment a single physical network or by using separate physical networks
(the latter is preferable).
For migration across IP subnets and for using separate pools of buffer and sockets, place traffic for
vMotion on the vMotion TCP/IP stack, and traffic for migration of powered-off virtual machines and
cloning on the Provisioning TCP/IP stack. See VMkernel Networking Layer.
n You can add and remove network adapters from a standard or distributed switch without affecting the
virtual machines or the network service that is running behind that switch. If you remove all the
running hardware, the virtual machines can still communicate among themselves. If you leave one
network adapter intact, all the virtual machines can still connect with the physical network.
n To protect your most sensitive virtual machines, deploy firewalls in virtual machines that route
between virtual networks with uplinks to physical networks and pure virtual networks with no uplinks.
n Physical network adapters connected to the same vSphere Standard Switch or vSphere Distributed
Switch should also be connected to the same physical network.
n Configure the same MTU on all VMkernel network adapters in a vSphere Distributed Switch. If
several VMkernel network adapters, configured with different MTUs, are connected to vSphere
distributed switches, you might experience network connectivity problems.
n Hosts on vSphere Distributed Switch 5.0 and Earlier Lose Connectivity to vCenter Server
n Virtual Machines Lose Connectivity After Changing the Uplink Failover Order of a Distributed Port
Group
n Unable to Add a Physical Adapter to a vSphere Distributed Switch That Has Network I/O Control
Enabled
n A Virtual Machine that Runs a VPN Client Causes Denial of Service for Virtual Machines on the Host
or Across a vSphere HA Cluster
n Virtual Machines on the Same Distributed Port Group and on Different Hosts Cannot Communicate
with Each Other
n Attempt to Power On a Migrated vApp Fails Because the Associated Protocol Profile Is Missing
n Networking Configuration Operation Is Rolled Back and a Host Is Disconnected from vCenter Server
Defining the Problem After you have isolated the symptoms of the problem, you must define the
Space problem space. Identify the software or hardware components that are
affected and might be causing the problem and those components that are
not involved.
Testing Possible When you know what the symptoms of the problem are and which
Solutions components are involved, test the solutions systematically until the problem
is resolved.
Identifying Symptoms
Before you attempt to resolve a problem in your implementation, you must identify precisely how it is
failing.
The first step in the troubleshooting process is to gather information that defines the specific symptoms of
what is happening. You might ask these questions when gathering this information:
n Can the affected task be divided into subtasks that you can evaluate separately?
n What has changed recently in the software or hardware that might be related to the failure?
To define the problem space in an implementation of vSphere, be aware of the components present. In
addition to VMware software, consider third-party software in use and which hardware is being used with
the VMware virtual hardware.
Recognizing the characteristics of the software and hardware elements and how they can impact the
problem, you can explore general problems that might be causing the symptoms.
n Incompatibility of components
Break down the process and consider each piece and the likelihood of its involvement separately. For
example, a case that is related to a virtual disk on local storage is probably unrelated to third-party router
configuration. However, a local disk controller setting might be contributing to the problem. If a component
is unrelated to the specific symptoms, you can probably eliminate it as a candidate for solution testing.
Think about what changed in the configuration recently before the problems started. Look for what is
common in the problem. If several problems started at the same time, you can probably trace all the
problems to the same cause.
With the information that you have gained about the symptoms and affected components, you can design
tests for pinpointing and resolving the problem. These tips might make this process more effective.
n Verify that each solution determines unequivocally whether the problem is fixed. Test each potential
solution but move on promptly if the fix does not resolve the problem.
n Develop and pursue a hierarchy of potential solutions based on likelihood. Systematically eliminate
each potential problem from the most likely to the least likely until the symptoms disappear.
n When testing potential solutions, change only one thing at a time. If your setup works after many
things are changed at once, you might not be able to discern which of those things made a difference.
n If the changes that you made for a solution do not help resolve the problem, return the
implementation to its previous status. If you do not return the implementation to its previous status,
new errors might be introduced.
n Find a similar implementation that is working and test it in parallel with the implementation that is not
working properly. Make changes on both systems at the same time until few differences or only one
difference remains between them.
Common Logs
The following logs are common to all deployments on Windows or Linux.
For Platform Services Controller node deployments, additional runtime logs are located at
C:\ProgramData\VMware\CIS\runtime\VMwareSTSService\logs.
Problem
The MAC addresses of virtual machines on the same broadcast domain or IP subnet are in conflict, or
vCenter Server generates a duplicate MAC address for a newly created virtual machine.
A virtual machine powers on and functions properly, but shares a MAC address with another virtual
machine. This situation might cause packet loss and other problems.
Cause
Virtual machines might have duplicate MAC addresses due to several reasons.
n Two vCenter Server instances with identical IDs generate overlapping MAC addresses for virtual
machine network adapters.
Each vCenter Server instance has an ID between 0 and 63 that is randomly generated at installation
time, but can be reconfigured after installation. vCenter Server uses the instance ID to generate MAC
addresses for the network adapters of the machine.
n A virtual machine has been transferred in power-off state from one vCenter Server instance to
another in the same network, for example, by using shared storage, and a new virtual machine
network adapter on the first vCenter Server receives the freed MAC address.
Solution
If you have an existing virtual machine with a conflicting MAC address, you must provide a unique
MAC address in the Virtual Hardware settings.
n Power off the virtual machine, configure the adapter to use a manual MAC address, and type the
new address.
n If you cannot power the virtual machine off for configuration, re-create the network adapter that is
in conflict with enabled manual MAC address assignment and type the new address. In the guest
operating system, set the same static IP address to the re-added adapter as before.
For information about configuring the network adapters of virtual machines, see the vSphere
Networking and vSphere Virtual Machine Administration documentation.
n If the vCenter Server instance generates the MAC addresses of virtual machines according to the
default allocation, VMware OUI, change the vCenter Server instance ID or use another allocation
method to resolve conflicts.
Note Changing the vCenter Server instance ID or switching to a different allocation scheme does
not resolve MAC address conflicts in existing virtual machines. Only virtual machines created or
network adapters added after the change receive addresses according to the new scheme.
For information about MAC address allocation schemes and setup, see the vSphere Networking
documentation.
Solution Description
Change the vCenter Server ID You can keep using the VMware OUI allocation scheme if your deployment
contains a small number of vCenter Server instances. According to this scheme,
a MAC address has the following format:
00:50:56:XX:YY:ZZ
Switch to prefix-based allocation You can use a custom OUI. For example, for a 02:12:34 locally administered
address range, MAC addresses have the form 02:12:34:XX:YY:ZZ. You can use
the fourth octet XX to distribute the OUI address space between the
vCenter Server instances. This structure results in 255 address clusters, each
cluster managed by a vCenter Server instance, and in about 65000 MAC
addresses per vCenter Server. For example, 02:12:34:01:YY:ZZ for vCenter
Server A, 02:12:34:02:YY:ZZ for vCenter Server B, and so on.
Prefix-based allocation is suitable for deployments of a larger scale.
For globally unique MAC addresses, the OUI must be registered in IEEE.
b Apply the new MAC address allocation scheme to an existing virtual machine in its Virtual
Hardware settings.
n Power off a virtual machine, configure the adapter to use a manual MAC address, revert to
automatic MAC address allocation, and power on the virtual machine.
n If the virtual machine is in production and you cannot power it off for configuration, after you
change the vCenter Server ID or the address allocation scheme, re-create the network
adapter in conflict with enabled automatic MAC address assignment. In the guest operating
system, set the same static IP address to the re-added adapter as before.
n Enforce MAC address regeneration when transferring a virtual machine between vCenter Server
instances by using the virtual machine files from a datastore.
a Power off a virtual machine, remove it from the inventory, and in its configuration file (.vmx), set
the ethernetX.addressType parameter to generated.
X next to ethernet stands for the sequence number of the virtual NIC in the virtual machine.
b Import the virtual machine from one vCenter Server system to another by registering the virtual
machine from a datastore in the target vCenter Server.
The virtual machine files can reside in a datastore that is shared between the two vCenter Server
instances or can be uploaded to a datastore that is accessible only from the target
vCenter Server system.
For information about registering a virtual machine from a datastore, see vSphere Virtual Machine
Administration.
While the virtual machine is starting up, an information icon appears on the virtual machine in the
vSphere Web Client.
d Right-click the virtual machine and select Guest OS > Answer Question.
The target vCenter Server re-generates the MAC address of the virtual machine. The new MAC
address starts with the VMware OUI 00:0c:29 and is based on the BIOS UUID of the virtual
machine. The BIOS UUID of the virtual machine is calculated from the BIOS UUID of the host.
n If the vCenter Server and hosts are version 6.0 and later and the vCenter Server instances are
connected in Enhanced Linked Mode, migrate virtual machines by using vMotion across
vCenter Server systems.
When a virtual machine is migrated across vCenter Server systems, the source vCenter Server adds
the MAC address of the virtual machine to a blacklist and does not assign them to other virtual
machines.
Problem
In the vSphere Web Client, after you assign a MAC address within the range 00:50:56:40:YY:ZZ –
00:50:56:7F:YY:ZZ to a virtual machine, attempts to power the virtual machine on fail with a status
message that the MAC address is in conflict.
Cause
You attempt to assign a MAC address which starts with the VMware OUI 00:50:56 and is within the
address range allocated for host VMkernel adapters on the vCenter Server system.
Solution
If you want to preserve the VMware OUI prefix, set a static MAC address within the range
00:50:56:00:00:00 – 00:50:56:3F:FF:FF. Otherwise, set an arbitrary MAC address whose prefix is
different from the VMware OUI one. For information about the ranges available for static MAC addresses
that have the VMware OUI prefix, see the vSphere Networking documentation.
Problem
n Attempts to remove a host from a vSphere distributed switch fail, and you receive a notification that
resources are still in use. The notification that you receive might look like the following:
n Attempts to remove a host proxy switch that still exists on the host from a previous networking
configuration fail. For example, you moved the host to a different data center or vCenter Server
system, or upgraded the ESXi and vCenter Server software, and created new networking
configuration. When trying to remove the host proxy switch, the operation fails because resources on
the proxy switch are still in use.
Cause
You cannot remove the host from the distributed switch or delete the host proxy switch because of the
following reasons.
Solution
Problem Solution
Cannot remove a 1 In the vSphere Web Client, navigate to the distributed switch.
host from a 2 On the Configure tab, select More > Ports.
distributed switch 3 Locate all ports that are still in use and check which VMkernel or virtual machine network adapters on
the host are still attached to the ports.
4 Migrate or delete the VMkernel and virtual machine network adapters that are still connected to the
switch.
5 Use the Add and Manage Hosts wizard in the vSphere Web Client to remove the host from the
switch.
After the host is removed, the host proxy switch is deleted automatically.
Problem
After you change the networking configuration of a port group on a vSphere Distributed Switch that
contains the VMkernel adapters for the management network, the hosts on the switch lose connectivity to
vCenter Server. In the vSphere Web Client the status of the hosts is nonresponsive.
Cause
On a vSphere Distributed Switch in vCenter Server that has networking rollback disabled, the port group
containing the VMkernel adapters for the management network is misconfigured in vCenter Server and
the invalid configuration is propagated to the hosts on the switch.
Note In vSphere networking rollback is enabled by default. However, you can enable or disable
rollbacks at the vCenter Server level. For more information see the vSphere Networking documentation.
Solution
1 From the Direct Console User Interface (DCUI) to an affected host, use the Restore vDS option from
the Network Restore Options menu to configure the uplinks and the ID of the VLAN for the
management network.
The DCUI creates a local ephemeral port and applies the VLAN and uplink configuration to the port.
The DCUI changes the VMkernel adapter for the management network to use the new host local port
to restore connectivity to vCenter Server.
After the host re-connects to vCenter Server, the vSphere Web Client displays a warning that some
hosts on the switch have different networking configuration from the configuration stored in vSphere
distributed switch.
2 In the vSphere Web Client, configure the distributed port group for the management network with
correct settings.
Situation Solution
You have altered the port group You can roll the configuration of the port group back one step. Right-click the port
configuration only once group, click Restore Configuration, and select Restore to previous
configuration.
You have backed up a valid You can restore the configuration of the port group by using the backup file. Right-
configuration of the port group click the port group, click Restore Configuration, and select Restore
configuration from a file.
You can also restore the configuration for the entire switch, including the port
group, from a backup file for the switch.
You have performed more than one You must provide valid settings for the port group manually.
configuration step and you do not have
a backup file
For information about networking rollback, recovery, and restore, see the vSphere Networking
documentation.
3 Migrate the VMkernel adapter for the management network from the host local ephemeral port to a
distributed port on the switch by using the Add and Manage Hosts wizard.
Unlike distributed ports, the ephemeral local port of the VMKernel has a non-numeric ID.
For information about handling VMkernel adapters through the Add and Manage Hosts wizard, see
the vSphere Networking documentation.
4 Apply the configuration of the distributed port group and VMkernel adapter from vCenter Server to the
host.
n Push the correct configuration of the distributed port group and VMkernel adapter from
vCenter Server to the host.
c From the Virtual switches list, select the distributed switch and click Rectify the state of the
selected distributed switch on the host.
n Wait until vCenter Server applies the settings within the next 24 hours.
Problem
After you change the networking configuration of a port group on a vSphere Distributed Switch 5.0 or
earlier that contains the VMkernel adapters for the management network, the hosts on the switch lose
connectivity to vCenter Server. In the vSphere Web Client the status of the hosts is nonresponsive.
Cause
On a vSphere Distributed Switch 5.0 and earlier in vCenter Server, the port group containing the
VMkernel adapters for the management network is misconfigured in vCenter Server and the invalid
configuration is propagated to the hosts on the switch.
Solution
3 In the vSphere Standard Switch view, create a new standard switch if the host does not have a
standard switch suitable for the management network.
b In the Add Network wizard, under Connection Types select Virtual Machine, and click Next.
d Under the Create a vSphere standard switch section, select one or more unoccupied physical
adapters on the host to carry the management traffic and click Next.
If all physical adapters are already busy with traffic from other switches, create the switch without
a physical network adapter connected. Later, remove the physical adapter for the management
network from the proxy switch of the distributed switch and add it to this standard switch.
e In the Port Group Properties section, type a network label that identifies the port group that you
are creating and optionally a VLAN ID.
f Click Finish.
4 In the vSphere Distributed Switch view, migrate the VMkernel adapter for the network to a standard
switch.
a Select the vSphere Distributed Switch view, and for the distributed switch, click Manage Virtual
Adapters.
b In the Manage Virtual Adapters wizard, select the VMkernel adapter from the list and click
Migrate.
c Select the newly created or another standard switch to migrate the adapter to, and click Next.
d Enter a network label that is unique in the scope of the host and optionally a VLAN ID for the
management network, and click Next.
e Review the settings on the target standard switch and click Finish.
5 In the vSphere Web Client, configure the distributed port group for the management network with
correct settings.
6 Migrate the VMkernel adapter for the management network from the standard switch to a port on the
distributed switch by using the Add and Manage Hosts wizard.
For information about the Add and Manage Hosts wizard, see the vSphere Networking
documentation.
7 If you have moved the physical adapter from the proxy switch to the standard switch, you can
reattach it to the distributed switch again by using the Add and Manage Hosts wizard.
Problem
No redundant physical NICs for a host are connected to a particular standard or a distributed switch, and
the following alarm appears:
Cause
Only one physical NIC on the host is connected to a certain standard or a distributed switch. The
redundant physical NICs are either down or are not assigned to the switch.
For example, assume that a host in your environment has physical NICs vmnic0 and vmnic1 connected to
vSwitch0, and the physical NIC vmnic1 goes offline, leaving only vmnic0 connected to vSwitch0. As a
result, the uplink redundancy for vSwitch0 is lost on the host.
Solution
Check which switch has lost uplink redundancy on the host. Connect at least one more physical NIC on
the host to this switch and reset the alarm to green. You can use the vSphere Web Client or the
ESXi Shell.
If a physical NIC is down, try to bring it back up by using the ESXi Shell on the host.
For information about using the networking commands in the ESXi Shell, see vSphere Command-Line
Interface Reference. For information about configuring networking on a host in the vSphere Web Client,
see vSphere Networking.
Problem
After you rearrange the uplinks in the failover groups for a distributed port group in vCenter Server, for
example, by using the vSphere Web Client, some virtual machines in the port group can no longer access
the external network.
Cause
After changing the failover order, many reasons might cause virtual machines to lose connectivity to the
external network.
n The host that runs the virtual machines does not have physical NICs associated with the uplinks that
are set to active or standby. All uplinks that are associated with physical NICs from the host for the
port group are moved to unused.
n A Link Aggregation Group (LAG) that has no physical NICs from the host is set as the only active
uplink according to the requirements for using LACP in vSphere.
n If the virtual machine traffic is separated in VLANs, the host physical adapters for the active uplinks
might be connected to trunk ports on the physical switch that do not handle traffic from these VLANs.
n If the port group is configured with IP hash load balancing policy, an active uplink adapter is
connected to a physical switch port that might not be in an EtherChannel.
You can examine the connectivity of the virtual machines in the port group to associated host uplinks and
uplink adapters from the central topology diagram of the distributed switch or from the proxy switch
diagram for the host.
Solution
n Restore the failover order with the uplink that is associated with a single physical NIC on the host
back to active.
n Create a port group with identical settings, make it use the valid uplink number for the host, and
migrate the virtual machine networking to the port group.
n Move the NIC to an uplink that participates in the active failover group.
You can use the vSphere Web Client to move the host physical NIC to another uplink.
n Use the Add and Manage Hosts wizard on the distributed switch.
c On the Select task page, select the Manage host networking option and select the host.
d To assign the NIC of the host to an active uplink, navigate to the Manage physical network
adapters page and associate the NIC to the switch uplink.
a Navigate to the host in the vSphere Web Client, and on the Configure tab, expand the
Networking menu.
c Click Manage the physical network adapters connected to the selected switch, and
move the NIC to the active uplink
Problem
You try to add a physical adapter with low speed, for example, 1 Gbps, to a vSphere Distributed Switch
that is connected to physical adapters with high speed, for example, 10 Gbps. Network I/O Control
version 3 is enabled on the switch and bandwidth reservations exist for one or more system traffic types,
such as vSphere management traffic, vSphere vMotion traffic, vSphere NFS traffic, and so on. The task
for adding the physical adapter fails with a status message that a parameter is incorrect.
Cause
Network I/O Control aligns the bandwidth that is available for reservation to the 10-Gbps speed of the
individual physical adapters that are already connected to the distributed switch. After you reserve a part
of this bandwidth, adding a physical adapter whose speed is less than 10 Gbps might not meet the
potential needs of a system traffic type.
For information about Network I/O Control version 3, see the vSphere Networking documentation.
Solution
4 Type the physical adapters that you want to use outside the scope of Network I/O Control as a
comma-separated list for the Net.IOControlPnicOptOut parameter.
6 In the vSphere Web Client, add the physical adapter to the distributed switch.
Problem
When you connect the network adapter of a virtual machine to an SR-IOV virtual function (VF), you create
a passthrough network adapter for the virtual machine. After the (VF) driver in the guest operating system
modifies the MAC address for the passthrough network adapter, the guest operating system shows that
the change is successful but the VM network adapter loses connectivity. Although the guest operating
system shows that the new MAC address is enabled, a log message in the /var/log/vmkernel.log file
indicates that the operation has failed.
Requested mac address change to new MAC address on port VM NIC port number, disallowed by vswitch
policy.
where
n new MAC address is the MAC address in the guest operation system.
n VM NIC port number is the port number of the VM network adapter in hexadecimal format.
Cause
The default security policy on the port group to which the passthrough network adapter is connected does
not allow changes in the MAC address in the guest operating system. As a result, the networking
interface in the guest operating system cannot acquire an IP address and loses connectivity.
Solution
u In the guest operating system, reset the interface to cause the passthrough network adapter to regain
its valid MAC address. If the interface is configured to use DHCP for address assignment, the
interface acquires an IP address automatically.
For example, on a Linux virtual machine run the ifconfig console command.
where X in ethX represents the sequence number of the virtual machine network adapter in the guest
operating system.
Problem
A virtual machine that is expected to send BPDU frames causes the traffic to the external network of the
virtual machines in the same port group to be blocked.
If the virtual machine runs on a host that is a part of a vSphere HA cluster, and the host becomes
network-isolated under certain conditions, you observe Denial of Service (DoS) on the hosts in the
cluster.
Cause
As a best practice, a physical switch port that is connected to an ESXi host has the Port Fast and BPDU
guard enabled to enforce the boundary of the Spanning Tree Protocol (STP). A standard or distributed
switch does not support STP, and it does not send any BPDU frames to the switch port. However, if any
BPDU frame from a compromised virtual machine arrives at a physical switch port facing an ESXi host ,
the BPDU guard feature disables the port to stop the frames from affecting the Spanning Tree Topology of
the network.
In certain cases a virtual machine is expected to send BPDU frames, for example, when deploying VPN
that is connected through a Windows bridge device or through a bridge function. If the physical switch
port paired with the physical adapter that handles the traffic from this virtual machine has the BPDU guard
on, the port is error-disabled, and the virtual machines and VMkernel adapters using the host physical
adapter cannot communicate with the external network anymore.
If the teaming and failover policy of the port group contains more active uplinks, the BPDU traffic is moved
to the adapter for the next active uplink. The new physical switch port becomes disabled, and more
workloads become unable to exchange packets with the network. Eventually, almost all entities on the
ESXi host might become unreachable.
If the virtual machine runs on a host that is a part of a vSphere HA cluster, and the host becomes
network-isolated because most of the physical switch ports connected to it are disabled, the active master
host in the cluster moves the BPDU sender virtual machine to another host. The virtual machine starts
disabling the physical switch ports connected to the new host. The migration across the vSphere HA
cluster eventually leads to accumulated DoS across the entire cluster.
Solution
n If the VPN software must continue its work on the virtual machine, allow the traffic out of the virtual
machine and configure the physical switch port individually to pass the BPDU frames.
Network
Device Configuration
Distributed or Set the Forged Transmit security property on the port group to Accept to allow BPDU frames to leave
standard switch the host and reach the physical switch port.
You can isolate the settings and the physical adapter for the VPN traffic by placing the virtual machine in
a separate port group and assigning the physical adapter to the group.
Caution Setting the Forged Transmit security property to Accept to enable a host to send BPDU
frames carries a security risk because a compromised virtual machine can perform spoofing attacks.
Note Do not enable the BPDU filter globally. If the BPDU filter is enabled globally, the Port Fast mode
becomes disabled and all physical switch ports perform the full set of STP functions.
n To deploy a bridge device between two virtual machine NICs connected to the same Layer 2 network,
allow the BPDU traffic out of the virtual machines and deactivate Port Fast and BPDU loop prevention
features.
Distributed or Set the Forged Transmit property of the security policy on the port groups to Accept to allow BPDU
standard switch frames to leave the host and reach the physical switch port.
You can isolate the settings and one or more physical adapters for the bridge traffic by placing the
virtual machine in a separate port group and assigning the physical adapters to the group.
Caution Setting the Forged Transmit security property to Accept to enable bridge deployment carries
a security risk because a compromised virtual machine can perform spoofing attacks.
Physical switch n Disable Port Fast on the ports to the virtual bridge device to run STP on them.
n Disable BPDU guard and filter on the ports facing the bridge device.
n Protect the environment from DoS attacks in any case by activating the BPDU filter on the ESXi host
or on the physical switch.
u On a host that does not have the Guest BPDU filter implemented enable the BPDU filter on the
physical switch port to the virtual bridge device.
Distributed or Set the Forged Transmit property of the security policy on the port group to Reject.
standard switch
Note Do not enable the BPDU filter globally. If the BPDU filter is enabled globally, the Port Fast
mode becomes disabled and all physical switch ports perform the full set of STP functions.
Problem
When a Windows virtual machine transmits UDP packets larger than 1024 bytes, you experience lower
than expected or oscillating throughput even when other traffic is negligible. In case of a video streaming
server, video playback pauses.
Cause
For every UDP packet larger than 1024 bytes, the Windows network stack waits for a transmit completion
interrupt before sending the next packet. vSphere does not provide a transparent workaround of the
situation.
Solution
n Increase the threshold in bytes at which Windows changes its behavior for UDP packets by modifying
the registry of the Windows guest OS.
b Add a value with the name FastSendDatagramThreshold of type DWORD equal to 1500.
For information about fixing this issue in the Windows registry, see
http://support.microsoft.com/kb/235257.
If the Windows virtual machine has a VMXNET3 vNIC adapter, configure one of the following
parameters in the .vmx file of the virtual machine. Use the vSphere Web Client, or directly modify
the .vmx file.
Increase the interrupt rate of the virtual machine to a higher rate than ethernetX.coalescingScheme rbc
expected packet rate. For example, if the expected packet rate is 15000 ethernetX.coalescingParams 16000
interrupts per second, set the interrupt rate to 16000 interrupts per second.
Set the ethernetX.coalescingScheme parameter to rbc and the
ethernetX.coalescingParams parameter to 16000. The default interrupt
rate is 4000 interrupts per second.
Disable coalescing for low throughput or latency-sensitive workloads. For ethernetX.coalescingScheme disabled
information about configuring low-latency workloads, see Best Practices for
Performance Tuning of Latency-Sensitive Workloads in vSphere VMs.
Revert to the coalescing algorithm from earlier ESXi releases. ethernetX.coalescingScheme calibrate
Note The ability to revert to the earlier algorithm will not be available in later
vSphere releases.
X next to ethernet stands for the sequence number of the vNIC in the virtual machine.
For more information about configuring parameters in the .vmx file, see the vSphere Virtual Machine
Administrationdocumentation.
This approach affects all virtual machines and all virtual machine NICs on the host.
You can edit the advanced system settings list for the host in the vSphere Web Client, or by using a
vCLI console command on the host from the ESXi Shell.
Set a default interrupt rate higher than the Net.CoalesceScheme /Net/CoalesceScheme rbc
expected packet rate. For example, set the Net.CoalesceParams /Net/CoalesceParams 16000
interrupt rate to 16000 if 15000 interrupts
are expected per second.
For information about configuring a host from the vSphere Web Client, see the vCenter Server and
Host Management documentation. For information about setting host properties by using a vCLI
command, refer to the vSphere Command-Line Interface Reference documentation.
Problem
Virtual machines that reside on different hosts and on the same port group are unable to communicate.
Pings from one virtual machine to another have no effect. You cannot migrate the virtual machines
between the hosts by using vMotion.
Cause
n There are no physical NICs on some of the hosts assigned to active or standby uplinks in the teaming
and failover order of the distributed port group.
n The physical NICs on the hosts that are assigned to the active or standby uplinks reside in different
VLANs on the physical switch. The physical NICs in different VLANs cannot see each other and thus
cannot communicate with each other.
Solution
n In the topology of the distributed switch, check which host does not have physical NICs assigned to
an active or standby uplink on the distributed port group. Assign at least one physical NIC on that
host to an active uplink on the port group.
n In the topology of the distributed switch, check the VLAN IDs of the physical NICs that are assigned
to the active uplinks on the distributed port group. On all hosts, assign physical NICs that are from the
same VLAN to an active uplink on the distributed port group.
n To verify that there is no problem at the physical layer, migrate the virtual machines to the same host
and check the communication between them. Verify that inbound and outbound ICMP traffic is
enabled in the guest OS. By default ICMP traffic is disabled in Windows Server 2008 and Windows
Server 2012.
Problem
After you cold migrate a vApp or a virtual machine to another data center or vCenter Server system, an
attempt to power it on fails. An error message states that a property cannot be initialized or allocated
because the network of the vApp or virtual machine does not have an associated network protocol profile.
Cannot initialize property 'property'. Network 'port group' has no associated network protocol profile.
Cannot allocate IP address for property 'property'. Network 'port group' has no associated network
protocol profile.
Cause
By using the OVF environment, the vApp or virtual machine retrieves network settings from a network
protocol profile that is associated with the port group of the vApp or virtual machine.
vCenter Server creates such a network protocol profile for you when you install the OVF of a vApp and
associates the profile with the port group that you specify during the installation.
The mapping between the protocol profile and port group is valid only in the scope of a data center. When
you move the vApp, the protocol profile is not transferred to the target data center because of the
following reasons:
n The network settings of the protocol profile might not be valid in the network environment of the target
data center.
n A port group that has the same name and is associated with another protocol profile might already
exist in the target data center, and vApps and virtual machines might be connected to this group.
Replacing the protocol profiles for the port group might affect the connectivity of these vApp and
virtual machines.
Solution
n Create a network protocol profile on the target data center or vCenter Server system with the required
network settings and associate the protocol profile with the port group to which the vApp or virtual
machine is connected. For example, this approach is suitable if the vApp or virtual machine is a
vCenter Server extension that uses the vCenter Extension vService.
For information about providing network settings to a vApp or virtual machine from a network protocol
profile, see the vSphere Networking documentation.
n Use the vSphere Web Client to export the OVF file of the vApp or virtual machine from the source
data center or vCenter Server system and deploy it on the target data center or vCenter Server
system.
When you use the vSphere Web Client to deploy the OVF file, the target vCenter Server system
creates the network protocol profile for the vApp.
For information about managing OVF files in the vSphere Web Client, see the vSphere Virtual
Machine Administration documentation.
Problem
Cause
Under stressful conditions on a host, that is, if many concurrent networking operations compete for limited
resources, the time to perform some of the operations might exceed the default timeout for rollback of
network configuration operations on the distributed switch. As a result, these operations are rolled back.
For example, such a condition might come up when you create a VMkernel adapter on a host that has a
very high number of switch ports or virtual adapters, all of which consume system resources on the host.
Solution
n Use the vSphere Web Client to increase the timeout for rollback on vCenter Server.
If you encounter the same problem again, increase the rollback timeout with 60 seconds
incrementally until the operation has enough time to succeed.
e Click OK.
n Increase the timeout for rollback by editing the vpxd.cfg configuration file.
If you encounter the same problem again, increase the rollback timeout with 60 seconds
incrementally until the operation has enough time to succeed.
a On a vCenter Server instance, navigate to the directory that contains the vpxd.cfg configuration
file.
c Under the <network> section, increase the timeout, in the <rollbackTimeout> element.
<config>
<vpxd>
<network>
<rollbackTimeout>60</rollbackTimeout>
</network>
</vpxd>
</config>