Table of Contents

Performance tuning guidelines for Windows Server 2016

Server Hardware Tuning
Performance considerations
Power considerations
Server Role Tuning
Active Directory Server
File Server
Hyper-V Server
Windows Server Containers
Remote Desktop Services
Web Server
Server Subsystem Tuning
Cache and memory tuning
Network subsystem tuning
Software Defined Networking (SDN) tuning
Storage subsystem tuning
PowerShell tuning
Script authoring considerations
Module authoring considerations
Additional tuning resources
 Performance Tuning Guidelines for Windows Server
2016
5/3/2017 1 min to read Edit Online

When you run a server system in your organization, you might have business needs not met using default server
settings. For example, you might need the lowest possible energy consumption, or the lowest possible latency, or
the maximum possible throughput on your server. This guide provides a set of guidelines that you can use to tune
the server settings in Windows Server 2016 and obtain incremental performance or energy efficiency gains,
especially when the nature of the workload varies little over time.
It is important that your tuning changes consider the hardware, the workload, the power budgets, and the
performance goals of your server. This guide describes each setting and its potential effect to help you make an
informed decision about its relevance to your system, workload, performance, and energy usage goals.

WARNING
Registry settings and tuning parameters changed significantly between versions of Windows Server. Be sure to use the latest
tuning guidelines to avoid unexpected results.

In this guide
This guide organizes performance and tuning guidance for Windows Server 2016 across three tuning categories:

SERVER HARDWARE SERVER ROLE SERVER SUBSYSTEM

Hardware performance considerations Active Directory Servers Cache and memory management

Hardware power considerations File Servers Networking subsystem

Hyper-V Servers Storage Spaces Direct (SSD)

Remote Desktop Services Software Defined Networking (SDN)

Web Servers

Windows Server Containers

Changes in this version

Sections added
Nano Server installation-type configuration considerations
Software Defined Networking, including HNV and SLB gateway configuration guidance
Storage Spaces Direct
HTTP1.1 and HTTP2
Windows Server Containers
Sections changed
Updates to Active Directory guidance section
Updates to File Server guidance section
Updates to Web Server guidance section
Updates to Hardware Power guidance section
Updates to PowerShell tuning guidance section
Significant updates to the Hyper-V guidance section
Performance Tuning for Workloads removed, pointers to relevant resources added to Additional Tuning
Resources article
Removal of dedicated storage sections, in favor of new Storage Spaces Direct section and canonical Technet
content
Removal of dedicated networking section, in favor of canonical Technet content
 Server Hardware Performance Considerations
4/24/2017 5 min to read Edit Online

The following section lists important items that you should consider when you choose server hardware. Following
these guidelines can help remove performance bottlenecks that might impede the servers performance.

Processor Recommendations
Choose 64-bit processors for servers. 64-bit processors have significantly more address space, and are required for
Windows Server 2016. No 32-bit editions of the operating system will be provided, but 32-bit applications will run
on the 64-bit Windows Server 2016 operating system.
To increase the computing resources in a server, you can use a processor with higher-frequency cores, or you can
increase the number of processor cores. If CPU is the limiting resource in the system, a core with 2x frequency
typically provides a greater performance improvement than two cores with 1x frequency.
Multiple cores are not expected to provide a perfect linear scaling, and the scaling factor can be even less if hyper-
threading is enabled because hyper-threading relies on sharing resources of the same physical core.

IMPORTANT
Match and scale the memory and I/O subsystem with the CPU performance, and vice versa.

Do not compare CPU frequencies across manufacturers and generations of processors because the comparison can
be a misleading indicator of speed.
For Hyper-V, make sure that the processor supports SLAT (Second Level Address Translation). It is implemented as
Extended Page Tables (EPT) by Intel and Nested Page Tables (NPT) by AMD. You can verify this feature is present by
using SystemInfo.exe on your server.

Cache Recommendations
Choose large L2 or L3 processor caches. On newer architectures, such as Haswell or Skylake, there is a unified Last
Level Cache (LLC) or an L4. The larger caches generally provide better performance, and they often play a bigger
role than raw CPU frequency.

Memory (RAM) and Paging Storage Recommendations

Increase the RAM to match your memory needs. When your computer runs low on memory and it needs more
immediately, Windows uses hard disk space to supplement system RAM through a procedure called paging. Too
much paging degrades the overall system performance. You can optimize paging by using the following guidelines
for page file placement:
Isolate the page file on its own storage device, or at least make sure it doesnt share the same storage
devices as other frequently accessed files. For example, place the page file and operating system files on
separate physical disk drives.
Place the page file on a drive that is not fault-tolerant. If the disk fails, a system crash is likely to occur. If you
place the page file on a fault-tolerant drive, remember that fault-tolerant systems are often slower to write
data because they write data to multiple locations.
 Use multiple disks or a disk array if you need additional disk bandwidth for paging. Do not place multiple
page files on different partitions of the same physical disk drive.

Peripheral Bus Recommendations

In Windows Server 2016, the primary storage and network interfaces should be PCI Express (PCIe) so servers with
PCIe buses are recommended. To avoid bus speed limitations, use PCIe x8 and higher slots for 10+ GB Ethernet
adapters.

Disk Recommendations
Choose disks with higher rotational speeds to reduce random request service times (~2 ms on average when you
compare 7,200- and 15,000-RPM drives) and to increase sequential request bandwidth. However, there are cost,
power, and other considerations associated with disks that have high rotational speeds.
2.5-inch enterprise-class disks can service a significantly larger number of random requests per second compared
to equivalent 3.5-inch drives.
Store frequently accessed data, especially sequentially accessed data, near the beginning of a disk because this
roughly corresponds to the outermost (fastest) tracks.
Consolidating small drives into fewer high-capacity drives can reduce overall storage performance. Fewer spindles
mean reduced request service concurrency; and therefore, potentially lower throughput and longer response times
(depending on the workload intensity).
The use of SSD and high speed flash disks is useful for read mostly disks with high I/O rates or latency sensitive I/O.
Boot disks are good candidates for the use of SSD or high speed flash disks as they can improve boot times
significantly.
NVMe SSDs offer superior performance with greater command queue depths, more efficient interrupt processing,
and greater efficiency for 4KB commands. This particularly benefits scenarios that requires heavy simultaneous I/O.

Network and Storage Adapter Recommendations

The following section lists the recommended characteristics for network and storage adapters for high-
performance servers. These settings can help prevent your networking or storage hardware from being a
bottleneck when they are under heavy load.
Certified adapter usage
Use an adapter that has passed the Windows Hardware Certification test suite.
64-bit capability
Adapters that are 64-bit-capable can perform direct memory access (DMA) operations to and from high physical
memory locations (greater than 4 GB). If the driver does not support DMA greater than 4 GB, the system double-
buffers the I/O to a physical address space of less than 4 GB.
Copper and fiber adapters
Copper adapters generally have the same performance as their fiber counterparts, and both copper and fiber are
available on some Fibre Channel adapters. Certain environments are better suited to copper adapters, whereas
other environments are better suited to fiber adapters.
Dual- or quad-port adapters
Multiport adapters are useful for servers that have a limited number of PCI slots.
To address SCSI limitations on the number of disks that can be connected to a SCSI bus, some adapters provide two
or four SCSI buses on a single adapter card. Fibre Channel adapters generally have no limits to the number of disks
that are connected to an adapter unless they are hidden behind a SCSI interface.
Serial Attached SCSI (SAS) and Serial ATA (SATA) adapters also have a limited number of connections because of
the serial nature of the protocols, but you can attach more disks by using switches.
Network adapters have this feature for load-balancing or failover scenarios. Using two single-port network
adapters usually yields better performance than using a single dual-port network adapter for the same workload.
PCI bus limitation can be a major factor in limiting performance for multiport adapters. Therefore, it is important to
consider placing them in a high-performing PCIe slot that provides enough bandwidth.
Interrupt moderation
Some adapters can moderate how frequently they interrupt the host processors to indicate activity or its
completion. Moderating interrupts can often result in reduced CPU load on the host, but, unless interrupt
moderation is performed intelligently; the CPU savings might increase latency.
Receive Side Scaling (RSS ) support
RSS enables packet receive-processing to scale with the number of available computer processors. This is
particularly important with 10 GB Ethernet and faster.
Offload capability and other advanced features such as message -signaled interrupt (MSI )-X
Offload-capable adapters offer CPU savings that yield improved performance.
Dynamic interrupt and deferred procedure call (DPC ) redirection
In Windows Server 2016, Numa I/O enables PCIe storage adapters to dynamically redirect interrupts and DPCs and
can help any multiprocessor system by improving workload partitioning, cache hit rates, and on-board hardware
interconnect usage for I/O-intensive workloads.

See Also
Server Hardware Power Considerations
Power and Performance Tuning
Processor Power Management Tuning
Recommended Balanced Plan Parameters
 Server Hardware Performance Considerations
4/24/2017 5 min to read Edit Online

The following section lists important items that you should consider when you choose server hardware. Following
these guidelines can help remove performance bottlenecks that might impede the servers performance.

Processor Recommendations
Choose 64-bit processors for servers. 64-bit processors have significantly more address space, and are required
for Windows Server 2016. No 32-bit editions of the operating system will be provided, but 32-bit applications will
run on the 64-bit Windows Server 2016 operating system.
To increase the computing resources in a server, you can use a processor with higher-frequency cores, or you can
increase the number of processor cores. If CPU is the limiting resource in the system, a core with 2x frequency
typically provides a greater performance improvement than two cores with 1x frequency.
Multiple cores are not expected to provide a perfect linear scaling, and the scaling factor can be even less if hyper-
threading is enabled because hyper-threading relies on sharing resources of the same physical core.

IMPORTANT
Match and scale the memory and I/O subsystem with the CPU performance, and vice versa.

Do not compare CPU frequencies across manufacturers and generations of processors because the comparison
can be a misleading indicator of speed.
For Hyper-V, make sure that the processor supports SLAT (Second Level Address Translation). It is implemented as
Extended Page Tables (EPT) by Intel and Nested Page Tables (NPT) by AMD. You can verify this feature is present
by using SystemInfo.exe on your server.

Cache Recommendations
Choose large L2 or L3 processor caches. On newer architectures, such as Haswell or Skylake, there is a unified Last
Level Cache (LLC) or an L4. The larger caches generally provide better performance, and they often play a bigger
role than raw CPU frequency.

Memory (RAM) and Paging Storage Recommendations

Increase the RAM to match your memory needs. When your computer runs low on memory and it needs more
immediately, Windows uses hard disk space to supplement system RAM through a procedure called paging. Too
much paging degrades the overall system performance. You can optimize paging by using the following
guidelines for page file placement:
Isolate the page file on its own storage device, or at least make sure it doesnt share the same storage
devices as other frequently accessed files. For example, place the page file and operating system files on
separate physical disk drives.
Place the page file on a drive that is not fault-tolerant. If the disk fails, a system crash is likely to occur. If you
place the page file on a fault-tolerant drive, remember that fault-tolerant systems are often slower to write
data because they write data to multiple locations.
 Use multiple disks or a disk array if you need additional disk bandwidth for paging. Do not place multiple
page files on different partitions of the same physical disk drive.

Peripheral Bus Recommendations

In Windows Server 2016, the primary storage and network interfaces should be PCI Express (PCIe) so servers with
PCIe buses are recommended. To avoid bus speed limitations, use PCIe x8 and higher slots for 10+ GB Ethernet
adapters.

Disk Recommendations
Choose disks with higher rotational speeds to reduce random request service times (~2 ms on average when you
compare 7,200- and 15,000-RPM drives) and to increase sequential request bandwidth. However, there are cost,
power, and other considerations associated with disks that have high rotational speeds.
2.5-inch enterprise-class disks can service a significantly larger number of random requests per second compared
to equivalent 3.5-inch drives.
Store frequently accessed data, especially sequentially accessed data, near the beginning of a disk because this
roughly corresponds to the outermost (fastest) tracks.
Consolidating small drives into fewer high-capacity drives can reduce overall storage performance. Fewer spindles
mean reduced request service concurrency; and therefore, potentially lower throughput and longer response
times (depending on the workload intensity).
The use of SSD and high speed flash disks is useful for read mostly disks with high I/O rates or latency sensitive
I/O. Boot disks are good candidates for the use of SSD or high speed flash disks as they can improve boot times
significantly.
NVMe SSDs offer superior performance with greater command queue depths, more efficient interrupt processing,
and greater efficiency for 4KB commands. This particularly benefits scenarios that requires heavy simultaneous
I/O.

Network and Storage Adapter Recommendations

The following section lists the recommended characteristics for network and storage adapters for high-
performance servers. These settings can help prevent your networking or storage hardware from being a
bottleneck when they are under heavy load.
Certified adapter usage
Use an adapter that has passed the Windows Hardware Certification test suite.
64-bit capability
Adapters that are 64-bit-capable can perform direct memory access (DMA) operations to and from high physical
memory locations (greater than 4 GB). If the driver does not support DMA greater than 4 GB, the system double-
buffers the I/O to a physical address space of less than 4 GB.
Copper and fiber adapters
Copper adapters generally have the same performance as their fiber counterparts, and both copper and fiber are
available on some Fibre Channel adapters. Certain environments are better suited to copper adapters, whereas
other environments are better suited to fiber adapters.
Dual- or quad-port adapters
Multiport adapters are useful for servers that have a limited number of PCI slots.
To address SCSI limitations on the number of disks that can be connected to a SCSI bus, some adapters provide
two or four SCSI buses on a single adapter card. Fibre Channel adapters generally have no limits to the number of
disks that are connected to an adapter unless they are hidden behind a SCSI interface.
Serial Attached SCSI (SAS) and Serial ATA (SATA) adapters also have a limited number of connections because of
the serial nature of the protocols, but you can attach more disks by using switches.
Network adapters have this feature for load-balancing or failover scenarios. Using two single-port network
adapters usually yields better performance than using a single dual-port network adapter for the same workload.
PCI bus limitation can be a major factor in limiting performance for multiport adapters. Therefore, it is important
to consider placing them in a high-performing PCIe slot that provides enough bandwidth.
Interrupt moderation
Some adapters can moderate how frequently they interrupt the host processors to indicate activity or its
completion. Moderating interrupts can often result in reduced CPU load on the host, but, unless interrupt
moderation is performed intelligently; the CPU savings might increase latency.
Receive Side Scaling (RSS ) support
RSS enables packet receive-processing to scale with the number of available computer processors. This is
particularly important with 10 GB Ethernet and faster.
Offload capability and other advanced features such as message -signaled interrupt (MSI )-X
Offload-capable adapters offer CPU savings that yield improved performance.
Dynamic interrupt and deferred procedure call (DPC ) redirection
In Windows Server 2016, Numa I/O enables PCIe storage adapters to dynamically redirect interrupts and DPCs
and can help any multiprocessor system by improving workload partitioning, cache hit rates, and on-board
hardware interconnect usage for I/O-intensive workloads.

See Also
Server Hardware Power Considerations
Power and Performance Tuning
Processor Power Management Tuning
Recommended Balanced Plan Parameters
 Server Hardware Power Considerations
4/24/2017 2 min to read Edit Online

It is important to recognize the increasing importance of energy efficiency in enterprise and data center
environments. High performance and low-energy usage are often conflicting goals, but by carefully selecting
server components, you can achieve the correct balance between them. The following sections lists guidelines for
power characteristics and capabilities of server hardware components.

Processor Recommendations
Frequency, operating voltage, cache size, and process technology affect the energy consumption of processors.
Processors have a thermal design point (TDP) rating that gives a basic indication of energy consumption relative
to other models.
In general, opt for the lowest TDP processor that will meet your performance goals. Also, newer generations of
processors are generally more energy efficient, and they may expose more power states for the Windows power
management algorithms, which enables better power management at all levels of performance. Or they may use
some of the new cooperative power management techniques that Microsoft has developed in partnership with
hardware manufacturers.
For more info on cooperative power management techniques, see the section named Collaborative Processor
Performance Control in the Advanced Configuration and Power Interface Specification.

Memory Recommendations
Memory accounts for an increasing fraction of the total system power. Many factors affect the energy
consumption of a memory DIMM, such as memory technology, error correction code (ECC), bus frequency,
capacity, density, and number of ranks. Therefore, it is best to compare expected power ratings before purchasing
large quantities of memory.
Low-power memory is now available, but you must consider the performance and cost trade-offs. If your server
will be paging, you should also factor in the energy cost of the paging disks.

Disks Recommendations
Higher RPM means increased energy consumption. SSD drives are more power efficient than rotational drives.
Also, 2.5-inch drives generally require less power than 3.5-inch drives.

Network and Storage Adapter Recommendations

Some adapters decrease energy consumption during idle periods. This is an important consideration for 10 Gb
networking adapters and high-bandwidth (4-8 Gb) storage links. Such devices can consume significant amounts
of energy.

Power Supply Recommendations

Improving power supply efficiency is a great way to reduce energy consumption without affecting performance.
High-efficiency power supplies can save many kilowatt-hours per year, per server.

Fan Recommendations
Fans, like power supplies, are an area where you can reduce energy consumption without affecting system
performance. Variable-speed fans can reduce RPM as the system load decreases, eliminating otherwise
unnecessary energy consumption.

USB devices Recommendations

Windows Server 2016 enables selective suspend for USB devices by default. However, a poorly written device
driver can still disrupt system energy efficiency by a sizeable margin. To avoid potential issues, disconnect USB
devices, disable them in the BIOS, or choose servers that do not require USB devices.

Remotely-managed Power Strip Recommendations

Power strips are not an integral part of server hardware, but they can make a large difference in the data center.
Measurements show that volume servers that are plugged in, but have been ostensibly powered off, may still
require up to 30 watts of power.
To avoid wasting electricity, you can deploy a remotely managed power strip for each rack of servers to
programmatically disconnect power from specific servers.

Processor terminology
The processor terminology used throughout this topic reflects the hierarchy of components available in the
following figure. Terms used from largest to smallest granularity of components are the following:
Processor socket
NUMA node
Core
Logical processor

See Also
Server Hardware Performance Considerations
Power and Performance Tuning
Processor Power Management Tuning
Recommended Balanced Plan Parameters
 Power and performance tuning
4/24/2017 13 min to read Edit Online

Energy efficiency is increasingly important in enterprise and data center environments, and it adds another set of
tradeoffs to the mix of configuration options.
Windows Server 2016 is optimized for excellent energy efficiency with minimum performance impact across a
wide range of customer workloads. Processor Power Management (PPM) Tuning for the Windows Server Balanced
Power Plan describes the workloads used for tuning the default parameters in Windows Server 2016, and
provides suggestions for customized tunings.
This section expands on energy-efficiency tradeoffs to help you make informed decisions if you need to adjust the
default power settings on your server. However, the majority of server hardware and workloads should not
require administrator power tuning when running Windows Server 2016.

Calculating server energy efficiency

When you tune your server for energy savings, you must also consider performance. Tuning affects performance
and power, sometimes in disproportionate amounts. For each possible adjustment, consider your power budget
and performance goals to determine whether the trade-off is acceptable.
You can calculate your server's energy efficiency ratio for a useful metric that incorporates power and
performance information. Energy efficiency is the ratio of work that is done to the average power that is required
during a specified amount of time.

You can use this metric to set practical goals that respect the tradeoff between power and performance. In
contrast, a goal of 10 percent energy savings across the data center fails to capture the corresponding effects on
performance and vice versa.
Similarly, if you tune your server to increase performance by 5 percent, and that results in 10 percent higher
energy consumption, the total result might or might not be acceptable for your business goals. The energy
efficiency metric allows for more informed decision making than power or performance metrics alone.

Measuring system energy consumption

You should establish a baseline power measurement before you tune your server for energy efficiency.
If your server has the necessary support, you can use the power metering and budgeting features in Windows
Server 2016 to view system-level energy consumption by using Performance Monitor.
One way to determine whether your server has support for metering and budgeting is to review the Windows
Server Catalog. If your server model qualifies for the new Enhanced Power Management qualification in the
Windows Hardware Certification Program, it is guaranteed to support the metering and budgeting functionality.
Another way to check for metering support is to manually look for the counters in Performance Monitor. Open
Performance Monitor, select Add Counters, and then locate the Power Meter counter group.
If named instances of power meters appear in the box labeled Instances of Selected Object, your platform
supports metering. The Power counter that shows power in watts appears in the selected counter group. The exact
derivation of the power data value is not specified. For example, it could be an instantaneous power draw or an
average power draw over some time interval.
If your server platform does not support metering, you can use a physical metering device connected to the power
supply input to measure system power draw or energy consumption.
To establish a baseline, you should measure the average power required at various system load points, from idle
to 100 percent (maximum throughput) to generate a load line. The following figure shows load lines for three
sample configurations:

You can use load lines to evaluate and compare the performance and energy consumption of configurations at all
load points. In this particular example, it is easy to see what the best configuration is. However, there can easily be
scenarios where one configuration works best for heavy workloads and one works best for light workloads.
You need to thoroughly understand your workload requirements to choose an optimal configuration. Dont
assume that when you find a good configuration, it will always remain optimal. You should measure system
utilization and energy consumption on a regular basis and after changes in workloads, workload levels, or server
hardware.

Diagnosing energy efficiency issues

PowerCfg.exe supports a command-line option that you can use to analyze the idle energy efficiency of your
server. When you run PowerCfg.exe with the /energy option, the tool performs a 60-second test to detect
potential energy efficiency issues. The tool generates a simple HTML report in the current directory.

IMPORTANT
To ensure an accurate analysis, make sure that all local apps are closed before you run PowerCfg.exe.

Shortened timer tick rates, drivers that lack power management support, and excessive CPU utilization are a few of
the behavioral issues that are detected by the powercfg /energy command. This tool provides a simple way to
identify and fix power management issues, potentially resulting in significant cost savings in a large datacenter.
For more info about PowerCfg.exe, see Using PowerCfg to Evaluate System Energy Efficiency.

Using power plans in Windows Server

Windows Server 2016 has three built-in power plans designed to meet different sets of business needs. These
plans provide a simple way for you to customize a server to meet power or performance goals. The following table
describes the plans, lists the common scenarios in which to use each plan, and gives some implementation details
for each plan.
 COMMON APPLICABLE IMPLEMENTATION
PLAN DESCRIPTION SCENARIOS HIGHLIGHTS

Balanced (recommended) Default setting. Targets General computing Matches capacity to

good energy efficiency with demand. Energy-saving
minimal performance features balance power and
impact. performance.

High Performance Increases performance at Low latency apps and app Processors are always locked
the cost of high energy code that is sensitive to at the highest performance
consumption. Power and processor performance state (including turbo
thermal limitations, changes frequencies). All cores are
operating expenses, and unparked. Thermal output
reliability considerations may be significant.
apply.

Power Saver Limits performance to save Deployments with limited Caps processor frequency at
energy and reduce power budgets and thermal a percentage of maximum (if
operating cost. Not constraints supported), and enables
recommended without other energy-saving
thorough testing to make features.
sure performance is
adequate.

These power plans exist in Windows for alternating current (AC) and direct current (DC) powered systems, but we
will assume that servers are always using an AC power source.
For more info on power plans and power policy configurations, see Power Policy Configuration and Deployment
in Windows.

NOTE
Some server manufactures have their own power management options available through the BIOS settings. If the operating
system does not have control over the power management, changing the power plans in Windows will not affect system
power and performance.

Tuning processor power management parameters

Each power plan represents a combination of numerous underlying power management parameters. The built-in
plans are three collections of recommended settings that cover a wide variety of workloads and scenarios.
However, we recognize that these plans will not meet every customers needs.
The following sections describe ways to tune some specific processor power management parameters to meet
goals not addressed by the three built-in plans. If you need to understand a wider array of power parameters, see
Power Policy Configuration and Deployment in Windows.

Processor performance boost mode

Intel Turbo Boost and AMD Turbo CORE technologies are features that allow processors to achieve additional
performance when it is most useful (that is, at high system loads). However, this feature increases CPU core energy
consumption, so Windows Server 2016 configures Turbo technologies based on the power policy that is in use
and the specific processor implementation.
Turbo is enabled for High Performance power plans on all Intel and AMD processors and it is disabled for Power
Saver power plans. For Balanced power plans on systems that rely on traditional P-state-based frequency
management, Turbo is enabled by default only if the platform supports the EPB register.
 NOTE
The EPB register is only supported in Intel Westmere and later processors.

For Intel Nehalem and AMD processors, Turbo is disabled by default on P-state-based platforms. However, if a
system supports Collaborative Processor Performance Control (CPPC), which is a new alternative mode of
performance communication between the operating system and the hardware (defined in ACPI 5.0), Turbo may be
engaged if the Windows operating system dynamically requests the hardware to deliver the highest possible
performance levels.
To enable or disable the Turbo Boost feature, the Processor Performance Boost Mode parameter must be
configured by the administrator or by the default parameter settings for the chosen power plan. Processor
Performance Boost Mode has five allowable values, as shown in Table 5.
For P-state-based control, the choices are Disabled, Enabled (Turbo is available to the hardware whenever nominal
performance is requested), and Efficient (Turbo is available only if the EPB register is implemented).
For CPPC-based control, the choices are Disabled, Efficient Enabled (Windows specifies the exact amount of Turbo
to provide), and Aggressive (Windows asks for maximum performance to enable Turbo).
In Windows Server 2016, the default value for Boost Mode is 3.

NAME P-STATE-BASED BEHAVIOR CPPC BEHAVIOR

0 (Disabled) Disabled Disabled

1 (Enabled) Enabled Efficient Enabled

2 (Aggressive) Enabled Aggressive

3 (Efficient Enabled) Efficient Efficient Enabled

4 (Efficient Aggressive) Efficient Aggressive

The following commands enable Processor Performance Boost Mode on the current power plan (specify the policy
by using a GUID alias):

Powercfg -setacvalueindex scheme_current sub_processor PERFBOOSTMODE 1

Powercfg -setactive scheme_current

IMPORTANT
You must run the powercfg -setactive command to enable the new settings. You do not need to reboot the server.

To set this value for power plans other than the currently selected plan, you can use aliases such as SCHEME_MAX
(Power Saver), SCHEME_MIN (High Performance), and SCHEME_BALANCED (Balanced) in place of
SCHEME_CURRENT. Replace scheme current in the powercfg -setactive commands previously shown with the
desired alias to enable that power plan.
For example, to adjust the Boost Mode in the Power Saver plan and make that Power Saver is the current plan, run
the following commands:
 Powercfg -setacvalueindex scheme_max sub_processor PERFBOOSTMODE 1
Powercfg -setactive scheme_max

Minimum and maximum processor performance state

Processors change between performance states (P-states) very quickly to match supply to demand, delivering
performance where necessary and saving energy when possible. If your server has specific high-performance or
minimum-power-consumption requirements, you might consider configuring the Minimum Processor
Performance State parameter or the Maximum Processor Performance State parameter.
The values for the Minimum Processor Performance State and Maximum Processor Performance State
parameters are expressed as a percentage of maximum processor frequency, with a value in the range 0 100.
If your server requires ultra-low latency, invariant CPU frequency (e.g., for repeatable testing), or the highest
performance levels, you might not want the processors switching to lower-performance states. For such a server,
you can cap the minimum processor performance state at 100 percent by using the following commands:

Powercfg -setacvalueindex scheme_current sub_processor PROCTHROTTLEMIN 100

Powercfg -setactive scheme_current

If your server requires lower energy consumption, you might want to cap the processor performance state at a
percentage of maximum. For example, you can restrict the processor to 75 percent of its maximum frequency by
using the following commands:

Powercfg -setacvalueindex scheme_current sub_processor PROCTHROTTLEMAX 75

Powercfg -setactive scheme_current

NOTE
Capping processor performance at a percentage of maximum requires processor support. Check the processor
documentation to determine whether such support exists, or view the Performance Monitor counter % of maximum
frequency in the Processor group to see if any frequency caps were applied.

Processor performance increase and decrease of thresholds and

policies
The speed at which a processor performance state increases or decreases is controlled by multiple parameters. The
following four parameters have the most visible impact:
Processor Performance Increase Threshold defines the utilization value above which a processors
performance state will increase. Larger values slow the rate of increase for the performance state in
response to increased activities.
Processor Performance Decrease Threshold defines the utilization value below which a processors
performance state will decrease. Larger values increase the rate of decrease for the performance state
during idle periods.
Processor Performance Increase Policy and Processor Performance Decrease Policy determine which
performance state should be set when a change happens. Single policy means it chooses the next state.
Rocket means the maximum or minimal power performance state. Ideal tries to find a balance between
power and performance.
For example, if your server requires ultra-low latency while still wanting to benefit from low power during idle
periods, you could quicken the performance state increase for any increase in load and slow the decrease when
load goes down. The following commands set the increase policy to Rocket for a faster state increase, and set the
decrease policy to Single. The increase and decrease thresholds are set to 10 and 8 respectively.

Powercfg.exe -setacvalueindex scheme_current sub_processor PERFINCPOL 2

Powercfg.exe -setacvalueindex scheme_current sub_processor PERFDECPOL 1
Powercfg.exe -setacvalueindex scheme_current sub_processor PERFINCTHRESHOLD 10
Powercfg.exe -setacvalueindex scheme_current sub_processor PERFDECTHRESHOLD 8
Powercfg.exe /setactive scheme_current

Processor performance core parking maximum and minimum cores

Core parking is a feature that was introduced in Windows Server 2008 R2. The processor power management
(PPM) engine and the scheduler work together to dynamically adjust the number of cores that are available to run
threads. The PPM engine chooses a minimum number of cores for the threads that will be scheduled.
Cores that are parked generally do not have any threads scheduled, and they will drop into very low power states
when they are not processing interrupts, DPCs, or other strictly affinitized work. The remaining cores are
responsible for the remainder of the workload. Core parking can potentially increase energy efficiency during
lower usage.
For most servers, the default core-parking behavior provides a reasonable balance of throughput and energy
efficiency. On processors where core parking may not show as much benefit on generic workloads, it can be
disabled by default.
If your server has specific core parking requirements, you can control the number of cores that are available to
park by using the Processor Performance Core Parking Maximum Cores parameter or the Processor
Performance Core Parking Minimum Cores parameter in Windows Server 2016.
One scenario that core parking isnt always optimal for is when there are one or more active threads affinitized to
a non-trivial subset of CPUs in a NUMA node (that is, more than 1 CPU, but less than the entire set of CPUs on the
node). When the core parking algorithm is picking cores to unpark (assuming an increase in workload intensity
occurs), it may not always pick the cores within the active affinitized subset (or subsets) to unpark, and thus may
end up unparking cores that wont actually be utilized.
The values for these parameters are percentages in the range 0 100. The Processor Performance Core Parking
Maximum Cores parameter controls the maximum percentage of cores that can be unparked (available to run
threads) at any time, while the Processor Performance Core Parking Minimum Cores parameter controls the
minimum percentage of cores that can be unparked. To turn off core parking, set the Processor Performance
Core Parking Minimum Cores parameter to 100 percent by using the following commands:

Powercfg -setacvalueindex scheme_current sub_processor CPMINCORES 100

Powercfg -setactive scheme_current

To reduce the number of schedulable cores to 50 percent of the maximum count, set the Processor Performance
Core Parking Maximum Cores parameter to 50 as follows:

Powercfg -setacvalueindex scheme_current sub_processor CPMAXCORES 50

Powercfg -setactive scheme_current

Processor performance core parking utility distribution

Utility Distribution is an algorithmic optimization in Windows Server 2016 that is designed to improve power
efficiency for some workloads. It tracks unmovable CPU activity (that is, DPCs, interrupts, or strictly affinitized
threads), and it predicts the future work on each processor based on the assumption that any movable work can
be distributed equally across all unparked cores.
Utility Distribution is enabled by default for the Balanced power plan for some processors. It can reduce processor
power consumption by lowering the requested CPU frequencies of workloads that are in a reasonably steady state.
However, Utility Distribution is not necessarily a good algorithmic choice for workloads that are subject to high
activity bursts or for programs where the workload quickly and randomly shifts across processors.
For such workloads, we recommend disabling Utility Distribution by using the following commands:

Powercfg -setacvalueindex scheme_current sub_processor DISTRIBUTEUTIL 0

Powercfg -setactive scheme_current

See Also
Server Hardware Performance Considerations
Server Hardware Power Considerations
Processor Power Management Tuning
Recommended Balanced Plan Parameters
 Processor Power Management (PPM) Tuning for the
Windows Server Balanced Power Plan
4/24/2017 9 min to read Edit Online

Starting with Windows Server 2008, Windows Server provides three power plans: Balanced, High Performance,
and Power Saver. The Balanced power plan is the default choice that aims to give the best energy efficiency for a
set of typical server workloads. This topic describes the workloads that have been used to determine the default
settings for the Balanced scheme for the past several releases of Windows.
If you run a server system that has dramatically different workload characteristics or performance and power
requirements than these workloads, you might want to consider tuning the default power settings (i.e., create a
custom power plan). One source of useful tuning information is the Server Hardware Power Considerations.
Alternately, you may decide that the High Performance power plan is the right choice for your environment,
recognizing that you will likely take a significant energy hit in exchange for some level of increased
responsiveness.

IMPORTANT
You should leverage the power policies that are included with Windows Server unless you have a specific need to create a
custom one and have a very good understanding that your results will vary depending on the characteristics of your
workload.

Windows Processor Power Tuning Methodology

Tested workloads
Workloads are selected to cover a best-effort set of typical Windows Server workloads. Obviously this set is not
intended to be representative of the entire breadth of real-world server environments.
The tuning in each power policy is data driven by the following five workloads:
IIS Web Server workload
A Microsoft internal benchmark called Web Fundamentals is used to optimize the energy efficiency of
platforms running IIS Web Server. The setup contains a web server and multiple clients that simulate the
web access traffic. The distribution of dynamic, static hot (in-memory), and static cold (disk access required)
web pages is based on statistical studies of production servers. To push the servers CPU cores to full
utilization (one end of the tested spectrum), the setup needs sufficiently fast network and disk resources.
SQL Server Database workload
The TPC-E benchmark is a popular benchmark for database performance analysis. It is used to generate an
OLTP workload for PPM tuning optimizations. This workload has significant disk I/O, and hence has a high
performance requirement for the storage system and memory size.
File Server workload
A Microsoft-developed benchmark called FSCT is used to generate an SMB file server workload. It creates a
large file set on the server and uses many client systems (actual or virtualized) to generate file open, close,
read and write operations. The operation mix is based on statistical studies of production servers. It stresses
CPU, disk, and network resources.
 SPECpower JAVA workload
SPECpower_ssj2008 is the first industry-standard SPEC benchmark that jointly evaluates power and
performance characteristics. It is a server-side Java workload with varying CPU load levels. It doesnt
require many disk or network resources, but it has certain requirements for memory bandwidth. Almost all
of the CPU activity is performed in user-mode; kernel-mode activity does not have much impact on the
benchmarks power and performance characteristics except for the power management decisions.
Application Server workload
The SAP-SD benchmark is used to generate an application server workload. A two-tier setup is used, with
the database and the application server on the same server host. This workload also utilizes response time
as a performance metric, which differs from other tested workloads. Thus it is used to verify the impact of
PPM parameters on responsiveness. Nevertheless, it is not intended to be representative of all latency-
sensitive production workloads.
All of the benchmarks except SPECpower were originally designed for performance analysis and were therefore
created to run at peak load levels. However, medium to light load levels are more common for real-world
production servers and are more interesting for Balanced plan optimizations. We intentionally run the
benchmarks at varying load levels from 100% down to 10% (in 10% steps) by using various throttling methods
(e.g., by reducing the number of active users/clients).
Hardware configurations
For each release of Windows, the most current production servers are used in the power plan analysis and
optimization process. In some cases, the tests were performed on pre-production systems whose release schedule
matched that of the next Windows release.
Given that most servers are sold with 1 to 4 processor sockets, and since scale-up servers are less likely to have
energy efficiency as a primary concern, the power plan optimization tests are primarily run on 2-socket and 4-
socket systems. The amount of RAM, disk, and network resources for each test are chosen to allow each system to
run all the way up to its full capacity, while taking into account the cost restrictions that would normally be in
place for real-world server environments, such as keeping the configurations reasonable.

IMPORTANT
Even though the system can run at its peak load, we typically optimize for lower load levels, since servers that consistently
run at their peak load levels would be well-advised to use the High Performance power plan unless energy efficiency is a
high priority.

Metrics
All of the tested benchmarks use throughput as the performance metric. Response Time is considered as an SLA
requirement for these workloads (except for SAP, where it is a primary metric). For example, a benchmark run is
considered valid if the mean or maximum response time is less than certain value.
Therefore, the PPM tuning analysis also uses throughput as its performance metric. At the highest load level
(100% CPU utilization), our goal is that the throughput should not decrease more than a few percent due to power
management optimizations. But the primary consideration is to maximize the power efficiency (as defined below)
at medium and low load levels.

Running the CPU cores at lower frequencies reduces energy consumption. However, lower frequencies typically
decrease throughput and increase response time. For the Balanced power plan, there is an intentional tradeoff of
responsiveness and power efficiency. The SAP workload tests, as well as the response time SLAs on the other
workloads, make sure that the response time increase doesnt exceed certain threshold (5% as an example) for
these specific workloads.

NOTE
If the workload uses response time as the performance metric, the system should either switch to the High Performance
power plan or change Balanced power plan as suggested in Recommended Balanced Power Plan Parameters for Quick
Response Time.

Tuning results
Starting with Windows Server 2008, Microsoft worked with Intel and AMD to optimize the PPM parameters for
the most up-to-date server processors for each Windows release. A tremendous number of PPM parameter
combinations were tested on each of the previously-discussed workloads to find the best power efficiency at
different load levels. As software algorithms were refined and as hardware power architectures evolved, each new
Windows Server always had better or equal power efficiency than its previous versions across the range of tested
workloads.
The following figure gives an example of the power efficiency under different TPC-E load levels on a 4-socket
production server running Windows Server 2008 R2. It shows an 8% improvement at medium load levels
compared to Windows Server 2008.

Customized Tuning Suggestions

If your primary workload characteristics differ significantly from the five workloads used for the default Balanced
power plan PPM tuning, you can experiment by altering one or more PPM parameters to find the best fit for your
environment.
Due to the number and complexity of parameters, this may be a challenging task, but if you are looking for the
best tradeoff between energy consumption and workload efficacy for your particular environment, it may be
worth the effort.
The complete set of tunable PPM parameters can be found in Processor power management tuning. Some of the
simplest power parameters to start with could be:
Processor Performance Increase Threshold and Processor Performance Increase Time larger
values slow the perf response to increased activity
Processor Performance Decrease Threshold large values quicken the power response to idle periods
Processor Performance Decrease Time larger values more gradually decrease perf during idle periods
Processor Performance Increase Policy the Single policy slows the perf response to increased and
sustained activity; the Rocket policy reacts quickly to increased activity
Processor Performance Decrease Policy the Single policy more gradually decreases perf over longer
idle periods; the Rocket policy drops power very quickly when entering an idle period
 IMPORTANT
Before starting any experiments, you should first understand your workloads, which will help you make the right PPM
parameter choices and reduce the tuning effort.

Understand high-level performance and power requirements

If your workload is real time (e.g., susceptible to glitching or other visible end-user impacts) or has very tight
responsiveness requirement (e.g., a stock brokerage), and if energy consumption is not a primary criteria for your
environment, you should probably just switch to the High Performance power plan. Otherwise, you should
understand the response time requirements of your workloads, and then tune the PPM parameters for the best
possible power efficiency that still meets those requirements.
Understand underlying workload characteristics
You should know your workloads and design the experiment parameter sets for tuning. For example, if
frequencies of the CPU cores need to be ramped very fast (perhaps you have a bursty workload with significant
idle periods, but you need very quick responsiveness when a new transaction comes along), then the processor
performance increase policy might need to be set to rocket (which, as the name implies, shoots the CPU core
frequency to its maximum value rather than stepping it up over a period of time).
If your workload is very bursty, the PPM check interval can be reduced to make the CPU frequency start stepping
up sooner after a burst arrives. If your workload doesnt have high thread concurrency, then core parking can be
enabled to force the workload to execute on a smaller number of cores, which could also potentially improve
processor cache hit ratios.
If you just want to increase CPU frequencies at medium utilization levels (i.e., not light workload levels), then
processor performance increase/decrease thresholds can be adjusted to not react until certain levels of activity are
observed.
Understand periodic behaviors
There may be different performance requirements for daytime and nighttime or over the weekends, or there may
be different workloads that run at different times. In this case, one set of PPM parameters might not be optimal for
all time periods. Since multiple custom power plans can be devised, it is possible to even tune for different time
periods and switch between power plans through scripts or other means of dynamic system configuration.
Again, this adds to the complexity of the optimization process, so it is a question of how much value will be gained
from this type of tuning, which will likely need to be repeated when there are significant hardware upgrades or
workload changes.
This is why Windows provides a Balanced power plan in the first place, because in many cases it is probably not
worth the effort of hand-tuning for a specific workload on a specific server.

See Also
Server Hardware Performance Considerations
Server Hardware Power Considerations
Power and Performance Tuning
Processor Power Management Tuning
Recommended Balanced Plan Parameters
 Recommended Balanced Power Plan Parameters for
Workloads Requiring Quick Response Times
4/24/2017 3 min to read Edit Online

The default Balanced power plan uses throughput as the performance metric for tuning. During the steady
state, throughput does not change with varying utilizations till the system is totally overloaded (~100%
utilization). As a result, the Balanced power plan favors power quite a lot with minimizing processor frequency
and maximizing utilization.
However response time could exponentially increase with utilization increases. Nowadays, the requirement of
quick response time has dramatically increased. Even though Microsoft suggested the users to switch to the High
Performance power plan when they need quick response time, some users do not want to lose the power benefit
during light to medium load levels. Hence, Microsoft provides the following set of suggested parameter changes
for the workloads requiring quick response time.

PARAMETER DESCRIPTION DEFAULT VALUE PROPOSED VALUE

Processor performance Utilization threshold above 90 60

increase threshold which the frequency is to
increase

Processor performance Utilization threshold below 80 40

decrease threshold which the frequency is to
decrease

Processor performance Number of PPM check 3 1

increase time windows before the
frequency is to increase

Processor performance How fast the frequency is to Single Ideal

increase policy increase

To set the proposed values, the users can run the following commands in a window with administrator:

Powercfg -setacvalueindex scheme_balanced sub_processor PERFINCTHRESHOLD 60

Powercfg -setacvalueindex scheme_balanced sub_processor PERFDECTHRESHOLD 40
Powercfg -setacvalueindex scheme_balanced sub_processor PERFINCTIME 1
Powercfg -setacvalueindex scheme_balanced sub_processor PERFINCPOL 0
Powercfg -setactive scheme_balanced

This change is based on the performance and power tradeoff analysis using the following workloads. For the
users who want to further fine tune the power efficiency with certain SLA requirements, please refer to Server
Hardware Performance Considerations.

NOTE
For additional recommendations and insight on leveraging power plans to tune virtualized workloads, read Hyper-v
Configuration
SPECpower JAVA workload
SPECpower_ssj2008, the most popular industry-standard SPEC benchmark for server power and performance
characteristics, is used to check the power impact. Since it only uses throughput as performance metric, the
default Balanced power plan provides the best power efficiency.
The proposed parameter change consumes slightly higher power at the light (i.e., <= 20%) load levels. But with
the higher load level, the difference increases, and it starts to consume same power as the High Performance
power plan after the 60% load level. To use the proposed change parameters, the users should be aware of the
power cost at medium to high load levels during their rack capacity planning.

GeekBench 3
GeekBench 3 is a cross-platform processor benchmark that separates the scores for single-core and multi-core
performance. It simulates a set of workloads including integer workloads (encryptions, compressions, image
processing, etc.), floating point workloads (modeling, fractal, image sharpening, image blurring, etc.) and memory
workloads (streaming).
Response time is a major measure in its score calculation. In our tested system, the default Balanced power plan
has ~18% regression in single-core tests and ~40% regression in multi-core tests compared to the High
Performance power plan. The proposed change removes these regressions.

DiskSpd
Diskspd is a command-line tool for storage benchmarking developed by Microsoft. It is widely used to generate a
variety of requests against storage systems for the storage performance analysis.
We set up a [Failover Cluster], and used Diskspd to generate random and sequential, and read and write IOs to the
local and remote storage systems with different IO sizes. Our tests show that the IO response time is very
sensitive to processor frequency under different power plans. The Balanced power plan could double the
response time of that from the High Performance power plan under certain workloads. The proposed change
removes most of the regressions.

IMPORTANT
Starting from Intel [Broadwell] processors running Windows Server 2016, most of the processor power management
decisions are made in the processor instead of OS level to achieve quicker adaption to the workload changes. The legacy
PPM parameters used by OS have minimal impact on the actual frequency decisions, except telling the processor if it should
favor power or performance, or capping the minimal and maximum frequencies. Hence, the proposed PPM parameter
change is only targeting to the pre-Broadwell systems.

See Also
Server Hardware Performance Considerations
Server Hardware Power Considerations
Power and Performance Tuning
Processor Power Management Tuning
Failover Cluster
 Performance tuning Active Directory Servers
4/24/2017 1 min to read Edit Online

Performance tuning Active Directory is focused on two goals:

Active Directory is optimally configured to service the load in the most efficient manner possible
Workloads submitted to Active Directory should be as efficient as possible
This requires proper attention to three separate areas:
Proper capacity planning ensuring sufficient hardware is in place to support existing load
Server side tuning configuring domain controllers to handle the load as efficiently as possible
Active Directory client/application tuning ensuring that clients and applications are using Active Directory in
an optimal fashion

Start with capacity planning

Properly deploying a sufficient number of domain controllers, in the right domain, in the right locales, and to
accommodate redundancy is critical to ensuring servicing client requests in a timely fashion. This is an in-depth
topic and outside of the scope of this guide. Readers readers are encouraged start to their Active Directory
performance tuning by reading and undestanding the recommendations and guidance found in Capacity Planning
for Active Directory Domain Services.

IMPORTANT
Proper configuration and sizing of Active Directory has a significant potential impact on overall system and workload
performance. Readers are highly encouraged to start by reading Capacity Planning for Active Directory Domain Services.

Updates and evolving recommendations

Massive improvements in both Active Directory and client performance optimizations have occurred over the last
several generations of the operating system and these efforts continue. We recommend that the most current
versions of the platform be deployed to get the benefits, including:
Increased reliability
Better performance
Better logging and tools to troubleshoot
However, we realize that this takes time and many environments are running in a scenario where 100% adoption
of the most current platform is impossible. Some improvements have been added to older versions of the platform
and well continue to add more.
We encourage you to stay up to date on the lastest news, guidance and best practices for managing ADDS by
following our team blog, "Ask the Directory Services Team".

See also
Hardware considerations
LDAP considerations
Proper placement of domain controllers and site considerations
Troubleshooting ADDS performance
Capacity Planning for Active Directory Domain Services
 Proper placement of domain controllers and site
considerations
4/24/2017 6 min to read Edit Online

Proper site definition is critical to performance. Clients falling out of site can experience poor performance for
authentications and queries. Furthermore, with the introduction of IPv6 on clients, the request can come from
either the IPv4 or the IPv6 address and Active Directory needs to have sites properly defined for IPv6. The
operating system prefers IPv6 to IPv4 when both are configured.
Starting in Windows Server 2008, the domain controller attempts to use name resolution to do a reverse lookup in
order to determine the site the client should be in. This can cause exhaustion of the ATQ Thread Pool and cause the
domain controller to become unresponsive. The appropriate resolution to this is to properly define the site
topology for IPv6. As a workaround, one can optimize the name resolution infrastructure to respond quickly to
domain controller requests. For more info see Windows Server 2008 or Windows Server 2008 R2 Domain
Controller delayed response to LDAP or Kerberos requests.
An additional area of consideration is locating Read/Write DCs for scenarios where RODCs are in use. Certain
operations require access to a writable Domain Controller or target a writable Domain Controller when a Read-
Only Domain Controller would suffice. Optimizing these scenarios would take two paths:
Contacting writable Domain Controllers when a Read-Only Domain Controller would suffice. This requires an
application code change.
Where a writable Domain Controller may be necessary. Place read-write Domain Controllers at central locations
to minimize latency.
For further information reference:
Application Compatibility with RODCs
Active Directory Service Interface (ADSI) and the Read Only Domain Controller (RODC) Avoiding performance
issues

Optimize for referrals

Referrals are how LDAP queries are redirected when the domain controller does not host a copy of the partition
queried. When a referral is returned, it contains the distinguished name of the partition, a DNS name, and a port
number. The client uses this information to continue the query on a server that hosts the partition. This is a
DCLocator scenario and all of the recommendations site definitions and domain controller placement is
maintained, but applications which depend on referrals are often overlooked. It is recommended to ensure AD
Topology including site definitions and domain controller placement properly reflects the needs of the client. Also,
this may include having domain controllers from multiple domains in a single site, tuning DNS settings, or
relocating the site of an application.

Optimization considerations for trusts

In an intra-forest scenario, trusts are processed according to the following domain hierarchy: Grand-Child Domain
-> Child Domain -> Forest Root Domain -> Child Domain -> Grand-Child Domain. This means that secure
channels at the forest root, and each parent, can become overloaded due to aggregation of authentication requests
transiting the DCs in the trust hierarchy. This may also incur delays in Active Directories of large geographical
dispersion when authentication also has to transit highly latent links to affect the above flow. Overloads can occur
in inter-forest and down-level trust scenarios. The following recommendations apply to all scenarios:
 Properly tune the MaxConcurrentAPI to support the load across the secure channel. For more info, see How
to do performance tuning for NTLM authentication by using the MaxConcurrentApi setting.
Create shortcut trusts as appropriate based on load.
Ensure that every domain controller in the domain is able to perform name resolution and communicate
with the domain controllers in the trusted domain.
Ensure locality considerations are taken into account for trusts.
Enable Kerberos where possible and minimize use of the secure channel to reduce risk of running into
MaxConcurrentAPI bottlenecks.
Cross domain trust scenarios are an area that has been consistently a pain point for many customers. Name
resolution and connectivity issues, often due to firewalls, cause resource exhaustion on the trusting domain
controller and impact all clients. Furthermore, an often overlooked scenario is optimizing access to trusted domain
controllers. The key areas to ensure this works properly are as follows:
Ensure the DNS and WINS name resolution that the trusting domain controllers are using can resolve an
accurate list of domain controllers for the trusted domain.
Statically added records have a tendency to become stale and reintroduce connectivity problems over
time. DNS forwards, Dynamic DNS, and merging WINS/DNS infrastructures are more maintainable
in the long run.
Ensure proper configuration of forwarders, conditional forwards, and secondary copies for both
forward and reverse lookup zones for every resource in the environment which a client may need to
access. Again, this requires manual maintenance and has a tendency to become stale. Consolidation
of infrastructures is ideal.
Domain controllers in the trusting domain will attempt to locate domain controllers in the trusted domain
that are in the same site first and then failback to the generic locators.
For more info on how DCLocator works, see Finding a Domain Controller in the Closest Site.
Converge site names between the trusted and trusting domains to reflect domain controller in the
same location. Ensure subnet and IP address mappings are properly linked to sites in both forests.
For more info, see Domain Locator Across a Forest Trust.
Ensure ports are open, according to DCLocator needs, for domain controller location. If firewalls exist
between the domains, ensure that the firewalls are properly configured for ALL trusts. If firewalls are
not open, the trusting domain controller will still attempt to access the trusted domain. If
communication fails for any reason, the trusting domain controller will eventually time out the
request to the trusted domain controller. However, these time outs can take several seconds per
request and can exhaust network ports on the trusting domain controller if the volume of incoming
requests is high. The client may experience the waits to timeout at the domain controller as hung
threads, which could translate to hung applications (if the application runs the request in the
foreground thread). For more info, see How to configure a firewall for domains and trusts.
Use DnsAvoidRegisterRecords to eliminate poorly performing or high-latency domain controllers,
such as those in satellite sites, from advertising to the generic locators. For more info, see How to
optimize the location of a domain controller or global catalog that resides outside of a client's site.
Note
There is a practical limit of about 50 to the number of domain controllers the client can consume.
These should be the most site-optimal and highest capacity domain controllers.
Consider placing domain controllers from trusted and trusting domains in the same physical location.
For all trust scenarios, credentials are routed according to the domain specified in the authentication requests. This
is also true for queries to the LookupAccountName and LsaLookupNames (as well as others, these are just the
most commonly used) APIs. When the domain parameters for these APIs are passed a NULL value, the domain
controller will attempt to find the account name specified in every trusted domain available.
Disable checking all available trusts when NULL domain is specified. How to restrict the lookup of isolated
names in external trusted domains by using the LsaLookupRestrictIsolatedNameLevel registry entry
Disable passing authentication requests with NULL domain specified across all available trusts. The
Lsass.exe process may stop responding if you have many external trusts on an Active Directory domain
controller

See also
Performance tuning Active Directory Servers
Hardware considerations
LDAP considerations
Troubleshooting ADDS performance
Capacity Planning for Active Directory Domain Services
 Hardware considerations in ADDS performance
tuning
4/24/2017 5 min to read Edit Online

IMPORTANT
The following is a summary of the key recommendations and considerations to optimize server hardware for Active Directory
workloads covered in greater depth in the Capacity Planning for Active Directory Domain Services article. Readers are highly
encouraged to review Capacity Planning for Active Directory Domain Services for a greater technical understanding and
implications of these recommendations.

Avoid going to disk

Active Directory caches as much of the database as memory allows. Fetching pages from memory are orders of
magnitude faster than going to physical media, whether the media is spindle or SSD based. Add more memory to
minimize disk I/O.
Active Directory Best Practices recommend putting enough RAM to load the entire DIT into memory, plus
accommodate the operating system and other installed applications, such as anti-virus, backup software,
monitoring, and so on.
For limitations of the legacy platforms, see Memory usage by the Lsass.exe process on domain
controllers that are running Windows Server 2003 or Windows 2000 Server.
Use the Memory\Long-Term Average Standby Cache Lifetime (s) > 30 minutes performance counter.
Put the operating system, logs, and the database on separate volumes. If all or the majority of the DIT can be
cached, once the cache is warmed and under a steady state, this becomes less relevant and offers a little
more flexibility in storage layout. In scenarios where the entire DIT cannot be cached, the importance of
splitting the operating system, logs, and database on separate volumes becomes more important.
Normally, I/O ratios to the DIT are about 90% read and 10% write. Scenarios where write I/O volumes
significantly exceed 10% - 20% are considered write-heavy. Write-heavy scenarios do not greatly benefit
from the Active Directory cache. To guarantee the transactional durability of data that is written to the
directory, Active Directory does not perform disk write caching. Instead, it commits all write operations to
the disk before it returns a successful completion status for an operation, unless there is an explicit request
not to do this. Therefore, fast disk I/O is important to the performance of write operations to Active
Directory. The following are hardware recommendations that might improve performance for these
scenarios:
Hardware RAID controllers
Increase the number of low-latency/high-RPM disks hosting the DIT and log files
Write caching on the controller
Review the disk subsystem performance individually for each volume. Most Active Directory scenarios are
predominantly read-based, thus the statistics on the volume hosting the DIT are the most important to
inspect. However, do not overlook monitoring the rest of the drives, including the operating system, and log
files drives. To determine if the domain controller is properly configured to avoid storage being the
bottleneck for performance, reference the section on Storage Subsystems for standards storage
 recommendations. Across many environments, the philosophy is to ensure that there is enough head room
to accommodate surges or spikes in load. These thresholds are warning thresholds where the head room to
accommodate surges or spikes in load becomes constrained and client responsiveness degrades. In short,
exceeding these thresholds is not bad in the short term (5 to 15 minutes a few times a day), however a
system running sustained with these sorts of statistics is not fully caching the database and may be over
taxed and should be investigated.
Database ==> Instances(lsass/NTDSA)\I/O Database Reads Averaged Latency < 15ms
Database ==> Instances(lsass/NTDSA)\I/O Database Reads/sec < 10
Database ==> Instances(lsass/NTDSA)\I/O Log Writes Averaged Latency < 10ms
Database ==> Instances(lsass/NTDSA)\I/O Log Writes/sec informational only.
To maintain consistency of data, all changes must be written to the log. There is no good or bad
number here, it is only a measure of how much the storage is supporting.
Plan non-core disk I/O loads, such as backup and anti-virus scans, for non-peak load periods. Also, use
backup and anti-virus solutions that support the low-priority I/O feature introduced in Windows Server
2008 to reduce competition with I/O needs of Active Directory.

Dont over tax the processors

Processors that dont have enough free cycles can cause long wait times on getting threads on to the processor for
execution. Across many environments, the philosophy is to ensure that there is enough head room to
accommodate surges or spikes in load to minimize impact on client responsiveness in these scenarios. In short,
exceeding the below thresholds is not bad in the short term (5 to 15 minutes a few times a day), however a system
running sustained with these sorts of statistics doesnt provide any head room to accommodate abnormal loads
and can easily be put into an over taxed scenario. Systems spending sustained periods above the thresholds
should be investigated to how to reduce processor loads.
For more info on how to select a processor, see Performance Tuning for Server Hardware.
Add hardware, optimize load, direct clients elsewhere, or remove load from the environment to reduce CPU
load.
Use the Processor Information(_Total)\% Processor Utilization < 60% performance counter.

Avoid overloading the network adapter

Just like with processors, excessive network adapter utilization will cause long wait times for the outbound traffic
to get on to the network. Active Directory tends to have small inbound requests and relatively to significantly
larger amounts of data returned to the client systems. Sent data far exceeds received data. Across many
environments, the philosophy is to ensure that there is enough head room to accommodate surges or spikes in
load. This threshold is a warning threshold where the head room to accommodate surges or spikes in load
becomes constrained and client responsiveness degrades. In short, exceeding these thresholds is not bad in the
short term (5 to 15 minutes a few times a day), however a system running sustained with these sorts of statistics is
over taxed and should be investigated.
For more info on how to tune the network subsystem, see Performance Tuning for Network Subsystems.
Use the Compare NetworkInterface(*)\Bytes Sent/Sec with NetworkInterface(*)\Current Bandwidth
performance counter. The ratio should be less than 60% utilized.

See also
Performance tuning Active Directory Servers
LDAP considerations
Proper placement of domain controllers and site considerations
Troubleshooting ADDS performance
Capacity Planning for Active Directory Domain Services
 LDAP considerations in ADDS performance tuning
4/24/2017 5 min to read Edit Online

IMPORTANT
The following is a summary of the key recommendations and considerations to optimize server hardware for Active Directory
workloads covered in greater depth in the Capacity Planning for Active Directory Domain Services article. Readers are highly
encouraged to review Capacity Planning for Active Directory Domain Services for a greater technical understanding and
implications of these recommendations.

Verify LDAP queries

Verify that LDAP queries conform with the creating efficient queries recommendations.
There is extensive documentation on MSDN about how to properly write, structure, and analyze queries for use
against Active Directory. For more info, see Creating More Efficient Microsoft Active Directory-Enabled
Applications.

Optimize LDAP page sizes

When returning results with multiple objects in response to client requests, the domain controller has to
temporarily store the result set in memory. Increasing page sizes will cause more memory usage and can age
items out of cache unnecessarily. In this case, the default settings are optimal. There are several scenarios where
recommendations were made to increase the page size settings. We recommend using the default values unless
specifically identified as inadequate.
When queries have many results, a limit of similar queries concurrently executed may be encountered. This occurs
as the LDAP server may deplete a global memory area known as the cookie pool. It may be necessary to increase
the size of the pool as discussed in How LDAP Server Cookies Are Handled.
To tune these settings, see Windows Server 2008 and newer domain controller returns only 5000 values in a LDAP
response.

Determine whether to add indices

Indexing attributes is useful when searching for objects that have the attribute name in a filter. Indexing can reduce
the number of objects that must be visited when evaluating the filter. However, this reduces the performance of
write operations because the index must be updated when the corresponding attribute is modified or added. It also
increases the size of the directory database, though the benefits often outweigh the cost of storage. Logging can be
used to find the expensive and inefficient queries. Once identified, consider indexing some attributes that are used
in the corresponding queries to improve the search performance. For more info on how Active Directory Searches
work, see How Active Directory Searches Work.
Scenarios that benefit in adding indices
Client load in requesting the data is generating significant CPU usage and the client query behavior cannot
be changed or optimized. By significant load, consider that it is showing itself in a Top 10 offender list in
Server Performance Advisor or the built-in Active Directory Data Collector Set and is using more than 1% of
CPU.
The client load is generating significant disk I/O on a server due to an unindexed attribute and the client
 query behavior cannot be changed or optimized.
A query is taking a long time and is not completing in an acceptable timeframe to the client due to lack of
covering indices.
Large volumes of queries with high durations are causing consumption and exhaustion of ATQ LDAP
Threads. Monitor the following performance counters:
NTDS\Request Latency This is subject to how long the request takes to process. Active Directory
times out requests after 120 seconds (default), however, the majority should run much faster and
extremely long running queries should get hidden in the overall numbers. Look for changes in this
baseline, rather than absolute thresholds.
Note
High values here can also be indicators of delays in proxying requests to other domains and CRL
checks.
NTDS\Estimated Queue Delay This should ideally be near 0 for optimal performance as this
means that requests spend no time waiting to be serviced.
These scenarios can be detected using one or more of the following approaches:
Determining Query Timing with the Statistics Control
Tracking Expensive and Inefficient Searches
Active Directory Diagnostics Data Collector Set in Performance Monitor (Son of SPA: AD Data Collector Sets
in Win2008 and beyond)
Microsoft Server Performance Advisor Active Directory Advisor Pack
Searches using any filter besides (objectClass=*) that use the Ancestors Index.
Other index considerations
Ensure that creating the index is the right solution to the problem after tuning the query has been exhausted
as an option. Sizing hardware properly is very important. Indices should be added only when the right fix is
to index the attribute, and not an attempt to obfuscate hardware problems.
Indices increase the size of the database by a minimum of the total size of the attribute being indexed. An
estimate of database growth can therefore be evaluated by taking the average size of the data in the
attribute and multiplying by the number of objects that will have the attribute populated. Generally this is
about a 1% increase in database size. For more info, see How the Data Store Works.
If search behavior is predominantly done at the organization unit level, consider indexing for containerized
searches.
Tuple indices are larger than normal indices, but it is much harder to estimate the size. Use normal indices
size estimates as the floor for growth, with a maximum of 20%. For more info, see How the Data Store
Works.
If search behavior is predominantly done at the organization unit level, consider indexing for containerized
searches.
Tuple Indices are needed to support medial search strings and final search strings. Tuple indices are not
needed for initial search strings.
Initial Search String (samAccountName=MYPC*)
Medial Search String - (samAccountName=*MYPC*)
Final Search String (samAccountName=*MYPC$)
 Creating an index will generate disk I/O while the index is being built. This is done on a background thread
with lower priority and incoming requests will be prioritized over the index build. If capacity planning for
the environment has been done correctly, this should be transparent. However, write-heavy scenarios or an
environment where the load on the domain controller storage is unknown could degrade client experience
and should be done off-hours.
Affects to replication traffic is minimal since building indices occurs locally.
For more info, see the following:
Creating More Efficient Microsoft Active Directory-Enabled Applications
Searching in Active Directory Domain Services
Indexed Attributes

See also
Performance tuning Active Directory Servers
Hardware considerations
Proper placement of domain controllers and site considerations
Troubleshooting ADDS performance
Capacity Planning for Active Directory Domain Services
 Troubleshooting Active Directory Domain Services
performance
4/24/2017 1 min to read Edit Online

For additional information on ADDS performance troubleshooting, see Monitoring Your Branch Office
Environment.

See also
Performance tuning Active Directory Servers
Hardware considerations
LDAP considerations
Proper placement of domain controllers and site considerations
Capacity Planning for Active Directory Domain Services
 Performance tuning for file servers
4/24/2017 6 min to read Edit Online

You should select the proper hardware to satisfy the expected file server load, considering average load, peak load,
capacity, growth plans, and response times. Hardware bottlenecks limit the effectiveness of software tuning.

General tuning parameters for clients

The following REG_DWORD registry settings can affect the performance of client computers that interact with SMB
file servers:
ConnectionCountPerNetworkInterface

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\ConnectionCountPerNetworkInterface

Applies to Windows 10, Windows 8.1, Windows 8, Windows Server 2016, Windows Server 2012 R2, and
Windows Server 2012
The default is 1, and we strongly recommend using the default. The valid range is 1-16. The maximum
number of connections per interface to be established with a server for non-RSS interfaces.
ConnectionCountPerRssNetworkInterface

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\ConnectionCountPerRssNetworkInterfac
e

Applies to Windows 10, Windows 8.1, Windows 8, Windows Server 2016, Windows Server 2012 R2, and
Windows Server 2012
The default is 4, and we strongly recommend using the default. The valid range is 1-16. The maximum
number of connections per interface to be established with a server for RSS interfaces.
ConnectionCountPerRdmaNetworkInterface

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\ConnectionCountPerRdmaNetworkInterfa
ce

Applies to Windows 10, Windows 8.1, Windows 8, Windows Server 2016, Windows Server 2012 R2, and
Windows Server 2012
The default is 2, and we strongly recommend using the default. The valid range is 1-16. The maximum
number of connections per interface to be established with a server for RDMA interfaces.
MaximumConnectionCountPerServer

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\MaximumConnectionCountPerServer

Applies to Windows 10, Windows 8.1, Windows 8, Windows Server 2016, Windows Server 2012 R2, and
Windows Server 2012
The default is 32, with a valid range from 1-64. The maximum number of connections to be established with
a single server running Windows Server 2012 across all interfaces.
DormantDirectoryTimeout

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DormantDirectoryTimeout

Applies to Windows 10, Windows 8.1, Windows 8, Windows Server 2016, Windows Server 2012 R2, and
Windows Server 2012
The default is 600 seconds. The maximum time server directory handles held open with directory leases.
FileInfoCacheLifetime

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\FileInfoCacheLifetime

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 10 seconds. The file information cache timeout period.
DirectoryCacheLifetime

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DirectoryCacheLifetime

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 10 seconds. This is the directory cache timeout.
Note
This parameter controls caching of directory metadata in the absence of directory leases.
DirectoryCacheEntrySizeMax

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DirectoryCacheEntrySizeMax

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 64 KB. This is the maximum size of directory cache entries.
FileNotFoundCacheLifetime

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\FileNotFoundCacheLifetime

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 5 seconds. The file not found cache timeout period.
CacheFileTimeout

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\CacheFileTimeout

Applies to Windows 8.1, Windows 8, Windows Server 2012, Windows Server 2012 R2, and Windows 7
The default is 10 seconds. This setting controls the length of time (in seconds) that the redirector will hold on
to cached data for a file after the last handle to the file is closed by an application.
DisableBandwidthThrottling

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DisableBandwidthThrottling

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 0. By default, the SMB redirector throttles throughput across high-latency network
connections, in some cases to avoid network-related timeouts. Setting this registry value to 1 disables this
throttling, enabling higher file transfer throughput over high-latency network connections.
DisableLargeMtu

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DisableLargeMtu

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 0 for Windows 8 only. In Windows 8, the SMB redirector transfers payloads as large as 1 MB
per request, which can improve file transfer speed. Setting this registry value to 1 limits the request size to
64 KB. You should evaluate the impact of this setting before applying it.
RequireSecuritySignature

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\RequireSecuritySignature

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 0, disabling SMB Signing. Changing this value to 1 enables SMB signing for all SMB
communication, preventing SMB communication with computers where SMB signing is disabled. SMB
signing can increase CPU cost and network round trips, but helps block man-in-the-middle attacks. If SMB
signing is not required, ensure that this registry value is 0 on all clients and servers.
For more info, see The Basics of SMB Signing.
FileInfoCacheEntriesMax

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\FileInfoCacheEntriesMax

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 64, with a valid range of 1 to 65536. This value is used to determine the amount of file
metadata that can be cached by the client. Increasing the value can reduce network traffic and increase
performance when a large number of files are accessed.
DirectoryCacheEntriesMax

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DirectoryCacheEntriesMax
 Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 16, with a valid range of 1 to 4096. This value is used to determine the amount of directory
information that can be cached by the client. Increasing the value can reduce network traffic and increase
performance when large directories are accessed.
FileNotFoundCacheEntriesMax

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\FileNotFoundCacheEntriesMax

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 128, with a valid range of 1 to 65536. This value is used to determine the amount of file name
information that can be cached by the client. Increasing the value can reduce network traffic and increase
performance when a large number of file names are accessed.
MaxCmds

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\MaxCmds

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 15. This parameter limits the number of outstanding requests on a session. Increasing the
value can use more memory, but it can improve performance by enabling a deeper request pipeline.
Increasing the value in conjunction with MaxMpxCt can also eliminate errors that are encountered due to
large numbers of outstanding long-term file requests, such as FindFirstChangeNotification calls. This
parameter does not affect connections with SMB 2.0 servers.
DormantFileLimit

HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DormantFileLimit

Applies to Windows 10, Windows 8.1, Windows 8, Windows 7, Windows Vista, Windows Server 2016,
Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008
The default is 1023. This parameter specifies the maximum number of files that should be left open on a
shared resource after the application has closed the file.
Client tuning example
The general tuning parameters for client computers can optimize a computer for accessing remote file shares,
particularly over some high-latency networks (such as branch offices, cross-datacenter communication, home
offices, and mobile broadband). The settings are not optimal or appropriate on all computers. You should evaluate
the impact of individual settings before applying them.

PARAMETER VALUE DEFAULT

DisableBandwidthThrottling 1 0

FileInfoCacheEntriesMax 32768 64

DirectoryCacheEntriesMax 4096 16
 PARAMETER VALUE DEFAULT

FileNotFoundCacheEntriesMax 32768 128

MaxCmds 32768 15

Starting in Windows 8, you can configure many of these SMB settings by using the Set-SmbClientConfiguration
and Set-SmbServerConfiguration Windows PowerShell cmdlets. Registry-only settings can be configured by
using Windows PowerShell as well.

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters"

RequireSecuritySignature -Value 0 -Force
 Performance tuning for SMB file servers
5/16/2017 6 min to read Edit Online

SMB configuration considerations

Do not enable any services or features that your file server and clients do not require. These might include SMB
signing, client-side caching, file system mini-filters, search service, scheduled tasks, NTFS encryption, NTFS
compression, IPSEC, firewall filters, Teredo, and SMB encryption.
Ensure that the BIOS and operating system power management modes are set as needed, which might include High
Performance mode or altered C-State. Ensure that the latest, most resilient, and fastest storage and networking
device drivers are installed.
Copying files is a common operation performed on a file server. Windows Server has several built-in file copy
utilities that you can run by using a command prompt. Robocopy is recommended. Introduced in Windows Server
2008 R2, the /mt option of Robocopy can significantly improve speed on remote file transfers by using multiple
threads when copying multiple small files. We also recommend using the /log option to reduce console output by
redirecting logs to a NUL device or to a file. When you use Xcopy, we recommend adding the /q and /k options to
your existing parameters. The former option reduces CPU overhead by reducing console output and the latter
reduces network traffic.

SMB performance tuning

File server performance and available tunings depend on the SMB protocol that is negotiated between each client
and the server, and on the deployed file server features. The highest protocol version currently available is SMB
3.1.1 in Windows Server 2016 and Windows 10. You can check which version of SMB is in use on your network by
using Windows PowerShell Get-SMBConnection on clients and Get-SMBSession | FL on servers.
SMB 3.0 protocol family
SMB 3.0 was introduced in Windows Server 2012 and further enhanced in Windows Server 2012 R2 (SMB 3.02)
and Windows Server 2016 (SMB 3.1.1). This version introduced technologies that may significantly improve
performance and availability of the file server. For more info, see SMB in Windows Server 2012 and 2012 R2 2012
and What's new in SMB 3.1.1.
SMB Direct
SMB Direct introduced the ability to use RDMA network interfaces for high throughput with low latency and low
CPU utilization.
Whenever SMB detects an RDMA-capable network, it automatically tries to use the RDMA capability. However, if for
any reason the SMB client fails to connect using the RDMA path, it will simply continue to use TCP/IP connections
instead. All RDMA interfaces that are compatible with SMB Direct are required to also implement a TCP/IP stack,
and SMB Multichannel is aware of that.
SMB Direct is not required in any SMB configuration, but it' s always recommended for those who want lower
latency and lower CPU utilization.
For more info about SMB Direct, see Improve Performance of a File Server with SMB Direct.
SMB Multichannel
SMB Multichannel allows file servers to use multiple network connections simultaneously and provides increased
throughput.
For more info about SMB Multichannel, see Deploy SMB Multichannel.
SMB Scale -Out
SMB Scale-out allows SMB 3.0 in a cluster configuration to show a share in all nodes of a cluster. This active/active
configuration makes it possible to scale file server clusters further, without a complex configuration with multiple
volumes, shares and cluster resources. The maximum share bandwidth is the total bandwidth of all file server
cluster nodes. The total bandwidth is no longer limited by the bandwidth of a single cluster node, but rather
depends on the capability of the backing storage system. You can increase the total bandwidth by adding nodes.
For more info about SMB Scale-Out, see Scale-Out File Server for Application Data Overview and the blog post To
scale out or not to scale out, that is the question.
Performance counters for SMB 3.0
The following SMB performance counters were introduced in Windows Server 2012, and they are considered a
base set of counters when you monitor the resource usage of SMB 2 and higher versions. Log the performance
counters to a local, raw (.blg) performance counter log. It is less expensive to collect all instances by using the
wildcard character (*), and then extract particular instances during post-processing by using Relog.exe.
SMB Client Shares
These counters display information about file shares on the server that are being accessed by a client that is
using SMB 2.0 or higher versions.
If you' re familiar with the regular disk counters in Windows, you might notice a certain resemblance. That' s
not by accident. The SMB client shares performance counters were designed to exactly match the disk
counters. This way you can easily reuse any guidance on application disk performance tuning you currently
have. For more info about counter mapping, see Per share client performance counters blog.
SMB Server Shares
These counters display information about the SMB 2.0 or higher file shares on the server.
SMB Server Sessions
These counters display information about SMB server sessions that are using SMB 2.0 or higher.
Turning on counters on server side (server shares or server sessions) may have significant performance
impact for high IO workloads.
Resume Key Filter
These counters display information about the Resume Key Filter.
SMB Direct Connection
These counters measure different aspects of connection activity. A computer can have multiple SMB Direct
connections. The SMB Direct Connection counters represent each connection as a pair of IP addresses and
ports, where the first IP address and port represent the connection's local endpoint, and the second IP
address and port represent the connection's remote endpoint.
Physical Disk, SMB, CSV FS performance counters relationships
For more info on how Physical Disk, SMB, and CSV FS (file system) counters are related, see the following
blog post: Cluster Shared Volume Performance Counters.

Tuning parameters for SMB file servers

The following REG_DWORD registry settings can affect the performance of SMB file servers:
Smb2CreditsMin and Smb2CreditsMax
 HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters\Smb2CreditsMin

HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters\Smb2CreditsMax

The defaults are 512 and 8192, respectively. These parameters allow the server to throttle client operation
concurrency dynamically within the specified boundaries. Some clients might achieve increased throughput
with higher concurrency limits, for example, copying files over high-bandwidth, high-latency links.

TIP
You can monitor SMB Client Shares\Credit Stalls /Sec to see if there are any issues with credits.

AdditionalCriticalWorkerThreads

HKLM\System\CurrentControlSet\Control\Session Manager\Executive\AdditionalCriticalWorkerThreads

The default is 0, which means that no additional critical kernel worker threads are added. This value affects
the number of threads that the file system cache uses for read-ahead and write-behind requests. Raising this
value can allow for more queued I/O in the storage subsystem, and it can improve I/O performance,
particularly on systems with many logical processors and powerful storage hardware.

TIP
The value may need to be increased if the amount of cache manager dirty data (performance counter Cache\Dirty
Pages) is growing to consume a large portion (over ~25%) of memory or if the system is doing lots of synchronous
read I/Os.

MaxThreadsPerQueue

HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters\MaxThreadsPerQueue

The default is 20. Increasing this value raises the number of threads that the file server can use to service
concurrent requests. When a large number of active connections need to be serviced, and hardware
resources, such as storage bandwidth, are sufficient, increasing the value can improve server scalability,
performance, and response times.

TIP
An indication that the value may need to be increased is if the SMB2 work queues are growing very large
(performance counter Server Work Queues\Queue Length\SMB2 NonBlocking *' is consistently above ~100).

AsynchronousCredits

HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters\MaxThreadsPerQueue

The default is 512. This parameter limits the number of concurrent asynchronous SMB commands that are
allowed on a single connection. Some cases (such as when there is a front-end server with a back-end IIS
server) require a large amount of concurrency (for file change notification requests, in particular). The value
of this entry can be increased to support these cases.
SMB server tuning example
The following settings can optimize a computer for file server performance in many cases. The settings are not
optimal or appropriate on all computers. You should evaluate the impact of individual settings before applying
them.

PARAMETER VALUE DEFAULT

AdditionalCriticalWorkerThreads 64 0

MaxThreadsPerQueue 64 20

SMB client performance monitor counters

For more info about SMB client counters, see Windows Server 2012 File Server Tip: New per-share SMB client
performance counters provide great insight.
 Performance Tuning NFS File Servers
4/24/2017 5 min to read Edit Online

Services for NFS model

The following sections provide information about the Microsoft Services for Network File System (NFS) model for
client-server communication. Since NFS v2 and NFS v3 are still the most widely deployed versions of the protocol,
all of the registry keys except for MaxConcurrentConnectionsPerIp apply to NFS v2 and NFS v3 only.
No registry tuning is required for NFS v4.1 protocol.
Service for NFS model overview
Microsoft Services for NFS provides a file-sharing solution for enterprises that have a mixed Windows and UNIX
environment. This communication model consists of client computers and a server. Applications on the client
request files that are located on the server through the redirector (Rdbss.sys) and NFS mini-redirector (Nfsrdr.sys).
The mini-redirector uses the NFS protocol to send its request through TCP/IP. The server receives multiple requests
from the clients through TCP/IP and routes the requests to the local file system (Ntfs.sys), which accesses the
storage stack.
The following figure shows the communication model for NFS.

Tuning parameters for NFS file servers

The following REG_DWORD registry settings can affect the performance of NFS file servers:
OptimalReads

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\OptimalReads

The default is 0. This parameter determines whether files are opened for FILE_RANDOM_ACCESS or for
FILE_SEQUENTIAL_ONLY, depending on the workload I/O characteristics. Set this value to 1 to force files to
be opened for FILE_RANDOM_ACCESS. FILE_RANDOM_ACCESS prevents the file system and cache manager
from prefetching.

NOTE
This setting must be carefully evaluated because it may have potential impact on system file cache grow.

RdWrHandleLifeTime
 HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\RdWrHandleLifeTime

The default is 5. This parameter controls the lifetime of an NFS cache entry in the file handle cache. The
parameter refers to cache entries that have an associated open NTFS file handle. Actual lifetime is
approximately equal to RdWrHandleLifeTime multiplied by RdWrThreadSleepTime. The minimum is 1 and
the maximum is 60.
RdWrNfsHandleLifeTime

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\RdWrNfsHandleLifeTime

The default is 5. This parameter controls the lifetime of an NFS cache entry in the file handle cache. The
parameter refers to cache entries that do not have an associated open NTFS file handle. Services for NFS
uses these cache entries to store file attributes for a file without keeping an open handle with the file system.
Actual lifetime is approximately equal to RdWrNfsHandleLifeTime multiplied by RdWrThreadSleepTime. The
minimum is 1 and the maximum is 60.
RdWrNfsReadHandlesLifeTime

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\RdWrNfsReadHandlesLifeTime

The default is 5. This parameter controls the lifetime of an NFS read cache entry in the file handle cache.
Actual lifetime is approximately equal to RdWrNfsReadHandlesLifeTime multiplied by
RdWrThreadSleepTime. The minimum is 1 and the maximum is 60.
RdWrThreadSleepTime

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\RdWrThreadSleepTime

The default is 5. This parameter controls the wait interval before running the cleanup thread on the file
handle cache. The value is in ticks, and it is non-deterministic. A tick is equivalent to approximately 100
nanoseconds. The minimum is 1 and the maximum is 60.
FileHandleCacheSizeinMB

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\FileHandleCacheSizeinMB

The default is 4. This parameter specifies the maximum memory to be consumed by file handle cache entries.
The minimum is 1 and the maximum is 1*1024*1024*1024 (1073741824).
LockFileHandleCacheInMemory

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\LockFileHandleCacheInMemory

The default is 0. This parameter specifies whether the physical pages that are allocated for the cache size
specified by FileHandleCacheSizeInMB are locked in memory. Setting this value to 1 enables this activity.
Pages are locked in memory (not paged to disk), which improves the performance of resolving file handles,
but reduces the memory that is available to applications.
MaxIcbNfsReadHandlesCacheSize
 HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\MaxIcbNfsReadHandlesCacheSize

The default is 64. This parameter specifies the maximum number of handles per volume for the read data
cache. Read cache entries are created only on systems that have more than 1 GB of memory. The minimum
is 0 and the maximum is 0xFFFFFFFF.
HandleSigningEnabled

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\HandleSigningEnabled

The default is 1. This parameter controls whether handles that are given out by NFS File Server are signed
cryptographically. Setting it to 0 disables handle signing.
RdWrNfsDeferredWritesFlushDelay

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\RdWrNfsDeferredWritesFlushDelay

The default is 60. This parameter is a soft timeout that controls the duration of NFS V3 UNSTABLE Write data
caching. The minimum is 1, and the maximum is 600. Actual lifetime is approximately equal to
RdWrNfsDeferredWritesFlushDelay multiplied by RdWrThreadSleepTime.
CacheAddFromCreateAndMkDir

HKLM\System\CurrentControlSet\Services\NfsServer\Parameters\CacheAddFromCreateAndMkDir

The default is 1 (enabled). This parameter controls whether handles that are opened during NFS V2 and V3
CREATE and MKDIR RPC procedure handlers are retained in the file handle cache. Set this value to 0 to
disable adding entries to the cache in CREATE and MKDIR code paths.
AdditionalDelayedWorkerThreads

HKLM\SYSTEM\CurrentControlSet\Control\SessionManager\Executive\AdditionalDelayedWorkerThreads

Increases the number of delayed worker threads that are created for the specified work queue. Delayed
worker threads process work items that are not considered time-critical and that can have their memory
stack paged out while waiting for work items. An insufficient number of threads reduces the rate at which
work items are serviced; a value that is too high consumes system resources unnecessarily.
NtfsDisable8dot3NameCreation

HKLM\System\CurrentControlSet\Control\FileSystem\NtfsDisable8dot3NameCreation

The default in Windows Server 2012 and Windows Server 2012 R2 is 2. In releases prior to Windows Server
2012, the default is 0. This parameter determines whether NTFS generates a short name in the 8dot3
(MSDOS) naming convention for long file names and for file names that contain characters from the
extended character set. If the value of this entry is 0, files can have two names: the name that the user
specifies and the short name that NTFS generates. If the user-specified name follows the 8dot3 naming
convention, NTFS does not generate a short name. A value of 2 means that this parameter can be configured
per volume.
 NOTE
The system volume has 8dot3 enabled by default. All other volumes in Windows Server 2012 and Windows Server
2012 R2 have 8dot3 disabled by default. Changing this value does not change the contents of a file, but it avoids the
short-name attribute creation for the file, which also changes how NTFS displays and manages the file. For most file
servers, the recommended setting is 1 (disabled).

NtfsDisableLastAccessUpdate

HKLM\System\CurrentControlSet\Control\FileSystem\NtfsDisableLastAccessUpdate

The default is 1. This system-global switch reduces disk I/O load and latencies by disabling the updating of
the date and time stamp for the last file or directory access.
MaxConcurrentConnectionsPerIp

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Rpcxdr\Parameters\MaxConcurrentConnectionsPerIp

The default value of the MaxConcurrentConnectionsPerIp parameter is 16. You can increase this value up to
a maximum of 8192 to increase the number of connections per IP address.
 Performance Tuning Hyper-V Servers
4/24/2017 1 min to read Edit Online

Hyper-V is the virtualization server role in Windows Server 2016. Virtualization servers can host multiple virtual
machines that are isolated from each other but share the underlying hardware resources by virtualizing the
processors, memory, and I/O devices. By consolidating servers onto a single machine, virtualization can improve
resource usage and energy efficiency and reduce the operational and maintenance costs of servers. In addition,
virtual machines and the management APIs offer more flexibility for managing resources, balancing load, and
provisioning systems.

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Terminology
4/24/2017 2 min to read Edit Online

This section summarizes key terminology specific to virtual machine technology that is used throughout this
performance tuning topic:

TERM DEFINITION

child partition Any virtual machine that is created by the root partition.

device virtualization A mechanism that lets a hardware resource be abstracted and

shared among multiple consumers.

emulated device A virtualized device that mimics an actual physical hardware

device so that guests can use the typical drivers for that
hardware device.

enlightenment An optimization to a guest operating system to make it aware

of virtual machine environments and tune its behavior for
virtual machines.

guest Software that is running in a partition. It can be a full-featured

operating system or a small, special-purpose kernel. The
hypervisor is guest-agnostic.

hypervisor A layer of software that sits above the hardware and below
one or more operating systems. Its primary job is to provide
isolated execution environments called partitions. Each
partition has its own set of virtualized hardware resources
(central processing unit or CPU, memory, and devices). The
hypervisor controls and arbitrates access to the underlying
hardware.

logical processor A processing unit that handles one thread of execution

(instruction stream). There can be one or more logical
processors per processor core and one or more cores per
processor socket.

passthrough disk access A representation of an entire physical disk as a virtual disk

within the guest. The data and commands are passed through
to the physical disk (through the root partitions native
storage stack) with no intervening processing by the virtual
stack.

root partition The root partition that is created first and owns all the
resources that the hypervisor does not, including most
devices and system memory. The root partition hosts the
virtualization stack and creates and manages the child
partitions.
 TERM DEFINITION

Hyper-V-specific device A virtualized device with no physical hardware analog, so

guests may need a driver (virtualization service client) to that
Hyper-V-specific device. The driver can use virtual machine
bus (VMBus) to communicate with the virtualized device
software in the root partition.

virtual machine A virtual computer that was created by software emulation

and has the same characteristics as a real computer.

virtual network switch (also referred to as a virtual switch) A virtual version of a

physical network switch. A virtual network can be configured
to provide access to local or external network resources for
one or more virtual machines.

virtual processor A virtual abstraction of a processor that is scheduled to run

on a logical processor. A virtual machine can have one or
more virtual processors.

virtualization service client (VSC) A software module that a guest loads to consume a resource
or service. For I/O devices, the virtualization service client can
be a device driver that the operating system kernel loads.

virtualization service provider (VSP) A provider exposed by the virtualization stack in the root
partition that provides resources or services such as I/O to a
child partition.

virtualization stack A collection of software components in the root partition that

work together to support virtual machines. The virtualization
stack works with and sits above the hypervisor. It also
provides management capabilities.

VMBus Channel-based communication mechanism used for inter-

partition communication and device enumeration on systems
with multiple active virtualized partitions. The VMBus is
installed with Hyper-V Integration Services.

See also
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Architecture
4/24/2017 1 min to read Edit Online

Hyper-V features a Type 1 hypervisor-based architecture. The hypervisor virtualizes processors and memory and
provides mechanisms for the virtualization stack in the root partition to manage child partitions (virtual machines)
and expose services such as I/O devices to the virtual machines.
The root partition owns and has direct access to the physical I/O devices. The virtualization stack in the root
partition provides a memory manager for virtual machines, management APIs, and virtualized I/O devices. It also
implements emulated devices such as the integrated device electronics (IDE) disk controller and PS/2 input device
port, and it supports Hyper-V-specific synthetic devices for increased performance and reduced overhead.

The Hyper-V-specific I/O architecture consists of virtualization service providers (VSPs) in the root partition and
virtualization service clients (VSCs) in the child partition. Each service is exposed as a device over VMBus, which
acts as an I/O bus and enables high-performance communication between virtual machines that use mechanisms
such as shared memory. The guest operating systems Plug and Play manager enumerates these devices,
including VMBus, and loads the appropriate device drivers (virtual service clients). Services other than I/O are also
exposed through this architecture.
Starting with Windows Server 2008, the operating system features enlightenments to optimize its behavior when
it is running in virtual machines. The benefits include reducing the cost of memory virtualization, improving
multicore scalability, and decreasing the background CPU usage of the guest operating system.
The following sections suggest best practices that yield increased performance on servers running Hyper-V role.

See also
Hyper-V terminology
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Configuration
4/24/2017 5 min to read Edit Online

Hardware selection
The hardware considerations for servers running Hyper-V generally resemble those of non-virtualized servers,
but servers running Hyper-V can exhibit increased CPU usage, consume more memory, and need larger I/O
bandwidth because of server consolidation.
Processors
Hyper-V in Windows Server 2016 presents the logical processors as one or more virtual processors to
each active virtual machine. Hyper-V now requires processors that support Second Level Address
Translation (SLAT) technologies such as Extended Page Tables (EPT) or Nested Page Tables (NPT).
Cache
Hyper-V can benefit from larger processor caches, especially for loads that have a large working set in
memory and in virtual machine configurations in which the ratio of virtual processors to logical processors
is high.
Memory
The physical server requires sufficient memory for the both the root and child partitions. The root partition
requires memory to efficiently perform I/Os on behalf of the virtual machines and operations such as a
virtual machine snapshot. Hyper-V ensures that sufficient memory is available to the root partition, and
allows remaining memory to be assigned to child partitions. Child partitions should be sized based on the
needs of the expected load for each virtual machine.
Storage
The storage hardware should have sufficient I/O bandwidth and capacity to meet the current and future
needs of the virtual machines that the physical server hosts. Consider these requirements when you select
storage controllers and disks and choose the RAID configuration. Placing virtual machines with highly disk-
intensive workloads on different physical disks will likely improve overall performance. For example, if four
virtual machines share a single disk and actively use it, each virtual machine can yield only 25 percent of
the bandwidth of that disk.

Power plan considerations

As a core technology, virtualization is a powerful tool useful in increasing server workload density, reducing the
number of required physical servers in your datacenter, increasing operational efficiency and reducing power
consumption costs. Power management is critical for cost management.
In an ideal datacenter environment, power consumption is managed by consolidating work onto machines until
theyre mostly busy and then turning off idle machines. If this approach is not practical, administrators can
leverage power plans on the physical hosts to ensure they do not consume more power than necessary.
Server power management techniques come with a cost, particularly as tenant workloads are not trusted to
dictate policy about the hosters physical infrastructure. The host layer software is left to infer how to maximize
throughput while minimizing power consumption. In mostly-idle machines, this can cause the physical
infrastructure to conclude that moderate power draw is appropriate, resulting in individual tenant workloads
running more slowly than they might otherwise.
Windows Server uses virtualization in a wide variety of scenarios. From a lightly loaded IIS Server to a moderately
busy SQL Server, to a cloud host with Hyper-V running hundreds of virtual machines per server. Each of these
scenarios may have unique hardware, software, and performance requirements. By default, Windows Server uses
and recommends the Balanced power plan which enables power conservation by scaling the processor
performance based on CPU utilization.
With the Balanced power plan, the highest power states (and lowest response latencies in tenant workloads) are
applied only when the physical host is relatively busy. If you value deterministic, low-latency response for all
tenant workloads, you should consider switching from the default Balanced power plan to the High
Performance power plan. The High Performance power plan will run the processors at full speed all the time,
effectively disabling Demand-Based Switching along with other power management techniques, and optimize for
performance over power savings.
For customers, who are satisfied with the cost savings from reducing the number of physical servers and want to
ensure they achieve maximum performance for their virtualized workloads, you should consider using the High
Performance power plan.
For additional recommendations and insight on leveraging power plans to optimize your infrastructure, read
Recommended Balanced Power Plan Parameters for Quick Response Times

Server Core installation option

Windows Server 2016 feature the Server Core installation option. Server Core offers a minimal environment for
hosting a select set of server roles including Hyper-V. It features a smaller disk footprint for the host OS, and a
smaller attack and servicing surface. Therefore, we highly recommend that Hyper-V virtualization servers use the
Server Core installation option.
A Server Core installation offers a console window only when the user is logged on, but Hyper-V exposes remote
management features including Windows Powershell so administrators can manage it remotely.

Dedicated server role

The root partition should be dedicated to Hyper-V. Running additional server roles on a server running Hyper-V
can adversely affect the performance of the virtualization server, especially if they consume significant CPU,
memory, or I/O bandwidth. Minimizing the server roles in the root partition has additional benefits such as
reducing the attack surface.
System administrators should consider carefully what software is installed in the root partition because some
software can adversely affect the overall performance of the server running Hyper-V.

Guest operating systems

Hyper-V supports and has been tuned for a number of different guest operating systems. The number of virtual
processors that are supported per guest depends on the guest operating system. For a list of the supported guest
operating systems, see Hyper-V Overview.

CPU statistics
Hyper-V publishes performance counters to help characterize the behavior of the virtualization server and report
the resource usage. The standard set of tools for viewing performance counters in Windows includes
Performance Monitor and Logman.exe, which can display and log the Hyper-V performance counters. The names
of the relevant counter objects are prefixed with Hyper-V.
You should always measure the CPU usage of the physical system by using the Hyper-V Hypervisor Logical
Processor performance counters. The CPU utilization counters that Task Manager and Performance Monitor
report in the root and child partitions do not reflect the actual physical CPU usage. Use the following performance
counters to monitor performance:
Hyper-V Hypervisor Logical Processor (*)\% Total Run Time The total non-idle time of the logical
processors
Hyper-V Hypervisor Logical Processor (*)\% Guest Run Time The time spent running cycles within a
guest or within the host
Hyper-V Hypervisor Logical Processor (*)\% Hypervisor Run Time The time spent running within the
hypervisor
Hyper-V Hypervisor Root Virtual Processor (*)\\* Measures the CPU usage of the root partition
Hyper-V Hypervisor Virtual Processor (*)\\* Measures the CPU usage of guest partitions

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Processor Performance
4/24/2017 3 min to read Edit Online

Virtual machine integration services

The Virtual Machine Integration Services include enlightened drivers for the Hyper-V-specific I/O devices, which
significantly reduces CPU overhead for I/O compared to emulated devices. You should install the latest version of
the Virtual Machine Integration Services in every supported virtual machine. The services decrease the CPU usage
of the guests, from idle guests to heavily used guests, and improves the I/O throughput. This is the first step in
tuning performance in a server running Hyper-V. For a list of supported guest operating systems, see Hyper-V
Overview.

Virtual processors
Hyper-V in Windows Server 2016 supports a maximum of 240 virtual processors per virtual machine. Virtual
machines that have loads that are not CPU intensive should be configured to use one virtual processor. This is
because of the additional overhead that is associated with multiple virtual processors, such as additional
synchronization costs in the guest operating system.
Increase the number of virtual processors if the virtual machine requires more than one CPU of processing under
peak load.

Background activity
Minimizing the background activity in idle virtual machines releases CPU cycles that can be used elsewhere by
other virtual machines. Windows guests typically use less than one percent of one CPU when they are idle. The
following are several best practices for minimizing the background CPU usage of a virtual machine:
Install the latest version of the Virtual Machine Integration Services.
Remove the emulated network adapter through the virtual machine settings dialog box (use the Microsoft
Hyper-V-specific adapter).
Remove unused devices such as the CD-ROM and COM port, or disconnect their media.
Keep the Windows guest operating system on the sign-in screen when it is not being used and disable the
screen saver.
Review the scheduled tasks and services that are enabled by default.
Review the ETW trace providers that are on by default by running logman.exe query -ets
Improve server applications to reduce periodic activity (such as timers).
Close Server Manager on both the host and guest operating systems.
Dont leave Hyper-V Manager running since it constantly refreshes the virtual machines thumbnail.
The following are additional best practices for configuring a client version of Windows in a virtual machine to
reduce the overall CPU usage:
Disable background services such as SuperFetch and Windows Search.
Disable scheduled tasks such as Scheduled Defrag.
Virtual NUMA
To enable virtualizing large scale-up workloads, Hyper-V in Windows Server 2016 expanded virtual machine scale
limits. A single virtual machine can be assigned up to 240 virtual processors and 12 TB of memory. When creating
such large virtual machines, memory from multiple NUMA nodes on the host system will likely be utilized. In such
virtual machine configuration, if virtual processors and memory are not allocated from the same NUMA node,
workloads may have bad performance due to the inability to take advantage of NUMA optimizations.
In Windows Server 2016, Hyper-V presents a virtual NUMA topology to virtual machines. By default, this virtual
NUMA topology is optimized to match the NUMA topology of the underlying host computer. Exposing a virtual
NUMA topology into a virtual machine allows the guest operating system and any NUMA-aware applications
running within it to take advantage of the NUMA performance optimizations, just as they would when running on
a physical computer.
There is no distinction between a virtual and a physical NUMA from the workloads perspective. Inside a virtual
machine, when a workload allocates local memory for data, and accesses that data in the same NUMA node, fast
local memory access results on the underlying physical system. Performance penalties due to remote memory
access are successfully avoided. Only NUMA-aware applications can benefit of vNUMA.
Microsoft SQL Server is an example of NUMA aware application. For more info, see Understanding Non-uniform
Memory Access.
Virtual NUMA and Dynamic Memory features cannot be used at the same time. A virtual machine that has
Dynamic Memory enabled effectively has only one virtual NUMA node, and no NUMA topology is presented to
the virtual machine regardless of the virtual NUMA settings.
For more info on Virtual NUMA, see Hyper-V Virtual NUMA Overview.

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Memory Performance
4/24/2017 2 min to read Edit Online

The hypervisor virtualizes the guest physical memory to isolate virtual machines from each other and to provide a
contiguous, zero-based memory space for each guest operating system, just as on non-virtualized systems.

Correct memory sizing for child partitions

You should size virtual machine memory as you typically do for server applications on a physical computer. You
must size it to reasonably handle the expected load at ordinary and peak times because insufficient memory can
significantly increase response times and CPU or I/O usage.
You can enable Dynamic Memory to allow Windows to size virtual machine memory dynamically. With Dynamic
Memory, if applications in the virtual machine experience problems making large sudden memory allocations,
you can increase the page file size for the virtual machine to ensure temporary backing while Dynamic Memory
responds to the memory pressure.
For more info on Dynamic Memory, see Hyper-V Dynamic Memory Overview and Hyper-V Dynamic Memory
Configuration Guide.
When running Windows in the child partition, you can use the following performance counters within a child
partition to identify whether the child partition is experiencing memory pressure and is likely to perform better
with a higher virtual machine memory size.

PERFORMANCE COUNTER SUGGESTED THRESHOLD VALUE

Memory Standby Cache Reserve Bytes Sum of Standby Cache Reserve Bytes and Free and Zero Page
List Bytes should be 200 MB or more on systems with 1 GB,
and 300 MB or more on systems with 2 GB or more of visible
RAM.

Memory Free & Zero Page List Bytes Sum of Standby Cache Reserve Bytes and Free and Zero Page
List Bytes should be 200 MB or more on systems with 1 GB,
and 300 MB or more on systems with 2 GB or more of visible
RAM.

Memory Pages Input/Sec Average over a 1-hour period is less than 10.

Correct memory sizing for root partition

The root partition must have sufficient memory to provide services such as I/O virtualization, virtual machine
snapshot, and management to support the child partitions.
Hyper-V in Windows Server 2016 monitors the runtime health of the root partitions management operating
system to determine how much memory can safely be allocated to child partitions, while still ensuring high
performance and reliability of the root partition.

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Storage I/O Performance
4/24/2017 17 min to read Edit Online

This section describes the different options and considerations for tuning storage I/O performance in a virtual
machine. The storage I/O path extends from the guest storage stack, through the host virtualization layer, to the
host storage stack, and then to the physical disk. Following are explanations about how optimizations are possible
at each of these stages.

Virtual controllers
Hyper-V offers three types of virtual controllers: IDE, SCSI, and Virtual host bus adapters (HBAs).

IDE
IDE controllers expose IDE disks to the virtual machine. The IDE controller is emulated, and it is the only controller
that is available for guest VMs running older version of Windows without the Virtual Machine Integration
Services. Disk I/O that is performed by using the IDE filter driver that is provided with the Virtual Machine
Integration Services is significantly better than the disk I/O performance that is provided with the emulated IDE
controller. We recommend that IDE disks be used only for the operating system disks because they have
performance limitations due to the maximum I/O size that can be issued to these devices.

SCSI (SAS controller)

SCSI controllers expose SCSI disks to the virtual machine, and each virtual SCSI controller can support up to 64
devices. For optimal performance, we recommend that you attach multiple disks to a single virtual SCSI controller
and create additional controllers only as they are required to scale the number of disks connected to the virtual
machine. SCSI path is not emulated which makes it the preferred controller for any disk other than the operating
system disk. In fact with Generation 2 VMs, it is the only type of controller possible. Introduced in Windows Server
2012 R2, this controller is reported as SAS to support shared VHDX.

Virtual Fibre Channel HBAs

Virtual Fibre Channel HBAs can be configured to allow direct access for virtual machines to Fibre Channel and
Fibre Channel over Ethernet (FCoE) LUNs. Virtual Fibre Channel disks bypass the NTFS file system in the root
partition, which reduces the CPU usage of storage I/O.
Large data drives and drives that are shared between multiple virtual machines (for guest clustering scenarios)
are prime candidates for virtual Fibre Channel disks.
Virtual Fibre Channel disks require one or more Fibre Channel host bus adapters (HBAs) to be installed on the
host. Each host HBA is required to use an HBA driver that supports the Windows Server 2016 Virtual Fibre
Channel/NPIV capabilities. The SAN fabric should support NPIV, and the HBA port(s) that are used for the virtual
Fibre Channel should be set up in a Fibre Channel topology that supports NPIV.
To maximize throughput on hosts that are installed with more than one HBA, we recommend that you configure
multiple virtual HBAs inside the Hyper-V virtual machine (up to four HBAs can be configured for each virtual
machine). Hyper-V will automatically make a best effort to balance virtual HBAs to host HBAs that access the
same virtual SAN.

Virtual disks
Disks can be exposed to the virtual machines through the virtual controllers. These disks could be virtual hard
disks that are file abstractions of a disk or a pass-through disk on the host.

Virtual hard disks

There are two virtual hard disk formats, VHD and VHDX. Each of these formats supports three types of virtual
hard disk files.

VHD format
The VHD format was the only virtual hard disk format that was supported by Hyper-V in past releases. Introduced
in Windows Server 2012, the VHD format has been modified to allow better alignment, which results in
significantly better performance on new large sector disks.
Any new VHD that is created on a Windows Server 2012 or newer has the optimal 4 KB alignment. This aligned
format is completely compatible with previous Windows Server operating systems. However, the alignment
property will be broken for new allocations from parsers that are not 4 KB alignment-aware (such as a VHD
parser from a previous version of Windows Server or a non-Microsoft parser).
Any VHD that is moved from a previous release does not automatically get converted to this new improved VHD
format.
To convert to new VHD format, run the following Windows PowerShell command:

Convert-VHD Path E:\vms\testvhd\test.vhd DestinationPath E:\vms\testvhd\test-converted.vhd

You can check the alignment property for all the VHDs on the system, and it should be converted to the optimal 4
KB alignment. You create a new VHD with the data from the original VHD by using the Create-from-Source
option.
To check for alignment by using Windows Powershell, examine the Alignment line, as shown below:

Get-VHD Path E:\vms\testvhd\test.vhd

Path : E:\vms\testvhd\test.vhd
VhdFormat : VHD
VhdType : Dynamic
FileSize : 69245440
Size : 10737418240
MinimumSize : 10735321088
LogicalSectorSize : 512
PhysicalSectorSize : 512
BlockSize : 2097152
ParentPath :
FragmentationPercentage : 10
Alignment : 0
Attached : False
DiskNumber :
IsDeleted : False
Number :

To verify alignment by using Windows PowerShell, examine the Alignment line, as shown below:
 Get-VHD Path E:\vms\testvhd\test-converted.vhd

Path : E:\vms\testvhd\test-converted.vhd
VhdFormat : VHD
VhdType : Dynamic
FileSize : 69369856
Size : 10737418240
MinimumSize : 10735321088
LogicalSectorSize : 512
PhysicalSectorSize : 512
BlockSize : 2097152
ParentPath :
FragmentationPercentage : 0
Alignment : 1
Attached : False
DiskNumber :
IsDeleted : False
Number :

VHDX format
VHDX is a new virtual hard disk format introduced in Windows Server 2012, which allows you to create resilient
high-performance virtual disks up to 64 terabytes. Benefits of this format include:
Support for virtual hard disk storage capacity of up to 64 terabytes.
Protection against data corruption during power failures by logging updates to the VHDX metadata
structures.
Ability to store custom metadata about a file, which a user might want to record, such as operating system
version or patches applied.
The VHDX format also provides the following performance benefits:
Improved alignment of the virtual hard disk format to work well on large sector disks.
Larger block sizes for dynamic and differential disks, which allows these disks to attune to the needs of the
workload.
4 KB logical sector virtual disk that allows for increased performance when used by applications and
workloads that are designed for 4 KB sectors.
Efficiency in representing data, which results in smaller file size and allows the underlying physical storage
device to reclaim unused space. (Trim requires pass-through or SCSI disks and trim-compatible hardware.)
When you upgrade to Windows Server 2016, we recommend that you convert all VHD files to the VHDX format
due to these benefits. The only scenario where it would make sense to keep the files in the VHD format is when a
virtual machine has the potential to be moved to a previous release of Hyper-V that does not support the VHDX
format.

Types of virtual hard disk files

There are three types of VHD files. The following sections are the performance characteristics and trade-offs
between the types.
The following recommendations should be taken into consideration with regards to selecting a VHD file type:
When using the VHD format, we recommend that you use the fixed type because it has better resiliency
and performance characteristics compared to the other VHD file types.
 When using the VHDX format, we recommend that you use the dynamic type because it offers resiliency
guarantees in addition to space savings that are associated with allocating space only when there is a need
to do so.
The fixed type is also recommended, irrespective of the format, when the storage on the hosting volume is
not actively monitored to ensure that sufficient disk space is present when expanding the VHD file at run
time.
Snapshots of a virtual machine create a differencing VHD to store writes to the disks. Having only a few
snapshots can elevate the CPU usage of storage I/Os, but might not noticeably affect performance except
in highly I/O-intensive server workloads. However, having a large chain of snapshots can noticeably affect
performance because reading from the VHD can require checking for the requested blocks in many
differencing VHDs. Keeping snapshot chains short is important for maintaining good disk I/O performance.

Fixed virtual hard disk type

Space for the VHD is first allocated when the VHD file is created. This type of VHD file is less likely to fragment,
which reduces the I/O throughput when a single I/O is split into multiple I/Os. It has the lowest CPU overhead of
the three VHD file types because reads and writes do not need to look up the mapping of the block.

Dynamic virtual hard disk type

Space for the VHD is allocated on demand. The blocks in the disk start as unallocated blocks and are not backed
by any actual space in the file. When a block is first written to, the virtualization stack must allocate space within
the VHD file for the block, and then update the metadata. This increases the number of necessary disk I/Os for the
Write and increases CPU usage. Reads and writes to existing blocks incur disk access and CPU overhead when
looking up the blocks mapping in the metadata.

Differencing virtual hard disk type

The VHD points to a parent VHD file. Any writes to blocks not written to result in space being allocated in the VHD
file, as with a dynamically expanding VHD. Reads are serviced from the VHD file if the block has been written to.
Otherwise, they are serviced from the parent VHD file. In both cases, the metadata is read to determine the
mapping of the block. Reads and Writes to this VHD can consume more CPU and result in more I/Os than a fixed
VHD file.

Block size considerations

Block size can significantly impact performance. It is optimal to match the block size to the allocation patterns of
the workload that is using the disk. For example, if an application is allocating in chunks of 16 MB, it would be
optimal to have a virtual hard disk block size of 16 MB. A block size of >2 MB is possible only on virtual hard
disks with the VHDX format. Having a larger block size than the allocation pattern for a random I/O workload will
significantly increase the space usage on the host.

Sector size implications

Most of the software industry has depended on disk sectors of 512 bytes, but the standard is moving to 4 KB disk
sectors. To reduce compatibility issues that might arise from a change in sector size, hard drive vendors are
introducing a transitional size referred to as 512 emulation drives (512e).
These emulation drives offer some of the advantages that are offered by 4 KB disk sector native drives, such as
improved format efficiency and an improved scheme for error correction codes (ECC). They come with fewer
compatibility issues that would occur by exposing a 4 KB sector size at the disk interface.
Support for 512e disks
A 512e disk can perform a write only in terms of a physical sectorthat is, it cannot directly write a 512byte
sector that is issued to it. The internal process in the disk that makes these writes possible follows these steps:
The disk reads the 4 KB physical sector to its internal cache, which contains the 512-byte logical sector
referred to in the write.
Data in the 4 KB buffer is modified to include the updated 512-byte sector.
The disk performs a write of the updated 4 KB buffer back to its physical sector on the disk.
This process is called read-modify-write (RMW). The overall performance impact of the RMW process depends on
the workloads. The RMW process causes performance degradation in virtual hard disks for the following reasons:
Dynamic and differencing virtual hard disks have a 512-byte sector bitmap in front of their data payload. In
addition, footer, header, and parent locators align to a 512-byte sector. It is common for the virtual hard
disk driver to issue 512-byte write commands to update these structures, resulting in the RMW process
described earlier.
Applications commonly issue reads and writes in multiples of 4 KB sizes (the default cluster size of NTFS).
Because there is a 512-byte sector bitmap in front of the data payload block of dynamic and differencing
virtual hard disks, the 4 KB blocks are not aligned to the physical 4 KB boundary. The following figure
shows a VHD 4 KB block (highlighted) that is not aligned with physical 4 KB boundary.

Each 4 KB write command that is issued by the current parser to update the payload data results in two reads for
two blocks on the disk, which are then updated and subsequently written back to the two disk blocks. Hyper-V in
Windows Server 2016 mitigates some of the performance effects on 512e disks on the VHD stack by preparing
the previously mentioned structures for alignment to 4 KB boundaries in the VHD format. This avoids the RMW
effect when accessing the data within the virtual hard disk file and when updating the virtual hard disk metadata
structures.
As mentioned earlier, VHDs that are copied from previous versions of Windows Server will not automatically be
aligned to 4 KB. You can manually convert them to optimally align by using the Copy from Source disk option
that is available in the VHD interfaces.
By default, VHDs are exposed with a physical sector size of 512 bytes. This is done to ensure that physical sector
size dependent applications are not impacted when the application and VHDs are moved from a previous version
of Windows Server.
By default, disks with the VHDX format are created with the 4 KB physical sector size to optimize their
performance profile regular disks and large sector disks. To make full use of 4 KB sectors its recommended to
use VHDX format.

Support for native 4 KB disks

Hyper-V in Windows Server 2012 R2 and beyond supports 4 KB native disks. But it is still possible to store VHD
disk on 4 KB native disk. This is done by implementing a software RMW algorithm in the virtual storage stack
layer that converts 512-byte access and update requests to corresponding 4 KB accesses and updates.
Because VHD file can only expose themselves as 512-byte logical sector size disks, it is very likely that there will
be applications that issue 512-byte I/O requests. In these cases, the RMW layer will satisfy these requests and
cause performance degradation. This is also true for a disk that is formatted with VHDX that has a logical sector
size of 512 bytes.
It is possible to configure a VHDX file to be exposed as a 4 KB logical sector size disk, and this would be an
optimal configuration for performance when the disk is hosted on a 4 KB native physical device. Care should be
taken to ensure that the guest and the application that is using the virtual disk are backed by the 4 KB logical
sector size. The VHDX formatting will work correctly on a 4 KB logical sector size device.

Pass-through disks
The VHD in a virtual machine can be mapped directly to a physical disk or logical unit number (LUN), instead of to
a VHD file. The benefit is that this configuration bypasses the NTFS file system in the root partition, which reduces
the CPU usage of storage I/O. The risk is that physical disks or LUNs can be more difficult to move between
machines than VHD files.
Pass-through disks should be avoided due to the limitations introduced with virtual machine migration scenarios.

Advanced storage features

Storage Quality of Service (QoS )
Starting in Windows Server 2012 R2, Hyper-V includes the ability to set certain quality-of-service (QoS)
parameters for storage on the virtual machines. Storage QoS provides storage performance isolation in a
multitenant environment and mechanisms to notify you when the storage I/O performance does not meet the
defined threshold to efficiently run your virtual machine workloads.
Storage QoS provides the ability to specify a maximum input/output operations per second (IOPS) value for your
virtual hard disk. An administrator can throttle the storage I/O to stop a tenant from consuming excessive storage
resources that may impact another tenant.
You can also set a minimum IOPS value. They will be notified when the IOPS to a specified virtual hard disk is
below a threshold that is needed for its optimal performance.
The virtual machine metrics infrastructure is also updated, with storage related parameters to allow the
administrator to monitor the performance and chargeback related parameters.
Maximum and minimum values are specified in terms of normalized IOPS where every 8 KB of data is counted as
an I/O.
Some of the limitations are as follows:
Only for virtual disks
Differencing disk cannot have parent virtual disk on a different volume
Replica - QoS for replica site configured separately from primary site
Shared VHDX is not supported
For more info on Storage Quality of Service, see Storage Quality of Service for Hyper-V.
NUMA I/O
Windows Server 2012 and beyond supports large virtual machines, and any large virtual machine configuration
(for example, a configuration with Microsoft SQL Server running with 64 virtual processors) will also need
scalability in terms of I/O throughput.
The following key improvements first introduced in the Windows Server 2012 storage stack and Hyper-V provide
the I/O scalability needs of large virtual machines:
 An increase in the number of communication channels created between the guest devices and host storage
stack.
A more efficient I/O completion mechanism involving interrupt distribution amongst the virtual processors
to avoid expensive interprocessor interruptions.
Introduced in Windows Server 2012, there are a few registry entries, located at
HKLM\System\CurrentControlSet\Enum\VMBUS\{device id}\{instance id}\StorChannel, that allow the number of
channels to be adjusted. They also align the virtual processors that handle the I/O completions to the virtual CPUs
that are assigned by the application to be the I/O processors. The registry settings are configured on a per-
adapter basis on the devices hardware key.
ChannelCount (DWORD) The total number of channels to use, with a maximum of 16. It defaults to a
ceiling, which is the number of virtual processors/16.
ChannelMask (QWORD) The processor affinity for the channels. If it is not set or is set to 0, it defaults to
the existing channel distribution algorithm that you use for normal storage or for networking channels.
This ensures that your storage channels wont conflict with your network channels.
Offloaded Data Transfer integration
Crucial maintenance tasks for VHDs, such as merge, move, and compact, depend copying large amounts of data.
The current method of copying data requires data to be read in and written to different locations, which can be a
time-consuming process. It also uses CPU and memory resources on the host, which could have been used to
service virtual machines.
Storage area network (SAN) vendors are working to provide near-instantaneous copy operations of large
amounts of data. This storage is designed to allow the system above the disks to specify the move of a specific
data set from one location to another. This hardware feature is known as an Offloaded Data Transfer.
Hyper-V in Windows Server 2012 and beyond supports Offload Data Transfer (ODX) operations so that these
operations can be passed from the guest operating system to the host hardware. This ensures that the workload
can use ODX-enabled storage as it would if it were running in a non-virtualized environment. The Hyper-V
storage stack also issues ODX operations during maintenance operations for VHDs such as merging disks and
storage migration meta-operations where large amounts of data are moved.
Unmap integration
Virtual hard disk files exist as files on a storage volume, and they share available space with other files. Because
the size of these files tends to be large, the space that they consume can grow quickly. Demand for more physical
storage affects the IT hardware budget. Its important to optimize the use of physical storage as much as possible.
Before Windows Server 2012, when applications delete content within a virtual hard disk, which effectively
abandoned the contents storage space, the Windows storage stack in the guest operating system and the Hyper-
V host had limitations that prevented this information from being communicated to the virtual hard disk and the
physical storage device. This prevented the Hyper-V storage stack from optimizing the space usage by the VHD-
based virtual disk files. It also prevented the underlying storage device from reclaiming the space that was
previously occupied by the deleted data.
Starting from Windows Server 2012, Hyper-V supports unmap notifications, which allow VHDX files to be more
efficient in representing that data within it. This results in smaller files size, and it allows the underlying physical
storage device to reclaim unused space.
Only Hyper-V-specific SCSI, enlightened IDE, and Virtual Fibre Channel controllers allow the unmap command
from the guest to reach the host virtual storage stack. On the virtual hard disks, only virtual disks formatted as
VHDX support unmap commands from the guest.
For these reasons, we recommend that you use VHDX files attached to a SCSI controller when not using Virtual
Fibre Channel disks.
See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Hyper-V Network I/O Performance
4/24/2017 1 min to read Edit Online

Server 2016 contains several improvements and new functionality to optimize network performance under
Hyper-V. Documentation on how to optimize network performance will be included in a future version of this
article.

Live Migration
Live Migration lets you to transparently move running virtual machines from one node of a failover cluster to
another node in the same cluster without a dropped network connection or perceived downtime.
Note
Failover Clustering requires shared storage for the cluster nodes.
The process of moving a running virtual machine can be divided into two major phases. The first phase copies the
memory of the virtual machine from the current host to the new host. The second phase transfers the virtual
machine state from the current host to the new host. The durations of both phases is greatly determined by the
speed at which data can be transferred from the current host to the new host.
Providing a dedicated network for live migration traffic helps minimize the time that is required to complete a live
migration, and it ensures consistent migration times.

Additionally, increasing the number of send and receive buffers on each network adapter that is involved in the
migration can improve migration performance.
Windows Server 2012 R2 introduced an option to speed up Live Migration by compressing memory before
transferring over the network or use Remote Direct Memory Access (RDMA), if your hardware supports it.

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
 Detecting bottlenecks in a virtualized environment
4/24/2017 3 min to read Edit Online

This section should give you some hints on what to monitor by using Performance Monitor and how to identify
where the problem might be when either the host or some of the virtual machines do not perform as you would
have expected.

Processor bottlenecks
Here are some common scenarios that could cause processor bottlenecks:
One or more logical processors are loaded
One or more virtual processors are loaded
You can use the following performance counters from the host:
Logical Processor Utilization - \Hyper-V Hypervisor Logical Processor(*)\% Total Run Time
Virtual Processor Utilization - \Hyper-V Hypervisor Virtual Processor(*)\% Total Run Time
Root Virtual Processor Utilization - \Hyper-V Hypervisor Root Virtual Processor(*)\% Total Run Time
If the Hyper-V Hypervisor Logical Processor(_Total)\% Total Runtime counter is over 90%, the host is
overloaded. You should add more processing power or move some virtual machines to a different host.
If the Hyper-V Hypervisor Virtual Processor(VM Name:VP x)\% Total Runtime counter is over 90% for all
virtual processors, you should do the following:
Verify that the host is not overloaded
Find out if the workload can leverage more virtual processors
Assign more virtual processors to the virtual machine
If Hyper-V Hypervisor Virtual Processor(VM Name:VP x)\% Total Runtime counter is over 90% for some,
but not all, of the virtual processors, you should do the following:
If your workload is receive network-intensive, you should consider using vRSS.
If the virtual machines are not running Windows Server 2012 R2, you should add more network adapters.
If your workload is storage-intensive, you should enable virtual NUMA and add more virtual disks.
If the Hyper-V Hypervisor Root Virtual Processor (Root VP x)\% Total Runtime counter is over 90% for
some, but not all, virtual processors and the Processor (x)\% Interrupt Time and Processor (x)\% DPC Time
counter approximately adds up to the value for the Root Virtual Processor(Root VP x)\% Total Runtime
counter, you should ensure enable VMQ on the network adapters.

Memory bottlenecks
Here are some common scenarios that could cause memory bottlenecks:
The host is not responsive.
Virtual machines cannot be started.
 Virtual machines run out of memory.
You can use the following performance counters from the host:
Memory\Available Mbytes
Hyper-V Dynamic Memory Balancer (*)\Available Memory
You can use the following performance counters from the virtual machine:
Memory\Available Mbytes
If the Memory\Available Mbytes and Hyper-V Dynamic Memory Balancer (*)\Available Memory counters
are low on the host, you should stop non-essential services and migrate one or more virtual machines to another
host.
If the Memory\Available Mbytes counter is low in the virtual machine, you should assign more memory to the
virtual machine. If you are using Dynamic Memory, you should increase the maximum memory setting.

Network bottlenecks
Here are some common scenarios that could cause network bottlenecks:
The host is network bound.
The virtual machine is network bound.
You can use the following performance counters from the host:
Network Interface(network adapter name)\Bytes/sec
You can use the following performance counters from the virtual machine:
Hyper-V Virtual Network Adapter (virtual machine name name<GUID>)\Bytes/sec
If the Physical NIC Bytes/sec counter is greater than or equal to 90% of capacity, you should add additional
network adapters, migrate virtual machines to another host, and configure Network QoS.
If the Hyper-V Virtual Network Adapter Bytes/sec counter is greater than or equal to 250 MBps, you should
add additional teamed network adapters in the virtual machine, enable vRSS, and use SR-IOV.
If your workloads cant meet their network latency, enable SR-IOV to present physical network adapter resources
to the virtual machine.

Storage bottlenecks
Here are some common scenarios that could cause storage bottlenecks:
The host and virtual machine operations are slow or time out.
The virtual machine is sluggish.
You can use the following performance counters from the host:
Physical Disk(disk letter)\Avg. disk sec/Read
Physical Disk(disk letter)\Avg. disk sec/Write
Physical Disk(disk letter)\Avg. disk read queue length
Physical Disk(disk letter)\Avg. disk write queue length
If latencies are consistently greater than 50ms, you should do the following:
Spread virtual machines across additional storage
Consider purchasing faster storage
Consider Tiered Storage Spaces, which was introduced in Windows Server 2012 R2
Consider using Storage QoS, which was introduced in Windows Server 2012 R2
Use VHDX

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Linux Virtual Machines
 Linux Virtual Machine Considerations
4/24/2017 2 min to read Edit Online

Linux and BSD virtual machines have additional considerations compared to Windows virtual machines in Hyper-
V.
The first consideration is whether Integration Services are present or if the VM is running merely on emulated
hardware with no enlightenment. A table of Linux and BSD releases that have built-in or downloadable
Integration Services is available in Supported Linux and FreeBSD virtual machines for Hyper-V on Windows.
These pages have grids of available Hyper-V features available to Linux distribution releases, and notes on those
features where applicable.
Even when the guest is running Integration Services, it can be configured with legacy hardware which does not
exhibit the best performance. For example, configure and use a virtual ethernet adapter for the guest instead of
using a legacy network adapter. With Windows Server 2016, advanced networking like SR-IOV are available as
well.

Linux Network Performance

Linux by default enables hardware acceleration and offloads by default. If vRSS is enabled in the properties of a
NIC on the host and the Linux guest has the capability to use vRSS the capability will be enabled. In Powershell
this same parameter can be changed with the EnableNetAdapterRSS command.
Similarly, the VMMQ (Virtual Switch RSS) feature can be enabled on the physical NIC used by the guest
Properties > Configure... > Advanced tab > set Virtual Switch RSS to Enabled or enable VMMQ in
Powershell using the following:

Set-VMNetworkAdapter -VMName **$VMName** -VmmqEnabled $True

In the guest additional TCP tuning can be performed by increasing limits. For the best performance spreading
workload over multiple CPUs and having deep workloads produces the best throughput, as virtualized workloads
will have higher latency than "bare metal" ones.
Some example tuning paramters that have been useful in network benchmarks include:

net.core.netdev_max_backlog = 30000
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.ipv4.tcp_wmem = 4096 12582912 33554432
net.ipv4.tcp_rmem = 4096 12582912 33554432
net.ipv4.tcp_max_syn_backlog = 80960
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_abort_on_overflow = 1

A useful tool for network microbenchmarks is ntttcp, which is available on both Linux and Windows. The Linux
version is open source and available from ntttcp-for-linux on github.com. The Windows version can be found in
the download center. When tuning workloads it is best to use as many streams as necessary to get the best
throughput. Using ntttcp to model traffic, the -P parameter sets the number of parallel connections used.
Linux Storage Performance
Some best practices, like the following, are listed on Best Practices for Running Linux on Hyper-V. The Linux
kernel has different I/O schedulers to reorder requests with different algorithms. NOOP is a first-in first-out
queue that passes the schedule decision to be made by the hypervisor. It is recommended to use NOOP as the
scheduler when running Linux virtual machine on Hyper-V. To change the scheduler for a specific device, in the
boot loader's configuration (/etc/grub.conf, for example), add elevator=noop to the kernel parameters, and then
restart.
Similar to networking, Linux guest performance with storage benefits the most from multiple queues with
enough depth to keep the host busy. Microbenchmarking storage performance is probably best with the fio
benchmark tool with the libaio engine.

See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
 Performance tuning Windows Server containers
5/4/2017 5 min to read Edit Online

Introduction
Windows Server 2016 is the first version of Windows to ship support for container technology built in to the OS. In
Server 2016, two types of containers are available: Windows Server Containers and Hyper-V Containers. Each
container type supports either the Server Core or Nano Server SKU of Windows Server 2016.
These configurations have different performance implications which we detail below to help you understand which
is right for your scenarios. In addition, we detail performance impacting configurations, and describe the tradeoffs
with each of those options.
Windows Server Container and Hyper-V Containers
Windows Server Container and Hyper-V containers offer many of the same portability and consistency benefits but
differ in terms of their isolation guarantees and performance characteristsics.
Windows Server Containers provide application isolation through process and namespace isolation technology.
A Windows Server container shares a kernel with the container host and all containers running on the host.
Hyper-V Containers expand on the isolation provided by Windows Server Containers by running each container
in a highly optimized virtual machine. In this configuration the kernel of the container host is not shared with the
Hyper-V Containers.
The additional isolation provided by Hyper-V containers is achieved in large part by a hypervisor layer of isolation
between the container and the container host. This affects container density as, unlike Windows Server Containers,
less sharing of system files and binaries can occur, resulting in an overall larger storage and memory footprint. In
addition there is the expected additional overhead in some network, storage io, and CPU paths.
Nano Server and Server Core
Windows Server Containers and Hyper-V containers offer support for Server Core and for a new installation option
available in Windows Server 2016 : Nano Server.
Nano Server is a remotely administered server operating system optimized for private clouds and datacenters. It is
similar to Windows Server in Server Core mode, but significantly smaller, has no local logon capability, and only
supports 64-bit applications, tools, and agents. It takes up far less disk space and starts faster.

Container Start Up Time

Container start up time is a key metric in many of the scenarios that containers offer the greatest benefit. As such,
understanding how to best optimize for container start up time is critical. Below are some tuning trade-offs to
understand to achieve improved start up time.
First Logon
Microsoft ships a base image for both Nano Server and Server Core. The base image which ships for Server Core
has been optimized by removing the start-up time overhead associated with first logon (OOBE). This is not the case
with Nano Server base image. However, this cost can be removed from Nano Server based images by committing
at least one layer to the container image. Subsequent container starts from the image will not incur the first logon
cost.
Scratch Space Location
Containers, by default, use a temporary scratch space on the container hosts system drive media for storage
during the lifetime of the running container. This serves as the containers system drive, and as such many of the
writes and reads done in container operation follow this path. For host systems where the system drive exists on
spinning disk magnetic media (HDDs) but faster storage media is available (faster HDDs or SSDs), it is possible to
move the container scratch space to a different drive. This is achieved by using the dockerd g command. This
command is global, and will affect all containers running on the system.
Nested Hyper-V Containers
Hyper-V for Windows Server 2016 introduces nested hypervisor support. That is, it is now possible to run a virtual
machine from within a virtual machine. This opens up many useful scenarios but also exaggerates some
performance impact that the hypervisor incurs, as there are two level of hypervisors running above the physical
host.
For containers, this has an impact when running a Hyper-V container inside a virtual machine. Since a Hyper-V
Container offers isolation through a hypervisor layer between itself and the container host, when the container host
is a Hyper-V based virtual machine, there is performance overhead associated in terms of container start-up time,
storage io, network io and throughput, and CPU.

Storage
Mounted Data Volumes
Containers offer the ability to use the container host system drive for the container scratch space. However, the
container scratch space has a life span equal to that of the container. That is, when the container is stopped, the
scratch space and all associated data goes away.
However, there are many scenarios in which having data persist independent of container lifetime is desired. In
these cases, we support mounting data volumes from the container host into the container. For Windows Server
Containers, there is neglible IO path overhead associated with mounted data volumes (near native performance).
However, when mounting data volumes into Hyper-V containers, there is some IO performance degradation in that
path. In addition, this impact is exaggerated when running Hyper-V containers inside of virtual machines.
Scratch Space
Both Windows Server Containers and Hyper-V containers provide a 20GB dynamic VHD for the container scratch
space by default. For both container types, the container OS takes up a portion of that space, and this is true for
every container started. Thus it is important to remember that every container started has some storage impact,
and depending on the workload can write up to 20GB of the backing storage media. Server storage configurations
should be designed with this in mind. (can we configure scratch size)

Networking
Windows Server Containers and Hyper-V containers offer a variety of networking modes to best suit the needs of
differing networking configurations. Each of these options present their own performance characteristics.
Windows Network Address Translation (WinNAT )
Each container will receive an IP address from an internal, private IP prefix (e.g. 172.16.0.0/12). Port forwarding /
mapping from the container host to container endpoints is supported. Docker creates a NAT network by default
when the dockerd first runs.
Of these three modes, the NAT configuration is the most expensive network IO path, but has the least amount of
configuration needed.
Windows Server containers use a Host vNIC to attach to the virtual switch. Hyper-V Containers use a Synthetic VM
NIC (not exposed to the Utility VM) to attach to the virtual switch. When containers are communicating with the
external network, packets are routed through WinNAT with address translations applied, which incurs some
overhead.
Transparent
Each container endpoint is directly connected to the physical network. IP addresses from the physical network can
be assigned statically or dynamically using an external DHCP server.
Transparent mode is the least expensive in terms of the network IO path, and external packets are directly passed
through to the container virtual NIC giving direct access to the external network.
L2 Bridge
Each container endpoint will be in the same IP subnet as the container host. The IP addresses must be assigned
statically from the same prefix as the container host. All container endpoints on the host will have the same MAC
address due to Layer-2 address translation.
L2 Bridge Mode is more performant than WinNAT mode as it provides direct access to the external network, but
less performant than Transparent mode as it also introduces MAC address translation.
 Performance Tuning Remote Desktop Session Hosts
4/24/2017 12 min to read Edit Online

This topic discusses how to select Remote Desktop Session Host (RD Session Host) hardware, tune the host, and
tune applications.
In this topic:
Selecting the proper hardware for performance
Tuning applications for Remote Desktop Session Host
Remote Desktop Session Host tuning parameters

Selecting the proper hardware for performance

For an RD Session Host server deployment, the choice of hardware is governed by the application set and how
users use them. The key factors that affect the number of users and their experience are CPU, memory, disk, and
graphics. This section contains additional guidelines that are specific to RD Session Host servers and is mostly
related to the multi-user environment of RD Session Host servers.
CPU configuration
CPU configuration is conceptually determined by multiplying the required CPU to support a session by the number
of sessions that the system is expected to support, while maintaining a buffer zone to handle temporary spikes.
Multiple logical processors can help reduce abnormal CPU congestion situations, which are usually caused by a few
overactive threads that are contained by a similar number of logical processors.
Therefore, the more logical processors on a system, the lower the cushion margin that must be built in to the CPU
usage estimate, which results in a larger percentage of active load per CPU. One important factor to remember is
that doubling the number of CPUs does not double CPU capacity.
Memory configuration
Memory configuration is dependent on the applications that users employ; however, the required amount of
memory can be estimated by using the following formula: TotalMem = OSMem + SessionMem * NS
OSMem is how much memory the operating system requires to run (such as system binary images, data structures,
and so on), SessionMem is how much memory processes running in one session require, and NS is the target
number of active sessions. The amount of required memory for a session is mostly determined by the private
memory reference set for applications and system processes that are running inside the session. Shared code or
data pages have little effect because only one copy is present on the system.
One interesting observation (assuming the disk system that is backing up the page file does not change) is that the
larger the number of concurrent active sessions the system plans to support, the bigger the per-session memory
allocation must be. If the amount of memory that is allocated per session is not increased, the number of page
faults that active sessions generate increases with the number of sessions. These faults eventually overwhelm the
I/O subsystem. By increasing the amount of memory that is allocated per session, the probability of incurring page
faults decreases, which helps reduce the overall rate of page faults.
Disk configuration
Storage is one of the most overlooked aspects when you configure RD Session Host servers, and it can be the most
common limitation in systems that are deployed in the field.
The disk activity that is generated on a typical RD Session Host server affects the following areas:
 System files and application binaries
Page files
User profiles and user data
Ideally, these areas should be backed up by distinct storage devices. Using striped RAID configurations or other
types of high-performance storage further improves performance. We highly recommend that you use storage
adapters with battery-backed write caching. Controllers with disk write caching offer improved support for
synchronous write operations. Because all users have a separate hive, synchronous write operations are
significantly more common on an RD Session Host server. Registry hives are periodically saved to disk by using
synchronous write operations. To enable these optimizations, from the Disk Management console, open the
Properties dialog box for the destination disk and, on the Policies tab, select the Enable write caching on the
disk and Turn off Windows write-cache buffer flushing on the device check boxes.
Network configuration
Network usage for an RD Session Host server includes two main categories:
RD Session Host connection traffic usage is determined almost exclusively by the drawing patterns that are
exhibited by the applications running inside the sessions and the redirected devices I/O traffic.
For example, applications handling text processing and data input consume bandwidth of approximately 10
to 100 kilobits per second, whereas rich graphics and video playback cause significant increases in
bandwidth usage.
Back-end connections such as roaming profiles, application access to file shares, database servers, e-mail
servers, and HTTP servers.
The volume and profile of network traffic is specific to each deployment.

Tuning applications for Remote Desktop Session Host

Most of the CPU usage on an RD Session Host server is driven by apps. Desktop apps are usually optimized toward
responsiveness with the goal of minimizing how long it takes an application to respond to a user request. However
in a server environment, it is equally important to minimize the total amount of CPU usage that is needed to
complete an action to avoid adversely affecting other sessions.
Consider the following suggestions when you configure apps that are to be used on an RD Session Host server:
Minimize background idle loop processing
Typical examples are disabling background grammar and spell check, data indexing for search, and
background saves.
Minimize how often an app performs a state check or update.
Disabling such behaviors or increasing the interval between polling iterations and timer firing significantly
benefits CPU usage because the effect of such activities is quickly amplified for many active sessions. Typical
examples are connection status icons and status bar information updates.
Minimize resource contention between apps by reducing their synchronization frequency.
Examples of such resources include registry keys and configuration files. Examples of application
components and features are status indicator (like shell notifications), background indexing or change
monitoring, and offline synchronization.
Disable unnecessary processes that are registered to start with user sign-in or a session startup.
These processes can significantly contribute to the cost of CPU usage when creating a new user session,
 which generally is a CPU-intensive process, and it can be very expensive in morning scenarios. Use
MsConfig.exe or MsInfo32.exe to obtain a list of processes that are started at user sign-in. For more detailed
info, you can use Autoruns for Windows.
For memory consumption, you should consider the following:
Verify that DLLs loaded by an app are not relocated.
Relocated DLLs can be verified by selecting Process DLL view, as shown in the following figure, by
using Process Explorer.
Here we can see that y.dll was relocated because x.dll already occupied its default base address and
ASLR was not enabled

If DLLs are relocated, it is impossible to share their code across sessions, which significantly increases
the footprint of a session. This is one of the most common memory-related performance issues on an
RD Session Host server.
For common language runtime (CLR) applications, use Native Image Generator (Ngen.exe) to increase page
sharing and reduce CPU overhead.
When possible, apply similar techniques to other similar execution engines.

Remote Desktop Session Host tuning parameters

Page file
Insufficient page file size can cause memory allocation failures in apps or system components. You can use the
memory-to-committed bytes performance counter to monitor how much committed virtual memory is on the
system.
Antivirus
Installing antivirus software on an RD Session Host server greatly affects overall system performance, especially
CPU usage. We highly recommend that you exclude from the active monitoring list all the folders that hold
temporary files, especially those that services and other system components generate.
Task Scheduler
Task Scheduler lets you examine the list of tasks that are scheduled for different events. For an RD Session Host
server, it is useful to focus specifically on the tasks that are configured to run on idle, at user sign-in, or on session
connect and disconnect. Because of the specifics of the deployment, many of these tasks might be unnecessary.
Desktop notification icons
Notification icons on the desktop can have fairly expensive refreshing mechanisms. You should disable any
notifications by removing the component that registers them from the startup list or by changing the configuration
on apps and system components to disable them. You can use Customize Notifications Icons to examine the list
of notifications that are available on the server.
RemoteFX data compression
Microsoft RemoteFX compression can be configured by using Group Policy under Computer Configuration >
Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop
Session Host > Remote Session Environment > Configure compression for RemoteFX data. Three values are
possible:
Optimized to use less memory Consumes the least amount of memory per session but has the lowest
compression ratio and therefore the highest bandwidth consumption.
Balances memory and network bandwidth Reduced bandwidth consumption while marginally
increasing memory consumption (approximately 200 KB per session).
Optimized to use less network bandwidth Further reduces network bandwidth usage at a cost of
approximately 2 MB per session. If you want to use this setting, you should assess the maximum number of
sessions and test to that level with this setting before you place the server in production.
You can also choose to not use a RemoteFX compression algorithm. Choosing to not use a RemoteFX compression
algorithm will use more network bandwidth, and it is only recommended if you are using a hardware device that is
designed to optimize network traffic. Even if you choose not to use a RemoteFX compression algorithm, some
graphics data will be compressed.
Device redirection
Device redirection can be configured by using Group Policy under Computer Configuration > Administrative
Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Device
and Resource Redirection or by using the Session Collection properties box in Server Manager.
Generally, device redirection increases how much network bandwidth RD Session Host server connections use
because data is exchanged between devices on the client computers and processes that are running in the server
session. The extent of the increase is a function of the frequency of operations that are performed by the
applications that are running on the server against the redirected devices.
Printer redirection and Plug and Play device redirection also increases CPU usage at sign-in. You can redirect
printers in two ways:
Matching printer driver-based redirection when a driver for the printer must be installed on the server.
Earlier releases of Windows Server used this method.
Introduced in Windows Server 2008, Easy Print printer driver redirection uses a common printer driver for
all printers.
We recommend the Easy Print method because it causes less CPU usage for printer installation at connection time.
The matching driver method causes increased CPU usage because it requires the spooler service to load different
drivers. For bandwidth usage, Easy Print causes slightly increased network bandwidth usage, but not significant
enough to offset the other performance, manageability, and reliability benefits.
Audio redirection causes a steady stream of network traffic. Audio redirection also enables users to run multimedia
apps that typically have high CPU consumption.
Client experience settings
By default, Remote Desktop Connection (RDC) automatically chooses the right experience setting based on the
suitability of the network connection between the server and client computers. We recommend that the RDC
configuration remain at Detect connection quality automatically.
For advanced users, RDC provides control over a range of settings that influence network bandwidth performance
for the Remote Desktop Services connection. You can access the following settings by using the Experience tab in
Remote Desktop Connection or as settings in the RDP file.
The following settings apply when connecting to any computer:
Disable wallpaper (Disable wallpaper:i:0) Does not show desktop wallpaper on redirected connections. This
setting can significantly reduce bandwidth usage if desktop wallpaper consists of an image or other content
with significant costs for drawing.
 Bitmap cache (Bitmapcachepersistenable:i:1) When this setting is enabled, it creates a client-side cache of
bitmaps that are rendered in the session. It provides a significant improvement on bandwidth usage, and it
should always be enabled (unless there are other security considerations).
Show contents of windows while dragging (Disable full window drag:i:1) When this setting is disabled, it
reduces bandwidth by displaying only the window frame instead of all the content when the window is
dragged.
Menu and window animation (Disable menu anims:i:1 and Disable cursor setting:i:1): When these settings
are disabled, it reduces bandwidth by disabling animation on menus (such as fading) and cursors.
Font smoothing (Allow font smoothing:i:0) Controls ClearType font-rendering support. When connecting to
computers running Windows 8 or Windows Server 2012 and above, enabling or disabling this setting does
not have a significant impact on bandwidth usage. However, for computers running versions earlier than
Windows 7 and Windows 2008 R2, enabling this setting affects network bandwidth consumption
significantly.
The following settings only apply when connecting to computers running Windows 7 and earlier operating system
versions:
Desktop composition This setting is supported only for a remote session to a computer running Windows
7 or Windows Server 2008 R2.
Visual styles (disable themes:i:1) When this setting is disabled, it reduces bandwidth by simplifying theme
drawings that use the Classic theme.
By using the Experience tab within Remote Desktop Connection, you can choose your connection speed to
influence network bandwidth performance. The following lists the options that are available to configure your
connection speed:
Detect connection quality automatically (Connection type:i:7) When this setting is enabled, Remote
Desktop Connection automatically chooses settings that will result in optimal user experience based on
connection quality. (This configuration is recommended when connecting to computers running Windows 8
or Windows Server 2012 and above).
Modem (56 Kbps) (Connection type:i:1) This setting enables persistent bitmap caching.
Low Speed Broadband (256 Kbps - 2 Mbps) (Connection type:i:2) This setting enables persistent bitmap
caching and visual styles.
Cellular/Satellite (2Mbps - 16 Mbps with high latency) (Connection type:i:3) This setting enables
desktop composition, persistent bitmap caching, visual styles, and desktop background.
High-speed broadband (2 Mbps 10 Mbps ) (Connection type:i:4) This setting enables desktop
composition, show contents of windows while dragging, menu and window animation, persistent bitmap
caching, visual styles, and desktop background.
WAN (10 Mbps or higher with high latency) (Connection type:i:5) This setting enables desktop
composition, show contents of windows while dragging, menu and window animation, persistent bitmap
caching, visual styles, and desktop background.
LAN (10 Mbps or higher) (Connection type:i:6) This setting enables desktop composition, show contents of
windows while dragging, menu and window animation, persistent bitmap caching, themes, and desktop
background.
Desktop Size
Desktop size for remote sessions can be controlled by using the Display tab in Remote Desktop Connection or by
using the RDP configuration file (desktopwidth:i:1152 and desktopheight:i:864). The larger the desktop size, the
greater the memory and bandwidth consumption that is associated with that session. The current maximum
desktop size is 4096 x 2048.
 Performance Tuning Remote Desktop Session Hosts
4/24/2017 12 min to read Edit Online

This topic discusses how to select Remote Desktop Session Host (RD Session Host) hardware, tune the host, and
tune applications.
In this topic:
Selecting the proper hardware for performance
Tuning applications for Remote Desktop Session Host
Remote Desktop Session Host tuning parameters

Selecting the proper hardware for performance

For an RD Session Host server deployment, the choice of hardware is governed by the application set and how
users use them. The key factors that affect the number of users and their experience are CPU, memory, disk, and
graphics. This section contains additional guidelines that are specific to RD Session Host servers and is mostly
related to the multi-user environment of RD Session Host servers.
CPU configuration
CPU configuration is conceptually determined by multiplying the required CPU to support a session by the number
of sessions that the system is expected to support, while maintaining a buffer zone to handle temporary spikes.
Multiple logical processors can help reduce abnormal CPU congestion situations, which are usually caused by a few
overactive threads that are contained by a similar number of logical processors.
Therefore, the more logical processors on a system, the lower the cushion margin that must be built in to the CPU
usage estimate, which results in a larger percentage of active load per CPU. One important factor to remember is
that doubling the number of CPUs does not double CPU capacity.
Memory configuration
Memory configuration is dependent on the applications that users employ; however, the required amount of
memory can be estimated by using the following formula: TotalMem = OSMem + SessionMem * NS
OSMem is how much memory the operating system requires to run (such as system binary images, data
structures, and so on), SessionMem is how much memory processes running in one session require, and NS is the
target number of active sessions. The amount of required memory for a session is mostly determined by the
private memory reference set for applications and system processes that are running inside the session. Shared
code or data pages have little effect because only one copy is present on the system.
One interesting observation (assuming the disk system that is backing up the page file does not change) is that the
larger the number of concurrent active sessions the system plans to support, the bigger the per-session memory
allocation must be. If the amount of memory that is allocated per session is not increased, the number of page
faults that active sessions generate increases with the number of sessions. These faults eventually overwhelm the
I/O subsystem. By increasing the amount of memory that is allocated per session, the probability of incurring page
faults decreases, which helps reduce the overall rate of page faults.
Disk configuration
Storage is one of the most overlooked aspects when you configure RD Session Host servers, and it can be the most
common limitation in systems that are deployed in the field.
The disk activity that is generated on a typical RD Session Host server affects the following areas:
 System files and application binaries
Page files
User profiles and user data
Ideally, these areas should be backed up by distinct storage devices. Using striped RAID configurations or other
types of high-performance storage further improves performance. We highly recommend that you use storage
adapters with battery-backed write caching. Controllers with disk write caching offer improved support for
synchronous write operations. Because all users have a separate hive, synchronous write operations are
significantly more common on an RD Session Host server. Registry hives are periodically saved to disk by using
synchronous write operations. To enable these optimizations, from the Disk Management console, open the
Properties dialog box for the destination disk and, on the Policies tab, select the Enable write caching on the
disk and Turn off Windows write-cache buffer flushing on the device check boxes.
Network configuration
Network usage for an RD Session Host server includes two main categories:
RD Session Host connection traffic usage is determined almost exclusively by the drawing patterns that are
exhibited by the applications running inside the sessions and the redirected devices I/O traffic.
For example, applications handling text processing and data input consume bandwidth of approximately 10
to 100 kilobits per second, whereas rich graphics and video playback cause significant increases in
bandwidth usage.
Back-end connections such as roaming profiles, application access to file shares, database servers, e-mail
servers, and HTTP servers.
The volume and profile of network traffic is specific to each deployment.

Tuning applications for Remote Desktop Session Host

Most of the CPU usage on an RD Session Host server is driven by apps. Desktop apps are usually optimized toward
responsiveness with the goal of minimizing how long it takes an application to respond to a user request. However
in a server environment, it is equally important to minimize the total amount of CPU usage that is needed to
complete an action to avoid adversely affecting other sessions.
Consider the following suggestions when you configure apps that are to be used on an RD Session Host server:
Minimize background idle loop processing
Typical examples are disabling background grammar and spell check, data indexing for search, and
background saves.
Minimize how often an app performs a state check or update.
Disabling such behaviors or increasing the interval between polling iterations and timer firing significantly
benefits CPU usage because the effect of such activities is quickly amplified for many active sessions. Typical
examples are connection status icons and status bar information updates.
Minimize resource contention between apps by reducing their synchronization frequency.
Examples of such resources include registry keys and configuration files. Examples of application
components and features are status indicator (like shell notifications), background indexing or change
monitoring, and offline synchronization.
Disable unnecessary processes that are registered to start with user sign-in or a session startup.
These processes can significantly contribute to the cost of CPU usage when creating a new user session,
 which generally is a CPU-intensive process, and it can be very expensive in morning scenarios. Use
MsConfig.exe or MsInfo32.exe to obtain a list of processes that are started at user sign-in. For more detailed
info, you can use Autoruns for Windows.
For memory consumption, you should consider the following:
Verify that DLLs loaded by an app are not relocated.
Relocated DLLs can be verified by selecting Process DLL view, as shown in the following figure, by
using Process Explorer.
Here we can see that y.dll was relocated because x.dll already occupied its default base address and
ASLR was not enabled

If DLLs are relocated, it is impossible to share their code across sessions, which significantly increases
the footprint of a session. This is one of the most common memory-related performance issues on an
RD Session Host server.
For common language runtime (CLR) applications, use Native Image Generator (Ngen.exe) to increase page
sharing and reduce CPU overhead.
When possible, apply similar techniques to other similar execution engines.

Remote Desktop Session Host tuning parameters

Page file
Insufficient page file size can cause memory allocation failures in apps or system components. You can use the
memory-to-committed bytes performance counter to monitor how much committed virtual memory is on the
system.
Antivirus
Installing antivirus software on an RD Session Host server greatly affects overall system performance, especially
CPU usage. We highly recommend that you exclude from the active monitoring list all the folders that hold
temporary files, especially those that services and other system components generate.
Task Scheduler
Task Scheduler lets you examine the list of tasks that are scheduled for different events. For an RD Session Host
server, it is useful to focus specifically on the tasks that are configured to run on idle, at user sign-in, or on session
connect and disconnect. Because of the specifics of the deployment, many of these tasks might be unnecessary.
Desktop notification icons
Notification icons on the desktop can have fairly expensive refreshing mechanisms. You should disable any
notifications by removing the component that registers them from the startup list or by changing the configuration
on apps and system components to disable them. You can use Customize Notifications Icons to examine the list
of notifications that are available on the server.
RemoteFX data compression
Microsoft RemoteFX compression can be configured by using Group Policy under Computer Configuration >
Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop
Session Host > Remote Session Environment > Configure compression for RemoteFX data. Three values
are possible:
Optimized to use less memory Consumes the least amount of memory per session but has the lowest
compression ratio and therefore the highest bandwidth consumption.
Balances memory and network bandwidth Reduced bandwidth consumption while marginally
increasing memory consumption (approximately 200 KB per session).
Optimized to use less network bandwidth Further reduces network bandwidth usage at a cost of
approximately 2 MB per session. If you want to use this setting, you should assess the maximum number of
sessions and test to that level with this setting before you place the server in production.
You can also choose to not use a RemoteFX compression algorithm. Choosing to not use a RemoteFX compression
algorithm will use more network bandwidth, and it is only recommended if you are using a hardware device that is
designed to optimize network traffic. Even if you choose not to use a RemoteFX compression algorithm, some
graphics data will be compressed.
Device redirection
Device redirection can be configured by using Group Policy under Computer Configuration > Administrative
Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host >
Device and Resource Redirection or by using the Session Collection properties box in Server Manager.
Generally, device redirection increases how much network bandwidth RD Session Host server connections use
because data is exchanged between devices on the client computers and processes that are running in the server
session. The extent of the increase is a function of the frequency of operations that are performed by the
applications that are running on the server against the redirected devices.
Printer redirection and Plug and Play device redirection also increases CPU usage at sign-in. You can redirect
printers in two ways:
Matching printer driver-based redirection when a driver for the printer must be installed on the server.
Earlier releases of Windows Server used this method.
Introduced in Windows Server 2008, Easy Print printer driver redirection uses a common printer driver for
all printers.
We recommend the Easy Print method because it causes less CPU usage for printer installation at connection time.
The matching driver method causes increased CPU usage because it requires the spooler service to load different
drivers. For bandwidth usage, Easy Print causes slightly increased network bandwidth usage, but not significant
enough to offset the other performance, manageability, and reliability benefits.
Audio redirection causes a steady stream of network traffic. Audio redirection also enables users to run multimedia
apps that typically have high CPU consumption.
Client experience settings
By default, Remote Desktop Connection (RDC) automatically chooses the right experience setting based on the
suitability of the network connection between the server and client computers. We recommend that the RDC
configuration remain at Detect connection quality automatically.
For advanced users, RDC provides control over a range of settings that influence network bandwidth performance
for the Remote Desktop Services connection. You can access the following settings by using the Experience tab in
Remote Desktop Connection or as settings in the RDP file.
The following settings apply when connecting to any computer:
Disable wallpaper (Disable wallpaper:i:0) Does not show desktop wallpaper on redirected connections. This
setting can significantly reduce bandwidth usage if desktop wallpaper consists of an image or other content
with significant costs for drawing.
 Bitmap cache (Bitmapcachepersistenable:i:1) When this setting is enabled, it creates a client-side cache of
bitmaps that are rendered in the session. It provides a significant improvement on bandwidth usage, and it
should always be enabled (unless there are other security considerations).
Show contents of windows while dragging (Disable full window drag:i:1) When this setting is disabled, it
reduces bandwidth by displaying only the window frame instead of all the content when the window is
dragged.
Menu and window animation (Disable menu anims:i:1 and Disable cursor setting:i:1): When these settings
are disabled, it reduces bandwidth by disabling animation on menus (such as fading) and cursors.
Font smoothing (Allow font smoothing:i:0) Controls ClearType font-rendering support. When connecting
to computers running Windows 8 or Windows Server 2012 and above, enabling or disabling this setting
does not have a significant impact on bandwidth usage. However, for computers running versions earlier
than Windows 7 and Windows 2008 R2, enabling this setting affects network bandwidth consumption
significantly.
The following settings only apply when connecting to computers running Windows 7 and earlier operating system
versions:
Desktop composition This setting is supported only for a remote session to a computer running Windows
7 or Windows Server 2008 R2.
Visual styles (disable themes:i:1) When this setting is disabled, it reduces bandwidth by simplifying theme
drawings that use the Classic theme.
By using the Experience tab within Remote Desktop Connection, you can choose your connection speed to
influence network bandwidth performance. The following lists the options that are available to configure your
connection speed:
Detect connection quality automatically (Connection type:i:7) When this setting is enabled, Remote
Desktop Connection automatically chooses settings that will result in optimal user experience based on
connection quality. (This configuration is recommended when connecting to computers running Windows 8
or Windows Server 2012 and above).
Modem (56 Kbps) (Connection type:i:1) This setting enables persistent bitmap caching.
Low Speed Broadband (256 Kbps - 2 Mbps) (Connection type:i:2) This setting enables persistent bitmap
caching and visual styles.
Cellular/Satellite (2Mbps - 16 Mbps with high latency) (Connection type:i:3) This setting enables
desktop composition, persistent bitmap caching, visual styles, and desktop background.
High-speed broadband (2 Mbps 10 Mbps ) (Connection type:i:4) This setting enables desktop
composition, show contents of windows while dragging, menu and window animation, persistent bitmap
caching, visual styles, and desktop background.
WAN (10 Mbps or higher with high latency) (Connection type:i:5) This setting enables desktop
composition, show contents of windows while dragging, menu and window animation, persistent bitmap
caching, visual styles, and desktop background.
LAN (10 Mbps or higher) (Connection type:i:6) This setting enables desktop composition, show contents of
windows while dragging, menu and window animation, persistent bitmap caching, themes, and desktop
background.
Desktop Size
Desktop size for remote sessions can be controlled by using the Display tab in Remote Desktop Connection or by
using the RDP configuration file (desktopwidth:i:1152 and desktopheight:i:864). The larger the desktop size, the
greater the memory and bandwidth consumption that is associated with that session. The current maximum
desktop size is 4096 x 2048.
 Performance Tuning Remote Desktop Virtualization
Hosts
4/24/2017 12 min to read Edit Online

Remote Desktop Virtualization Host (RD Virtualization Host) is a role service that supports Virtual Desktop
Infrastructure (VDI) scenarios and lets multiple concurrent users run Windows-based applications in virtual
machines that are hosted on a server running Windows Server 2016 and Hyper-V.
Windows Server 2016 supports two types of virtual desktops, personal virtual desktops and pooled virtual
desktops.
In this topic:
General considerations
Performance optimizations

General considerations
Storage
Storage is the most likely performance bottleneck, and it is important to size your storage to properly handle the
I/O load that is generated by virtual machine state changes. If a pilot or simulation is not feasible, a good guideline
is to provision one disk spindle for four active virtual machines. Use disk configurations that have good write
performance (such as RAID 1+0).
When appropriate, use Disk Deduplication and caching to reduce the disk read load and to enable your storage
solution to speed up performance by caching a significant portion of the image.
Data Deduplication and VDI
Introduced in Windows Server 2012 R2, Data Deduplication supports optimization of open files. In order to use
virtual machines running on a deduplicated volume, the virtual machine files need to be stored on a separate host
from the Hyper-V host. If Hyper-V and deduplication are running on the same machine, the two features will
contend for system resources and negatively impact overall performance.
The volume must also be configured to use the Virtual Desktop Infrastructure (VDI) deduplication optimization
type. You can configure this by using Server Manager (File and Storage Services -> Volumes -> Dedup
Settings) or by using the following Windows PowerShell command:

Enable-DedupVolume <volume> -UsageType HyperV

Note
Data Deduplication optimization of open files is supported only for VDI scenarios with Hyper-V using remote
storage over SMB 3.0.
Memory
Server memory usage is driven by three main factors:
Operating system overhead
Hyper-V service overhead per virtual machine
Memory allocated to each virtual machine
For a typical knowledge worker workload, guest virtual machines running x86 Window 8 or Windows 8.1 should be
given ~512 MB of memory as the baseline. However, Dynamic Memory will likely increase the guest virtual
machines memory to about 800 MB, depending on the workload. For x64, we see about 800 MB starting,
increasing to 1024 MB.
Therefore, it is important to provide enough server memory to satisfy the memory that is required by the expected
number of guest virtual machines, plus allow a sufficient amount of memory for the server.
CPU
When you plan server capacity for an RD Virtualization Host server, the number of virtual machines per physical
core will depend on the nature of the workload. As a starting point, it is reasonable to plan 12 virtual machines per
physical core, and then run the appropriate scenarios to validate performance and density. Higher density may be
achievable depending on the specifics of the workload.
We recommend enabling hyper-threading, but be sure to calculate the oversubscription ratio based on the number
of physical cores and not the number of logical processors. This ensures the expected level of performance on a per
CPU basis.
Virtual GPU
Microsoft RemoteFX for RD Virtualization Host delivers a rich graphics experience for Virtual Desktop Infrastructure
(VDI) through host-side remoting, a render-capture-encode pipeline, a highly efficient GPU-based encode, throttling
based on client activity, and a DirectX-enabled virtual GPU. RemoteFX for RD Virtualization Host upgrades the
virtual GPU from DirectX9 to DirectX11. It also improves the user experience by supporting more monitors at
higher resolutions.
The RemoteFX DirectX11 experience is available without a hardware GPU, through a software-emulated driver.
Although this software GPU provides a good experience, the RemoteFX virtual graphics processing unit (VGPU)
adds a hardware accelerated experience to virtual desktops.
To take advantage of the RemoteFX VGPU experience on a server running Windows Server 2016, you need a GPU
driver (such as DirectX11.1 or WDDM 1.2) on the host server. For more information about GPU offerings to use
with RemoteFX for RD Virtualization Host, contact your GPU provider.
If you use the RemoteFX virtual GPU in your VDI deployment, the deployment capacity will vary based on usage
scenarios and hardware configuration. When you plan your deployment, consider the following:
Number of GPUs on your system
Video memory capacity on the GPUs
Processor and hardware resources on your system
RemoteFX server system memory
For every virtual desktop enabled with a virtual GPU, RemoteFX uses system memory in the guest operating system
and in the RemoteFX-enabled server. The hypervisor guarantees the availability of system memory for a guest
operating system. On the server, each virtual GPU-enabled virtual desktop needs to advertise its system memory
requirement to the hypervisor. When the virtual GPU-enabled virtual desktop is starting, the hypervisor reserves
additional system memory in the RemoteFX-enabled server for the VGPU-enabled virtual desktop.
The memory requirement for the RemoteFX-enabled server is dynamic because the amount of memory consumed
on the RemoteFX-enabled server is dependent on the number of monitors that are associated with the VGPU-
enabled virtual desktops and the maximum resolution for those monitors.
RemoteFX server GPU video memory
Every virtual GPU-enabled virtual desktop uses the video memory in the GPU hardware on the host server to
render the desktop. In addition to rendering, the video memory is used by a codec to compress the rendered screen.
The amount of memory needed is directly based on the amount of monitors that are provisioned to the virtual
machine.
The video memory that is reserved varies based on the number of monitors and the system screen resolution.
Some users may require a higher screen resolution for specific tasks. There is greater scalability with lower
resolution settings if all other settings remain constant.
RemoteFX processor
The hypervisor schedules the RemoteFX-enabled server and the virtual GPU-enabled virtual desktops on the CPU.
Unlike the system memory, there isnt information that is related to additional resources that RemoteFX needs to
share with the hypervisor. The additional CPU overhead that RemoteFX brings into the virtual GPU-enabled virtual
desktop is related to running the virtual GPU driver and a user-mode Remote Desktop Protocol stack.
On the RemoteFX-enabled server, the overhead is increased, because the system runs an additional process
(rdvgm.exe) per virtual GPU-enabled virtual desktop. This process uses the graphics device driver to run commands
on the GPU. The codec also uses the CPUs for compressing the screen data that needs to be sent back to the client.
More virtual processors mean a better user experience. We recommend allocating at least two virtual CPUs per
virtual GPU-enabled virtual desktop. We also recommend using the x64 architecture for virtual GPU-enabled virtual
desktops because the performance on x64 virtual machines is better compared to x86 virtual machines.
RemoteFX GPU processing power
For every virtual GPU-enabled virtual desktop, there is a corresponding DirectX process running on the RemoteFX-
enabled server. This process replays all the graphics commands that it receives from the RemoteFX virtual desktop
onto the physical GPU. For the physical GPU, it is equivalent to simultaneously running multiple DirectX
applications.
Typically, graphics devices and drivers are tuned to run a few applications on the desktop. RemoteFX stretches the
GPUs to be used in a unique manner. To measure how the GPU is performing on a RemoteFX server, performance
counters have been added to measure the GPU response to RemoteFX requests.
Usually when a GPU resource is low on resources, Read and Write operations to the GPU take a long time to
complete. By using performance counters, administrators can take preventative action, eliminating the possibility of
any downtime for their end users.
The following performance counters are available on the RemoteFX server to measure the virtual GPU performance:
RemoteFX graphics
Frames Skipped/Second - Insufficient Client Resources Number of frames skipped per second due to
insufficient client resources
Graphics Compression Ratio Ratio of the number of bytes encoded to the number of bytes input
RemoteFX root GPU management
Resources: TDRs in Server GPUs Total number of times that the TDR times out in the GPU on the server
Resources: Virtual machines running RemoteFX Total number of virtual machines that have the
RemoteFX 3D Video Adapter installed
VRAM: Available MB per GPU Amount of dedicated video memory that is not being used
VRAM: Reserved % per GPU Percent of dedicated video memory that has been reserved for RemoteFX
RemoteFX software
Capture Rate for monitor [1-4] Displays the RemoteFX capture rate for monitors 1-4
Compression Ratio Deprecated in Windows 8 and replaced by Graphics Compression Ratio
 Delayed Frames/sec Number of frames per second where graphics data was not sent within a certain
amount of time
GPU response time from Capture Latency measured within RemoteFX Capture (in microseconds) for GPU
operations to complete
GPU response time from Render Latency measured within RemoteFX Render (in microseconds) for GPU
operations to complete
Output Bytes Total number of RemoteFX output bytes
Waiting for client count/sec Deprecated in Windows 8 and replaced by Frames Skipped/Second -
Insufficient Client Resources
RemoteFX vGPU management
Resources: TDRs local to virtual machines Total number of TDRs that have occurred in this virtual
machine (TDRs that the server propagated to the virtual machines are not included)
Resources: TDRs propagated by Server Total number of TDRs that occurred on the server and that have
been propagated to the virtual machine
RemoteFX virtual machine vGPU performance
Data: Invoked presents/sec Total number (in seconds) of present operations to be rendered to the desktop
of the virtual machine per second
Data: Outgoing presents/sec Total number of present operations sent by the virtual machine to the server
GPU per second
Data: Read bytes/sec Total number of read bytes from the RemoteFX-enabled server per second
Data: Send bytes/sec Total number of bytes sent to the RemoteFX-enabled server GPU per second
DMA: Communication buffers average latency (sec) Average amount of time (in seconds) spent in the
communication buffers
DMA: DMA buffer latency (sec) Amount of time (in seconds) from when the DMA is submitted until
completed
DMA: Queue length DMA Queue length for a RemoteFX 3D Video Adapter
Resources: TDR timeouts per GPU Count of TDR timeouts that have occurred per GPU on the virtual
machine
Resources: TDR timeouts per GPU engine Count of TDR timeouts that have occurred per GPU engine on
the virtual machine
In addition to the RemoteFX virtual GPU performance counters, you can also measure the GPU utilization by using
Process Explorer, which shows video memory usage and the GPU utilization.

Performance optimizations
Dynamic Memory
Dynamic Memory enables more efficiently utilization of the memory resources of the server running Hyper-V by
balancing how memory is distributed between running virtual machines. Memory can be dynamically reallocated
between virtual machines in response to their changing workloads.
Dynamic Memory enables you to increase virtual machine density with the resources you already have without
sacrificing performance or scalability. The result is more efficient use of expensive server hardware resources, which
can translate into easier management and lower costs.
On guest operating systems running Windows 8 and above with virtual processors that span multiple logical
processors, consider the tradeoff between running with Dynamic Memory to help minimize memory usage and
disabling Dynamic Memory to improve the performance of an application that is computer-topology aware. Such
an application can leverage the topology information to make scheduling and memory allocation decisions.
Tiered Storage
RD Virtualization Host supports tiered storage for virtual desktop pools. The physical computer that is shared by all
pooled virtual desktops within a collection can use a small-size, high-performance storage solution, such as a
mirrored solid-state drive (SSD). The pooled virtual desktops can be placed on less expensive, traditional storage
such as RAID 1+0.
The physical computer should be placed on a SSD is because most of the read-I/Os from pooled virtual desktops go
to the management operating system. Therefore, the storage that is used by the physical computer must sustain
much higher read I/Os per second.
This deployment configuration assures cost effective performance where performance is needed. The SSD provides
higher performance on a smaller size disk (~20 GB per collection, depending on the configuration). Traditional
storage for pooled virtual desktops (RAID 1+0) uses about 3 GB per virtual machine.
CSV cache
Failover Clustering in Windows Server 2012 and above provides caching on Cluster Shared Volumes (CSV). This is
extremely beneficial for pooled virtual desktop collections where the majority of the read I/Os come from the
management operating system. The CSV cache provides higher performance by several orders of magnitude
because it caches blocks that are read more than once and delivers them from system memory, which reduces the
I/O. For more info on CSV cache, see How to Enable CSV Cache.
Pooled virtual desktops
By default, pooled virtual desktops are rolled back to the pristine state after a user signs out, so any changes made
to the Windows operating system since the last user sign-in are abandoned.
Although its possible to disable the rollback, it is still a temporary condition because typically a pooled virtual
desktop collection is re-created due to various updates to the virtual desktop template.
It makes sense to turn off Windows features and services that depend on persistent state. Additionally, it makes
sense to turn off services that are primarily for non-enterprise scenarios.
Each specific service should be evaluated appropriately prior to any broad deployment. The following are some
initial things to consider:

SERVICE WHY?

Auto update Pooled virtual desktops are updated by re-creating the virtual
desktop template.

Offline files Virtual desktops are always online and connected from a
networking point-of-view.

Background defrag File-system changes are discarded after a user signs off (due to
a rollback to the pristine state or re-creating the virtual
desktop template, which results in re-creating all pooled virtual
desktops).

Hibernate or sleep No such concept for VDI

 SERVICE WHY?

Bug check memory dump No such concept for pooled virtual desktops. A bug-check
pooled virtual desktop will start from the pristine state.

WLAN autoconfig There is no WiFi device interface for VDI

Windows Media Player network sharing service Consumer centric service

Home group provider Consumer centric service

Internet connection sharing Consumer centric service

Media Center extended services Consumer centric service

Note
This list is not meant to be a complete list, because any changes will affect the intended goals and scenarios. For
more info, see Hot off the presses, get it now, the Windows 8 VDI optimization script, courtesy of PFE!.
Note
SuperFetch in Windows 8 is enabled by default. It is VDI-aware and should not be disabled. SuperFetch can further
reduce memory consumption through memory page sharing, which is beneficial for VDI. Pooled virtual desktops
running Windows 7, SuperFetch should be disabled, but for personal virtual desktops running Windows 7, it should
be left on.
 Performance Tuning Remote Desktop Gateways
4/24/2017 2 min to read Edit Online

Note
In Windows 8+ and Windows Server 2012 R2+, Remote Desktop Gateway (RD Gateway) supports TCP, UDP, and
the legacy RPC transports. Most of the following data is regarding the legacy RPC transport. If the legacy RPC
transport is not being used, this section is not applicable.
This topic describes the performance-related parameters that help improve the performance of a customer
deployment and the tunings that rely on the customers network usage patterns.
At its core, RD Gateway performs many packet forwarding operations between Remote Desktop Connection
instances and the RD Session Host server instances within the customers network.
Note
The following parameters apply to RPC transport only.
Internet Information Services (IIS) and RD Gateway export the following registry parameters to help improve
system performance in the RD Gateway.
Thread tunings
Maxiothreads

HKLM\Software\Microsoft\Terminal Server Gateway\Maxiothreads (REG_DWORD)

This app-specific thread pool specifies the number of threads that RD Gateway creates to handle incoming
requests. If this registry setting is present, it takes effect. The number of threads equals the number of logical
processes. If the number of logical processors is less than 5, the default is 5 threads.
MaxPoolThreads

HKLM\System\CurrentControlSet\Services\InetInfo\Parameters\MaxPoolThreads (REG_DWORD)

This parameter specifies the number of IIS pool threads to create per logical processor. The IIS pool threads
watch the network for requests and process all incoming requests. The MaxPoolThreads count does not
include threads that RD Gateway consumes. The default value is 4.
Remote procedure call tunings for RD Gateway
The following parameters can help tune the remote procedure calls (RPC) that are received by Remote Desktop
Connection and RD Gateway computers. Changing the windows helps throttle how much data is flowing through
each connection and can improve performance for RPC over HTTP v2 scenarios.
ServerReceiveWindow

HKLM\Software\Microsoft\Rpc\ServerReceiveWindow (REG_DWORD)

The default value is 64 KB. This value specifies the window that the server uses for data that is received from
the RPC proxy. The minimum value is set to 8 KB, and the maximum value is set at 1 GB. If a value is not
present, the default value is used. When changes are made to this value, IIS must be restarted for the change
to take effect.
 ServerReceiveWindow

HKLM\Software\Microsoft\Rpc\ServerReceiveWindow (REG_DWORD)

The default value is 64 KB. This value specifies the window that the client uses for data that is received from
the RPC proxy. The minimum value is 8 KB, and the maximum value is 1 GB. If a value is not present, the
default value is used.

Monitoring and data collection

The following list of performance counters is considered a base set of counters when you monitor the resource
usage on the RD Gateway:
\Terminal Service Gateway\*
\RPC/HTTP Proxy\*
\RPC/HTTP Proxy Per Server\*
\Web Service\*
\W3SVC_W3WP\*
\IPv4\*
\Memory\*
\Network Interface(*)\*
\Process(*)\*
\Processor Information(*)\*
\Synchronization(*)\*
\System\*
\TCPv4\*
The following performance counters are applicable only for legacy RPC transport:
\RPC/HTTP Proxy\* RPC
\RPC/HTTP Proxy Per Server\* RPC
\Web Service\* RPC
\W3SVC_W3WP\* RPC
Note
If applicable, add the \IPv6\* and \TCPv6\* objects.ReplaceThisText
 Performance Tuning Web Servers
4/24/2017 2 min to read Edit Online

This topic describes performance tuning methods and recommendations for Windows Server 2016 web servers.

Selecting the proper hardware for performance

It is important to select the proper hardware to satisfy the expected web load, considering average load, peak load,
capacity, growth plans, and response times. Hardware bottlenecks limit the effectiveness of software tuning.
Performance Tuning for Server Hardware provides recommendations for hardware to avoid the following
performance constraints:
Slow CPUs offer limited processing power for CPU intensive workloads such as ASP, ASP.NET, and TLS
scenarios.
A small L2 or L3/LLC processor cache might adversely affect performance.
A limited amount of memory affects the number of sites that can be hosted, how many dynamic content
scripts (such as ASP.NET) can be stored, and the number of application pools or worker processes.
Networking becomes a bottleneck because of an inefficient network adapter.
The file system becomes a bottleneck because of an inefficient disk subsystem or storage adapter.

Operating system best practices

If possible, start with a clean installation of the operating system. Upgrading the software can leave outdated,
unwanted, or suboptimal registry settings and previously installed services and applications that consume
resources if they are started automatically. If another operating system is installed and you must keep it, you
should install the new operating system on a different partition. Otherwise, the new installation overwrites the
settings under %Program Files%\Common Files.
To reduce disk access interference, place the system page file, operating system, web data, ASP template cache, and
the Internet Information Services (IIS) log on separate physical disks, if possible.
To reduce contention for system resources, install Microsoft SQL Server and IIS on different servers, if possible.
Avoid installing non-essential services and applications. In some cases, it might be worthwhile to disable services
that are not required on a system.

NTFS file system settings

The system-global switch NtfsDisableLastAccessUpdate (REG_DWORD) 1 is located under
HKLM\System\CurrentControlSet\Control\FileSystem and is set by default to 1. This switch reduces disk I/O
load and latencies by disabling date and time stamp updating for the last file or directory access. Clean installations
of Windows Server 2016, Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and
Windows Server 2008 enable this setting by default, and you do not need to adjust it. Earlier versions of Windows
did not set this key. If your server is running an earlier version of Windows, or it was upgraded to Windows Server
2016, Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, or Windows Server 2008, you
should enable this setting.
Disabling the updates is effective when you are using large data sets (or many hosts) that contain thousands of
directories. We recommend that you use IIS logging instead if you maintain this information only for Web
administration.

WARNING
Some applications, such as incremental backup utilities, rely on this update information, and they do not function correctly
without it.

See also
IIS 10.0 performance tuning
HTTP 1.1/2 tuning
 Tuning IIS 10.0
4/24/2017 23 min to read Edit Online

Internet Information Services (IIS) 10.0 is included with Windows Server 2016. It uses a process model similar to
that of IIS 8.5 and IIS 7.0. A kernel-mode web driver (http.sys) receives and routes HTTP requests, and satisfies
requests from its response cache. Worker processes register for URL subspaces, and http.sys routes the request to
the appropriate process (or set of processes for application pools).
HTTP.sys is responsible for connection management and request handling. The request can be served from the
HTTP.sys cache or passed to a worker process for further handling. Multiple worker processes can be configured,
which provides isolation at a reduced cost. For more info on how request handling works, see the following figure:

HTTP.sys includes a response cache. When a request matches an entry in the response cache, HTTP.sys sends the
cache response directly from kernel mode. Some web application platforms, such as ASP.NET, provide mechanisms
to enable any dynamic content to be cached in the kernel-mode cache. The static file handler in IIS 10.0
automatically caches frequently requested files in http.sys.
Because a web server has kernel-mode and user-mode components, both components must be tuned for optimal
performance. Therefore, tuning IIS 10.0 for a specific workload includes configuring the following:
HTTP.sys and the associated kernel-mode cache
Worker processes and user-mode IIS, including the application pool configuration
Certain tuning parameters that affect performance
The following sections discuss how to configure the kernel-mode and user-mode aspects of IIS 10.0.

Kernel-mode settings
Performance-related HTTP.sys settings fall into two broad categories: cache management and connection and
request management. All registry settings are stored under the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters

Note
If the HTTP service is already running, you must restart it for the changes to take effect.

Cache management settings

One benefit that HTTP.sys provides is a kernel-mode cache. If the response is in the kernel-mode cache, you can
satisfy an HTTP request entirely from the kernel mode, which significantly lowers the CPU cost of handling the
request. However, the kernel-mode cache of IIS 10.0 is based on physical memory, and the cost of an entry is the
memory that it occupies.
An entry in the cache is helpful only when it is used. However, the entry always consumes physical memory,
whether or not the entry is being used. You must evaluate the usefulness of an item in the cache (the savings from
being able to serve it from the cache) and its cost (the physical memory occupied) over the lifetime of the entry by
considering the available resources (CPU and physical memory) and the workload requirements. HTTP.sys tries to
keep only useful, actively accessed items in the cache, but you can increase the performance of the web server by
tuning the HTTP.sys cache for particular workloads.
The following are some useful settings for the HTTP.sys kernel-mode cache:
UriEnableCache Default value: 1
A non-zero value enables the kernel-mode response and fragment caching. For most workloads, the cache
should remain enabled. Consider disabling the cache if you expect a very low response and fragment
caching.
UriMaxCacheMegabyteCount Default value: 0
A non-zero value that specifies the maximum memory that is available to the kernel-mode cache. The
default value, 0, enables the system to automatically adjust how much memory is available to the cache.
Note
Specifying the size sets only the maximum, and the system might not let the cache grow to the maximum set
size.
UriMaxUriBytes Default value: 262144 bytes (256 KB)
The maximum size of an entry in the kernel-mode cache. Responses or fragments larger than this are not
cached. If you have enough memory, consider increasing the limit. If memory is limited and large entries are
crowding out smaller ones, it might be helpful to lower the limit.
UriScavengerPeriod Default value: 120 seconds
The HTTP.sys cache is periodically scanned by a scavenger, and entries that are not accessed between
scavenger scans are removed. Setting the scavenger period to a high value reduces the number of
scavenger scans. However, the cache memory usage might increase because older, less frequently accessed
entries can remain in the cache. Setting the period too low causes more frequent scavenger scans, and it can
result in too many flushes and cache churn.

Request and connection management settings

In Windows Server 2016, HTTP.sys manages connections automatically. The following registry settings are no
longer used:
MaxConnections

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\MaxConnections

IdleConnectionsHighMark

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\IdleConnectionsHighMark

IdleConnectionsLowMark

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\IdleConnectionsLowMark
 IdleListTrimmerPeriod

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\IdleListTrimmerPeriod

RequestBufferLookasideDepth

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\RequestBufferLookasideDepth

InternalRequestLookasideDepth

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Http\Parameters\InternalRequestLookasideDepth

User-mode settings
The settings in this section affect the IIS 10.0 worker process behavior. Most of these settings can be found in the
following XML configuration file:
%SystemRoot%\system32\inetsrv\config\applicationHost.config
Use Appcmd.exe, the IIS 10.0 Management Console, the WebAdministration or IISAdministration PowerShell
Cmdlets to change them. Most settings are automatically detected, and they do not require a restart of the IIS 10.0
worker processes or web application server. For more info about the applicationHost.config file, see Introduction to
ApplicationHost.config.

Ideal CPU setting for NUMA hardware

Starting from Windows 2016, IIS 10.0 supports automatic ideal CPU assignment for its thread pool threads to
enhance the performance and scalability on NUMA hardware. This feature is enabled by default and can be
configured through the following registry key:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\InetInfo\Parameters\ThreadPoolUseIdealCpu

With this feature enabled, IIS thread manager makes its best effort to evenly distribute IIS thread pool threads
across all CPUs in all NUMA nodes based on their current loads. In general, it is recommended to keep this default
setting unchanged for NUMA hardware.
Note
The ideal CPU setting is different from the worker process NUMA node assignment settings
(numaNodeAssignment and numaNodeAffinityMode) introduced in CPU Settings for an Application Pool. The ideal
CPU setting affects how IIS distributes its thread pool threads, while the worker process NUMA node assignment
settings determine on which NUMA node a worker process starts.

User-mode cache behavior settings

This section describes the settings that affect caching behavior in IIS 10.0. The user-mode cache is implemented as
a module that listens to the global caching events that are raised by the integrated pipeline. To completely disable
the user-mode cache, remove the FileCacheModule (cachfile.dll) module from the list of installed modules in the
system.webServer/globalModules configuration section in applicationHost.config.
system.webServer/caching
 ATTRIBUTE DESCRIPTION DEFAULT

Enabled Disables the user-mode IIS cache when True

set to False. When the cache hit rate is
very small, you can disable the cache
completely to avoid the overhead that
is associated with the cache code path.
Disabling the user-mode cache does
not disable the kernel-mode cache.

enableKernelCache Disables the kernel-mode cache when True

set to False.

maxCacheSize Limits the IIS user-mode cache size to 0

the specified size in Megabytes. IIS
adjusts the default depending on
available memory. Choose the value
carefully based on the size of the set of
frequently accessed files versus the
amount of RAM or the IIS process
address space.

maxResponseSize Caches files up to the specified size. The 262144

actual value depends on the number
and size of the largest files in the data
set versus the available RAM. Caching
large, frequently requested files can
reduce CPU usage, disk access, and
associated latencies.

Compression behavior settings

IIS starting from 7.0 compresses static content by default. Also, compression of dynamic content is enabled by
default when the DynamicCompressionModule is installed. Compression reduces bandwidth usage but increases
CPU usage. Compressed content is cached in the kernel-mode cache if possible. Starting from 8.5, IIS lets
compression be controlled independently for static and dynamic content. Static content typically refers to content
that does not change, such as GIF or HTM files. Dynamic content is typically generated by scripts or code on the
server, that is, ASP.NET pages. You can customize the classification of any particular extension as static or dynamic.
To completely disable compression, remove StaticCompressionModule and DynamicCompressionModule from the
list of modules in the system.webServer/globalModules section in applicationHost.config.
system.webServer/httpCompression

ATTRIBUTE DESCRIPTION DEFAULT

staticCompression-EnableCpuUsage Enables or disables compression if the 50, 100, 50, and 90 respectively
current percentage CPU usage goes
staticCompression-DisableCpuUsage above or below specified limits.

dynamicCompression-EnableCpuUsage Starting with IIS 7.0, compression is

automatically disabled if steady-state
dynamicCompression-DisableCpuUsage CPU increases above the disable
threshold. Compression is enabled if
CPU drops below the enable threshold.
 ATTRIBUTE DESCRIPTION DEFAULT

directory Specifies the directory in which %SystemDrive%\inetpub\temp\IIS

compressed versions of static files are Temporary Compressed Files
temporarily stored and cached.
Consider moving this directory off the
system drive if it is accessed frequently.

doDiskSpaceLimiting Specifies whether a limit exists for how True

much disk space all compressed files can
occupy. Compressed files are stored in
the compression directory that is
specified by the directory attribute.

maxDiskSpaceUsage Specifies the number of bytes of disk 100 MB

space that compressed files can occupy
in the compression directory.

This setting might need to be increased

if the total size of all compressed
content is too large.

system.webServer/urlCompression

ATTRIBUTE DESCRIPTION DEFAULT

doStaticCompression Specifies whether static content is True

compressed.

doDynamicCompression Specifies whether dynamic content is True

compressed.

Note
For servers running IIS 10.0 that have low average CPU usage, consider enabling compression for dynamic content,
especially if responses are large. This should first be done in a test environment to assess the effect on the CPU
usage from the baseline.
Tuning the default document list
The default document module handles HTTP requests for the root of a directory and translates them into requests
for a specific file, such as Default.htm or Index.htm. On average, around 25 percent of all requests on the Internet
go through the default document path. This varies significantly for individual sites. When an HTTP request does not
specify a file name, the default document module searches the list of allowed default documents for each name in
the file system. This can adversely affect performance, especially if reaching the content requires making a network
round trip or touching a disk.
You can avoid the overhead by selectively disabling default documents and by reducing or ordering the list of
documents. For websites that use a default document, you should reduce the list to only the default document
types that are used. Additionally, order the list so that it begins with the most frequently accessed default document
file name.
You can selectively set the default document behavior on particular URLs by customizing the configuration inside a
location tag in applicationHost.config or by inserting a web.config file directly in the content directory. This allows a
hybrid approach, which enables default documents only where they are necessary and sets the list to the correct
file name for each URL.
To disable default documents completely, remove DefaultDocumentModule from the list of modules in the
system.webServer/globalModules section in applicationHost.config.
system.webServer/defaultDocument

ATTRIBUTE DESCRIPTION DEFAULT

enabled Specifies that default documents are True

enabled.

<files> element Specifies the file names that are The default list is Default.htm,
configured as default documents. Default.asp, Index.htm, Index.html,
Iisstart.htm, and Default.aspx.

Central binary logging

When the server session has numerous URL groups under it, the process of creating hundreds of formatted log
files for individual URL groups and writing the log data to a disk can quickly consume valuable CPU and memory
resources, thereby creating performance and scalability issues. Centralized binary logging minimizes the amount of
system resources that are used for logging, while at the same time providing detailed log data for organizations
that require it. Parsing binary-format logs requires a post-processing tool.
You can enable central binary logging by setting the centralLogFileMode attribute to CentralBinary and setting the
enabled attribute to True. Consider moving the location of the central log file off the system partition and onto a
dedicated logging drive to avoid contention between system activities and logging activities.
system.applicationHost/log

ATTRIBUTE DESCRIPTION DEFAULT

centralLogFileMode Specifies the logging mode for a server. Site

Change this value to CentralBinary to
enable central binary logging.

system.applicationHost/log/centralBinaryLogFile

ATTRIBUTE DESCRIPTION DEFAULT

enabled Specifies whether central binary logging False

is enabled.

directory Specifies the directory where log entries %SystemDrive%\inetpub\logs\LogFiles

are written.

Application and site tunings

The following settings relate to application pool and site tunings.
system.applicationHost/applicationPools/applicationPoolDefaults

ATTRIBUTE DESCRIPTION DEFAULT

 ATTRIBUTE DESCRIPTION DEFAULT

queueLength Indicates to HTTP.sys how many 1000

requests are queued for an application
pool before future requests are rejected.
When the value for this property is
exceeded, IIS rejects subsequent
requests with a 503 error.

Consider increasing this for applications

that communicate with high-latency
back-end data stores if 503 errors are
observed.

enable32BitAppOnWin64 When True, enables a 32-bit application False

to run on a computer that has a 64-bit
processor.

Consider enabling 32-bit mode if

memory consumption is a concern.
Because pointer sizes and instruction
sizes are smaller, 32-bit applications use
less memory than 64-bit applications.
The drawback to running 32-bit
applications on a 64-bit computer is
that user-mode address space is limited
to 4 GB.

system.applicationHost/sites/VirtualDirectoryDefault

ATTRIBUTE DESCRIPTION DEFAULT

allowSubDirConfig Specifies whether IIS looks for True

web.config files in content directories
lower than the current level (True) or
does not look for web.config files in
content directories lower than the
current level (False). By imposing a
simple limitation, which allows
configuration only in virtual directories,
IIS 10.0 can know that, unless
/<name>.htm is a virtual directory, it
should not look for a configuration file.
Skipping the additional file operations
can significantly improve performance
of websites that have a very large set of
randomly accessed static content.

Managing IIS 10.0 modules

IIS 10.0 has been factored into multiple, user-extensible modules to support a modular structure. This factorization
has a small cost. For each module the integrated pipeline must call the module for every event that is relevant to
the module. This happens regardless of whether the module must do any work. You can conserve CPU cycles and
memory by removing all modules that are not relevant to a particular website.
A web server that is tuned for simple static files might include only the following five modules: UriCacheModule,
HttpCacheModule, StaticFileModule, AnonymousAuthenticationModule, and HttpLoggingModule.
To remove modules from applicationHost.config, remove all references to the module from the
system.webServer/handlers and system.webServer/modules sections in addition to removing the module
declaration in system.webServer/globalModules.

Classic ASP settings

The major cost of processing a classic ASP request includes initializing a script engine, compiling the requested ASP
script into an ASP template, and executing the template on the script engine. While the template execution cost
depends on the complexity of the requested ASP script, IIS classic ASP module can cache script engines in memory
and cache templates in both memory and disk (only if in-memory template cache overflows) to boost performance
in CPU-bound scenarios.
The following settings are used to configure the classic ASP template cache and script engine cache, and they do
not affect ASP.NET settings.
system.webServer/asp/cache

ATTRIBUTE DESCRIPTION DEFAULT

diskTemplateCacheDirectory The name of the directory that ASP uses %SystemDrive%\inetpub\temp\ASP

to store compiled templates when the Compiled Templates
in-memory cache overflows.

Recommendation: Set to a directory

that is not heavily used, for example, a
drive that is not shared with the
operating system, IIS log, or other
frequently accessed content.

maxDiskTemplateCacheFiles Specifies the maximum number of 2000

compiled ASP templates that can be
cached on disk.

Recommendation: Set to the maximum

value of 0x7FFFFFFF.

scriptFileCacheSize This attribute specifies the maximum 500

number of compiled ASP templates that
can be cached in memory.

Recommendation: Set to at least as

many as the number of frequently-
requested ASP scripts served by an
application pool. If possible, set to as
many ASP templates as memory limits
allow.

scriptEngineCacheMax Specifies the maximum number of script 250

engines that will keep cached in
memory.

Recommendation: Set to at least as

many as the number of frequently-
requested ASP scripts served by an
application pool. If possible, set to as
many script engines as the memory
limit allows.

system.webServer/asp/limits
 ATTRIBUTE DESCRIPTION DEFAULT

processorThreadMax Specifies the maximum number of 25

worker threads per processor that ASP
can create. Increase if the current
setting is insufficient to handle the load,
which can cause errors when it is
serving requests or cause under-usage
of CPU resources.

system.webServer/asp/comPlus

ATTRIBUTE DESCRIPTION DEFAULT

executeInMta Set to True if errors or failures are False

detected while IIS is serving ASP
content. This can occur, for example,
when hosting multiple isolated sites in
which each site runs under its own
worker process. Errors are typically
reported from COM+ in the Event
Viewer. This setting enables the multi-
threaded apartment model in ASP.

ASP.NET concurrency setting

ASP.NET 3.5
By default, ASP.NET limits request concurrency to reduce steady-state memory consumption on the server. High
concurrency applications might need to adjust some settings to improve overall performance. You can change this
setting in aspnet.config file:

<system.web>
<applicationPool maxConcurrentRequestsPerCPU=5000/>
</system.web>

The following setting is useful to fully use resources on a system:

maxConcurrentRequestPerCpu Default value: 5000
This setting limits the maximum number of concurrently executing ASP.NET requests on a system. The
default value is conservative to reduce memory consumption of ASP.NET applications. Consider increasing
this limit on systems that run applications that perform long, synchronous I/O operations. Otherwise, users
can experience high latency because of queuing or request failures due to exceeding queue limits under a
high load when the default setting is used.
ASP.NET 4.6
Besides the maxConcurrentRequestPerCpu setting, ASP.NET 4.7 also provides settings to improve the performance
in the applications which heavily rely on asynchronous operation. The setting can be changed in aspnet.config file.

<system.web>
<applicationPool percentCpuLimit=90 percentCpuLimitMinActiveRequestPerCpu=100/>
</system.web>

percentCpuLimit Default value: 90 Asynchronous request has some scalability issues when a huge load
(beyond the hardware capabilities) is put on such scenario. The problem is due to the nature of allocation on
 asynchronous scenarios. In these conditions, allocation will happen when the asynchronous operation starts,
and it will be consumed when it completes. By that time, its very possible the objects have been moved to
generation 1 or 2 by GC. When this happens, increasing the load will show increase on request per second (rps)
until a point. Once we pass that point, the time spent in GC will start to become a problem and the rps will start
to dip, having a negative scaling effect. To fix the problem, when the cpu usage exceeds percentCpuLimit setting,
requests will be sent to the ASP.NET native queue.
percentCpuLimitMinActiveRequestPerCpu Default value: 100 CPU throttling(percentCpuLimit setting) is not
based on number of requests but on how expensive they are. As a result, there could be just a few CPU-
intensive requests causing a backup in the native queue with no way to empty it aside from incoming requests.
To solve this problme, percentCpuLimitMinActiveRequestPerCpu can be used to ensure a minimum number of
requests are being served before throttling kicks in.

Worker process and recycling options

You can configure options for recycling IIS worker processes and provide practical solutions to acute situations or
events without requiring intervention or resetting a service or computer. Such situations and events include
memory leaks, increasing memory load, or unresponsive or idle worker processes. Under ordinary conditions,
recycling options might not be needed and recycling can be turned off or the system can be configured to recycle
very infrequently.
You can enable process recycling for a particular application by adding attributes to the
recycling/periodicRestart element. The recycle event can be triggered by several events including memory
usage, a fixed number of requests, and a fixed time period. When a worker process is recycled, the queued and
executing requests are drained, and a new process is simultaneously started to service new requests. The
recycling/periodicRestart element is per-application, which means that each attribute in the following table is
partitioned on a per-application basis.
system.applicationHost/applicationPools/ApplicationPoolDefaults/recycling/periodicRestart

ATTRIBUTE DESCRIPTION DEFAULT

memory Enable process recycling if virtual 0

memory consumption exceeds the
specified limit in kilobytes. This is a
useful setting for 32-bit computers that
have a small, 2 GB address space. It can
help avoid failed requests due to out-
of-memory errors.

privateMemory Enable process recycling if private 0

memory allocations exceed a specified
limit in kilobytes.

requests Enable process recycling after a certain 0

number of requests.

time Enable process recycling after a specified 29:00:00

time period.

Dynamic worker-process page-out tuning

Starting in Windows Server 2012 R2, IIS offers the option of configuring worker process to suspend after they have
been idle for a while (in addition to the option of terminate, which existed since IIS 7).
The main purpose of both the idle worker process page-out and idle worker process termination features is to
conserve memory utilization on the server, since a site can consume a lot of memory even if its just sitting there,
listening. Depending on the technology used on the site (static content vs ASP.NET vs other frameworks), the
memory used can be anywhere from about 10 MB to hundreds of MBs, and this means that if your server is
configured with many sites, figuring out the most effective settings for your sites can dramatically improve
performance of both active and suspended sites.
Before we go into specifics, we must keep in mind that if there are no memory constraints, then its probably best
to simply set the sites to never suspend or terminate. After all, theres little value in terminating a worker process if
its the only one on the machine.
Note
In case the site runs unstable code, such as code with a memory leak, or otherwise unstable, setting the site to
terminate on idle can be a quick-and-dirty alternative to fixing the code bug. This isnt something we would
encourage, but in a crunch, it may be better to use this feature as a clean-up mechanism while a more permanent
solution is in the works.]
Another factor to consider is that if the site does use a lot of memory, then the suspension process itself takes a toll,
because the computer has to write the data used by the worker process to disk. If the worker process is using a
large chunk of memory, then suspending it might be more expensive than the cost of having to wait for it to start
back up.
To make the best of the worker process suspension feature, you need to review your sites in each application pool,
and decide which should be suspended, which should be terminated, and which should be active indefinitely. For
each action and each site, you need to figure out the ideal time-out period.
Ideally, the sites that you will configure for suspension or termination are those that have visitors every day, but not
enough to warrant keeping it active all the time. These are usually sites with around 20 unique visitors a day or less.
You can analyze the traffic patterns using the sites log files and calculate the average daily traffic.
Keep in mind that once a specific user connects to the site, they will typically stay on it for at least a while, making
additional requests, and so just counting daily requests may not accurately reflect the real traffic patterns. To get a
more accurate reading, you can also use a tool, such as Microsoft Excel, to calculate the average time between
requests. For example:

REQUEST URL REQUEST TIME DELTA

1 /SourceSilverLight/Geosourc 10:01
e.web/grosource.html

2 /SourceSilverLight/Geosourc 10:10 0:09

e.web/sliverlight.js

3 /SourceSilverLight/Geosourc 10:11 0:01

e.web/clientbin/geo/1.aspx

4 /lClientAccessPolicy.xml 10:12 0:01

5 / 10:23 0:11
SourceSilverLight/Geosource
webService/Service.asmx

6 / 11:50 1:27
SourceSilverLight/Geosource.
web/GeoSearchServer.
 REQUEST URL REQUEST TIME DELTA

7 /rest/Services/CachedService 12:50 1:00

s/Silverlight_load_la

8 /rest/Services/CachedService 12:51 0:01

s/Silverlight_basemap.

9 /rest/Services/DynamicServic 12:59 0:08

e/ Silverlight_basemap.

10 /rest/Services/CachedService 13:40 0:41

s/Ortho_2004_cache.as...

11 /rest/Services/CachedService 13:40 0:00

s/Ortho_2005_cache.js

12 /rest/Services/CachedService 13:41 0:01

s/OrthoBaseEngine.aspx

The hard part, though, is figuring out what setting to apply to make sense. In our case, the site gets a bunch of
requests from users, and the table above shows that a total of 4 unique sessions occurred in a period of 4 hours.
With the default settings for worker process suspension of the application pool, the site would be terminated after
the default timeout of 20 minutes, which means each of these users would experience the site spin-up cycle. This
makes it an ideal candidate for worker process suspension, because for most of the time, the site is idle, and so
suspending it would conserve resources, and allow the users to reach the site almost instantly.
A final, and very important note about this is that disk performance is crucial for this feature. Because the
suspension and wake-up process involve writing and reading large amount of data to the hard drive, we strongly
recommend using a fast disk for this. Solid State Drives (SSDs) are ideal and highly recommended for this, and you
should make sure that the Windows page file is stored on it (if the operating system itself is not installed on the
SSD, configure the operating system to move the page file to it).
Whether you use an SSD or not, we also recommend fixing the size of the page file to accommodate writing the
page-out data to it without file-resizing. Page-file resizing might happen when the operating system needs to store
data in the page file, because by default, Windows is configured to automatically adjust its size based on need. By
setting the size to a fixed one, you can prevent resizing and improve performance a lot.
To configure a pre-fixed page file size, you need to calculate its ideal size, which depends on how many sites you
will be suspending, and how much memory they consume. If the average is 200 MB for an active worker process
and you have 500 sites on the servers that will be suspending, then the page file should be at least (200 * 500) MB
over the base size of the page file (so base + 100 GB in our example).
Note
When sites are suspended, they will consume approximately 6 MB each, so in our case, memory usage if all sites
are suspended would be around 3 GB. In reality, though, youre probably never going to have them all suspended
at the same time.

Transport Layer Security tuning parameters

The use of Transport Layer Security (TLS) imposes additional CPU cost. The most expensive component of TLS is
the cost of establishing a session establishment because it involves a full handshake. Reconnection, encryption, and
decryption also add to the cost. For better TLS performance, do the following:
Enable HTTP keep-alives for TLS sessions. This eliminates the session establishment costs.
 Reuse sessions when appropriate, especially with non-keep-alive traffic.
Selectively apply encryption only to pages or parts of the site that need it, rather to the entire site.
Note
Larger keys provide more security, but they also use more CPU time.
All components might not need to be encrypted. However, mixing plain HTTP and HTTPS might result in a
pop-up warning that not all content on the page is secure.

Internet Server Application Programming Interface (ISAPI)

No special tuning parameters are needed for ISAPI applications. If you write a private ISAPI extension, make sure
that it is written for performance and resource use.

Managed code tuning guidelines

The integrated pipeline model in IIS 10.0 enables a high degree of flexibility and extensibility. Custom modules that
are implemented in native or managed code can be inserted into the pipeline, or they can replace existing modules.
Although this extensibility model offers convenience and simplicity, you should be careful before you insert new
managed modules that hook into global events. Adding a global managed module means that all requests,
including static file requests, must touch managed code. Custom modules are susceptible to events such as
garbage collection. In addition, custom modules add significant CPU cost due to marshaling data between native
and managed code. If possible, you should set preCondition to managedHandler for managed module.
To get better cold startup performance, make sure that you precompile the ASP.NET web site or leverage IIS
Application Initialization feature to warm up the application.
If session state is not needed, make sure that you turn it off for each page.
If there are many I/O bound operations, try to use asynchronous version of relevant APIs which will give you much
better scalability.
Also using Output Cache properly will also boost the performance of your web site.
When you run multiple hosts that contain ASP.NET scripts in isolated mode (one application pool per site), monitor
the memory usage. Make sure that the server has enough RAM for the expected number of concurrently running
application pools. Consider using multiple application domains instead of multiple isolated processes.

Other issues that affect IIS performance

The following issues can affect IIS performance:
Installation of filters that are not cache-aware
The installation of a filter that is not HTTP-cache-aware causes IIS to completely disable caching, which
results in poor performance. ISAPI filters that were written before IIS 6.0 can cause this behavior.
Common Gateway Interface (CGI) requests
For performance reasons, the use of CGI applications to serve requests is not recommended with IIS.
Frequently creating and deleting CGI processes involves significant overhead. Better alternatives include
using FastCGI, ISAPI application scripts and ASP or ASP.NET scripts. Isolation is available for each of these
options.

See also
Web Server performance tuning
HTTP 1.1/2 tuning
 Performance Tuning HTTP 1.1/2
4/24/2017 1 min to read Edit Online

HTTP/2 is meant to improve performance on the client side (e.g., page load time on a browser). On the server, it
may represent a slight increase in CPU cost. Whereas the server no longer requires a single TCP connection for
every request, some of that state will now be kept in the HTTP layer. Furthermore, HTTP/2 has header compression,
which represents additional CPU load.
Some situations require an HTTP/1.1 fallback (resetting the HTTP/2 connection and instead establishing a new
connection to use HTTP/1.1). In particular, TLS renegotiation and HTTP authentication (other than Basic and Digest)
require HTTP/1.1 fallback. Even though this adds overhead, these operations already imply some delay and so are
not particularly performance-sensitive.

See also
Web Server performance tuning
IIS 10.0 performance tuning
 Performance Tuning Cache and Memory Manager
4/24/2017 2 min to read Edit Online

By default, Windows caches file data that is read from disks and written to disks. This implies that read operations
read file data from an area in system memory, known as the system file cache, rather than from the physical disk.
Correspondingly, write operations write file data to the system file cache rather than to the disk, and this type of
cache is referred to as a write-back cache. Caching is managed per file object. Caching occurs under the direction of
the Cache Manager, which operates continuously while Windows is running.
File data in the system file cache is written to the disk at intervals determined by the operating system. Flushed
pages stay either in system cache working set (when FILE_FLAG_RANDOM_ACCESS is set and file handle wasnt
closed) or on the standby list where these become part of available memory.
The policy of delaying the writing of the data to the file and holding it in the cache until the cache is flushed is called
lazy writing, and it is triggered by the Cache Manager at a determinate time interval. The time at which a block of
file data is flushed is partially based on the amount of time it has been stored in the cache and the amount of time
since the data was last accessed in a read operation. This ensures that file data that is frequently read will stay
accessible in the system file cache for the maximum amount of time.
This file data caching process is illustrated in the following figure:

As depicted by the solid arrows in the preceding figure, a 256 KB region of data is read into a 256 KB cache slot in
system address space when it is first requested by the Cache Manager during a file read operation. A user-mode
process then copies the data in this slot to its own address space. When the process has completed its data access,
it writes the altered data back to the same slot in the system cache, as shown by the dotted arrow between the
process address space and the system cache. When the Cache Manager has determined that the data will no longer
be needed for a certain amount of time, it writes the altered data back to the file on the disk, as shown by the dotted
arrow between the system cache and the disk.
In this section:
Cache and Memory Manager Potential Performance Issues
Cache and Memory Manager Improvements in Windows Server 2016
 Troubleshoot Cache and Memory Manager
Performance Issues
4/24/2017 2 min to read Edit Online

Before Windows Server 2012, two primary potential issues caused system file cache to grow until available
memory was almost depleted under certain workloads. When this situation results in the system being sluggish,
you can determine whether the server is facing one of these issues.

Counters to monitor
Memory\Long-Term Average Standby Cache Lifetime (s) < 1800 seconds
Memory\Available Mbytes is low
Memory\System Cache Resident Bytes
If Memory\Available Mbytes is low and at the same time Memory\System Cache Resident Bytes is consuming
significant part of the physical memory, you can use RAMMAP to find out what the cache is being used for.

System file cache contains NTFS metafile data structures

This problem is indicated by a very high number of active Metafile pages in RAMMAP output, as shown in the
following figure. This problem might have been observed on busy servers with millions of files being accessed,
thereby resulting in caching NTFS metafile data not being released from the cache.

The problem used to be mitigated by DynCache tool. In Windows Server 2012+, the architecture has been
redesigned and this problem should no longer exist.

System file cache contains memory mapped files

This problem is indicated by very high number of active Mapped file pages in RAMMAP output. This usually
indicates that some application on the server is opening a lot of large files using CreateFile API with
FILE_FLAG_RANDOM_ACCESS flag set.
This issue is described in detail in KB article 2549369. FILE_FLAG_RANDOM_ACCESS flag is a hint for Cache
Manager to keep mapped views of the file in memory as long as possible (until Memory Manager doesnt signal
low memory condition). At the same time, this flag instructs Cache Manager to disable prefetching of file data.
This situation has been mitigated to some extent by working set trimming improvements in Windows Server
2012+, but the issue itself needs to be primarily addressed by the application vendor by not using
FILE_FLAG_RANDOM_ACCESS. An alternative solution for the app vendor might be to use low memory priority
when accessing the files. This can be achieved using the SetThreadInformation API. Pages that are accessed at low
memory priority are removed from the working set more aggressively.
Cache Manager, starting in Windows Server 2016 further mitigates this by ignoring FILE_FLAG_RANDOM_ACCESS
when making trimming decisions, so it is treated just like any other file opened without the
FILE_FLAG_RANDOM_ACCESS flag (Cache Manager still honors this flag to disable prefetching of file data). You can
still cause system cache bloat if you have large number of files opened with this flag and accessed in truly random
fashion. It is highly recommended that FILE_FLAG_RANDOM_ACCESS not be used by applications.
 Cache and Memory Manager Improvements
4/24/2017 1 min to read Edit Online

This topic describes Cache Manager and Memory Manager improvements in Windows Server 2012 and 2016.

Cache Manager improvements in in Windows Server 2016

Cache Manager also added support for true Asynchronous Cached Reads. This could potentially improve the
performance of an application if it relies heavily on asynchronous cached reads. While most in-box filesystems have
supported async-cached reads for a while, there were often performance limitations due to various design choices
related to handling of thread-pools and filesystems' internal work queues. With support from kernel-proper, Cache
Manager now hides all the thread-pool and work queue management complexities from filesystems making it
more efficient at handling asynchronous cached reads.Cache Manager has one set of control datastructures for
each of (system supported maximum) VHD-nesting levels to maximize parallelism.

Cache Manager improvements in Windows Server 2012

In addition to Cache Manager enhancements to read ahead logic for sequential workloads, a new API
CcSetReadAheadGranularityEx was added to let file system drivers, such as SMB, change their read ahead
parameters. It allows better throughput for remote file scenarios by sending multiple small-sized read ahead
requests instead of sending a single large read ahead request. Only kernel components, such as file system drivers,
can programmatically configure these values on a per-file basis.

Memory Manager improvements in Windows Server 2012

Enabling page combining may reduce memory usage on servers which have a lot of private, pageable pages with
identical contents. For example, servers running multiple instances of the same memory-intensive app, or a single
app that works with highly repetitive data, might be good candidates to try page combining. The downside of
enabling page combining is increased CPU usage.
Here are some examples of server roles where page combining is unlikely to give much benefit:
File servers (most of the memory is consumed by file pages which are not private and therefore not
combinable)
Microsoft SQL Servers that are configured to use AWE or large pages (most of the memory is private but
non-pageable)
Page combining is disabled by default but can be enabled by using the Enable-MMAgent Windows PowerShell
cmdlet. Page combining was added in Windows Server 2012.
 Network Subsystem Performance Tuning
4/24/2017 1 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic for an overview of the network subsystem and for links to other topics in this guide.

NOTE
In addition to this topic, the following sections of this guide provide performance tuning recommendations for network
devices and the network stack.
Choosing a Network Adapter
Configure the Order of Network Interfaces
Performance Tuning Network Adapters
Network-Related Performance Counters
Performance Tools for Network Workloads

Performance tuning the network subsystem, particularly for network intensive workloads, can involve each layer
of the network architecture, which is also called the network stack. These layers are broadly divided into the
following sections.
1. Network interface. This is the lowest layer in the network stack, and contains the network driver that
communicates directly with the network adapter.
2. Network Driver Interface Specification (NDIS). NDIS exposes interfaces for the driver below it and for
the layers above it, such as the Protocol Stack.
3. Protocol Stack. The protocol stack implements protocols such as TCP/IP and UDP/IP. These layers expose
the transport layer interface for layers above them.
4. System Drivers. These are typically clients that use a transport data extension (TDX) or Winsock Kernel
(WSK) interface to expose interfaces to user-mode applications. The WSK interface was introduced in
Windows Server 2008 and Windows Vista, and it is exposed by AFD.sys. The interface improves
performance by eliminating the switching between user mode and kernel mode.
5. User-Mode Applications. These are typically Microsoft solutions or custom applications.
The table below provides a vertical illustration of the layers of the network stack, including examples of items that
run in each layer.
 Choosing a Network Adapter
4/24/2017 10 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic to learn some of the features of network adapters that might affect your purchasing choices.
Network-intensive applications require high-performance network adapters. This section explores some
considerations for choosing network adapters, as well as how to configure different network adapter settings to
achieve the best network performance.

TIP
You can configure network adapter settings by using Windows PowerShell. For more information, see Network Adapter
Cmdlets in Windows PowerShell.

Offload Capabilities
Offloading tasks from the central processing unit (CPU) to the network adapter can reduce CPU usage on the
server, which improves the overall system performance.
The network stack in Microsoft products can offload one or more tasks to a network adapter if you select a network
adapter that has the appropriate offload capabilities. The following table provides a brief overview of different
offload capabilities that are available in Windows Server 2016.

OFFLOAD TYPE DESCRIPTION

Checksum calculation for TCP The network stack can offload the calculation and validation of
Transmission Control Protocol (TCP) checksums on send and
receive code paths. It can also offload the calculation and
validation of IPv4 and IPv6 checksums on send and receive
code paths.

Checksum calculation for UDP The network stack can offload the calculation and validation of
User Datagram Protocol (UDP) checksums on send and
receive code paths.

Checksum calculation for IPv4 The network stack can offload the calculation and validation of
IPv4 checksums on send and receive code paths.

Checksum calculation for IPv6 The network stack can offload the calculation and validation of
IPv6 checksums on send and receive code paths.

Segmentation of large TCP packets The TCP/IP transport layer supports Large Send Offload v2
(LSOv2). With LSOv2, the TCP/IP transport layer can offload
the segmentation of large TCP packets to the network
adapter.
 OFFLOAD TYPE DESCRIPTION

Receive Side Scaling (RSS) RSS is a network driver technology that enables the efficient
distribution of network receive processing across multiple
CPUs in multiprocessor systems. More detail about RSS is
provided later in this topic.

Receive Segment Coalescing (RSC) RSC is the ability to group packets together to minimize the
header processing that is necessary for the host to perform. A
maximum of 64 KB of received payload can be coalesced into a
single larger packet for processing. More detail about RSC is
provided later in this topic.

Receive Side Scaling

Windows Server 2016, Windows Server 2012, Windows Server 2012 R2, Windows Server 2008 R2, and Windows
Server 2008 support Receive Side Scaling (RSS).
Some servers are configured with multiple logical processors that share hardware resources (such as a physical
core) and which are treated as Simultaneous Multi-Threading (SMT) peers. Intel Hyper-Threading Technology is an
example. RSS directs network processing to up to one logical processor per core. For example, on a server with Intel
Hyper-Threading, 4 cores, and 8 logical processors, RSS uses no more than 4 logical processors for network
processing.
RSS distributes incoming network I/O packets among logical processors so that packets which belong to the same
TCP connection are processed on the same logical processor, which preserves ordering.
RSS also load balances UDP unicast and multicast traffic, and it routes related flows (which are determined by
hashing the source and destination addresses) to the same logical processor, preserving the order of related
arrivals. This helps improve scalability and performance for receive-intensive scenarios for servers that have fewer
network adapters than they do eligible logical processors.
Configuring RSS
In Windows Server 2016, you can configure RSS by using Windows PowerShell cmdlets and RSS profiles.
You can define RSS profiles by using the Profile parameter of the Set-NetAdapterRss Windows PowerShell
cmdlet.
Windows PowerShell commands for RSS configuration
The following cmdlets allow you to see and modify RSS parameters per network adapter.

NOTE
For a detailed command reference for each cmdlet, including syntax and parameters, you can click the following links. In
addition, you can pass the cmdlet name to Get-Help at the Windows PowerShell prompt for details on each command.

Disable-NetAdapterRss. This command disables RSS on the network adapter that you specify.
Enable-NetAdapterRss. This command enables RSS on the network adapter that you specify.
Get-NetAdapterRss. This command retrieves RSS properties of the network adapter that you specify.
Set-NetAdapterRss. This command sets the RSS properties on the network adapter that you specify.
RSS profiles
You can use the Profile parameter of the Set-NetAdapterRss cmdlet to specify which logical processors are
assigned to which network adapter. Available values for this parameter are:
 Closest. Logical processor numbers that are near the network adapters base RSS processor are preferred.
With this profile, the operating system might rebalance logical processors dynamically based on load.
ClosestStatic. Logical processor numbers near the network adapters base RSS processor are preferred.
With this profile, the operating system does not rebalance logical processors dynamically based on load.
NUMA. Logical processor numbers are generally selected on different NUMA nodes to distribute the load.
With this profile, the operating system might rebalance logical processors dynamically based on load.
NUMAStatic. This is the default profile. Logical processor numbers are generally selected on different
NUMA nodes to distribute the load. With this profile, the operating system will not rebalance logical
processors dynamically based on load.
Conservative. RSS uses as few processors as possible to sustain the load. This option helps reduce the
number of interrupts.
Depending on the scenario and the workload characteristics, you can also use other parameters of the Set-
NetAdapterRss Windows PowerShell cmdlet to specify the following:
On a per-network adapter basis, how many logical processors can be used for RSS.
The starting offset for the range of logical processors.
The node from which the network adapter allocates memory.
Following are the additional Set-NetAdapterRss parameters that you can use to configure RSS:

NOTE
In the example syntax for each parameter below, the network adapter name Ethernet is used as an example value for the
Name parameter of the Set-NetAdapterRss command. When you run the cmdlet, ensure that the network adapter name
that you use is appropriate for your environment.

* MaxProcessors: Sets the maximum number of RSS processors to be used. This ensures that application
traffic is bound to a maximum number of processors on a given interface. Example syntax:
Set-NetAdapterRss Name Ethernet MaxProcessors <value>

* BaseProcessorGroup: Sets the base processor group of a NUMA node. This impacts the processor array
that is used by RSS. Example syntax:
Set-NetAdapterRss Name Ethernet BaseProcessorGroup <value>

* MaxProcessorGroup: Sets the Max processor group of a NUMA node. This impacts the processor array
that is used by RSS. Setting this would restrict a maximum processor group so that load balancing is aligned
within a k-group. Example syntax:
Set-NetAdapterRss Name Ethernet MaxProcessorGroup <value>

* BaseProcessorNumber: Sets the base processor number of a NUMA node. This impacts the processor
array that is used by RSS. This allows partitioning processors across network adapters. This is the first logical
processor in the range of RSS processors that is assigned to each adapter. Example syntax:
Set-NetAdapterRss Name Ethernet BaseProcessorNumber <Byte Value>

* NumaNode: The NUMA node that each network adapter can allocate memory from. This can be within a
k-group or from different k-groups. Example syntax:
Set-NetAdapterRss Name Ethernet NumaNodeID <value>

* NumberofReceiveQueues: If your logical processors seem to be underutilized for receive traffic (for
 example, as viewed in Task Manager), you can try increasing the number of RSS queues from the default of
2 to the maximum that is supported by your network adapter. Your network adapter may have options to
change the number of RSS queues as part of the driver. Example syntax:
Set-NetAdapterRss Name Ethernet NumberOfReceiveQueues <value>

For more information, click the following link to download Scalable Networking: Eliminating the Receive Processing
BottleneckIntroducing RSS in Word format.
Understanding RSS Performance
Tuning RSS requires understanding the configuration and the load-balancing logic. To verify that the RSS settings
have taken effect, you can review the output when you run the Get-NetAdapterRss Windows PowerShell cmdlet.
Following is example output of this cmdlet.

PS C:\Users\Administrator> get-netadapterrss
Name : testnic 2
InterfaceDescription : Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #66
Enabled : True
NumberOfReceiveQueues : 2
Profile : NUMAStatic
BaseProcessor: [Group:Number] : 0:0
MaxProcessor: [Group:Number] : 0:15
MaxProcessors : 8

IndirectionTable: [Group:Number]:
0:0 0:4 0:0 0:4 0:0 0:4 0:0 0:4

(# indirection table entries are a power of 2 and based on # of processors)

0:0 0:4 0:0 0:4 0:0 0:4 0:0 0:4

In addition to echoing parameters that were set, the key aspect of the output is the indirection table output. The
indirection table displays the hash table buckets that are used to distribute incoming traffic. In this example, the n:c
notation designates the Numa K-Group:CPU index pair that is used to direct incoming traffic. We see exactly 2
unique entries (0:0 and 0:4), which represent k-group 0/cpu0 and k-group 0/cpu 4, respectively.
There is only one k-group for this system (k-group 0) and a n (where n <= 128) indirection table entry. Because the
number of receive queues is set to 2, only 2 processors (0:0, 0:4) are chosen - even though maximum processors is
set to 8. In effect, the indirection table is hashing incoming traffic to only use 2 CPUs out of the 8 that are available.
To fully utilize the CPUs, the number of RSS Receive Queues must be equal to or greater than Max Processors. In
the previous example, the Receive Queue should be set to 8 or greater.
NIC Teaming and RSS
RSS can be enabled on a network adapter that is teamed with another network interface card using NIC Teaming. In
this scenario, only the underlying physical network adapter can be configured to use RSS. A user cannot set RSS
cmdlets on the teamed network adapter.
Receive Segment Coalescing (RSC )
Receive Segment Coalescing (RSC) helps performance by reducing the number of IP headers that are processed for
a given amount of received data. It should be used to help scale the performance of received data by grouping (or
coalescing) the smaller packets into larger units.
This approach can affect latency with benefits mostly seen in throughput gains. RSC is recommended to increase
throughput for received heavy workloads. Consider deploying network adapters that support RSC.
On these network adapters, ensure that RSC is on (this is the default setting), unless you have specific workloads
(for example, low latency, low throughput networking) that show benefit from RSC being off.
Understanding RSC Diagnostics
You can diagnose RSC by using the Windows PowerShell cmdlets Get-NetAdapterRsc and Get-
NetAdapterStatistics.
Following is example output when you run the Get-NetAdapterRsc cmdlet.

PS C:\Users\Administrator> Get-NetAdapterRsc

Name IPv4Enabled IPv6Enabled IPv4Operational IPv6Operational

IPv4FailureReason IPv6Failure
Reason
---- ----------- ----------- --------------- --------------- ----------------- ----
--------
Ethernet True False True False NoFailure
NicProperties

The Get cmdlet shows whether RSC is enabled in the interface and whether TCP enables RSC to be in an
operational state. The failure reason provides details about the failure to enable RSC on that interface.
In the previous scenario, IPv4 RSC is supported and operational in the interface. To understand diagnostic failures,
one can see the coalesced bytes or exceptions caused. This provides an indication of the coalescing issues.
Following is example output when you run the Get-NetAdapterStatistics cmdlet.

PS C:\Users\Administrator> $x = Get-NetAdapterStatistics myAdapter

PS C:\Users\Administrator> $x.rscstatistics

CoalescedBytes : 0
CoalescedPackets : 0
CoalescingEvents : 0
CoalescingExceptions : 0

RSC and Virtualization

RSC is only supported in the physical host when the host network adapter is not bound to the Hyper-V Virtual
Switch. RSC is disabled by the operating system when the host is bound to the Hyper-V Virtual Switch. In addition,
virtual machines do not get the benefit of RSC because virtual network adapters do not support RSC.
RSC can be enabled for a virtual machine when Single Root Input/Output Virtualization (SR-IOV) is enabled. In this
case, virtual functions support RSC capability; hence, virtual machines also receive the benefit of RSC.

Network Adapter Resources

A few network adapters actively manage their resources to achieve optimum performance. Several network
adapters allow you to manually configure resources by using the Advanced Networking tab for the adapter. For
such adapters, you can set the values of a number of parameters, including the number of receive buffers and send
buffers.
Configuring network adapter resources is simplified by the use of the following Windows PowerShell cmdlets.
Get-NetAdapterAdvancedProperty
Set-NetAdapterAdvancedProperty
Enable-NetAdapter
Enable-NetAdapterBinding
Enable-NetAdapterChecksumOffload
 Enable-NetAdapterIPSecOffload
Enable-NetAdapterLso
Enable-NetAdapterPowerManagement
Enable-NetAdapterQos
Enable-NetAdapterRDMA
Enable-NetAdapterSriov
For more information, see Network Adapter Cmdlets in Windows PowerShell.
For links to all topics in this guide, see Network Subsystem Performance Tuning.
 Configure the Order of Network Interfaces
4/24/2017 1 min to read Edit Online

Applies To: Windows Server 2016

In Windows Server 2016 and Windows 10, you can use the interface metric to configure the order of network
interfaces.
This is different than in previous versions of Windows and Windows Server, which allowed you to configure the
binding order of network adapters by using either the user interface or the commands
INetCfgComponentBindings::MoveBefore and INetCfgComponentBindings::MoveAfter. These two
methods for ordering network interfaces are not available in Windows Server 2016 and Windows 10.
Instead, you can use the new method for setting the enumerated order of network adapters by configuring the
interface metric of each adapter. You can configure the interface metric by using the Set-NetIPInterface Windows
PowerShell command.
When network traffic routes are chosen and you have configured the InterfaceMetric parameter of the Set-
NetIPInterface command, the overall metric that is used to determine the interface preference is the sum of the
route metric and the interface metric. Typically, the interface metric gives preference to a particular interface, such
as using wired if both wired and wireless are available.
The following Windows PowerShell command example shows use of this parameter.

Set-NetIPInterface -InterfaceIndex 12 -InterfaceMetric 15

The order in which adapters appear in a list is determined by the IPv4 or IPv6 interface metric. For more
information, see GetAdaptersAddresses function.
For links to all topics in this guide, see Network Subsystem Performance Tuning.
 Performance Tuning Network Adapters
4/24/2017 7 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic to performance tune network adapters that are installed in computers that are running
Windows Server 2016.
Determining the correct tuning settings for your network adapter depend on the following variables:
The network adapter and its feature set
The type of workload performed by the server
The server hardware and software resources
Your performance goals for the server
If your network adapter provides tuning options, you can optimize network throughput and resource usage to
achieve optimum throughput based on the parameters described above.
The following sections describe some of your performance tuning options.

Enabling Offload Features

Turning on network adapter offload features is usually beneficial. Sometimes, however, the network adapter is not
powerful enough to handle the offload capabilities with high throughput.

IMPORTANT
Do not use the offload features IPsec Task Offload or TCP Chimney Offload. These technologies are deprecated in
Windows Server 2016, and might adversely affect server and networking performance. In addition, these technologies might
not be supported by Microsoft in the future.

For example, enabling segmentation offload can reduce the maximum sustainable throughput on some network
adapters because of limited hardware resources. However, if the reduced throughput is not expected to be a
limitation, you should enable offload capabilities, even for this type of network adapter.

NOTE
Some network adapters require offload features to be independently enabled for send and receive paths.

Enabling Receive Side Scaling (RSS) for Web Servers

RSS can improve web scalability and performance when there are fewer network adapters than logical processors
on the server. When all the web traffic is going through the RSS-capable network adapters, incoming web requests
from different connections can be simultaneously processed across different CPUs.
It is important to note that due to the logic in RSS and Hypertext Transfer Protocol (HTTP) for load distribution,
performance might be severely degraded if a non-RSS-capable network adapter accepts web traffic on a server that
has one or more RSS-capable network adapters. In this circumstance, you should use RSS-capable network
adapters or disable RSS on the network adapter properties Advanced Properties tab. To determine whether a
network adapter is RSS-capable, you can view the RSS information on the network adapter properties Advanced
Properties tab.
RSS Profiles and RSS Queues
The default RSS predefined profile is NUMA Static, which changes the default behavior from previous versions of
the operating system. To get started with RSS Profiles, you can review the available profiles to understand when
they are beneficial and how they apply to your network environment and hardware.
For example, if you open Task Manager and review the logical processors on your server, and they seem to be
underutilized for receive traffic, you can try increasing the number of RSS queues from the default of 2 to the
maximum that is supported by your network adapter. Your network adapter might have options to change the
number of RSS queues as part of the driver.

Increasing Network Adapter Resources

For network adapters that allow manual configuration of resources, such as receive and send buffers, you should
increase the allocated resources.
Some network adapters set their receive buffers low to conserve allocated memory from the host. The low value
results in dropped packets and decreased performance. Therefore, for receive-intensive scenarios, we recommend
that you increase the receive buffer value to the maximum.

NOTE
If a network adapter does not expose manual resource configuration, it either dynamically configures the resources, or the
resources are set to a fixed value that cannot be changed.

Enabling Interrupt Moderation

To control interrupt moderation, some network adapters expose different interrupt moderation levels, buffer
coalescing parameters (sometimes separately for send and receive buffers), or both.
You should consider interrupt moderation for CPU-bound workloads, and consider the trade-off between the host
CPU savings and latency versus the increased host CPU savings because of more interrupts and less latency. If the
network adapter does not perform interrupt moderation, but it does expose buffer coalescing, increasing the
number of coalesced buffers allows more buffers per send or receive, which improves performance.

Performance Tuning for Low Latency Packet Processing

Many network adapters provide options to optimize operating system-induced latency. Latency is the elapsed time
between the network driver processing an incoming packet and the network driver sending the packet back. This
time is usually measured in microseconds. For comparison, the transmission time for packet transmissions over
long distances is usually measured in milliseconds (an order of magnitude larger). This tuning will not reduce the
time a packet spends in transit.
Following are some performance tuning suggestions for microsecond-sensitive networks.
Set the computer BIOS to High Performance, with C-states disabled. However, note that this is system and
BIOS dependent, and some systems will provide higher performance if the operating system controls power
management. You can check and adjust your power management settings from Settings or by using the
powercfg command. For more information, see Powercfg Command-Line Options
Set the operating system power management profile to High Performance System. Note that this will not
work properly if the system BIOS has been set to disable operating system control of power management.
Enable Static Offloads, for example, UDP Checksums, TCP Checksums, and Send Large Offload (LSO).
 Enable RSS if the traffic is multi-streamed, such as high-volume multicast receive.
Disable the Interrupt Moderation setting for network card drivers that require the lowest possible latency.
Remember, this can use more CPU time and it represents a tradeoff.
Handle network adapter interrupts and DPCs on a core processor that shares CPU cache with the core that is
being used by the program (user thread) that is handling the packet. CPU affinity tuning can be used to
direct a process to certain logical processors in conjunction with RSS configuration to accomplish this. Using
the same core for the interrupt, DPC, and user mode thread exhibits worse performance as load increases
because the ISR, DPC, and thread contend for the use of the core.

System Management Interrupts

Many hardware systems use System Management Interrupts (SMI) for a variety of maintenance functions,
including reporting of error correction code (ECC) memory errors, legacy USB compatibility, fan control, and BIOS
controlled power management.
The SMI is the highest priority interrupt on the system and places the CPU in a management mode, which preempts
all other activity while it runs an interrupt service routine, typically contained in BIOS.
Unfortunately, this can result in latency spikes of 100 microseconds or more.
If you need to achieve the lowest latency, you should request a BIOS version from your hardware provider that
reduces SMIs to the lowest degree possible. These are frequently referred to as low latency BIOS or SMI free
BIOS. In some cases, it is not possible for a hardware platform to eliminate SMI activity altogether because it is
used to control essential functions (for example, cooling fans).

NOTE
The operating system can exert no control over SMIs because the logical processor is running in a special maintenance mode,
which prevents operating system intervention.

Performance Tuning TCP

You can performance tune TCP using the following items.
TCP Receive Window Auto -Tuning
Prior to Windows Server 2008, the network stack used a fixed-size receive-side window that limited the overall
potential throughput for connections. One of the most significant changes to the TCP stack is TCP receive window
auto-tuning.
You can calculate the total throughput of a single connection when you use this fixed size default as:
Total achievable throughput in bytes = TCP window * (1 / connection latency)
For example, the total achievable throughput is only 51 Mbps on a 1 GB connection with 10 ms latency (a
reasonable value for a large corporate network infrastructure).
With auto-tuning, however, the receive-side window is adjustable, and it can grow to meet the demands of the
sender. It is possible for a connection to achieve a full line rate of a 1 GB connection. Network usage scenarios that
might have been limited in the past by the total achievable throughput of TCP connections can now fully use the
network.
Deprecated TCP parameters
The following registry settings from Windows Server 2003 are no longer supported, and are ignored in later
versions.
All of these settings had the following registry location:

```
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters
```

TcpWindowSize
NumTcbTablePartitions
MaxHashTableSize
Windows Filtering Platform
The Windows Filtering Platform (WFP) that was introduced in Windows Vista and Windows Server 2008 provides
APIs to non-Microsoft independent software vendors (ISVs) to create packet processing filters. Examples include
firewall and antivirus software.

NOTE
A poorly written WFP filter can significantly decrease a servers networking performance. For more information, see Porting
Packet-Processing Drivers and Apps to WFP in the Windows Dev Center.

For links to all topics in this guide, see Network Subsystem Performance Tuning.
 Network-Related Performance Counters
4/24/2017 1 min to read Edit Online

Applies To: Windows Server 2016

This topic lists the counters that are relevant to managing network performance, and contains the following
sections.
Resource Utilization
Potential Network Problems
Receive Side Coalescing (RSC) performance

Resource Utilization
The following performance counters are relevant to network resource utilization.
IPv4, IPv6
Datagrams Received/sec
Datagrams Sent/sec
TCPv4, TCPv6
Segments Received/sec
Segments Sent/sec
Segments Retransmitted/sec
Network Interface(), Network Adapter(\)
Bytes Received/sec
Bytes Sent/sec
Packets Received/sec
Packets Sent/sec
Output Queue Length
This counter is the length of the output packet queue (in packets). If this is longer than 2, delays occur.
You should find the bottleneck and eliminate it if you can. Because NDIS queues the requests, this
length should always be 0.
Processor Information
% Processor Time
Interrupts/sec
DPCs Queued/sec
This counter is an average rate at which DPCs were added to the logical processor's DPC queue. Each
logical processor has its own DPC queue. This counter measures the rate at which DPCs are added to
 the queue, not the number of DPCs in the queue. It displays the difference between the values that
were observed in the last two samples, divided by the duration of the sample interval.

Potential Network Problems

The following performance counters are relevant to potential network problems.
Network Interface(), Network Adapter(\)
Packets Received Discarded
Packets Received Errors
Packets Outbound Discarded
Packets Outbound Errors
WFPv4, WFPv6
Packets Discarded/sec
UDPv4, UDPv6
Datagrams Received Errors
TCPv4, TCPv6
Connection Failures
Connections Reset
Network QoS Policy
Packets dropped
Packets dropped/sec
Per Processor Network Interface Card Activity
Low Resource Receive Indications/sec
Low Resource Received Packets/sec
Microsoft Winsock BSP
Dropped Datagrams
Dropped Datagrams/sec
Rejected Connections
Rejected Connections/sec

Receive Side Coalescing (RSC) performance

The following performance counters are relevant to RSC performance.
Network Adapter(*)
TCP Active RSC Connections
TCP RSC Average Packet Size
TCP RSC Coalesced Packets/sec
TCP RSC Exceptions/sec
For links to all topics in this guide, see Network Subsystem Performance Tuning.
 Performance Tools for Network Workloads
4/24/2017 1 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic to learn about performance tools.

This topic contains sections about the Client to Server Traffic tool, TCP/IP Window Size, and Microsoft Server
Performance Advisor.

Client to Server Traffic tool

The Client to Server Traffic (ctsTraffic) tool provides you with the ability to create and verify network traffic.
For more information, and to download the tool, see ctsTraffic (Client-To-Server Traffic).

TCP/IP Window Size

For 1 GB adapters, the settings shown in the previous table should provide good throughput because NTttcp sets
the default TCP window size to 64 K through a specific logical processor option (SO_RCVBUF) for the connection.
This provides good performance on a low-latency network.
In contrast, for high-latency networks or for 10 GB adapters, the default TCP window size value for NTttcp yields
less than optimal performance. In both cases, you must adjust the TCP window size to allow for the larger
bandwidth delay product.
You can statically set the TCP window size to a large value by using the -rb option. This option disables TCP
Window Auto-Tuning, and we recommend using it only if the user fully understands the resultant change in TCP/IP
behavior. By default, the TCP window size is set at a sufficient value and adjusts only under heavy load or over
high-latency links.

Microsoft Server Performance Advisor

Microsoft Server Performance Advisor (SPA) helps IT administrators collect metrics to identify, compare, and
diagnose potential performance issues in a Windows Server 2016, Windows Server 2012 R2, Windows Server
2012, Windows Server 2008 R2, or Windows Server 2008 deployment.
SPA generates comprehensive diagnostic reports and charts, and it provides recommendations to help you quickly
analyze issues and develop corrective actions.
For more information and to download the advisor, see Microsoft Server Performance Advisor in the Windows
Hardware Dev Center.
For links to all topics in this guide, see Network Subsystem Performance Tuning.
 NIC Teaming
4/24/2017 7 min to read Edit Online

Applies To: Windows Server 2016

This topic provides an overview of Network Interface Card (NIC) Teaming in Windows Server 2016.

NOTE
In addition to this topic, the following NIC Teaming content is available.
NIC Teaming in Virtual Machines (VMs)
NIC Teaming and Virtual Local Area Networks (VLANs)
NIC Teaming MAC Address Use and Management
Troubleshooting NIC Teaming
Create a New NIC Team on a Host Computer or VM
NIC Teaming (NetLBFO) Cmdlets in Windows PowerShell
TechNet Gallery Download: Windows Server 2016 NIC and Switch Embedded Teaming User Guide

NIC Teaming Overview

NIC Teaming allows you to group between one and thirty-two physical Ethernet network adapters into one or
more software-based virtual network adapters. These virtual network adapters provide fast performance and fault
tolerance in the event of a network adapter failure.
NIC Team member network adapters must all be installed in the same physical host computer to be placed in a
team.

NOTE
A NIC team that contains only one network adapter cannot provide load balancing and failover; however with one network
adapter, you can use NIC Teaming for separation of network traffic when you are also using virtual Local Area Networks
(VLANs).

When you configure network adapters into a NIC team, they are connected into the NIC teaming solution common
core, which then presents one or more virtual adapters (also called team NICs [tNICs] or team interfaces) to the
operating system. Windows Server 2016 supports up to 32 team interfaces per team. There are a variety of
algorithms that distribute outbound traffic (load) between the NICs.
The following illustration depicts a NIC Team with multiple tNICs.
In addition, you can connect your teamed NICs to the same switch or to different switches. If you connect NICs to
different switches, both switches must be on the same subnet.

NIC Teaming Availability

NIC Teaming is available in all versions of Windows Server 2016. In addition, you can use Windows PowerShell
commands, Remote Desktop, and Remote Server Administration Tools to manage NIC Teaming from computers
that are running a client operating system upon which the tools are supported.

Supported and Unsupported NICs for NIC Teaming

You can use any Ethernet NIC that has passed the Windows Hardware Qualification and Logo test (WHQL tests) in
a NIC Team in Windows Server 2016.
The following NICs cannot be placed in a NIC team.
Hyper-V virtual network adapters that are Hyper-V Virtual Switch ports exposed as NICs in the host
partition.

IMPORTANT
Hyper-V virtual NICs that are exposed in the host partition (vNICs) must not be placed in a team. Teaming of vNICs
inside of the host partition is not supported in any configuration or combination. Attempts to team vNICs might
cause a complete loss of communication if network failures occur.

The kernel debug network adapter (KDNIC).

NICs that are being used for network boot.
NICs that use technologies other than Ethernet, such as WWAN, WLAN/Wi-Fi, Bluetooth, and Infiniband,
including Internet Protocol over Infiniband (IPoIB) NICs.

NIC Teaming Compatibility

NIC teaming is compatible with all networking technologies in Windows Server 2016 with the following
exceptions.
Single-root I/O virtualization (SR-IOV). For SR-IOV, data is delivered directly to the NIC without passing
it through the networking stack (in the host operating system, in the case of virtualization). Therefore, it is
not possible for the NIC team to inspect or redirect the data to another path in the team.
Native host Quality of Service (QoS). When QoS policies are set on a native or host system and those
policies invoke minimum bandwidth limitations, the overall throughput for a NIC team will be less than it
would be without the bandwidth policies in place.
 TCP Chimney. TCP Chimney is not supported with NIC teaming because TCP Chimney offloads the entire
networking stack directly to the NIC.
802.1X Authentication. 802.1X Authentication should not be used with NIC Teaming. Some switches do
not permit the configuration of both 802.1X Authentication and NIC Teaming on the same port.
To learn about using NIC Teaming within virtual machines (VMs) that are running on a Hyper-V host, see NIC
Teaming in Virtual Machines (VMs).

NIC Teaming and Virtual Machine Queues (VMQs)

VMQ and NIC Teaming work well together; VMQ should be enabled anytime Hyper-V is enabled. Depending on
the switch configuration mode and the load distribution algorithm, NIC teaming will either present VMQ
capabilities to the Hyper-V switch that show the number of queues available to be the smallest number of queues
supported by any adapter in the team (Min-queues mode) or the total number of queues available across all team
members (Sum-of-Queues mode).
Specifically, if the team is in Switch-Independent teaming mode and the Load Distribution is set to Hyper-V Port
mode or Dynamic mode, then the number of queues reported is the sum of all the queues available from the team
members (Sum-of-Queues mode); otherwise the number of queues reported is the smallest number of queues
supported by any member of the team (Min-Queues mode).
Here's why:
When the switch independent team is in Hyper-V Port mode or Dynamic mode the inbound traffic for a
Hyper-V switch port (VM) will always arrive on the same team member. The host can predict/control which
member will receive the traffic for a particular VM so NIC Teaming can be more thoughtful about which
VMQ Queues to allocate on a particular team member. NIC Teaming, working with the Hyper-V switch, will
set the VMQ for a VM on exactly one team member and know that inbound traffic will hit that queue.
When the team is in any switch dependent mode (static teaming or LACP teaming), the switch that the team
is connected to controls the inbound traffic distribution. The host's NIC Teaming software can't predict
which team member will get the inbound traffic for a VM and it may be that the switch distributes the traffic
for a VM across all team members. As a result the NIC Teaming software, working with the Hyper-V switch,
programs a queue for the VM on every team member, not just one team member.
When the team is in switch-independent mode and is using an address hash load distribution algorithm, the
inbound traffic will always come in on one NIC (the primary team member) - all of it on just one team
member. Since other team members aren't dealing with inbound traffic they get programmed with the
same queues as the primary member so that if the primary member fails any other team member can be
used to pick up the inbound traffic and the queues are already in place.
Most NICs have queues that can be used for either Receive Side Scaling (RSS) or VMQ, but not both at the same
time. Some VMQ settings appear to be settings for RSS queues but are really settings on the generic queues that
both RSS and VMQ use depending on which feature is presently in use. Each NIC has, in its advanced properties,
values for RssBaseProcNumber and \MaxRssProcessors. Following are a few VMQ settings that provide better
system performance.
Ideally each NIC should have the *RssBaseProcNumber set to an even number greater than or equal to two
(2). This is because the first physical processor, Core 0 (logical processors 0 and 1), typically does most of
the system processing so the network processing should be steered away from this physical processor.
(Some machine architectures don't have two logical processors per physical processor so for such machines
the base processor should be greater than or equal to 1. If in doubt assume your host is using a 2 logical
processor per physical processor architecture.)
If the team is in Sum-of-Queues mode the team members' processors should be, to the extent practical,
 non-overlapping. For example, in a 4-core host (8 logical processors) with a team of 2 10Gbps NICs, you
could set the first one to use base processor of 2 and to use 4 cores; the second would be set to use base
processor 6 and use 2 cores.
If the team is in Min-Queues mode the processor sets used by the team members must be identical.

NIC Teaming and Hyper-V Network Virtualization (HNV)

NIC Teaming is fully compatible with Hyper-V Network Virtualization (HNV). The HNV management system
provides information to the NIC Teaming driver that allows NIC Teaming to distribute the load in a way that is
optimized for the HNV traffic.

NIC Teaming and Live Migration

NIC Teaming in VMs does not affect Live Migration. The same rules exist for Live Migration whether or not NIC
Teaming is configured in the VM.

See Also
NIC Teaming in Virtual Machines (VMs)
 NIC Teaming in Virtual Machines (VMs)
4/24/2017 2 min to read Edit Online

Applies To: Windows Server 2016

This topic provides information about using NIC Teaming within Hyper-V VMs, and contains the following sections.
NIC Teaming Configuration Requirements
NIC Teaming with SR-IOV-Capable Network Adapters

NIC Teaming Configuration Requirements

If you want to use NIC Teaming in a VM, you must connect the virtual network adapters in the VM to external
Hyper-V Virtual Switches only; virtual network adapters that are connected to internal or private Hyper-V Virtual
Switches are not able to connect to the switch when they are in a team, and networking fails for the VM.
The Windows Server 2016 NIC Teaming solution supports teams with two members in VMs. Larger teams can be
created but such teams are not supported. Every team member must be connected to a different external Hyper-V
Virtual Switch, and the VM's networking interfaces must be configured to allow teaming. For more information, see
Create a New NIC Team in a VM.
NIC Teams within a VM must have their Teaming mode configured as Switch Independent. In addition, Load
Balancing mode for the NIC Team in a VM must be configured with the Address Hash distribution mode.
Configuring NIC Teaming in a VM with two virtual network adapters that are connected to different external Hyper-
V Virtual Switches allows the VM to sustain network connectivity even in the circumstance when one of the physical
network adapters connected to one virtual switch fails or gets disconnected.

NIC Teaming with SR-IOV-Capable Network Adapters

This failover protection is particularly important when working with Single Root I/O Virtualization (SR-IOV),
because SR-IOV traffic doesn't go through the Hyper-V Virtual Switch and cannot be protected by a NIC team in or
under the Hyper-V host. With the VM NIC Teaming option, you can configure two external Hyper-V Virtual
Switches, each connected to its own SR-IOV-capable NIC.

Each VM can have a virtual function (VF) from one or both SR-IOV NICs and, in the event of a NIC disconnect,
failover from the primary VF to the back-up adapter (VF). Alternately, the VM may have a VF from one NIC and a
non-VF vmNIC connected to another virtual switch. If the NIC associated with the VF gets disconnected, the traffic
can failover to the other switch without loss of connectivity.
Because failover between NICs in a VM might result in traffic being sent with the MAC address of the other vmNIC,
each Hyper-V Virtual Switch port associated with a VM that is using NIC Teaming must be set to allow teaming. To
discover how to enable NIC Teaming in the VM, see Create a New NIC Team in a VM.

See Also
NIC Teaming and Virtual Local Area Networks (VLANs)
Create a New NIC Team in a VM
NIC Teaming
 NIC Teaming and Virtual Local Area Networks
(VLANs)
4/24/2017 2 min to read Edit Online

Applies To: Windows Server 2016

This topic provides information about using NIC Teaming with virtual Local Area Networks (VLANs) on both host
computers and VMs, and includes the following sections.
Team interfaces and VLANs
Using VLANs with NIC Teaming in a VM
Managing network interfaces and VLANs

Team interfaces and VLANs

When you use NIC Teaming, the only reason to create multiple team interfaces is to logically divide inbound traffic
by virtual LAN (VLAN). Creating multiple team interfaces allows a host to be connected to different VLANs at the
same time.
When you use NIC Teaming with VLANs, ensure that you configure your environment using the following
guidelines.
1. Before you enable NIC Teaming, configure the physical switch ports connected to the teaming host to use
trunk (promiscuous) mode. The physical switch should pass all traffic to the host for filtering without
modifying the traffic.
2. Do not configure VLAN filters on the NICs by using the NIC advanced properties settings. Let the NIC
Teaming software or the Hyper-V Virtual Switch (if present) perform VLAN filtering.

Using VLANs with NIC Teaming in a VM

When a team is connected to a Hyper-V Virtual Switch, all VLAN segregation must be done in the Hyper-V Virtual
Switch rather than in NIC Teaming.
You can use the following guidelines when planning to use VLANs in a VM that is configured with a NIC Team.
The preferred method of supporting multiple VLANs in a VM is to configure the VM with multiple ports on
the Hyper-V Virtual Switch and associate each port with a VLAN. Never team these ports in the VM because
doing so will cause network communication problems.
If the VM has multiple SR-IOV Virtual Functions (VFs), ensure that they are on the same VLAN before
teaming them in the VM. It's easily possible to configure the different VFs to be on different VLANs and
doing so will cause network communication problems.

Managing network interfaces and VLANs

If you must have more than one VLAN exposed into a guest operating system, consider renaming the Ethernet
interfaces to clarify which VLAN is assigned to the interface. For example, if the Ethernet interface is associated
with VLAN 12 and the Ethernet 2 interface is associated with VLAN 48, rename the interface Ethernet to
EthernetVLAN12 and the other to EthernetVLAN48. You can rename interfaces by using the Windows
PowerShell command Rename-NetAdapter or by performing the following procedure.
To rename a network adapter
1. In Server Manager, in the Properties pane for the server whose network adapters you want to rename, click
the link to the right of the network adapter name. For example, if the network adapter is named Ethernet,
click either the IP address or other blue text to the right of the name Ethernet. The Network Connections
folder opens.
2. Right-click the network adapter that you want to rename, and select Rename.
3. Type the new name for the network adapter and press ENTER.

See Also
NIC Teaming MAC Address Use and Management
NIC Teaming
 NIC Teaming MAC Address Use and Management
4/24/2017 3 min to read Edit Online

Applies To: Windows Server 2016

When you configure a NIC Team with switch independent mode and either address hash or dynamic load
distribution, the team uses the media access control (MAC) address of the primary NIC Team member on outbound
traffic. The primary NIC Team member is a network adapter selected by the operating system from the initial set of
team members.
The primary team member is the first team member to bind to the team after you create it or after the host
computer is restarted. Because the primary team member might change in a non-deterministic manner at each
boot, NIC disable/enable action, or other reconfiguration activities, the primary team member might change, and
the MAC address of the team might vary.
In most situations this doesn't cause problems, but there are a few cases where issues might arise.
If the primary team member is removed from the team and then placed into operation there may be a MAC
address conflict. To resolve this conflict, disable and then enable the team interface. The process of disabling and
then enabling the team interface causes the interface to select a new MAC address from the remaining team
members, thereby eliminating the MAC address conflict.
You can set the MAC address of the NIC team to a specific MAC address by setting it in the primary team interface,
just as you can do when configuring the MAC address of any physical NIC.

MAC address use on transmitted packets

When you configure a NIC Team in switch independent mode and either address hash or dynamic load
distribution, the packets from a single source (such as a single VM) is simultaneously distributed across multiple
team members. To prevent the switches from getting confused and to prevent MAC flapping alarms, the source
MAC address is replaced with a different MAC address on the frames transmitted on team members other than the
primary team member. Because of this, each team member uses a different MAC address, and MAC address
conflicts are prevented unless and until failure occurs.
When a failure is detected on the primary NIC, the NIC Teaming software starts using the primary team member's
MAC address on the team member that is chosen to serve as the temporary primary team member (i.e., the one
that will now appear to the switch as the primary team member). This change only applies to traffic that was going
to be sent on the primary team member with the primary team member's MAC address as its source MAC address.
Other traffic continues to be sent with whatever source MAC address it would have used prior to the failure.
Following are lists that describe NIC Teaming MAC address replacement behavior, based on how the team is
configured:
1. In Switch Independent mode with Address Hash distribution
All ARP and NS packets are sent on the primary team member
All traffic sent on NICs other than the primary team member are sent with the source MAC address
modified to match the NIC on which they are sent
All traffic sent on the primary team member is sent with the original source MAC address (which may
be the team's source MAC address)
2. In Switch Independent mode with Hyper-V Port distribution
Every vmSwitch port is affinitized to a team member
Every packet is sent on the team member to which the port is affinitized
No source MAC replacement is done
3. In Switch Independent mode with Dynamic distribution
Every vmSwitch port is affinitized to a team member
All ARP/NS packets are sent on the team member to which the port is affinitized
Packets sent on the team member that is the affinitized team member have no source MAC address
replacement done
Packets sent on a team member other than the affinitized team member will have source MAC
address replacement done
4. In Switch Dependent mode (all distributions)
No source MAC address replacement is performed

See Also
Create a New NIC Team on a Host Computer or VM
NIC Teaming
 Create a New NIC Team on a Host Computer or VM
4/24/2017 8 min to read Edit Online

Applies To: Windows Server 2016

This topic provides information about NIC Teaming configuration so that you understand the selections you must
make when you are configuring a new NIC Team. This topic contains the following sections.
Choosing a Teaming Mode
Choosing a Load Balancing Mode
Choosing a Standby Adapter Setting
Using the Primary Team Interface Property

NOTE
If you already understand these configuration items, you can use the following procedures to configure NIC Teaming.
Create a New NIC Team in a VM
Create a New NIC Team

When you create a new NIC Team, you must configure the following NIC Team properties.
Team name
Member adapters
Teaming mode
Load balancing mode
Standby adapter
You can also optionally configure the primary team interface and configure a virtual LAN (VLAN) number.
These NIC Team properties are displayed in the following illustration, which contains example values for some NIC
Team properties.
Choosing a Teaming Mode
The options for Teaming mode are Switch Independent, Static Teaming, and Link Aggregation Control
Protocol (LACP). Both Static Teaming and LACP are Switch Dependent modes. For best NIC Team performance
with all three Teaming modes, it is recommended that you use a Load Balancing mode of Dynamic distribution.
Switch Independent
With Switch Independent mode, the switch or switches to which the NIC Team members are connected are
unaware of the presence of the NIC team and do not determine how to distribute network traffic to NIC Team
members - instead, the NIC Team distributes inbound network traffic across the NIC Team members.
When you use Switch Independent mode with Dynamic distribution, the network traffic load is distributed based
on the TCP Ports address hash as modified by the Dynamic load balancing algorithm. The Dynamic load balancing
algorithm redistributes flows to optimize team member bandwidth utilization so that individual flow transmissions
can move from one active team member to another. The algorithm takes into account the small possibility that
redistributing traffic could cause out-of-order delivery of packets, so it takes steps to minimize that possibility.
Switch Dependent
With Switch Dependent modes, the switch to which the NIC Team members are connected determines how to
distribute the inbound network traffic among the NIC Team members. The switch has complete independence to
determine how to distribute the network traffic across the NIC Team members.

IMPORTANT
Switch dependent teaming requires that all team members are connected to the same physical switch or a multi-chassis
switch that shares a switch ID among the multiple chassis.
Static Teaming requires you to manually configure both the switch and the host to identify which links form the
team. Because this is a statically configured solution, there is no additional protocol to assist the switch and the
host to identify incorrectly plugged cables or other errors that could cause the team to fail to perform. This mode is
typically supported by server-class switches.
Unlike Static Teaming, LACP Teaming mode dynamically identifies links that are connected between the host and
the switch. This dynamic connection enables the automatic creation of a team and, in theory but rarely in practice,
the expansion and reduction of a team simply by the transmission or receipt of LACP packets from the peer entity.
All server-class switches support LACP, and all require the network operator to administratively enable LACP on
the switch port. When you configure a Teaming mode of LACP, NIC Teaming always operates in LACP's Active
mode with a short timer. No option is presently available to modify the timer or change the LACP mode.
When you use Switch Dependent modes with Dynamic distribution, the network traffic load is distributed based on
the TransportPorts address hash as modified by the Dynamic load balancing algorithm. The Dynamic load
balancing algorithm redistributes flows to optimize team member bandwidth utilization. Individual flow
transmissions can move from one active team member to another as part of the dynamic distribution. The
algorithm takes into account the small possibility that redistributing traffic could cause out-of-order delivery of
packets, so it takes steps to minimize that possibility.
As with all switch dependent configurations, the switch determines how to distribute the inbound traffic among
the team members. The switch is expected to do a reasonable job of distributing the traffic across the team
members but it has complete independence to determine how it does so.

Choosing a Load Balancing Mode

The options for Load Balancing distribution mode are Address Hash, Hyper-V Port, and Dynamic.
Address Hash
This Load Balancing mode creates a hash that is based on address components of the packet. It then assigns
packets that have that hash value to one of the available adapters. Usually this mechanism alone is sufficient to
create a reasonable balance across the available adapters.
You can use Windows PowerShell to specify values for the following hashing function components.
Source and destination TCP ports and source and destination IP addresses. This is the default when you
select Address Hash as the Load Balancing mode.
Source and destination IP addresses only.
Source and destination MAC addresses only.
The TCP ports hash creates the most granular distribution of traffic streams, resulting in smaller streams that can
be independently moved between NIC team members. However, you cannot use the TCP ports hash for traffic that
is not TCP or UDP-based, or where the TCP and UDP ports are hidden from the stack, such as with IPsec-protected
traffic. In these cases, the hash automatically uses the IP address hash or, if the traffic is not IP traffic, the MAC
address hash is used.
Hyper-V Port
There is an advantage in using Hyper-V Port mode for NIC Teams that are configured on Hyper-V hosts. Because
VMs have independent MAC addresses, the VM's MAC address - or the port the VM is connected to on the Hyper-
V Virtual Switch - can be the basis upon which to divide network traffic between NIC Team members.
 IMPORTANT
NIC Teams that you create within VMs cannot be configured with the Hyper-V Port load balancing mode. Use the Address
Hash load balancing mode instead.

Because the adjacent switch always sees a particular MAC address on one port, the switch distributes the ingress
load (the traffic from the switch to the host) on multiple links based on the destination MAC (VM MAC) address.
This is particularly useful when Virtual Machine Queues (VMQs) are used, because a queue can be placed on the
specific NIC where the traffic is expected to arrive.
However, if the host has only a few VMs, this mode might not be granular enough to achieve a well-balanced
distribution. This mode will also always limit a single VM (i.e., the traffic from a single switch port) to the
bandwidth that is available on a single interface. NIC Teaming uses the Hyper-V Virtual Switch Port as the identifier
instead of using the source MAC address because, in some instances, a VM might be configured with more than
one MAC address on a switch port.
Dynamic
This Load balancing mode utilizes the best aspects of each of the other two modes and combines them into a
single mode:
Outbound loads are distributed based on a hash of the TCP Ports and IP addresses. Dynamic mode also
rebalances loads in real time so that a given outbound flow may move back and forth between team
members.
Inbound loads are distributed in the same manner as the Hyper-V port mode.
The outbound loads in this mode are dynamically balanced based on the concept of flowlets. Just as human speech
has natural breaks at the ends of words and sentences, TCP flows (TCP communication streams) also have
naturally occurring breaks. The portion of a TCP flow between two such breaks is referred to as a flowlet.
When the dynamic mode algorithm detects that a flowlet boundary has been encountered - such as when a break
of sufficient length has occurred in the TCP flow - the algorithm automatically rebalances the flow to another team
member if appropriate. In some circumstances the algorithm might also periodically rebalance flows that do not
contain any flowlets. Because of this, the affinity between TCP flow and team member can change at any time as
the dynamic balancing algorithm works to balance the workload of the team members.
Whether the team is configured with Switch Independent or one of the Switch Dependent modes, it is
recommended that you use Dynamic distribution mode for best performance.
There is an exception to this rule when the NIC Team has just two team members, is configured in Switch
Independent mode, and has Active/Standby mode enabled, with one NIC active and the other configured for
Standby. With this NIC Team configuration, Address Hash distribution provides slightly better performance than
Dynamic distribution.

Choosing a Standby Adapter Setting

The options for Standby Adapter are None (all adapters Active) or your selection of a specific network adapter in
the NIC Team that will act as a Standby adapter, while other unselected team members are Active. When you
configure a NIC as a Standby adapter, no network traffic is sent to or processed by the adapter unless and until the
moment when an Active NIC fails. After an Active NIC fails, the Standby NIC becomes active, and processes
network traffic. If and when all team members are restored to service, the standby team member is returned to
standby status.
If you have a two-NIC team and you choose to configure one NIC as a Standby adapter, you lose the bandwidth
aggregation advantages that exist with two active NICs.
 IMPORTANT
You do not need to designate a Standby Adapter to achieve fault tolerance; fault tolerance is always present whenever there
are at least two network adapters in a NIC Team.

Using the Primary Team Interface Property

To access the Primary Team Interface dialog box, you must click the link that is highlighted in the illustration below.

After you click the highlighted link, the following New Team Interface dialog box opens.
If you are using VLANs, you can use this dialog box to specify a VLAN number.
Whether or not you are using VLANs, you can specify a tNIC name for the NIC Team.

See Also
Create a New NIC Team in a VM
NIC Teaming
 Create a New NIC Team
4/24/2017 2 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic to create a new NIC Team on a host computer or in a Hyper-V virtual machine (VM) that is
running Windows Server 2016.

IMPORTANT
If you are creating a new NIC Team in a VM, review the topic Create a New NIC Team in a VM before you perform this
procedure.

Membership in Administrators, or equivalent, is the minimum required to perform this procedure.

To create a new NIC Team
1. In Server Manager, click Local Server.
2. In the Properties pane, in the first column, locate NIC Teaming, and then click the link Disabled to the
right. The NIC Teaming dialog box opens.

3. In Adapters and Interfaces, select the network adapters that you want to add to a NIC Team. For example,
if you want to add the adapters Ethernet 2 and Ethernet 3 to a new NIC Team, make the selection per the
illustration below.
4. Click TASKS, and then click Add to New Team.

5. The New team dialog box opens and displays network adapters and team members. In Team name, type a
name for the new NIC Team, and then click Additional properties.

6. In Additional properties, select values for Teaming mode, Load balancing mode, and Standby
 adapter. In most cases, the highest performing load balancing mode is Dynamic. For more detailed
explanations of these modes, see the topic Create a New NIC Team on a Host Computer or VM.

IMPORTANT
If you are configuring a NIC Team in a virtual machine (VM), you must select a Teaming mode of Switch
Independent and a Load balancing mode of Address Hash.

7. If you want to configure the primary team interface name or assign a VLAN number to the NIC Team, click
the link to the right of Primary team interface. The New team interface dialog box opens.
8. Depending on your requirements, take one of the following actions:
To provide a tNIC interface name, type an interface name.
To configure VLAN membership, click Specific VLAN. Type the VLAN information in the first section
of the dialog box, which is highlighted in the illustration below. For example, if you want to add this
NIC Team to the accounting VLAN number 44, Type Accounting 44 - VLAN. Next, to the right of
Specific VLAN, type the VLAN number that you want to use. For example, type 44.

9. Click OK.

See Also
Create a New NIC Team on a Host Computer or VM
Create a New NIC Team in a VM
NIC Teaming
 Create a New NIC Team in a VM
4/24/2017 3 min to read Edit Online

Applies To: Windows Server 2016

You can use this topic to connect a virtual machine (VM) to Hyper-V Virtual Switches in a manner that is
consistent with NIC Teaming requirements within VMs. You can also use this topic to create a new NIC team in a
VM.
This topic contains the following sections.
Network configuration requirements
Configure the physical and virtual network
Create a NIC Team

Network configuration requirements

The physical switch, Hyper-V Virtual Switch, local area network (LAN), and NIC Teaming requirements for creating
a NIC Team in a VM are:
The computer that is running Hyper-V must have two or more network adapters.
If the network adapters are connected to multiple physical switches, the physical switches must be on the
same Layer 2 subnet.
You must use Hyper-V Manager or Windows PowerShell commands to create two external Hyper-V Virtual
Switches, each of which is connected to a different physical network adapter.
The VM in which you want to configure NIC Teaming must be connected to both external virtual switches
that you create.
The Windows Server 2016 NIC Teaming solution supports teams with two members in VMs. Larger teams
can be created but such teams are not supported.
NIC Teams within a VM must have their Teaming mode configured as Switch Independent. In addition,
Load Balancing mode for the NIC Team in a VM must be configured with the Address Hash distribution
mode.

Configure the physical and virtual network

Before you perform this procedure, you must deploy a Hyper-V host with two network adapters that are
connected to different physical switches, and you must configure the network adapters with IP addresses that are
from the same IP address range.
By using the following procedure, you can create two external Hyper-V Virtual Switches, connect a VM to the
switches, and then configure the VM connections to the switches.

IMPORTANT
This procedure does not include instructions on how to create a VM.
Membership in Administrators, or equivalent, is the minimum required to perform this procedure.
To create a virtual switch and connect a VM
1. On the Hyper-V host, open Hyper-V Manager, and then click Virtual Switch Manager.

2. Virtual Switch Manager opens. In What type of virtual switch do you want to create?, ensure that
External is selected, and then click Create Virtual Switch.
3. The Virtual Switch Properties page opens. Type a Name for the virtual switch, and add Notes as needed.
4. In Connection type, in External network, select the physical network adapter to which you want to
attach the virtual switch.
5. Configure additional switch properties so that they are correct for your deployment, and then click OK.
6. Create a second external virtual switch by repeating the previous steps. Connect the second external switch
to a different network adapter.
7. Open Hyper-V Manager. In Virtual Machines, right-click the VM that you want to configure, and then click
Settings. The VM Settings dialog box opens.

IMPORTANT
Ensure that the VM is not started. If it is started, perform shutdown before configuring the VM.

8. In Hardware, click Network Adapter.

9. In Network Adapter properties, select one of the virtual switches that you created in previous steps, and
then click Apply.
10. In Hardware, click to expand the plus sign (+) next to Network Adapter. Click Advanced Features.

NOTE
Steps 10 through 12 demonstrate how to enable NIC Teaming by using the graphical user interface. You can also
enable NIC Teaming by running the following Windows PowerShell command:
Set-VMNetworkAdapter -VMName <VMname> -AllowTeaming On
11. In Advanced Features, scroll down to NIC Teaming.
12. In NIC Teaming, click to select Enable this network adapter to be part of a team in the guest
operating system. Click OK.
13. To add a second network adapter, in Hyper-V Manager, in Virtual Machines, right-click the same VM, and
then click Settings. The VM Settings dialog box opens.
14. In Add Hardware, click Network Adapter, and then click Add.
15. In Network Adapter properties, select the second virtual switch that you created in previous steps, and
then click Apply.
16. In Hardware, click to expand the plus sign (+) next to Network Adapter. Click Advanced Features.
17. In Advanced Features, scroll down to NIC Teaming.
18. In NIC Teaming, click to select Enable this network adapter to be part of a team in the guest
operating system. Click OK.
You can now start and log on to your VM to create your new NIC Team.

Create a NIC Team

To create a new NIC Team in your VM, follow the instructions in the topic Create a New NIC Team.

See Also
Create a New NIC Team on a Host Computer or VM
NIC Teaming
 Troubleshooting NIC Teaming
4/24/2017 2 min to read Edit Online

Applies To: Windows Server 2016

This topic provides information about troubleshooting NIC Teaming, and contains the following sections, which
describe possible causes of issues with NIC Teaming.
Hardware that doesn't conform to specification
Physical switch security features
Disabling and Enabling with Windows PowerShell

Hardware that doesn't conform to specification

When hardware implementations of standard protocols do not conform to specification, NIC Teaming performance
might be affected.
During normal operation, NIC Teaming may send packets from the same IP address, yet with multiple different
source media access control (MAC) addresses. According to protocol standards, the receivers of these packets must
resolve the IP address of the host or VM to a specific MAC address rather than responding to the MAC address
from which the packet was received. Clients that correctly implement the address resolution protocols, IPv4's
Address Resolution Protocol (ARP) or IPv6's neighbor discovery protocol (NDP), will send packets with the correct
destination MAC address (the MAC address of the VM or host that owns that IP address).
Some embedded hardware, however, does not correctly implement the address resolution protocols, and also
might not explicitly resolve an IP address to a MAC address using ARP or NDP. A storage area network (SAN)
controller is an example of a device that might perform in this manner. Non-conforming devices copy the source
MAC address that is contained in a received packet and use that as the destination MAC address in the
corresponding outgoing packets.
This results in packets being sent to the wrong destination MAC address. Because of this, the packets are dropped
by the Hyper-V Virtual Switch because they don't match any known destination.
If you are having trouble connecting to SAN controllers or other embedded hardware, you should take packet
captures and determine whether your hardware is correctly implementing ARP or NDP, and contact your hardware
vendor for support.

Physical switch security features

Depending on configuration, NIC Teaming may send packets from the same IP address with multiple different
source MAC addresses. This can trip up security features on the physical switch such as dynamic ARP inspection or
IP source guard, especially if the physical switch is not aware that the ports are part of a team. This can occur if you
configure NIC Teaming in Switch Independent mode. You should inspect the switch logs to determine whether
switch security features are causing connectivity problems with NIC Teaming.

Disabling and enabling network adapters by using Windows PowerShell

A common reason for a NIC Team to fail is that the team interface is disabled. In many cases, the interface is
disabled by accident when the following Windows PowerShell sequence of commands is run:
 Disable-NetAdapter *
Enable-NetAdapter *

This sequence of commands does not enable all of the NetAdapters that it disabled.
This is because disabling all of the underlying physical member NICs causes the NIC team interface to be removed
and no longer show up in Get-NetAdapter. Because of this, the Enable-NetAdapter \* command does not enable
the NIC Team, because that adapter is removed.
The Enable-NetAdapter \* command does, however, enable the member NICs, which then (after a short time)
causes the team interface to be recreated. In this circumstance, the team interface is still in a "disabled" state
because it has not been re-enabled. Enabling the team interface after it is recreated will allow network traffic to
begin to flow again.

See Also
NIC Teaming
 Performance Tuning Software Defined Networks
4/24/2017 3 min to read Edit Online

Software Defined Networking (SDN) in Windows Server 2016 is made up of a combination of a Network
Controller, Hyper-V Hosts, Software Load Balancer Gateways and HNV Gateways. For tuning of each of these
components refer to the following sections:

Network Controller
The network controller is a Windows Server role which must be enabled on Virtual Machines running on hosts that
are configured to use SDN and are controlled by the network controller.
Three Network Controller enabled VMs are sufficient for high availability and maximum performance. Each VM
must be sized according to the guidelines provided in the SDN infrastructure virtual machine role requirements
section of the Plan Software Defined Networking topic.
SDN Quality of Service (QoS )
To ensure virtual machine traffic is prioritized effectively and fairly, it is recommended that you configure SDN QoS
on the workload virtual machines. For more information on configuring SDN QoS, refer to the Configure QoS for a
Tenant VM Network Adapter topic.

Hyper-V Host Networking

The guidance provided in the Hyper-V network I/O performance section of the Performance Tuning for Hyper-V
Servers guide is applicable when SDN is used, however this section covers additional guidelines that must be
followed to ensure the best performance when using SDN.
Physical Network Adapter (NIC ) Teaming
For best performance and fail-over capabilities, it is recommended that you configure the physical network
adapters to be teamed. When using SDN you must create the team with Switch Embedded Teaming (SET).
The optimal number of team members is two as virtualized traffic will be spread across both of the team members
for both inbound and outbound directions. You can have more than two team members; however inbound traffic
will be spread over at most two of the adapters. Outbound traffic will always be spread across all adapters if the
default of dynamic load balancing remains configured on the virtual switch.
Encapsulation Offloads
SDN relies on encapsulation of packets to to virtualize the network. For optimal performance, it is important that
the network adapter supports hardware offload for the encapsulation format that is used. There is no significant
performance benefit of one encapsulation format over another. The default encapsulation format when the
network controller is used is VXLAN.
You can determine which encapsulation format is being used through the network controller with the following
PowerShell cmdlet:

(Get-NetworkControllerVirtualNetworkConfiguration -connectionuri
$uri).properties.networkvirtualizationprotocol

For best performance, if VXLAN is returned then you must make sure your physical network adapters support
VXLAN task offload. If NVGRE is returned, then your physical network adapters must support NVGRE task offload.
MTU
Encapsulation results in extra bytes being added to each packet. In order to avoid fragmentation of these packets,
the physical network must be configured to use jumbo frames. An MTU value of 9234 is the recommended size for
either VXLAN or NVGRE and must be configured on the physical switch for the physical interfaces of the host ports
(L2) and the router interfaces (L3) of the VLANs over which encapsulated packets will be sent. This includes the
Transit, HNV Provider and Management networks.
MTU on the Hyper-V host is configured through the network adapter, and the Network Controller Host Agent
running on the Hyper-V host will adjust for the encapsulation overhead automatically if supported by the network
adapter driver.
Once traffic egresses from the virtual network via a Gateway, the encapsulation is removed and the original MTU as
sent from the VM is used.
Single Root IO Virtualization (SR -IOV )
SDN is implemented on the Hyper-V host using a forwarding switch extension in the virtual switch. For this switch
extension to process packets, SR-IOV must not be used on virtual network interfaces that are configured for use
with the network controller as it causes VM traffic to bypass the virtual switch.
SR-IOV can still be enabled on the virtual switch if desired and can be used by VM network adapters that are not
controlled by the network controller. These SR-IOV VMs can coexist on the same virtual switch as network
controller controlled VMs which do not use SR-IOV.
If you are using 40Gbit network adapters it is recommended that you enable SR-IOV on the virtual switch for the
Software Load Balancing (SLB) Gateways to achieve maximum throughput. This is covered in more detail in the
Software Load Balancer Gateways section.

HNV Gateways
You can find information on tuning HNV Gateways for use with SDN in the HVN Gateways section.

Software Load Balancer (SLB)

SLB Gateways can only be used with the Network Controller and SDN. You can find more information on tuning
SDN for use iwth SLB Gateways in the Software Load Balancer Gateways section.
 HNV Gateway Performance Tuning in Software
Defined Networks
4/24/2017 6 min to read Edit Online

This topic provides hardware specifications and configuration recommendations for servers that are running
Hyper-V and hosting Windows Server Gateway virtual machines, in addition to configuration parameters for
Windows Server Gateway virtual machines (VMs). To extract best performance from Windows Server gateway VMs,
it is expected that these guidelines will be followed. The following sections contain hardware and configuration
requirements when you deploy Windows Server Gateway.
1. Hyper-V hardware recommendations
2. Hyper-V host configuration
3. Windows Server gateway VM configuration

Hyper-V hardware recommendations

Following is the recommended minimum hardware configuration for each server that is running Windows Server
2016 and Hyper-V.

SERVER COMPONENT SPECIFICATION

Central Processing Unit (CPU) Non-Uniform Memory Architecture (NUMA) nodes: 2

If there are multiple Windows Server gateway VMs on the
host, for best performance, each gateway VM should have full
access to one NUMA node. And it should be different from the
NUMA node used by the host physical adapter.

Cores per NUMA node 2

Hyper-Threading Disabled. Hyper-Threading does not improve the performance

of Windows Server Gateway.

Random Access Memory (RAM) 48 GB

Network Interface Cards (NICs) Two 10 GB NICs,The gateway performance will depend on the
line rate. If the line rate is less than 10Gbps, the gateway
tunnel throughput numbers will also go down by the same
factor.

Ensure that the number of virtual processors that are assigned to a Windows Server Gateway VM does not exceed
the number of processors on the NUMA node. For example, if a NUMA node has 8 cores, the number of virtual
processors should be less than or equal to 8. For best performance, it should be 8. To find out the number of NUMA
nodes and the number of cores per NUMA node, run the following Windows PowerShell script on each Hyper-V
host:
 $nodes = [object[]] $(gwmi Namespace root\virtualization\v2 -Class MSVM_NumaNode)
$cores = ($nodes | Measure-Object NumberOfProcessorCores -sum).Sum
$lps = ($nodes | Measure-Object NumberOfLogicalProcessors -sum).Sum

Write-Host "Number of NUMA Nodes: ", $nodes.count

Write-Host ("Total Number of Cores: ", $cores)
Write-Host ("Total Number of Logical Processors: ", $lps)

IMPORTANT
Allocating virtual processors across NUMA nodes might have a negative performance impact on Windows Server Gateway.
Running multiple VMs, each of which has virtual processors from one NUMA node, likely provides better aggregate
performance than a single VM to which all virtual processors are assigned.

One gateway VM with eight virtual processors and at least 8GB RAM is recommended when selecting the number
of gateway VMs to install on each Hyper-V host when each NUMA node has eight cores. In this case, one NUMA
node is dedicated to the host machine.

Hyper-V Host configuration

Following is the recommended configuration for each server that is running Windows Server 2016 and Hyper-V
and whose workload is to run Windows Server Gateway VMs. These configuration instructions include the use of
Windows PowerShell command examples. These examples contain placeholders for actual values that you need to
provide when you run the commands in your environment. For example, network adapter name placeholders are
NIC1 and NIC2. When you run commands that use these placeholders, utilize the actual names of the network
adapters on your servers rather than using the placeholders, or the commands will fail.

NOTE
To run the following Windows PowerShell commands, you must be a member of the Administrators group.

CONFIGURATION ITEM WINDOWS POWERSHELL CONFIGURATION

Switch Embedded Teaming When you create a vswitch with multiple network adapters, it
automatically enabled switch embedded teaming for those
adapters.
New-VMSwitch -Name TeamedvSwitch -NetAdapterName "NIC
1","NIC 2"
Traditional teaming through LBFO is not supported with SDN
in Windows Server 2016. Switch Embedded Teaming allows
you to use the same set of NICs for your virtual traffic and
RDMA traffic. This was not supported with NIC teaming based
on LBFO.

Interrupt Moderation on physical NICs Use default settings. To check the configuration, you can use
the following Windows PowerShell command:
Get-NetAdapterAdvancedProperty
 CONFIGURATION ITEM WINDOWS POWERSHELL CONFIGURATION

Receive Buffers size on physical NICs You can verify whether the physical NICs support the
configuration of this parameter by running the command
Get-NetAdapterAdvancedProperty . If they do not support
this parameter, the output from the command does not
include the property Receive Buffers. If NICs do support this
parameter, you can use the following Windows PowerShell
command to set the Receive Buffers size:
Set-NetAdapterAdvancedProperty NIC1 DisplayName
Receive Buffers DisplayValue 3000

Send Buffers size on physical NICs You can verify whether the physical NICs support the
configuration of this parameter by running the command
Get-NetAdapterAdvancedProperty . If the NICs do not
support this parameter, the output from the command does
not include the property Send Buffers. If NICs do support
this parameter, you can use the following Windows PowerShell
command to set the Send Buffers size:
Set-NetAdapterAdvancedProperty NIC1 DisplayName
Transmit Buffers DisplayValue 3000

Receive Side Scaling (RSS) on physical NICs You can verify whether your physical NICs have RSS enabled
by running the Windows PowerShell command Get-
NetAdapterRss. You can use the following Windows
PowerShell commands to enable and configure RSS on your
network adapters:
Enable-NetAdapterRss NIC1,NIC2
Set-NetAdapterRss NIC1,NIC2
NumberOfReceiveQueues 16 -MaxProcessors
NOTE: If VMMQ or VMQ is enabled, RSS does not have to be
enabled on the physical network adapters. You can enable it
on the host virtual network adapters

VMMQ To enable VMMQ for a VM, run the following command:

Set-VmNetworkAdapter -VMName <gateway vm name>,-
VrssEnabled $true -VmmqEnabled $true
NOTE: Not all network adapters support VMMQ. Currently, it
is supported on Chelsio T5 and T6, Mellanox CX-3 and CX-4,
and QLogic 45xxx series

Virtual Machine Queue (VMQ) on the NIC Team You can enable VMQ on your SET team by using the following
Windows PowerShell command:
Enable-NetAdapterVmq
NOTE: This should be enabled only if the HW does not
support VMMQ. If supported, VMMQ should be enabled for
better performance.

NOTE
VMQ and vRSS come into picture only when the load on the VM is high and the CPU is being utilized to the maximum. Only
then will at least one processor core max out. VMQ and vRSS will then be beneficial to help spread the processing load across
multiple cores. This is not applicable for IPsec traffic as IPsec traffic is confined to a single core.

Windows Server Gateway VM configuration

On both Hyper-V hosts, you can configure multiple VMs that are configured as gateways with Windows Server
Gateway. You can use Virtual Switch Manager to create a Hyper-V Virtual Switch that is bound to the NIC team on
the Hyper-V host. Note that for best performance, you should deploy a single gateway VM on a Hyper-V host.
Following is the recommended configuration for each Windows Server Gateway VM.

CONFIGURATION ITEM WINDOWS POWERSHELL CONFIGURATION

Memory 8 GB

Number of virtual network adapters 3 NICs with the following specific uses: 1 for Management that
is used by the management operating system, 1 External that
provides access to external networks, 1 that is Internal that
provides access to internal networks only.

Receive Side Scaling (RSS) You can keep the default RSS settings for the Management
NIC. The following example configuration is for a VM that has
8 virtual processors. For the External and Internal NICs, you
can enable RSS with BaseProcNumber set to 0 and
MaxRssProcessors set to 8 using the following Windows
PowerShell command:
Set-NetAdapterRss Internal,External
BaseProcNumber 0 MaxProcessorNumber 8

Send side buffer You can keep the default Send Side Buffer settings for the
Management NIC. For both the Internal and External NICs you
can configure the Send Side Buffer with 32 MB of RAM by
using the following Windows PowerShell command:
Set-NetAdapterAdvancedProperty Internal,External
DisplayName Send Buffer Size DisplayValue 32MB

Receive Side buffer You can keep the default Receive Side Buffer settings for the
Management NIC. For both the Internal and External NICs,
you can configure the Receive Side Buffer with 16 MB of RAM
by using the following Windows PowerShell command:
Set-NetAdapterAdvancedProperty Internal,External
DisplayName Receive Buffer Size DisplayValue
16MB

Forward Optimization You can keep the default Forward Optimization settings for
the Management NIC. For both the Internal and External NICs,
you can enable Forward Optimization by using the following
Windows PowerShell command:
Set-NetAdapterAdvancedProperty Internal,External
DisplayName Forward Optimization DisplayValue 1
 SLB Gateway Performance Tuning in Software
Defined Networks
4/24/2017 2 min to read Edit Online

Software load balancing is provided by a combination of a load balancer manager in the Network Controller VMs,
the Hyper-V Virtual Switch and a set of Load Balancer Multixplexor (Mux) VMs.
No additional performance tuning is required to configure the Network Controller or the Hyper-V host for load
balancing beyond what is described in the Software Defined Networking section, unless you will be using SR-IOV
for the Muxes as described below.

SLB Mux VM Configuration

SLB Mux virtual machines are deployed in an Active-Active configuration. This means that every Mux VM that is
deployed and added to the Network Controller can process incoming requests. Thus, the total aggregate
throughput of all of the connections is only limited by the number of Mux VMs that you have deployed.
An individual connection to a Virtual IP (VIP) will always be sent to the same Mux, assuming the number of muxes
remains constant, and as a result its throughput will be limited to the throughput of a single Mux VM. Muxes only
process the inbound traffic that is destined to a VIP. Response packets go directly from the VM that is sending the
response to the physical switch which forwards it on to the client.
In some cases when the source of the request originates from an SDN host that is added to the same Network
Controller that manages the VIP, further optimization of the inbound path for the request is also performed which
enables most packets to travel directly from the client to the server, bypassing the Mux VM entirely. No additional
configuration is required for this optimization to take place.
Each SLB Mux VM must be sized according to the guidelines provided in the SDN infrastructure virtual machine
role requirements section of the Plan Software Defined Networking topic.

Single Root IO virtualization (SR-IOV)

When using 40Gbit Ethernet, the ability for the virtual switch to process packets for the Mux VM becomes the
limiting factor for Mux VM throughput. Because of this it is recommended that SR-IOV be enabled on the SLB VM's
VM Network Adapter to ensure that the virtual switch is not the bottleneck.
To enable SR-IOV, you must enable it on the virtual switch when the virtual switch is created. In this example, we
are creating a virtual switch with switch embedded teaming (SET) and SR-IOV:

new-vmswitch -Name SDNSwitch -EnableEmbeddedTeaming $true -NetAdapterName @("NIC1", "NIC2") -EnableIOV

$true

Then, it must be enabled on the virtual network adapter(s) of the SLB Mux VM which process the data traffic. In this
example, SR-IOV is being enabled on all adapters:

get-vmnetworkadapter -VMName SLBMUX1 | set-vmnetworkadapter -IovWeight 50

 Performance tuning for Storage Spaces Direct
4/24/2017 2 min to read Edit Online

Storage Spaces Direct, a Windows Server-based software-defined storage solution, automatically tunes its
performance, obviating the need to manually specify column counts, the cache configuration of the hardware you
use, and other factors that must be set manually with shared SAS storage solutions. For background info, see
Storage Spaces Direct in Windows Server 2016.
The Storage Spaces Direct Software Storage Bus Cache is automatically configured based on the types of storage
present in the system. Three types recognized: HDD, SSD and NVMe. The cache claims the fastest storage for read
and/or write caching, as appropriate, and uses the slower storage for persistent storage of data.
The following table summarizes the defaults:

STORAGE TYPES CACHE CONFIGURATION

Any Single Type If there is only one type of storage present, the Software
Storage Bus Cache isn't configured.

SSD+HDD or NVMe+HDD The fastest storage is configured as the cache layer and caches
both reads and writes.

SSD+SSD or NVMe+NVMe These fast+fast options are targeted to combinations of higher

and lower endurance storage, for instance 10 drive writes per
day (DWPD) NAND flash SSD for cache and 1.5 DWPD NAND
flash SSD for capacity. They're enabled by giving Storage
Spaces Direct a set of Model strings to identify cache devices
with. For more information see the Enable-
StorageSpacesDirect cmdlet reference ( CacheDeviceModel ).

In a fast+fast system, only writes are cached. Reads aren't

cached.

Note that caching over an SSD or NVMe device defaults to write caching, only. The intention is that since the
capacity device is fast, there is limited value in moving read content to the cache devices. There are cases where this
may not hold, though care should be taken since enabling read cache may unnecessarily consume cache device
endurance for no increase in performance. Examples may include:
NVme+SSD Enabling read cache will allow read IO to take advantage of the PCIe connectivity and/or higher
IOPS performance of the NVMe devices as compared to the aggregated SSD.
This may be true for bandwidth-oriented scenarios due to the relative bandwidth capabilities of the NVMe
devices vs. the HBA connecting to the SSD. It may not be true for IOPS-oriented scenarios where CPU costs of
IOPS may limit systems before the increased performance can be realized.
NVMe+NVMe Similarly, if the read capability of the cache NVMe are greater than the combined capacity
NVMe, there may be value in enabling read cache.
Good cases for read cache in these configurations are expected to be unusual.
To view and alter the cache configuration, use the Get-ClusterStorageSpacesDirect and Set-
ClusterStorageSpacesDirect cmdlets. The CacheModeHDD and CacheModeSSD properties define how the cache
operates on capacity media of the indicated type.

See also
Understanding Storage Spaces Direct
Planning Storage Spaces Direct
Performance tuning for file servers
Software-Defined Storage Design Considerations Guide (for Windows Server 2012 R2 and shared SAS storage)
 Frequently Asked Questions about Storage Replica
5/31/2017 10 min to read Edit Online

Applies To: Windows Server 2016

This topic contains answers to frequently asked questions (FAQs) about Storage Replica.

Is Storage Replica supported on Nano Server?

Yes.

NOTE
You must use the Storage Nano Server package during setup. For more information about deploying Nano Server, see
Getting Started with Nano Server.

Install Storage Replica on Nano Server using PowerShell remoting as follows:

1. Add the Nano server to your client trust list.

NOTE
This step is only necessary if the computer is not a member of an Active Directory Domain Services forest or in an
untrusted forest. It adds NTLM support to PSSession remoting, which is disabled by default for security reasons. For
more information, see PowerShell Remoting Security Considerations.

Set-Item WSMan:\localhost\Client\TrustedHosts "<computer name of Nano Server>"

2. To install the Storage Replica feature, run the following cmdlet from a management computer:

Install-windowsfeature -Name storage-replica,RSAT-Storage-Replica -ComputerName <nano server> -Restart -

IncludeManagementTools

Using the Test-SRTopology cmdlet with Nano Server in Windows Server 2016 requires remote script
invocation with CredSSP. Unlike other Storage Replica cmdlets, Test-SRTopology requires running locally on
the source server.
On the Nano server (through a remote PSSession) :

NOTE
CREDSSP is needed for Kerberos double-hop support in the Test-SRTopology cmdlet, and not needed by other
Storage Replica cmdlets, which handle distributed system credentials automatically. Using CREDSSP is not
recommended under typical circumstances. For an alternative to CREDSSP, review the following Microsoft blog post:
"PowerShell Remoting Kerberos Double Hop Solved Securely" -
https://blogs.technet.microsoft.com/ashleymcglone/2016/08/30/powershell-remoting-kerberos-double-hop-solved-
securely/
 Enable-WSManCredSSP -role server

On the management computer:

Enable-WSManCredSSP Client -DelegateComputer <remote server name>

$CustomCred = Get-Credential

Invoke-Command -ComputerName sr-srv01 -ScriptBlock { Test-SRTopology <commands> } -Authentication

Credssp -Credential $CustomCred

Then copy the results to your management computer or share the path. Because Nano lacks the necessary
graphical libraries, you can use Test-SRTopology to process the results and give you a report file with charts.
For example:

Test-SRTopology -GenerateReport -DataPath \\sr-srv05\c$\temp

How do I see the progress of replication during initial sync?

The Event 1237 messages shown in the Storage Replica Admin even log on the destination server show number of
bytes copied and bytes remaining every 10 seconds. You can also use the Storage Replica performance counter on
the destination showing \Storage Replica Statistics\Total Bytes Received for one or more replicated volumes.
You can also query the replication group using Windows PowerShell. For instance, this sample command gets the
name of the groups on the destination then queries one group named Replication 2 every 10 seconds to show
progress:

Get-SRGroup

do{
$r=(Get-SRGroup -Name "Replication 2").replicas
[System.Console]::Write("Number of remaining bytes {0}`n", $r.NumOfBytesRemaining)
Start-Sleep 10
}until($r.ReplicationStatus -eq 'ContinuouslyReplicating')
Write-Output "Replica Status: "$r.replicationstatus

Can I specify specific network interfaces to be used for replication?

Yes, using Set-SRNetworkConstraint . This cmdlet operates at the interface layer and be used on both cluster and
non-cluster scenarios.
For example, with a standalone server (on each node):

Get-SRPartnership

Get-NetIPConfiguration

Note the gateway and interface information (on both servers) and the partnership directions. Then run:
 Set-SRNetworkConstraint -SourceComputerName sr-srv06 -SourceRGName rg02 -
SourceNWInterface 2 -DestinationComputerName sr-srv05 -DestinationNWInterface 3 -DestinationRGName rg01

Get-SRNetworkConstraint

Update-SmbMultichannelConnection

For configuring network constraints on a stretch cluster:

Set-SRNetworkConstraint -SourceComputerName sr-srv01 -SourceRGName group1 -SourceNWInterface "Cluster Network

1","Cluster Network 2" -DestinationComputerName sr-srv03 -DestinationRGName group2 -DestinationNWInterface
"Cluster Network 1","Cluster Network 2"

Can I configure one-to-many replication or transitive (A to B to C)

replication?
Not in Windows Server 2016. This release only supports one to one replication of a server, cluster, or stretch cluster
node. This may change in a later release. You can of course configure replication between various servers of a
specific volume pair, in either direction. For instance, Server 1 can replicate its D volume to server 2, and its E
volume from Server 3.

Can I grow or shrink replicated volumes replicated by Storage Replica?

You can grow (extend) volumes, but not shrink them. By default, Storage Replica prevents administrators from
extending replicated volumes; use the Set-SRGroup -AllowVolumeResize $TRUE option on the source group, prior to
resizing. For example:
1. Use against the source computer: Set-SRGroup -Name YourRG -AllowVolumeResize $TRUE
2. Grow the volume using whatever technique you prefer
3. Use against the source computer: Set-SRGroup -Name YourRG -AllowVolumeResize $FALSE

Can I bring a destination volume online for read-only access?

Not in Windows Server 2016. Storage Replica dismounts the destination volume and its drive letter or mount point
when replication begins. This may change in a later release.

Can I configure Scale-out File Server (SOFS) in a stretch cluster?

While technically possible, this is not a recommended configuration in Windows Server 2016 due to the lack of site
awareness in the compute nodes contacting the SOFS. If using campus-distance networking, where latencies are
typically sub-millisecond, this configuration typically works works without issues.
If configuring cluster-to-cluster replication, Storage Replica fully supports Scale-out File Servers, including the use
of Storage Spaces Direct, when replicating between two clusters.

Can I configure Storage Spaces Direct in a stretch cluster with Storage

Replica?
This is not a supported configuration in Windows Server 2016. This may change in a later release. If configuring
cluster-to-cluster replication, Storage Replica fully supports Scale Out File Servers and Hyper-V Servers, including
the use of Storage Spaces Direct.
How do I configure asynchronous replication?
Specify New-SRPartnership -ReplicationMode and provide argument Asynchronous. By default, all replication in
Storage Replica is synchronous. You can also change the mode with Set-SRPartnership -ReplicationMode .

How do I prevent automatic failover of a stretch cluster?

To prevent automatic failover, you can use PowerShell to configure
Get-ClusterNode -Name "NodeName").NodeWeight=0 . This removes the vote on each node in the disaster recovery site.
Then you can use Start-ClusterNode -PreventQuorum on nodes in the primary site and
Start-ClusterNode -ForceQuorum on nodes in the disaster site to force failover. There is no graphical option for
preventing automatic failover, and preventing automatic failover is not recommended.

How do I disable virtual machine resiliency?

To prevent the new Hyper-V virtual machine resiliency feature from running and therefore pausing virtual
machines instead of failing them over to the disaster recovery site, run (Get-Cluster).ResiliencyDefaultPeriod=0

How can I reduce time for initial synchronization?

You can use thin-provisioned storage as one way to speed up initial sync times. Storage Replica queries for and
automatically uses thin-provisioned storage, including non-clustered Storage Spaces, Hyper-V dynamic disks, and
SAN LUNs.
You can also use seeded data volumes to reduce bandwidth usage and sometimes time, by ensuring that the
destination volume has some subset of data from the primary - via a restored backup, old snapshot, previous
replication, copied files, etc. - then using the Seeded option in Failover Cluster Manager or New-SRPartnership . If the
volume is mostly empty, using seeded sync may reduce time and bandwidth usage.

Can I delegate users to administer replication?

You can use the Grant-SRDelegation cmdlet in Windows Server 2016. This allows you to set specific users in server
to server, cluster to cluster, and stretch cluster replication scenarios as having the permissions to create, modify, or
remove replication, without being a member of the local administrators group. For example:

Grant-SRDelegation -UserName contso\tonywang

The cmdlet will remind you that the user needs to log off and on of the server they are planning to administer in
order for the change to take effect. You can use Get-SRDelegation and Revoke-SRDelegation to further control this.

What are my backup and restore options for replicated volumes?

Storage Replica supports backing up and restoring the source volume. It also supports creating and restoring
snapshots of the source volume. You cannot backup or restore the destination volume while protected by Storage
Replica, as it is not mounted nor accessible. If you experience a disaster where the source volume is lost, using
Set-SRPartnership to promote the previous destination volume to now be a read/writable source will allow you to
backup or restore that volume. You can also remove replication with Remove-SRPartnership and Remove-SRGroup to
remount that volume as read/writable. To create periodic application consistent snapshots, you can use
VSSADMIN.EXE on the source server to snapshot replicated data volumes. For example, where you are replicating
the F: volume with Storage Replica:

vssadmin create shadow /for=F:

Then, after you switch replication direction, remove replication, or are simply still on the same source volume, you
can restore any snapshot to its point in time. For example, still using F:

vssadmin list shadows

vssadmin revert shadow /shadow={shadown copy ID GUID listed previously}

You can also schedule this tool to run periodically using a scheduled task. For more information on using VSS,
review Vssadmin. There is no need or value in backing up the log volumes. Attempting to do so will be ignored by
VSS. Use of Windows Server Backup, Microsoft Azure Backup, Microsoft DPM, or other snapshot, VSS, virtual
machine, or file-based technologies are supported by Storage Replica as long as they operate within the volume
layer. Storage Replica does not support block-based backup and restore.

Can I configure replication to restrict bandwidth usage?

Yes, via the SMB bandwidth limiter. This is a global setting for all Storage Replica traffic and therefore affects all
replication from this server. Typically, this is needed only with Storage Replica initial sync setup, where all the
volume data must transfer. If needed after initial sync, your network bandwidth is too low for your IO workload;
reduce the IO or increase the bandwidth.
This should only be used with asynchronous replication (note: initial sync is always asynchronous even if you have
specified synchronous). You can also use network QoS policies to shape Storage Replica traffic. Use of highly
matched seeded Storage Replica replication will also lower overall initial sync bandwidth usage considerably.
To set the bandwidth limit, use:

Set-SmbBandwidthLimit -Category StorageReplication -BytesPerSecond x

To see the bandwidth limit, use:

Get-SmbBandwidthLimit -Category StorageReplication

To remove the bandwidth limit, use:

Remove-SmbBandwidthLimit -Category StorageReplication

What network ports does Storage Replica require?

Storage Replica relies on SMB and WSMAN for its replication and management. This means the following ports are
required:
445 (SMB - replication transport protocol) 5445 (iWARP SMB - only needed when using iWARP RDMA networking)
5895 (WSManHTTP - Management protocol for WMI/CIM/PowerShell)
Note: The Test-SRTopology cmdlet requires ICMPv4/ICMPv6, but not for replication or management.

What are the log volume best practices?

Storage Replica relies on the log for all write performance. Log performance critical to replication performance. You
can use DISKSPD.EXE to measure your log performance along with Performance Counters to analyze existing data
volume performance and ensure that the log volume performs as well or better than the data volume. You should
always use flash media like SSD on log volumes. You must never allow any other workloads to run on the log
volume, the same way you would never allow other workloads to run on SQL database log volumes. The size of the
log or log volume has no real effect on performance, only on recovery time.
Microsoft recommends that the log storage be faster than the data storage. Log volumes must never be used for
other workloads.

Why would you choose a stretch cluster versus cluster-to-cluster versus

server-to-server topology?
Storage Replica comes in three main configurations: strech cluster, cluster-to-cluster, and server-to-server. There
are different advantages to each.
The stretch cluster topology is ideal for workloads requiring automatic failover with orchestration, such as Hyper-V
private cloud clusters and SQL Server FCI. It also has a built-in graphical interface using Failover Cluster Manager. It
utilizes the classic asymmetric cluster shared storage architecture of Storage Spaces, SAN, iSCSI, and RAID via
presistent reservation. You can run this with as few as 2 nodes.
The cluster-to-cluster topology uses two separate clusters and is ideal for administrators who want manual failover,
especially when the second site is provisioned for disaster recovery and not everyday usage. Orchestration is
manual. Unlike stretch cluster, Storage Spaces Direct can be used in this configuration. You can run this with as few
as four nodes.
The server-to-server topology is ideal for customers running hardware that cannot be clustered. It requires manual
failover and orchestration. It is ideal for inexpensive deployments between branch offices and central datacenters,
especially when using asynchronous replication. This configuration can often replace instances of DFSR-protected
File Servers used for single-master disaster recovery scenarios.
In all cases, the topologies support both running on physical hardware as well as virtual machines. When in virtual
machines, the underlying hypervisor doesn't require Hyper-V; it can be VMware, KVM, Xen, etc.
Storage Replica also has a server-to-self mode, where you point replication to two different volumes on the same
computer.

How do I report an issue with Storage Replica or this guide?

For technical assistance with Storage Replica, you can post at the Microsoft TechNet forums. You can also email
[email protected] for questions on Storage Replica or issues with this documentation. The
https://windowsserver.uservoice.com site is preferred for design change requests, as it allows your fellow
customers to provide support and feedback for your ideas.

Related Topics
Storage Replica Overview
Stretch Cluster Replication Using Shared Storage
Server to Server Storage Replication
Cluster to Cluster Storage Replication
Storage Replica: Known Issues

See Also
Storage Overview
Storage Spaces Direct in Windows Server 2016
 Advanced Data Deduplication settings
4/24/2017 12 min to read Edit Online

Applies to Windows Server 2016

This document describes how to modify advanced Data Deduplication settings. For recommended workloads, the
default settings should be sufficient. The main reason to modify these settings is to improve Data Deduplication's
performance with other kinds of workloads.

Modifying Data Deduplication job schedules

The default Data Deduplication job schedules are designed to work well for recommended workloads and be as
non-intrusive as possible (excluding the Priority Optimization job that is enabled for the Backup usage type). When
workloads have large resource requirements, it is possible to ensure that jobs run only during idle hours, or to
reduce or increase the amount of system resources that a Data Deduplication job is allowed to consume.
Changing a Data Deduplication schedule
Data Deduplication jobs are scheduled via Windows Task Scheduler and can be viewed and edited there under the
path Microsoft\Windows\Deduplication. Data Deduplication includes several cmdlets that make scheduling easy.
Get-DedupSchedule shows the current scheduled jobs.
New-DedupSchedule creates a new scheduled job.
Set-DedupSchedule modifies an existing scheduled job.
Remove-DedupSchedule removes a scheduled job.

The most common reason for changing when Data Deduplication jobs run is to ensure that jobs run during off
hours. The following step-by-step example shows how to modify the Data Deduplication schedule for a sunny day
scenario: a hyper-converged Hyper-V host that is idle on weekends and after 7:00 PM on weeknights. To change the
schedule, run the following PowerShell cmdlets in an Administrator context.
1. Disable the scheduled hourly Optimization jobs.

Set-DedupSchedule -Name BackgroundOptimization -Enabled $false

Set-DedupSchedule -Name PriorityOptimization -Enabled $false

2. Remove the currently scheduled Garbage Collection and Integrity Scrubbing jobs.

Get-DedupSchedule -Type GarbageCollection | ForEach-Object { Remove-DedupSchedule -InputObject $_ }

Get-DedupSchedule -Type Scrubbing | ForEach-Object { Remove-DedupSchedule -InputObject $_ }

3. Create a nightly Optimization job that runs at 7:00 PM with high priority and all the CPUs and memory
available on the system.

New-DedupSchedule -Name "NightlyOptimization" -Type Optimization -DurationHours 11 -Memory 100 -Cores

100 -Priority High -Days @(1,2,3,4,5) -Start (Get-Date "2016-08-08 19:00:00")
 NOTE
The date part of the System.Datetime provided to -Start is irrelevant (as long as it's in the past), but the time
part specifies when the job should start.

4. Create a weekly Garbage Collection job that runs on Saturday starting at 7:00 AM with high priority and all
the CPUs and memory available on the system.

New-DedupSchedule -Name "WeeklyGarbageCollection" -Type GarbageCollection -DurationHours 23 -Memory 100

-Cores 100 -Priority High -Days @(6) -Start (Get-Date "2016-08-13 07:00:00")

5. Create a weekly Integrity Scrubbing job that runs on Sunday starting at 7 AM with high priority and all the
CPUs and memory available on the system.

New-DedupSchedule -Name "WeeklyIntegrityScrubbing" -Type Scrubbing -DurationHours 23 -Memory 100 -Cores

100 -Priority High -Days @(0) -Start (Get-Date "2016-08-14 07:00:00")

Available job-wide settings

You can toggle the following settings for new or scheduled Data Deduplication jobs:

WHY WOULD YOU WANT TO

PARAMETER NAME DEFINITION ACCEPTED VALUES SET THIS VALUE?

Type The type of the job that Optimization This value is required
should be scheduled GarbageCollection because it is the type of job
Scrubbing that you want to have be
scheduled. This value cannot
be changed after the task
has been scheduled.

Priority The system priority of the High This value helps the system
scheduled job Medium determine how to allocate
Low CPU time. High will use
more CPU time, low will use
less.

Days The days that the job is An array of integers 0-6 Scheduled tasks have to run
scheduled representing the days of the on at least one day.
week:
0 = Sunday
1 = Monday
2 = Tuesday
3 = Wednesday
4 = Thursday
5 = Friday
6 = Saturday

Cores The percentage of cores on Integers 0-100 (indicates a To control what level of
the system that a job should percentage) impact a job will have on the
use compute resources on the
system
 WHY WOULD YOU WANT TO
PARAMETER NAME DEFINITION ACCEPTED VALUES SET THIS VALUE?

DurationHours The maximum number of Positive integers To prevent a job for running
hours a job should be into a workload's non-idle
allowed to run hours

Enabled Whether the job will run True/false To disable a job without
removing it

Full For scheduling a full Garbage Switch (true/false) By default, every fourth job
Collection job is a full Garbage Collection
job. With this switch, you can
schedule full Garbage
Collection to run more
frequently.

InputOutputThrottle Specifies the amount of Integers 0-100 (indicates a Throttling ensures that jobs
input/output throttling percentage) don't interfere with other
applied to the job I/O-intensive processes.

Memory The percentage of memory Integers 0-100 (indicates a To control what level of
on the system that a job percentage) impact the job will have on
should use the memory resources of the
system

Name The name of the scheduled String A job must have a uniquely
job identifiable name.

ReadOnly Indicates that the scrubbing Switch (true/false) You want to manually
job processes and reports on restore files that sit on bad
corruptions that it finds, but sections of the disk.
does not run any repair
actions

Start Specifies the time a job System.DateTime The date part of the
should start System.Datetime provided
to Start is irrelevant (as long
as it's in the past), but the
time part specifies when the
job should start.

StopWhenSystemBusy Specifies whether Data Switch (True/False) This switch gives you the
Deduplication should stop if ability to control the
the system is busy behavior of Data
Deduplication--this is
especially important if you
want to run Data
Deduplication while your
workload is not idle.

Modifying Data Deduplication volume-wide settings

Toggling volume settings
You can set the volume-wide default settings for Data Deduplication via the usage type that you select when you
enable a deduplication for a volume. Data Deduplication includes cmdlets that make editing volume-wide settings
easy:
 Get-DedupVolume
Set-DedupVolume

The main reasons to modify the volume settings from the selected usage type are to improve read performance for
specific files (such as multimedia or other file types that are already compressed) or to fine-tune Data Deduplication
for better optimization for your specific workload. The following example shows how to modify the Data
Deduplication volume settings for a workload that most closely resembles a general purpose file server workload,
but uses large files that change frequently.
1. See the current volume settings for Cluster Shared Volume 1.

Get-DedupVolume -Volume C:\ClusterStorage\Volume1 | Select *

2. Enable OptimizePartialFiles on Cluster Shared Volume 1 so that the MinimumFileAge policy applies to
sections of the file rather than the whole file. This ensures that the majority of the file gets optimized even
though sections of the file change regularly.

Set-DedupVolume -Volume C:\ClusterStorage\Volume1 -OptimizePartialFiles

Available volume -wide settings

WHY WOULD YOU WANT TO
SETTING NAME DEFINITION ACCEPTED VALUES MODIFY THIS VALUE?

ChunkRedundancyThreshold The number of times that a Positive integers The main reason to modify
chunk is referenced before a this number is to increase
chunk is duplicated into the the savings rate for volumes
hotspot section of the with high duplication. In
Chunk Store. The value of general, the default value
the hotspot section is that (100) is the recommended
so-called "hot" chunks that setting, and you shouldn't
are referenced frequently need to modify this.
have multiple access paths
to improve access time.

ExcludeFileType File types that are excluded Array of file extensions Some file types, particularly
from optimization multimedia or files that are
already compressed, do not
benefit very much from
being optimized. This setting
allows you to configure
which types are excluded.

ExcludeFolder Specifies folder paths that Array of folder paths If you want to improve
should not be considered for performance or keep content
optimization in particular paths from
being optimized, you can
exclude certain paths on the
volume from consideration
for optimization.
 WHY WOULD YOU WANT TO
SETTING NAME DEFINITION ACCEPTED VALUES MODIFY THIS VALUE?

InputOutputScale Specifies the level of IO Positive integers ranging 1- The main reason to modify
parallelization (IO queues) 36 this value is to decrease the
for Data Deduplication to impact on the performance
use on a volume during a of a high IO workload by
post-processing job restricting the number of IO
queues that Data
Deduplication is allowed to
use on a volume. Note that
modifying this setting from
the default may cause Data
Deduplication's post-
processing jobs to run
slowly.

MinimumFileAgeDays Number of days after the file Positive integers (inclusive of The Default and HyperV
is created before the file is zero) usage types set this value to
considered to be in-policy 3 to maximize performance
for optimization. on hot or recently created
files. You may want to
modify this if you want Data
Deduplication to be more
aggressive or if you do not
care about the extra latency
associated with
deduplication.

MinimumFileSize Minimum file size that a file Positive integers (bytes) The main reason to change
must have to be considered greater than 32 KB this value is to exclude small
in-policy for optimization files that may have limited
optimization value to
conserve compute time.

NoCompress Whether the chunks should True/False Some types of files,

be compressed before being particularly multimedia files
put into the Chunk Store and already compressed file
types, may not compress
well. This setting allows you
to turn off compression for
all files on the volume. This
would be ideal if you are
optimizing a dataset that has
a lot of files that are already
compressed.

NoCompressionFileType File types whose chunks Array of file extensions Some types of files,
should not be compressed particularly multimedia files
before going into the Chunk and already compressed file
Store types, may not compress
well. This setting allows
compression to be turned off
for those files, saving CPU
resources.
 WHY WOULD YOU WANT TO
SETTING NAME DEFINITION ACCEPTED VALUES MODIFY THIS VALUE?

OptimizeInUseFiles When enabled, files that True/false Enable this setting if your
have active handles against workload keeps files open for
them will be considered as extended periods of time. If
in-policy for optimization. this setting is not enabled, a
file would never get
optimized if the workload
has an open handle to it,
even if it's only occasionally
appending data at the end.

OptimizePartialFiles When enabled, the True/false Enable this setting if your

MinimumFileAge value workload works with large,
applies to segments of a file often edited files where most
rather than to the whole file. of the file content is
untouched. If this setting is
not enabled, these files
would never get optimized
because they keep getting
changed, even though most
of the file content is ready to
be optimized.

Verify When enabled, if the hash of True/false This is an integrity feature

a chunk matches a chunk we that ensures that the
already have in our Chunk hashing algorithm that
Store, the chunks are compares chunks does not
compared byte-by-byte to make a mistake by
ensure they are identical. comparing two chunks of
data that are actually
different but have the same
hash. In practice, it is
extremely improbable that
this would ever happen.
Enabling the verification
feature adds significant
overhead to the optimization
job.

Modifying Data Deduplication system-wide settings

Data Deduplication has additional system-wide settings that can be configured via the registry. These settings apply
to all of the jobs and volumes that run on the system. Extra care must be given whenever editing the registry.
For example, you may want to disable full Garbage Collection. More information about why this may be useful for
your scenario can be found in Frequently asked questions. To edit the registry with PowerShell:
If Data Deduplication is running in a cluster:

Set-ItemProperty -Path HKLM:\System\CurrentControlSet\Services\ddpsvc\Settings -Name DeepGCInterval -

Type DWord -Value 0xFFFFFFFF
Set-ItemProperty -Path HKLM:\CLUSTER\Dedup -Name DeepGCInterval -Type DWord -Value 0xFFFFFFFF

If Data Deduplication is not running in a cluster:

 Set-ItemProperty -Path HKLM:\System\CurrentControlSet\Services\ddpsvc\Settings -Name DeepGCInterval -
Type DWord -Value 0xFFFFFFFF

Available system-wide settings

WHY WOULD YOU WANT TO
SETTING NAME DEFINITION ACCEPTED VALUES CHANGE THIS?

WlmMemoryOverPercentThr This setting allows jobs to Positive integers (a value of If you have another task that
eshold use more memory than Data 300 means 300% or 3 times) will stop if Data
Deduplication judges to Deduplication takes more
actually be available. For memory
example, a setting of 300
would mean that the job
would have to use three
times the assigned memory
to get canceled.

DeepGCInterval This setting configures the Integers (-1 indicates See this frequently asked
interval at which regular disabled) question
Garbage Collection jobs
become full Garbage
Collection jobs. A setting of
n would mean that every nth
job was a full Garbage
Collection job. Note that full
Garbage Collection is always
disabled (regardless of the
registry value) for volumes
with the Backup Usage Type.
Start-DedupJob -Type
GarbageCollection -Full
may be used if full Garbage
Collection is desired on a
Backup volume.

Frequently asked questions

I changed a Data Deduplication setting, and now jobs are slow or don't finish, or my workload
performance has decreased. Why?
These settings give you a lot of power to control how Data Deduplication runs. Use them responsibly, and monitor
performance.
I want to run a Data Deduplication job right now, but I don't want to create a new schedule--can I do
this?
Yes, all jobs can be run manually.
What is the difference between full and regular Garbage Collection?
There are two types of Garbage Collection:
Regular Garbage Collection uses a statistical algorithm to find large unreferenced chunks that meet a certain
criteria (low in memory and IOPs). Regular Garbage Collection compacts a chunk store container only if a
minimum percentage of the chunks are unreferenced. This type of Garbage Collection runs much faster and uses
fewer resources than full Garbage Collection. The default schedule of the regular Garbage Collection job is to run
once a week.
Full Garbage Collection does a much more thorough job of finding unreferenced chunks and freeing more disk
space. Full Garbage Collection compacts every container even if just a single chunk in the container is
unreferenced. Full Garbage Collection will also free space that may have been in use if there was a crash or
 power failure during an Optimization job. Full Garbage Collection jobs will recover 100 percent of the available
space that can be recovered on a deduplicated volume at the cost of requiring more time and system resources
compared to a regular Garbage Collection job. The full Garbage Collection job will typically find and release up
to 5 percent more of the unreferenced data than a regular Garbage Collection job. The default schedule of the
full Garbage Collection job is to run every fourth time Garbage Collection is scheduled.
Why would I want to disable full Garbage Collection?
Garbage Collection could adversely affect the volume's lifetime shadow copies and the size of incremental
backup. High churn or I/O-intensive workloads may see a degradation in performance by full Garbage Collection
jobs.
You can manually run a full Garbage Collection job from PowerShell to clean up leaks if you know your system
crashed.
 Performance Tuning for Powershell
5/3/2017 1 min to read Edit Online

This document discusses general guidelines for achieving the best possible performance of PowerShell 5.1. Some
of the issues described in this document may be addressed in future versions.
This document does not describe best practices.
The guidance in this document should be applied in a thoughtful manner.
Performance is often not an issue, saving 10ms or 100ms might go completely unnoticed.
Some guidance in this document describes atypical PowerShell usage that may be confusing and unfamiliar to
some PowerShell users.
The following topics provide specific guidance.
Script Authoring Considerations
Module Authoring Considerations
 PowerShell scripting performance considerations
5/3/2017 2 min to read Edit Online

PowerShell scripts that leverage .NET directly and avoid the pipeline tend to be faster than idiomatic PowerShell.
Idiomatic PowerShell typically uses cmdlets and PowerShell functions heavily, often leveraging the pipeline, and
dropping down into .NET only when necessary.

NOTE
Many of the techniques described here are not idiomatic PowerShell and may reduce the readability of a PowerShell script.
Script authors are advised to use idiomatic PowerShell unless performance dictates otherwise.

Suppressing Output
There are many ways to avoid writing objects to the pipeline:

$null = $arrayList.Add($item)
[void]$arrayList.Add($item)

Assignment to $null or casting to [void] are roughly equivalent and should generally be preferred where
performance matters.

$arrayList.Add($item) > $null

File redirection to $null is nearly as good as the previous alternatives, most scripts would never notice the
difference. Depending on the scenario, file redirection does introduce a little bit of overhead though.

$arrayList.Add($item) | Out-Null

Piping to Out-Null has significant overhead when compared to the alternatives. It should be avoiding in
performance sensitive code.

$null = . {
$arrayList.Add($item)
$arrayList.Add(42)
}

Introducing a script block and calling it (using dot sourcing or otherwise) then assigning the result to $null is a
convenient technique for suppressing the output of a large block of script. This technique performs roughly as well
as piping to Out-Null and should be avoided in performance sensitive script. The extra overhead in this example
comes from the creation of and invoking a script block that was previously inline script.

Array Addition
Generating a list of items is often done using an array with the addition operator:
 $results = @()
$results += Do-Something
$results += Do-SomethingElse
$results

This can be very inefficent because arrays are immutable. Each addition to the array actually creates a new array
big enough to hold all elements of both the left and right operands, then copies the elements of both operands into
the new array. For small collections, this overhead may not matter. For large collections, this can definitely be an
issue.
There are a couple of alternatives. If you don't actually require an array, instead consider using an ArrayList:

$results = [System.Collections.ArrayList]::new()
$results.AddRange((Do-Something))
$results.AddRange((Do-SomethingElse))
$results

If you do require an array, you can use your own ArrayList and simply call ArrayList.ToArray when you want the
array. Alternatively, you can let PowerShell create the ArrayList and Array for you:

$results = @(
Do-Something
Do-SomethingElse
)

In this example, PowerShell creates an ArrayList to hold the results written to the pipeline inside the array
expression. Just before assigning to $results , PowerShell converts the ArrayList to an object[] .

Processing Large Files

The idiomatic way to process a file in PowerShell might look something like:

Get-Content $path | Where-Object { $_.Length -gt 10 }

This can be nearly an order of magnitude slower than using .NET APIs directly:

try
{
$stream = [System.IO.StreamReader]::new($path)
while ($line = $stream.ReadLine())
{
if ($line.Length -gt 10)
{
$line
}
}
}
finally
{
$stream.Dispose()
}

Avoid Write-Host
It is generally considered poor practice to write output directly to the console, but when it makes sense, many
scripts use Write-Host .
If you must write many messages to the console, Write-Host can be an order of magnitude slower than
[Console]::WriteLine() .
 PowerShell module authoring considerations
5/3/2017 1 min to read Edit Online

This document includes some guidelines related to how a module is authored for best performance.

Module Manifest Authoring

A module manifest that does not use the following guidelines can have a noticeable impact on general PowerShell
performance even if the module is not used in a session.
Command auto-discovery analyzes each module to determine which commands the module exports and this
analysis can be expensive. The results of module analysis are cached per user, but the cache isn't available on first
run, which is a typical scenario with containers. During module analysis, if the exported commands can be fully
determined from the manifest, more expensive analysis of the module can be avoided.
Guidelines
In the module manifest, do not use wildcards in the AliasesToExport , CmdletsToExport , and
FunctionsToExport entries.

If the module does not export commands of a particular type, specify this explicitly in the manifest by
specifying @() . A missing or $null entry is equivalent to specifying the wildcard * .
The following should be avoided where possible:

@{
FunctionsToExport = '*'

# Also avoid omitting an entry, it is equivalent to using a wildcard

# CmdletsToExport = '*'
# AliasesToExport = '*'
}

Instead, use:

@{
FunctionsToExport = 'Format-Hex', 'Format-Octal'
CmdletsToExport = @() # Specify an empty array, not $null
AliasesToExport = @() # Also ensure all three entries are present
}

Avoid CDXML
When deciding how to implement your module, there are three primary choices:
Binary (usually C#)
Script (PowerShell)
CDXML (an XML file wrapping CIM)
If the speed of loading your module is important, CDXML is roughly an order of magnitude slower than a binary
module.
A binary module loads the fastest because it is compiled ahead of time and can use NGen to JIT compile once per
machine.
A script module typically loads a bit more slowly than a binary module because PowerShell must parse the script
before compiling and executing it.
A CDXML module is typically much slower than a script module because it must first parse an XML file which then
generates quite a bit of PowerShell script that is then parsed and compiled.
 Additional performance tuning resources
4/24/2017 1 min to read Edit Online

Use the links in this topic to learn more about the concepts that were discussed in this tuning guide.

Microsoft Windows Server Websites

Windows Server Catalog
Windows Sysinternals
Transaction Processing Performance Council
Windows Assessment and Deployment Kit

Power Management Tuning Resources

Power Policy Configuration and Deployment in Windows
Using PowerCfg to Evaluate System Energy Efficiency
Interrupt-Affinity Policy Tool

Networking Subsystem Tuning Resources

Scalable Networking: Eliminating the Receive Processing BottleneckIntroducing RSS
Windows Filtering Platform
Networking Deployment Guide: Deploying High-Speed Networking Features

Storage Subsystem Tuning Resources

Disk Subsystem Performance Analysis for Windows (Parts of this document are out of date, but many of the
general observations and guidelines captured are still accurate and relevant.)

File Server Tuning Resources

Performance Tuning Guidelines for Microsoft Services for Network File System
[MS-FSSO]: File Access Services System Overview
How to disable the TCP autotuning diagnostic tool

Active Directory Server Tuning Resources

Active Directory Performance
How to configure Active Directory diagnostic event logging in Windows Server 2003 and in Windows 2000
Server

Virtualization Server Tuning Resources

Whats New in Hyper-V in Windows Server 2016
 Hyper-V Dynamic Memory Configuration Guide
NUMA Node Balancing
Hyper-V WMI Provider
Hyper-V WMI Classes
About Virtual Machines and Guest Operating Systems
Optimizing and Troubleshooting Hyper-V Storage
Optimizing and Troubleshooting Hyper-V Networking

Print Server Tuning Resources

Print Server Scalability and Capacity Planning

Server Workload Tuning Resources

Performance Tuning for NTttcp
Ttcp
How to use NTttcp to Test Network Performance
Using the File Server Capactiy Tool
Using the SPECsfs2008 File Server
Performance Tuning for the Sales and Distribution Workload
Performance Tuning for Online Transaction Processing (OLTP)
How to: Configure SQL Server to Use Soft-NUMA
How to: Map TCP/IP Ports to NUMA Nodes
ALTER SERVER CONFIGURATION (Transact-SQL)

Server Tuning Tools

Microsoft Server Performance Advisor

Performance Tuning Guidelines for previous versions of Windows

Server
Use the performance tuning guidelines to improve performance for older versions of Windows Server.
Heres a list of performance tuning guidelines for previous versions of Windows Server:
Performance Tuning Guidelines for Windows Server 2012 R2
Performance Tuning Guidelines for Windows Server 2012
Performance Tuning Guidelines for Windows Server 2008 R2
Performance Tuning Guidelines for Windows Server 2008