Dell PowerEdge Concepts and Features

DELL POWEREDGE
CONCEPTS AND
FEATURES
PARTICIPANT GUIDE
PARTICIPANT GUIDE
Dell PowerEdge Concepts and Features
© Copyright 2023 Dell Inc Page 2

Table of Contents
Dell PowerEdge Concepts and Features 6
Portfolio Overview 7
Introduction to Server Portfolio 7
Dell PowerEdge Naming Convention 7
Service Tag and Asset Tag 8
PowerEdge 13G Servers 10
PowerEdge 13G Specifications 12
PowerEdge 13G Control Panel 15
PowerEdge 14G Control Panel 20
Server Components 25
Introduction 25
Processors 27
Memory 33
Power 59
Cooling 68
Networking 97
Graphic Processing Units (GPUs) 106
Expansion Card 115
Storage 120
Introduction to Server Security 187

Server Security Overview 187
Trusted Platform Module 189
Trusted Platform Module 2.0 192
Configuring the TPM 193
Silicon-Based Hardware Root of Trust (RoT) 194

BIOS Live Scanning 195
Intel Boot Guard 197
Boot Guard Process 200
Automated BIOS Recovery 201
Secure Boot 202
Secure Boot Policy 204
Secure Erase for Self Encrypting Disks (SEDs) 204
System Lockdown mode in iDRAC 205
Dell Technologies Secured Component Verification 206
Configuration Validation 208

Overview 208
Hardware Configuration Error Messages 209
HWC8010 and HW8011 Error Interpretations 211
Dell PowerEdge Concepts and Features - Appendix 213

RAID in System Setup Utility 213
BOSS CLI Commands 213
BOSS Management on OMSA 214
BOSS Management on System Setup Utility 214
BOSS Management on iDRAC 215
Rear Facing Module 215
PowerEdge 750xa GPU Configurations 215
RAID in OMSA 216
RAID in Lifecycle Controller 217
iDRAC Liquid Leak Error 217
Single DIMM Blanks 218
PowerEdge XR11 Fans and Heatsinks 218
15G Memory Population Guidelines 219
PowerEdge Thermals - FAQs 221
Different RIO Cards 226
NVMe Support 226

Dell PowerEdge Concepts and Features - Appendix 227
cStack Cable on PowerEdge 13G server 227

Portfolio Overview

Portfolio Overview
Portfolio Overview
Introduction to Server Portfolio
Movie:
The web version of this content contains a movie.
Click the link below to watch an introduction to the Dell server portfolio.
https://edutube.dell.com/Player.aspx?autoplay=true&vno=ZICcxeiVyyvuY3T8JG72
ng
Dell PowerEdge Naming Convention
Dell PowerEdge servers with common design components are identified by the
server model name.
The server naming convention provides insight into the form factor, class of
system, generation, and the CPU socket count.
Three Number Naming Convention

Portfolio Overview
Four Number Naming Convention
Important:
• The PowerEdge XE family of servers is purpose-built
for complex, emerging workloads that require high-
performance and large storage. For example, the
PowerEdge XE8545.
• The PowerEdge XR family of servers is ruggedized,
industrial-grade servers intended for extreme
environments. For example, PowerEdge XR11/XR12.
Service Tag and Asset Tag
The Dell service tag is a seven-character identifier that is unique to the product.
• The service tag of a PowerEdge server is a pullout tab also known as an
Enterprise Service Tag (EST). ESTs are typically located on the front or rear of
the chassis.
• Information about the service tag can also be found on a sticker typically on the
side of the chassis, and in the server BIOS.

Portfolio Overview
All Dell PowerEdge servers have a Service Tag and can have an Asset Tag
added.
The Asset Tag is an empty field within BIOS where you can input your own
identifying information such as the system’s security number or location ID.
Movie:
The web version of this content contains a movie.
Important: In addition to the service tag, there is an Express

Service Code which is a 10-digit or more numeric version of the
service tag. The code can be typed into a telephone for call
routing.
Click the link below to watch a demo video on locating the service tag in a
PowerEdge server.
https://edutube.dell.com/Player.aspx?autoplay=true&vno=DnD1V|@$@|UUKQ2lR
tWtNqDYMw

Portfolio Overview
PowerEdge 13G Servers
The PowerEdge 13G servers feature:

• Incredible density across a range of resources enabling highly flexible data
center scaling.
• Intel Xeon processors.
• PowerEdge RAID Controller 9 series cards for server storage expansion.
• Flexible storage options, including in-box storage tiering1.
• Reliability, Availability, and Storage (RAS) characteristics.
Below are several of 13G PowerEdge servers.
PowerEdge R630
PowerEdge R630
1A tiered storage solution enables organizations to move older, less important,

and infrequently accessed data to less expensive storage solutions, while
maintaining active and important data on high-performing storage media.

Portfolio Overview
PowerEdge R730xd
PowerEdge R730xd
PowerEdge R930
PowerEdge R930
Tip: For more information about the PowerEdge 13G servers, go

to dell.com/support

Portfolio Overview
PowerEdge 13G Specifications
1: Processor: PowerEdge R930 of the 13th generation supports up to x4 Intel

Xeon E7-8800 product family processors.
2: Power Supplies: 1100 W DC power supply
Titanium efficiency 750 W AC power supply
Platinum efficiency 495 W, 750 W, 1100 W AC power supply
3: Expansion and Network: Up to x7 PCIe 3.0 slots, supports a dedicated RAID

card slot and a dedicated NDC slot.
Embedded NIC- 1 GbE x4 (this port is for management only). The NDC is present
in different versions ranging from 4x 1 GB, 2x 1 Gb + 2x 10 GB or 4x 10 GB.

Portfolio Overview
4: Embedded Management: IPMI 2.0 compliant
iDRAC8 with Lifecycle Controller
iDRAC8 Express (default) or Enterprise (upgradable):

• 8 GB vFlash media (optional)
• 16 GB vFlash media (optional)
5: Memory:
Supported in the form of RDIMMs and LRDIMMs.
Up to 3 TB (24 DIMM slots): 4 GB/8 GB/16 GB/32 GB/64 GB/128 GB
DDR4 up to 2400 MT/s
6:
Listed below are the PERC9 cards that are supported in the PowerEdge 13G
systems:
• Internal controllers: PERC S130 (Software RAID), PERC H330, PERC H730,
PERC H730P, HBA330 (no RAID internal HBA)
• External HBAs (RAID): PERC H830
• External HBAs (non-RAID): 12 Gbps SAS HBA
Following are the specifications of PowerEdge 13G servers.
Memory: Supported in the form of RDIMMs and LRDIMMs.
Up to 3 TB (24 DIMM slots): 4 GB/8 GB/16 GB/32 GB/64 GB/128 GB.
DDR4 up to 2400 MT/s.
Storage Controllers: Listed below are the PERC9 cards that are supported in the
PowerEdge 13G systems:
• Internal controllers: PERC S130 (Software RAID), PERC H330, PERC H730,
PERC H730P, HBA330 (no RAID internal HBA)
• External HBAs (RAID): PERC H830
• External HBAs (non-RAID): 12 Gbps SAS HBA

Portfolio Overview
Power Supplies: 1100 W DC power supply
Titanium efficiency 750 W AC power supply
Platinum efficiency 495 W, 750 W, 1100 W AC power supply
Embedded Management: IPMI 2.0 compliant
iDRAC8 with Lifecycle Controller
iDRAC8 Express (default) or Enterprise (upgradable):
• 8 GB vFlash media (optional) - all systems will get shipped with a vflash but
only systems that have the enterprise license will be able to use them.
• 16 GB vFlash media (optional) - all systems will ship with 8GB but customers
can request the 16GB vflash as an upgrade.
Expansion and Network: Up to x7 PCIe 3.0 slots, supports a dedicated RAID

card slot and a dedicated NDC slot.
Embedded NIC- 1 GbE x4 (his is the dedicated port for the iDRAC) - The NDC is
present in different versions ranging from 4x 1 GB, 2x 1 Gb + 2x 10 GB or 4x 10
GB.
Processor: PowerEdge R930 of the 13th generation supports up to x4 Intel Xeon

E7-8800 product family processors.

Portfolio Overview
PowerEdge 13G Control Panel
The table below explains components on the front panel of the PowerEdge server.
Part Number Description
1 Power Button
2 NMI Button
3 System Identification Button
4 LCD Menu Buttons
5 LCD Panel
6 VGA Connector
7 SD vFlash card slot
8 Service Tag
9 USB and iDRAC Direct Connector
10 USB Connector

Portfolio Overview
In comparison to the PowerEdge 13G servers, the PowerEdge 14G servers

feature:
• The Intel Xeon scalable processor that replaces the Xeon E5 and E7 versions
found on the 13G platform.
• An optional new bezel that includes the LCD panel, to support additional
airflow for devices with higher power consumption.
• PowerEdge RAID Controller 10 series cards that support advanced
capabilities.
• Randomized passwords for the iDRAC account, rather than the default
credentials that 13G servers support.
• The ability to use M.2 SATA devices as boot cards (BOSS cards), requiring
Operating System support.
Below are several 14G PowerEdge servers.
PowerEdge R640
PowerEdge R640
PowerEdge R740xd

Portfolio Overview
PowerEdge R740xd
PowerEdge R940
PowerEdge R940
PowerEdge R940xa
All the systems will ship with a bezel but the customer has the choice of
purchasing the bezel with or without LCD.
PowerEdge R940xa with front bezel

Portfolio Overview

to dell.com/support
1: Management and Automation: PowerEdge 14G systems provide a single-

management layer for all platforms, reducing the time and effort that is needed to
manage large-scale environments. PowerEdge 14G systems support iDRAC9.
iDRAC9 supports the Group Manager feature that enables users to have multiple
console experience and offers simplified basic iDRAC management.

Portfolio Overview
2: Intel C620 chipset: PowerEdge 14G systems include the Intel Lewisburg as the
Platform Controller Hub (PCH) chip. The Integrated Intel® Ethernet with scalable
iWARP RDMA in the Intel® C620 series chipset provides up to four 10 GBPS
high-speed Ethernet ports for high data throughputs and low-latency. Ideal for
storage, data intensive, and connected IoT solutions.
3: The Intel® Xeon® scalable processor family supports 2933 MT/s memory. As
an example, the PowerEdge R740 and R740xd support two DIMMs per channel at
2933 MT/s with these processors.
4: GPU/FPGA mainstream: Expanded GPU capacity with additional accelerator

cards to maximize workload.
Supports up to 3 x 300 W or 6 x 150 W accelerator cards maximizes workload
acceleration.
Management and Automation: PowerEdge 14G systems provide a single-

management layer for all platforms, reducing the time and effort that is needed to
manage large-scale environments. PowerEdge 14G systems support iDRAC9
(this feature is only available with the Data Center and Enterprise license).
iDRAC9 supports the Group Manager feature that enables users to have multiple
console experience and offers simplified basic iDRAC management.
Intel C620 chipset: PowerEdge 14G systems include the Intel Lewisburg as the
Platform Controller Hub (PCH) chip. The Integrated Intel® Ethernet with scalable
iWARP RDMA in the Intel® C620 series chipset provides up to four 10 GBPS
high-speed Ethernet ports for high data throughputs and low-latency. Ideal for
storage, data intensive, and connected IoT solutions.
GPU/FPGA mainstream: Expanded GPU capacity with additional accelerator

cards to maximize workload.
Supports up to 3 x 300 W or 6 x 150 W accelerator cards maximizes workload

acceleration.
Processor: The Intel® Xeon® scalable processor family supports 2933 MT/s
memory. As an example, the PowerEdge R740 and R740xd support two DIMMs
per channel at 2993 MT/s with these processors.

Portfolio Overview
PowerEdge 14G Control Panel
Control Panel View
The image below highlights the control panels that are located at the front of a
PowerEdge 14G system.
The front view of the PowerEdge R740 is used as an example.
Control Panel Components

Portfolio Overview
In comparison to the PowerEdge 14G servers, the PowerEdge 15G servers

feature:
• Products purpose-built for diverse environments to support workloads that are
accelerated, computational-heavy, based on Artificial Intelligence and other
edge technologies.
• A cyber resilient architecture and a well-established Root of Trust (RoT)2, to
ensure a complete cycle of security for the server.
• Uniquely designed chassis that support ducted fans to improve energy
consumption.
• Support for Open Compute Project (OCP) NIC 3.0 technology.
Below are several 15G PowerEdge servers.
PowerEdge R750
PowerEdge R750
2 Root of Trust is a concept that starts a chain of trust that ensures systems boot
with a legitimate code at every step of the boot process. RoT is controlled by the
iDRAC9.

Portfolio Overview
PowerEdge XE8545
PowerEdge XE8545
PowerEdge XR11
PowerEdge XR11
PowerEdge R6515
PowerEdge R6515

to dell.com/support

Portfolio Overview
1: Direct Liquid Cooling (DLC): DLC is introduced in the PowerEdge 15G systems
and features a leak-sensing technology to identify and resolve issues faster. The
DLC technology is supported only in the PowerEdge R650, PowerEdge R750,
PowerEdge R750xa and PowerEdge C6520 servers.
2: PowerEdge RAID Controllers: Support for PERC 10 and PERC 11 cards for
enhanced RAID performance.
To view the list of PERC types for Dell systems, visit the List of PowerEdge RAID
Controller (PERC) types for Dell systems document in dell.com/support
3: Configuration Validation: The 15G PowerEdge systems support interconnect

flexibility and advanced iDRAC management features to collect precise system
configuration information and report configuration errors.

Portfolio Overview
4: Memory: The 15G PowerEdge servers support the Intel Optane persistent
memory, that support up to 16 DIMMS per CPU.
Intel Optane persistent memory is also known as Barlow Pass. Click here for more
information on Barlow pass and different configuration.
Direct Liquid Cooling (DLC): DLC is introduced in the PowerEdge 15G systems
and features a leak-sensing technology to identify and resolve issues faster. The
DLC technology is supported only in the PowerEdge R650, PowerEdge R750,
PowerEdge R750xa, and PowerEdge C6520 servers.
PowerEdge RAID Controllers (PERC): Support for PERC 10 and PERC 11

cards for enhanced RAID performance.
To view the list of PERC types for Dell systems, visit the list of PowerEdge RAID
Controller (PERC) types for Dell systems document in dell.com/support.
Memory: The 15G PowerEdge servers support the Intel Optane persistent
memory, that support up to 16 DIMMS per CPU.
Intel Optane persistent memory is also known as Barlow Pass. Click here for more
information on Barlow pass and different configuration.
Configuration Validation: The 15G PowerEdge systems support interconnect

flexibility and advanced iDRAC management features to collect precise system
configuration information and report configuration errors.

Server Components
Server Components
Introduction
Server components are broadly identified into three categories:

• Hot swap components
• Field Replacement Units (FRUs)
• Customer Replacement Units (CRUs)
Hot Swap Components
Hot swap components can be Hot swappable or hot-pluggable.

• Hot swapping: Describes replacing components while the system is still in
operation.
• Hot plugging: Describes the addition of components that would expand the
system while the system is still in operation.
Hot swap components are designated by orange tabs or markings.
Hot swap components enable zero system downtime for failures and
serviceability.
Examples of some Hot swap components are fans, disks, and Power Supply Units
(PSUs).
Examples of Hot swap components.

Server Components
Field Replacement Units
A field replacement unit (FRU) is a server component or assembly that requires

the entire chassis to be powered off to service.
FRUs are replaced by a user or technician without having to send the entire
product or system to a repair facility.
FRU is marked as blue color. Blue color indicates that the system must be
shutdown to replace this component.
Image of an FRU fan.
Customer Replacement Units
Some component parts are designed for easy customer removal and replacement;
such parts are designated as Customer Self-Replaceable (CSR) or Customer
Replaceable Unit (CRU).
When during the troubleshooting, a Dell Technician determines that the repair can
be accomplished with a CSR/CRU designated part, Dell ships the designated part

Server Components
directly to the customer, which allows customers to replace parts at their own
convenience.
Important: The assembly and disassembly videos of the server

components are available in the document "Server Components-
Assembly and Disassembly videos" under Resources section.
Processors
Intel Xeon Scalable Processor Family
The Intel processor uses a metal naming convention to designate the different
levels of available features. The levels are Platinum, Gold, Silver, and Bronze.
1: Intel® Xeon® Platinum processors offer the industry best performance for
mission-critical and hybrid cloud workloads, real-time analytics, machine learning,
and artificial intelligence. The platinum processors offer monumental leaps in I/O,
memory, storage, and network technologies.
The platinum processors support:

• Up to 40 CPU cores.
• 8+ socket configurations.
• Up to 12 TB memory.
2: Intel® Xeon® Gold processors offer high performance, advanced reliability, and
hardware-enhanced security. The gold processors are optimized for demanding
data-centers, hybrid-cloud compute, network, and storage workloads.

Server Components
The gold processors support:

• Up to four socket configurations.
• Up to 6 TB memory.
3: Intel® Xeon® Silver processors offer the hardware-enhanced performance and

security that is required for data center compute, network, and storage. The silver
processors are optimal for midsized and growing IT organizations.
The silver processors support:

• Up to two socket configurations.
• Up to 1.5 TB memory.
4: Intel® Xeon® Bronze processors provide optimized performance for small

businesses and basic storage servers.
The bronze processors support:

• Up to eight CPU cores.
• Up to two socket configurations.
• Up to 1.5 TB memory.
• Platinum: Intel® Xeon® Platinum processors offer the industry best

performance for mission-critical and hybrid cloud workloads, real-time
analytics, machine learning, and artificial intelligence. The platinum processors
offer monumental leaps in I/O, memory, storage, and network technologies.
The platinum processors support:
− Up to 40 CPU cores.
− 8+ socket configurations.
− Up to 12 TB memory.
• Gold: Intel® Xeon® Gold processors offer high performance, advanced
reliability, and hardware-enhanced security. The gold processors are optimized
for demanding data-centers, hybrid-cloud compute, network, and storage
workloads.
The gold processors support:

Server Components
− Up to four socket configurations.
− Up to 6 TB memory.
• Silver: Intel® Xeon® Silver processors offer the hardware-enhanced
performance and security that is required for data center compute, network,
and storage. The silver processors are optimal for midsized and growing IT
organizations.
The silver processors support:
− Up to two socket configurations.
− Up to 1.5 TB memory.
• Bronze: Intel® Xeon® Bronze processors provide optimized performance for
small businesses and basic storage servers.
The bronze processors support:
− Up to eight CPU cores.

− Up to two socket configurations.
− Up to 1.5 TB memory.
AMD Processors
AMD Processor

Server Components
The Dell PowerEdge server portfolio is powered by the third generation AMD
EPYC™ Processors. The AMD processors system on a chip (SOC) is the next-
generation data center processor supporting socket compatibility with socket
infrastructure. The AMD Milan processor is based on a new enhanced Zen2 CPU
core with integrated I/O controllers.
The AMD Milan processor:
• Offers significant performance improvement from current generation
production.
• Has 128 PCIe lanes, eight-channel memory, and dual-socket configurations.
• Lowers cost through an optimal balance of compute, memory, I/O, and
security.
• Offers one I/O memory die which removes internal bottleneck for lower latency.
• Has up to 64 CPU cores per processor.
• Interchip global memory interconnect (xGMI2) up to 64 lanes.
• Has Secure Encrypted Virtualization(SEV) which provides 509 unique
hypervisor keys.
• Has two restrictions.
− The RTC/CMOS is built into the CPU, similar to previous PowerEdge AMD
servers. RTC/CMOS will be lost when CPU1 in server is removed or
reinstalled.
− AMD does not support early boot. No error message is displayed when
there is no memory that is populated in the system.
PowerEdge 15G Processors
The PowerEdge 15G servers also support third-generation Intel Xeon scalable
processors.
The Intel® Xeon® Processor has increased performance and incremental memory
options.
The Xeon scalable processor supports usages from entry designs based on Intel
Xeon Silver processors to advanced capabilities offered in the new Intel Xeon
Platinum processor.
The third-generation Intel Xeon scalable processor supports:

Server Components
• Increased memory capacity with up to 8 channels and up to 256 GB DDR4

DIMM support.
• Enhanced memory performance with support for up to 3200MT/s DIMMs (2
DPC).
• Intel Optane persistent memory 200 series module (Intel Optane Persistent
Memory 200 Series, up to 512 GB modules) up to 6 TB of total system
memory/socket DDR+PMM.
• Faster I/O with PCI express 4 and up to 64 lanes (per socket) at 16 GT/s.
• Faster Intel Ultra Path Interconnect (UPI) with 3 Intel UPI at 11.2 GT/s
(supported in gold and platinum options).
The 15G server with Intel processors and heatsinks has an additional anti-tilt
feature to prevent tilting of the heatsink assembly. The plastic nuts secure the
heatsinks to the system board.
Anti-tilt feature in a PowerEdge 15G Intel processor and heatsink assembly.

Server Components
Processor Settings
The Processor Settings option is used to view and configure various processor
settings. The Processor Settings can be accessed through System Setup utility.
Processor Settings.
Go to System Setup Main Menu > System BIOS > Processor Settings.
Some of the available Processor Settings options are:

• Logical Processor: This option is set to Enabled by default. If this option is set
to Enabled, the BIOS displays all the logical processors. If this option is set
to Disabled, the BIOS displays only one logical processor per core.
• QPI Speed: Controls QuickPath Interconnect (QPI) data rate settings.
• Virtualization Technology: User can enable or disable the additional
hardware capabilities provided for virtualization. This option is set to Enabled
by default.
• Address Translation Service (ATS): This option provides an interface
between CPU and DMA Memory Management to a chipset's Address
Translation and Protection Table to translate DMA addresses to host
addresses. This option is set to Enabled by default.
• Adjacent Cache Line Prefetch: This option optimizes the system for
applications that need high utilization of sequential memory access. This option
is set to Enabled by default. User can disable this option for applications that
need high utilization of random memory access.
• Hardware Prefetcher: When set to enable, the processor is able to prefetch
extra cache lines for every memory request.

Server Components
• X2Apic Mode: x2APIC architecture extends processor addressability and

enhances interrupt delivery performance. User can enable or disable the
X2Apic mode.
• Number of Cores per Processor: Number of cores per processor controls the
number of enabled cores in each processor. This option is set to All by default.
• Processor Core Speed: Processor core speed specifies the maximum core
frequency of the processor.
For more information, visit below link:

https://www.dell.com/support/manuals/en-in/poweredge-
r630/r630_om_pub/processor-settings-details?guid=guid-5fc21783-239f-4469-
be5f-153eb5403270&lang=en-us
Memory
Server Memory Module
Dell PowerEdge servers run on Error-Correcting Code (ECC) memory. The ECC
memory can test and correct any memory errors without the processor or the user
being aware of these operations. ECC corrects the errors without interrupting
other operations on the server.
Memory Comparison
The table below highlights the differences in memory features across the three
generations of Dell PowerEdge servers.

Server Components
Server Generation 13G 14G 15G
Number of 12 DIMMs per 12 DIMMs per 16 DIMMs per

Supported DIMMs CPU CPU CPU
RAM Size 1 X 4 GB 1 X 8 GB 1 X 8 GB
DIMM Type RDIMM RDIMM RDIMM

LRDIMM LRDIMM LRDIMM
NVDIMM-N NVDIMM-BP
Supported 4 Channels 6 Channels 8 Channels

Channels per
Processor
Intel Persistent N/A Apache Pass Barlow Pass

Memory 3DXPoint3 3DXPoint4
Supported Transfer 2400 MT/s 2933 MT/s 3200 MT/s

Speed 2133 MT/s 2666 MT/s 2933 MT/s
1866 MT/s 2400 MT/s 2666 MT/s
1600 MT/s 2133 MT/s
1866 MT/s
3 The 3DXPoint is a type of nonvolatile memory.

4 The 3DXPoint is a type of nonvolatile memory.

Server Components
Note: NVDIMM-N is not supported on Dell 15G servers.
CPU and Memory Architecture Comparison
The common CPU architecture features that evolve are:
• The number of memory channels available:

− PowerEdge 15G servers consist of eight memory channels with dual slots
per channel.
− 15G supports flexible memory configurations ranging from system
capacities of 8 GB (minimum) to 4 TB (maximum).
• The number of DIMMs supported:
− PowerEdge 15G servers can support up to 32 DIMMs.

− Each server has eight memory channels per processor, with each channel
supporting up to 2 DIMMs.

Server Components
15G CPU and Memory Architecture
15G CPU and memory layout.
The 15G CPU and memory architecture:

• Supports Intel third-generation Xeon Scalable processors.
• Manages data flow between the memory channels5 through memory
controllers on the digital circuit.
• Enables memory interleaving for efficient memory accesses across multiple
DIMMs.
• Provides eight memory channels with up to two memory slots per channel and
sixteen memory modules per processor.
5Memory channels are the physical layer on which the data travels between the
CPU and memory modules.

Server Components
Memory Layout
13G Memory Layout
13G memory channels and their relation to DIMM slots.

Server Components
Features of 13G memory layout are:

• Supports up to 12 DIMM slots per CPU.
− Four memory channels per CPU with three memory slots per channel.
• In a dual-processor configuration, the memory configuration for each processor
should be identical.6
− For single-processor systems, sockets A1 to A12 are available.
− For dual-processor systems, sockets A1 to A12 and sockets B1 to B12 are
available.
• Populating four memory modules per processor (one DIMM per channel) at a
time maximizes the performance.
• Click here for the guidelines on installing the memory modules on a 13G
server.
14G Memory Layout
Some of the features of 14G memory layout are:

− Six memory channels per CPU with two memory slots per channel.
6 If socket A1 is populated for processor 1, then populate socket B1 for processor

2 with an identical DIMM.
7 If socket A1 is populated for processor 1, then populate socket B1 for processor

Server Components

available.
• Populating six identical memory modules per processor (one DIMM per
channel) at a time maximizes the performance.
server.
15G Memory Layout
Some of the features of 15G memory layout are:

− Eight memory channels per CPU with two memory slots per channel.
• DIMMs should be populated in from the A1 slot up to the A16 slot in sequential
order.

Server Components

available.
server.
Memory Settings
The Dell server memory settings can be accessed through the Lifecycle Controller
(LCC) System Setup option.
8If socket A1 is populated for processor 1, then populate socket B1 for processor

Server Components
Important: Administrators can also change memory settings via

the iDRAC9 GUI. The iDRAC9 is often a customer preferred
method to configure memory settings. Using the iDRAC9 GUI
avoids taking systems down to change BIOS settings until a
downtime is identified. Going into the LCC or F2 to the BIOS to
configure memory settings are both seen as a backup for
customers. Customers will configure memory through the iDRAC9
and then schedule the job to edit the BIOS settings on next
reboot.
Step 1
Reboot the system to view the various server management interfaces.
Step 2
Press F2 key in the keyboard. The system will enter the System Setup page.

Server Components
Step 3
Select System BIOS option.

Server Components
Step 4
Select Memory Settings option.
Memory Settings

Server Components
The memory options for the PowerEdge servers include:

• System Memory Testing: Indicates if the system memory tests are run during
system boot.
• Memory Operating Mode: Indicates current memory operating mode.
9Memory modes can add redundancy and fail-over features to the server.
• Node Interleaving: Specifies if Non-Uniform Memory Architecture10 (NUMA) is

supported.
• There are differences in the Memory Settings between 13G, 14G, and 15G
PowerEdge Servers. Click here to learn how to discover the supported memory
configurations for the different types of PowerEdge Servers.
9This option is set to Optimizer Mode by default.

10If this field is set to Enabled, memory interleaving is supported if a symmetric
memory configuration is installed.
If the field is set to Disabled, the system supports NUMA (asymmetric) memory
configurations. This option is set to Disabled by default.

Server Components
Memory Modes
System Memory settings page.
The Dell PowerEdge chipset allows different operating modes for the memory to
be set in the BIOS.
The various memory modes available on the Dell PowerEdge servers are:
• Optimizer Mode: The memory controllers run independently of each other.
• Mirror Mode: The system supports memory mirroring if identical memory
modules are installed in two channels.

Server Components
• Advanced ECC Mode: The two memory channels closest to the processor
(channel 0 & 1) are combined to form a single 128-bit channel.
• Spare Mode: One rank11 per memory channel is reserved as a spare.
• Dell Fault Resilient Mode: An area of the memory that is established as fault
resilient and is used by a VMware vSphere hypervisor or other services to
maximize the availability.
Optimizer Mode:
In Optimizer mode, all three channels are populated with memory modules. This
mode permits a larger total memory capacity but does not support SDDC with x8-
based memory modules.
It is recommended to populate all three channels with identical memory but each
channel can have a different size DIMM. The larger DIMM has to be installed in
the first slot and the configuration has to be the same across all three channels. In
a dual-processor configuration, the memory configuration for each processor must
be identical. Optimizer mode is the only mode to support mixed memory sizes.
11A memory rank is a block or area of data that is created using some, or all, of
the memory chips on a module.

Server Components
Any configurations not following the above rules may generate error messages or
not POST at all. For more detail read the initial release notes: Installing and
configuring DDR3 Memory
It is recommended to populate all three DIMM slots on servers with three DIMM
slots per channel to take advantage of memory interleaving to get maximum
performance. While a single UDIMM per channel gives slightly better performance
than an RDIMM, RDIMMs give better performance when multiple DIMM per
channel are installed.
Optimizer is used if just one DIMM for each processor is configured. A minimal
single-channel configuration of 1 GB memory modules per processor is also
supported in this mode. Minimum to POST would be one DIMM and in the first slot
and just CPU 1 installed.
Memory Mirroring:
The PowerEdge system supports memory mirroring if identical memory modules

are installed in the two channels closest to the processor (CH 0 & 1) and no
memory that is installed in the channel furthest from the processor (CH 2).
Mirroring must be enabled in the System Setup program. The total available
system memory is one-half of the total populated memory, any operating system
will only show half the physical memory installed.
Advanced ECC (Lockstep) Mode
In Advanced ECC (Lockstep) mode, the two channels closest to the processor
(CH 0 & 1) are combined to form one 128-bit channel. This mode supports Single
Device Data Correction (SDDC) for both x4 and x8 based memory modules.
Memory modules must be identical in size, speed, and technology in the slots on
channel 0 and 1. Channel 2 has to be empty or option will not be available in the
System Setup program.
Using the Intel 5500 and 5520 chipset with Intel 55xx and 56xx processors
channels 0 and 1 are combined which enables 8-bit error correction instead of 4-
bit in normal Advances ECC (not lockstep). SDDC gives the ability to recover from
more types of single and multibit memory errors. The third channel and
corresponding memory slots cannot be used but full amount of installed physical
memory will be accessible to the operating system.
Note: 14G and 15G servers do not support advanced ECC mode.

Server Components
Dell Fault Resilient mode
Dell Fault Resilient Mode (FRM) is a Memory Operating mode available on the
BIOS settings of high-end yx2x Dell PowerEdge servers and later. This mode
establishes an area of memory that is fault resilient and protects the hypervisor
against uncorrectable memory errors, and safeguards the system from becoming
unresponsive. Systems with ESXi that supports the FRM feature can load the
operating system kernel to maximize system availability and or critical applications
or services.
Memory Spare Mode
Memory Spare mode can be further classified into two types.
• Single-rank sparing mode: It allocates one rank per channel as a spare. This
mode requires a population of two ranks or more per channel.
• Multi-rank sparing mode: It allocates two ranks per channel as a spare. This
mode requires a population of three ranks or more per channel.
When single rank memory sparing is enabled, the system memory available to the
operating system is reduced by one rank per channel. See the example below.

Server Components
Note: If excessive correctable errors occur in a rank or channel,

they are moved to the spare area. The operating system is running
to prevent errors from causing an uncorrectable failure.
Post Package Repair (PPR) is a self-healing memory
enhancement that works to repair the issue by disabling the
location/address at the hardware layer, enabling a spare memory
row to be used instead.
Intel Persistent Memory
The Intel Optane Persistent Memory (Barlow Pass) solution retains data during a
power loss, system shutdown, or system errors. Barlow Pass (BPS) uses
persistent memory as storage, rather than traditional memory.
Features of the Intel DC Persistent memory Module (DCPMM) are:
• Creates a unique new memory tier to reduce latencies and optimize workloads.
• Provides disruptive storage class memory cell technology (3DxPoint) that
resides on the DDR memory interface.
• Provides large memory footprints of 128 GB, 256 GB, and 512 GB.
• Enables in-memory data to survive a soft reset or a hard reboot (power loss).
• Provides minimal latency and faster storage for large amounts of memory.

Server Components
Note: Barlow Pass is the second generation of Intel Optane

Persistent Memory. Apache Pass is the first generation of Intel
Optane Persistent Memory.
Apache Pass supports 14G platforms and Barlow Pass DIMMs
support the 15G platforms. DIMMs are not compatible across
generations.
Intel Persistent Memory DIMM Layout
The image represents the Intel Persistent memory DIMM layout.

Server Components
Barlow Pass Advantages
The Barlow Pass architecture consists of two-tier memory and storage hierarchy
to address the data performance and storage challenges. The advantages of the
hierarchical approach are:
• Provides a unique combination of affordable large capacity and support for

data persistence.
• Optimizes the resources for efficient data access and storage.
• Provides higher performance (up to 3200 MT/s) with low latency DRAMs.
• Creates larger memory capacity (up to 4TB per CPU) to store and protect data
in DRAM.
• Enables in-memory computing for large datasets.
• Leverages the speed and proximity from the technologies nearer to the CPU.

Server Components
Industry Mixed Memory Combinations
Barlow pass supports the following mixed memory combinations.
Memory Population Configurations with Barlow Pass
BPS Memory Configuration
The memory modes can be used only if the DIMMs are RDIMMs with capacity of
32GB or lesser.
Config RDIMM + BPS Number Number Number App- Memory

number (Configuration) of CPU of of BPS Direct Mode
per RDIMMs per CPU Mode
system per CPU
BPS1 4+4 1 or 2 4 4 Y Y
BPS2 6+1 1 or 2 6 1 Y
BPS3 8+1 1 or 2 8 1 Y
BPS4 8+4 1 or 2 8 4 Y Y

Server Components
BPS5 8+8 1 or 2 8 8 Y Y
BPS6 12+2 1 or 2 12 2 Y
BPS Processor Configuration
Each row on the below chart represents a different valid memory configuration for
mixing Barlow Pass (B) and RDIMMs (R).
Important: Click here to review recommended guidelines for

installing Intel Persistent Memory 200 Series (BPS) in a
PowerEdge server.

Server Components
Operational Modes
Intel Persistent Memory can be configured into two different modes.
DIMMs can be used in Memory Mode or App Direct Mode12.
Users can configure the memory modes and update it in the BIOS.
12The mode can be changed through the BIOS settings: F2 -> System BIOS ->
Memory Settings -> Persistent Memory -> Intel Persistent Memory -> Region
Configuration.

Server Components
Memory Mode
Memory mode uses the DIMM as

memory only.
Features of Memory mode:

• The system memory functions as
volatile system memory.
• Memory mode is transparent to
the OS and Applications.13
• Memory mode do not change
any software/application on the
system.
Advantages of Memory mode are:

• BPS DIMMs are used as volatile
system memory when configured
in Memory Mode.
• All DDR4 standard DIMMs are
used as cache (near memory).
• Asynchronous DRAM Refresh
(ADR) is required for all modes.
Memory mode architecture.

Intel® Optane™ DC Persistent Memory
Module(DCPMM)
13They access DIMMs as system memory, and will not have control or direct
access to DDR4 DIMMs that are used for caching.

Server Components
Memory Mode setting in System BIOS
AppDirect Mode
AppDirect mode architecture. The NVDIMM in the image represents BPS.
The App Direct mode is the default memory mode on the BIOS. AppDirect mode
uses the DIMM as storage.
Features of App Direct mode:
• Provides larger storage capacity, higher endurance, low latency and traditional
read/write.

Server Components
• Works with existing file systems to access the files. Two major methods to
access the files are Block method14 and PMEM method15.
• Cache lines are accessed using load or store instructions.
• Application is responsible for flushing data out of CPU cache into persistence
guaranteed memory buffers.
Advantages of AppDirect mode are:
• Persistent memory aware file systems, operating system and applications can
access BPS DIMMs in AppDirect mode.
• BPS DIMMs are used as persistent memory when configured in AppDirect
Mode.
Important: By default, the memory is in memory mode. The user

must change it to persistent memory in the iDRAC or BIOS.
Memory Mode and Operational Mode Differences
In Memory mode, the BIOS and operating system list the capacity of the Optane
memory and not the RDIMMs. The RDIMMS are used as cache for the Optane
DIMMs when running in memory mode.
14 The block method is slower and is similar to traditional storage access. The
block size is configurable at the operating system level.
15 PMEM method uses the full technology potential, but requires the application to
be optimized.

Server Components
In AppDirect mode, 632 GB is the full amount of memory available but only 128
GB of it is volatile. The system uses the rest of the memory as persistent storage.
The image shows the difference between the memory capacity that is available when running in
Memory mode and AppDirect mode.
Important: Dell offers Intel Optane Persistent Memory (PMem), a

nonvolatile memory that has the same form factor as a standard
DDR4 DIMM. PMem can be configured into two modes: Memory
Mode and App Mode. Click here to review the Dell PMem 100
Series User's Guide and the information on how to install and
remove the persistent memory.

Server Components
Power
Power Supply Unit
Hot-swappable Dell 800 W PSU.
In most cases16, the power supply unit (PSU) is a hot-swappable component that
provides power redundancy support on PowerEdge servers.
All the Dell PowerEdge servers support a minimum of two PSUs.
The Dell PSUs offer:

• Automatic load balancing and fail-over.
• Power distribution management capabilities.
• Regular firmware updates.
16The PSUs shipped in the PowerEdge 200-500 server series are not hot-
swappable. Not all Dell PowerEdge support a minimum of two PSUs. Some low-
end PowerEdge servers have a single PSU.

Server Components
Important: Before performing a hot swap on a PSU, be sure to

verify the failed PSU in the iDRAC.
PSU Configuration Modes
The PSUs on PowerEdge servers can be configured in different modes based on

the customer requirements17.
The redundant power supply is implemented in PowerEdge servers to keep the

servers running when the primary PSU fails to provide the necessary power
supply and to minimize the risk of a complete server shutdown. The Power
Configuration panel uses the iDRAC9 (and newer versions) interface to configure
redundant power supplies for servers. The type of power supply configuration or
the redundancy mode depends on the server chassis and the number of PSUs.
Grid Redundant
The Grid Redundant configuration is also known as 1 + 1 configuration.
In grid redundant mode, the hot spare18 feature is disabled and the power output
is distributed equally across both power supplies. The Power Factor Correction
17 Not all Dell PowerEdge support a minimum of two PSUs. Low end PowerEdge
servers have a single PSU. For example, the PowerEdge R230 does not support
the multiple PSU feature and redundancy.
18 When the hot spare feature is enabled, one of the redundant PSUs is switched
to the sleep state. The active PSU supports 100 percent of the system load, thus
operating at higher efficiency. The PSU in the sleep state monitors the output

Server Components
(PFC) is disabled by default, to reduce power consumption when the system is on

standby. However, if a single PSU fails, the power drops down.
Power supplies are divided into Grid A and Grid B. If a grid or a PSU on one grid
fails, but the PSUs on the second grid are functional, the system does not shut
down. Grid redundancy depends on the system configuration.
Dual power supply configuration with grid redundant option.
No Redundancy
The no redundancy configuration is also known as a 2 + 0 configuration. For

example: Adding two power supplies together to create 1140 W (2 x 570 W). With
no redundancy configuration, the hot spare19 feature is enabled and Power Factor
voltage of the active PSU. If the output voltage of the active PSU drops, the PSU
in the sleep state returns to an active output state.
19 When the hot spare feature is enabled, one of the redundant PSUs is switched
to the sleep state. The active PSU supports 100 percent of the system load, thus
operating at higher efficiency. The PSU in the sleep state monitors the output
voltage of the active PSU. If the output voltage of the active PSU drops, the PSU
in the sleep state returns to an active output state.

Server Components
Correction (PFC) is disabled by default, to reduce power consumption when the

system is in standby. However, if a single PSU fails, the system may shut down.
Configuration with no redundancy.
PSU Redundancy Types
The PSU redundancy mode depends on the server type and the number of PSUs
in the system.
Redundancy Mode Option Description
1+0 Non-Redundant No redundancy.
1+1 Input or PSU Provides redundancy for

Redundant input failures (failed power
cables, PDU, grid) and
redundancy for PSU
failures.
2+0 Non-Redundant No redundancy.

Provides support for
configurations that exceed
the output capacity of a
single PSU.
The table shows PSU redundancy modes that are available with Dell PowerEdge servers.

Server Components
Power Capping
The Power capping option is used to limit the amount of power consumed by a
server.
When power cap policy is enabled, it enforces a user-defined power limits on the
system. If power-capping is not enabled, the default hardware power-protection
policy is used. This power-protection policy is independent of the user-defined
policy. The system performance is dynamically adjusted to maintain power
consumption close to the specified threshold.
Enabling power capping through iDRAC.

Go to Configuration > Power Management > Power Cap Policy
Advantages of capping are:

• Power budget threshold can be specified to reduce consumption.
• System performance is dynamically adjusted to maintain the power
consumption below the user-specified threshold.
• Reduces peak power consumption of the server.
• Using OpenManage Enterprise Power Manager you can optimizes the overall
data center power efficiency by setting power caps on individual system or by
groups of servers.
• The power limit can be set through the iDRAC UI.

Server Components
Mixing Power Supplies
PSU of PowerEdge 14G and PowerEdge 15G.
PowerEdge servers support mixed PSU configurations. The criteria to implement

mixed PSU configurations are:
• PSUs must have the same type of label20.

• PSUs must have the same maximum output power.
• Mixing PSUs from different generations of PowerEdge servers is not
supported.
• Mixing PSUs from different generations or from the same generation but with
different power wattages results in a mismatch condition. When a mismatch
20 For example, the Extended Power Performance (EPP) label.

Server Components
condition occurs the lower wattage PSU is disabled and a warning condition is
triggered.
Important: In the 15G PowerEdge PSUs, the wattage label is

present on the handle wing. In the PowerEdge 13G/14G PSUs, the
wattage label is present on the chassis.
PSU LED Indicator Behavior
The table details the PSU LED indicator behaviors.
A 1400 W PSU with active LED.
LED Color LED behavior Description
Solid Green PSU is in a healthy state
Blinking Amber, 2 sec ON, PSU is in a faulty state

1 sec Off

Server Components
Blinking Green with 4 Hz PSU mismatch

rate for five times and
then stay OFF.
Blinking Green with 4 Hz Firmware update

rate.
OFF No power
PSU Firmware Updates
The PSU firmware can be updated through the Lifecycle Controller (LCC). Click
here to review how to update a server Power Supply Unit firmware, including a
video walk-through of the procedure.
Example of a PSU errors on iDRAC due to mixed PSU configuration.
The following precautions should be taken while updating the firmware:
• During the update process, the server powers off.

• Powering on may prematurely lead to a corrupted PSU firmware and can brick
the power supply.
− Do not power on the server as the iDRAC and the Lifecycle Controller are
running background operations during this power off.
• A firmware update cannot be performed on a mixed PSU configuration. The
PSU may receive the wrong firmware image due to mixed configuration.
• To update PSU firmware of different wattages:

Server Components
− Update the firmware on a single PSU.

− Remove PSU 1 and apply updates to PSU 2. Next, remove PSU 2 and
apply updates to PSU 1.
• If matching PSUs are plugged into mixed line voltage, the low line PSU triggers
an error.
PSU Blanks
Location of PSU in Dell PowerEdge R750 (15G).
To maintain an efficient airflow for system cooling, all servers with an empty PSU
slot require PSU blank plates. PSU blanks avoid the loss of cooling airflow. If the
PSU blanks are missing, the system temperature might increase and result in
component failures.
In 15G servers, the PSUs are located in the rear of the system. The PSUs are on
the opposite side of each other for better airflow within the chassis.

Server Components
Cooling
Fans and Types of Fans
High-performance fan used in the Dell PowerEdge R6525 server.
A server consists of multiple fans. When a fan fails, the remaining fans take up the
load.
The cooling fans dissipate the heat generated by the functioning of the server21.
These fans cool the processors, expansion cards, and memory modules.
The fans ramp up when:
• The system cover is removed.

• The system is powered on.
• The iDRAC is rebooted.
21Some servers may not have hot-swappable fans (Hot swapping is the
replacement or adding of components to a system unit system without stopping,
shutting down, or rebooting the system). If no hot-swappable fans are available
and if a fan fails, the iDRAC ramps up the existing fans, similar to systems with
hot-swap fans. However, the failed fan cannot be replaced until the system has
been powered off as the fans must have their cables that are disconnected from
the system board.

Server Components
Types of Fans
The Dell PowerEdge servers use Standard fans, High-Performance fans, and Very
High-Performance fans based on the server configurations22.
Dell PowerEdge servers come with different chassis dimensions and they can be
1U, 2U, and so on. Based on the chassis dimension and design the fan dimension
may vary as well.
If a system has six fans and one of the fans fails, the iDRAC ramps up the
remaining fans. It keeps the temperature within the chassis at a set level. (It
should be noted that if the temperature is already well below the required level, the
iDRAC may not ramp up the remaining fans.)
Once the failed fan has been replaced, the iDRAC tests the new fan. It slowly
decreases the speed of the existing fans while increasing the speed of the new fan
until they are all operating at the correct speed.
Removal of the chassis covers often results in the fans ramping up as the cover is
used to deflect airflow throughout the system. If the cover is removed a certain
amount of airflow is lost so the iDRAC, upon detecting that the cover has been
removed, it ramps up the fan-in an effort to increase the airflow across the
components to maintain required temperatures.
Also when the system is first powered on, the temperatures take a few seconds to
be recorded. As a fail-safe procedure, the iDRAC ramps the fans up and then
bring them back down as temperature status is analyzed.
Should a fan fail, errors are posted and the remaining fans pick up the additional
workload. However, based on the temperature within the chassis, the remaining
fans may or may not increase their speed.
22A user can see the thermal restrictions matrix or the technical guide of the
server for more information.

Server Components
The orange color denotes a hot-swappable component.
Some servers, especially the lower ranges may not have hot-swap fans. In that
case, if a fan fails, the iDRAC ramps up the existing fans like for systems that have
hot-swap fans, but the failed fan cannot be replaced until the system has been
powered off as the fans must have their cables that are disconnected from the
system board.
Standard Fans (STD)
Standard fan dimension for:
• R6525 is 40 mm (1.57 inches) x 40 mm (1.57 inches) x 56

mm (2.20 inches).
• R7525 is 60 mm (2.36 inches) x 60 mm (2.36 inches) x 38
mm (1.49 inches).
High Performance Fans (HPR) Standard fan
High performance and very high-performance silver grade fans.
The HPR fans provide a higher airflow rate. HPR fans are required in 12 x 3.5”,
rear-storage configurations and most GPU configurations.
High-performance fan dimension for:

Server Components
• R6525 is 40 mm (1.57 inches) x 40 mm (1.57 inches) x 56 mm (2.20 inches).

Very High Performance Fans (VHP)
High performance and very high performance gold grade fans.
VHP fans require front 16x NVMe drives or 8x NVMe, 16x SAS drives with GPU
configurations.
Very high-performance fan dimension for:

Important: New cooling fans come with the High-Performance

Gold and Silver Grade label, while the older cooling fans have the
High-Performance label.

Server Components
Important: High-performance fans can be distinguished by the

color of the label, gold, or silver. Mixing of STD, HPR (Silver), or
HPR (Gold) fan is not supported.
PowerEdge 15G Fans
Some of the 15th generation Dell Intel servers use standard, high-performance
silver grade, or high-performance gold grade fans, dependent on the configuration.
Dell PowerEdge servers such as R250, R350, T350, T150, XR11 and XR12 use
non hot pluggable single rotor fans.

Server Components
Heatsinks
The type of heatsink that is used is based on the CPU TDP23 and GPU
configurations.
Some PowerEdge servers have unique fan positioning in the chassis. For
example, the Dell PowerEdge XR11 has two fans that are located towards the
middle of the chassis. It has an extended heatsink design for optimum cooling.
Did you know?
In, certain single CPU configurations (non-GPU or nonrear-drive) only four fans
are required to be installed in the fan bay.24
23 Thermal Design Power (TDP) is measured in watts and is the maximum amount
of heat that is generated by a GPU or CPU. There are multiple types of CPU
heatsinks available including standard (STD), T-type, and full height heatsinks.

Server Components
Unique fan position and extended heatsink that is used in the PowerEdge XR11.
To remove the heatsink: The heatsink and processor are too hot to touch for some
time after the system has been powered off. Allow the heatsink and processor to
cool down before handling them.
1. Ensure that all four Anti-Tilt wires are in the locked position (outward position),
and then using a Torx number T30 screwdriver, loosen.
• Loosen the first screw three turns.
• Loosen the screw diagonally opposite to the screw you loosened first.
• Repeat the procedure for the remaining two screws.
• Return to the first screw to loosen it completely.
24In such configurations, only four fans are required to cool the system. For the
other two fan sockets, two fan blanks are required to be installed in fan bays 1 and
2. The number of fans that are required depends on the server model and
configuration.

Server Components
2. Set the Anti-Tilt wires to the unlocked position (inward position).

3. Lift the processor and heatsink module (PHM) from the system and set the
PHM aside with the processor side facing up.

Server Components
Memory DIMM Blank and GPU Air Shroud

Server Components
DIMM blanks on empty DIMM slots help regulate air flow through the CPU and
DIMM area.
For some servers, the air shroud looks like piano keys25 that drop down into empty
DIMM slots to stop airflow from being wasted. However, systems with midrange
storage require DIMM blanks for empty DIMM slots since the trays do not contain
piano keys.
Some of the PowerEdge servers like the XE8545 have separate GPU air shrouds
for better airflow and heat dissipation.
GPU Heatsinks and Fans
Dell specialized servers also known as XE servers can have advanced GPU
configurations. These systems generate a lot of heat and require custom
solutions.
25 This means that they do not require single DIMM blanks on empty DIMM slots.

Server Components
The Dell PowerEdge XE8545 supports up to 4x NVIDIA A100s GPUs and NVLink
in an air-cooled chassis.
The GPUs are cooled with the help of hot-pluggable GPU fans and specialized
heatsinks for each of the GPUs.
Important: Click the following link to get a copy of a guide showing

the steps involved in removing the Nvidia A100 GPU assembly. -
https://dell.bravais.com/s/KlmMp4D2b4cYCkDwPpcK.
Dell Multi Vector Cooling
The features of Dell Multi Vector Cooling 2.0 are:

• Patented adaptive closed loop power capping.
• Patented baseline fan speed algorithm.
• It provides custom delta-t design that enables users to specify the outlet
temperature (Requires iDRAC Datacenter).
• It provides custom PCIe inlet temperature, airflow control and, other custom
cooling capabilities for PCIe devices (Requires iDRAC Datacenter).
Design Innovation
The advanced thermal design streamlines the airflow pathway in the chassis and
directs the appropriate volume of air to components that require a constant air
supply.

Server Components
The design minimizes the fan and system power consumption while maintaining
the system temperature.
Thermal Design and Cooling Components
The latest thermal design and cooling components features are:
• It has a split PSU design to improve airflow through the system.

• It enables optimized PCIe connectivity from CPU to rear PCIe slots.
• It has a smaller PSU form factor (except 2400 W PSU).
• It provides additional “exhaust lanes” for hot airflow from CPUs.
• It provides enhanced performance that is required for cooling higher thermal
design power (TDP) CPUs.

Server Components
Server Exhaust Temperature
iDRAC Cooling Configuration settings page where a user can change the exhaust temperature
limit.
Dell PowerEdge servers use advanced thermal control algorithms to maintain

system temperatures at reliable levels while minimizing fan speed26 and system
airflow.
User Exhaust Temperature Control Options:
• Instances occur that require exhaust temperatures to be reduced, typically to

accommodate legacy power distribution units (PDUs), power cables, or Top of
Rack (ToR) switches.
• The fan speed and exhaust temperature settings are available using HII (BIOS-
based iDRAC settings), iDRAC web UI, and RACADM.
Users can apply custom fan speeds when using interfaces such as: iDRAC UI,
BIOS setup (F2), and RACADM.
26This minimization of system fan speeds and airflow can result in high exhaust
temperatures may be of concern to some users.

Server Components
HVAC - Heating, Ventilation, and Air Conditioning
As computing demands grow, so do data centers, and with this growth comes
huge amounts of heat27 that must be managed efficiently. Many data centers start
out as a few racks in a server room, adding more equipment over time. Without
taking cooling factors into account, data center HVAC management can become
difficult.
27When data centers are exposed to heat, servers start to slow down or
malfunction altogether. The same thing happens when the server rooms are too
cold. The ideal temperature for the data center depends on the size and amount of
heat that is emitted, but operating within this ideal temperature range is crucial for
overall performance.

Server Components
Hot aisle containment (HAC) guides the hot air (red arrows) into a system unit room air handler
(CRAH) which then recirculates the flow into cool air (blue arrows).

Server Components
Dell PowerEdge Liquid Cooling
Multiple PowerEdge servers with new Intel and AMD processors support the Dell Technologies
DLC.
Direct Liquid Cooling (DLC) solution manages the growing thermal challenges.
Dell DLC solutions28 cool the CPU with warm liquid, which has the capacity to
transfer heat up to 4X more than the capacity of air cooling.
Because DLC solutions are more efficient at extracting heat, it reduces the burden
on server system fans and the data center’s cooling infrastructure.
The PowerEdge servers below offer DLC cooling on the newest Intel and AMD
processors:
• C6520
• C6420
• C6525
28DLC solution is more efficient at extracting heat, reducing the burden on server
system fans and the data center’s cooling infrastructure.

Server Components
• R6525
• R7525
• R650
• R750
• R750xa
Important: The DLC solution is not offered as an After Point of

Sale (APOS) kit. Liquid cooling hardware settings cannot be added
as an upgrade post purchase.
Discussion Block
Discussion Notes:
• DLC improves overall data center Power Utilization Efficiency (PUE).

• DLC improves the power efficiency of servers and eliminates the need for
costly cooling infrastructures such as chillers and Computer Room Air
Conditioning (CRAC) units. It lowers the overall cost and improves the Total
Cost of Ownership (TCO).
• DLC improves the life of the IT infrastructure.

Server Components
Direct Liquid Cooling Technology
DLC example of a cold plate and coolant loop. Monolithic is used in the 15G rack servers and
modular is used in the Dell PowerEdge C6420 and C6520 servers.
DLC uses the exceptional thermal capacity of liquid to absorb and remove the
heat that is created by new high-power processors. Cold plates are attached
directly to the processors. The coolant captures and removes the heat from the
system to a heat exchanger in the rack or row.
This heat load is removed from the data center using a warm water loop,
potentially bypassing the expensive chiller system. By replacing (or
supplementing) conventional air-cooling with higher-efficient liquid cooling, the
overall operational efficiency of the data center is improved.

Server Components
New Features and Solutions
Leak Sensing Technology
Leak Sense technology provides customers with the knowledge that potential
issues are found and reported quickly.
If a coolant leak occurs, the system’s leak sensor logs an alert29 in the iDRAC
system.
29Three errors can be reported: small leak (warning), large leak (critical), leak
sensor error (warning – indicates the issue with the leak detection board) on the
iDRAC. These error detections can be configured to take meaningful actions using
tools like OpenManage Enterprise.

Server Components
POD Solution
POD solution containing two outer racks with node-level DLC and one middle In-Row Cooler.
A node-level DLC solution captures between 50%-60% of a server’s internal heat

(depending on the configuration).
The Dell rack-level POD solution30 concept is designed for total heat capture.
The POD solution contains front and back containment for racks of DLC servers,
plus an In-Row Cooler that is integrated between the IT racks to capture any
remaining heat.
Liquid cooling systems follow two basic architectures.
Monolithic Architecture
The Dell PowerEdge R650, R750, and R750xa follow the monolithic architecture.
30A pod or a cluster is a set of system units that are linked by high-speed
networks into a single unit.

Server Components
In the monolithic architecture, the Liquid Leak Sensor board connects to the
Complex Programmable Logic Device (CPLD) using the Liquid Cooling Rear I/O
board.
Modular Architecture
The Dell PowerEdge C6520 follows the modular architecture.
In the modular architecture, the Liquid Leak Sensor board connects directly to the
Complex Programmable Logic Device (CPLD).

Server Components
Discussion Notes:
• The Liquid Cooling Rear I/O (LC RIO) board is a component specific to the
monolithic architecture only (PowerEdge R650, R750, and R750xa).
• A high-level overview of the liquid cooling process:
− Depending on the platform, the Liquid Leak Sensor (LLS) board, connects
to the immediate upstream entity (LC RIO board for the monolithic
architecture or the CPLD for the modular architecture) using an alert cable.
− The message is then forwarded to the iDRAC using the SPI-X registers,
and the error is logged.
• If a leak develops in a particular cold plate and the detection cable is not
engaged, then the alarm signal will not be received. A disengaged alert cable
is reported as an error in the iDRAC logs.
• Here, the context of a 'modular architecture' does not include blade servers.
The Dell PowerEdge MX750c system does not support a liquid cooling
configuration.

Server Components
Liquid Cooling Setup
Liquid cooling is supported internally and externally in the 14G C6420 and some
15G PowerEdge servers.
Internal Support for DLC (with Leak Detection capabilities)
Within the sled, liquid cooling support is offered by:

• Cold Plates
• Liquid Leak Sensor (LLS) Board
• Liquid Cooling Rear I/O Board (specific to the monolithic architecture only)
• Liquid Cooling Module
External Support for Liquid Cooling
External support for liquid cooling is common for both the monolithic and modular
architectures.
Outside the rack, liquid cooling support is offered by:

• Rack manifolds:
− Coolant tubes come out of each sled in the server and connect to a
manifold. The manifold further connects to a Coolant Distribution Unit.
− The racks for liquid-cooled systems must be at least 1200 mm (47.24
inches) deep x 750 mm (29.52 inches) wide to allow the installation of the
manifolds with the Power Distribution Units (PDUs).
• Coolant Distribution Unit (CDU):
− CDUs connect to the rack manifold to pump coolant to the racks and
exchange heat from the servers with facility water.

Server Components
Liquid Cooling Connection Chain
Overview of the DLC with leak detection
Color code for cold water flow.
Color code for hot water flow.

Server Components
DLC Ecosystem
The image below shows the high-level overview of the DLC ecosystem.
DLC ecosystem
Discussion Block
Discussion Notes:
• CPU Cold Plates:

The CoolIT cold plates are a passive cooling solution that is managed by
centralized pumping architectures. Designed for use with the Intel Ice Lake
processors, the cold plates replace the traditional heatsink assembly.
The cold plates are sold and supported by Dell.
• Rack Manifold:
Made with reliable stainless steel and 100% nondrip quick disconnects,
rack manifolds can be arranged horizontally or vertically for a manual
connection at the front or rear of the rack.

Server Components
• Heat Exchanger:
Also called as Coolant Distribution Units (CDU), the liquid-to-liquid heat
exchangers can support either one rack or a group of racks for a cooling
solution.
CoolIT Systems Rack DLC product line offers various Heat ExchangeModules
depending on load requirements and availability of facility water, including CHx
(Liquid-to-Liquid), AHx (Liquid-toAir), and custom options.
• Liquid Leak Sensor (LLS) Board:
A Liquid Leak Sensor (LLS) is a mechanism that detects leaks within the liquid
cooling system in the system.
The LLS mechanism can determine if a leak is small (0.02 ml) or large (0.2 ml).
• Liquid Cooling Rear I/O board:
The Liquid Cooling Rear I/O board is a new design, specific to the monolithic
architecture. The Liquid Leak Sensor (LLS) board connects to the Liquid
Cooling rear I/O board using an alert cable.
• Liquid Cooling Module:
The liquid cooling module assembly consists of:

x2 leak detection boards connected to the external liquid cooling system using
x2 coolant tubes.
An alert cable connecting the liquid cooling module to the Liquid Cooling rear
I/O board.
Benefits of Liquid Cooling
The benefits of DLC implementation are:
• Increased System Cooling Capacity - DLC enables system configurations

that may not be possible with air cooling alone, such as high thermal design
power (TDP) CPUs, dense storage, and add-in cards.
• Improved Energy Efficiency (PUE) – The DLC cold plate solution reduces
energy costs by up to 45% relative to cooled air. It helps extend the life of
existing air infrastructure.
• Higher Compute Density – For the Intel Ice Lake based C6520 system, DLC
cooling supports up to 25% more cores per rack. For the AMD Milan based

Server Components
C6525 system (with backplane configuration supporting storage drives), DLC

cooling enables 2x the core count over air-cooling alone.
• Swift Serviceability – The CPU DLC cold plate solution attaches with four
screws, making service quick and simple.
Fresh Air Environment
The Dell Chiller-Less Fresh Air solution brings air into the data center from the
outside to support the cooling systems. The Dell Fresh Air Solution:
• It handles operating temperatures up to 45°c (113-degrees fahrenheits).

• It withstands up to 90 percent relative humidity.
• It eliminates compressor-based cooling during certain parts of the year.
Dell has validated a portfolio of servers, storage, networking, and power

infrastructure that has been developed for sustained operation at temperatures
ranging from 5°C (-23°F) to 45°C (113°F) and allowable humidity from 5 to 90
percent.
Dell Fresh Air 2.0 hardware includes specific configurations that can operate at
higher temperature and humidity levels and use clean outside air for air intake
instead of tightly controlled air conditioning (AC) from a cold aisle.
Fresh Air Environment Deployment Restrictions
The general configuration and device restrictions for deployment in a fresh air
environment are listed below.
Click here to learn more about Dell PowerEdge thermals.

Server Components
1:
• High-power PCIe cards (>75 W that use AUX cable, such as: GPU) Lower
power cards could also be excluded based on system limitations.
• Third-party PCIe card (any power levels).
2: Intel Optane DIMM.
3:
• Platforms with middrive and rear-drive configurations.

• PCIe SSD/NVMe drives.
• M.2/BOSS drives.
4: Tape backup unit (tape drives).
5: Power supply configuration that is not redundant.
Presence of following devices prevents the system to be fresh air compliant:

Server Components
• Power supply configuration that is not redundant.

• Intel Optane DIMM.
• High-power PCIe cards (>75 W that use AUX cable, such as, GPU) Lower
• High-power PCIe cards (>75 W that use AUX cable, such as, GPU) Lower
• Third-party PCIe card (any power levels).
• Tape Backup Unit (Tape Drives).
• Platforms with middrives and rear-drive configurations.
• PCIe SSD/NVMe drives.
• M.2/BOSS device.
• Each platform in addition has specific CPU TDP limits (that is CPU > x TDP is
not fresh air compliant) and is listed in platform-specific constraints.
• Status of the system fresh air compliance is reflected in iDRAC interface under
Cooling -> Temperature section.

Server Components
Networking
Network Daughter Card
NDC used in the Dell PowerEdge 12G, 13G, and 14G servers.
Older generation servers used a network interface card (NIC) built into the system
board. When upgrading or changing the NIC technology, users would install a
PCIe network interface controller in one of the PCIe slots in the server.
With the Dell PowerEdge 12G, 13G and 14G servers, the NICs are based on a
daughter card.31 Users can easily change network requirements as they evolve.
31The Network Daughter Card (NDC) is a custom form factor mezzanine card that
contains a complete NIC subsystem.

Server Components
A Dell Network Daughter Card (NDC)32 enables the user to choose the right
network fabric without using a valuable PCI slot. It presents an easy upgrade path
from 1 GbE to 25 GbE LAN speeds.
OCP Card
OCP card in a PowerEdge server.
The Open Compute Project (OCP)33 cards are network cards that connect to the
PCI bus. They are physically smaller than the Industry Standard Architecture (ISA)
expansion card and often connect to a dedicated connector on the system board.
The OCP card was introduced with the Dell PowerEdge 15G servers.
32 The NDC typical includes the features and behavior of a traditional LOM (LAN
on Motherboard) subsystem. It includes the added benefit of flexibility in terms of
providing the choice to customers to choose their favorite network types, speed,
and vendors.
33 The Open Compute Project (OCP) is an organization that shares designs of
data center products and best practices among companies. The designs and
projects include server designs, data storage, rack designs, open networking
switches, and so on

Server Components
The following are benefits of the OCP card:
• It is a removable networking card.

• Flexibility for customer to choose interconnect (10 GB, 25 GB, 50 GB, and so
on).
• It does not consume a regular PCIe slot.
• Replaces Network daughter Card (NDC) from previous generation servers.
• Physically smaller than the ISA expansion card, and often connect to a
dedicated connector on the system board.
Important: The OCP and the NDC cards are not a hot-swappable
component.
SNAP I/O
The above image shows balanced I/O with SNAP I/O.
A Storage Network Architecture and Parallel I/O or SNAP I/O network interface
controller consists of a PCIe card.
The SNAP I/O adapters enable both CPUs within a dual-socket server to connect
directly to the network through its own dedicated PCIe interface.
SNAP I/O results in low latency, CPU utilization and higher network throughput.
Some of the advantages of the SNAP I/O are:

Server Components
• The NIC is directly connected to both CPU sockets.

• The SNAP I/O bypasses the Ultra Path Interconnect (UPI).
• It provides maximum use of the available resources.
Important: Only certain riser configurations in the Dell PowerEdge

15G platforms have SNAP I/O. For example, they are available in
both Dell PowerEdge R650 and R750 but it depends the riser
configuration that is purchased by the user.
The card used in the 15G AMD servers (Dell PowerEdge R6525
and R7525) is called SNAP I/O. Whereas the card that is used in
the Intel-based 15G servers (Dell PowerEdge C6520) is called
SNAPI. They have the same architecture.

Server Components
SNAP I/O Cards for PowerEdge Servers
Below are some of the images of the SNAP I/O cards.
SNAP I/O ConnectX-5 Card
The image below is of a SNAP I/O ConnecX-5 dual-port 100 GbE only adapter. It
supports PCIe Gen3/Gen4 x16. It is supported by the iDRAC and the Lifecycle
Controller.
SNAP I/O ConnecX-5.
SNAP I/O ConnectX-6 Card
The bottom-left image is of a primary SNAP I/O card, and the bottom right is an
auxiliary card. The left image is the SNAP I/O ConnectX-6 single port VPI HDR
adapter. It supports PCIe Gen4 x16 and PCIe Gen3 x32 (with auxiliary card). The
iDRAC and Lifecycle Controller do not support this card.

Server Components
On the left, is a SNAP I/O ConnectX-6 single port VPI HDR adapter and on the right is an auxiliary
card.
SNAP I/O ConnectX-6 in a Dell PowerEdge Server
The image below shows the SNAP I/O ConnectX-6 card along with the auxiliary
card which is installed in a Dell PowerEdge server.
SNAP I/O ConnectX-6 card.

Server Components
SNAPI
The above image shows SNAPI-capable NIC directly connected to both CPUs bypassing QPI and
UPI34. It frees up bandwidth for applications and improves latency.
34 The Intel Ultra Path Interconnect (UPI) is a point-to-point processor

interconnect. UPI was developed by Intel which replaced the Intel QuickPath
Interconnect (QPI) in Xeon Skylake-SP platforms starting in 2017.

Server Components
Both SNAP I/O and SNAPI35 (also know as socket direct cards) are similar in how
they function. However, they connect to the CPU differently.
• SNAP I/O plugs into the system board.

• SNAPI uses a cable that connects to a riser.
The following components are required to enable SNAPI or SNAP I/O

configuration:
• SNAPI capable NIC.

• SNAPI capable risers.
• SNAPI cable that is connected to CPU 2 (In the Dell PowerEdge C6520, riser 1
B comes with a dedicated SNAPI cable connector that connects into the
system board).
Rear Input Output and LOM
The Rear I/O and the LAN on Motherboard (LOM) cards are available with
PowerEdge 15G servers.
35System for NUMA Aligned Partitioned I/O (SNAPI) allows an I/O device to
connect directly with multiple upstream CPU sockets. It bypasses inter-CPU
socket link usage and associated overheads such as NUMA latency penalty.

Server Components
RIO and LOM cards on a system board.
The Rear I/O card (RIO) consists of:

• iDRAC port36
• Video Graphics Array (VGA)
• USB
• ID Button
• Chassis intrusion switch cable
• Optional serial connector
PowerEdge 15G servers have the option of using the LOM, OCP, or both.
The LOM consists of a Broadcom 5720 1Gb dual port interface.
36 RIO has an iDRAC port, but the iDRAC chipset is on the system board.

Server Components
Important: When a user purchases a PowerEdge 15G server with

the Direct Liquid Cooling configuration, they get a different RIO
card. The alternate board does not come with a VGA port.
Graphic Processing Units (GPUs)
Accelerator Cards
Artificial Intelligence (AI) has different computing needs compared to traditional

CPUs. To fill intensive applications performance gaps, accelerators help to
remove server processing roadblocks.
GPUs
A Graphic Processing Unit (GPU) card offload compute-intensive portions37 of the

application to the GPU, while the remainder of the code still runs on the CPU.
Examples of GPU computation in a server:
• Vectored-floating point operations (games or movie rendering process).

• General purpose simulations (intensive number crunching).
37 A GPU typically has thousands of cores that are designed for efficient execution
of mathematical functions.

Server Components
CPUs consist of minimal cores optimized for serial processing, while GPUs consist of thousands of
smaller, more efficient cores designed for parallel performance.
FPGAs
Field Programmable Gate Arrays (FPGA) accelerators are cards can be

dynamically reprogrammed with a data path that exactly matches workloads, such
as data analytics, image inference, encryption, and compression.
In the image example, Intel FPGA Accelerated Network Function moves load
balancing, QoS and classifying tasks away from the CPU load.

Server Components
Intel FPGA card programmed to augment the capabilities of Virtual Network Functions running on
a carrier cloud.
ASICs
Application-Specific Integrated Circuits (ASICs) are cards with silicon devices built
for specific purpose such as graph computing with massively parallel, low-
precision floating-point computing.
Intelligent Processor Unit (IPU) chips in Graphcore are an AI processor.

Server Components
A Graphcore card with embedded IPUs, specifically designed for artificial intelligence.
Tip: The Graphcore IPU supports the PowerEdge R6525 and DSS
8440.

Server Components
PowerEdge Server Accelerator Support
Dell Technologies supports a broad array of server accelerators in Dell

PowerEdge servers.
Manufactu Model
rer
NVIDIA • Nvidia A100

GPU • Nvidia A40
• Nvidia A30
• Nvidia A10
• Quadro RTX 6000
• Quadro RTX 8000
• T4 Tensor Core
• Tesla V100/S
• Link: https://www.nvidia.com/en-us/data-center/a100/
AMD GPU • MI100

• Link: https://www.amd.com/system/files/documents/amd-cdna-
whitepaper.pdf
Intel FPGA • Stratix 10 SX

• Arria 10 GX
• Link:
https://www.intel.com/content/www/us/en/products/details/fpga/pl
atforms/pac.html
Xilinx • Xilinx U200

FPGA • Link: https://www.xilinx.com/products/boards-and-
kits/alveo/u200.html#solutions

Server Components
Graphcore Link: https://www.graphcore.ai/products/ipu

IPUs
Example: Click here to view an example of GPU configuration.
Tip: GPU product documentation:

https://www.delltechnologies.com/en-us/servers/server-
accelerators.htm#tab0=0
GPU Use Cases
Choosing GPUs and other accelerated architectures and products is a key

decision IT teams have in their hands. Once the decision is made, for the
appropriate workloads, then infrastructure strategy and product choices are
addressed.
The NVIDIA CUDA Toolkit includes GPU-accelerated libraries, debugging and

optimization tools. C/C++ compiler, and runtime library for application deployment.
• Machine and deep learning: Accelerate AI by enabling the parallel processing

power required to speed both training and inferencing workloads.
• Accelerated database: Accelerate speed aggregations, sorts, and grouping
operations to solve complex analytics operations that overload traditional
databases.
• Predictive analysis: Accelerate dynamic correlation and predictive outcomes
with greater speed, accuracy, and scale.
• Streaming data: The Internet of Things (IoT) - accelerate simultaneous
ingestion, exploration, and visualization of streaming data for real-time
analysis.

Server Components
• Financial modeling: Accelerate the HPC and artificial intelligence (AI) industry
to leverage massive datasets to better understand risk and return.
• Modeling and simulation: Provide modeling and simulation for early evaluation,
fast testing of design modifications enabling more iterations.
• Signal processing: Enable providers to model and analyze signal data streams
coming in from computers, radios, videos, and cell phones in real time.
• Visualization: Enhance performance for 3D visualization applications such as
computer-aided design, enabling software to draw models in real time as the
user moves them.
• Seismic processing: Oil and Gas - accelerate extraction information from
massive seismic data stores, speeding time to results and lowering costs.
Compute Unified Device Architecture
CUDA concept showing a four SM GPU and an eight SM GPU. Source:

https://developer.nvidia.com/blog/cuda-refresher-getting-started-with-cuda/
Compute Unified Device Architecture (CUDA) is the NVIDIA parallel computing

platform and programming model.
CUDA divides work into small independent work and solves independently among
the CUDA blocks.

Server Components
Tip: The NVIDIA CUDA Toolkit is a management software that

provides a development environment for creating high-
performance GPU-accelerated applications. To access the NVIDIA
CUDA Toolkit, go to https://developer.nvidia.com/cuda-downloads
To access the driver download portal of NVIDIA, go to
https://www.nvidia.com/Download/index.aspx
CUDA requires a supported version of Linux with a GCC complier and toolchain,
or Microsoft Windows or Microsoft Visual Studio, depending on the OS used.
The NVIDIA CUDA Toolkit includes GPU-accelerated libraries, debugging and

optimization tools. C/C++ compiler, and runtime library for application deployment.
CUDA Installation Instructions
The links that are provided below detail CUDA installation instructions by
operating system.
To access the NVIDIA CUDA

installation guide for Linux, go to
https://docs.nvidia.com/cuda/cuda-
installation-guide-linux/index.html
To access the CUDA installation guide

for Microsoft Windows, go to
https://docs.nvidia.com/cuda/cuda-
installation-guide-microsoft-
windows/index.html
To access the CUDA installation

instructions for VMware, go to
https://docs.vmware.com/en/VMware-
vSphere-Bitfusion/4.0/Example-
Guide/GUID-1E535E93-156A-4E27-
9DE3-0941F1D635EF.html

Server Components
System Board Inlet Temperature
iDRAC UI, System > Cooling > Temperatures.
The system board inlet temperature shouldbetween the minimum and maximum
warning threshold38.
System board temperature that exceeds thresholds degrades the GPU

performance.
If a system board inlet temperature warning message is logged, the GPUs lower
the power consumption39 to avoid thermal damage.
If the inlet temperature exceeds the critical threshold, a message is logged.
Server Upgrade - GPU Kit
The GPU full-length kit, half-length kit, and the GPU power cable kit are kits
available for customers. Depending on the kit ordered, the respective components
are available.
38 The range is optimal for GPU performance. The iDRAC sets the thermal
warning threshold when the GPU is installed.
39 Lowering the power consumption results in lower GPU performance.

Server Components
Use SolVe to generate the upgrade procedure for the GPU kit.
SolVe Online home page: https://solve.dell.com/solve/home
Caution: Do not install GPUs, network cards, or other PCIe

devices on your system that are not validated and tested by Dell.
Damage caused by unauthorized and invalidated hardware
installation will null and void the system warranty.
Warning: Consumer-Grade GPUs should not be installed or used

in the Enterprise Server products.
Expansion Card
PCIe Overview
Theoretical bandwidth for PCIe lanes.

Server Components
A PCIe40 (peripheral component interconnect express) is the bus that connects

supported peripheral components.
The slot size determines the maximum number of PCIe lanes.
Bandwidth scales linearly, so a x4 lane connection has twice the bandwidth of a x1

lane connection.
An x1, x4, x8, or x16 card can use a x16 lane slot. A system board can have
multiple slot types and support different PCIe versions.
PCIe Slots and Processors
PowerEdge R750 CPU and PCIe lanes. The PowerEdge R750 supports many riser and PCIe lane
configurations.
40 PCIe 3 and PCIe 4 are the bus types common today.

Server Components
The system board processors control the PCIe slots. Also, the system board
chipset may support PCIe slots.
Dell PowerEdge 14G servers support PCIe 3. PowerEdge 15G servers support
PCIe 3 and PCIe 4.
For example: the PowerEdge R750, with the use of expansion card risers, can
support up to 48 x 4.0 PCIe lanes per CPU.
PCIe Server Peripherals
Dell BOSS card.
Common PowerEdge server peripherals:

• BOSS
• PERC

Server Components
• Network Interface Card (NIC)

• Accelerators:
− Graphics Processing Units (GPUs)41

− Field Programmable Gate Arrays (FPGAs)42
− Intelligence Processing Unit (IPU)43
PCIe Card Form Factor
The PowerEdge server supports many different PCIe card form factors. The
graphic shows the standard full-length, half-length, and low profile dimensions.
Also, PowerEdge servers may support other form factors such as half-length, and
half-height (HLHH).
41 Co‑processors designed to accelerate compute performance.

42 Specific characteristics to run certain types of algorithms up to 1000X faster
than traditional software solutions.
43 Emphasize graph computing with massively parallel, low-precision floating point
computing.

Server Components
PCIe standard form factor dimensions.
Risers
Riser examples. Supported risers for the PowerEdge R750xa.
Riser cards enable users to install additional expansion cards for the server.
Riser cards can accommodate all servers, including 1U servers.

Server Components
Storage
Introduction to Server Storage
The Dell PowerEdge servers support storage options to

access, manage, secure, and provide compute services
for the data.
The type of storage option varies based on the cost,

performance, size, and storage space. This topic
provides an insight into the different types of storage
devices that a PowerEdge server supports.
Non-Volatile Memory express (NVMe)
Non-Volatile Memory express (NVMe) is a logical device interface for accessing

nonvolatile storage media that attach through a PCI Express (PCIe) bus or switch.
Tip: The NVMe devices may require installation of additional

devices such as PCIe extenders.

Server Components
Advantages
Dell NVMe storage drive in carrier and NVMe card.
Advantages of NVMe storage devices are:

• Enables PCIe lanes to communicate with flash media
• Provides higher input and output operations per second
• Eliminates the SCSI bottleneck caused due to array latency
• Reduces the latency of flash-based infrastructure
• Lowers power consumption of the server
14G servers use generation 3 PCIe technology to write to the drives using four
lanes. 15G servers use PCIe Gen 4 and 2 lanes to each drive.

Server Components
Cabling
The rear drive slot numbering differs based on the cable configuration of the rear
backplane.
NVMe cabling.
If the rear backplane is connected directly to the PERC Controller or system

board, then the rear drive slots start at 0.

Server Components
Form Factor
NVMe PCIe SSDs use a 2.5 inch (U.2) in carrier.
NVMe PCIe SSDs use a 2.5 inch (U.2) and an add-in controller (AIC) form factor.
The NVMe PCIe SSD U.2 installs into a carrier. The NVMe PCIe SSD AIC form
factor installs into the appropriate system board slot.
Paddle Card - 15G Only
Paddle cards connect an NVMe backplane to the system board using cables. The
paddle card interfaces with the system board chipset.
When paddle cards are used, the onboard s150 is controlling the NVMe disks.
Paddle cards are similar to risers, but they do not have the riser cage. Paddle
cards provide efficient data management on systems with many storage devices.
The paddle cards are only available with certain riser configurations. Not all
systems will come with paddle boards, it is dependent on the users configuration.
For example, in the PowerEdge R750, the configuration that supports 24 X 2.5"
hard drives with the backplane has paddle cards for efficient management.
Installing the R4 paddle card on PowerEdge R750xa.

Server Components
1:
Paddle cards used in Dell PowerEdge servers.

Server Components
Location of paddle card in 15G server.
Paddle card views

Server Components
Internal Dual SD Module
Internal Dual SD Module with dual SD cards.
The Internal Dual SD Module (IDSDM) provides a redundant SD-card module for
embedded hypervisors. Users configure the IDSDM for storage or as the operating
system boot partition.
Features of the IDSDM card are:
• Maintains mirror mode configuration by using SD cards in both slots to provide

redundancy

Server Components
• Provides high performance and data protection44

• Provides active mode and standby mode options for dual SD cards
• Supports single card operation without redundancy
Important:
When the redundancy option is set to Mirror Mode, the information
is replicated from one SD card to another.
ESXi 7.0 must be at 7.0 U2c to avoid potential issues with writing to
the SD cards. Find more information about partition intermittently
breaks in the VMware knowledge base.
The IDSDM also supports a vFlash card with an iDRAC Enterprise
license.
vFlash only ships with the IDSDM on 14G servers. The PowerEdge
15G servers no longer support vFlash with the iDRAC.
Boot Optimized Storage Solution (BOSS)
Dell offers two types of Boot Optimized Storage Solution (BOSS) cards.
BOSS-S1
BOSS-S1 is a simple RAID solution card designed specifically for booting a

server's operating system.
44 The data is written on both cards, but the data is read from the first card. If the
first card fails or is removed, the second card automatically becomes active.

Server Components
Features of BOSS-S1 are:
• Supports up to two 6 Gbps M.2 SATA drives.

• M.2 devices are read-intensive with 240 GB or 480 GB capacity.
• Creates a single virtual disk from the available hard drives.
• Available in the low-profile and full-height form factors.
Click here to learn more about the Dell BOSS-S1 card through the Dell Boot
Optimized Server Storage-S1 User's Guide.
BOSS-S2
Dell Technologies Boot Optimized Storage Solution-S2 (BOSS-S2) is a RAID

solution card that is designed for booting a server's operating system. The Dell
15G and future servers support BOSS-S2 cards. BOSS-S2 provides
enhancements over the BOSS-S1. Some of the features are:

Server Components
• Supports up to two 80 mm (3.14 in) M.2 SATA solid-state devices.

• M.2 devices are read-intensive with 240 GB or 480 GB capacity.
• Provides a rear-facing module for quick and easy accessibility to the M.2
SSDs.
• Enables full hot-plug support.
• Connects using PCIe Gen 2.0 with four lanes.
• Supports two SATA generation 3 ports.
Click here to learn more about the Dell BOSS-S2 through the Dell Technologies
Boot Optimized Storage Solution-S2 User's Guide.
Important: Configure and manage BOSS cards through the iDRAC,

System Setup Utility, OMSA, and BOSS CLI.

Server Components
BOSS Management on iDRAC
BOSS Management on System Setup Utility

Server Components
BOSS Management on OMSA
BOSS CLI Commands
Command Description Example Usage
info -o hba Display BOSS-S1 controller mvsetup info -o hba

information
info -o vd Display virtual drive mvsetup info -o vd

information
info -o pd Display physical drive ./mvcli info -o pd

information
smart -p <PD_ID> Display SMART information ./mvcli smart -p 0

of physical drive
event Display controller events ./mvcli event
RAID
Redundant Array of Independent/Inexpensive Disks (RAID) combines multiple

hard drive components into a single logical unit for the purposes of data
redundancy and data protection.

Server Components
Data is distributed across the drives in several ways known as RAID levels. Based
on the customer requirements, the RAID levels can be configured for optimal
performance. Click here to learn more about available RAID levels and
specifications.
1:
The features of RAID 0 are:
• Enables the system to write data across multiple

hard drives instead of one hard drive.
• Divides each hard drive storage space into 64 KB
stripes.
• Enhanced performance as multiple hard drives are
accessed simultaneously, but does not provide data
redundancy.
The advantages of RAID 0 are:

• Performance boost for read and write operations due to the striping of data
across multiple disks.
• Increases the total size of available space that is presented to the operating
system.
The disadvantages of RAID 0 are:

• Does not provide redundancy/duplication of data.
• The failure of one of the disks leads to entire data loss.

Server Components
2:
• Data written to one disk is simultaneously written to

another disk.
• If one disk fails, the contents of the other disk can be
used to run the system and rebuild the failed hard
drive.
• Provides data redundancy as the contents of the
disk are written to a second disk.

• Improves read performance since different blocks of data can be accessed
from all the disks simultaneously.
• A multithreaded process can access block 1 from disk 1 and block 2 from disk
2 at once thereby increasing the read speed.
• Ideal for mission critical storage and hosting operating systems.

• Write performance is reduced since all the drives must be updated whenever
new data is written.
• Disk space is wasted to duplicate the data thereby increasing the cost to
storage ratio.

Server Components
3:
• Combines distributed parity45 with disk striping.

• Provides redundancy for one hard drive failure without duplicating the contents
of entire hard drive.

• Provides high read performance.
• Efficient use of drive capacity.
• If one disk fails, the data is created from the parity information that was created
for redundancy.

• Longer rebuild of data as compared to RAID 0 due to parity recalculation.
45Parity data is redundant data that is generated to provide fault tolerance within
certain RAID levels.

Server Components
4:
RAID 6 uses the concept of dual parity with block-level disk striping. RAID 6 allows
two disk failures without duplicating the contents of entire physical disks. The disk
capacity is calculated by n-2. If there are four disks, then the virtual disk capacity
is the total size of two disks.
The minimum number of disks required to configure RAID 6 is four. RAID 6 can
have a maximum of 32 drives.

• Provides data redundancy.
• The dual parity provides high read performance.

• Write performance decreases due to dual parity calculations.
• Additional cost required due to two disks dedicated parity.

Server Components
5:
An example of RAID 10.
RAID 10 combines RAID 0 and RAID 1 with a minimum of four disks. In RAID 10,
two disks are striped and mirrored onto two other disks, creating a single array of
disk drives.

• High read performance.
• Good fault-tolerance.
• Provides faster data recovery due to data redundancy.
• Faster rebuild time as compared to other RAID levels.

• Expensive and complex to setup as compared to other RAID levels.
• Essentially uses only half of its storage capacity.
• One drive can fail from each mirrored set, but two drives cannot fail within the
same RAID 1 set. Failure of two drives in the same span leads to data loss.

Server Components
6: RAID 50 (RAID 5+0), a type of nested RAID level, combines the block-level
striping of RAID 0 with the distributed parity of RAID 5.
The minimum number of disks required to configure RAID 50 is six.

• Multiple disk failures are tolerated before the data loss.
• Provides better fault-tolerance than other RAID levels.

• Failure of two drives in the same span leads to data loss.
7: Features of RAID 60 are:
• Parity is distributed across all drives in the array.

Server Components
• Fault Tolerance - Disk errors, dual disk failures.

• Advantage - Data redundancy, high read performance.
• Disadvantage - Write performance decrease due to dual parity calculations.
Extra cost due to 2 disk equivalent devoted to parity.
Important: RAID levels can be configured through the Lifecycle

Controller, System Setup Utility, and OMSA.

Server Components
RAID 0
RAID 0 uses the concept of striping that allows data to be written across multiple
hard drives instead of one physical disk. RAID 0 involves the partitioning of each
physical disk storage space into 64 KB stripes.
The minimum number of disks required to configure RAID 0 is two. RAID 0 can
• Performance boost for read and write operations due to the striping of data
across multiple disks.
• Increases the total size of available space that is presented to the operating
system.

Server Components
• Does not provide redundancy/duplication of data.

• The failure of one of the disks leads to entire data loss.
RAID 1
RAID 1 uses the concept of data mirroring. Data is mirrored or cloned to other
disks so that if one of the disks fails, the other one can be used.
The minimum number of disks required to configure RAID 1 is two. RAID 1 can
• Improves read performance since different blocks of data can be accessed

from all the disks simultaneously.
• A multithreaded process can access block 1 from disk 1 and block 2 from disk
2 at once thereby increasing the read speed.
• Ideal for mission critical storage and hosting operating systems.

Server Components
• Write performance is reduced since all the drives must be updated whenever
new data is written.
• Disk space is wasted to duplicate the data thereby increasing the cost to
storage ratio.
RAID 5
RAID 5 uses the concept of distributed parity with block-level disk striping. RAID 5
stripes data blocks across multiple disks like RAID 0 while storing parity
information. The disk capacity is calculated by n-1. If there are three disks, then
the virtual disk capacity is the total size of two disks.
The minimum number of disks required to configure RAID 5 is three. RAID 5 can

Server Components
• Provides high read performance.

• Efficient use of drive capacity.
• If one disk fails, the data is created from the parity information that was created
for redundancy.
• Longer rebuild of data as compared to RAID 0 due to parity recalculation.
RAID 6
RAID 6 uses the concept of dual parity with block-level disk striping. RAID 6 allows
two disk failures without duplicating the contents of entire physical disks. The disk
capacity is calculated by n-2. If there are four disks, then the virtual disk capacity
is the total size of two disks.
• Provides data redundancy.

• The dual parity provides high read performance.

Server Components
• Write performance decreases due to dual parity calculations.

• Additional cost required due to two disks dedicated parity.
RAID 10
RAID 10 combines RAID 0 and RAID 1 with a minimum of four disks. In RAID 10,
two disks are striped and mirrored onto two other disks, creating a single array of
disk drives.

• Good fault-tolerance.
• Provides faster data recovery due to data redundancy.
• Faster rebuild time as compared to other RAID levels.
• Expensive and complex to setup as compared to other RAID levels.

Server Components
• Essentially uses only half of its storage capacity.

• One drive can fail from each mirrored set, but two drives cannot fail within the
same RAID 1 set. Failure of two drives in the same span leads to data loss.
RAID 50
RAID 50 (RAID 5+0), a type of nested RAID level, combines the block-level
striping of RAID 0 with the distributed parity of RAID 5.
The minimum number of disks required to configure RAID 50 is six.
• Multiple disk failures are tolerated before the data loss.

• Provides better fault-tolerance than other RAID levels.
• Failure of two drives in the same span leads to data loss.

Server Components
RAID 60
RAID 60 (6+0), a type of nested RAID level, combines the block-level striping of
RAID 0 with the dual distributed parity of RAID 6.
The minimum number of disks required to configure RAID 60 is eight.
• High data redundancy.

• High disk space consumption.

Server Components
RAID Level Comparison
The table highlights the major differences between each RAID level.
RAID 0 RAID 1 RAID 5 RAID 6 RAID 10 RAID RAID 60

50
Descript Data Disk Data Data RAID 0 RAID 0 RAID 0

ion striping mirroring striping striping and and and
(no data with with two RAID 1 RAID 5 RAID 6
protectio distribute distribut combine combin combine
n) d parity ed parity d ed d
blocks
Number 1-32 2 3-32 4-32 4-192 6-192 8-192

of hard
drives
(PERC
9)
Number 1-32 2 3-32 4-32 4-240 6-240 6-240

of hard
drives
(PERC
10)
Fault None Single Single Double One per Disk Disk

Toleran disk disk disk mirror set errors, errors,
ce failure failure failure single dual disk
disk failures
failures
Benefit Highest Data Best Highest Highest Provid Data

performa protectio balance data performa es redundan
nce n of cost, protectio nce with increas cy, high
through performa n data ed fault read
redunda nce, and through protectio toleran performa
ncy data redunda n ce nce
protection ncy

Server Components
Hot Spare
System Setup Utility displaying the hot spare options.
Hot spares46 are dedicated standby disks. When a hard drive that is used in a
virtual disk fails, the assigned hot spare47 is activated to replace the failed hard
drive without interrupting the system or requiring any intervention. When a hot
spare is activated, it rebuilds the data for all redundant virtual disks that were
using the failed hard drive.
46 A hot spare must be at least as large as the drive it is to replace, and a hot
spare must be the same drive type (SAS/SATA) as the drive it is to replace.
47 Hot spare cannot be assigned to 7200 RPM disks to replace 10 drives. It also
cannot be assigned to hard drives with large size.

Server Components
The types of hot spare are:
• Global Hot Spare - Assigned to all virtual disks on the controller.

• Dedicated Hot Spare - Assigned to only one virtual disk on the controller.
• Persistent Hot Spare - Once enabled, any slots with hot spares configured
automatically become persistent hot spare slots48. If a hot spare disk fails or is
removed, a replacement disk that is inserted into the same slot automatically
becomes a hot spare with the same properties.
• Revertible Hot Spare - The replace member functionality allows a previously
commissioned hot spare to revert to a usable hot spare. When a disk failure
occurs within a virtual disk, an assigned hot spare, dedicated, or global, is
commissioned and begins rebuilding until the virtual disk is optimal.
Possible RAID Level Migrations
The table illustrates possible RAID Level Migrations.
Source Target Number of Number Capacity Description

RAID RAID Physical of Expansion
Level Level Disks Physical Possible
(Beginning) Disks
(End)
RAID 0 RAID 0 1 or more 2 or more Yes Increases

capacity by
adding disks.
48The PERC 10 series can be configured so that the system backplane or storage
enclosure disk slots are dedicated as hot spare slots. This feature can be enabled
using the Dell OpenManage storage management application.

Server Components
RAID 0 RAID 1 1 2 Yes Converts a

nonredundant
virtual disk into
a mirrored
virtual disk by
adding one
disk.
RAID 0 RAID 5 1 or more 3 or more Yes Adds distributed

parity
redundancy; at
least one disk
must be added.
RAID 0 RAID 6 1 or more 4 or more Yes Adds dual

distributed
parity
redundancy; at
least two disks
must be added.
RAID 1 RAID 0 2 2 or more Yes Removes

redundancy
while increasing
capacity.
RAID 1 RAID 5 2 3 or more Yes Maintains

redundancy
while adding
capacity.
RAID 1 RAID 6 2 4 or more Yes Adds dual

distributed
parity
redundancy and
adds capacity.

Server Components
RAID 5 RAID 0 3 or more 2 or more Yes Converts to a

nonredundant
virtual disk and
reclaims disk
space that is
used for
distributed
parity data; one
disk can be
removed.

capacity by
adding disks.
RAID 5 RAID 6 3 or more 4 or more Yes Adds dual

distributed
parity
redundancy; at
least one disk
needs to be
added.
RAID 6 RAID 0 4 or more 2 or more Yes Converts to a

nonredundant
virtual disk and
reclaims disk
space that is
used for
distributed
parity data; two
disks can be
removed.

Server Components
RAID 6 RAID 5 4 or more 3 or more Yes Removes one

set of parity
data and
reclaims disk
space that is
used for it; one
disk can be
removed.

capacity by
adding disks.
RAID 10 RAID 4 or more 6 or more Yes Increases

10 capacity by
adding disks;
an even
number of disks
must be added.
PERC Overview
The Dell PowerEdge RAID Controller (PERC) is a series of RAID disk storage
controllers which support SAS, SATA hard drives, and Solid-State Drives (SSDs).
NVMe hardware RAID support is available with the PERC 11 (H755N front,
H755MX and H755 adapter).
H745 Adapter

Server Components
1 Heat sink
2 Battery
3 Battery cable connector
4 SAS or SATA backplane connector B
5 SAS or SATA backplane connector A
6 PCIe Connector

Server Components
H745 Front
1 PCIe cable connector
2 Battery
3 Heat sink
7 Power connector

Server Components
H345 Adapter
1 Heat sink
4 PCIe connector

Server Components
H345 Front
2 Heat sink
5 Power connector

Server Components
H755 Adapter
1 Heat sink
2 PCIe connector
3 Battery
4 Backplane connector A
5 Backplane connector B

Server Components
H755 Front
1 Battery
2 PCIe input connector
3 Power card edge connector
4 Heat sink

Server Components
H755 NVMe
1 Battery
4 Heat sink

Server Components
H755MX
The PERC11 controller introduces new features that boost performance. PERC11
supports the PCIe Gen4 host interface and the upgraded DDR4 8GB 2666MT/s
cache memory. However, the greatest addition to this generation of technology is
the inclusion of NVMe hardware RAID support. NVMe hardware RAID support is
available on the H755N front, H755MX and H755 adapter form factors.
The H755MX cable is used and shipping in modular systems such as with the
MX700 chassis.
Important: The PERC H755MX does not support the MX5016s

storage sled. The customers who want to use the MX5016s should
use the HBA300MMZ (manages internal disks only) or jumbo
PERC (manages both internal and storage sled disks). Refer to
PERC Shows as Missing in R740 / R740xd After Parts
Replacement available on Dell support site to learn more.
The PowerEdge Expandable RAID Controller (PERC) 11 series49 consist of

SAS/SATA and NVMe controllers.
• PERC 10 controllers have three form factors: mini monolithic, MX, and adapter.
49 The PERC 11 provides reliability, high performance and fault-tolerant disk

subsystem management. PERC controllers include support for RAID levels 0, 1, 5,
6, 10, 50, and 60.

Server Components
• The mini-monolithic controller plugs in to the system board and the backplane
using a cStack cable.
• The MX version connects to the system board through the mini mezzanine slot.
• The PCIe adapter plugs into the server riser card. The backplane is then
connected using an internal SAS cable.
• PERC 1150 and PERC 1051 controllers replace the mini monolithic form factor
with the front PERC. This redesign has shorter cabling between the PERC,
backplane, and system board.
• PERC cabling is used for all data communication including Storage Enclosure
Services (SES), SAS, PCIe, and I2C data.
50 PERC 11 cards are an upgraded version of PERC 10 cards and are used with
Intel 15G PowerEdge servers.
51 PERC 10 cards were released with PowerEdge 14G and 15G servers.

Server Components
Below are different types of PERC 10.6 and 11.1 cards supported by PowerEdge
15G servers.
1: Features of PERC H755 Adapter
1 Heatsink
2 PCIe connector
3 Battery

Server Components
1 Heatsink
2 Battery
6 PCIe connector

Server Components
3: Features of PERC H745 Front
2 Battery
3 Heatsink
7 Power connector

Server Components
1 Heatsink
4 PCIe connector

Server Components
5: Features of PERC H345 Front
2 Heatsink
5 Power connector

Server Components
6: Features of PERC H755 Front SAS
1 Battery
2 PCIe input connector
4 Heatsink

Server Components
7: Features of PERC H755N Front NVMe
1 Battery
4 Heatsink

Server Components
8: PERC H755 MX
NVMe support requires universal backplane.
The PERC H755MX does not support the MX5016s storage sled. The customers
want to use the MX5016s should use the HBA300MMZ (manages internal disks
only) or jumbo PERC (manages both internal and storage sled disks).
The PERC 11 provides reliability, high performance and fault-tolerant disk

subsystem management. PERC controllers include support for RAID levels 0, 1, 5,
6, 10, 50, and 60.

Server Components
PERC H755 adapter
The features of PERC H755 adapter are:
PERC H755 adapter.
1. Heatsink
2. PCle connector
3. Battery
4. Backplane connector A
5. Backplane connector B
6. Battery cable connector

Server Components
PERC H755 front SAS
The features of PERC H755 front SAS are:
PERC H755 front SAS.
1. Battery
2. PCle input connector
3. Power card edge connector
4. Heatsink

Server Components
PERC H755 front NVMe
PERC H755N front NVMe.
The features of PERC H755N front NVMe are:
1. Battery
2. PCle cable connector
4. Heatsink

Server Components
PERC H755 MX adapter
PERC H755 MX adapter.
The features of PERC H755 MX adapter are:
1. Battery under cover

2. Heatsink
3. PCle cable connector
PERC H750 adapter SAS
The features of PERC H750 adapter SAS are:
PERC H750 adapter SAS.
1. Heatsink
2. Battery
5. PCle connector

Server Components
1. Heat sink
2. Backplane connector
4. PCle connector
PERC H355 front SAS
The features of PERC H355 front SAS are:
PERC H355 front SAS.
1. PCIe input connector

2. Heatsink

Server Components
1. Heatsink
3. PCle connector

Server Components
NVMe Support with PERC 11
The PERC 11 features that support NVMe are:

• The PERC 11 supports Dell PCIe Gen3 and Gen4 NVMe devices.
• The PERC 11 controller supports up to 8 direct-attached NVMe drives. Links to
devices are x2 width for all NVMe Devices, even for drives that support x4.
• The PERC H755N is a non-transparent bridge that acts as a root port to NVMe
drives.
• A virtual disk (VD) made of NVMe drives is displayed as SCSI VD to the host
operating system.

Server Components
• A PERC translates the SCSI instructions and passes the instructions to the
NVMe drives.
• Windows Device Manager lists all the NVMe drives.
The image shows how the Windows Device Manager lists the NVMe drives.
PERC Configuration Modes
In the Dell 15G servers, PERC has two options for enclosure configuration mode:
Unified Mode and Split Mode.
• The Unified Mode is the default mode.

• Use Split Mode for a high-performance with no failover or High Availability (HA)
functionality.
Edit enclosure mode
Go to Storage Configuration. Expand Enclosure Configuration.

Server Components
Select the mode
In Unified Mode, all 24 drives are connected to the front controller.
The split mode is indicated as <X:Y>. By default, the split mode is a <12:12> split.
X slots are assigned to one controller and Y slots are assigned to a different
controller.
Once the mode is selected, click Add to Pending Operations.

Server Components
Apply Changes
Once added to pending operations, click Apply Now to initiate the configuration
operation.
After the job is completed, a cold reboot is required to apply the changes.

Server Components
PERC Card Matrix
PERC 10 Specifications
Feature PERC H345 PERC H745 PERC H745P MX
RAID Levels 0, 1, 10 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60
Cache N/A 4 GB DDR4 2133 8 GB DDR4 2133

memory Mhz cache Mhz cache
Drives type 3 Gbps SATA, 6 3 Gbps SATA, 6 3 Gbps SATA, 6

Gbps SATA or Gbps SATA or Gbps SATA or
SAS, and 12 Gbps SAS, and 12 Gbps SAS, and 12 Gbps
SAS SAS SAS
PCIe support Generation 3 Generation 3 Generation 3

Technical Specification of PERC 10 Cards.
PERC 11 Specification
Feature PERC H755 PERC H755 Front PERC H755N

Adapter SAS Front NVMe
RAID Levels 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60
Cache 8 GB DDR4 2666 8 GB DDR4 2666 8 GB DDR4 2666

memory MT/s cache MT/s cache MT/s cache
Drives type 3 Gbps SATA, 6 3 Gbps SATA, 6 Gen3 (8 GT/s) and

Gbps SATA or Gbps SATA or Gen4 (16 GT/s)
SAS, and 12 Gbps SAS, and 12 Gbps NVMe
SAS, Gen3 (8 SAS
GT/s), and Gen4
(16 GT/s) NVMe
PCIe support Generation 4 Generation 4 Generation 4

Technical Specification of PERC 11 Cards

Server Components
For detailed specifications on the PERC cards visit dell.com/support.
Ask: List few PERC cards from PERC 10.6 series which support 14G and 15G
both.
Accept answers and discuss. Ensure that the following points are covered:
• PERC 10.6 supports SAS or SATA Only.

• H345 Controller has no battery nor DDR. The PERC H345 supports only RAID
0,1,10 and SysPDs.
• H740/H745/H840 has two modes, RAID Mode and eHBA mode.
− RAID Mode: All RAID Levels. VDs' start at VD0 and counts up.
− eHBA Mode: RAID 0, 1, 10 and EPD-PTs. VDs' start at VD239 and counts
down.
• The PERC 11.1 supports SAS/SATA or NVMe depending on the configuration
of the system.
• Single RAID mode that supports all RAID levels and EPD-PTs.
− VDs' start at VD239 and counts down.

Server Components

Server Components
Backplanes

Server Components
PowerEdge backplane.
A backplane is a high-speed communication circuit board52 that provides a single

interface for storage devices, better logical control, and faster connection speeds.
Some of the features of the server backplane are:
• HDDs/SSDs connect to the front of the backplane.

• SATA/SAS/NVMe cables with power and I2C data cables connect to the rear
of the backplane.
• Cables connect from the front control panel to the backplane.
• Drives fail to power on when the backplane loses power connection.
• System fails to connect to the storage drives when the connection between the
backplane and the system controller is interrupted.
• The backplane configuration may differ based on the system design.
The backplane is the board where the HDDs/SSDs connect. Logically splitting into
two separate backplanes to mirror channels creates a single point of failure so if
multiple HDDs lose connection the backplane should be investigated. All the
drives not powering on indicates that the backplane may have lost its power. Also
the drives may be powered on but the system reports that it cannot connect to any
or some of the drives. The data connection failing between the backplane and the
system board or disk controller may cause the issue.
An expander enables support for more drives.
If the system does not have a backplane, users can cable drives directly to the
system board.
52 14G and 15G SAS backplanes meet 12 Gbps SAS and 6 Gbps SATA
requirements. 15G NVMe backplanes meet 16 GTs PCIe Gen4 requirements with
2 lanes per drive. 14G NVMe backplanes meet 8 GTs PCIe Gen3 requirements
with 4 lanes per drive.

Server Components
PowerEdge servers with a front PERC (fPERC) card
A front mounting fPERC module of PowerEdge R650 server.

Server Components
Rear mounting front PERC module of PowerEdge R750 server.
Universal Backplane
The features of a universal backplane are:
1. DST_SA2 (backplane to front PERC).

2. DST_PB2 (PCIe/NVMe connector).
3. DST_PA2 (PCIe/NVMe connector).
4. DST_SA1 (PERC to backplane).
5. BP_PWR_1 (backplane power and signal cable to system board).
7. DST_PB1 (PCIe/NVMe connector).
Example of cabling in PowerEdge R740 server using a monolithic card

Server Components
Important: Refer to NVMe and I/O Topologies for Next Generation

Dell PowerEdge Servers for I/O interconnect topologies spanning
the latest PowerEdge rack server portfolio. For example, the Dell
PowerEdge R750 Installation and Service Manual lists cables and
cable routing in the R750.

Introduction to Server Security
Server Security Overview
Server security focuses on protecting the data and resources that are
stored in servers.
Server security is critical for securing the IT infrastructure.
The security approach is built-in and not bolted-on53.
The latest PowerEdge servers support an enhanced cyber resilient

architecture that uses silicon-based Root of Trust (RoT)54 to increase the
server security.
53 The modern IT security approach is engineered and integrated as an

entity, not bolted on as an afterthought.
54 Root of Trust is a concept that starts a chain of trust needed to ensure
computers boot with legitimate code.

1: Cryptographically Verified Trusted Booting: Anchors end-to-end

server safety and overall data center security. It includes features like
silicon-based RoT, digitally signed firmware, and automatic BIOS
recovery.
2/3: Secure Boot: Checks the cryptographic signatures of Unified

Extensible Firmware Interface (UEFI) drivers and other code that is loaded
before the Operating System runs.
4: iDRAC Credential Vault: A secure storage space for credentials,

certificates, and other sensitive data that is encrypted with a silicon-based
key unique to each server.
5: Dynamic System Lockdown: Secures any system configuration and

firmware from malicious or unintended changes while alerting users to any
attempted system changes.
6: Enterprise Key Management: Delivers a central key management

solution to manage data-at-rest across the organization.
7: System Erase: Allows users to easily retire or repurpose the latest

PowerEdge servers by securely and quickly wiping data from storage
drives and other embedded nonvolatile memory.
8: Supply Chain Security: Provides supply chain assurance by ensuring

there is no product tampering or counterfeit components before shipping
products to the users.
• Cryptographically Verified Trusted Booting: Anchors end-to-end

server safety and overall data center security. It includes features like
silicon-based RoT, digitally signed firmware, and automatic BIOS
recovery.
• Secure Boot: Checks the cryptographic signatures of Unified
Extensible Firmware Interface (UEFI) drivers and other code that is
loaded before the Operating System runs.
• iDRAC Credential Vault: A secure storage space for credentials,
certificates, and other sensitive data that is encrypted with a silicon-
based key unique to each server.

• Dynamic System Lockdown: Secures any system configuration and

firmware from malicious or unintended changes while alerting users to
any attempted system changes.
• Enterprise Key Management: Delivers a central key management
solution to manage data-at-rest across the organization.
• System Erase: Allows users to easily retire or repurpose the latest
PowerEdge servers by securely and quickly wiping data from storage
drives and other embedded nonvolatile memory.
• Supply Chain Security: Provides supply chain assurance by ensuring
there is no product tampering or counterfeit components before
shipping products to the users.
Trusted Platform Module
The Trusted Platform Module (TPM) is a hardware security device that

provides the server with the ability to create cryptographic keys. The
cryptographic keys are used for encryption and decryption.
TPM employs a specialized chip55 that stores protected key information in

a tamper-proof medium to authenticate the host system hardware.
55 The chip includes a unique endorsement key that is baked into the
module during manufacturing, like a digital fingerprint to establish the
trustworthiness of data and applications. This cross-platform solution
engages at the lowest level of system operation, protecting against
unauthorized firmware and software modifications that can undermine
system integrity.

When TPM is enabled on a device, the resident operating system works

together with the device to encrypt the hard drives. TPMs are passive
devices.56 So, they do not have the intelligence to communicate.
The TPM cannot be removed from one system board and installed on
another system board.
1: Hashing: Used to convert the input (string of characters) of any length

to a fixed size value which represents the original string using an
algorithm.
2: Random Number Generation: A technique of generating a sequence

of numbers/symbols that cannot be predicted by anything other than a
random chance.
56 TPMs only receive commands and return responses.

3: Asymmetric Key Generation: A technique of generating two keys - a

public key and a private key which are used in cryptography.
4: Asymmetric Encryption/Decryption (Public Key Cryptography):

This technique uses the public key (known to all users) and the private key
(known only to the owner) for encryption and decryption.
The TPM security features are:

• Hashing: Used to convert the input (string of characters) of any length
to a fixed size value which represents the original string using an
algorithm.
• Random Number Generation: A technique of generating a sequence
of numbers/symbols that cannot be predicted by anything other than a
random chance.
• Asymmetric Key Generation: A technique of generating two keys - a
public key and a private key which are used in cryptography.
• Asymmetric Encryption/Decryption (Public Key Cryptography): This
technique uses the public key (known to all users) and the private key
(known only to the owner) for encryption and decryption.

Trusted Platform Module 2.0
Introduction to TPM 2.0
TPM 2.0 supports newer hash

algorithms, SHA-256, which can
improve driver signing and key
generation performance.
SHA-256 is 256 bits or 32 bytes

whereas SHA-1 is 20 bytes. The
SHA-1 algorithm is being
deprecated in favor of stronger
algorithms, such as SHA-256.
The advantages of TPM 2.0 over

TPM 1.2 are:
• TPM 2.0 supports the SHA-256
hashing algorithm, whereas
TPM 1.2 supports only SHA-1,
which has known vulnerabilities.
• TPM 2.0 also supports newer
operating systems: Windows
Server 2016 and later versions as well as Ubuntu 16.04 and later
versions.
TPM 2.0 is not fully supported in legacy BIOS mode because there is
no pointer to TPM logs in legacy BIOS mode.
TPM 2.0 Hierarchies
Each domain or hierarchy of TPM has its own resources and controls.
TPM 2.0 supports three domains:

• Security:
Storage Hierarchy- The storage hierarchy is used by the end user or
by the enterprise IT department for general cryptographic usage.
• Platform:

Platform Hierarchy- The platform hierarchy is under the control of the

platform manufacturer. The hierarchy includes functions that protect
platform or firmware integrity.
• Privacy:
Endorsement Hierarchy- The endorsement hierarchy is used when the

user has privacy concerns. The endorsement administrator has access
to some protected TPM commands and functionalities.
Configuring the TPM
Different settings are used on Windows Server 2012R2, Windows Server

2016 and later versions to match the operating system capabilities.
Windows Server 2016 and later versions:
The BIOS settings need certain modifications to fully leverage the

Windows Server 2016. Modifying the BIOS enables the server for the TPM
guarded host deployment that is required to run shielded virtual machines.
Guarded hosts57 and shielded Virtual Machines58 are new to Windows
Server 2016 and later versions.
57 Guarded hosts can run shielded Virtual Machines (VMs).

58 Shielded VMs are supported by a set of security controls that protect
against rootkits and bootkits.

Accessing the TPM using the System Setup menu.
Silicon-Based Hardware Root of Trust (RoT)
Silicon-based RoT is the latest security feature implemented on

PowerEdge servers. The concept is rooted in the foundational security
and protection of each PowerEdge server.
The PowerEdge 15G servers use a Silicon-based RoT to attest to the

integrity of the code running. The servers ensure that no unauthorized
BIOS or firmware codes run. If the code is replaced with malware, the
server cannot execute the code.
The iDRAC is responsible for RoT and verifies BIOS SPI code before
allowing host chipset & CPU to run any code.
RoT Purpose
The silicon-based RoT starts a chain of trust to ensure systems boot with
a legitimate BIOS code. If the performed BIOS code has been verified as
legitimate, those credentials are trusted by the execution of each
subsequent code.

RoT Operation
1. On a server, the silicon chip acts to validate that the BIOS is legitimate
by checking its encrypted signature.
2. This encrypted signature (a Dell encryption key) is burned into silicon
during the manufacturing process and cannot be changed.
The only way to make the Root of Trust robust is to do it in hardware. The
read-only encryption keys are burned into PowerEdge servers at the
factory. These keys cannot be changed or erased. When the server
powers on, the hardware chip verifies that the BIOS code is legitimate
from Dell using the key that is burned into silicon in the factory.
Verifying the RoT
A failure to verify that the BIOS is legitimate results in a shutdown of the

server and the user is notified in the log. The BIOS recovery process can
be initiated by the user. If the RoT is validated successfully, the rest of the
BIOS modules are validated by using a chain of trust procedure until
control is handed off to the operating system or the hypervisor.
BIOS Live Scanning
• The BIOS Live Scanning feature enables users to scan the system
BIOS once POST is completed. This task can be run once or can be
set up on a schedule.
• The scan period could be once a week, once a month or once in a year
(adjustable by end user).
• The BIOS Live Scanning is a licensed feature available only with
iDRAC Datacenter license in 15G systems only.

Image of the iDRAC UI with the BIOS Live Scanning option highlighted.

Intel Boot Guard
The 14G and the 15G PowerEdge servers support the Intel Boot Guard
verified boot feature. Boot Guard protects the server BIOS.
• Basic Input Output System (BIOS) is implicitly a critical element of any
solution stack that includes risks59 while updating.
• The BIOS persists between power cycles, becoming a potentially
attractive target for malicious attacks.
• Attacks against the BIOS are typically hard to detect. Attacks run
before the operating system and other security software loads. This
mechanism leaves a platform or organization exposed to further threat
or performance issues.
1: What is the function of Boot Guard?
59Due to this, some users hesitate to perform scheduled updates during a

server life cycle.

Boot Guard is a processor feature that prevents the system from running
the firmware images that are not released by the manufacturer. It also
allows the BIOS or UEFI to verify that the BIOS is not compromised before
booting.
2: How does Boot Guard operate?
In the Boot Guard verification method, the CPU compares the current
BIOS or UEFI firmware image with an official hash-generated version of
the image that is stored on PowerEdge servers.
3: How to identify Boot Guard occurrence?
If a Boot Guard event is activated, the BIOS or UEFI feature is

immediately activated and an attempt to recover a backup BIOS or UEFI
occurs.
Discussion
What is the function of Boot Guard?
Boot Guard is a processor feature that prevents the system from running
the firmware images that are not released by the manufacturer. It also
allows the BIOS or UEFI to verify that the BIOS is not compromised before
booting.
How does Boot Guard operate?
Discussion
Dell's BootGuard verification method involves comparing the BIOS image

against the official hash that is generated and stored on Dell's servers.
Basic Input Output System (BIOS) is implicitly a critical element of any

solution stack and since the BIOS persists between power cycles it poses

a potentially attractive target for malicious attacks. Attacks against the

BIOS are typically hard to detect because they run before the operating
system and other security software loads.
Due to the critical nature of the BIOS and the perceived risks of updating,
some customers hesitate to perform scheduled updates during a server
lifecycle. This can leave a platform or organization exposed to even further
threat or performance issues. For this reason, we have implemented
multiple new features including Boot Guard.
Protection at the Chipset: The Dell 14th generation of PowerEdge

servers supports the Intel Boot Guard verified boot feature. The Boot
Guard extends the platform root of trust to the Platform Controller Hub
(PCH). The PCH contains One-Time Programmable (OTP) fuses that are
burned by the Dell factory during the manufacturing process with the
selected Boot Guard policy and the hash of the Master Public Key. The
Key Manifest on the BIOS SPI flash is signed by the Dell Master OEM
key, and delegates authority to the Boot Policy Manifest key. Then the
Boot Policy Manifest authorizes the Initial Boot Block (IBB), which is the
first BIOS code module to run at the reset vector. If the IBB fails
authentication, Boot Guard shuts down the system and does not allow it to
boot. Each BIOS module contains a hash value of the next module in the
chain and uses the hash value to validate the next module. The IBB
validates (SEC+PEI) before handing off control to it. The (SEC+PEI) then
validates (PEI+MRC) and (PEI+MRC) further validates the (DXE+BDS)
modules. After that point, the UEFI Secure Boot, if enabled, can extend
the root of trust to the remaining BIOS, third-party UEFI drivers, and
operating system loader.

Boot Guard Process
Boot Guard - Protection at the Chipset
1. The Boot Guard extends the platform RoT to the Platform Controller
Hub (PCH).
2. The PCH contains One-Time Programmable (OTP) fuses that are
burned at the Dell factory during the manufacturing process.
3. The OTP contains the selected Boot Guard policy and the hash of the
master public key.
4. The key manifest on the BIOS SPI flash is signed by the Dell master
Original Equipment Manufacturer (OEM) key and delegates authority
to the boot policy manifest key.
5. Each BIOS module contains a hash value of the next module in the
chain and uses the hash value to validate the next module.
Boot Guard Event in the LCC Log
If the Boot Guard event detects any issue in the BIOS image before
booting, it immediately activates the BIOS or UEFI recovery feature and
attempts to recover a backup BIOS or UEFI.
The Boot Guard event and the subsequent events that perform the BIOS
or UEFI recovery are captured in the Lifecycle Controller log as
highlighted in the image.
BIOS recovery logs in the Lifecycle Controller.

Automated BIOS Recovery
If BIOS is corrupted60, a method to recover the BIOS is needed.
A BIOS/UEFI recovery can be initiated in two ways, either through Boot

Guard or when the BIOS detects corruption.
There are two BIOS ROMs in the system, one that is 32MB (for the normal
full-sized BIOS) and another 16MB recovery ROM.
The iDRAC stores a backup BIOS image for BIOS recovery.
The iDRAC orchestrates the entire end-to-end recovery process.
The BIOS recovery occurs in three steps:

1. The iDRAC extracts the recovery BIOS image from a Dell Update
Package (DUP) saved in the control panel during factory install. The
DUP programs the recovery ROM to boot instead of the traditional
BIOS.
2. Once the system boots the recovery ROM61, a BIOS update job is
launched to flash the primary ROM with the full BIOS image using the
latest BIOS DUP known to the system.
3. After the BIOS update completes, the recovery ROM is disabled, the
system reboots back to the primary ROM.
60 BIOS corruption can either be due to a malicious attack, due to a power

loss during the update process, or due to any other unforeseen event.
61 Read Only Memory

Tip: For more information about automated BIOS recovery,

go to 15G Boot Guard and Automatic BIOS recovery page
in dell.com/support
Secure Boot
As software security breaches are becoming more frequent and

incognitive, system administrators must deploy a wider variety of
defenses, such as Secure Boot.
The UEFI Secure Boot is a technology that secures the boot process by
verifying if the drivers and operating system loaders are signed by the key
that is authorized by the firmware.
1: What is Secure Boot?
Secure Boot is a system BIOS feature that guards against attacks by

preventing the execution of unauthorized code in the preboot environment.
It provides an improved way for the BIOS to authenticate each component
in the system using certificates or policies during the boot process.
2: How does Secure Boot operate?
The BIOS authenticates each module that is run during the boot process
using certificates in the Secure Boot policy. Before the system BIOS loads

a module into memory, Secure Boot checks whether the module has the
authorization to run the system. This is done by launching various code
modules, such as: device firmware, diagnostics, and operating system
loaders.
3: How is Secure Boot enabled?
Secure Boot is disabled by default. It is unavailable unless the Boot mode

is set to UEFI. Only UEFI-based system can be booted when it is enabled.
What is Secure Boot?
Secure Boot is a system BIOS feature that guards against attacks by

preventing the execution of unauthorized code in the preboot environment.
It provides an improved way for the BIOS to authenticate each component
in the system using certificates or policies during the boot process.
How does Secure Boot operate?
The BIOS authenticates each module that is run during the boot process
using certificates in the Secure Boot policy. Before the system BIOS loads
a module into memory, Secure Boot checks whether the module has the
authorization to run the system. This is done by launching various code
modules, such as device firmware, diagnostics, and operating system
loaders.
How is Secure Boot enabled?
Secure Boot is disabled by default. It is unavailable unless the Boot mode

is set to UEFI. Only UEFI-based system can be booted when it is enabled.
How to identify Boot Guard occurrence?
If a Boot Guard event is activated, the BIOS/UEFI feature is immediately

activated and an attempt to recover a backup BIOS/UEFI occurs.

Secure Boot Policy
The Secure Boot policy allows a user to specify the policy or digital
signature that BIOS uses to authenticate. The policy can be classified as:
• Standard: BIOS uses the default set of certificates to validate the
drivers and operating system loaders during the boot process. By
default, the Secure Boot Policy is set to Standard.
• Custom: BIOS uses the specific set of certificates that can be
imported or deleted from the standard certificates to validate the
drivers and operating system loaders during the boot process.
The Secure Boot policies in the latest technology of Dell servers are
described in terms of various modes.
Secure Boot Policy Modes
Secure Erase for Self Encrypting Disks (SEDs)
Secure Erase is the process of permanently erasing all data on Self-

Encrypting Disks (SEDs) and resetting their security attributes.
Secure Erase is used to reset the security attributes when an SED is

inaccessible due to lost or forgotten paraphrase.
The process of Secure Erase can be identified when data on SEDs is

completely erased and reset to the default state.

Important: Instant Secure Erase (ISE) is used to instantly

erase user data. The PowerEdge 14G and 15G servers
offer ISE-capable drives by default.
ISE drives use the same encryption technology as SEDs but
do not allow the encryption key to be secured. The
encryption technology allows the drive to be repurposed and
securely erased using the cryptographic erase function.
System Lockdown mode in iDRAC
When OpenManage Essentials introduced the notion of configuration drift

for both hardware changes and firmware changes, customers began
asking if there was a way to prevent any changes from happening in the
first place. This is how Lockdown Mode started. The Lockdown option is
selected in the iDRAC GUI and used to prevent any changes to firmware
or hardware settings during normal operations.
When the System Lockdown Mode is enabled, only some configuration

changes are allowed.
Image shows the system lockdown mode in the highlighted box.

Important: The Lifecycle Controller cannot be booted using

the F10 key as the key is disabled and therefore no
firmware updates can occur. Users can enter the BIOS on
POST by using the F2 key. But, users cannot make any
changes since read only mode is enabled.
Dell Technologies Secured Component Verification
Dell Technologies Secured Component Verification (SCV) is a supply

chain assurance offering that enables users to verify that the PowerEdge
server received on order, matches the factory manufactured model.
Secured Component Verification process
How does the SCV work?

• To validate components in a certificate containing the unique system
component IDs is generated during factory assembly process.
• This certificate is signed in the Dell factory and is stored in iDRAC9,
later used by the SCV application.
• The SCV application validates the system inventory against the SCV
certificate.

Tip: For more information on Secured Component

Verification (SCV), visit the Dell Secured Component
Verification Reference Guide for Servers page in
dell.com/support

Configuration Validation
Overview
Configuration Validation is a feature in PowerEdge

servers that works to modularize interchangeable
pieces of a server, and allows the user to manage
the varying, leveraged parts into unique
configurations.
Configuration Validation:
• Ensures a system is aware of the hardware in

the server.
• Holds information about Self-Describing I/O (SDIO) of backplanes,
risers and internal cards (PERC, BOSS and so on).
• Provides detailed information on the source and destination of internal
chassis cable connections.
• Utilizes the unique identifier given to each piece of hardware in system,
so users can recognize each piece individually.
The Configuration Validation feature enables the iDRAC platform to:
• Collect the precise configuration of the hardware components.

• Compare the collected configuration with the expected configuration,
which allows the user to identify errors in platform configuration.
Important: Configuration Validation is available only in the

PowerEdge 15G servers.

Hardware Configuration Error Messages
Configuration Validation displays the following error messages when a

component is misconfigured.
• Error Message - HWC8010: Occurs when there are one or two issues
in the configuration.
• Error Message - HWC8011: Occurs when there are multiple issues in
the configuration.
The product specific user manuals provide additional details about these
errors.
Error Messages
Error Message HWC8011.
Error Message HWC8010.

Error Messages Resolved
Message when error HWC8010 is resolved.
Message when error HWC8011 is resolved.

HWC8010 and HW8011 Error Interpretations
The following table highlights the HWC8010 and HW8011 error messages
along with the interpretation of the error.
Error Definition Interpretation Example
Configuration A configured • Configuration Configuration

Error element within Error is Error: Slimline
the closest reported if the SL2 and BP4
match contains customer or There is a
something that the onsite configuration
does not match technician error between
the Dell qualified incorrectly the SL2 and
configuration. configures the BP4.
system; or if
there is an
issue in the
system during
the factory
dispatch.
• The key action
is to verify the
cable routing
and fix the
errors.

Configuration The iDRAC • The cases that Configuration

Missing interface found a are reported Missing: CPU2
configured are due to the There is a
component cables being missing
missing within damaged or configuration in
the closest must be CPU2.
match detected. replaced.
• The key action
is to guide the
customer to
verify the
missing
cables.
• If all cables
are present
and
connected,
then call out
for a
replacement.
Communication A configured • The Communication

Error element is controllers can Error:
present, but communicate Backplane 2
communication over a cable, The controllers
over the but the have an issue to
management communicatio communicate
interface is not n is over backplane
working. unsuccessful 2.
over the
management
interface.

Dell PowerEdge Concepts and Features - Appendix
RAID in System Setup Utility
BOSS CLI Commands
Command Description Example Usage
info -o hba Display BOSS-S1 controller mvsetup info -o hba

information
info -o vd Display virtual drive mvsetup info -o vd

information
info -o pd Display physical drive ./mvcli info -o pd

information

smart -p <PD_ID> Display SMART information ./mvcli smart -p 0

of physical drive
event Display controller events ./mvcli event
BOSS Management on OMSA
BOSS Management on System Setup Utility

BOSS Management on iDRAC
Rear Facing Module
The Online Course Contains an Interaction Here.
[Detailed description of the Interaction for Guides]
PowerEdge 750xa GPU Configurations
The Dell PowerEdge 750xa supports two GPU configurations in the chassis front
end.
• Four GPUs configuration
− In this configuration, each riser has two GPUs.
• Two GPUs configuration

− In this configuration, each riser has one GPU. The empty slots should be
installed with a dummy GPU module.
− For example, if GPUs are installed in slot 31 and slot 33, then dummy GPU
modules must be installed in slot 32 and slot 34. Similarly, if two GPUs are
installed in slot 33 and slot 34, then dummy GPU modules must be installed
in empty slots 31 and slot 32.
RAID in OMSA

RAID in Lifecycle Controller
iDRAC Liquid Leak Error
The iDRAC GUI reporting a leak detection in the server.

Single DIMM Blanks
PowerEdge XR11 Fans and Heatsinks

15G Memory Population Guidelines
Dell recommended guidelines for installing memory module on a 15G server are:
• All DIMMs must be DDR4.
• RDIMMs and LRDIMMs must not be mixed.
• DRAMs with x4 and x8 based memory modules can be mixed.
• In Optimizer Mode, the DRAM controllers operate independently in the 64-bit
mode and provide optimized memory
performance.
• If memory modules with different speeds are installed, they will operate at the
speed of the slowest installed memory module(s) or slower, depending on the
system DIMM configuration.
• Populate all the sockets with white release tabs first and then the black release
tabs.
• Memory modules of different capacities can be mixed, provided other memory
population rules are followed.
• When mixing memory modules with different capacities, populate the sockets
with memory modules with the highest capacity first.
− For example, if the customer wants to mix 8 GB and 16 GB memory
modules, populate 16 GB memory modules in the sockets with white
release tabs and 8 GB memory modules in the sockets with black release
tabs.
should be identical.
• Mixing of more than two memory module capacities in a system is not
supported.
• Unbalanced or odd memory configuration results in a performance loss and
system may not identify the memory modules being installed, so always
populate memory channels identically with equal DIMMs for best performance.
• Supported RDIMM / LRDIMM configurations are 1, 2, 4, 6, 8, 12, or 16 DIMMs
per processor.

Dell recommended guidelines for installing memory module on a 14G server are:
• All DIMMs must be DDR4.
• RDIMMs and LRDIMMs/NVDIMMs and LRDIMMs should not be mixed.
• NVDIMMs and RDIMMs can be mixed.
• 64 GB LRDIMMs should not be mixed with 128 GB LRDIMMs.
• Up to two RDIMMs and LRDIMMs can be populated per channel regardless of
rank count.
• A maximum of two different ranked DIMMs can be populated in a channel
regardless of rank count.
speed of the slowest installed memory module(s) or slower, depending on the
• Populate all the sockets with white release tabs first, followed by the black
release tabs.
with memory modules with the highest capacity first.
modules, populate 16 GB memory modules in the sockets with white
release tabs and 8 GB memory modules in the sockets with black release
tabs.
supported.
• Unbalanced memory configurations will result in a performance loss so always
populate memory channels identically with identical DIMMs for best
performance.

Dell recommended guidelines for installing memory modules on a 13G server are:
• RDIMMs and LRDIMMs must not be mixed.
• Up to three dual-rank or single-rank RDIMMs can be populated per channel.
• Up to three LRDIMMs can be populated per channel regardless of rank count.
speed of the slowest installed memory module(s) or slower, depending on
• Populate memory module sockets only if a processor is installed.
• Populate all the sockets with white release tabs first, followed by the black
release tabs, and then the green release tabs.
with memory modules with highest capacity first.
modules, populate 8 GB memory modules in the sockets with white release
tabs and 4 GB memory modules in the sockets with black release tabs.
supported.
• Populate four memory modules per processor (one DIMM per channel) at a
time to maximize performance.
PowerEdge Thermals - FAQs
Why does my system fan speed change?
In general, fan speed changes due to change in system ambient temperature or

hardware component temperatures. Fan speeds increase in order to maintain

system reliability by ensuring components operate below their maximum

temperature specifications. Most frequently, fan speeds increase due to the
workload running on the CPU. Other hardware components with increased cooling
requirements include GPU and FPGA cards, third-party PCIe adapters, high-end
networking adapters, and high-density storage configurations.
Why are my fans running at very high speed or at full speed?
Fans may be running at high or full speed for various reasons. The workload
running on the server can result in high CPU utilization and thus an increase in
cooling requirement. If the system is idle and fans are still at full speed, then either
a hardware option (such as a high-power card or a third-party PCIe adapter)
presents in the server requires that full fan speed, or there is a failure of sensor
communication, a fan failure, or operation of a server without chassis cover and/or
air regulating shroud. Some systems require blanks for nonpopulated hard drive
slots, DIMM slots and/or CPU. Cooling for certain components may be
compromised if these blanks are missing, resulting in higher fan speeds.
Why cannot I lower my fan speeds?
Thermal algorithms define the minimum system fan speeds based on ambient
temperature, system configuration and system utilization. Allowing the user to
reduce fan speed could put system cooling at risk, potentially causing system
thermal-related failures. The only instance in which the user can reduce system
fan speed is when a third-party PCIe adapter card is part of the configuration for
which a thermal algorithm provides cooling based on limited information from the
card. This response may result in overcooling of the card. In this case, the user
can turn off the fan response that is associated with this card, or define a custom
airflow value for the card (iDRAC Web interface or RACADM). Turning off third-
party Card fan response may reduce the fan speed if other components within the
system are not requesting a higher fan speed than the response requested by the
third-party PCIe adapter card. It is not recommended to turn off this response
unless the user is aware of the cooling requirement of the adapter card.
I hear a fan spinning but my server is not powered ON. Is that expected?
Some server platforms are designed to allow one particular fan-in the system to
power ON when the system is in standby (AUX) state (AC plugged in, but power
button not pressed). This fan may run under some system inlet ambient conditions

to ensure cooling for onboard network devices that may be active in system AUX
state.
How many fans are supported on this server?
Some platform configurations require only a limited subset of fans to be present in

the server. For example, some 2-socket platforms may allow a lower fan count
when configured with a single CPU. If a configuration does not require full fan
population, a fan blank must be installed to prevent air recirculation within the
system. Most configurations generally require full fan population. If the
configuration has fewer fans than required, a fan failure log is generated in iDRAC.
Conversely, if there are more fans present than required, the additional fans may
operate at their lowest speed, with no additional response to system utilization. It
is not recommended to populate more or fewer fans than the required quantity for
optimum cooling operation of the server. Some platforms also support different fan
types (Standard Performance vs. High Performance) and are represented as Type
1 or 2. A label on the individual fan carrier indicates if the fan is standard or high
performance. There is no indication of the installed fan type in the iDRAC user
interfaces. This feature may be added in a future iDRAC release. It is not
recommended to swap standard fans with high-performance fans, unless allowed
by the platform configuration.
What is fan Pulse Width Modulation (PWM)?
Fan speeds are expressed in Revolution Per Minute (RPM) but the input signal
that drives the fan to run at different speed is expressed as PWM (Pulse Width
Modulation). PWM can be any number between 0% and 100%. It should be noted
that a PWM of 0% generally does not mean that a fan is OFF. 0% is typically
defined as the fan’s lowest operational speed. Conversely, at 100% PWM, fans
run at the maximum RPM. The relationship between fan PWM and RPM is linear.
What custom cooling options are available to users?
Various custom thermal settings are available and accessible using iDRAC
interfaces like RACADM, iDRAC UI, and BIOS HII browser. These thermal options
include, Custom Thermal Profiles (Maximum Performance, Maximum
Performance per Watt, Sound Cap); custom fan speed options (minimum fan
speed, fan speed offsets); and reduced Exhaust Temperature settings. In addition,
custom airflow settings can be applied to third-party PCIe adapter cards through
RACADM and iDRAC UI interfaces. The easiest way to access these options is to

connect to the iDRAC Web UI of the server and go to Cooling -> Configure Fans -
> Fan Configuration.
What is the Sound Cap option in Custom Thermal Profiles?
Sound Cap is a new feature of PowerEdge 14G servers. Sound Cap was
developed in response to customer requests and is for specialized environments
in which minimizing acoustical output is a higher criteria than peak raw
performance. Sound Cap limits, or “caps”, CPU power consumption and thus fan
speed, resulting in a lower acoustical ceiling. Its application is unique for
acoustical deployments and may result in reduced system performance.
Why are PCIe adapter cards installed based on a slot priority requirement in
the server?
There are various reasons that slot restrictions exist for certain cards. Some
common ones are:
• Certain slots are limited by PCIe lane width (like x4, x8, x16).
• Mechanically, a card may fit only in certain locations. This can be based on
such as whether the card is single wide vs. double wide, or standard-length or
full-length card.
• Cabling that is connected to the card that requires the card to be in a certain
location for optimum cable routing.
• Cooling limitations in certain slots, such as airflow limitations may cause a
Cooling or Thermal priority.
Where can I find more information about PCIe adapter card cooling on the
server?
The best place to look for this information is within the iDRAC Web UI. From the
iDRAC home screen, select Cooling -> Fan Overview -> Configure Fans. Then
scroll down to see the “PCIe Airflow Settings”. This section displays all the PCIe
adapter slots present in the system and the maximum airflow in LFM (Linear Feet
per Minute) available at each slot (when all fans are at full speed). This section
also indicates if a particular PCIe adapter card is considered a third-party Card,
and if so, what LFM is being provided. The user has the option to customize the
airflow based on the card specifications. This feature is new with PowerEdge 14G
servers and is an industry first.

Why is the top cover of my system hot and is that an indication of potential
cooling problem? OR Why are CPU temperatures high? OR Why is the air
coming out of the server so hot?
The system top cover may get hot in local regions above the CPU heatsinks or
near the back of the system. This occurs most commonly in dense systems and in
1U servers. The localized heating of the top cover is due to the close proximity of
the cover to the CPU heatsink or to the heated exhaust air at the rear of the
system. The surface- and exhaust temperatures should not exceed safety limits of
70°C. Components such as CPUs, GPUs, and general board components are
designed to run at higher temperatures without impact component or system
reliability. Users wanting to review or adjust system temperatures or exhaust air
temperature can use Custom Thermal Settings through various iDRAC interfaces
to increase fan speed (and thus system cooling) by applying any one of the Fan
Speed Offset, Minimum Fan Speed, and/or Custom Exhaust Temperature options.
Does my platform support GPUs?
Many high-power compute GPU devices that are passively cooled require
platform-specific configuration restrictions, and those are allowed only on a limited
number of platforms. Lower power (such as less than 75 W) PCIe adapters are
supported on all platforms. See platform-specific limitations to ensure compliance.
Why are different CPU heat sinks required in some configurations?
Some platforms require different CPU heat sinks based on the installed CPU TDP
or other specific hardware options. For example, shorter heat sinks and a different
air shroud are required in the R740 and R740xd to allow for proper GPU cooling.
See the individual platform details for specific information.

Different RIO Cards
NVMe Support
NVMe Drives PCIe Generation
Intel Gen 4
P5500/P5600
Kioxia Gen 4
CD6/CM6
Samsung Gen 4
1733/1735

cStack Cable on PowerEdge 13G server

Latency
Latency is the response time or the period of time that a component waits
for an answer from another component. Latency is the time it takes for the
storage to respond to a request.
Parity
Parity is small data that is used to connect to larger data blocks and
recover data when a disk failure occurs.
vFlash card
vFlash cards provide a shared storage space between the server system
and its iDRAC.

© Copyright 2020 Dell Inc. Page 228

Dell PowerEdge Concepts and Features - Participant Guide

Uploaded by

Document Informationclick to expand document information

Document Informationclick to expand document information

Copyright:

Available Formats

Dell PowerEdge Concepts and Features - Participant Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dell PowerEdge Concepts and Features - Participant Guide

Uploaded by

Copyright:

Available Formats

DELL POWEREDGE

© Copyright 2023 Dell Inc Page 2