Document #: IBDN-DCDG-0707

www.belden.com
Acronyms

10 GbE 10 Gigabit Ethernet LO laser-optimized

ANSI American National Standards m meter
Institute
MDA main distribution area
CAT 6 Category 6
mi mile
CAT 6A Augmented Category 6
mm millimeter
CO central office
MPO multifiber push-on
CP consolidation point
NIC network interface card
CRAC computer room air conditioner
NOC network operations center
dB decibels
OM optical multimode
dc direct current
OS optical singlemode
EDA equipment distribution area
PCB printed circuit board
EF entrance facility
PP patch panel
EIA Electronic Industries Alliance
RJ registered jack
EMI electromagnetic interference
SAN storage area network
ER equipment room
SFF small form factor
ft foot/feet
SP service provider
FXM FiberExpress Manager
sq. square
GbE Gigabit Ethernet
TE telecommunications enclosure
Gb/s gigabits per second
TIA Telecommunications Industry
HD high density Association
HDA horizontal distribution area TR telecommunications room
IDC insulation displacement contact TSB Telecommunications Systems
Bulletin
in inch
U rack unit (also RU)
ISL inter-switch link
UHD ultra high density
ISO International Organization for
Standardization W/sq. ft watts per square foot
km kilometer WA work area
kw-hr kilowatt-hour ZDA zone distribution area
Table of Contents

1. INTRODUCTION ...................................................................................................................... 1

2. SCOPE...................................................................................................................................... 3

3. SPACES ................................................................................................................................... 4

4. PATHWAYS ............................................................................................................................. 7

5. CABINETS .............................................................................................................................12

6. POWERING AND COOLING ................................................................................................16

7. CABLING SYSTEM DESIGN ...............................................................................................19

8. COPPER INNOVATIONS .....................................................................................................31

9. FIBER INNOVATIONS ..........................................................................................................37

10. CERTIFICATION....................................................................................................................43

REFERENCES ...............................................................................................................................45
Figures

Figure 1: Commercial building spaces (side view) ......................................................................... 6

Figure 2: Data center spaces (top view) .......................................................................................... 6
Figure 3: Raised floor pathway example ......................................................................................... 8
Figure 4: Overhead pathway example ............................................................................................. 9
Figure 5: TIA recommended cabinet layout (top view) .................................................................11
Figure 6: High-density switch cabinets ..........................................................................................12
Figure 7: High-density server cabinet ............................................................................................13
®
Figure 8: Belden High-density Racking System – 2304 port capacity (1152 in each rack) .....14
Figure 9: Belden FiberExpress Manager – 3840 fiber capacity ...................................................15
Figure 10: Belden High-density Open Frame Enclosure with perforated cabinet doors ..............18
Figure 11: Interconnect setup—modular cords link switches to patch panels on
each HDA rack.................................................................................................................20
Figure 12: Data center cabling system example .............................................................................21
Figure 13: Centralized topology using rack-mount patch panels ...................................................23
Figure 14: Cross-connect setup—centralized Any-to-any connectivity between
2304 switch ports and 2304 servers/storage units .......................................................24
Figure 15: Centralized topology using wall-mount patch panels to maximize floor space...........25
Figure 16: Cross-connect setup using wall-mount panels..............................................................26
Figure 17: In-row topology ................................................................................................................27
Figure 18: Distributed topology.........................................................................................................28
Figure 19: Pigtail connections to servers/storage units ..................................................................30
®
Figure 20: Belden 10GX CAT 6A cable with RoundFleX technology ..........................................32
Figure 21: Belden 10GX CAT 6A IDC connecting block with X-Pair technology..........................33
Figure 22: Belden 10GX IDC System (1296 x 1296 ports) ............................................................33
Figure 23: Belden 10GX CAT 6A module with FleXPoint, MatriX IDC,
and X-Bar technologies...................................................................................................34
Figure 24: Belden 10GX CAT 6A patch panel with 48 10GX modules .........................................35
Figure 25: Belden patch cord management tool .............................................................................35
Figure 26: Belden equipment harnesses .........................................................................................36
Figure 27: Belden FiberExpress secure/keyed LC system ............................................................38
Figure 28: Belden FiberExpress pre-connectorized assemblies ...................................................39
®
Figure 29: Belden Optimax field-installable connectors and tool kit ............................................ 40
Figure 30: Belden FiberExpress Manager (FXM) ...........................................................................41
Figure 31: Belden FXM Shelf 1U and connector module ...............................................................42
Figure 32: Belden’s IBDN Structured Cabling System Certification Program...............................44
1. Introduction

1.1 The term data center is used to describe a specialized facility for housing a large
number of servers, storage devices, networking equipment and communications
links. Past (and current) equivalents to a data center include mainframe computer
rooms and telephone company central offices (COs). In all cases, the facility serves
as a centralized operations center for processing, storage, and communications.

1.2 Like a commercial office building, a data center can serve a single organization,
such as a bank or an Internet merchant. Alternatively, the owner of the data center
can lease out space to multiple organizations, enabling them to place (or collocate)
their equipment in a securely managed facility. A variation of this model has the
owner leasing the hardware as well, with the “tenant” responsible for running their
application(s) on the leased server(s).

1.3 Regardless of the type of ownership, all data centers have similar needs, since their
goal is the same—to provide secure and uninterrupted availability to high-
performance computing, storage, networking, and communications resources.

1.4 The ever-increasing density of equipment makes a data center environment more
challenging than a typical telecommunications room. For example, a single
equipment or cabling rack can house over 100 blade servers or serve as a
termination point for more than 3000 optical fibers. A data center can be designed
for hundreds or thousands of such racks.

1.5 As a result of this equipment and cabling density, data center design must place
particular emphasis on such factors as:

• Adequate facility space (present and future)

• Power (operational and backup)
• Cooling (general and rack-specific)
• Cabling pathways
• Equipment racks
• Cabling system (components and design)

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 1
 1.6 To assist design professionals involved with data center infrastructure, the
Telecommunications Industry Association (TIA) has issued a standard called
ANSI/TIA-942-2005: Telecommunications Infrastructure Standard for Data Centers.
This document includes contributions from more than 60 organizations, including
data center owners, consultants, and product manufacturers.

1.7 The investment of time and materials associated with data center deployment is
considerable. New servers, storage devices, network gear, and communications
links are routinely added to maintain or improve services, all without interrupting
existing operations.

1.8 To summarize, everything has to work when moving bits into, around, and out of the
data center—and the cabling infrastructure is where the bits move. While most
network equipment is replaced in 3 to 5 year cycles, cabling is expected to serve
multiple generations of devices over a 10 to 25 year period. A well-designed cabling
infrastructure provides the needed flexibility and scalability when reconfiguring or
expanding data center services.

1.9 Belden’s role as the premier cabling system innovator dates back over a century to
the beginning of electronic communications. Our corporate history spans the
introduction of the telephone, computers, and of course, structured cabling. We
continue to provide solutions to large-scale infrastructure projects around the world
sponsored by governments, financial organizations, telephone companies, research
centers, and other institutions, both private and public.

2007 Edition Data Center Design Guidelines

Page 2 © Belden, Inc. 2007
2. Scope

2.1 The contents of this document, Belden’s Data Center Cabling Guidelines, have been
updated to incorporate best practices guidance from the TIA as well as other
sources of expertise.

2.2 The scope of these guidelines is to aid in the design and implementation of
structured cabling systems in data centers and similar facilities, such as network
equipment or computer rooms. These guidelines are not intended to replace or
supersede any existing of future cabling standards or any existing applicable codes
and regulations.

2.3 We hope you find this document useful in planning your high-density infrastructure.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 3
3. Spaces

3.1 When discussing spaces within a commercial building, the largest allocation is made
for users and their work areas (WAs). In a data center environment, the equivalent
of the WA is the equipment rack, which may be enclosed and referred to as a
cabinet or an enclosure.

NOTE: The terms “rack” and “cabinet” are used interchangeably in this document
to refer to support frames for housing any combination of network
equipment, power units, and cabling connectivity (e.g., patch panels,
modular cords).

3.2 Although equipment cabinets are similar in appearance and dimensions, their
contents determine where they are located within the data center. The types of
network devices commonly associated with data center cabinets include:

• Servers
• Storage devices
• Switches (general network, server cluster, storage network, management)
• Load balancers
• Routers
• Special-purpose appliances (security, acceleration)

3.3 In a commercial building, WAs and floors are used to group and organize individuals
by function or department. Similarly, entrance facilities (EFs), equipment rooms
(ERs), telecommunications rooms (TRs), and telecommunications enclosures (TEs)
provide for the distribution of networking equipment and cabling infrastructure in
commercial buildings (see Figure 1).

3.4 The TIA has adopted a similar hierarchy for the deployment of equipment cabinets
within data centers. Cabinets are grouped together based on the function of their
contents. This makes it easier to manage and expand operations, since space is
allocated and reserved for each type of cabinet.

2007 Edition Data Center Design Guidelines

Page 4 © Belden, Inc. 2007
 3.5 The spaces assigned for equipment in a data center (see Figure 2) can be
compared to the spaces in a commercial building as follows:

• The entrance room in a data center is the equivalent of a commercial building

EF and is used to house equipment associated with access provider
communications circuits.
• The main distribution area (MDA) in a data center is the equivalent of a
commercial building ER.
• The horizontal distribution area (HDA) in a data center is the equivalent of a
commercial building TR or TE.
• The zone distribution area (ZDA) in a data center is the equivalent of a
commercial building consolidation point (CP).
• The equipment distribution area (EDA) in a data center is the equivalent of a
commercial building WA (each EDA is assigned to a single equipment rack or
cabinet just as each WA is assigned to a single individual).

NOTE: The combined space allocated for the MDA, HDA, ZDA, and EDA cabinets
is referred to as the Computer Room.

3.6 Additional data center spaces serving a supporting role include:

• Office space for data center support staff

• Network Operations Center (NOC) to monitor and manage data center devices
and traffic
• Dedicated TR for office space and NOC equipment
• Electrical and mechanical rooms
• Storage room for parts and equipment
• Configuration/staging room to test and prepare devices for deployment
• Shipping/receiving room
• Secondary entrance room for supplemental or backup communications circuits
to access providers

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 5
 Figure 1: Commercial building spaces (side view)

Horizontal Cable
WA
TE th
TR 4 floor
WA
CP

rd
WA Horizontal Cable TR 3 floor

Backbone Cables

nd
WA Horizontal Cable TR 2 floor

st
Backbone Cable 1 floor
EF ER

Figure 2: Data center spaces (top view)

Access Primary Entrance

Providers Facility

Backbone Cable

Support Area TR
Backbone MDA
Cable
Horizontal Cable

Offices, Operations Center,

Support Rooms

Backbone Cable
Backbone Cable

HDA
HDA
Horizontal Cable
Horizontal Cable

Horizontal Cable ZDA

Horizontal Cable or
EDA EDA Zone Area Cord

EDA
Computer Room

2007 Edition Data Center Design Guidelines

Page 6 © Belden, Inc. 2007
 3.7 Architectural considerations when designing data center spaces include the
following:

• Room and area sizing

• Ceiling height
• Lighting
• Floor loading
• Signage
• Door locations and sizing
• Access clearances for personnel and equipment

3.8 Security and availability considerations include:

• Entrance/exit access controls and surveillance

• Resources for fire prevention, detection, suppression, and containment
• Preparedness for damaging climactic events (floods, lightning strikes, seismic
movements)
• Redundant/failover spaces, pathways, cabling, power, network devices, and
communications links

4. Pathways

4.1 Three types of pathways are generally required to service network equipment
cabinets in a data center environment:

• Air circulation pathways

• Power cable pathways
• Network cable pathways

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 7
 4.2 It is best to avoid sharing pathways to minimize interference between these three
services. For example, dense or improperly routed power and network cable
pathways in an access floor can restrict the upward flow of cold air to equipment
cabinets through the perforated floor tiles. As well, the power cable can be a source
of interference to nearby copper-based network cable, with the coupled noise
resulting in corrupted frames and diminished network performance.

4.3 In some cases, network equipment vendors design their cabinet layouts in a manner
that enforces adequate separation between power and network cable. For example,
a server cabinet may be optimized for power cable entry from below and network
cable entry from above. Such an arrangement would require both raised floor and
overhead (ceiling-supported) pathways.

Figure 3: Raised floor pathway example

2007 Edition Data Center Design Guidelines

Page 8 © Belden, Inc. 2007
 Figure 4: Overhead pathway example

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 9
 4.4 If both power and copper-based network cable must be routed underfloor or
overhead, it is recommended to use separate pathways for each, with the distances
between the pathways based on such criteria as:

• Quantity and type of electrical circuits

• Shielded vs. unshielded power and network cables
• Type(s) of pathways used (e.g., metallic conduit vs. wire basket cable trays)
• Electrical and building code requirements

Example

The TIA recommends a minimum separation of 100 mm (4 in) when 31 to 60

shielded single-phase circuits (110/240 volt, 20 amp) are adjacent to network cables.
If the power cables are unshielded and grounded metal trays are not used, the
recommendation is to double the separation to a minimum of 200 mm (8 in).

4.5 The TIA recommends a row-based arrangement of cabinets in a data center, with
the fronts of equipment racks facing each other in one row (cold aisle with perforated
tiles) and the backs facing each other in both adjacent rows (hot aisles with non-
perforated tiles), as shown in Figure 5. In this arrangement, lower-density power
cable pathways are routed through cold aisles to optimize airflow and higher-density
network cable pathways are placed in the hot aisles. Similarly, cold air enters from
the front of the cabinets in the cold aisles and exits from the back of the cabinets in
the hot aisles. Air circulation can be passive or forced (e.g., using fans to pull in cold
air or expel hot air).

2007 Edition Data Center Design Guidelines

Page 10 © Belden, Inc. 2007
 Figure 5: TIA recommended cabinet layout (top view)

4.6 It is desirable to keep pathways separate even in cases where optical fiber network
cable is used and electrical interference is not an issue. For example, the setup and
reconfiguration of equipment cabinets is simplified when all power cables are
underfloor and all network cables are overhead.

4.7 In cases where both copper and optical fiber network cable share a pathway, the
two types of cable should be grouped separately. Whenever possible, optical fiber
cable should rest on top of copper cable to avoid excessive optical loss due to
mechanical stress on the fibers.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 11
5. Cabinets

5.1 The terms cabinet, rack, and enclosure are all used to describe the support frame
for data center equipment and cabling. The current generation of cabinets can be
classified into two types; both specialized for data center environments:

• Equipment cabinets with integrated power strips and airflow/cooling options

customized for high-density/high-power devices
• Ultra-high density cable termination cabinets with capacity equal to the number
of cabling runs to multiple equipment cabinets

NOTE: Cabinets dedicated to switching equipment are typically wider than server
or storage cabinets (e.g., 762 mm [30 in] vs. 610 mm [24 in]) to
accommodate the larger volume of cables and modular cords.

Figure 6: High-density switch cabinets

2007 Edition Data Center Design Guidelines

Page 12 © Belden, Inc. 2007
 Figure 7: High-density server cabinet

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 13
 ®
Figure 8: Belden High-density Racking System – 2304 port capacity (1152 in each rack)

2007 Edition Data Center Design Guidelines

Page 14 © Belden, Inc. 2007
 ® ®
Figure 9: Belden FiberExpress Manager – 3840 fiber capacity

5.2 In addition to the alternating cold and hot aisle configuration described in the
previous section, TIA considerations for cabinets include the following:

• Avoiding network equipment with side-to-side or bottom-top airflow (front-back

airflow is best suited to cold aisle/hot aisle setup)
• Recommended rack height of 2.1 m (7 ft)
• Width and depth suitable for planned contents of each cabinet
• Recommended clearance of 1.2 m (4 ft) in the front and 1 m (3 ft) in the back of
each cabinet
• In a row of racks, 250 mm (10 in) vertical cable managers between adjacent
racks and 150 mm (6 in) vertical cable managers at row ends

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 15
6. Powering and Cooling

6.1 Power consumption and energy costs are significant factors for any data center
owner. High-density components make it possible to group significant computing,
storage, or networking resources into a single cabinet. However, savings from
reduce space requirements are offset by the cost of maintaining acceptable
operating temperatures. Accordingly, watts per square foot (W/sq. ft) is a commonly
used measure to indicate data center operating cost and equipment density. Typical
values vary from 50 W/sq. ft to 300 W/sq. ft, with advice to plan for values as high as
500 W/sq. ft.

6.2 It is possible to configure a single blade server cabinet that requires 30,000 watts of
power to operate. However, the thermal engineering and cost of cooling for such a
setup make it impractical to consider this level of equipment density.

6.3 Estimates suggest that for each kilowatt of power used by cabinet equipment, a
second kilowatt is needed for cooling. Since data centers operate continuously, the
challenge for designers is to find the right balance between space and energy
consumption. If equipment is spread out over many cabinets, the cooling
requirements per cabinet are reduced, but the size of the space to be cooled
increases.

The following example illustrates the yearly cost associated with data center
powering and cooling for a single 10,000 watt (10 kilowatt) cabinet (assuming
powering and cooling costs are equal).

If the cost of electricity is 5 cents per kilowatt-hour (kw-hr) then:

10 kw x 24 hours x 365 days x $0.05 x 2 (powering & cooling) = $8760/year

At 10 cents/kw-hr, the cost is $17,520/year.

At 15 cents/kw-hr, the cost is $26,280/year.

2007 Edition Data Center Design Guidelines

Page 16 © Belden, Inc. 2007
 6.4 An allocation of 1.5 sq. m – 2.3 sq. m (16 sq. ft – 25 sq. ft) per cabinet is common,
with three types of powering available for network equipment:

• Single-phase power (e.g., 120 volt, 15 or 20 amps)

• Three-phase power (e.g., 208 volt, 20 or 30 amps)
• Direct current (dc) power (e.g., -48 volt dc)

6.5 Historically, dc powering has been associated with telephone equipment and COs.
However, some data center equipment vendors promote it as a means of reducing
power consumption, equipment cooling, and rack space for power supplies.

The following are two examples of available power at a cabinet, using a single-phase
or a three-phase circuit:

• Single-phase, 120 volt, 20 amp circuit

o 120 x 20 = 2400 watts
o Using a safety factor of 0.8 (i.e., derating of 20%),
2400 x 0.8 = 1920 watts
• Three-phase, 208 volt, 30 amp circuit

o 208 x 30 x 1.732 (square root of 3) = 10,808 watts

o Using a safety factor of 0.8,
10,808 x 0.8 = 8646 watts

6.6 Multiple vendors offer high-capacity network devices equipped with power supplies
rated at 4000, 6000, or 8000 watts. Alternatively, 40 single unit (1U) rack-mount
servers equipped with 300 watt power supplies gives a rating of 12,000 watts. Such
examples illustrate the need to carefully plan the contents and locations of high
power consumption cabinets in data centers.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 17
 6.7 Data center cooling options range from computer room air conditioners (CRACs) to
chilled water piping inside equipment cabinets. In general, thermostats and fans are
used to monitor and maintain acceptable operating temperatures. Perforated or
slotted cabinet doors and optimized routing of modular cords also serve to maximize
cabinet ventilation.

Figure 10: Belden High-density Open Frame Enclosure with perforated cabinet doors

2007 Edition Data Center Design Guidelines

Page 18 © Belden, Inc. 2007
7. Cabling System Design

7.1 The bulk of the cabling infrastructure in a typical data center serves to connect end
devices (servers, storage arrays) to switches in a single-level or multi-level
hierarchy. This is comparable to most commercial buildings, where horizontal
cabling to user work areas makes up the largest quantity of premises cabling. The
biggest difference between the two environments is density.

Example:

A dense cabling system design for a 9.3 sq. m (100 sq. ft) office may specify six
cabling runs to accommodate all expected network devices for its occupant.
Compare this to a typical EDA equipment cabinet with 24 servers, each equipped
with 2 network interface cards (NICs), requiring 48 cabling runs to serve a 1.5 sq. m
(16 sq. ft) area.

7.2 Simply terminating a large number of cables at a cabinet is not enough. Cables
represent the static side of a run, since they are not typically changed after initial
installation. The challenging part is the modular cords, which are constantly
accessed to make or change port connections (see Figure 11).

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 19
 Figure 11: Interconnect setup—modular cords link switches to patch panels on each HDA rack

2007 Edition Data Center Design Guidelines

Page 20 © Belden, Inc. 2007
 7.3 As stated in the introduction to this document, the useful life of a data center cabling
system is expected to span several generations of network electronics. This is
especially true for devices such as servers and storage units, which are added or
replaced far more frequently than networking equipment such as Ethernet or Fibre
Channel switches and core routers. Therefore, the biggest challenge is to design the
connectivity architecture between HDAs, ZDAs, and EDAs (see Figure 12).

Figure 12: Data center cabling system example

Access Primary
Providers Entrance Facility
• Workstation outlets
• Copper/fiber cable
• Copper/fiber PPs
• Copper/fiber cordage Backbone Cable

• HD IDC blocks
Support Area • Fiber PP
TR
• Coax connectivity
Backbone MDA (for DS-1, DS-3 circuits)
Cable • Racks for SP equipment
Horizontal Cable
(584 mm [23 in])
• Core equipment cabinets
Offices, Operations Center,
(762 mm [30 in])
Support Rooms

• CAT 6/6A HD PP • LO 50-
m cable

(24 – 144 ports)
Backbone Cable Backbone Cable
• CAT 6/6A cordage
• Fiber PP
• Fiber cordage HDA
• Switch equipment HDA
cabinets Horizontal Cable • CAT 6/6A UHD PP
(762 mm [30 in]) Horizontal Cable (up to 288 ports)
• CAT 6/6A cordage
Horizontal Cable (up to 7.6 m [25 ft])
ZDA • Fiber PP
• Fiber cordage
• CAT 6/6A HD PP
• LC fiber cordage
(24 – 144 ports) Horizontal Cable or
EDA (up to 7.6 m [25 ft])
• CAT 6/6A cordage EDA Zone Area Cord
(up to 3 m [10 ft])
• FXM shelf 1U
(12-72 strands) • CAT 6/6A cable
• LC fiber cordage • LO 50-
m cable EDA
(up to 3 m [10 ft])
• Server equipment Computer Room
cabinets
(610 mm [24 in])

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 21
 7.4 TIA guidelines for data center cabling distances are summarized as follows (see
Figure 12).

• Copper or fiber horizontal cable (e.g., from a switch cabinet in an HDA to an

equipment cabinet in an EDA) shall not be longer than 90 m (295 ft). A total of
10 m (33 ft) is allocated for the modular cords used at both ends, resulting in a
port-to-port maximum channel length of 100 m (328 ft).
NOTE: If a ZDA is used with copper cabling between an HDA and an EDA,
the ZDA should be located at least 15 m (49 ft) from the HDA. This is
to minimize interference caused by multiple connection points in
close proximity. The maximum channel length from HDA to EDA
remains 100 m (328 ft).

• Copper backbone cabling channels (e.g., from a core switch in the MDA to a
second-tier switch cabinet in an HDA) shall not exceed 100 m (328 ft). The
HDA should be located at least 15 m (49 ft) from the MDA to minimize
interference caused by multiple connection points in close proximity.

PLEASE NOTE:

Although TIA specifies a minimum of 15 m (49 ft) between copper

® ®
connection points, Belden IBDN Systems can tolerate distances as
short as 5 m (16 ft), avoiding the need for slack cable management.

• Unlike copper cabling, allowable lengths for fiber backbone cabling channels
vary by application (e.g., 2 Gb/s vs 4 Gb/s Fiber Channel, Gigabit Ethernet
[GbE] vs 10 Gigabit Ethernet [10 GbE]). In general, 150 m to 300 m (492 ft to
984 ft) is specified by standards when using laser-optimized (LO) 50-
m
multimode fiber (OM3).
NOTE: Longer runs are possible for specific application by using enhanced
(standard exceeding) LO multimode fiber (e.g., 500 m [1640 ft] for
10 GbE) or singlemode (OS1, OS2) fiber (e.g., 10 km [6.2 mi] for
Fibre Channel). As well, adapter and switch vendors may offer
multiple fiber transceiver options—with varying distance limits—for
their products.

• If HDAs are omitted and cabling runs are from the MDA directly to an EDA, a
maximum channel length of 300 m (984 ft) is allowed for optical fiber cabling
and 100 m (328 ft) is allowed for copper cabling.

2007 Edition Data Center Design Guidelines

Page 22 © Belden, Inc. 2007
 7.5 Three topologies can be used for designing the connectivity architecture in a data
center, as follows:

7.5.1 Option 1: Centralized topology

All switches are grouped together in one area. The cabling infrastructure is used to
connect switch ports to servers/storage units. Uses the most cabling. Most flexible
for management (entire data center is managed as a single network entity). Switch
port use is maximized (any port can connect to any server/storage unit).

Figure 13: Centralized topology using rack-mount patch panels

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 23
 Figure 14: Cross-connect setup—centralized
Any-to-any connectivity between 2304 switch ports and 2304 servers/storage units

2007 Edition Data Center Design Guidelines

Page 24 © Belden, Inc. 2007
 Figure 15: Centralized topology using wall-mount patch panels to maximize floor space

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 25
 Figure 16: Cross-connect setup using wall-mount panels

2007 Edition Data Center Design Guidelines

Page 26 © Belden, Inc. 2007
 7.5.2 Option 2: Partially centralized (in-row) topology

Each row of server/storage cabinets has a set of switches in a cabinet at the end of
the row. Less cabling is used than with centralized topology. Moderate flexibility for
management (each row is managed as a separate network entity). Some switch
ports may remain unused (ports are dedicated to the row).

Figure 17: In-row topology

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 27
 7.5.3 Option 3: Distributed (in-cabinet/top of rack) topology

Each server/storage cabinet is equipped with a switch. Uses the least amount of
cabling. Least flexible for management (each cabinet is managed as a separate
network entity). Many switch ports may remain unused (ports are dedicated to the
cabinet).

Figure 18: Distributed topology

2007 Edition Data Center Design Guidelines

Page 28 © Belden, Inc. 2007
 7.6 A general design recommendation for all cabling systems is to manage active device
connectivity using passive cabling components. For example, a centralized copper
or fiber patch panel can terminate cables connected to cabinets populated with
servers or storage arrays. An adjacent patch panel can do the same for switch
cabinets. Connecting any server or storage array to any switch port in the data
center would consist of manipulating a copper or fiber patch cord. In addition to ease
of use, other advantages to this setup are:

• Isolation of network electronics. After initial cabling to the cabinets, you would
not need to touch the equipment to make or change connections. The central
patch panels can even be in another area or room of the data center.
• Investment protection. Changing cabinet equipment would not require any
changes to the cabling system. The existing cabling would connect to the
port(s) in the new device and be managed as usual from the central patch
panels.

7.7 A second design recommendation is to consider wall-mount patch panels instead of

rack-mount ones in order to maximize available floor space in the data center.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 29
 7.8 A third recommendation is to minimize modular cord and cabling bundles in each
EDA equipment cabinet (or ZDA distribution cabinet) by using pigtails (cords with an
RJ-45 connector at one end and wire pairs at the other end). Since they are cut to
precise lengths from equipment ports to the cabinet-based patch panel(s), pigtails
eliminate the need for slack management, which can be a problem with uniform-
length patch cords. By placing the patch panels at the top of an equipment cabinet
(or bottom, depending on the location of cabling pathways), excess cables and
modular cords within the cabinet are eliminated, improving airflow and maintenance
access (see Figure 19).

NOTE: EDA patch panels are the equivalent of telecommunications outlets in

office work areas.

Figure 19: Pigtail connections to servers/storage units

2007 Edition Data Center Design Guidelines

Page 30 © Belden, Inc. 2007
8. Copper Innovations

8.1 The two types of networks associated with data center implementations are Ethernet
for server networking and Fiber Channel for storage networking. While it is possible
to use copper cabling for Fiber Channel connections, the distance limitations (e.g.,
33 m [108 ft] at 1 Gb/s, 17 m [56 ft] at 2 Gb/s) make it impractical to consider copper
as a suitable medium for storage area networks (SANs).

8.2 In the current data center standard, which was published in 2005, the TIA
recommends Category 6 (CAT 6) cabling runs. Innovations in design and
manufacturing have since improved and the current industry recommendation is to
install Augmented Category 6 (CAT 6A), which is expected to be specified in the
next version of the data center standard.

8.3 Copper is suitable for server networking, where 100 m (328 ft) port-to-port (channel)
runs are possible for the full range of Ethernet data rates, including GbE and
10 GbE.

NOTE: The term 10GBASE-T is also used to describe 10GbE operations over
twisted-pair copper cabling.

8.4 Installed CAT 6 cabling infrastructure can support the operation of 10GBASE-T only
up to 55 m (180 ft) under certain conditions as detailed in TIA TSB-155. To
overcome these limitations, TIA is developing the standard for CAT 6A, to be
published as TIA/EIA-568-B.2-10. CAT 6A will support the operation of 10GBASE-T
over a maximum of 100 m (328 ft) of twisted-pair copper cabling with up to four
connectors, as well as the low-power “short-reach mode” version of 10GBASE-T
over a maximum of 30 m (98 ft) of twisted-pair copper cabling with up to two
connectors.

8.5 The need for 10 GbE is greatest in inter-switch links (ISLs), which are used to make
distributed switch ports function as a single switch. Instead of a one-time purchase
of a very large switch, multiple smaller GbE switches can be acquired as needed
and linked together using 10 GbE. In such cases, CAT 6 or CAT 6A can be used for
GbE links to servers and CAT 6A (or fiber) can be used for 10GbE ISLs.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 31
 8.6 Some of Belden’s innovations for copper cabling infrastructure include:

8.6.1 Second-generation CAT 6A cable—smaller outside diameter (7.49 mm

[0.295 in]), pair-to-pair and pair-to-jacket isolation

® ®
Figure 20: Belden 10GX CAT 6A cable with RoundFleX technology

2007 Edition Data Center Design Guidelines

Page 32 © Belden, Inc. 2007
 8.6.2 CAT 6A insulation displacement contact (IDC) connecting block for
jumper wire-style terminations and management—15 dB better than the limit
specified in standards and nearly transparent to signal transmission (appears
to be a continuous run of cable)

® ®
Figure 21: Belden 10GX CAT 6A IDC connecting block with X-Pair technology

Figure 22: Belden 10GX IDC System (1296 x 1296 ports)

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 33
 8.6.3 CAT 6A FleXPoint printed circuit board (PCB)—replaces traditional
pins in RJ-45 modules for improved performance

® ®
Figure 23: Belden 10GX CAT 6A module with FleXPoint, MatriX IDC, and X-Bar technologies

2007 Edition Data Center Design Guidelines

Page 34 © Belden, Inc. 2007
 8.6.4 CAT 6A ultra high-density (UHD) rack-mount patch panel—48 ports,
1U rack space

® ®
Figure 24: Belden 10GX CAT 6A patch panel with 48 10GX modules

8.6.5 Patch cord management tool—avoids disrupting adjacent cords when

connecting or disconnecting a patch cord from a switch or patch panel

Figure 25: Belden patch cord management tool

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 35
 8.6.6 Equipment cable harness—factory-assembled, dressed, and labeled with
port numbers to fit specific equipment vendor products

®
Figure 26: Belden equipment harnesses

2007 Edition Data Center Design Guidelines

Page 36 © Belden, Inc. 2007
9. Fiber Innovations

9.1 Optical fiber cabling can be used exclusively or in conjunction with copper cabling in
the data center. Its advantages include electromagnetic interference (EMI) immunity,
flexibility, smaller cable and patch cord diameters, and allowable spans exceeding
100 m (328 ft). Its principal disadvantage is the higher cost of provisioning devices
with fiber interfaces instead of copper ones. In many cases, both in commercial and
data center environments, switches are provisioned with fiber ISL (or uplink) ports
and copper end-node ports.

9.2 Multiple types of optical fiber have been introduced over the years and the current
TIA recommendation is to use LO 50-
m multimode fiber for maximum flexibility.
Similarly, many types of fiber connectors are available, with LC small form factor
(SFF) connectors favored by equipment vendors for their small size (compared to
previous generation products such as SC).

9.3 From an installation perspective, two options are generally considered for data
center fiber deployment:

• Factory-made pre-connectorized assemblies for “plug-and-play” connectivity.

Used in cases where rapid connectivity is needed or there is a lack of fiber
installation expertise.
• Field-installable connectors for mounting on individual fibers.

9.4 In some environments (e.g., military, financial), security is an additional concern,

since the fiber infrastructure usually transports aggregate data (e.g., all traffic
between sites or groups of servers). In such cases, secure connectors and
transparent enclosures can be used to minimize the risk of data theft through
physical access to fiber connection points.

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 37
 9.5 Some of Belden’s innovations for fiber cabling infrastructure include:

9.5.1 Secure/keyed LC connectors and transparent modules for high-security

environments. Six different color and keying options for segregating network
segments (e.g., storage, backbone, departmental)

® ®
Figure 27: Belden FiberExpress secure/keyed LC system

2007 Edition Data Center Design Guidelines

Page 38 © Belden, Inc. 2007
 ®
9.5.2 Multifiber push-on (MPO) connectivity and FiberExpress Bar, both.
introduced in 1997. Pre-connectorized fiber assemblies that snap together for
rapid deployment situations (e.g., disaster recovery) or locations lacking
individuals with fiber installation experience

®
Figure 28: Belden FiberExpress pre-connectorized assemblies

24-fiber trunk cable with MPO connectors

12-fiber MPO-LC fan-out assembly

FiberExpress Bar

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 39
 ®
9.5.3 Optimax , introduced in 1996. Universal field-installable fiber connectors and
and tool kits provide an easy-to-learn means of connectorizing all types
of fiber cables

®
Figure 29: Belden Optimax field-installable connectors and tool kit

2007 Edition Data Center Design Guidelines

Page 40 © Belden, Inc. 2007
 9.5.4 High-density fiber cables and enclosures with capacity and manageability for
over 3800 fibers per cabinet

® ®
Figure 30: Belden FiberExpress Manager (FXM)

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 41
 ®
Figure 31: Belden FXM Shelf 1U and connector module

FXM Shelf 1U

Left side
Right side

Connector module

2007 Edition Data Center Design Guidelines

Page 42 © Belden, Inc. 2007
10. Certification

10.1 Unlike network devices, a cabling infrastructure does not arrive at a site fully
assembled. As well, the upper limits of a network may be increased months or years
after the initial installation (e.g., migration from GbE to 10 GbE networking).

10.2 The purpose of certification is to provide customers the assurance that their installed
cabling system performs as expected over time. A combination of manufacturing,
design, and installation expertise is required to deliver this assurance.

®
10.3 Belden’s IBDN Structured Cabling System Certification Program provides
comprehensive training to our business partners and comprehensive
product/application warrantees to our customers. The result is a 25-year extended
product warranty for individual components and a lifetime application assurance for
the installed cabling system as a whole.

10.4 Some of the numbers associated with certification include:

• 30,000+ IBDN Certified sites to date in the Americas

• 6 million+ IBDN Certified cabling runs worldwide to date
• 55,000+ students trained by the IBDN Training Center worldwide to date

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 43
 ®
Figure 32: Belden’s IBDN Structured Cabling System Certification Program

10.5 To summarize, the message to our customers is as follows:

• Your IBDN Certified System will support any current or future standards-defined
application that is designed to operate over the cabling system you have
purchased.
• If your IBDN Certified System is unable to support an approved current or
future application, Belden and its partners will correct the failure at no cost to
you, including materials and labor.

2007 Edition Data Center Design Guidelines

Page 44 © Belden, Inc. 2007
References

AFCOM (Association for Computer Operations Management)

http://www.afcom.com

ANSI/TIA-942-2005: Telecommunications Infrastructure Standard for Data Centers

(April 2005)
http://www.tiaonline.org

Belden IBDN Design Guide, Issue 8

http://www.BeldenIBDN.com

Belden Optical Fiber Design Guide, Issue 4

http://www.BeldenIBDN.com

BICSI-002: Data Center Design and Implementation Best Practices

(draft, under development at the time of this writing)
http://www.bicsi.org

DatacenterDynamics
http://www.datacenterdynamics.com

Ethernet Alliance
http://www.ethernetalliance.org

Fibre Channel Industry Association

http://www.fibrechannel.org

The Uptime Institute

http://www.uptimeinstitute.org

Data Center Design Guidelines 2007 Edition

© Belden, Inc. 2007 Page 45