Telecommunications Infrastructure

Standard for Data Centers

ANSI/TIA-942

Chris DiMinico
[email protected] IEEE 802.3 HSSG
MC Communications
Data Center Standards
„ ANSI/TIA-942 Telecommunications
Infrastructure Standard for Data Centers
‹ Co-chairs: Chris DiMinico & Jonathan Jew
‹ Published 2005 – available through TIA at
www.tiaonline.org
„ ANSI/NECA/BICSI-002 Data Center Design
and Implementation Best Practices
‹ co-chairs: Jonathan Jew & John Kacperski
‹ best practices – complements TIA-942 –
2007 target
IEEE 802.3 HSSG
Who Developed TIA-942?
„ Developed by the TIA TR-42.1.1 Network
Distribution Nodes subcommittee as Project
No. 3-0092
„ Participants included:
‹ Architecture & Engineering Firms
‹ Consultants
‹ End Users
‹ Manufacturers

IEEE 802.3 HSSG

Purpose of TIA-942
„ Encourage early participation of telecom
designers in data center design process
„ Fill a void by providing standards for planning
of data centers, computer rooms, server
rooms, and similar spaces.
„ The standard encompasses much more than
just telecommunications infrastructure.
„ Close to half of the technical content deals
with facility specifications.

IEEE 802.3 HSSG

Purpose of TIA-942
„ Define a standard telecommunications
infrastructure for data centers
‹ Structured cabling system for data centers using
standardized architecture and media
‹ Accommodates a wide range of applications (LAN,
WAN, SAN, channels, consoles)
‹ Accommodates current and future protocols
(e.g., 10+ GbE )
‹ Replaces unstructured point-to-point cabling that
uses different cabling for different applications

IEEE 802.3 HSSG

Purpose of TIA-942
„ Specifications for data center
telecommunications pathways and spaces
„ Recommendations on media and distance for
applications over structured cabling
„ Establish a standard for data center tiers to
replace several proprietary standards. The
TIA data center tier standard is:
• A tool to evaluate existing data centers
• A tool to communicate design
requirements

IEEE 802.3 HSSG

Unstructured Cabling
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA AB AC AD AE AF AG AH AI AJ AK AL AM AN AO AP AQ AR AS

Heater - A/C Heater - A/C Heater - A/C

1 STORAGE POMONA
FI L E F IL E FIL E
POMONA
DESK CABINET
F IL E

TABLE TABLE A/C UNIT CABI NET CABINET CABINET CABINET

A/C UNIT C D ROM HP
247-7589 SCANNER PRINTER

2 #7 #6
TABLE

AT &T
He ater - A/C

DEFINIT Y

CONS OLE
ROU TE R
UNIT

CI SCO
3
N ETW OR K N ETW OR K
N ETWORK CABINET
4 CABINET
H8
CABINET
E7
IBM
CAB
NETW OR K NETWORK A6
CABINET CABINET
NETWORK

HALON
NETWOR K

NETWOR K

NETWOR K
NETWORK

NE TWORK

NE TWORK

NE TWORK
CAB INE T

CABINET

C ABINET

CAB INE T

C ABINET

C6
CABI NET

CABI NET

CABI NET

N ETW OR K NETWORK N ETW ORK B7 N ETW OR K PANEL

I7
I8

I6

I4

NETWORK IBM
I5

I3

I2

I1

CABINET CABINET CABINET CABINET

5 C ABINET
DESK

H7 G7 F6 IBM N ETWOR K A5
E6
3745-A11 NETW OR K
CABINET
3746-900 CABINET
N ETW ORK C5 B6
N ETW OR K N ETWOR K NETWORK
CABINET CABINET C ABINET NETWORK CABINET
6 H6 G6 F5 C ABINET NETW OR K N ETWOR K A4 F/X
E5 IBM CABINET CABINET
247-7543
NETWORK N ETW ORK C4 B5 N ETW OR K
NETWORK 3745-41D
CABINET C ABINET NETWORK IBM CABINET
CABIN ET
7 H5
G5
F4 C ABINET
E4
3745-17A NETW OR K
CABINET
N ETWOR K
CABINET
A3 HALON
ABORT
NETWORK C3 B4 N ETW OR K HALON
NETWORK N ETW ORK
CABIN ET CABINET RELEASE
TAPE RACK CABIN ET C ABINET NETWORK IBM
TAPE RACK G4
8 H4 F3 C ABINET
E3
SAA
3745-L13 NETW OR K
CABINET
N ETWOR K
CABINET
A2

NETWORK NETWORK N ETW ORK CABIN ET C2 B3

PA C BELL
C ABI NE T
POMONA

CABIN ET C ABINET NETWORK D2 NETWORK

A/C UNIT

CABIN ET

A1
G3 F2 C ABINET NETW OR K
9 H3
E2 CABINET
N ETWOR K
CABINET
#8

SAA
NETWORK LAN C1 B2
NETWORK CABIN ET
CABIN ET CABINET NETWORK
CABIN ET D1
G2 F1 C ABINET
H2 N ETWOR K
10 E1 CABINET
B1
IBM NETWORK NETWORK
CABIN ET CABIN ET
IBM S-390 G1
IBM H1
11 DASD DASD R34

12
247-7538
13
LOCKER

INSTALL A CABLE WHEN YOU NEED IT

3174 3174 3174 3174 3174 3174
14 STORAGE
LIEBERT
STORAGE
CABINET CABINET CABINET CABINET CABINET CABINET

IBM TABLE PDU -C

IBM

(SINGLE-USE, UNORGANIZED CABLING)

TAPE RACK CABINET CABINET
AS -400 AS-400
STORAGE
C ABINE T

15

16

IEEE 802.3 HSSG

Structured Cabling
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA AB AC AD AE AF AG AH AI AJ AK AL AM AN AO AP AQ AR AS

Heater - A/C Heater - A/C Heater - A/C

1 STORAGE POMONA
FIL E FIL E FIL E
POMONA
DESK CABINET
FILE

TABLE TABLE A/C UNIT CABINET CABINET CABINET CABINET

A/C UNIT CD ROM HP
247-7589 SCANN ER PRINTER
2 #7 #6
TABLE

AT&T
DEFINITY
Heater - A/C

CON SOLE
ROUTE R
UNIT

CISCO
3
NETWORK
FIBER MDF COPPER NETWORK IBM 3745s NETWOR K

4 CABINET
& CORE
C ABINET

MDF C ABINET IBM

H8 NETWORK NETWORK A6
E7 CAB
C ABINET
NETWOR K

CABINET
NE TWORK

NE TWORK

NE TWORK
H ALON
NETWORK

NETWORK

NETWORK
NETWORK
CAB INE T
CABI NET

CABIN ET

CAB IN ET

C6
C ABI NET

CABINET

CABINET

C ABINET
NETWORK NETWORK NETWOR K B7 NETWOR K PANEL

IDF
I8

I7

I6

I4

N ETWORK
I5

I3

I2

I1
C ABINET CABINET C ABINET IBM C ABINET
5 F6 CABINET
DESK

H7 G7 IBM NETWORK A5
E6 3745-A11 NETWORK
C ABINET CABINET
3746-900 B6
NETWORK NETWORK NETWOR K C5 NETWOR K
CABIN ET C ABINET N ETWORK C ABINET
C ABINET
F/X
6 IDF H6 G6 F5 CABINET
E5 IBM
NETWORK
C ABINET
NETWORK
CABINET
A4

247-7543
N ETWOR K C4 B5 NETWOR K
N ETWOR K NETWORK 3745-41D
C ABINET C ABINET CABIN ET N ETWORK IBM C ABINET

7 H5
G5 F4 CABINET
E4
3745-17A NETWORK
C ABINET
NETWORK
CABINET
A3 H ALON
ABORT
NETWORK C3 B4 NETWOR K H ALON
N ETWOR K NETWORK
CABIN ET N ETWORK C ABINET RELEASE
T APE RACK TAPE RACK C ABINET CABIN ET IBM
8 H4 G4 F3 CABINET
E3
SAA
3745-L13 NETWORK
C ABINET
NETWORK
CABINET
A2

NETWORK NETWORK CABINET C2 B3

N ETWOR K

PAC B ELL
C A BIN ET
POMONA

CABIN ET CABIN ET N ETWORK D2

A/C UNIT

C ABINET

A1
G3 F2 CABINET NETWORK
9 H3
E2
IDF
C ABINET IDF NETWORK
CABINET
#8

SAA

Mainframe N ETWOR K
C ABINET
NETWORK
CABIN ET
LAN
CABINET N ETWORK
C ABINET
D1
C1 B2

10
H2 G2 F1 CABINET
E1
NETWORK
IDF
CABINET
B1
IDF
IBM N ETWOR K NETWORK

IBM IBM S-390 IDF IDF

C ABINET
H1
IDF IDF
CABIN ET
G1

11 DASD DASD R34

12
247-7538

13

STRUCTURED CABLING SYSTEM (ORGANIZED,

LOC KER
3174 3174 3174 3174 3174 3174
14 LIEBERT CABINET CABINET CABINET CABINET CABINET CABINET

REUSABLE, FLEXIBLE CABLING)

T ABLE STORAGE PDU- C STORAGE
IBM IBM
TAPE RACK CABINET CABINET
AS-400 AS-400
S TOR AGE
C AB IN ET

15

16

IEEE 802.3 HSSG

Design Elements
„ Cabling Design
„ Facility Design
„ Network Design
„ Informative annex’s: Provide best practices
‹ Annex A - Cabling Design Considerations
‹ Annex B- Telecommunications infrastructure
administration
‹ Annex C-Access provider information
‹ Annex D- Coordination of equipment plans with
other engineers
‹ Annex E- Data center space considerations
‹ Annex F- Site selection

IEEE 802.3 HSSG

Design Elements
„ Cabling Design:

‹ Copper and fiber cabling performance

‹ Connectors, cables, distribution hardware
‹ Cabling distances
‹ Space management
„ Facility Design:

‹ Data center sizing

‹ Power distribution methodologies
‹ Pathways and spaces
‹ HVAC, security, operations, and administration.
‹ Flexibility, scalability, reliability and space
management
IEEE 802.3 HSSG
Design Elements
„ Network Design:

‹ Support of legacy systems

‹ Enable rapid deployment of new and emerging
technologies such as 10 GbE and 10+ GbE
copper and fiber applications.

IEEE 802.3 HSSG

 Relationship of Spaces
BUILDING SITE
BUILDING SHELL
GENERAL OFFICE TELECOM ROOMS & OFFICE BUILDING
SPACE EQUIPMENT ROOMS SUPPORT SPACE
for spaces outside data
center

DATA CENTER
SUPPORT ENTRANCE DATA CENTER
STAFF ROOM(S) ELECTRICAL &
OFFICES MECHANICAL ROOMS

OPERATIONS TELECOM ROOM(S)

STORAGE ROOMS
CENTER for data center
& LOADING DOCKS
support spaces

COMPUTER ROOM

IEEE 802.3 HSSG

 Data Center Topology
Entrance Room
Entrance Carriers Active Equip Carriers
Facility interconnect

Offices, Operations
Center, Support Rooms
Secondary Entrance Room

Active Equip Active Equip Carriers

interconnect
MC
MC
interconnect Carriers

Back bone
Cabling

HC
IC IC
HC HC
IC ICHC
Zone
Horizontal Distribution
Cabling
interconnect
interconnect interconnect interconnect
Active Equip Active Equip Active Equip
Storage Area Active Equip
Devices
Data Center
IEEE 802.3 HSSG
 TIA-942 Spaces
„ Entrance Room (ER) - location of interface
with campus and carrier entrance facilities
„ Main Distribution Area (MDA) – location of
main cross-connect (MC)
„ Horizontal Distribution Area (HDA) – location
of horizontal cross-connect (HC)
„ Zone Distribution Area (ZDA) – location of
zone outlet (ZO) or consolidation point (CP)
„ Equipment Distribution Area (EDA) – location
of equipment cabinets and racks

IEEE 802.3 HSSG

Data Center Cabling
Main
MC
MC Distribution
Area

Backbone
Cabling
Horizontal Horizontal
Cabling HC
IC ICHC Distribution
Area

Zone
interconnect Distribution
Area

Equipment
interconnect interconnect Active Equip Distribution
Active Equip Active Equip Area

Horizontal cabling is the cabling from the horizontal cross-connect (in the
main distribution area or horizontal distribution area) to the outlet in the
equipment distribution area or zone distribution area.
IEEE 802.3 HSSG
Horizontal and Backbone Cabling
„ Recognized Cables:
‹ a) 100-ohm twisted-pair cable (ANSI/TIA/EIA-568-
B.2), category 6 recommended (ANSI/TIA/EIA-
568-B.2-1)
‹ b) multimode optical fiber cable, either 62.5/125
micron or 50/125 micron (ANSI/TIA/EIA-568-B.3),
50/125 micron 850 nm laser optimized multimode
fiber is recommended (ANSI/TIA-568-3-1)
‹ c)singlemode optical fiber cable (ANSI/TIA/EIA-
568-B.3)
‹ d)75-ohm (734 and 735 type) coaxial cable
(Telcordia Technologies GR-139-CORE)

IEEE 802.3 HSSG

Horizontal cabling distances
„ The maximum horizontal distance is 90 m
independent of media type.
„ The maximum channel distance including
equipment cords is 100 m.
„ The maximum cabling distance in a data
center not containing a horizontal
distribution area is:
‹ 300 m for an optical fiber channel including
equipment cords.
‹ 100 m for copper cabling including equipment
cords.

IEEE 802.3 HSSG

Backbone Cabling
„ Includes cabling from MDA to ER, HDA
„ Optional cabling between HDAs allowed
„ Maximum backbone cable lengths depend on
applications to be supported
„ Centralized optical fiber cabling supported
with interconnect, splice, or pull-through at
the HDA
„ Star topology with no intermediate cross-
connects
„ Various topologies permit redundancy and
flexibility to support various data center sizes

IEEE 802.3 HSSG

Computer Room & Entrance Room
Requirements
„ Min clear height of 2.6m/8.5 ft
„ Min door size 1m/3ft wide 2.13/7ft high
„ Min dist floor loading 7.2 kPA/150lbf/ft2,
recommended min 12 kPA/250 lbf/ft2
„ 20 degrees C to 25 degrees C
„ 40% to 55% relative humidity (reduces ESD)
„ Any sprinkler systems must be pre-action
system
„ Common bonding network (CBN) –
equipotential ground reference

IEEE 802.3 HSSG

Main Distribution Area
„ Location of Main Cross-Connect (MC), the
central point of distribution for data center
structured cabling system
„ Centrally located to avoid exceeding
maximum distance restrictions (typically for E-
1s, E-3s, T-1s and T-3s)
„ Install separate racks for Fiber, UTP, and
coaxial cable distribution

IEEE 802.3 HSSG

Horizontal Distribution Area
„ Location of Horizontal Cross-Connect (HC),
the distribution point for cabling to equipment
distribution area
„ Distribution LAN, SAN, KVM switches and
console servers located in HDA
„ MDA may also include an HC for nearby
equipment distribution area
„ Number of HDAs depends on the density of
cabling and the size of the data center

IEEE 802.3 HSSG

Horizontal Distribution Area
„ The capacity of the cable tray system and the
size of the cross-connect creates practical
limits on the size of the HC
„ Guideline is maximum of 2,000 4-pair UTP or
coax cable terminations per HDA
„ Arrange patch bays to minimize patch cable
lengths and to simplify cable management
‹ Separate racks for fiber, UTP, and coax
‹ Locate switches and patch panels to
minimize patch cord lengths

IEEE 802.3 HSSG

 Zone Distribution Area
„ Rack, cabinet, or under floor enclosure that houses a
zone outlet (ZO) or consolidation point (CP)
„ ZO - structured cabling termination for floor-standing
equipment that cannot accept patch panels (e.g.
mainframes and large servers).
„ CP - intermediate termination point (e.g. cabling to
areas where floor plan is uncertain or dynamic)
„ No cross-connects within the ZDA
„ No active equipment shall be located in the ZDA
„ Maximum of 144 connections in a ZDA
„ Maximum of one ZDA within a horizontal cable run

IEEE 802.3 HSSG

Equipment Cabinets
„ Front rails of cabinets
must be recessed to
provide adequate room
for patch cables and
wire managers
„ Recommend 1-to-1
ratio of patching to
cable management
„ Arrange switches and
patch panels to
minimize patching
between cabinets &
racks
„ Perforated tiles at front
of cabinets
„ One edge of cabinets
placed at edge of tile
IEEE 802.3 HSSG
 Raised Floor
„ Better appearance than overhead cabling.
„ Allows higher power densities, better control
of cooling, and more flexibility in location of
cooling equipment
„ Most stand-alone computer systems are
designed for cabling from below
„ Coordinate under floor cabling with
mechanical & electrical engineers
„ Recommend wire basket cable trays in hot
aisles for telecom cabling

IEEE 802.3 HSSG

Example of Wire Basket Cable Trays
For Cabling Under Raised Floor
24"
RAISED FLOOR TILE 1.375"
DIVIDER
18"
9" 5"
COPPER
4"
FIBER CABLING
SUPPORT STRUT 1"

12" 18"
36"
5"
COPPER
4"
FIBER CABLING
SUPPORT STRUT 1"

Open Area
13.5"

24"
On Center

IEEE 802.3 HSSG

 Overhead Cable Trays
„ Less expensive than raised floor systems
„ Cable trays can be attached to the top of
racks and cabinets (if they are uniform in
height)
„ Cable trays suspended from the ceiling
provides more flexibility for supporting
cabinets/racks of various heights and for
adding and removing cabinets/racks.
„ Cable trays can be installed with several
layers
„ Coordinate location with lighting, ducts,
overhead conduits, overhead power
distribution IEEE 802.3 HSSG
Overhead Cable Tray Example
3 Layer cable tray system:
„ Bottom layer – signal
„ Middle layer – power
„ Top layer – fiber
„ Signal Reference Grid in
brackets attached to lower
layer of trays
„ Fiber patch cables may be
in fiber duct attached to
threaded rods

IEEE 802.3 HSSG

Infrastructure Administration
„ Informative annex with TIA-606-A standards
compliant labeling scheme for all
components.
„ Labeling scheme extended for use in data
centers
„ Cabinets and racks labeled by location using
tile grid or row/position identifiers
„ All cabinets, racks, patch panels, cables, and
patch cords should be labeled

IEEE 802.3 HSSG

 Labeling Example "X" COORDINATE
AA AB AC AD AE AF AG AH AI AJ AK AL AM AN AO AP AQ AR AS AT AU AV AW AX AY AZ BA BB BC BD

01

02
Cat 6 x 24
03

04

05

06
RIGHT
FRONT
07
CORNER CABINET AQ03
08 CABINET AJ05
09

10

11

12

13

"Y" COORDINATE

Patch Panel for 24 Cat 6 from 1st Panel in Cab AJ05 to 2nd Panel in Cab AQ03
AJ05-A to AQ03-B Ports 01-24
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

A A

IEEE 802.3 HSSG

Site Selection
„ Informative annex with guidelines for
selection of a site for a data center
‹ Architectural
‹ Electrical
‹ Mechanical
‹ Telecommunications
‹ Security
‹ Other

IEEE 802.3 HSSG

Facilities Specifications & Tiers
„ Informative annex with general architectural,
structural, electrical, mechanical, and
telecommunications recommendations requirements
„ Annex includes detailed architectural, security,
electrical, mechanical, and telecommunications
recommendations for each Tier
„ Recommended specifications by tier are a uniform
way to rate aspects of a data center design and are a
starting point for initiating design requirements with
qualified architects and engineers

IEEE 802.3 HSSG

Data Center Tiers
„ Tier 1 – basic data center
‹ no redundancy
„ Tier 2 – redundant components
‹ Single distribution path with redundant
components
„ Tier 3 – concurrently maintainable
‹ Multiple distribution paths with only one
active
„ Tier 4 – fault tolerant
‹ Multiple active distribution paths

IEEE 802.3 HSSG

 Redundant Topologies
Carriers Carriers Carriers Carriers

Secondary
Primary Customer
Customer
Maintenance Hole
Maintenance Hole
(Tier 1 and higher)
(Tier 2 and higher)
Tie
r3

Tier 1

Tier 3
2
Data Center Tier

Primary Entrance Tier 3 Secondary

Room Entrance Room
(Tier 1 and higher) (Tier 3 and higher)

Tie
r4

Tier 1

Tier 4
Offices, Operations
Center, Support r3
Rooms Tie

Secondary Dist
Main Dist Area
Telecom Room Tier 4 Area
(Tier 1 and higher)
(Optional for Tier 4)

Computer
Room
Horiz Dist Area

Optional Horiz Dist Area Horiz Dist Area

Cabling Zone Dist Area

Equip Dist Equip Dist Equip Dist

Area Area Area

IEEE 802.3 HSSG

Data Center Tiers
„ Higher tiers correspond to higher availability, but also
have higher construction costs
„ Data Center can have different tier ratings for
different portions of its infrastructure (architectural,
security, mechanical, electrical, telecommunications)
„ The overall rating for the data center is equal to the
lowest tier rating
„ Capacity of systems may need to be upgraded to
maintain tier rating as data center load increases
„ Human error and operating procedures have a major
impact on availability

IEEE 802.3 HSSG

Conclusion
ƒ TIA-942 is the first standard to specifically addresses data
center infrastructure.
ƒ Primarily a telecom infrastructure standard, but about half
of the content deals with facility requirements.
ƒ Provides a flexible and manageable structured cabling
system using standard media.
ƒ Builds on existing standards, when applicable
ƒ Guidelines on a wide range of subjects useful to someone
designing or managing a data center.
ƒ An official tiering standard for determining the quality of a
center. A way to objectively compare one center with
another.

IEEE 802.3 HSSG