Data Center Auditing

What you need to know about your DC infrastructure

Volkmar Bend DCDC

TÜV Informationstechnik GmbH
Member of TÜV NORD GROUP
AUDIT
An audit is a systematic and independent examination of records, documents and vouchers of an object or
an organization to ascertain how far the statements and disclosures present a true and fair view of the
object of the audit. The auditor perceives and recognizes the propositions before him / her for examination,
obtains evidence, evaluates the same and formulates an opinion on the basis of his judgement which is
communicated through his audit report.

Any subject matter may be audited. Audits provide third party assurance to various stakeholders that the
subject matter is free from material misstatement. Areas which are commonly audited include: Compliance
audit, internal controls, quality management, project management.

As a result of an audit, stakeholders may effectively evaluate and improve the effectiveness of risk manage-
ment, control, and the governance process over the subject matter.

Source: Wikipedia
TÜVIT TSI INTERNATIONAL – AUDIT AND CERTIFICATION PROJECTS

UK GER

LUX
A UK
ES CH Iran
US China

Saudi UAE Thailand

Arabia
Singapore

South
Africa
CONTENTS

 Reasons for an audit Reasons:

Insecurity about the current status of a data
 Benefits center, i.e. level of resilience, survivability, code
conformance. Or as a basis for a refurbishment
 Nature and scope or expansion project. Or to be used as an
internal or external proof of quality (marketing
 On-site inspection tool).
 More than a checklist
 Result and conclusion
RISK POTENTIALS

 Force Majeure
 Technical faults
 Criminal acts
 Negligence
AVAILABILITY

Data center requirements 24/7

PRINCIPLES OF AVAILABILITY

 Fault tolerance  Prioritization

 Redundancy  Transparency
 Separation  Automatism
 Robustness  Autonomy
 Scalability  Diversification
AVAILABILITY
 There are requirements by authorities, statutory organizations,
supervisory boards, etc., published in form of guidelines, laws, Availability
codes, regulations, …
 Priority are heath and safety issues
 Examples Protection of
– Building codes property

– Fire protection
– etc.
 There are recommendations regarding the protection of property Protection of humans

 Definition of availability??
CONTENTS

 Reasons for an audit Benefits:

Gaining detailed information about the data
 Benefits center’s current status, fault tolerances,
uncovering potential weaknesses to avoid any
 Nature and scope downtime, providing recommendations
regarding enhancements and / or potential
 On-site inspection alternatives.
 More than a checklist
 Result and conclusion
BENEFITS

Industry Customers and markets

Market position
Vendors
Bank
Courts of law

Liabilities Basel II/III

IT-operator
Conditions
Confidence
Board Insurance company
THERE ARE MANY DESIGN GUIDES …
EXCERPTS FROM THE TIA-942

Contradiction
AUDIT CATALOGUE
 Conclusion: The basis for an audit should be defined in a way that inspections will be compre-
hensive and results reproducible.
 Examples from the audit catalogue:
L1 L2 L3 L4
CONTENTS

 Reasons for an audit Nature and scope:

An independent third party analysis, neutral
 Benefits and vendor independent, carried out by trained
 Nature and scope and experiences data center professionals, pre-
ferably with an engineering background, involv-
 On-site inspection ing the client’s staff or representatives.
Covering all relevant fields including site and
 More than a checklist building, electrical and mechanical systems,
security systems, cabling, organization and do-
 Result and conclusion
cumentation, considering recognized national
and international data center standards.
IMPULSES

Important:
Holistic
approach
ENVIRONMENT
Avoidance of
 Flooding areas
 Major traffic arteries
 Explosion hazards
 Airborne contaminants
 Sources of vibration
 Political targets
 Event venues
 Etc.
CONSTRUCTION
 Protection of incoming supply lines
 Arrangement of rooms
 Constructive fire protection
 Constructive water protection
 Protection against intrusion
 Lightning protection
 Spatial separations
 Etc.
FIRE PROTECTION
 Central panel
 Smoke and other detectors
 VESDA systems
 Fire suppression systems
 Fire prevention systems
 Fire dampers
 Etc.
SECURITY SYSTEMS
 Access control system
 Gathering of data
 Coding
 Intrusion protection system
 Detectors
 CCTV
 Security zones
 Security personnel
 Etc.
ENERGY SUPPLY
 TN-S Net
 Redundancies
 Transformers
 UPS
 Generator
 Fuel storage
 Cable pathways
 SPDs
 Etc.
HVAC
 CRAC units
 Chillers
 Cooling towers / heat exchangers
 Piping and valves
 Leakage detection
 Ventilation and air filtering
 BAS
 Etc.
ORGANIZATION
 Maintenance + repairs
 Proper operation
 Responsibilities
 Security inspections
 Coordination between IT + FAC
 Testing
 Training
 Etc.
DOCUMENTATION
 Security concept
 Environmental analysis (min. 1.5 mile radius)
 DR concepts
 Floor plans
 Schematics
 Installation layouts
 Energy- and AC requirements
 List of alarms
 Etc.
2 SITES
 2 Data centers at
 2 Locations
 2 Supply paths and
 Redundant connections
 With different environmental risks
CABLING
 Redundant WAN links
 Separation of power and data cables
 Installation of cables
 Rack built-up
 Rack feeds
CONTENTS

 Reasons for an audit On-site inspection:

An onsite inspection by auditing experts as an
 Benefits essential auditing component after evaluation
of submitted documents to verify the present
 Nature and scope conditions, incl. testing and an assessment of
capacities, constraints and operation proce-
 On-site inspection dures.
 More than a checklist
 Result and conclusion
BASIC REQUIREMENTS
Documents Personnel
 current  Knowledgeable contact persons
 Originator  Access to all buildings, systems, and
 Hard copy and/or digital components
 Documents  Potentially with prior NDA
– Legend
– Scale (with floor plans)
– Date
– Suitable scale
CHECKING OF DOCUMENTS – COMMENTS

 Check on completeness
 Distribution of documents according to disciplines
 Analysis of security concept in combination with enclosed plans and schematics
 Comparison of descriptions to TSI requirements. Non-conformities, undocumented
implementations and misleading explanations will be collected in a comments list.
 Check of documents as preparation for on-site inspection
DOCUMENTATION

Risk analysis Site plan Schematics Name plate information of

Security concept Floor plans  EAC important components,
Fire protection concept Sections  Intrusion detection e.g. transformers, UPS,
DR plan  Fire detection batteries, gen sets, chillers,
Environmental analysis Projections onto floor plan of  Fire suppression cooling towers, etc.
 main supply pathways  Energy supply Energy balance
 security zones  Mechanical supply Cooling capacity balance
 intrusion detectors  Ventilation Acceptance certificates
 EAC components Room list
 CCTV cameras Door / Windows schedule
 misc. sensors, e.g. leakage
TECHNICAL KNOW-HOW
 An interdisciplinary team of technical experts will audit and evaluate the data center, e.g.
from the following fields:

– Electrical engineering
– Mechanical engineering
– Electronic security systems
– Architects
– Physicists
– Information technology
– Cabling specialists
ON-SITE INSPECTION
 Audit  Level 2: 2 auditors 1 day
– Environment  > Level 2: 3-4 auditors 1-3 days
– All IT-rooms
– All support rooms  Discussion of concepts and
– All adjacent rooms implementation with local
– Control room technical staff or planners
– Pathways  Triggering of alarms
– Roof  Photos of special situations
– Raised floor and risers
CONTENTS

 Reasons for an audit Not just checklist:

A customized format but based on well document-
 Benefits ed procedures, taking into account the data center’s
specific characteristics by analyzing and evaluating
 Nature and scope all aspects, using an engineering-based and protect-
ion-objective approach.
 On-site inspection
 More than a checklist
 Result and conclusion
THE AUDIT AND CERTIFICATION SCHEME
Audit catalogue
Design guides

UPTIME TIER BICSI 002 TIA 942 BITKOM EN 50600

THE PROBLEM WITH AN AUDIT

Some things can be

measured precisely,
different to the quality
of a data center …
THE PILLARS OF INFRASTRUCTURAL MEASURES
Smoke detectors
Temperature sensors
EAC
etc.

Detection
Precaution
SPSs
Intrusion protection
UPS Reaction
etc.
Forwarding of alarms
but also Fire suppression
Planning Switching between power sources
certification etc.
METHODOLOGY: COMPLY OR EXPLAIN

Goal: Protection by other objects

Avoidance of major
traffic arteries with an Course of traffic artery
increased risk of the
transport of hazardous
Speed limit
goods (risk analysis) Distance to
major traffic User frequency
artery
Type of construction Type of road

Room arrangement Accident statistics

CONTENTS

 Reasons for an audit Outcome:

The evaluation reports must be comprehensive,
 Benefits concise and pragmatic, making practical re-
 Nature and scope commendations for improvements and suggest-
ions for a realistic implementation, specific to
 On-site inspection the audited facility, with reproducible conclus-
ions.
 More than a checklist
 Result and conclusion
TYPICAL PROBLEMS (EXAMPLES)

 Fire stops
 Fire loads (boxes, waste cans)
 Security deficiencies / differences in quality
 Monitoring of intrusion attempts and protection against intrusion are different things

 Congestion of raised floor

 A property „following“ a standard
does not conform to the standard
TYPICAL PROBLEMS (EXAMPLES)

 Grounding connections
 Human mistakes (design, installation, operation, maintenance)
 Missing sensibility of the personnel
 Reaction to alarms
 Insufficient reserves
 Documentation is not up-to-date
PROJECT X

 ABC powder fire extinguishers

 1 ceiling mounted smoke detector + ASD
 Gas suppression system (3 bottles insufficient)
 EAC tokens in office drawer
 Missing drip trays underneath pipes
PROJECT Y

 Position fire dampers

 Caps on ceiling mounted smoke detectors
 Gas suppression system
 Technical building with wooden stairs
 Missing screw in major flange
 Tilted pedestals in raised floor
AUDIT AS BASIS OF A CERTIFICATION
The advantages of a certification:
 Creating trust with your clients.
 Provision of a proof of quality for monitoring institutions / internal revision /
accountants.
 You are improving and securing the quality of your services.
 Generation of a competitive advantage in the industry.
VERIFYING IS BETTER THAN A VERBAL STATEMENT
A LOOK TO THE FUTURE

What will be the future international standards for Data Centers (and also for audits)?

The ISO/IEC committee has decided on May 18th 2017 to develop documents for sustainable
ICT facilities and infrastructure, such as data centers. These documents will use the EN 50600
as the basis and will also consider other standards and best practices.

Resolutions adopted at the 18 May 2017 JTC 1/SC 39 Plenary in Sunnyvale, Ca.
AT THE CONCLUSION
YOUR POINT OF CONTACT

TÜV Informationstechnik GmbH

Langemarckstrasse 20
45141 Essen, Germany

Volkmar Bend DCDC

M.Arch.(USA) Dipl.-Ing.Arch.(TH)
Data Center Auditor
IT Infrastructure
+49-201-8999-579
[email protected]