TSM As A Data Protection Solution
TSM As A Data Protection Solution
Mary Lovelace
Gerd Becker
Rosane Langnor
Mikael Lindstrom
Pia Nymann
Felipe Peres
Norbert Pott
Julien Sauvanet
Gokhan Yildirim
ibm.com/redbooks
International Technical Support Organization
August 2014
SG24-8134-00
Note: Before using this information and the product it supports, read the information in “Notices” on
page ix.
This edition applies to Version 7, Release 1, of IBM Tivoli Storage Manager family of products.
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Contents v
6.5.1 Challenges the solution addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.5.2 Solution architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.5.3 Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.5.4 Use scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.5.5 Hardware and software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Appendix C. VSS and Tivoli Storage Manager related product concepts . . . . . . . . . 327
Contents vii
viii IBM Tivoli Storage Manager as a Data Protection Solution
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not grant you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any
manner serve as an endorsement of those websites. The materials at those websites are not part of the
materials for this IBM product and use of those websites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.
Any performance data contained herein was determined in a controlled environment. Therefore, the results
obtained in other operating environments may vary significantly. Some measurements may have been made
on development-level systems and there is no guarantee that these measurements will be the same on
generally available systems. Furthermore, some measurements may have been estimated through
extrapolation. Actual results may vary. Users of this document should verify the applicable data for their
specific environment.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
Active Cloud Engine® HACMP™ Redbooks®
AIX® HyperFactor® Redbooks (logo) ®
AS/400® IBM® RS/6000®
Cognos® Informix® Storwize®
DB2® Lotus® System Storage®
developerWorks® Lotus Notes® System x®
Domino® Notes® System z®
DS8000® OS/390® Tivoli®
FICON® PowerHA® Tivoli Storage Manager FastBack®
FlashCopy® PowerVM® Workload Partitions Manager™
Global Technology Services® ProtecTIER® XIV®
GPFS™ pureScale® z/OS®
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Ultrium, the LTO Logo and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and
other countries.
Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.
Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its
affiliates.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, or service names may be trademarks or service marks of others.
When you hear IBM® Tivoli® Storage Manager, the first thing that you typically think of is data
backup. Tivoli Storage Manager is the premier storage management solution for mixed
platform environments.
Businesses face a tidal wave of information and data that seems to increase daily. The ability
to successfully and efficiently manage information and data has become imperative. The
Tivoli Storage Manager family of products helps businesses successfully gain better control
and efficiently manage the information tidal wave through significant enhancements in
multiple facets of data protection.
Tivoli Storage Manager is a highly scalable and available data protection solution. It takes
data protection scalability to the next level with a relational database, which is based on IBM
DB2® technology. Greater availability is delivered through enhancements such as online,
automated database reorganization.
This IBM Redbooks® publication describes the evolving set of data-protection challenges and
how capabilities in Tivoli Storage Manager can best be used to address those challenges.
This book is more than merely a description of new and changed functions in Tivoli Storage
Manager; it is a guide to use for your overall data protection solution.
Authors
This book was produced by a team of specialists from around the world working at the
International Technical Support Organization, Poughkeepsie Center.
Gerd Becker is a Project Manager for EMPALIS GmbH, an IBM Business Partner in
Germany. He has more than 30 years of IT experience, including over 15 years of experience
with storage management products such as DFSMS and Tivoli Storage Manager. His areas
of expertise include IBM Tivoli Storage Manager implementation projects and education at
customer sites, including mainframe environments (IBM OS/390®, VSE, VM, and Linux for
zSeries). He holds several certifications, including technical and sales, and is an IBM Tivoli
Certified Instructor. He has developed and taught several storage classes for IBM Education
Services in Germany, Switzerland, and Austria, and for qSkills in Nuernberg. He has been
Chairman of the Guide Share Europe (GSE) Storage - User Group for more than six years.
Gerd is an author of these IBM Redbooks publications: IBM Tivoli Storage Manager Version
5.3 Technical Guide, SG24-6638; and Certification Guide Series: IBM Tivoli Storage Manager
V6.1, SG24-7781. Gerd participated in the beta test for Tivoli Storage Manager from Version
5.5 through 6.4, and participated in the EAP Program for Tivoli Storage Manager Version 7.1.
Gerd is a member of the Tivoli Storage Manager Advisory Council.
Pia Nymann is an IBM Certified Senior IT Specialist working in Systems Lab Services in
Denmark, which is part of IBM Systems Technology Group. She has 15 years of experience
in the IT industry and has several years of Tivoli Storage software experience including
designing and implementing Tivoli Storage Manager Solutions. This includes working on
various platforms and applications. She used her skills during that time by supporting and
educating many clients about the Tivoli Storage Manager software. Pia has worked with
various areas of the storage management discipline, including Tivoli Productivity Center,
storage analysis, and service management. This is her first Redbooks residency.
Felipe Peres is an Advisory Product Service Specialist for IBM Technical Support Services in
Brazil. He has four years of experience with Tivoli Storage Manager working as L1 Support.
His main areas of expertise include supporting and managing Tivoli Storage Manager
environments and all components, and Tivoli Data Protection and Fastback components. In
addition to working as remote support with Tivoli Storage Manager L1, his other activities
include local support, implementing Tivoli Storage Manager solutions, and creating and
teaching Tivoli Storage Manager workshops. He has more than 10 Tivoli Storage Manager
Certifications and he is an IBM Certified Solution Advisor for Tivoli Storage Solutions V3, IBM
Certified ADP for IBM Service Management Tivoli Storage Management V4, Tivoli Storage
Productivity Center Deployment Professional, and Fastback Deployment Professional. He
holds a degree in System Analysis, with a focus on data security, from São Paulo State
Technological College (FATEC).
Norbert Pott is an IBM Tivoli Storage Manager Support Specialist in Germany. He works for
the Tivoli Storage Manager back-end support team, providing support to customers
worldwide. He has 32 years of experience with IBM, over 23 years of experience in IT, and
more than 16 years of experience with the Tivoli Storage Manager product, starting with
ADSM Version 2.1.5. His areas of expertise include Tivoli Storage Manager client
development skill and in-depth knowledge of problem determination. He is an author of
several Redbooks publications: IBM Tivoli Storage Manager Version 5.3 Technical Workshop
Presentation Guide, SG24-6774; IBM Tivoli Storage Manager Implementation Guide,
SG24-5416; IBM Tivoli Storage Management Concepts, SG24-4877; IBM Tivoli Storage
Manager Versions 5.4 and 5.5 Technical Guide, SG24-7447; Tivoli Storage Manager V6.1
Technical Guide, SG24-7718; and Certification Study Guide Series: Tivoli Storage Manager
V6.1, SG24-7781.
Gokhan Yildirim is a Principal Specialist of Storage Team. He works for Garanti Bank, one
of the largest banks in Turkey. He has 28 years of IT experience and has worked for Garanti
Bank since 1997. Gokhan started his career as a Programmer for System 38, IBM AS/400®,
i Series RPG, Mainframe PL/1, PC Systems Visual Basic, and C. He has more than ten years
of Tivoli Storage Manager Administration knowledge with an IBM Certified Administrator for
Tivoli Storage Manager certification, and has expertise in other related products like tape
systems, disk systems, and virtual tape libraries (VTLs).
Ella Buslovich
Larry Coyne
Diane Sherman
Erica Wazewski
Debbie Willmschen
International Technical Support Organization
Jason Basler
Dave Cannon
Colin Dawson
Fraser Macintosh
David McClelland
Kathy Mitton
Dominic Mueller-Wicke
Ian Smith
Jim Smith
Daniel Wolfe
Justin Youngblood
Chris Zaremba
IBM Tivoli Storage Manager
Tommy Hueber
SWG Lab Services Managing Consultant
Stephane Criachi
Concat AG
Richard Spurlock
Cobalt Iron
Preface xiii
Now you can become a published author, too!
Here’s an opportunity to spotlight your skills, grow your career, and become a published
author—all at the same time! Join an ITSO residency project and help write a book in your
area of expertise, while honing your experience using leading-edge technologies. Your efforts
will help to increase product acceptance and customer satisfaction, as you expand your
network of technical contacts and relationships. Residencies run from two to six weeks in
length, and you can participate either in person or as a remote resident working from your
home base.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:
Use the online Contact us review Redbooks form found at:
ibm.com/redbooks
Send your comments in an email to:
[email protected]
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
In this chapter, we look at the type of data that is generated today and how Tivoli Storage
Manager is used as a data protection solution to manage that data.
As data grows, storage administrators are challenged to complete backup operations within
established backup windows. More data in the backup system means a longer amount of time
is necessary to recover when something goes wrong, increasing the risks.
One solution to all this data growth has been to buy more storage, as the cost of the storage
itself decreases over time. But the cost of housing, powering, cooling, and managing these
devices is exploding.
And, of course, businesses are always changing. Storage administrators need to adapt to any
number of changes in their environments, from bringing new technologies, applications and
data sources online, to complying with new corporate and government data management
mandates.
Greater efficiency and effectiveness in core data management functions (backup, archiving
and ensuring continuous operations) can have a direct and positive impact on the business.
Expectations and requirements for data protection have changed dramatically in recent years:
Where it was once possible to operate with backup windows that lasted eight hours or
more, those windows have shrunk or even disappeared.
Where backups used to be daily (and the loss of 24 hours worth of data was acceptable)
backups for core applications are now much more frequent, with far less tolerance for data
loss.
The IT infrastructure has become much more complex and distributed, with different types
of operating systems and applications. Where all data once was treated the same, many
users and applications now have unique restore requirements.
Figure 1-1 IBM characterizes big data by its volume, velocity and variety
In this book, we look at the challenges the three V’s impose on the backup/archive
environment. We cover the ways that you can use Tivoli Storage Manager to provide effective
and efficient protection of your data (protecting the right data at the right times), helping you
to not increase the amount of storage you have.
1.1.3 Virtualization
More enterprises are using virtualization as a means to reduce cost and improve efficiency.
Virtualization is taking place in many areas of the IT Infrastructure such as servers, storage,
and networking. The backups also need to follow this trend and administrators need to
change to a smarter way of doing these backups.
In this book, we also cover how Tivoli Storage Manager features can be used to run smarter
backups of your smarter infrastructure
You ask yourself how you can improve your backup process through these ways:
Improved performance
Reduced time of execution
Improved data integrity
Reduced physical space to store backup data
Reduced LAN/WAN bandwidth consumption
A huge contributor to data growth is the repeated duplication of large amounts of data every
time you perform a full backup. In an IBM holistic approach, one option is to avoid data growth
from unnecessary data duplication, by only backing up data that has changed since the last
backup. Another option is to determine what types of data you have and categorize it so that
you can manage it most effectively, by moving less frequently accessed data to lower-cost
tiers of storage, and by automatically moving older data to the correct tier of storage and
deleting data that you no longer need or want. This can shorten your backup cycles and
improve application performance. Finally, you can compress and deduplicate the data that
you put into your data protection and retention systems.
The goal is to protect the organization’s data from failures. Protection is provided across a
wide range of operating systems and applications, running on hardware as different as
notebooks and mainframes. Figure 1-2 on page 7 shows the IBM family of products that helps
you cope with today’s challenges on storage management. These are covered in more details
in the next chapters.
More often, as disk storage technology evolves, other solutions become possible and are
being incorporated to the product. Deduplication and snapshots are examples of Tivoli
Storage Manager features that take advantage of the disk subsystem architecture.
However, tapes cannot be discarded. Tape technology continues to evolve, increasing its
speed and capacity. It is still the storage device suitable for many applications and business
requirements. It is best suited for the backup of large sequential files, such as big databases
and digital images, and to store data that needs to be kept for a long period of time to meet
regulatory requirements.
We discuss several scenarios of how Tivoli Storage Manager uses disk and tape for a better
solution.
Businesses face a tidal wave of information and data that seems to increase daily. The ability
to successfully and efficiently manage information and data has become imperative. The IBM
Tivoli Storage Manager family of products helps businesses successfully gain better control
and efficiently manage the information tidal wave with significant enhancements in multiple
facets of data protection.
From its inception, Tivoli Storage Manager has been a highly scalable and available data
protection solution. Tivoli Storage Manager takes data protection scalability to the next level
with a new relational database, based on IBM DB2 technology. Greater availability is
delivered through enhancements such as online, automated database reorganization. In
addition, the increased scalability and the ability to use the latest in server technology helps
deliver increased performance of backup and recovery processes for business critical
recovery point objective (RPO) and recovery time objective needs
Another important tool is the IBM Tivoli Storage Manager monitoring and reporting feature
that can be installed on IBM AIX®, Linux on IBM System x®, and Microsoft Windows
platforms, but can monitor a Tivoli Storage Manager server running on any platform.
You can view the historical reports to determine if any issues or trends need attention, such
as uncontrolled growth over time. You can also view workspaces that are being monitored to
see the Tivoli Storage Manager server IDs, database size, agent status, client node status,
scheduled events, and so on.
The reporting component, sometimes referred to as Tivoli Common Reporting, reports on the
retrieved historical data. IBM Tivoli Monitoring acts as a monitoring application that provides
workspaces for you to monitor near real-time information.
1.3.2 Summary
Our goal for the reader of this book is that, when you have gone through the material in this
book and studied the solutions we describe, you will be able to build a data protection solution
that meets your unique requirements by using the Tivoli Storage Manager family of products.
As you go through the material in this book, keep in mind the following product highlights,
based on Tivoli Storage Manager 6.3, described in the ESG Lab Validation Report1:
Tivoli Storage Manager progressive incremental backup, combined with client- and
source-side data deduplication technology provided an impressive 95% data reduction
factor (more than 19:1) over just 11 days of backups.
DB2 continues to provide an enterprise-class, scalable back end for Tivoli Storage
Manager. ESG Lab confirmed that Tivoli Storage Manager can scale to more than four
billion objects under management while providing higher performance and more
functionality.
Tivoli Storage Manager offers hot Tivoli Storage Manager server disaster recovery, and
demonstrated the ability to replicate deduplicated data sets from a live Tivoli Storage
Manager server to a hot standby server. Tivoli Storage Manager 6.3 provided seamless
and transparent restores for clients after failover.
Tivoli Storage Manager showed many improvements to ease of use and management.
The Tivoli Integrated Portal provided a common repository for all Tivoli Storage Manager
interfaces, with a “single pane of glass” for management of an enterprise Tivoli Storage
Manager environment. ESG Lab also tested automated client deployment, with no
scripting or manual commands needed.
When you decide to use the Tivoli Storage Manager product, you can rely on 20 years of data
protection technologies that, with current versions, allow you to implement data protection
solutions for your current and future requirements.
1
http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=WH&infotype=SA&appname=SWGE_TI_SG_USEN&htm
lfid=TIL14047USEN&attachment=TIL14047USEN.PDF (IBM Tivoli Storage Manager 6.3: Deduplication, Remote Site
Recovery, and Scalability, November 2012)
Tivoli Storage Manager provides centralized, automated data protection to help reduce the
risks that are associated with data loss. This highly scalable software helps you manage more
data with less infrastructure and simplified administration. You can save money, improve
Disaster Recovery Manager offers various options to configure, control, and automatically
generate a disaster recovery plan file. The plan contains the information, scripts, and
procedures required to automate restoration and help ensure quick and reliable recovery of
data after a disaster. The scripts contain the commands necessary to rebuild the Tivoli
Storage Manager server.
One of the key features of Tivoli Storage Manager Disaster Recovery Manager is the ability to
track media in various states, such as onsite, in transit, or in a vault. The media movement
features of Disaster Recovery Manager assist greatly with the daily tasks of sending disaster
recovery media offsite, and receiving expired media onsite for reuse. With these features, the
system administrator can quickly locate all available copies of data.
Disaster Recovery Manager helps maintain business continuity by taking care of these
functions:
Establishing and helping to automate a thorough server disaster recovery plan, clients can
then subsequently restore their data from the server if required, and can continue their
daily backup procedures
Ensuring that vital site-specific information is available in the same plan
Automating vital recovery steps to return the Tivoli Storage Manager server and backup
environment to normal operation
Managing and identifying offsite media required for recovery
Tracking and reporting destroyed systems in the event of a disaster
Storing client configuration information and assigning client recovery priorities
With Disaster Recovery Manager, you can recover at an alternate site, on a replacement
system with another hardware configuration
The web-based dashboard (Figure 2-3 on page 19) aggregates information about the backup
environment into a unified display. This information includes the systems, virtual machines
and applications being backed up, backup servers, and storage repositories.
With the visibility that Tivoli Storage Manager Operations Center provides, you can more
easily manage backup tasks running across multiple backup servers. You can see how well
those tasks were carried out. The health and capacity of backup systems, and also task
fulfillment, is evident at a glance. This enables you to see whether an overnight backup went
smoothly, or which backups require investigation and possible corrective action. The
web-based GUI can help you more easily accomplish all of this anytime, anywhere (for
example, from a desktop computer, a laptop, or even a browser-enabled mobile device).
Tivoli Storage Manager Operations Center enables administrators to view the backup history
for individual systems, focusing on key error messages. It proactively detects problems and
brings them to the attention of the backup administrator through the GUI. It also provides
essential information needed for resolution.
Tivoli Storage Manager Operations Center enables almost any IT team member to handle
everyday backup administration tasks, presenting information in an easy-to-understand
format. In many cases, non-specialists can investigate and resolve backup issues, which
enables these tasks to be accomplished faster and frees experts to focus on new projects.
The solution includes a built-in service for assigning problems among support teams,
significantly enhancing team collaboration in order to speed, simplify and lower the cost of
backup problem resolution. For more complex issues that still require an expert backup
administrator to solve, Tivoli Storage Manager Operations Center can help drive a quick and
effective resolution process. A pop-up command builder can be used from the web
dashboard. This command builder enables experts to take necessary corrective actions
quickly, then updates the dashboard to reflect the fix status. This helps improve backup,
archive, and restore problem-resolution times, and to reduce complexity.
'!
'
! ,
'
"
+
#$%&
& *
!'
(
&
! )*
Tivoli Storage Manager for Databases includes Data Protection for Oracle, which interfaces
with Oracle Recovery Manager (RMAN), to support Oracle backup and restore utilities. It also
includes Data Protection for Microsoft SQL Server, which enables users to perform online
backups of SQL databases to Tivoli Storage Manager servers.
Note: DB2 and IBM Informix® are not included in Tivoli Storage Manager for Databases
because these IBM database products include Tivoli Storage Manager agents.
You can continue running primary applications on your database servers while backup jobs
are running. You can restore data to and from offline storage using automated tasks, utilities,
and interfaces. This software performs online, consistent, and centralized backups to help you
avoid downtime, protect vital enterprise data and minimize operational costs.
Tivoli Storage Manager for Virtual Environments protects VMware vSphere and vCloud virtual
machines by offloading backup workloads to a centralized server and enabling near-instant
recovery. It enables you to protect data without the need for a traditional backup window. You
can protect the massive amounts of information that virtual machines generate without
impacting the physical resources of the VMware server. Multiple virtual machines are
supported with one Tivoli Storage Manager agent. LAN-free data transfer from the VMware
server’s storage to the backup server is supported. This preserves bandwidth for other users.
Tivoli Storage Manager for Virtual Environments integrates with and extends the role of Tivoli
Storage Manager in providing data protection services to virtualized applications in
production environments. These include backup and recovery, online database and
application protection, disaster recovery, data reduction, and bare machine recovery.
Overhead is eliminated with centralized vStorage backup. The VMware vStorage APIs for
Data Protection technology is supported, which simplifies and optimizes data protection. This
technology conducts operations on the backup and management servers rather than the
virtual machines, greatly reducing system overhead and any disruption that backups might
cause to virtualized applications.
A broad range of virtualized applications are supported and may be combined with
application-aware software such as IBM Tivoli Storage Manager for Databases or IBM Tivoli
Storage Manager for Mail to provide application-consistent snapshot backups.
VMware vStorage APIs for Data Protection are used, including block-level incremental forever
backups based on VMware’s Changed Block Tracking that take a nondisruptive snapshot at
the virtual machine image level. The need for traditional backup windows is eliminated by
continuously capturing data changes at the block level.
Tivoli Storage Manager for Virtual Environments retrieves data from image-level backups. It
provides flexible recovery options for file-level, volume-level or virtual machine (VM)
image-level recovery using a single backup of a virtual machine image.
In a file restore operation, the administrator launches the Tivoli Storage Manager for Virtual
Environments restore on the vStorage backup server, accesses a point-in-time view of the
data in the storage pool using Tivoli Storage Manager and performs a drag-and-drop of the
desired files. The file restore operation can also be initiated from within the virtual guest
environment. This feature is only available for Windows and Linux operating systems.
For a full disk volume restore, Tivoli Storage Manager for Virtual Environments mounts the
point-in-time snapshot for that volume (mounted to the recovery volume) and makes it
available immediately to users and applications. The actual data recovery happens in the
background. All requests to write to and read from the volume are handled first, providing full,
near-normal performance during the recovery process. This feature is only available for
Windows and Linux operating systems.
VM image-level restores provide the recovery of not only the data, but the virtualized
computing environment, from operating systems, applications and patches to upgrades and
custom configurations.
Within these versions, data protection of the following applications can be configured with
Tivoli Storage FlashCopy Manager:
IBM DB2, IBM DB2 with SAP
Oracle, Oracle with SAP
Microsoft Exchange and Microsoft SQL Server
In addition, other applications can be supported on IBM AIX, HP-UX, Linux, Solaris, and
Microsoft Windows platforms with script customizing.
These capabilities are achieved through the use of advanced storage hardware snapshot
technology to help create a high performance, low impact application data protection solution.
Tivoli Storage FlashCopy Manager is easy to install, configure, deploy, and seamlessly
integrates with various storage systems such as: IBM System Storage® DS8000®; IBM XIV®
Storage System products; IBM Storwize V7000 and Storwize V3700; IBM System Storage N
series and NetApp systems; IBM System Storage SAN Volume Controller; and all IBM and
non IBM devices supported by the SAN Volume Controller.
The following list identifies the storage solutions that are natively supported with the Tivoli
Storage FlashCopy Manager software:
IBM XIV Storage System
IBM Storwize V7000
IBM System Storage SAN Volume Controller
IBM System Storage DS8000
IBM System Storage N series
NetApp systems
By using Rocket Software and the Rocket Device Adapter Pack for IBM Tivoli Storage
FlashCopy Manager for UNIX and Linux, new storage solutions can be used with Tivoli
Storage FlashCopy Manager, such as EMC Symmetrix VMAX/DMX (TimeFinder Snapshot
and Clone).
Rocket Device Adapter Pack (RDAP) for IBM Tivoli Storage FlashCopy Manager is a plug-in
for IBM Tivoli Storage FlashCopy Manager. With RDAP for Tivoli Storage FlashCopy
Manager, you can exploit Tivoli Storage FlashCopy Manager advanced snapshot-based data
protection capabilities for application data that resides on EMC Symmetrix storage devices.
RDAP for Tivoli Storage FlashCopy Manager supports TimeFinder/Snap and VP Snap, and
mount support with device access masking ability.
For more information about storage solutions supported for Tivoli Storage FlashCopy
Manager see this website:
http://www.ibm.com/support/docview.wss?uid=swg21455924
In addition to those devices, Tivoli Storage FlashCopy Manager V3.2 and later on Windows
supports any storage system that is Microsoft Volume Shadow Copy Services (VSS) capable
by using the VSS system provider or a VSS hardware provider that strictly adheres to the
Microsoft VSS Provider interface.
Tivoli Storage Manager for Enterprise Resource Planning integrates with database-specific
utilities of IBM DB2 for Linux, UNIX and Windows, Oracle, and SAP BR*Tools.
Tivoli Storage Manager for Space Management contains pre-migration tools that send a copy
of the file to be migrated to the Tivoli Storage Manager server prior to migration, making
storage space available and allowing for scheduled data transfer over the network.
You can predefine the stub file size. This feature eliminates the recall of the entire file for
programs that only browse the first part of a file.
It is more integrated when used with the IBM General Parallel File System (GPFS™) policy
engine. Files managed by Tivoli Storage Manager for Space Management can be moved
from one storage pool (such as fast RAID) to another storage pool (such as slower SATA
disks). The file path remains the same, including the file name, directory and file system
name.
Tivoli Storage Manager HSM for Windows controls disk storage requirements by
automatically migrating rarely used files. It moves low-activity or inactive files to a hierarchy of
lower-cost storage options. You gain an efficient storage capacity management solution that
is based on rules-driven migration policies. Tivoli Storage Manager HSM for Windows has the
following functions:
Enhances user productivity by offering rapid access to archived data. It also helps
maximize available disk space for users.
Minimizes backup times and resource usage by focusing on active files.
Migrates Microsoft Windows files based on file name, dates of creation, last access and
modification. It also transparently recalls files to the original host system as needed.
Takes advantage of an IBM Tivoli Storage Manager server for a hierarchy of storage tiers
and reliable backup and recovery functions.
Provides automatic threshold migration, which helps maintain a certain amount of free
space on protected file systems.
Tivoli Storage Manager for Storage Area Networks works with client computers and servers
to make data transfers over a storage area network (SAN). It helps SAN-connected backup
clients and Tivoli Storage Manager servers optimize their direct network connections to
storage. As a result server CPU utilization is reduced.
Several types of backups are available: full system (installation image), volume group, file
system, file or directory, and raw logical volume.
Tivoli Storage Manager for System Backup and Recovery offers the following functions:
Supports configuration of network ports to communicate through firewalls, provides
pass-thru support for AIX operating system language locales.
Allows for the storage of backup objects into an IBM Tivoli Storage Manager server, Tivoli
Storage Manager for System Backup and Recovery can back up any non-rootvg data.
Integrates network boot and installation features in support of IBM RS/6000® SP systems.
It also provides Offline Mirror Backup options to enable simultaneous user and system
access to active copies of data.
IBM Tivoli Storage Manager for z/OS Media V6.3 and IBM Tivoli Storage Manager Extended
Edition for z/OS Media V6.3 enable IBM Tivoli Storage Manager V6.3 servers, running on AIX
and Linux on IBM System z®, to access various FICON attached tape or DASD resources on
z/OS.
Tivoli Storage Manager provides access to SCSI and Fibre Channel storage; through Tivoli
Storage Manager for z/OS Media, it also provides access to FICON attached storage.
Tivoli Storage Manager for z/OS Media receives data from and sends data to a Tivoli Storage
Manager V6.3 server, performing I/O to tape or DASD using sequential file access on behalf
of the Tivoli Storage Manager V6.3 server.
Tivoli Storage Manager for z/OS Media provides improved sequential-access file
implementation and supports the same storage devices as Tivoli Storage Manager for z/OS
V5.5 server. It also interacts with z/OS Data Facility Storage Management Subsystem
(DFSMS) and the Tape Management System in the same way as Tivoli Storage Manager for
z/OS V5.5 server.
The database of a Tivoli Storage Manager V5 server that is running on a z/OS system can be
migrated to a V6.3 server that runs on AIX or Linux on System z. After the upgrade, z/OS
users can continue to access data stored on tape volumes whose contents are accessed by
using FICON attached storage devices through the new Tivoli Storage Manager for z/OS
Media connector.
Microsoft Exchange data objects can be recovered from virtually any Microsoft Exchange
database, even corrupt databases. It enables recovery of objects that were previously
considered unrecoverable, such as deleted mail messages or address books that were lost
because of synchronization errors.
The downtime that is associated with data recovery is minimized, thus improving service
levels. The software supports Microsoft Exchange 2003, 2007, and 2010. Objects can be
restored directly to an Exchange Server or sent to a user-defined destination using Simple
Mail Transfer Protocol (SMTP).
This product is specifically designed for desktop and laptop computers. It provides continuous
protection for key corporate information also. By backing up your most important files as they
are saved. It does not wait for a scheduled interval to back up files.
Tivoli Storage Manager FastBack for Workstations helps minimize recovery times and
maximize the granularity of recoverable data by providing user-initiated recovery. It helps
avoid an impact on employee productivity following a data loss or system failure.
Tivoli Storage Manager FastBack Center is a convenient bundle of three IBM software
products: Tivoli Storage Manager FastBack, IBM Tivoli Storage Manager FastBack for Bare
Machine Recovery, and IBM Tivoli Storage Manager FastBack for Microsoft Exchange.
It helps organizations of all sizes meet a wide range of data management challenges for
complex, distributed infrastructures. You can deploy the advanced management tools you
need for each of your individual data protection requirements without having to worry about
individual product licenses. Tivoli Storage Manager Suite for Unified Recovery offers these
functions:
Provides extensive data protection for a wide range of systems including virtual machines,
file servers, email, databases, mainframes and even desktops. This bundled solution
allows you to use the right data protection tool for each of your requirements.
Reduces costs and simplifies procurement and deployment with per terabyte capacity
licensing. You can deploy any of 10 solution components, in any location and quantity, with
a simplified license that measures only the amount of data being managed.
Scales to meet the recovery needs of any size organization by managing up to four billion
data objects on a single server. This solution supports more than 50 operating system
versions and hundreds of server and storage devices.
Manages the entire suite of products from a single user console, you can configure,
manage, upgrade, report and monitor all 10 products from a single administration
interface.
NetApp
The Netapp enhancements in Tivoli Storage Manager V6.4 are as follows:
Tivoli Storage Manager V6.4 supports snapshot-assisted progressive incremental backup.
This is usually used with NetApp’s SnapMirror replication to take backups from a remote
SnapMirror site. Tivoli Storage Manager can now use NetApp snapshot facility to take a
progressive incremental backup.
Tivoli Storage Manager V6.4 supports snapshot-assisted progressive incremental backup
operations now for vFilers (virtual controllers).
Now Tivoli Storage Manager V6.4 provides more ready-to-use Cognos reports. Users can
customize existing Cognos reports or rapidly create new reports with a drag-and-drop
function.
Also, starting with this version of Tivoli Storage FlashCopy Manager, you can use Rocket
Device Adapter for IBM Tivoli Storage FlashCopy Manager. This brings you the support of
more storage vendors such as EMC disk bays.
For more information about Rocket Device Adapter software, go to the following address:
http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=8
97/ENUS213-567&appname=USN
2.4 Conclusion
As data and storage management becomes more sophisticated, it also becomes more
complicated. Business needs change daily. Regulations exist about backup, archiving,
disaster recovery, copy version, and retention; satisfying rules, laws, and regulations, is not
easy for companies. The key to properly managing a complex storage environment is to
approach it strategically.
In this book, we discuss many solutions by using a strategic approach that can make storage
management easier, more efficient, and more effective.
Data protection has become a critical priority in business today. As our planet gets smarter,
our computing systems must become smarter too. Systems must become more automated,
adaptive, and robust. Data protection and retention solutions need to efficiently store and
manage growing data volumes while keeping data secure and available. This section
discusses various Tivoli Storage Manager tools and functions available to protect your data.
Data protection can be done at several different layers and places and for many different data
types. This chapter discusses the Tivoli Storage Manager tools, explaining where and at what
layer they are acting to effectively protect your data.
The system state backup consists of data from several VSS writers. When the files that
belong to the System Writer change, an incremental backup is used for these files. Using
incremental backup on the System Writer files reduces the amount of time to back up the
system state. A full backup is used for all other system state data.
By default, a progressive incremental backup is used for the System Writer files in the system
state. If you want to run a full backup of all system state data, you can specify
include.systemstate mc_name in the client options file (dsm.opt), where mc_name is the name
of the management class with copy group mode absolute.
The list of startable system state and system services components are dynamic and can
change depending on service pack and operating system features installed. Tivoli Storage
Manager allows for the dynamic discovery and back up of these components.
Back up Automated System Recovery (ASR) files in preparation for recovering the Windows
disk configuration information and system state in case a catastrophic system or hardware
failure occurs.
Tivoli Storage Manager generates the ASR files in the \adsm.sys\ASR staging directory on the
system drive of your local workstation and stores these files in the ASR file space on the Tivoli
Storage Manager server.
To recover from a machine disaster, use a system restore utility that is appropriate for your
platform or build a new replacement to prepare the recovery using the Tivoli Storage Manager
backup-archive client.
TBMR
The backup-archive client can be used together with the third-party product TBMR for
recovery. Cristie TBMR is powerful, easy-to-use software that integrates with IBM Tivoli
Storage Manager. It provides rapid, automatic machine recovery to an identical state following
failure of physical hardware or corruption of an operating system.
TBMR helps users of Tivoli Storage Manager recover their operating systems directly from a
Tivoli Storage Manager backup. No additional backup is required and users can invoke the
Tivoli Storage Manager tracks all of the backups at a file level. It has no concept of a full
backup with dependent incrementals or differentials. Because of the Tivoli Storage Manager
powerful relational database, it does not require periodic full backups.
This methodology reduces network and storage resource consumption and lowers the overall
cost of storage management. Tivoli Storage Manager’s file level progressive backup
methodology is far superior to other traditional backup methods such as full-plus-incremental
or full-plus-differential, because progressive incremental backups are never redundant.
Image backup
An image backup is a block-by-block copy, single object backup of a volume (typically a UNIX
file system or raw logical volume, or Windows drive) on a Tivoli Storage Manager client. Being
able to restore an entire volume as one object can lead to faster recoveries. Image backup is
available at the time of writing on AIX, HP-UX, Oracle Solaris, Linux, and Windows client
platforms.
With image backup, the Tivoli Storage Manager server does not track individual files in the file
system image. File system images are tracked as individual objects and the management
class policy is applied to the file system image as a whole.
To restore an image backup of a volume, the Tivoli Storage Manager client must be able to
obtain an exclusive lock on the volume being restored.
Journal-based backup
Journal-based backup provides an alternative to traditional progressive incremental backup,
which under certain circumstances can dramatically increase overall backup performance.
As the name already implies, journal-based backups have no effect on archive processing.
The main difference between journal-based backup and progressive incremental backup is
the method in which the list of backup candidate objects is derived.
A backup candidate list specifies objects for a particular file system that are to be backed up,
expired, or updated on the Tivoli Storage Manager server by a backup-archive client.
The progressive incremental backup operation derives the backup candidate list by building
and comparing the list of active previously backed-up objects stored on the Tivoli Storage
Manager server with the list of objects currently residing on the local file system.
The server list is obtained over the network and the local list is obtained by scanning the local
file system. Objects that exist in the local list but do not exist in the server list are added as
backup candidates to the candidate list.
Objects that exist in both lists but differ in some way (such as attributes, policy, and size) are
also added as backup candidates unless only the Tivoli Storage Manager database attributes
differ, in which case they are added as attribute update candidates. Objects that exist in the
server list but not in the local list are added as expiration candidates.
Figure 3-1 on page 38 describes how the journal engine keeps track of changes in the file
system and communicates with the client during backup. You can place the journal databases
for different disks on a single volume or separate volumes.
Journal
Engine
(includes file TSM Server
System monitor Backup
journaled
entries
The Tivoli Storage Manager backup-archive client obtains the backup candidate list by
contacting the journal-based backup daemon. This is a local background process that
manages and maintains a journal database of change activity for each file system being
journaled.
Because the backup-archive client does not carry out the initial metadata conversation, the
backup-archive client does not have to sit idle. The backup-archive client can begin sending
the files to the Tivoli Storage Manager server as soon as the journal-based backup is
initiated. This means faster backup times and less backup-archive client idle time.
Previously, this file list construction and processing could severely impact any Tivoli Storage
Manager backup-archive clients that were memory-bound or CPU-bound.
To help address this problem, you can use subfile backups. When a client’s file has been
previously backed up, any subsequent backups are typically made of the portion of the client's
file that has changed (a subfile), rather than the entire file. A base file is represented by a
backup of the entire file and is the file on which subfiles are dependent. If the changes to a file
are extensive, a user can request a backup on the entire file. A new base file is established on
which subsequent subfile backups are dependent.
The adaptive subfile backup function is still available but is being superseded by the client
side deduplication function to reduce the data amount during the backup and archive
processing. For more information about client side deduplication, see “Client-side data
deduplication” on page 40.
Compression
You have the option to specify whether each client should compress its files or other objects
before sending them to the Tivoli Storage Manager server. Compression is available for both
backup and archive operations. Enabling client compression will decrease the network traffic
between client and server (because it sends a smaller quantity of data) at the expense of
requiring client CPU resources to perform the operation.
Therefore, the decision to enable client compression must be made individually for each
configuration. If using client compression, then the client will also automatically decompress
any objects that are sent back to it from the server when the reverse restore or retrieve
operation is requested. Objects that are compressed also ultimately take up less storage
space in the Tivoli Storage Manager server storage pools, reducing resource requirements.
If you do not enable client compression, the files will be sent at their full size to the Tivoli
Storage Manager server. Most sequential storage devices, like tape drives, can perform
hardware compression. If this is the case, you still can get the benefit of reduced space
required in the storage pool. If a client has already-compressed the files, then a
compression-enabled tape drive normally will not be able to compress them further.
Compression rates vary considerably depending on the type of data presented.
For further data reduction, you can enable client-side data deduplication and compression
together. Each extent is compressed before it is sent to the server. Compression saves
space, but it increases the processing time on the client workstation.
Include-exclude lists
You can create an include-exclude list to exclude a specific file or groups of files from backup
services, and to assign specific management classes to files. Tivoli Storage Manager backs
up any file that is not explicitly excluded.
The management class concept gives Tivoli Storage Manager granular control over how and
where each file is backed up. The include-exclude list is a set of references local to each
client that controls which files are backed up and what management class is used. If you do
not select an explicit management class, Tivoli Storage Manager uses a designated default
management class.
The include-exclude list is a very powerful tool for specifying exactly what a client should back
up, if any additional processing should be applied and where the data is stored.
For applications that use the Tivoli Storage Manager API, the deduplication cache should not
be used because of the potential for backup failures that are caused by the cache being out of
sync with the Tivoli Storage Manager server. If multiple, concurrent Tivoli Storage Manager
client sessions are configured (such as with a Tivoli Storage Manager for VMware vStorage
backup server), there must be a separate cache that is configured for each session.
Data encryption
To improve the security of stored data, the backup-archive client implements an optional
encryption function, which allows for encrypting data before it is sent to the Tivoli Storage
Manager server. This helps secure backed up-data during transmission, and it means that the
data stored on the Tivoli Storage Manager server is encrypted and thus is unreadable by any
malicious intruders.
For the strongest possible encryption, you can specify to use 128-bit Advanced Encryption
Standard (AES) data encryption, with the encryptiontype option. The user can choose which
files are subject to encryption through include and exclude processing. The encryption uses a
simple key management system, which means that the user either must remember the
encryption key password during restore or store it locally on the client system.
The encryption processing is the last task on the client system before the data is sent to the
server. Other client operations such as compression happen before encryption is done.
Encryption works for backup as well as for archive. Tivoli Storage Manager will not attempt to
deduplicate client-encrypted data.
LAN-free data movement makes LAN bandwidth available for other uses and decreases the
load on the Tivoli Storage Manager server, allowing it to support a greater number of
concurrent client connections.
The key component of Tivoli Storage Manager for Storage Area Networks is the storage
agent. You install the storage agent on a client system that shares storage resources with the
Tivoli Storage Manager server.
The storage agent can support several clients while installed on only one of the clients. You
can point a Tivoli Storage Manager client to an off-host storage agent. To do this, you do not
need to install the storage agent except on the machine that is writing to the SAN.
Figure 3-2 shows SAN data movement with the lanfreecommmethod option.
The storage agent communicates with the server to obtain and store database information
and to coordinate device and volume access. The server and client coordinate and transfer
data access through the SAN. The client uses the storage agent for operations where
appropriate. For example, if a SAN path is defined, the client (by means of the storage agent)
transfers data on that path. If a failure occurs on the SAN path, failover occurs, and the client
uses its LAN connection to the Tivoli Storage Manager server and moves the client data over
the LAN.
The storage agent can send the data directly to the server using the LAN control paths
between the storage agent and the server. An example is a LAN-free storage pool that is
updated to read-only after the client connects to the server and obtains its initial policy
information. The storage agent, instead of failing the operation, sends the data to the server. If
the storage hierarchy is configured so that the next storage pool destination is available, the
server performs the operation.
Tivoli Storage Manager supports SAN-attached device sharing in the following environments:
Tivoli Storage Manager native library management support consists of an Automated
Cartridge System Library Software (ACSLS), SCSI, or 349x library manager and library
clients or just a library manager.
Shared disk storage using a FILE library and the integration of IBM General Parallel File
System. IBM General Parallel File System is the preferred option for operating systems on
which it is supported.
VSS is supported with Data Protection for Exchange and Data Protection for SQL that can be
used in conjunction with Tivoli Storage FlashCopy Manager to allow for offload VSS backup
operations.
A VSS backup uses Microsoft Volume Shadow Copy Service technology to produce an online
snapshot (point-in-time consistent copies) of the data. It allows a snapshot of large amounts
of data at once. During a VSS backup, the application server is not in backup mode for an
extended period of time, because the length of time to perform the snapshot is usually
measured in seconds and not hours.
The VSS provider implementation determines which type of shadow copy (clone,
copy-on-write, or redirect-on-write) is created.
For more information about VSS and Tivoli Storage Manager related product concepts see
Appendix B, “Hierarchical storage management (HSM)” on page 323.
You need to run the wizard only on one node in the cluster; the wizard creates the necessary
services on all nodes in the cluster. To ensure that any node can perform backups if any other
node fails, the wizard copies the registry data to all nodes in the cluster.
The wizard can configure only one cluster group at a time. If you have multiple cluster groups
to protect, run the wizard as many times as needed to configure the client to back up each
group.
Tivoli Storage Manager is able to support the most virtual architectures on the market today.
We can protect the data in the virtual environment at different levels depending on the virtual
layer used. We divide the protection into three levels:
Off-host backup of virtual machines
Tivoli Storage Manager for Virtual Environments provides off-host backup capabilities for
VMware environments using the VMware vStorage APIs for Data Protection (VADP). A
single backup can be used for both complete virtual machine (VM) recoveries, and
file-level recovery. The capabilities of Tivoli Storage Manager for Virtual Environments are
further enhanced to provide additional scalability. This includes new capabilities to allow
for an incremental-forever backup model, and the ability to use multiple sessions to back
up virtual machines in parallel.
The Tivoli Storage Manager client archive command archives a single file, selected files, or
all files in a directory and its subdirectories on a server. Archive files that you want to preserve
in their current condition. To release storage space on your workstation, delete files as you
archive them by using the deletefiles option. Retrieve the archived files to your workstation
when you need them again.
Archiving is a special kind of data protection, sometimes assigned the terms of long time
retention and compliant archiving. But in the context of Tivoli Storage Manager, the archive
function is sometimes also used to protect a special kind of data such as database log files or
SAP application data in a specific way. Archive can also be treated as data protection of
important data that must be stored in a secure way and for a long time.
Retention
Backup files are stored and retained based on versioning criteria; archived objects are stored
and retained on a number of days basis.
There is a possibility to extend the retention period by using a mechanism called deletion
hold. If a hold is placed on an object through the client API, the object is not deleted until the
hold is released. See the Backup-Archive Clients Installation and User's Guide for your
operating system platform:
http://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.0/com.ibm.itsm.client.doc/r_
pdf_clients.html?lang=en
There is no limit to how often you alternate holding and releasing an object. An object can
have only one hold on it at a time, so if you attempt to hold an object that is already held, you
will get an error message.
If an object is held, it is not deleted whether or not data retention protection is active. If an
object is not held, it is handled according to existing processing such as normal expiration,
data retention protection, or event-based retention. Data that is in deletion hold status can be
exported. The hold status will be preserved when the data is imported to another system.
See 2.3, “Tivoli Storage Manager recent enhancements” on page 29 for a brief introduction to
the features in Tivoli Storage Manager introduced in the Tivoli Storage Manager V6.4 and
Tivoli Storage Manager V7.1 releases.
This section describes available tools to protect data on NAS and Storwize devices. Also
included is an explanation on how the Tivoli Storage Manager client protects files stored
within GPFS by using the FlashCopy mechanism at the hardware level.
Tivoli Storage Manager Extended Edition offers the ability to do full/differential file system
image backups and restore of servers that support the NDMP protocol. You can back up
directly to the Tivoli Storage Manager hierarchy and also implement Disaster Recovery
Manager, as it now supports NAS storage. Multiple backup and restore operations can be
performed in parallel.
The NetApp filer has an embedded operating system. Tivoli Storage Manager backup is done
either by allowing the backup-archive client to access files over the network (NFS/CIFS) or
through NDMP. This causes major performance problems in the case of big filers with large
file systems, as the backup cannot be completed within the backup window.
NetApp provides an API (Snapshot Difference API) that takes two snapshots, the base
snapshot and the difference snapshot, as input and returns a list of all files that have been
added, deleted, modified, and renamed between the two snapshots. The API handles files,
directories, ACLs, streams, hard links and symbolic links, and supports both traditional and
FlexVol volumes.
SnapMirror to Tape
Tivoli Storage Manager V6 leverages NetApp SnapMirror to Tape technology, enabling data
centers to use Tivoli Storage Manager Network Data Management Protocol (NDMP) support
to back up FAS and IBM N series volumes directly to tape or across the network to any
storage device managed by Tivoli Storage Manager. This can be done without having to scan
each volume to find new and changed files, helping reduce the amount of time customers
need to back up their FAS and IBM N series systems, which can make the backup process
much faster.
You use the NDMP SnapMirror to Tape feature as a disaster recovery option for copying very
large NetApp file systems to secondary storage. With a parameter option on the BACKUP and
RESTORE NODE commands, you can back up and restore file systems using SnapMirror to
Tape. Consider the following guidelines before you use it as a backup method:
You cannot initiate a SnapMirror to Tape backup or restore operation from the Tivoli
Storage Manager Web client, command-line client, or the Operation Center.
You cannot perform differential backups of SnapMirror images.
You cannot perform a directory-level backup using SnapMirror to Tape, because Tivoli
Storage Manager does not permit a SMTape backup operation on a server virtual file
space.
You cannot perform an NDMP file-level restore operation from SnapMirror to Tape images.
Therefore, a table of contents is never created during SnapMirror to Tape image backups.
In a conventional backup method, the backup software can back up data from the shares
exported from Storwize V7000 Unified system file modules mounted on the NFS or CIFS
clients. Backup software is installed on each NFS or CIFS client and data is backed up over
the network to the backup server.
The integrated Tivoli Storage Manager solution with Storwize V7000 Unified system file
modules provides the following advantages:
Tivoli Storage Manager client is pre installed on the file modules of the Storwize V7000
Unified system. Data is directly backed up from file modules to the backup server.
Storwize V7000 Unified system file modules have an integrated IBM General Parallel File
System (IBM GPFS) policy scan engine to scan the inode table of a file system, which is
much faster than the conventional system calls, reducing the overall time required for the
backup.
Storwize V7000 Unified system leverages both its file modules for backing up the data to
improve performance and to distribute load across both the file modules.
Storwize V7000 Unified system file modules can establish several sessions with a Tivoli
Storage Manager server.
An existing Tivoli Storage Manager server environment can be leveraged to back up
Storwize V7000 Unified system.
Multiple Tivoli Storage Manager servers can be configured to back up different GPFS file
systems to spread backup workload across multiple backup servers.
Note: The use of Tivoli Storage Manager backup-archive client and Space Management
client in combination with GPFS is supported only on the AIX and Linux platforms.
Note: A suggestion is to use the same Tivoli Storage Manager Server for the
backup-archive client and the Space Management client. In this way, you can take
advantage of the integration between the clients for inline copy and stub restore.
The two main types of storage snapshots are copy-on-write snapshot and split-mirror
snapshot. Utilities are available that can automatically generate either type. For more
information, see the following website:
http://www.ibm.com/developerworks/tivoli/library/t-snaptsm1/index.html
The FlashCopy is crash-consistent on the operating system level. For application consistency
the Tivoli Storage FlashCopy Manager has a built-in capability to support several commonly
used applications. On the Microsoft platform, we the Volume Shadow Copy Service to make
consistent application snapshots. See 3.1.4, “Microsoft Volume Shadow Copy Service” on
page 42 for more information.
Figure 3-3 Integrate applications and IBM storage: Tivoli Storage FlashCopy Manager
With Tivoli Storage FlashCopy Manager, we can perform near-instant application aware
snapshot backups, with minimal performance impact for IBM DB2, Oracle, SAP, Microsoft
SQL Server and Exchange, and VMware virtual machines. On the storage system side, it
integrates with IBM Storwize family, IBM System Storage DS8000, IBM System Storage SAN
Volume Controller, XIV Storage System, IBM N series, and NetApp. We support FlashCopy
on AIX, Solaris, HP-UX, Linux, and Microsoft Windows. More information about platform
requirements is at the following address:
http://www.ibm.com/support/docview.wss?uid=swg21427692
Tivoli Storage FlashCopy Manager can now be used with other storage vendors such as EMC
by leveraging the Rocket Device Adapter for IBM Tivoli Storage FlashCopy Manager. The
Rocket Device Adapter is a device agent interface. Rocket Device Adapters for Tivoli Storage
FlashCopy Manager extend backup and restore operations seamlessly to the native
operations of EMC systems
Tivoli Storage FlashCopy Manager can be configured either as a stand-alone solution where
FlashCopies are stored only in the storage system or as a Tivoli Storage Manager integrated
solution where the FlashCopy is transferred to the Tivoli Storage Manager server storage
hierarchy.
Another implementation is to integrate Tivoli Storage FlashCopy Manager with Metro Mirror
and Global Mirror functionality. This is available on AIX, HP-UX, Linux, and Solaris with IBM
System Storage SAN Volume Controller, IBM System Storage Storwize V7000, and IBM XIV
Storage Systems. This provides application consistent backup and restore at a remote facility
for improved disaster recovery.
You do not need to know where the data actually is. When you request a file, Tivoli Storage
Manager gets the location of the objects to restore from its database.
You can restore using the command-line interface (CLI), GUI, or web browser interface. The
browser interface enables you to initiate the restore remotely so that you do not have to be
physically located at the system being restored.
Point-in-time
A point-in-time operation restores the specified objects to the state that existed at a specific
date and time. A point-in-time restore is supported on the file space, directory, or file level.
You must specify a sufficiently long retention period in the management class to enable this to
occur. To provide a point-in-time restore capability for, say, up to one month previously, set the
following parameters:
VEREXISTS and VERDELETED to NOLIMIT
RETEXTRA and RETONLY to at least 31 days
This way, the number of times that the files change in the restorable period does not matter
because you will always have enough versions stored to be able to perform the restore.
Data migrated by hierarchical storage management (HSM) clients and archive data are not
permitted in active data pools. As updated versions of backup data continue to be stored in
active-data pools, older versions are deactivated and removed during reclamation
processing.
Find more information about active data pools, see “Active-data pools” on page 60.
Snapshot restore
Snapshot capabilities are achieved through the use of advanced storage-specific hardware
snapshot technology to help create a high performance, low impact application data
protection solution. To deliver data protection for business-critical databases restores in
minutes instead of hours.
3.2.2 Retrieve
The retrieve operation obtains copies of archived files from the Tivoli Storage Manager
server. You can specify either selected files or whole directories to retrieve archived files. The
description option enables you to search for the descriptions assigned to the archive package
when it was made; you may decide to put the files into the same directory from which they
were archived, or into a different directory.
3.2.3 Recall
This is a brief description of the recall function to bring back a migrated file from IBM Tivoli
Storage Manager to its original place on the local file system.
Tivoli Storage Manager provides disaster recovery of the Tivoli Storage Manager server with
its Disaster Recovery Manager function. Disaster Recovery Manager offers various options to
configure, control, and automatically generate a disaster recovery plan containing the
information, scripts, and procedures to automate recovery of the Tivoli Storage Manager
server, and help ensure quick recovery of data after a disaster.
With electronic tape vaulting, the Tivoli Storage Manager server will have an alternate
location to store primary and copy storage pools as though they are directly attached. The
Tivoli Storage Manager server can first write a copy of disk storage pool data to tape pools at
the remote site (Datacenter #2), then the data can be migrated to the tape storage pools at
the primary site (Datacenter #1). See Figure 3-4.
LAN/WAN
TCP/IP
Datacenter #1 Datacenter #2
Data Flow
Depending on your configuration (and whether remote disk replication is being used in
conjunction with electronic tape vaulting), you might choose to back up the Tivoli Storage
Manager database and configuration files by this method. This ensures a copy of the data is
stored at both sites and that the Tivoli Storage Manager server can rapidly recover at the
remote site. If remote disk replication is used for mirroring of the Tivoli Storage Manager
database and storage pools, the Tivoli Storage Manager server can be recovered quickly,
without any loss of client data. A peer-to-peer configuration can be used to balance the load
of Tivoli Storage Manager services in the enterprise and provide data protection and rapid
recovery for a failure at either site. A major consideration with electronic vaulting is
synchronization of metadata in the database with data in the storage pool. If the metadata is
replicated before the data, invalid database references might occur at the target site.
Table 3-1 summarizes several of these technologies. The use of such technologies also might
depend on a particular vendor’s replication or vaulting solution. For example, the IBM DS8000
uses PPRC to achieve data replication. Metro Mirror is supported by ESCON links, which can
be further extended by DWDM or WAN channel extension.
Storage area networks (SANs) overcome the distance limitations of other storage channel
protocols, such as SCSI. Longwave laser GBICs (available on most SAN hubs), switches, and
directors enable a transmission distance of up to 10 kilometers (11 kilometers when you
include switch to host connections) when used with 9-micron diameter single-mode optical
fiber. Shortwave GBICs use multi-mode fiber and is the ideal choice for shorter distance (less
than 500 meters from transmitter to receiver or vice versa).
CWDM or DWDM is a way to open up the conventional optical fiber bandwidth by breaking it
up into many channels, each at a particular optical wavelength (a unique color of light). Each
wavelength can carry a signal at any bit rate less than an upper limit defined by the
electronics, typically up to several gigabits per second. Dense wavelength division
multiplexing (DWDMs) are implemented in areas that have dark fiber available through telco
and service providers. The DWDM is deployed as part of the physical layer. It is therefore
WAN and IP based channel extenders typically use telecommunication lines for data transfer
and therefore enable application and recovery sites to be located longer distances apart. The
use of WAN and IP channel extenders provides the separation for disaster recovery purposes
and avoids various barriers imposed when customers do not have a “right of way” to lay their
fiber cable. WAN and IP channel extenders generally compress the data before sending it
over the transport network, however the compression ratio must be determined based on the
application characteristics and the distance.
Network attached storage (NAS) and iSCSI solutions are beginning to offer low cost IP based
storage. Copies of Tivoli Storage Manager storage pools and the Tivoli Storage Manager
database can be stored at a remote site using IP based storage to offer a low cost
implementation while using existing infrastructure. Configurations can include Tivoli Storage
Manager clients attached to iSCSI-based data, backing up to a Tivoli Storage Manager server
or Tivoli Storage Manager servers using iSCSI based storage as storage pools.
For a detailed overview of technologies, products, costs, and preferred practices with distance
solutions, see Introduction to Storage Area Networks and System Networking, SG24-5470.
Also consider that collocation on copy storage pools will result in more partially filled volumes
and probably increased offsite reclamation activity. Collocation typically results in a partially
filled sequential volume for each client or client file space. This might be acceptable for
primary storage pools because these partially filled volumes remain available and can be
filled during the next migration process. However, for copy storage pools this might be
unacceptable because the storage pool backups are usually made to be taken offsite
immediately. If you use collocation for copy storage pools, you will have to decide between
these courses of action:
Taking more partially filled volumes offsite, thereby increasing the reclamation activity
when the reclamation threshold is lowered or reached
Leaving these partially filled volumes onsite until they fill and risk not having an offsite
copy of the data on these volumes.
With collocation disabled for a copy storage pool, typically there will be only a few partially
filled volumes after storage pool backups to the copy storage pool are complete. Consider
carefully before using collocation for copy storage pools. Even if you use collocation for your
primary storage pools, you might want to disable collocation for copy storage pools. Or, you
might want to restrict collocation on copy storage pools to certain critical clients, as identified
by the Business Impact Analysis.
When an offsite volume is reclaimed, the files on the volume are rewritten to another copy
storage pool volume that is onsite. The Tivoli Storage Manager server copies valid files
contained on the offsite volumes being reclaimed, from the original files in the primary storage
pools. In this way, the server can reclaim offsite copy storage pool volumes without having to
recall and mount these volumes. Logically, these files are moved back to the onsite location.
The new volume must be moved offsite as soon as possible. However, the files have not been
physically deleted from the original offsite volume. In the event of a disaster occurring before
the newly written copy storage pool volume has been taken offsite, these files can still be
recovered from the offsite volume, provided that it has not already been reused (based on the
REUSEDELAY parameter) and the database backup that you use for recovery references the
files on the offsite volume.
The REUSEDELAY parameter specifies the number of days that must elapse before a volume
can be reused or returned to scratch status after all files are expired, deleted, or moved from
the volume.
The server reclaims an offsite volume that has reached the reclamation threshold as follows:
1. The server determines which files on the volume are still valid.
2. The server obtains these valid files from a primary storage pool, or if necessary, from an
onsite volume of a copy storage pool.
3. The server writes the files to one or more volumes in the copy storage pool and updates
the database. If a file is an aggregate file with unused space, the unused space is
removed during this process.
4. A message is issued indicating that the offsite volume was reclaimed.
5. The newly written volumes are then marked to be sent offsite, and after this has occurred,
the reclaimed volume can be returned to an onsite scratch pool.
Volumes with the access value of offsite are eligible for reclamation if the amount of empty
space on a volume exceeds the reclamation threshold for the copy storage pool. The default
reclamation threshold for copy storage pools is 100%, which means that reclamation is not
performed.
If you plan to make daily storage pool backups to a copy storage pool, then mark all new
volumes in the copy storage pool as offsite and send them to the offsite storage location. This
strategy works well with one consideration; if you are using automatic reclamation (the
reclamation threshold is less than 100%). Each day’s storage pool backups will create a
number of new copy storage pool volumes, the last one being only partially filled. If the
percentage of empty space on this partially filled volume is higher than the reclamation
percentage, this volume becomes eligible for reclamation as soon as you mark it offsite. The
reclamation process causes a new volume to be created with the same files on it. The volume
you take offsite is then empty according to the Tivoli Storage Manager database. If you do not
recognize what is happening, you can perpetuate this process by marking the new partially
filled volume offsite.
If you send copy storage pool volumes offsite, the best approach is to control copy storage
pool reclamation by using the default value of 100. This turns off reclamation for the copy
Depending on your data expiration patterns, you might not need to do reclamation of offsite
volumes each day. You might choose to perform offsite reclamation on a less frequent basis.
For example, suppose you send copy storage pool volumes to and from your offsite storage
location once a week. You can run reclamation for the copy storage pool weekly, so that as
offsite volumes become empty they are sent back for reuse.
When you perform reclamation for offsite volumes, use the following sequence:
1. Back up your primary storage pools to copy storage pools.
2. Turn on reclamation for copy storage pools by lowering the reclamation threshold below
100%.
3. When reclamation processing completes, turn off reclamation for copy storage pools by
raising the reclamation threshold to 100%.
4. Mark any newly created copy storage pool volumes as offsite and then move them to the
offsite location.
This sequence ensures that the files on the new copy storage pool volumes are sent offsite,
and are not inadvertently kept onsite because of reclamation.
Attention: If collocation is enabled and reclamation occurs, the server tries to reclaim the
files for each client node or client file space onto a minimal number of volumes.
We discuss this function in detail in 6.3, “Data protection solution using node replication
feature” on page 128.
You can deduplicate any type of data except encrypted data. You can deduplicate client
backup and archive data, Tivoli Data Protection data, and so on. Tivoli Storage Manager can
deduplicate whole files as well as files that are members of an aggregate. You can
deduplicate data that has already been stored. No additional backup, archive, or migration is
required.
Deduplicated
Client 1 A disk storage
A pool stores
unique chunks
Node File to reduce disk
Deduplication utilization
B
Client 1 A
Client 2 B Server Client 2 B
Client 3 C A
Tape copy pool
C TSM Database stores A, B, and
C individually to
B avoid
Client 3 C performance
degradation
Files A, B and C have C during restore
common data
Figure 3-5 Deduplication overview
If the storage pool is configured with a sequential file device class, it is eligible for storage pool
deduplication. Data deduplication and its benefits are described in 6.2, “Disk-to-disk data
protection solution using deduplication” on page 113.
When a user tries to restore, retrieve, or export file data, the requested file is obtained from a
primary storage pool wherever possible. Stored objects are only restored from copy storage
pools when the expected primary storage pool volume is unavailable.
A primary storage pool can use random access storage (DISK device class) or sequential
access storage (for example, tape, optical, or FILE device classes).
The copy storage pool may contain copies of all files that appear in the primary storage pool.
A copy storage pool provides recovery from partial and complete failures of a primary storage
pool.
A partial failure would be, for example, if a single tape in a primary storage pool is damaged
by a failing drive, or simply has too many read or write errors.
Copy pools
Onsite volumes support both, media and disaster recovery. If a media failure is detected for a
primary storage pool volume, objects will be automatically accessed using the copy storage
pool volume.
Offsite volumes are used for disaster recovery. If a media failure is detected for a primary
storage pool volume and no onsite copy volume is available, offsite volumes need to be
retrieved at the local site for the object to become accessible again. Typically they are stored
at the offsite location together with a database backup to allow for a complete recovery of the
server in the event of a disaster.
Active-data pools
With active-data pools (ADP), Tivoli Storage Manager administrators can segregate active
and inactive files in the servers storage hierarchy. With ADP, you can organize active data that
is likely to be restored in a way that the restore operations do not need to access tapes
containing inactive data. If you keep many versions of files, searching tapes for active
versions of files that are mixed with inactive versions can take a long time. By storing only the
active versions together on tapes, the time spent to position past the inactive files is
eliminated. For Tivoli Storage Manager, this means keeping only the active versions of files in
a storage pool strictly defined for such a purpose.
Setting up an active-data pool depends on whether your intended use is for faster client
restore or onsite and offsite storage of active data. Active-data pools can use any type of
sequential-access storage (for example, a tape device class or FILE device class). However,
the precise benefits of an active-data pool depends on the specific device type associated
with the pool. As mentioned in “Snapshot restore” on page 51, active-data pools associated
with a FILE device class are ideal for fast client restores because FILE volumes do not have
to be physically mounted and because the server does not have to position past inactive files
Active-data pools that use removable media, such as tape or optical, offer similar benefits.
Although tapes need to be mounted, the server does not have to position past inactive files.
However, the primary benefit of using removable media in active-data pools is the reduction of
the number of volumes used for onsite and offsite storage. If you vault data electronically to a
remote location, an active-data pool associated with a SERVER device class lets you save
bandwidth by copying and restoring only active data.
You can also use active data pool volumes, electronically vaulted, at an offsite location for
increased protection of your data in case of a server disaster.
When cache is disabled and migration occurs, the server migrates the files to the next storage
pool and erases the files from the disk storage pool. By default, the system disables caching
for each disk storage pool because of the potential effects of cache on backup performance.
The advantage of using cache for a disk storage pool is that cache can improve how quickly
the server retrieves some files. When you use cache, a copy of the file remains on disk
storage after the server migrates the primary file to another storage pool. You may want to
consider using a disk storage pool with cache enabled for storing space-managed files that
are frequently accessed by clients.
Figure 3-8 shows how simultaneous writes to primary pools and up to three copy pools and
active-data pools occur during the following operations:
Client backup, archive, and space-management operations
Server import operations
Storage pool migration
With Tivoli Storage Manager, you can now write data simultaneously to copy storage pools
and active-data pools during server data-migration processes.
The simultaneous-write function during migration can reduce the amount of time required to
back up storage pools or copy active data. Data that is simultaneously written to copy storage
pools or active-data pools during migration is not copied again to the copy storage pools or
active-data pools. For example, suppose that you migrate all the data in your primary
random-access disk storage pool nightly and then back up your primary storage pools. By
using the simultaneous-write function during migration, you can significantly reduce the
amount of time required for backup operations. You can also use the simultaneous-write
function during migration if you have many client nodes and the number of mount points that
are required to perform the simultaneous-write function during client store operations is
unacceptable. If mounting and demounting tapes when writing data simultaneously during
client store operations is taking too much time, consider writing data simultaneously during
migration. With Tivoli Storage Manager V6.2, you can specify the simultaneous-write function
for a primary storage pool if it is the target for any of the eligible operations (client store
sessions, server import processes, and server data-migration processes).
Similar to client-side simultaneous write during data store operations, a copy storage pool list
is created at the beginning of a process and is active for the life of the process. While the
process is active, the copy storage pools in the list are written to.
Simultaneous write migration is not supported for the server data movements, such as
reclamation, move data, move node data, or cloning.
Simultaneous writes are not supported for LAN-free backups, or when a NAS backup is
writing a TOC file.
Carefully consider the use of simultaneous writes. Because the data is written to the copy
storage pool and primary storage pool simultaneously, the backup performance is only as
good as the slowest device being used for any of the pools.
GPFS provides the file and block locking necessary to share a physical device among
multiple hosts. The shared disk storage pool allows for storage agents and servers to access
FILE devclass storage from multiple physical systems on the LAN. This helps reduce server
utilization and enables more concurrent operations by distributing them across multiple
storage agents.
Concurrent access to volumes improves restore performance by allowing two or more client
sessions, two or more storage agents, or a combination of client sessions and storage agents
to access the same volume at the same time. Multiple client sessions and storage agents can
read a FILE volume concurrently. In addition, one client session or storage agent can write to
the volume while it is being read.
As part of the planning process, decide whether to use LAN-free data movement and whether
you want to use client-side data deduplication, server-side deduplication, or both.
Traditionally, disaster recovery plans include daily offsite tape backups that are picked up from
the local site and transported by a courier to a secure facility, which is often a tape vaulting
service provider. Vaulting of tapes at offsite locations can provide a secure means to protect
data in the event of a disaster at the primary site. To recover from a disaster, you must know
the location of offsite recovery media. Tivoli Storage Manager Disaster Recovery Manager
(DRM) helps determine which volumes to move offsite and back onsite and tracks the location
of the volumes. Disaster Recovery Manager is included in Tivoli Storage Manager Extended
Edition.
With tape vaulting, you can back up primary storage pools to a copy storage pool and then
send the copy storage pool volumes offsite. You can track these copy storage pool volumes
by changing their access mode to offsite, and updating the volume history to identify their
location. If an offsite volume becomes expired, the server does not immediately return the
volume to the scratch pool. The delay prevents the empty volumes from being deleted from
the database, making it easier to determine which volumes must be returned to the onsite
location. Disaster Recovery Manager handles all of this automatically.
Storage pools will be arranged in a storage hierarchy, which consists of at least one primary
storage pool to which a client node backs up, archives, or migrate data. Typically, data is
stored initially in a disk storage pool for fast client restores, and then moved to a tape-based
storage pool, which is slower to access but which has greater capacity and can be more cost
efficient. The location of all data objects is automatically tracked within the server database.
You can set up your devices so that the server automatically moves data from one device to
another, or one media type to another. The selection can be based on characteristics such as
file size or storage capacity. A typical implementation might have a disk storage pool with a
subordinate sequential (tape) storage pool. When a client backs up a file, the server might
initially store the file on disk according to the policy for that file. Later, the server might move
the file to tape when the disk becomes full. This action by the server is called migration. You
can also place a size limit on files that are stored on disk, so that large files are stored initially
on tape instead of on disk.
You can organize the server’s storage pools into one or more hierarchical structures, as
Figure 3-10 on page 65 shows.
more expensive
less expensive
slower
Another option to consider for your storage pool hierarchy is IBM tape cartridges and drives,
(3592, 1120, 1130 and 1140) which can be configured for an optimal combination of access
time and storage capacity.
Migration of files from disk to sequential storage pool volumes is particularly useful because
the server migrates all the files for a group of nodes or a single node together. This gives you
partial collocation for clients. Migration of files is especially helpful if you decide not to enable
collocation for sequential storage pools.
This storage hierarchy allows flexibility in a number of ways. For example, you can set policy
to have clients send their backup data to disks for faster backup operations, then later have
the server automatically migrate the data to tape.
Data hierarchies can be customized to suit business needs. As Figure 3-11 shows, server
Grey’s data can go directly from disk to tape. Server Cyan’s data might require more complex
retention to keep it available longer. This data might need to go from disk to file to tape.
Grey Cyan
Using a VTL, you can create variable numbers of drives and volumes because they are only
logical entities within the VTL. The ability to create more drives and volumes increases the
capability for parallelism, giving you more simultaneous mounts and tape I/O.
VTLs use SCSI and Fibre Channel interfaces to interact with applications. Because VTLs
emulate tape drives, libraries, and volumes, an application such as Tivoli Storage Manager
cannot distinguish a VTL from real tape hardware unless the library is identified as a VTL.
With enhancements available in Version 6.3, you can define a library as a virtual tape library
(VTL) to Tivoli Storage Manager. VTLs primarily use disk subsystems to internally store data.
Because they do not use tape media, you can exceed the capabilities of a physical tape
library when using VTL storage. Using a VTL, you can define many volumes and drives which
provides for greater flexibility in the storage environment and increases productivity by
allowing more simultaneous mounts and tape I/O.
There are some considerations for defining a library as a virtual tape library, including
enhancements for performance and setup of your hardware. Defining a VTL to the Tivoli
Storage Manager server can help improve performance because the server handles mount
point processing for VTLs differently than real tape libraries. The physical limitations for real
tape hardware are not applicable to a VTL, affording options for better scalability.
You can use a VTL for any virtual tape library when the following conditions are both true:
There is no mixed media involved in the VTL. Only one type and generation of drive and
media is emulated in the library.
Every server and storage agent with access to the VTL has paths that are defined for all
drives in the library.
If either of these conditions is not met, any mount performance advantage from defining a
VTL library to the Tivoli Storage Manager server can be reduced or negated. VTLs are
compatible with earlier versions of both library clients and storage agents. The library client or
storage agent is not affected by the type of library that is used for storage. If mixed media and
path conditions are true for a SCSI library, it can be defined or updated as LIBTYPE=VTL.
Because VTLs do not have the physical limitations that real tape hardware does, their
capacity for storage is more flexible. The concept of storage capacity in a virtual tape library is
different from capacity in physical tape hardware. In a physical tape library, each volume has
a defined capacity, and the library’s capacity is defined in terms of the total number of
volumes in the library. The capacity of a VTL, alternatively, is defined in terms of total
available disk space. You can increase or decrease the number and size of volumes on disk.
This variability affects what it means to run out of space in a VTL. For example, a volume in a
VTL can run out of space before reaching its assigned capacity if the total underlying disk
runs out of space. In this situation, the server can receive an end-of-volume message without
any warning, resulting in backup failures. When out-of-space errors and backup failures
occur, disk space is usually still available in the VTL. It is hidden in volumes that are not in
To help prevent out-of-space errors, ensure that any SCSI library that you update to
LIBTYPE=VTL is updated with the RELABELSCRATCH parameter set to YES. The
RELABELSCRATCH option enables the server to overwrite the label for any volume that is
deleted and to return the volume to scratch status in the library. The RELABELSCRATCH
parameter defaults to YES for any library defined as a VTL.
Drive configuration in a VTL is variable, depending on the needs of your environment. Most
VTL environments use as many drives as possible to maximize the number of concurrent
tape operations. A single tape mount in a VTL environment is typically faster than a physical
tape mount. However, using many drives increases the amount of time that the Tivoli Storage
Manager server requires when a mount is requested. The selection process takes longer
because the number of drives that are defined in a single library object in the server
increases. Virtual tape mounts can take as long or longer than physical tape mounts
depending on the number of drives in the VTL.
For best results, create VTLs with 300 - 500 drives each. If more drives are required, you can
logically partition the VTL into multiple libraries and assign drives to each library. Operating
system and SAN hardware configurations could impose limitations on the number of devices
that can be utilized within the VTL library.
Figure 3-12 shows the advanced tiering approach with virtual tape libraries. See 6.4, “Tivoli
Storage Manager together with ProtecTIER” on page 135.
Figure 3-12 Advanced storage tiering with virtual tape library
Explaining each of the data storage management components, or the effects of the retention
parameters defined in these components on the data stored in the server is far beyond the
scope of this book. But, we do provide a brief overview of how client data is stored.
Tivoli Storage Manager policies are rules that determine how the client data is stored and
managed. The rules include where the data is initially stored, how many backup versions are
kept, how long archive copies are kept, and so on.
You can have multiple policies and assign the various policies as needed to specific clients, or
even to specific files. Policies Assign a location in server storage where data is initially stored.
Server storage is divided into storage pools that are groups of storage volumes.
Server storage can include hard disk, optical, and tape volumes.
Clients use Tivoli Storage Manager to store data for any of the following purposes:
Backup and restore
The backup process copies data from client systems to server storage to ensure against
loss of data that is regularly changed. A policy includes the number of versions and the
retention time for those versions. The server retains versions of a file according to this
policy, and replaces older versions of the file with newer versions.
Archive and retrieve
The archive process copies data from client systems to server storage for long-term
storage. The process can optionally delete the archived files from the client systems. The
server retains archive copies according to the policy for archive retention time. A client can
retrieve an archived copy of a file.
Backup set recovery
Backup set recovery is the creation of a complete set of backed-up files for a client. The
set of files is called a backup set. A backup set is created on the server from the most
recently backed-up files that are already stored in server storage for the client. The policy
for the backup set consists of the retention time that you choose when you create the
backup set.
You can copy a backup set onto compatible portable media, which can then be taken
directly to the client for rapid recovery without the use of a network and without having to
communicate with the Tivoli Storage Manager server.
Migration and recall
Migration is a function of the Tivoli Storage Manager for Space Management (for
supported UNIX and Linux systems) and Tivoli Storage Manager HSM for Windows
programs. It frees client storage space by copying files from client systems to server
storage. On the client, the program replaces the original file with a stub file that points to
the migrated file in server storage. Files are recalled to the client systems when needed.
The process of migrating and retrieving data through these programs is transparent to
users and applications, other than a possible degradation in performance as compared to
data stored on locally attached, tier one disk. Policy determines when files are considered
for automatic migration. On the UNIX or Linux systems that support the Tivoli Storage
Manager for Space Management program, policies determine whether files must be
backed up to the server before being migrated. Space management is also integrated with
backup. If the file to be backed up is already migrated to server storage, the file is backed
up from there.
Files remain in server storage until they expire (retention policies are met) and expiration
processing occurs, or until they are deleted from server storage. A file expires because of
criteria that are set in policy. For example, the criteria include the number of versions that are
allowed for a file and the number of days that elapsed since a file was deleted from the client’s
file system.
LDAP authentication
Instead of the embedded password protection of client data, the Tivoli Storage Manager
server can use an external LDAP directory or Microsoft Active Directory to provide login
authentication for administrators and nodes. Figure 3-14 shows the communication flow.
IBM Tivoli Directory Server and Windows Active Directory are the current supported LDAP
servers. LDAP-authenticated passwords give you an extra level of security by being
case-sensitive, offering advanced password rule enforcement, and a centralized server on
which to authenticate them. You must have an LDAP directory or Microsoft Active Directory
on which to authenticate the password.
With TLS/SSL industry-standard communications, you can encrypt all traffic between the
backup-archive client, the administrative command-line clients, and the IBM Tivoli Storage
Manager server. You can use either self-signed or vendor-acquired SSL certificates. TSL/SSL
is available for LAN-free and server-to-server communication too.
Data encryption
To heighten security for Tivoli Storage Manager sessions, data sent can be encrypted.
Encryption happens on the client side. The file is password-protected so that it is only the
client that can unlock the file during a restore. Encryption exists for both the backup-archive
client and API client. Figure 3-16 shows the transport from the client to the server.
Data sent to the Tivoli Storage Manager server during backup and archive operations are
encrypted with standard 128-bit AES or 56-bit DES encryption. The data that you include is
stored in encrypted form, and encryption does not affect the amount of data that is sent or
received.
For WAN implementations of Tivoli Storage Manager across public networks, data encryption
complements the TLS/SSL communication and completes data security for Tivoli Storage
Manager.
Encryption and client-side data deduplication are mutually exclusive. If you enable both
encryption and client-side data deduplication, encryption operations complete and client-side
data deduplication is ignored. Encryption is incompatible with either client-side or server-side
deduplication. Encrypted files, and files that are eligible for client-side data deduplication, can
be processed in the same operation, but are done in separate transactions.
When you install the product, it includes administrative scripts that have an example order of
execution for scheduling administrative commands. The sample scripts are in scripts.smp
and include the following examples:
Command parameter substitution
SQL SELECT statements that you specify when the script is processed
Command execution control, such as PARALLEL and SERIAL processing options
Conditional logic flow statements:
– The IF clause: This clause determines how processing should proceed based on the
current return code value.
– The EXIT statement: This statement ends script processing
– The GOTO and LABEL statement: This statement directs logic flow to continue
processing
– Comments
You can convert the sample scripts into a maintenance script according to your needs.
The Tivoli Storage Manager also supports macros that are called by the administrative client.
A macro is a file that contains one or more administrative client commands. You can run a
macro only from the administrative client in batch or interactive modes, not from the server.
Macros are stored as a file on the administrative client. Macros are not distributed across
servers and cannot be scheduled on the server.
Admin
Proposed Solution 1 Acquire client packages
Center
3
Define / update policy and schedule
TSM administrator
Define the list of nodes to be 10 views update results
4 updated and send them to server
5 dsmc retrieve package -postschedcmd=deployclient Retrieve client package from the server
(including the bootstrap program)
You can view the historical reports to determine if any issues or trends need attention, such
as uncontrolled growth over time. You can also view workspace that is monitored to see the
Tivoli Storage Manager server IDs, database size, agent status, client node status, scheduled
events, and so on.
The reporting component, sometimes referred to as Tivoli Common Reporting, reports on the
retrieved historical data. IBM Tivoli Monitoring acts as a monitoring application that provides
workspace for you to monitor near real-time information.
The Tivoli Storage Manager monitoring agent communicates with the Tivoli Storage Manager
reporting and monitoring server to retrieve data from its database and return this data to the
Tivoli Monitoring server. The Tivoli Storage Manager monitoring agent communicates with the
existing IBM Tivoli Monitoring infrastructure, or Tivoli Storage Manager reporting and
monitoring server to retrieve data from its database and return this data to the Tivoli
Monitoring server.
User ID:
DB2 database AIX Linux
db2inst1
Windows
Tivoli Monitoring
for Tivoli Storage db2admin
Manager agent
instances
Library sharing
When Tivoli Storage Managers share a storage device, a single server, the library manager,
controls device operations. These operations include mount, dismount, volume ownership,
and library inventory. Other servers and other library clients use server-to-server
communications to contact the library manager and request device service. Data moves over
the SAN between servers and the storage device. Figure 3-19 on page 75 shows a
configuration with a single library client, library server, tape library, the communication paths
and data paths.
The inventory of media volumes in the shared library device is partitioned among servers.
Either one Tivoli Storage Manager server owns a particular volume, or the volume is in the
global scratch pool. No one server owns the scratch pool at any one time.
Only one Tivoli Storage Manager server accesses each tape drive at a time. Drive access is
serialized and controlled so that servers do not dismount other servers’ volumes or writes to
drives where other servers mount their volumes.
Database backup
More options to protect the database exist today than previously. The standard way is still
valid but as the database grows we need a smarter way to get the backup and restore times
decreased. Multiple concurrent data streams while backing up or restoring the database can
be used or Tivoli Storage FlashCopy Manager can protect the database.
See 5.2.2, “Database backup and restore” on page 96 for further details.
HADR is not intended to replace the database backup, but can increase the availability of
Tivoli Storage Manager server application. See “DB2 High Availability Disaster Recovery
(HADR)” on page 104.
On Microsoft Windows, the Tivoli Storage Manager server can be set up as part of the
Microsoft Cluster for Windows resource group. Each Tivoli Storage Manager server instance
is configured in its own resource group.
A more detailed discussion of this topic is available in “Clustering services” on page 100.
Database reorganization
The Tivoli Storage Manager server database reorganizes itself automatically in order to
enhance performance and usage of the database. The process reconstructs rows to eliminate
fragmented data, and compacts information. If the automatic table and index reorganization is
affecting server performance, you can manually schedule reorganizations.
See the Database size, database reorganization, and performance considerations for Tivoli
Storage Manager Version 6 servers technote for details about reorganization:
http://www.ibm.com/support/docview.wss?uid=swg21452146
Keep in mind that this is not an implementation guide and we use the graphic only to explain
the concepts. Your real schedules, of course, might look different, and might change
depending on how much processing power the server has and the workload. For a detailed
introduction to the topic, see the following resources:
IBM Tivoli Storage Manager Implementation Guide, SG24-5416
http://www.redbooks.ibm.com/abstracts/sg245416.html
IBM Tivoli Storage Management Concepts, SG24-4877
http://www.redbooks.ibm.com/abstracts/sg244877.html
A general daily lifecycle starts with the client backup operation. Here we do not distinguish
between progressive incremental or some data protection agent activity. You usually dedicate
some time of the day to the backup or archive operations. The best approach is not to not
interrupt the backup task with server maintenance operations.
When the backup window is closed, the server can start its housekeeping. Housekeeping
consists of the following tasks:
Backup storage pools: Typically, the first process after the backup activity completes is to
back up the storage pools to be sure valid copies are available for onsite and offsite
storage.
Database backup: A current database backup makes sure you have a copy of your
database in case something happens and you need to restore. The backup taken after the
storage pool backup does have the correct pointers to the objects managed by the server.
After reclamation completes, you are at the end of your housekeeping tasks. You might have
scheduled another database backup while continuing with your server operation, waiting for
the next backup window to open. With Tivoli Storage Manager V6, additional server
housekeeping tasks are introduced.
Identify duplicates
Replicate nodes
Now if you go back to the wheel of life, you see how we integrate those processes in the daily
operation. We start the identify duplicates process during the ingesting of client data so that
we have more time to find the duplicates during the day. Because of the resources required
for the identify process and the file expiration process, you might want to be sure that identify
and expiration do not run in parallel.
This is why in the sample lifecycle we wait for the expiration to complete before we start the
node replication, another job that is database-intensive. Depending on your implementation,
you might not take local storage pool copies for your deduplication storage pool and rely on
node replication. If that is the case, you can adjust the server process scheduling to your
needs, for example copy your non-deduplication storage pools while running the identify
duplicate processes in parallel. In any case, be sure that you dedicated the required
resources to your server so that the server is able to complete the housekeeping tasks.
There is a recorded Storage Technical Exchange (STE) session that discusses how to
perform important, system validations on your Tivoli Storage Manager V6 environment. It
explains how the basic steps can be performed manually or automatically. Several
components, including the Tivoli Storage Manager Administration Center, Tivoli Common
Reporting, Tivoli Enterprise Portal and custom SQL queries are presented, you can find the
presentation at the following address:
http://www.ibm.com/support/docview.wss?uid=swg27021836
Also, several videos are available about the Tivoli Storage Manager Operation Center:
https://www.youtube.com/watch?v=5l0wtmCmwkA
This chapter looks at those challenges, and helps to categorize them, thus allowing you to
take advantage of the entire toolkit portfolio to adjust for current and future challenges.
This chapter has an explanation of how the team that wrote this book approached the task to
provide an alternate view of the product to you: looking at Tivoli Storage Manager as a data
protection solution in addition to a backup product. You will find pointers to solutions the team
developed to meet your business requirements.
We searched for the best way to explain the transition from what was considered a
backup/archive product to a full data protection solution, using well known Tivoli Storage
Manager technologies. The goal was to provide possible solutions and the information to
allow you to design and take advantage of new solutions combining available technologies.
The initial approach was to introduce something we named solution tiering. We tried to bring
RTO/RPO objectives into some relation to implementation cost. What we developed on the
whiteboard was the chart you see in Figure 4-1, the solution tiering chart.
RPO/RTO $
Sketch
The plan was to tweak this sketch to our needs and then have the graphic designer convert
this to an impressive graphic. Now, when we started reviewing all types of information to use
for the book, we came across the Disaster Recovery Tiers that were defined by the SHARE
user group in 1992. This is graphically represented in Disaster Recovery Strategies with Tivoli
Storage Manager, SG24-6844:
http://www.redbooks.ibm.com/abstracts/sg246844.html
The message we learned from this is that the jargon, when discussing data protection, has
changed, but the challenges have existed for a long time. The other message was that for all
of this time, the Tivoli Storage Manager product was available to address these challenges.
For example, although the 1992 concept of “big data” does not look challenging today, the
Tivoli Storage Manager product at that time provided the desired level of protection to your
data, as it does today. So although you might see further references to that model throughout
the book, the team decided to return to the drawing board and devise an alternate approach
to introduce the changed mind-set of Tivoli Storage Manager as a data protection solution.
As a next step in our process, we decided to remove challenges such as financial aspects,
from our consideration. For example, it is nothing new that full high availability (HA) comes at
a price. Our next step was to concentrate on possible data protection architectures.
An online version of the matrix is available under the Additional Materials tab at the IBM
Redbooks website:
http://www.redbooks.ibm.com/Redbooks.nsf/pages/addmats
Keep in mind that this matrix is something the team agreed upon, and when you review and
apply filters, you might want to recategorize the technical and business challenges based on
your individual business challenges.
From there, we defined categories that can help you find the right data protection solution for
your specific task.
Then we needed to determine how to put the information in the matrix. Initially we thought it
would be a perfect tool for what we planned to use it for. We introduced more rows to refer to
We next identified the following challenge categories, creating a subset of the original matrix
tailored to each challenge:
Data reduction
Data availability
Security
Disaster recovery
Virtualization
Again, this was the team’s view of how to logically group the challenges, features, and
solutions together. In the following sections of this chapter, and in the remainder of this book
you will find the matrix subset for each category, guidance for how to interpret it, and
references to extra resources, either in this publication or externally.
We hope you find the matrix as useful as we found it during the development of the book.
On the left are the business challenges that relate to the challenge category. On the right are
the technical challenges.
In the middle are the related Tivoli Storage Manager toolkit features we identified. You are
most likely familiar with them because some have been available for a long time.
The matrix lists sample solutions that allow you to push your data protection limits. From
there, you can find references to the sample implementations that demonstrate how creative
you can be, implementing the exact data protection solution that fits your needs.
Depending on the specific challenge you have, you might want to look at either of the
available solutions to reduce your recovery time, or to help you protect your virtualized
environments, or help you with both. Search for an X (as in Figure 4-4 on page 86) in either
the solution or toolkit row to match your challenge column. When you find a match, it can help
you with the specific challenge category.
Here we look at the data reduction challenge, and as you might expect, you see many well
known toolkit options to reduce network bandwidth and therefore increase network efficiency.
Now if a standard implementation no longer fits your needs, look at the sample solutions we
provide; they might be what you are looking for.
You might wonder why you see server maintenance listed under the data reduction challenge.
Here it was the author’s common understanding that you need a well maintained and healthy
server to be able to achieve your goals.
Solution references
Data reduction is a common feature of several market products. One of the most efficient
ways to reduce the amount of data that is backed up these days is to use data deduplication.
Tivoli Storage Manager has built-in data deduplication features at both the client and server
side. In addition, it can also interact with external deduplication systems such as ProtecTIER
when Tivoli Storage Manager built-in deduplication does not fit.
We describe two examples of data reduction, based on different technologies; both use Tivoli
Storage products. See the following sections:
6.3, “Data protection solution using node replication feature” on page 128
6.4, “Tivoli Storage Manager together with ProtecTIER” on page 135
Toolkit references
Tivoli Storage Manager has data reduction features such as progressive incremental forever
backups, data compression, or subfile backup along with the other features listed in the matrix
snapshot. These features are described in Chapter 3, “Data protection with Tivoli Storage
Manager” on page 33.
Solution references
All solutions that are described in Chapter 6, “Tivoli Storage Manager Technologies and
Solutions” on page 111 meet this data availability category because it is the main purpose of
data protection. An aspect that might change regarding data availability is how fast you need
the data back to your system. Depending on this recovery time objective criteria, you will
probably decide to use a disk-to-disk backup approach or another tape-based solution.
Although, consider that the faster your system is, the more expansive the solution is, as
shown in Figure 4-2 on page 83.
In this book, we do not describe all security options in detail. IBM Tivoli Storage Manager:
Building a Secure Environment, SG24-7505 covers the various security features of Tivoli
Storage Manager. It shows how to use them, together with preferred practice principles to
design, implement, and administer a more secure backup management environment. The
book covers passwords, administrative levels of control, the vital role of encryption, and
procedures for managing offsite data, among other topics. The book is at this location:
http://www.redbooks.ibm.com/abstracts/sg247505.html
For information about improved user authentication and management by integration with
Lightweight Directory Access Protocol (LDAP), go to the IBM Knowledge Center:
http://www.ibm.com/support/knowledgecenter/SSGSG7_6.4.1/com.ibm.itsm.srv.doc/c_mgc
linod_managepwlogin.html
Solution references
Disaster recovery has become an entire pane of a data protection solution and must be
carefully designed to fulfill the recovery time objectives (RTO) and recovery point objectives
(RPO). These are two of the most relevant pieces of information when you build a data
protection solution; they are basically the business impact.
Toolkit references
The first image that comes to mind regarding disaster recovery is to make sure to protect your
Tivoli Storage Manager server database, which is key to your protection solution. Tivoli
Storage Manager server infrastructure protection is described in 5.2, “Protecting the Tivoli
Storage Manager server database” on page 95. It contains (or refers you to other resources
that contain) several creative solutions to protect your database.
By virtualizing your systems, you separate the physical from logical, you manage and utilize
IT resources as a cohesive and holistic unit that is constantly adjusting, reallocating, and
responding as fast as the business dictates.
In the first phase of virtualization, when it was a cutting-edge technology, development or test
environments were considered good candidates, while real production environments were
not, mainly because of performance concerns. Today, with the advent of hypervisor-based
virtualization, like IBM PowerVM hypervisor, VMware ESXi, or Microsoft Hyper-V, this
perception has radically changed.
Virtualization has evolved and can now handle more data while delivering better performance
than in its first phase. Virtualized infrastructures vary in size, therefore they have different
requirements for data protection. The Tivoli Storage Manager family of products takes this
into consideration and provides a set of scalable solutions to address every need.
You might need to use a more conventional approach to protect your data, even in a virtual
machine. In this case you can take advantage of the progressive incremental forever method
combined with the journal-based backup.
Therefore we can not talk about virtualization data protection without talking about cloud data
protection.
The cloud, beyond virtualization, is the ability to dynamically move resources between
different hosts. It adds the ability to create resource pools from virtualized assets, manage a
group of VMs as a single entity, and provide self-service capabilities and usage based
resource metering.
These requirements were considered when the latest version of Tivoli Storage Manager
products were built (for instance, Tivoli Storage Manager for Virtual Environment Data
Protection for VMware and Tivoli Storage FlashCopy Manager for VMware). These two
products bring the required flexibility to handle the data protection of such a dynamic
infrastructure that is the cloud.
Solution references
Details about various sample solution scenarios are in 7.2, “Virtualization: VMware Data
Protection solution” on page 144.
Toolkit references
Tivoli Storage Manager toolkit is greatly improved (since V6.2 and even more with V7.1) to
handle the virtual environment. Although the traditional features work within a virtual
environment, the Tivoli Storage Manager for Virtual Environment product brings a complete
set of enhanced features that you can use to handle data protection of virtualized systems
from the hypervisor layer.
More information about the traditional tools is in Chapter 3, “Data protection with Tivoli
Storage Manager” on page 33. An explanation of how the enhancements work is in 2.2.3,
“Tivoli Storage Manager for Virtual Environments: Data Protection for VMware” on page 21
and 2.3, “Tivoli Storage Manager recent enhancements” on page 29.
4.3 Summary
In this chapter we introduce the solution matrix, explain how to use the matrix to find the
solution for the data protection challenge you want to meet. We use the matrix to reference
you from this chapter to solutions and Tivoli Storage Manager toolkit features that we
document in this book.
In Figure 5-1, our challenge matrix shows solutions and toolkit options related to server
infrastructure protection and disaster recovery (DR). We discuss these in this chapter.
For a long time, performing regular database and storage pool backups to tape volumes,
which are then moved to an offsite location, was the most common and implemented method.
Later in this chapter, we describe the traditional server protection techniques and strategies,
and how to protect the database, with snapshot technology, which is now possible with the
move to the underlying DB2 database.
The Tivoli Storage Manager infrastructure consists of the database and the setup files that
are needed to recover the database and client data. The setup files include, for example, the
active log and the archive log. Client data includes data that is backed up, archived, and
migrated to primary storage pools.
Figure 5-2 on page 95 shows the business continuity tiers and the relation between recovery
time objectives and the cost to safeguard against data loss. With Tivoli Storage Manager, you
can configure the server infrastructure protection to meet each business continuity objective
you require.
The Tivoli Storage Manager application can participate in and be protected by upper tier
solutions in a manner similar to other critical applications. Define your current or server
recovery requirements as early as possible during your planning phase.
Further discussion of mirroring techniques to protect the database log files is in 5.3.1,
“Protecting the Tivoli Storage Manager database log files” on page 106.
$& '!
(()
*+)((
!
!"
#!
$!!%
A special client is embedded within the Tivoli Storage Manager server to perform the backup
and the data flows through the server to the media that will store the backup. Backups can be
initiated manually by using the dsmserv backup db command or can be automatic by using
database monitoring algorithms. The restore is later performed with the dsmserv restore db
command, which identifies the appropriate backup media using the Tivoli Storage Manager
volume history file, and restores the database from it.
Although standard Tivoli Storage Manager server backup operations (as described in
“Multistream database backup” on page 97) can now be multithreaded, they can still take
many hours to backup or restore a multi-terabyte (TB) database. Performance of ongoing
server operations can be reduced while the database is being backed up. A larger database
takes longer to restore, so the server is offline for longer in the event of a Tivoli Storage
Manager database restore.
To solve the backup or restore of a multi-terabyte database problem, you can use the Tivoli
Storage FlashCopy Manager product to provide point-in-time snapshot backup and recovery
capability for the Tivoli Storage Manager’s underlying database that is functionally equivalent
to a standard database backup or restore, or to alleviate the impact of the increased size of
Tivoli Storage Manager databases that can be caused when adopting the latest Tivoli Storage
Manager capabilities, most notably storage pool data deduplication.
& '% !
*+*
$%
& '%
"&.
" # "&.
!
Figure 5-5 Tivoli Storage Manager Server database backup architecture using Tivoli Storage
FlashCopy Manager
Tivoli Storage FlashCopy Manager can be used in conjunction with a Tivoli Storage Manager
server to “offload” traditional backup processing from the production application servers.
Snapshot backups are made by Tivoli Storage FlashCopy Manager on the production server
and subsequently offloaded (backed up) to the Tivoli Storage Manager backup infrastructure.
In this case backups are performed through an additional server called the backup server
(also termed offload server), which performs the actual backup. Because the backup
operation is offloaded to the backup server, the production server is free from nearly all the
performance impact. The production server’s processor time is dedicated for the actual
application tasks, so performance of application users is not affected during backup. When
performing offloaded backups, Tivoli Storage FlashCopy Manager is used in conjunction with
additional Tivoli Storage Manager modules to provide application consistency and perform
the backup from the backup server to Tivoli Storage Manager.
You can extend this approach and as an example perform disk-only backups to an additional
set of FlashCopy devices. Disk-only backups can then be used to provide a fast restore option
for onsite recovery. For offsite protection, or to protect against failure of the storage device
that holds the database they should be used in conjunction with “traditional” Tivoli Storage
Manager database backup and restore techniques or with the “offloaded” backups shown
above.
The methods for backup to and restore from disk-only backups and offloaded backups are
described in the developerWorks Tivoli Storage Manager wiki at the following website:
http://ibm.co/1k614j2
More information about databases (not only Tivoli Storage Manager databases) is in 7.6.3,
“Protect very large databases with Tivoli Storage FlashCopy Manager” on page 245.
Disaster Recovery Manager offers various options to configure, control, and automatically
generate a disaster recovery plan containing the information, scripts, and procedures needed
to automate recovery of the Tivoli Storage Manager server, and help ensure quick recovery of
data after a disaster.
Figure 5-6 illustrates a basic Disaster Recovery Manager strategy.The courier in the figure
represents shipping physical tape cartridges to a disaster recovery location. Today the courier
can represent the data moving from one location to another, and which can be physical tapes,
electronically copied utilizing virtual volumes, and if you think about storage pools you can
use replication. There are several ways you can implement this, but fundamentally you ship
your data to a remote location to protect from a disaster.
The Disaster Recovery Manager media management function manages the movement of
database backup and copy storage pool tapes to and from offsite storage, and performs
expiration of Tivoli Storage Manager database backup series. Disaster Recovery Manager
also manages and tracks the media on which the data is stored, so that data can be easily
located if disaster strikes.
Clustering services
Clustering is a high-availability solution that minimizes or eliminates many potential sources
of downtime. It is the most common technique to achieve high availability (HA), by introducing
redundancy in software, hardware, and data. In a failure, the clustering software immediately
starts the application on the standby system without requiring administrative intervention.
Cold standby Second node is a backup, Data from first system Few hours
installed/configured only can be backed-up and
when first node fails. Upon restored on second
failure, it is activated. system as required.
Hot standby Software is installed and Data mirrored near Few seconds
running on both nodes. The real time and both
software on the second systems have identical
system is running but not data. Data replication
processing. is typically done
through software.
Active-Active (load Both first and second Data replication Zero failover
balanced) systems are active and happens time
processing requests in through software and
parallel. is bi-directional.
Figure 5-8 shows a sample cluster configuration: nodes A, B, and C are active, D is passive
(standby). Node D can take over from A, B, or C.
After the problem with node B is fixed, node D can failback to node B as shown in
Figure 5-10. Nodes A, B, and C are active again, node D is standby.
A server cluster maintains a consistent, updated copy of the cluster database on all active
nodes.
The Tivoli Storage Manager server can be configured to work with any HA clustering solution,
but is tested and documented for Microsoft Cluster for Windows, IBM HACMP™ and IBM
PowerHA® on AIX, and Tivoli System Automation for Multiplatforms (SA MP) on Linux
environments.
The following Cluster support for IBM Tivoli Storage Manager server technote has details of
what to consider if you plan to use the Tivoli Storage Manager server with other HA products:
http://www-01.ibm.com/support/docview.wss?uid=swg21609772
On AIX, you have options to cluster the Tivoli Storage Manager database and log files:
Logical Volume Manager (LVM) Mirroring
Live Partition Mobility (LPM)
PowerHA on AIX (HACMP)
What you choose depends on how automated the fail over of the Tivoli Storage Manager
application should be, where LVM is less automated than LPM, which again is less automated
than PowerHA.
In Linux environments, there are several cluster software solutions to choose from, such as
Red Hat Cluster Suite or SteelEye LifeKeeper.
Oracle Solaris Cluster and HP UNIX Serviceguard are valid cluster software solutions for their
respective platforms, Oracle Solaris or HP UNIX.
A solution that applies to clustering of all these platforms is Veritas Cluster Server. It provides
application cluster capabilities to systems running other applications, including databases or
network file sharing.
To protect cluster storage resources, a Tivoli Storage Manager client can be used inside or
outside the cluster. Consider the following information:
Outside the cluster, cluster storage resources can be mapped to the client, which will then
“follow” the resource to whichever node it may be assigned to.
The disadvantage to being outside the cluster is that backup of data over network
protocols such as NFS or CIFS is slower than the backup of a local file system.
Placing the client within the cluster allows for faster backups and is supported by Tivoli
Storage Manager’s incremental forever and other fault tolerant features.
Additional references
A recorded Storage Technical Exchange session discusses the Tivoli Storage Manager
server in a clustered and high availability environment, which has further information,
installation examples, and references to more information. You can find the session here:
http://www.ibm.com/support/docview.wss?uid=swg27036541
HADR is one part of the technique described in Electronic vaulting using deduplicated remote
copy storage pools, available on the Tivoli Storage Manager wiki pages:
http://ibm.co/1o9YyDv
Also see 6.3, “Data protection solution using node replication feature” on page 128.
The other part of the technique uses remotely attached copy storage pools. Here we
concentrate on the HADR side and show how you can use it to prevent delay in restoring the
database, in case of a disaster.
When you plan to use HADR, you need to understand that HADR is a DB2 function outside of
Tivoli Storage Manager, and requires to be licensed separately. Your Tivoli Storage Manager
server is aware of the underlying DB2, but neither DB2 nor HADR is aware of the Tivoli
Storage Manager server.
As shown with Figure 5-11 on page 105, HADR establishes a standby copy of the primary
database, with the desired synchronous log synchronization mode for the Tivoli Storage
Manager database. The HADR standby database is in constant rollforward mode to stay
synchronized with the primary server.
DB2 HADR is a supported and well known feature of DB2. Typically the standby server is
initialized with a restore of a remote mounted full offline database backup. After they are
started, the primary and standby server databases synchronize using IP to ship database
updates.
The activity log1 messages, from a failover test in the lab, as shown with Figure 5-12 on
page 106, document how fast the standby server can restart in an HADR environment after
the primary server becomes unavailable.
In our lab environment, the Tivoli Storage Manager server had the following results:
Starts 36 seconds after shutdown.
Is unavailable only for 1 minute 22 seconds.
Although these numbers were generated under lab conditions, and your results might vary,
they show how fast the server can start by starting the warm standby server.
1
The activity log contains messages that are normally sent to the server console during server options.
The active log files record transactions that are in progress on the server.
DB2 HADR helps you to meet your business continuity needs and is part of advanced
electronic vaulting implementations.
Imagine hardware issues where the write order of database pages to the actual hardware
might be affected.
As a result of this, if the active logs are affected (such as a partial write to the storage), this
represents a single point of failure where the log data necessary to “reconcile” the database
for transaction consistency (crash recovery) is not available.
Having a mirror for the active log provides a higher degree of protection. The protection that is
provided results in the active log data being less likely to also be affected by a hardware write
issue.
For more information, see the following IBM Knowledge Center topics:
Protecting the database and infrastructure setup files:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/c_pro
tect_db-reclog_ovr.html
Protecting the active, archive, and archive failover logs:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_rec
ovprotect.html
To specify the file path and name for a volume history file, use multiple VOLUMEHISTORY
entries. Tivoli Storage Manager stores duplicate volume histories in all files that are specified
with VOLUMEHISTORY options. To find the required volume history information during a
database restore operation, the server tries to open volume history files in the order in which
the VOLUMEHISTORY entries occur in the server options file.
If configured, Disaster Recovery Manager saves a copy of the volume history file in its
disaster recovery plan file.
For more information, see the “Protecting the volume history file” topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_save_v
olhist.html
The following device configuration information is stored in the Tivoli Storage Manager
database and updated in the device configuration files:
Devices class definitions
Library definitions
Drive definitions
The device information must match the devices configured on the system where the restore
operation can be performed. You might have to edit those commands in an existing file so that
they match.
To specify the file path and name for a device configuration file, use the DEVCONFIG server
option. To specify more than one path and name, use multiple DEVCONIG entries. Tivoli
Storage Manager stores duplicate device configuration information in all the files that are
specified with DEVCONFIG options. To find the required device configuration information
during a database restore operation, the server tries to open device configuration files in the
order in which the DEVCONFIG entries occur in the server options file. If the server cannot
read a file, the server tries to open the next device configuration file.
To ensure the availability of device configuration information, use one or more of these steps:
Store at least one copy of the device configuration file offsite or on a disk separate from
the database.
Store a printout of the file offsite.
Store a copy of the file offsite with your database backups and volume history file.
Store a remote copy of the file, for example, on an NFS-mounted file system.
For more information, see the “Protecting the device configuration file” topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_save_d
evconfig.html
To ensure the availability of server options file, use one or more of these steps:
Store at least one copy of the server options file offsite or on a disk separate from the
database.
Store a printout of the file offsite.
Store a copy of the file offsite with your database backups and device configuration file.
Store a remote copy of the file, for example, on an NFS-mounted file system.
If configured, Disaster Recovery Manager automatically saves a copy of the server options
file in its disaster recovery plan file.
For more information, see the “Protecting the server options file” topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_save_o
pt_db_reclog_info.html
You can determine the following information from the recovery log:
Directory where the recovery log is located
Amount of disk space required
If you lose the recovery log, you lose the changes that were made since the last database
backup. If configured, Disaster Recovery Manager helps you save database and recovery log
information.
For more information, see the “Protecting information about the database and recovery logs”
topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_dblog_
info.html
The cert.kdb file includes the server’s public key, which allows the client to encrypt data. The
digital certificate file cannot be stored in the server database because the Global Security Kit
(GSKit) requires a separate file in a certain format. The cert256.arm file is generated by the
V6.3 server for distribution to the V6.3 clients.
Keep backup copies of the cert.kdb and cert256.arm files in a secure location. If both of the
original files and any copies are lost or corrupted, you can generate a new certificate file.
If client data object encryption is in use and the encryption key is not available, data cannot be
restored or retrieved under any circumstance. When using ENABLECLIENTENCRYPTKEY
for encryption, the encryption key is stored on the server database. This means that for
objects using this method, the server database must exist and have the proper values for the
objects for a proper restore operation. Ensure that you back up the server database
frequently to prevent data loss.
For more information, see the “Protecting the Secure Sockets Layer digital certificate file”
topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/c_protec
t_ssl_certkdb.html
You can use server-to-server communications to store copies of the recovery plan on a
remote target server, in addition to traditional disk-based files.
Storing recovery plan files on a target server provides the following advantages:
A central repository for recovery plan files
Automatic expiration of plan files
Query capabilities for displaying information about plan files and their contents
Fast retrieval of a recovery plan file if a disaster occurs
You can also store the recovery plan locally, printed, or on a disk.
For more information, see the “Protecting the disaster recovery plan” topic:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.srv.doc/t_protec
t_drp.html
5.4 Summary
Protecting your Tivoli Storage Manager server infrastructure involves more than a backup
copy of your Tivoli storage Manager database. Be sure to have multiple current copies of
infrastructure objects necessary for a database restore operation. Because the Tivoli Storage
Manager proprietary database has been replaced by a DB2 database, you now have even
more options to protect the database. Protecting only the database is not enough, however.
You must also have the infrastructure files if you need to restore your DB.
In this chapter, we offer several examples of how to solve these problems with various Tivoli
Storage Manager technologies and solutions.
First, in 6.2, “Disk-to-disk data protection solution using deduplication” on page 113 we
explain how to implement a Tivoli Storage Manager solution using data deduplication. We
show you how it works, and how important it is for your business.
Second, we describe in 6.3, “Data protection solution using node replication feature” on
page 128, how to enlarge the architecture described previously by adding new Tivoli Storage
Manager components and functionality.
Third, in 6.5, “Tivoli Storage Manager as a virtual appliance” on page 136 we discuss how you
can implement a fully virtualized data protection and restore solution based on Tivoli Storage
Manager products.
With an increasing data growth and an increased tape media handling, your business
requires a change where data deduplication is a technology that removes redundant data to
reduce the storage capacity requirement for retaining the data. When deduplication
technology is applied to data protection, it can provide a highly effective means for reducing
overall cost of a data protection solution.
Deduplication provides additional data reduction capabilities and can be used in addition to
the progressive incremental and compression capabilities already available in the product.
Figure 6-1 shows the data reduction features in Tivoli Storage Manager.
Figure 6-1 Overview of the data reduction features in Tivoli Storage Manager
This section describes the benefits of deduplication and provides guidance for how to make
effective use of the Tivoli Storage Manager deduplication feature as part of a well-designed
data protection solution. We also provide information for reference purposes and guidance
during the planning of other deployments of Tivoli Storage Manager. The architectures are not
intended to be a suitable configuration for all situations, and our intention is to describe the
benefits of deduplication and with guidance on, how to make effective use of the Tivoli
Storage Manager deduplication feature as part of a well-designed data protection solution.
Remember, this document does not replace the need to carefully plan and design the
elements of your own implementation of Tivoli Storage Manager.
Summary
The following list is a short summary of benefits that the Tivoli Storage Manager deduplication
solution provides:
Decreases the amount of storage capacity required to contain data growth. Remember,
cost reduction is the result of data reduction; deduplication is just one of several methods
that provides for data reduction (such as progressive incremental backup). The goal is
overall data reduction when all techniques are combined, rather than just on the
deduplication ratio.
Provides an affordable solution that can be expanded in the future and that delivers a rapid
return on investment (ROI) and a lower total cost of ownership (TCO) to justify the
investment.
Network bandwidth optimization (incremental, data deduplication) sends less data over
the network.
Reduces cost in labor with less or no tape media handling at all.
Figure 6-2 on page 115 illustrates the architecture that includes two primary storage pool
hierarchies. The first is a deduplicated sequential-file storage pool. Data remains in this pool
for its entire retention and is never allowed to migrate to another storage pool.
The remainder of this section summarizes key aspects of the Tivoli Storage Manager
architecture where the client backups are ingested over the LAN to two primary storage pool
destinations.
A deduplicated primary file-based disk storage pool is used for a majority of clients with a mix
of both client-side and server-side deduplication. Objects are stored in this pool for their entire
retention. A random disk-based primary storage pool is used as clients ingest into a random
disk storage pool which cannot use deduplication.
The following considerations help to determine which clients use this deduplicated storage
pool:
Fast restore times are needed without the delay that is associated with tape mounts, and
data spread across multiple tapes.
The data responds well to deduplication in terms of the amount of reduction.
Data is not encrypted on the client.
Having a second copy storage pool using disk (that can also be deduplicated) is another
option. Server-side deduplication is a two-step process:
1. Duplicate identification
2. Removal of the excess data during a subsequent data movement process such as
reclamation or migration
The second step can be done after a storage pool backup copy is created. See the
description of the deduprequiresbackup option at the IBM Tivoli Storage Manager Version
V7.1 page:
http://pic.dhe.ibm.com/infocenter/tsminfo/v7r1/index.jsp
When using server-side deduplication, schedule the storage pool backup process prior to the
reclamation processing to ensure that there is minimal overhead when copying the data. After
identify duplicate has run, the data is not deduplicated but it is redefined such that it can be
reconstructed and dehydrated during the subsequent data movement operation. Sufficient
time must be allotted for the scheduled storage pool backup to complete before the start of
the schedule for reclamation.
When using client-side deduplication, the storage pool backup processing always occurs after
data has been deduplicated. This requires deduplicated data to be reconstructed during the
copy (if the copy storage pool is not also deduplicated). The reconstruction processing can
result in storage pool backup processing, which is slower when compared with storage pool
backup processing of data that has not been deduplicated. For planning purposes, estimate
that the duration of storage pool backup will be doubled for data which is already
deduplicated.
A secondary copy is suggested, preferably using node replication, but not required. However,
with disk-to-disk data protection, the primary storage pool data remains on disk until it
expires. You can achieve a significant reduction of disk storage if the primary storage pool is
configured for deduplication.
Unlike compression, deduplication can take advantage of a pattern that occurs multiple times
within a collection of data. With compression, a single instance of a pattern is represented by
a smaller amount of data that is used to algorithmically re-create the original data pattern.
Compression cannot take advantage of data redundancy for patterns that recur throughout
the collection of data, and this significantly reduces the potential reduction capability.
However, compression can be combined with deduplication to take advantage of both
techniques and further reduce the required amount of data storage beyond what would be
required by using just one technique or the other.
The implementation of Tivoli Storage Manager deduplication applies only to the FILE device
class (sequential-access disk) storage pools, and can be used with primary, copy, or
active-data pools.
Unlike other backup products, Tivoli Storage Manager provides a substantial advantage in
data reduction through its incremental-forever technology. Combined with deduplication,
compression, exclusion of specified objects, and appropriate retention policies, Tivoli Storage
Manager provides highly effective data reduction.
Therefore, the business objectives should be clearly defined and understood when you
consider how to measure data reduction effectiveness. If reduction of storage and
infrastructure costs is the ultimate goal, the focus will be on overall data reduction
effectiveness, with data deduplication effectiveness as one component.
Server-side deduplication
With server-side deduplication, all processing of redundant data occurs on the Tivoli Storage
Manager server, after the data is backed up. Server-side deduplication is also called
target-side deduplication. The key characteristics of server-side deduplication are as follows:
Duplicate data is identified after backup data has been transferred to the storage pool
volume.
Both server-side and client-side deduplication can be used together within the same
storage pool, and reduce data across backups regardless of which type of deduplication is
used.
Duplicate identification processing must run regularly on the server, and consumes Tivoli
Storage Manager server processor and Tivoli Storage Manager database resources.
Storage pool data reduction is not realized until data from the deduplication storage pool is
moved to another storage pool volume, usually through a reclamation process, but can
also occur during a Tivoli Storage Manager MOVE DATA process.
Client-side deduplication
Client-side deduplication processes the redundant data during the backup process on the
host system where the source data is located. The net results of deduplication are virtually
the same as with server-side deduplication, except that the storage savings are realized
immediately, because only the unique data needs to be sent to the server in its entirety. Data
that is duplicate requires only a small signature to be sent to the Tivoli Storage Manager
server. Client-side duplication is especially effective when conserving bandwidth between the
Tivoli Storage Manager client and server is important.
For the backup-archive client (including VMware backup), a suggestion is to always configure
a cache when you use client-side deduplication. For applications that use the Tivoli Storage
Manager API, the deduplication cache should not be used because of the potential for backup
failures that are caused by the cache being out of sync with the Tivoli Storage Manager
server. If multiple, concurrent Tivoli Storage Manager client sessions are configured (such as
with a Tivoli Storage Manager for VMware vStorage backup server), a separate cache must
be configured for each session.
If the fastest restore performance from disk is a high priority, then restore performance
benchmarking should be done to determine whether the effects of deduplication can be
accommodated. Table 6-1 compares the restore performance of small and large object
workloads across several storage scenarios.
Tape Typically slower due to tape mounts Typically faster due to streaming
and seeks capabilities of modern tape drives
Deduplicated disk Faster than tape, slower than Slowest since data must be
non-deduplicated disk rehydrated, when compared to tape
which is fast for streaming large
objects that are not spread across
many tapes.
An optimal balance can be made between Tivoli Storage Manager deduplication and storage
appliance deduplication. Both techniques can be used in the same environment for separate
storage hierarchies or in separate Tivoli Storage Manager server instances. For example,
Tivoli Storage Manager client-side deduplication is an ideal choice for backing up remote
environments, either to a local Tivoli Storage Manager server or to a central data center. Tivoli
Alternatively, within a large data center, a separate Tivoli Storage Manager server can be
designated for backing up a critical subset of all hosts that use Tivoli Storage Manager
deduplication. The remaining hosts can back up to a separate Tivoli Storage Manager server
instance that uses a deduplicating appliance, such as ProtecTIER, for its primary storage
pool and also supports replication of the deduplicated data.
Tivoli Storage Manager deduplication should not be used in the same storage hierarchy as a
deduplicating appliance. For a deduplicating VTL, the Tivoli Storage Manager storage pool
data must be “rehydrated” before moving to the VTL (as with any tape device), and there is no
data reduction as a result of the Tivoli Storage Manager deduplication; rather it will be
re-deduplicated by the VTL. For a deduplicating NAS device, a FILE device type can be
created on the NAS. However, because the data is already deduplicated by Tivoli Storage
Manager, there is little to no additional data reduction possible by the NAS device.
6.2.11 Factors when deciding between Tivoli Storage Manager and appliance
deduplication
Scale, scope, and cost are the three major factors to consider when you are deciding which
deduplication technology to use.
Scale
The Tivoli Storage Manager deduplication technology is a scalable solution, which uses
software technology that heavily uses Tivoli Storage Manager database transactions. The
deduplication processing has an impact on daily server processes such as reclamation and
storage pool backup. For a specific Tivoli Storage Manager server hardware configuration (for
example, Tivoli Storage Manager database disk speed, processor and memory capability,
and storage pool device speeds), a practical guideline can be followed regarding the amount
of data that can be backed up using deduplication.
The practical guideline described is not built in to the product, and can vary based on the
capabilities of the hardware which is used. At the time of writing this book, the amount of
protected data is presented as a guideline with the purpose of keeping the size of the Tivoli
Storage Manager database below the recommended size of 4 TB. A database of this size
(4 TB) corresponds roughly to 400 TB of protected data (original data plus all retained
versions). There is no harm in occasionally exceeding the limit for daily ingest, which is
prescribed with the goal of allowing enough time each day for the Tivoli Storage Manager
Server maintenance tasks to run efficiently. Regularly exceeding the practical limit on daily
ingest for your specific hardware might impact the ability to achieve the maximum possible
amount of data reduction, or cause backup durations to run longer than you want.
Deduplication appliances have dedicated resources for deduplication processing and do not
have a direct impact on Tivoli Storage Manager server performance and scalability. If you
want to scale up a single Tivoli Storage Manager server instance as much as possible,
beyond approximately 400 TB of protected data (original data plus all retained versions), then
consider appliance deduplication. However, often a more cost-effective approach is to scale
The amount of data that can be ingested each day depends on the capability of the hardware
that is used for the Tivoli Storage Manager server. A primary consideration is the speed of the
disk used for the Tivoli Storage Manager database. To a lesser degree, the speed of the disk
used for the storage pool is also important. A high-end Tivoli Storage Manager server that
uses solid-state drives (SSD) for the database and serial-attached SCSI (SAS) disk drives for
the storage pool can handle up to 20 TB per day with server-side deduplication, and 30 TB
per day using client-side deduplication.
Scope
The scope of Tivoli Storage Manager deduplication is limited to a single Tivoli Storage
Manager server instance and more precisely within a Tivoli Storage Manager storage pool. A
single, shared deduplication appliance can provide deduplication across multiple Tivoli
Storage Manager servers. When Tivoli Storage Manager node replication is used in a
many-to-one architecture, such as with branch offices, the deduplicated storage pool on the
replication target can deduplicate across the data incoming from the multiple source servers.
Cost
Tivoli Storage Manager deduplication functionality is embedded in the product without an
additional software license cost. In fact Tivoli Storage Manager software license costs will
reduce when capacity-based licensing is in force because the capacity is calculated after
deduplication has occurred. An important consideration is that hardware resources must be
appropriately sized and configured. Anticipate extra expense when you plan a Tivoli Storage
Manager server configuration that will be used with deduplication. However, these additional
costs can easily be offset by the savings in disk storage. Also, the software license costs are
reduced when capacity-based pricing is in effect.
Deduplication appliances are priced for the performance and capability that they provide, and
generally are considered more expensive per gigabyte than the hardware requirements for
Tivoli Storage Manager native deduplication. A detailed cost comparison should be done to
determine the most cost-effective solution.
The following conditions lead to effective use of Tivoli Storage Manager deduplication:
Need for reduction of the disk space required for backup storage.
Need for remote backups over limited bandwidth connections.
Use of Tivoli Storage Manager node replication for disaster recovery across
geographically dispersed locations.
Total amount of backup data and ingested data per day are within the recommended limits
of less than 400 TB total and 30 TB per day for each Tivoli Storage Manager server
instance (to mitigate this you can deploy a second Tivoli Storage Manager server).
Either a disk-to-disk backup should be configured (where the final destination of backup
data is on a deduplicating disk storage pool) or data should reside in the FILE storage pool
The Tivoli Storage Manager internal database plays a central role in enabling deduplication
technology. Deduplication requires extra database capacity to be available. In addition, there
is a significant increase in the frequency of references to records in the database during many
Tivoli Storage Manager operations including backup, restore, duplicate identification, and
reclamation. These demands on the database require that the database disk storage be
capable of sustaining higher rates of I/O operations than are required without the use of
deduplication.
As a result, planning for resources used by the Tivoli Storage Manager database is critical for
a successful deduplication deployment.
The following web page (“Determining the impact of deduplication on Tivoli Storage Manager
server database and storage pools”) provides information for estimating the amount of disk
storage that will be required for your Tivoli Storage Manager database. It also has formulas for
estimating database size, based on the volume of data to be stored.
http://www.ibm.com/support/docview.wss?uid=swg21596944
As a simplified guideline for taking a rough estimate, you can plan for 10 GB of database
storage for every 1 TB of data that will be protected in deduplicated storage pools.
The estimation guidelines are approximate, because actual requirements depend on many
factors including those that cannot be predicted in advance (for example, a change in the data
backup rate, the exact amount of backup data, and other factors).
Tip: A suggestion is to begin with an active log size of 120 GB and monitor the space
usage and adjust the size of the active log as needed.
In general, systems based on SSD technology and SAS disk drives, and Fibre Channel
provide the best capabilities in terms of increased IOPS. Because the claims of disk
manufacturers are not always reliable, we suggest that you measure actual IOPS of a disk
system before implementing a new Tivoli Storage Manager database. The highest levels of
daily ingest (greater than 8 TB per day) will require the database to use solid-state storage.
Details about how to configure high performing disk storage are beyond the scope of this
document. Consider the following key points when you configure disk storage for the Tivoli
Storage Manager database:
The disk used for the Tivoli Storage Manager database should be configured according to
best practices for a transactional database.
Low-latency, enterprise-class disk devices or storage subsystems should be used for the
Tivoli Storage Manager database.
Disk devices or storage systems that are capable of a minimum of approximately 3000
IOPS are suggested for the Tivoli Storage Manager Database disk device. Consider an
extra 1000 IOPS per TB of daily ingested data (pre-deduplication). Lower-performing disk
devices can be used, but performance might not be optimal. See the Deduplication FAQs
for an example configuration:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli+S
torage+Manager/page/Deduplication+FAQ
Disk I/O should be distributed over as many disk devices and controllers as possible.
Tivoli Storage Manager database and logs should be configured on separate disk volumes
(LUNS), and should not share disk volumes with the Tivoli Storage Manager storage pool
or any other application or file system.
6.2.21 Processor
The use of deduplication requires additional processor resources on the Tivoli Storage
Manager server, particularly for performing the task of duplicate identification. Consider using
a minimum of at least eight (2.2 GHz or equivalent) processor cores in any Tivoli Storage
Manager server that is configured for deduplication.
A suggested minimum CPU requirement is the equivalent of one 2.2 GHz CPU core per
backup process with client-side deduplication. As an example, a system with a single-socket,
quad-core, 2.2 Ghz processor that is used 75% or less during the backup window can be a
good candidate to use client-side deduplication.
Typically a combination of both client-side and server-side data deduplication is the most
appropriate. Here are some further points to consider:
Server-side deduplication is a two-step process of duplicate data identification followed by
reclamation to remove the duplicate data. Client-side deduplication stores the data directly
in a deduplicated format, eliminating the need for the extra reclamation processing.
Deduplication on the client can be combined with compression to provide the largest
possible storage savings.
Client-side deduplication processing can increase backup durations. Expect increased
backup durations if network bandwidth is not restrictive. A doubling of backup durations is
a reasonable estimate when using client-side deduplication in an environment that is not
constrained by the network. In addition, if you will be creating a secondary copy using
storage pool backup where the copy storage pool is not using deduplication, the data
movement will take longer because of the extra processing required to reconstruct the
deduplicated data.
Client-side deduplication can outperform server-side deduplication with a high-performing
Tivoli Storage Manager server configuration and a low-latency network connection
between the clients and server. In addition, when combining deduplication with node
replication, client-side deduplication stores data on the Tivoli Storage Manager server in a
deduplicated state that is ready for immediate replication that will take advantage of the
DB2 high availability and disaster recovery with electronic vaulting solution is still valuable.
Now, with node replication and the disaster recovery functionality it brings, it is the future
Tivoli Storage Manager disaster recovery solution.
Node replication aims to help simplify disaster recovery from Tivoli Storage Manager with a
hot standby Tivoli Storage Manager server. In the ideal world, no requirement exists to
recover Tivoli Storage Manager at a remote site, recall the copy pool volumes, and perform
time-consuming restores from those tapes. Node replication allows for the backup data to be
replicated to a target server, meaning that it is ready and waiting when the disaster recovery
begins.
However, like all technologies, you need to be aware of certain aspects. Although there are
huge benefits to node replication, it does require careful configuration to ensure that you get
the best results from it.
If you have bandwidth constraints, consider reducing the amount of data sent across the wire.
You can work on this in several ways:
Consider transmitting deduplicated data. While the node replication process is working,
after deduplicates are identified, only the deduplicated chunks of data will be replicated,
resulting in potentially a huge savings.
Identify which of your servers has the most pressing recovery point objective (RTO) and
recovery time objective (RTO) at a disaster recovery. If bandwidth dictates that you can
replicate only a subset of your business data, ensure that you are using that bandwidth
intelligently and make sure it is backing up the most important subset. Remember this
aspect: Even if bandwidth is not a concern, replicate the critical nodes first.
Active versions of backup data generally are considered to be more important than the
inactive versions. By setting up an initial replication of only active data, you can avoid the
overloading of your network or servers had you replicated both active and inactive data.
Configure your replication rules to replicate active data first. After the initial replication of
the active data completes, configure the replication rules to replicate all versions of the
data. During the next scheduled replication, any new active versions, including all inactive
versions, will be replicated. The files that were active but are now inactive are not
replicated again.
If you can wait for the initial replication process to complete, finish configuring the nodes
and begin or schedule a replication. To ensure that the replication process proceeds as
expected, monitor the progress of the replication by issuing a QUERY PROCESS command.
Summary information in the output includes the amount of data replicated and the process
duration. Use the summary information statistics to determine whether the values of the
controlled test match the actual replication values. If the values match, continue the
replication. If the values do not match and you are concerned about the amount of time
that replication is taking, consider one of the replication methods described in the next
section (“Planning for incremental replication” on page 129).
Daily incremental replications typically do not require as much time to complete as the initial
replication. However, if the initial replication took four days to complete, and during that time
you were ingesting data daily, your first scheduled replication might require more time than
When should you run daily incremental replications? Do not run replication during client
backup windows because the data that is replicated is stored at the same time. Wait until after
the client backup completes and run replication during the window allotted for server
maintenance. You should also avoid running replication at the same time that expiration is
running.
Consider replication as a peer process for storage pool backups and schedule replication
accordingly. If you are replicating from deduplication enabled storage pools, run processes in
the following order (waiting for each to complete before proceeding to the next):
1. Identify duplicates: This process divides files into extents, potentially reducing the amount
of data to be sent to the target server when replication occurs.
2. Replication: Only file extents that do not exist on the target server are sent during
replication, reducing the required bandwidth and improving performance. Replication
performance improves as more deduplicated data is stored on the target server. When
more extents are stored on the target server, the probability that a duplicate will be found
for an extent increases.
3. Reclamation: Reclamation removes and links duplicated extents. Running reclamation
after replication completes improves performance because replication does not need
access additional linked extents.
For details about scheduling replication along with other server processes, see the following
Technote:
http://www-01.ibm.com/support/docview.wss?uid=swg21419733
When setting the MAXSESSIONS value, consider the number of logical and physical drives that
can be dedicated to the replication process. To access a sequential-access volume, Tivoli
Storage Manager uses a mount point and, if the device type is not FILE, a physical drive. The
number of available mount points and drives depends on the following factors:
Other Tivoli Storage Manager and system activity
The mount limits of the device classes for the sequential access storage pools that are
involved
Ensure that sufficient mount points and drives are available to allow node replication
processes to complete. Each replication session might need a mount point on the source
and target replication servers for storage pool volumes. If the device type is not FILE, each
session might also need a drive on both the source and target replication servers.
When setting a value for MAXSESSIONS on the REPLICATE NODE command, consider the
available bandwidth and the processor capacity of the source and target replication
servers.
To avoid replication sessions waiting for a mount point in deduplication enabled storage pools,
you may need to adjust the number of mount points allowed. A low mount limit might cause
replication sessions already holding mount points to needlessly wait for additional mount
points. For example, suppose that the mount limit is set to 10 and that all sessions already
have a metadata volume mounted and that each session needs to mount another volume. In
these circumstances, the sessions will stall while waiting for a mount point that is not
available. Other sessions or server processes that are using mount points in the device class
can also aggravate this problem.
To avoid having replication sessions wait for a mount point in deduplication-enabled storage
pools, use the following formula to set the mount limit in the device class to which the
deduplication enabled storage pool is assigned:
NUMOPENVOLSALLOWED * MAXSESSIONS
The server option NUMOPENVOLSALLOWED specifies the number of input FILE volumes in
a deduplicated storage pool that can be open at one time. The default value is 10. You specify
the MAXSESSIONS parameter on the REPLICATE NODE command. The default value for the
parameter is 10. If you use the defaults for both values, then update the mount limit in the
device class to 100. This value should be sufficient to allow replication and other sessions or
processes to acquire mount points. Be aware, however, that running other sessions and
processes that use mount points from the same device class as the replication process will
use mount points and may affect replication performance.
If your mount limit is already set to a value that is enough to satisfy the number of mount
points needed for replication then no action is necessary.
With an increasing need for high availability and business continuity, the Tivoli Storage
Manager replication is an effective tool for overall cost of a high availability data protection
solution. Node replication can offer a huge advantage over the traditional way of recovering
data during a disaster recovery, but be sure you understand that it must be configured
correctly to ensure you get the results you need and expect.
If a disaster occurs and the source replication server is temporarily unavailable, client nodes
can recover their data from the target replication server. If the source replication server
cannot be recovered, client nodes can be converted for backup operations on the target
replication server. Node replication is the process of incrementally copying or replicating client
node data from one Tivoli Storage Manager server to another for the purpose of disaster
recovery.
See the “Synchronization of source and target replication server after role change” technote:
http://www.ibm.com/support/docview.wss?uid=swg21628618
Your benefits
Tivoli Storage Manager node replication solution provides the following benefits:
A backup solution with disaster recovery protection that does not involve tape
Allows for a hot standby server at a remote location.
By combining with deduplication, it can reduce bandwidth that is required
Less tape media handling and decreased labor
High availability
Business continuity
Allows for a many-to-one replication solution where server from multiple branch offices
replicate to a common hub server
Figure 6-3 on page 133 illustrates the simulated architecture that includes two Tivoli Storage
Manager servers at separate sites and their primary storage pool hierarchies. Once a day,
these two sites simultaneously replicate their data to each other over a high-speed WAN
connection. Each server acts as both a backup target for clients that are local to the site, and
a replication target for the Tivoli Storage Manager server from the other site. Each site has
roughly the same storage hierarchy and uses the same domain names.
Solution description
How does node replication process work? Node replication is incrementally copying, or
replicating, data that belongs to backup-archive client nodes. Data is replicated from one
Tivoli Storage Manager server to another server. The server from which client node data is
replicated is the source replication server. The server to which client node data is replicated is
the target replication server.
Replication provides a tuning option that controls the number of simultaneous replication
sessions to use. Tuning this option can increase throughput to take advantage of available
bandwidth. Another option is to run more than one replication process, with each process
working on a different group of nodes.
A server can function as the source of replicated data for some client nodes and as the target
of replicated data for other client nodes. The purpose of replication is to maintain the same
level of files on the source and the target replication servers.
As part of replication processing, client node data that was deleted from the source
replication server is also deleted from the target replication server. When client node data is
replicated, only the data that is not on the target replication server is copied. Replication
processing involves the interaction of replication rules, states, and modes:
Deletes data on the target server that has been deleted on the source server
Can recover client data directly from the hot standby server (unlike virtual volumes)
Can be used with or without data deduplication
Can have multiple servers replicate to one target server
Only Tivoli Storage Manager V6.3 servers and later can be used for node replication.
However, data for client nodes that are running a client level V6.3 or earlier can be replicated.
Data that was stored on a Tivoli Storage Manager V6.2 or earlier server before it was
upgraded to V6.3 can be replicated.
All new data ingested each day into the deduplicated storage pool hierarchy is replicated
using Tivoli Storage Manager node replication to the other Tivoli Storage Manager server with
the following details:
Node replication occurs after duplicate identification has been completed so that
bandwidth savings from deduplication is possible.
Multiple identify duplicate data sessions are used to increase throughput.
Preferred practices are implemented that separate the activities of data ingestion and the
tasks of server data maintenance are divided into distinct windows each day. This includes
ordering the data maintenance tasks optimally to avoid resource contention.
Different disk storage is used for holding the Tivoli Storage Manager database and Tivoli
Storage Manager storage pools. A faster disk is used for the database; a less-expensive
slower disk is used for the storage pools.
The daily processing cycle we prefer for replication combined with deduplication is as follows:
1. Ingest new backup data.
2. Run identify duplicates if necessary (not already deduplicated by the clients). Note that the
identify duplicates processing can be overlapped with the backup ingest.
3. Run replicate node (Tivoli Storage Manager database backup can run in parallel).
4. Expire inventory.
5. Start reclamation.
That book describes how you can combine the advanced capabilities and features of Tivoli
Storage Manager with the powerful performance-enhancing and cost-reducing capabilities of
the ProtecTIER product. The book covers topics such as what to consider (in the planning
stage) to correctly implement an IBM ProtecTIER environment that is integrated with IBM
Tivoli Storage Manager.
This appliance-like deployment of Tivoli Storage Manager Data Protection solution design
makes sense to consider for a fully virtualized environment.
We show how Tivoli Storage Manager Server and data mover can run on the same machine,
and elaborate on the capability of such installation. This example does not describe the
highest performing environment possible; the same performance considerations still apply.
Here is the link to the latest optimizing performance for servers and clients publication:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20St
orage%20Manager/page/Optimizing%20performance%20for%20servers%20and%20clients
Rather, it describes one example environment to serve as an aid to Tivoli Storage Manager
data protection solution deployment in a fully virtualized environment.
The solution discussed here is possible only because Tivoli Storage Manager server and
Backup-Archive client are both supported in a virtual environment.
As a reminder, see the following address for the list of Tivoli Storage Manager supported
features in a virtual environment (IBM Tivoli Storage Manager guest support for Virtual
Machines and Virtualization):
http://www-01.ibm.com/support/docview.wss?uid=swg21239546
Notice that the design here focuses on VMware as a virtualization layer, but can also be done
using other virtualization providers as the web page indicates. Again, see that web page to be
sure that your virtualization layer is supported.
Virtualization has several features that Tivoli Storage Manager can use. So, having the data
protection solution integrated to the virtual environment makes sense. All challenges that
Tivoli Storage Manager addresses in a physical environment can also be addressed in a
virtual environment, as per the product virtualization support.
Virtual Infrastructure
Production TSM server
LAN 1Ge DR Server
EVA
One VM WARDEN TSM Node Replication Scorpio 2
with TSM server & DR TSM server
Ironthrone hosts our vBS
set of 20 VM to be
TSM
backedup
deduplication
SAN
Production Data
(datastores)
Notice in this example the possibility to interconnect the virtualized Tivoli Storage Manager
server to an external server, using node replication, thus to send the data from the virtualized
infrastructure, for disaster recovery purposes, for instance.
To learn more about the IBM Tivoli Storage Manager server and datamover running in a
Virtual Environment Sample Solution, see the following document:
http://ibm.co/1kZWbpZ
This document provides a sample architecture describing a consolidation of the Tivoli Storage
Manager components such as Tivoli Storage Manager server and Tivoli Storage Manager for
Virtual Environments on one virtual machine.
The virtualization allows you to easily deploy new Tivoli Storage Manager Server and Data
Protection for VMware data mover to protect more virtual machines.
By moving the virtual machine that holds the Tivoli Storage Manager server and Data
Protection for VMware data mover, you can distribute the backup load and optimize the data
transfer within the virtual machine, ESXi hosts and network components. In addition, you can
easily move the Tivoli Storage Manager server database volumes, storage pool volumes, to
another data store more evenly distribute the load generated by virtual machine data
protection.
Tivoli Storage Manager deduplication technology can be also enabled to reduce the amount
of data transferred, however it generates an extra load on the ESXi host that is hosting the
virtual machine where the Tivoli Storage Manager server and Data Protection for VMware
data mover is running. When enabling deduplication, see the following web page to
understand the required resources:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20St
orage%20Manager/page/Deduplication
Scope
This solution applies to every VMware based virtualization infrastructure.
The number of resources that are required by Tivoli Storage Manager Server and data mover
must be calculated to fit the environment.
Cost
When running Tivoli Storage Manager Server and Data Protection for VMware within a virtual
machine, no dedicated or separate hardware is required, therefore potential savings can be
obtained by avoiding the investment on dedicated and separate hardware.
As you would do in any other Tivoli Storage Manager installation, enabling the data
deduplication can save space in storage pools, thus reducing the license costs.
The number of virtual machines that can be protected with this type of solution depends on
the virtual machine disk size and the daily changed data rate, which gives the amount of data
to be ingested daily by the Tivoli Storage Manager server.
See the Tivoli Storage Manager wiki for documentation about data deduplication:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20St
orage%20Manager/page/Deduplication
Systems virtualization is one of the most important transformations today. Tivoli Storage
Manager understands this and offers a range of suitable products to assist you in this change.
This chapter explains the various ways of protecting data in a virtualized environment:
Using in-guest Tivoli Storage Manager agents as you would do for any physical machine
Using off-host Tivoli Storage Manager agents such as Tivoli Storage Manager
backup-archive client and its native “full-vm” backup feature
Using off-load backups (leverage hardware snapshot technology) such as Tivoli Storage
FlashCopy Manager
Regardless of the virtualization platform, and understanding the hypervisor platform, the
same technical challenges arise regarding data protection.
Virtual resource mobility
Hardware resource sharing
Figure 7-1 on page 143 shows the features that Tivoli Storage Manager provides to protect
your data in a virtualized environment, while answering these common challenges.
This subset of Tivoli Storage Manager tools can be used when protecting the virtual
environment. However, there are differences depending of the hypervisor distributor.
We describe Tivoli Storage Manager toolkit benefits throughout the following sections:
7.2, “Virtualization: VMware Data Protection solution” on page 144
7.3, “Virtualization: Hyper-V host-based backup” on page 158
7.4, “Virtualization: In-guest backup” on page 167
Note: The file-level backup is not supported for Linux virtual machines. Use the
in-guest Tivoli Storage Manager agent in addition to the full-vm backup method to have
the same level of protection.
Tivoli Storage Manager for Virtual Environments Data Protection for VMware
This option is an upgrade of the Tivoli Storage Manager backup-archive client off host
backup option described previously. When installing Tivoli Storage Manager for Virtual
Environments Data Protection for VMware, an enablement file upgrades the
backup-archive client, adding incremental and incremental forever backups capability. It
also brings a set of recovery features and a vSphere client plug-in.
Tivoli Storage Manager for Virtual Environments Data protection for VMware brings
recovery agents that allow extracting data to be restored from a unique backup, whatever
the type of restore operation. That is, even if the backup is a block level backup, by using
the Data Protection for VMware recovery agent you can restore files, volumes, and
machines. These operations can be done for both Linux and Windows virtual machines.
Tivoli Storage FlashCopy Manager for VMware
This option consists of making the snapshot at the data store level, using Tivoli Storage
FlashCopy Manager for VMware. This solution is based on hardware snapshot available at
the back-end disk subsystem.
The recovery possibilities are virtual disk, virtual machine, and data store. File level
restores are also provided by attaching a VDisk from a backup to an existing VM.
Note: You must know your recovery requirements before choosing your backup method.
The restore function is strongly influenced by initially choosing the right Tivoli Storage
Manager backup option.
Here are the identified recovery requests for our scenario environment:
Ability to recover single object from the Active directory database
Ability to recover virtual machine with consistent MS-SQL database
Ability to recover the SAP DB2 database and Oracle database at a specific point in time
Ability to recover mailboxes from Exchange Server
Ability to recover files from file server
Build a solution that can be part of a disaster recovery scenario
Note: This sample solution depicts only virtual resources because we are talking about
virtualization. In some cases, you might use a physical machine such as vStorage Backup
Server, backup proxy, or proxied Storage Agent. Then, add Tivoli Storage Manager for
SAN to products list, it allows you to transfer data from the back-end disks to Tivoli Storage
Manager storage hierarchy using LAN-free transport method. See “LAN-free data
movement” on page 41 for more information.
0 %
"#" !
$
! !
$ %&
!
&' !
($! ! !
''
Figure 7-2 Sample architecture of Tivoli Storage Manager products protecting VMware environment
For more details about FlashCopy and hardware support, see “Snapshot at the storage
hardware layer” on page 49.
ESXi
ESXi provides a virtualization platform for a large variety of open systems such as Windows
and Linux. ESXi uses a special file system to store virtual machines called Virtual Machine
File System (VMFS). VMFS is a clustered file system that can be accessed concurrently by
up to 32 ESXi hosts. A VMFS partition can be spread over 32 logical volumes called extents.
It can have a maximum size of 64 TB; the maximum file size on a VMFS partition is 2 TB.
On a VMFS partition, ESXi stores all the files of which a virtual machine consists:
Virtual machine configuration file (.vmx)
Virtual machine BIOS file (.nvram)
Virtual disk files or raw device mappings (.vmdk)
Virtual machine log files (.log)
The scope of a data store is a data center. The data store is uniquely named within the data
center. It is advisable to have a one-to-one relationship between LUNs and data stores. One
LUN should contain only one data store. One data store should not be spread over more than
one LUN.
vSphere management
For the management of vSphere, VMware provides two components:
vCenter server is a management suite to manage multiple ESXi hosts and to provide
advanced functions (such as on-line migrations, cloning, and high availability).
vSphere client is Windows software to connect to either single ESXi hosts or vCenter
Server that provides a graphical user interface.
Tivoli Storage Manager backup-archive client is able to protect the entire virtual machine
using vStorage API for Data Protection (VADP). Depending of how the virtual machine is set
up, you might need to use an alternate solution to protect the data that resides on this virtual
machine.
For instance, when using pRDM or bus sharing, the snapshot can no longer be taken, so you
need to use another data protection solution. pRDM and bus sharing can be required when
configuring clustered operating systems or clustered application within virtual machines.
Because all Tivoli Storage Manager agents used in this example are deduplication-aware,
see 3.3.3, “Server-side data deduplication” on page 57 to understand how the deduplication
works on both Tivoli Storage Manager client and Tivoli Storage Manager server side.
For more information about Tivoli Storage Manager server protection, see 5.1, “Protecting the
Tivoli Storage Manager server infrastructure” on page 94.
The disk storage pool is used to store the control files (CTL) sent by Tivoli Storage Manager
for Virtual Environment backups. Its size is roughly estimated to 1% of the amount of VMware
data to be protected.
The virtual tape library contains all the backups from the VMware data protection agents and
in-guest agents. The ProtecTier brings inline deduplication capabilities, thus allowing us to
save storage space without any space overhead. It also provides fast access to virtual
machine data blocks in case of file level restore, instant volume restore, and virtual machine
restore.
In this scenario, we don’t describe the possibility to replicate the data using node replication.
See 6.3, “Data protection solution using node replication feature” on page 128 for a detailed
description on how to implement this function.
Tivoli Storage Manager for Virtual Environment, because of its backup format (block based)
and strategy (single pass backup, multiple recovery level), requires to store its data on fast
access media to enable some of its recovery features. Table 7-1 shows the features available
depending on where the data resides. Tivoli Storage Manager for Virtual Environments Data
Protection for VMware supports tape and virtual tape library (VTL) devices with the
restrictions listed in this document:
http://www.ibm.com/support/docview.wss?uid=swg27021081
Table 7-1 Feature availability based on vStorage backup server and storage type
Storage and features Disk or file Virtual tape Physical tape
Tivoli Storage Manager for Virtual Environments: Data Protection for VMware
Tivoli Storage Manager for Virtual Environments Data Protection for VMware provides
protection of virtual machines, to provide a crash consistent recovery of the virtual machine
whatever the virtual machine content. It provides the full or incremental backup feature, at a
block level, based on Changed Block Tracking (CBT) feature that provides VMware through
VMware API Data Protection (VADP).
The Microsoft SQL Server and Microsoft Exchange hosted applications can be protected with
Tivoli Storage Manager for Virtual Environment. The feature to protect those two applications
natively in Tivoli Storage Manager for Virtual Environment is named self-contained application
protection. It enables the transaction log management without any in-guest agent installation.
The machine, which runs the full virtual machine backup, injects a code to be executed within
the virtual machine to clean up the logs after backup completes successfully.
The self-contained application protection must be carefully used, based on the recovery
granularity you want to have. If you need to recover to a point in time through transaction logs
(for Microsoft SQL Server), individual database (for Microsoft SQL Server), or individual
mailboxes (Exchange), plan to use the in-guest data protection agent. The same advice
applies when your applications are clustered.
There is another case where you can install it, typically when the virtual machine has pRDM
attached. These disks are skipped by Tivoli Storage Manager for Virtual Environment
full-virtual machine backups.
Tivoli Storage FlashCopy Manager for VMware optionally integrates with Tivoli Storage
Manager for Virtual Environments to store VMware image backups on Tivoli Storage Manager
server storage. With Tivoli Storage FlashCopy Manager for VMware, you can create off-host
backups for VMware virtual machines in a vSphere environment. Backups are generated at
the VMware data store level and restores can be done at the data store, virtual machine,
virtual disk or file level. A command-line interface (CLI), the Data Protection for VMware
command-line interface, and a vCenter GUI plug-in, the Data Protection for VMware vCenter
plug-in, are provided. Tivoli Storage FlashCopy Manager for VMware protects the virtual
infrastructure through automated data protection and recovery of your virtual machines. It
offers an easy-to-use interface to help you manage backup and recovery of virtual machines
in a multiple VMware ESX server environment. With Tivoli Storage FlashCopy Manager for
VMware, you can create off-host storage hardware snapshot backups from VMware VMs.
The following features are provided by Tivoli Storage FlashCopy Manager for VMware:
Backup, restore, and disaster recovery operations for virtual machines are streamlined
and simplified.
File system consistent backups are provided and the backup window of the virtual
machine is reduced by using hardware snapshots of complete data stores in combination
with offloaded backups to Tivoli Storage Manager.
The combination of Tivoli FlashCopy Manager for VMware and Tivoli Storage Manager for
Virtual Environment fits when planning for a disaster recovery scenario.
For the challenges, we describe how the data is backed up, how you can recover, and explain
their interaction with the disaster recovery plan when required.
We apply the same strategy to protect the virtual machine hosting the File Server. This allow
us to save backup time, the amount of data to be processed, and Virtual machine (ESXi
hosts) computing resources as compared to a traditional in-guest backup-archive client
Tivoli Storage Manager for Virtual Environments Data Protection for VMware is also the
solution we use to protect the virtual machines with custom applications. In addition, we use a
custom pre-freeze script to quiesce the I/O before the snapshot occurs. This ensures the
application consistency. We take advantage of the instant volume recovery feature from the
Data Protection for VMware recovery agent, and in the meantime provide the ability restore
the entire machine in case of disaster recovery. For such a custom application, having the
entire virtual machine snapshot makes sense so that you keep all possible (and most likely
poorly identified) dependencies between the operating system and this custom application.
For a virtual machine that runs Microsoft Active Directory, we use Tivoli Storage Manager
backup-archive client because it allows us to recover individual active directory objects from
accidental corruption or deletion. This operation is then online, without impacting any other
Active Directory activities. To learn more about restoring active directory object, see the
“Restore Windows individual Active Directory objects” topic in the IBM Tivoli Storage
Manager for Windows Backup-Archive Clients Installation and User’s Guide.
In our scenario, based on the assumption saying that Microsoft SQL Server recovery includes
the entire virtual machine restore, we use Tivoli Storage Manager for Virtual Environments
Data Protection for VMware snapshots, with the self-contained application protection feature
enabled. We assume in this case that running version are among the supported ones:
Microsoft SQL Server 2008, Microsoft SQL Server 2008 R2, Microsoft SQL Server 2012.
Self-contained application protection has a mechanism to truncate the logs once the backup
ends, therefore the application never runs out of log space. This avoids installing and
maintaining in guest data protection for the application agent. However, in the recovery of
individual databases, log files are not possible without restoring the entire virtual machine.
You might have a recovery strategy stating that you need to be able to restore partial
Microsoft SQL databases, or you must provide a point-in-time recovery capability. If so, you
might choose to implement Tivoli Data Protection for Microsoft SQL server instead. If you
want to recover individual Microsoft SQL databases from a VM backup, you must use both
Data Protection for VMware and the IBM Tivoli Storage Manager for Databases: Data
Protection for Microsoft SQL Server V7.1.
These statements apply to Microsoft Exchange. See IBM Tivoli Storage Manager for Virtual
Environments: Data Protection for VMware User’s Guide for more details about self-contained
application protection:
http://www.ibm.com/support/knowledgecenter/api/content/SSGSG7_7.1.0/com.ibm.itsm.v
e.doc/b_ve_user_guide.pdf
Next, we discuss Microsoft Exchange Cluster defined over multiple virtual machines. In our
case these virtual machines have physical raw device mapping (pRDM) disks. To protect this
cluster, we implement Tivoli Storage FlashCopy Manager within the virtual machines.
Because we know that we have a disaster recovery strategy, we set up Tivoli Storage
FlashCopy Manager with the Tivoli Storage Manager support instead of stand-alone mode.
Indeed stand-alone mode creates VSS-based snapshots and manages them on the server
that is performing the backup. The VSS backup, using Microsoft Volume Shadow Copy
Service, produces an online snapshot (point-in-time consistent copy) of Exchange. The Tivoli
Storage Manager support mode means that backups can be stored either locally or on the
Tivoli Storage Manager server. The Tivoli Storage FlashCopy Manager software protects and
manages Exchange by using a Tivoli Storage Manager server. With Tivoli Storage Manager,
you can also offload your backups to another computer and move the data to the Tivoli
Storage Manager server.
By installing and using Tivoli Storage FlashCopy Manager in the virtual machines to protect
our Microsoft Exchange cluster, we are able to protect the data stored on pRDM disks, which
are not supported by VMware snapshot operations. We also meet the mailbox's recovery
requirement. Even better, we can restore individual mail using the mailbox restore browser
utility, this either using local snapshot copy (from FlashCopy) or using offloaded copies stored
on the Tivoli Storage Manager server.
The Linux virtual machine hosting the SAP DB2 database uses IBM System Storage N series
disks, which are assigned to ESXi hosts as Fibre Channel LUNs. These LUNs are dedicated
to the Linux in a pRDM mode to fulfill performance requirements. We use Tivoli Storage
FlashCopy Manager for the SAP DB2 database protection. DB2 backups to Tivoli Storage
Manager can be done with either Tivoli Storage Manager for Enterprise Resource Planning
environment or the Tivoli Storage Manager agent that is included with DB2.
Tivoli Storage FlashCopy Manager either initiates a tape backup from the snapshot target set
when the snapshot completes or backs up a previously generated snapshot. We use the
second mode to offload snapshots onto Tivoli Storage Manager server, on a regular basis for
the disaster recovery purpose, relying on IBM System Storage N series to restore SAP DB2
database from snapshot when needed. If an older backup is needed, we restore the backup
from the Tivoli Storage Manager server.
To protect the Oracle database, we use Oracle Recovery Manager (RMAN); Data Protection
for Oracle provides an interface between Oracle Media Management API calls and Tivoli
Storage Manager API routines. RMAN provides consistent and secure backup, restore, and
recovery performance for Oracle databases. While the Oracle RMAN initiates a backup or
restore, Data Protection for Oracle acts as the interface to the Tivoli Storage Manager server.
The Tivoli Storage Manager server then applies our storage management policies to the data.
Data Protection for Oracle implements the Oracle defined Media Management application
program interface (SBTAPI) 2.0.
Thereby we meet the restore requirements and send the data out of the production
environment to the Tivoli Storage Manager server, so we meet the disaster recovery
objectives.
On top of all these backup methodologies we implement Tivoli Storage FlashCopy Manager
for VMware, to protect all the data that resides on VMware data stores in an easy and efficient
way, to optimize as much as possible the Disaster Recovery. Tivoli Storage FlashCopy
Manager for VMware allows backing up all data stored on the VMware data stores, and
excludes those on pRDM disks (toleration mode only, explicitly excluded). This allows a fast
and easy recovery.
Using Tivoli Storage FlashCopy Manager backup methodology, restoring all virtual machines
with their configuration and layout is streamlined, except for those with pRDM.
For the pRDM content, because we are using Tivoli Storage FlashCopy Manager in guest as
a data protection solution, we need to separately restore that data from the hardware
snapshots, either from the back-end disks or from the Tivoli Storage Manager server because
it must be offloaded to a Tivoli Storage Manager managed storage.
On the Tivoli Storage Manager server side, all data from our backup strategy is duplicated
using Tivoli Storage Manager Server node replication to another Tivoli Storage Manager
server, to make them available in case of disaster. In addition, the data is also available for
any purpose other than disaster recovery. For more information about node replication, see
6.3, “Data protection solution using node replication feature” on page 128.
Backup
In these scenarios, we examine various components and data protection methods that fit to
the data type to be protected. Data protection methods hereafter consist of backup, recovery,
and backup frequency (controlled by schedules).
Virtual Environment backups
Incremental forever backup uses Tivoli Storage Manager grouping technology to create
synthetic-full recovery points by combining required blocks from previous backups with the
changes from daily incremental backups. An incremental forever backup strategy
minimizes backup windows while providing faster recovery of your data. Data Protection
for VMware provides a backup strategy called incremental forever. Rather than scheduling
weekly (periodic) full backups, this backup solution requires only one initial full backup.
After, an ongoing (forever) sequence of incremental backups occurs.
The filtering options available (exclude.vmdisk) are used to exclude the virtual machine or
virtual machine disks that Tivoli Storage Manager for virtual environment is not supposed
to protect, because they are protected by another in-guest solution. We also use the
performance options (vmmaxparallel, vmlimitperdatastore, and vmlimitperhost) to
decrease the backup window. To save network bandwidth, we enable both compression
and deduplication on Tivoli Storage Manager client side.
Virtual machine running Microsoft Active Directory
Active directory objects are part of the systemstate backup. Every day we run an
incremental backup of the machine, including the systemstate.
DB2 redologs and datafiles backup
Tivoli Storage Manager agent will be installed in the virtual machine to allow DB2
database to send its DB2 logs by using the Tivoli Storage Manager Backup-Archive Client
API. Those logs will be sent directly on the Tivoli Storage Manager server.
Using Tivoli Storage FlashCopy Manager with IBM System Storage N series and NetApp
storage system, we protect the DB2 databases.
Because the IBM System Storage N series and NetApp storage systems can be attached
to hosts using SAN, the following tasks can be completed:
– Use Tivoli Storage FlashCopy Manager to complete snapshots.
– Offload the Tivoli Storage Manager backup of the FlashCopy data to an auxiliary host.
– Use the FlashCopy data that backs up DB2, SAP with DB2, to restore data.
– Create database clones.
We set up policies within the IBM System Storage N series storage systems that delete
snapshots created with Tivoli Storage FlashCopy Manager. For this purpose, Tivoli
Storage FlashCopy Manager periodically checks whether backups on the storage system
remain valid. This checking process is referred to as reconciliation.
Exchange backup
In our case the Exchange server databases are in a Database Availability Group (DAG)
environment, and we back up the databases to a common node, therefore we set up a
Database Availability Group node name (DAGNODE).
For Microsoft Exchange Server databases in a Database Availability Group (DAG)
environment, several online copies of a database are maintained for high availability. To
reduce the number of database backups that are made, we set up to back up the copies of
Restore
The restore scenarios depend on how the backup is done.
Using Tivoli Storage Manager for Virtual Environment, we can recover data from the full
virtual machine backup and, when necessary, extract either files or an entire volume. For the
entire volume restore we can use the instant restore volume method if we need to bring up
the service as fast as possible. In that case, the volume is accessible for everyone after a few
minutes, while a background process runs to restore all the content. Or, we can restore only
the specific disk by using the dsmc restore vm command, and the following option:
:vmdk=disklabel
Depending of the task you want to initiate, you can chose among vStorage Backup Server,
virtual machine, or the Data Protection for VMware vCenter plug-in as Table 7-2 describes.
Table 7-2 Availability of Data Protection for VMware operation depending of the location
Location vStorage Backup Virtual machine vCenter plug-in
task Server
Instant Volume Not available Using Data Protection for Not available
restore VMware recovery agent
File Level restore Using Data Protection Using Data Protection for Not available
for VMware recovery VMware recovery agent
agent
Using Tivoli Storage FlashCopy Manager, we take advantage of the near-instant restore
feature, that allows you to recover the data from the local snapshot as well as use the
backups stored on the Tivoli Storage Manager server.
With the Data Protection for Oracle and RMAN, the table, table space, logs, or any other
point-in-time recovery is satisfied. When needed, the data is sent back from Tivoli Storage
Manager to the virtual machine using RMAN commands, via the Data Protection for Oracle
API.
When restoring data with Tivoli Storage FlashCopy Manager for VMware, the following
options are available:
Restore to either the original data store or to a different data store at the virtual machine
level.
Restore a single virtual disk to the original location or a different virtual machine. This
restore occurs by attaching a virtual disk from within a backup to a target virtual machine.
Attach single virtual disks in the backup to the original or a different virtual machine to
enable file level restore operations.
Restore one or more data stores by using the instant_restore command or from the
GUI. You can select which data store to restore and check the dependencies of virtual
machines that are stored in this data store to other data stores in the environment. If a
distributed virtual machine is present, a list of extra data stores are identified and you can
select more data stores to restore for consistency.
Schedules
To have a central management of the schedules, we use the Tivoli Storage Manager server
central scheduler. All the Tivoli Storage Manager agents deployed in our solution can be
triggered by the Tivoli Storage Manager server scheduler, then take advantage of the built-in
event status and other reporting features.
For Tivoli Storage FlashCopy Manager tasks (Microsoft Exchange and SAP DB2 in our case)
and Data Protection for Oracle, we create Tivoli Storage Manager schedules, managed by the
Tivoli Storage Manager server and not using local scheduler agent, like Windows Task
Scheduler, even if it is supported for Tivoli Storage FlashCopy Manager. Thus we keep control
and track all events from a single source, the Tivoli Storage Manager server.
For Tivoli Storage Manager for VMware and in-guest backup-archive client, the schedule plan
is simplified. We need to create only one type of schedule since we use an incremental
forever backup strategy.
For the Tivoli Storage Manager for VMware features, like Tivoli Storage Manager for Virtual
Environments Data Protection for VMware and Tivoli Storage FlashCopy Manager for
VMware, you can control, monitor, and restart the schedules using the vCenter plug-in as
shown in Figure 7-3 on page 157. However, notice that only those created with the plug-in
can be managed by the plug-in. If some tasks are created directly on the Tivoli Storage
Manager server, they will not be available within the vCenter plug-in.
In our case, we always control the event, whatever the type of backup, from the Tivoli Storage
Manager server.
The following schedules are defined to the Tivoli Storage Manager server:
Oracle archivelogs backups, period=2 perunits=hours dayofweek=any weekofmonth=any
action=command obj=”<backup archivelog>”
Oracle incremental backups, dayofweek=mon,tue,wed,thur,fri,sat weekofmonth=any
action=command obj=”<backup incremental level 1>”
Oracle full backups, dayofweek=sun weekofmonth=any action=command obj=”<backup
incremental level 0>”
FlashCopy incremental backups, dayofweek=mon,tue,wed,thur,fri,sat action=command
obj=”<flashcopy incremental>”
FlashCopy full backups, dayofweek=sun weekofmonth=sun action=command
obj=”<flashcopy full>”
Tivoli Storage Manager for Virtual Environment backups; period=1 perunits=day
dayofweek=any weekofmonth=any action=backup subaction=vm
option=”-mode=incremental”
In guest backup-archive client backup, period=1 perunits=day dayofweek=any
weekofmonth=any action=incremental
Tivoli Storage FlashCopy Manager for VMware requires the installation of a Linux vStorage
Backup Server.
The Hyper-V cluster can service different kinds of VM operating systems and applications.
The data that need protection may require more levels of protection in order to satisfy both
disaster recovery capabilities and to make sure that the data backup is consistent across the
infrastructure. Also consider the level of restore granularity.
The hypervisor layer is an interface to the hardware. It is isolated from all other layers, such as
VMs and base operating systems. Hypervisor is done by Windows operating system, starting
with Windows 2008, which have a Hyper-V role installed (at this time Windows 2008R2,
Windows 2012). This is known as the parent partition. The guest VM operating systems,
known as the child partitions, can be Windows operating system running 64 bit, 32 bit, or
Linux.
Figure 7-4 illustrates a Hyper-V clustered environment running different kinds of VMs. It could
be Microsoft SQL servers running on each Hyper-V node or a file server and print services.
The data is stored on a shared storage system. You must have only one Tivoli Storage
Manager client installed on each physical machine, that is the base Hyper-V host system, not
on the virtual machines.
Hyper-V has its own VSS writer which means that you have the ability to get application
consistent backups of virtual machines from the host level. Some conditions must be met to
ensure this works as expected. Look up the appropriate documentation on the VSS writer of
the application used.
The basic operation is that when the Hyper-V VSS writer is triggered by Tivoli Storage
Manager, it propagates the requests to the internal Hyper-V VSS requestor of the VM. The
VM then in turn notifies all of its registered VSS writers (Exchange, SQL-Server, and more)
that a backup is taking place.
The most common need for application-consistent backups is the use of database servers.
Even though Microsoft SQL and Exchange server have their own VSS writer, this VSS
mechanism will provide a crash consistent backup only. For proper database protection and
advanced recovery possibilities, Tivoli Data Protection for Application within the virtual
machine should be used.
If no VSS writer is available for the application you are running the backup will be crash
consistent from which the application might or might not recover. In this situation, the advice
is to install the Tivoli Storage Manager backup-archive client on the VM and maybe run a pre-
and post-script to ensure application consistency.
Not all situations require an application-consistent backup. File and print servers are fine with
crash consistent and possibly inconsistent backups, therefore full virtual machine backup is
fine, with or without VSS enabled.
For the VMs that do not support VSS, such as Linux and Windows 2000, the saved state is
used. Hyper-V will make a crash-consistent snapshot. The saved state is also used if the
backup integration service is not installed on the VM. If VSS writer in the hypervisor is unable
to communicate with VSS through integration components, its default behavior is to put the
VM into a saved state before a snapshot of host volumes are taken for backup.
Note: Pass-through and iSCSI disks are not visible to the base operating system
configured with the Hyper-V role. Therefore, the data on these type of disks must be
backed up by the Tivoli Storage Manager that is installed on the VM.
Client deduplication can be enabled to reduce the data transferred to the Tivoli Storage
Manager server. The default value (CLIENTDEDUPTXNLIMIT) for the maximum file size to
be deduplicated is 300 GB, which should be greater than the size of the files sent to the Tivoli
Storage Manager server. The policy used for the data must have a destination storage pool
that is deduplication-enabled. See “Client-side data deduplication” on page 40.
Backup
Tivoli Storage Manager backs up or restores all files that are associated with a guest virtual
machine. Tivoli Storage Manager server uses the file grouping to keep all files that comprise a
virtual machine together as one virtual entity. Normal versioning in the central configured
policies are used. Figure 7-5 shows the GUI for backup the VMs.
Figure 7-5 Tivoli Storage Manager client backup GUI interface with Hyper-V integration
The virtual machines must be backed up to the same client node in Tivoli Storage Manager
using the ASNODE parameter. In that way, all VM backups are kept under the same node name
and it is easier to manage if the VMs move to another physical Hyper-V node in the cluster.
Backing up multiple virtual machines onto the same Tivoli Storage Manager target node
streamlines the management of backup and recovery operation. It simplifies the scheduling
tasks because only one event must be created. That simplifies the recovery and the backup
operations because virtual machines are stored within a target node.
Backing up a file server is straight forward because the data is consistent with a normal VSS
snapshot. Because this is a full backup of the VM data, the size of the disks and the backup
time must be considered.
For VMs that do not have applications that are installed, such as a print server service, using
the VSS snapshot might be the only solution that you need. In fact, you might consider
running a new full backup only when the server is software-updated. Otherwise, the crash
consistent backup is adequate.
For application servers like Microsoft SQL or Microsoft Exchange we are able to take an
application consistent VSS snapshot. It might be adequate to only backup the database at the
host level once a day. If you need to make several backups during the day to lower the RPO
you probably need to back up the database and its transaction logs on the VM using Tivoli
Storage Manager for databases software.
To use this backup combination, see “In-guest backup capability” on page 164.
Figure 7-7 Tivoli Storage Manager client restore GUI interface with Hyper-V integration
The VM is restored to the cluster node that is listed as the current owner. It should be restored
only by a backup-archive client installed on the cluster node that is the current owner of the
VM. To determine the VM configurations of the cluster, use Windows and select Server
Manager Features Failover Cluster Manager Cluster name Services and
Applications. Use the Services and Applications window to change which node is the owner
of the VM before you restore the VM.
When using the command line to restore all the Hyper-V, backup objects are displayed on the
first page. After selecting the objects that you want to restore, the second page lists the
backup details for the active and inactive objects.
When a restore operation is complete, the existing virtual machine is stopped (if it is running)
and all existing files (for example, VHD) are deleted. When the backup snapshot is restored,
the virtual machine that existed when the files were backed up is re-created. This includes
any Hyper-V snapshots that existed when the files were backed up.
File level restore granularity is possible only if you have backed up the files from the VM with
the Tivoli Storage Manager backup-archive client installed. This might be important for file
servers where you need this kind of restore granularity.
For servers like print servers where a crash-consistent backup is adequate, the restore is
quick to get the VM running again from the host-based snapshot backup. In this case, you
would not install a Tivoli Storage Manager backup-archive client because you would not need
to restore single files or directories.
With application servers like Microsoft SQL servers, more options need to be considered.
These applications tend to have many transactions running during time. It is crucial that the
state of the data is consistent and can be regenerated to a specific point in time. For these
If the guest OS of the VM is supported by Tivoli Storage Manager, we are able to back up the
data on a file level basis. If the VM is running an application that is supported by a Tivoli
Storage Manager Data Protection client, the data can be backed up also with this client.
Backup data is sent across the LAN from the VM to the Tivoli Storage Manager server
storage hierarchy, as shown in Figure 7-8.
If you are able to complete a full backup of all VMs every day and the RPO is adequate, do
only the host-based VMFull backup. In most cases, this is not an option and we suggest that
you combine the two options (host-based and in-guest backup) to get as much flexibility into
the solution as you need.
Figure 7-9 illustrates the solution design. Tivoli Storage FlashCopy Manager can be used as
a stand-alone product or integrate with a Tivoli Storage Manager server for advanced
archiving, versioning, and scheduling capabilities. This specific solution focuses on the
stand-alone installation for managing local snapshots.
The IBM Storwize family products and SAN Volume Controller systems provide internal
storage as well as external storage virtualization to make it possible to integrate with and
manage existing heterogeneous storage, along with the internal storage from a single
interface.
Combining the capabilities of Tivoli Storage Manager can protect your complete Hyper-V
environment.
The -vmlist="vm1,vm2" option can filter which VMs to include in the backup.
In-guest scheduling
The Tivoli Storage Manager client scheduler is configured on the VM. All schedules that
the VM should run (file-level or database backup) is configured on the Tivoli Storage
Manager server.
Scheduling Tivoli Storage FlashCopy Manager copies
Tivoli Storage FlashCopy Manager has its own scheduler that you can use. Or if you want
to centralize the initiation and reporting on the schedules, you can use the Tivoli Storage
Manager client scheduler to automate the copies.
Solution description
In-guest backups are started in the virtual machine and provide the following features:
Virtual machines (without applications)
Virtual machines with MS-SQL server and Exchange Server
Virtual machines with applications such as Oracle or DB2
Virtual machines with physical raw disks
Virtual machines with any custom applications
If the guest OS of the VM is supported by Tivoli Storage Manager, we are able to back up the
data on a file level basis. If the VM is running an application that is supported by a Tivoli
Storage Manager Data Protection client, the data can be backed up with this client also.
Backup data is send across the LAN from the VM to the Tivoli Storage Manager server
storage hierarchy.
Backup
The same product backup capabilities that apply to a physical host, also apply to a virtual
machine. All the standard function of the Tivoli Storage Manager backup-archive client can be
used, such as progressive incremental forever, journal-based, and image backup. The VSS
support of Microsoft operating systems is also supported.
If the virtual machine is part of a Microsoft Shadow Copy Services solution, we are able to
support the disks in resource groups for backup.
You can use a server-side deduplication storage pool as a destination for your backup data to
get the data reduction in the Tivoli Storage Manager storage pool.
Restore granularity
We can restore single files, directories, system state, or image level backups. The standard
Tivoli Storage Manager backup-archive client GUI or command line can be used.
Although these steps seem easy, several factors must be considered to be sure the process
is working on every type of virtual machine you have in your environment.
System backup and recovery solutions can be used to make the recovery process easier and
more robust against change. One example is the Network Installation Manager (NIM) server
in AIX. The NIM server can keep mksysb backups of the AIX system of a VM and make the
recovery procedure easier. For the given hypervisor, other solutions might be considered to
incorporate it to a complete disaster recovery plan of the IT infrastructure.
Figure 7-11 on page 170 shows how data is kept on its own set of LUNs separate from the
LUNs that the hypervisor manages, otherwise FlashCopy Manager will not be recognized as
the FlashCopy enabled disks.
Scheduling
In a virtual environment, you can easily have 10 - 50 VMs running on a single physical host.
Each VM will have its own Tivoli Storage Manager client scheduler running to trigger the daily
incremental backup job to back up the file system, services database, and applications.
The backup window and randomization used by Tivoli Storage Manager scheduler can help
you distribute the backup load on the physical host to minimize the resources used at a
specific time.
Another add-on option is to use journal-based backup. It uses a change journal maintained by
the Tivoli Storage Manager journal service process running in the Tivoli Storage Manager
backup-archive client. Because we do not need to use resources to scan the entire file
system of the VM, it runs much quicker and fewer less CPU cycles to complete the backup.
On Linux, journal-based backup is supported on Ext2, Ext3, Ext4; XFS, ReiserFS, JFS, VxFS,
and NSS, and for a local file system shared through NFS. GPFS is not supported for
journal-based backups.
As you see from the matrix in Chapter 4, “Tivoli Storage Manager challenge matrix” on
page 81, the online backups can help you meet the recovery time objective (RTO) and
recovery point objective (RPO). To identify the recovery requirements, answer the following
questions:
How critical are your databases and applications to your business?
Is it acceptable to lose any database or application activity? How much time and changes
are acceptable to lose?
What is the maximum acceptable downtime for your databases and applications?
Is it necessary to perform a restore right up to a point of failure?
What is the backup window? How many resources are available for the backup?
Must the database or application be available only during commercial hours or working
days, or does it have to operate on a 24x7x365 basis?
Are there peak periods of database and application utilization? How frequently does the
data change during peak and nonpeak periods?
Do you have two or more databases or applications that must maintain a logical
consistency?
What are the legal requirements for your backup routines? Are you required to retain
backups for a long period of time (more than your normal retention period)?
The answers to these questions can help you determine a backup strategy.
Tivoli Storage Manager for Mail and Tivoli Storage Manager for Databases enable data
protection of your mail and databases while they are online. They automate data protection,
enable hot backups without shutting down the application and improve data restore
performance.
7.5.1 Tivoli Storage Manager for Databases: Data Protection for Oracle
The Tivoli Data Protection for Oracle application client provides an integrated solution for
performing backup and restore operations on Oracle databases. It is a client application that
provides backup of online databases and restore of databases to the original or different
location.
IBM Tivoli Storage Manager for Databases helps protect Oracle Databases data no matter
where it is stored. You can continue running primary applications on your database servers,
while they back up and restore data to and from offline storage using automated tasks,
utilities and interfaces. This software performs online, consistent, and centralized backups to
help you avoid downtime, protect vital enterprise data, and minimize operational costs.
RMAN provides the interface to the Oracle database and the functions for backup, restore,
and recovery. It does not provide any storage management capabilities and must be
integrated with other storage management products such as Tivoli Storage Manager to
provide a complete enterprise wide storage management solution.
Solution architecture
This section describes the solution components for using Tivoli Data Protection for Oracle to
protect your Oracle database.
Databases
A database consists of one or more logical units called table spaces. On the physical layer, the
database has data files, control files, and optionally a password file.
Table spaces
When a database is created, a data dictionary is created in the system table space. Although
It is not required to create additional table space, additional table spaces are suggested for
user data. The data dictionary is critical to the operation of the database because it records,
verifies, and conducts ongoing work. The system table space is always online and cannot be
taken offline because the data dictionary must always be available to Oracle.
Data files
A table space is a logical grouping of data storage called data files. A data file can be a file or
a raw device. A table space can have a mixture of both files and raw devices as data files.
Backup can be performed on a logical level (database and table spaces) or on a physical level
(data files).
Control files
Control files keep information about the physical structure of the database and log files. They
are commonly multiplexed and are defined in the initialization parameter files. Keep at least
three copies on separate disks. Just like the data dictionary, it is important to make backups of
your control files regularly. Losing the control files makes recovery much more complicated.
RMAN command
The RMAN command is the database administrator’s interface to RMAN. It invokes a
command-line interface that provides a scripting language (operating system independent) for
performing backup and recovery operations. RMAN can be executed either interactively,
where a command prompt is displayed and additional RMAN commands entered, or in batch
mode, where an RMAN command file is executed.
Target database
The target database is the Oracle database instance on which RMAN executes specified
backup, restore, and recovery actions. When the RMAN command is run, it connects to the
target database. The target database is specified by using RMAN parameters.
Communication channel
RMAN can perform backup and restore functions to either local disk or to external media
management products such as an Tivoli Storage Manager server through the libobk.a library
provided by Tivoli Data Protection for Oracle. These I/O operations are performed over a
communication channel that defines the device to be used for the operation. The channel is
used by RMAN to send or receive backup data to and from the I/O device. For backup and
restore operations, you must allocate a channel before the operation is performed. A channel
corresponds to a single device. With the Tivoli Data Protection for Oracle, a channel is a
single session to a Tivoli Storage Manager server using the SBT API. Multiple channels can
be allocated. RMAN provides a multiplexing feature that enables parallel data streams to be
sent over multiple allocated channels to maximize backup and recovery performance.
Recovery catalog
The recovery catalog is the repository for information about backup objects created by
RMAN. It is an Oracle database instance, separate from the target databases, and can
contain information for multiple target databases. The data stored in the recovery catalog
comprises structural information about the target databases to back up and restore. The
recovery catalog contains the following information:
Physical schema of a target database
You must register the target database at the recovery catalog to define the physical
schema of the target database. RMAN needs to know about any structural change of the
target database, and obtains this information from the target database control file.
Database backup history
RMAN backs up databases, table spaces, data files, control files, and archive logs to the
Tivoli Storage Manager server. Details of these backup objects held on Tivoli Storage
Manager is stored in the recovery catalog.
For restore operations, RMAN queries the recovery catalog to determine which files to restore
from Tivoli Storage Manager. For a backup operation, RMAN backs up the objects specified
in the command to Tivoli Storage Manager. In both cases, the data is transferred on the
previously defined channel.
RMAN
3 Recovery catalog
5
SGA
Server 4
Process
Server
Process
channel
Server Process
Tape
Solution description
Planning is one of the most important areas for consideration before beginning to use Tivoli
Storage Manager for database. It is important that the database administrator and the Tivoli
Storage Manager administrator work together to anticipate the circumstances in which
recovery will be required, and also the resource and configuration requirements. These ideas
apply to all types of databases.
Backup requirements
A backup strategy is only one part of your overall data management plan. You must consider
how important your data is to the function (or even existence) of your organization. The less
time that your organization can function without its data, the more important that data is to
you. Your system must be designed in such a way to keep important data available when a
failure occurs. Reliance on backups is not necessarily sufficient. Before you design a backup
strategy, define the requirements that the strategy must satisfy. These are some factors to
consider when you define the requirements for your backup strategy:
Types of events (the categories of incidents that may occur)
Speed of recovery (how quickly you need to be able to recover)
Backup windows (the periods of time at which backups can be performed)
Recovery points (to which points in time you need to be able to recover)
Units of recovery (which other tables and files need to be recovered to the same point in
time)
None of these on their own can guarantee the availability of your data, but in combination they
can reduce the impact of a failure.
The Tivoli Storage Manager server cannot and does not know what length of time a client
program needs to keep the data objects. This must be done by the client.
Because the policy requirements for Oracle backups differ from the settings you want for
regular Tivoli Storage Manager backup clients, a different management class must be defined
within Tivoli Storage Manager for managing these Oracle backups. There are two ways to
implement this different management class setup:
Define a new management class within an existing policy domain.
Define a separate policy domain where the default management class contains the
required settings.
Crosscheck utility
The crosscheck utility verifies whether backups still exist on disk or Tivoli Storage Manager.
RMAN does not delete backup entries that it could not find, but instead marks them as
expired. If the backup was erroneously marked expired, for example, because Tivoli Storage
Manager was unavailable or misconfigured, the crosscheck utility will mark it available the
next time it is run if the backup still exists.
A backup object that is the active version or in the active state will never be purged from Tivoli
Storage Manager storage, that is, it never expires. It must first be inactivated by the Tivoli
Storage Manager client program. The Tivoli Storage Manager client program can do this by
manually deleting the backup object or sending a new version of the backup object. When a
backup object becomes inactive or moves into the inactive state, it is still accessible by the
Tivoli Storage Manager client. A main difference between active and inactive is that an active
object becomes inactive due to a client operation. An inactive object becomes expired
automatically by the Tivoli Storage Manager server as soon as it exceeds its retention criteria.
Changing from inactive to expired does not require a client operation. There is no way for a
backup object to change back to the active state once it has become inactive. When a backup
archive object moves into the expired state, it is no longer accessible by the Tivoli Storage
Manager client. Additionally, there is no way for the backup object to change back to the
inactive state once it has become expired. If the retention for the backup object is set to retain
zero inactive objects (retextra=1,verdel=0) or to retain inactive copies for zero days
(retextra=0, retonly=0), the active backup object will change to the expired state as soon as
the active backup is inactivated
Use scenarios
This section describes additional scenarios for using Tivoli Data Protection for Oracle.
Back up the database using Tivoli Data Protection for Oracle and RMAN
When you execute the backup command, you create one or more backup sets. A backup set,
as shown in Figure 7-13 on page 180, which is a logical construction, contains one or more
physical backup pieces. Backup pieces are operating system files that contain the backed up
data files, control files, or archived redo logs. You cannot split a file across different backup
sets or mix archived redo logs and data files into a single backup set.
Database
backup piece
Full backup
A full backup is a non-incremental backup of one or more data files. A full backup has no
effect on incremental backups and is not considered to be part of the incremental strategy.
If the database is in ARCHIVELOG mode, you can choose to do full backup while the
database is online or offline. If the database is in NOARCHIVELOG mode, the database must
be closed by a clean shutdown. Full backups can be taken of the following items:
Data files
Table spaces
Databases
Control files
Archive logs
Incremental backup
RMAN can incrementally back up databases at the individual block level. An incremental
backup is a backup of one or more data files and contains only those blocks that have been
modified since a previous backup at the same or lower level.
The multilevel incremental backup feature allows you to create various levels of incremental
backups. Each level is denoted by an integer, with 0 being the lowest backup level. An
incremental backup performed at a given level backs up only those blocks that have been
modified since the last backup at the same or lower level.
Incremental backup of control files or archived logs is not supported. There are two types of
incremental backups: non-cumulative and cumulative.
Figure 7-14 on page 182 illustrates part of a monthly cycle of non-cumulative incremental
backups. The cycle is based on backup levels 0, 1, and 2. A weekly backup is performed at
level 0 on Sunday, incremental backups level 2 are performed on Monday through
Wednesday and on Friday and Saturday, and weekly incremental backups at level 1 on each
Thursday.
Note that the first incremental backup must be a level 0 backup that contains all used blocks.
A cumulative backup at level 2 will contain all blocks changed since the most recent level 1
backup, copying all blocks changed since the base level 0 backup only if a previous level 1 is
unavailable. In contrast to a cumulative backup, a differential backup at level 2 determines
which level 1 or level 2 backup occurred most recently and copies all blocks changed since
that backup.
Restore operations
Recovering a database, table space, or data file is a two-stage process. The object is first
restored and then it is recovered. The restore process restores the necessary full or
incremental level 0 backups. Incremental backups at levels greater than 0 are not restored.
These are restored during the subsequent recovery process. By default, the objects are
restored to their original location as specified in the recovery catalog. An alternative location
can also be specified if required. RMAN uses the recovery catalog to select the most current
backup sets for use in the restore. The mode in which the database is running determines
whether a consistent or inconsistent restore operation can be performed.
If the control files are lost, they must be restored before other restore operations can be
performed. Only after restoring the control files can the target database be mounted and the
other restore operations started. A restore operation is typically followed by a recovery
operation.
Recovery Operation
Recovery is a process whereby a restored file is made available, either to the most current
state or to a specific point in time. Once the data files are restored, they have to be made
consistent with each other. The recover command is used to perform media recovery and to
apply incremental backups. During the recovery process, RMAN automatically restores the
archived redo logs as required. If RMAN has a choice between restoring incremental backup
sets or applying redo logs, it always uses the incremental backups. If overlapping levels of
incremental backup are available, the lowest level of incremental backup, the one covering
the longest period of time, is chosen automatically.
For database recovery, the database must be started but not open. Archive log backup sets
are restored as needed to perform a recovery. They are restored to the current archive log
destination as specified in the init.ora file
Before enabling LAN-free support, you must install the Tivoli Storage Manager Managed
System for SAN Storage Agent on the same system as Data Protection for Oracle. If your
database is considered very large, see 7.6, “Big data: Structured, very large databases” on
page 231.
See the IBM Tivoli Storage Manager for SAN for your operating environment for more
information about LAN-free requirements.
Considerations
Consider the following information:
The deduplication and enablelanfree options are mutually exclusive. Therefore, you can
use either one option or the other, but not both options together.
The deduplication and enableclientencryptkey options are also mutually exclusive.
Therefore, you can use either one option or the other, but not both options together.
More information about deduplication is in 3.3.3, “Server-side data deduplication” on page 57.
Additional information
A description of the offering is provided, along with links to installation resources and
additional informations about the product.
System requirements
The detailed system requirements for Tivoli Storage Manager for Databases and other
maintenance updates are available from the All Requirement Documents web page:
http://www.ibm.com/support/docview.wss?uid=swg21218747
Documentation updates
For information that was unavailable at the time of publication, see the following web page:
http://www.ibm.com/support/docview.wss?uid=swg27022321
7.5.2 Tivoli Storage Manager for Databases: Data Protection for Microsoft SQL
Tivoli Data Protection for SQL Server enables you to perform online backups and restores of
Microsoft SQL Server databases to Tivoli Storage Manager Server storage using either
command-line or graphical interfaces on Windows servers, in a stand-alone or clustered
environment. Use Tivoli Storage Manager for Databases: Data Protection for Microsoft SQL
Server to protect business-critical databases with automated tasks, utilities, and interfaces
Solution architecture
Data Protection for Microsoft SQL Server performs online backups and restores of Microsoft
SQL databases to Tivoli Storage Manager storage or local shadow volumes. You can perform
backups and restores using a command-line or graphical user interface (GUI).
Data Protection for Microsoft SQL operations use the Tivoli Storage Manager application
programming interface (API) to communicate with the Tivoli Storage Manager server, and use
the SQL API to communicate with SQL Server. In addition to using these APIs, Data
Protection for Microsoft SQL VSS operations use the Tivoli Storage Manager backup-archive
client (VSS Requestor) and Microsoft Volume Shadow Copy Service to produce an online
snapshot (point-in-time consistent copy) of SQL data that can be stored on local shadow
volumes or on Tivoli Storage Manager server storage.
Data transfer between client and server can be done via LAN (TCP/IP) or SAN (LAN-free
backups).
Tivoli
Storage
SQL API SQL Data API TSM Manager
Server Protection
Server
A Legacy backup creates a copy of all or part of a SQL database or logs on Tivoli Storage
Manager storage media.
Data Protection for SQL provides selection mechanisms and the logic that are required to
back up and restore SQL data. When you initiate a Legacy backup operation, Data Protection
for SQL completes the following actions:
1. Begins a session with a Tivoli Storage Manager server using the Tivoli Storage Manager
API and information contained in a client options file.
2. Starts a session with the SQL Server using the SQL-SMO interface.
3. Instructs the SQL Server using the SQL VDI interface to begin a backup of the selected
database objects.
4. Receives data from the SQL Server and sends it to the Tivoli Storage Manager server.
5. Informs the SQL Server that the backup is complete.
6. Ends the Tivoli Storage Manager server and SQL Server sessions.
When a backup is performed, Tivoli Storage Manager server retains information about the
SQL Server and database. This information is available for query and restore operations after
the backup is completed. The information about the names and sizes of the database file
groups and files is stored along with the database data, as a sub-object. This sub-object is
referred to as metadata.
For local VSS backups you must have a licensed version of IBM Tivoli Storage FlashCopy
Manager installed on your system. We cover this option in 7.5.6, “Tivoli Storage FlashCopy
Manager” on page 222.
When performing VSS backups and moving data to Tivoli Storage Manager server storage,
sufficient space on local snapshot volumes is still temporarily required to hold the snapshot.
For SQL data backed up to Tivoli Storage Manager server storage, the SQL data on the
snapshot volume is sent to the Tivoli Storage Manager server. After the data transfer to the
server is complete, the snapshot volume is made available for reuse.
The Tivoli Storage Manager backup-archive client serves as the VSS requestor that
communicates with VSS to access the SQL data to create shadow copies of SQL databases.
Data Protection for SQL serves as a front end for VSS backup operations. Because of the role
that the backup-archive client performs as the VSS requestor, features such as LAN-free
backup, client-side deduplication, data encryption, and data compression require that options
related to these features be specified in the backup-archive client options file for VSS
operations.
Microsoft
SQL Server Source
volumes
VSS/VDS VSS/VDS
Tivoli Storage
FlashCopy Manager
Use scenarios
Different backup strategies are available depending on specific requirements regarding
network traffic, backup window and acceptable restore times. Table 7-3 on page 190
summarizes these strategies.
Full backup only This approach is best for SQL databases that are relatively small because it X X
implies that the entire database is backed up each time. Each full backup
takes longer to perform, but the restore process is most efficient because
only the most recent (or other appropriate) full backup need be restored. This
is the appropriate strategy for system databases such as master, model, and
msdb due to their normally small size.
Full plus log A full plus transaction log backup strategy is commonly used when the X X
backup normal backup window or network capacity cannot support a full backup
each time. In such cases, a periodic full backup followed by a series of log
backups allows the backup window and network traffic to be minimized.
The full backups can be done during low usage times when a larger backup
window and increased network traffic can be tolerated. The restore process
becomes more complex, however, because a full backup, as well as
subsequent log backups, must be restored. It is also possible to do a
point-in-time restore to restore a transaction log to a specified date and time.
You can apply Legacy log backups after a full VSS backup has been
restored. To do this, you must leave the database in a recovering state by
specifying /recovery=no in the CLI or by making sure that the Recovery
option in the GUI Restore Databases or Restore Groups/Files is not selected
when restoring the VSS backup.
Full plus This strategy can be used between full backups. A differential database X X
differential backup can save both time and space. Space is saved because the backup
backup consists of only the changed portions of a database since the last full backup
(it is cumulative). Time is saved because you can avoid applying all individual
log backups within that time to the operation. This applies to restore
operations too; only the last differential backup (latest version) must be
restored.
Although VSS supports full backups only, Legacy differential backups can be
applied to the VSS full backup. To do this, you must leave the database in a
recovering state by specifying /recovery=no in the CLI or by making sure that
the Recovery option is not selected when restoring the VSS backup.
Full plus This strategy allows for a faster restore scenario by reducing the number of X X
differential plus transactions that may need to be restored and applied. If, for example, a full
log backup Legacy or VSS backup is done weekly, a differential nightly, and a log backup
every four hours, the restore would involve the full backup, a differential, and
at most five log backups. However, simply a full plus log backup scheme on
the same cycle could require a full plus up to 41 log backups to be restored
(six days times six log backups per day plus up to five backups on the day
the full backup was done).
Although VSS supports full backups only, Legacy log backups and Legacy
differential backups can be applied to the VSS full backup
File or group When a group is created on the SQL Server, database files are identified with X
backups that group. The group used for the group backup is dependant on the group
to which the database files are defined. Use a file backup strategy when it is
impractical to backup an entire database because of size and accompanying
time and performance issues.
When performing restore operations for a file or file group, provide a separate
backup of the transaction log.
File or group options can also save both backup and restore time in cases
when certain tables or indexes have more updates than others and need to
be backed up more often. It is time-effective to place such data in their own
file group or files and then back up only those items.
An AlwaysOn Availability Group can contain a set of primary databases and one to four
copies of the set of primary databases, called secondary databases.
Databases in an availability group are called availability databases, and they fail over
together as a group.
An AlwaysOn Availability Group requires SQL Server instances on Windows Failover Cluster
nodes. An availability group can have several replicas. For example, availability group 1 might
have replicas node1, node2, and node3.
A cluster node might be an availability replica for one or more availability groups. For
example, node1 might be a replica for availability group 1 and another availability group. For
the secondary replica, read-only is an option that can be set at the availability group level.
The AlwaysOn Node is used to manage backups of availability databases. When working in a
Tivoli Storage Manager environment, the AlwaysOn Node should be common in a Windows
Failover Cluster. This presence enables the management of backups of an availability
database in a single location, regardless of the replica that is used to perform the backup.
For all backup operations of secondary availability replicas, the secondary replicas must be in
the synchronized or synchronizing state.
For information about setting up a LAN-free environment, see IBM Tivoli Storage Manager for
SAN for Windows Storage Agent User's Guide:
http://pic.dhe.ibm.com/infocenter/tsminfo/v7r1/topic/com.ibm.itsm.sta.doc/welcome.html
Before installing Data Protection for SQL, ensure that your system meets the minimum
software and operating requirements. Details of the software and operating system
requirements for Data Protection for SQL can change over time. For current software
requirements, see the Tivoli Storage Manager for Databases - All Requirements Documents
website:
http://www.ibm.com/support/docview.wss?uid=swg21218747
Tivoli Data Protection for Lotus Domino is not intended as a substitute for the standard Tivoli
Storage Manager backup-archive client. Tivoli Data Protection for Domino cannot be used to
back up or restore any non-database data, such as Notes ID files, or notes.ini, or any other
system configuration files. Those files need to be backed up by the Tivoli Storage Manager
backup-archive client. Therefore, the two client types work together to provide full data
protection for your Notes environment. The Tivoli Data Protection for Lotus Domino
application client and the Tivoli Storage Manager backup-archive client can run
simultaneously on the same Domino server, however, they are totally separate clients as far
as the Tivoli Storage Manager server is concerned.
Client benefits
IBM Tivoli Storage Manager for Mail protects data on email servers running IBM Lotus
Domino. This software module for IBM Tivoli Storage Manager enables data protection of
your mail databases while they are online. It automates data protection, enables hot backups
without shutting down the application and improves data restore performance. Tivoli Storage
Manager for Mail does these tasks:
Uses APIs provided by mail application vendors to perform online hot backups and
improve restores without shutting down the mail application.
Helps protect the growing amount of new and changing data that should be securely
backed up to help maintain continuous availability.
Exploits the Lotus Domino transaction logging feature, which enables the capture and
logging of database changes, resulting in less-frequent full backups.
Backs up Lotus Domino NSF databases.
Maintains multiple backup versions of Domino databases.
Archives Lotus Domino transaction log files when archival logging is in effect.
Restores backup versions of a Lotus Domino database and apply changes since the last
backup from the transaction log.
Restores Domino databases to a specific point in time.
Restores one or more archived transaction log files.
Expires database backups automatically based on version limit and retention period.
Expires archived transaction log files when no longer needed.
Automates scheduled backups.
Restores Domino databases to an alternate server or partition.
Accesses Data Protection for Domino remotely using the Tivoli Storage Manager Web
client.
Accesses Data Protection for Domino using the client GUI based on Oracle Java.
Accesses Data Protection for Domino using the command-line interface.
Domino
Logs
Transaction
Tivoli Storage
Logs
DP Manager
For Repository
Domino API TCP/IP
API
Lotus
Domino
Domino Tivoli Storage
Backup Manager
Server
Domino
Database
Tivoli Storage Manager
Database and Logs
Figure 7-18 Tivoli Storage Manager for Mail - Data Protection for Domino
The backup and recovery API in Domino provides the capability to perform online full backups
of individual databases and archives of the transaction log when archival logging is in effect.
When archival logging is used on the Domino server, it archives the transaction log files and
retrieves them as required for a database recovery. Database backups are archived. The
Tivoli Data Protection for Lotus Domino application client provides a command line interface
for performing backups and restores. The application client commands are issued from a
command prompt. On Windows, Tivoli Data Protection for Lotus Domino also provides a GUI,
which supports most of the functions of the application client; transaction log files are stored
in Tivoli Storage Manager storage. A transaction log captures database changes for logged
databases, so full database backups are not required as frequently.
Data Protection for Domino provides support for two types of Domino databases:
NSF and DB2.
A database can be restored to the production server under a temporary name and the
desired document can be copied to the appropriate database. If, for performance reasons, the
production server cannot be used in the restore process, the database can be restored to an
alternate server and copied to the production server. You should perform alternate server
restores when possible to reduce demands on the production Domino server. Alternate
server restores can be performed to an alternate partition or to a separate Domino server.
Note: Because transaction log file names are unique, they will not expire because of
version limit.
Archived transaction log files are retained on the Tivoli Storage Manager server as long as a
database backup exists that needs these log files for a complete recovery.
Use scenarios
Different backup strategies are available depending on the database used with Domino
Server.
Archival logging allows transaction log data to be archived on the Tivoli Storage Manager
server so that changes to logged databases can be stored on the Tivoli Storage Manager
server without having to perform a full backup. This allows a strategy with less frequent full
database backups because changes to logged databases are available for restore in the
archived transaction log files.
The archivelog command backs up Domino transaction log files when archival logging is in
effect on the Domino server. The command queries the Domino server to determine if any log
extents are ready for archiving. If so, the log files are backed up to Tivoli Storage Manager
server storage, and the Domino server is notified of their availability for reuse.
In addition, high and low threshold values can be specified as a percentage of the log
capacity to control whether log files should be archived when the command is run. This allows
the command to be scheduled regularly to protect against a log full condition but to actually
do the archive only if the log is getting close to being full.
Each backup takes longer to perform, but the restore process is most efficient because only
the most recent (or other appropriate) full backup needs to be restored.
Note: You can apply updates to the restored database from the transaction log if the log
has not wrapped since the backup was performed. If the log has wrapped, the attempt to
apply logs fails
The archivelog command captures changes to all logged databases in between full backups
of selected databases. To restore a database to its most recent state, restore the most recent
database backup and specify /applylogs when activating the restored database. This
automatically restores the necessary archived transaction log files so that updates for the
database can be applied.
Note: The DB2 commands do not return information about whether a backup to a Tivoli
Storage Manager server was compressed, encrypted, sent LAN-free or deduplicated.
To restore a DB2 enabled Notes database to its most recent time, first select the most recent
backup from a DB2 Group backup or from a full DB2 database backup that contains the DB2
enabled Notes database (if available). If the most recent DB2 Group backup is not available,
restore the DB2 Group from the most recent full DB2 database backup. This type of restore is
Environments that contain both NSF and DB2 enabled Notes databases
Domino environments that contain both NSF and DB2 enabled Notes databases can
implement the following backup strategy:
Perform full DB2 database backups and NSF selective backups on a regular basis.
Perform routine incremental backups of NSF databases to inactivate backup copies that
have been deleted from the Domino server.
Perform regular DB2 Group backups if the DB2 database is enabled for rollforward
recovery.
Perform routine archiving of the transaction log files if archival transaction logging is
enabled on the Domino server.
Routinely inactivate the Domino server log file and routinely inactivate (and delete) DB2
objects from the Tivoli Storage Manager server.
Additional information
A description of the offering is provided, with links to installation resources and additional
informations about the product.
System requirements
The detailed system requirements for this release, and other V6.4 maintenance updates, are
available from the Tivoli Storage Manager for Mail - All Requirement Documents:
http://www.ibm.com/support/docview.wss?uid=swg21219345
7.5.4 Tivoli Storage Manager for Mail: Data Protection for Microsoft
Exchange Server
Tivoli Storage Manager for Mail: Data Protection for Microsoft Exchange Server provides
online backups and restores of Microsoft Exchange Server components to Tivoli Storage
Manager storage. Data Protection for Microsoft Exchange Server provides a connection
between an Exchange Server and a Tivoli Storage Manager server which allows Exchange
data to be protected and managed by Tivoli Storage Manager. Data Protection for Microsoft
Exchange Server protects Exchange Server data and improves the availability of Exchange
databases.
Support: Starting with Exchange Server 2010, Microsoft no longer supports legacy-style
(streaming) backups. Only VSS-style backups are supported. Tivoli Storage Manager 7.1
supports only Microsoft Exchange Servers 2010 and 2013. For these Exchange servers
only VSS backups are supported. References to Legacy backups in this book are
supported by Tivoli Storage Manager 6.4.1 for Microsoft Exchange Server 2007.
Today, email systems play a key role in making an enterprise successful. Businesses are
often severely impacted when email service is down, even if other production services are
running. Consequently, ensuring email server availability is a critical business concern.
In the face of such constraints, certain requirements must be addressed, such as these:
Fast recovery
Fast backups
Zero impact, high performance backups
Intelligent management of these backups
Data Protection for Microsoft Exchange helps protect and manage Exchange Server
environments by facilitating the backup, restore, and recovery of Exchange Server data.
Solution architecture
Data Protection for Microsoft Exchange performs online backup and restore operations of
Microsoft Exchange Server databases (Exchange Server 2010 and 2013) to Tivoli Storage
Manager storage or local shadow volumes. You can perform backups and restores using a
command-line interface (CLI) or graphical user interface (GUI). See your Exchange Server
documentation for complete, detailed information about the backup and restore process of
Microsoft Exchange Servers.
Beginning with Exchange Server 2010 Microsoft no longer supports the Microsoft Legacy API
(streaming) for backup and restore operations. It only supports the use of VSS for the backup
and restore.
Data Protection for Microsoft Exchange operations use the Tivoli Storage Manager
application programming interface (API) to communicate with the Tivoli Storage Manager
server, and use the Exchange API to communicate with Exchange Server. In addition to using
these APIs, Data Protection for Microsoft Exchange VSS operations use the Tivoli Storage
Manager backup-archive client (VSS Requestor) and Microsoft Volume Shadow Copy
Service to produce an online snapshot (point-in-time consistent copy) of Exchange data that
can be stored on local shadow volumes or on Tivoli Storage Manager server storage.
Solution description
Data Protection for Exchange provides a Legacy method and a VSS method for backing up
data.
Data Protection for Exchange tracks and stores mailbox location history, which is used to
automate mailbox restore operations. This causes a delay before each backup. In a small or
centralized Active Directory environment, the delay might be a few seconds or minutes. In
large or geographically disperse Active Directory environments, the delay might be more time.
If you do not plan to use mailbox restore, you can safely disable mailbox history.
When you initiate a Legacy backup operation, Data Protection for Exchange completes the
following actions:
1. Begins a session with a Tivoli Storage Manager server using the Tivoli Storage Manager
API and information contained in a client options file.
2. Informs the Exchange Server that a backup is ready to begin.
3. Receives data from the Exchange Server and sends it to the Tivoli Storage Manager
server.
4. Informs the Exchange Server that the backup is complete.
5. Ends the Tivoli Storage Manager server session.
Data Protection for Microsoft Exchange provides backup and restore functions for the
Exchange storage groups and associated transaction logs. Data Protection for Microsoft
Exchange does not provide a complete disaster recovery solution for an Exchange Server. In
a disaster recovery situation, Data Protection for Microsoft Exchange restores local
continuous replication (LCR) storage groups. Other files need to be restored in a disaster
recovery situation. See your Microsoft Exchange Server documentation for disaster recovery
considerations.
Personal folders and personal address books that are stored on Microsoft Outlook clients are
not protected by Data Protection for Microsoft Exchange. The Tivoli Storage Manager
backup-archive client can be used on the Outlook client platform to back up and restore these
files. Because the Outlook client normally keeps these files locked when running, you should
stop the Outlook client before backing up or restoring these files. Because Tivoli Storage
Manager Backup-Archive client provides open file support, you might be able to back up and
restore these files while the Outlook client is running.
Figure 7-19 summarizes the components of a Tivoli Storage Manager with Data Protection for
Exchange solution providing VSS backup restore services.
M ic ro s o ft
E xc h a n g e S e rve r Source
volumes
V S S/V DS V S S/V DS
T SM fo r M a il – Da ta
T iv o li S to ra g e M a n a g e r
P ro te c tio n fo r
Snapshot B a c k u p -A rc h ive c lie n t
E xc h a n g e (Target (re m o te )
volumes)
T iv o li S to ra g e
F la s h Co p y M a n a g e r
T iv o li S to ra g e M a n a g e r
B a c k u p -A rc h ive c lie n t T iv o li S to ra g e
(lo c a l) M a n a g e r S e rv e r
Figure 7-19 Data Protection for Exchange with Tivoli Storage Manager integration
A VSS backup means that the Exchange Server is not in backup mode for an extended period
of time. The length of time to perform the snapshot is usually measured in seconds, not
hours. In addition, a VSS backup allows a snapshot of large amounts of data at one time
because the snapshot works at the volume level. VSS backups can be stored on Tivoli
Storage Manager server storage or local VSS shadow volumes. Both of these storage
destinations require that sufficient space be available for the snapshot. VSS backups stored
locally on VSS shadow volumes are directly accessible by the Exchange system (as long as
sufficient space is available for the snapshot).
These types of local VSS backups are faster for a couple of reasons:
Because of the way the snapshots are managed
Because the data is not placed into Tivoli Storage Manager server storage
Restoring these backups is also fast because the Exchange data is not transferred from Tivoli
Storage Manager server storage over the network.
For local VSS backups you must have a licensed version of IBM Tivoli Storage FlashCopy
Manager installed on your system.
When performing VSS backups and moving data to Tivoli Storage Manager server storage,
sufficient space on local snapshot volumes is still temporarily required to hold the snapshot.
For Exchange data backed up to Tivoli Storage Manager server storage, the Exchange data
on the snapshot volume is sent to the Tivoli Storage Manager server. After the data transfer to
the server is complete, the snapshot volume is made available for reuse. If you are storing
VSS backups locally and the maximum number of local backup versions to be maintained (as
specified by the Tivoli Storage Manager policy) is reached, the oldest backup version is
expired to create the snapshot for the backup to Tivoli Storage Manager server storage.
The Tivoli Storage Manager backup-archive client serves as the VSS requestor that
communicates with VSS to access the Exchange data to create shadow copies of Exchange
storage groups. Data Protection for Exchange serves as a front end for VSS backup
operations. Because of the role that the backup-archive client performs as the VSS requestor,
features such as LAN-free backup, client-side deduplication, data encryption, and data
compression require that options related to these features be specified in the backup-archive
client options file for VSS operations.
Some VSS backup characteristics differ from Legacy backup characteristics. Examples of
these differences are the backup characteristics for types supported, the backup granularity,
and the backup storage location options.
Legacy restore
When a legacy restore is initiated from the Data Protection for Exchange GUI, you are
prompted to either dismount the databases or cancel the restore operation. If the restore is
run from the CLI, the Exchange Administrator must first dismount the necessary databases.
Data Protection for Exchange performs the following actions to restore an Exchange
database or storage group.
1. It starts a session with the Tivoli Storage Manager Server.
2. It informs the Exchange Server that a restore is about to start.
3. It restores the specified database and log files from the Tivoli Storage Manager server.
The logs are restored to a temporary location as specified by the Exchange Administrator.
4. It informs the Exchange Server that the restore has completed. You have the option of
either starting the recovery or mounting the database (when the recovery completes).
5. It ends the Tivoli Storage Manager server session.
Depending on what backup strategy you choose, restoring an Exchange Storage Group can
involve restoring multiple backup objects from the Tivoli Storage Manager server. For details
about restoring Legacy backups, see Data Protection for Microsoft Exchange Server
Installation and User’s Guide, SC32-9058.
VSS restore
Only backups that are made through VSS can be restored through VSS. Therefore,
incremental or differential Legacy backups cannot be restored with this method, and neither
can full or copy Legacy backups. Because of current Microsoft restrictions, a Recovery
Storage Group (RSG) restore from VSS snapshot backups is not supported. Site Replication
Service (SRS) and Key Management Service (KMS) also cannot be restored with VSS. A
VSS restore will be directed to the same drive letters and paths as when the backup was run.
Use scenarios
Depending on your specific requirements regarding network traffic, backup window, and
acceptable restore times, you might choose to follow different backup strategies. It is
important to understand all aspects of Exchange Server disaster recovery, and backup
considerations recommended by Microsoft.
Table 7-4 summarizes these methods and can help you plan your backup strategy.
Table 7-4 Backup strategies with Data Protection for Microsoft Exchange Server
Backup type Use Legacy VSS
Full backup This type backs up the specified storage group or database, and X X
associated transaction logs. The Exchange Server deletes the
committed log files after the storage group or database, and logs are
successfully checked for integrity and backed up. If the storage group
or database is not mounted, the backup fails and the transaction logs
are not truncated
Copy backup This type is similar to a full backup except that transaction log files X X
are not deleted after the backup. A copy backup is used to make a
full backup of the Exchange Server storage group without disrupting
any backup procedures that use a incremental or differential backup
Database copy backup This type backs up only the specified database and its associated X
transaction logs. The transaction log files are not deleted after the
backup. A database copy backup is used to make a special full
backup of the database without disrupting any backup procedures
that use incremental or differential backups
Incremental backup This type backs up only transaction logs. The Exchange Server X X
deletes the committed log files after they are successfully backed up.
These log files are not deleted if the backup fails. Restoration of an
Exchange Server storage group or database from an incremental
backup requires the following tasks:
1. Restore of the last full backup
2. Restore of any other incremental backups that are performed
between the full backup and this incremental backup
3. Restore of this incremental backup
The log files are not deleted if storage groups or databases are not
mounted.
Differential backup This type backs up transaction logs. The log files are not deleted. X X
When you perform a full backup followed by only differential backups,
the last full backup and the latest differential backup has all data
required to bring the storage group back to the most recent state.
This type of backup is also called a cumulative incremental backup.
Restoring an Exchange Server storage group from a differential
backup requires the following tasks:
1. Restore of the last full backup
2. Restore of this differential backup, but no other differential
backups
Circular logging: When circular logging is enabled, you cannot use differential or
incremental backups. Data loss might occur if the log wraps before an incremental or
differential backup is finished. If you choose a backup strategy that involves incremental or
differential backups, you must disable circular logging for the Exchange storage group or
database from the Exchange Administrator program. See your Microsoft Exchange Server
documentation for more information about circular logging.
When you are considering backup strategies, use the following guidelines:
Do not use incremental and differential backups together.
If you choose a strategy that involves incremental or differential backups, circular logging
must be disabled on the storage groups or databases of the Exchange Server.
For Exchange Server 2010 and 2013, consider using DAG database replication
technologies. See your Microsoft documentation for details regarding this technology. For
an additional Database Availability Group (DAG) backup strategy, set up all DAG members
to back up all of the database copies. Use the /MIN and /PREFERDAGPAS flags.
Details of the hardware and software requirements change over time because of
maintenance updates and the addition of operating system, application, and other software
currency support.
For the most current requirements, see the hardware and software requirements technote
that is associated with your level of software. This technote is available from the following site:
http://www.ibm.com/support/docview.wss?uid=swg21219345
Administration
Administration
Assistant
Assistant
Client
Server
Media mgmt
library for
Oracle TSM
ProLe API
backint
Tivoli Storage
Manager server
dsm.opt
init<SID>.sap Init,SID>.utl
dsmsys.opt
Client benefits
IBM Tivoli Storage Manager for Enterprise Resource Planning protects your vital SAP system
data. It provides automated data protection for mySAP and SAP R/3 environments. You can
improve availability of your SAP database servers and reduce your administration workload.
Tivoli Storage Manager for Enterprise Resource Planning offers these benefits:
Delivers business value by protecting SAP system data efficiently and consistently.
Helps increase productivity by reducing repetitive administrative tasks.
Is SAP certified for heterogeneous environments.
Provides more efficient backup of very large SAP databases.
Integrates with database-specific utilities of IBM DB2 for Linux, UNIX and Windows,
Oracle and SAP BR*Tools.
As demonstrated in Figure 7-21, SAP backup and recovery utilities center on database
objects where more than 90% of the data resides on an SAP database server. As a result,
Data Protection for SAP backs up and restores data files, control files, and online or offline
redo logs.
T a b le s p a c e file s
D a ta b a s e c o n tro l file s
Backup/restore TSM
O n lin e re d o lo g s with BR* Tools via S e rv e r
DP for SA P
O fflin e re d o lo g s
O ra c le e x e c u ta b le s
S A P e x e c u ta b le s
Backup with
TSM backup-archive client
A ll o th e r file s
D B 2 d a ta b a s e c o n te n ts
(d a ta b lo c k s )
SAP
D a ta b a s e D a ta b a s e c o n tro l file s TSM
S e rv e r B ackup /restore S e rv e r
V ia D P fo r S A P
O fflin e D B 2 lo g file s
D B 2 e x e c u ta b le s
S A P e x e c u ta b le s B acku p w ith
T S M backu p-archive
A ll o th e r file s client
Other files (such as SAP and Oracle or DB2 executable files) can be backed up using the IBM
Tivoli Storage Manager Backup-Archive Client. This is important for disaster recovery
purposes, because all SAP and Oracle or DB2 executable files must be available before using
Data Protection for SAP to restore and recover the database.
With the integration, it is possible to follow ERP backup/restore procedures and to use the
integrated SAP database utilities BRBACKUP, BRARCHIVE, BRRESTORE and SAPDBA for
backup and restore. Other SAP related files (executables) are backed up by using Tivoli
Storage Manager standard techniques for file backup and restore, for example, incremental
backup, file filtering, and point-in-time recovery.
C onfiguration profile
file
BRRECOVER
Tivoli Data
SAP BRBACKUP
Protection for
DBA BRARCHIVE
SAP
BRRESTORE
Figure 7-24 shows the Tivoli Data Protection for SAP for Oracle function and interfaces.
Figure 7-24 Tivoli Data Protection for SAP for Oracle overview
Figure 7-25 shows the data interface between Oracle Databases and Tivoli Storage Manager
using Data Protection for SAP for Oracle using backint interface.
Figure 7-25 Data Protection for SAP for Oracle using Backint
Notes:
Using this method, the chosen data files are sent to Tivoli Storage Manager one by one.
No compression or block checking are performed at this level.
When a database is in backup mode, the amount of redo logs written to disk increases.
This is because Oracle writes the entire dirty block to the disk, not just the updated
data.
In some cases, when the backup routine fails for any reason, the data file remains in
active backup mode. This can cause some performance impact and additional I/O to
the disk.
Figure 7-26 shows the data interface between Oracle Database and Tivoli Storage Manager
using Data Protection for Oracle for SAP using RMAN.
Oracle database
Tivoli Storage
Manager server
Oracle instance
Data files (server process)
Control files
Redo logs
Tivoli Data
Protection for
Offline SAP
Redo logs
BR*Tools RMAN
/oracle/<sid>/ Catalog DB
saparchive
/ oracle/<sid>/
Control files
backup
Figure 7-26 Data Protection for SAP for Oracle using RMAN
Figure 7-27 shows the data interface between DB2 Databases and Tivoli Storage Manager
using Data Protection for SAP for DB2.
The archiving of DB2 offline log files is provided by the SAP tool BRARCHIVE. The retrieval of
DB2 offline log files is provided by the SAP tool BRRESTORE and by the Data Protection for
SAP tool BackOM. As of DB2 Version 9.X, offline log files can be archived and retrieved with
the DB2 built-in Log Manager. The DB2 Command-Line Processor (CLP) interprets
commands for the DB2 database and passes control to a DB2 Server Process. In the case of
Data Protection for SAP, the LOAD <libraryname> option causes DB2 to invoke the Data
Protection for SAP shared library. This process kicks off the backup or restore, loads the
library dynamically, and communicates with it through the TivoliStorage Manager API. For
starting a backup or restore, the DB2 CLP communicates with the DB2 Server Process,
providing the Server Process with the relevant information for processing the database
Backup and recovery tools are either an integrated component of a relational database
system, such as Oracle Recovery Manager (RMAN), DB2 backup/restore functions, or may
be installed as an independent software package developed by a third party (such as SAP
BR*Tools for Oracle).
Aside from performing backup and recovery operations, backup managers integrated with
media management systems can manage and enforce backup storage policies and backup
retention policies. The backup retention policies control retention and deletion of backup
versions stored on the backup media. Backup storage policies determine the destination
media to be used for the particular backup objects, such as backup images of table spaces,
and backups of archived logs can be stored on different media, as dictated by the storage
policy.
The basic functions supported by the backup management tools allow you to do these tasks:
Automate data protection tasks and allow for backup of database objects using various
methods and techniques (such as online or offline backup and full or partial backup).
Provide automatic or semi-automatic restore and recovery functions specific to a particular
RDBMS. A backup repository is used for the decision making about which backup image
to restore (for example, backup manager picks the last version of full backup and
determines which archive redo log backups will be needed for rollforward recovery).
Use a backup repository to keep track of the location of backup images and their time
stamps. The information from the backup repository is retrieved whenever the user sends
a list of the history of backup operations. The backup repository records are also used
whenever backup manager has to determine which backup objects will be retrieved during
the automatic database restore or recovery.
Control the retention and expiration of backup versions according to retention policies. The
retention policy may be defined by either the number of days the backup objects are to be
retained or by a number of backup versions to be kept.
Database managers provide interfaces to transfer backup data to backup media or third-party
media management systems such as IBM Tivoli Storage Manager. The media interfaces
either are based on open standards (such as DB2 XBSA) or can be specific for the particular
RDBMS (such as BR*Tools BACKINT interface or Oracle RMAN SBT interface). Interface
adapters for particular backup management systems are developed by the media
management systems vendors. This conceptual relationship is shown in Figure 7-28.
Figure 7-28 System architecture: database backup adapters for Tivoli Storage Manager
Note: IBM DB2 UDB provides integrated Tivoli Storage Manager backup support, so it can
directly call the Tivoli Storage Manager API client to transfer data to Tivoli Storage
Manager server.
Backup adapters
The following Tivoli Storage Manager clients provide backup adapters for Oracle, MS-SQL,
IBM DB2 UDB, and SAP MaxDB databases:
General purpose Tivoli Storage Manager backup adapters for databases:
– IBM Tivoli Storage Manager for Databases - Oracle:
Tivoli Storage Manager backup adapter for Oracle RMAN implements Oracle Media
Management API, Secure Backup to Tape (SBT) API V2.0.
– IBM Tivoli Storage Manager for Databases
Microsoft SQL Server: Tivoli Storage Manager adapter for Microsoft SQL Server
implements Microsoft SQL Server Virtual Device Interface (VDI), which enables the
Microsoft SQL backups to be transferred to Tivoli Storage Manager server. It supports
both earlier and VSS backups.
– IBM DB2 built-in Tivoli Storage Manager support
DB2 UDB provides an integrated support for Tivoli Storage Manager. Thus, the Tivoli
Storage Manager API client is the only component that is required to enable transfers
of DB2 backup data to the Tivoli Storage Manager server.
– IBM ADINT/TSM
The Tivoli Storage Manager adapter is similar to Tivoli Storage Manager for ERP and
provides seamless integration with SAP MaxDB backup, restore, and recover utilities.
ADINT/TSM is a service offering with central hotline support sold by Tivoli Services
organizations in the countries. For more information see the following site:
http://www-05.ibm.com/de/entwicklung/adint_tsm/
SAP-oriented Tivoli Storage Manager backup adapters for databases:
– Tivoli Storage Manager for ERP - Oracle
This software package implements the Tivoli Storage Manager backup adapters for
Oracle RMAN (SBTAPI) and for SAP BR*Tools (BACKINT). Tivoli Storage Manager for
ERP (Oracle) can serve as a backup adapter either between SAP BR*Tools and Tivoli
Storage Manager server or between Oracle RMAN and Tivoli Storage Manager server.
– Tivoli Storage Manager for ERP - DB2
This implements shared libraries that can integrate DB2 UDB backup tools with Tivoli
Storage Manager server.
Table 7-5 Summary of Tivoli Storage Manager adapters for DB2 UDB, Oracle, and MaxDB
Adapter or DB2 UDB Oracle RMAN SAP SAP MaxDB
backup tool BR*Tools for
Oracle
Built-in Tivoli Yes (API) Not available Not available Not available
Storage Manager
Support
Tivoli Storage Not available Yes (SBTAPI) Not available Not available
Manager for
Databases
(Oracle)
Tivoli Storage Yes (API) Not available Not available Not available
Manager for ERP
(DB2)
All the backup solutions mentioned can be integrated with the advanced backup techniques
such as LAN-free backup, parallel transfer of backup data to and from Tivoli Storage Manager
server, or multiplexing. Implementation of these techniques can significantly reduce backup
and restore times and eliminate the impact of backup data transfers on LAN throughput
A storage area network (SAN) is a high-speed network that connects different kinds of data
storage devices, such as disks subsystems, tape libraries, and juke boxes with associated
data servers. The primary purpose of the SAN is to transfer data between systems and
storage elements. For more information about SAN, see Introduction to Storage Area
Networks and System Networking, SG24-5470. The SAN feature with the storage agent
provides a great opportunity for lowering the backup window, reducing the traffic on the LAN,
and reducing the use of the IBM Tivoli Storage Manager server. You can install the storage
agent on a client machine that shares storage resources with the Tivoli Storage Manager
server or on a client machine that does not share storage resources but is connected to a
client machine that does share storage resources with the Tivoli Storage Manager server.
For more information, see IBM Tivoli Storage Manager for SAN for AIX: Storage Agent User’s
Guide, SC32-0129.
IBM Tivoli Storage Manager for Enterprise Resource Planning is a simple, scalable data
protection solution for SAP HANA and SAP ERP. SAP HANA’s memory-to-disk backups
create export files that must be preserved in case recovery is required. Tivoli Storage
Manager for ERP includes a one-step command that automates SAP HANA backup and
Tivoli Storage Manager data protection. This is a simple process that preserves SAP HANA
data safely in Tivoli Storage Manager. Full and incremental (log only) backups can be
performed, so you can make frequent backups throughout the day to help reduce the risk of
data loss. Parallel multiplexed sessions can be configured for faster backup processing as the
SAP HANA environment grows.
Recovery is also simplified. The most recent Tivoli Storage Manager managed SAP HANA
backup files can be recovered to their home directory with one command. Alternately,
administrators can select from a list of available backups or restore to alternate systems.
Tivoli Storage Manager for ERP reduces complexity by enabling all files associated with a
SAP HANA backup to be restored as a single unit. Tivoli Storage Manager data protection
solutions can help clients implement consistent, integrated backup policies for SAP HANA,
SAP ERP, GPFS and other critical components.
For information about the Data Protection for SAP HANA component, see this location:
http://www.ibm.com/support/knowledgecenter/SSZHVN_6.4.1/com.ibm.itsm.erp.doc/welco
me.html
Additional information
For more specific information, see the following Tivoli Storage Management web pages:
Tivoli Storage Manager for Enterprise Resource Planning:
http://www.ibm.com/software/tivoli/products/storage-mgr-erp/
IBM ADINT/TSM
http://www-05.ibm.com/de/entwicklung/adint_tsm/index.html
Tivoli Storage Manager for Databases
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManagerfor
Databases.html
The applications supported by Tivoli Storage FlashCopy Manager are IBM DB2, Oracle,
Microsoft Exchange, and Microsoft SQL Server, and the VMware vSphere platform. In
addition, IBM DB2 and Oracle databases are supported for use either with or without SAP
environments. Other applications can be supported on IBM AIX, HP-UX, Linux, Solaris, and
Microsoft Windows platforms with script customization.
These capabilities are achieved through the utilization of advanced storage specific hardware
snapshot technology to help create a high-performance, low-impact application data
protection solution. Tivoli Storage FlashCopy Manager is easy to install, configure, and
In addition to the above devices, Tivoli Storage FlashCopy Manager on Windows supports
any storage system that is VSS capable, by using the Microsoft Volume Shadow Copy
Services (VSS) system provider or a VSS Hardware Provider that strictly adheres to the
Microsoft VSS Provider interface.
Client benefits
Tivoli Storage FlashCopy Manager can help with these challenges in the following ways:
Performing and managing frequent, near-instant, non-disruptive, application-aware
backups and restores.
Using advanced snapshot technologies in IBM storage systems.
Improving availability of IBM DB2, SAP, Oracle, Microsoft Exchange, Microsoft SQL and
other application data
Supporting application development, data mining, and so on through database cloning
Integrating seamlessly with Tivoli Storage Manager to enable a full range of data lifecycle
and recovery management.
Solution architecture
Tivoli Storage FlashCopy Manager is available for three platforms:
Tivoli Storage FlashCopy Manager for UNIX and Linux:
Tivoli Storage FlashCopy Manager uses the copy services capabilities of intelligent disk
subsystems to create point-in-time copies. These are application-aware copies
(FlashCopy or snapshot) of the production data.
This copy is then retained on disk as backup allowing for a fast restore operation
(Flashback). Tivoli Storage FlashCopy Manager also allows mounting the copy on an
auxiliary server (backup server) as a logical copy. This copy (instead of the original data
on the production server) is made accessible for further processing. This processing
includes creating a tape backup or performing backup verification functions (for example,
the Database Verify Utility).
Tivoli Storage FlashCopy Manager for Windows:
Tivoli Storage FlashCopy Manager provides the tools and information needed to create
and manage volume-level snapshots of Microsoft SQL Server and Microsoft Exchange
server data. Snapshots are created while the applications remain online.
Figure 7-29 shows the components of the Tivoli Storage FlashCopy Manager. Tivoli Storage
FlashCopy Manager can now be used with other storage vendors such as EMC by leveraging
the Rocket Device Adapter for IBM Tivoli Storage FlashCopy Manager. For more information
about the Rocket Device Adapter see 2.2.5, “Tivoli Storage FlashCopy Manager” on page 23.
Tivoli Storage
Application FlashCopy Manager Online, near instant
System Local snapshot backups with
Application Snapshot minimal performance
Data Versions
impact
Snapshot
Backup
Sn High performance, near
a
Re psho instant restore capability
sto t
re
Oracle
Integrated with Storage
DB2
SAP
Hardware snapshots
SQL Server
Exchange Server For Various Storage Simplified deployment
Custom Apps
SVC With Optional
File Systems V7000 Database Cloning
V5000
TSM Backup
VMware
V3700 Integration
XIV
DS8000
N-Series * Via Rocket Adapter
NetApp **VSS Integration
EMC*
Other**
Solution description
This section provides a description about Tivoli Storage FlashCopy Manager for Windows
and Tivoli Storage FlashCopy Manager for UNIX. For information about Tivoli Storage
FlashCopy Manager for VMware see 7.1, “Common virtualization challenges” on page 142.
Tivoli Storage FlashCopy Manager for Windows
Tivoli Storage FlashCopy Manager for Windows supports five VSS backup types for Microsoft
Exchange Servers: full, copy, incremental, differential, and copy without integrity check. With
each backup type, there are options to specify the recovery preferences handled by
Exchange after the restore. Depending on the backup type selection, Exchange performs
tasks such as deleting committed log files, integrity checking (ESEUTIL), log replay, and
mounting the databases.
Tivoli Storage FlashCopy Manager for Windows provides four types of VSS restores:
Both Exchange and SQL (three types):
– VSS Restore: This restores VSS backups from a remote Tivoli Storage Manager
server.
– VSS Fast Restore: This restores VSS backups from local shadow volumes using file
level copy mechanisms. In general, restores conclude within minutes instead of hours.
– VSS Instant Restore: This restores by copying snapshots from IBM Tivoli Storage
FlashCopy Manager volumes back to the original source volumes with
hardware-assisted volume-level copy mechanisms. The application returns to normal
operation as soon as the volume-level copy starts and the log replay is complete.
Exchange only (one type):
– Mailbox Restore from VSS backups: This provides granular recovery of any mailbox or
item from IBM Tivoli Storage FlashCopy Manager for Windows VSS-based snapshot
backups. The ability to restore individual mailbox and granular mailbox items is a new
feature for Microsoft Exchange Server 2007 or later using IBM Tivoli Storage
FlashCopy Manager for Windows. IBM Tivoli Storage FlashCopy Manager for Windows
maintains mailbox location history. For backups taken with prior versions to Data
Protection for Exchange 6.1, no mailbox location history is available. When restoring
from these prior version backups, if the mailbox to be restored from has been moved or
deleted since the time of the backup, the /mailboxoriglocation parameter is necessary.
A VSS Instant Restore overwrites the entire contents of the source volumes. However,
overwriting the source volumes can be avoided by selecting the Disable VSS Instant
Restore option. This option bypasses volume-level copy and uses file-level copy instead to
restore the files from a VSS backup that resides on local shadow volumes.
Provide advanced protection Snapshot backup and restore of a full database including and
and restoration of DB2 excluding log files
databases on UNIX platforms Backup and restore of individual partitions for multi-partition
DB2 databases
Support of databases that are mirrored (between sites) using
Logical Volume Manager (LVM) mirroring technology
Provide snapshot backup and Support of databases that are mirrored (between sites) using
restore of a full SAP database LVM mirroring technology
running on DB2 or Oracle on
UNIX platforms
Integration with Tivoli Storage Extend data protection and retention off the source storage
Manager system for long-term data management and offsite disaster
recovery capabilities
Take advantage of automated data reduction and data
management policies that can reduce overall costs
Additional information
These resources have examples of Tivoli Storage FlashCopy Manager implementation.
IBM Tivoli Storage FlashCopy Manager 2.2 for Windows Backup & Recovery Solution for
Microsoft SQL Server 2008 and Exchange Server 2010 on the IBM XIV Storage System:
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101840
SAP with IBM Tivoli Storage FlashCopy Manager for VMware and IBM XIV and IBM
Storwize V7000 storage systems:
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102290
So how do you back up very large databases? The answer is by careful planning. We need to
find ways to complete the backup and the restore within a shortened time frame. As the data
grows the backup window and recovery time object can become too long and new
technologies need to be put into play to satisfy business needs.
So the challenge is to have a solution to back up and restore data as quickly as possible, with
lowest impact on the production data. This should be done at the lowest cost even when you
experience significant data growth.
The matrix Figure 7-32 shows which Tivoli Storage Manager solutions that can address these
challenges when protecting very large databases.
To meet the challenges we need to look at the drivers to move the data from the primary
production storage to the Tivoli Storage Manager back-end storage. For this particular type of
data we look at the tired storage hierarchy which can be a disk to disk, disk to tape, or a disk
to VTL solution. In our Tivoli Storage Manager toolkit, we will make use of hardware-based
snapshot technology and LAN-free data transfer. Because VLDBs do not change the data
structure, we can lower the footprint by using different ways of deduplication functions.
FlashCopy
E
Capacity
T AP
L or
VT
K,
DIS
Hours
Days
Disk
mySAP Subsyst
DB
Server
* till start of recovery
In this section, we describe what options exist to protect large amounts of data stored in
structured databases with a Tivoli Storage Manager solution.
There are discussions about which solution is best for backup data in general, whether it
should be disk-based or tape-based. Some reports might indicate VTL as being the most cost
efficient and others indicate tape. In reality, this must be considered case by case. Maybe we
need not choose at all. A combination of disk and tape might be the optimal strategy for many
storage administrators to address various needs. It all comes down to good planning and a
fundamental understanding of the data to be protected and also what the backup window and
recovery time objectives are.
A VTL or disk-based backup can provide the performance that is needed to prioritize the
recall of files for high risk applications. As data backed up to disk becomes infrequently or
never accessed, it should be moved to tape for long-term retention. You must define what
long term means to your company. Tape technology can provide data security, compliance,
and offline protection (against viruses, hackers, system errors, and so on) and a long term,
low-cost archive repository.
The LAN-free data transfer concepts are the same for all devices supported for that purpose.
We give a short description in this section about LAN-free backups to tape. The concepts will
also apply to disk and VTL, which we describe later.
LAN-free backup should be the standard way to backup big data with Tivoli Storage Manager.
To achieve the best overall performance, you should consider the type of data on the client.
LAN-free backup makes substantial use of the LAN for control and other information
exchange. When trying to decide if LAN-free is appropriate for your environment, consider the
following common factors:
A congested network (LAN): This factor includes overall network congestion as well as any
network limitations between the client and the server machines.
Constrained server: Streaming data to tape over the SAN can be faster than using the
network, if the client systems have access to the SAN storage resources.
Existing SAN storage resources
Type of data to be backed up
Number of mount points available
Because the LAN-free path is used for sending the actual data, and not the metadata, a client
workload, which has proportionately more metadata than data to send, will in general see
less benefit from using the LAN-free path. Conversely, a client workload that spends most of
its time sending data will see a greater benefit from using LAN-free backup. From this
information, you can extrapolate that large file sizes are better suited for LAN-free backup.
Important: In this topic we discuss LAN-free backups to tape in very large database
environments. For more information about protecting databases, see 7.5, “Application and
email servers” on page 171.
Solution benefits
Using LAN-free to tape should give some immediate benefits:
Improved performance reducing backup time.
Decreased recovery time of data compared to transferring data across the LAN.
LAN-free data movement makes LAN bandwidth available for other uses and decreases
the load on the Tivoli Storage Manager server, allowing it to support a greater number of
concurrent client connections.
Exploit SAN infrastructure to relieve LAN.
Control of data as it grows.
Support for open system connectivity.
Heterogeneous operating system platforms can share the same storage devices, which
allows for better use of storage and reductions in storage costs.
LAN-free architecture
Figure 7-34 shows a typical LAN-free architecture.
Configuration preparations
Before you set up a LAN-free configuration, you should have a good understanding of the
essential pieces of information and components. This section discusses these pieces.
To assist you in setting up LAN-free data movement, identify or be aware of these items:
The client machine on which LAN-free is to be set up
After you decide to use LAN-free backup, consider where to install the Storage Agent. The
Storage Agent is installed most usually on the actual client machine, having the Storage
Agent running on a separate system is also possible. For informations about having the
Storage Agent run on a separate machine, see product documentation:
http://pic.dhe.ibm.com/infocenter/tsminfo/v6r4/topic/com.ibm.itsm.sta.doc/t_tsm
san.html
The Tivoli Storage Manager server to be used
The version of the Tivoli Storage Manager server and the Storage Agent must match. You
can find the compatibility matrix at the following location:
http://www-01.ibm.com/support/docview.wss?uid=swg21302789
The type of library sharing method to be used
The SAN-attached storage devices that you have (or plan to have) in your environment
most likely determines the type of library sharing method you will use. IBM tape devices or
non IBM storage devices that are supported by the Tivoli Storage Manager device driver
can be used for sharing. These device types include SCSI, ACSLS, external, 349x tape
libraries and virtual tape libraries (VTL). For more information about LAN-free to VTL, see
7.6.2, “LAN-free backups to VTL” on page 238.
The device names
The device names for the SAN-attached storage devices are required when defining the
path on the server. SAN-attached devices might appear with different device names on
each host that is attached to the same SAN.
A proper management class destination
For more information about planning and configuring LAN-free, see the following location:
http://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.0/com.ibm.itsm.ic.doc/welcom
e.html?lang=en
Solution description
The solution in Figure 7-34 on page 234 shows how the data goes across the SAN instead of
LAN. SANs provide an alternative path for data movement between the IBM Tivoli Storage
Manager client and server. LAN-free data transfer exploits this SAN path by enabling the IBM
Tivoli Storage Manager client to back up and restore data directly to and from SAN-attached
storage. Storage is shared between the Tivoli Storage Manager library server and client. The
Tivoli Storage Manager library manager server is responsible for managing the tape library.
The following steps are involved in a Tivoli Storage Manager LAN-free backup:
1. The Backup-Archive Client begins a backup operation. The Tivoli Storage Manager server
reports policy information to the client, including whether a destination is LAN-free. As the
client assigns policy settings for files during backup processing, it uses the Storage Agent
to send the data via LAN-free when the destination for that policy is LAN-free enabled.
2. The Storage Agent receives data for those files backed up by the client and assigned to
policy settings that use a LAN-free enabled storage pool. The Storage Agent sends a
request for a volume mount to the Library Manager server.
3. A request is sent from the Library Manager to the storage device to mount the appropriate
media.
Note: LAN-free data movement takes precedence over client-side data deduplication. If
LAN-free data movement occurs during client-side data deduplication, client-side data
deduplication is turned off, and a message is issued in the error log.
All other tasks, which are metadata-related, use the LAN path. Therefore, depending on the
profile of the backup to be performed, the proportion of time spent transferring over the LAN
versus the SAN differs.
LAN-free is typically ideal for transferring large amounts of data, such as when you backup a
database. However, LAN-free data movement is not optimal for short bursts of data or for
small files, such as archived log files. For that reason, do not use LAN-free data movement for
archived transaction log files.
Table 7-7 Hardware and software requirements for LAN-free client support
Servers Backup-archive clients Tivoli Data Protection
AIX AIX R3
z/OS MS-SQL
See the Tivoli Storage Manager Server-Client Compatibility and Upgrade Considerations
technote:
http://www.ibm.com/support/docview.wss?uid=swg21053218
Although Tivoli Storage Manager has supported disk storage pools for many years, there are
some obvious advantages to using a virtual tape library in place of, or as a supplement to,
your disk storage pools. For example, you do not need to assign a dedicated disk pool for
every server.
Backup and recovery from disk can solve all of the listed challenges in Figure 7-32 on
page 231. An effective disk-based backup and recovery environment requires the application
to know how to take advantage of the random access capabilities of disk like the Tivoli
Storage Manager application does.
You can use any supported virtual tape library when the following conditions are true:
There is no mixed media involved in the VTL.
Only one type and generation of drive and media is emulated in the library.
Every server and storage agent with access to the VTL has paths that are defined for all
drives in the library.
If any of these conditions are not met, any mount performance advantage, from defining a
VTL library to the Tivoli Storage Manager server, can be reduced or negated.
VTLs are compatible with earlier versions of both library clients and storage agents. The
library client or storage agent is not affected by the type of library that is used for storage. If
mixed media and path conditions are true for a SCSI library, it can be defined or updated as
LIBTYPE=VTL.
Solution architecture
By emulating an actual tape library, the VTL enables Tivoli Storage Manager to function
identically with virtual tape as it does with physical tape, except that it does it much faster and
much more reliable.
We are in this solution using the IBM TS7650G ProtecTIER as an example of a LAN-free to
VTL environment. Figure 7-36 on page 240 shows the backup servers that are connected to
FC Switch and ProtecTIER as a data destination. Backup servers can be either Tivoli Storage
Manager servers or Storage Agents.
Backup
Array
Backup
Servers
Figure 7-36 Tivoli Storage Manager with ProtecTIER Solution
With the IBM TS7650G, you can either share a virtual tape library between many servers and
storage agents or you can create several virtual tape libraries, one or more for each server.
This means you are not required to dedicate storage to every server, because the VTL that
uses ProtecTIER technology can assign capacity as required.
More important, the TS7650G’s unique IBM HyperFactor® data deduplication technology
provides up to 25:1 reduction of storage space required, delivering the capacity to retain
months of backup images at a total cost of ownership (TCO), less than that of actual tape.
The TS7650G ProtecTIER deduplication technology was designed to work seamlessly with
Tivoli Storage Manager to offer remarkable improvements in speed and reliability, and to
greatly reduce the TCO of a disk-based backup and recovery environment.
Solution description
The solution provided uses ProtecTIER as the VTL with deduplication enabled. Combining
the advanced capabilities and features of Tivoli Storage Manager with the powerful
capabilities of the ProtecTIER product provide IT organizations a cost-effective way to
improve the performance, reliability, and scalability of data protection.
LAN-free backups are simpler with the ProtecTIER product because there are increased tape
resources and fewer hardware restrictions. Because ProtecTIER is a VTL, it has the
advantage of presenting greatly increased tape resources to the backup server. This feature
is available with only Tivoli Storage Manager V6.3 or later. So, you are able to perform
LAN-free backups to the ProtecTIER server without the limitations normally applied to these
backups, such as tape drive availability. If you have many LAN-free clients already, then it is
possible that your LAN-free backup windows are dictated not entirely by business needs but
also by hardware availability. With the ProtecTIER product and its maximum of 256 virtual
tape drives per ProtecTIER node, you can virtually eliminate any previous hardware
restrictions, and schedule your backups as and when they are required by your business
needs.
The backup servers (Tivoli Storage Manager Server or Tivoli Storage Manager Storage
Agent) must have a dedicated host bus adapter (HBA) port or ports for the ProtecTIER VTL.
This port or ports can be shared with a physical tape library. However, the physical tape
library must not be in the same storage area network (SAN) zone as the VTL. When it is not
possible to dedicate HBA ports for VTL and physical tape library, you should have different
zones to separate the traffic.
Port sharing: Although sharing a Fibre Channel (FC) port between physical and virtual
tape is possible when they are in different SAN zones, you should avoid sharing a port with
a SAN-attached disk. For more information see the following technote:
http://www.ibm.com/support/docview.wss?uid=swg21194590
The ProtecTIER frees storage administrators from shuffling backup jobs and tuning tape
libraries and operations for optimal performance. Regardless of the amount of data being
streamed, virtual tape libraries back up data at the speed of the target disk.
In addition, although virtual tape libraries accept and process all tape operation commands,
they are not hit with the delays associated with allocating tape cartridges, robotic movements,
media mounting, media positioning, or media eject operations. In a tape-based environment,
these delays can last from a few seconds to a few minutes per tape.
You might be able to reduce your current backup window by taking full advantage of the
throughput performance capabilities of the ProtecTIER product. It is easy to define a greater
number of virtual drives and to schedule backups to run at the same time to maximize the
number of allowable parallel tape operations on ProtecTIER servers.
Collocation
Collocation means that all of the data for a node or node group is contained on the same set
of virtual cartridges. Because you do not have any of the restrictions as you do with physical
cartridges normally associated with this feature (such as media and slot consumption), you
can enable the option with benefit.
Taking advantage of Tivoli Storage Manager multiplexing capabilities and parallel sessions
increases backup performance but has a devastating impact of slowing down restore
performance significantly. This is primarily due to the extra time required to read through
backup images and skipping over unwanted data from other backup jobs combined into that
image.
The VTL eliminates the delays associated with moving, loading, and mounting of cartridges
and the seek times inherent to physical tape.
By recovering data from VTLs at the speed of disk, Tivoli Storage Manager can deliver an
order of magnitude improvement in recovery performance and, more important, reduce
downtime and its associated costs.
Migrating data: Do not expect an effective deduplication when you migrate your existing
data from physical tape to the ProtecTIER repository if the data was originally backed up
without suggested practices in place. Use the most current version of the ProtecTIER
product so that you implement the appropriate Tivoli Storage Manager parser, which
maximizes your overall deduplication factoring ratio.
Reducing costs
Most TCO studies compare only the cost of automated tape libraries and their associated
media to the cost of large disk storage systems with virtual tape library servers and software.
These hard costs are only part of the total equation. These TCO studies are weak on
operational cost and resort to making generalized assumptions. Strategic Research
Corporation recently completed a study that includes hard costs as well as the cost to
administer failed backups (that first-time backup error rate problem that tape struggles with),
Combining the data protection capabilities of Tivoli Storage Manager and Virtual Tape Library
Solutions with the ProtecTIER reduces the time and cost associated with managing backup
and recovery processes, and eliminates cumbersome tape media management and
maintenance activities like tape head cleaning and tape re-tensioning.
There is no doubt that the need to maintain data on physical tape media will continue for the
foreseeable future. However, supplementing Tivoli Storage Manager with the ProtecTIER
technology can enable IT managers to consolidate tape library resources, lowering hardware
replacement costs, implementation and training costs, remote vaulting services costs, and
significant annual drive and library maintenance costs.
Improving reliability
If you are looking for the best way to increase the reliability of your Tivoli Storage Manager
environment, begin by reducing the cause of most failed backup jobs, which is tape.
Tapes, tape drives, and the robotic tape libraries are mechanical devices and are prone to
failures, including these:
Tape drive failure
Media errors
Tape getting stuck in a drive
Cannot read tape due to aging or normal wear and tear
Tape not loaded
Wrong tapes loaded
Tape demagnetized
Robotic arm jam
The reliability of tape is a big issue and frequently causes backup jobs to fail, requiring
manual intervention and time spent restarting jobs to maintain the necessary level of data
protection.
Use scenarios
The LAN-free data movement in itself is fairly simple in its usage. However, when data arrives
at the destination VTL, options can be used for data reduction and disaster recovery
purposes.
In most Tivoli Storage Manager environments, backup data is written to tape and physically
transported to offsite storage or a hot site location for disaster recovery protection. This
manually intensive operation is required because electronically transporting the huge amount
of backup data generated daily is cost prohibitive.
With the ProtecTIER advanced HyperFactor data de-duplication capabilities, moving data to
and from remote locations can be done quickly and affordably. By significantly shrinking the
amount of data sent, the ProtecTIER technology reduces the bandwidth that is required to
electronically vault data to remote locations. This enables IT organizations to replicate
selected mission-critical backup data from a primary location over a WAN to a secure offsite
location.
You can use ProtecTIER replication to offload tape cloning to your secondary site. Many
users are replicating their data from the primary site to the secondary (DR) site, and then
moving it from the disk-based repository onto physical tape cartridges for long-term retention.
One of the advantages of this practice at the secondary site is that it shifts the burden of
cloning to physical tape from the production environment to the DR site location. The DR site
cloning operation uses the cartridge replicas at the ProtecTIER VTL shelf of the destination.
The process imitates the commonly used physical process for the transportation of physical
cartridges from the primary site to a DR site. This feature is effective in single domain backup
deployments because in these environments the backup application servers at both sites
share the catalog and can be concurrently connected to the ProtecTIER systems. The
replication visibility switch control feature is used in these environments. The cartridges to be
cloned are moved from the primary repository to the secondary repository and then cloned to
physical tapes.
A typical site-to-site disaster recovery model is shown in Figure 7-37 on page 245. A backup
server can either be Tivoli Storage Manager server or Storage Agent.
Future N ative
B ackup S ignificant bandwidth reduction
P rotecTIE R
S erver R eplication
Secondary Site
R epresented
capacity
P rotecT IE R P hysical
B ackup Gateway capacity
S erver
7.6.3 Protect very large databases with Tivoli Storage FlashCopy Manager
This section describes how to protect very large databases with Tivoli Storage FlashCopy
Manager integrated with Tivoli Storage Manager. The solution is based on an example but the
problem description applies to most other environments of this type, and the same concepts
can be used for that particular database system you have. We point out what to consider and
which functionalities are available to protect these types of environments, both on the client
side and on the Tivoli Storage Manager server side.
Solution benefits
Several obvious benefits to this solution should be considered. Some of them that are listed
result from the exploitation of LAN-free and ProtecTIER, based on the integration with Tivoli
Storage Manager:
RTO is improved.
Backup window is shorter.
Off-loaded backup to Tivoli Storage Manager server so that the production server is not
involved in the backup and therefore zero load on production server.
ProtecTIER provides deduplication to backup data to lower storage costs.
LAN-free to VTL removes the backup off the LAN to the SAN.
LAN-free takes backup loads away from the Tivoli Storage Manager server CPU and I/O.
The backup and recovery time is almost instantaneous with FlashCopy and FlashBack. The
restore time can vary depending on how many transactions need to be rolled forward after the
FlashBack has occurred.
Solution architecture
Big database environments are based on an enterprise server and storage infrastructure. The
solution in Figure 7-38 covers the protection of an environment based on the AIX platform
with SAP running on a DB2 database system. The concepts also apply to an Oracle
environment, so if you are using Oracle by itself or with SAP you have the same options as
described here.
DC1 DC2
Tivoli
Production Storage
Backup
Production Standby Manager
server proxy server
server server
PowerHA
Tota S
l to r age N3700
TS7650G A
13 12 11 10 9 8 7 6 5 4 3 2 1 0
SVC1 SVC2
C O M P A C T C O M PA C T
0 2 4 0 2 4
1 3 5
1 3 5
FlashCopy Space 1
FlashCopy Space 2
DS8x00 DS8x00
In this type of environment, you normally keep the production data in two data centers with
some sort of replication or mirroring between them. In this case, we use AIX LVM mirroring to
have a synchronized copy of the SAP database data on both sites. Another option is to exploit
storage replication (Metro Mirror or Global Mirror) to replicate the database volumes to the
remote site. The two servers are set up in a PowerHA where the standby server can take over
As a back-end solution for Tivoli Storage Manager server storage, we chose IBM ProtecTIER
gateway. This is the most scalable VTL solution with hardware deduplication capabilities that
IBM has today. Read 7.6.2, “LAN-free backups to VTL” on page 238 for more information
about LAN-free to VTL solutions.
Solution description
We installed Tivoli Storage FlashCopy Manager on the targeted SAP source servers to
provide the FlashCopy backup to disk. We use the technology of the advanced storage
subsystems and the SAP integration provided by Tivoli Storage FlashCopy Manager. When
the FlashCopy has been completed the disk is then mounted to a backup proxy server to
either verify the data or to copy the data to ProtecTIER so it can be managed by the Tivoli
Storage Manager server policies.
All copy services functions used by IBM Tivoli Storage FlashCopy Manager are at the volume
or LUN level. In addition, multiple volumes that are organized into volume groups require IBM
Tivoli Storage FlashCopy Manager to process these volume groups consistently. As a result,
non-application data residing on a volume group that is processed by IBM Tivoli Storage
FlashCopy Manager is included in the backup. Similarly, all data that resides on a volume
group that is being restored is overwritten. This means that the disk configuration must be
planned to be sure that the FlashCopy will include the correct data.
Because SAP environments are fully integrated with DB2, the DB2 backup command is used.
DB2 notifies Tivoli Storage FlashCopy Manager of the current environment in order to enable
Tivoli Storage FlashCopy Manager to implement the appropriate workflow. Tivoli Storage
FlashCopy Manager supports single partition databases, and logically or physically
partitioned databases on journaled file systems (JFS and JFS2). Supported DB2 backup
options are documented in the DB2 user publications. Tivoli Storage FlashCopy Manager
supports these DB2 backup functions:
Full database backups, both online and offline
Backups of selected database partitions
Backups of database partitions including or excluding database logs
Snapshots are created using space efficient FlashCopy. This minimizes the space
requirement on the DS8x00 to store the FlashCopy snapshots. However we are then also
restricted to have access to both the source volume and the FlashCopy volume to make a
restore. If we do a full copy of the source volume to the target FlashCopy volume we do not
need the source volume for restore. But it would require the same space in both the source
and target volumes and we need to copy the data before it will be available for restore. The
copy frequency would then be limited to how long time the copy will take before making
another FlashCopy.
The schedules can run at 00:30 in DC2 for DR; and at 08:30, 12:30, and 16:30 a schedule
can run in DC1. If the working hours of the company is 08:30 to 17:30, then at a maximum,
you roll forward 4 hours of logs from the production day. We suggest that you combine the
FlashCopy snapshots with the Tivoli Data Protection for ERP to Tivoli Storage Manager
server at DC2, which is closest to the Tivoli Storage Manager server storage hierarchy.
Table 7-8 shows source and target volumes and which location is used at the scheduled time.
Scenarios
Several components are involved in the backup and restore process of data when using Tivoli
Storage FlashCopy Manager:
Application and database layer (SAP DB2)
DB2 initiates the process. The database is put into backup mode to make a consistent
FlashCopy.
Backup and restore initiator
Tivoli Storage FlashCopy Manager is responsible for triggering the FlashCopy process
and for managing the retention policies of the FlashCopy snapshots.
Virtualization layer
The CIMOM agent resides on the SAN Volume Controller clustered system and is the
interface used to communicate with the SAN Volume Controller and trigger the FlashCopy
command.
The process stops here if you do not want to make a backup to the Tivoli Storage Manager
server.
Backup using Tivoli Storage FlashCopy Manager and Tivoli Storage Manager
server
If you want to make a copy of the database to Tivoli Storage Manager you can proceed with
the following steps. The process is shown in Figure 7-40 on page 250 indicated in blue.
1. Tivoli Storage FlashCopy Manager on the backup proxy server notifies DB2 that a backup
can occur.
2. DB2 initiates a backup using standard Tivoli Storage Manager methods. Tivoli Storage
Manager thinks the backup is occurring on the production server.
3. The DB2 backup uses the Tivoli Storage Manager for SAN agent to run a LAN-free
backup to VTL.
Steps for the recovery process as shown in Figure 7-41 are as follows:
1. DB2 issues standard recovery commands of db2 restore or db2 recover and interfaces
with Tivoli Data Protection for Tivoli Storage Manager to invoke Tivoli Storage Manager for
the restore.
2. Tivoli Data Protection for Tivoli Storage Manager interfaces with Tivoli Storage Manager
for SAN to perform a LAN-free restore from the tape drive connected to the production
server.
3. Tape data is transferred to the production server.
4. Production DB2 is recovered according to the recovery request.
Figure 7-42 shows the communication flow to recover data from a FlashCopy.
Steps for the recovery process as shown in Figure 7-42 are as follows:
1. DB2 issues special recovery commands to invoke Tivoli Storage FlashCopy Manager.
2. The device manager on the production server communicates with the CIMOM on the SAN
Volume Controller to request the FlashBack operation
3. The FlashBack is performed so that the tracks unchanged before the FlashCopy recovery
point are used to overwrite the changed tracks in the production database.
When you perform a restore in an LVM mirrored environment, the mirror is re-created
automatically. But the resynchronization of the volume groups is not performed. This
operation consumes numerous CPU and I/O resources. This leads to a degraded operation,
for example, in case of log recovery of the database instance was restored. Thus the
re-creation and the resynchronization of the LVM mirror must be done separately when you
decide it can run.
Forward recovery can start immediately after a FlashCopy relationship is established to apply
the transaction logs including those from the Tivoli Data Protection based backup and rolling
the database forward to a point in time.
The database at the primary server is now fully accessible, and all applications on the primary
server can start.
The SAP System Copy guidelines describe several extra actions to be performed in the
copied SAP system, as in the following examples:
Disable Remote Function Call (RFC) destinations.
Disable batch-job processing.
Some of these actions defuse the cloned system that is they anticipate the running of SAP
tasks that are planned in the production system, but that must not be performed or even
repeated in the cloned system (for example, data transfer to or from other applications, batch
jobs or spool jobs). Tivoli Storage FlashCopy Manager provides the ability to automatically
run scripts before and after clone creation and before the cloned SAP system is started.
Tivoli Storage FlashCopy Manager uses the FlashCopy function of the SAN Volume
Controller for database cloning. This method eliminates downtime and minimizes the impact
on the production database.
For FlashCopy backup, the physical volume identification numbers (PVIDs) are not changed.
For FlashCopy cloning, the PVIDs of the FlashCopy target disks are automatically changed
by the Tivoli Storage FlashCopy Manager software. You can have several cloned databases
of one source database running on one host.
With Tivoli Storage FlashCopy Manager, a cloning process can be started with an online or
offline source database. For online Tivoli Storage FlashCopy Manager cloning, the source
database is suspended for a short time. The suspension occurs when the storage system
creates its FlashCopy or snapshot of the source database.
The cloned database (target database) can have the same database name as the source
database. The cloned database can also be renamed to any valid database name during the
Tivoli Storage FlashCopy Manager cloning process. Tivoli Storage FlashCopy Manager
requires the cloned database to be created on a different database server than the source
database server, regardless of whether the clone database name is changed.
The cloning function is started from the command line on the production system. As shown in
Figure 7-43, the steps are as follows:
1. Start cloning on the production system.
2. The preprocessing scripts run against the clone database. This task is optional and
depends on available preprocessing scripts on the clone. The scripts are not part of the
Tivoli Storage FlashCopy Manager software.
3. The Device Manager on the production server notifies the CIMOM interface on the SAN
Volume Controller to initiate a FlashCopy of the volumes.
4. SAN Volume Controller sets up needed pointers for FlashCopy to occur.
5. A consistent FlashCopy or snapshot backup, including database logs, is created on the
storage server.
6. Tivoli Storage FlashCopy Manager on the production server notifies Tivoli Storage
FlashCopy Manager on the clone server that the FlashCopy is complete.
7. Mount the cloned production database on the clone system and rename the PVIDs, logical
volumes and volume groups.
When you clone databases and use SAN Volume Controller, the space-efficient disks can be
used as a target for FlashCopy cloning operations, but there are restrictions on the FlashCopy
backups. You can complete cloning operations from the cloning source volumes. If you want
to complete FlashCopy backup and FlashCopy cloning from the same source disks, use full
target disks.
Tivoli Storage FlashCopy Manager can also create a database clone on a remote storage
system. See “Metro Mirror for backup or cloning to remote site” (the next section).
Note: Cloning a FlashCopy Backup image is not supported. If multiple clones are required,
these are always created from the same running production or source system, not from an
offline backup image.
Figure 7-44 illustrates the hosts and volumes involved in remote mirroring that uses Metro
and Global mirrors and where FlashCopy Manager is installed to make FlashCopy snapshots
on both sites.
Long distance
SAN Volume Controller fabric SAN Volume Controller
primary cluster primary cluster
Figure 7-44 Remote mirroring using Metro Mirror and Global Mirror sources
Figure 7-45 shows the data movement using Metro Mirror or Global Mirror and the data
transfer to the Tivoli Storage Manager server storage hierarchy.
FlashCopy
Metro/Global Mirror
Production Standby
TSM Server
A use scenario might be to keep FlashCopy snapshots on the primary site and the secondary
site. Different configurations of the target volumes provide more flexibility in how the storage
capacity is being used:
FlashCopy target volumes that are smaller than the source volumes that have been
created for the database to enable space efficient FlashCopy snapshots on the primary
system.
FlashCopy target volumes that have the same size as the source volumes to enable full
and incremental FlashCopy snapshots on the secondary SAN Volume Controller cluster.
These copies can also be used for cloning the database.
If you want to create FlashCopy backups or clones on both the primary and the secondary
storage system, the Tivoli Storage FlashCopy Manager configuration must provide the
information of when to perform which kind of backup or clone when running the schedule.
The notion is that restoring from the FlashCopy snapshots will only occur if the replicated data
to remote site include the error that you want to overcome. Or it could be that you do not want
to make a fail over to the remote site and need to recover the data on the primary site.
So when would you make use of the FlashCopy snapshots in the disaster site? Definitely for
backup to Tivoli Storage Manager server or cloning processes. The restore of the remote
FlashCopy snapshots will be used if:
The primary source LUN is destroyed or corrupted and the replication of the data is
corrupted as well.
Since the data cannot be recovered from the primary target FlashCopy snapshots
because we used Space-efficient FlashCopy volumes we need to recover it from the
disaster site
The primary site is down and a fail over to the disaster site did not work and you need to
recover the data on the disaster site.
Note: For environments with a SAN Volume Controller version 6.1 and earlier, Tivoli
Storage FlashCopy Manager must stop and deactivate the Global Mirror and Metro Mirror
before running a restore operation.
Additional information
For more information, see the following resources:
Tivoli Storage FlashCopy Manager; system requirements and supported environments:
http://www.ibm.com/software/tivoli/products/storage-flashcopy-mgr/
IBM System Storage SAN Volume Controller:
http://www.ibm.com/systems/storage/software/virtualization/svc/index.html
Implementing the IBM System Storage SAN Volume Controller, SG24-7933 (description
and FlashCopy):
http://publib-b.boulder.ibm.com/abstracts/sg247933.html
Best Practices for Tivoli Storage FlashCopy Manager in a SAN Volume Controller Metro
Mirror environment:
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102196
Infrastructure Solutions: Design, Manage, and Optimize a 20 TB SAP NetWeaver
Business Intelligence Data Warehouse, SG24-7289:
http://www.redbooks.ibm.com/abstracts/SG247289.html
IBM ProtecTIER Implementation and Best Practices Guide, SG24-8025:
http://www.redbooks.ibm.com/redpieces/abstracts/sg248025.html
Tivoli Storage FlashCopy Manager reconciliation:
http://www-01.ibm.com/support/docview.wss?uid=swg21618086
The big challenge is to find the best way to protect this environment, which sees millions of
new files being written every day to single file systems, making sure backups complete
successfully in a timely manner and that data is restorable also in a timely manner. New
technologies should be considered when you want to back up files systems that have millions
of files.
In this section, we explain what components you can use with Tivoli Storage Manager to back
up unstructured data, both on local file systems and in remote NAS devices, based on the
matrix shown in Figure 7-46. Starting at the most traditional backup method of progressive
incremental backups, we describe other alternatives for the backup of large file systems.
For several reasons, you might have use different procedures to do a successful and reliable
backup, always with the goal to be prepared for a fast and effective restore. The various
functions are described in the next topics. Figure 7-47 on page 260 shows the procedures
and functions that are available and can be combined (analogy is a gear shifter knob). Notice
that progressive incremental and GPFS policy-driven backups are grouped together. The
Tivoli Storage Manager backup-archive client is integrated with the GPFS command mmbackup
to provide a scalable and performant backup solution for GPFS file systems. Therefore the
progressive incremental backup process as provided from Tivoli Storage Manager
backup-archive client can be replaced by mmbackup, provided from GPFS.
135
Journal-Based Software-Based
2
Backup Snapshots
Snapshot- 24R
TSM FastBack
Assisted
3 Progressive 4 Hardware-
Incremental Based
Snapshots
GPFS Policy-
FlashCopy Manager
Driven Backup
R Restore
Figure 7-47 Like a gear shift, use the methodologies for backup
Progressive incremental backup relies on a local file system scan and a query of active
entries in the Tivoli Storage Manager server database.
Solution architecture
Preferred storage pool destination is disk or file, because the small files that are sent directly
to tape can cause tape drive contention. This is also true for the journal-based backup
solution in 7.7.2, “Journal-based backups” on page 264.
Running an incremental backup, the backup-archive client first inspects the file on the local
systems and compares it with files that are already backed up on the server and have not
been changed on the client. If it finds a file on the client that has been changed, it backs up
that file.
The progressive incremental backup shown in Figure 7-48 has these steps:
1. Client queries the server for a current view of file system (dsmc incremental).
2. Server returns a list of files for the entire file system (db query).
3. Client scans and compares the local file system.
4. Client creates transactions of files for backup.
During the inspection phase (while the client is checking if files are changed), the session
remains idle. Because the object inspected number is high and the session is idle during that
time, your connection is closed by the Tivoli Storage Manager server based on the value of
For very large file systems, the time spent to inspect the file system might add so much time
to the overall time of the backup that the available backup window is exceeded. You might
need to implement other options, as discussed in this book. The time required to enumerate
the files is done at relatively the same speed that the dir command would list all the files on
the computer. Another reason is that if the number of files is very large, more memory will be
required. Tivoli Storage Manager requires approximately 300 bytes of memory per file to
inspect. When the number of files is very large, more memory will be used and can affect
performance.
Use scenarios
You can use progressive incremental to backup and recover CIFS/NFS file system data on
NAS devices with progressive incremental as shown in Figure 7-49.
There are two ways to set up progressive incrementals with CIFS/NFS file systems:
Use a Tivoli Storage Manager backup-archive client to back up and restore data, by using
CIFS or NFS to access files from the backup-archive client. Data can be stored on the
Tivoli Storage Manager server with file-level granularity, by using the progressive
incremental backup method. The data is stored in the Tivoli Storage Manager storage
hierarchy and can be migrated, reclaimed, and backed up to a copy storage pool.
This method increases processor usage when the Tivoli Storage Manager client accesses
individual files. Data flows through the Tivoli Storage Manager client and through the Tivoli
Storage Manager server, unless a LAN-free configuration is used.
Use a Tivoli Storage Manager backup-archive client running on the NAS device, if you can
use external programs with the NAS operating system. Data can be stored on the Tivoli
Storage Manager server with file-level granularity using progressive-incremental backup.
The data is stored in the Tivoli Storage Manager storage hierarchy and can be migrated,
reclaimed, and backed up to a copy storage pool.
This method decreases processor usage of CIFS or NFS. Data flows through the Tivoli
Storage Manager client and through the Tivoli Storage Manager server, unless a LAN-free
configuration is used.
The Tivoli Storage Manager backup-archive client can be configured to protect files which are
accessed with the Network File System (NFS) protocol.
Backup performance is better when you install the backup-archive client where the file system
physically resides, but sometimes it is necessary to access file systems using NFS for
purposes of backup and recovery. The Tivoli Storage Manager UNIX and Linux
backup-archive client can back up, archive, restore and retrieve file data using an NFS mount.
This includes all versions of the NFS protocol.
The Windows client backs up the CIFS share definition under the root directory, the mapped
CIFS share, or the UNC name. This support requires that the NAS file server is running DATA
ONTAP software which presents CIFS shares to remote clients as ordinary remote NTFS
shares.
The output in Example 7-3 is displayed when the root directory (and share definition) is
backed up.
Network traffic between the client and the server is also reduced because less pre-backup
communication happens. The backup-archive client will still send data as usual to the server
to be stored and at this time the network traffic is the same without journaling.
Because the backup-archive does not carry out the initial metadata conversation, it does not
sit idle, waiting for the list of files to be backed up. The backup-archive client begins to send
the files to the Tivoli Storage Manager as soon as the journal-based backup is initiated. This
means faster backup times and less backup-archive client idle time.
Solution description
With journal-based backups, the file system and Tivoli Storage Manager database scans are
not performed to determine which files to backup. Instead, a local database is created with all
changes that occur in the file system before backups. Figure 7-50 on page 265 explains how
journal-based backups work.
A change journal is a database of file system change information. The Tivoli Storage
Manager Journal daemon creates and maintains a change journal for each file system that is
being journaled, as specified in the Tivoli Storage Manager Journal Service tsmjbbd.ini
configuration file.
A backup of a particular file system will be journal-based when the Tivoli Storage Manager
journal daemon has been installed and configured to journal the particular file system, and a
valid journal has been established for the file system. Journal-based backup is enabled by
successfully completing a full incremental backup.
The primary difference between traditional incremental backup and journal-based backup is
the method used for backup and expiration candidates.
A journal database will be created in the client server and sometimes its settings need to be
refined for better performance or troubleshooting. See the following technote to estimate the
size of a journal database:
http://www.ibm.com/support/docview.wss?uid=swg21213920
Use scenarios
You will use journal-based backup when backing up file systems with small or moderate
amounts of change activity between backup cycles. If you have many file changes between
backup cycles, you will have very large change journals. Large change journals might create
memory and performance problems that can negate the benefits of journal-based backup.
For example, creating, deleting, renaming, or moving very large directory trees can also
negate the benefit of using journal-based backup instead of normal incremental backup.
Initial successful progressive incremental backup of entire file system must be performed to
enable the journal.
Solution description
You can back up a logical volume as a single object (image backup) on your system. The
traditional static image backup prevents write access to the volume by other system
applications during the operation.
You must be a root/administrative user to perform this task on AIX, HP-UX, Linux, Solaris and
Windows operating systems, and image backup does not apply to Mac OS X. You can
perform the backup only on formatted volumes; not on raw logical volumes. Windows uses
VSS to create snapshot of volume.
Use scenarios
We describe two backup methods to be used with image backups that allow you to perform a
point-in-time restore of your file systems and improve backup and restore performance. You
can always use image backups alone but then you do not have the possibility of restoring
single files and it will generate larger backup volumes.
Restore your data by performing an incremental restore. Before you start the restore, select
the image plus incremental directories and files, and delete inactive files from local options in
the Restore Options window. During the restore, the client does the following steps:
1. Restores the most recent image on the server.
2. Deletes all of the files restored in the previous step that are inactive on the server. These
are files that existed at the time of the image backup, but were subsequently deleted and
recorded by a later incremental backup.
3. Restores new and changed files from the incremental backups.
Restore your volume by performing an incremental restore. Before you start the restore,
select the Image plus incremental directories and files option in the Restore Options window.
This first restores the most recent image and then restores all of the incremental backups
performed since that date.
Perform full image backups periodically, which improves restore time. The file system can
have no previous full incremental backups.
Incremental-by-date image backup does not inactivate files on the server. When you restore
an image with the incremental option, files deleted after the original image backup are
present after the restore.
Files are expired on the server when they are Files are not expired on the server. After the
deleted from the file system. On restore, you have image incremental restore completes, all files
an option to delete files that are expired on the that are deleted on the file system after the image
server from the image. backup are present after the restore. If file
systems are running at or near capacity, an
out-of-space condition might result.
Incremental backup time is the same as regular Incremental image backup is faster because the
incremental backups. client does not query the server for each file that
is copied.
Restore is much faster compared to a full Restore is much faster compared to a full
incremental file system restore. incremental file system restore.
Directories deleted from the file system after the Directories and files deleted from the file system
last image backup are not expired. after the last full image backup are not expired.
Note: These are only suggested numbers, which can be used as a baseline.
Table 7-10 RTO table showing greater than (>) and less than (<) values
Client data
Medium: > 4 hours but Progressive backup Progressive backup Progressive backup
< 24 hours + journal + journal + journal
+ image backup + local snapshot / non
Tivoli Storage
Manager
Solution architecture
The NDMP backup and restore features are fully integrated with Tivoli Storage Manager
Extended Edition server and client. No extra software is required on the server, client, or NAS
appliance. When doing backups and restores, the NAS device and the Tivoli Storage
Manager server and client all have specific roles, as shown in Figure 7-51 on page 270.
During backup and restore operations, data flows directly between the NAS appliance and the
tape drive. NDMP for NAS backup uses either an SCSI-attached tape device local to the NAS
appliance, or a SAN-attached SCSI or Automated Cartridge System Library Software
(ACSLS) device that can be shared with the Tivoli Storage Manager server. Library robotics
can be controlled directly by the Tivoli Storage Manager server or by passing SCSI
commands through the NAS file server.
Solution description
Tivoli Storage Manager combines NDMP methodology with existing Tivoli Storage Manager
infrastructure and capabilities allowing the following items:
Administrative authority and remote access
Policy management
Library and drive sharing
Tape management
System and operation management (scheduling, reporting, logging)
NDMP backup methods are either full or differential. Full backup includes all files within a
NAS file system or directory tree. Differential backup includes all files that have changed
since the most recent full backup. If differential backup is specified and full backup not found,
a full backup is performed.
Tivoli Storage Manager tracks file-system image backups and can perform NDMP file-level
restores using Direct Access Recovery (DAR).
During a restore of individual files (as opposed to directory tree or file system), Tivoli Storage
Manager accesses TOC to get position information for each file to be restored. Tivoli Storage
Manager initiates a DAR operation, providing position information for each file. NAS device
positions directly to each file, avoiding scan of the entire image.
If the full path of the file is known, use the server’s RESTORE NODE command. The difference is
that the restore will not be a DAR restore, just a scan.
Figure 7-52 shows how backup and restore work with the table of contents.
DB
Tape Library
Storage location of TOC is specified in Tivoli Storage Manager policy and we suggest that it
stays on disk storage for fast access.
When performing a file-level restore for a NAS file server, the table of contents (TOC) is
loaded into temporary database tables in order to choose files to restore. The amount of
space required depends on the average length of the file and directory names and the
average depth of the directory structure. The amount of space also depends on the file server
vendor and whether non-English characters are used in file and directory names.
In general, the amount of space required is about 280 bytes for each file or directory in the
TOC when using English file and directory names. If the NAS file server is Network Appliance
and contains non-English file or directory names, the amount of space required is about 340
bytes for each non-English file or directory in the TOC.
Tivoli Storage Manager versioning applies to the complete NDMP dump. Tivoli Storage
Manager server is not aware of the single objects included in the NDMP dump (except when
reading the TOC).
Offsite protection
The following operations are supported with NDMP:
Back up storage pool.
Restore storage pool or volume.
Move data:
– Intra-pool for space recovery
– Inter-pool for migration to new device type
Disaster Recovery Manager has support for NDMP data.
Restore node uses copy pool if primary data is not accessible.
Use scenarios
Tivoli Storage Manager provides two of the most common configurations that use NDMP for
backing up and managing NAS file servers.
1 2
Legend:
TCP/IP Connection
Control
Data Flow
NDMP filer-to-server
Another variation is to use Tivoli Storage Manager hierarchy to send backups from NAS
devices using Network Data Management Protocol (NDMP).
With the filer-to-server configuration, the library is attached to the DMA (Tivoli Storage
Manager server). The NAS device does not have access to the library, as shown in
Figure 7-54 on page 274. This configuration is also known as 3-way NDMP backup. The
backup data from the NAS device is transferred over the network (TCP/IP) to the Tivoli
Storage Manager server.
The NDMPCONTROLPORT option specifies the port number to be used for internal
communications for certain NDMP operations. The Tivoli Storage Manager server does not
function as a general purpose NDMP tape server.
Specify the port number to be used for internal communications for certain NDMP operations.
The port number must be in the range of 1024 - 32767. The default is 10000.
Firewall considerations are more stringent than they are for filer-to-attached-library because
communications can be initiated by either the Tivoli Storage Manager server or the NAS file
server. NDMP tape servers run as threads within the Tivoli Storage Manager server and the
tape server accepts connections on port of 10001.
This port number can be changed through the options file (Example 7-4) in the Tivoli Storage
Manager server options file shown in “Hardware and software requirements” on page 192.
During NDMP filer-to-server backup operations, you can use the NDMPPREFDATAINTERFACE
option to specify which network interface the Tivoli Storage Manager server uses to receive
NDMP backup data. The value for this option is a host name or IPv4 address that is
associated with one of the active network interfaces of the system on which the Tivoli Storage
Manager server is running. This interface must be IPv4-enabled.
Before using this option, verify that your NAS device supports NDMP operations that use a
different network interface for NDMP control and NDMP data connections. NDMP control
Figure 7-55 shows the NAS file server with control of the library.
Tape Library
Tivoli Storage
Manager
Web Client
1
(Optional)
2
Legend:
TCP/IP Connection
Control
Data Flow
1 Robotics Control
NAS File System
2 Drive Access
Figure 7-55 NAS file server controls the library
For the procedure, see the Quick NDMP Setup for Tivoli Storage Manager white paper:
https://www-304.ibm.com/software/brandcatalog/ismlibrary/details?catalog.label=1TW
10SM36#
NDMP operations require a specialized device class that is not set up in the default
configuration. Therefore, you must define it yourself. Indicate this information in the device
class definition:
Specify NAS as the value for the DEVTYPE parameter.
Specify the MOUNTRETENTION=0 setting, which is required for NDMP operations.
Specify a value for the ESTCAPACITY parameter. This should correspond to the nominal
capacity of the tape media being used for the tests.
NDMP media requires its own distinctly formatted storage pool. Only one data format is
available for general use, NDMPDUMP. Use dataformat=ndmpdump when you define the
storage pools for NDMP backups.
For NDMP backups, rather than use the standard backup client we use a data mover, which
provides the Tivoli Storage Manager server with a communication path to the NAS filer. Each
data mover must be associated with a node, which is the first parameter after the define
datamover command. The data format must be specified as ndmpdump. The usual low level
address is 10000.
When it is time to define library and drive paths, whether you use configuration 1 or 2 will
determine how you set them up:
For configuration 1 the Tivoli Storage Manager server will control the library robotics, so a
path must be defined between the server instance and the robotics port.
For configuration 2, robotics control is done by the NAS filer, so a path must be defined
between the NAS node and the library.
Note: The device name to use in this case is the device parameter used by the NAS server
to identify the library robotics.
Tape drives can be dynamically shared in the Tivoli Storage Manager with the use of library
manager. See Figure 7-56 for details.
Library
Library control Data flow
Manager
Client Metadata
NAS
Client Device
Client
LAN
NAS
Device
SAN Drive 1
Drive 2
Storage Drive 3
Agent
Drive 4
Client
Library Tape
Client Client Library
Storage
Client Agent
For NAS and IBM System Storage N series file servers that are running ONTAP 7.3.0, or
later, you can use the snapdiff option to start the snapshot differencing backup from NetApp
when you run a full-volume incremental backup. Using this option reduces memory usage
and is faster.
Consider the following restrictions when running a full-volume incremental backup using the
snapdiff option, to ensure that data is backed up when it should be.
A file is excluded because of an exclude rule in the include-exclude file. Tivoli Storage
Manager runs a backup of the current snapshot with that exclude rule in effect. This
happens when you have not made changes to the file, but you have removed the rule that
excluded the file. NetApp will not detect this include-exclude change because it detects
only file changes between two snapshots.
The NetApp software (not Tivoli Storage Manager) determines what is a changed object. If
you run a full volume backup of an NFS-mounted or a CIFS-mapped NetApp or N series
volume, all the snapshots under the snapshot directory might also be backed up. To avoid this
situation, you can do one of the following actions:
Run NDMP backups.
Run backups using the snapshotroot option.
Run incremental backups using the snapdiff option.
Tip: If you run an incremental backup using the snapdiff option and you schedule
periodic incremental backups, use the createnewbase=yes option with the snapdiff
option to create a base snapshot and use it as a source to run an incremental backup.
Note: The .snapshot directory is not backed up for some versions of Red Hat Linux, so
you are not required to exclude it.
Snapshot differencing is a method that Tivoli Storage Manager provides in combination with
the application interface that NetApp supports to use the snapshot functionality to determine
what data changed and now must be backed up.
With Tivoli Storage Manager Version 6.4 we have a full function solution package for NetApp
and N series file servers, which can use snapshot functions to provide an optimized backup
strategy. This includes the following items:
File Access Protocol
– Data OnTAP 7.3.3 and 8.1 have a feature named File Access Protocol on the snapshot
differencing API, but not on OnTAP 8.0.
– Enables snapshot differencing to handle file names with 7-bit ASCII characters and
also non 7-bit ASCII character correctly either through NFS or CIFS.
Supports NetApp SnapDiff vFiler in ONTAP 8.1.1.
Snapshot differential backup of SnapMirror-restricted (read-only) mirror volumes.
This gives you overall flexibility to back up the network-attached file systems that are hosted
on a NetApp or N series system.
Also know that you can use your standardized backup methodology which is incremental
forever on a file level basis.
Solution description
As mentioned in Chapter 3, “Data protection with Tivoli Storage Manager” on page 33, the
NetApp snapshot difference API can be used to remove the long scan time Tivoli Storage
Manager incremental backup needs to find the data that needs to be backed up.
The N series or NetApp volumes should be connected through NFS or CIFS over a NAS
network to Linux, AIX, or Windows Backup-Archive client hosts. On Linux or AIX, files are
accessed through NFS using UNIX file semanticsm and with secure access using Kerberos
authentication. On Windows, files are accessed through CIFS shares using Windows file
semantics with NTFS Security style.
Data OnTAP 7.3 or later is supported. Only entire volumes can be specified on the
incremental command. Read-only volumes are not supported, unless in a SnapMirror
relationship. Qtrees and subdirectories are not supported. vFilers are now also supported.
SnapDiff Backup
TSM Server
vol 1
SnapMirror support
You can use NetApp SnapDiff backup processing in conjunction with NetApp SnapMirror
replication to back up NetApp or N series production or remote filer volumes.
The overview in Figure 7-58 on page 281 shows how the components relate.
SnapDiff Backup
Via File Sharing
TSM
(CIFS/ NFS)
Server
Snapshot
Scheduler
vol 1 vol 1_mirror
vol 2 vol 2_mirror
The new support allows snapshot differencing backup of SnapMirror restricted (read-only)
mirror volumes. By default, Tivoli Storage Manager creates base and difference snapshots on
volumes being backed up, which is not possible on read-only mirror volumes.
A typical configuration is to offload the backups from the source filer by creating backups of
the source volumes by using the replicated volume snapshots stored on the destination filer.
Ordinarily, backing up a destination filer presents a problem because creating a snapshot
differencing backup requires that a new snapshot be created on the volume that you are
backing up. The destination filer volumes that mirror the contents of the source volumes are
read-only volumes, so snapshots cannot be created on them.
To overcome this read-only restriction, Tivoli Storage Manager provides client configuration
options so you can use the existing base and differential snapshots on the read-only
destination volume to back up changes to the Tivoli Storage Manager server.
For more Information about the N series functions in Tivoli Storage Manager, see IBM Tivoli
Storage Manager for Windows Backup-Archive Clients Installation and User's Guide,
SC23-9792.
Use scenarios
NetApp Snapshot Differencing was implemented for environments with “big fat filers” that
house millions of files, with unacceptable backup windows. Using incremental backup by
NetApp, the Tivoli Storage Manager Client does not need to crawl the file space looking for
changed files, but instead queries the ONTAP OS on a filer for files that have changed since
the last -snapdiff or -diffsnapshot backup.
This method specifically increases the speed of incremental backups of filers with many files,
and with a small percentage of changed files. In a classic incremental backup, the Tivoli
Storage Manager Client can spend hours on a large file system, searching for changed files.
In both extremes, NetApp Snapshot Differencing performs better than classic incremental, but
especially so when relatively few files have been changed. Testing was also performed on a
The solution is combined with your traditional incremental forever backup strategy. For the
backup window, this means that scan time is reduced to find what data has changed. You can
compare it with journal-based backup, where the compare-time is also reduced. To determine
what time-slice in your overall backup time the compare time is, query the summary table, as
shown in Example 7-5, to find the idle time amount.
The first incremental backup taken with the -snapdiff option creates the base snapshot, to
which the next incremental backup will reference. After this point, the idle time in the summary
table should tend to zero.
When you use the incremental backup with snapdiff option, remember the side effects that
can occur. A file change or metadata change results in file backup. Mass directory changes
are handled on the directory level so that only an incremental backup of that directory takes
place. If a file is deleted from the local system, the active version is expired on the Tivoli
Storage Manager server. That means the full version and retention control on an object basis
will be considered. A file that is unexpectedly removed from the Tivoli Storage Manager
server results in a new file backup. Because the Tivoli Storage Manager server is not
returning information about current version, events such as file deletion from Tivoli Storage
Manager server, new include or exclude rules, copy mode, and frequency are not detected
and honored.
If you have an environment with NetApp or IBM System Storage N series filers in a
SnapMirror relationship, perform the incremental backup with the snapdiff option from the
system to which the snapshot is synchronized. That means, refer to the snapshot, created on
the primary system and copied to the SnapMirror system, with these options:
-useexistingbase, -diffsnapshot, -basesnasphotname, and -diffsnapshotname.
Example 7-6 shows how these options are used in the command to back up the latest nightly
snapshot, created by the default snapshot scheduler.
With this solution you are able to manage your files and objects on a single object level. The
interface for backup and restore is the standard backup-archive client. Command line, GUI
client, and web client can all be used. Each version of each file will be stored as a single
object in the database. It fits in the standard configuration, so that no special knowledge for
the backup administrator is necessary.
Table 7-11 lists which levels of the 6.2, 6.3, 6.4, and 7.1 IBM Tivoli Storage Manager
Windows, AIX, and Linux x86/x86_64 Backup-Archive clients are supported with which levels
of NetApp Data ONTAP for the NetApp snapshot-assisted progressive incremental
functionality.
6.2.0 and 6.2.1 Windows, AIX 64-bit 7.3.0, 7.3.1, and 7.3.2
6.2.x (where x is 2 or higher) Windows, AIX 64-bit, and Linux 7.3.y (where y is 0 or higher)2
x86 and 8.0.1
Additional information
For further information about implementing the snapshot differencing solution, see the
following resources:
Using the IBM System Storage N series with IBM Tivoli Storage Manager, SG24-7243:
http://www.redbooks.ibm.com/abstracts/sg247243.html
Snapshot Differencing Caveats page in IBM developerWorks highlights peculiarities in the
way the NetApp snapshot differencing works as compared to standard backups:
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli%20St
orage%20Manager/page/Snapshot%20Differencing%20Caveats?lang=en
In case of restoring data, selective restore operations can be performed without the need to
restore a full backup.
With the Tivoli Storage Manager server, administrators of SONAS can do these tasks:
Perform a scheduled full-incremental backup of a file system.
Perform an individual full-incremental backup of a file system started manually
Restore several single file system objects, objects according to file pattern, whole
directories, and all subdirectories or a whole file systems from Tivoli Storage Manager
server.
Query previous or active backup sessions.
Show the log files generated by backup and restore sessions.
SONAS provides clustered serviceability to its users. In case of a failure of interface nodes,
access to data is provided through the alternate interface nodes.
If helper backup nodes fail during the active backup session, all backup processes of that
interface node are redistributed to the remaining backup nodes and backup continues to
the Tivoli Storage Manager server.
If the primary backup node fails during the active backup session, the entire backup fails.
You will need to restart the backup again after the interface node becomes healthy.
Because the restore occurs only on a single interface node, if that particular interface node
fails during the active restore session, the entire restore process fails. You must restart the
restore process after the interface node becomes healthy.
If the management node fails during the backup or restore session, the backup or restore
session continues. After the management node returns to a good state, the status of
backup or restore request along with the corresponding result is reported to the
management node.
Solution architecture
The Tivoli Storage Manager client is directly installed on the SONAS interface nodes; data is
backed up directly from the interface nodes to the Tivoli Storage Manager server as shown in
Figure 7-59 on page 285. SONAS integrated the GPFS policy scan engine internally to scan
the inode table of a file system. This is much faster than the conventional system calls,
reducing the overall time required for the backup.
Client network
TSM clients on
interface nodes
Hierarchy managed by
Tivoli Storage Manager
technology
SONAS TSM client is preinstalled on the interface nodes and back up data
directly from the interface nodes of SONAS to the TSM backup server.
Multiple Tivoli Storage Manager servers can back up a SONAS system. All Tivoli Storage
Manager servers are connected to the network and reachable from the interface nodes of
SONAS. It is possible to have one Tivoli Storage Manager server configured per file system
but it is not possible to have multiple Tivoli Storage Manager servers backing up the same file
system. All Tivoli Storage Manager servers do not have to be on the same release. However,
verify that they all are compatible with the Tivoli Storage Manager client version on SONAS.
The mmbackup command does not need to check each file against the Tivoli Storage Manager
server to identify changes. It uses the policy engine and a shadow database of Tivoli Storage
Manager inventory to identify changes in the file system. File changes are tracked by GPFS.
Each File Module then operates on its own file-lists, by default spawning one thread. The
mmbackup command calls the /usr/lpp/mmfs/bin/mmexectsmcmd script, which uses the Tivoli
Storage Manager commands shown in Example 7-7.
Several scripts are provided to define the File Modules in the backup, the relationship of
which file system needs to be backed up to which Tivoli Storage Manager server, and to start
and stop backups or restores.
Use scenarios
The integrated Tivoli Storage Manager client solution enables enterprises to back up and
restore data in a minimum time period. It increases efficiency, performance, and reliability of
the backup and restore operations. Taking a full backup every time ensures reliability of
backups. However, not all the data changes every time a new backup is invoked, and the time
required for a full backup increases the backup window. To reduce duplication of backed up
data, you need to back up only those files and objects that have been changed or created
since the last backup. This identification can be done with a full file system traversal using
standard system calls, but the time increases linearly with the number of objects in the file
system. SONAS integrated IBM General Parallel File System (IBM GPFS) policy engine
scans the inode table of a file system, which is much faster than the conventional system calls
reducing the overall time required for the backup. SONAS also leverages its multiple interface
nodes in the backup process to further reduce the time needed for backups.
Tivoli Storage Manager client software is preinstalled on SONAS on each of the interface
nodes. This is done during the SONAS software installation. Not all interface nodes need to
participate in the backup and restore activities. All interface nodes that are configured for
backup purposes are called backup nodes. The first backup node that is configured with Tivoli
Storage Manager Server is called the primary backup node. Other backup nodes are called
as helper backup nodes.
See the Integrating IBM SONAS with IBM Tivoli Storage Manager white paper for more
details:
http://public.dhe.ibm.com/partnerworld/pub/whitepaper/19356.pdf
GPFS provides a single name space across all pools. Files in the same directory can be in
different pools. Files are placed in storage pools at creation time using placement policies.
Files can be moved between pools based on migration policies and files can be removed
based on given policies.
GPFS implements an engine for fast file system scans and file-system monitoring, which is
also known as policy engine. The policy engine implements a user programmable interface
which is called the GPFS policy rule language. This section focuses on the use of this engine
in conjunction with the Tivoli Storage Manager for Space Management client.
GPFS 3.2 introduces a new feature called external storage pools. You can set up external
storage pools and GPFS policies allowing the GPFS policy manager to coordinate file
migrations from a native GPFS online pool to external pools on the Tivoli Storage Manager
server. The GPFS policy manager invokes the migration through the HSM client command
line interface.
The migration candidate selection is identical to the GPFS native pool-to-pool migration rule.
The Policy Engine uses scripts to call the dsmmigrate -filelist Tivoli Storage Manager
command for the migration of files from a native storage pool to the Tivoli Storage Manager
server.
An HSM solution typically moves the file’s data to back-end storage and leaves a small stub
file back in the local storage. The stub file consumes minimal space but leaves all metadata
information on the local storage in such a way that for a user or a program, the file looks like a
normal local stored file. When the user or a program accesses the file, the HSM solution
automatically recalls (moves back) the file’s data from the back-end storage and gives the
reading application access to the file when all the data is back online. The back-end storage
for Tivoli Storage Manager HSM is the Tivoli Storage Manager server, which handles tier2
(disk storage) and tier3 (tape storage) of the storage hierarchy. That means Tivoli Storage
Manager HSM virtually extends the managed file system with the space that is provided by
the Tivoli Storage Manager server. Migrating files to the Tivoli Storage Manager server frees
space for new data on your local file system and takes advantage of lower-cost storage
resources that are available in your network environment.
Setting up external storage pools and GPFS policies allows GPFS policy manager to
coordinate file migrations from a native GPFS online pool to external pools on the Tivoli
Storage Manager server. The GPFS policy manager invokes the migration through the HSM
client CLI.
HSM client on AIX and Linux GPFS, and AIX JFS2 support LAN-free data transfer. With
assistance from GPFS, the need for a file system scan with HSM is eliminated, allowing Tivoli
Storage Manager for Space Management to migrate data more efficiently.
V7000 Unified
IBM Storwize V7000 Unified is a unified storage product, providing file storage through
network-attached storage (NAS) protocols, and block storage through block storage protocols
such as Fibre Channel and iSCSI. The NAS part of V7000 Unified allows storing files through
standardized protocols such as NFS and CIFS.
V7000 Unified includes backup module with Tivoli Storage Manager backup client (V 6.3). It
can be configured to back up data to external Tivoli Storage Manager server.
Incremental backups are performed per file system via LAN interfaces, and exploit IBM Active
Cloud Engine® for fast identification of files. Tivoli Storage Manager client architecture is
shown in Figure 7-61.
The backup is scheduled on V7000 Unified that starts the mmbackup utility. That utility invokes
Tivoli Storage Manager backup client to copy files to the Tivoli Storage Manager server. The
backup client can be configured to run on both file modules in parallel.
Restoring files, directories, or file systems can be done through the GUI or CLI, but only one
restore process at a time.
V7000 Unified can also be configured to automatically move a file between different tiers of
storage. The integrated information lifecycle management (ILM) function allows the moving of
files between disk-tiers, which are managed by the V7000 system. Furthermore the HSM
capability allows moving files from the V7000 managed disk storage to an external Tivoli
Storage Manager server while keeping the access to the file transparent.
The V7000 Unified includes Tivoli Storage Manager HSM client for migrating files from V7000
Unified managed disk to tape, which is attached to an external Tivoli Storage Manager server.
The local Active Cloud Engine, which is an integral part of the V7000 Unified, in combination
with the Tivoli Storage Manager HSM client and Tivoli Storage Manager server is used to
perform policy-based migration of files from V7000 managed disk to external tape, managed
by an external Tivoli Storage Manager server. For example, policies can be based on age or
last-access time of files, whereby files older than 1 year or files that have not been accessed
for the last 30 days can be migrated to an external Tivoli Storage Manager server with
attached tape storage.
The Tivoli Storage Manager HSM client can use the same Tivoli Storage Manager server as
the Tivoli Storage Manager backup client, which is configured for the file system. Using the
same Tivoli Storage Manager server for file system backup and migration enables the inline
backup function (when a file that was migrated is backed up, the file will not be recalled and
copied again to the Tivoli Storage Manager server). Instead the Tivoli Storage Manager
server performs an inline copy of the file from the HSM storage pool to the backup storage
pool of the Tivoli Storage Manager server. This improves operational efficiency and reduces
the amount of data being transferred over the network.
HSM is enabled on file-system basis and operates on the entire file system. Policies and rules
can be configured to operate on a file set basis. If multiple file systems are configured for
HSM, one or more Tivoli Storage Manager servers can be used as a migration target,
however a single file system can have only one migration destination server.
HSM can be configured to run on one or both file modules of the V7000 Unified system. The
recommendation is to configure Tivoli Storage Manager HSM to run on all file modules for all
file systems, because the workload (file migration) is distributed among the file modules
(approximately 100 files per file module for each chunk).
When files are migrated from the disk storage to an external Tivoli Storage Manager server,
these files remain represented in the name space of the file share, facilitating transparent
access from a user perspective. If the user accesses a migrated file, the HSM client performs
a recall of the file from the Tivoli Storage Manager server.
See the IBM Storwize V7000 Unified - TSM HSM Overview and Implementation white paper,
which shows how to implement HSM in V7000 Unified:
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102116
Use the NDMP SnapMirror to Tape feature as a disaster recovery option for copying very
large NetApp file systems to secondary storage. For most NetApp file systems, use the
standard NDMP full or differential backup method.
Solution architecture
Tivoli Storage Manager NetApp and N series integration enhancements address the backup
challenges associated with large filers. NDMP SnapMirror to Tape feature provides an
efficient disaster recovery or vaulting solution.
NDMP backups of very large NAS devices can be slow. SnapMirror to Tape, also called
SMTAPE at ONTAP 8, is a block-level movement of a flexible volume or volume. The function
is similar to backup-archive client backup image, which is covered in 7.7.3, “Image backups”
on page 266. The data written to tape is a block by block copy of the file system. No time is
wasted to search through the file system. The block structure is preserved so that
deduplicated data savings are kept. Another advantage is that the whole snapshot structure is
saved and will be available after restore process. Because the disk geometry is used,
restoration of traditional volume to unlike geometry can be very slow. The implementation is
provided by extending NDMP functions.
The transport layer for the backup and restore with SnapMirror to Tape uses the standard
NDMP protocol. You have the choice to control the data flow. Either, filer-to-tape, which
means LAN-free data transfer, or filer-to-server through LAN can be used, controlled by the
destination storage pool definition. Only the additional option type=snapm is necessary to
switch between traditional NDMP backup and SnapMirror to Tape. Every backup process will
create a separate backup version of the volume on the Tivoli Storage Manager server. Even if
To restore a volume, and it is always a whole volume, it must be prepared and put in restricted
mode. After the restore, it will automatically switch to normal mode. The destination of the
retrieval must use the same or later version of Data ONTAP. There is a different file system
format between traditional volume and the Flexvol volume; the FlexVol volume cannot be
restored to a traditional volume and vice versa. Restoring to the same disk geometry as
backup should be done for best performance results.
Information about SnapMirror copies can be displayed with the following command:
Query NASbackup Type=SNAPM
SnapMirror to Tape is a preferred solution for disaster recovery or offsite vaulting. It is not a
replacement for traditional file-based backup like incremental forever or full or differential
Tivoli Storage Manager NDMP backup.
Use scenarios
If you need a fast disaster protection of your NetApp or N series filers, SnapMirror to Tape is a
valid solution. You can do a quick protection of whole volumes for fast restores. The big
advantage is that all snapshots including all pointers will be available after restore.
Consider that this kind of data protection is almost an addition to any of the standard
protection solutions you will use. For example, you will do a daily NDMP based file level
backup and at the weekend an additional backup with SnapMirror to Tape will be scheduled.
Or you can take a daily incremental file level backup with the snapdiff option instead. Even if
different node names are used and the data resides on different storage pool targets, the data
will fit together after the restore process. In the case of a disaster, you will restore the broken
volumes completely from the SnapMirror to Tape image and later on the files, which are
changed after the time that the SnapMirror to Tape backup was taken. You can either use the
files saved with NDMP or the incremental backup. If restoring the files from the incremental
backup, the option -ifnewer can help you select the involved objects.
The -ifnewer option replaces an existing file with the latest backup version only if the backup
version is newer than the existing file.
Figure 7-63 shows how the solutions work together. While the standard backup method for all
filers is NDMP and can also be used for archiving, SnapMirror to Tape should be used for
disaster protection.
Hardware requirements
The hardware requirements are as follows:
Tivoli Storage Manager Server on Windows, UNIX, or Linux
NetApp FAS or IBM System Storage N series
Software requirements
The software requirements are as follows:
Tivoli Storage Manager Server Version 6.1 or later
Tivoli Storage Manager Backup-Archive Client Version 6.4 or later
Data ONTAP Version 7.4 or later
References
For more information about implementing the SnapMirror to Tape solution see Using the IBM
System Storage N series with IBM Tivoli Storage Manager, SG24-7243:
http://www.redbooks.ibm.com/abstracts/sg247243.html
As companies expand operations into new markets, the percentage of total corporate data in
remote offices is increasing. However, many companies are not adequately protecting these
assets. Obviously this is a very complicated problem and can be extremely expensive to solve
completely, considering all that can go wrong with remote office data:
Accidentally or maliciously deleted files: You need a way to easily and quickly restore
these assets at the file level from a local backup copy of the data.
Virus, worm, or hacker attack: You need to restore your systems and data to a point in time
before the attack.
Corrupted database: You need to ensure that your backup data is “application-consistent,”
meaning that all parts of recent, complex database transactions are included in the
backup.
Disk or server crash: You need to be able to quickly restore at the volume level, and to
restore operating systems and applications, even on different hardware.
Local or regional disaster: You need a recent copy of your data in another location that is
outside the potential disaster area, and ability to quickly restore the affected workloads.
Combine these challenges with the lack of skilled and dedicated IT staff in most remote
offices and you can see that much business risk is associated with data in these offices.
We put these challenges together in our solution matrix, as described in Chapter 4, “Tivoli
Storage Manager challenge matrix” on page 81.
IBM offers solutions for remote office data protection and recovery that are automated, easy
to use, and cost-effective. These solutions offer enterprise-class data protection at small
business prices, and integrate seamless with core data storage management solutions in the
data center, such as Tivoli Storage Manager.
Depending on what kind of data is stored in the remote office, the corresponding data
protections solution should be used. If you have file servers, big or small, and many or few
objects, then incremental backup or journal-based backup might be the best choice. See
7.7.1, “Progressive incremental backups” on page 260.
If you have applications, databases, and mail servers to protect, you can use the application
protection like Tivoli Storage Manager for ERP and others. If you have a virtual environment,
you can use the Tivoli Storage Manager for Virtual Environments to protect this kind of data.
For all kinds of data protection consider that the bandwidth to you data center might be limited
and you need any kind of data reduction for the data transfer.
To reduce the amount of data sent over the network you can use these solutions:
Client-side deduplication
Compression
Subfile backup (only Windows based client)
Hardware WAN accelerator like Cisco or Riverbed
Trying to recover data in remote offices from tape backups can also be problematic and often
requires the help of central IT staff or outside contractors. If you use incremental backups to
reduce the time needed to run the backup job, you must restore the last full backup and then
each sequential incremental tape, all of which can increase your recovery time while
increasing complexity and risk. In many cases, only the most essential data can be
recovered. Successful recovery from tape backups also assumes that all backup operations
were completed successfully, with no tape errors, labeling errors, or tape loss. Tape is a
point-in-time technology, so it can work well for recovering from a variety of data losses.
However, the time needed to perform a recovery makes tape a poor choice for remote offices.
And given the infrequency of backup jobs, tape probably does not address your more
business-critical recovery needs.
For this reason, we show a solution, how you can protect your remote office clients locally,
and have also a valid disaster protection of your whole remote office in your data center.
NODE4
DB
Datacenter
Servers
NODE1
CHICAGO_SRV
PHOENIX_SRV DALLAS_SRV
NODE4 data
NODE3 and NODE2 data
Remote Office
Servers
NODE3 data
DB DB
NODE5 data
ATLANTA_SRV
DB
NODE2
NODE3
NODE5
Install a Tivoli Storage Manager instance on your remote site. Use a disk-only solution and
keep the total amount of data in disk storage pools. For deduplication purposes, they must be
FILE disk storage pools. All your local clients are attached to this Tivoli Storage Manager
server. The disk hardware should be reliable storage and RAID-protected to prevent failures
and outages. The database and recovery logs should also reside on reliable midrange
storage systems like Storwize family. To protect the remote Tivoli Storage Manager server
against disaster outages, we do a copy of the database and storage pools with
server-to-server virtual volumes to the central Tivoli Storage Manager server in the data
center.
Now we have the helpful and improved function of node replication. This will help us to protect
the data in a way that the objects and the corresponding metadata will be stored on a target
server. There is no need to send the database backup as virtual volumes over WAN to an
on-premises data center or to IBM SoftLayer, however, for increasing the level of protection,
this is still possible. If you decide to do that, keep in mind that you should also prepare the
disaster recovery manager plan file and send it as a virtual volume to the data center. In the
case of a disaster outage, you can connect the remote clients directly to the Tivoli Storage
Manager server in the data center, without recovery of the server database and storage pools
in the remote location. The limitations and challenges in the case of recovery are the same as
described in “Recovery considerations” on page 295.
Sometimes, important clients need a very fast recovery during a disaster situation. In this
case, it is possible to generate a backup set from the data, replicate data to the target server
in the data center, and store it on a storage device, which will be accessed locally on the
client. The local backup set restore will be used to recover the data on the client in the remote
office. Figure 7-66 on page 297 shows how this is done.
DB
D atacen ter
S erv ers
NO DE 1
backup db, planfile C H IC A GO_S RV
use virtual volum es
P HO E N IX _S R V D A LLA S _S RV
N O D E 4 data
N O D E 3 and N O D E 2 data
R em ote O ffic e
S erv ers
N O D E 3 data
DB DB
N O D E 5 data
generate backup db, planfile
bac k ups et use virtual volum es
from
replic ated A T LA N T A _S R V
data
DB
NO DE 2
NO DE 3
NO DE 5
Figure 7-66 Restore local backup set from replicated node data
If you decide to send deduplicated data with node replication to your data center, see
Guidelines for node replication at the following web page:
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli%20Stora
ge%20Manager/page/Guidelines%20for%20node%20replication?lang=en
7.8.3 Remote office server connected to data center server with 3-site copy
For special requirements, you should have three copies of the backup data and they must be
stored at different places for disaster protection reasons. As described in the previous
section, we have only a two-site data protection, even if we copy the data with node
replication to the data center. How can we achieve this challenge? Maybe you consider a
solution where you replicate the already-replicated data to the next level, in this case to the
disaster protection site. However, this is not possible today. With node replication you can
copy the data to only one target.
How can we solve this problem? The Tivoli Storage Manager architecture can easily solve
this problem. We use the standard copy pool mechanism, which gives the choice to store the
copy storage pool on the primary site using online attached devices such as sequential disk,
virtual tape, or physical tape. But we can also use the disaster recover functionality to send
the copy pool data offsite.
D A L L A S _ S RV
P HO E N IX _ S R V N O D E 4 data
N O D E 3 and N O D E 2 data
R em o te O ffice
S erv ers
N O D E 3 data
DB DB
N O D E 5 data
backup db, planfile
use virtual volum es
DB
NO DE 2
D is as ter R ec ov ery M anager
offs ite D a ta ons ite
D e live r y
courier
notm ountable
NO DE 3
vault
m ountable
D a ta
vaultretrieve
D e live r y
S ite 3
onsiteretrieve
courierretrieve
Figure 7-67 3-site data protection with node replication and disaster recovery manager
If a server outage occurs, you have different places, where the data is stored for recovery. If it
is only a local incident, you can restore from the Tivoli Storage Manager server on the remote
site. If the server on the remote site is unavailable, you can recover from the data center site,
by using either the replicated data or the local backup set. If the data center is also affected,
you must first recover the data center from the disaster recovery site. In this case, it is a
multi-step procedure, which is well-planned, during the setup of the disaster protection
procedures.
This section shows solutions to successfully protect the data and its value.
In our solutions matrix, we put those challenges together. See Figure 7-69 on page 300 to
see, the combination.
Client benefit
The solution offers these benefits:
Secure data, secure data transfer, secure data access, secure data store
Reduced backup time
Fast recovery
Remote office and mobile data management
Reduced RPO, increased backup frequency
Business continuity
Solution architecture
This solution describes how you can build a secure environment to protect your valuable
business data, even when you are in a remote office or traveling to client but have low network
bandwidth. We show available options to use the traditional way with a daily based backup or
the more efficient version to protect your data continuously.
Figure 7-69 on page 300 shows the challenge and how to solve it. In this environment, you
have remote locations and mobile devices, which must be protected by the centrally located
Tivoli Storage Manager infrastructure.
In addition to data backup, archive copies of data can also be created using Tivoli Storage
Manager. Archive creates an additional copy of data and stores it for a specific amount of
time, known as the retention period. Tivoli Storage Manager archives are not expired until the
retention period is past, even if the original files are deleted from the client system.
Therefore, the difference between backup and archive is that backup creates and controls
multiple backup versions; archive creates an additional file that is retained for a specific
period of time.
Transport encryption
While sending data to the Tivoli Storage Manager Server, transport encryption is a key point
and is necessary.
Before a communication session between the Tivoli Storage Manager client and the Tivoli
Storage Manager server begins, an authentication handshaking process occurs with
authentication tickets and mutual algorithms. The Tivoli Storage Manager security protocol is
modeled after the Kerberos network authentication protocol, which is a highly respected
method for secure sign-on cryptography. The client uses its password as part of an encryption
key, and does not send the password over the network. Each session key is unique, so
replaying a session stream will not result in a sign-on to the Tivoli Storage Manager server.
This significantly lowers the chance of a Tivoli Storage Manager session being hijacked by an
outside user.
To heighten security for Tivoli Storage Manager sessions, data sent to the Tivoli Storage
Manager server during backup and archive operations can be encrypted with standard DES
56-bit encryption. The solution is backup-archive client, built-in and integrated with Tivoli
Storage Manager server. Alternatively, you can set the ENCRYPTIONTYPE option to
AES128 or DES56 in the client options file (dsm.opt).
Compression
Tivoli Storage Manager client compression option helps to instruct the Tivoli Storage Manager
client to compress files before sending them to the Tivoli Storage Manager server. This
industry-standard compression gives us effective bandwidth use.
Compressing files reduces the file data storage space and can improve throughput over slow
networks with a powerful client. The throughput, however, can be degraded when running on
a slow client system using a fast network, because software compression uses significant
client CPU resources and costs additional elapsed time.
If the compression option is set to YES, the compression processing can be controlled in the
following ways:
Use the include.compression option to include files within a broad group of excluded files
for compression processing.
Use the exclude.compression option to exclude specific files or groups of files from
compression processing, especially for the objects that are already compressed or
encrypted, such as GIF, JPG, ZIP, MP3, and others.
If the Tivoli Storage Manager client compression and encryption is used for the same file
during backup, the file is first compressed and then encrypted, which results in a smaller file.
On restore, the file is decrypted first and then decompressed.
The IBM Tivoli Storage Manager Backup-Archive Client, included with the server, provides
the operational backup and archive functions. The client implements the patented progressive
incremental backup methodology, deduplication, adaptive subfile backup technology, and
unique record retention methods for backup and archive functions.
The backup-archive clients are implemented as multi-session clients, which means that they
are able to take advantage of the multi-threading capabilities of modern operating systems.
Deduplication
Deduplication is also another bandwidth and data reduction technique. Tivoli Storage
Manager helps to decrease the rate of amount of storage capacity required to contain data
growth with a built-in data deduplication feature that helps eliminate redundant data.
Deduplication eliminates redundant data chunks when client-side deduplication is used. This
can enable significantly less data to be moved over the network, and significantly more
backup data to be stored on disk.
Figure 7-70 shows our solution approach to get benefits from Tivoli Storage Manager client
deduplication.
'
"
($
)#
#"
&'
&(
&(
37
&! 3$
7$
&!
"
#$
370 *#0"
&'
&(
&'
&!
!"##
$#
%
You can get more benefit from deduplication if you combine it with Tivoli Storage Manager
Node replication for Disaster Recovery, which is discussed as another solution in this book.
Subfile backup
Mobile and remote computer branches have limited access to the infrastructure that serves
the rest of the organization. Some limitations include being attached to the corporate network
with reduced bandwidth, limited connect time, and minimal assistance to perform the backup.
This limited access both increases the criticality of storage management services and limits
the applicability of traditional methods and policies. Tivoli Storage Manager helps resolve
these problems with its adaptive subfile backup feature which reduces the amount of data
transferred while backing up changed files.
The changed file portion is backed up as a differential backup relative to the last complete
backup of the file (base or reference file). It is called a delta file. All changes since the last
complete backup of the file are included in this delta file. In the case of a restore operation,
this feature allows for restoring the whole file by restoring only two subfile components, one
delta file, and the last complete backup of the whole file.
For further information, and best practices for recovering Windows Server 2008, Windows
Server 2008 R2, Windows 7, and Windows Vista, see the following document:
http://ibm.co/1ptFRi7
Another solution is the product from Cristie Software Limited, which also addresses this
challenge.
TBMR is a software package from Cristie Software that offers these benefits:
Leverages a customer’s investment in Tivoli Storage Manager to provide a fully automated
method of recovering or cloning a system that is running Microsoft Windows operating
system, Linux, Oracle Solaris, and AIX to the same or new hardware.
Complements your Tivoli Storage Manager backup and recovery strategy by offering a
solution to recover, migrate, or clone your operating systems without the need to perform a
separate backup. TBMR helps to maximize your recovery flexibility by supporting recovery
to dissimilar hardware or virtual machines, which in turn minimizes business disruption.
Tivoli Storage Manager is an IBM enterprise class backup and archiving software. TBMR
works exclusively with Tivoli Storage Manager and is by IBM and its channel partners
worldwide as the preferred BMR solution for Tivoli Storage Manager.
The main features of the BMR solution for Tivoli are as follows:
Fully integrated with Tivoli Storage Manager
Rebuild a server OS in under ten minutes
Uses the Tivoli Storage Manager backups for disaster recovery
Multiple servers can be recovered simultaneously
TBMR is the ONLY software that can recover direct from the Tivoli Storage Manager
server without doing a separate backup
Continuously protected files can be backed up to a local drive. This means that backup copies
are created even when network conditions prevent storing backup copies on remote storage
locations. Continuously protected files can also be stored on remote storage locations, when
network connections allow. If a remote location is not available when you change a
continuously protected file, the FastBack for Workstations client makes a backup copy on that
device as soon as the device becomes available. Scheduled backup copies are created on
the interval that you configure (hourly, weekly, daily, or monthly). If the remote device for
scheduled backups is not available at the time of the backup, the FastBack for Workstations
client makes backup copies on the remote location as soon as that device becomes available.
Figure 7-72 on page 306 shows a backup paradigm using a hybrid approach.
Figure 7-72 Flashback for Workstations backup paradigm using a hybrid approach
IBM Tivoli FastBack for Workstations is designed to comprehensively protect end users
workstations. It hooks into the file system, so that new or changed files, applications, or
directories are detected and are then copied to a local storage as shown in Figure 7-73. Files
can also be replicated continuously or on a scheduled basis to a secondary media type or
target. This can be a file-share, a target from a WebDev server, a Tivoli Storage Manager
server, or a removable disk device (external hard drive, USB flash drive, and others). A user
can maintain the configurations of the software or this can be controlled from a central
management console. Although control of the configuration can be delegated to an
administrator, users can still recover their own data. This can reduce the number of
restoration requests for help desks and allow IT administrators to focus on other tasks.
WA N
n – LAN/
catio
Repli
File Server
Replic
ation –
LAN/W
AN
Replication
Re
plic
ati
on
– htt
p/h
ttp
s TSM Server
Web target
Bandwidth throttling
By specifying throttle settings and network rules for Tivoli Storage Manager FastBack for
Workstations, you can manage the bandwidth usage in the networks that you specified.
Use the Network Rules settings to manage bandwidth usage for each network. When a
network is accessed, Tivoli Storage Manager FastBack for Workstations uses the first rule in
the list that matches the network. As a result, the throttle setting does not require a manual
update every time Tivoli Storage Manager FastBack for Workstations accesses a different
network. When a new network is detected, a default network rule is created. This default rule
is added to the end of the network rule list.
This feature helps you control the data transfer to your remote system and not to overload the
connection, which might be a tight WAN connection.
For information about configuring bandwidth throttling, see IBM Tivoli Storage Manager
FastBack for Workstations Version 6.3 Client Installation and User's Guide, SC27-2809.
Central management
Key to providing a unified data protection infrastructure is the ability to centrally control and
manage data protection operations. Tivoli Storage Manager FastBack for Workstations V6.3
provides a centralized management interface that can help administrators manage thousands
of laptops and desktops. This central management interface runs in the same integrated
solutions console in which the Tivoli Storage Manager Administration Center runs, enabling
the management of data protection policies across the data center, remote office, and mobile
Figure 7-74 Central administration in one instance of the Tivoli Integrated Solutions Console
Continuos data Real-time data protection Simplified storage management can save IT
protection and user labor.
Restore to point-in-time Multiple versions of the Continuous data protection provides data
files retained integrity when viruses and corruption attack
systems.
Backs up only changed Less data is transferred Optimizes bandwidth and network transfer
files of data.
Backs up your files the Real-time data protection Continuously protects versions of the files to
moment they change allow customers choice of recovery points.
Backs up files to local Stand-alone protection Ability to write-protect data locally even
cache when not connected in case of virus,
corruptions, logical error or user error.
Remote and local disk Choice of backup Ability to send data to mix of backup
support devices devices: disk, NAS, USB, local partition,
LUN from SAN.
Additional information
For hints and tips, see Deploying FastBack for Workstations In an Enterprise Environment:
http://www-01.ibm.com/support/docview.wss?uid=swg21638113
How can the Tivoli Storage Manager product help you with your data protection challenges?
To help you to meet your requirements for business-valuable processes and applications that
are related to recovery time objective (RTO) and recovery point objective (RPO), Tivoli
Storage Manager assists with intelligent storage architecture methods. Tivoli Storage
Manager can act as an interface between intraday and interday protection, as shown in
Figure A-1.
Intradayprotection
CDR
Continuous
CDP
DataReplication
Seconds
Continuous
Application DataProtection
Transparent
ClusterMirroring
Application DynamicThinProvisioning
AwareGroup DynamicTiering
Snapshot
StorageMigrationConsolidation
Hours
StorageVirtualization
TSM
VTL
ProtecTIER
Diskpool
Tape
Backup&
Days
Archive
Interdayprotection
low costs high costs
The Tivoli Storage Management layer interfaces between the application layer and the
storage layer for the application and data-specific backup and recovery. You can take
advantage of this and configure a disk and tape based solution dependent on the particular
application.
Gather the information in a spreadsheet using information from the activity log, the summary
table and query commands (see Figure A-4 on page 317).
The output is presented in a comma separated format, where it can be used for further
calculations. Example A-2 shows output.
You can also create a batch query from an existing Tivoli Storage Manager instance, as
shown in Example A-3.
The data can now be imported in a spreadsheet program for analysis as shown in Figure A-2.
Figure A-2 sample spreadsheet for analysis of the output from query node
Valuable information is also in the occupancy table of the Tivoli Storage Manager Server
database. To query this information, use the administrative query command (Example A-4).
Example A-4 Sample query command to get information about the occupancy for backup and archive
query auditocc
You also can collect data from the accounting log or from the schedule log of the clients.
To evaluate the value of the information about client backups, check the schdule log.
Example A-5 shows a dsmsched.log file.
For further data analytics, review the data that is collected in the accounting log. A sample of
the comma separated content of the accounting log is shown in Example A-6.
In our case, we also collected information about the application and databases, such as size
of the database from the application system, and stored it in a separate database such as
mysql. We can use this daily collected data for our analysis. If Tivoli Storage Manager is not
running, you must find a way to collect this data manually.
The spreadsheet can look like the table in Figure A-4. You can expand this information list
based on your requirements. For example, you might decide to add some numbers for typical
restore requirements.
Indicators are helpful numbers or characters that can help you find categories and structures
of the data, and can help you sort and organize the data. The indicators are described in a
legend. Figure A-5 shows sample indicators in a legend.
Figure A-5 Legend of sample indicator fields and assigned Indicator Values
The value of the indicators to the data can be done manually or by intelligent formulas. In our
example, the value of the indicator is done manually, because during the process we were
able to decide the value to assign.
In our example, we use database size as an indicator field and assign these values:
0 for file systems without databases
1 for small databases up to 10 GB
2 for databases between 10 and 20 GB
3 for databases up to 50 GB
4 for databases up to 500 GB
5 for large databases up to 3 TB
6 for huge databases above the limit
For network categories, we have another indicator field with these assigned values:
1 for 100 Mb connection
2 for 1 Gb
3 for 10 Gb
With the summary of all indicators, you can identify your service requirements. With these
results you can build your storage categories. In our case (Figure A-6 on page 320), these are
the values:
Values 0, 1, and 2 represent a category that uses a traditional storage hierarchy.
Value 3 sends the data to disk and migrates to virtual tape library (VTL).
Value 4 are the nodes to do the backup through LAN direct to the VTL.
Value 5 will do it LAN-free.
Value 6 is for huge database systems that will use LAN-free data transfer direct to physical
tape.
Another way might be to implement a disk-to-disk backup strategy data replication to another
server if the data protection requirements will be identified to a high level of security for
special systems that are represented by Tivoli Storage Manager nodes.
The indicators may represent your service requirements. Identify the factors that are
important for your backup and restore strategy:
If you want to meet fast restore requirements for a huge amount of data in few objects,
LAN-free data transfer to tape might be your solution.
If you have millions of small files to protect, incremental backup to random disk pools is a
possible implementation.
If you want to take advantage of data deduplication, put the data on sequential file storage
pools with data deduplication and leave it there for a disk only solution.
If you want to have extra protection for your deduplicated data, take advantage of node
replication.
If the amount is too big, migrate the data from the random disk storage pool to physical
tape.
After you categorize the tiers of protection you want to implement, you can use Tivoli Storage
Manager to implement the data protection solution to fit your needs. And of course you can
configure any combination of them, even in a single server, taking advantage of the powerful
Tivoli Storage Manager policy management. Depending on the values of the spreadsheet,
and mostly of the indicators, now you can categorize your nodes and assign them to Tivoli
Storage Manager objects such as domains. You can use the Indicator fields to summarize as
in these examples:
Indicator value 1 in IndicatorFiled 1 (database size) could represent a category that
uses a traditional storage hierarchy
Indicator value 2 in Indicator Filed1 (database size) if you want to implement a
disk-to-disk backup strategy with data replication to another server.
This is why we refer to this as Tivoli Storage Manager policy-based dynamic tiering.
Figure A-6 shows our project’s data flow to solve the tiered storage approach.
The message here is that you can be creative and innovative when you use the Tivoli Storage
Manager toolbox for your data protection solution.
Business processes change frequently. As a result, you want to adjust the service level
requirements for RTO and RPO dynamically. To assist with this, you can take advantage of
the centralized functions for the policy and tiered storage management that is available with
the Tivoli Storage Manager product.
You can use the existing mechanisms for data migration, data copy, data deduplication, and
data replication. The copy and replication functions are used to create redundant data for
disaster protection and offsite vaulting. Migration is used to change the storage tier to another
level in the storage hierarchy, for example to move the data to new storage technology or, in
the case of lifecycle management, for long-term retention to high capacity, low cost, and lower
RTO physical tape. Successful dynamic tiering requires that the defined storage classes for
diskpool, filepool, virtual tape library, and physical tape are transparently integrated and
controlled in the Tivoli Storage Manager server. Avoid special functions of the storage
hardware, for example, VTL tape caching or post deduplication. Be sure that the server
controls the data management, and that the service level requirements are guaranteed at
recovery time. Special functions of the storage hardware might influence the effectiveness of
the Tivoli Storage Manager system or the operating of it.
The storage tiers can handle the calculated amount of data that must be moved in and out
and must be stored in the data pools. Each tier is a collection of storage pools and storage
pool volumes that represent an amount of available space. They are stretched over all layers
in the hierarchy, including disk, VTL, and physical tape and can be adjusted to your needs.
You can scale to the calculated data amount from the top to the bottom of the storage
hierarchy without a change in the basic technical concept.
This provides you with the necessary investment protection of the overall solution for the life
of the data.
Parallelism and distribution are the keys to independent modular storage tiers, giving you
enhanced redundancy and linear scalability.
In effect, HSM turns the fast disk drives into caches for the slower mass storage devices. The
HSM for Windows client monitors the way files are used and lets you automate policies as to
which files can safely be moved (migrated) to slower devices and which files should stay on
the hard disks.
The HSM for Windows client manages the migration of individual files, files from parts of file
systems, or complete file systems, to remote storage in Tivoli Storage Manager. Migrated files
can be accessed, opened, and updated by the Windows application corresponding to the file
extension.
The HSM for Windows client includes a graphical user interface (HSM for Windows client
GUI) that you use to define and run migration jobs, threshold migration, reconciliation,
searches and file retrieval, and to define general settings. You can also do many of these
tasks using HSM for Windows client commands from a Command Prompt window.
Transparent
Again, from a user or running process perspective, all files in the local file system are actually
available. Directory listings and other commands that do not require access to the entire file
will appear exactly as they would without the HSM client. When a migrated file is needed by
an application or command, the operating system initiates a transparent recall for the file to
the Tivoli Storage Manager server. This helps to more easily locate and access offloaded
data without administrator interaction. The transparency is not affected by the capability of
keeping separate versions of migrated files because a user-initiated recall always gets the
latest version. Retrieval is transparently invoked by actions resulting in an open call to the file:
1. Double-click the file from an Explorer window.
2. Select File Open on the files icon from the appropriate program, like Windows Explorer
or any other file navigator.
If a file is recalled from the server, in the next scheduled archiving run, the retrieved document
is replaced by a shortcut (“re-stubbed”). No additional version is stored on the server. An MD5
key is computed for the retrieved document. This MD5 key is compared with the MD5 key
stored in the migrated document. If the two MD5 keys match, the file is only replaced with a
stub, otherwise a new version of the file is stored in the repository. In this way, the file is
migrated again only when necessary (for example, if it changed on the client).
Files migrated to the Tivoli Storage Manager server using the HSM Client for Windows are
retained on the server for the length of time defined in the Retain Version field of the archive
copy group. You should set this field according to your needs and the space available. This
field can be set to NOLIMIT, which means the migrated files are kept on the server indefinitely,
regardless of whether the original is deleted from the client. If you set this field to a lesser
value, be careful of the possibility that the stub file still exists on the client, when the migrated
file on the server has expired. Upon backup of a stub file, the stub file becomes the active
copy of the data, marking the original copy inactive. Depending on your policy settings, the
original file can be processed by expiration, making it impossible to restore from a backup.
Selective
Selective recalls of migrated files are performed by the HSM Client GUI or command line.
These interfaces provide powerful controls of what to retrieve and where. They provide the
ability to recall different versions of the files that were sent to the Tivoli Storage Manager
archive pool, or files where the stub file has been deleted from the client system. A selective
recall can be directed to the same or different directory from the one where the file was
originally migrated. Selective recalls can be submitted only by the Tivoli Storage Manager
HSM for Windows administrator.
When a file is migrated from your local system to Tivoli Storage Manager storage, a
placeholder, or stub file, is created in place of the original file. Stub files contain the necessary
The HSM client provides space management services for locally mounted file systems, and it
migrates regular files only. It does not migrate character-special files, block-special files,
named pipe files, or directories.
File migration, unlike file backup, does not protect against accidental file deletion, file
corruption, or disk failure. Continue to back up your files regardless of whether they reside on
your local file system or are migrated to Tivoli Storage Manager storage. The IBM Tivoli
Storage Manager backup-archive client is used to back up and restore migrated files in the
same manner as you would back up and restore files that reside on your local file system. If
you accidentally delete stub files from your local file system, or if you lose your local file
system, you can restore the stub files or the complete files.
For planned processes, such as storing a large group of files in storage and returning them to
your local file system for processing, use the archive and retrieve processes. The
backup-archive client is used to archive and retrieve copies of migrated files in the same
manner as you would archive and retrieve copies of files that reside on your local file system.
The HSM client functions for threshold migration, demand migration, selective migration, and
selective and transparent recall include processing GPFS file systems containing multiple
space-managed storage pools.
The HSM client has both a graphical user interface (the HSM GUI) and commands you can
run from a shell. You can also use the commands in scripts and cron jobs.
Because the Data Protection Client and the DSM agent are in a proxy relationship, they are
able to communicate using client-to-client communication. As a result, during the backup
operation, the Data Protection Client communicates with the DSM agent, which in turn
communicates with the Volume Shadow Copy Service. The Volume Shadow Copy Service
then communicates with the Application Server (such as Microsoft Exchange Server or
Microsoft SQL Server) via the application VSS Writer to start the snapshot.
The Data Protection client supports the following types of VSS backups:
Local
This backup type creates a persistent shadow copy that resides on a snapshot volume.
Although this type of backup is managed by Tivoli Storage Manager policy, the actual data
resides on volumes local to the VSS Provider and not on Tivoli Storage Manager
controlled storage. The VSS Provider software controls how that shadow copy is created
and maintained, and the Tivoli Storage Manager policy manages its lifecycle.
Tivoli Storage Manager
This backup type creates a copy of application data on Tivoli Storage Manager server
storage. This data from an application server VSS snapshot is copied to the Tivoli Storage
Manager server as backup data. This data is also managed by Tivoli Storage Manager
policy. When this backup is complete, the snapshot volumes are released.
Both
This backup type creates two copies of application server data. One copy resides on local
volumes as a snapshot and another copy (taken from the snapshot copy) is sent to Tivoli
Storage Manager server storage.
Off-loaded backup
This backup type refers to a Tivoli Storage Manager backup that is done from a system
other than the application server. To accomplish this, a shadow copy volume is imported to
This backup type means the VSS snapshot is both transportable and non-persistent.
Appendix C. VSS and Tivoli Storage Manager related product concepts 329
330 Tivoli Storage Manager as a Data Protection Solution
D
Select the Additional materials and open the directory that corresponds with the IBM
Redbooks form number, SG248134.
The publications listed in this section are considered particularly suitable for a more detailed
discussion of the topics covered in this book.
IBM Redbooks
The following IBM Redbooks publications provide additional information about the topic in this
document. Note that some publications referenced in this list might be available in softcopy
only.
IBM ProtecTIER Implementation and Best Practices Guide, SG24-8025
IBM Tivoli Storage Manager: Building a Secure Environment, SG24-7505
Infrastructure Solutions: Design, Manage, and Optimize a 20 TB SAP NetWeaver
Business Intelligence Data Warehouse, SG24-7289
SONAS Concepts, Architecture, and Planning Guide, SG24-7963
You can search for, view, download or order these documents and other Redbooks,
Redpapers, Web Docs, draft and additional materials, at the following website:
ibm.com/redbooks
Other publications
These publications are also relevant as further information sources:
Tivoli Storage Manager for Enterprise Resource Planning - Data Protection for SAP,
Installation and User’s Guide for DB2, SC33-6341
Tivoli Storage Manager for Enterprise Resource Planning - Data Protection for SAP,
Installation and User’s Guide for Oracle, SC33-6340
IBM Tivoli Storage FlashCopy Manager V3.2: Installation and User's Guide on UNIX and
Linux, SC27-4005
IBM SONAS Introduction and Planning Guide, GA32-0716
Provides solutions for When you hear IBM Tivoli Storage Manager, the first thing that you
typically think of is data backup. Tivoli Storage Manager is the premier INTERNATIONAL
common data
storage management solution for mixed platform environments. TECHNICAL
protection challenges
Businesses face a tidal wave of information and data that seems to
SUPPORT
increase daily. The ability to successfully and efficiently manage ORGANIZATION
Includes challenges
information and data has become imperative. The Tivoli Storage
and solution matrices
Manager family of products helps businesses successfully gain better
control and efficiently manage the information tidal wave through
Effectively use the significant enhancements in multiple facets of data protection.
Tivoli Storage BUILDING TECHNICAL
Tivoli Storage Manager is a highly scalable and available data INFORMATION BASED ON
Manager Toolkit protection solution. It takes data protection scalability to the next level PRACTICAL EXPERIENCE
with a relational database, which is based on IBM DB2 technology.
Greater availability is delivered through enhancements such as online, IBM Redbooks are developed
automated database reorganization. by the IBM International
This IBM Redbooks publication describes the evolving set of Technical Support
data-protection challenges and how capabilities in Tivoli Storage Organization. Experts from
Manager can best be used to address those challenges. This book is
IBM, Customers and Partners
from around the world create
more than merely a description of new and changed functions in Tivoli timely technical information
Storage Manager; it is a guide to use for your overall data protection based on realistic scenarios.
solution. Specific recommendations
are provided to help you
implement IT solutions more
effectively in your
environment.