Front cover

Grid Computing
Products and Services
Grid computing functionality

Products for grid computing

Services and solutions

Luis Ferreira
Mariano Batista
Sebastien Fibra
Chin Yau Lee
Carlos Alexandre Queiroz Silva
Joao Almeida
Fabiano Lucchese
Nam Keung

ibm.com/redbooks
International Technical Support Organization

Grid Computing Products and Services

August 2005

SG24-6650-00
 Note: Before using this information and the product it supports, read the information in,
“Notices” on page xiii.

First Edition (August 2005)

This edition applies to the capability of the IBM, ISVs, and open source products used to build a
grid computing solution.

© Copyright International Business Machines Corporation 2005. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP
Schedule Contract with IBM Corp.
Contents

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

Part 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.1 Grid computing as an evolution of distributed computing . . . . . . . . . . 4
1.1.2 Grid computing models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.1.3 Grid and its relationship with other IT components . . . . . . . . . . . . . . 11
1.2 Categories and functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.1 Categorization of components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2.2 Functionality on grid computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.3 Grid computing products and players . . . . . . . . . . . . . . . . . . . . . . . . 20

Part 2. Functionality and products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Chapter 2. Web services core/hosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2 Grid standards and toolkits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.1 Grid standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.2 OGSA Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3 Grid application environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.2 WebSphere Extended Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3.3 WebSphere Portal Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3.4 WebSphere MQ Family. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.5 WebSphere Messaging Resources. . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.6 WebSphere Business Integration Messaging Broker . . . . . . . . . . . . 38
2.4 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

© Copyright IBM Corp. 2005. All rights reserved. iii

 Chapter 3. Workload virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1 Scheduling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2 Grid scheduling and policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3 How to choose the appropriate scheduler. . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4 Mapping user requirements into products . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4.1 Tivoli Workload Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4.2 LoadLeveler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.3 DataSynapse GridServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.4 Platform LSF (Load Sharing Facility) . . . . . . . . . . . . . . . . . . . . . . . . 50
3.4.5 Platform LSF MultiCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4.6 Platform LSF HPC (High-Performance Computing) . . . . . . . . . . . . . 51
3.4.7 Platform Symphony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4.8 United Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4.9 Altair PBS Professional . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.4.10 OpenPBS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4.11 Condor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4.12 Community Scheduler Framework (CSF) . . . . . . . . . . . . . . . . . . . . 57
3.4.13 Maui Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.5 Architecture examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.5.1 Desktop grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.5.2 Server grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Chapter 4. Information virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2 Information virtualization layers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2.1 Block data virtualization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2.2 File system virtualization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2.3 Information virtualization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.3 Requirements and products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3.1 Defining requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3.2 Block data layer products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3.3 Files system layer products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3.4 Information layer products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.4 Case study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.4.1 Problem overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.4.2 Grid solution planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.4.3 Federation of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Chapter 5. Other disciplines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2 Provisioning and orchestration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.1 Provisioning and orchestration products . . . . . . . . . . . . . . . . . . . . . . 88
5.2.2 Provisioning and orchestration case study . . . . . . . . . . . . . . . . . . . . 89

iv Grid Computing Products and Services

 5.2.3 Correlated products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.3 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.3.1 Firewalls, VPNs, and virtual organizations . . . . . . . . . . . . . . . . . . . . 93
5.3.2 Security products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.3.3 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.4 Directory Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.4.1 Directory Services products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.4.2 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.5 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.5.1 Licensing products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.6 Billing and metering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Part 3. Bundles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Chapter 6. Virtualization Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.2 Virtualization Engine suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.2.1 IBM Virtualization Engine Suite for Servers . . . . . . . . . . . . . . . . . . 108
6.2.2 IBM Virtualization Engine Suite for Storage . . . . . . . . . . . . . . . . . . 110
6.3 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Chapter 7. ISV suites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.2 Platform Computing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.3 DataSynapse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.4 United Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Part 4. Services and solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Chapter 8. Services and solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8.2 Services portfolio for grid computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8.2.1 Grid Innovation Workshop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
8.2.2 Grid Strategy Engagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
8.2.3 Grid Design Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
8.2.4 Grid Software Implementation Services . . . . . . . . . . . . . . . . . . . . . 127
8.2.5 Grid Training and Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.2.6 IBM support line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.3 Solutions portfolio for grid computing . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
8.3.1 IBM Grid Value at Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8.3.2 Analytics Acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.3.3 Design Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Contents v
 8.3.4 Engineering Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
8.3.5 Geophysical Processing and Analysis . . . . . . . . . . . . . . . . . . . . . . 134
8.3.6 Information Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8.3.7 IT Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
8.3.8 Research Collaboration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
8.3.9 Grid Clash Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.3.10 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.4 Services approach to Grid Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Part 5. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Appendix A. Surrounding initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

World Community Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Globus Alliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Global Grid Forum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
OASIS and WSRF TC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
W3C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
TeraGrid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Appendix B. World Community Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Technical features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
High level design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Products used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
More information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
World Community Grid Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Example of the World Community Grid Screen Saver . . . . . . . . . . . . . . . 162
www.worldcommunitygrid.org . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Appendix C. Products reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Big picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
DB2 Content Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
LoadLeveler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Tivoli Access Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Tivoli Enterprise Console®. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Tivoli Intelligent Orchestrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Tivoli License Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Tivoli Management Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Tivoli Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Tivoli NetView. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Tivoli Provisioning Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

vi Grid Computing Products and Services

Tivoli SAN FS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Tivoli Service Level Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Tivoli Workload Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
TotalStorage SAN Volume Controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Virtualization Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
WebSphere Extended Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
WebSphere Information Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
WebSphere MQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
WebSphere Portal Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Contents vii
viii Grid Computing Products and Services
Figures

1-1 Grid stack products’ portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1-2 Product evaluation criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1-3 WebSphere Information Integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2-1 OGSA architecture model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2-2 Web Services based Grid Application Model . . . . . . . . . . . . . . . . . . . . . 27
2-3 Globus-based Grid Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3-1 Focus of scheduling environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3-2 LoadLeveler components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3-3 GridServer architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3-4 Simplistic view of Condor scheduling. . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3-5 Proposed Desktop Grid setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3-6 Current environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3-7 Proposed Architecture for “xyz” Corporation . . . . . . . . . . . . . . . . . . . . . 63
4-1 Virtualization layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4-2 Block data virtualization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4-3 File layer virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4-4 Information virtualization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4-5 Example for a product overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4-6 Block data layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4-7 File system layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4-8 Information layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4-9 “xyz” Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4-10 Federated using WebSphere Information Integration . . . . . . . . . . . . . . 84
5-1 A typical Internet banking infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . 89
5-2 A bank VPN infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6-1 Groups of virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
8-1 Grid computing services roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
8-2 Analytics Acceleration architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8-3 Engineering Design architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
8-4 Geophysical Processing and Analysis architecture . . . . . . . . . . . . . . . 136
8-5 Research Collaboration architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 138
B-1 World Community Grid - Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
B-2 World Community Grid: High Level Solution Design . . . . . . . . . . . . . . 155
B-3 World Community Grid: Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 158
B-4 World Community Grid: Architecture with products . . . . . . . . . . . . . . . 161
B-5 World Community Grid Agent Interface . . . . . . . . . . . . . . . . . . . . . . . . 162
B-6 World Community Grid Screen Saver . . . . . . . . . . . . . . . . . . . . . . . . . 163
B-7 World Community Grid Web Site - Homepage . . . . . . . . . . . . . . . . . . 164

© Copyright IBM Corp. 2005. All rights reserved. ix

 B-8 World Community Grid Web Site - Project Statistics . . . . . . . . . . . . . . 165
B-9 World Community Grid: Member Area . . . . . . . . . . . . . . . . . . . . . . . . . 166
C-1 DB2 Content Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
C-2 GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
C-3 LoadLeveler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
C-4 Tivoli Access Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
C-5 Tivoli Enterprise Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
C-6 Tivoli Intelligent Orchestrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
C-7 Tivoli License Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
C-8 Tivoli Management Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
C-9 Tivoli Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
C-10 Tivoli NetView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
C-11 Tivoli Provisioning Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
C-12 Tivoli SAN FS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
C-13 Tivoli Service Level Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
C-14 Tivoli Workload Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
C-15 TotalStorage San Volume Controller . . . . . . . . . . . . . . . . . . . . . . . . . . 194
C-16 Virtualization Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
C-17 WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
C-18 WebSphere Extended Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
C-19 WebSphere Information Integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
C-20 WebSphere MQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
C-21 WebSphere Portal Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

x Grid Computing Products and Services

Tables

2-1 Comparing WBI-MQ and WAS-MR . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

8-1 Grid computing services overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
8-2 Grid solutions overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
8-3 Offering availability by industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
C-1 Web Services Core/Hosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
C-2 Workload virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
C-3 Information virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
C-4 Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C-5 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C-6 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C-7 License management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C-8 DB2 Content Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
C-9 GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
C-10 LoadLeveler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
C-11 Tivoli Access Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
C-12 Tivoli Enterprise Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
C-13 Tivoli Intelligent Orchestrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
C-14 Tivoli License Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
C-15 Tivoli Management Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
C-16 Tivoli Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
C-17 Tivoli NetView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
C-18 Tivoli Provisioning Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
C-19 Tivoli SAN FS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
C-20 Tivoli Service Level Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
C-21 Tivoli Workload Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
C-22 TotalStorage San Volume Controller . . . . . . . . . . . . . . . . . . . . . . . . . . 194
C-23 Virtualization Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
C-24 WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
C-25 WebSphere Extended Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
C-26 WebSphere Information Integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
C-27 WebSphere MQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
C-28 WebSphere Portal Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

© Copyright IBM Corp. 2005. All rights reserved. xi

xii Grid Computing Products and Services
Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any IBM intellectual property right may be used instead. However, it is the user's
responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not give you any license to these patents. You can send license
inquiries, in writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES
THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at
any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on
the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the
sample programs are written. These examples have not been thoroughly tested under all conditions. IBM,
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to IBM for the purposes of
developing, using, marketing, or distributing application programs conforming to IBM's application
programming interfaces.

© Copyright IBM Corp. 2005. All rights reserved. xiii

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AFS® iSeries™ RS/6000®
AIX 5L™ LoadLeveler® Storage Tank™
AIX® Lotus Notes® Tivoli Enterprise™
CICS® Lotus® Tivoli Enterprise Console®
Domino® MQSeries® Tivoli®
DB2 Universal Database™ NetView® TotalStorage®
DB2® Notes® TCS®
Eserver® OmniFind™ Virtualization Engine™
Eserver® OS/2® WebSphere®
FlashCopy® pSeries® World Community Grid™
HACMP™ QuickPlace® xSeries®
IBM® Redbooks™ z/OS®
i5/OS™ Redbooks (logo) ™ zSeries®

The following terms are trademarks of other companies:

EJB, Java, JavaBeans, Jini, JDBC, JSP, J2EE, Solaris, Sun, Sun Microsystems, and all Java-based
trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Nina, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United
States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, and service names may be trademarks or service marks of others.

xiv Grid Computing Products and Services

Preface

This IBM® Redbook is part of a series of documents related to grid computing

that IBM is presenting to the IT community to it and all its players: clients,
industry leaders, emerging enterprises, universities, and producers of
technology. It is mainly oriented to IT architects or those who have the
responsibility of analyzing the capabilities of the products to build a grid solution.

We describe the functions of grid computing and the categorizations of the

components within it. IBM, ISVs, and open source products will be explained.
Some of these products are not specific for grid, but they can be used in a grid
computing environment. We hope that this redbook helps you select functions
and products and shows you how grid can fit into your IT picture as new products
and services for grid are introduced.

The redbook is organized into the following parts:

Part 1, “Introduction” on page 1

Introduces the concept of grid computing and provides the terminology the
readers will use in this redbook.

Part 2, “Functionality and products” on page 21

Explores the functionality associated with certain products and product families.
It also presents some examples of the utilization of these products in the grid
computing environment.

Part 3, “Bundles” on page 103

Presents grid computing product families that can be used to build a complete
grid solution. These bundles, or suites, often implement most of the core grid
disciplines.

Part 4, “Services and solutions” on page 119

Gives an overview of the grid computing offerings and services that IBM
provides.

Part 5, “Appendixes” on page 143

Explorers the following additional grid computing related topics:
򐂰 Appendix A, “Surrounding initiatives” on page 145
򐂰 Appendix B, “World Community Grid” on page 149
򐂰 Appendix C, “Products reference” on page 169

© Copyright IBM Corp. 2005. All rights reserved. xv

The team that wrote this redbook
This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization, Austin Center.

Luis Ferreira, also known as “Luix”, is a Senior Software Engineer at IBM

Americas, working on grid computing architecture projects. He has 20 years of
experience with UNIX®-like operating systems in design, architecture and
implementation, and holds a Master of Science degree in Systems Engineering
from Universidade Federal do Rio de Janeiro in Brazil. Before joining IBM
Americas, Luis worked at International Technical Support Organization as a
Linux® and Grid Computing Project Leader, Tivoli® Systems as a Certified Tivoli
Consultant, at IBM Brazil as a Certified IT Specialist, and at Cobra
Computadores as a kernel developer and operating systems designer.

Mariano Batista is a Certified IT Architect. He has been working for IBM for
more than 11 years. He is usually called to do pre-sales activities in multi-brand
projects that have strong integration and complex technologies. Mariano got his
certification as ITA in Middleware and Application Services in December 2002
and was invited to be part of the ITA Certification Board at the same time. In the
last six years, he has worked for Sales and Distribution as a Solution IT Architect
doing pre-sales activities for multi-industry clients, mainly in Argentina, with
coverage on SSA (Spanish South America) and Latin America. He has been part
of the on demand pre-sales IT Architects team since 2003 and the e-business
Solutions Technical Sales unit since 2001 in the Americas. His current
responsibilities include a broad range of IT solutions and technologies; grid
computing is one of them. Mariano represents IBM in internal and external
conferences, events, and articles published in newspapers and magazines in
Argentina related to grid computing. Before becoming a formal IT architect,
Mariano was part of the Industry Systems Business Unit for Financial Services
Sector in Argentina. He was responsible for important engagements in
e-business solutions for Financial Services Sector industries. He was part of the
team that implemented the first Interactive Financial Services project in Latin
America, and he was the IT Architect and technical leader for the second phase
of that project. He started at IBM in 1993 as a C/C++ developer in the Advanced
Solutions Unit of IBM Argentina. He also provided AIX® and system
management support for that unit. Mariano received his Bachelor of Computer
Sciences degree with honors at the Universidad del Salvador in Argentina in
1994.

Sebastien Fibra is a grid specialist at the IBM Advanced Technical Support

(ATS) Design Center for on demand business, within the Product and Solutions
Support Center (PSSC) in Montpellier, France. He received a M.S. degree from a
top-ranked French computing engineering school (ENSEEIHT, Toulouse) in
1996. After his graduation, he worked for Matra, providing consulting and support

xvi Grid Computing Products and Services

for CAD solutions for space applications and aeronautics, spending 16 months in
Boston, Massachusetts. Prior to his current position in IBM, he was a senior
technical consultant for a Web agency in Paris. He joined IBM in 2003 and is the
lead for the deployment and operations of the grid connecting the network of IBM
Design Centers worldwide.

Chin Yau Lee works as an Advisory Technical Specialist in grid computing for
IBM ASEAN/South Asia. He holds a Honours Degree in Computing and
Information System from the University of Staffordshire. He has been using Linux
since 1996 and had a few years of experience as a UNIX and Linux engineer
before joining IBM. His areas of expertise includes High Performance Linux and
UNIX, UNIX Systems Administration, High Availability solutions, Internet based
solutions, and grid computing architectures, which he has been actively working
on for the last four years. He is also an IBM Certified Advance Technical Expert
on AIX, Sun™ Certified System/Network Administrator and Red Hat Certified
Engineer. He co-authored the redbook Deploying Linux on IBM ^ pSeries
Clusters, SG24-7014.

Carlos Alexandre Queiroz Silva is an independent consultant working for Alex

Microsystems. He has been working with grid computing, Jini™, and J2EE™
technologies since 2000. Currently, he is earning a Master's degree at
Universidade de São Paulo as a distributed systems and network specialist. He
has published articles at several congresses, such as middleware2003, SBRC,
grid computing, and parallel applications events. Carlos is an active developer of
InteGrade, which can be found at http://gsd.ime.usp.br/integrade.

Joao Almeida is an IT Specialist with IBM Portugal. He is a Red Hat Certified

Engineer and has four years of experience in Linux AIX and pSeries®. Working
in implementation services and technical support, his areas of expertise include
Linux, AIX, HACMP™, WebSphere®, and grid computing.

Fabiano Lucchese is the business director of Sparsi Computing in Grid

(http://www.sparsi.com) and works as a grid computing consultor in a number
of nation-wide projects. In 1994, Fabiano was admitted to the Computer
Engineering undergraduate course of the State University of Campinas, Brazil
and in mid-1997, he moved to France to finish his undergraduate studies at the
Central School of Lyon. Also in France, he pursued graduate-level studies in
Industrial Automation. Back in Brazil, he joined Unisoma Mathematics for
Productivity, where he worked as a software engineer on the development of
image processing and optimization systems. From 2000 to 2002, he joined the
Faculty of Electrical and Computer Engineering of the State University of
Campinas as a graduate student and acquired an M.S. degree on Computer
Engineering for developing a task scheduling algorithm for balancing processing
loads on heterogeneous grids. Fabiano took part in publishing the redbook Grid
Services Programming and Application Enablement, SG24-6100.

Preface xvii
 Nam Keung is a senior technical consultant for IBM in Austin, Texas. He has
worked in the area of AIX ISDN communications, AIX SOM/DSOM development,
AIX multimedia development, NT clustering technology, and Java™ performance.
His current assignment involves helping IBM business partners and solution
providers in their efforts to port and deploy applications to the pSeries platform.
He also consults in performance tuning and other educational needs for the
pSeries platform.

Acknowledgements
Thanks to the following people for their contributions:
Joanne Luedtke, Lupe Brown, Cheryl Pecchia, Arzu Gucer, Chris Blatchley,
Wade Wallace, Ella Buslovich
International Technical Support Organization, IBM

Tony White
Worldwide Grid Computing Technical Sales Business Unit Executive, IBM

Ronald Watkins
Worldwide Grid Computing Business Development Executive, Public Sector, IBM

Matthew P Haynos
Program Director, Grid Computing Strategy, IBM Somers

Matthew Friedman
Worldwide Grid Marketing Executive

David Chisholm
Manager Grid/VE, Linux, and Digital Media, IBM Americas

Stephen Gordon
Industry Solutions Executive, IBM Beaverton

Al Hamid
Executive IT Architect and STSM, Grid/OSS Worldwide Leader, BCS

Peter Nielsen
Linux & Grid EBO, IBM Global Services

Chris Reech, Viktors Berstis, Richard Strysniewicz, Jeff Mausolf, Dennis Spexet
IBM Global Services / e-Technology Center, Grid Computing Initiative

Nina® Wilner
Grid Computing - IT Technical Architect LifeSciences, IBM Austin

Bill Bovermann
World Community Grid™ Executive, BCS Seattle

xviii Grid Computing Products and Services

Lee B Wilson
Technical Sales Specialist, IBM Dallas

Susan Malaika
STSM (Web, XML, Grid, DB2®, Data), IBM Silicon Valley Laboratory

Allen Luniewski
Information Management On Demand Strategy, IBM Almaden

Chris Dawson
IT Architect, Grid Community of Practice Lead, IBM Bethesda

Alexei Chirokikh
Consulting IT Architect, IBM Rochester

Robert Vrablik
Grid Computing Strategy and Planning, IBM Poughkeepsie

Barbara DeLibero
Worldwide Sales Executive - Grid, IBM Waltham

Dr. Alexei Chirokikh

Public Sector Business Consulting Services IBM

Judith Adolino
WW Linux and Grid Services EBO, IBM Global Services Boca Raton

Joe Zhou
Grid Computing, XML and Relational conversion, IBM Southbury

Dikran S Meliksetian
STSM - High Performance On Demand Solutions, IBM Southbury

Takanori Seki
Distinguished Engineer - Grid Computing Business, Asia Pacific, IBM Japan

Tomoari Yasuda
Grid Specialist, IBM Japan

Yann Guerin
EMEA Grid Computing TSM, IBM France

Herve Sabrie
Manager, EMEA Design Center & Solution Centers, IBM France

Ruth Harada
Alliances Manager, IBM Brazil

Preface xix
 Katia Pessanha
Universities Alliances Manager, IBM Brazil

Luiz Roberto Rocha

Grid Specialist, IBM Brazil

Srikrishnan Sundararajan
IBM India Software Labs

Clive Harris
Senior Architect, IBM UK

Dr. Manfred Oevers

Life Science Solutions Development and Grid Computing, IBM UK

Cliff Vars
Information Integration Technology Solutions, IBM Austin

Ellen Stokes
STSM, Grid Computing, IBM Austin

Pete Henry
Client Executive, ISV & Developer Relations IBM

Dr. Carl Kesselman

University of Southern California/Information Sciences Institute

Antonio Saverio Rincon Mungioli

Escola Politécnica of Universidade de São Paulo, Brazil

Elton Minetto
Universidade Comunitária Regional de Chapecó, Brazil

Edward Walker, Ph.D.

Texas Advanced Computing Center, University of Texas at Austin

Karen Miller
University of Wisconsin - Madison

Michael Humphrey
Altair Engineering, Inc.

Wayne Slater, David Smith, Marnie Biles

Platform Computing Inc.

Mary Bass, Piush Patel

United Devices Inc.

xx Grid Computing Products and Services

 Tony Bishop, Jennifer Smith, Pat Aughavin
DataSynapse Inc.

Become a published author

Join us for a two- to six-week residency program! Help write an IBM Redbook
dealing with specific products or solutions, while getting hands-on experience
with leading-edge technologies. You'll team with IBM technical professionals,
Business Partners, and clients.

Your efforts will help increase product acceptance and client satisfaction. As a
bonus, you'll develop a network of contacts in IBM development labs, and
increase your productivity and marketability.

Find out more about the residency program, browse the residency index, and
apply online at:
ibm.com/redbooks/residencies.html

Comments welcome
Your comments are important to us!

We want our Redbooks™ to be as helpful as possible. Send us your comments

about this or other Redbooks in one of the following ways:
򐂰 Use the online Contact us review redbook form found at:
ibm.com/redbooks
򐂰 Send your comments in an e-mail to:
[email protected]
򐂰 Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. JN9B Building 003 Internal Zip 2834
11400 Burnet Road
Austin, Texas 78758-3493

Preface xxi
xxii Grid Computing Products and Services
 Part 1

Part 1 Introduction

© Copyright IBM Corp. 2005. All rights reserved. 1

2 Grid Computing Products and Services
 1

Chapter 1. Introduction
This chapter discusses the following:
򐂰 Concepts of grid computing
򐂰 Terminology
򐂰 Classification of the grid disciplines that will be the basis of the analysis of the
products and scenarios

© Copyright IBM Corp. 2005. All rights reserved. 3

1.1 Introduction
Grid computing has been evolving since its formal definition in the early 90's. At
that time, grid computing was a concept and started to have incipient
implementations in universities. Today, it is a reality that can be used not only in
academic or scientific environments, but in commercial bids to provide leading
edge solutions to clients. As stated in the Introduction to Grid Computing with
Globus, SG24-6895, “grid computing is distributed computing taken to the next
evolutionary level”.

Several things have happened from the time of initial concept and academic
implementation. Over the following years, grid was used in laboratories and
gained maturity as the concept became a model. That model started a new way
of thinking in the IT community and helped solve requirements for end users and
the consumers of the IT industry. Definitions like server grid, desktop grid,
data/information grid, and others appeared.

The grid paradigm inspired several technical people outside the places where the
concept was born. It was integrated into products of both incipient and well
established IT enterprises. The paradigm opened the way to real pieces of
technology that could be used for commercial purposes.

With grid computing, we can observe a very interesting evolution: What started in
papers and in projects led by universities became a conceptual architecture that
can be used with commercial targets to enrich a user’s experience and to help
with the requirements of the enterprises that consume IT.

1.1.1 Grid computing as an evolution of distributed computing

Grid computing is an evolution of distributed computing that allows you, through
open standards, to see independent and physically scattered computing
resources as though they were a unique large virtual computer. This is a simple
definition of grid computing, but good enough for explaining grid in a few words to
a new comer, or to someone who does not know very much about IT.

In a more formal and complete definition, grid computing can be seen as a

distributed computing model that supports the concept of virtual dynamic
organizations by providing secure and coordinated access and sharing of
heterogeneous and geographically distributed resources, such as applications,
data, processor power, network bandwidth, storage capacity and others, over a
network and across organizational boundaries, using a set of open standards
and protocols. Grid users see these resources as a large virtual computer.

4 Grid Computing Products and Services

If we analyze this definition deeper, we will find several important elements:

Distributed computing model...

Grid is based on distributed computing. It means that the infrastructure is not
centralized, because its resources are geographically distributed. This implies
some kind of communication between them.

...over a network...
The network is the physical structure that interconnects the distributed
resources. A network generally only provides the pipes to connect some
component with other components. However, in order to create comprehensive
and interesting communications, it is necessary to adopt some protocols and
open standards.

...using open standards and protocols...

Open standards and protocols provide the mechanisms needed to create
communications between components developed by different vendors. They also
allow the developers to concentrate on the business logic rather than on the
method of programming communication routines.

...using shared resources...

A distributed computing infrastructure provides the ability to share resources.
Initially, we see resources as CPU, disks, and memory. However, resources can
also involve other elements, such as networks, databases, storage devices, and
so on. Sharing resources through a network allows you to have a homogeneous
system composed of heterogeneous resources. A request can use a resource
that is outside of the organization to which it belongs. It brings forth the concept
of "virtual dynamic organizations".

... inside a virtual organization...

An example of a virtual organization is a department within an university or an
university in a community. When a user requires a service from the grid, it can
get the solicited resource from a second party that belongs to another
organization. For a certain time, these two parties will belong to the same
organization, a virtual organization intended to provide a resource to the
requester. A virtual organization needs to be dynamic, because if the requester
needs more resources than the provider can provide, then the grid should search
for more resources, expanding that organization to deliver what the requester is
asking. This dynamism applies as well when the requester asks again for the
same kind of resources and the grid provides what is available at that moment.

Chapter 1. Introduction 5
 ...with secure access...
The implementation of virtual dynamic organizations requires secure
mechanisms to ensure protection of all shared elements. If a user requires CPU
time to run a certain program, it should not jeopardize any grid component.

Security also implies a mechanism to guarantee the identity of the actors,

meaning authentication of the consumers and providers. Security is also related
to authorization, which means that as soon as a consumer or provider has been
authenticated, it will use only the resources for which it has proper rights. There
are additional implications related to security, like keeping up-to-date the list of
users and resources, or the Access Control Lists (ACLs).

The integrity of the information is needed to ensure that the information that is
transferred in the network is not modified after delivery. Security has to enforce
privacy as well, to make sure that only those who have the right to see the
information will be allowed to.

... and coordinated access...

The notion of coordination brings forth the idea of a scheduler for jobs,
processes, or higher level units of processing like tasks. Implementing
mechanisms to provide quality of service (QoS) comes with coordination.
Although QoS is not an original quality of grid, it is obvious that if a grid can
ensure QoS, it will become more effective in fulfilling business objectives. For
example, it is possible to implement a grid with regular users and premium users.
These premium users, who are willing to benefit from a higher QoS, can be billed
for the exclusive services provided by the grid, which guarantees them that their
jobs will be scheduled on time by increasing their priority. These are only a few
qualities needed to provide an acceptable QoS.

Coordinated access is required in many ways. If someone submits a job to the

grid that can be parallelized, it might be desirable to have the grid split the job
into a number of subjobs, process these smaller chunks of computation, and
return a unique consolidated result to the requester. On the other hand, if the
requester prefers to receive the results as they are coming, the grid should allow
it. Both examples require coordination features for a single actor. But grids must
provide coordination to all the requesters, producers, consumers, and donors.

Grids must also implement policies to satisfy the requirements of the consumers
and to use the capabilities of the donors without exceeding the levels that they
stated. Coordination requires the implementation of policies, the registration of
the capabilities of the donors, some metering abilities, and some intelligence to
optimize the usage of the whole grid. It should plan for potential cases, such as a
machine turned off during non-working hours or days. If the grid has premium
users, it has to give a higher priority to their requests

6 Grid Computing Products and Services

1.1.2 Grid computing models
It is possible to identify models of grids based on the type of service that their
donors give and that their requesters demand.

There are also some fundamental grid models based on the type of basic
services provided. Resources can basically be computing power, provided by
servers or individual computers, data storage capacity, provided by information
and data repositories, or network bandwidth, provided by networked
infrastructures. Of course, a grid can be conceived to handle several types of
resources. It is also possible to implement higher, value-add services based on
one or more of these models. These grids expand the principles of the basic
models, leveraging and combining them in innovative way to provide more
advanced services.

Computational grid
A computational grid is an infrastructure that allows resources to donate
computing power to the grid whenever the workload demands. This infrastructure
is suitable for applications that demand:
򐂰 As much processing power as possible or additional processing power during
certain periods of time and a single machine cannot provide it or at too much
cost.
򐂰 It is usually associated with resource scavenging in desktops machines and
underused servers.

Such a grid takes advantage of the idle resources in the virtual organization. Just
think about the cycles of CPU that are unused when a typical user browses on
the Internet, reads an e-mail, or creates office documents like presentations or
word processing files. Additionally, it would allow to reuse the infrastructure and
the client application for more than one project at the same time.

Examples of server-oriented grids

Let us consider a person who uses complex spreadsheets to execute heavy
financial calculations. Sometimes this kind of spreadsheet needs to iterate as
many times as possible to get closer to the desired result. The more iterations,
the more accurate the results of the execution will be. That process is often
applied to carry out profitability analysis.

Profitability analysis and stocks evaluations for shareholders, or loan approvals,

can be carried out with a server grid as well. Usually, these operations are
executed in the customer assistant’s desktop machines and it is possible to
improve the response time to the client by using a server-oriented grid to run this
kind of application. The users who are in contact with the client could submit a

Chapter 1. Introduction 7
 job to the grid, it would use idle resources to fulfill the job, and it would return
results faster than if the user had used only his own computer.

Another interesting application area is the petroleum industry. Geologists gather

information in oil fields, which are almost always away from the data centers.
Therefore, these users have very powerful workstations with high level graphics
and processing capabilities, since they need to analyze the composition, geology,
and streams in the reservoir locally. This reservoir study, which encompasses
seismic and geology analysis, requires high processing power. A fast response
from the system is important to the geologists and engineers on the field, who
have to make important real-time decisions for the oil company. Despite the great
high-end workstations that exist today, more processing power is sometimes
needed to be more effective on the field. A server grid provides the ability to
submit jobs from the field that are executed in the main, regional, or local
datacenter. In that way, the activity of the professionals in the oil field can be
improved and the time to take decisions that impact on business is reduced.

Examples of desktop-oriented grids

This model is used in World Community Grid, and it could be implemented with
commercial objectives within an enterprise as well. An enterprise could
implement a desktop grid to free some server resources. It might be possible to
identify some activities that usually run on servers and could be run on other
computers to free the servers from these activities. It would allow the servers to
have more processing power during the peak activities that can be run only on
this server.

In this kind of grid, it is more complex to guarantee the time needed to get a
result. For this reason, it is primarily intended for applications that can afford to
deliver results with no critical time commitment. Although it is called a desktop
grid, some distinct coordination points, typically deployed on server machines,
play an important role in this solution; scheduling, directory services, dispatching,
disassembly, and assembly mechanisms should be provided. Another important
characteristic of this grid that could be very appreciated by its adopters is the
provision of a toolkit or framework to create applications.

Note: World Community Grid's mission is to create the largest public

computing grid benefiting humanity. More information about World Community
Grid can be found in Appendix B, “World Community Grid” on page 149 and at
the following Web site:
http://www.worldcommunitygrid.org/

8 Grid Computing Products and Services

Data grid
Within a data grid infrastructure are the components used to provide grid
capabilities to the data and information virtualization disciplines. It provides the
ability to supply homogeneous access to heterogeneous repositories of data. It
allows the data consumers to see an unified image of the respective information
or data spread across different resources, potentially based on different
technologies.

Information Infrastructure
Information is usually defined as "meaningful data". Meaningful data is often
associated with the unit of information that means something useful to the end
user. The vision of different repositories of information as though they were a
single one illustrates the idea of information integration. Information grid is
intended to integrate different sources of information in a comprehensive way. It
allows applications and users to see a single database that hides the complexity
of accessing multiple databases. The information spread across them is
published by the grid as though it were really centralized.

This grid should implement connectors to the final databases, which surely are
heterogeneous and geographically distributed. It is desirable as well to have
replication and caching mechanisms that make the management and usage of
the IT infrastructure more efficient.

File system and block data infrastructure

File system and block data infrastructure for grid follows the main principle of the
information infrastructure but at a lower level. While the information infrastructure
provides unified access to information, the file system and block data
infrastructure provides unified access to files or chunks of data.

Data is not always meaningful to the end user. However, it is the unit of work of
operating systems and hardware devices. Data in this context means blocks of
data or files.

SAN-FS is one of these technologies. It implements mechanisms to see storage

components from different vendors as a single disk in a Storage Area Network.
SAN-FS is an example of a basic data grid implemented at the hardware level.

Network grid
In a typical corporate network, computers are very often permanently connected
to it while using only a portion of its bandwidth. Every machine, servers and
desktops, has underused network bandwidth, which can be considered as an
idle resource. When a given user or machine requires more resources from the
network, a bottleneck is reached.

Chapter 1. Introduction 9
 Examples
In a file transfer using FTP, the server side of it allows no more than two or three
simultaneous connections from the same requester. The traffic may be limited by
the server itself just to avoid the situation where a given user gets a great portion
of the server’s bandwidth. This is necessary because there could be other
requesters asking the same server for its resources. The server has to
implement some way to ensure a balanced competition among their requesters.
This is fair from the server point of view, but from the requester standpoint, things
may be seen a bit differently. If the desktop tries to download a file from a FTP
server that limits the bandwidth just in case there are other requesters competing
for the same resource, some idle network resources will appear on the requester
side. Imagine a desktop connected through a 100 Mbps connection to a server
that perhaps provides one or more 1 Gbps connections. If the limitation is set up
to 1 Mbps per connection, the desktop will have theoretically 99 Mbps free, ready
to use. The server has to leave some free network resources to satisfy other
users’ requirements. But the desktop could use its free resources. We know that
there could be additional factors to consider, like WAN bandwidth limitations,
routers, and so on. But, just to keep it simple, we are showing this example in a
LAN environment, or in an environment without other physical restrictions than
the bandwidth of the nodes.

In this example, the desktop could connect to other servers in the same
infrastructure to simultaneously download another portion of the file. This way,
the first server would not modify its policies to attend users’ requests, the second
server could provide more data to the desktop, and the desktop would more
effectively use its own network interface.

This implementation requires some intelligence in the desktop side, perhaps a

fancier file transfer protocol or a smarter FTP client.

From the server side, this requires the duplication of the pool of files on each
server. Perhaps the servers are geographically distributed, in a WAN
environment. In this case, it is obvious that storage is cheaper than network.
Therefore, it is possible to have the files downloaded from several servers.

This type of grid is sometimes called “distributed peer to peer” grid, network grid
communication grid, or “grid delivery”. We can see some resemblance to the well
known peer-to-peer (P2P) programs. However, a network grid has some more
efficient mechanisms to provide services. For example, it can provide a
centralized directory of files, or the directory itself can be part of the client side.

IBM has implemented an internal network grid called IBM downloadGrid with
services similar to the one described above, which allowed IBM to reduce the
transatlantic network traffic.

10 Grid Computing Products and Services

 Note: More information about the IBM downloadGrid can be found at the
following Web site:
http://www.research.ibm.com/journal/sj/434/meliksetian.html

In order to implement a well balanced network grid, it is necessary to analyze the

whole infrastructure, the network topology, the behavior of the users, and the cost
of the most critical connections, as well as the way the contents are used. It is
recommended that the CPU that performs strong processing activities is as close
as possible to the repository of contents.

Multipurpose grid
The multipurpose grid is perhaps the more common implementation in the future
of grid computing. The infrastructure of this grid should be adaptive enough to
provide any of the grid models. It could be implemented as well as a meta-grid
with abilities to route the requests to the grid that supports the right model to fulfill
them.

1.1.3 Grid and its relationship with other IT components

This section discusses the relationship between grid computing and other
components of IT. Some of those components are technologies, while others are
concepts or paradigms. It is important to see that grid is related to all of them in
some way. As we will see, none of the following items is exclusively grid. In most
of the cases, grid leverages them and is leveraged by them.

Grid and deep computing

Deep computing, as well known as high performance computing (HPC), is a
technology that uses clusters of machines to execute high demanding CPU
tasks. A deep computing cluster is generally implemented with machines
connected through high speed networks. In these cases, the job that the cluster
runs can be split and parallelized into small pieces that run in the cluster’s nodes.

A deep computer infrastructure can be part of a grid, and a grid can be the
mechanism to provide deep computing resources. Grid provides something that
a typical deep computing cluster does not: access to heterogeneous resources.

Grid and on demand

On demand business is IBM’s vision of the IT industry and market. It
acknowledges that the flexibility of the markets will demand flexibility in
enterprises’ business processes. Flexible, adaptable, and resilient business
processes require a flexible IT operating environment. The operating
environment has to be virtualized; it should be based on open standards to be

Chapter 1. Introduction 11
 integrable and must have autonomic capabilities. Grid computing leverages on
demand because it is a fundamental component to achieve the highest degree of
virtualization, one of the key factors of the on demand operating environment.
Grid is also leveraged when its implementation complies with open standards; it
is integrated into the business processes of the organization, and its components
have autonomic abilities.

Grid and UMI

In an on demand enterprise, once the business and the IT infrastructure are
flexible, it is possible to think of new ways to make the IT services and other
enterprise services profitable. UMI (Utility Management Infrastructure) is a model
or specification of how the IT services can be acquired and paid as though they
were a service provided by a traditional utility. Utilities use the “pay per usage”
payment method. UMI conceptually establishes what components a utility must
have to provide IT services and their consumers pay per usage. It means
variable costs for the clients. It also means, for the IT utility provider, that the
infrastructure can be reused and shared. Grid computing capabilities are a
reasonable option to implement an IT utility, but not the only choice. Grid
provides provisioning, metering, billing, and mechanisms to use idle resources,
sharing of portions of resources, and security and management. All of them are
required qualities to implement an infrastructure aligned with UMI.

Grid and virtualization

Virtualization is the ability to provide a unified vision of a set of resources. These
resources may be geographically distributed, run on different technologies, and
developed by different vendors. Virtualization is one of the key elements of the on
demand operating environment, and essential on grid implementations. More
information about on demand operating environment can be found at:
http://www.redbooks.ibm.com/

Grid and provisioning

As explained in “Provisioning” on page 86, provisioning mechanisms can provide
on demand additional resources to the grid. Alternately, the grid can provide
resources as well. Both can live independently, coexist in the same infrastructure,
and be leveraged if they are well integrated.

1.2 Categories and functions

Grid computing is a concept that can be studied in several ways. The information
contained in this redbook has been classified and analyzed according to cross
criteria, categories, and functions.

12 Grid Computing Products and Services

 Categories are related to the layers in a grid stack products’ portfolio, as shown in
Figure 1-1. Categories help the reader relate products with their role and scope
within a grid. IT Architects will find the categories to be a technical reference to
help them find the right products for their solutions. Categories will also be useful
to understand the scope of the professional services that should be implemented
to satisfy the requirements of a solution. IT architects must consider business,
functional, and non-functional requirements that should be fulfilled by the
infrastructure and the services that a product and a category of the grid layer can
provide.

1.2.1 Categorization of components

The layer structure, or grid stack (Figure 1-1), of the grid components is an
creation of the traditional IT industry.

Applications
Content Management
Billing and Metering
Orchestration and Provisioning
Task Scheduling
Workload Management
Systems Management
Software licensing
Job Scheduling
Structured Data Virtualization
File and Block Data Virtualization
Grid Middleware

Figure 1-1 Grid stack products’ portfolio

Application
This layer describes an environment for developed applications to take
advantage of the whole set of features provided by the grid. Applications can
make use of all layers in the stack, through classes, APIs, frameworks, toolkits, or
Software Development Kit (SDKs).

Chapter 1. Introduction 13
 The application layer is also associated with the applications developed for a
specific industry.

Content management
Content management provides another degree of virtualization to data and
information within a grid. It is related to the handling of digital media, like video,
audio, images, or streams. These are not structured data, and need to be treated
in a different way. It usually requires specific features, such as searching an
image database for the most similar picture to the one given in input.

Billing and metering

Metering is a desirable service in a grid. It helps quantify the efficiency and
performance of the grid, and the usage of its resources. It is the base of billing
activities, which affect a price to the resource consumption units delivered by the
grid. Provisioning products have some basic capabilities of metering and billing
to register the events related to the resource’s provision.

Orchestration and provisioning

Provisioning is the ability to transform a basic inactive resource into an mobilized
resource for applications, by installing and configuring all the required software in
an automated way. Orchestration is designed to define and optimize the
concurrent provisioning activities in an infrastructure, including the return to the
initial state of the resources once their utilization has been ended.

Task scheduling
Task scheduling provides an environment to run small pieces of execution that,
combined, create the unit of work that the requester needs to execute. The
decomposition of this unit of work into smaller pieces can consist of parallel,
short running tasks or of tasks that have to be run in an special order within an
automated workflow.

Workload management
Workload management is a mechanism designed to balance the workload
among different resources. It is defined during the setup and used at run time.
The workload management can be dynamic depending on the real time workload
of the involved components and the policies applied to them. This layer may be
associated with a Service Level Agreement (SLA). In fact, if a SLA exists, the
workload management layer should be aware of it in order to provide the right
power to the right resource at the right time.

14 Grid Computing Products and Services

System management
Management is vital in every IT infrastructure. Distributed computing
environments require even more sophisticated tools and policies.

Several of the activities behind the scenes are related to management and
administration. Almost all layers in the grid stack have a strong relationship with
management. The middleware has to be configured and enabled. Security has to
be set up and security logs have to be sent to the system management. Block,
file, and structured data virtualization have to be configured and adapted to
optimize their utilization. License and Workload management are obvious
management activities. Provisioning, orchestration, billing, metering, and the rest
of the layers in the grid stack have to be configured and managed as well. The
Tivoli Suite is an example of management infrastructure. It is intended for a
traditional IT environment, but can clearly be integrated into a grid.

Software licensing
License management is a technology that allow you to control the licenses that
are in use for a certain period of time. It provides elements to determine which
machine is the most suitable candidate for a task, based on the software installed
on it.

As grid is based on a dynamic infrastructure, it may require that certain software

products run for a limited time, then stay on standby for a certain period.

Job scheduling
Job scheduling is used to optimize the execution of jobs onto a grid. In this
context, a job can be understood as the computational work needed to deliver a
meaningful result within the context of an application. Its execution is governed
by policies that rule the basic scheduling and prioritizing of jobs to be run.

Structured data virtualization

Structured data virtualization provides a unified access to heterogeneous
databases as though they were a large and single database. It supplies
grid-based access to structured data sources of information, like databases,
documental databases, XML, or flat files with certain structure and objects.

This is a higher level of data virtualization. A grid can create an unified vision of
the different repositories of information. The consumers of this service should
see a single database that in fact is composed of aggregations and associations
of other databases.

Chapter 1. Introduction 15
 File and block data virtualization layer
File and block data virtualization mechanisms are related to the lowest layers of
data services provided by the grid. They can live at the hardware level,
associated with blocks of data managed by storage devices.

In the block data virtualization case, the actor is the hardware itself and its
drivers. The components create the unified vision at the operating system level,
which, in turn, will create a file system on top of the block data grid. File data
virtualization is provided by file system implementations.

Grid middleware
The grid middleware provides the integration among the grid components. It is
the keystone software that provides basic services to grid components. These
basic services are integration mechanisms based on standards, description, and
creation of services. The middleware also includes the elements that allow the
grid to interact with its own components.

What about grid security?

Security is a service that is present across the whole grid stack. This explains
why it does not appear in the Figure 1-1 on page 13. Its mission is to guarantee
the integrity of the grid, of its resources, and of its users interactions, by
preventing unauthorized access. Security provides authentication, authorization,
integrity, and privacy to the operations carried out by the grid.

1.2.2 Functionality on grid computing

Functions group high level product roles to better understand the details and the
capabilities of the products. A classification per function, also known as
disciplines, will be used throughout all chapters of this redbook. They are the
attributes an IT Architect will look for to understand the main features a product
brings to the grid. This classification, as shown in Figure 1-2 on page 17,
presents four elements:
򐂰 Web Services Core/Hosting
򐂰 Workload virtualization
򐂰 Information virtualization
򐂰 Other Disciplines

Examples of other disciplines would be security, provisioning, billing, and

metering.

16 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure 1-2 Product evaluation criteria

Products charts based on their disciplines

Products covered in this redbook support the functions described above to a
certain point. The width of each bar (0 to 3) on Figure 1-2 reflects our perception
of the product in a grid computing environment, as follows:
򐂰 0: The product has no features of this discipline.
򐂰 1: The product has some features of this discipline.
򐂰 2: The product has most of the features of this discipline.
򐂰 3: The product was developed and designed to completely cover the
discipline.

As an example, you will find in Figure 1-3 on page 18 the product chart for the
WebSphere Information Integrator, which illustrates our understanding of the
support it provides for the functions specified in this redbook. The chart is to be
read as follows:
򐂰 Information virtualization
3: The product was developed and designed to completely cover the
discipline. WebSphere Information Integrator is a data federation software
that provides structured access to heterogeneous types of data.

Chapter 1. Introduction 17
 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure 1-3 WebSphere Information Integrator

Product charts of most of the products described in this redbook are in

Appendix C, “Products reference” on page 169.

Web services core/hosting

This is the application environment, the arena where applications live, provide
services to requesters, and consume services from the grid. The environment is
intended to facilitate the life cycle of the grid-enabled applications, in other words,
how they are developed, run, and maintained.

Examples of the capabilities that an application environment can provide are a

run-time environment for the application itself, monitoring tools, security features,
and APIs to access the lower layers of the grid stack (toolkits). These toolkits
should provide a common set of utilities to improve the development time
experience. For example, a toolkit could provide a set of tools to build
applications that can be parallelized in an easy way, either to be further deployed
on a desktop grid or on a server grid, or high-level interfaces to handle grid
resources and submit jobs to them. Other tools include components to split the
unit of works and consolidate the results.

18 Grid Computing Products and Services

For more information about Web Services Core/Hosting, see Chapter 2, “Web
services core/hosting” on page 23.

Workload virtualization
Scheduling, the main matter of workload virtualization, is the ability to execute a
unit of work at the most favorable moment under certain policies. Scheduling can
be defined at a single machine level (within a grid) and at grid level (over several
computation pools); this is what meta-scheduling is about. The unit of work and
terminology are defined according to the type of application environment. It is
normally possible to schedule jobs, tasks, or services. Just to simplify matters,
we can generically consider it as an “activity”.

A scheduler’s main role is to ensure that some activity will be carried out over a
certain period of time. The scheduler has to consider the workload of the
resources that are candidates to run some activity. It would not make sense to
submit a consequent job to a machine that is already heavily loaded, while other
machines just sit in the grid with idle resources. If the machine accepts that
workload, it should delay the execution of its current and new activities with a
lower priority.

For more information about Workload virtualization, see Chapter 3, “Workload

virtualization” on page 41.

Information virtualization
Information virtualization encompasses the different types of data infrastructure
for grid and the different types of products available to implement them, in order
to enable the unified vision of different repositories of data. The virtualization can
be done at the block data, file, or information levels.

The block data virtualization provides the unified access directly from hardware
or from a driver of a given hardware that runs on a given operating system. The
consumer of this data is usually the operating system itself, and the unit of work
is a raw block of data in a given storage device. It allows operating systems to
see physical resources that are not in the same device as a single logical volume
or file system. Then the operating system can emulate a regular file system on
top of the services provided by the lower layers. These components create a
logical low level structure on top of the physical structure of the storage and
publish it to the operating systems.

The file virtualization level is associated with file systems. In this case, the unit of
work is a file. A file can be meaningful by itself, like a spreadsheet or a word
processing document; however, applications usually need a combination of files
to map the data into meaningful information.

Chapter 1. Introduction 19
 The information layer is the highest level of the information virtualization
discipline. It is also known as the structured data layer and it provides the ability
to integrate information from heterogeneous resources with heterogeneous
formats by accessing them using heterogeneous protocols and publishing them
to the applications in a homogeneous way, thereby enabling the applications, or
consumers of this information, to have a unified vision of the information
repository.

For more information about information virtualization, see Chapter 4,

“Information virtualization” on page 65.

Other disciplines
Examples of other disciplines are security, provisioning, billing, and metering.
Explanations about these disciplines can be found in Chapter 5, “Other
disciplines” on page 85.

1.2.3 Grid computing products and players

IBM Business Partners — resellers and distributors, consultants and integrators,
and developers — all play critical roles in bringing the benefits of grid computing
to both large enterprise and mid-market clients, across a variety of industries.

The products presented in this redbook are the main products suitable for a grid
computing environment to date. We will not discuss the displayed products as
grid products, or grid-enabled products, but as products with useful and strong
capabilities that are qualified to be part of a grid computing solution. Some of
these products are IBM products, but some of them are also developed by
Independent Software Vendor (ISVs) or Open Source communities. These ISVs,
also presented in Chapter 7, “ISV suites” on page 113, can be integrated through
IBM professional services as well.

IBM has established key business relationships with leading middleware ISVs to
provide clients with the most robust grid solutions in the industry. Each plays an
important role in IBM Grid Computing solutions. More information can be found
at the following Web site:
http://www.ibm.com/grid/grid_partners/index.shtml

20 Grid Computing Products and Services

 Part 2

Part 2 Functionality
and products

© Copyright IBM Corp. 2005. All rights reserved. 21

22 Grid Computing Products and Services
 2

Chapter 2. Web services core/hosting

This chapter discusses the following:
򐂰 Grid application development toolkits
򐂰 Grid application hosting environment products
򐂰 Grid standards, as elaborated by the Grid community, and how Grid services
are merging with Web services

© Copyright IBM Corp. 2005. All rights reserved. 23

2.1 Introduction
This chapter discusses products and technologies that can be used to support
the execution of grid applications, or constitute a hosting environment for these
applications.

The Grid Standards and Toolkits section focuses on low-to-middle level software
toolkits that support the building of grids, but are not meant to be usable right “out
of the box”. Their purpose is to make grid oriented programming easier by
providing developers with APIs they can call from within their own code, in order
to implement grid features.

The Grid Application Environments section aims at describing what products are
eligible for grid application deployment, according to grid standards, such as
Open Grid Services Architecture (OGSA).

2.2 Grid standards and toolkits

A development toolkit, also known as a software developer's kit (SDK), is a set of
programs used by a computer programmer to write application programs.
Typically, an SDK can include a visual screen builder, a code editor, a compiler, a
linker, and some other facilities.

A grid application development toolkit focuses on providing fundamental enabling

technology, for grid users to securely share computing power, data, and other
resources across corporate, institutional, and geographic boundaries, without
sacrificing local autonomy. Underlying features to achieve these tasks include
information infrastructure, dynamic resource discovery and allocation, data
management, communication, fault detection, security, and portability.

Most grid application development toolkits already abide by grid standards, as

defined by the Global Grid Forum (GGF) and main contributors, such as the
Globus Alliance, OASIS, and IBM. The latest contributions, the Web Services -
Resource Framework and Notification (WSRF and WS-Notification, respectively)
are currently being reviewed by the Oasis consortium for standardization.

2.2.1 Grid standards

Grid computing uses open standards and protocols to enable the virtualization of
distributed computing resources to create a single system image across
heterogeneous, geographically dispersed IT environments. In order to guarantee
grid components’ and applications’ portability and interoperability, a number of

24 Grid Computing Products and Services

rules and open standards, which new grid developments are expected to comply
with, have been published and refined.

To better understand how OGSA and WSRF combine to become a grid standard,
one has to consider what steps are required to design a technology:
1. Defining the technology: What is the technology all about? What need is it
expected to fulfill? What are its limitations? What is needed to build this
technology?
2. Specifying the technology: Specifications for the components, definition of
properties and attributes for the objects, and detailed description of the
communication protocols and interfaces involved.
3. Implementation: A reference implementation to support the specifications and
to demonstrate both the feasibility and the relevance of the technology.

As far as grid technologies are concerned, OGSA addresses the first issue
(definition), WSRF addresses the second one (specifications), and the Globus
Toolkit 4.0, for example, is a reference implementation of WSRF.

Open Grid Services Architecture (OGSA)

OGSA was designed as an attempt to standardize interfaces and behaviors for
distributed systems by:
1. Refactoring the Globus protocol suite to enable a common base and expose
key capabilities.
This was achieved by generalizing and redesigning protocols and
mechanisms already identified as key for grid technologies, such as
authentication, policy and lifetime management, notification, service naming,
and reliable invocation.
2. Extending new technical requirements, such as concurrent access to
resources and management tools.
3. Introducing the notion of “Grid Service”.
The goal here was to define grid entities by their interface and behavior, to
unify access to programs and resources, and make it easier for grid architects
to achieve virtualization.
4. Embracing key Web services technologies to adopt a standard Interface
Definition Language.

The best way to look at OGSA is to consider it as a layered model, just like the
OSI reference model (see Figure 2-1 on page 26).

Chapter 2. Web services core/hosting 25

 Applications

OGSA Architected Grid Services

Web Services-Resource Framework

OGSA Enabled OGSA Enabled OGSA Enabled OGSA Enabled OGSA Enabled OGSA Enabled
Security Workflow Database File Systems Directory Messaging

OGSA Enabled OGSA Enabled OGSA Enabled

Servers Storage Network

Figure 2-1 OGSA architecture model

Web Services: Resource Framework (WSRF)

“The purpose of the Web Services Resource Framework (WSRF) TC is to define
a generic and open framework for modeling and accessing stateful resources
using Web services. This includes mechanisms to describe views on the state, to
support management of the state through properties associated with the Web
service, and to describe how these mechanisms are extensible to groups of Web
services.” -- OASIS Web Services Resource Framework Technical Committee.

Note: WSRF aims to solve the concerns that the Grid community has
expressed about former OGSA specifications, namely OGSI. Its main purpose
is to achieve the convergence of Grid services and Web services. It was
announced by the Globus Alliance and IBM in conjunction with HP on January
2004. In March 2004, the WSRF specifications was submitted to OASIS.

More information can be found at:

http://www.globus.org/wsrf/
http://www.oasis-open.org/home/index.php

26 Grid Computing Products and Services

The main improvements consisted of:
򐂰 Splitting the monolithic OGSI into a more meaningful subset of specifications,
in order to provide a collection of specifications that can be used either
individually or in combinations
򐂰 Improving compatibility of Grid services with existing Web services tooling,
hereby defining an architecture more clearly aligned with the general
evolution of Web services
򐂰 Refining Grid services specifications to make a clear distinction between the
service that operates a resource and the resource itself

More to the point, WSRF introduces both a design pattern to specify how to use
Web services to access “stateful” components, and a message-based
publish-subscribe mechanism to Web services.

From a Grid perspective, this means all components of a Grid architecture can
be described via WSRF as stateful resources accessible through one or more
Web services, whether they are physical entities (processor, disk drive,
communication link, and so on) or logical construct (authorization, job,
subscription, and so on), real or virtual, static or dynamic (through lifetime
management).

Figure 2-2 gives an overview of the Web Services based Grid Application Model.

Application-Specific Services

Program Execution Data Services

Grid Services

Web Services-Resource Framework (WS-RF)

Web Services Messaging, Security, …

Figure 2-2 Web Services based Grid Application Model

Chapter 2. Web services core/hosting 27

 For further informtion about Grid standards, check the following Web sites:
http://www.gridforum.org
http://www.globus.org
http://www.oasis-open.org

2.2.2 OGSA Implementations

OGSA implementations are grid development toolkits that enable programmers
to build OGSA-compliant grid solutions. Toolkits are available for the following
programing languages, some of them discussed in this section:
򐂰 Java/C++ (the Globus Toolkit)
http://www.globus.org/toolkit/
򐂰 Perl (WSRF::Lite)
http://www.sve.man.ac.uk/Research/AtoZ/ILCT
򐂰 Python (pyGlobus)
http://www-itg.lbl.gov/gtg/projects/pyGlobus/
򐂰 .Net (WSRF.Net)
http://www.cs.virginia.edu/~gsw2c/wsrf.net.html

The Globus Toolkit

The Globus Toolkit provides an open source reference implementation of basic
services for building a grid infrastructure and for deploying grid applications.

Note: The Globus Toolkit is an open source software developed by the Globus
Alliance. More information can be found at the following Web site:
http://www.globus.org/

The Toolkit relies on the open standard protocols described above, and provides
APIs to allocate and manage shared resources in a secure framework. It
provides the core services necessary to address the Grid Computing challenges
introduced earlier.

The IBM Grid Toolbox V3 for Multiplatforms

The IBM Grid Toolbox V3 for Multiplatforms is a platform for grid programmers to
develop and test grid services and applications, and for grid administrators to
host them.

28 Grid Computing Products and Services

 Note: The IBM Grid Toolbox is a commercial release of the Globus Toolkit 3.0
with IBM added value. More information can be found at:
http://www.ibm.com/grid/solutions/grid_toolbox.shtml

In addition to GT 3.0 features, the Toolbox provides the following add-ons:

򐂰 An integrated wizard-based installer (InstallShield technology)
򐂰 A grid service Java/J2EE run-time environment based on the GGF Grid
Service specification for hosting grid services
This environment replaces the open source stand-alone container with an
embedded WebSphere Express Application Server (see 2.3.1, “WebSphere
Application Server” on page 30 for more information).
򐂰 A Web-based management application used to manage services within the
run-time environment
򐂰 A set of configuration and administration commands
򐂰 Development tools to build, package, and use grid services
򐂰 Common (base) grid services for:
– Discovery via Service Group
– Policy Management
– Common Management Models (CMM)
򐂰 An Information Center, including tutorials to assist with the education and
understanding of the technologies and capabilities packaged within the
product
򐂰 Sample grid services and applications demonstrating key capabilities

Besides, IBM Grid Toolbox includes an embedded relational database used for:
򐂰 Reliable File Transfer service journaling, which is required for recoverability
򐂰 Backing store for Policy services
򐂰 Backing store for Service Group service

Finally, IBM Grid Toolbox embeds OpenJMS for the JMS based notification
framework.

The Platform Globus Toolkit

The Platform Globus Toolkit is a commercial distribution of the Globus Toolkit 3.0.

Chapter 2. Web services core/hosting 29

 Note: In addition to the base set of components and services available in
GT3, Platform Globus Toolkit includes significant extensions based upon the
Community Scheduler Framework (CSF), and connectors that provide
interoperability with a wide variety of third-party workload management
systems and grid offerings. More information can be found at:
http://www.platform.com/Products/Platform.Globus.Toolkit/Product.Informat
ion/Features.Benefits.htm/

Here is a list of features provided by the Platform Globus Toolkit:

򐂰 Round-robin job scheduling
򐂰 Advance reservation booking, query, and control
򐂰 Reservation-based job scheduling
򐂰 Job throttling support for increased reliability
򐂰 File staging support
򐂰 Interoperability with a wide range of third-party workload management
systems and grid service providers
򐂰 Scalability
򐂰 Job persistence
򐂰 Load balancing between multiple Platform LSF clusters
򐂰 Intelligent scheduling based on host type specification
򐂰 Kerberos Security Support

2.3 Grid application environments

The application environment is the arena where applications live, provide
services to requesters, and consume services from the grid. The environment is
intended to facilitate the life cycle of the grid-enabled applications or, in other
words, how they are developed, run, and maintained.

2.3.1 WebSphere Application Server

WebSphere Application Server provides an operating environment with
advanced performance and availability capabilities in support of dynamic
application environments.

30 Grid Computing Products and Services

 Note: WebSphere Application Server is the foundation of the WebSphere
software family, the industry's premier Java-based application platform,
integrating enterprise data and transactions for the dynamic e-business world.
More information can be found at:
http://www.ibm.com/software/websphere/index.html

More specifically, and from a grid standpoint, WebSphere Application Server is

bringing programmers a Java run-time environment to support grid
implementations, and monitoring capabilities to automatically manage the
application workload and route traffic to one server or another according to its
workload at a given time.

In addition to its support of Web services standards, WebSphere Application

Server comes with a set of tools and Java APIs that enable, for example, the
development of portal-type Web applications, such as a job submission portal.
On top of the job submission features, a programmer could implement a full grid
environment by enforcing security with user and project authentication,
policy-based authorization, proxy delegation and certificates management, and
by adding capabilities such as input data staging, computing resource selection,
and multi-format job output retrieval, all of it accessible through a simple Web
browser, as described in the Figure 2-3.

User
Browser

HTTP
Java
Servlet

Java Certificate
Web Service Authority

HTTP Java Enterprise

Server Java Database
Page Bean

Globus
Grid
Java
Resource
CoG Kit

View Controller Model

WebSphere Application Server

Figure 2-3 Globus-based Grid Portal

Chapter 2. Web services core/hosting 31

 In this case, features such as the Grid Registration service are implemented as a
Web application and based on the Model View Controller (MVC) design pattern.
The separation of presentation logic and business logic is made by defining three
parts, Model, View, and Controller, hosted by WebSphere Application Server.

The Controller manages and controls all interactions between the user and the
application. Usually, the Controller is a servlet that receives an HTTP request
when the user clicks on a link in the browser. Then, the Controller passes the
input parameters to the Model that does the work. This Grid Registration service
is made of several Controller servlets that implement methods specific to each
user action.

The Model encapsulates the business logic, rules, and data, and does the
business processing, usually implemented by Java classes, Java Beans, or
Enterprise Java Beans (EJBs). The Grid Registration service can use a session
EJB™ to store and fetch data from the database.

Called by the Controller, the View part uses the results of the business process
and constructs the response to be presented to the user, usually implemented
through Java Server Pages (JSPs). JSPs build dynamic HTML code and send it
as an HTTP response to the user's browser.

Here, WebSphere Application Server interacts with the Globus Toolkit through
the Globus Java Commodity Grid (CoG) Kit, which provides a set of APIs to
handle Globus resources; it is especially useful here to create certificates signed
by the Certificate Authority, and exchange information with the grid resource
manager in order to submit a job.

Of course, these are only a few hints about what can be created using
WebSphere Application Server as the hosting keystone; further developments
could extend the reach of the application to other products, in order for it to
include job scheduling, resource provisioning, and other grid features.

2.3.2 WebSphere Extended Deployment

WebSphere Extended Deployment (WAS-XD) is an extension of the WebSphere
Application Server to support intensive workloads. WAS-XD allows the splitting of
a given task into several tasks, and run each piece in different application servers
that belong to the same domain. This is an implementation of grid computing,
specifically, a server grid with parallel tasks. Applications have to include specific
calls to WAS-XD services and have to be adapted to this paradigm in order to be
split in smaller and parallel pieces.

Once the WAS-XD engine is running, it finds those applications that can be
placed or split in different application servers. Based on policies, it assigns

32 Grid Computing Products and Services

 pieces of execution to the right application server. It allows you to use
underutilized application servers, to provide additional capabilities during
workload peaks, and to improve the quality of services for the applications
running in the WAS-XD domain.

WAS-XD provides dynamic allocation of resources, and virtualizes pool of

resources and applications. It has autonomic characteristics, since it provides
real-time information about the current status in regards to the policies, and acts
according to them. WAS-XD allows you to apply business goals to WebSphere
Application Server applications. The operator has a graphic console that shows
the current status of the WAS-XD. It gives a new design pattern of development
that is specially designed for high performance and reliable transaction
processing.

Note: WebSphere Extended Deployment is an extension of the WebSphere

Application Server to support intensive workloads. More information can be
found at:
http://www.ibm.com/software/webservers/appserv/extend/features/

2.3.3 WebSphere Portal Server

WebSphere Portal Server is IBM’s platform to create, run, and maintain
corporate portals. This type of portals manages transactional information, and
integrates information originating from different sources, like databases, existing
systems, content providers, business partners, and so on.

Note: WebSphere Portal Server is part of the WebSphere software family.

More information can be found at:
http://www.ibm.com/software/genservers/portal/

This kind of platform requires a sophisticated degree of integration to provide

comprehensive Web access to heterogeneous sources of information. This
integration is achieved through the use of “portlets”, which are standardized
objects, loosely coupled, that have the required logic to look for some source of
information and publish that information to the portal. The portal engine provides
the required connectors and run-time services to run and contain the portlet
itself, to manage the portlet’s connector with the foreign source of information,
and to get the portlet’s output and display it in the Web page. WebSphere Portal
Server provides services to implement this programming model.

In terms of hosting, WebSphere Portal Server maintains the environment where

the portlets live. This container is a specialized portlet’s application server. All
portlets share the same communication protocols with the portlet container.

Chapter 2. Web services core/hosting 33

 WebSphere Portal Server also provides common services to the portlets on top
of those provided by the application server itself, like authentication,
authorization, connection pooling, and so on. For example, WebSphere Portal
Server provides personalization of content, and interaction between the user and
applications, processes, and content. WebSphere Portal Server implements a
single point of interactive access to all the information that a user needs and the
enterprise publishes through a comprehensive and unified Web interface.

Information and the path to this information can be personalized according to the
user and type of user who is accessing it. Personalization services also take into
account the kind of device that is being used to access the portal, and their policy
can be modelled as a matrix of users and devices. Devices can be typical
browsers, low end workstation browsers, public Web browsers (which impacts
the information’s confidentiality), cell phones and PDAs using WAP, pagers,
interactive voice response systems (IVRs), scripts automatically retrieving
content, and so on. Defining how the information is accessed and formatted
depending on the device is crucial to enable a user to have different profiles and
preferences based on the device that is being used.

Additional services provided to enhance the portal are, for example, tools to
search topics or certain content in information managed by the portal. Portal
Document Manager is a portlet that allows users to share, view, and organize
files. On the other hand, Productivity Components allow users to create, edit,
convert, and view documents like word processing files and spreadsheets.
WebSphere Portal Server also includes collaborative tools, like Instant
Messaging or People Finder Portlet, which bring advanced navigation
capabilities over a corporate directory. My Lotus® Workspace (QuickPlace®)
portlet manages workplaces, which are virtual desktops managed by the portal.
WebConferencing Portlet provides a suite to manage online meetings. Users can
schedule or join meetings directly from the portlet, with a few mouse clicks.
Finally, WebSphere Portal Server can provide external search capabilities in
DB2, Lotus Notes®, and Lotus Domino® databases, HTML and text documents,
and external Web search engines.

Note: More information about DB2 can be found at:

http://www.ibm.com/software/data/db2/

Note: More information about Lotus Notes, Lotus Domino, and Lotus
Workspace can be found at:
http://www.lotus.com/

34 Grid Computing Products and Services

What does a Web portal bring to a grid?
In the same way, a Web portal is an entry point to access and use information, or,
to existing systems, a portal can be an entry point to a grid, as soon as
interaction is required between the user of the grid and the infrastructure itself. A
given user can submit a job to the grid from its Web browser through the portal,
or schedule a time and book resources to run it. The output of this job can be
returned by the grid to a PDA, or a cell phone. The grid itself can notify a user
about the execution of a job by e-mail, by a SMS message, or by a message on
voice mail. These proactive features are brought to the grid by the portal, in
addition to more classical notifications like a Web page.

WebSphere Portal Server can implement high level services to track specific
actions taken by the users. This may be particularly useful to prevent abusive
usage of the grid, like repetitive high priority jobs whose results are not retrieved
immediately. It is vital to know how relevant a job submitted to the grid is, so that
more important jobs can have precedence if need be. As a result, grid schedulers
and workload managers implement sophisticated policies to assign job priorities
and book resources in advance, and dynamically adapt their policies. This is an
example of how sophisticated but typical portal characteristics can improve the
user experience and the grid efficiency.

In some cases, portals are necessary rather than just desirable for a grid. For
example, in a IBM Grid Toolbox or Globus Toolkit grid, all grid nodes must have
the software installed to be registered to the grid and submit jobs. A way to plug
these machines into a IBM Grid Toolbox or Globus Toolkit grid is to build
“submission clients” that can run on the desktops and connect them to the grid.
Another more scalable and easy to administer solution would be to create a Web
portal that connects the user with the grid. It would be necessary to build a portlet
that allows you to submit jobs, to schedule their execution and inquire about their
status, to book CPU time in advance, and so on.

All the features described above are typical qualities of the Web Services Core
and Hosting ecosystem from the user interaction standpoint. WebSphere Portal
Server, IBM Grid Toolbox, and the WebSphere Application Server family provide
all the tools necessary to implement them and integrate people, besides
applications, with the grid.

Chapter 2. Web services core/hosting 35

2.3.4 WebSphere MQ Family
The WebSphere MQ family includes many products, offering a range of
capabilities. It allows you to exchange information across different platforms,
integrating new and existing business applications.

Note: WebSphere MQ Family is part the WebSphere software solutions. More

information can be found at:
http://www.ibm.com/software/integration/wmq/

WebSphere MQ Messaging
WebSphere MQ Messaging (formerly MQSeries®) provides any-to-any
connectivity from desktop to mainframe, through business quality messaging,
supporting several platforms. MQ supplies a reliable messaging system and APIs
for several languages on the supported platforms. By using MQ, an application
can interact with another, whatever protocols and operating systems are used by
either one. The developer only needs to create the interface to implement the
syntax and semantic of the message. MQ provides the connectivity, and the
application brings the logic to create and interpret messages.

What do WebSphere MQ products bring to grid?

WebSphere MQ products are components that are part of a traditional IT
environment, as soon as applications need reliable connectivity. Easy to
implement and with a short learning curve across different platforms, they are a
good option to be considered when the need arises to simplify asynchronous
communications between a number of heterogeneous applications using a
messaging system.

On the other hand, MQ has Web services connectors that can use SOAP and
XML to receive and transfer messages. Moreover, if an application that exposes
MQ interface to the rest of the infrastructure needs to connect to a grid, it may be
relevant to use MQ capabilities as an entry point to the grid, for example, by
implementing a grid gateway that sits waiting for MQ messages on one side and
sends grid messages on the other side. This would bring a substantial benefit to
the architecture by making it unnecessary to modify the interfaces of the
applications that submit jobs to the grid, while guaranteeing the reliability of the
information transmitted as messages.

In a different context, a grid can have several entry points: standard Web
services, a portal, and ad-hoc connectors, which may be implemented with MQ.
For example, some platforms do not support Web services, or applications can
be developed in a programming language that was not designed to easily create
SOAP content. In this case, it is still possible to leverage the applications and to
protect existing investments while enabling the building of a grid: instead of

36 Grid Computing Products and Services

 creating several new pieces of code, one for each application that could submit a
job to the grid, MQ and its connectors can be used to implement a gateway
between the applications and the grid that accepts messages on the applications
side and passes them to the grid.

2.3.5 WebSphere Messaging Resources

WebSphere Application Server V6 includes new and additional capabilities
related to messaging that can be used as a Java Messaging Service (JMS)
provider by J2EE applications running on the application server. WebSphere
Application Server V6 still supports other JMS providers, such as WebSphere
MQ. Messaging capabilities, using standard interfaces such as those provided by
JMS, enables an Enterprise Service Bus (ESB), to aid in integration among
requesters and providers within a virtual organization. Requesters and providers
are usually applications and systems that need to communicate to provide new
services to the end users. Integration between requesters and providers usually
requires “n-to-n” connections and messaging management. It implies building
several interfaces. An ESB allows the creation of a single interface for every
provider and requester to access the ESB itself. Then the ESB translates the
message to the appropriate protocol for the requester or the provider that the
message was sent to. In that way, the ESB reduces the number of interfaces,
eases the administration and maintenance of the interfaces, and allows new
requesters and providers to be plugged in with less effort.

These capabilities, in conjunction with WebSphere MQ, provide an ESB ready to

use by J2EE applications running on WebSphere Application Server. Several
connectors are available, such as HTML, HTTP, SOAP, Web Services, JMS,
CICS®, SAP-BAPI, XLM, RMI, SQL, and so on, while WebSphere MQ provides
the support for the JMS internal mechanism. These messaging resources can be
accessed as Java beans.

The grid environment can access the ESB's services instead of accessing
individual service providers with ad-hoc mechanisms. In fact, if an ESB already
exists in an organization, and the services to be provided by the grid are
accessible through the ESB, then the grid should access these services through
the ESB.

Note: WebSphere Messaging Resources (WAS-MR) is a feature of

WebSphere Application Server V6. More information can be found at:
http://www.ibm.com/developerworks/websphere/techjournal/0501_reinitz/0501
_reinitz.html

Chapter 2. Web services core/hosting 37

 Table 2-1 on page 39 shows a comparison overview between WebSphere
Business Integration Messaging Broker (WBI-MQ) and WebSphere Messaging
Resources (WAS-MR).

2.3.6 WebSphere Business Integration Messaging Broker

WebSphere Business Integration Messaging Broker (WBI-MQ), formerly MQ
Integrator, is a hub of integration that consolidates and integrates the interfaces
of messaging among several systems. It helps to build an Enterprise Service Bus
(ESB). It is fully based on WebSphere MQ (WAS-MQ) and exports its services to
applications through WAS-MQ connections and Java APIs. WAS-MQ only
provides the connection or the link between requesters and providers of the
services, while WBI-MQ adds powerful logic to transform the messages. It is
common in a corporate environment that different applications have to
communicate with others and several interfaces have to be developed. The
number of interfaces is usually “n-to-n”, where n is the number of systems. Even
with WAS-MQ, the applications have to implement all the messaging
transformations in a “n-to-n” way, because WAS-MQ only gives the link between
them. The messaging transformation is still the responsibility of the applications.
On the other hand, WBI-MQ implements a hub in such way that the applications
to be integrated only need to create one interface, to the hub, that works as an
ESB. WBI-MQ provides tools to create the transformation rules and a repository
to keep them alive. Then, messages are transformed on-the-fly.

WBI-MQ can be used in a grid environment as the ESB of a corporate grid. The
grid environment can access ESB's services instead of accessing the service
providers with ad-hoc mechanisms. In fact, if an ESB already exists in an
organization, and the services to be provided by the grid are accessible through
the ESB, then the grid should access these services through the ESB. A grid can
also be a service provider of the ESB of an organization.

Note: WebSphere Business Integration Messaging Broker is a hub of

integration that consolidates and integrates the interfaces of messaging
among several systems. More information can be found at:
http://www.ibm.com/software/integration/wbimessagebroker/

Comparing WBI-MQ and WAS-MR

Table 2-1 on page 39 shows a comparison overview between WebSphere
Business Integration Messaging Broker (WBI-MQ) and WebSphere Messaging
Resources (WAS-MR).

38 Grid Computing Products and Services

 Table 2-1 Comparing WBI-MQ and WAS-MR
WebSphere Business Integration WebSphere Messaging Resources
Messaging Broker

A whole product. A feature of WebSphere Application

Server V6.

Implements a middleware outside the Implements a JMS mechanism to the

application server, but accessible by the application server.
application server and other applications.

Uses WAS-MQ internally and externally. Uses WAS-MQ internally and supports
MQ externally wrapped in JMS messages.

Publishes services through WAS-MQ Publishes services as JMS services and

queues. These services can be wrapped Java beans.
in Java classes.

Provides basic connectors out of the box Provides several connectors out of the
while others have to be added. box.

Most of the connections with providers Usually connections with providers can be
should be treated as MQ messages. treated in the original protocols.

High availability is implemented with its High availability is implemented on the

own tools. application server.

Messaging transformation is provided by Messaging transformation is provided by

rules by using the tools provided. These code, by using provided templates.
rules are stored in an internal database
that acts like a persistent repository.

2.4 Conclusion
In this chapter, we briefly described the standards and products associated with
the building of basic grid computing infrastructures. Some basic products for the
Web Services Core/Hosting discipline are part of the WebSphere family. Most of
their capabilities are well known from the traditional IT perceptive and can be
integrated with a variety of other products (for example, IBM Grid Toolbox
includes WebSphere Application Server Express Edition).

Chapter 2. Web services core/hosting 39

40 Grid Computing Products and Services
 3

Chapter 3. Workload virtualization

This chapter discusses the following:
򐂰 Concepts, technologies, and features related to workload virtualization
򐂰 Scheduler role on grid environment
򐂰 How to choose the appropriate grid scheduler
򐂰 Products that support grid scheduling

© Copyright IBM Corp. 2005. All rights reserved. 41

3.1 Scheduling
Scheduling is the core of workload virtualization in a grid solution, as it is the
mediator or the middleman that manages the distribution of jobs for workload
balance. This can be performed as simply as taking the next available resource,
but often this task involves prioritizing job queues, managing the load, finding
workarounds when encountering reserved resources, and monitoring progress.

Typically, a job scheduler is a program or application responsible for initiating and

managing jobs automatically by processing job control statements prepared by
the submitter. Some job schedulers also offer a graphical interface to monitor and
analyze a job’s execution.

Some of the typical features of a job scheduler are:

򐂰 Receive a job submission for processing
򐂰 Monitor jobs, checking for completions or failures
򐂰 Advanced features
– Performance Monitoring
– Jobs backfilling
– Jobs status and report
– Jobs priority

Jobs schedulers
Job schedulers are able to submit, control, and monitor the workload of jobs
submitted in a network of computers. The jobs submitted are ran based on
priorities and scheduling algorithms that are typically defined by the submitter.
Some of the most popular job schedulers are IBM LoadLeveler®, Platform LSF,
Portable Batch System (PBS), and Condor.

Figure 3-1 on page 43 points out the focus of the chapter and where job
schedulers are typically used.

42 Grid Computing Products and Services

 Security

User

Meta CSF
Job Submit
Scheduler IBM Loadleveler
Platform's LSF
OpenPBS
etc
Scheduler Scheduler

Application

Data access

Figure 3-1 Focus of scheduling environment

3.2 Grid scheduling and policies

There might be different levels of schedulers within a grid environment. For
example, a cluster could be represented as a single resource and it may have its
own scheduler to manage its nodes. A higher level scheduler (sometimes called
a meta-scheduler) might be used to schedule work to be accomplished on
multiple clusters, while each cluster's scheduler handles the actual scheduling of
work on its individual nodes. The main goal is to optimize the allocation of jobs to
resources and achieve better performance.

Here is a snapshot of some different types of schedulers:

򐂰 Gang scheduler
The gang scheduler permits a set of processes, or multiple threads from a
single process, to be scheduled concurrently as a group.

Chapter 3. Workload virtualization 43

 򐂰 Grid engine
A scheduling system that accepts job submissions and schedules them for
execution on the appropriate nodes of a grid based on resource management
policies.
򐂰 Meta-scheduler
It provides an interface and adapters to the lower level schedulers. May
perform load balancing of workloads across multiple systems. Each system
would then have its own local scheduler to determine how its job queue is
processed.

3.3 How to choose the appropriate scheduler

For choosing the most suitable scheduler for a particular application, one should
fully understand the nature of such an application.
򐂰 Is the application CPU bound? Can it be split into smaller units?
򐂰 Can it run on multiple platforms?
򐂰 Can the computations be split and carried out in any order?
򐂰 Does it mostly read data rather than write it and in which quantities?
򐂰 Is this a one-time execution or an application that is run over and over?
򐂰 Are portions of the results reusable in another runs of the application?
򐂰 Does the application involve large amounts of input data?
򐂰 Does the application involve receiving changing data?
򐂰 Does the application involve intensive communication?

Knowing the inner details of an application, and therefore knowing that this
application can be “gridified”, lead us to choose a best fit for our scheduling
requirements. Such requirements should also take into account operational
issues like the ones addressed in the following questions.
򐂰 What are the user requirements?
User requirements are the most critical factors here; some users jobs may ran
for days or maybe weeks before they complete. Users might want to have
some checkpoints features in the scheduler so that it can resume without
restarting the whole job. Status reporting and monitoring might be the area in
which the user wants to focus on.
Some additional user requirements relies on workload scheduling activities,
priority algorithms and policies, automatic rescheduling, load balancing,
performance, and capacity.

44 Grid Computing Products and Services

 򐂰 What are the hardware and operating systems requirements?
While this is not the most important question in defining the appropriate
scheduler, we do know that some schedulers do not support multiple
platforms.
򐂰 How do the users submit their jobs?
This question gives an insight into what kind of interfaces the schedulers
support. Portals are very common and popular today. Most schedulers
support portals access for submitting jobs.
򐂰 What is the data access used for the job run?
Most often when we submit a job, some amount of data needs to be
transferred to the grid. Eventually, users use a mechanism such as “GridFTP”
to transfer such data. In many cases, where data needs to be shared and
accessed by multiple nodes, it is necessary to have some mechanism for
managing multi-party access to data chunks.

3.4 Mapping user requirements into products

In most cases, we need to directly map the users requirements into the
appropriate product. There are two broad categorization of schedulers: the
server-oriented grid and the desktop-oriented grid. Desktop grids are typically
used in environments where CPU harvesting is deployed, trying to make use of
idle workstations for processing. On the other hand, server-oriented grids are
used in environments where specialized and high-performance machines are
fully (or almost fully) available for the grid. In both cases, schedulers are
responsible for managing the workload across the computing resources for
efficient job execution.

3.4.1 Tivoli Workload Scheduler

While workload management might be under the same umbrella as scheduling, it
has some particularities that should be clearly noted. Workload management
typically provides more functionality for managing the job that is being submitted,
allowing it to be easily moved from one node to another. One such product is IBM
Tivoli Workload Scheduler. It is calendar based, and has a choreographer
scheduler that automates, monitors, and controls the flow of work through an
enterprise's entire IT infrastructure on both local and remote systems. From a
single point of control, the suite analyzes the status of the production work and
drives the processing of the workload according to installation business policies.
It supports a multiple end-user environment, enabling distributed processing and
control across sites and departments within an enterprise.

Chapter 3. Workload virtualization 45

 Note: Tivoli Workload Scheduler is part of the IBM Tivoli family of products.
For more information, refer to the following Web page:
http://www.ibm.com/software/sw-bycategory/tivoli/

Tivoli Workload Scheduler for Virtualized Data Centers V8.2 extends the
capabilities of IBM Tivoli Workload Scheduler to workload optimization and grid
computing. It provides an enterprise-level scheduling solution that helps you
perform cross-platform, cross-domain, and cross-enterprise scheduling by
integrating business applications across grid, mainframe, and traditional
distributed environments. It also provides fault tolerance, scalability, and
outstanding workload performance. Workload Scheduler for Virtualized Data
Centers V8.2 provides the following features:
򐂰 Centralized control: All scheduling objects are defined and stored on a master
scheduling controller.
򐂰 Delegation of work: Scheduling duties are sent from the master to other
workstations in the network.
򐂰 Fault tolerance: In the event of a loss of network connectivity with the master,
other workstations continue to run jobs.
򐂰 Remote administration: Manipulation of workload can be done remotely using
the Job Scheduling Console.

Automatic driver
Workload Scheduler provides leading-edge solutions to problems in a production
workload management. It can automate, plan, and control the processing of an
enterprise’s entire production workload. It functions as an “automatic driver” for
the production workload, maximizing the throughput of work and optimizing the
resources usage. When Workload Scheduler interfaces with other system
management products, it actually takes part in a more comprehensive integrated
automation and systems management platform.

IBM Tivoli Workload Scheduler can manage the complete flow of work through
an enterprise’s entire operation, on both local and remote systems.

Single point of control

From a single point of control, Workload Scheduler analyzes the status of the
production work and drives the processing of the workload according to
installation business policies. It supports a multiple end-user environment,
enabling distributed processing and control across sites and departments within
an enterprise.

46 Grid Computing Products and Services

3.4.2 LoadLeveler
LoadLeveler is another popular scheduling software, typically deployed on IBM
^ pSeries hardware. It is also bundled with the Cluster 1600 solutions to
provide job management. It has tight integration with other software from IBM,
such as Engineering and Scientific Subroutine Library (ESSL), Parallel ESSL
(PESSL), and Parallel Environment (PE).

Note: LoadLeveler, from IBM, is a distributed network-wide job management

facility designed to dynamically schedule work on IBM UNIX servers, such as
the IBM ^ pSeries and IBM RS/6000® systems. For more information,
refer to the following Web site:
http://www.ibm.com/servers/eserver/clusters/software/

Important: Linux machines and AIX 5L™ machines can now exist in the same
LoadLeveler cluster. For more information, refer to the following Web site:
http://publib.boulder.ibm.com/infocenter/clresctr/index.jsp?topic=/com.ib
m.cluster.loadl.doc/doc_updates/ll3_2.1update.html

LoadLeveler allows users to optimize workload execution and performance on a

pool of pServer machines by matching the jobs’ processing needs with the
available resources. LoadLeveler schedules jobs both on clusters and on a High
Performance Computing (HPC) environment, and provides functions for
submitting and processing jobs quickly and efficiently in a dynamic environment.

The scheduler manages both serial and parallel jobs over the grid nodes. In this
particular case, this distributed environment consists of a pool of machines or
servers, often referred to as a LoadLeveler cluster. Machines in the pool may be
of three basic types: desktop workstations available for batch jobs (usually when
not in use by their owner), dedicated servers, and parallel machines.

The allocation of jobs depends on the availability of resources within the cluster
and on a set of rules defined by the LoadLeveler administrator. A user submits a
job using a job command file, and the LoadLeveler scheduler attempts to find
resources to satisfy the requirements of the job.

Here are the LoadLeveler scheduling features:

򐂰 Checkpoint/Restart: Saves the state of a running job so the job can be
suspended and re-started from the same place later.
򐂰 Parallel Job Suspension: Suspends a parallel job in favor of a higher-priority
job, then resumes the suspended job at the same place.
򐂰 Backfill Scheduling: Improves resource utilization by reducing idle time.

Chapter 3. Workload virtualization 47

 򐂰 Runs lower priority jobs in the queue on available resources.
򐂰 Runs the higher priority jobs as soon as the required resources become
available; it schedules jobs to maximize resource utilization and minimize job
completion time.
򐂰 Jobs are scheduled based on resource availability and user-defined rules to
match processing needs with resources.
򐂰 Machines can be of different types: Dedicated servers, parallel machines, and
idle desktops.
򐂰 Both serial and parallel jobs can be scheduled.

Figure 3-2 shows the different components in LoadLeveler.

Submitting Machines
• Submit, query or cancel jobs
• May be outside the LoadLeveler cluster

Scheduler
• Receives jobs submitted by users
• Manages each job to completion & maintains information
• Asks Central Manager to find appropriate machine(s)

Central Manager
• Examines each job’s requirements & determines
most appropriate machine(s) to run the job
• Central Manager is not a single point of failure

Executing Machines
• Run the jobs

Figure 3-2 LoadLeveler components

3.4.3 DataSynapse GridServer

GridServer is a highly scalable software infrastructure that allows application
services to operate in a virtualized fashion, unattached to specific hardware
resources. Client applications submit requests to the grid environment and
GridServer dynamically provisions services to respond to the request. Multiple
client applications can submit multiple requests in parallel and GridServer can
dynamically create multiple service instances to handle requests in parallel on
different grid nodes. This architecture is therefore highly scalable in both speed
and throughput. For example, a single client will see scalable performance gains

48 Grid Computing Products and Services

in the processing of multiple requests, and many applications and users will see
scalable throughput of the aggregate load.

GridServer has four different components in its architecture:.

1. Grid Clients: The components that submit service requests to grid, also
called Drivers.
2. Engines: The processes that host and run services on the grid nodes.
3. Brokers: The component that provides request queuing, scheduling, and
load-balancing. Brokers are also responsible for managing Engines.
4. Directors: The component that assigns Grid Clients and engines to Brokers
based on policy. They manage and load balance Engines across available
Brokers.

Figure 3-3 presents a logical architecture that supports a N-Tier model of

Brokers, Directors, and Engines. High-availability hot failover with checkpointing
of method level workload processing is also available.

Client
Engine
Engine Daemon

Director

GridServer
Broker Engine Instance
Manager

Engine Instance

Figure 3-3 GridServer architecture

Engines and Grid Clients log in to the Director and are authenticated; the
Director then routes Engines and Grid Clients to available Brokers. Grid Clients
submit requests through the Broker. Engines receive work requests from the
Broker, and in most cases, exchange data directly with the Engine processes.
This allows the system to be highly scalable. Since the Brokers manage all work
requests, load balancing is optimal and resilience is built into the system. Brokers
manage Engines and Grid Clients, and schedule work via lightweight messages.

Chapter 3. Workload virtualization 49

 Grid Clients and Engines can exchange work data directly when using Direct
Data Transfer. In addition to providing a programming model and runtime
environment for Grid Services, GridServer also includes a highly manageable
operating environment. There is a Web-based console, the GridServer
Administration Tool, and a SOAP-based administration interface. Both allow you
to deploy Services, manage the workloads running on the Grid, and configure
the GridServer environment.

Note: GridServer is a software infrastructure that allows application services

to operate in a virtualized fashion, unattached to specific hardware resources.
For more information, refer to the following Web site:
http://www.datasynapse.com/

3.4.4 Platform LSF (Load Sharing Facility)

Platform LSF is software for managing and accelerating batch workload
processing for compute- and data-intensive applications. With Platform LSF, an
organization can intelligently schedule and guarantee the completion of batch
workload across a distributed, virtualized IT environment. Platform LSF fully
utilizes all IT resources regardless of operating system, including desktops,
servers, and mainframes, to ensure policy-driven, prioritized service levels for
always-on access to resources. Underpinning Platform LSF is the
production-proven, open, robust Virtual Execution Machine (VEM) architecture
that sets the benchmark for performance and scalability across heterogeneous
environments.

Platform LSF Features are:

򐂰 High performing, open scalable architecture
򐂰 Comprehensive set of intelligent scheduling policies, including:
– Fairshare
– Preemption
– Advance Reservation
– Resource Reservation
– SLA Scheduling
򐂰 Advanced self-management
򐂰 Heterogeneous platform support
򐂰 Extensive application support
򐂰 Comprehensive, extensible, and standards-based security

Note: Platform LSF is part of the Platform LSF family of products. For more
information, refer to the following Web site:
http://www.platform.com/products/LSFfamily/

50 Grid Computing Products and Services

3.4.5 Platform LSF MultiCluster
Platform LSF MultiCluster extends an organization's reach to share virtualized
resources beyond a single Platform LSF cluster to span geographical locations.
With Platform LSF MultiCluster, local ownership and control is maintained,
ensuring priority access to any local cluster while providing global access across
an enterprise grid. Organizations using Platform LSF MultiCluster complete
workload processing faster with increased computing power, enhancing
productivity and speeding time to results.

Note: Platform LSF MultiCluster is part of the Platform LSF family of products.
For more information, refer to the following Web site:
http://www.platform.com/products/LSFfamily/

3.4.6 Platform LSF HPC (High-Performance Computing)

Platform LSF HPC is software for managing and accelerating High Performance
Computing (HPC) mission-critical workloads. With Platform LSF HPC, you can
intelligently schedule parallel and serial workloads, providing you with the
capability of solving large, grand challenge problems while utilizing the available
computing resources at maximum capacity. It enables you to take full advantage
of high performance network interconnects available on clustered systems and
supercomputers.

Platform LSF HPC features topology-based scheduling that enables maximum

application performance for industry leading interconnects. Underpinning
Platform LSF HPC is the robust Virtual Execution Machine (VEM). More than a
distributed application technology, the VEM is a standards based, high
performance, and system-level architecture. The VEM provides a true virtual
environment that is ideal for critical workload management. This workload-centric
approach offers many performance, reliability, and management advantages.
Currently powering a variety of demanding industries and critical applications,
the production-proven VEM is the ideal architecture to handle the ever-increasing
demand for application performance.

Platform HPC Features:

򐂰 Optimized application, system, and hardware performance
򐂰 Enhanced accounting, auditing, and control
򐂰 Advanced self-management
򐂰 Commercial grade system scalability and reliability
򐂰 Extensive hardware support
򐂰 Comprehensive, extensible, and standards-based security

Chapter 3. Workload virtualization 51

 Note: Platform LSF HPC is part of the Platform LSF family of products. For
more information, refer to the following Web site:
http://www.platform.com/products/LSFfamily/

3.4.7 Platform Symphony

Platform Symphony is software that distributes and virtualizes compute-intensive
application services and processes across existing heterogeneous IT resources,
creating a shared, scalable, and fault-tolerant infrastructure, delivering faster and
reliable application performance.

Platform Symphony’ service-level grid infrastructure virtualizes compute

intensive applications, improving their performance, scalability, and reliability.
Platform Symphony dynamically allocates resources to satisfy the needs of
single or multiple applications and their workloads across individual or multiple
lines-of-businesses (LOB).

Platform Symphony Features:

򐂰 Support for the widest range of workloads found in financial services
򐂰 True service-level virtualization for greater application reliability
򐂰 Global, enterprise-class scalability for optimal performance and failover
򐂰 Guaranteed service execution of all application workloads
򐂰 A single common infrastructure (VEM) for rapid migration from the desktop to
the data center
򐂰 The ability to deploy at any IT level from the application to the line-of-business
(LOB) and throughout the enterprise
򐂰 A self-healing architecture lowering administrative costs
򐂰 Comprehensive, extensible, and standards-based security

Note: For more informtion about Platform Symphony, refer to the following
Web site:
http://www.platform.com/Products/Platform.Symphony/

3.4.8 United Devices

United Devices offers a software platform for deploying enterprise grids on
heterogeneous and geographically distributed resources, such as clusters,
servers, and desktops. The Smallpox Analyzer from http://www.grid.org was
developed using United Devices software.

52 Grid Computing Products and Services

 Note: The grid solution from United Devices is part of the Grid MP platform
family of products and services. For more information, refer to the following
Web site:
http://www.ud.com/solutions/

The United Devices Grid MP platform allows organizations to build a computation

grid from a group of different resources. A GUI management console provides
the ability to submit and monitor jobs as well as perform various administration
functions. The product is useful for CPU intensive grid jobs that are self
contained and that use small amounts of data, as well as for management of
distributed resources traditionally controlled by DRMS or other tools. By adjusting
polling times accordingly, the Grid MP platform demonstrates its scalability,
handling a very large number of systems. The architecture is designed to scale
from a few hundred devices to several hundred thousand devices to a million
devices.

The Grid MP platform includes a workload scheduler that balances the

computational needs of application users with the usage preferences of each
device that is participating in the network. The system supports multiple
hardware architectures and operating systems in the various desktops that serve
as resources, while remaining transparent to the user or job submitter. It also
includes a centralized scheduling engine that balances massive computational
demands with a supply of compatible devices available in the grid. Selection of a
job at dispatch time depends on the connecting device's capability, job priority,
device availability, usage preferences, and other constraints associated with the
job. In this way, Grid MP ensures that the job is sent to the devices that is best
suited to run it. When a job is dispatched, its meta-data is also sent. The files
containing actual data are uploaded or downloaded to/from the file service. The
Dispatch Service also automatically reschedules jobs that have failed either due
to device shutdown or job timeouts.

Jobs are automatically rescheduled when resources are overloaded or

disconnected. Job priorities are automatically escalated based on the time since
their last dispatch. In addition, users are able to manually suspend, migrate, or
preempt jobs. Unlike traditional workload management systems that bind jobs to
a specific resource at job submission time, Grid MP transparently schedules jobs
to a variety of heterogeneous resources by binding to them at dispatch time. This
dynamic binding balances workload more efficiently and vastly improves
resource utilization.

Administrators can specify load thresholds to limit usage of device

subcomponents, such as CPU, memory, and disk space. If a device exceeds its
specified threshold, the grid application running on it is suspended until the load
falls below the threshold.

Chapter 3. Workload virtualization 53

 In addition to traditional job scheduling built into the grid platform, United Devices
offers a Unified Job Scheduling (UJS) product that allows organizations to
manage isolated, dedicated Distributed Resource Managers (DRMs) from a
single tool. This product preserves departmental control of resources while
enabling the sharing of resource groups that previously operated in silos, with
peaks and valleys in demand. Unified Job Scheduling provides virtualized
access to all resources: systems, applications, and data. The architecture is
based on open Web services standards that enable the creation of a true
service-oriented architecture (SOA) for access to an enterprise-wide set of
resources.

3.4.9 Altair PBS Professional

Altair PBS Professional is a workload management solution suitable for
deployment in the most demanding production HPC environments. PBS
Professional has the availability, redundancy, and resiliency features required for
uninterrupted services.

PBS Professional is most often deployed across a set of server systems within
an HPC datacenter environment. PBS Professional is a capable solution for large
complex heterogeneous HPC datacenter deployments. PBS Professional can
also be deployed across networks of workstations or personal computers. In
cycle harvesting mode, PBS Professional provides significantly higher hardware
utilization rates by placing batch jobs onto desktop systems when they are not
being utilized for interactive work. PBS Professional is widely deployed
throughout a variety of market segments.

PBS Professional provides three essential benefits:

1. All HPC assets (hardware and software licenses) are federated into a virtual
pool of resources in an effort to maximize utilization of these assets.
2. The entire workload is then intelligently scheduled to run within this virtual
pool based on predefined business policies, which determine how these HPC
assets are to be shared.
3. The scientists and engineers are presented a consistent user interface for
submitting and managing all types of workloads to all of the computing
resources. This allows the users to concentrate on their work, not their
computing environment.

54 Grid Computing Products and Services

 Note: PBS Professional is the commercial version of the original Portable
Batch System (PBS) developed for NASA. PBS Professional is developed by
Altair Grid Technologies (a subsidiary of Altair Engineering). Visit the following
Web sites to learn more about PBS Professional:
http://www.altair.com/
http://www.pbspro.com/

3.4.10 OpenPBS
OpenPBS is a simple workload management solution intended for deployment
on small clusters comprised of dedicated homogeneous server nodes. OpenPBS
is distributed in source code form and users will need to compile the software for
each platform.

The end-user license agreement associated with OpenPBS is not a GPL license
agreement. The OpenPBS license agreement allows any end user to utilize the
software without any costs, but prohibits any entity from redistributing OpenPBS
in conjunction with any commercial product (hardware, software, or service) that
is not free of charge.

Altair does not offer professional software support for OpenPBS. However, there
is an extremely large community of OpenPBS users worldwide and some level of
support is facilitated through this open source community. The
http://www.openpbs.org Web site presents a product comparison between
OpenPBS and PBS Professional.

OpenPBS provides three essential benefits:

1. All HPC computer resources are federated into a virtual pool of resources in
an effort to maximize utilization of these computers.
2. The entire workload is then scheduled to run within this virtual pool based on
simple scheduling algorithms.
3. The scientists and engineers are presented a consistent user interface for
submitting and managing all types of workloads to all of the computing
resources. This allows the users to concentrate on their work, not their
computing environment.

Chapter 3. Workload virtualization 55

 Note: OpenPBS is a freeware distribution based on the original Portable
Batch System (PBS) developed for NASA. OpenPBS is owned and distributed
by Altair Grid Technologies (a subsidiary of Altair Engineering). OpenPBS is
downloadable at no cost to end-users in source code form. Visit the following
Web site to learn more about OpenPBS and to download the software:
http://www.openpbs.org

3.4.11 Condor
Condor, commonly used by universities and research and development
companies, is a freely available, open source, software package that leverages
the use of dedicated or nondedicated resources.

Note: Condor is a research project developed by the University of Wisconsin

Madison. For more information, refer to the following Web site:
http://www.cs.wisc.edu/condor/

The Condor environment is suited to providing large amounts of computing

power over significant periods of time. This has been termed High Throughput
Computing (HTC) in contrast to High Performance Computing (HPC) which aims
to provide peerless computing power over shorter periods through the use of
special dedicated hardware or software infrastructures.

Condor converts collections of distributed workstations and dedicated clusters

into a distributed high-throughput computing facility. Condor matches jobs with
available machines using “Class Ads” for flexible resource matchmaking that
provides resource management in a distributed environment with decentralized
ownership of resources.

Figure 3-4 on page 57 is a very high level representation of how Condor works.

56 Grid Computing Products and Services

 C e n tra l C o n d o r C o lle c to r
M anager C o n d o r N e g o tia to r

S u b m it E xe c u tio n
M a c h in e M a c h in e

C h e c k p o in t
D a ta

Figure 3-4 Simplistic view of Condor scheduling

3.4.12 Community Scheduler Framework (CSF)

The Community Scheduler Framework (CSF) is an open-source implementation
of a number of grid services, which together perform the functions of a grid
metascheduler or community scheduler. CSF provides basic capabilities for
scheduling and can be used as a development toolkit for implementing
community schedulers.

Note: For more informtion about CSF, please refer to the following Web sites:
http://www.platform.com/Products/Platform.Globus.Toolkit/Product.Informat
ion/Features.Benefits.htm
http://sourceforge.net/projects/gcsf/

Platform Computing has contributed the CSF open source metascheduler to the
Globus Toolkit. CSF classes can be extended to provide more domain specific
community schedulers and support many different kinds of grid deployment
models. Examples of grid level scheduling algorithms include scheduling across
multiple clusters within a virtual organization, co-scheduling across multiple
resource managers, scheduling based on SLAs, and economic scheduling
models. By making use of the open source CSF, grid scheduling implementations
ensure that they interact with resource managers using standard interfaces
without needing to know all of the underlying details of the specification or having
to implement the protocol themselves.

Chapter 3. Workload virtualization 57

 The CSF is composed of several features:
򐂰 Round-robin job scheduling
򐂰 Advance reservation booking, query, and control
򐂰 Reservation-based job scheduling
򐂰 Job throttling support for increased reliability
򐂰 File staging support

3.4.13 Maui Scheduler

The Maui Scheduler is a job scheduler that allows the administrator to have
broad control over the different ways that jobs are submitted for scheduling. It
also supports features such as advance reservation, Quality of Service (QoS)
levels, backfill, and allocation management.

Note: Maui Scheduler is an open source software that is used to implement

an advanced reservation HPC parallel batch scheduler. For more information,
please refer to the following Web site:
http://sourceforge.net/projects/mauischeduler

It has an advanced scheduling mechanism, with backfilling, and it allows jobs

with a lower priority to execute prior to a higher priority only if it does not delay
the start of a prioritized job.

Advance features that Maui adds to a scheduler include:

򐂰 Extensive job priority policies
򐂰 Job advance reservation support
򐂰 QoS Support
򐂰 Meta-scheduling Interface
򐂰 Different types of fair share policies
򐂰 Different types of back filling policies

Scheduling versus Advance Reservation

There are many types of scheduling algorithms, and we often need a way to
reserve resources in advance. This is specially true when a single demanding job
needs to be executed and the grid is out of available resources: this calls for
starvation. To solve this problem, some schedulers have implemented a way to
stop and hold smaller jobs so that the starving job can be started. Looking at this
algorithm, it will affect all the jobs in the queue, not just the starving large jobs.

58 Grid Computing Products and Services

 To tackle this same problem, Maui implemented an algorithm called advance
reservation, which is able to allocate resources in the future, just as though you
were looking through a calendar and booking time.

Besides using Maui, starvation can also be resolved using a preemptive job
scheduler, but this is not an easy task, as this implies a rather precise
performance prediction schema.

3.5 Architecture examples

In this section, we present some examples of the use of schedulers. As shown in
the earlier sections, there are quite a few types of schedulers, with different
applicability and functionality.

3.5.1 Desktop grid

A research company “xyz” indicated that the computing power they have in the
lab is insufficient for the massive jobs they run. Their research focuses on DNA
data processing and the evolution of cancer. While there are many software
packages able to handle this computational need, they lack the scalability to grow
to millions of organisms that the researchers are looking forward to analyzing.

To meet these requirements, the proposed solution architecture must be able to

scale well and the data should be split into multiple small pieces for processing
before joining back together for analysis and results.

Current environment
The researchers are currently running their application on a dedicated
multi-processor machine, which is insufficient to achieve effective results on the
jobs. Also, the jobs they are running often “hog” the computing resources in the
lab, which is a shared facility. However, there are a number of student computer
labs that are idle at night.

Proposed grid environment

It is proposed to enable the application on the desktop computers of the
company as well. The application will then use United Devices’ Grid MP product
as the grid middleware to manage, submit, and collect the jobs.

The proposed Desktop grid environment, shown in Figure 3-5 on page 60, uses
the United Devices grid framework for scheduling, monitoring, and jobs
coordination across all the desktops.

Chapter 3. Workload virtualization 59

 Lab Lab

Agent Agent
Agent Agent

Agent Agent
Agent

Job Split

Lab O pen M P
Server United Devices

Agent Agent

Results
Job
Agent

Figure 3-5 Proposed Desktop Grid setup

3.5.2 Server grid

The “xyz” Corporation needs to leverage grid computing technology to maximize
their computing resources on site to better perform research and analysis. They
would like to poll all their servers together and expect to use a scheduler to
performance load balancing and allocation of the available computing resources.

Current situation
In the current environment, the clients have the following infrastructure that they
use as a file server, printer server, workstations, and others.

The current IT environment in this organization is as follows:

򐂰 Workstations running Linux all over the organization.
򐂰 NFS is used as a file sharing backbone.
򐂰 User programs are written in a mixture of C, C++, and FORTRAN, and mostly
compiled under the GNU Compiler Collection (GCC).

60 Grid Computing Products and Services

򐂰 There are future plans to deploy applications such as Fluent, abacus, and so
on.

In the current environment shown in Figure 3-6, the researchers in “xyz”

Corporation need to access the system via telnet or ssh to submit a job. In case
the server is busy or high in workload, the users log into another server. Once a
suitable server is identified, the user will run the job with the corresponding data
files available through the NFS.

50
.........

Manual Query
Job Submission

Telnet/SSH

Researchers
Figure 3-6 Current environment

Pain of the current infrastructure

At a glance, we can identify that there are some areas of improvement in the
current infrastructure. The researchers are spending much of their time
identifying a suitable machine for their job, instead of focusing on their core
competencies.
򐂰 Researchers are spending precious time finding a machine to execute a job.
򐂰 Security concerns, with so many users logging in all the servers.
򐂰 Management control, with the growing number of workstations.
򐂰 Users with urgent jobs do not get higher priority over the resources.

Chapter 3. Workload virtualization 61

 Proposed solution
The “xyz” Corporation’s IT team plans to use a grid-enabled scheduler to create
a virtualized compute environment for the researchers to use to submit jobs so
that they see the resources as a single pool.

Components of grid
The grid consists of a number of different components that are suitable for a wide
range of specific needs, but in this case, we first propose the addition of
scheduling software, possibly OpenPBS.

In selecting a suitable scheduler for “xyz” Corporation, we need to evaluate some

basic requirements for the solution. As we know, the basic feature of a scheduler
is to be the “traffic” controller of all jobs.

The basic features that this scheduler must provide are:

򐂰 Queue a job that a user submits
򐂰 Evaluate the job requirements
򐂰 Select the appropriate host to run the jobs
򐂰 Dispatch the job to a suitable machine to run
򐂰 Monitor and inform the user when the job is successfully completed

Also, by placing all the various machines under the control of the scheduler, it
automates the task of job dispatching, saving time and optimizing the use of
hardware.

Administration
By placing the machines under the administration of the scheduler, the selection
of servers/workstations to perform a certain task is now left to the scheduler.
Users will also not be required to have access to all the servers to run the jobs.
The scheduler will be running the job on the user’s behalf.

Hardware and software

As part of the solution, we propose an additional server be installed with the job
scheduler software. OpenPBS is chosen, as it meets the “xyz” Corporation
requirements and it also provides additional features, such as multiple queue
support. This will enable the “xyz” Corporation to manage the jobs more
efficiently, breaking their priority jobs from the lesser.

Also, OpenPBS has the capability for file staging, providing users with the ability
to specify any files that need to be copied onto the execution host before the job
runs, and any that need to be moved out after the job completes. The job will be
scheduled to run only after the required files have been successfully transferred.

62 Grid Computing Products and Services

Advantages
Based on the proposed solution for “xyz” Corporation, as shown in Figure 3-7,
they could now harness and use all their computing power and resources:
򐂰 Researchers could now focus on their research and not look for servers to
submit their job.
򐂰 Job scheduler will automatically run the jobs based the priority assigned.
򐂰 All the computing resources in the organization could be harnessed for the
computing power.
򐂰 The solution is scalable, and additional nodes can be easily added to the
schedulers.
򐂰 Better administration work, with less user access (such as telnet/ssh) into
the systems.

Open PBS
Server
Job Scheduler

Researchers/SSH

Figure 3-7 Proposed Architecture for “xyz” Corporation

Tip: Altair PBS Professional, a workload management solution suitable for

deployment in the most demanding production HPC environments, is also a
good option to be used in this example.

Chapter 3. Workload virtualization 63

 Future roadmap
With the current solution, “xyz” Corporation has effectively created virtualized
computing power. This is a major step toward the creation of a virtualized
environment. Some of the possible extensions to the current solution are as
follows:
򐂰 Data Federation
򐂰 Web Portal for submitting jobs

A Web Portal will give the users the capability to submit jobs and manage the
environment via Web services. Using this service, the users effectively have an
interface to select the type of jobs, the platform, and the priority of the jobs. For
more information about these products, see Chapter 2, “Web services
core/hosting” on page 23.

64 Grid Computing Products and Services

 4

Chapter 4. Information virtualization

The following topics are discussed in this chapter:
򐂰 Types of information infrastructure
򐂰 Overview of products available for this discipline
򐂰 Mapping user requirements to the type of information infrastructure

© Copyright IBM Corp. 2005. All rights reserved. 65

4.1 Introduction
Information virtualization is the process by which data and information stored in
distributed and unrelated devices is managed as a single and concise set of
data. As such, virtualization can naturally be accomplished by data grids, as the
distributed and heterogeneous nature of such grids make them suitable for
unifying decentralized resources based on distinct technologies to provide the
final user with a single vision of the data.

The virtualization process can take place in different levels of abstractions and,
as such, data is treated in this context as a multiple-layer concept. The next
section describes the data layers that we consider in this chapter and presents
basic products that provide data virtualization functionality.

4.2 Information virtualization layers

Figure 4-1 presents the three basic levels of virtualization typical found in an
information infrastructure. The next sections briefly describe each one of them.

Information
Information
Resources

File
File System
System

Block
Block Data
Data

Figure 4-1 Virtualization layers

Attention: We use the term virtualization here in the sense of aggregating

distinct entities to become one virtual entity. Other subdivision of layers can be
defined depending on the architectural blueprint been used.

66 Grid Computing Products and Services

4.2.1 Block data virtualization layer
Block data infrastructure for grids provides unified access to chunks or blocks of
data. As presented in Figure 4-2, this layer provides an abstraction over how raw
data is stored on independent storage devices, offering operating systems with a
unified vision of such data.

The main advantages of such independence between computers and storage

devices are that it:
򐂰 Reduces the complexity and costs of managing storage
򐂰 Increases the utilization of existing capacity
򐂰 Centralizes the management of multiple storage controllers

The layer between computers and storage devices hide the complexity of storage
and helps to increase the utilization. It also helps to reduce the costs of
managing by centralizing the management of the multiple storage devices or
subsystems.

Block Data Virtualization

Storage Devices

Figure 4-2 Block data virtualization layer

4.2.2 File system virtualization layer

The file system infrastructure follows the main principle of the data block
infrastructure but at a higher level, providing unified access to files, as presented
in Figure 4-3 on page 68.

A number of solutions have been created to handle the problem of remote data
access in homogeneous and heterogeneous environments. Network file
systems, for example, are a convenient solution for sharing data across computer
nodes and a typical example is the NFS protocol. Such file systems make
possible the access to remote files in a networked environment as though they
were locally available.

Chapter 4. Information virtualization 67

 File System Virtualization

File Systems

Figure 4-3 File layer virtualization

4.2.3 Information virtualization layer

Information is usually defined as “meaningful data”, from the perspective of the
end user. This layer provides an abstraction over disparate and distributed
information sources, such as a Database Management System (DBMS), flat files
(for example, comma-separated files), structured files (for example, XML
documents), or a Content Management System.

The information virtualization layer also refers to the ability to federate or

integrate data and information from heterogeneous resources into a unified
repository. The whole idea is to present a single view of the information (see
Figure 4-4).

Information Virtualization

Structured Content
DBMS Flat File Management
File
System

Figure 4-4 Information virtualization layer

Some parallel databases are usually regarded as information virtualization tools,

but in general, these products only support their own branded engines, and as
such cannot be considered grid-enabled products.

68 Grid Computing Products and Services

 Attention: it is natural to think that information virtualization software provides
virtualization in the two lower levels, as well as that file system virtualization
software provides data-block virtualization. However, this is not always true, as
we can see in most distributed database implementations.

4.3 Requirements and products

It is one of the IT architect’s role to choose the right products according to the
solution’s requirements. In this section, we discuss how to depict the main
requirements that a solution may require and which products are available for
providing such requirements.

4.3.1 Defining requirements

In the context presented above, answering the following questions is imperative:
Where is the data stored?
Data to be accessed is stored on a single remote system
or distributed across multiple networked systems.
How heterogeneous is the data source?
The data to be accessed is stored on different types of
source systems (for example, DBMS, Content
Management Systems, and flat files). It also
encompasses data coming from sources on different
hardware bases and from the same "kind of" source, but
provided by distinct products, and possibly from different
vendors.
What are the data formats?
Data types and formats used by the source systems differ
in format and type.
What are the data semantics?
Data that is owned by different domains (for example,
business units) might differ in semantics (for example,
homonym for entity customer).
How is the network performance?
Bandwidth with a given latency.
How are the security policies?
Ownership of data and procedures for authentication and
authorization differs among source systems.

Chapter 4. Information virtualization 69

 Once a better understanding of the problem is acquired, the architect should
define the layer of abstraction in which the solution is best fit. This is the first step
when choosing the products to adopt in a particular implementation, as
information, file system and data block products are usually part of unrelated
families of products. In some cases, multi-layer solutions might be required.
Some examples of solutions for each layer are:
򐂰 Information virtualization layer
Distinct servers could be the repository of data, for example databases,
images, videos, or audio streams. Using IBM WebSphere Information
Integration framework, we could federate the different data into a single
database. A federated database creates references or links to the actual data.
End users and applications will have a single view of all the different type of
data.
򐂰 File system virtualization layer
A solution that has servers that see several storage devices as though they
were one device. The servers could share out some of its folder to another
department using a protocol such as NFS.
򐂰 Block data virtualization layer
A solution that has servers that see several storage devices as though they
were one device. The servers are connected to storage managed by SAN
Volume Controller, which manages file systems on multiple SAN storages.

The examples described above are not mutually exclusive and can be combined
if the solution requires it.

Products overview
Figure 4-5 provides an overview of some Information virtualization products in
each layer.

IBM® WebSphere® Information Integration

Information
DB2 Universal Database
GPFS

File System NFS V4

SAN File System

SAN Volume Controller

Block data
Tivoli Storage Resource Manager

Figure 4-5 Example for a product overview

At the File System Layer GPFS, NFS Version 4, SAN File System, Network
Attached Storage (NAS), and Andrew File System (AFS®) provide an abstraction

70 Grid Computing Products and Services

 for the underlying files system. It shields user and application from the complexity
that is associated with accessing remote data that might be stored in disparate
file systems.

At the block data layer, Tivoli Storage Resource Manager and IBM SAN Volume
Controller provide an abstraction of storage devices. It permits a level of
indirection between storage devices, as seen by computers and storage devices
exported by storage systems.

4.3.2 Block data layer products

At the block data layer, as shown in Figure 4-6, Tivoli Storage Resource Manager
and IBM SAN Volume Controller TotalStorage® provide an abstraction of storage
devices.

Resources

Block
Block Data
Data Layer
Layer
Figure 4-6 Block data layer

Storage Area Network

SANs originated to overcome the problems with network attached storage (NAS)
devices, which - like ordinary servers - are difficult to manage and difficult to
expand in capacity. SAN is made up of a number of fabric switches connected in
a network. The most common form of SAN uses the Fibre Channel fabric
protocol (with Fibre Channel switches). Connected to the SAN will be one or
more disk array controllers and one or more servers. The SAN allows the storage
space on the hard disks in the disk array controllers to be shared among the
servers.

Clients can use SAN capabilities to reduce the complexity that is associated with
managing large amounts of (probably) heterogeneous storage devices.
Furthermore, it provides them with greater flexibility in allocating storage space.

Several products are available, with different virtualization capabilities.

Chapter 4. Information virtualization 71

 SAN Volume Controller
The SAN Volume Controller (“TotalStorage SAN Volume Controller” on page 193)
(SAN VC) is a storage solution that enables SAN attached storage to be
managed as one large pool rather than many individual islands of capacity, thus
allowing clients to increase utilization of these storage resources.

Note: SAN Volume Controller is part of the IBM TotalStorage Open Software
Family. The TotalStorage Open Software family is a set of comprehensive and
flexible storage software solutions that can help enterprises address their
storage management challenges today and is designed to help enterprises
take steps toward evolving to an On Demand storage environment. For more
information, please refer to the following Web site:
http://www.ibm.com/servers/storage/software/virtualization/svc/

With SAN VC, users can allocate their storage to applications based on
application requirements instead of constraints in subsystem connectivity and
functional capabilities. Besides this function, it also provides a centralized
storage management capability.

This central point of control will increase storage administrator productivity by

more flexible disk management, for example, dynamic creation and expansion of
logical volumes. Also, the SAN VC provides a common platform for advanced
copy services, such as FlashCopy® and Remote Mirroring that allow source and
target volumes to be located on heterogeneous hardware platforms. With
powerful data migration facilities as part of the base virtualization software, SAN
Volume Controller can be added into existing SAN environments without
requiring data movement from current physical locations. Once in place,
transparent migration of data is supported (that is, data typically can be moved
without interrupting host applications).

The SAN Volume Controller is a storage solution integrating specialized

virtualization software with clustered hardware engines. The solution uses
customized IBM ^ xSeries® servers running an operating environment
based on a Linux kernel. Industry standard host bus adapters (HBAs) will
interface with the SAN fabric. Because the SAN VC is designed to work in an
open environment with heterogeneous disk systems, clients will be able to deploy
flexible storage solutions without concern for proprietary architectures. The
flexible nature of the SAN Volume Controller enables clients to grow in
performance and capacity according to their needs.

IBM Tivoli Storage Manager

IBM Tivoli Storage Manager protects an organization's data against hardware
failures and other errors by storing backup and archive copies of data in offline
storage.

72 Grid Computing Products and Services

 Note: IBM Tivoli Storage Manager is part of the IBM TotalStorage Open
Software Family. The TotalStorage Open Software family is a set of
comprehensive and flexible storage software solutions that can help
enterprises address their storage management challenges today and is
designed to help enterprises take steps toward evolving to an On Demand
storage environment. For more information, please refer to the following Web
site:
http://www.ibm.com/software/tivoli/products/storage-mgr/

It can scale to protect hundreds of computers, from laptops to mainframes,

running a dozen different operating systems, connected via the Internet, WANs,
or LANs. The centralized Web-based management, smart data move and store
techniques, and comprehensive policy-based automation work together to
minimize data protection administration costs and the impact on both computers
and networks. Optional modules enable business-critical applications that must
run 24x7x365 to utilize IBM Tivoli Storage Manager centralized data protection
with no interruption to their service. The features of IBM Tivoli Storage Manager
are as followed:
򐂰 Backup
Provides the ability to back up numerous versions of files to the hierarchy of
storage. In case of a failure or someone accidentally deletes a file, the chosen
version of the data can be easily restored. It supports also progressive
backup.
򐂰 Tape resource sharing
Enables multiple Tivoli Storage Manager servers to use the same tape library
and drives. This improves tape hardware asset utilization, recovery
performance, and tape hardware asset utilization.
򐂰 Network-free rapid recovery
Supports high-speed client data recovery directly from a tape or CD-ROM.
This minimizes recovery time by eliminating the use of network and central
services resources.
򐂰 LAN-free data transfer
Effectively exploits SAN environments by moving back-end office and IT data
transfers from the communication network to a dedicated data network or
SAN. Internet Protocol (IP) communication bandwidth can then be used to
improve service levels for end users and clients.

Chapter 4. Information virtualization 73

 򐂰 Dynamic multi threaded transfer
Permits multiple data transfers, to and from the same Tivoli Storage Manager
server. This new feature boosts performance backups to more than three
times faster than the rate of a single-threaded session. This speed is achieved
because the number of IBM Tivoli Storage Manager data transfer sessions is
transparently optimized based on available system resources.
򐂰 Adaptive differencing technology
Changes the way data is transferred throughout the enterprise. Data is
transferred by byte, block, or file level, based on data size and network
characteristics. This newly patented technology supports a variety of
connectivity strategies, including LANs, WANs, SANs, Internet, and dial-up
connections. Adaptive Differencing technology is designed for mobile
computer users and other users with a need to minimize data transmitted over
the network.
򐂰 Enterprise administration
Simplifies centralized control across multiple Tivoli Storage Manager
implementations without sacrificing network performance. This enables
high-performance backups to locally attached storage devices using a
minimum of network resources.

IBM Tivoli Storage Resource Manager

IBM Tivoli Storage Resource Manager is a comprehensive capacity
management solution for heterogeneous storage environments. It includes
enterprise wide reporting and monitoring, policy based management, and
automated capacity provisioning for Directed Attached Storage (DAS), Network
Attached Storage (NAS), and Storage Area Network (SAN) environments.
.

Note: IBM Tivoli Storage Resource Manager is part of the IBM TotalStorage
Open Software Family. The TotalStorage Open Software family is a set of
comprehensive and flexible storage software solutions that can help
enterprises address their storage management challenges today and is
designed to help enterprises take steps toward evolving to an On Demand
storage environment. For more information, please refer to the following Web
site:
http://www.ibm.com/software/tivoli/products/storage-resource-mgr/

74 Grid Computing Products and Services

 And because it is policy-based, through its autonomic self-healing capabilities, it
can detect potential problems and automatically make adjustments based on the
policies and actions you have established. Tivoli Storage Resource Manager can
help you:
򐂰 Maximize storage return on investment
With Tivoli Storage Resource Manager, you can leverage and optimize your
existing storage resources and perform storage management with a high level
of control.
򐂰 Maximize storage utilization
Tivoli Storage Resource Manager lets you find unused space and reduce
wasted space by identifying outdated or nonessential data.
򐂰 Effectively manage more storage with the same staff
Because Tivoli Storage Resource Manager automatically collects and reports
on storage assets according to the policies and thresholds you set, you spend
less time diagnosing and fixing storage problems. Automatically taking
corrective action based on policies can help administrators sustain the health
of your storage infrastructure to maintain application availability through
autonomic self-healing behavior

4.3.3 Files system layer products

At the file system layer, as shown in Figure 4-7, GPFS, NFS Version 4, and SAN
File System provide an abstraction of the underlying files system. It shields users
and applications from the complexity that is associated with accessing remote
data that might be stored in distinct file systems.
Resources

File
File System
System Layer
Layer

Figure 4-7 File system layer

Chapter 4. Information virtualization 75

 NFS Version 4
The Network File System Version 4 is a distributed file system based on the
previous releases with a more robust error recovery and a new transport protocol
for heterogeneous environment.

Note: The NFS version 4 protocol is a further revision of the NFS protocol
version 2 (RFC1094) and 3 (RFC1813). For more information, please refer to
the following Web sites:
http://www.ietf.org/rfc/rfc3530.txt
http://www.nfsv4.org/

It offers the following features:

򐂰 Strong security and a new protocol for file locking, supporting sophisticated
security mechanisms. These mechanisms include Kerberos 5 and SPKM3, in
addition to traditional AUTH_SYS security. A new API is provided to allow
easy addition of new security mechanisms in the future.
򐂰 File delegation: The NFS Version 4 client can access and modify a file in its
own cache without sending any network requests to the server, until the
server indicates via a callback that another client wishes to access a file. This
reduces the amount of traffic between clients and server considerably in
cases where no other clients wish to access a set of files concurrently.
򐂰 NFS Version 4 introduces protocol support for file migration and replication.

IBM General Parallel File System

IBM General Parallel File System (“GPFS” on page 174) provides global access
to files.

Note: The IBM GPFS, originally developed for SP clusters, can today provide
fast data access from all nodes in a cluster of IBM UNIX servers. For more
information, please refer to the following Web sites:
http://www.ibm.com/servers/eserver/clusters/software/gpfs.html
http://www.ibm.com/servers/eserver/pseries/software/sp/gpfs.html

Files created in GPFS can be accessed from every node that runs GPFS code.
For those nodes not running GPFS, files will still be accessible by using NFS. A
parallel file system not only offers performance advantages by eliminating the
limitation of a single server for file services, but also offers a great deal of
flexibility. With a parallel file system, since all nodes can “see” the file system, it is
easier to move applications from one node to another. GPFS also has the ability
to define data groups that are located under the same “single point of failure”
unit. This allows the data access path to be automatically routed if any of the data

76 Grid Computing Products and Services

groups should fail. The file system is also able to bring itself down to read-only
status to protect data integrity if required. This is especially valid for high
availability solutions, if unrecoverable errors occur in the storage servers.

Sharing the same file system among several nodes has the benefit of increasing
the maximum I/O bandwidth that otherwise would be limited by the maximum
local I/O bandwidth of the single server.

SAN FS
SAN FS (“Tivoli SAN FS” on page 189) provides file system virtualization that
can help reduce complexity by treating storage resources as a single, common
pool. The design of SAN FS is also meant to support a heterogeneous
environment.

Note: SAN FS is part of the IBM TotalStorage Open Software Family. The
TotalStorage Open Software family is a set of comprehensive and flexible
storage software solutions that can help enterprises address their storage
management challenges today and is designed to help enterprises take steps
toward evolving to an on demand storage environment. For more information,
please refer to the following Web site:
http://www.ibm.com/servers/storage/software/virtualization/sfs/

The meta-information about the files, such as file location, security, and the
locking mechanism, are managed by the SAN FS. In many traditional file
systems, the metadata resides within individual servers, which can create
limitations in sharing and accessing data across servers or across file systems.
By managing the metadata on the storage network using a metadata server,
instead of within individual application servers, the design of the SAN File
System helps move intelligence out of individual servers onto the storage
network, so that it can be available to any application server in the network.
Some of the relevant features are:
򐂰 Provide central point for data and storage management, with a single file
system with consistent policies, management, and monitoring capabilities.
򐂰 Storage pools can be designed to represent differentiated classes of service
for different business areas: a critical storage pool can have the data stored in
the most expensive storage while the staging data can be kept in an older
disk.
򐂰 SAN FS allows you to manage physical storage as a pool, with options to
transparently add, remove, and change the application servers.

Chapter 4. Information virtualization 77

 So how does it really work? There is a client code that needs to be running on
the application servers, and interacts with the SAN File System metadata server
to retrieve the metadata. Once the client code receives the metadata, it then
accesses the file data directly through the SAN. No changes to the applications
need to be made.

4.3.4 Information layer products

At the information layer, as shown in Figure 4-8, IBM WebSphere Information
Integration and DB2 UDB provide an abstraction over disparate and distributed
data sources, such as a Database Management System (DBMS), flat files (for
example, comma separated file), structured files (for example, XML documents),
or a Content Management System.

Resources Information
Information Layer
Layer

Figure 4-8 Information layer

WebSphere Information Integration Family

WebSphere Information Integration provides a strategic framework that gives
companies real-time, integrated access to business information: structured and
unstructured, public and private, and mainframe and distributed.

Federation of data in this sense means that end users and applications are able
to access any data, in any format, on any heterogeneous platform. It also allows
end users and applications to query and manipulate different types of data,
organized in a structured database format, all at the same time

In addition to federation capabilities, WebSphere Information Integration family

includes replication, enterprise searching, and publishing capabilities.

In short, the WebSphere Information Integration family provides:

򐂰 Data federation: The ability to provide unified access to different repositories
of information running in a heterogeneous environment. It also allows the
application to connect to other vendor relational databases, document
databases like Domino, XML, flat files, digital media, and archiving
databases, among others.

78 Grid Computing Products and Services

򐂰 Replication: The ability to replicate content from foreign repositories to local
repositories and vice-versa. A solution could require that functionality mostly
in cases where there is WAN communication between the consumers and the
repositories. A local replica of the content would provide better response time
and less usage of the bandwidth.

Attention: Do not confuse WebSphere Information Integration with

WebSphere Information Integrator.
򐂰 WebSphere Information Integration is the family of products and the
solution portfolio.
򐂰 WebSphere Information Integrator is the product and the foundation for the
information integration framework.

The following WebSphere Information Integration products are fundamental

when building an information layer in a grid solution:
򐂰 WebSphere Information Integrator
It provides the foundation for a strategic information integration framework,
enabling the federation and virtualization of information in an organization
(see “WebSphere Information Integrator” on page 199).
򐂰 WebSphere Information Integrator Content Edition
It enhances the ability to virtualize unstructured data, helping applications
access and work with a broad range of unstructured information sources.
򐂰 WebSphere Information Integrator OmniFind™ Edition
It provides enterprise search middleware for powering intranets, extranets,
and corporate public Web sites.

Note: The WebSphere Information Integration family provides a framework

that includes data and content federation, replication, event publishing, and
enterprise search. For more information, please refer to the following Web
site:
http://www.ibm.com/software/data/integration/

DB2 Product Family

DB2 Universal Database™ is the database management system that delivers a
flexible and cost-effective database platform to build robust on demand business
applications. DB2 UDB further leverages your resources with broad support for
open standards and popular development platforms, such as J2EE and
Microsoft® .NET. The DB2 UDB family also includes solutions tailored for specific
needs, such as business intelligence and advanced tooling. Whether your

Chapter 4. Information virtualization 79

 business is large or small, DB2 UDB has a solution built and priced to meet your
unique needs.

Note: For more information about the DB2 Product Family, please refer to:
http://www.ibm.com/software/data/db2/

Database partitioning feature (DPF)

DPF is a feature to enable scaling of a single database to more than one server.
It offers the necessary scalability to distribute a large database over multiple
partitions (logical and physical) using a shared-nothing architecture. DPF can be
beneficial for environments consisting of more than one server.

DPF partition consists of its own data, indexes, configuration files, and
transaction logs. A partitioned database is a database with two or more
partitions. Tables can then be located in one or more database partitions.
Processors associated with each database partition are used to satisfy table
requests. Data retrieval and update requests are decomposed automatically into
sub-requests, and executed in parallel among the applicable database partitions.

Note: A client who has purchased and installed DB2 Universal Database
Enterprise Server Edition (ESE) can create a partitioned instance by using the
database partitioning feature (DPF). More information about DPF can be
found at:
http://www.ibm.com/developerworks/db2/library/techarticle/dm-0504mcarthur
/
http://publib.boulder.ibm.com/infocenter/db2help/index.jsp?topic=/com.ibm
.db2.udb.concepts.doc/concepts/about_dpf.htm

DB2 Content Manager

The IBM DB2 Content Manager (“DB2 Content Manager” on page 173) is an
enterprise content management engine for images, graphics, video, Web
content, and compliance documents. It provides digital asset management and
content integration.

Note: DB2 Content Manager is part of the DB2 Content Management family of
products, which helps manage, share, integrate, and deliver critical business
information on demand. For more information, please refer to the following
Web site:
http://www.ibm.com/software/data/cm/

80 Grid Computing Products and Services

 DB2 Information Integrator for Content
IBM DB2 Information Integrator for Content is an information integration
infrastructure that gives applications access to sources from inside and outside
the enterprise.

Note: DB2 Information Integrator for Content is part of the DB2 family of
products. DB2 Information Integrator for Content provides a strategic
framework to help clients speed new application deployment and control IT
costs. It provides a framework that includes data and content federation,
replication, event publishing, and enterprise search. For more information,
please refer to the following Web site:
http://www.ibm.com/software/data/eip/

From a Web browser or a portal, knowledge workers can concurrently access

data and content from multiple sources, expanding the reach of their analysis
and improving their productivity. DB2 Information Integrator for Content provides:
򐂰 A 'federated search', direct connectors, and simultaneous access across the
multiple repositories.
򐂰 IBM Lotus Extended Search for scalable, distributed, parallel searches -
across Domino and Microsoft mail, applications and indexes, LDAP, File
system and Web content - returns results to a single Web browser interface.
򐂰 Advanced Web crawling and information mining.
򐂰 Workflow integration for items accessed by Information Integrator for Content.
򐂰 An application development toolkit with a consolidated set of APIs for rapid
portal application.
򐂰 Broad platform support.
򐂰 Broad support for relational databases.

4.4 Case study

The “xyz” Corporation has engaged their engineers to plan, design, and
implement a solution to enable them to share data between the different
departments. The “xyz” Corporation is a silicon design company that has an
heterogeneous computing environment. Each department has its own data
repository spread over several countries. The CIO has decided to have a
common IT budget for the company and it should share information across
departments by grid technologies. The core business today runs on top of the
UNIX operating system and each department has its own server that runs the

Chapter 4. Information virtualization 81

 Windows®-based operating system. Additionally, there are several Linux servers
supporting the engineering and manufacturing department.

4.4.1 Problem overview

Each of the departments in the countries that the company operate in does its
own research and stores their findings and information locally. Users from
another country can only gain access to the data by asking their colleague in the
other country for it, so there is redundancy of work at times.

At the management level, the CIO wants to evaluate the use of a virtualization
technology to improve the research time and achieve higher resource utilization.
Information grids support heterogeneous platforms and provide data sharing and
large-scale collaboration. Figure 4-9 illustrates the current infrastructure that
“xyz” Corporation has.

F ile F ile DB F ile F ile DB

DB F o ld e r F o ld e r DB

N e tw o rk
F a c ility A F a c ility C

F ile F ile DB

DB

F a c ility B

Figure 4-9 “xyz” Infrastructure

4.4.2 Grid solution planning

Before we put the data grid solution together, let us evaluate the findings that we
got from the client and categorize them into “Current Environment” and
“Challenges”.

82 Grid Computing Products and Services

 Current environment
򐂰 Heterogeneous operating environment consisting of UNIX, Linux, and
Windows-based operating systems.
򐂰 Environment is distributed, with branches over multiple locations having their
own data repositories.
򐂰 Each of the departments has their own IT budget and thus there are
heterogeneous storage solutions: SAN, NAS, and directed attached storage.

Challenges
򐂰 Engineers working on similar projects are located in distinct sites, but the
research repository data is only available to the main team.
򐂰 Access to data today is performed on a case-to-case basis, and data sharing
between different research center is low.
򐂰 The file sharing protocols are different: The UNIX and LINUX machines
access shared files using Network File System (NFS) and the Microsoft
based machines access shared files using the Server Messages Block (SMB)
protocol.

Having the above scenario in mind, IT engineers assigned by the CIO will work
together to come up with a solution and implement it as a pilot data grid project to
share information between two of “xyz” Corporation’s branches. An information
grid in this case will enable engineers and other departments to share and view
data regardless of where it is located or resides. It will also provides access to
heterogeneous files, databases, and storage systems, and shares files through
multiple protocols.

4.4.3 Federation of data

Looking at the success of optimizing the infrastructure using a data grid, the CIO
of “xyz” Corporation is now looking at building a knowledge portal for sharing
information across the company. With this portal, the CIO also aims to give users
the ability to search for data from multiple data sources regardless of the location
and data type.

Current environment
򐂰 There are multiple databases deployed throughout the corporation.
򐂰 There are also documents in text files and images.
򐂰 Users have to go through each database to search for files and information,
and there is no universal data repository of the information.

Chapter 4. Information virtualization 83

 IT engineers from “xyz” Corporation evaluated multiple products. The team is
looking for a product that can federate data that they have without having to
change the database platform. The existing infrastructure should remain and the
portal should be present as an interface for the users to query the data.

After evaluation, the team decided to go for WebSphere Information Integration,

as it allows them to federate data from structured and unstructured data into a
single database. Therefore, for the pilot project, the team has decided to federate
their databases.

The current final solution is shown in Figure 4-10.

Facility A Facility B Facility C

Virtualized-NFS, SMB, SQL Access

WebSphere®
Information Integration

File File DB2 MySQL Oracle DB

Figure 4-10 Federated using WebSphere Information Integration

84 Grid Computing Products and Services

 5

Chapter 5. Other disciplines

This chapter discusses the following:
򐂰 Provisioning and orchestration
򐂰 Security
򐂰 Directory Services
򐂰 Licensing
򐂰 Billing and metering

© Copyright IBM Corp. 2005. All rights reserved. 85

5.1 Introduction
When building a grid infrastructure, there are other components that we should
take into consideration at the design stage. The components will directly affect
the future operation of the infrastructure. Some of the components are:
򐂰 Provisioning and orchestration
򐂰 Security
򐂰 Licensing
򐂰 Monitoring
򐂰 Billing and metering

5.2 Provisioning and orchestration

Provisioning is responsible for allocating new machines automatically while
orchestration is responsible for provisioning resources in a time line (or along the
time line). Orchestration is dynamic and on demand, and adapts to the real-time
requirements of the enterprise. For example, provisioning can automate a task to
install a server; orchestration can automate a task to put all the right provisioned
resources in the right place in the right time.

It is quite usual that, on typical IT environments, workload varies broadly, with

some peaks during the night, others during the day, and so on. You need to
provision servers, storage, and infrastructure in order to support the highest
workload. It means that these servers, storage, and IT infrastructure are going to
be highly underutilized when those applications are not at their peak.
Additionally, the processes for deploying these servers and the underlying
software are mainly manual. This also creates another problem: The ability of a
company to respond to changes is relatively slow because of the manual
processes that are used to deploy and provision an application, or add capacity
to a server farm, or to accommodate the increased utilization and peak
workloads that you see.

Provisioning
Provisioning handles the coordination and allocation of managed resources,
such as servers or storage within an on demand environment.

These resources are provisioned dynamically since they are required for
satisfying the needs of a business application. Provisioning is done by allocating
managed resources from a “resource pool". This gives the IT organization the
flexibility to dynamically respond to the needs of the business. Resources can be
dynamically allocated from the pool, then de-allocated and returned to the pool

86 Grid Computing Products and Services

after the resource is no longer needed for a specific business application. Once
de-allocated and returned to the pool, the resource then becomes readily
available to be allocated to another application, where resources might be
needed to ensure that the application’s performance goals and commitments are
met.

In addition to provisioning hardware resources, operating systems images,

middleware (such as Web applications), server software, and DB software can
be managed by the provisioning component of the framework. The provisioning
component provides the "what to do" piece of automation to ensure that a newly
provisioned resource is made fully usable to support the business goals.
Provisioning and workload management work together to provide resources
when needed. Given a pool of servers, the provisioning component will attempt
to maximize the effectiveness of the pool by increasing or decreasing the number
of servers assigned to a given cluster.

Orchestration
Orchestration extends the provisioning model by providing a sensing, capacity
planning, and responsive software application to dynamically meet service level
agreements. Because of this sensing and responsive software architecture,
orchestration enables dynamic and automated allocation of server capacity on
demand. Intelligent Orchestrator, by dynamically allocating capacity to
applications that require it, improves utilization of underutilized computing
systems. Orchestrator reallocates existing server capacity to support
unpredictable application demands. It improves application service levels without
investing in additional capacity. It increases the value of IT server investments by
making server capacity available to all applications under its management. Using
Intelligent Orchestrator, a data center can accurately predict impending capacity
requirements, and facilitate dynamic infrastructure reallocation of server and
network resources. Orchestration initiates the activities necessary to
automatically maintain server availability and meet service level agreements. It
provides the why, where, and when of a complete solution.

Why
By monitoring the applications under its control, an orchestrator senses
degrading performance and determines the actions that have to be taken.

Where
Because solutions are monitored closely, an orchestrator determines where a
server needs to be deployed and automatically instructs the provisioning function
to deploy a server to the proper cluster and configure all the network attributes to
keep the solution within adequate service levels.

Chapter 5. Other disciplines 87

 When
By means of its capacity management capabilities, an orchestrator predicts when
resources will become available or be needed. It starts the provisioning process
just-in-time to meet an application’s growing or decreasing workload
requirements.

5.2.1 Provisioning and orchestration products

IBM has a complete solution environment for provisioning and orchestration. The
product family is composed IBM Provisioning Manager and IBM Tivoli Intelligent
Orchestrator.

IBM Tivoli Provisioning Manager

The systems provisioning capability introduces the concept of a pool of
resources that are shared between different workloads. Tivoli Provisioning
Manager provides the necessary tools for making machines available in a broad,
grid-enabled infrastructure. Provisioning of systems is a sequence of activities
that need to happen in a specific order. The activities have dependencies on
previous activities. Typical activities include installing operating systems,
remotely booting networks, configuring networks such as Virtual Private
Networks (VPN) and storage environments like Storage Area Network (SAN) or
Network Attached Storage (NAS), and "deprovisioning" resources that are no
longer needed back into an available pool.

Automating the execution of these processes is a huge savings in terms of staff,

enabling the leveraging of the highly skilled staff for other tasks more aligned to
your business strategy, rather than continuously repeating traditional tasks.

Note: IBM Tivoli Provisioning Manager now enables on demand computing

across the entire data center through server, storage, and network
automation. For more information, please refer to the following Web site:
http://www.ibm.com/software/tivoli/products/prov-mgr/

IBM Tivoli Intelligent Orchestrator

IBM Tivoli Intelligent Orchestrator (“Tivoli Intelligent Orchestrator” on page 180)
enables staff to sense and respond to changes in the environment so that the
infrastructure meets the service level commitments.

The IBM Tivoli Orchestration solution makes it possible for the staff to monitor the
response times that clients are actually getting from the business applications. If
the system senses that is about to break that minimum required service level, it
will direct IBM Tivoli Provisioning Manager to provision more IT resources to
support the application that those clients are using. IBM Tivoli Intelligent

88 Grid Computing Products and Services

 Orchestrator does this automatically and is fully integrated with IBM Tivoli
Provisioning Manager. When IBM Tivoli Intelligent Orchestrator perceives that
the response time is back in the normal range, it can actually deprovision some
resources, so that those excess or underutilized resources can then be
redeployed and used somewhere else in the site as appropriate.

Note: IBM Tivoli Intelligent Orchestrator extends the benefits of the IBM Tivoli
Provisioning Manager. For more information, please refer to the following Web
site:
http://www.ibm.com/software/tivoli/products/intell-orch/

5.2.2 Provisioning and orchestration case study

This case is based on a hypothetical Internet banking application. Grid
environments sites are spread out across the multiple bank locations. in
Figure 5-1, we have a typical IT infrastructure of an Internet bank system.

Figure 5-1 A typical Internet banking infrastructure

Chapter 5. Other disciplines 89

 This three tier environment has a firewall that separates each layer, in order to
protect them from intruders. The several site locations are interlinked on the
same private network, creating a virtual private network (VPN), as seen in
Figure 5-2.

Figure 5-2 A bank VPN infrastructure

90 Grid Computing Products and Services

 Internet banking application
Internet banking applications typically have a great number of clients and
requests.

When the current Web server pool is not enough to respond to clients’ requests
in an adequate amount of time, the orchestration mechanism takes over and
searches for underutilized resources on other company’s sites. Under manual
processing, these new machines will be added one by one, demanding many
operators and much time. Using the Provisioning Manager, the deployment of
new machines could be automated, demanding less time and no operator for the
reallocation.

The operations described above are managed and made by IBM Tivoli
Provisioning and IBM Tivoli Orchestration. Tivoli Provisioning is responsible for
allocating new machines automatically while orchestration provides a dynamic
and on demand environment for applying the company’s best practices.

5.2.3 Correlated products

The IBM Cluster Systems Management (CSM) can be considered part of the
provisioning discipline for grids. One of the most important features of CSM is to
monitor both hardware and software events, and to trigger automated recovery
actions when appropriate. These event monitoring and automated recovery
features enhance the efficiency of the administrator and the reliability of the
environment by enabling rapid problem detection and resolution.

IBM Cluster Systems Management

The purpose of the IBM Cluster Systems Management (CSM) product is to
provide broad management capabilities for a multiple nodes environment, such
as clusters and grids. CSM provides management from a single point of control.
These include resource monitoring, automated monitoring and operation, remote
hardware control, remote command execution, security, configuration file
management, parallel network installation, and diagnostics.

CSM provides a variety of other benefits:

򐂰 CSM helps administrators deploy their nodes rapidly by automating many
configuration tasks and by leveraging existing Open Source products.
򐂰 CSM provides efficient monitoring of the environment resources without
overwhelming network bandwidth.
򐂰 The automated error detection CSM provides helps catch problems before
they impact the environment, and assists with rapid resolution and recovery of
problems that occur.

Chapter 5. Other disciplines 91

 򐂰 CSM has an architecture and modular construction that maximizes flexibility
so your distributed solution can evolve and grow as your needs change.
򐂰 You can use CSM to manage a homogeneous or a heterogeneous
environment.

Note: IBM Cluster Systems Management provides a robust, powerful, and

centralized way to manage a large number of nodes from a single point of
control. For more information, please refer to the following Web site:
http://www.ibm.com/servers/eserver/clusters/software/

5.3 Security
This section discuss security and its relationship with grid computing. In grid or in
any distributed computing system, security plays a major role when ensuring the
quality of service delivered. We can divide security into several subjects to
analyze how a grid can satisfy basic security requirements:
Authentication The grid has to “be aware“ of the identity of the users that
interact with it, either providing or using resources.
Authorization The grid has to restrict access to its resources to the
users that are eligible to access it.
Integrity Data exchanged among grid nodes should not be subject
to tampering.
Privacy The grid must provide facilities for assuring the privacy of
the information exchanged between nodes when
required.

In order to implement a comprehensive security infrastructure, with centralized

definitions for resources, which usually are at different levels, some additional
features may be recommended, for example, in a UNIX infrastructure, with
operating systems that are part of the grid to run jobs, and, for example, for some
specific nodes with WebSphere Application Server, we can have a LDAP Server
with the definition of the users, passwords, and more resources. In that way, the
UNIX operating systems and WebSphere nodes can request the LDAP Server
when an user or application is trying to log into the grid. This centralized
mechanism may be better than creating a set of users in WebSphere and
another set of users on the UNIX boxes and maintaining two sets of users. This
implementation may be part of a single sign-on strategy.

Other desirable mechanisms that a grid could implement are related to the
trusted relationships that have to be established among different virtual
organizations that belong to the same grid at a certain time. Globus Toolkit and

92 Grid Computing Products and Services

 IBM Grid Toolbox require digital certificates to include a node into a grid. This
requires a certification authority and a means of certificate validation. In a grid
that has resources from different real organizations, some traditional IT
mechanisms should be implemented to share resources, like a virtual private
network (VPN). In more restricted grids, or in the communication between two
independent grids, there should be mutually trusted mechanisms to ensure
security, or, in other words, one grid should be able to rely on the policies of the
other grid and vice-versa.

5.3.1 Firewalls, VPNs, and virtual organizations

Grid allows you to share resources between real organizations by creating a
virtual organization. The way to create an effective virtual organization is often
carried out through traditional IT mechanisms. This implies the use of firewalls,
VPNs, portals, and others.

As an example, consider the scenario in which a corporate user submits a job to

a computational grid that runs in a partner enterprise: in this case, a trusted
relationship prior to the job’s submission exists, and this trusted relationship
between two corporations includes the cross-execution of jobs in their grids.
There are some traditional IT components that allow the joining of both IT
infrastructures for specific services: leased lines and other hard communication
links are examples of hardware components used to join different private
networks.

On the software side, there are some alternatives: probably the most traditional is
a SSL-based virtual private network (VPN). This consists of an encrypted,
secure, and mutually trusted connection, which establishes a transparent tunnel
of communication between two nodes that belong to different network contexts.
In this sense, a VPN may be used to join two real organizations into one virtual
organization for the implementation of a single grid.

Additional security requirements and interesting capabilities for grid are mutual
authentication among different organizations and delegation of authentication so
that a process becomes more efficient when a computational activity involves a
number of spread resources.

Chapter 5. Other disciplines 93

5.3.2 Security products
In this section, we present a set of security products for grid.

Tivoli Access Manager

Tivoli Access Manager (“Tivoli Access Manager” on page 177) implements
security mechanisms based on predefined policies. A policy is a higher level rule
that details a set of behaviors for several systems in different situations; it means
that the user can define policies rather than just permissions. Tivoli Access
Manager provides a security infrastructure to an IT installation, and its security
services can be invoked from Web servers, application servers, databases,
operating systems, domain controllers, and others. Tivoli Access Manager gives
you the opportunity to implement a real security layer that crosses most of the
other software layers, instead of giving just simple rules for the different
subsystems that belong to an infrastructure.

Tivoli Access Manager security mechanisms can be integrated in a grid to

provide comprehensive security services to its resources. Tivoli Access Manager
uses open standards to provide its services. As we can see, these features are
very important for complex environments. These are examples of typical IT
components that can be integrated into a distributed environment.

Tivoli Access Manager for e-business

Provides unified authentication and authorization to an IT infrastructure. Tivoli
Access Manager for e-business supports single sign-on to mainframe, UNIX
operating systems, Microsoft foundation, Web servers and Web application
servers, and telnet. It exports a number of APIs to be used from .NET and J2EE
applications. It offers flexibility in design time, as it can be used with plug-ins in
Web servers and in highly scalable proxy architecture. All these services allow
the developers to concentrate on the business logic, avoiding spending time on
the design of security mechanisms. From the administrator’s point of view, Tivoli
Access Manager provides an unified and centralized security mechanism to the
whole infrastructure. It also gives benefits in time and cost to the IT management
staff.

Note: Tivoli Access Manager for e-business is part of the IBM Tivoli Access
Manager family of products. For more information, please refer to the following
Web site:
http://www.ibm.com/software/tivoli/products/access-mgr-e-bus/

Note: More information about .NET can be found at:

http://www.microsoft.com/net/

94 Grid Computing Products and Services

Tivoli Access Manager for Business Integration
Tivoli Access Manager for Business Integration is a multi-platform security
management solution. It was specifically designed for the WebSphere MQ family
and it basically upgrades MQ’s native security services. The data-protection
provided by Tivoli Access Manager for Business Integration is at the application
level, so it is not necessary to modify or recompile the MQ applications that use
Tivoli Access Manager for Business Integration. This level of protection covers
the link and channel level, which are related to the confidentiality and integrity of
the messages when they are in transit between two systems. Tivoli Access
Manager for Business Integration also gives security mechanisms that are used
when the messages are resident in a queue to the control of MQ.

Note: Tivoli Access Manager for Business Integration is part of the IBM Tivoli
Access Manager family of products. For more information, please refer to the
following Web site:
http://www.ibm.com/software/tivoli/products/access-mgr-bus-integration/

Tivoli Access Manager for Operating Systems

Tivoli Access Manager for Operating Systems is a powerful security system that
locks resources and prevents unauthorized access. It is simple to use and it is
intended to secure files, applications, and even other operating platforms. This
product is useful for protecting the infrastructure even from internal unauthorized
access, blocking and protecting the environment from both inside and outside
suspicious activities. It provides audit mechanisms to midrange operating
systems that are similar to those provided by the mainframe, like world-class
protection, auditing, and tracking of the actions of every user. It also provides a
tool for self-auditing, Personal Universal Auditing, that audits users, programs,
files, ports, resources, and systems.

Note: Tivoli Access Manager for Operating Systems is part of the IBM Tivoli
Access Manager family of products. For more information, please refer to the
following Web site:
http://www.ibm.com/software/tivoli/products/access-mgr-operating-sys/

Grid Security Infrastructure (GSI)

Globus Toolkit and IBM Grid Toolbox provide an implementation of Grid Security
Infrastructure (GSI). GSI uses digital certificates to provide mutual
authentication, single sign-on, and privacy of the information exchange between
two grid nodes.

Chapter 5. Other disciplines 95

 Note: GSI is a set of libraries and tools, provided with the Globus Toolkit, for
accomplishing secure authentication over an open network. It is possible to
use the authentication portion of the Globus Toolkit via two applications:
GSI-enabled Secure Shell (SSH) and GSI-enabled FTP (gsiftp). For more
information, please refer to the following Web sites:
http://www.globus.org/security/v1.1/standalone.html
http://www-unix.globus.org/toolkit/docs/3.2/security.html
http://publib.boulder.ibm.com/eserver/v1r1/en_US/index.htm?info/ogsainfo/
secure.htm
http://www-unix.globus.org/toolkit/docs/3.2/gsi/key/index.html

The single sign-on mechanism across organizational boundaries includes

delegation of credentials. That mechanism allows the secure execution of
computational activities that involve resources scattered over multiple sites.

Mutual authentication occurs when two parties have their own certificates signed
by their respective CAs (certification authorities), and those actors also trust the
signature of the other’s CAs in the other’s certificates. This mechanism allows
you to pass over the boundaries of an organization.

GSI also implements delegation of an authentication mechanism through a

proxy. Typical situations where this mechanism is useful are, for example, when
some computational activity in the grid requires several mutual authentications in
several resources. That computational activity may be generated by a user or by
local/remote agents on behalf of the user; this process might demand the
reentering of the certificate’s passphrase every time a resource is accessed. A
proxy can be useful in automating this process by delegation: in this context, it is
just a new certificate with new public and private keys, and that certificate is
signed by the owner instead of the CA. Then, the proxy’s certificate is used on
behalf of the user’s certificate.

Further explanation of these processes and GSI can be found in the redbook
Introduction to Grid Computing with Globus, SG24-6895 (at Redbooks Web site)
and at the Globus Alliance Web site:
http://www.redbooks.ibm.com/
http://www.globus.org

Enterprise Identity Mapping

Enterprise Identity Mapping (EIM) is part of the Autonomic Computing initiative
for Server Groups IBM ^® security. It provides a mapping between the
users of several systems and the EIM identity, by associating the user registered
in every system with the identity. EIM implements a list or directory of
users/passwords/systems and has connectors to access them. The benefit for

96 Grid Computing Products and Services

 the user, the developer, and the infrastructure itself is that a given application that
accesses different subsystems, all of them with the same human user but with
different users to the systems' point of view, will not need to place, cache, or send
a user and password for each subsystem. Instead, the application can access
the EIM repository and then call EIM every time a new user/password is needed
in the flow. EIM can be considered as a basic single-sign-on mechanism for IBM
and non-IBM platforms.

The EIM Domain controller is based on a LDAP Server. In that server, EIM stores
the identifiers, associations, and registries for the different systems. The
registries contain a user and password for a given identity in a given system.

EIM can help in a grid environment by providing unified identities to users that
have access with different user and passwords to heterogeneous systems.

Note: Enterprise Identity Mapping is part of the Autonomic Computing

initiative for Server Groups IBM ^ security. For more information,
please refer to the following Web site:
http://www.ibm.com/servers/eserver/security/eim/

5.3.3 Conclusions
In summary, we can say that there are not yet a wide range of products
specifically designed for security in grids. However, most traditional IT products
may be integrated into a grid system. That is because a grid can be seen as a
new way of using traditional IT components. Therefore, the ability to integrate
traditional IT components with specifically designed components for grid is
associated with those products capabilities.

A grid surely will require sophisticated security mechanisms due to its intrinsic
complexity and distribution of resources. For that reason, we recommend having
a security framework or a similar security infrastructure, based on open
standards, to unify the security policies and ease the definition and management
of complex rules. Toolkits like Globus Toolkit and IBM Grid Toolbox require digital
certificates, but they usually provide only services to applications, and the grid
has resources other than applications. At this point, a security framework and
other mechanisms are vital for the health of the grid.

Chapter 5. Other disciplines 97

5.4 Directory Services
With the idea that a grid can be made up of many heterogeneous resources, it is
vital to have a method of capturing what the grid looks like. One would want to be
able to perform queries on both static and dynamic attributes of a grid. The static
attributes may include the machines enrolled in a grid, the operation system
version, CPU speed, physical memory, and so on. The dynamic attributes may
include what machines are available, disk space available, current load, and so
on. The larger the grid, the more important it is to be able to store and retrieve
this kind of information. Some of grid products include this capability as part of
the solution, such as the Globus Toolkit.

Most of the time, Directory Services are interrelated with security disciplines and
products (please refer to 5.3, “Security” on page 92).

5.4.1 Directory Services products

In this section, we present the Directory Services products.

Tivoli Directory Server

Tivoli Directory Server (TDS) is a component of the IBM identity management
solution. It is the foundation for deploying comprehensive identity management
applications and advanced software architectures like grids. TDS implements the
Internet Engineering Task Force (IETF) LDAP V3 specifications. It also includes
enhancements added by IBM in functional and performance areas. TDS uses
IBM DB2 Universal Database as backup storage to provide per-LDAP operation
transaction integrity, high performance operations, and online backup and restore
capability.

Note: Tivoli Directory Server is a component of the IBM identity management

solution. For more information, please refer to the following Web site:
http://www.ibm.com/software/tivoli/products/directory-server/

Tivoli Directory Integrator

Tivoli Directory Integrator (TDI) is one of the products of the IBM identity
management solution. It can integrate different directories of resources across
the enterprise. These repositories can be placed on directories, databases, and
applications.

TDI is a suitable solution for those organizations with heterogeneous repositories

of resources that prefer keeping those repositories but providing a unified and
integrated vision to the applications that use their services. TDI can be useful as
well for these organizations when they decide to consolidate the heterogeneous

98 Grid Computing Products and Services

 repositories because the consolidation activities can be done while TDI hides
that complexity to the applications that use their capabilities. In other words, TDI
can be used as a central hub of a decentralized directory infrastructure, while it
can help to centralize it if the organization needs it for some reason.

TDI has built-in connectors to heterogeneous repositories of resources and

provide Java APIs to modify or extend these connectors.

TDI can synchronize repositories, provide a consistent structure of directories,

even though those repositories were different, and can implement rules to
expose the right resource to right consumer.

It supports Web services, connectors to LDAP, JDBC™, JMS, XML, and JNDI,
among others, and includes the IBM Tivoli Directory Server, which is the IBM
directory of resources for mid-range platforms.

Note: Tivoli Directory Integrator (TDI) is one of the products of the IBM identity
management solution For more information, please refer to the following Web
site:
http://www.ibm.com/software/tivoli/products/directory-integrator

5.4.2 Conclusions
Directory Services are used in a grid environment to make the search and
discovery of resources placed in different directories or repositories easier by
providing a unified vision of the directory to the grid environment or the user that
asks for certain services from the grid.

5.5 Licensing
In most organizations, a main portion of their investment in information
technology infrastructure is dedicated to software licensing. For this reason, it is
not surprising that most companies are looking for a better way to manage their
software licenses. This leads us to a new term: Software Asset Management.
This is a process of managing software assets, which includes budget planning,
software life cycle, such as upgrades, and others.

One of the concerns about grid implementations is related to the licensing of

software components. This is especially true with Independent Software Vendor
(ISV) applications, as most of them tie their licenses to processor count, host
identification (ID), and processor type. When this happens, it limits the growth
and the scalability of a grid.

Chapter 5. Other disciplines 99

 Some of the commonly heard terms about licensing are listed below:
򐂰 Site license
򐂰 Node locked license
򐂰 Floating license

Site license
Under site license, the company typically pays one single amount to have
unlimited use of the software: the company is allowed to add additional servers
without worrying about paying new fees or a nominal fee could be charged based
on the amount of new servers. This is commonly used in data centers where the
software is extensively used, and helps build up an agile environment, as users
can run their applications anywhere. On the other hand, this is generally a very
expensive solution.

Node locked license

One of the most commonly deployed licensing method is node locked license,
which basically means that the application is constrained to certain hosts. Often,
the ISV will generate a key for that applications based on MAC address, host ID,
CPU ID, number of processors it has, type of processor, and others. In such a
scenario, when a grid has to be deployed, the architect will require some
additional planning and the grid platform has to keep track of which server has
which applications.

Floating license
In the floating license method or model, a certain number of licenses is shared
within a set of nodes. Most often, there is a license manager server controlling
and keeping track of the licenses distributed. Applications request an available
license from the license manager when it is started, and once completed, it will
return it back. However, if the licenses are insufficient, additional applications
instances are unable to execute. This model is very effective and very commonly
used in grid applications (if the application supports it), allowing growth and
scalability.

100 Grid Computing Products and Services

5.5.1 Licensing products
In this section, we present a licensing product that may be integrated into a grid
system.

IBM Tivoli License Manager

IBM Tivoli License Manager offers an effective way to manage multiple product
licenses by means of company-tailored policies. IBM Tivoli License Manager
(ITLM) helps with the following tasks:
򐂰 Information collected about the products installed, using an inventory scan
technology
򐂰 Keeping track of the in-use licenses, available licenses, and procured licenses
򐂰 Monitoring the start and stop of a software on any machine
򐂰 Monitoring of license usage patterns and informing administrators
򐂰 Assigning licenses to systems
򐂰 Provides historical and real-time reports of the software usage

Note: License Manager is part of the IBM Tivoli software family to automate
the software licensing and utilization, and provides advanced reporting and
inventory capabilities. For more information, please refer to the following Web
site:
http://www.ibm.com/software/tivoli/products/license-mgr/

5.5.2 Conclusion
As more software vendors start to adopt grid into their software, the licensing
strategy may expand. Major vendors have software licensing models catered
towards grid environments. In extreme cases, a license can be managed by
provisioning techniques. In this case, the software is installed on the server in
which the execution is to be performed and, after the job is completed, the
application is removed.

Chapter 5. Other disciplines 101

5.6 Billing and metering
Some implementations may require the use of metering and billing techniques,
such as utility-like services, so that the owner of the resources made available is
somehow compensated by providing the resource.

Some of the commonly used methods to accomplish billing and metering today
includes:
򐂰 Scheduler accounting logs
򐂰 Licensing management

Scheduler accounting logs

When jobs are submitted, queued and pushed out for execution by job
schedulers, many lot entries are generated. In this way, logs usually contain very
useful information, such as when the job is started, when it ends, which
machines are selected, and how long the execution took.

Licensing management
Licensing management is one of the easiest methods to monitor the usage of
software, manually or through a software, such a IBM Tivoli License Manager.
Usage can be tracked based on the application installation rather than the
application’s executions. This may sound ineffective, but it provides the quickest
method to deploy billing and metering. Products such as IBM Tivoli Orchestration
and IBM Tivoli Provisioning Manager have some features for monitoring the
usage, which can be used for billing and metering as well.

5.6.1 Conclusion
As utility-like services get more popular, there will be a greater demand for billing
and metering software with better defined control. This is especially true in light
of electronic utility computing, also known as “e-utility computing”, in which
someone can use the computing power through simple and dynamic access, in
an on-demand basis, and be billed at a later stage.

102 Grid Computing Products and Services

 Part 3

Part 3 Bundles

© Copyright IBM Corp. 2005. All rights reserved. 103

104 Grid Computing Products and Services
 6

Chapter 6. Virtualization Engine

This chapter discusses the following topic:
򐂰 Virtualization Engine™

© Copyright IBM Corp. 2005. All rights reserved. 105

6.1 Introduction
Virtualization Engine (VE) is an IBM suite, a set of systems, products, and
technologies that work together in a comprehensive way to provide a seamless
and unified side of the IT infrastructure. This reduces the complexity of allocating
new resources to applications, as all of these resources are dealt within the same
fashion. It also allows easier monitoring and management of the IT resources.

Note: The IBM Virtualization Engine utilizes key IBM virtualization

technologies, making a comprehensive approach available to enterprise-wide
virtualization that is consistent across heterogeneous environments.

Complete information about IBM Virtualization Solutions are presented at the

following Web site:
http://www.ibm.com/servers/eserver/about/virtualization/index.html

As shown on Figure 6-1, VE has three major groups of virtualization technologies

and products:
򐂰 Systems services
򐂰 Systems technologies
򐂰 Operating systems

S y s te m s
S e r v ic e s

IB M
V irtu a liz a tio n
E n g in e ™
S y s te m s O p e r a tin g
T e c h n o lo g ie s S y s te m s

Figure 6-1 Groups of virtualization

System services
System services’ products and technologies are higher level components in the
stack of infrastructure layers. This set of components is intended to create the
environment to run applications, manage the complexity of the technology, and
create a unified repository of storage resources. There are two main groups of
elements: those related to servers and those related to storage.

106 Grid Computing Products and Services

 System technologies
System technologies of VE are the technologies available in IBM ^
machines. These components provide virtualization intelligence in the processor,
including micro partition abilities, memory, I/O, and network. These technologies
are embedded in IBM hardware, and they are delivered in the IBM ^ and
TotalStorage brands. Examples of these technologies are Hypervisor, Virtual I/O,
and Virtual LAN, available in Power5 machines.

Operating systems
VE features work on the operating system level, as they provide the ability to run
multiple operating systems on a single box. These characteristics are available in
pSeries, iSeries™ through LPAR and DLPAR technologies, and zSeries®, which
allows LPAR and VM images to run different operating systems. xSeries
operating systems can be virtualized through VMware software
(http://www.vmware.com/) that can be integrated into a VE infrastructure.
Examples of the operating systems that can use these features are z/OS®,
i5/OS™, AIX 5L, Linux, and Windows.

6.2 Virtualization Engine suites

Virtualization Engine suite provides provisioning resources, workload
management, monitoring, and a console for management and administration.
This console is Web-based and has a sort of “dashboard” to manage the VE
infrastructure (see “Virtualization Engine console” on page 108). Drill-down
features and an intuitive user interface to identify different situations and better
understand the infrastructure’s behavior are included. Additionally, VE also
provides management capabilities for the grid and for its individual components.

There are two suites that make up Virtualization Engine systems services:
򐂰 IBM Virtualization Engine Suite for Servers
򐂰 IBM Virtualization Engine Suite for Storage

The main differentiator of VE is that it provides services for the whole stack of
infrastructure levels, creating an integrated environment to run and manage
applications, and to provide, virtualize, and manage resources.

Chapter 6. Virtualization Engine 107

 Virtualization Engine console
The Virtualization Engine console is based on the IBM Integrated Solutions
Console framework to provide a consolidated view for managing your virtualized
enterprise resources. The Virtualization Engine console is used to manage your
IT environment by looking at your systems beyond operating system boundaries
and helping to better maximize resource sharing. It provides two powerful
cross-platform applications: the health center and the launch-pad. The health
center provides a single view point to perform key system and storage monitoring
and management. The launch-pad provides a single view to launch key
Virtualization Engine related systems and storage management consoles.

6.2.1 IBM Virtualization Engine Suite for Servers

Virtualization Engine Suite for Servers, also known as Virtualization Engine
Services Multiplatform, helps provide virtualization and management of
resources across select systems, both in an individual server and across IBM
and select non-IBM systems. This suite delivers tools for running grid
applications as well as provisioning, management, and workload management.
as Virtualization Engine Services Multiplatform consists of the following systems
services:
򐂰 VE Console (see “Virtualization Engine console” on page 108
򐂰 IBM Enterprise Workload Manager
򐂰 IBM Director Multiplatform
򐂰 IBM Tivoli Provisioning Manager
򐂰 IBM Grid Toolbox V3 for Multiplatforms

IBM Enterprise Workload Manager

Enterprise Workload Manager (EWLM) enables you to automatically monitor and
manage multi-tiered, distributed, heterogeneous, or homogeneous workloads
across an IT infrastructure to better achieve defined business goals for end-user
services. These capabilities allow you to identify work requests based on service
class definitions, track performance of those requests across server and
subsystem boundaries, and manage the underlying physical and network
resources to set specified performance goals for each service class.

EWLM is intended to support application performance, making the right

resources available to the applications that need them, in real time, usually
during peak workloads. EWLM collects ARM messages sent by the resources;
that way, EWLM can report metrics related to performance goals as business
priority, response time, and others.

EWLM can also work jointly with Tivoli Intelligent Orchestrator (TIO). TIO can
deploy the EWLM agent in the resources that it is allocating through provisioning
and orchestration. Those resources are allocated in a EWLM domain, so EWLM

108 Grid Computing Products and Services

will recognize them and will optimize their utilization based on the designed
policies.

EWLM can be integrated with WebSphere Application Server’s applications, as it

uses open interfaces based on ARM. WebSphere and DB2 offer instrumentation
data through ARM, so they can provide information to EWLM in order to be
managed by it. The user defines the policies and transaction classes, which can
be used in all the supported platforms. EWLM is able to monitor the behavior of
WebSphere Application Server and DB2. Those are the input used to make
workload balancing decisions based on business policies and infrastructure
parameters, like server utilization, transaction response time, and topology.
EWLM provides a Web based console with drill-down capabilities to inspect
resources.

IBM Director Multiplatform

Director Multiplatform helps deliver a common, consistent, and cross-platform
systems management solution for IBM servers, storage, and operating systems.
It provides a single administrative console for management tasks (operating
system, storage management, distributed systems management, and platform
management), a common management infrastructure for upward integration with
Tivoli, and a management foundation for the on demand architecture.

Using Director Multiplatform, many of the administrator's manual tasks can be

automated to proactively and remotely manage systems. Tasks such as
discovery, event logs and action plans, file transfer, inventory collection, process
management, resource monitors and thresholds are among those tasks that fall
into this category. Additionally, the predictive and proactive capabilities
associated with alerting and real-time system diagnostics help maximize server
uptime and reduce service downtime costs.

IBM Tivoli Provisioning Manager

See “IBM Tivoli Provisioning Manager” on page 88.

IBM Grid Toolbox V3 for Multiplatforms

Grid Toolbox V3 for Multiplatforms, based on the Globus Toolkit V3.0 and the
Open Grid Services Architecture (OGSA), delivers the infrastructure to help
rapidly develop, deploy, and manage distributed resources. Grid Toolbox V3 uses
a service model approach for its architecture, which keeps the details of the
implementation separate from the interface, and is based on the Open Grid
Services Infrastructure (OGSI). For more information about Globus Toolkit and
OGSA, see 2.2, “Grid standards and toolkits” on page 24.

Chapter 6. Virtualization Engine 109

6.2.2 IBM Virtualization Engine Suite for Storage
Virtualization Engine Suite for Storage can complement the Virtualization Engine
Suite for Servers, and consists of the following separately orderable products:
򐂰 IBM TotalStorage Productivity Center
򐂰 IBM TotalStorage SAN Volume Controller
򐂰 IBM TotalStorage SAN File System

IBM TotalStorage Productivity Center

TotalStorage Productivity Center is an open storage infrastructure management
solution designed to help reduce the effort of managing complex storage
infrastructures, to help improve storage capacity utilization, and to help increase
administrative efficiency.

IBM TotalStorage SAN Volume Controller

SANs today create a fixed relationship between storage devices and servers.
TotalStorage SAN Volume Controller (SVC) implements a virtualization layer
between storage devices and servers. The servers can see virtual disks in the
SAN, and SVC maps those virtual disks in the real physical devices. For more
information about SVC, see 4.3.2, “Block data layer products” on page 71.

IBM TotalStorage SAN File System

Once the block data is virtualized in storage pools managed by SVC, servers can
create their own file systems. However, it is common that several servers share
file systems in an organization. TotalStorage SAN File System (SAN-FS) is a
product that creates that virtual and shared file system. Instead of using other
classic technologies to share file systems through the network and have several
points of management, SAN-FS provides a shared file system for servers
through the SAN itself, with a single point of management and name space. For
more information about SAN-FS, see 4.3.2, “Block data layer products” on
page 71.

6.3 Conclusion
As mentioned before in this redbook, there are several traditional IT components
that can fit into a grid. Moreover, that grid can be improved and leveraged by
these traditional IT components, although they were not specifically designed for
grid. This is an example of the traditional IT technology tools, or non-grid tools,
that are included in provisioning, Virtual I/O, Virtual LAN, micro-partitioning,
monitoring, and so on. Several of these technologies provide services to the grid
computing environment’s underlaying layers.

110 Grid Computing Products and Services

VE utilizes the key IBM virtualization technologies and products while grid
computing is perhaps the most advanced degree of virtualization, as it enables
multi-organization resource virtualization.

VE also provides sophisticated management tools, going further than just

virtualization functions. It also creates a foundation for advanced automation
within IBM ^ and TotalStorage products, as well as non-IBM hardware in
specific subjects.

Chapter 6. Virtualization Engine 111

112 Grid Computing Products and Services
 7

Chapter 7. ISV suites

This chapter discusses the following bundles:
򐂰 Platform Computing
򐂰 DataSynapse
򐂰 United Devices

© Copyright IBM Corp. 2005. All rights reserved. 113

7.1 Introduction
Bundles for grid computing are those sets of products and tools that allow for the
development, deployment, execution, and management of a grid. Most of the
bundles’ components enumerated in this chapter already have been described in
this redbook, so we will not describe them here in detail.

7.2 Platform Computing

Platform Computing has a complete set of tools to develop and run grid
solutions. Platform’s products address the scalability, security, scheduling, and
management attributes of a grid infrastructure. Mainly oriented to server grid,
Platform also contributes to the Global Grid Forum (GGF), adheres to the Open
Grid Services Architecture (OGSA), and supports several operating systems and
hardware.

See 3.4.4, “Platform LSF (Load Sharing Facility)” on page 50, 3.4.5, “Platform
LSF MultiCluster” on page 51, 3.4.6, “Platform LSF HPC (High-Performance
Computing)” on page 51, and 3.4.7, “Platform Symphony” on page 52 for more
informtion about this topic.

Note: Complete information about Platform Computing can be found at:

http://www.platform.com/

Platform offers two different types of bundles, including a full set of grid and
distributed computing products and a family bundle of products.

The bundle of grid and distributing computing products include:

򐂰 Platform LSF
Platform LSF is one of Platform's distributed computing products and is one
of the core products in the family bundle that manages and accelerates batch
workload processing for compute- and data-intensive applications. With
Platform LSF, an organization can intelligently schedule and guarantee the
completion of batch workload across a distributed, virtualized IT environment.
Platform LSF fully utilizes all IT resources regardless of operating system,
including desktops, servers and mainframes, to ensure policy-driven,
prioritized service levels for always-on access to resources. Underpinning
Platform LSF is the production-proven, open, robust Virtual Execution
Machine (VEM) architecture that sets the benchmark for performance and
scalability across heterogeneous environments. Platform LSF has an

114 Grid Computing Products and Services

 extensive set of scheduling capabilities, including Fairshare, Preemption,
Advance Reservation, Resource Reservation, and SLA-based scheduling.
򐂰 Platform LSF HPC
Platform LSF HPC is one of Platform's distributed computing products and is
one of the core products in the family bundle. Platform LSF HPC is software
for high-performance computing needs. This software allows organizations to
intelligently schedule parallel and serial workloads, providing the ability to
solve large, grand challenging problems while utilizing the available
computing resources at maximum capacity. It takes full advantage of high
performance network interconnects available on clustered systems and
supercomputers.
򐂰 Platform Symphony
Platform Symphony is one of Platform's grid and distributed computing
products. Platform Symphony is a comprehensive grid solution that is
enterprise-class software that distributes and virtualizes compute-intensive
application services and processes across existing heterogeneous IT
resources, creating a shared, scalable, and fault-tolerant infrastructure,
delivering faster, more reliable application performance while reducing cost.
Platform Symphony is engineered to provide business-critical applications
compute power on demand. This is accomplished by decoupling application
compute demands from dedicated hardware found in unreliable and
underutilized in LOB IT silos. Underpinning Platform Symphony is the robust
Virtual Execution Machine (VEM). More than simply a distributed application
technology, the VEM is a standards-based, high performance, system-level
architecture. The VEM resides below the application layer, providing a true
virtual environment that is ideal for critical workload management. This
workload-centric approach offers many performance, reliability and
management advantages when compared to other approaches

The second family bundle of products includes:

򐂰 Platform LSF
򐂰 Platform LSF HPC
򐂰 Platform LSF MultiCluster
Platform LSF MultiCluster extends an organization's reach to share
virtualized resources beyond a single Platform LSF cluster to span
geographical locations. With Platform LSF MultiCluster, local ownership and
control is maintained, ensuring priority access to any local cluster while
providing global access across an enterprise grid. Organizations using
Platform LSF MultiCluster complete workload processing faster with
increased computing power, enhancing productivity and speeding time to
results.

Chapter 7. ISV suites 115

 򐂰 Platform LSF License Scheduler
Platform LSF License Scheduler optimizes the usage of all application
licenses across Platform LSF clusters by allocating a virtualized pool of
licenses to users based on an organization's established distribution policy.
Using Platform LSF License Scheduler, organizations can realize dramatic
reductions in additional application license purchases and increase overall
access to license resources.
򐂰 Platform LSF Reports
Platform LSF Reports provides a number of key Platform LSF cluster
performance metrics in a series of reports, which provide insight into cluster
utilization. Using Platform LSF Reports, administrators are able to improve
overall cluster performance and troubleshoot configuration problems.
򐂰 Platform LSF Analytics
Platform LSF Analytics uses workload and license management data to
provide design centers with analytics support for project planning decisions.
Platform LSF Analytics assists engineering managers to estimate project
completion times and provision hardware and licenses more accurately,
ensuring project priorities are being met. Platform LSF Analytics scales for
large Platform LSF clusters and multiple sites around the world.

Components of these bundles are supported on several platforms. Platform also

has its Globus Toolkit implementation. Platform Globus Toolkit supports OGSA,
and it can be integrated within a traditional Platform architecture.

7.3 DataSynapse
DataSynapse's GridServer software provides open standards compliant,
pre-manufactured services across various grid computing disciplines:
scheduling, workload management, data virtualization, and grid middleware.
DataSynapse enables organizations to create "Service Grids" that support
stateful, persistent, transactional processing of high concurrency and traditional
parallel execution workloads.

GridServer enables the broadest set of enterprise applications to run on a shared

infrastructure, unlocking performance bottlenecks across the enterprise for
improved productivity and responsiveness. It creates a virtual application
execution environment that dynamically fulfills service level requirements across
virtual resources, delivering substantially higher service levels for radically lower
costs.

116 Grid Computing Products and Services

 Note: Complete information about DataSynapse's GridServer software can be
found at:
http://www.datasynapse.com/

DataSynapse GridServer provides a service scheduler that enables applications

with transactional or HPC type workloads to be executed in a service oriented
concurrent or parallel execution paradigm, including BPEL workflow support for
event driven scheduling. Refer to 3.4.3, “DataSynapse GridServer” on page 48,
which shows the GridServer components’ architecture.

GridServer also offers an essential role in the Information virtualization discipline.

GridServer provides a standards based, multi-language distributed,
synchronized in memory/disk cache that supports native cache loaders,
transaction manager plug-ins, spring framework transaction manager support of
POJOs, and EIS access via JBDC, EAI, Web Service, and Application calls. It
also supports connection pooling and it can dynamically provision additional
connections on-the-fly.

7.4 United Devices

United Devices offers a portfolio of products for developing, implementing, and
managing diverse and geographically distributed grids of heterogeneous
resources. In addition, its family of products operate the world's largest grid for
massive-scale research projects, such as the grid.org project (see
http://www.grid.org/) and the World Community Grid. For more information
about World Community Grid, refer to Appendix B, “World Community Grid” on
page 149.

The United Devices Grid MP platform family of products can operate grids of any
size as well as aggregate resources on a network (including clusters, servers,
desktops, and laptops) to create a high-performance compute grid for powering a
wide range of applications. United Devices serves clients in a variety of
industries, including life sciences, manufacturing, government, industrial
engineering, and others.

Note: Complete information about United Devices can be found at:

http://www.ud.com/

Grid MP Enterprise from United Devices is able to manage groups of resources,

from a single cluster to large-scale multi-resource grid. The package includes the
MP services and agents supporting Windows, Linux, AIX, UNIX, Solaris™, and

Chapter 7. ISV suites 117

 Mac OS. The product is suitable for highly parallelizable, CPU intensive
computational activities that are “self contained.” This means that a given task
can run without further communication with external resources, up to its
completion. Grid MP is designed to scale up to hundreds of thousands of
devices. Its scalability has been proven with its global grid projects on grid.org, a
grid with over 3 million devices.

In addition, Grid MP also features the following:

򐂰 SDK for applications development that allows for rapid migration of
applications to the grid.
򐂰 Provides an environment in an encrypted form. Furthermore, applications are
run in a secure sandbox environment on all grid nodes.
򐂰 Web-based management interface for administrative users.
򐂰 Workload and scheduling capabilities. For example, it can balance the job
assignation for different donors based on the amount of computing resources,
such as memory or CPU power, made available to the grid. Thus, the unit of
work that is sent to the donors may have different weights based on the
donors’ capabilities. In short, Grid MP assigns jobs to the most appropriate
resource available. Jobs can also be rescheduled if the donors have not
returned the results after a certain period of time.
򐂰 Features a Unified Job Scheduling (UJS) product that allows organizations to
manage isolated, dedicated Distributed Resource Managers (DRMs) from a
single tool. This product preserves departmental control of resources while
enabling the sharing of resource groups that previously operated in silos, with
peaks and valleys in demand.
򐂰 Provides a data warehouse and reporting tool, MP Insight, which allows
organizations to analyze current capacity and utilization information in order
to make more informed purchasing and capacity planning decisions. MP
Insight collects information related to applications, jobs, resources, and users
in the grid and extracts this information into a data warehouse, where it is
made available to users and administrators in the form of metrics-based
reports.

118 Grid Computing Products and Services

 Part 4

Part 4 Services and

solutions

© Copyright IBM Corp. 2005. All rights reserved. 119

120 Grid Computing Products and Services
 8

Chapter 8. Services and solutions

This chapter discusses the following:
򐂰 Services portfolio for grid computing
򐂰 Solutions portfolio for grid computing

© Copyright IBM Corp. 2005. All rights reserved. 121

8.1 Introduction
IBM is committed to open standards and is working with The Globus Alliance
open source development community, the Global Grid Forum, and the
Interoperable Informatics Infrastructure Consortium (I3C) to promote the
adoption of open standards and accelerate the availability of grid offerings that
IBM and its business partners can provide in the grid computing business.

Note: IBM and business partners work together with clients to leverage grid
computing solutions in their existing IT infrastructure. It is important to mention
that grid computing is not an out-of-the-box product: building a grid is the
result of an assessment and architecture development carried out in a
client-tailored fashion.

Important: The following Web site provides a search capability to access all
IBM announcement letters, part numbers, sales manual, press releases,
Redbooks, white papers and catalogs:
http://www.ibm.com/common/ssi

IBM Global Services

IBM Global Services has experienced grid consultants and architects engaged in
grid projects worldwide. With this experience, IBM can help clients in all
industries enhance resiliency, scalability, stability, high availability, security, and
evolve into On Demand Business with their grid deployments. IBM Global
Services is widely recognized as the largest services company in the world and it
is in 164 countries or regions. For more information, please refer to the following
Web site:
http://www.ibm.com/igs

8.2 Services portfolio for grid computing

IBM’s main focus is the creation, development, and manufacture of the industry’s
most advanced information technologies. This includes computer systems,
software, networking systems, storage devices, and microelectronics. IBM Global
Services translates these advanced technologies into business value for clients
and helps make information technology easy to acquire and manage. To learn
more about the services portfolio, visit:
http://www.ibm.com/services/
http://www.ibm.com/linux/grid/solutions/

122 Grid Computing Products and Services

Table 8-1 provides a overview of the services IBM has specifically designed for
grid computing.

Table 8-1 Grid computing services overview

Services Description

Grid Typically a 3~5 days workshop, its main goal is to provide a grid
Innovation computing design overview that can be used as a basis for follow-up
Workshop engagements.

Grid Strategy Engagement service that defines the vision for a client to leverage
Engagement innovative grid computing solutions. It also identifies how grid
computing technologies can impact an organization’s existing
enterprise architecture and business model.

Grid Design Design service that precedes grid computing implementation

Services services. Its main goal is to provide a detailed design to be
implemented afterwards.

Grid Software Service in which implementation, testing, and deployment activities of

Implementatio the grid computing environment, outlined and designed by the
n Services previous services, take place.

Grid Training Training and education offering available to complement and leverage
and Education the skills of the interveners in the grid computing implementation
roadmap.

The graphic in Figure 8-1 on page 124 shows an example roadmap for grid
computing implementation services.

Chapter 8. Services and solutions 123

 Grid Strategy Grid Design Grid Implementation
Engagement Services Services
Project 1 Project 1
Grid Innovation
Workshop

Grid Design
Services Grid Implementation
Project 2 Services
Project 2

Grid Design
Services Grid Implementation
Project n Services
Project n

Grid Implementation Roadmap

Education and Training / Support services

Figure 8-1 Grid computing services roadmap

8.2.1 Grid Innovation Workshop

Grid Innovation Workshop is a service designed to help companies and
universities understand the innovation and potential of grid computing.
Customized for each organization, these workshops are designed to provide
tools that help in the identification of the benefits of grid computing, and define a
roadmap for its implementation.

Description
This service is designed with the objective of identifying high-level business
goals, objectives, and critical success factors. Typically, a workshop that lasts
from three to five days, is designed to define potential stakeholders and
understand goals, needs, and expectations. This gives you a chance to identify
and prioritize actionable pilot opportunities that are aligned with the business
goals, objectives, and critical success factors. One of the objectives in this
service is to develop a grid pilot plan that includes process identification and
implementation timelines that provide awareness regarding the benefits and
technology of grid.

This workshop also lets you define and provide an introduction to grid computing
concepts, evolution, and adoption frameworks. It also provides a high-level
introduction to grid architecture and technologies, and introduces potential

124 Grid Computing Products and Services

 opportunities (current processes, proposed processes, economic justification,
business impact, and technological impact).

These workshops focus on designing a grid environment based on open

standards, tailored for homogeneous and heterogeneous IT architectures,
instead of being limited to certain software or hardware. This allows an
optimization of the existing infrastructure, making the individual optimization of
resources possible.

The main goal of this service is to have every stakeholder (IT representatives,
product representatives, and decision makers) and IBM sitting at the same table,
in order to be able to successfully outline an engagement for the steps of the grid
implementation and to be able to successfully deploy the strategy that is outlined
during this workshop.

For more information, refer to the following Web site:

http://www.ibm.com/grid/solutions/innovation_workshop.shtml

8.2.2 Grid Strategy Engagement

Grid Strategy Engagement provides the tools for starting most typical grid
engagements.

Description
This service provides the initial steps necessary to evaluate a client's current IT
environment and gives the client the information necessary to alter their overall
IT strategy based on their business needs. This service is also key to typical
follow-on engagements, where a more detailed grid design is developed and
implemented.

The service addresses the following major areas of focus:

򐂰 Engagement and project management
򐂰 Understanding the client business environment and objectives
򐂰 Understanding the current IT environment and plans
򐂰 Future vision definition
򐂰 Development of grid applications, data, infrastructure, and management
strategies
򐂰 Documentation of grid applications, data, infrastructure, and security and
privacy policies and requirements
򐂰 Evaluation of identified applications, data stores, IT infrastructure, and plans
򐂰 Evaluation of the ability to manage a grid environment

Chapter 8. Services and solutions 125

 򐂰 Review of security controls
򐂰 Assessment of application, host, and network security
򐂰 Evaluation of IT services, IT cost, and value, and the current strategy versus
the grid strategy
򐂰 Evaluation of alternative solution approaches and organizational change
readiness
򐂰 Initiative identification, justification, and phasing
򐂰 Transition planning

Using this service will allow the client to arrive at a logical and defensible plan for
using grid technologies to support and enhance the client business plans.
Positioned at the front of the full life cycle of grid services, it is followed by 8.2.3,
“Grid Design Services” on page 126 and 8.2.4, “Grid Software Implementation
Services” on page 127. It also provides a thorough review of all aspects of the
client IT environment and is also a way of linking the client business strategy with
a complete IT strategy, including the client IT and business organization: systems
management, security, applications development, organizational change, testing,
and business continuity.

8.2.3 Grid Design Services

Grid Design Services provides the means for designing custom grid
environments.

Description
This service begins with the grid solution architecture overview and is narrowed
to two progressively more detailed levels of design granularity:
1. First level design: The grid macro design provides a product free design that
may also include the design of processes and a high level organizational
structure.
2. Second level of design: The grid micro design is the physical or node level
design that provides a more detailed product mapping overview of the grid
architecture.

If processes are included in the scope of the design, the micro design will also
include process workflow and a detailed design of the organization that will
implement the processes. The intent of the grid micro design is to produce a
design in enough detail to implement the grid solution.

These design phases provide the steps necessary to design a grid from the
logical architecture level, down to a detailed design where vendor products are

126 Grid Computing Products and Services

 selected. These steps will also provide tools to follow-on phases where the grid is
implemented and operated.

Grid Design Services addresses the following major areas of focus for a
successful grid design:
򐂰 Engagement and project management
򐂰 Understanding the enterprise business environment and objectives as well as
the current IT environment and plans
򐂰 Documenting grid application and infrastructure requirements
򐂰 Providing architectures describing grid applications, middleware,
infrastructure, and security
򐂰 Designing macro level organization and technology solutions
򐂰 Development of physical models for applications, grid middleware, grid
infrastructure, and grid management tools
򐂰 Conducting static testing
򐂰 Training and user support definition
򐂰 High level implementation planning

Positioned at the mid-point of the life cycle of grid services, following 8.2.2, “Grid
Strategy Engagement” on page 125 and followed by 8.2.4, “Grid Software
Implementation Services” on page 127, Grid Design Services provides a way of
linking the grid infrastructure to other aspects of the enterprise IT and business
organization at key design decision points, like systems management, security,
applications development, organizational change, testing, and business
continuity.

8.2.4 Grid Software Implementation Services

Grid Software Implementation Services provide detailed planning and physical
installation of a grid environment. It offers the ability to implement custom grid
solutions using proven project planning principles.

Description
The phases of the service provide the steps necessary to implement a grid
environment, from the initial project planning, through piloting the environment,
and completing with the migration into operations.

Chapter 8. Services and solutions 127

 The grid implementation service addresses the following major areas of focus for
a successful grid solution:
򐂰 Engagement and project management
򐂰 Detailed implementation planning
򐂰 Non-production environment preparation
򐂰 Infrastructure management procedure preparation
򐂰 System components assembly
򐂰 Testing preparation
򐂰 Pilot deployment
򐂰 Transition planning
򐂰 Infrastructure and application customization
򐂰 Data migration
򐂰 Production deployment
򐂰 Engagement completion
򐂰 Delivery

Using this service will allow a client to implement previously planned applications
modifications and install a grid infrastructure.

Positioned at the end of the life cycle of grid services following 8.2.2, “Grid
Strategy Engagement” on page 125, and 8.2.3, “Grid Design Services” on
page 126, Grid Software Implementation Services provides the tools to
implement the grid environment achieved from the planning in previous services.

8.2.5 Grid Training and Education

Grid Training and Education is a service offered by IBM IT education services
that provides clients with the know-how needed to successfully develop,
implement, manage, and operate a grid environment. There are several grid
related courses and lectures available. For more information, visit:
http://www.ibm.com/services/learning/us/catalog/

8.2.6 IBM support line

IBM offers enterprise-grade support for almost every component that can be
used in a grid solution, via the Internet and voice support 24 hours a day, 7 days
a week. This support ranges from answering usage questions to identifying
problems, all revolving around the client’s needs. IBM offers world-class support
as a standard offering, with several options, including the ability to customize

128 Grid Computing Products and Services

 support according to the skills and experience of the client’s current IT staff.
Support line is IBM’s support offering for e-infrastructure software. It is an
established worldwide proven service that covers IBM and non-IBM software
products. For more information about the IBM support line, refer to the following
Web site:
http://www.ibm.com/services/its/us/supportline.html

8.3 Solutions portfolio for grid computing

Table 8-2 provides an overview of the IBM grid solutions, pointing out the
advantages for each offering.

Table 8-2 Grid solutions overview

Solution Description

IBM Grid Value at Work IBM's approach to grid value quantification

Analytics Acceleration Reduce time and effort required to analyze data; accelerate
product development, client insight applications, and
accelerate time-to-results

Design Collaboration Leverage idle hardware, speed data capture, and virtualize
disparate systems

Engineering Design Reduce time to market, leverage idle hardware, speed data
capture, and virtualize disparate systems

Geophysical Processing Improve efficiency, speed on seismic data analysis, and

and Analysis enhance competitiveness and agility

Information Accessibility Maximize exploitation of existing data resources and assets,

achieve unified data access and simplified integration,
secure and simplify data access, and optimize storage and
integration of heterogeneous resources

IT Optimization Share large-scale raw data among researchers and across

disciplines; simplify data access and integration in a flexible,
extensible infrastructure

Research Collaboration Reduce overhead and increase capacity, flexibility,

scalability, resiliency; exploit underutilized computing,
storage, and human resources

Grid Clash analysis Optimize and accelerate clash analysis

Chapter 8. Services and solutions 129

8.3.1 IBM Grid Value at Work
Before engaging a grid project, clients need to assess the value of their
investment. IBM Business Consulting Services, using this offering, can provide
quantifiable business value output to help businesses meet the challenges that
face them.

Description
This is an assessment offering. Its main goals are to predict the business value of
the investment in a grid environment, validate the proposed grid model,
determine the optimal grid blueprint, and monitor IT level indicators of financial
value to efficiently manage the infrastructure.

IBM Grid Value at Work assists in developing a grid deployment roadmap. The
roadmap includes components for total cost of ownership analysis, grid capacity
planning, and business value estimation. It provides the flexibility to focus on one
or more of these components to determine the appropriate investment levels for
grid computing.

Additionally, the IBM Grid Value at Work service helps identify the grid’s most
appropriate uses by analyzing the comprehensive business value of the
implementation across all deployment stages and quantifying the following
information:
򐂰 Business value of improving the speed, accuracy, frequency, or throughput of
the analysis of results for business analytics
򐂰 IT infrastructure cost savings and potential benefits from faster time to market
or improved design quality for research, engineering, and design
򐂰 Cost savings from virtualization of the IT infrastructure and value derived from
exploiting existing, underutilized resources for IT infrastructure optimization
򐂰 Cost savings from integrated information access and shared usage of IT
resources across departments and organizations

This offering provides a deep examination of the critical value components

examined by an executive staff. In particular, it features four analysis modules
that can be used collectively or individually:
1. Grid capacity planner: Helps to identify the applications that are best
positioned to work in a grid
2. TCO estimator: Helps to identify the expected cost savings, as well as the
expected expenses of a deployment
3. Business value estimator: Helps to identify opportunities for companies to
extract the maximum value out of applications on a grid

130 Grid Computing Products and Services

 4. Business case builder: Incorporates outputs from other modules within the
Grid Value at Work tool to present a comprehensive, view of the proposed
grid

With Grid Value at Work, companies can establish strong business cases to
justify investment on a grid and help the clients to meet their business needs in a
more competitive world.

8.3.2 Analytics Acceleration

The Analytics Acceleration offering allows businesses to run their existing
analytics software, whether custom-built management systems, commercial
applications, or a combination of both. It is an open and flexible architecture
supporting a wide range of packaged and custom analytical applications. It can
include SAS Credit Scoring Application software from SAS
(http://www.sas.com/) and relies on grid middleware from DataSynapse to
create distributed sets of virtualized resources.

Description
Working with partners in its industry, IBM has built a grid offering that can deal
with the most demanding applications today. Clients can now reach a new level
of analysis with much more accurate results and yet with less investment in
hardware resources.

This offering is particularly optimized for computing intensive tasks, mainly for the
financial sector. Its main characteristic is that it intends to use every kind of
hardware resources that the infrastructure provides: servers, clusters,
mainframes, and desktops.

Figure 8-2 on page 132 provides an overview of a typical architecture for the
Analytics Acceleration offering.

Chapter 8. Services and solutions 131

 Grid Servers

Avaki / DataSynapse

Resources Resources
Pools Pools

SAS - Credit Scoring SAS - Credit Scoring

Application Application

DB2 Information Integration

Unstructured Structured
Data Data

Figure 8-2 Analytics Acceleration architecture

8.3.3 Design Collaboration

The Design Collaboration offering aims to leverage idle hardware, speed data
capture, and virtualize disparate systems.

Description
Most automobile and aerospace manufacturers use a leading edge design tool
that has many complex features, but some partners may not want to purchase
and maintain multiple design tools for different projects. Using another tool might
require additional testing and analysis by design partners and the original
equipment manufacturer, adding time and cost to the project. Also, most

132 Grid Computing Products and Services

 participants in the automotive and aerospace design process must work with
restricted IT budgets, and some design partners may have limited access to
highly trained IT experts.

This service provides tools to help clients in automotive and aerospace

engineering benefit from a grid implementation, and it can help reduce the time
required to capture, compile, and analyze research data and accelerate product
development.

The solution also includes a Grid Innovation Workshop, a pilot implementation,

and a complete portfolio of IBM Global Services solutions.

8.3.4 Engineering Design

In an intensely competitive marketplace, automobile and aerospace companies
must achieve faster time to market by decreasing the turnaround time for product
design.

The Engineering Design offering aims to reduce time to market, leverage idle
hardware, speed data capture, and virtualize disparate systems.

Description
Engineering Design uses the latest grid technologies, including the Open Grid
Services Architecture (OGSA), which merges the open protocols used for grid
computing with the protocols used for Web services. It also provides the latest
software from leading grid software companies, such as Platform Computing,
and the latest open source grid technologies from The Globus Alliance.

Figure 8-3 on page 134 provides an overview of a typical architecture for the
Engineering Design offering.

Chapter 8. Services and solutions 133

 Figure 8-3 Engineering Design architecture

8.3.5 Geophysical Processing and Analysis

As processing requirements continue to grow, more and more petroleum
companies are taking advantage of parallelization and clustering technologies.
Grid computing can be used to facilitate application parallelization in which an
application is divided into independent tasks that can then be executed
concurrently, reducing the time to completion. Clusters link separate nodes to
create a single, powerful, and scalable system, and grid technologies can help to
increase the overall compute resource sharing and utilization in such clusters.

The Geophysical Processing and Analysis offering aims to improve efficiency

and speed on seismic data analysis in order to enhance competitiveness and
agility.

134 Grid Computing Products and Services

Description
IBM has a comprehensive approach to help determine the most appropriate
combination of technologies for geophysical processing and analysis in the
petroleum markets. WebSphere Web application servers, advanced computer
cluster offerings, powerful database technologies, application integration
approaches, and knowledge management techniques from IBM can help reduce
the time and effort required to capture, compile, and analyze seismic and
reservoir data, which can accelerate asset decision time.

Application parallelization can facilitate grid computing when used for

geophysical processing and analysis. We use industry-standard messaging
technologies, such as the Message Passing Interface, OpenMP, Parallel Virtual
Machine, and pthreads, to parallelize applications.

IBM’s relationships with Platform Computing, DataSynapse, and United Devices

allow clients to use a single product for application parallelization and distributed
execution. These companies offer products that provide job and task-level
scheduling for low-overhead job submission as well as for interactive
computations. It also offers job and task-level schedulers, along with the Globus
Toolkit, to support commercial application environments.

Figure 8-4 on page 136 provides an overview of a typical architecture for the
Geophysical Processing and Analysis offering.

Chapter 8. Services and solutions 135

 Figure 8-4 Geophysical Processing and Analysis architecture

8.3.6 Information Accessibility

The Information Accessibility offering helps to reduce the time and effort required
to capture, compile, and analyze research data, and do product development.

Description
IBM has scalable tools and systems to help life sciences organizations and
governments benefit from a grid implementation. The offering can achieve a
seamless integration of data and resources and leverage existing IT
environments to provide centralized data access from multiple data sources.

136 Grid Computing Products and Services

 WebSphere Information Integration family provides a virtualized data access for
research, educational, and governmental entities by unifying and merging data
from several heterogeneous data sources, and offering this data in a virtualized
layer that applications can access without being concerned with where, how, and
when this data is gathered and stored.

8.3.7 IT Optimization
In a global economy, businesses increasingly rely on IT infrastructures that drive
business strategy, evolution, and success. Streamlining these components and
the supporting organizational processes is an essential yet complex undertaking,
which can be accomplished through grid computing.

The IT Optimization offering helps to share large-scale raw data among

researchers and across disciplines; it simplifies data access and integration.

Description
Grid computing can enable consolidation of applications, networks, and other
data and can help reduce tension between existing systems and politics between
departments. This new approach can help organizations share resources and
improve resource management. Grid computing can and should be used to
optimize existing traditional IT environments, leveraging management and
monitoring automated provisioning and building a on demand computing
environment. Using the tools and benefits provided by grid computing, clients
can leverage their existing IT environment to a more productive level, increasing
the return of the investment in these IT environments.

8.3.8 Research Collaboration

In response to the challenges of information access, research organizations and
universities are redefining their information sharing methodologies, as well as
redesigning their IT infrastructures to be more effective in the new research
environment.

The Research Collaboration offering helps to reduce resource overhead and

increase capacity, flexibility, scalability, and resiliency.

Description
Grid computing can help these organizations manage large-scale data sharing.
This new approach helps maximize the use of existing data resources and
makes both structured and unstructured data available across a department,
organization, or consortium. In addition, grid computing can help researchers
secure data access and optimize storage; plus, IBM has scalable tools and
systems to help research organizations and universities benefit from a grid

Chapter 8. Services and solutions 137

 implementation for research collaboration. To address researchers’ special
security needs, this service also offers a complete line of security and systems
management solutions to support a certifiable, multilevel security infrastructure
using Tivoli Management products.

Figure 8-5 provides an overview of a typical architecture for the Research

Collaboration offering.

Figure 8-5 Research Collaboration architecture

138 Grid Computing Products and Services

8.3.9 Grid Clash Analysis
Waiting until the product is complete to run clash tests creates delays in getting
the product to market and limitations on the number of clash analysis jobs, which
creates quality issues.

The Grid Clash Analysis offering helps to optimize and accelerate clash analysis.

Description
This offering is designed to offer an increase of productivity to existing clash
analysis environments. Typically, clash analysis is a very time consuming part of
the design process for, for example, cars, aircraft, and other equipment. Using
grid, IBM can provide a noticeable improvement to this process, providing
increasing advantages to its clients.

Services for implementing the clash offering may include:

򐂰 Assessing the current usage of clash analysis in the client environment (Grid
Strategy Assessment)
򐂰 Determining any software gaps (Grid Application Readiness Assessment)
򐂰 Planning the pilot implementation (see 8.2.1, “Grid Innovation Workshop” on
page 124)
򐂰 Implementing the pilot using IBM Global Services custom code (Custom
Application Development)
򐂰 Implementing performance enhancement steps in the grid scheduler
environment (see 8.3.7, “IT Optimization” on page 137)
򐂰 Verifying performance results and documenting open issues (Testing
Services)
򐂰 Transferring skills to client personnel (see 8.2.5, “Grid Training and Education”
on page 128)
򐂰 Defining the next steps to implement a production grid (see 8.2.2, “Grid
Strategy Engagement” on page 125)

8.3.10 Summary
Figure 8-3 on page 140 presents the availability of the above offerings’
availability by industry. Some of offerings are customized for specific industries
and others are in fact cross-industry.

Chapter 8. Services and solutions 139

 Table 8-3 Offering availability by industry
Financial Petroleum Aerospace Life
Services and Energy Automotive Sciences,
Industry Research,
and
Government

Analytics Offering Offering Offering Offering

Acceleration Available Available Available Available

Design Offering Offering

Collaboration Available Available

Engineering Offering
Design Available

Geophysical Offering Offering

Processing Available Available
and Analysis

Information Offering Offering

Accessibility Available Available

IT Offering Offering Offering Offering

Optimization Available Available Available Available

Research Offering
Collaboration Available

Clash Offering
Analysis Available

IBM Grid Offering Offering Offering Offering

Innovation Available Available Available Available
Workshop

For complete list of the grid computing offerings, refer to the following Web site:
http://www.ibm.com/grid/solutions/index.shtml

8.4 Services approach to Grid Projects

IBM applies the best methods and practices to access, plan, and implement the
solutions and services designed.

These solutions and offering should be focused on the business need and should
be addressed with a solution independent approach and oriented to offer the
best business value throughout the project’s life.

140 Grid Computing Products and Services

The evaluation of the grid middleware options should be focused on a “best tool
for the job” basis and the use of this middleware should be balanced with the grid
middleware ISV and IBM SWG Product Set to provide the best solution for client
satisfaction.

Chapter 8. Services and solutions 141

142 Grid Computing Products and Services
 Part 5

Part 5 Appendixes

© Copyright IBM Corp. 2005. All rights reserved. 143

144 Grid Computing Products and Services
 A

Appendix A. Surrounding initiatives

This appendix provides the following:
򐂰 Initiatives that contribute to the grid community by sharing resources,
programs, development tools, and open-source products.

© Copyright IBM Corp. 2005. All rights reserved. 145

World Community Grid
The World Community Grid’s mission is to create the largest public computing
grid benefiting humanity (see Appendix B, “World Community Grid” on
page 149). The work is built on the belief that technological innovation combined
with visionary scientific research and large-scale voluntaryism can change our
world for the better. For more information, please refer to the following Web site:
http://www.worldcommunitygrid.org/

Globus Alliance
The Globus Alliance conducts research and development to create fundamental
technologies for grid computing. The alliance is formed by a group of sponsors
and collaborators from around the world. The core team is based at the Argonne
National Laboratory and other worldwide institutions (see
http://www.globus.org/about/team.html for more information).

The Globus Toolkit is being developed by the Globus Alliance and many others
all over the world. For more information, please refer to the following Web site:
http://www.globus.org/

Global Grid Forum

The Global Grid Forum (GGF) is a community-initiated organization of thousands
of active participants from industry and research. GGF's primary objectives in
this organization is to promote development, deployment, and implementation of
grid technologies through creation of technical specifications, user experiences,
and implementation guidelines. For more information, please refer to the
following Web site:
http://www.gridforum.org/

146 Grid Computing Products and Services

OASIS and WSRF TC
Organization for the Advancement of Structured Information Standards (OASIS)
is an international consortium that drives the development and adoption of
e-business standards and technologies. For more information, please refer to the
following Web site:
http://www.oasis-open.org/

The OASIS Web Services Resource Framework (WSRF) Technical Committee

(TC) works on the definition of a generic, royalty-free, open framework for
modeling and accessing stateful resources (required for grid computing) using
Web services. For more information, please refer to the following Web site:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf

W3C
The World Wide Web Consortium (W3C) is an international consortium that
develops specifications and guidelines for Web technologies. For more
information, please refer to the following Web site:
http://www.w3.org/

TeraGrid
TeraGrid is a initiative to build and deploy the world's largest, distributed
infrastructure for open scientific research. It combinations three programs:
National Science Foundation (NSF) Terascale initiative: Terascale Computing
System (TCS®), Distributed Terascale Facility (DTF) and Extensible Terascale
Facility (ETF). For more information, please refer to the following Web site:
http://www.teragrid.org/
http://www.nsf.gov/

Appendix A. Surrounding initiatives 147

148 Grid Computing Products and Services
 B

Appendix B. World Community Grid

This appendix provides the following:
򐂰 Informtion about the IBM World Community Grid initiative, whose purpose is
to contribute to humanitarian projects by providing unused compute cycles
through a world-scale grid.
򐂰 An overview of the grid architecture design and of the organization that would
be the most efficient one to leverage the World Community Grid.

© Copyright IBM Corp. 2005. All rights reserved. 149

Motivations
Over the last few years, grid computing has established itself as an invaluable
technology to speed up highly-demanding computing tasks. By joining together
individual computing resources, it creates a large system with massive
computational power. Despite its embryonic state, a number of sectors, such as
life science, finance, geology, and industry, already benefit from this technology.

But beyond business concerns, grid computing also provides a way for
volunteers to donate unused processing time on their computers, to solve large
and complex problems for the benefit of humanity. Discovering treatments for
killer viruses, delivering more accurate forecasts of looming natural disasters,
and helping researchers identify cures for widespread diseases are only a few
ways of how individual computers could be used in this scenario.

Being firmly convinced that grid computing is poised for philanthropic initiatives
around the world, IBM is committed to providing the technical infrastructure
required to support a philanthropic world-scale grid: the World Community Grid.
The vision of this initiative is to create the world's largest public computing grid to
tackle carefully selected humanitarian projects. The name was chosen to
represent the endeavor's reach (worldwide), volunteer and scientific collaboration
(community), and technical underpinnings (grid computing).

The motivation behind the World Community Grid is leadership. IBM and a group
of leading philanthropic foundations will provide this leadership, crucial expertise,
resources, and guidance, and the philanthropic missions of these foundations will
be strengthened by the power of grid computing.

The official Web site of the World Community Grid can be found at:
http://www.worldcommunitygrid.org/

Technical features
To match expectations and provide a high quality of service, the World
Community Grid has the following properties:
򐂰 Desktop grid model
򐂰 Easy to volunteer
򐂰 High-performance
򐂰 Reliable
򐂰 Scalable
򐂰 Secure
򐂰 Entertaining

150 Grid Computing Products and Services

Desktop grid model
Basically, this translates into a grid architecture featuring:
򐂰 An Agent, that is, a program running on contributing computers to grid-enable
them and take advantage of their unused processing cycles when idle
򐂰 A Server, which is in charge of dispatching the jobs and collecting the results

Easy to volunteer
Previous large-audience PC grids projects have established that most volunteers
are driven, first, by the sense that they are contributing their resources to a good
cause and, second, by the ease, both functional and financial, of contributing the
PC's power.

High-performance
Considering the complexity of the problems to be submitted by the research
community, and the huge processing power their resolution implies, the World
Community Grid must be designed to:
򐂰 Store very large amounts of data
򐂰 Avoid bottlenecks and minimize latency time
򐂰 Handle over one million of contributing computers from the launch

Reliable
The World Community Grid must be reliable in terms of:
򐂰 Availability of the platform
Arrangements must be made to avoid any interruption of service (server
redundancy, 24x7 technical support).
򐂰 Dependability of the returned results
Computing tasks should be submitted a number of times, to several
computers, and returned results compared with each other to make sure that
faulty computations are discarded.

Scalable
As a large part of the 650 million PCs worldwide is expected to participate in the
future, the World Community Grid must anticipate a far larger number of
contributors. As a corollary, such a computing power increase implies that the
World Community Grid will have the ability to handle several grid-enabled
research projects, simultaneously deployed on the same platform.

Appendix B. World Community Grid 151

 Secure
Security aspects have to be taken into account for both the World Community
Grid Agent and the servers:
򐂰 The Agent must not represent a security issue for contributing individuals in
any way, and should contain a public key to enforce the validity of incoming
data and computation code.
򐂰 On the other hand, the grid-enablement process of a research project should
include an encryption phase of all data to be processed, as well as the
computation code sent to the agents.

Of course, since the World Community Grid infrastructure is accessed from the
Internet, drastic additional security measures must be considered to prevent any
server from being compromised (firewall and other security devices).

Entertaining
Finally, the World Community Grid should provide potential contributors with
various collateral features to encourage participation, and to back up
communication efforts aimed at advertising the World Community Grid:
򐂰 A few measures for the participants to evaluate their contribution:
– Total CPU time contributed
– Number of tasks completed
– An arbitrary system of points, based on the computing power of the device
and on the time spent computing results
򐂰 A user-friendly interface for the agent, including a task execution progress bar
and the measures described above; the Agent should also come with a
screen saver displaying the computer’s World Community Grid-related
activity.
򐂰 A Web site (http://www.worldcommunitygrid.org) to showcase the World
Community Grid and to:
– Spread the word about the World Community Grid and about the research
projects currently using it
– Register contributors and make the Agent available for download
– Provide additional entertaining features, some of them collaborative, such
such as team creation, forums, global statistics, and so on

152 Grid Computing Products and Services

Organization
As presented in Figure B-1, four components can be identified that support the
World Community Grid project: funding, technical resources, an advisory board
of scientists and grid specialists, and a communication plan to promote the use of
the World Community Grid by the widest possible population.

Figure B-1 World Community Grid - Basics

Funding
The World Community Grid will initially be sponsored by IBM and various
foundations invited to join the effort, to cover the following activities:
򐂰 Infrastructure hosting with IBM Global Services, including all necessary
hardware and software
򐂰 The development and ongoing maintenance of the World Community Grid
Web site
򐂰 Application preparation and loading of the individual research projects onto
the grid
򐂰 Ongoing support of the World Community Grid community through forums,
online content, and e-mail
򐂰 Public relations and marketing activities to attract World Community Grid
volunteers and research projects

The ultimate goal will be to establish an independent, self-sustaining, and

not-for-profit organization.

Technical resources
These resources include hardware, software, and qualified people to operate
them, as developed in the previous section.

Appendix B. World Community Grid 153

 World Community Grid Advisory Board
A board of advisors chaired by an IBM executive will search for research
challenges that could improve the health and safety of human beings. This board
will be composed of prominent scientists or representatives from research and
philanthropic organizations, and of grid computing specialists and researchers
who will provide their technical expertise. The advisory board will tackle difficult
decisions: evaluating and prioritizing proposals to tap the grid, balancing
pressing projects like the containment of new infectious agents against the need
to make progress on longer-standing problems, and weighing the merits of “big
science” proposals against requests of individual researchers. It will also develop
specific evaluation criteria based on the overarching objective of improving
human welfare.

Projects will be evaluated and prioritized based on several factors including:

Need Which proposed projects will help populations with the
greatest needs?
Scope Which proposed projects might have the greatest impact,
both in terms of breadth and depth?
Suitability Which proposals are best technologically suited for grid
computing?

IBM's current guidelines for donations will serve as a starting point in determining
the organizations that are eligible to apply for the computational power.

The list of members of the board can be found at:

http://www.worldcommunitygrid.org/about_us/advisory_board.html

Marketing and communication

In order for the World Community Grid to become widely visible, a
communication and media plan will be created and all supporting material will be
provided by the different actors involved in the project.

High level design

From the requirements above, it appears that the IT infrastructure to establish the
World Community Grid is based upon three components:
򐂰 Clients
򐂰 Servers
򐂰 Administrators

154 Grid Computing Products and Services

From a global perspective, the World Community Grid architecture should
resemble the one shown in Figure B-2.

Internet Connected
World Community Grid
Volunteers

Desktop PC

Laptop computer

Laptop computer

Engineering
Workstation

Data

Fibre Attached Storage

Grid Servers

Data
World Community Grid
Server Infrastructure
Application Servers Database Servers

World Community Grid

Web Site Servers

Figure B-2 World Community Grid: High Level Solution Design

On the client side

Large numbers of volunteers who are willing to download and install the grid
desktop program, enabling them to donate the computing power of their idle PCs
and servers.

On the server side

A large server infrastructure to act as the “brains” of the grid.

Appendix B. World Community Grid 155

 The Web servers
The infrastructure that hosts the http://www.worldgrid.org Web site, from
where individuals and organizations can learn more about the projects, register
and download the grid-participant program, and join Web-based forums to
communicate with other volunteers.

In order to handle the heavy load that is expected once several millions of people
are aware of the World Community Grid and start to use it frequently, it has been
decided to split the treatment of static and dynamic pages between two groups of
dedicated servers. The static pages, served by the front-end Web servers,
contain informtion about the World Community Grid and on the research projects
running on it, FAQs, troubleshooting tips, legal mentions, and so on. On the other
hand, the dynamic pages will be served by the back-end application servers and
deal mainly with user-oriented services, such as global, by project and individual
statistics, member authentication, forums, and so on.

The grid servers

These servers will dispatch tasks to grid-enabled computers and will retrieve and
compile the results. They will be in charge of making sure that a task has been
completed and that the returned result is correct, for example, by submitting the
same task to various computers running the World Community Grid Agent, and
by comparing the returned results with one another.

For security reasons, the grid software running on these servers will also keep
track of the submitted tasks so that only computing resources identified as valid
devices registered through the agent interface are allowed to return a result.

The storage resources

Storage is a crucial issue for the World Community Grid, and will have to handle
two types of data.
򐂰 Desktop Grid data
This data can be categorized into application data and user data:
– Grid-enabled applications data
• Computation code
• Pre-processed projects input data
• Global statistics / project advancement
– Users data
• Project subscriptions
• Individual statistics

156 Grid Computing Products and Services

򐂰 Registered members information
These data originate from two sources:
– From World Community Grid contributors
• Through the World Community Grid registration form that every user
has to fill in when installing the World Community Grid Agent (member
name and password, e-mail address for password recovery and
information updates, personal information, and address for statistic by
geography)
• Through the device creation form (resource contribution schedule, type
of internet connection, and so on)
• When using the World Community Grid collaborative features (team
management, forums, and so on)
– From the grid database
• Consolidated individual statistics for integration into the World
Community Grid Web site

In order to balance the load and minimize the risks of server failure, and because
this data is accessed from distinct servers, the decision was made to distribute
the data over two platforms, according to their type.

Figure B-3 on page 158 summarizes all the above considerations.

Appendix B. World Community Grid 157

 Figure B-3 World Community Grid: Architecture

The administration team

Of course, this platform cannot be installed, configured, hosted, and maintained
without a team of qualified IT professionals, who will also take the applications
identified by the World Community Grid advisory board and prepare them to run
on the grid.

Products used
In order to complete the design of the World Community Grid, hardware and
software products have to be chosen for each architectural component.

World Community Grid Web Servers

According to the architecture design, these servers are to be the front ends of the
World Community Grid Web site; their purpose is to serve static pages only, and
forward dynamic pages requests to the World Community Grid Application
Servers.

For reasons of costs and maintenance, the selected servers were IBM ^
xSeries, with Linux as the operating system.

158 Grid Computing Products and Services

As for the Web server software, it seemed a natural choice to go with WebSphere
Application Server; this product may seem inappropriate, since it is to be used
only for serving static pages, but on the other hand, it provides network
deployment and administration features that are a definite value-add, especially
in terms of implementation and hosting of the World Community Grid
infrastructure. Moreover, it would not make much sense to choose different
products for the front-end servers and the back-end servers.

World Community Grid Application Servers

The purpose of these servers is to relieve the front ends of all dynamic content,
including database access and code execution.

For reasons listed above, these servers should be IBM ^ xSeries running
Linux as well.

The WebSphere Application Server is IBM's premier Web application server. It

provides servlet and JSP™ processors, which includes support for JSP
templates. It supports J2EE applications containing Enterprise Java Beans
(EJBs) along with Bean and Container Managed Persistence (BMP/CMP). It also
includes support for Web services through various protocols.

World Community Grid Storage Resources

These servers are to handle all data storage related activity for both the World
Community Grid Web site and the grid-enabled applications.

Since a considerable amount of data is expected, the selected servers to host

both databases were IBM Shark Enterprise Storage Servers running DB2
database software on AIX/pSeries technologies. In addition to their robustness
and proven maturity, they fit perfectly in an all-IBM servers architecture, and can
be easily managed by IBM Managed Storage Services (MSS).

MSS offers flexible and scalable storage networking solutions for midrange
server environments. Storage capacity "on demand," backup/restore, and
management services are available as fee-based services. Hosted storage
resources receive the benefits of Storage Area Network (SAN) and Network
Attached Storage (NAS) technologies as part of an integrated solution.

Appendix B. World Community Grid 159

 World Community Grid platform
These servers are the core of the World Community Grid. In order to make these
servers homogenous with the rest of the infrastructure, they were also specified
as xServers running Linux.

In terms of software, the grid infrastructure product that would be part of the
World Community Grid had to demonstrate sizeable client counts and satisfy as
many of the following criteria as possible:
򐂰 General considerations
– Good level of maturity, especially on open Internet usage
򐂰 Project-related considerations
– Ability to run multiple applications concurrently, depending on the
contributor preferences
– Contributed resource evaluation system included (to keep track of the
points)
– Collaborative features provided (teams)
򐂰 Agent features
– Non-intrusive technology
– Easy installation and configuration of software
– Low disk space, memory, and bandwidth required
– Ability to communicate through a firewall
– Screensaver
– Automatic update of the client software
򐂰 Security issues
– Protection against spoofing and man-in-the-middle attacks
– Communication secured by mutual handshake and encryption

After reviewing several products, and because previous collaborations had

turned out to be definite success (in particular, the Smallpox Research Grid
Project), IBM decided to trust the United Devices MP platform. For more
information, please refer to the following Web site:
http://www.worldcommunitygrid.org/projects_showcase/projects_archive.html

160 Grid Computing Products and Services

Figure B-4 World Community Grid: Architecture with products

More information
To conclude this presentation of the World Community Grid, we thought that it
would be relevant to consider some aspects of the project from a contributor
standpoint, in particular the visible part of the World Community Grid as it is
today.

World Community Grid Agent

The Agent is the program that runs on every contributing resource. Its interface
displays the progress of the task currently being processed, the estimated total
points of the device, and its properties as a computing resource, among other
information. Communications between the Agent and the World Community Grid

Appendix B. World Community Grid 161

 Servers are encrypted and authenticated using a public key system, which
prevents falsification of data by third parties masquerading as legitimate users,
as well as delivery of a computer virus. Additionally, the user can configure the
Agent so that it does not use more hard disk than specified and can limit
processing and communications to certain time windows.

Figure B-5 World Community Grid Agent Interface

Example of the World Community Grid Screen Saver

When a contributing computer goes idle, the World Community Grid Screen
Saver requisitions resources as permitted by the computer owner to resume its
work, while providing graphical insights on the task being processed, as shown in
Figure B-6 on page 163.

162 Grid Computing Products and Services

Figure B-6 World Community Grid Screen Saver

www.worldcommunitygrid.org
Below are a few screen captures of the actual the World Community Grid Web
site; this is solely intended as a glimpse of what kind of information should be
provided to encourage potential participants, and to illustrate some of the
previously discussed features.

The World Community Grid home page (Figure B-7 on page 164) gives quick
access to the main sections of the Web site, and underlines the message the
World Community Grid is conveying: “You can help change the world”.

Appendix B. World Community Grid 163

Figure B-7 World Community Grid Web Site - Homepage

The World Community Grid Web site provides visitors with global in-depth
statistics on the community contribution. These stats can be consolidated by
various criteria, in particular by projects (see Figure B-8 on page 165).

164 Grid Computing Products and Services

Figure B-8 World Community Grid Web Site - Project Statistics

The first project is known as Human Proteome. Human Proteome, in short, is a

study that includes a method of determining the final shape and function of
proteins to ultimately combat a number of diseases. Human Proteome is not an
end in itself, but rather it is a stepping stone to potentially provide a cure for a
variety of diseases.

The World Community Grid Web site has a member area, for registered
contributors to measure their individual contribution and manage their devices.

Appendix B. World Community Grid 165

 Statistics of the teams they belong to are also displayed in this area (see
Figure B-9).

Figure B-9 World Community Grid: Member Area

166 Grid Computing Products and Services

Conclusion
The World Community Grid has been designed as an ambitious project from the
start, and is likely to expand at a fast rate as more and more people around the
world join the community of contributors.

In terms of measures, the goal is to enlist two million computers in the World
Community Grid in the first year, running up to six applications, increasing that to
eight million computers in the third year with more applications.

Successfully harnessing the value of grid computing on a grand, philanthropic

scale requires three elements. Two of them – the technical infrastructure and the
massive volunteer population – exist today. The third element, a guiding
organization tapping these powerful forces for the greater good of humanity, is
the World Community Grid.

Appendix B. World Community Grid 167

168 Grid Computing Products and Services
 C

Appendix C. Products reference

This appendix provides the following:
򐂰 A list of some products that can be part of any grid computing solution
򐂰 A briefing of each product, including our analysis of its functionality
򐂰 Links for more information about products

© Copyright IBM Corp. 2005. All rights reserved. 169

Big picture
Theoretically, any product can be used in a grid solution. The products presented
here are the most popular ones or the grid-enabled ones. There is no product
that by itself can provide a complete (or turn-key) grid solution; therefore, the
product features must complement each other in order to build a grid
environment. The product choice depends on the IT architect and the analysis of
the functionality to match the architecture needs.

Note: There is no magic formula to consider if a product can be used in grid

solutions or not. It depends on the functionality of the product, and the
functionality will determine if the product is suitable for the environment.

Products covered in this redbook support (or not) the functions described in
1.2.2, “Functionality on grid computing” on page 16. Figures presented in this
appendix illustrate our understanding of the functions each product can
provide in a grid environment. It is not a prescription, but just a big picture
about each product.

The width of each bar (0 to 3) reflects our perception of the importance of the
product in a grid computing environment. Figures are to be read as follows:
򐂰 0: The product has no features of this discipline.
򐂰 1: The product has some features of this discipline.
򐂰 2: The product has most of the features of this discipline.
򐂰 3: The product was developed and designed to completely cover the
discipline.

Important: The following Web site provides a search capability to access all
IBM announcement letters, part numbers, sales manual, press releases,
Redbooks, white papers, and catalogs:
http://www.ibm.com/common/ssi

Products per functionality

The following tables present the products distributed by functionality, but the
products’ information is placed in alphabetic order.

Web services core/hosting

Web services core/hosting products presented in this appendix are listed in
Table C-1 on page 171.

170 Grid Computing Products and Services

Table C-1 Web Services Core/Hosting
Products Distributor

WebSphere Application Server IBM

WebSphere Portal Server IBM

WebSphere MQ IBM

WebSphere Extended Deployment IBM

Workload virtualization
Workload virtualization products presented in this appendix are listed in
Table C-2.

Table C-2 Workload virtualization

Products Distributor

Tivoli WorkLoad Scheduler IBM

LoadLeveler IBM

Information virtualization
Information virtualization products presented in this appendix are listed in
Table C-3.

Table C-3 Information virtualization

Products Distributor

WebSphere Information Integrator IBM

DB2 Content Manager IBM

IBM General Parallel File System (GPFS) IBM

TotalStorage SAN File System (SAN-FS) IBM

SAN Volume Controller (SVC) IBM

Appendix C. Products reference 171

 Other disciplines
Products for provisioning that are presented in this appendix are listed in
Table C-4.

Table C-4 Provisioning

Products Distributor

Tivoli Provisioning Manager IBM

Tivoli Intelligent Orchestrator IBM

Products for security that are presented in this appendix are listed in Table C-5.

Table C-5 Security

Products Distributor

Tivoli Access Manager IBM

Monitoring products for monitoring that are presented in this appendix are listed
in Table C-6.

Table C-6 Monitoring

Products Distributor

Tivoli NetView® IBM

Tivoli Monitoring IBM

Tivoli Management Framework IBM

Tivoli Enterprise™ Console IBM

Tivoli Service Level Advisor IBM

The products for licensing that are presented in this appendix are listed in
Table C-7.

Table C-7 License management

Products Distributor

Tivoli License Manager IBM

172 Grid Computing Products and Services

DB2 Content Manager
DB2 Content Manager is a product for storing and managing digital media, such
as images, video, sound, and non-structured data that has to be related in a
structured way.

Figure C-1 and Table C-8 on page 174 present the importance of DB2 Content
Manager in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-1 DB2 Content Manager

Appendix C. Products reference 173

 Table C-8 DB2 Content Manager
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information IBM DB2 Content Manager provides content management for

virtualization images, graphics, video, Web content, and compliance
documents. Apart from that, it also has the capability to provide
imaging, digital asset management, Web content management,
and content integration.

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.ibm.com/software/data/cm/

GPFS
IBM General Parallel File System (GPFS) is a file system designed to have high
performance and high availability. Originally developed for RS/6000 SP clusters,
it has been ported to Linux and is currently used as one of the first choices in grid
solutions for distributed data storage.

Figure C-2 on page 175 and Table C-9 on page 175 present the importance of
GPFS in the grid computing environment.

174 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-2 GPFS

Table C-9 GPFS

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information Allows parallel access to multiple different physical storage within

virtualization the file system. It is designed as a high throughput file system
with global name space.

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.ibm.com/servers/eserver/clusters/software/gpfs.html
http://www.ibm.com/servers/eserver/pseries/software/sp/gpfs.html

Appendix C. Products reference 175

LoadLeveler
IBM LoadLeveler is a High Performance Computing job management system that
allows users to optimize workload execution and performance on pool of IBM
^ pSeries machines by matching the jobs processing needs with the
available resources. LoadLeveler schedules jobs in a HPC environment, and
provides functions for submitting and processing jobs quickly and efficiently in a
dynamic environment.

Figure C-3 and Table C-10 on page 177 present the importance of the IBM
LoadLeveler in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-3 LoadLeveler

176 Grid Computing Products and Services

 Table C-10 LoadLeveler
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload Supports gang scheduling, which implements time sharing

virtualization features that allows multiple jobs to execute simultaneously on a
group of nodes, and also allows multiple jobs to timeshare on a
single node.

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Provides GUI management for submitting jobs

For more information, see:

http://publib.boulder.ibm.com/clresctr/windows/public/llbooks.html

Tivoli Access Manager

Tivoli Access Manager is a security product that provides a single point of user
authentication and authorization administration. Access Manager provides a
single point of authentication and authorization to Web-based resources, and
provides standards-based APIs that allow Web application servers to access
Tivoli Access Manager security services.

Figure C-4 on page 178 and Table C-11 on page 178 present the importance of
Tivoli Access Manager in the grid computing environment.

Appendix C. Products reference 177

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-4 Tivoli Access Manager

Table C-11 Tivoli Access Manager

Discipline Description

Web Services There are APIs and configuration tools.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Implements a strong security framework that can be used in

traditional IT and grid.

For more information, see:

http://www.ibm.com/software/tivoli/products/access-mgr-e-bus/

178 Grid Computing Products and Services

Tivoli Enterprise Console®
Tivoli Enterprise Console is the focal point of events from all Tivoli products.

Figure C-5 and Table C-12 present the importance of Tivoli Enterprise Console
in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-5 Tivoli Enterprise Console

Table C-12 Tivoli Enterprise Console

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Provides automated problem diagnosis and resolution to improve

system performance and reduce support costs.

Appendix C. Products reference 179

 For more information, see:
http://www.ibm.com/software/tivoli/products/enterprise-console/

Tivoli Intelligent Orchestrator

Tivoli Intelligent Orchestrator extends the benefits of the Tivoli Provisioning
Manager. It intelligently and dynamically issues instructions to Tivoli Provisioning
Manager, which then uses automation workflow to maintain server availability
and meet required service levels. By constantly monitoring resources and
requirements for anticipated peak workloads, it can then trigger the appropriate
response.

Figure C-6 and Table C-13 on page 181 present the importance of Tivoli
Intelligent Orchestrator in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-6 Tivoli Intelligent Orchestrator

180 Grid Computing Products and Services

 Table C-13 Tivoli Intelligent Orchestrator
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Combined with Provisioning Manager, Orchestrator enable grid

environments with intelligent and dynamic provisioning, creating
and managing an automatic provisioning infrastructure.

For more information, see:

http://www.ibm.com/software/tivoli/products/intell-orch/

Tivoli License Manager

Tivoli License Manager automates software licensing and utilization, and
provides advanced reporting and inventory capabilities across an IT
infrastructure.

Figure C-7 on page 182 and Table C-14 on page 182 present the importance of
Tivoli License Manager in the grid computing environment.

Appendix C. Products reference 181

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-7 Tivoli License Manager

Table C-14 Tivoli License Manager

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Provides automatic license management usage and also

reporting and inventory.

For more information, see:

http://www.ibm.com/software/tivoli/products/license-mgr/

182 Grid Computing Products and Services

Tivoli Management Framework
Tivoli Management Framework is the base component for the Tivoli product line.
Using Tivoli Management Framework and a combination of Tivoli Enterprise
applications, administrators can manage large distributed networks with multiple
operating systems, various network services, diverse system tasks, and
constantly changing nodes and users. Tivoli Management Framework provides a
set of common services or features that are used by the Tivoli Enterprise
applications installed on Tivoli Management Framework.

Figure C-8 and Table C-15 on page 184 present the importance of Tivoli
Management Framework in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-8 Tivoli Management Framework

Appendix C. Products reference 183

 Table C-15 Tivoli Management Framework
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Builds a framework as a foundation for a suite of management

applications that make systems and network management easy.

For more information, see:

http://www.ibm.com/software/tivoli/products/mgt-framework/

Tivoli Monitoring
IBM Tivoli Monitoring provides the means to monitor several IT infrastructure
resources. It detects bottlenecks and potential problems. It also provides
automated best practices out of the box that proactively monitor critical hardware
and software, including middleware, applications, and databases.

Figure C-9 on page 185 and Table C-16 on page 185 present the importance of
Tivoli Monitoring in the grid computing environment.

184 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-9 Tivoli Monitoring

Table C-16 Tivoli Monitoring

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Provides monitoring for essential system resources.

For more information, see:

http://www.ibm.com/software/tivoli/products/monitor/

Appendix C. Products reference 185

Tivoli NetView
IBM Tivoli NetView extends traditional network management to ensure the
availability of critical business systems and to provide rapid resolution of
problems. It discovers TCP/IP networks, displays network topologies, correlates
and manages events and SNMP traps, monitors network health, and gathers
performance data.

Figure C-10 and Table C-16 on page 185 present the importance of Tivoli
NetView in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-10 Tivoli NetView

186 Grid Computing Products and Services

 Table C-17 Tivoli NetView
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Extends traditional network management to ensure the

availability of critical business systems and to provide rapid
resolution of problems.

For more information, see:

http://www.ibm.com/software/tivoli/products/netview/

Tivoli Provisioning Manager

IBM Tivoli Provisioning Manager is designed to help automate infrastructure
deployment: It automates the manual provisioning and deployment process using
pre-built "industry best practice" workflows to provide control and configuration of
major vendor products. Tivoli Provisioning Manager automates manual tasks of
provisioning and configuring servers, operating systems, middleware,
applications, storage, and network devices.

Figure C-11 on page 188 and Table C-18 on page 188 present the importance of
Tivoli Provisioning Manager in the grid computing environment.

Appendix C. Products reference 187

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-11 Tivoli Provisioning Manager

Table C-18 Tivoli Provisioning Manager

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Developed to automate every provisioning task.

For more information, see:

http://www.ibm.com/software/tivoli/products/prov-mgr/

188 Grid Computing Products and Services

Tivoli SAN FS
Based on the Storage Tank™ technology developed by IBM research, SAN FS is
designed to provide a network-based heterogeneous file system for data sharing
and centralized policy-based storage management in an open environment. The
SAN File System provides a common file system for UNIX, Windows, and Linux
servers, with a single global namespace to help provide data sharing across
servers. It enables host systems to plug into a common SAN-wide file structure.

Figure C-12 and Table C-19 on page 190 present the importance of Tivoli SAN
FS in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-12 Tivoli SAN FS

Appendix C. Products reference 189

 Table C-19 Tivoli SAN FS
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information SAN FS file system virtualization treats storage as a single,

virtualization common pool with heterogeneous platform support. Storage
space can be allocated and managed dynamically.

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.storage.ibm.com/software/virtualization/sfs/index.html

Tivoli Service Level Advisor

Tivoli Service Level Adviser is a service level management product that simplifies
and automates the process of managing service level agreements. It provides
the tools for proactively managing and reporting on service levels from across the
management infrastructure.

Figure C-13 on page 191 and Table C-20 on page 191 present the importance of
Tivoli Service Level Advisor in the grid computing environment.

190 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-13 Tivoli Service Level Advisor

Table C-20 Tivoli Service Level Advisor

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Designed to provide predictive service level management

capabilities by enabling you to proactively predict when SLA
violations are likely to occur and then take corrective actions to
avoid an SLA violation.

For more information, see:

http://www.ibm.com/software/tivoli/products/service-level-advisor/

Appendix C. Products reference 191

Tivoli Workload Scheduler
IBM Tivoli Workload Scheduler (ITWS) is a calendar based choreographer
scheduler that automates, monitors, and controls the flow of work through an
enterprise's entire IT infrastructure on both local and remote systems. From a
single point of control, the suite analyzes the status of the production work and
drives the processing of the workload according to installation business policies.
It supports a multiple-end-user environment, enabling distributed processing and
control across sites and departments within your enterprise.

Figure C-14 and Table C-21 on page 193 present the importance of Tivoli
Workload Scheduler in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-14 Tivoli Workload Scheduler

192 Grid Computing Products and Services

 Table C-21 Tivoli Workload Scheduler
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload Provides an enterprise-level scheduling solution that enables

virtualization clients to perform cross-platform, cross-domain, and
cross-enterprise scheduling by integrating business applications
across grid, mainframe, HPC, and traditional distributed
environments.

Information No relevant features are available in this discipline.

virtualization

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.ibm.com/software/tivoli/products/scheduler/

TotalStorage SAN Volume Controller

IBM TotalStorage SAN Volume Controller (SVC) is designed to reduce the
complexity and costs of managing storage networks. It allows users to virtualize
their storage and helps increase the utilization of existing capacity and
centralizes the management of multiple controllers in an open-system SAN
environment. It is designed to increase the flexibility of your storage infrastructure
by enabling changes to the physical storage with minimal or no disruption to
applications.

Figure C-15 on page 194 and Table C-22 on page 194 present the importance of
TotalStorage SAN Volume Controller in the grid computing environment.

Appendix C. Products reference 193

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-15 TotalStorage San Volume Controller

Table C-22 TotalStorage San Volume Controller

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information IBM SAN VC provides a virtualization layer between physical

virtualization storages and manages them as a whole. Users can and might be
transparently accessing data spread access different storages,
giving better usage patterns.

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.ibm.com/servers/storage/software/virtualization/svc/

194 Grid Computing Products and Services

Virtualization Engine
Virtualization Engine (VE) provides a logical rather than physical view of data,
computing power, storage capacity, and other resources, while automating the
management of these resources based on business goals.

Figure C-16 and Table C-23 on page 196 present the importance of
Virtualization Engine in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-16 Virtualization Engine

Appendix C. Products reference 195

 Table C-23 Virtualization Engine
Discipline Description

Web Services Has the IBM Grid Toolbox and is complemented by the
Core/Hosting WebSphere family of products.

Workload Provides several scheduling components.

virtualization

Information Implements some functionality through IBM Grid toolbox and can
virtualization be complemented by WebSphere Information Integrator.

Other Disciplines Implements features like provisioning, security, monitoring, and

resource management.

For more information, see:

http://www.ibm.com/servers/eserver/about/virtualization/?ca=vedemot&met=web
&me=escallout

WebSphere Application Server

IBM has integrated grid computing capabilities into WebSphere Application
Server. One of the key capabilities integrated is a “traffic-cop” load balancer,
which routes WebSphere workload traffic across an application server cluster to
deliver optimal performance. WebSphere Application Server provides a run-time
environment that supports WSRF-compliant implementations of OGSA.

Figure C-17 on page 197 and Table C-24 on page 197 present the importance of
WebSphere Application Server in the grid computing environment.

196 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-17 WebSphere Application Server

Table C-24 WebSphere Application Server

Discipline Description

Web Services Designed to implement every feature in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Flexible management and security model with policies and
authentication.

For more information, see:

http://www.redbooks.ibm.com/redpapers/pdfs/redp3740.pdf
http://www.ibm.com/websphere/

Appendix C. Products reference 197

WebSphere Extended Deployment
WebSphere Extended Deployment is optional add-on software technology to
WebSphere Application Server Network Deployment, delivering a new class of
features for distributed platforms. This product allows an application to be split
and run on different application servers, thereby enabling you to execute, in
parallel, subsets of the application logic. As a result, the whole task can be
executed faster than if it were executed in just a single box.

Figure C-17 on page 197 and Table C-24 on page 197 present the importance of
WebSphere Extended Deployment in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-18 WebSphere Extended Deployment

198 Grid Computing Products and Services

 Table C-25 WebSphere Extended Deployment
Discipline Description

Web Services Designed to implement every feature in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Flexible services distribution, management, and security model

with policies and authentication.

For more information, see:

http://www.ibm.com/software/webservers/appserv/extend/features/
http://www.ibm.com/websphere/

WebSphere Information Integrator

WebSphere Information Integrator is a product that provides the means to
integrate information. It provides a SQL interface for applications that need to
access distributed and heterogeneous data in virtually any data source.

Figure C-19 on page 200 and Table C-26 on page 200 present the importance of
WebSphere Information Integrator in the grid computing environment.

Appendix C. Products reference 199

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-19 WebSphere Information Integrator

Table C-26 WebSphere Information Integrator

Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information WebSphere Information Integrator is a data federation software

virtualization that provides structured access to heterogeneous types of data.

Other Disciplines No relevant features are available in this discipline.

For more information, see:

http://www.ibm.com/software/data/integration/

200 Grid Computing Products and Services

WebSphere MQ
WebSphere MQ allows you to exchange information across different platforms,
integrating new and existing business applications. WebSphere MQ Messaging
provides any-to-any connectivity from desktop to mainframe, through business
quality messaging, supporting over 35 platforms.

Figure C-20 and Table C-27 on page 202 present the importance of WebSphere
MQ in the grid computing environment.

Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-20 WebSphere MQ

Appendix C. Products reference 201

 Table C-27 WebSphere MQ
Discipline Description

Web Services No relevant features are available in this discipline.

Core/Hosting

Workload Can provide asynchronous message passing.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Provides a messaging passing mechanism.

For more information, see:

http://www.ibm.com/software/integration/wmq/

WebSphere Portal Server

WebSphere Portal Server provides a single point of access to applications,
application content, processes, and people in your network. It allows you to
establish customized portals for your employees, Business Partners, and clients.
The framework architecture implemented in this product provides a unified
access point to internal and external Web applications, as well as portal access
to other existing applications. In this way, users sign on to the portal and receive
personalized Web pages.

Figure C-21 on page 203 and Table C-28 on page 203 present the importance of
WebSphere Portal Server in the grid computing environment.

202 Grid Computing Products and Services

 Functionality on Grid Computing

Web Services
Core/Hosting

Workload
Virtualization

Information
Virtualization

Other Disciplines

Figure C-21 WebSphere Portal Server

Table C-28 WebSphere Portal Server

Discipline Description

Web Services Portal capability on the top of an application server.

Core/Hosting

Workload No relevant features are available in this discipline.

virtualization

Information No relevant features are available in this discipline.

virtualization

Other Disciplines Integration capability with other applications.

For more information, see:

http://www.ibm.com/software/genservers/portal/
http://www.ibm.com/websphere/

Appendix C. Products reference 203

204 Grid Computing Products and Services
Related publications

The publications listed in this section are considered particularly suitable for a
more detailed discussion of the topics covered in this redbook.

IBM Redbooks
For informtion about ordering these publications, see “How to get IBM Redbooks”
on page 216. Note that some of the documents referenced here may be available
in softcopy only.
򐂰 Fundamentals of Grid Computing, REDP-3613
򐂰 Globus Toolkit 3.0 Quick Start, REDP-3697
򐂰 Grid Computing with the IBM Grid Toolbox, SG24-6332
򐂰 Grid Computing on Research and Education, SG24-6649
򐂰 Grid Services Programming and Application Enablement, SG24-6100
򐂰 Introduction to Grid Computing with Globus, SG24-6895

Other publications
These publications are also relevant as further information sources:
򐂰 Abbas, Grid Computing: A Practical Guide to Technology and Applications,
Charles River Media, 2003, ISBN 1584502762
򐂰 Berman, et al (editors), Grid Computing: Making The Global Infrastructure a
Reality, Wiley, John & Sons, Incorporated, 2003, ISBN 0470853190
򐂰 Cockburn, Writing Effective Use Cases, Pearson Education, 2000, ISBN
0201702258
򐂰 Foster, et al (editors), The Grid 2: Blueprint for a New Computing
Infrastructure, Morgan Kaufmann Publishers, 2003, ISBN 1558609334
򐂰 “GPFS: A Shared-Disk File System for Large Computing Clusters” by Frank
Schmuck, Roger Haskin in Proceedings of the Conference on File and
Storage Technologies, 2002
򐂰 Joseph, et al, Grid Computing, Prentice Hall Professional Technical
Reference, 2003, ISBN 0131456601

© Copyright IBM Corp. 2005. All rights reserved. 205

Online resources
These Web sites and URLs are also relevant as further information sources:
򐂰 Accelrys home page
http://www.accelrys.com
򐂰 Altair Engineering
http://www.altair.com
򐂰 Apple: MacOS X
http://www.apple.com/macosx
򐂰 Cisco Systems
http://www.cisco.com
򐂰 Citrix home page
http://www.citrix.com
򐂰 Community Scheduler Framework
http://sourceforge.net/projects/gcsf
򐂰 Condor
http://www.cs.wisc.edu/condor
򐂰 The CrossGrid Project
http://www.eu-crossgrid.org
򐂰 The DataGrid Project
http://www.eu-datagrid.org
http://www.eu-egee.org
򐂰 DataSynapse
http://www.datasynapse.com/
򐂰 DataSynapse GridServer Technology
http://www.datasynapse.com/solutions/tech.html
򐂰 DB2 Content Management
http://www.ibm.com/software/data/cm
򐂰 WebSphere Information Integration
http://www.ibm.com/software/data/integration
򐂰 Database partitioning feature (DPF)
http://publib.boulder.ibm.com/infocenter/db2help/index.jsp?topic=/com.ibm.d
b2.udb.concepts.doc/concepts/about_dpf.htm

206 Grid Computing Products and Services

򐂰 DB2 Product Family
http://www.ibm.com/software/data/db2
򐂰 DCGrid Platform
http://www.entropia.com/dcgrid_platform.asp
򐂰 DCGrid Product Family
http://www.entropia.com/dcgrid_product_family.asp
򐂰 Design and implementation of an enterprise grid
http://www.research.ibm.com/journal/sj/434/meliksetian.html
򐂰 developerWorks: Analytics Acceleration Grid Environment, Part 1
http://www.ibm.com/developerworks/grid/library/gr-aage1/
򐂰 developerWorks: Analytics Acceleration Grid Environment, Part 2
http://www.ibm.com/developerworks/grid/library/gr-aage2/
򐂰 developerWorks: Build grid portals with Grid Portal Toolkit 3
http://www.ibm.com/developerworks/grid/library/gr-gridport/
򐂰 developerWorks: Community Scheduler Framework to implement grid
meta-schedulers
http://www.ibm.com/developerworks/grid/library/gr-meta.html
򐂰 developerWorks: Community Scheduler Framework to create and submit grid
jobs, Part 2
http://www.ibm.com/developerworks/grid/library/gr-meta2.html
򐂰 developerWorks: Enable existing applications for grid
http://www.ibm.com/developerworks/grid/library/gr-exist/
򐂰 developerWorks: Geographically dispersed grid, Part 1: Aligning computation
and data
http://www.ibm.com/developerworks/grid/library/gr-gdg1/
򐂰 developerWorks: Globus Toolkit 4 Early Access: WSRF
http://www.ibm.com/developerworks/grid/library/gr-gt4early/
򐂰 developerWorks: Grid computing
http://www.ibm.com/developerworks/grid/
򐂰 developerWorks: Grid watch: Column summary
http://www.ibm.com/developerworks/grid/library/gr-watchcol.html
򐂰 developerWorks: The information grid: Article summary
http://www.ibm.com/developerworks/grid/library/gr-infocol.html

Related publications 207

 򐂰 developerWorks: Orchestrating grid workloads -- neither feast nor famine
http://www.ibm.com/developerworks/grid/library/gr-feast/
򐂰 developerWorks: Six strategies for grid application enablement: Article
summary
http://www.ibm.com/developerworks/grid/library/gr-sixcol.html
򐂰 developerWorks: zSeries and iSeries servers in the grid domain
http://www.ibm.com/developerworks/grid/library/gr-ziseries/
򐂰 Develop your grid service with the IBM Grid Toolbox
http://www.ibm.com/developerworks/grid/library/gr-develop
򐂰 Enterprise Identity Mapping
http://www.ibm.com/servers/eserver/security/eim
򐂰 Gartner home page
http://www.gartner.com
򐂰 General Parallel File System for AIX 5L
http://www.ibm.com/servers/eserver/pseries/software/sp/gpfs.html
򐂰 General Parallel File System for Linux
http://www.ibm.com/servers/eserver/clusters/software/gpfs.html
򐂰 Global Grid Forum
http://www.gridforum.org
򐂰 Globus Alliance
http://www.globus.org
򐂰 Globus Alliance Team
http://www.globus.org/about/team.html
򐂰 The Globus Toolkit
http://www-unix.globus.org/toolkit
http://www.globus.org/toolkit
򐂰 Globus Toolkit Security Documentation
http://www-unix.globus.org/toolkit/docs/3.2/security.html
򐂰 GRID.org
http://www.grid.org
򐂰 The Community Scheduler Framework
http://gcsf.sourceforge.net

208 Grid Computing Products and Services

򐂰 Grid Engine
http://gridengine.sunsource.net
򐂰 GridFTP Protocol and Software
http://www-fp.globus.org/datagrid/gridftp.html
򐂰 GridFTP: Protocol Extensions to FTP for the Grid
http://www.globus.org/research/papers/GridftpSpec02.doc
򐂰 The Gridlab Project
http://www.gridlab.org/
򐂰 Grid MP Solutions Overview
http://www.ud.com/solutions
򐂰 Grid Security Infrastructure
http://www-unix.globus.org/toolkit/docs/3.2/gsi/key/index.html
򐂰 GUMS Certificate Management
http://www.atlasgrid.bnl.gov/testbed/gums/introduction.shtml
򐂰 GUMS Overview
http://grid.racf.bnl.gov/GUMS/guide_introduction.html
򐂰 Grid User Management Systems
http://www.atlasgrid.bnl.gov/testbed/gums
򐂰 HP-UX 11i home page
http://www.hp.com/products1/unix/operating
򐂰 IBM DB2 Information Integrator for Content: Overview
http://www.ibm.com/software/data/eip
򐂰 IBM ^ Cluster software
http://www.ibm.com/servers/eserver/clusters/software
򐂰 IBM ^ Software Information Center
http://publib.boulder.ibm.com/eserver/v1r1/en_US/index.htm?info/ogsainfo
򐂰 IBM Global Services
http://www.ibm.com/services
http://www.ibm.com/igs
򐂰 IBM Grid computing: Business Partners
http://www.ibm.com/grid/grid_partners/index.shtml
򐂰 IBM Grid computing: Solutions
http://www.ibm.com/linux/grid/solutions

Related publications 209

 򐂰 IBM Grid Innovation Workshop
http://www.ibm.com/grid/solutions/innovation_workshop.shtml
򐂰 IBM Grid Solutions
http://www.ibm.com/grid/solutions/index.shtml
򐂰 IBM Grid Toolbox
http://www.ibm.com/grid/solutions/grid_toolbox.shtml
򐂰 IBM Informix
http://www.ibm.com/software/data/informix
򐂰 IBM IT Education Services
http://www.ibm.com/services/learning/us/catalog
򐂰 IBM LoadLeveler
http://publib.boulder.ibm.com/clresctr/windows/public/llbooks.html
򐂰 IBM Offering Information: User settings
http://www.ibm.com/common/ssi
򐂰 IBM Operational Support Services - Support Line
http://www.ibm.com/services/its/us/supportline.html
򐂰 IBM SAN File System: Overview
http://www.ibm.com/servers/storage/software/virtualization/sfs
򐂰 IBM SAN Volume Controller: Overview
http://www.ibm.com/servers/storage/software/virtualization/svc/index.html
򐂰 IBM System Journal - Design and implementation of an enterprise grid
http://www.research.ibm.com/journal/sj/434/meliksetian.html
򐂰 IBM Tivoli Access Manager for Business Integration
http://www.ibm.com/software/tivoli/products/access-mgr-bus-integration
򐂰 IBM Tivoli Access Manager for e-business
http://www.ibm.com/software/tivoli/products/access-mgr-e-bus
򐂰 IBM Tivoli Access Manager for Operating Systems
http://www.ibm.com/software/tivoli/products/access-mgr-operating-sys
򐂰 IBM Tivoli Directory Integrator
http://www.ibm.com/software/tivoli/products/directory-integrator
򐂰 IBM Tivoli Directory Server
http://www.ibm.com/software/tivoli/products/directory-server

210 Grid Computing Products and Services

򐂰 IBM Tivoli Enterprise Console
http://www.ibm.com/software/tivoli/products/enterprise-console
򐂰 IBM Tivoli Intelligent Orchestrator
http://www.ibm.com/software/tivoli/products/intell-orch
򐂰 IBM Tivoli License Manager
http://www.ibm.com/software/tivoli/products/license-mgr
򐂰 IBM Tivoli Management Framework
http://www.ibm.com/software/tivoli/products/mgt-framework
򐂰 IBM Tivoli Monitoring
http://www.ibm.com/software/tivoli/products/monitor
򐂰 IBM Tivoli NetView
http://www.ibm.com/software/tivoli/products/netview
򐂰 IBM Tivoli Provisioning Manager
http://www.ibm.com/software/tivoli/products/prov-mgr
򐂰 IBM Tivoli Service Level Advisor
http://www.ibm.com/software/tivoli/products/service-level-advisor
򐂰 IBM Tivoli software
http://www.ibm.com/software/sw-bycategory/tivoli
򐂰 IBM Tivoli Storage Manager: Product Overview
http://www.ibm.com/software/tivoli/products/storage-mgr
򐂰 IBM Tivoli Storage Resource Manager
http://www.ibm.com/software/tivoli/products/storage-resource-mgr
򐂰 IBM Tivoli Workload Scheduler
http://www.ibm.com/software/tivoli/products/scheduler
򐂰 IBM Virtualization Solutions
http://www.ibm.com/servers/eserver/about/virtualization/?ca=vedemot&m
http://www.ibm.com/servers/eserver/about/virtualization/index.html
http://www.ibm.com/servers/eserver/about/virtualization/?ca=vedemot&met=web
&me=escallout
򐂰 IBM WebSphere Developer Technical Journal: Building an Enterprise Service
Bus with WebSphere Application Server V6 -- Part 1, found at:
http://www.ibm.com/developerworks/websphere/techjournal/0501_reinitz/0501_r
einitz.html

Related publications 211

 򐂰 Install the DB2 UDB data partitioning feature on Linux
http://www.ibm.com/developerworks/db2/library/techarticle/dm-0504mcarthur/
򐂰 InteGrade
http://gsd.ime.usp.br/integrade
򐂰 JDBC technology
http://java.sun.com/products/jdbc
򐂰 LoadLeveler 3.2.1 documentation updates
http://publib.boulder.ibm.com/infocenter/clresctr/index.jsp?topic=/com.ibm.
cluster.loadl.doc/doc_updates/ll3_2.1update.html
򐂰 Lotus Software
http://www.lotus.com
򐂰 Maui Scheduler
http://sourceforge.net/projects/mauischeduler
򐂰 Microsoft .NET
http://www.microsoft.com/net
򐂰 Microsoft Office Online
http://office.microsoft.com/en-us/default.aspx
򐂰 National Science Foundation
http://www.nsf.gov
򐂰 The National Science Foundation Middleware Initiative
http://www.nsf-middleware.org/
򐂰 Network File System (NFS) Version 4 Protocol
http://www.ietf.org/rfc/rfc3530.txt
򐂰 NFS Version 4
http://www.nfsv4.org
򐂰 Nortel
http://www.nortel.com
򐂰 OASIS
http://www.oasis-open.org
http://www.oasis-open.org/home/index.php
򐂰 OASIS Web Services Resource Framework (WSRF) TC
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf

212 Grid Computing Products and Services

򐂰 OGSA-DAI project
http://www.ogsadai.org.uk
򐂰 OGSI::Lite and WSRF::Lite - Perl Grid Services
http://www.sve.man.ac.uk/Research/AtoZ/ILCT
򐂰 OpenPBS
http://www.openpbs.org/about.html
򐂰 Oracle home page
http://www.oracle.com
򐂰 PBS Professional
http://www.pbspro.com
http://www.openpbs.org/about_pbspro.html
򐂰 Platform Computing
http://www.platform.com/
򐂰 Platform Globus Toolkit
http://www.platform.com/Products/Platform.Globus.Toolkit/Product.Informatio
n/Features.Benefits.htm
򐂰 Platform LSF
http://www.platform.com/products/LSF
򐂰 Platform LSF Family of Products
http://www.platform.com/products/LSFfamily
򐂰 Platform LSF License Scheduler
http://www.platform.com/products/LSF/addons.asp#license
򐂰 Platform Symphony
http://www.platform.com/Products/Platform.Symphony/
򐂰 Portable Batch System
http://www.openpbs.org
򐂰 Python Globus
http://www-itg.lbl.gov/gtg/projects/pyGlobus
򐂰 SAN File Systems
http://www.storage.ibm.com/software/virtualization/sfs/index.html
򐂰 SAN Volume Controller
http://www.ibm.com/servers/storage/software/virtualization/svc
򐂰 SAS Institute, Inc.

Related publications 213

 http://www.sas.com/
򐂰 SETI@Home
http://setiathome.ssl.berkeley.edu
򐂰 The Smallpox Research Grid
http://www.grid.org/projects/smallpox
򐂰 Solaris Operating System
http://wwws.sun.com/software/solaris
򐂰 Sparsi
http://www.sparsi.com
򐂰 SQL Server
http://www.microsoft.com/sql
򐂰 Stand-Alone Grid Security Infrastructure (GSI)
http://www.globus.org/security/v1.1/standalone.html
򐂰 Sun: Grid Computing Solutions
http://wwws.sun.com/software/grid
򐂰 Sybase home page
http://www.sybase.com
򐂰 Teragrid
http://www.teragrid.org
򐂰 Unicore
http://unicore.sourceforge.net
򐂰 United Devices
http://www.ud.com
򐂰 United Devices Grid MP
http://www.ud.com/solutions/fabs
򐂰 VMWare home page
http://www.vmware.com
򐂰 The Virtual Data Toolkit
http://www.cs.wisc.edu/vdt/
򐂰 WebSphere Application Server Performance Advisors Technology Preview
http://www.ibm.com/developerworks/websphere/downloads/performance_advisors.
html

214 Grid Computing Products and Services

򐂰 WebSphere Business Integration
http://www.ibm.com/developerworks/websphere/zones/businessintegration/bigpi
cture.html
򐂰 WebSphere Business Integration Message Broker
http://www.ibm.com/software/integration/wbimessagebroker
򐂰 WebSphere Business Integration Server
http://www.ibm.com/software/integration/wbiserver
򐂰 WebSphere Extended Deployment
http://www.ibm.com/software/webservers/appserv/extend/features
򐂰 WebSphere MQ
http://www.ibm.com/software/integration/wmq
򐂰 WebSphere Portal for Multiplatforms
http://www.ibm.com/software/genservers/portal
򐂰 WebSphere Product Overview
http://www.redbooks.ibm.com/redpapers/pdfs/redp3740.pdf
򐂰 WebSphere software
http://www.ibm.com/software/websphere/index.html
http://www.ibm.com/websphere
򐂰 World Community Grid
http://www.worldcommunitygrid.org
http://www.worldgrid.org
򐂰 World Community Grid: Advisory Board
http://www.worldcommunitygrid.org/about_us/advisory_board.html
򐂰 World Community Grid: Project Archive
http://www.worldcommunitygrid.org/projects_showcase/projects_archive.html
򐂰 World Wide Web Consortium (W3C)
http://www.w3.org
򐂰 The WS-Resource Framework
http://www.globus.org/wsrf
򐂰 WSFRF.NET
http://www.cs.virginia.edu/~gsw2c/wsrf.net.html

Related publications 215

How to get IBM Redbooks
You can search for, view, or download Redbooks, Redpapers, Hints and Tips,
draft publications and Additional materials, as well as order hardcopy Redbooks
or CD-ROMs, at this Web site:
ibm.com/redbooks

Help from IBM

IBM Support and downloads
ibm.com/support

IBM Global Services

ibm.com/services

216 Grid Computing Products and Services

Glossary

API. Application Programming Interface. Globus MDS. Monitoring and Discovery System.

CIFS. Common Internet file system, the protocol Globus MJS. Managed Job Service.
formerly known as Server Message Block (SMB)
and, before that, as the BAF protocol (after its Globus RFT. Reliable File Transfer.
original creator, Dr. Barry Feigenbaum). CIFS is a
protocol for file and device sharing across a network. Globus UHE. User Hosting Environment.

CMM. Common Management Models. Globus. A collaborative project centered at

Argonne National Laboratory that is focused on
DAS. Direct Attached Storage. enabling the application of grid concepts to
computing.
DHCP. Dynamic Host Configuration Protocol is a
client-server networking protocol. A DHCP server GNU. GNU’s Not Unix.
provides configuration parameters specific to the
DHCP client host requesting, generally, information GPFS. General Parallel File System is a type of
required by the host to participate on the Internet mountable networked file systems.
network. DHCP also provides a mechanism for
allocation of IP addresses to hosts. GRAM. Globus Resource Allocation Manager,
which requests and uses remote system resources
DNS The Domain Name System is a system that for jobs.
stores information about host names and domain
names on networks, such as the Internet. GRIS. Grid Resource Information Service.

EJB. Enterprise Java Beans, a set of GSI. The Grid Security Infrastructure contains
JavaBeans™, currently being developed by Sun to components to secure your grid network.
support development of distributed applications.
HPC High Performance Computing consists of
FTP. File Transmission Protocol, a standard computing applications on (parallel)
Internet protocol to exchange files on top of TCP/IP. supercomputers and computer clusters.

GCC. GNU Compiler Collection, which currently HTC. High Throughput Computing.
contains front ends for C, C++, Objective-C, Fortran,
Java, and Ada, as well as libraries for these HTML. Hypertext Markup Language, which is a
languages (libstdc++, libgcj, and so on). document type consisting of text and tags.

GDC. Grid Domain Controller. HTTP. Hypertext Transfer Protocol, which is an

application protocol.
GGF. The Global Grid Forum was founded in 2001
when the merger of regional grid organizations JDBC. Java Database Connectivity. Similar to
created a single worldwide one. ODBC, this set of application programming
interfaces (APIs) provides a standard mechanism to
GIIS. Grid Index Information Services. allow Java applets access to a database.

© Copyright IBM Corp. 2005. All rights reserved. 217

JMS. Java Messaging System. OGSA-DAI. Open Grid Services Architecture -
Data Access Integration. A project developed by the
JSP. Java Server Pages, which is an extensible UK Database Task Force whose objective is to
Web technology that uses static data, JSP elements, provide a standard interface for a distributed query
and server-side Java objects to generate dynamic processing system to access data in different
content for a client. databases.

LDAP. Lightweight Directory Access Protocol. OGSI. Open Grid Services Infrastructure. A
IETF term for an TCP/IP based network protocol that minimal set of grid services and supporting
enables access to a Directory System Agent. infrastructure on top of which OGSA systems and
applications can be built or extended.
MMJFS. Managed Job Factory
Service, which is a job manager that provides an P2G. Peer to Group.
interface for requesting and using remote resources
for the jobs. Peer to Peer .A network that does not rely on
dedicated servers for communication but instead
MVC. Model View Controller. mostly uses direct connections between clients
(peers). A pure peer-to-peer network does not have
NAS. Network Attached Storage. NAS is storage in the notion of clients or servers, but only equal peer
a form that is readily accessible on a network, as nodes that simultaneously function as both "clients"
opposed to direct-attached storage. and "servers" to the other nodes on the network.

NFS. Network File System. A client/server RLS. Replica Location Service.

application designed by Sun Microsystems™ that
allows all network users to access shared files SAN. A Storage Area Network is a high-speed
stored on computers of different types. special-purpose network (or subnetwork) that
interconnects different kinds of data storage devices
OASIS. Organization for the Advancement of with associated data servers on behalf of a larger
Structured Information Standards. network of users.

ODBC. Open Database Connectivity. Open SDK. Software Development Kit. A programming
standard developed by Microsoft aimed at providing package that enables a programmer to develop
a consistent interface for relational databases, applications for a specific platform.
regardless of which database engine is used (for
example, SQL Server, Oracle, or MySQL). SMB. Server Message Block. A file sharing
protocol used in DOS, OS/2®, and early versions of
OGSA. The Open Grid Services Architecture is a Windows.
standard that sets the basis for communication in
grids across virtual organizations. OGSA marries SOA. Service-Oriented Architecture is an
open standards and grid computing protocols with architecture that represents software functionality as
Web Services, bringing together the ability to share discoverable services on the network.
computing resources with the ability to provide
application interoperability over the Internet. SOAP. Simple Object Access Protocol. A method
invented by Microsoft to use RPC over the internet
via HTTP calls.

218 Grid Computing Products and Services

SQL. Structured Query Language. A type of
programming language used to construct database
queries and perform updates and other
maintenance of relational databases. SQL is not a
full-fledged language that can create stand-alone
applications, but it is strong enough to create
interactive routines in other database programs.

SSL. Secure Socket Layer is a security protocol

that provides communication privacy. SSL enables
client/server applications to communicate in a way
that is designed to prevent eavesdropping,
tampering, and message forgery. SSL was
developed by Netscape Communications Corp. and
RSA Data Security.

VO. Virtual Organization. A virtual entity whose

users and servers are geographically apart but
share their resources collectively as a larger grid.
The users of the grid can be organized dynamically
into a number of virtual organizations, each with
different policy requirements.

VPN. Virtual Private Network. A network that is

constructed by using public wires to connect nodes,
using encryption and other security mechanisms to
ensure that only authorized users can access the
network and that the data cannot be intercepted.

WAN. Wide Area Network. It can be made up of

interconnected smaller networks spread throughout
a building, a state, or the entire globe.

Web Services. A way of providing computational

capabilities using standard Internet protocols and
architectural elements

WSRF. Web Services Resource Framework.

Defines a generic and open framework for modeling
and accessing stateful resources using Web
services.

XML. Extensible Markup Language. Modification

of the SGML standard. In contrast to SGML
documents, XML documents may exist without
having their schema described in a document type
definition.

Glossary 219
220 Grid Computing Products and Services
Index
distributed peer to peer grid
A see Network grid
Access Control Lists
see ACL
ACL 6 E
AFS 70 Enterprise Identity Mapping 96
AIX 159 EWLM 108
Altair PBS Professional 54
API 13, 18, 76, 177
ARM 108
F
federated database 70
authentication 6, 16, 92–94, 177 File System Virtualization Layer 70, 75
authorization 6, 16, 92, 94, 177 File systems
AFS 70
B NAS 70
billing 15, 102 SAN 70, 75
firewall 152, 160
FTP 10
C Functionality
CA 96
Billing and metering 85
certificates 93, 95, 97
Directory Services 85
certification authorities
Information Virtualization 16–17, 19
see CA
Information virtualization 65
cluster 11, 43, 47, 135, 196
Licensing 85
Condor 56
Other Disciplines 16, 20
CSF 57
Provisioning and orchestration 85
Scheduling 16, 19
D Security 85
Data grid 9 Web Services Core/Hosting 16, 18, 23
Database 80 Workload Virtualization 41
Database partitioning feature (DPF) 80
DataSynapse GridServer 48, 116, 135, 173
DB2 Content Manager 80
G
Gang Scheduler 43
DB2 Information Integrator for Content 81
General Parallel File System
DB2 Product Family
see IBM General Parallel File System
Database partitioning feature (DPF) 80
GGF 114
DB2 Content Manager 80
Global Grid Forum 122
DB2 Information Integrator for Content 81
Globus Alliance 122, 133
DB2 UDB 78–79
Globus Toolkit 92, 97, 116, 135, 174
DBMS 78
GPFS
deep computing
see IBM General Parallel File System
see HPC
Grid computing models
Desktop grid 45, 59, 150
Data grid 9
Directed Attached Storage
Desktop grid 45, 59
see DAS

© Copyright IBM Corp. 2005. All rights reserved. 221

 Multipurpose grid 11 IBM Managed Storage Services 159
Network grid 9 IBM Shark Enterprise Storage Servers 159
Server grid 7, 45, 60, 114 IBM Tivoli Access Manager 94, 172, 177
grid delivery IBM Tivoli Directory Integrator 98
see Network grid IBM Tivoli Directory Server 98
Grid Engine 44 IBM Tivoli Enterprise Console 172, 179
Grid stack product’s portfolio IBM Tivoli Intelligent Orchestrator 180
Application 13 IBM Tivoli Intelligent Think Dynamic Orchestrator
Billing and metering 14 88, 172
Content management 14 IBM Tivoli License Manager 101, 172, 181
File and block data virtualization 16 IBM Tivoli Management Framework 172, 183
Grid middleware 16 IBM Tivoli Monitoring 172, 184
Job scheduling 15 IBM Tivoli NetView 172, 186
Orchestration and provisioning 14 IBM Tivoli Provisioning Manager 88, 108–109, 172,
Software licensing 15 187
Structured data virtualization 15 IBM Tivoli Service Level Advisor 172, 190
System management 15 IBM Tivoli Storage Manager 72
Task scheduling 14 IBM Tivoli Storage Resource Manager 74
Workload management 14 IBM Tivoli Suite 15
GridFTP 45 IBM Tivoli WorkLoad Scheduler 45, 171, 192
GSI 95 IBM TotalStorage Productivity Center 110
IBM TotalStorage SAN File System 9, 77, 110, 171,
189
H IBM TotalStorage SAN Volume Controller 72, 110,
High Performance Computing
171, 193
see HPC
IBM Virtualization Engine 106, 195
High Throughput Computing
IBM Virtualization Engine Suite for Servers 108
see HTC
IBM WebSphere Application Server 196
HPC 11, 47, 56
IBM WebSphere MQ 201
HTC 56
IBM WebSphere Portal Server 202
Independent Software Vendor
I see ISV
I3C 122 Information grid
i5/OS 107 see Data grid
IBM AIX 107 Interoperable Informatics Infrastructure Consortium
IBM Cluster Systems Management 91 see I3C
IBM DB2 109, 159 ISV 20, 99
IBM DB2 Content Manager 80 DataSynapse 48, 116, 131
IBM DB2 Information Integrator for Content 81 Platform Computing 50, 114, 133, 135
IBM Director Multiplatform 109 United Devices 52, 59, 117, 135
IBM downloadGrid 10
IBM Enterprise Workload Manager 108
IBM eServer pSeries 47, 176 J
Java
IBM eServer xSeries 72, 158
EJB 159
IBM General Parallel File System 70, 75–76, 171,
J2EE 94, 159
174
JSP 159
IBM Global Services 122, 133, 153
servlet 159
IBM Grid Toolbox 93, 97, 109
IBM LoadLeveler 47, 171, 176

222 Grid Computing Products and Services

K Platform Symphony 52, 115
Kerberos 76 Products
Altair PBS Professional 54
Condor 56
L CSF 57
LDAP 81, 92
DataSynapse GridServer 48, 116, 135, 173
license management 15
Enterprise Identity Mapping 96
Linux 72, 83, 107, 158, 160, 189
Globus Toolkit 92, 116, 135, 174
load balancing 44, 60
IBM Cluster Systems Management 91
IBM DB2 109, 159
M IBM DB2 Content Manager 80, 171, 173
Maui Scheduler 58 IBM DB2 Information Integrator for Content 81
Message Passing Interface 135 IBM Director Multiplatform 109
metadata 77 IBM Enterprise Workload Manager 108
meta-scheduler 43–44 IBM General Parallel File System 76, 171, 174
metering 15, 102 IBM Grid Toolbox 93, 109
Multipurpose grid 11 IBM LoadLeveler 47, 171, 176
IBM Tivoli Access Manager 94, 172, 177
IBM Tivoli Directory Integrator 98
N IBM Tivoli Directory Server 98
NAS 70–71, 74, 83, 159
IBM Tivoli Enterprise Console 172, 179
Network grid 9
IBM Tivoli Intelligent Orchestrator 180
NFS 70, 75–76, 83
IBM Tivoli Intelligent Think Dynamic Orchestra-
tor 88, 172
O IBM Tivoli License Manager 101, 172, 181
OGSA 114, 116, 133, 196 IBM Tivoli Management Framework 172, 183
on demand 11–12, 72–74, 77, 122, 159 IBM Tivoli Monitoring 172, 184
open source 56 IBM Tivoli NetView 172, 186
open standards 4, 11, 94, 122 IBM Tivoli Provisioning Manager 88, 108–109,
OpenMP 135 172, 187
OpenPBS 55, 62 IBM Tivoli Service Level Advisor 172, 190
orchestration 15, 87 IBM Tivoli Storage Manager 72
IBM Tivoli Storage Resource Manager 74
IBM Tivoli WorkLoad Scheduler 45, 171, 192
P
P2P 10 IBM TotalStorage Productivity Center 110
Parallel Virtual Machine 135 IBM TotalStorage SAN File System 9, 77, 110,
pay per usage 12 171, 189
peer-to-peer IBM TotalStorage SAN Volume Controller 72,
see P2P 110, 171, 193
Platform Computing 133, 135 IBM Virtualization Engine 106, 195
Platform Load Sharing Facility IBM Virtualization Engine Suite for Servers 108
see Platform LSF Maui Scheduler 58
Platform LSF 50, 114 NFS 60, 76
Platform LSF Analytics 116 OpenPBS 55, 62
Platform LSF HPC 51, 115 Platform LSF 50, 114
Platform LSF License Scheduler 116 Platform LSF Analytics 116
Platform LSF MultiCluster 51, 115 Platform LSF HPC 51, 115
Platform LSF Reports 116 Platform LSF License Scheduler 116
Platform LSF MultiCluster 51, 115

Index 223
 Platform LSF Reports 116 Grid Design Services 123, 126
Platform Symphony 52, 115 Grid Innovation Workshop 123–124
United Devices 52 Grid Software Implementation Services 123,
United Devices MP 53, 117, 160 127
Web Services Core/Hosting Grid Strategy Engagement 123, 125
IBM WebSphere Application Server 196 Grid Training and Education 123, 128
IBM WebSphere MQ 201 IBM Managed Storage Services 159
IBM WebSphere Portal Server 202 IBM Support Line 128
WebSphere Application Server 30, 92, 159, 171 Shared Resources
WebSphere Business Integration Messaging CPU 5
Broker 38 databases 5
WebSphere Extended Deployment 32, 171, disks 5
198 memory 5
WebSphere Information Integration 84, 137, network 5
171, 199 storage devices 5
WebSphere Information Integrator 79 single sign-on 94–95
WebSphere Information Integrator Content Edi- SLA 14
tion 79 Smallpox Project 160
WebSphere Information Integrator OmniFind SMB 83
Edition 79 Software Development Kit
WebSphere Messaging Resources 37 see SDK
WebSphere MQ 171 Solutions
WebSphere MQ Family 36 Analytics Acceleration 129, 131, 140
WebSphere MQ Messaging 36 Clash Analisys 140
WebSphere Portal Server 33, 171 Design Collaboration 129, 132, 140
provisioning 15, 86, 88, 107–108, 110, 187 Engineering Design 129, 133, 140
pthreads 135 Geophysical Processing and Analysis 129,
134, 140
Grid Clash Analysis 129, 139
Q IBM Grid Innovation Workshop 140
QoS 6, 58
IBM Grid Value at Work 129–130
quality of service
Information Accessibility 129, 136, 140
see QoS
IT Optimization 129, 137, 140
Research Collaboration 129, 137, 140
R Storage Area Network 9
Redbooks Web site 216 see SAN
Contact us xxi Storage Virtualization Layer 71
resource pool 86
T
S TCO 130
SAN 71, 74, 83, 110, 159, 193
SAN Volume Controller
see SVC U
UMI 12
SDK 13, 118
United Devices 52, 59, 135
Server grid 7, 45, 60, 114
United Devices MP 53, 117, 160
Service Level Agreement
UNIX 83, 92, 189
see SLA
Utility Management Infrastructure
Services
see UMI

224 Grid Computing Products and Services

V
virtual computer 4
virtual organization 7, 93
virtual private network
see VPN
VMWare 107
VPN 90, 93

W
Web services 133
WebSphere Application Server 30, 92, 159, 171
WebSphere Business Integration Messaging Broker
38
WebSphere Extended Deployment 32, 171
WebSphere Information Integration 78
WebSphere Information Integrator 79
WebSphere Information Integrator Content Edition
79
WebSphere Information Integrator OmniFind Edition
79
WebSphere Messaging Resources 37
WebSphere MQ 171
WebSphere MQ Family 36
WebSphere MQ Messaging 36
WebSphere Portal Server 33, 171
workload management 15, 45, 53, 87, 107–108
World Community Grid 149, 151–154, 158, 167
Agent 161
Screen Saver 162
Servers 161
world-scale grid 150
WSRF 196

X
XML 15

Z
z/OS 107

Index 225
226 Grid Computing Products and Services
Grid Computing Products and Services
(0.5” spine)
0.475”<->0.875”
250 <-> 459 pages
 Back cover ®

Products and Services for

Grid Computing

Grid computing This IBM Redbook is part of a series of documents related to grid
functionality computing that IBM is presenting to the IT community to enrich it and INTERNATIONAL
all its players: clients, industry leaders, emerging enterprises, TECHNICAL
Products for grid
universities, and producers of technology. It is mainly oriented to IT SUPPORT
architects or those who have the responsibility of analyzing the
computing capabilities of the product used to build a grid solution.
ORGANIZATION

Services and We describe the functions of grid computing and the categorizations
of the components within it. IBM, ISVs, and open source products will
solutions be explained. Some of these products are not specific for grid, but they BUILDING TECHNICAL
can be used in a grid computing environment. We hope that this INFORMATION BASED ON
redbook helps you select functions and products and shows you how PRACTICAL EXPERIENCE
grid can fit into your IT picture as new products and services for grid
are introduced.
IBM Redbooks are developed by
the IBM International Technical
Part 1 introduces the concept of grid computing and provides the Support Organization. Experts
terminology the readers will use in this redbook. from IBM, Customers and
Partners from around the world
Part 2 explores the functionality associated with certain products and create timely technical
product families. It also presents some examples of the utilization of information based on realistic
these products in the grid computing environment. scenarios. Specific
recommendations are provided
Part 3 presents grid computing product families that can be used to to help you implement IT
build a complete grid solution. These bundles, or suites, often solutions more effectively in
implement most of the core grid disciplines. your environment.

Part 4 gives an overview of the grid computing offerings and services

that IBM provides.
For more information:
ibm.com/redbooks

SG24-6650-00 ISBN 0738491780