CS6703 - GRID AND CLOUD

COMPUTING
OBJECTIVES:
The student should be made to:
Understand how Grid computing helps in solving large scale

scientific problems.

Gain knowledge on the concept of virtualization that is fundamental

to cloud computing.

Learn how to program the grid and the cloud.

Understand the security issues in the grid and the cloud environment.
UNIT I INTRODUCTION (9)

Evolution of Distributed computing: Scalable computing over the Internet Technologies

for network based systems clusters of cooperative computers - Grid computing

Infrastructures cloud computing - service oriented architecture Introduction to Grid

Architecture and standards Elements of Grid Overview of Grid Architecture.

UNIT II GRID SERVICES (9)

Introduction to Open Grid Services Architecture (OGSA) Motivation Functionality

Requirements Practical & Detailed view of OGSA/OGSI Data intensive grid service

models OGSA services.

UNIT III VIRTUALIZATION (9)

Cloud deployment models: public, private, hybrid, community Categories of cloud

computing: Everything as a service: Infrastructure, platform, software - Pros and Cons of

cloud computing Implementation levels of virtualization virtualization structure

virtualization of CPU, Memory and I/O devices virtual clusters and Resource Management

Virtualization for data center automation.

UNIT IV PROGRAMMING MODEL (9)

Open source grid middleware packages Globus Toolkit (GT4) Architecture , Configuration

Usage of Globus Main components and Programming model - Introduction to Hadoop

Framework - Mapreduce, Input splitting, map and reduce functions, specifying input and

output parameters, configuring and running a job Design of Hadoop file system, HDFS

concepts, command line and java interface, dataflow of File read & File write.
UNIT V SECURITY (9)

Trust models for Grid security environment Authentication and Authorization methods

Grid security infrastructure Cloud Infrastructure security: network, host and application

level aspects of data security, provider data and its security, Identity and access

management architecture, IAM practices in the cloud, SaaS, PaaS, IaaS availability in the

cloud, Key privacy issues in the cloud.

OUTCOMES:

At the end of the course, the student should be able to:

Apply grid computing techniques to solve large scale scientific problems.

Apply the concept of virtualization.

Use the grid and cloud tool kits.

Apply the security models in the grid and the cloud environment.

TEXT BOOK:

1. Kai Hwang, Geoffery C. Fox and Jack J. Dongarra, Distributed and

Cloud Computing: Clusters, Grids, Clouds and the Future of Internet,

First Edition, Morgan Kaufman Publisher, an Imprint of Elsevier, 2012.

REFERENCES:
1. Jason Venner, Pro Hadoop- Build Scalable, Distributed Applications in the
Cloud, A Press, 2009
2. Tom White, Hadoop The Definitive Guide, First Edition. OReilly, 2009.
3. Bart Jacob (Editor), Introduction to Grid Computing, IBM Red Books, Vervante,
2005
4. Ian Foster, Carl Kesselman, The Grid: Blueprint for a New Computing
Infrastructure, 2nd Edition, Morgan Kaufmann.
5. Frederic Magoules and Jie Pan, Introduction to Grid Computing CRC Press, 2009.
6. Daniel Minoli, A Networking Approach to Grid Computing, John Wiley
Publication, 2005.
7. Barry Wilkinson, Grid Computing: Techniques and Applications, Chapman and
Hall, CRC, Taylor and Francis Group, 2010
Distributed Computing.
Distributed computing is a field of computer science that studies

distributed systems.

A distributed system is a model in which components located on

networked computers communicate and coordinate their actions by

passing messages.

The components interact with each other in order to achieve a

common goal.
Evolution of Distributed computing
Over the past 60 years, computing technology has undergone a series of platform and
environment changes.
In this section, we assess evolutionary changes in machine architecture, operating system
platform, network connectivity, and application workload.
Instead of using a centralized computer to solve computational problems, a parallel and
distributed computing system uses multiple computers to solve large-scale problems over
the Internet.
Thus, distributed computing becomes data-intensive and network-centric. This section
identifies the applications of modern computer systems that practice parallel and
distributed computing.
These large-scale Internet applications have significantly enhanced the quality of life and
information services in society today.
The Age of Internet Computing
Billions of people use the Internet every day. As a result, supercomputer sites and large
data centers must provide high-performance computing services to huge numbers of
Internet users concurrently.
Because of this high demand, the Linpack Benchmark for high-performance computing
(HPC) applications is no longer optimal for measuring system performance.
The emergence of computing clouds instead demands high-throughput computing
(HTC) systems built with parallel and distributed computing technologies .
We have to upgrade data centers using fast servers, storage systems, and high-
bandwidth networks.
The purpose is to advance network-based computing and web services with the
emerging new technologies.
SCALABLE COMPUTING OVER THE INTERNET
The Platform Evolution Computer technology has gone through five generations of development, with
each generation lasting from 10 to 20 years.
Successive generations are overlapped in about 10 years. For instance, from 1950 to 1970, a handful of
mainframes, including the IBM 360 and CDC 6400, were built to satisfy the demands of large
businesses and government organizations.
From 1960 to 1980, lower-cost minicomputers such as the DEC PDP 11 and VAX Series became
popular among small businesses and on college campuses.
From 1970 to 1990, we saw widespread use of personal computers built with VLSI microprocessors.
From 1980 to 2000, massive numbers of portable computers and pervasive devices appeared in both
wired and wireless applications.
Since 1990, the use of both HPC and HTC systems hidden in clusters, grids, or Internet clouds has
proliferated.
These systems are employed by both consumers and high-end web-scale computing and information
services.
 The general computing trend is to leverage shared web resources and massive amounts
of data over the Internet.
On the HPC side, supercomputers (massively parallel processors or MPPs) are
gradually replaced by clusters of cooperative computers out of a desire to share
computing resources.
The cluster is often a collection of homogeneous compute nodes that are physically
connected in close range to one another.
On the HTC side, peer-to-peer (P2P) networks are formed for distributed file sharing
and content delivery applications.
A P2P system is built over many client machines.
Peer machines are globally distributed in nature.
P2P, cloud computing, and web service platforms are more focused on HTC
applications than on HPC applications.
Clustering and P2P technologies lead to the development of computational grids or data
grids.
Fig1: Evolutionary trend toward parallel, distributed, and cloud computing with
clusters, MPPs, P2P networks, grids, clouds, web services, and the Internet of Things
When the Internet was introduced in 1969, Leonard Klienrock of UCLA declared: As of now, computer
networks are still in their infancy, but as they grow up and become sophisticated, we will probably see the
spread of computer utilities, which like present electric and telephone utilities, will service individual
homes and offices across the country.
Many people have redefined the term computer since that time.
In 1984, John Gage of Sun Microsystems created the slogan, The network is the computer.
In 2008, David Patterson of UC Berkeley said, The data center is the computer. There are dramatic
differences between developing software for millions to use as a service versus distributing software to run
on their PCs.
Recently, Rajkumar Buyya of Melbourne University simply said: The cloud is the computer. This
book covers clusters, MPPs, P2P networks, grids, clouds, web services, social networks, and the IoT.
In fact, the differences among clusters, grids, P2P systems, and clouds may blur in the future.
Some people view clouds as grids or clusters with modest changes through virtualization.
Others feel the changes could be major, since clouds are anticipated to process huge data sets generated by
the traditional Internet, social networks, and the future IoT.
The distinctions and dependencies among all distributed and cloud systems models will become clearer
and more transparent.
Computing Paradigm Distinctions

The high-technology community has argued for many years about the precise

definitions of centralized computing, parallel computing, distributed computing, and

cloud computing.

In general, distributed computing is the opposite of centralized computing.

The field of parallel computing.

Distributed System Models and Enabling Technologies

overlaps with distributed computing to a great extent, and cloud computing overlap

with distributed, centralized, and parallel computing.

 Centralized computing
This is a computing paradigm by which all computer resources are centralized in one
physical system.
All resources (processors, memory, and storage) are fully shared and tightly coupled
within one integrated OS.
Many data centers and supercomputers are centralized systems, but they are used in
parallel, distributed, and cloud computing applications.
PARALLEL COMPUTING

In parallel computing, all processors are either tightly coupled with centralized
shared memory or loosely coupled with distributed memory.
Some authors refer to this discipline as parallel processing .
Interprocessor communication is accomplished through shared memory or via
message passing.
A computer system capable of parallel computing is commonly known as a
parallel computer Programs running in a parallel computer are called parallel
programs.
The process of writing parallel programs is often referred to as parallel
programming .
DISTRIBUTED COMPUTING

This is a field of computer science/engineering that studies distributed systems.

A distributed system consists of multiple autonomous computers, each having its own

private memory, communicating through a computer network.

Information exchange in a distributed system is accomplished through message

passing.

A computer program that runs in a distributed system is known as a distributed

program.

The process of writing distributed programs is referred to as distributed

programming.
What is Cloud?

The term Cloud refers to a Network or Internet. In other words, we can say that Cloud

is something, which is present at remote location.

Cloud can provide services over public and private networks, i.e., WAN, LAN or VPN.

Applications such as e-mail, web conferencing, customer relationship management

(CRM) execute on cloud.

What is Cloud Computing?

Cloud Computing refers to manipulating, configuring, and accessing the hardware and
software resources remotely.

It offers online data storage, infrastructure, and application.

Cloud computing offers platform independency, as the software is not required to be

installed locally on the PC. Hence, the Cloud Computing is making our business
applications mobile and collaborative.
Grid Computing:
Grid computing is the concept of distributed computing technologies for

computing resource sharing among participants in a virtualized collection of

organization.

Grid computing system is the ability to provide the quality of service

requirements necessary for the end-user community.

QOS provided by the grid like performance, availability, management aspects,

business value and flexibility in pricing.

DISTRIBUTED SYSTEM
The mid-1990s, technologies for building P2P networks and networks of clusters have
been consolidated into many national projects designed to establish wide area
computing infrastructures, known as computational grids or data grids.
Internet clouds are the result of moving desktop computing to service-oriented
computing using server clusters and huge database Scalable Computing over the
Internet at data centres.
Massively distributed systems are intended to exploit a high degree of parallelism or
concurrency among many machines.
In October 2010, the highest performing cluster machine was built in China with 86016
CPU processor cores and 3,211,264 GPU cores in a Tianhe-1A system.
The largest computational grid connects up to hundreds of server clusters.
 A typical P2P network may involve millions of client machines working simultaneously.
Experimental cloud computing clusters have been built with thousands of processing nodes.
Both HPC and HTC systems emphasize parallelism and distributed computing.
Future HPC and HTC systems must be able to satisfy this huge demand in computing power in
terms of throughput, efficiency, scalability, and reliability.
The system efficiency is decided by speed, programming, and energy factors (i.e., throughput per
watt of energy consumed). Meeting these goals requires to yield the following design objectives:

EFFICIENCY measures the utilization rate of resources in an execution

model by exploiting massive parallelism in HPC.

For HTC, efficiency is more closely related to job throughput,

data access, storage, and power efficiency.
DEPENDABILITY :measures the reliability and self-management from the chip to
the system and application levels.

The purpose is to provide high-throughput service with Quality of

Service (QoS) assurance, even under failure conditions.

ADAPTATION : In the programming model measures the ability to support

billions of job requests over massive data sets and virtualized

cloud resources under various workload and service models.

FLEXIBILITY : In application deployment measures the ability of distributed

systems to run well in both HPC (science and engineering) and
HTC (business) applications.
SCALABLE COMPUTING TRENDS AND NEW PARADIGMS:
Several predictable trends in technology are known to drive computing applications.

In fact, designers and programmers want to predict the technological capabilities of future
systems.
For instance, Jim Grays paper, Rules of Thumb in Data Engineering, is an excellent
example of how technology affects applications and vice versa.
In addition, Moores law indicates that processor speed doubles every 18 months. Although
Moores law has been proven valid over the last 30 years, it is difficult to say whether it will
continue to be true in the future.
Gilders law indicates that network bandwidth has doubled each year in the past. Will that
trend continue in the future?

The tremendous price/performance ratio of commodity hardware was driven by the desktop,

notebook, and tablet computing markets.

This has also driven the adoption and use of commodity technologies in large-scale

computing.
INNOVATIVE APPLICATIONS

Both HPC and HTC systems desire transparency in many application aspects.

For example, data access, resource allocation, process location, concurrency in

execution, job replication, and failure recovery should be made transparent to

both users and system management.

Almost all applications demand computing economics, web-scale data collection,

system reliability, and scalable performance.

For example, distributed transaction processing is often practiced in the banking

and finance industry.

 Transactions represent 90 percent of the existing market for reliable banking systems.
Users must deal with multiple database servers in distributed transactions.
Maintaining the consistency of replicated transaction records is vital in real-time
banking services.
THE TREND TOWARDS UTILITY COMPUTING:

All grid/cloud platforms are regarded as utility service providers.

However, cloud computing offers a broader concept than utility computing.

Distributed cloud applications run on any available servers in some edge

networks.

The vision of computer utilities in modern distributed computing systems.

SCALABLE COMPUTING TRENDS AND NEW PARADIGMS:
Several predictable trends in technology are known to drive computing applications. In fact,
designers and programmers want to predict the technological capabilities of future systems.
For instance, Jim Grays paper, Rules of Thumb in Data Engineering, is an excellent
example of how technology affects applications and vice versa.
In addition, Moores law indicates that processor speed doubles every 18 months. Although
Moores law has been proven valid over the last 30 years, it is difficult to say whether it will
continue to be true in the future.
Gilders law indicates that network bandwidth has doubled each year in the past. Will that
trend continue in the future?

The tremendous price/performance ratio of commodity hardware was driven by the desktop,

notebook, and tablet computing markets.

This has also driven the adoption and use of commodity technologies in large-scale

computing.