Cloud computing

Module 1
Distributed System Models and Enabling Technologies

Topics:

1.1 Scalable Computing over the Internet

1.2 Technologies for Network-based Systems
1.3 System Models for Distributed and Cloud Computing
1.4 Software Environments for Distributed Systems and Clouds
1.5 Performance, Security, and Energy Efficiency

1.1 Scalable Computing Over the Internet

 Over the past 60 years, computing technology has undergone a series of platform and
environment changes.
 In this section, we assess evolutionary changes in machine architecture, operating system
platform, network connectivity, and application workload. Instead of using a centralized
computer to solve computational problems,
 a parallel and distributed computing system uses multiple computers to solve large-scale
problems over the Internet.
 Thus, distributed computing becomes data-intensive and network-centric.
 This section identifies the applications of modern computer systems that practice parallel
and distributed computing. These large-scale Internet applications have significantly
enhanced the quality of life and information services in society today.

1.1.1 The Age of Internet Computing

 Billions of people use the Internet every day. As a result, supercomputer sites and large
data centers must provide high-performance computing services to huge numbers of
Internet users concurrently.
 Because of this high demand, the Linpack Benchmark for high-performance computing
(HPC) applications is no longer optimal for measuring system performance.

1
Cse bkec
 Cloud computing

 We have to upgrade data centers using fast servers, storage systems, and high-bandwidth
networks. The purpose is to advance network-based computing and web services with the
emerging new technologies.

1.1.1.1 The Platform Evolution

 First Generation (1950-1970): Mainframes like IBM 360 and CDC 6400.
 Second Generation (1960-1980): Minicomputers like DEC PDP 11 and VAX.
 Third Generation (1970-1990): Personal computers with VLSI microprocessors.
 Fourth Generation (1980-2000): Portable and wireless computing devices.
 Fifth Generation (1990-present): HPC and HTC systems in clusters, grids, and cloud
computing
 The general computing trend is to leverage shared web resources and massive amounts of
data over the Internet. Figure 1.1 illustrates the evolution of HPC and HTC systems.
 On the HPC side, supercomputers (massively parallel processors or MPPs) are gradually
replaced by clusters of cooperative computers out of a desire to share computing
resources.
 The cluster is often a collection of homogeneous compute nodes that are physically
connected in close range to one another.

FIGURE 1.1 Evolutionary trend toward parallel, distributed, and cloud computing with clusters,
MPPs, P2P networks, grids, clouds, web services, and the Internet of Things.
 On the HTC side, peer-to-peer (P2P) networks are formed for distributed file sharing and
content delivery applications.
 A P2P system is built over many client machines Peer machines are globally distributed
in nature.
 P2P, cloud computing, and web service platforms are more focused on HTC applications
than on HPC applications.

2
Cse bkec
 Cloud computing

 Clustering and P2P technologies lead to the development of computational grids or data
grids.

1.1.1.2 High-Performance Computing

 For many years, HPC systems emphasize the raw speed performance. The speed of HPC
systems has increased from G flops in the early 1990s to now P flops in 2010. This
improvement was driven mainly by the demands from scientific, engineering, and
manufacturing communities.
 For example, the Top 500 most powerful computer systems in the world are measured by
floating-point speed in Lin pack benchmark results.
 However, the number of supercomputer users is limited to less than 10% of all computer
users.

1.1.1.3 High-Throughput Computing

 The development of market-oriented high-end computing systems is undergoing a
strategic change from an HPC paradigm to an HTC paradigm. This HTC paradigm pays
more attention to high-flux computing.
 The main application for high-flux computing is in Internet searches and web services by
millions or more users simultaneously.
 The performance goal thus shifts to measure high throughput or the number of tasks
completed per unit of time..

1.1.1.4 Three New Computing Paradigms

 As Figure 1.1 illustrates, with the introduction of SOA, Web 2.0 services become
available.
 Advances in virtualization make it possible to see the growth of Internet clouds as a new
computing paradigm.
 The maturity of radio-frequency identification (RFID), Global Positioning System (GPS),
and sensor technologies has triggered the development of the Internet of Things (IoT).
These new paradigms are only briefly introduced here.

1.1.1.5 Computing Paradigm Distinctions

 The high-technology community has argued for many years about the precise definitions
of centralized computing, parallel computing, distributed computing, and cloud
computing.
 In general, distributed computing is the opposite of centralized computing.

 Centralized computing This is a computing paradigm by which all computer resources

are centralized in one physical system. All resources (processors, memory, and storage)
are fully shared and tightly coupled within one integrated OS.

3
Cse bkec
 Cloud computing

 Parallel computing In parallel computing, all processors are either tightly coupled with
centralized shared memory or loosely coupled with distributed memory. Some authors
refer to this discipline as parallel processing.
 Distributed computing This is a field of computer science/engineering that studies
distributed systems. A distributed system consists of multiple autonomous computers,
each having its own private memory, communicating through a computer network.
 Cloud computing An Internet cloud of resources can be either a centralized or a
distributed computing system. The cloud applies parallel or distributed computing, or
both. Clouds can be built with physical or virtualized resources over large data centers
that are centralized or distributed.

1.1.1.6 Distributed System Families

 Since the mid-1990s, technologies for building P2P networks and networks of clusters
have been consolidated into many national projects designed to establish wide area
computing infrastructures, known as computational grids or data grids.
 Recently, we have witnessed a surge in interest in exploring Internet cloud resources for
data-intensive applications.
 Internet clouds are the result of moving desktop computing to service-oriented computing
using server clusters and huge databases at data centers.
 This chapter introduces the basics of various parallel and distributed families. Grids and
clouds are disparity systems that place great emphasis on resource sharing in hardware,
software, and data sets.
 The largest computational grid connects up to hundreds of server clusters A typical P2P
network may involve millions of client machines working simultaneously. Experimental
cloud computing clusters have been built with thousands of processing nodes.

Future Computing Needs and Design Objectives

 Efficiency: Maximizing parallelism, job throughput, and power efficiency.
 Dependability: Ensuring reliability and Quality of Service (QoS).
 Adaptation: Scaling to billions of requests over vast data sets.
 Flexibility: Supporting both HPC (scientific/engineering) and HTC (business)
Applications.

1.1.2 Scalable Computing Trends and New Paradigms

 Several predictable trends in technology are known to drive computing applications. In
fact, designers and programmers want to predict the technological capabilities of future
systems.
 For instance, Jim Gray’s paper, “Rules of Thumb in Data Engineering,” is an excellent
example of how technology affects applications and vice versa.

4
Cse bkec
 Cloud computing

 In addition, Moore’s law indicates that processor speed doubles every 18 months.
Although Moore’s law has been proven valid over the last 30 years, it is difficult to say
whether it will continue to be true in the future.

1.1.2.1 Degrees of Parallelism

 Bit-level (BLP): Transition from 4-bit to 64-bit CPUs.

 Instruction-level (ILP): Techniques like pipelining, superscalar processing, and
multithreading.
 Data-level (DLP): SIMD and vector processing for efficient parallel execution.
 Task-level (TLP): Parallel tasks on multicore processors, though challenging to program.
 Job-level (JLP): High-level parallelism in distributed systems, integrating finegrain
parallelism.

1.1.2.2 Innovative Applications

 Both HPC and HTC systems desire transparency in many application aspects. For
example, data access, resource allocation, process location, concurrency in execution, job
replication, and failure recovery should be made transparent to both users and system
management.
 highlights a few key applications that have driven the development of parallel and
distributed systems over the years.
 These applications spread across many important domains in science, engineering,
business, education, health care, traffic control, Internet and web services, military, and
government applications. Table

Applications of High-Performance and High-Throughput Systems

 Almost all applications demand computing economics, web-scale data collection, system
reliability, and scalable performance. For example, distributed transaction processing is
often practiced in the banking and finance industry.

5
Cse bkec
 Cloud computing

 Transactions represent 90 percent of the existing market for reliable banking systems.
Users must deal with multiple database servers in distributed transactions. Maintaining
the consistency of replicated transaction records is crucial in real-time banking services. .

1.1.2.3 The Trend Toward Utility Computing

 Figure 1.2 identifies major computing paradigms to facilitate the study of distributed
systems and their applications. These paradigms share some common characteristics.
 First, they are all ubiquitous in daily life. Reliability and scalability are two major design
objectives in these computing models.
 Second, they are aimed at autonomic operations that can be self-organized to support
dynamic discovery. Finally, these paradigms are composable with QoS and SLAs
(service-level agreements). These paradigms and their attributes realize the computer
utility vision.

FIGURE 1.2 The vision of computer utilities in modern distributed computing systems.

 Utility computing focuses on a business model in which customers receive computing

resources from a paid service provider. All grid/cloud platforms are regarded as utility
service providers. However, cloud computing offers a broader concept than utility
computing.
 Distributed cloud applications run on any available servers in some edge networks. Major
technological challenges include all aspects of computer science and engineering.
 For example, users demand new network-efficient processors, scalable memory and
storage schemes, distributed OSes, middleware for machine virtualization, new
programming models, effective resource management, and application program
development.

1.1.2.4 The Hype Cycle of New Technologies

 Any new and emerging computing and information technology may go through a hype
cycle, as illustrated in Figure 1.3.
 This cycle shows the expectations for the technology at five different stages. The
expectations rise sharply from the trigger period to a high peak of inflated expectations.

6
Cse bkec
 Cloud computing

 Through a short period of disillusionment, the expectation may drop to a valley and then
increase steadily over a long enlightenment period to a plateau of productivity.
 The number of years for an emerging technology to reach a certain stage is marked by
special symbols. The hollow circles indicate technologies that will reach mainstream
adoption in two years.
 The gray circles represent technologies that will reach mainstream adoption in two to five
years. The solid circles represent those that require five to 10 years to reach mainstream
adoption, and the triangles denote those that require more than 10 years.

FIGURE 1.3 Hype cycle for Emerging Technologies, 2010.Hype Cycle Disclaimer

 The Hype Cycle is copyrighted 2010 by Gartner, Inc. and its affiliates and is reused with
permission.
 Hype Cycles are graphical representations of the relative maturity of technologies, IT
methodologies and management disciplines.
 They are intended solely as a research tool, and not as a specific guide to action. Gartner
disclaims all warranties, express or implied, with respect to this research, including any
warranties of merchantability or fitness for a particular purpose.
 This Hype Cycle graphic was published by Gartner, Inc. as part of a larger research note
and should be evaluated in the context of the entire report. The Gartner report is available
at.
 The hype cycle in Figure 1.3 shows the technology status as of August 2010.

1.1.3 The Internet of Things and Cyber-Physical Systems

 In this section, we will discuss two Internet development trends: the Internet of Things
[48] and cyber-physical systems. These evolutionary trends emphasize the extension of

7
Cse bkec
 Cloud computing

the Internet to everyday objects. We will only cover the basics of these concepts here; we
will discuss them in more detail in Chapter 9.

1.1.3.1 The Internet of Things

IoT extends the Internet to everyday objects, interconnecting devices, tools, and computers via
sensors, RFID, and GPS.
History: Introduced in 1999 at MIT, IoT enables communication between objects and people.
IPv6 Impact: With 2¹²⁸ IP addresses, IoT can assign unique addresses to all objects, tracking up
to 100 trillion static or moving objects.
Communication Models:
 H2H (Human-to-Human)
 H2T (Human-to-Thing)
 T2T (Thing-to-Thing)
Development & Challenges:
 IoT is in its early stages, mainly advancing in Asia and Europe.
 Cloud computing is expected to enhance efficiency, intelligence, and scalability in IoT
interactions.
 Smart Earth Vision: IoT aims to create intelligent cities, clean energy, better healthcare,
and sustainable environments.

1.1.3.2 Cyber-Physical Systems

 A cyber-physical system (CPS) is the result of interaction between computational
processes and the physical world. A CPS integrates “cyber” (heterogeneous,
asynchronous) with “physical” (concurrent and information-dense) objects.
 A CPS merges the “3C” technologies of computation, communication, and control into
an intelligent closed feedback system between the physical world and the information
world, a concept which is actively explored in the United States.
 The IoT emphasizes various networking connections among physical objects, while the
CPS emphasizes exploration of virtual reality (VR) applications in the physical world.
We may transform how we interact with the physical world just like the Internet
transformed how we interact with the virtual world. We will study IoT, CPS, and their
relationship to cloud computing

8
Cse bkec
 Cloud computing

1.2 Technologies for Network-Based Systems

 With the concept of scalable computing under our belt, it’s time to explore hardware,
software, and network technologies for distributed computing system design and
applications.
 In particular, we will focus on viable approaches to building distributed operating
systems for handling massive parallelism in a distributed environment.

1.2.1 Multicore CPUs and Multithreading Technologies

 Consider the growth of component and network technologies over the past 30 years. They
are crucial to the development of HPC and HTC systems. In Figure 1.4, processor speed
is measured in millions of instructions per second (MIPS) and network bandwidth is
measured in megabits per second (Mbps) or gigabits per second (Gbps). The unit GE
refers to 1 Gbps Ethernet bandwidth.

FIGURE 1.4 Improvement in processor and network technologies over 33 years.

1.2.1.1 Advances in CPU Processors

 Today, advanced CPUs or microprocessor chips assume a multicore architecture with
dual, quad, six, or more processing cores. These processors exploit parallelism at ILP and
TLP levels.
 Processor speed growth is plotted in the upper curve in Figure 1.4 across generations of
microprocessors or CMPs. We see growth from 1 MIPS for the VAX 780 in 1978 to
1,800 MIPS for the Intel Pentium 4 in 2002, up to a 22,000 MIPS peak for the Sun
Niagara 2 in 2008.
 Both multi-core CPU and many-core GPU processors can handle multiple instruction
threads at different magnitudes today.
 Figure 1.5 shows the architecture of a typical multicore processor. Each core is
essentially a processor with its own private cache (L1 cache).

9
Cse bkec
 Cloud computing

 Multiple cores are housed in the same chip with an L2 cache that is shared by all cores. In
the future, multiple CMPs could be built on the same CPU chip with even the L3 cache
on the chip.
 Multicore and multithreaded CPUs are equipped with many high-end processors,
including the Intel i7, Xeon, AMD Opteron, Sun Niagara, IBM Power 6, and X cell
processors. Each core could be also multithreaded.
 For example, the Niagara II is built with eight cores with eight threads handled by each
core. This implies that the maximum ILP and TLP that can be exploited in Niagara is 64
(8 × 8 = 64). In 2011, the Intel Core i7 990x has reported 159,000 MIPS execution rate as
shown in the upper- most square in Figure 1.4.

FIGURE 1.5 Schematic of a modern multicore CPU chip using a hierarchy of caches, where L1
cache is private to each core, on-chip L2 cache is shared and L3 cache or DRAM Is off the chip.

1.2.1.2 Multicore CPU and Many-Core GPU Architectures

 Multicore CPUs may increase from the tens of cores to hundreds or more in the future.
But the CPU has reached its limit in terms of exploiting massive DLP due to the
aforementioned memory wall problem. This has triggered the development of many-core
GPUs with hundreds or more thin cores.
 Both IA-32 and IA-64 instruction set architectures are built into commercial CPUs. Now,
x-86 processors have been extended to serve HPC and HTC systems in some high-end
server processors.

1.2.1.3 Multithreading Technology

 Consider in Figure 1.6 the dispatch of five independent threads of instructions to four
pipelined data paths (functional units) in each of the following five processor categories,
from left to right: a four-issue
 superscalar processor,
 a fine-grain multithreaded processor,
 a coarse-grain multithreaded processor,
 a two-core CMP, and
 a simultaneous multithreaded (SMT) processor.

10
Cse bkec
 Cloud computing

 The superscalar processor is single-threaded with four functional units. Each of the three
multithreaded processors is four-way multithreaded over four functional data paths.
 In the dual-core processor, assume two processing cores, each a single-threaded two-way
superscalar processor.

FIGURE 1.6 Five micro-architectures in modern CPU processors, that exploit ILP and TLP
supported by multicore and multithreading technologies.

 Instructions from different threads are distinguished by specific shading patterns for
instructions from five independent threads. Typical instruction scheduling patterns are
shown here. Only instructions from the same thread are executed in a superscalar
processor.
 Fine-grain multithreading switches the execution of instructions from different threads
per cycle.
 Course-grain multithreading executes many instructions from the same thread for quite
a few cycles before switching to another thread.
 The multi core CMP executes instructions from different threads completely.
 The SMT allows simultaneous scheduling of instructions from different threads in the
same cycle.

1.2.2 GPU Computing to Exascale and Beyond

 A GPU is a graphics coprocessor or accelerator mounted on a computer’s graphics card

or video card. A GPU offloads the CPU from tedious graphics tasks in video editing
applications. The world’s first GPU, the GeForce 256, was marketed by NVIDIA in
1999.
 These GPU chips can process a minimum of 10 million polygons per second, and are
used in nearly every computer on the market today.

11
Cse bkec
 Cloud computing

 Unlike CPUs, GPUs have a throughput architecture that exploits massive parallelism by
executing many concurrent threads slowly, instead of executing a single long thread in a
conventional microprocessor very quickly.

1.2.2.1 How GPUs Work

 Early GPUs functioned as coprocessors attached to the CPU. Today, the NVIDIA GPU
has been upgraded to 128 cores on a single chip. Furthermore, each core on a GPU can
handle eight threads of instructions.
 This translates to having up to 1,024 threads executed concurrently on a single GPU. This
is true massive parallelism, compared to only a few threads that can be handled by a
conventional CPU. The CPU is optimized for latency caches, while the GPU is optimized
to deliver much higher throughput with explicit management of on-chip memory.

1.2.2.2 GPU Programming Model

 Figure 1.7 shows the interaction between a CPU and GPU in performing parallel
execution of floating-point operations concurrently. The CPU is the conventional
multicore processor with limited parallelism to exploit. The GPU has a many-core
architecture that has hundreds of simple processing cores organized as multiprocessors.
 Each core can have one or more threads. Essentially, the CPU’s floating-point kernel
computation role is largely offloaded to the many-core GPU. The CPU instructs the GPU
to perform massive data processing. The bandwidth must be matched between the on-
board main memory and the on-chip GPU memory.
 This process is carried out in NVIDIA’s CUDA programming using the GeForce 8800 or
Tesla and Fermi GPUs.

12
Cse bkec
 Cloud computing

FIGURE 1.7 The use of a GPU along with a CPU for massively parallel execution in hundreds or
thousands of processing cores.

1.2.2.3 Power Efficiency of the GPU

 Bill Dally of Stanford University considers power and massive parallelism as the major
benefits of GPUs over CPUs for the future. By extrapolating current technology and
computer architecture, it was estimated that 60 Gflops/watt per core is needed to run an
exaflops system (see Figure 1.10).
 Power constrains what we can put in a CPU or GPU chip. Dally has estimated that the
CPU chip consumes about 2 nJ/instruction, while the GPU chip requires 200
pJ/instruction, which is 1/10 less than that of the CPU. The CPU is optimized for latency
in caches and memory, while the GPU is optimized for throughput with explicit
management of on-chip memory.

1.2.3 Memory, Storage, and Wide-Area Networking

1.2.3.1 Memory Technology

 The upper curve in Figure 1.10 plots the growth of DRAM chip capacity from 16 KB in
1976 to 64 GB in 2011. This shows that memory chips have experienced a 4x increase in
capacity every three years. Memory access time did not improve much in the past.
 In fact, the memory wall problem is getting worse as the processor gets faster. For hard
drives, capacity increased from 260 MB in 1981 to 250 GB in 2004. The Seagate
Barracuda XT hard drive reached 3 TB in 2011. This represents an approximately 10x
increase in capacity every eight years. The capacity increase of disk arrays will be even
greater in the years to come.
 Faster processor speed and larger memory capacity result in a wider gap between
processors and memory. The memory wall may become even worse a problem limiting
the CPU performance in the future.
1.2.3.2 Disks and Storage Technology
 Beyond 2011, disks or disk arrays have exceeded 3 TB in capacity. The lower curve in
Figure 1.10 shows the disk storage growth in 7 orders of magnitude in 33 years.
 The rapid growth of flash memory and solid-state drives (SSDs) also impacts the future
of HPC and HTC systems. The mortality rate of SSD is not bad at all.
 A typical SSD can handle 300,000 to 1 million write cycles per block.

13
Cse bkec
 Cloud computing

1.2.3.3 System-Area Interconnects

 The nodes in small clusters are mostly interconnected by an Ethernet switch or a local
area network (LAN). As Figure 1.11 shows, a LAN typically is used to connect client
hosts to big servers. A storage area network (SAN) connects servers to network storage
such as disk arrays. Network attached storage (NAS) connects client hosts directly to the
disk arrays. All three types of networks often appear in a large cluster built with
commercial network components

1.2.3.4 Wide-Area Networking

 The lower curve in Figure 1.10 plots the rapid growth of Ethernet bandwidth from 10
Mbps in 1979 to 1 Gbps in 1999, and 40 ~ 100 GE in 2011. It has been speculated that 1
Tbps network links will become available by 2013. According to Berman, Fox, and Hey
[6], network links with 1,000, 1,000, 100, 10, and 1 Gbps bandwidths were reported,
respectively, for international, national, organization, optical desktop, and copper desktop
connections in 2006.
 An increase factor of two per year on network performance was reported, which is faster
than Moore’s law on CPU speed doubling every 18 months.

14
Cse bkec
 Cloud computing

 The implication is that more computers will be used concurrently in the future. High-
bandwidth networking increases the capability of building massively distributed systems.
 The IDC 2010 report predicted that both InfiniBand and Ethernet will be the two major
interconnect choices in the HPC arena. Most data centers are using Gigabit Ethernet as
the inter connect in their server clusters.

1.2.4 Virtual Machines and Virtualization Middleware

 A conventional computer has a single OS image. This offers a rigid architecture that
tightly couples application software to a specific hardware platform. Some software
running well on one machine may not be executable on another platform with a different
instruction set under a fixed OS.
 Virtual machines (VMs) offer novel solutions to underutilized resources, application
inflexibility, software manageability, and security concerns in existing physical
machines.
 Today, to build large clusters, grids, and clouds, we need to access large amounts of
computing, storage, and networking resources in a virtualized manner. We need to
aggregate those resources, and hopefully, offer a single system image. In particular, a
cloud of provisioned resources must rely on virtualization of processors, memory, and
I/O facilities dynamically. We will cover virtualization in Chapter 3. However, the basic
concepts of virtualized resources, such as VMs, virtual storage, and virtual networking
and their virtualization software or middleware, need to be introduced first. Figure 1.12
illustrates the architectures of three VM configurations.

FIGURE 1.12 Three VM architectures in (b), (c), and (d), compared with the traditional physical
machine shown in (a). C

1.2.4.1 Virtual Machines

 In Figure 1.12, the host machine is equipped with the physical hardware, as shown at the
bottom of the figure. An example is an x-86 architecture desktop running its installed
Windows OS, as shown in part (a) of the figure. The VM can be provisioned for any

15
Cse bkec
 Cloud computing

hardware system. The VM is built with virtual resources managed by a guest OS to run a
specific application.
 Between the VMs and the host platform, one needs to deploy a middleware layer called a
virtual machine monitor (VMM). Figure 1.12(b) shows a native VM installed with the
use of a VMM called a hypervisor in privileged mode. For example, the hardware has x-
86 architecture running the Windows system.
 The guest OS could be a Linux system and the hypervisor is the XEN system developed
at Cambridge University. This hypervisor approach is also called bare-metal VM,
because the hypervisor handles the bare hardware (CPU, memory, and I/O) directly.
Another architecture is the host VM shown in Figure 1.12(c). Here the VMM runs in
nonprivileged mode.
 The host OS need not be modified. The VM can also be implemented with a dual mode,
as shown in Figure 1.12(d). Part of the VMM runs at the user level and another part runs
at the supervisor level. In this case, the host OS may have to be modified to some extent.
Multiple VMs can be ported to a given hardware system to support the virtualization
process.

1.2.4.2 VM Primitive Operations

 The VMM provides the VM abstraction to the guest OS. With full virtualization, the
VMM exports a VM abstraction identical to the physical machine so that a standard OS
such as Windows 2000 or Linux can run just as it would on the physical hardware. Low-
level VMM operations are indicated by Mendel Rosenblum [41] and illustrated in Figure
1.13.
 Multiplexing: Multiple VMs run on a single hardware machine. Suspension
 & Storage: VM state is saved for later use. Resumption: Suspended VM can be restored
on a different machine.

16
Cse bkec
 Cloud computing

 Migration: VM moves across platforms, enhancing flexibility.

 Benefits: Improved resource utilization, reduced server sprawl, increased efficiency
(VMware reports 60–80% server utilization).

1.2.5 Data Center Virtualization for Cloud Computing

 In this section, we discuss basic architecture and design considerations of data centers.
Cloud architecture is built with commodity hardware and network devices.
 Almost all cloud platforms choose the popular x86 processors.
 Low-cost terabyte disks and Gigabit Ethernet are used to build data centers. Data center
design emphasizes the performance/price ratio over speed performance alone. In other
words, storage and energy efficiency are more important than shear speed performance.
 Figure 1.13 shows the server growth and cost breakdown of data centers over the past 15
years. Worldwide, about 43 million servers are in use as of 2010. The cost of utilities
exceeds the cost of hardware after three years.

1.2.5.1 Data Center Growth and Cost Breakdown

 A large data center may be built with thousands of servers. Smaller data centers are
typically built with hundreds of servers. The cost to build and maintain data center
servers has increased over the years.
 According to a 2009 IDC report typically only 30 percent of data center costs goes
toward purchasing IT equipment (such as servers and disks), 33 percent is attributed to
the chiller, 18 percent to the uninterruptible power supply (UPS), 9 percent to computer
room air conditioning (CRAC), and the remaining 7 percent to power distribution,
lighting, and transformer costs.
 Thus, about 60 percent of the cost to run a data center is allocated to management and
maintenance. The server purchase cost did not increase much with time. The cost of
electricity and cooling did increase from 5 percent to 14 percent in 15 years.

1.2.5.2 Low-Cost Design Philosophy

 High-end switches or routers may be too cost-prohibitive for building data centers. Thus,
using high-bandwidth networks may not fit the economics of cloud computing.
 Given a fixed budget, commodity switches and networks are more desirable in data
centers.
 Similarly, using commodity x86 servers is more desired over expensive mainframes. The
software layer handles network traffic balancing, fault tolerance, and expandability.
 Currently, nearly all cloud computing data centers use Ethernet as their fundamental
network technology.

1.2.5.3 Convergence of Technologies

17
Cse bkec
 Cloud computing

 Essentially, cloud computing is enabled by the convergence of technologies in four areas:

(1) hardware virtualization and multi-core chips, (2) utility and grid computing, (3) SOA,
Web 2.0, and WS mashups, and (4) atonomic computing and data center automation.
 Hardware virtualization and multicore chips enable the existence of dynamic
configurations in the cloud. Utility and grid computing technologies lay the necessary
foundation for computing clouds. Recent advances in SOA, Web 2.0, and mashups of
platforms are pushing the cloud another step forward.
 Iterative MapReduce extends MapReduce to support a broader range of data mining
algorithms commonly used in scientific applications. The cloud runs on an extremely
large cluster of commodity computers. Internal to each cluster node, multithreading is
practiced with a large number of cores in many-core GPU clusters.

1.3 System Models for Distributed and Cloud Computing

 Distributed and cloud computing systems are built over a large number of autonomous
computer nodes. These node machines are interconnected by SANs, LANs, or WANs in a
hierarchical manner.
 With today’s networking technology, a few LAN switches can easily connect hundreds of
machines as a working cluster. A WAN can connect many local clusters to form a very
large cluster of clusters. In this sense, one can build a massive system with millions of
computers connected to edge networks.
 Massive systems are considered highly scalable, and can reach web-scale connectivity,
either physically or logically. In Table 1.2, massive systems are classified into four
groups: clusters, P2P networks, computing grids, and Internet clouds over huge data
centers. In terms of node number, these four system classes may involve hundreds,
thousands, or even millions of computers as participating nodes.
 These machines work collectively, cooperatively, or collaboratively at various levels. The
table entries characterize these four system classes in various technical and application
aspects.

18
Cse bkec
 Cloud computing

1.3.1 Clusters of Cooperative Computers

 A computing cluster consists of interconnected stand-alone computers which work
cooperatively as a single integrated computing resource. In the past, clustered computer
systems have demonstrated impressive results in handling heavy workloads with large
data sets.

1.3.1.1 Cluster Architecture

 Figure 1.15 shows the architecture of a typical server cluster built around a low-latency,
high-bandwidth interconnection network.
 This network can be as simple as a SAN (e.g., Myrinet) or a LAN (e.g., Ethernet). To
build a larger cluster with more nodes, the interconnection network can be built with
multiple levels of Gigabit Ethernet, Myrinet, or InfiniBand switches.
 Through hierarchical construction using a SAN, LAN, or WAN, one can build scalable
clusters with an increasing number of nodes.
 The cluster is connected to the Internet via a virtual private network (VPN) gateway. The
gateway IP address locates the cluster.
 The system image of a computer is decided by the way the OS manages the shared cluster
resources.

19
Cse bkec
 Cloud computing

 Most clusters have loosely coupled node computers. All resources of a server node are
managed by their own OS.
 Thus, most clusters have multiple system images as a result of having many autonomous
nodes under different OS control.

FIGURE 1.15 A cluster of servers interconnected by a high-bandwidth SAN or LAN with shared
I/O devices and disk arrays; the cluster acts as a single computer attached to the Internet.

1.3.1.2 Single-System Image

 Greg Pfister [38] has indicated that an ideal cluster should merge multiple system images
into a single-system image (SSI).
 Cluster designers desire a cluster operating system or some middleware to support SSI at
various levels, including the sharing of CPUs, memory, and I/O across all cluster nodes.
 An SSI is an illusion created by software or hardware that presents a collection of
resources as one integrated, powerful resource.
 SSI makes the cluster appear like a single machine to the user. A cluster with multiple
system images is nothing but a collection of independent computers.

1.3.1.3 Hardware, Software, and Middleware Support

 we will discuss cluster design principles for both small and large clusters. Clusters
exploring massive parallelism are commonly known as MPPs. Almost all HPC clusters in
the Top 500 list are also MPPs.
 The building blocks are computer nodes (PCs, workstations, servers, or SMP), special
communication software such as PVM or MPI, and a network interface card in each
computer node. Most clusters run under the Linux OS.
 The computer nodes are interconnected by a high-bandwidth network (such as Gigabit
Ethernet, Myrinet, InfiniBand, etc.).
 Special cluster middleware supports are needed to create SSI or high availability (HA).
Both sequential and parallel applications can run on the cluster, and special parallel
environments are needed to facilitate use of the cluster resources.

20
Cse bkec
 Cloud computing

 For example, distributed memory has multiple images. Users may want all distributed
memory to be shared by all servers by forming distributed shared memory (DSM).
 Many SSI features are expensive or difficult to achieve at various cluster operational
levels. Instead of achieving SSI, many clusters are loosely coupled machines. Using
virtualization, one can build many virtual

1.3.1.4 Major Cluster Design Issues

 Unfortunately, a cluster-wide OS for complete resource sharing is not available yet.

Middleware or OS extensions were developed at the user space to achieve SSI at selected
functional levels. Without this middleware, cluster nodes cannot work together
effectively to achieve cooperative computing.
 The software environments and applications must rely on the middleware to achieve high
performance.

1.3.2 Grid Computing Infrastructures

 In the past 30 years, users have experienced a natural growth path from Internet to web
and grid computing services. Internet services such as the Telnet command enables a
local computer to connect to a remote computer.
 A web service such as HTTP enables remote access of remote web pages. Grid
computing is envisioned to allow close interaction among applications running on distant
computers simultaneously.
 Forbes Magazine has projected the global growth of the IT-based economy from $1
trillion in 2001 to $20 trillion by 2015. The evolution from Internet to web and grid
services is certainly playing a major role in this growth.

1.3.2.1 Computational Grids

 Like an electric utility power grid, a computing grid offers an infrastructure that couples
computers, software/middleware, special instruments, and people and sensors together.
The grid is often constructed across LAN, WAN, or Internet backbone networks at a
regional, national, or global scale.
 Enterprises or organizations present grids as integrated computing resources. Personal
computers, laptops, and PDAs can be used as access devices to a grid system. Figure 1.16
shows an example computational grid built over multiple resource sites owned by
different organizations.
 The grid is built across various IP broadband networks including LANs and WANs
already used by enterprises or organizations over the Internet. The grid is presented to

21
Cse bkec
 Cloud computing

users as an integrated resource pool as shown in the upper half of the figure.

FIGURE 1.16 Computational grid or data grid providing computing utility, data, and information
services through resource sharing and cooperation among participating organizations.

1.3.2.2 Grid Families

 Grid technology demands new distributed computing models, software/middleware

support, network protocols, and hardware infrastructures. National grid projects are
followed by industrial grid platform development by IBM, Microsoft, Sun, HP, Dell,
Cisco, EMC, Platform Computing, and others.
 New grid service providers (GSPs) and new grid applications have emerged rapidly,
similar to the growth of Internet and web services in the past two decades. In Table 1.4,
grid systems are classified in essentially two categories: computational or data grids and
P2P grids. Computing or data grids are built primarily at the national level

22
Cse bkec
 Cloud computing

1.3.3 Peer-to-Peer Network Families

 An example of a well-established distributed system is the client-server architecture. In

this scenario, client machines (PCs and workstations) are connected to a central server for
compute, e-mail, file access, and database applications.
 The P2P architecture offers a distributed model of networked systems. First, a P2P
network is client-oriented instead of server-oriented. In this section, P2P systems are
introduced at the physical level and overlay networks at the logical level.

1.3.3.1 P2P Systems

 In a P2P system, every node acts as both a client and a server, providing part of the
system resources. Peer machines are simply client computers connected to the Internet.
All client machines act autonomously to join or leave the system freely.
 This implies that no master-slave relationship exists among the peers. No central
coordination or central database is needed. In other words, no peer machine has a global
view of the entire P2P system. The system is self-organizing with distributed control.
 Figure 1.17 shows the architecture of a P2P network at two abstraction levels. Initially,
the peers are totally unrelated.
 The physical network is simply an ad hoc network formed at various Internet domains
randomly using the TCP/IP and NAI protocols. Thus, the physical network varies in size
and topology dynamically due to the free membership in the P2P network.

FIGURE 1.17 The structure of a P2P system by mapping a physical IP network to an overlay
network built with virtual links.

1.3.3.2 Overlay Networks

 Data items or files are distributed in the participating peers. Based on communication or
file-sharing needs, the peer IDs form an overlay network at the logical level.

23
Cse bkec
 Cloud computing

 This overlay is a virtual network formed by mapping each physical machine with its ID,
logically, through a virtual mapping as shown in Figure 1.17.
 When a new peer joins the system, its peer ID is added as a node in the overlay network.
When an existing peer leaves the system, its peer ID is removed from the overlay
network automatically.
 Therefore, it is the P2P overlay network that characterizes the logical connectivity among
the peers.
 There are two types of overlay networks: unstructured and structured. An unstructured
overlay network is characterized by a random graph. There is no fixed route to send
messages or files among the nodes. Often, flooding is applied to send a query to all nodes
in an unstructured overlay, thus resulting in heavy network traffic and nondeterministic
search results

1.3.3.3 P2P Application Families

 Based on application, P2P networks are classified into four groups, as shown in Table
1.5. The first family is for distributed file sharing of digital contents (music, videos, etc.)
on the P2P network. This includes many popular P2P networks such as Gnutella, Napster,
and BitTorrent, among others.
 Collaboration P2P networks include MSN or Skype chatting, instant messaging, and
collaborative design, among others. The third family is for distributed P2P computing in
specific applications. For example, SETI@home provides 25 Tflops of distributed
computing power, collectively, over 3 million Internet host machines.
 Other P2P platforms, such as JXTA, .NET, and FightingAID@home, support naming,
discovery, communication, security, and resource aggregation in some P2P applications.

1.3.3.4 P2P Computing Challenges

 Heterogeneity: Hardware, software, and network incompatibilities.

 Scalability: Handling increased workloads.
 Security & Privacy: Lack of trust, copyright concerns.
 Performance Issues: Data location, routing efficiency, load balancing.
 Fault Tolerance: Replication prevents single points of failure.

1.3.4 Cloud Computing Over the Internet

 Gordon Bell, Jim Gray, and Alex Szalay [5] have advocated: “Computational science is
changing to be data-intensive.
 Supercomputers must be balanced systems, not just CPU farms but also petascale I/O and
networking arrays.” In the future, working with large data sets will typically mean

24
Cse bkec
 Cloud computing

sending the computations (programs) to the data, rather than copying the data to the
workstations.
 This reflects the trend in IT of moving computing and data from desktops to large data
centers, where there is on-demand provision of software, hardware, and data as a service.
 This data explosion has promoted the idea of cloud computing.

1.3.4.1 Internet Clouds

 Cloud computing applies a virtualized platform with elastic resources on demand by

provisioning hardware, software, and data sets dynamically (see Figure 1.18).
 The idea is to move desktop computing to a service-oriented platform using server
clusters and huge databases at data centers.
 Cloud computing leverages its low cost and simplicity to benefit both users and
providers. Machine virtualization has enabled such cost-effectiveness.
 Cloud computing intends to satisfy many user applications simultaneously. The cloud
ecosystem must be designed to be secure, trustworthy, and dependable.
 Some computer users think of the cloud as a centralized resource pool. Others consider
the cloud to be a server cluster which practices distributed computing over all the servers
used.

FIGURE 1.18 Virtualized resources from data centers to form an Internet cloud, provisioned
with hardware, software, storage, network, and services for paid users to run their applications

1.3.4.2 The Cloud Landscape

 Traditionally, a distributed computing system tends to be owned and operated by an

autonomous administrative domain (e.g., a research laboratory or company) for on-
premises computing needs.
 However, these traditional systems have encountered several performance bottlenecks:
constant system maintenance, poor utilization, and increasing costs associated with
hardware/software upgrades.
 Cloud computing as an on-demand computing paradigm resolves or relieves us from
these problems.

25
Cse bkec
 Cloud computing

• Infrastructure as a Service (IaaS)

This model puts together infrastructures demanded by users—namely servers, storage, networks,
and the data center fabric.
The user can deploy and run on multiple VMs running guest OSes on specific applications.
The user does not manage or control the underlying cloud infrastructure, but can specify when to
request and release the needed resources.

• Platform as a Service (PaaS)

This model enables the user to deploy user-built applications onto a virtualized cloud platform.
PaaS includes middleware, databases, development tools, and some runtime support such as Web
2.0 and Java.
The platform includes both hardware and software integrated with specific programming
interfaces. The provider supplies the API and software tools (e.g., Java, Python, Web 2.0, .NET).
The user is freed from managing the cloud infrastructure.

• Software as a Service (SaaS)

This refers to browser-initiated application software over thousands of paid cloud customers.
The SaaS model applies to business processes, industry applications, consumer relationship
management (CRM), enterprise resources planning (ERP), human resources (HR), and
collaborative applications.
On the customer side, there is no upfront investment in servers or software licensing. On the
provider side, costs are rather low, compared with conventional hosting of user applications.

The following list highlights eight reasons to adapt the cloud for upgraded Internet
applications and web services:
1. Desired location in areas with protected space and higher energy efficiency
2. Sharing of peak-load capacity among a large pool of users, improving overall utilization
3. Separation of infrastructure maintenance duties from domain-specific application development
4. Significant reduction in cloud computing cost, compared with traditional computing
paradigms5. Cloud computing programming and application development
6. Service and data discovery and content/service distribution
7. Privacy, security, copyright, and reliability issues
8. Service agreements, business models, and pricing policies

26
Cse bkec