ITC4205: Introduction To

Cloud Computing
Distributed System
• A distributed system is a collection of autonomous computers or nodes that work
together as a unified system, interconnected through a network. These nodes
collaborate and communicate with each other to achieve a common goal, such as
sharing resources, performing tasks, or solving complex problems.
• In a distributed system, each node operates independently and has its own
processing power, memory, and storage. The nodes interact with each other by
exchanging messages, sharing data, and coordinating their actions. This
decentralized nature allows for better scalability, fault tolerance, and improved
performance compared to a single centralized system.
• Distributed systems can be found in various domains, including cloud computing,
web services, scientific research, financial transactions, and many others. They
enable the efficient utilization of resources, parallel processing, and effective
distribution of workloads.
Distributed System
Overall, a distributed system is an infrastructure that enables multiple
computers or nodes to work together as a unified system, leveraging
their individual capabilities and collaborating through network
communication to achieve common goals efficiently and reliably.
Distributed System Models
1. Client-Server Model: The client-server model is a fundamental
distributed system model where the system is divided into two main
components: clients and servers. Clients initiate requests for services or
resources, while servers respond to these requests. This model allows for
centralized management and provides a straightforward approach to
building distributed systems. Examples include web applications, where
clients (web browsers) request data from servers (web servers).
2. Peer-to-Peer (P2P) Model: The peer-to-peer model is a decentralized
distributed system model where all participating computers, known as
peers, have equal capabilities and responsibilities. Peers communicate
and collaborate directly with each other to share resources and perform
tasks. P2P systems are often used for file sharing, content distribution,
and collaborative applications. Examples include BitTorrent for file
sharing and blockchain networks.
Distributed System Models
3. Publish-Subscribe Model: The publish-subscribe model is based on event-
driven communication. Publishers generate events or messages and send
them to a message broker or event bus. Subscribers register their
interest in specific types of events and receive them from the broker. This
model allows for loosely coupled communication and is well-suited for
real-time systems, IoT applications, and messaging systems.
4. Master-Slave Model: The master-slave model involves a centralized
controller, known as the master, that manages and coordinates multiple
slave nodes. The master assigns tasks to the slaves, and they execute
those tasks. The master maintains overall control and monitors the
progress of the distributed system. This model is commonly used in
parallel computing, where the master divides a large problem into
smaller subproblems that are solved by the slave nodes.
Distributed System Models
5. Hybrid Model: The hybrid model combines elements of both the
client-server and peer-to-peer models. It involves a combination of
centralized servers and decentralized peer-to-peer interactions. This
model allows for the benefits of both approaches, providing
scalability, fault-tolerance, and efficient resource utilization. Hybrid
models are commonly used in large-scale systems, such as content
delivery networks (CDNs) or distributed databases.
Distributed System Models
• These distributed system models offer different approaches to
organizing and managing resources, communication, and
coordination in distributed environments. They cater to various
requirements and use cases, allowing developers to choose the most
suitable model based on factors such as scalability, fault tolerance,
and system architecture.
• By understanding these models, developers can design distributed
systems that effectively leverage the advantages of distributed
computing, such as scalability, reliability, and resource sharing. Each
model has its strengths and considerations, and selecting the
appropriate model depends on the specific requirements of the
system being developed.
Parallel Computing
• Parallel computing refers to the simultaneous execution of tasks or
computations using multiple processors or computing units. It involves
dividing a problem into smaller subproblems and solving them
concurrently to achieve faster and more efficient processing.
• To understand parallel computing, let's consider a simple analogy: Imagine
you have a large number of apples that need to be sorted based on their
color. If you were doing this task alone, you would examine each apple
individually, determine its color, and place it in the appropriate category.
This sequential approach would take a considerable amount of time.
• Now, let's consider parallel computing. Instead of sorting the apples alone,
you have a team of friends working with you. Each person takes a portion
of the apples and sorts them independently based on color. This
simultaneous sorting drastically reduces the time required to complete the
task.
Parallel Computing
Parallel Computing
• In parallel computing, multiple processors or computing units work together on
different parts of a problem simultaneously. Each processor performs
computations independently on a portion of the data, making progress
concurrently. The individual results are then combined to obtain the final
outcome.
• Parallel computing can significantly improve performance and speed up
computations for tasks that can be divided into independent subproblems. This
approach is particularly useful for computationally intensive applications, such as
scientific simulations, data analysis, image processing, and machine learning.
• However, it's important to note that not all tasks can be parallelized effectively.
Some tasks have dependencies or require sequential execution, which limits their
parallelizability. Additionally, the effectiveness of parallel computing depends on
factors such as the size of the problem, the availability of resources, and the
efficiency of communication and synchronization among the processors.
Parallel Computing
• Parallel computing can be achieved in different ways, including
shared-memory parallelism and distributed-memory parallelism. In
shared-memory parallelism, multiple processors share a common
memory space and can directly access and modify the same data. In
distributed-memory parallelism, each processor has its own separate
memory, and communication between processors occurs through
message passing.
• To leverage parallel computing effectively, developers use specialized
programming models, libraries, and frameworks that provide
abstractions and tools for managing parallel tasks, distributing data,
and coordinating the execution. Examples include OpenMP, MPI,
CUDA, and frameworks like Apache Hadoop and Apache Spark.
Shared Memory vs Distributed Memory
Parallelism
Parallel Computing
In summary, parallel computing involves the simultaneous execution of
tasks or computations using multiple processors or computing units. It
divides a problem into smaller subproblems and solves them
concurrently, resulting in faster processing and improved performance.
By harnessing parallelism, we can tackle complex computations more
efficiently and achieve significant speedups in various domains.
Virtualization
• Virtualization refers to the creation of virtual versions or representations of
various computing resources, such as servers, operating systems, storage devices,
or networks. It enables the sharing and allocation of physical resources among
multiple users or applications, allowing them to operate as if they had dedicated
access to those resources.
• To better understand virtualization, let's consider the analogy of a shared office
space. Imagine a large office building with multiple rooms, desks, and facilities. In
a traditional setup, each person or organization would have their own dedicated
office space. This is similar to how computing resources were traditionally
allocated.
• Now, let's introduce virtualization. Instead of having dedicated office spaces, the
building is divided into virtual offices using partitions or virtual walls. Each virtual
office can be customized and used by different individuals or organizations, while
physically sharing the same infrastructure.
Virtualization
• In computing, virtualization works in a similar way. It allows the
creation of virtual machines (VMs) or virtual environments that run
on a single physical server or computer. Each VM operates as a
separate and independent entity, with its own operating system,
applications, and resources, even though they are sharing the same
underlying hardware.
Virtualization
Virtualization
• A hypervisor, also known as a virtual machine monitor (VMM), is a
software or firmware component that enables the creation and
management of virtual machines (VMs) on a physical computer or server. It
acts as an intermediary layer between the underlying hardware and the
virtual machines running on it.
• To understand the role of a hypervisor, let's continue with the analogy of a
shared office space. Imagine that the virtual offices we discussed earlier in
the context of virtualization are managed by an office manager who
ensures smooth operations and manages the resources.
• Similarly, a hypervisor serves as the manager for virtual machines. It
provides an abstraction layer that virtualizes the underlying hardware,
allowing multiple virtual machines to run independently on the same
physical server.
Functions of Hypervisor
1.Resource Allocation: The hypervisor allocates physical hardware resources, such as CPU,
memory, storage, and network, to each virtual machine. It ensures that the resources are
fairly distributed among the virtual machines and prevents them from interfering with
one another.
2.Isolation: The hypervisor enforces isolation between virtual machines, ensuring that
each VM operates independently and does not impact the performance or stability of
other VMs. This isolation prevents a VM from accessing or affecting resources assigned
to another VM.
3.Hardware Abstraction: The hypervisor presents a virtualized view of the hardware to
each virtual machine, making it appear as if the VM has direct access to dedicated
resources. This abstraction allows the VMs to run different operating systems and
software applications as if they were running on separate physical machines.
4.VM Management: The hypervisor provides management capabilities for virtual
machines, allowing administrators to create, configure, start, stop, and monitor VMs. It
also facilitates tasks such as VM migration, snapshots, and resource adjustments.
5.Performance Optimization: The hypervisor optimizes resource utilization by dynamically
allocating resources based on the needs of each virtual machine. It can adjust resource
allocation in real-time to ensure efficient utilization and maximize overall system
performance.
Benefits of Virtualization
1. Server Consolidation: Virtualization enables the consolidation of
multiple physical servers into a single server running multiple virtual
machines. This allows better utilization of hardware resources and
reduces the number of physical servers required.
2. Resource Allocation and Isolation: Virtualization allows for efficient
allocation of resources, such as CPU, memory, and storage, among
different virtual machines. It also provides isolation between virtual
machines, ensuring that the performance or issues in one virtual
machine do not affect others.
3. Flexibility and Agility: Virtualization provides flexibility in terms of
deploying, managing, and migrating virtual machines. It allows for easy
scaling up or down of resources as needed, and it simplifies the process
of creating and provisioning new virtual machines.
Benefits of Virtualization
4. Testing and Development: Virtualization facilitates the creation of
isolated virtual environments for testing and development
purposes. Developers can create multiple virtual machines with
different configurations, allowing them to test software or
experiment without affecting the production environment.
5. Disaster Recovery and High Availability: Virtualization enables the
creation of backups and snapshots of virtual machines, making
disaster recovery easier. It also allows for features like live
migration, where virtual machines can be moved from one physical
server to another without downtime, providing high availability.
Virtualization (Summary)
• Virtualization can be implemented at various levels, including server
virtualization, storage virtualization, network virtualization, and desktop
virtualization. Each level offers different benefits and use cases.
• Popular virtualization technologies and platforms include VMware,
Microsoft Hyper-V, KVM, and Xen. These tools provide the necessary
software and management capabilities to create and manage virtual
machines and virtual environments.
• In summary, virtualization is the creation of virtual versions of computing
resources, allowing for efficient sharing, allocation, and management of
physical resources. It provides benefits such as server consolidation,
resource allocation, flexibility, testing and development environments, and
disaster recovery. Virtualization plays a crucial role in modern data centers
and cloud computing, enabling efficient and scalable utilization of
resources.
Grid Computing
• Grid computing is a distributed computing model that involves the coordination and
sharing of computing resources across multiple computers or "nodes" that are
geographically dispersed. It enables the pooling of resources, such as processing power,
storage, and data, to collectively solve complex problems or perform large-scale
computations.
• To better understand grid computing, let's consider an analogy: a power grid. Imagine a
network of power stations generating electricity that is distributed and shared across a
large area. Each power station contributes to the overall electricity supply, and
consumers can access the power they need from any available source. Grid computing
operates in a similar manner but with computing resources instead of electrical power.
• In a grid computing system, individual computers or servers, often referred to as
"nodes," contribute their unused processing power and storage capacity to a shared
pool. These nodes are interconnected through a network, allowing them to
communicate and collaborate with one another. The grid infrastructure manages and
coordinates the allocation of resources, scheduling tasks, and distributing data across the
nodes.
Advantages of Grid Computing
1.Resource Sharing: Grid computing enables the efficient utilization of computing
resources by sharing them across multiple nodes. Nodes can contribute their unused
processing power and storage capacity to the grid, allowing others to access and utilize
those resources.
2.Scalability: Grid computing provides scalability by allowing additional nodes to join the
grid. This increases the overall computing power and resources available to handle larger
workloads or more demanding tasks.
3.Collaboration: Grid computing enables collaboration among different organizations or
research groups. They can contribute their resources and expertise to tackle complex
problems that require significant computational power or data analysis.
4.High Performance: By leveraging the collective computing power of multiple nodes, grid
computing can deliver high performance and faster execution times for computationally
intensive tasks.
5.Cost Efficiency: Grid computing allows organizations to optimize resource utilization and
avoid the need for dedicated infrastructure. Instead of investing in costly individual
servers or data centers, they can leverage existing resources within the grid, resulting in
cost savings.
Grid Computing Summary
• Grid computing finds applications in various fields, including scientific research,
engineering simulations, weather forecasting, drug discovery, and financial modeling. It
enables researchers, scientists, and organizations to access significant computing power
and resources that would be impractical to deploy and maintain individually.
• Standards and protocols such as the Open Grid Services Architecture (OGSA) and the
Globus Toolkit have been developed to facilitate the implementation and management
of grid computing environments. These tools provide the necessary infrastructure,
middleware, and scheduling mechanisms to coordinate and optimize resource usage in
grid systems.
• In summary, grid computing is a distributed computing model that enables the
coordination and sharing of computing resources across geographically dispersed nodes.
It offers resource sharing, scalability, collaboration, high performance, and cost efficiency.
Grid computing finds applications in various scientific, research, and data-intensive
domains, providing access to significant computing power and resources.
Microsoft Azure
• Microsoft Azure is a cloud computing platform that offers a wide range of
services and capabilities to build, deploy, and manage applications in the cloud. It
provides the necessary infrastructure and tools to implement various cloud
computing architectures, making it a versatile platform for organizations looking
to leverage cloud technologies.
• Azure provides a flexible and scalable infrastructure that includes virtual
machines, storage, and networking capabilities. It also offers a variety of platform
services, such as databases, analytics, artificial intelligence, and internet of things
(IoT) solutions. In addition, Azure includes development tools, management
tools, and a global network of data centers to support the deployment and
operation of cloud-based applications.
• Azure follows a pay-as-you-go pricing model, allowing organizations to scale
resources up or down based on their needs, making it suitable for both small-
scale projects and enterprise-level deployments.
Amazon AWS
• Amazon Web Services (AWS) is a comprehensive cloud computing platform provided by Amazon.
It offers a wide range of cloud services and solutions that enable businesses and individuals to
build, deploy, and manage applications and services in the cloud.
• AWS provides a global infrastructure of data centers, offering compute power, storage, and
networking capabilities across various regions worldwide. It offers a diverse set of services,
including compute (e.g., Amazon EC2 for virtual servers), storage (e.g., Amazon S3 for object
storage), databases (e.g., Amazon RDS for managed databases), networking (e.g., Amazon VPC for
virtual private cloud), security (e.g., AWS Identity and Access Management), analytics (e.g.,
Amazon Redshift for data warehousing), machine learning (e.g., Amazon SageMaker), and many
more.
• AWS allows users to choose and combine these services based on their specific needs, providing
flexibility, scalability, and cost-effectiveness. It offers a pay-as-you-go pricing model, allowing
users to pay only for the resources they consume without any upfront costs or long-term
commitments.
• AWS has gained significant popularity and market share in the cloud computing industry due to its
extensive service portfolio, global presence, and reliability. It caters to a wide range of customers,
from startups to large enterprises, and supports various use cases, such as web applications,
mobile applications, big data processing, artificial intelligence, and more.
Google MapReduce
• MapReduce is a programming model and framework developed by Google
for processing and analyzing large datasets in a distributed manner. It was
designed to run on Google's cloud infrastructure, known as Google Cloud
Platform (GCP).
• In the MapReduce architecture, data processing tasks are divided into two
main phases: the "Map" phase and the "Reduce" phase. The Map phase
involves splitting the input data into smaller chunks and processing them in
parallel across multiple nodes in the cloud. The Reduce phase combines
the intermediate results generated by the Map phase to produce the final
output.
• Google MapReduce takes advantage of the cloud's scalability and
distributed computing capabilities. It automatically handles the distribution
of data and computation across multiple nodes, allowing for efficient
processing of large datasets. The architecture provides fault tolerance, as it
can handle node failures and ensure the completion of tasks.