UNIT 3

VIRTUALIZATION INFRASTRUCTURE AND DOCKER

Desktop Virtualization – Network Virtualization – Storage Virtualization – System-level

of Operating Virtualization – Application Virtualization – Virtual clusters and Resource
Management – Containers vs. Virtual Machines – Introduction to Docker – Docker
Components – Docker Container – Docker Images and Repositories.

What is desktop virtualization?

Desktop virtualization creates a software-based (or virtual) version of an end user’s
desktop environment and operating system (OS) that is decoupled from the end user’s
computing device or client. This enables the user to access his or her desktop from
any computing device.

Desktop virtualization deployment models

Virtual desktop infrastructure (VDI)

In VDI deployment model, the operating system runs on a virtual machine (VM)
hosted on a server in a data center. The desktop image travels over the network to
the end user’s device, where the end user can interact with the desktop (and the
underlying applications and operating system) as if they were local.

VDI gives each user his or her own dedicated VM running its own operating system.
The operating system resources—drivers, CPUs, memory, etc.—operate from a
software layer called a hypervisor that mimics their output, manages the resource
allocation to multiple VMs, and allows them to run side by side on the same server.
A key benefit of VDI is that it can deliver the Windows 10 desktop and operating
system to the end user’s devices. However, because VDI supports only one user per
Windows 10 instance, it requires a separate VM for each Windows 10 user.

Remote desktop services (RDS)

In RDS—also known as Remote Desktop Session Host (RDSH)—users remotely access

desktops and Windows applications through the Microsoft Windows Server operating
system. Applications and desktop images are served via Microsoft Remote Desktop
Protocol (RDP). Formerly known as Microsoft Terminal Server, this product has
remained largely unchanged since its initial release.

From the end user’s perspective, RDS and VDI are identical. But because one instance
of Windows Server can support as many simultaneous users as the server hardware
can handle, RDS can be a more cost-effective desktop virtualization option. It’s also
worth noting applications tested or certified to run on Windows 10 may not be tested
or certified to run on the Windows Server OS.

Desktop-as-a-Service (DaaS)

In DaaS, VMs are hosted on a cloud-based backend by a third-party provider. DaaS is

readily scalable, can be more flexible than on-premise solutions, and generally deploys
faster than many other desktop virtualization options.

Like other types of cloud desktop virtualization, DaaS shares many of the
general benefits of cloud computing, including support for fluctuating workloads and
changing storage demands, usage-based pricing, and the ability to make applications
and data accessible from almost any internet-connected device. The chief drawback
to DaaS is that features and configurations are not always as customizable as required.
Choosing a model
VDI is a popular choice because it offers a virtualized version of a familiar computing
model—physical desktop computing. But implementing VDI requires you to manage
all aspects of the infrastructure yourself, including the hardware, operating systems
and applications, and hypervisor and associated software. This can be challenging if
your VDI experience and expertise is limited. Purchasing all infrastructure components
can require a larger upfront investment.

RDS/RDSH can be a solid choice if it supports the specific applications you need to run
and your end users only need access to those applications, not full Windows desktops.
RDS offers greater end-user density per server than VDI, and systems are usually
cheaper and more scalable than full VDI environments. Your staff does need the
requisite skill set and experience to administer and manage RDS/RDSH technology,
however.

DaaS is currently gaining in popularity as IT teams grow more comfortable with shared
desktops and shared applications. Overall, it tends to be the most cost-effective
option. It’s also the easiest to administer, requiring little in-house expertise in
managing infrastructure or VDI. It’s readily scalable and involves operating
expenditures rather than capital expenditures, a more affordable cost structure for
many businesses.
Benefits of desktop virtualization

Virtualizing desktops provides many potential benefits that can vary depending upon
the deployment model you choose.

Simpler administration. Desktop virtualization can make it easier for IT teams to

manage employee computing needs. Your business can maintain a single VM template
for employees within similar roles or functions instead of maintaining individual
computers that must be reconfigured, updated, or patched whenever software
changes need to be made. This saves time and IT resources.

Cost savings. Many virtual desktop solutions allow you to shift more of your IT
budget from capital expenditures to operating expenditures. Because compute-
intensive applications require less processing power when they’re delivered via VMs
hosted on a data center server, desktop virtualization can extend the life of older or
less powerful end-user devices. On-premise virtual desktop solutions may require a
significant initial investment in server hardware, hypervisor software, and other
infrastructure, making cloud-based DaaS—wherein you simply pay a regular usage-
based charge—a more attractive option.
Improved productivity. Desktop virtualization makes it easier for employees to
access enterprise computing resources. They can work anytime, anywhere, from any
supported device with an Internet connection.

Support for a broad variety of device types. Virtual desktops can support remote
desktop access from a wide variety of devices, including laptop and desktop
computers, thin clients, zero clients, tablets, and even some mobile phones. You can
use virtual desktops to deliver workstation-like experiences and access to the full
desktop anywhere, anytime, regardless of the operating system native to the end user
device.

Stronger security. In desktop virtualization, the desktop image is abstracted and

separated from the physical hardware used to access it, and the VM used to deliver
the desktop image can be a tightly controlled environment managed by the enterprise
IT department.

Agility and scalability. It’s quick and easy to deploy new VMs or serve new
applications whenever necessary, and it is just as easy to delete them when they’re
no longer needed.

Better end-user experiences. When you implement desktop virtualization, your

end users will enjoy a feature-rich experience without sacrificing functionality they’ve
come to rely on, like printing or access to USB ports.
Desktop virtualization software

The software required for delivering virtual desktops depends on the virtualization
method you chose.

With virtual desktop infrastructure (VDI), the desktop operating system (most
commonly Microsoft Windows) runs and is managed in the data center. Hypervisor
software runs on the host server, delivering access to a VM to each end user over
the network. Connection broker software is required to authenticate users, connect
each to a virtual machine, monitor activity levels, and reassign the VM when the
connection is terminated. Connection brokers may be bundled with, or purchased
separately from, the hypervisor.

Remote desktop services (RDS/RDSH) can be implemented using utilities that are
bundled with the Microsoft Windows Server operating system.

If you choose a Desktop-as-a-Service (DaaS) solution, all software installation,

configuration, and maintenance will be handled by the DaaS cloud-hosted service
provider. This includes applications, operating systems, files, and user preferences.
Network Virtualization in Cloud Computing

Network Virtualization is a process of logically grouping physical networks and making

them operate as single or multiple independent networks called Virtual Networks.

General Architecture Of Network Virtualization

Tools for Network Virtualization :
1. Physical switch OS –
It is where the OS must have the functionality of network virtualization.
2. Hypervisor –
It is which uses third-party software or built-in networking and the
functionalities of network virtualization.
The basic functionality of the OS is to give the application or the executing process
with a simple set of instructions. System calls that are generated by the OS and
executed through the libc library are comparable to the service primitives given at
the interface between the application and the network through the SAP (Service
Access Point).
The hypervisor is used to create a virtual switch and configuring virtual networks on
it. The third-party software is installed onto the hypervisor and it replaces the native
networking functionality of the hypervisor. A hypervisor allows us to have various
VMs all working optimally on a single piece of computer hardware.
Functions of Network Virtualization :
 It enables the functional grouping of nodes in a virtual network.
 It enables the virtual network to share network resources.
 It allows communication between nodes in a virtual network without
routing of frames.
 It restricts management traffic.
 It enforces routing for communication between virtual networks.
Network Virtualization in Virtual Data Center :
1. Physical Network
  Physical components: Network adapters, switches, bridges, repeaters,
routers and hubs.
 Grants connectivity among physical servers running a hypervisor, between
physical servers and storage systems and between physical servers and
clients.
2. VM Network
 Consists of virtual switches.
 Provides connectivity to hypervisor kernel.
 Connects to the physical network.
 Resides inside the physical server.

Network Virtualization In VDC

Advantages of Network Virtualization :
Improves manageability –
 Grouping and regrouping of nodes are eased.
 Configuration of VM is allowed from a centralized management
workstation using management software.
Reduces CAPEX –
 The requirement to set up separate physical networks for different node
groups is reduced.
Improves utilization –
 Multiple VMs are enabled to share the same physical network which
enhances the utilization of network resource.
Enhances performance –
 Network broadcast is restricted and VM performance is improved.
Enhances security –
 Sensitive data is isolated from one VM to another VM.
 Access to nodes is restricted in a VM from another VM.
Disadvantages of Network Virtualization :
 It needs to manage IT in the abstract.
  It needs to coexist with physical devices in a cloud-integrated hybrid
environment.
 Increased complexity.
 Upfront cost.
 Possible learning curve.
Examples of Network Virtualization :
Virtual LAN (VLAN) –
 The performance and speed of busy networks can be improved by VLAN.
 VLAN can simplify additions or any changes to the network.
Network Overlays –
 A framework is provided by an encapsulation protocol called VXLAN for
overlaying virtualized layer 2 networks over layer 3 networks.
 The Generic Network Virtualization Encapsulation protocol (GENEVE)
provides a new way to encapsulation designed to provide control-plane
independence between the endpoints of the tunnel.
Network Virtualization Platform: VMware NSX –
 VMware NSX Data Center transports the components of networking and
security such as switching, firewalling and routing that are defined and
consumed in software.
 It transports the operational model of a virtual machine (VM) for the
network.
Applications of Network Virtualization :
 Network virtualization may be used in the development of application
testing to mimic real-world hardware and system software.
 It helps us to integrate several physical networks into a single network or
separate single physical networks into multiple analytical networks.
 In the field of application performance engineering, network virtualization
allows the simulation of connections between applications, services,
dependencies, and end-users for software testing.
 It helps us to deploy applications in a quicker time frame, thereby
supporting a faster go-to-market.
 Network virtualization helps the software testing teams to derive actual
results with expected instances and congestion issues in a networked
environment.

Storage Virtualization in Cloud Computing – Types & Benefits

Storage virtualization in Cloud Computing is nothing but the sharing of physical

storage into multiple storage devices which further appears to be a single storage
device.
What is Storage Virtualization in Cloud Computing?

Storage virtualization in Cloud Computing is nothing but the sharing of physical

storage into multiple storage devices which further appears to be a single storage
device. It can be also called as a group of an available storage device which simply
manages from a central console.
This virtualization provides numerous benefits such as easy backup, achieving, and
recovery of the data.

This whole process requires very less time and works in an efficient manner. Storage
virtualization in Cloud Computing does not show the actual complexity of the
Storage Area Network (SAN). This virtualization is applicable to all levels of SAN.
Why Storage Virtualization should be implemented?

Following are the reasons shows why we storage virtualization in Cloud Computing
implements:

 If this virtualization implements in IT environment it will improve the

management of the storage. As each and everything will properly store
and manage there won’t be any congestion and the task will perform
quickly.
 There will be very less downtime as the storage availability is better. All
these problems eliminate with the help of an automated management
system.
 Storage virtualization will provide better storage utilization as storing most
information in a particular place can cause loss of data, congestion, and
any other problems. So, properly dividing storage and storing data can be
useful.
Types of Storage Virtualization
Here, we are going to list down all the storage virtualization in Cloud Computing;

 Hardware Assisted Virtualization

 Kernel Level Virtualization
 Hypervisor Virtualization
 Para-Virtualization
 Full Virtualization
i. Hardware Assisted Virtualization

This type of virtualization requires hardware support. It is similar to full Para-

virtualization. Here, the unmodified OS can run as hardware support for virtualization
and we can also use to handle hardware access requests and protect operations.
ii. Kernel Level Virtualization

It runs a separate version of the Linux Kernel. Kernel level allows running multiple
servers in a single host. It uses a device driver to communicate between main Linux
Kernel and the virtual machine. This virtualization is a special form of Server
Virtualization.
iii. Hypervisor Virtualization

A hypervisor is a layer between the Operating system and hardware. With the help
of hypervisor multiple operating systems can work. Moreover, it provides
features and necessary services which help OS to work properly.
iv. Para-Virtualization
It is based on hypervisor which handles emulation and trapping of software. Here, the
guest operating system is modified before installing it to any further machine. The
modified system communicates directly with the hypervisor and improves the
performance.

v. Full Virtualization

This virtualization is similar to Para-Virtualization. In this, the hypervisor traps the

machine operations which is used by the operating system to perform the
operations. After trapping the operations, it emulates in particular software and the
status codes returned.
Storage Virtualization Risks

i. Limited Adoption

The one-third of the enterprise is reporting in a computer economics survey that they
are increasing the funds for storage virtualization. There are some understanding of
adoption rates, return of investment and the cost of ownership.

ii. Problems in Naming

Before very less VMS was used but now there has been a rapid growth of VMS which
makes it difficult to distinguish between the important and the important VMS. To
make it more future proof building a naming system and sharing with it with all
involved parties should be done.
iii. Failure

The failure occurs due to downtime and data loss. The installation of VMware which
hosts crucial services becomes a single point of failure. So to eliminate this threat
the protection of virtual machine data should prioritize to the top.
 Storage Virtualization – Risk & Methods
Methods of Storage Virtualization

i. File-based Storage Virtualization

This type of virtualization is used for a specific purpose and can apply to network-
attached storage (NAS) system.

File-based storage virtualization in Cloud Computing utilizes server message block or

network file system protocols and with its help of it breaks the dependency in a normal
network attached storage array.

This is done between the data being accessed and the location of the physical memory.
It also provides a benefit of better handling file migration in the background which
improves the performance.

ii. Block-based Virtual Storage

The Block based virtual storage is more widely used than the virtual storage system
as the virtual storage system is sometimes used for a specific purpose. The block-
based virtual storage system uses logical storage such as drive partition from the
physical memory in a storage device.

It also abstracts the logical storage such as a hard disk drive or any solid state memory
device. This also allows the virtualization management software to get familiar with
the capacity of the available device and split them into shared resources to assign.

Address Space Remapping

Storage virtualization in Cloud Computing helps to achieve location independence by

utilizing the physical location of the data. This system provides the space to the
customer to store their data and handles the process of mapping.
It is possible that the output of virtualizations can cascade as an input for a higher
level of virtualizations. This leads to the fact that it is possible to have multiple layers
of virtualizations mapping.

Why Storage Virtualization is Important?

i. Performs Tasks

The appliances of storage virtualization are responsible for several tasks such as
heterogeneous replication and federation. These devices lineup in front of arrays and
create a common interface for the host.

This allows the administrator to mix and match the protocols and array which are
behind the appliances

ii. WAN Management

It does not send multiple copies of the similar data over WAN. The WAN accelerator
use to cache the data and send it LAN speed without changing the performance of
WAN.

iii. Disaster Recovery

Storage virtualization in Cloud Computing can increase disk utilization and is flexible.
This ameliorates disaster recovery and the continuity of the business.

iv. Storage Tiering

Storage tiering is a technique which monitors and selects the most commonly used
data and put it on its highest performing storage pool. The least used data is put in
the weakest performance storage pool.

Storage Virtualization – Advantages and Importance

Advantages of Storage Virtualization

Let’s discuss some benefits of Storage Virtualization in Cloud Computing:

i. Easy Retrieval and Upload of Data

In storage virtualization, the data quickly retrieve from virtual storage. It is as easy
as accessing a file on the local computer. The data store very easily with the help of
some application and an internet connection which is an easy task.

ii. Better Management

The data can be migrated based upon the utilization such as the data which is
frequently used can be stored on a high-performance storage system. However, the
data which is rarely used can be placed on a bit slower system.

This is an example of a battery management system and the customer won’t face
any issue regarding storage.

iii. Security
In storage virtualization, the data stores in different place and secure with maximum
security. If any disaster takes place the data can be retrieved from some other place
and it won’t affect the customer.

The security has the ability to meet the real utilization necessities rather than
providing additional storage.

How Storage Virtualization Apply?

Following are the different ways for storage applies to the virtualization:

 Host-Based
 Network-Based
 Array-Based
i. Host-Based Storage Virtualization

Here, all the virtualizations and management is done at the host level with the help
of software and physical storage, it can be any device or array.

The host is made up of multiple hosts which present virtual drives of a set to the
guest machines. Doesn’t matter whether they are VMs in an enterprise or PCs.

ii. Network-Based Storage Virtualization

Network-based storage virtualization is the most common form which are using
nowadays. Devices such as a smart switch or purpose-built server connect to all the
storage device in a fibre channel storage network and present the storage as a
virtual pool.
iii. Array-Based Storage Virtualization

Here the storage array provides different types of storage which are physical and used
as storage tiers. The software is available which handles the amount of storage tier
made up of solid-state drives hard drives.

The storage virtualization technique is now common among the users as it has
its own benefits. With the help of storage virtualization in Cloud Computing, all the
drives can combine with a single centrally managed resource.
Moreover, it allows modifying and making changes without downtime. This provides
flexibility to the customer by making data migration flexible.

System-
level of Operating Virtualization
System-
level of Operating Virtualization
SYSTEM-LEVEL OF OPERATING VIRTUALIZATION

Operating system-based Virtualization refers to an operating system feature in

which the kernel enables the existence of various isolated user-space instances.
The installation of virtualization software also refers to Operating system-based
virtualization. It is installed over a pre-existing operating system and that operating
system is called the host operating system.

In this virtualization, a user installs the virtualization software in the operating

system of his system like any other program and utilizes this application to operate
and generate various virtual machines. Here, the virtualization software allows
direct access to any of the created virtual machines to the user. As the host OS can
provide hardware devices with the mandatory support, operating system
virtualization may affect compatibility issues of hardware even when the hardware
driver is not allocated to the virtualization software.
Virtualization software is able to convert hardware IT resources that require unique
software for operation into virtualized IT resources. As the host OS is a complete
operating system in itself, many OS-based services are available as organizational
management and administration tools can be utilized for the virtualization host
management.

Some major operating system-based services are mentioned below:

1. Backup and Recovery.
2. Security Management.
3. Integration to Directory Services.
Various major operations of Operating System Based Virtualization are described
below:

1. Hardware capabilities can be employed, such as the network connection

and CPU.
2. Connected peripherals with which it can interact, such as a webcam,
printer, keyboard, or Scanners.
 3. Data that can be read or written, such as files, folders, and network
shares.
The Operating system may have the capability to allow or deny access to such
resources based on which the program requests them and the user account in the
context of which it runs. OS may also hide these resources, which leads that when
a computer program computes them, they do not appear in the enumeration
results. Nevertheless, from a programming perspective, the computer program has
interacted with those resources and the operating system has managed an act of
interaction.

With operating-system-virtualization or containerization, it is probable to run

programs within containers, to which only parts of these resources are allocated. A
program that is expected to perceive the whole computer, once run inside a
container, can only see the allocated resources and believes them to be all that is
available. Several containers can be formed on each operating system, to each of
which a subset of the computer’s resources is allocated. Each container may
include many computer programs. These programs may run parallel or distinctly,
even interrelate with each other.

features of operating system-based virtualization are:

 Resource isolation: Operating system-based virtualization provides a

high level of resource isolation, which allows each container to have its
own set of resources, including CPU, memory, and I/O bandwidth.
 Lightweight: Containers are lightweight compared to traditional virtual
machines as they share the same host operating system, resulting in
faster startup and lower resource usage.
 Portability: Containers are highly portable, making it easy to move
them from one environment to another without needing to modify the
underlying application.
 Scalability: Containers can be easily scaled up or down based on the
application requirements, allowing applications to be highly responsive to
changes in demand.
 Security: Containers provide a high level of security by isolating the
containerized application from the host operating system and other
containers running on the same system.
 Reduced O+verhead: Containers incur less overhead than traditional
virtual machines, as they do not need to emulate a full hardware
environment.
 Easy Management: Containers are easy to manage, as they can be
started, stopped, and monitored using simple commands.
Operating system-based virtualization can raise demands and problems related to
performance overhead, such as:

1. The host operating system employs CPU, memory, and other hardware IT
resources.
2. Hardware-related calls from guest operating systems need to navigate
numerous layers to and from the hardware, which shrinkage overall
performance.
3. Licenses are frequently essential for host operating systems, in addition
to individual licenses for each of their guest operating systems.
Advantages of Operating System-Based Virtualization:
 Resource Efficiency: Operating system-based virtualization allows for
greater resource efficiency as containers do not need to emulate a
complete hardware environment, which reduces resource overhead.
 High Scalability: Containers can be quickly and easily scaled up or
down depending on the demand, which makes it easy to respond to
changes in the workload.Easy Management: Containers are easy to
manage as they can be managed through simple commands, which
makes it easy to deploy and maintain large numbers of containers.
Reduced Costs: Operating system-based virtualization can significantly
reduce costs, as it requires fewer resources and infrastructure than
traditional virtual machines.
 Faster Deployment: Containers can be deployed quickly, reducing the
time required to launch new applications or update existing ones.
 Portability: Containers are highly portable, making it easy to move
them from one environment to another without requiring changes to the
underlying application.
Disadvantages of Operating System-Based Virtualization:
 Security: Operating system-based virtualization may pose security risks
as containers share the same host operating system, which means that a
security breach in one container could potentially affect all other
containers running on the same system.
 Limited Isolation: Containers may not provide complete isolation
between applications, which can lead to performance degradation or
resource contention.
 Complexity: Operating system-based virtualization can be complex to
set up and manage, requiring specialized skills and knowledge.
 Dependency Issues: Containers may have dependency issues with
other containers or the host operating system, which can lead to
compatibility issues and hinder deployment.
  Limited Hardware Access: Containers may have limited access to
hardware resources, which can limit their ability to perform certain tasks
or applications that require direct hardware access.

3.4 VIRTUAL CLUSTERS AND RES OURCE MAN AGEMENT

3.4.1 Physical versus Virtual Clusters

3.4.1.1 Fast Dep loyment and Effective Scheduling

3.4.1.2 H igh-P erformance Virtual St orag e

3.4.2 Live VM Migrat ion Step s and P erformance Effects

3.4.3 M igrat ion of Memory, Files, and Network Resources

3.4.3.1 M emory Migrat ion

3.4.3.2 File Syst em Migration

3.4.3.3 Net work M igrat ion

3.4.4 Dynamic Dep loyment of Virtual Clust ers

3.4 VIRTUAL CLUSTERS AND RES OURCE MAN AGEMENT

 A physical clust er is a collect ion of servers (physical

machines) int erconnected by a physical net work such as a
LAN.
 This chapt er – int ro to – virtual clusters, its prop ert ies and
potent ial app licat ions.
 Three crit ical d esign issues of virtual clusters:
o live migrat ion of VMs
o memory and file migrations
o Dynamic dep loyment of virtual clust ers.
3.4.1 Physical versus Virtual Clusters

 Virtual clust ers are built with VM s installed at distributed

servers from one or more physical clust ers.
 The VMs in a virtual cluster are interconnected logically
by a virtual net work across seve ral phys ical net works.
 Figure 3.18 illustrates the concepts of virtual clusters and
physical clust ers.

 Each virtual cluster is formed with physical machines or a VM

host ed by mult iple phy sical clust ers.
 The virtual cluster bound aries are shown as dist inct
boundaries.

The provisioning of VM s to a virtual clust er is done dynamically to

have the following prop ert ies:

o The virtual cluster nodes can b e either physical or virtual

machines. Multip le VMs running wit h different OSes can
be deployed on the same physical node.
o A VM runs with a guest OS, which is often different from
the host OS, that manages the resources in the physical
machine, where the VM is implemented.
o The purpose of using VMs is to consolid ate multip le
functionalities on the same server. This will greatly
enhance server utilizat ion and app lication flex ibility.
 o VMs can be colonized (rep licat ed) in mult iple servers for
the purp ose of promot ing d istribut ed parallelism, fault
tolerance, and d isast er recovery.
o The size (numb er of nod es) of a virt ual cluster can grow
or shrink dynamically, similar to the way an overlay
network varies in size in a p eer -to-peer (P2P) network.
o The failure of any physical nod es may disab le some VMs
inst alled on the failing nod es. But t he failure of VMs will
not pull down the host system.

Since syst em virtualizat ion has been widely used, it is necessary

to

o effectively manage VM s running on a mass of physical

comput ing nodes (also called virtual clust ers) and
o build a high-p erformance virt ualized comput ing
environment .

This involves

o virtual cluster dep loyment,

o monit oring and manag ement over large -scale clust ers,
o resource scheduling
o load balancing
o server consolid ation
o fault t olerance
 Figure 3.19 shows the concept of a virt ual clust er b ased on
applicat ion part itioning or customization.

The different colors in the figure represent the nod es in d ifferent

virtual clusters.

 Issues to consid er – how to efficient ly store the larg e number

of VM images in the system.

 There are common inst allat ions for most users or app licat ions,
such as op erating syst ems or user -level p rogramming
libraries.
 These software packages can be preinst alled as temp lat es
(called template VMs).
 With these t emplates, users can build their own software
stacks.
 New OS inst ances can b e cop ied from the temp lat e VM.
 User -sp ecific comp onent s such as programming libraries and
applicat ions can be inst alled to those instances.

 Three physical clust ers are shown on the left side of Figure
3.18.
 Four virtual clust ers are created on the right , over the
physical clust ers.
 The physical machines are also called host systems.
 In contrast, the VMs are guest sys tems.
 The host and guest syst ems may run with d ifferent operating
systems.
 Each VM can be inst alled on a remote server or rep licat ed on
mult iple servers belong ing to the same or different physical
clusters.
 The bound ary of a virtual cluster can change as VM nodes are
added, removed, or migrat ed dynamically over t ime.

3.4.1.1 Fast Deployment and E ffective Scheduling

 The syst em should have the capab ility of fast deployment.

 Here, dep loyment means two thing s:
o to construct and d istribut e software st acks (OS,
libraries, applications) to a physical node inside clust ers
as fast as possib le,
 o to quickly switch runtime environments from one user’s
virtual cluster to another user’s virt ual cluster.

 If one user finishes using his system, the corresp onding

virtual clust er should shut down or suspend quickly to save
the resources to run other VMs for othe r users.

 The concept of “green comput ing” has attracted much

attent ion recent ly.

 Approaches
o Focus on saving the energy cost of component s in
individual workstat ions
o Apply clust er-wid e energy-efficient techniques on
homog eneous workstat ions and specific app licat ions.

 The live migrat ion of VM s allows workloads of one node to

transfer to another node.
 Problem –
o it does not guarantee that VMs can randomly migrate
among themselves.
o potent ial overhead caused by live migrat ions of VM s
 overhead also affects cluster utilizat ion,
throughput, and QoS issues.
o challenge – d etermine how t o design migrat ion
strat egies to imp lement green comput ing without
influencing the performance of clust ers.
o Another advantag e of virtualizat ion is load balancing
of applicat ions in a virtual cluster.
o Load balancing can b e achieved using
 the load index and
 frequency of user logins.
o The automatic scale -up and scale-d own mechanism of a
virtual cluster can b e imp lement ed based on this mod el.
o Consequently,
 we can increase the resource ut ilization of nod es
and
 short en the response t ime of syst ems.

 Mapping VMs onto the most appropriat e physical nod e should

promote performance.
  Dynamically adjust ing loads among nodes by live mig rat ion of
VMs is desired, when the load s on cluster nodes becom e quit e
unbalanced.

3.4.1.2 High-Performance Virtual Storage

 The temp lat e VM can be distributed to several physical host s

in the cluster to customize the VMs.

 Basically, there are four st eps to d eploy a group of VMs ont o

a targ et clust er:
o preparing the disk image,
o configuring the VMs,
o choosing the dest inat ion nodes, and
o executing the VM deployment command on every host .

 Every VM is configured with

o a name,
o disk image,
o network setting, and
o allocat ed CPU and memory.

 Each VM configurat ion is record ed into a file.

 Most configurat ion items use the same sett ing s, while some
of them, such as UUI D, VM name, and IP address, are assigned
with automatically calculated values.

 The d eployment princip le is to fulfill the VM requirement and

to balance workloads among the whole host network.

3.4.2 Live VM Migration Steps and Performance Effe cts

 In a clust er built with mixed nodes of host and guest syst ems,
the normal method of op erat ion is to run everything on the
physical machine.
 Virtual clust ers can be app lied in computat ional grid s, cloud
platforms, and high -p erformance computing (HPC) systems.
 Virtual clustering provid es dynamic resources that can be
quickly put together upon user demand or after a node failure.
 In particular, virtual clustering plays a key role in cloud
comput ing.
 There are four ways to manage a virtual cluster.
o cluster manager resid es on a guest syst em
o Cluster manager resid es on the host syst ems. The host -
based manager supervises the guest systems and can
restart the guest system on another phy sical machine.
o Use an ind epend ent cluster manag er on both the host
and guest syst ems – issue – makes infrastructure
management more complex.
o Use an int egrated clust er on the guest and host systems.
This means the manag er must b e designed to d istinguish
between virtualized resources and physical resources.

 A VM can be in one of the following four st ates.

o An inactive state is d efined by the virt ualizat ion
platform, under which the VM is not enab led.
o An active state refers to a VM that has been
inst ant iat ed at the virtualizat ion platform to perform a
real task.
o A pause d state corresponds t o a VM that has been
inst ant iat ed but disab led to process a task or p aused in
a wait ing stat e.
o A VM enters the suspe nde d state if its machine file and
virtual r esources are st ored back to the disk.

 VMs can b e live -migrat ed from one physical machine to

another;
 When a VM fails, one VM can be rep laced by another VM on a
different node, as long as they both run with the same guest
OS.
 The migration copies the VM st ate file from the st orag e area
to the host machine

 Figure 3.20 shows the process of life migration of a VM from

host A to host B. (six st eps)
 Steps 0 and 1: St art migrat ion.
o Makes preparations for the migrat ion, includ ing
determining the migrat ing VM and the destination host.
o Migrat ion –
 manual done by user or
 automatically started by strateg ies such as load
balancing and server consolid ation.

 Steps 2: Transfer memory.

o Since the whole ex ecut ion stat e of the VM is stored in
memory, send ing the VM’s memory to the dest inat ion
node ensures cont inuit y of t he service provid ed by the
VM.
o All of the memory data is transferred in the first round,
and
o Then t he migration controller recopies the memory
data which is chang ed in the last round.
o These st eps keep it erat ing until all modified data is
copied to destinat ion node.
  Step 3: Suspend the VM and copy the last portion of the data.
o The migrating VM’s ex ecut ion is susp ended when t he last
round’s memory data is transferred.
o Other non-memory data such as CPU an d network st ates
are sent.
o During this step, the VM is stopped and its app licat ions
will no longer run.
o This “service unavailab le” t ime is called the “downtime”
of mig rat ion, which should be as short as possib le so
that it can be neglig ible to users.

 Steps 4 and 5: Commit and act ivat e the new host .

o After all the need ed data is cop ied to the d est inat ion
host, the VM reloads the stat es and recovers the
execution of programs in the d est ination host, and the
service provid ed by this VM cont inues.
o Redirect the net work connect ion to t he new VM and clear
the dependency to the source host.
o Finally remove the original VM from the source host.

 When a VM runs a live service, it is necessary to ensure that

the mig rat ion has
o negligib le downt ime,
o the lowest network band width consumption possib le, and
o a reasonab le t otal migrat ion t ime.
o Migrat ion should not d isrupt ot her act ive services
resid ing in the same host through resource cont ention
(e.g ., CPU, network band width).

3.4.3 Migratio n of Memory, Files , and Networ k Resour ces

 Since clust ers have a high init ial cost of ownership,

o (includ ing space, power cond itioning, and cooling
equipment,)
o leasing or sharing access to a common clust er is an
attractive solut ion when demands vary over t ime .
 Shared clust ers offer econom ies of scale and more effect ive
utilizat ion of resources by mult iplex ing.
 Early configuration and management systems focus on
expressive and scalable mechanisms for d efining clusters for
specific typ es of service, and physically part it ion cluster
nodes among those types.
  When one syst em migrat es t o another physical nod e, we
should consider the following issues.
o Memory M igrat ion
o File System Migration
o Net work M igrat ion
o Live Migrat ion of VM Using Xen

3.4.3.1 Memory Migration

 This is one of the most import ant aspect s of VM migrat ion.

 Moving the memory inst ance of a VM from one physical host
to another
o depend upon the characteristics of applicat ion/workload s
supported by the guest OS.

 Memory migrat ion can be in a rang e of hundreds of megabyt es

to a few gigab ytes in a typical system today, and
 it needs to b e done in an efficient manner.

 The Internet Suspend- Resume (ISR) technique exploit s

temporal locality as memory states are likely to have
consid erab le overlap in the susp ended and the resumed
inst ances of a VM.

 Temporal locality refers to the fact that the memory stat es

differ only by the amount of work done since a VM was last
susp ended before being init iat ed for migrat ion.

 To exploit temporal locality, each file in the file system is

represented as a tree of small sub files .
 A copy of this tree ex ists in both the suspend ed and resumed
VM inst ances.
 The advant age of using a tree -based representat ion of files is
that the caching ensures the transmission of only those files
which have been change d.
 The ISR technique deals with situat ions where the mig rat ion
of live machines is not a necessity.
 Predictab ly, the downtime (the period during which the
service is unavailab le due t o there b eing no current ly
executing instance of a VM) is high, com pared to some of the
other t echniques d iscussed lat er.
3.4.3.2 File System Migrat ion

 To support VM migrat ion, a system must provid e each VM with

a consistent , locat ion -ind epend ent view of the file system that
is availab le on all host s.

Solut ion 1

 A simp le way to achieve this is to provid e each VM with its

own virtual d isk which the file system is mapp ed to and
transp ort the cont ents of this virtual disk along with the other
states of the VM.
 However, due to the current trend of high capacity disks,
migration of the cont ent s of an ent ire disk over a network is
not a viab le solution.

Solut ion 2

 Another way is to have a g lob al file system across all machines

where a VM could be located. This way removes the need to
copy files from one machine to another because all files are
network accessib le.

Solut ion 3

 A distribut ed file system is used in I SR serving as a transp ort

mechanism for prop agat ing a susp ended VM stat e.
 The actual file systems themselves are not mapped ont o the
distribut ed file system.
 Instead, the VMM only accesses it s local file system.
 The relevant VM files are explicitly copied into the local file
syst em for a resume op erat ion and taken out of the local file
syst em for a susp end operat ion.
 This approach relieves developers from the comple x ities of
implement ing several different file syst em calls for different
distribut ed file systems.
 It also essentially disassociates the VMM from any particular
distribut ed file system semant ics.
 However, this d ecoup ling means that the VMM has to store the
content s of each VM’s virtual d isks in it s local files, which
have to be moved around with the other st ate informat ion of
that VM.
Solut ion 4

 In smart copying, the VMM exploit s spatial locality.

 to transmit only the differe nce between the two file
systems at suspending and resuming locat ions.
 This technique significantly reduces the amount of actual
physical data that has to be moved.

Solut ion 5 – proactive st ate transfer solut ion – predict new location

 In situations where there is no locality t o e xploit , a different

approach is to synthesize much of the state at the
resuming site.
 On many systems, user files only form a small fraction of the
actual dat a on d isk.
 Operating syst em and application software account for the
majorit y of st orag e sp ace.
 The proact ive st ate t ransfer solut ion works in those cases
where the resuming sit e can b e p redicted with reasonable
confidence.

3.4.3.3 Network Migration

 A migrat ing VM should maint ain all open net work connect ions
without relying on forwarding mechanisms on the original host
or on support from mob ility or redirect ion mechanisms.

 To enab le remote syst ems to locat e and communicat e with a

VM, each VM must be assigned a virtual IP address known
to other ent ities.
 This address can be distinct from the IP addre ss of the host
machine where the VM is current ly locat ed.
 Each VM can also have its own d ist inct virtual MAC address .
 The VMM maint ains a mapp ing of the virtual IP and MAC
addresses t o their corresp onding VMs. ( ARP Tab le )

Solut ion 1 – Virtual IP and MAC ad dress

 In general, a migrat ing VM includ es all the prot ocol stat es and
carries its IP address with it .
 If the source and destination machines of a VM migration are
typically connect ed to a sing le swit ched LAN, an unsolicit ed
 ARP rep ly from t he migrating h ost is provided advert ising t hat
the IP has moved to a new location.
 This solves the op en network connect ion problem by
reconfiguring all the peers to send future packet s to a new
locat ion.
 Although a few p ackets t hat have alread y been transmitt ed
might be lost, there are no ot her prob lems with this
mechanism.

Solut ion 2 – Switched Networks

 Alternat ively, on a switched network, the migrating OS can

keep it s original Ethernet MAC address and rely on the network
switch to d etect its move to a new port.

Solut ion 3 – Live migrat ion

 Live migrat ion means moving a VM from one physical node to

another while keep ing its OS environment and app licat ions
unbroken.
 This cap ability is used to provid e efficient
o online system maintenance,
o reconfigurat ion,
o load balanc ing, and
o proact ive fault tolerance.

 It also provides desirab le features to satisfy requirements

for com put ing resources in mod ern comput ing syst ems,
including server consolidat ion, performance isolat ion,
and ease of management.
 Note:
o Trad itional migrat ion suspends VMs before the
transp ortat ion and then resumes them at the end of the
process.
o By imp ort ing the precopy mechanism, a VM could be live
migrated without st opping the VM and keep the
applicat ions running during the mig ration.

Solut ion 3 – Clust er Environment with net work -accessib le storage

syst em, like storage area network (SAN) or network att ached
storage (NAS)
  Only memory and CPU status needs t o be transferred from the
source node to the target node.

Solut ion 4 – precopy approach

 Precopy approach – first transfers all memory pages, and then

only copies mod ified pag es during t he last round it eratively.

Solut ion 5 – Postcopy approach

 Here, all memory pag es are transferred only once during the
whole migrat ion process and the baseline total migrat ion t ime
is reduced.
 But the downt ime is much higher than that of precopy due t o
the lat ency of fet ching pages from the source nod e before the
VM can be resumed on the target.

 With the advent of multicore or many -core machines,

abundant CPU resources are available which can be used to
compress page frames and the amount of transferred data can
be significant ly reduced .
 Memory compression alg orithms typically have little memory
overhead.
 Decompression is also simp le and very fast

3.4.4 Dynamic De ployme nt of Vir tual Clusters

  Table 3.5 summarizes four virtual cluster research projects –
We briefly introduce them here just to identify their design
object ives and rep ort ed result s.
o Cellular Disco at Stanford is a virtual clust er built in a
shared-memory mult iprocessor syst em.
o INRIA virt ual cluster was built to test parallel alg orithm
performance.
o COD was d eveloped at Duke University to support
dynamic resource allocat ion wit h a virtual cluster
management system.
o VIOLIN clusters was built at Purdue University using
mult iple VM clust ering to prove the advantag e of
dynamic ad aptat ion.

Difference between Virtual Machines and

Containers
Virtual machines and Containers are two ways of deploying multiple, isolated
services on a single platform.
Virtual Machine:

It runs on top of an emulating software called the hypervisor which sits between
the hardware and the virtual machine. The hypervisor is the key to enabling
virtualization. It manages the sharing of physical resources into virtual
machines. Each virtual machine runs its guest operating system. They are less
agile and have lower portability than containers.

Container:
It sits on the top of a physical server and its host operating system. They
share a common operating system that requires care and feeding for bug
fixes and patches. They are more agile and have higher portability than virtual
machines.
Let’s see the difference between Virtual machines and Containers.
SNo. Virtual Machines(VM) Containers

1 VM is a piece of software While a container is

that allows you to install software that allows
other software inside of different functionalities
it so you control it of an application
virtually as opposed to independently.
installing the software
directly on the computer.

2. Applications running on While applications

a VM system, or running in a container
hypervisor, can run environment share a
different OS. single OS.

3. VM virtualizes the While containers

computer system, virtualize the operating
meaning its hardware. system, or the software
only.

4. VM size is very large, While the size of the

generally in gigabytes. container is very light,
 generally a few hundred
megabytes, though it
may vary as per use.

5. VM takes longer to run While containers take far

than containers, the less time to run.
exact time depending on
the underlying hardware.

6. VM uses a lot of system While containers require

memory. very less memory.

7. VM is more secure, as While containers are less

the underlying hardware secure, as the
isn’t shared between virtualization is software-
processes. based, and memory is
shared.

8. VMs are useful when we While containers are

require all of the OS useful when we are
resources to run various required to maximize the
applications. running applications
using minimal servers.

9. Examples of Type 1 Examples of containers

hypervisors are KVM, are RancherOS,
Xen, and VMware. PhotonOS, and
Virtualbox is a Type 2 Containers by Docker.
hypervisor
Introduction to Docker
Docker is a set of platforms as a service (PaaS) products that use the
Operating system level virtualization to deliver software in packages called
containers. Containers are isolated from one another and bundle their own
software, libraries, and configuration files; they can communicate with each
other through well-defined channels. All containers are run by a single
operating system kernel and therefore use fewer resources than a virtual
machine.

Difference between Docker Containers and Virtual Machines

1. Docker Containers
 Docker Containers contain binaries, libraries, and configuration files
along with the application itself.
 They don’t contain a guest OS for each container and rely on the
underlying OS kernel, which makes the containers lightweight.
 Containers share resources with other containers in the same host
OS and provide OS-level process isolation.
2. Virtual Machines
 Virtual Machines (VMs) run on Hypervisors, which allow multiple
Virtual Machines to run on a single machine along with its own
operating system.
 Each VM has its own copy of an operating system along with the
application and necessary binaries, which makes it significantly
larger and it requires more resources.
 They provide Hardware-level process isolation and are slow to boot.

Important Terminologies in Docker

1. Docker Image
 It is a file, comprised of multiple layers, used to execute code in a
Docker container.
 They are a set of instructions used to create docker containers.
2. Docker Container
 It is a runtime instance of an image.
 Allows developers to package applications with all parts needed such
as libraries and other dependencies.
3. Docker file
 It is a text document that contains necessary commands which on
execution helps assemble a Docker Image.
 Docker image is created using a Docker file.
4. Docker Engine
 The software that hosts the containers is named Docker Engine.
 Docker Engine is a client-server based application
 The docker engine has 3 main components:
 Server: It is responsible for creating and managing Docker
images, containers, networks, and volumes on the Docker.
It is referred to as a daemon process.
 REST API: It specifies how the applications can interact
with the Server and instructs it what to do.
 Client: The Client is a docker command-line interface
(CLI), that allows us to interact with Docker using the
docker commands.
5. Docker Hub
 Docker Hub is the official online repository where you can find other
Docker Images that are available for use.
 It makes it easy to find, manage, and share container images with
others.

Installing Docker on Ubuntu

1. Remove old version of Docker

$ sudo apt-get remove docker docker-engine docker.io containerd runc

2. Installing Docker Engine

$ sudo apt-get update

$ sudo apt-get install \

ca-certificates \
curl \
gnupg \
lsb-release

$ sudo mkdir -p /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o
/etc/apt/keyrings/docker.gpg
$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg]
https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

$ sudo groupadd docker

$ sudo usermod -aG docker $USER
Check if docker is successfully installed in your system
$ sudo docker run hello-world

Create an application in Docker

1. Create a folder with 2 files (Dockerfile and main.py file) in it.

 Dockerfile
 main.py
2. Edit main.py with the below code.

 Python3
#!/usr/bin/env python3

print("Docker and GFG rock!")

3. Edit Dockerfile with the below commands.

FROM python:latest
COPY main.py /
CMD [ "python", "./main.py" ]

4. Create a Docker image.

Once you have created and edited the main.py file and the Dockerfile, create
your image to contain your application.
$ sudo docker build -t python-test .
The ‘-t’ option allows to define the name of your image. ‘python-test’ is the
name we have chosen for the image.
5. Run the Docker image
Once the image is created, your code is ready to launch.
$ sudo docker run python-test

Push an image to Docker Hub

1. Create an Account on Docker Hub.

2. Click on the “Create Repository” button, put the name of the file, and click
on “Create”.
3. Now will “tag our image” and “push it to the Docker Hub repository” which
we just created.

Now, run the below command to list docker images:

$ docker images

The above will give us this result

REPOSITORY TAG IMAGE_ID CREATED SIZE afrozchakure/python-test latest

c7857f97ebbd 2 hours ago 933MB

Image ID is used to tag the image. The syntax to tag the image is:
docker tag <image-id> <your dockerhub username>/python-test:latest
$ docker tag c7857f97ebbd afrozchakure/python-test:latest

4. Push image to Docker Hub repository

$ docker push afrozchakure/python-test

Fetch and run the image from Docker Hub

1. To remove all versions of a particular image from our local system, we use
the Image ID for it.
$ docker rmi -f af939ee31fdc

2. Now run the image, it will fetch the image from the docker hub if it doesn’t
exist on your local machine.
$ docker run afrozchakure/python-test
Docker overview
Docker is an open platform for developing, shipping, and running applications.
Docker enables you to separate your applications from your infrastructure so you
can deliver software quickly. With Docker, you can manage your infrastructure in the
same ways you manage your applications. By taking advantage of Docker's
methodologies for shipping, testing, and deploying code, you can significantly reduce
the delay between writing code and running it in production.

The Docker platform

Docker provides the ability to package and run an application in a loosely isolated
environment called a container. The isolation and security lets you to run many
containers simultaneously on a given host. Containers are lightweight and contain
everything needed to run the application, so you don't need to rely on what's
installed on the host. You can share containers while you work, and be sure that
everyone you share with gets the same container that works in the same way.

Docker provides tooling and a platform to manage the lifecycle of your containers:

 Develop your application and its supporting components using containers.

 The container becomes the unit for distributing and testing your application.
 When you're ready, deploy your application into your production environment, as a
container or an orchestrated service. This works the same whether your production
environment is a local data center, a cloud provider, or a hybrid of the two.

What can I use Docker for?

Fast, consistent delivery of your applications

Docker streamlines the development lifecycle by allowing developers to work in
standardized environments using local containers which provide your applications
and services. Containers are great for continuous integration and continuous delivery
(CI/CD) workflows.

Consider the following example scenario:

 Your developers write code locally and share their work with their colleagues using
Docker containers.
 They use Docker to push their applications into a test environment and run
automated and manual tests.
 When developers find bugs, they can fix them in the development environment and
redeploy them to the test environment for testing and validation.
  When testing is complete, getting the fix to the customer is as simple as pushing the
updated image to the production environment.

Responsive deployment and scaling

Docker's container-based platform allows for highly portable workloads. Docker

containers can run on a developer's local laptop, on physical or virtual machines in a
data center, on cloud providers, or in a mixture of environments.

Docker's portability and lightweight nature also make it easy to dynamically manage
workloads, scaling up or tearing down applications and services as business needs
dictate, in near real time.

Running more workloads on the same hardware

Docker is lightweight and fast. It provides a viable, cost-effective alternative to

hypervisor-based virtual machines, so you can use more of your server capacity to
achieve your business goals. Docker is perfect for high density environments and for
small and medium deployments where you need to do more with fewer resources.

Docker architecture
Docker uses a client-server architecture. The Docker client talks to the Docker
daemon, which does the heavy lifting of building, running, and distributing your
Docker containers. The Docker client and daemon can run on the same system, or
you can connect a Docker client to a remote Docker daemon. The Docker client and
daemon communicate using a REST API, over UNIX sockets or a network interface.
Another Docker client is Docker Compose, that lets you work with applications
consisting of a set of containers.
The Docker daemon

The Docker daemon ( dockerd ) listens for Docker API requests and manages Docker
objects such as images, containers, networks, and volumes. A daemon can also
communicate with other daemons to manage Docker services.

The Docker client

The Docker client ( docker ) is the primary way that many Docker users interact with
Docker. When you use commands such as docker run , the client sends these
commands to dockerd , which carries them out. The docker command uses the Docker
API. The Docker client can communicate with more than one daemon.

Docker Desktop

Docker Desktop is an easy-to-install application for your Mac, Windows or Linux

environment that enables you to build and share containerized applications and
microservices. Docker Desktop includes the Docker daemon ( dockerd ), the Docker
client ( docker ), Docker Compose, Docker Content Trust, Kubernetes, and Credential
Helper. For more information, see Docker Desktop.

Docker registries
A Docker registry stores Docker images. Docker Hub is a public registry that anyone
can use, and Docker looks for images on Docker Hub by default. You can even run
your own private registry.

When you use the docker pull or docker run commands, Docker pulls the required
images from your configured registry. When you use the docker push command,
Docker pushes your image to your configured registry.

Docker objects

When you use Docker, you are creating and using images, containers, networks,
volumes, plugins, and other objects. This section is a brief overview of some of
those objects.

Images
An image is a read-only template with instructions for creating a Docker container.
Often, an image is based on another image, with some additional customization. For
example, you may build an image which is based on the ubuntu image, but installs
the Apache web server and your application, as well as the configuration details
needed to make your application run.
You might create your own images or you might only use those created by others
and published in a registry. To build your own image, you create a Dockerfile with a
simple syntax for defining the steps needed to create the image and run it. Each
instruction in a Dockerfile creates a layer in the image. When you change the
Dockerfile and rebuild the image, only those layers which have changed are rebuilt.
This is part of what makes images so lightweight, small, and fast, when compared to
other virtualization technologies.

Containers
A container is a runnable instance of an image. You can create, start, stop, move, or
delete a container using the Docker API or CLI. You can connect a container to one
or more networks, attach storage to it, or even create a new image based on its
current state.

By default, a container is relatively well isolated from other containers and its host
machine. You can control how isolated a container's network, storage, or other
underlying subsystems are from other containers or from the host machine.

A container is defined by its image as well as any configuration options you provide
to it when you create or start it. When a container is removed, any changes to its
state that aren't stored in persistent storage disappear.

Example docker run command

The following command runs an ubuntu container, attaches interactively to your local
command-line session, and runs /bin/bash .

$ docker run -i -t ubuntu /bin/bash

When you run this command, the following happens (assuming you are using the
default registry configuration):

1. If you don't have the ubuntu image locally, Docker pulls it from your
configured registry, as though you had run docker pull ubuntu manually.
2. Docker creates a new container, as though you had run a docker container
create command manually.

3. Docker allocates a read-write filesystem to the container, as its final layer.

This allows a running container to create or modify files and directories in its
local filesystem.
 4. Docker creates a network interface to connect the container to the default
network, since you didn't specify any networking options. This includes
assigning an IP address to the container. By default, containers can connect
to external networks using the host machine's network connection.

5. Docker starts the container and executes /bin/bash . Because the container is
running interactively and attached to your terminal (due to the -i and -
t flags), you can provide input using your keyboard while Docker logs the
output to your terminal.
6. When you run exit to terminate the /bin/bash command, the container stops
but isn't removed. You can start it again or remove it.
The underlying technology
Docker is written in the Go programming languageopen_in_new and takes
advantage of several features of the Linux kernel to deliver its functionality. Docker
uses a technology called namespaces to provide the isolated workspace called the
container. When you run a container, Docker creates a set of namespaces for that
container.

These namespaces provide a layer of isolation. Each aspect of a container runs in a

separate namespace and its access is limited to that namespace.

Docker – Using Public Repositories To Host

Docker Images
Docker is a software platform for creating isolated virtualized environments
for building, deploying, and testing applications with ease. In this tutorial, we
will learn how to host public repositories on docker hub which is a hosted
repository service provided by Docker for finding and sharing
container images. Just like GitHub allows the hosting of code of our
application, DockerHub allows the hosting of Images of our applications.
Docker Image
In order to run these applications, we first need to create an image of the
current application state. An image can be sometimes referred to as a snapshot
of our project. Images are read-only in nature and consist of file that contains
the source code, libraries, dependencies, tools, and other files needed for an
application to run.
A Docker image is a read-only template that contains a set of instructions for
creating a container which can run on the Docker platform.
Docker Registry

A Docker registry is a service that stores and manages Docker images. Docker
registry could be hosted by a third party, as a public or private registry. Some
examples of Docker registries are as follows:
 Docker Hub
 GitLab
 AWS Container Registry
 Google Container Registry
 Docker – Private Registries

Docker Repository
A Docker repository is a collection of different Docker images with the same
name, that have different tags. Tags basically are identifiers of the image within
a repository.
In this tutorial, we will use Docker Hub to host our repositories, which is free
for public use.

Steps
Step 1. Creating An Account On Docker Hub. Go to DockerHub and create a
new account or log in to your existing account.

Step 2. Creating A Repository (optional)

On the docker hub, you can create a repository by clicking on create repository
button. Give the repository a name and description and make sure it is marked
as public. This step is not necessary when you are hosting a public repository.
It is used while hosting a private one.

Repository Name image widget.

Step 3. Build a Docker Image

Now we will generate a basic express application and create an Image out of
it.
$ mkdir express-app && cd express-app

$ npx express-generator -e
Now, create a Dockerfile for the application and copy the content as shown
below:
$ touch Dockerfile

FROM node:16

# Create app directory

WORKDIR /usr/src/app

# Install app dependencies

# A wildcard is used to ensure both package.json

# AND package-lock.json are copied
# where available (npm@5+)

COPY package*.json ./

RUN npm install

# If you are building your code for production

# RUN npm ci --only=production

# Bundle app source

COPY . .

EXPOSE 3000

CMD [ "npm", "start" ]

You can now build this Dockerfile with the docker build command.
$ docker build -t rhythmshandlya/express-app .

One thing to notice is as did not specify the tag name, it will be given the :
latest tag.
Step 4. Run This Image Locally
$ docker run -p 3000:3000 rhythmshandlya/express-app
Step 5. Push Image to docker hub. To push a local Image to the docker hub
we will need to log in to the docker hub with our terminal.
$ docker login

$ docker push rhythmshandlya/express-app

Step 6. Playing With Tags. We can make changes to this application and give
it a version tag of 0.0.1
$ docker build -t rhythmshandlya/express-app:0.0.1 .

$ docker push rhythmshandlya/express-app:0.0.1

Output:

Now that we have hosted our image in public anyone can pull and run them on
their machines.
$ docker pull rhythmshandlya/express-app:latest

OR

$ docker push rhythmshandlya/express-app:0.0.1