Computer System Administration

(ECC 4209)

Lecture 3
(Linux Virtualization)

[email protected]
1
 Contents
1. Introduction to Linux Virtualization
2. Finding the right virtualization technology
3. Using Linux repository managers
4. Building effective environments using VirtualBox
5. Building containers with LXC
6. How and when to closely manage VMs

2
Introduction to Linux Virtualization
• Here are two virtualization facts that need to be known
– Linux absolutely dominates the virtual space
– Virtualization makes it easier to learn any technology
• This lecture will introduce the dominant enterprise
virtualization technologies currently in use.
• Also explore virtualized environment where for safely
learn Linux administration skills

3
 What is Virtualization?
• Once upon a time when a new server is required, need to
perform this:
– to research
– request budget approval
– negotiate
– order
– safely house
– provision
– launch a brand-new server machine
• The process from start to finish could take months
• When increasing demand on that service threatened to
overwhelm the server’s capacity, need to start the whole
thing over again, hoping to eventually get the
capacity/demand balance right 4
 What is Virtualization?
(continue)
• A common scenario would see a company providing multiple
but codependent services, each run on its own hardware
– E.g. frontend web server deployed along with a backend database
– Soon end up with one server deeply underused and one (next to it on
the rack) unable to keep up
• Imagine to securely share compute, memory, storage, and
networking resources of a single high-capacity server among
multiple services
• Imagine being able to carve virtual server instances out of
that physical server by assigning them only the level of
resources they need, and then instantly adjusting capacity to
meet changing demands
5
 What is Virtualization?
(continue)
• Now imagine being able to efficiently pack dozens of those
virtual computers running multiple OS onto a single
bare-metal server so that absolutely nothing is ever wasted
• Imagine then being able to have those VMs automatically
spill over onto other physical servers as the first ones fill up
• Imagine the convenience of being able to kill a VM that’s
failed or in need of an update, and replace it so quickly that
users might never realize anything has changed
– it is called virtualization
– so attractive that it now dominates the enterprise computing world!

6
 What is Virtualization?
(continue)
• Now most local or cloud-based server loads are running on
some kind of virtualization technology and the OS running
the vast majority of those virtual workloads is Linux
• Figure 2.1 shows VM clients ofhardware host with
connectivity to each other and to a larger network through
external router
Network connectivity

VM VM VM
VM
(web (web (Database
(firewall)
server) server) )

Host Operating System

Rout
Hardware Layer er
(compute, storage, networking)

Figure 2.1: VM clients of a hardware host with connectivity to each 7

other and to a larger network through an external router
 What is Virtualization?
(continue)
• Amazon Web Services (AWS), lets customers rent capacity
on (Linux) servers hosting millions of VMs
– run workloads, including many most popular online services
• Figure 2.2 shows how an AWS Elastic Compute Cloud (EC2)
VM instance serves as a hub for a full range of storage,
database, and networking tools
• Virtualization uses isolated space (namespace) on a physical
computer where a guest OS can be installed and then fooled
into thinking that it’s all alone on its own computer
(remember OS process?)
• Guest operating systems can share network connections so
that their administrators can log in remotely to perform their
work exactly as they would on traditional machines
8
 EC2
Security
Group

Internet

Figure 2.2: A typical cloud computing workload centered around AWS’s Elastic
Cloud Compute (EC2) VM instances on Amazon Web Services
9
 Virtualization Approaches
•
(Hypervisors)
Hypervisors controls host system hardware to some extent,
providing each guest OS the resources it needs (Figure 2.3)
• Guest machines are run as system processes, but with
virtualized access to hardware resources
• AWS servers built on open source Xen hypervisor (now KVM).
• Other important hypervisor platforms include VMware ESXi,
KVM, and Microsoft’s Hyper-V (WSL2 run on Hyper-V)
Network
connectivity
Guest 1 Guest 2 Guest 3 Guest 4
Guest Guest Guest
OS OS OS
Hypervisor

Rout Linux Kernel

er
Hardware Layer
(compute, storage, networking)
10
Figure 2.3: A type 2 hypervisor architecture showing full OS installed on
each guest with some special administration duties delegated to Guest1
 Virtualization Approaches
(Containers)
• Containers is extremely lightweight virtual servers that, rather
than running as full OS, share the underlying kernel of their
host OS (see Figure 2.4)
• Containers are built from scripts, created and launched in
seconds, easily and reliably shared across networks
• Popular container technology are Docker and LXC
Network
connectivity
Contain Contain Contain Contain
er er er er

LXC host software

Linux Kernel
Rout
er
Hardware Layer
(compute, storage, networking)
Figure 2.4 LXC architecture showing access between the LXC 11

environment
 Virtualization Considerations
• Full-sized hypervisors like Xen and KVM (via management
frontend like Libvirt) are normally used for enterprise-sized
deployments involving large fleets of Linux VMs
• VirtualBox (and VMware’s Player) are perfect for testing and
experimenting with live operating systems, one or two at a
time, without the need to install them to actual PCs
– their relatively high overhead makes them unsuitable for most
production environments
• Container technologies like LXC and Docker are lightweight
and can be provisioned and launched in mere seconds.
– LXC containers are particularly well suited to playing with new
technologies and safely building OS software stacks.
– container technology running countless dynamic, integrated fleets of
containers as part of vast microservices architectures
12
Software Package Management
• Package manager called Advanced Package Tool (APT)
• Package managers connect computers to online software
repositories (mostly free and open source)
• The manager, which comes installed with Linux by default,
has a number of jobs:
– Maintains a local index to track repositories and their contents
– Tracks the status of all the software installed on local machine
– Ensures that all available updates are applied to installed software
– Ensures that software dependencies (other software packages or
configuration parameters required by the package) are met for new
applications before they’re installed
– Handles installing and removing software packages

13
 Online (master) software
repository

Provide mirrors with updated packages

Maintain software index on PC

Transfer and install software packages

Update (patch) installed packages

Repository mirror servers Linux PC

(for faster, more efficient downloads)

Figure 2.5: The relationships among master software repositories, mirror download
servers, and Linux running on an end user machine

14
Package Manager and Distro
Package Manager Distribution

DPKG/APT Debian
Ubuntu
Mint
Kali Linux

RPM/YUM Red Hat Enterprise Linux

Fedora
Rocky Linux
RPM/YaST SUSE Linux
OpenSUSE

Nix NixOS

GNU Guix GNU Guix System

15
 Working with Debian package
manager
• DEB or RPM package file is available for Skype on Linux
• Install it from the command line using dpkg commad:
– Use the –i flag (for install)
• This example assumes that the package is saved to the
Downloads directory in user account:
– $ cd /home/<username>/Downloads
– # dpkg -i skypeforlinux-64.deb
• Please note that dpkg only installs a package
– will notify any dependencies that need to be installed but it will not
install them
• Package Management System like APT or YUM is a set of
tools that help install, remove, and change packages easily
– APT is a clever dpkg
– YUM is a clever rpm 16
 Installing VirtualBox
• Getting VirtualBox installed on an Ubuntu machine:
– #apt update
– #apt install virtualbox
• Prompt # means this command requires admin privileges,
and normally accessed by prefacing command with sudo
– $sudo apt install virtualbox
• Install Extension Pack on Ubuntu for VirtualBox adds-on:
– $sudo apt install virtualbox-guest-additions-iso
• APT knew virtualbox when added it to the install command
– because a VirtualBox package is part of an online repository with
which APT is already familiar
• For searching available software using keywords
– $apt search sensors
– $apt show lm-sensors
17
 Unknown Repositories
• If software not known please add to the repository manually to
the /etc/apt/sources.list or use
add-apt-repository command:
– $sudo apt install software-properties-common
– $sudo add-apt-repository [options] repository
• To see available options for add-apt-repository command
– $man add-apt-repository
• Authenticated installation (for MongoDB)
– $sudo apt-key adv --keyserver
hkp://keyserver.ubuntu.com:80 --recv
9DA31620334BD75D9DCB49F368818C72E52529D4
– $sudo add-apt-repository 'deb [arch=amd64]
https://repo.mongodb.org/apt/ubuntu
bionic/mongodb-org/4.0 multiverse'
18
 Default Repositories
• In Ubuntu, the default repositories are:
– Main : Free and open-source software from the Ubuntu developers
– Universe : Free and open-source software from the Linux community
– Restricted : Device drivers and other software under licensing from
manufacturers
– Multiverse : All other software under copyright or legal restriction
• In Debian, the default repositories are:
– Main : Free licensed software
– Contrib : Free software that depends on non-free licensed software
– Non-free : Software with restrictions on use and distribution

19
 Defining a Virtual Machine (VM)
• VirtualBox enabled customization of VM’s “hardware” as DIY
computer or PC building exercise
• After clicking “New” in the VirtualBox menu, provide VM name
• Figure 2.6, VirtualBox should be able to correctly populate the
Type and Version fields automatically
• “Type” and “Version” won’t install an actual OS, but are used
to apply appropriate hardware emulation settings
• On the next screen, allocate RAM to VM
– the default amount (768 MB) should be fine
– leave enough for the host machine and any other VMs that might
already live on it
– Even if each VM is only using the default amount of memory, two or
three of them can start to eat away at the RAM needed for normal host
operations
20
 Defining a Virtual Machine (VM)
• The next screen (Figure 2.8) provides choice of hard disk
file-type format for the disk
• Unless planning to eventually export the disk to use with
some other virtualization environment, the default VirtualBox
Disk Image (VDI) format will work fine
• I’ve also never regretted going with the default Dynamically
Allocated option (Figure 2.9) to determine how the virtual
drive will consume space on the host.
• Here dynamic means space on the host storage disk will be
allocated to the VM only as needed.
– Should the VM disk usage remain low, less host space will be
allocated.

21
 Defining a Virtual Machine (VM)
• A fixed-sized disk, will be given its maximum amount of space
right away, regardless of how much it’s actually using.
– If Linux is chosen VirtualBox will probably offer only 8 GB of total disk
size on the next screen (Figure 2.10)
– If Windows is chosen as OS, the default choice would have been 25 GB
• When done, click Create, and the new VM will appear in the list
of VMs on the left side of the VirtualBox manager
– But that’s only the machine, now need an OS to bring it to life

22
 The New
VM button

VirtualBox will try

to guess the OS

Figure 2.6: The Hard Disk screen. Note how, in this case, the non-default
Use an Existing Virtual Hard Disk File radio button is selected.
23
Figure 2.7: The Hard Disk screen. Note how, in this case, the non-default
Use an Existing Virtual Hard Disk File radio button is selected.
24
Figure 2.8: Virtual hard disks can be created using a number of formats.
VDI is fine for VMs that will be used only within VirtualBox.
25
Figure 2.9: Dynamically allocated virtual disks will only consume
as much space on their host’s devices as they need
26
Figure 2.10: Virtual disk can be as large as 2 TB or the maximum
free space on the host device.
27
 Installing an OS
• Now that new VM’s virtual hardware profile, install a new OS:
– Download a file (in ISO format) containing the image of the Linux distro
– Boot the new VM using a virtual DVD drive containing the downloaded
ISO
– Work through the standard OS installation process
– Boot the VM and launch the OS that installed previously
• Once settled on a distribution, need to download an .ISO file
containing the OS files and installation program
• In the case of Ubuntu, go to the https://ubuntu.com website
• If using this VM for administration tasks, then the small and
fast server version is probably a better choice than Desktop

28
 Installing an OS
• Large files can sometimes become corrupted during the
download process
• It’s always a good idea to immediately calculate the checksum
(or hash) for the .ISO downloaded for verification
• Get the appropriate SHA or MD5 checksum, which is a long
string looking something like this:
– 4375b73e3a1aa305a36320ffd7484682922262b3
• In the case of Ubuntu, web page at http://releases.ubuntu.com
– clicking the directory matching the downloaded version, and then
clicking one of the links to a checksum (MD5SUM or SHA1SUM)
• Compare the appropriate string from that page with the results
of a command run from the same directory:
– $sha1sum ubuntu-16.04.2-server-amd64.iso
– If they matched, it’s fine but if not then need to download the .ISO again
29
 Installing an OS
• Once ISO file is in place, head back to VirtualBox
• With the VM just created highlighted in the left panel, click the
green Start button at the top of the app
– select an ISO file from the file system to use as a virtual DVD drive
• The new VM will read this DVD and launch an OS installation
• When everything is installed, there still might be a few more
things to take care of before successfully boot into VM

30
 Installing an OS
• With VM highlighted, click the yellow Settings icon
– Configure VM’s environment and hardware settings
• By Clicking Network allows to define network connectivity
– For VM to have full internet access through the host machine’s NIC,
select Bridged Adapter from the Attached to drop-down and then select
the name of the host’s adapter (as shown in Figure 2.11)
– Alternatively, uses NAT option

Figure 2.11: The

Network tab of
the Settings
dialog where type
of network
interface (or
interfaces) to use
for VM
31
Cloning and Sharing VirtualBox VM
• One of the most obvious advantages of working with VMs is
the ability to quickly access a fresh, clean OS environment
• Why not keep the original VM in its clean post-install state and
create an identical clone whenever want to do work?
• Select the (stopped) VM want to use as a master copy, click
the Machine menu link, and then click Clone
• Confirm the name for the clone and then, after clicking Next
– Full Clone (entirely new file copies are created for the new VM)
– Linked Clone (new VM will share all the base files with its master while
maintaining any new work separately)
• Selecting the Linked option will go much faster and take up
much less room on hard disk
– Downside is unable to move this clone to a different computer later
• Now click Clone, and a new VM appears in the VM panel
– Start it and then log in using the same credentials set on the master32
Managing VMs From Command Line
• VirtualBox comes with its own command-line shell that’s
invoked using vboxmanage
• Why bother with the command line?
– Allows to work on remote servers
• To see how vboxmanage works, use list VMs to list all the VMs
currently available on the system
– $ vboxmanage list vms
• Cloning VMs (Kali Linux template VM, naming copy newkali)
– $vboxmanage clonevm --register Kali-Linux-template
--name newkali
• Need to convert the VM to some standardized file format Open
Virtualization Format (OVA):
– $vboxmanage export website-project -o website.ova
• After transfering OVA file to remote machine: 33
– $vboxmanage import website.ova
 Working with Linux Containers
(LXC)
• VirtualBox is great for running operations requiring
– Linux kernel access
– using security features like SELinux
– need GUI desktop sessions
– testing OS like Windows
• But if need fast access to a clean Linux environment and not
looking for any special release version, then use LXC instead
• This is because LXC containers skillfully share many system
resources with both the host and other containers
– they work like full-bore, standalone servers, using only minimal storage
space and memory

34
 Getting Started with LXC
• Install LXC on Ubuntu workstation
– #apt update
– #apt install lxc
• The -n value sets the name for the container, and -t tells LXC to
build the container with Ubuntu template:
– $sudo lxc-create -n myContainer -t myContainer
– $sudo DOWNLOAD_KEYSERVER="keyserver.ubuntu.com"
lxc-create -t download -n myContainer
• There are actually quite a few templates available, see from this
listing of the /usr/share/lxc/templates/ directory:
– $ls /usr/share/lxc/templates/

35
 Creating the First Container
• If decided to create a CentOS container, then make a note of
the final few lines of the output, as it contains information
about the password to log in:
– #lxc-create -n centos_lxc -t centos
[...]
The temporary root password is stored in:
'/var/lib/lxc/centos_lxc/tmp_root_pass'

36
 Creating the First Container
• Log in using the user name root and the password contained
in the tmp_root_pass file
• For Ubuntu template, then use ubuntu for both user name
and password
• For anything serious, change that password right away:
– $passwd
Changing password for ubuntu.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
• Now use lxc-ls --fancy to check the status :
– #lxc-ls –fancy
NAME STATE AUTOSTART GROUPS IPV4 IPV6
myContainer STOPPED 0 - - - 37
 Creating the First Container
• Start container using lxc-start command with -d and -n
– $sudo lxc-start -d -n myContainer
– $sudo lxc-ls --fancy
NAME STATE AUTOSTART GROUPS IPV4 IPV6
myContainer RUNNING 0 - 10.0.3.142 -
• This time, the container is running and has been given an IP
address (10.0.3.142).
• Use this address to log in using a secure shell (SSH) and
launch a root shell session within a running container using
lxc-attach command:
– #lxc-attach -n myContainer
– root@myContainer:/#
• list the container’s network interfaces:
– root@myContainer:/# ip addr
38
 Starting the First Container
• When done looking around the new container, either run exit to
log out leaving the container running
– root@myContainer:/# exit
• Exit or shut down the container using shutdown -h now
– The -h flag I added to shutdown before stands for halt
– If I used r instead, rather than shutting down for good, the container
would reboot
• Let’s run reboot and then try to log in again right away to see
how long it takes for the container to get back up on its feet:
– root@myContainer:/# shutdown -r now
• Container can be stopped and started in mere seconds!
– #lxc-attach -n myContainer

39
 Container Tips & Tricks
• From here on, until type exit, perform sudo full time:
– $ sudo su
[sudo] password for username:
#
• Now change directory to /var/lib/lxc/, and list the
contents and see a directory with the name of the container
– #cd /var/lib/lxc
– #ls
myContainer
• Move to container directory, and list its contents
• There’ll be a file called config and a directory called rootfs
(the fs stands for file system)
– #cd myContainer
– #ls
40
config rootfs
 Container Tips & Tricks
• Check the rootfs directory
– #cd rootfs
– #ls
bin dev home lib64 mnt proc run srv tmp var
boot etc lib media opt root sbin sys usr
• All those subdirectories that fill rootfs, all part of the Linux
Filesystem Hierarchy Standard (FHS).
– It’s container’s main root (/) directory but within the host’s file system
• With admin permissions on the host, browse through those
directories and edit any files necessary
– even when the container isn’t running
• Able to do all kinds of things with this access
• Suppose accidently lock yourself out on a container
– navigate through the file system fixing the configuration file that were
messed up, and getting back to work
41
 Summary
• Hypervisors like VirtualBox provide an environment where
virtual OS can safely access hardware resources, whereas
lightweight containers share their host’s software kernel
• Linux package managers like APT and RPM (Yum) oversee
the installation and administration of software from curated
online repositories using a regularly updated index that mirrors
the state of the remote repository
• Getting a VM going in VirtualBox requires defining its virtual
hardware environment, downloading an OS image, and
installing the OS on the VM
• Easily clone, share, and administer VirtualBox VMs from the
command line with vboxmanage command
• LXC containers are built on predefined, distro-based templates
• LXC data is stored within the host file system, making it easy
to administer and maintain 42
•
Key Terminologies
Virtualization is the logical sharing of compute, storage, and
networking resources among multiple processes, allowing
each to run as if it’s a standalone physical computer
• A hypervisor is software running on a host machine that
exposes system resources to a guest layer, allowing the
launching and administration of fullstack guest VMs
• A container is a VM that, instead of full-stack, lives on top of
(and shares) the host machine’s core OS kernel
• Containers are extremely easy to launch and kill, according to
short-term need
• A dynamically allocated virtual drive in VirtualBox takes up
only as much space on the physical drives as the VM uses
• A fixed-size disk, by contrast, takes up the maximum space no
matter how much data is there
• A software repository is a location where digital resources can43
be stored
 Command-line review
• apt install virtualbox uses APT to install a software
package from a remote repository
• dpkg -i skypeforlinux-64.deb directly installs a downloaded
Debian package on a Ubuntu machine
• wget https://example.com/document-to-download uses the
wget command line program to download a file
• apt update syncs the local software index with what’s
available from online repositories
• md5sum ubuntu-16.04.2-server-amd64.iso calculates the
checksum for a downloaded file for verification
• vboxmanage clonevm Kali-Linux-template --name newkali
uses the vboxmanage tool to clone an existing VM
• lxc-start -d -n myContainer starts an existing container
• ip addr displays info NIC (including their IP addresses)
44
• exit leaves a shell session without shutting down container
 References
• Linux in Action, David Clinton:
– https://www.manning.com/books/linux-in-action
• Learning Modern Linux, Michael Hausenblas:
– https://www.oreilly.com/library/view/learning-modern-linux/97
81098108939/
• Linux Administration Best Practices, Scott Alan Miller:
– https://www.packtpub.com/product/linux-administration-best-
practices/9781800568792
• Linux Containers and Virtualization: A Kernel
Perspective:
– https://link.springer.com/book/10.1007/978-1-4842-6283-2
45
 References
(continue)
• Hardware and Software Support for Virtualization,
Edouard Bugnion:
– https://www.morganclaypool.com/doi/abs/10.2200/S00754E
D1V01Y201701CAC038

46