SERVER + NOTES, COTD.

Linux Logical Volume Management (LVM):

This is the new system for managing disks. Previously, sysadmins had to go through a very laborious
process. Managing disk space was previously a significant task for sysadmins. Running out of disk space
used to be the start of a very long and complex series of tasks to increase the space available to a disk
partition. It also required taking the system offline.

This involved installing a new hard drive, booting to a recovery or single user mode, creating a partition
and a file system on the new hard drive using temporary mount points to move the data from the too
small file system to the new larger one, then changing the content of the etk and fstab files to reflect the
correct device name for the new partition, then rebooting to remount the new file system on the correct
mount point.

LVM allows for very flexible disk space management, providing features like the ability to add disk space
to a logical volume and its file system while that file system is mounted and active. It also allows for the
collection of multiple physical hard drives and partitions into a single volume group, which can then be
divided into more logical volumes - it makes life a little easier.

Linux Logical Volume Manager also allows reducing the amount of disk space allocated to a logical
volume, but there are requirements:

First, the volume must be unmounted.

Second, the file system itself must be reduced before the volume where it resides can be reduced. NOTE:
the file system must allow resizing for this feature to work! The Ext2 EXT3 and EXT4 file systems all allow
both offline, (unmounted) and online (mounted) resizing when increasing a file system's size, as well as
offline resizing when REDUCING the size. Check the file systems details to verify whether they can be
resized at all, and especially whether they can be resized while online.

Always run new distributions in a VirtualBox virtual machine for a few days to ensure you won't run into
any devastating unforeseen problems when installing it in production machines.
A third of the way through a Fedora dsitro installation, a user ran out of space on the file system.
VirtualBox detected the "out of space condition" and paused the virtual machine, displaying an error
message indicating the problem. This was not due to the virtual disk being too small. The logical volume
on the host computer was running out of space, so the virtual disk of the virtual machine didn't have
enough space to expand onto the host's logical volume. Most modern distros use logical volume
management by default, so if there is free space available in the volume group, one can assign additional
disk space to the appropriate logical volume, then expand the file system of the host on the fly.

Linux logical volume management makes it possible to recover without losing any data and without
having to restart a time-consuming installation. This means no need to reformat the entire hard drive
and reinstall the operating system or even reboot: simply assign some available space to the appropriate
logical volume and resize the file system -- all the while the file system is online and running the
installation program. After resizing the logical volume and the file system, it's possible to resume running
the virtual machine and the installation will continue as if no problems have occurred.

This has happened to a lot of people.

The structure of a logical volume manager disk environment:

Volume management enables combining multiple individual hard drives or disk partitions into a single
volume group (here labeled VG), so the volume group can be subdivided into logical volumes or LV or
even used as a single large volume. Regular file systems such as ext3 or ext4 can then be created on a
logical volume.

In the following figure two complete physical hard drives and one partition from a third hard drive have
all been combined into a single volume group. Two logical volumes have been created from the space
and the volume group, and a file system such as Ext 3 or 4 has been created on each of the two logical
volumes. Thus, adding disk space to a host is straightforward, but done relatively infrequently these
days.
The basic steps:

Create an entirely new volume group or add new space to an existing volume group and either expand
an existing logical volume or create a new one.

Sometimes, it's necessary to add a new logical volume to a host. For example, after noticing that the
directory containing virtual disks for VirtualBox virtual machines is filling up the home file system, create
a new logical volume in which to store the virtual machine data, including the virtual disks.

This frees up a lot of space in the home file system and allows management of the disk space for VMs
independently.

Basic steps for adding a new logical volume:

 One might have to install a new hard drive, but creating a partition on that hard drive is optional.
 Create a physical volume, a PV of the complete hard drive or of a partition on the hard drive.
 Assign the new physical volume to an existing volume group (VG), or create a new volume
group.
 Create a new logical volume (LV) from the space in the volume group.
 Create a file system on the new logical volume.
 Add appropriate entries to the etcd fstab for mounting the file system.
 Mount the file system.
An example that demonstrates how to use a CLI to extend an existing volume group for more space,
create a new logical volume in that space, and create a file system on the logical volume. This procedure
can be performed on a running mounted file system. However, only the ext3 and ext4 file systems can be
resized on the fly on a running mounted file system. No other file systems, including btrfs and ZFS can be
resized. If there isn’t enough space in the volume group on the existing hard drives to add the desired
amount of space, it might be necessary to add a new hard drive and create the space to add the logical
volume. First the physical hard drive must be installed, then create a new physical volume PV:

pvcreate /dev/hdd
This assumes the new hard drive is assigned as /dev/HDD. It's not necessary to create a partition on the
new drive. Creating the physical volume, which will be recognized by the logical volume manager, can be
performed on a newly-installed raw disk or on a Linux partition Type 83. If the entire hard drive will be
used, creating a partition first doesn't offer any real advantages - it just uses disk space for metadata that
could otherwise be used as part of the PV.

Extending an existing volume group rather than creating a new one: After the physical volume has been
created, extend the existing volume group (VG) to include the space on the new PV. In this example, the
existing volume group is named MyVG01.

vgextend /dev/MyVG01 /dev/HDD

lvcreate -L +50G --name Stuff MyVG01

mkfs t ext4/dev/MyVG01/stuff.
Next, add a file system label, to make it easy to identify the file system in case of some kind of disk-
related problem:

e2label /dev/MyVG01/Stuff Stuff

Verify the volume has been created correctly, using the df, LVS, and Vgs commands:

df
lvs
vgs
To resize a file system:
 1. Install a new hard drive.
2. Optionally, create a partition on the hard drive.
3. Create a physical volume (PV) of the complete hard drive (or partition on the hard drive).
4. Assign the new physical volume to an existing volume group (VG), or create a new volume group.
5. Create one or more logical volumes (LVs) from the space in the volume group, or expand an
existing logical volume with some or all of the new space in the volume group.
6. If a new logical volume was created, create a file system on it.
7. If adding space to an existing logical volume, use the resize to command to enlarge the file
system to fill the space in a logical volume.
8. Add appropriate entries to the etc fs tab for mounting the file system.
9. Lastly, mount the file system.
To add entries to the etc/fstab file and mount a volume in Linux, you need to first
identify the device you want to mount using the lsblk command. Then, you can create
a mount point directory using mkdir command. Next, you edit the etc/fstab file using a
text editor like nano or vim to add an entry specifying the device, mount point, file
system type, and mount options. Save the file and then use the mount -a command to
mount all entries in the etc/fstab file. This process allows the system to automatically
mount the volume at boot and provides a consistent path for accessing the volume.

An example of how to resize an existing logical volume in an LVM environment using the CL to add about
50GB of space to the file system. This procedure can only be used on a mounted live file system with the
Linux 2.6 kernel or higher, as well as the ext3 or ext4 file systems.

Lvextend -L +50G /dev/MyVG01/Stuff

To resize a file system:

Resize2fs /dev/MyVG01/Stuff

It’s not recommended to do this on any mission-critical system, but it can be done – even on the root file
system. Only the ext3 and ext4 file systems can be resized on the fly on a running mounted file system.
Many other file systems, including btrfs and ZFS cannot be resized.
 NTFS

NTFS is the equivalent to the win 95 File Allocation Tables, but with improvements to performance and
security.

The OS uses this system to create and maintain a file system on a storage drive or device. It organizes the
data into files to control how data (files) is named, stored, retrieved and updated, as well as metadata
associated with those files, such as file ownership and user permissions.

NTFS is just one type of file system among systems differentiated by the OS and drive with which they’re
associated.

There's also a Distributed File System (DFS), where files are stored across multiple servers, but accessed
and handled just as if they were stored locally. A DFS enables multiple users to easily share data and files
on a network and provides redundancy. Microsoft Windows and some removable storage devices use
NTFS to organize name and store files. NTFS is an option for formatting SSDs, where the speed of NTFS is
particularly useful.

HDDs, USBs, and MicroSD cards used with windows also benefit from NTFS. Depending upon the storage
capacity of the device, the OS used, and the type of drive, a different file system might be preferable,
such as Fat32 or Extended FAT.

Each file system has benefits and some drawbacks. For example, security and permissions are much
more advanced with NTFS, ExFAT, and FAT32. On the other hand, FAT32 and exFAT work much better
with non-windows OSes, such as Mac and Linux.
All Microsoft OSes from Windows XP forward use NTFS version 3.1 as their main file system, but NTFS is
also used on external drives because it has the capacity those drives need supporting large files and
partition sizes.

NTFS can support up to eight petabytes of volumes and files on Windows Server 2019 and Windows 10.
The theoretical limit for the individual file size NTFS can support is 16 bytes -one kilobyte.

When installing an OS, the user chooses a file system, and when formatting an SSD or an HDD, users
choose a file system to use. Formatting each type of drive is slightly different, but both are compatible
with NTFS. When an HDD is formatted (initialized), it's divided into partitions. These are major divisions
of the hard drives, physical space. Within each partition, the OS tracks all stored files.

Using NTFS, each file is stored on the HDD in one or more indivisible “clusters” – disk spaces of a
predefined uniform size, ranging from 512 bytes to 64kB. Windows NT provides a recommended default
cluster size based on overall drive size. Thus, for example, a four-gigabyte drive has a default cluster size
of four kilobytes.

Because these clusters are indivisible, even the smallest file uses one cluster, and a 4.1kB file would use
two clusters or eight kilobytes on a four-kilobyte cluster system. Cluster sizes are based upon balancing
maximal disk space use and minimal disk accesses to retrieve a given file. Generally, the larger the drive,
the larger the default cluster size, because it's assumed that a user will prefer fewer disk accesses and
better performance at the expense of less efficient space use.

When a file is created in NTFS, a record of that file is created in the Master File Table (MFT). This record is
used to locate possibly scattered clusters for a file. NTFS looks contiguous space to hold all of a file’s
clusters, but that isn't always available.

Along with its content, every file contains metadata – a description of its attributes. Among these are file
permissions and encryption, one of the distinguishing characteristics of NTFS compared to FAT.

NTFS Advantages:
 Organizational efficiency - NTFS uses a B-tree directory scheme to keep track of file clusters. This
efficiently sorts and the organizes files.
 Accessible data - NTFS stores file, cluster and other data in the MFT, not just an overall governing
table like FAT.
 File size - NTFS supports very large files.
 User permissions – NTFS uses an access control list to restrict file access.
 Integrated file compression - file sizes can be shrunk to provide more storage space.
 Unicode file naming - NTFS uses a natural file naming convention, which also allows for longer
file names and a wider array of characters.
 Security for data removable and non-removable disks.
 Requires less storage – NTFS supports sparse files, replacing empty information and long strings
of zeros with metadata that takes up less storage.
 Easy volume access - NTFS uses mounted volumes, which means disk volumes can be accessed
as normal folders in the file system.
  Greater control - NTFS uses disk quotas, granting organizations the ability to limit the amount of
storage granted to users.
 Easy logging - the MFT logs and audits files on the drive with a journaling file system, so
administrators can track files that have been deleted, added or changed. Transactions are all
recorded in a file system journal.
 Reliability - data and files can quickly be restored after a system failure or error because NTFS
maintains the consistency of the file system. It's a fault tolerant system with an MFT mirror file
that the system can reference if the first MFT gets corrupted.

NTFS Disadvantages:
 Limited OS compatibility – NTFS is only fully compatible with Windows OSes: it's read-only with
non-windows OS.
 Limited device support - many removable devices don't support NTFS, including Android
smartphones, DVD players, digital cameras, media players, smart TVs and printers.

Microsoft developed Fat32 before NTFS, making it the oldest of the three file systems, so it's generally
considered less efficient than NTFS, allowing a maximum file size of four gigabytes and 32 gigabyte
volumes in Windows.

Fat 32 is a lot easier to format than NTFS and simpler in some other ways: its file allocation table is a less
complex way to organize files than the MFT and NTFS and because it's simpler to use fat, 32 is more
compatible with Non-windows OSes and it's used where NTFS generally isn't, such as with smart TVs and
digital cameras and those kind of digital peripherals. Fat 32 works with every version of Mac, Linux and
Windows. NTFS is read-only with both Mac and Linux.

For example, Fat 32 can only store files up to four gigabytes, while fat can handle file sizes of 16
exabytes. Exfat does require additional software to work with Mac and Linux systems, but it's more
compatible with them than NTFS. It's ideal for when users need a larger file size than Fat32 can afford
but needs more compatibility than NTFS.

The journaling file system in NTFS also makes it possible to use the journal to repair data corruption,
which is something FAT cannot do. The MFT in NTFS holds more information about the files being held in
fat file allocation tables, making for better file indexing and cluster organization.