Linux File System

A file system is a structured method of storing and managing data—including files, directories, and metadata—on your machine. Think of it like a library. If thousands of books were scattered around, finding one would be hard. But in an organized structure, like labeled shelves, locating a book becomes easy.

This article aims to simplify the complexities of Linux file systems, guiding beginners through their layers, characteristics, and implementations. By shedding light on these nuances, we empower users to make informed choices in navigating the dynamic landscape of Linux operating systems.

What is the Linux File System

A Linux file system is a set of processes that controls how, where, and when data is stored or retrieved from storage devices. It manages data systematically on disk drives or partitions, and each partition in Linux has its own file system because Linux treats everything as a file, including devices and applications.. Like Windows uses C: and D: drives, Linux uses mount points, but everything appears under the root / directory. In Linux, everything is treated as a file, including devices and applications.

In this article, we will be focusing on the file system for hard disks on a Linux OSand discuss which type of file system is suitable.

Linux File System Structure

The architecture of a file system comprises three layers mentioned below.

1. Logical File System:

The Logical File System acts as the interface between the user applications and the file system itself. It facilitates essential operations such as opening, reading, and closing files. Essentially, it serves as the user-friendly front-end, ensuring that applications can interact with the file system in a way that aligns with user expectations.

2. Virtual File System:

The Virtual File System (VFS) is a crucial layer that enables the concurrent operation of multiple instances of physical file systems. It provides a standardized interface, allowing different file systems to coexist and operate simultaneously. This layer abstracts the underlying complexities, ensuring compatibility and cohesion between various file system implementations.

3. Physical File System:

The Physical File System is responsible for the tangible management and storage of physical memory blocks on the disk. It handles the low-level details of storing and retrieving data, interacting directly with the hardware components. This layer ensures the efficient allocation and utilization of physical storage resources, contributing to the overall performance and reliability of the file system.

Together, these layers form a cohesive architecture, orchestrating the organized and efficient handling of data in the Linux operating system.

Characteristics of a File System

A file system defines the rules and structures for how data is organized, stored, accessed, and managed on a storage device.

• Space Management: How the data is stored on a storage device. Pertaining to the memory blocks and fragmentation practices applied in it.
• Filename: A file system may have certain restrictions to file names such as the name length, the use of special characters, and case sensitive-ness.
• Directory: The directories/folders may store files in a linear or hierarchical manner while maintaining an index table of all the files contained in that directory or subdirectory.
• Metadata: For each file stored, the file system stores various information about that file's existence such as its data length, its access permissions, device type, modified date-time, and other attributes. This is called metadata.
• Utilities: File systems provide features for initializing, deleting, renaming, moving, copying, backup, recovery, and control access of files and folders.
• Design: Due to their implementations, file systems have limitations on the amount of data they can store.

Some important terms:

Understanding these key terms is essential before exploring various Linux file system implementations for disk storage.

1) Journaling:

Journaling file systems keep a log called the journal, that keeps track of the changes made to a file but not yet permanently committed to the disk so that in case of a system failure the lost changes can be brought back. Journaling works like a checklist:

1. Log changes in the journal.
2. Apply changes to the disk.
3. Mark them as complete.

Journaling can be configured in three different modes, each offering a trade-off between reliability and performance. The Journal mode is the most reliable as it logs both file data and metadata, ensuring the highest level of data integrity. However, it is also the slowest mode due to the extensive logging process. The Ordered mode, on the other hand, logs only the metadata, with the file data being written before the metadata. This provides a balanced approach between data safety and system performance. Lastly, the Writeback mode logs only metadata without enforcing any order between file data and metadata writes. While it is the fastest journaling mode, it is also the least safe, as it increases the risk of data corruption in the event of a crash.

2) Versioning:

Versioning file systems store previously saved versions of a file, i.e., the copies of a file are stored based on previous commits to the disk in a minutely or hourly manner to create a backup.

3) Inode:

The index node is the representation of any file or directory based on the parameters - size, permission, ownership, and location of the file and directory.

Now, we come to part where we discuss the various implementations of the file system in Linux for disk storage devices.

Linux File Systems: 

Here are some linux file systems:

Note: Cluster and distributed file systems will not be included for simplicity.

1) ext (Extended File System): 

Implemented in 1992, it is the first file system specifically designed for Linux. It is the first member of the ext family of file systems.

2) ext2: 

The second ext was developed in 1993. It is a non-journaling file system that became known for its efficient handling of flash drives and SSDs. It solved the problems of separate timestamp for access, inode modification and data modification. Due to not being journaled, it is slow to load at boot time.

3) Xiafs: 

Also developed in 1993, Xiafs was developed as an alternative but lacked the power and functionality of ext2. Due to limited features and scalability, it is no longer in use.

4) ext3: 

Introduced in 1999, ext3 brought in journaling capabilities, offering improved reliability. Unlike ext2, it avoided long boot-time checks after an improper shutdown. It also supported online file system growth and HTree indexing, making it efficient for large directories.

5) JFS (Journaled File System):

First created by IBM in 1990, the original JFS was taken to open source to be implemented for Linux in 199 it is Known for its ability to perform well under varied loads JFS performs well under different kinds of load but is not commonly used anymore due to the release of ext4 in 2006 which gives better performance.

6) ReiserFS: 

It is a journal file system developed in 2001. Despite its earlier issues, it has tail packing as a scheme to reduce internal fragmentation. It uses a B+ Tree that gives less than linear time in directory lookups and updates. It was the default file system in SUSE Linux till version 6.4, until switching to ext3 in 2006 for version 10.2.

7) XFS: 

XFS is a 64-bit journaling file system and was ported to Linux in 2001. It now acts as the default file system for many Linux distributions. It provides features like snapshots, online defragmentation, sparse files, variable block sizes, and excellent capacity. It also excels at parallel I/O operations.

8) SquashFS: 

Developed in 2002, this file system is read-only and is used only with embedded systems where low overhead is needed.

9) Reiser4: 

It is an incremental model to ReiserFS. It was developed in 2004. However, it is not widely adapted or supported on many Linux distributions.

10) ext4: 

The fourth ext developed in 2006, is a journaling file system. It has backward compatibility with ext3 and ext2 and it provides several other features, some of which are persistent pre-allocation, unlimited number of subdirectories, metadata checksumming and large file size. ext4 is the default file system for many Linux distributions and also has compatibility with Windows and Macintosh.

11) btrfs (Better/Butter/B-tree FS): 

It was developed in 2007. It provides many features such as snapshotting, drive pooling, data scrubbing, self-healing and online defragmentation. It is the default file system for Fedora Workstation.

12) bcachefs: 

This is a copy-on-write file system that was first announced in 2015 with the goal of performing better than btrfs and ext4. Its features include full filesystem encryption, native compression, snapshots, and 64-bit check summing.

13) Others:

Linux also has support for file systems of operating systems such as NTFS and exFAT, but these do not support standard Unix permission settings. They are mostly used for interoperability with other operating systems.

File Systems Comparison:

Please note that there are more criteria than the ones listed in the table. This table is supposed to give you an idea of how file systems have evolved.

Observations:

We see that XFS, ext4 and btrfs perform the best of all the other file systems. In fact, btrfs looks as if it's almost the best. Despite that, the ext family of file systems has been the default for most Linux distributions for a long time. So, what is it that made the developers choose ext4 as the default rather than btrfs or XFS? Since ext4 is so important for this discussion, let's describe it a bit more.

ext4 in Linux File System

Ext4 was designed to be backward compatible with ext3 and ext2, its previous generations. It's better than the previous generations in the following ways:

• It provides a large file system as described in the table above.
• Utilizes extents that improve large file performance and reduces fragmentation.
• Provides persistent pre-allocation which guarantees space allocation and contiguous memory.
• Delayed allocation improves performance and reduces fragmentation by effectively allocating larger amounts of data at a time.
• It uses HTree indices to allow unlimited number of subdirectories.
• Performs journal checksumming which allows the file system to realize that some of its entries are invalid or out of order after a crash.
• Support for time-of-creation timestamps and improved timestamps to induce granularity.
• Transparent encryption.
• Allows cleaning of inode tables in background which in turn speeds initialization. The process is called lazy initialization.
• Enables writing barriers by default. Which ensures that file system metadata is correctly written and ordered on disk, even when write caches lose power.

There are still some features in the process of developing like metadata checksumming, first-class quota supports, and large allocation blocks.

However, ext4 has some limitations. Ext4 does not guarantee the integrity of your data, if the data is corrupted while already on disk then it has no way of detecting or repairing such corruption. The ext4 file system cannot secure deletion of files, which is supposed to cause overwriting of files upon deletion. It results in sensitive data ending up in the file-system journal.

XFS performs highly well for large filesystems and high degrees of concurrency. So XFS is stable, yet there's not a solid borderline that would make you choose it over ext4 since both work about the same. Unless you want a file system that directly solves a problem of ext4 like having capacity > 50TiB.

Btrfs on the other hand, despite offering features like multiple device management, per-block checksumming, asynchronous replication and inline compression, does not perform the best in many common use cases as compared to ext4 and XFS. Several of its features can be buggy and result in reduced performance and data loss. 

Some Hands On Example on Linux File System

For example, if our use_case is to set up a server that will first store and serve large multimedia files (videos and audios). In that case we have to prioritize efficient speed and use of storage space. 

According to this requirement the XFS file system would be a better choice. Because we know that XFS is optimized for large files and can work on high volumes of data transfer which in general makes it ideal for media servers.

Following steps to use it:

Step 1: Installing XFS utilities package on Linux system. 

sudo apt-get install xfsprogs

Step 2: Create a partition to format as XFS.

For example:   `/dev/sda1`

This can be done using tool like `fdisk`.

Step 3: Format the partition as XFS.

sudo mkfs.xfs /dev/sda1 -f

We have formatted partition using XFS filesystem. (Used -f for forcefully to avoid error or warning) .

Step 4: Mount the XFS partition to a directory we want.

sudo mount /dev/sda1 /mnt/jayesh_xfs_partition

 We have mounted XFS partition to a directory `/mnt/jayesh_xfs_partition`, (you can create your own directory.)

Step 5: To verify the mount.

df -h

Conclusion:

In this article we discussed Linux file system in operating systems, delving into its layers, characteristics, and the architecture of Linux file systems. It provides a thorough exploration of various options, from ext to contemporary choices like ext4, XFS, and btrfs. The comparison table highlights the superior performance of XFS, ext4, and btrfs, with ext4 standing out for its backward compatibility and design enhancements. The article wisely recommends ext4 as the default for general users unless specific needs dictate alternatives, citing instances where XFS excels for large media files. In essence, the article serves as a practical guide for users to navigate the complexities of file systems, emphasizing the reliable nature of ext4 for most use cases while acknowledging niche applications for other systems.