Chapter 11: File System

Implementation

Operating System Concepts – 10h Silberschatz, Galvin and Gagne

 Outline
▪ File-System Structure
▪ File-System Operations
▪ Directory Implementation
▪ Allocation Methods
▪ Free-Space Management
▪ Efficiency and Performance
▪ Recovery

Operating System Concepts – 10th 14.2 Silberschatz, Galvin and Gagne

 Objectives
▪ Describe the details of implementing local file systems and
directory structures
▪ Discuss block allocation and free-block algorithms and trade-
offs
▪ Explore file system efficiency and performance issues
▪ Look at recovery from file system failures

Operating System Concepts – 10th 14.3 Silberschatz, Galvin and Gagne

 File-System Structure
▪ File structure
• Logical storage unit
• Collection of related information
▪ File system resides on secondary storage (disks)
• Provided user interface to storage, mapping logical to
physical
• Provides efficient and convenient access to disk by
allowing data to be stored, located retrieved easily
▪ Disk provides in-place rewrite and random access
• I/O transfers performed in blocks of sectors (usually 512
bytes)
▪ File control block (FCB) – storage structure consisting of
information about a file
▪ Device driver controls the physical device
▪ File system organized into layers

Operating System Concepts – 10th 14.4 Silberschatz, Galvin and Gagne

 Layered File System

Operating System Concepts – 10th 14.5 Silberschatz, Galvin and Gagne

 File System Layers
▪ Device drivers manage I/O devices at the I/O control layer
Given commands like
read drive1, cylinder 72, track 2, sector 10, into memory
location 1060
Outputs low-level hardware specific commands to hardware
controller
▪ Basic file system given command like “retrieve block 123”
translates to device driver
▪ Also manages memory buffers and caches (allocation, freeing,
replacement)
• Buffers hold data in transit
• Caches hold frequently used data
▪ File organization module understands files, logical address, and
physical blocks
▪ Translates logical block # to physical block #
▪ Manages free space, disk allocation

Operating System Concepts – 10th 14.6 Silberschatz, Galvin and Gagne

 File System Layers (Cont.)
▪ Logical file system manages metadata information
• Translates file name into file number, file handle,
location by maintaining file control blocks (inodes in
UNIX)
• Directory management
• Protection
▪ Layering useful for reducing complexity and
redundancy, but adds overhead and can decrease
performance
▪ Logical layers can be implemented by any coding
method according to OS designer

Operating System Concepts – 10th 14.7 Silberschatz, Galvin and Gagne

 File System Layers (Cont.)
▪ Many file systems, sometimes many within an operating system
• Each with its own format:
• CD-ROM is ISO 9660;
• Unix has UFS, FFS;
• Windows has FAT, FAT32, NTFS as well as floppy, CD, DVD
Blu-ray,
• Linux has more than 130 types, with extended file system
ext3 and ext4 leading; plus distributed file systems, etc.)
• New ones still arriving – ZFS, GoogleFS, Oracle ASM, FUSE

Operating System Concepts – 10th 14.8 Silberschatz, Galvin and Gagne

 File-System Operations
▪ We have system calls at the API level, but how do we
implement their functions?
• On-disk and in-memory structures
▪ Boot control block contains info needed by system to boot OS
from that volume
• Needed if volume contains OS, usually first block of volume
▪ Volume control block (superblock, master file table) contains
volume details
• Total # of blocks, # of free blocks, block size, free block
pointers or array
▪ Directory structure organizes the files
• Names and inode numbers, master file table

Operating System Concepts – 10th 14.9 Silberschatz, Galvin and Gagne

 File Control Block (FCB)
▪ OS maintains FCB per file, which contains many
details about the file
• Typically, inode number, permissions, size, dates
• Example

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 In-Memory File System Structures
▪ Mount table storing file system mounts, mount points,
file system types
▪ System-wide open-file table contains a copy of the FCB of
each file and other info
▪ Per-process open-file table contains pointers to
appropriate entries in system-wide open-file table as
well as other info

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 In-Memory File System Structures (Cont.)

• Figure 12-3(a) refers to opening a file

• Figure 12-3(b) refers to reading a file

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Directory Implementation
▪ Linear list of file names with pointer to the data blocks
• Simple to program
• Time-consuming to execute
4 Linear search time
4 Could keep ordered alphabetically via linked list
or use B+ tree
▪ Hash Table – linear list with hash data structure
• Decreases directory search time
• Collisions – situations where two file names hash to
the same location
• Only good if entries are fixed size, or use chained-
overflow method

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Allocation Method
▪ An allocation method refers to how disk blocks are
allocated for files:
• Contiguous
• Linked
• Indexed

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Contiguous Allocation Method
▪ An allocation method refers to how disk blocks are
allocated for files:
▪ Each file occupies set of contiguous blocks
• Best performance in most cases
• Simple – only starting location (block #) and
length (number of blocks) are required
• Problems include:
4 Finding space on the disk for a file,
4 Knowing file size,
4 External fragmentation, need for compaction
off-line (downtime) or on-line

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Contiguous Allocation (Cont.)
▪ Mapping from logical to
physical (block size =512
bytes)
Q

LA/512

▪ Block to be accessed = starting

address + Q
▪ Displacement into block = R

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Extent-Based Systems
▪ Many newer file systems (i.e., Veritas File System) use a
modified contiguous allocation scheme
▪ Extent-based file systems allocate disk blocks in extents
▪ An extent is a contiguous block of disks
• Extents are allocated for file allocation
• A file consists of one or more extents

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Linked Allocation
▪ Each file is a linked list of blocks
▪ File ends at nil pointer
▪ No external fragmentation
▪ Each block contains pointer to next block
▪ No compaction, external fragmentation
▪ Free space management system called when new block
needed
▪ Improve efficiency by clustering blocks into groups but
increases internal fragmentation
▪ Reliability can be a problem
▪ Locating a block can take many I/Os and disk seeks

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Linked Allocation Example
▪ Each file is a linked list of disk blocks: blocks may be
scattered anywhere on the disk
▪ Scheme

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Linked Allocation (Cont.)
▪ Mapping

Q
LA/
511
R
▪ Block to be accessed is the Qth block in the linked
chain of blocks representing the file.
▪ Displacement into block = R + 1

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 FAT Allocation Method
▪ Beginning of volume has table, indexed by block number
▪ Much like a linked list, but faster on disk and cacheable
▪ New block allocation simple

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 File-Allocation Table

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation Method
▪ Each file has its own index block(s) of pointers to its data blocks
▪ Logical view

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Example of Indexed Allocation

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation – Small Files
▪ Need index table
▪ Random access
▪ Dynamic access without external fragmentation, but have
overhead of index block
▪ Mapping from logical to physical in a file of maximum size
of 256K bytes and block size of 512 bytes. We need only 1
block for index table
Q

LA/512

R
▪ Calculation:
• Q = displacement into index table
• R = displacement into block

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation – Large Files
▪ Mapping from logical to physical in a file of
unbounded length (block size of 512 words)
• Linked scheme – Link blocks of index table (no
limit on size)
• Multi-level indexing
• Combined Scheme

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 ▪ Linked Scheme: An index block is normally one disk block. Thus, it can be read and written
directly by itself. To allow for large files, we may link together several index blocks.
▪ Multilevel index: A variant of the linked representation is to use a first level index block
to point to a set of second – level index blocks.
▪ Combined scheme:
▪ Another alternative, used in the UFS, is to keep the first, say, 15 pointers of the index block in
the file’s inode.
▪ The first 12 of these pointers point to direct blocks; that is for small ( no more than 12 blocks)
files do not need a separate index block
▪ The next pointer is the address of a single indirect block.
▪ The single indirect block is an index block, containing not data, but rather the addresses of
blocks that do contain data.
▪ Then there is a double indirect block pointer, which contains the address of a block that
contain pointers to the actual data blocks. The last pointer would contain pointers to the actual
data blocks.
▪ The last pointer would contain the address of a triple indirect block.

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation – Linked Scheme
▪ Link blocks of index table (no limit on size)
Q1
LA / (512 x
511)
R1
▪ Outer-level mapping scheme
• Q1 = block of index table
• R1 is used as follows
Q2
R1 /
512
R2
• Inner-level mapping scheme
4 Q2 = displacement into block of index table
4 R2 displacement into block of file

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation – Two-level Scheme
▪ Two-level index (4K blocks could store 1,024 four-byte pointers
in outer index -> 1,048,567 data blocks and file size of up to
4GB)
Q1
LA / (512 x
512)
R1

▪ Mapping scheme for outer-index:

• Q1 = displacement into outer-index
• R1 is used as follows:
Q2
R1 /
512 R2

• Mapping scheme for index level:

4 Q2 = displacement into block of index table
4 R2 displacement into block of file

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Indexed Allocation – Two-Level Scheme

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Combined Scheme : UNIX UFS
▪ 4K bytes per block, 32-bit addresses

▪ More index blocks than can be addressed with 32-bit file pointer

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Performance
▪ Best method depends on file access type
• Contiguous great for sequential and random
▪ Linked good for sequential, not random
▪ Declare access type at creation
• Select either contiguous or linked
▪ Indexed more complex
• Single block access could require 2 index block reads then
data block read
• Clustering can help improve throughput, reduce CPU
overhead
▪ For NVM, no disk head so different algorithms and optimizations
needed
• Using old algorithm uses many CPU cycles trying to avoid
non-existent head movement
• Goal is to reduce CPU cycles and overall path needed for I/O

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Free-Space Management
▪ File system maintains free-space list to track available
blocks/clusters
• (Using term “block” for simplicity)
▪ Bit vector or bit map (n blocks) n
01 2 -
1⇒ 1
…
block[i]
bit free

[i]
= 0 ⇒
block[i]
occupied

Block number calculation

(number of bits per
word) *
(number of 0-value
words) +
offset of first
CPUs have instructions 1 bit offset within word of first
to return
“1” bit
Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne
 Free-Space Management
▪ File system maintains free-space list to track available blocks
▪ Bit vector or bit map (n blocks)
n
01 21 ⇒ -
bi … 1
block[i]
t[ free



i] 0 ⇒
= extra
▪ Bit map requires block[i]
space
• Example: occupied
block size = 4KB = 212 bytes
disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory

▪ Easy to get contiguous files

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Linked Free Space List on Disk
▪ Linked list (free list)
• Cannot get contiguous
space easily
• No waste. Linked Free
Space List on Disk of
space
• No need to traverse the
entire list (if # free
blocks recorded)

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Free-Space Management (Cont.)
▪ Grouping
• Modify linked list to store address of next n-1 free
blocks in first free block, plus a pointer to next block
that contains free-block-pointers (like this one)

▪ Counting
• Because space is frequently contiguously used and
freed, with contiguous-allocation allocation, extents,
or clustering
4 Keep address of first free block and count of
following free blocks
4 Free space list then has entries containing
addresses and counts

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Efficiency and Performance
▪ Efficiency dependent on:
• Disk allocation and directory algorithms
• Types of data kept in file’s directory entry
• Pre-allocation or as-needed allocation of metadata
structures
• Fixed-size or varying-size data structures

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Efficiency and Performance (Cont.)
▪ Performance
• Keeping data and metadata close together
• Buffer cache – separate section of main memory for
frequently used blocks

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Page Cache
▪ A page cache caches pages rather than disk blocks
using virtual memory techniques and addresses
▪ Memory-mapped I/O uses a page cache
▪ Routine I/O through the file system uses the buffer
(disk) cache
▪ This leads to the following figure

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 I/O Without a Unified Buffer Cache

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Unified Buffer Cache
▪ A unified buffer cache uses the same page cache to
cache both memory-mapped pages and ordinary file
system I/O to avoid double caching
▪ But which caches get priority, and what replacement
algorithms to use?
▪ LRU is used..

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 I/O Using a Unified Buffer Cache

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Another issue that can effect the performance of I/O is whether writes to
the file system occur synchronous or asynchronously.
• Synchronous writes sometimes requested by apps or needed by
OS
4 No buffering / caching – writes must hit disk before
acknowledgement
4 Asynchronous writes more common, buffer-able, faster
• Free-behind and read-ahead – techniques to optimize sequential
access
– Free-behind - removes block from buffer as soon as next
block is requested.
– Read-ahead - request block and several subsequent blocks
are read and cached.
• Reads frequently slower than writes

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Recovery
▪ Consistency checking – compares data in directory
structure with data blocks on disk, and tries to fix
inconsistencies
• Can be slow and sometimes fails
▪ Use system programs to back up data from disk to
another storage device (magnetic tape, other magnetic
disk, optical)
▪ Recover lost file or disk by restoring data from backup

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 Log Structured File Systems
▪ Log structured (or journaling) file systems record each metadata
update to the file system as a transaction
▪ All transactions are written to a log
• A transaction is considered committed once it is written to
the log (sequentially)
• Sometimes to a separate device or section of disk
• However, the file system may not yet be updated
▪ The transactions in the log are asynchronously written to the file
system structures
• When the file system structures are modified, the transaction
is removed from the log
▪ If the file system crashes, all remaining transactions in the log
must still be performed
▪ Faster recovery from crash, removes chance of inconsistency of
metadata

Operating System Concepts – 10th 14. Silberschatz, Galvin and Gagne

 End of Chapter 11

Operating System Concepts – 10h Silberschatz, Galvin and Gagne