Chapter 11:

File System Implementation

Operating System Concepts with Java – 8th Edition 11.1 Silberschatz, Galvin and Gagne ©2009
 Chapter 11: File System Implementation

„ File-System Structure
„ File-System Implementation
„ Directory Implementation
„ Allocation Methods
„ Free-Space Management
„ Efficiency and Performance
„ Recovery
„ NFS
„ Example: WAFL File System

Operating System Concepts with Java – 8th Edition 11.2 Silberschatz, Galvin and Gagne ©2009
 Objectives

„ To describe the details of implementing local file systems and directory

structures

„ To describe the implementation of remote file systems

„ To discuss block allocation and free-block algorithms and trade-offs

Operating System Concepts with Java – 8th Edition 11.3 Silberschatz, Galvin and Gagne ©2009
 File-System Structure

„ File structure
z Logical storage unit
z Collection of related information

„ File system organized into layers

„ File system resides on secondary storage (disks)

z Provides efficient and convenient access to disk by allowing data to
be stored, located retrieved easily

„ File control block – storage structure consisting of information about

a file

„ Device driver controls the physical device

Operating System Concepts with Java – 8th Edition 11.4 Silberschatz, Galvin and Gagne ©2009
 Layered File System

Operating System Concepts with Java – 8th Edition 11.5 Silberschatz, Galvin and Gagne ©2009
 File-System Implementation

„ Boot control block contains info needed by system to boot OS from

that volume

„ Volume control block contains volume details

„ Directory structure organizes the files

„ Per-file File Control Block (FCB) contains many details about the file

Operating System Concepts with Java – 8th Edition 11.6 Silberschatz, Galvin and Gagne ©2009
 A Typical File Control Block

Operating System Concepts with Java – 8th Edition 11.7 Silberschatz, Galvin and Gagne ©2009
 In-Memory File System Structures

„ The following figure illustrates the necessary file system structures

provided by the operating systems.

„ Figure 12-3(a) refers to opening a file.

„ Figure 12-3(b) refers to reading a file.

Operating System Concepts with Java – 8th Edition 11.8 Silberschatz, Galvin and Gagne ©2009
 In-Memory File System Structures

Operating System Concepts with Java – 8th Edition 11.9 Silberschatz, Galvin and Gagne ©2009
 Virtual File Systems

„ Virtual File Systems (VFS) provide an object-oriented way of

implementing file systems.

„ VFS allows the same system call interface (the API) to be used for
different types of file systems.

„ The API is to the VFS interface, rather than any specific type of file
system.

Operating System Concepts with Java – 8th Edition 11.10 Silberschatz, Galvin and Gagne ©2009
 Schematic View of Virtual File System

Operating System Concepts with Java – 8th Edition 11.11 Silberschatz, Galvin and Gagne ©2009
 Directory Implementation

„ Linear list of file names with pointer to the data blocks.

z simple to program
z time-consuming to execute

„ Hash Table – linear list with hash data structure.

z decreases directory search time
z collisions – situations where two file names hash to the same
location
z fixed size

Operating System Concepts with Java – 8th Edition 11.12 Silberschatz, Galvin and Gagne ©2009
 Allocation Methods

„ An allocation method refers to how disk blocks are allocated for files:

„ Contiguous allocation

„ Linked allocation

„ Indexed allocation

Operating System Concepts with Java – 8th Edition 11.13 Silberschatz, Galvin and Gagne ©2009
 Contiguous Allocation

„ Each file occupies a set of contiguous blocks on the disk

„ Simple – only starting location (block #) and length (number of blocks)

are required

„ Random access

„ Wasteful of space (dynamic storage-allocation problem)

„ Files cannot grow

Operating System Concepts with Java – 8th Edition 11.14 Silberschatz, Galvin and Gagne ©2009
 Contiguous Allocation

„ Mapping from logical to physical

LA/512

Block to be accessed = ! + starting address

Displacement into block = R

Operating System Concepts with Java – 8th Edition 11.15 Silberschatz, Galvin and Gagne ©2009
 Contiguous Allocation of Disk Space

Operating System Concepts with Java – 8th Edition 11.16 Silberschatz, Galvin and Gagne ©2009
 Extent-Based Systems

„ Many newer file systems (i.e., Veritas File System) use a modified
contiguous allocation scheme

„ Extent-based file systems allocate disk blocks in extents

„ An extent is a contiguous block of disks

z Extents are allocated for file allocation
z A file consists of one or more extents

Operating System Concepts with Java – 8th Edition 11.17 Silberschatz, Galvin and Gagne ©2009
 Linked Allocation

„ Each file is a linked list of disk blocks: blocks may be scattered

anywhere on the disk.

block = pointer

Operating System Concepts with Java – 8th Edition 11.18 Silberschatz, Galvin and Gagne ©2009
 Linked Allocation (Cont.)

„ Simple – need only starting address

„ Free-space management system – no waste of space
„ No random access
„ Mapping

Q
LA/511
R

Block to be accessed is the Qth block in the linked chain of

blocks representing the file.
Displacement into block = R + 1
„ File-allocation table (FAT) – disk-space allocation used by MS-
DOS and OS/2.

Operating System Concepts with Java – 8th Edition 11.19 Silberschatz, Galvin and Gagne ©2009
 Linked Allocation

Operating System Concepts with Java – 8th Edition 11.20 Silberschatz, Galvin and Gagne ©2009
 File-Allocation Table

Operating System Concepts with Java – 8th Edition 11.21 Silberschatz, Galvin and Gagne ©2009
 Indexed Allocation

„ Brings all pointers together into the index block

„ Logical view

index table

Operating System Concepts with Java – 8th Edition 11.22 Silberschatz, Galvin and Gagne ©2009
 Example of Indexed Allocation

Operating System Concepts with Java – 8th Edition 11.23 Silberschatz, Galvin and Gagne ©2009
 Indexed Allocation (Cont.)
„ Need index table

„ Random access

„ Dynamic access without external fragmentation, but have overhead of

index block

„ Mapping from logical to physical in a file of maximum size of 256K

words and block size of 512 words. We need only 1 block for index
table
Q
LA/512
R

Q = displacement into index table

R = displacement into block

Operating System Concepts with Java – 8th Edition 11.24 Silberschatz, Galvin and Gagne ©2009
 Indexed Allocation – Mapping (Cont.)
„ Mapping from logical to physical in a file of unbounded length
(block size of 512 words)

„ Linked scheme – Link blocks of index table (no limit on size)

Q1
LA / (512 x 511)
R1
Q1 = block of index table
R1 is used as follows:
Q2
R1 / 512
R2

Q2 = displacement into block of index table

R2 displacement into block of file:

Operating System Concepts with Java – 8th Edition 11.25 Silberschatz, Galvin and Gagne ©2009
 Indexed Allocation – Mapping (Cont.)

„ Two-level index (maximum file size is 5123)

Q1
LA / (512 x 512)
R1

Q1 = displacement into outer-index

R1 is used as follows:
Q2
R1 / 512
R2

Q2 = displacement into block of index table

R2 displacement into block of file:

Operating System Concepts with Java – 8th Edition 11.26 Silberschatz, Galvin and Gagne ©2009
 Indexed Allocation – Mapping (Cont.)

outer-index

index table file

Operating System Concepts with Java – 8th Edition 11.27 Silberschatz, Galvin and Gagne ©2009
 Combined Scheme:
UNIX UFS (4K bytes per block)

Operating System Concepts with Java – 8th Edition 11.28 Silberschatz, Galvin and Gagne ©2009
 Free-Space Management

„ Bit vector (n blocks)

0 1 2 n-1
…

678
0 ⇒ block[i] free
bit[i] =
1 ⇒ block[i] occupied

Block number calculation

(number of bits per word) *

(number of 0-value words) +
offset of first 1 bit

Operating System Concepts with Java – 8th Edition 11.29 Silberschatz, Galvin and Gagne ©2009
 Free-Space Management (Cont.)

„ Bit map requires extra space

z Example:
block size = 212 bytes
disk size = 230 bytes (1 gigabyte)
n = 230/212 = 218 bits (or 32K bytes)
„ Easy to get contiguous files

„ Linked list (free list)

z Cannot get contiguous space easily
z No waste of space
„ Grouping

„ Counting

Operating System Concepts with Java – 8th Edition 11.30 Silberschatz, Galvin and Gagne ©2009
 Free-Space Management (Cont.)
„ Need to protect:
z Pointer to free list
z Bit map
 Must be kept on disk
 Copy in memory and disk may differ
 Cannot allow for block[i] to have a situation where bit[i] =
1 in memory and bit[i] = 0 on disk
z Solution:
 Set bit[i] = 1 in disk
 Allocate block[i]
 Set bit[i] = 1 in memory

Operating System Concepts with Java – 8th Edition 11.31 Silberschatz, Galvin and Gagne ©2009
 Directory Implementation

„ Linear list of file names with pointer to the data blocks

z simple to program
z time-consuming to execute

„ Hash Table – linear list with hash data structure

z decreases directory search time
z collisions – situations where two file names hash to the same
location
z fixed size

Operating System Concepts with Java – 8th Edition 11.32 Silberschatz, Galvin and Gagne ©2009
 Linked Free Space List on Disk

Operating System Concepts with Java – 8th Edition 11.33 Silberschatz, Galvin and Gagne ©2009
 Efficiency and Performance

„ Efficiency dependent on:

z disk allocation and directory algorithms
z types of data kept in file’s directory entry

„ Performance
z disk cache – separate section of main memory for frequently used
blocks
z free-behind and read-ahead – techniques to optimize sequential
access
z improve PC performance by dedicating section of memory as
virtual disk, or RAM disk

Operating System Concepts with Java – 8th Edition 11.34 Silberschatz, Galvin and Gagne ©2009
 Page Cache

„ A page cache caches pages rather than disk blocks using virtual
memory techniques

„ Memory-mapped I/O uses a page cache

„ Routine I/O through the file system uses the buffer (disk) cache

„ This leads to the following figure

Operating System Concepts with Java – 8th Edition 11.35 Silberschatz, Galvin and Gagne ©2009
 I/O Without a Unified Buffer Cache

Operating System Concepts with Java – 8th Edition 11.36 Silberschatz, Galvin and Gagne ©2009
 Unified Buffer Cache

„ A unified buffer cache uses the same page cache to cache both
memory-mapped pages and ordinary file system I/O

Operating System Concepts with Java – 8th Edition 11.37 Silberschatz, Galvin and Gagne ©2009
 I/O Using a Unified Buffer Cache

Operating System Concepts with Java – 8th Edition 11.38 Silberschatz, Galvin and Gagne ©2009
 Recovery

„ Consistency checking – compares data in directory structure with

data blocks on disk, and tries to fix inconsistencies

„ Use system programs to back up data from disk to another storage

device (magnetic tape, other magnetic disk, optical)

„ Recover lost file or disk by restoring data from backup

Operating System Concepts with Java – 8th Edition 11.39 Silberschatz, Galvin and Gagne ©2009
 Log Structured File Systems

„ Log structured (or journaling) file systems record each update to

the file system as a transaction.

„ All transactions are written to a log.

z A transaction is considered committed once it is written to the log.
z However, the file system may not yet be updated.

„ The transactions in the log are asynchronously written to the file

system.
z When the file system is modified, the transaction is removed from
the log.

„ If the file system crashes, all remaining transactions in the log must
still be performed.

Operating System Concepts with Java – 8th Edition 11.40 Silberschatz, Galvin and Gagne ©2009
 The Sun Network File System (NFS)

„ An implementation and a specification of a software system for

accessing remote files across LANs (or WANs).

„ The implementation is part of the Solaris and SunOS operating

systems running on Sun workstations using an unreliable datagram
protocol (UDP/IP protocol and Ethernet.

Operating System Concepts with Java – 8th Edition 11.41 Silberschatz, Galvin and Gagne ©2009
 NFS (Cont.)
„ Interconnected workstations viewed as a set of independent machines
with independent file systems, which allows sharing among these file
systems in a transparent manner.
z A remote directory is mounted over a local file system directory.
 The mounted directory looks like an integral subtree of the
local file system, replacing the subtree descending from the
local directory.
z Specification of the remote directory for the mount operation is
nontransparent; the host name of the remote directory has to be
provided.
 Files in the remote directory can then be accessed in a
transparent manner.
z Subject to access-rights accreditation, potentially any file system
(or directory within a file system), can be mounted remotely on top
of any local directory.

Operating System Concepts with Java – 8th Edition 11.42 Silberschatz, Galvin and Gagne ©2009
 NFS (Cont.)

„ NFS is designed to operate in a heterogeneous environment of

different machines, operating systems, and network architectures; the
NFS specifications independent of these media.

„ This independence is achieved through the use of RPC primitives built

on top of an External Data Representation (XDR) protocol used
between two implementation-independent interfaces.

„ The NFS specification distinguishes between the services provided by

a mount mechanism and the actual remote-file-access services.

Operating System Concepts with Java – 8th Edition 11.43 Silberschatz, Galvin and Gagne ©2009
 Three Independent File Systems

Operating System Concepts with Java – 8th Edition 11.44 Silberschatz, Galvin and Gagne ©2009
 Mounting in NFS

Mounts Cascading mounts

Operating System Concepts with Java – 8th Edition 11.45 Silberschatz, Galvin and Gagne ©2009
 NFS Mount Protocol

„ Establishes initial logical connection between server and client

„ Mount operation includes name of remote directory to be mounted and
name of server machine storing it
z Mount request is mapped to corresponding RPC and forwarded to
mount server running on server machine
z Export list – specifies local file systems that server exports for
mounting, along with names of machines that are permitted to
mount them
„ Following a mount request that conforms to its export list, the server
returns a file handle—a key for further accesses
„ File handle – a file-system identifier, and an inode number to identify
the mounted directory within the exported file system
„ The mount operation changes only the user’s view and does not affect
the server side

Operating System Concepts with Java – 8th Edition 11.46 Silberschatz, Galvin and Gagne ©2009
 NFS Protocol
„ Provides a set of remote procedure calls for remote file operations.
The procedures support the following operations:
z searching for a file within a directory
z reading a set of directory entries
z manipulating links and directories
z accessing file attributes
z reading and writing files

„ NFS servers are stateless; each request has to provide a full set of
arguments
(NFS V4 is just coming available – very different, stateful)

„ Modified data must be committed to the server’s disk before results are
returned to the client (lose advantages of caching)

„ The NFS protocol does not provide concurrency-control mechanisms

Operating System Concepts with Java – 8th Edition 11.47 Silberschatz, Galvin and Gagne ©2009
 Three Major Layers of NFS Architecture

„ UNIX file-system interface (based on the open, read, write, and close
calls, and file descriptors)

„ Virtual File System (VFS) layer – distinguishes local files from remote
ones, and local files are further distinguished according to their file-
system types
z The VFS activates file-system-specific operations to handle local
requests according to their file-system types
z Calls the NFS protocol procedures for remote requests

„ NFS service layer – bottom layer of the architecture

z Implements the NFS protocol

Operating System Concepts with Java – 8th Edition 11.48 Silberschatz, Galvin and Gagne ©2009
 Schematic View of NFS Architecture

Operating System Concepts with Java – 8th Edition 11.49 Silberschatz, Galvin and Gagne ©2009
 NFS Path-Name Translation

„ Performed by breaking the path into component names and performing

a separate NFS lookup call for every pair of component name and
directory vnode

„ To make lookup faster, a directory name lookup cache on the client’s

side holds the vnodes for remote directory names

Operating System Concepts with Java – 8th Edition 11.50 Silberschatz, Galvin and Gagne ©2009
 NFS Remote Operations

„ Nearly one-to-one correspondence between regular UNIX system calls

and the NFS protocol RPCs (except opening and closing files)

„ NFS adheres to the remote-service paradigm, but employs buffering

and caching techniques for the sake of performance

„ File-blocks cache – when a file is opened, the kernel checks with the
remote server whether to fetch or revalidate the cached attributes
z Cached file blocks are used only if the corresponding cached
attributes are up to date

„ File-attribute cache – the attribute cache is updated whenever new

attributes arrive from the server

„ Clients do not free delayed-write blocks until the server confirms that
the data have been written to disk

Operating System Concepts with Java – 8th Edition 11.51 Silberschatz, Galvin and Gagne ©2009
 Example: WAFL File System

„ Used on Network Appliance “Filers” – distributed file system appliances

„ “Write-anywhere file layout”

„ Serves up NFS, CIFS, http, ftp

„ Random I/O optimized, write optimized

z NVRAM for write caching

„ Similar to Berkeley Fast File System, with extensive modifications

Operating System Concepts with Java – 8th Edition 11.52 Silberschatz, Galvin and Gagne ©2009
 The WAFL File Layout

Operating System Concepts with Java – 8th Edition 11.53 Silberschatz, Galvin and Gagne ©2009
 Snapshots in WAFL

Operating System Concepts with Java – 8th Edition 11.54 Silberschatz, Galvin and Gagne ©2009
 End of Chapter 11

Operating System Concepts with Java – 8th Edition 11.55 Silberschatz, Galvin and Gagne ©2009