Chapter 11: File System

Implementation

Adapted to COP4610 by Robert van Engelen

 File-System Structure
 File structure concept:
 A logical storage
unit
 A collection of
related information
 File system implements
file structures
 File system resides
on secondary
storage (disks)
 File system structure
is organized into
layers

Layered file system structure

perating System Concepts – 7th Edition, Jan 1, 2005 11.2 Silberschatz, Galvin and Gagne
 File-System Structure (cont’d)
 Logical file system -
manages meta-data about
the file-system structure in
file-control blocks
 File organization module -
maintains information about
files and translates logical
block addresses into
physical block addresses
 Basic file system - issues
generic commands to the
driver, e.g. read drive1,
cylinder 73, track 2, sector
10
 Device drivers - lowest level
of I/O control with interrupt
handlers to transfer data
between memory and disk,
e.g. read block 123

Layered file system structure

perating System Concepts – 7th Edition, Jan 1, 2005 11.3 Silberschatz, Galvin and Gagne
 File-Control Blocks

 The logical file system maintains structures

consisting of information about a file: file-control
block (FCB)

Typical FCB

perating System Concepts – 7th Edition, Jan 1, 2005 11.4 Silberschatz, Galvin and Gagne
 In-Memory File System Structures

Operations
performed
when
opening a file

Operations
performed
when reading
the file after
opening

perating System Concepts – 7th Edition, Jan 1, 2005 11.5 Silberschatz, Galvin and Gagne
 Virtual File Systems
 Virtual File Systems (VFS)
provide an object-oriented
way of implementing file
systems
 VFS allows the same
system call interface
(the API) to be used for
different types of file
systems
 VFS architecture
supports different object
types, e.g. file objects,
directory objects, whole
file system
 The API is to the VFS
interface, e.g:
open()
read()
write()
mmap()

perating System Concepts – 7th Edition, Jan 1, 2005 11.6 Silberschatz, Galvin and Gagne
 Directory Implementation

How to
efficiently
implement
directory
structures?

perating System Concepts – 7th Edition, Jan 1, 2005 11.7 Silberschatz, Galvin and Gagne
 Directory Implementation (cont’d)

 Linear list of file names with pointer to the data

blocks
 Simple to program
 Time-consuming to search for file name in
(long) lists
 Hash Table – linear list with hash data structure
 Decreases directory search time
 Collisions – situations where two file names
hash to the same location
 Enlarge the hash table
 Or use fixed size with overflow chains

perating System Concepts – 7th Edition, Jan 1, 2005 11.8 Silberschatz, Galvin and Gagne
 Allocation Methods

How to
efficiently
allocate data
blocks for
files on disk?

perating System Concepts – 7th Edition, Jan 1, 2005 11.9 Silberschatz, Galvin and Gagne
 Allocation Methods (cont’d)

 Three allocation methods:

1. Contiguous allocation

2. Linked allocation

3. Indexed allocation

perating System Concepts – 7th Edition, Jan 1, 2005 11.10 Silberschatz, Galvin and Gagne
 Contiguous Allocation
 Each file occupies a set of
contiguous blocks on the
disk
 Pros:
 Simple – only starting
location (block #) and
length (number of
blocks) are required
 Random access
 Cons:
 Wasteful of space
(dynamic storage-
allocation problem)
 External
fragmentation: may
need to compact
space
 Files cannot grow

perating System Concepts – 7th Edition, Jan 1, 2005 11.11 Silberschatz, Galvin and Gagne
 Contiguous Allocation (cont’d)

 Mapping from logical address LA to physical

address (B,D) with block number B and
displacement D
 Suppose block size is 512 bytes:
Quotient Q
LA/512
Remainder
R

 B = Q + starting address
 D=R

perating System Concepts – 7th Edition, Jan 1, 2005 11.12 Silberschatz, Galvin and Gagne
 Contiguous Allocation: Extent-Based
Systems

 Some newer file systems (i.e. high-performance

Veritas File System) use a modified contiguous
allocation scheme

 Extent-based file systems allocate disk blocks in

extents

 An extent is a contiguous chunk of blocks (similar to

clusters)
 Extents are allocated when the file grows
 Extents are linked
 A file consists of one or more extents
 Extent size can be set by owner of file

perating System Concepts – 7th Edition, Jan 1, 2005 11.13 Silberschatz, Galvin and Gagne
 Linked Allocation
 Each file is a linked list of
disk blocks
 Blocks may be scattered
anywhere on the disk

block = pointer

 Pros:
 Simple – need only
starting address
 Free-space management
system – no waste of
space
 Cons:
 No efficient random
access
perating System Concepts – 7th Edition, Jan 1, 2005 11.14 Silberschatz, Galvin and Gagne
 Linked Allocation (Cont.)

 Mapping logical address LA to physical address

(B,D) with block number B and displacement D
 Suppose block size is 512 bytes and each block
contains 4 bytes reserved for pointer to next
block:
Quotient Q
LA/508
Remainder R

 B = Qth block in the linked chain of blocks

 D=R+4

perating System Concepts – 7th Edition, Jan 1, 2005 11.15 Silberschatz, Galvin and Gagne
 File-Allocation Table (FAT)
Variation on
linked list
allocation

FAT is
located in
contiguous
space on
disk

Each entry
corresponds
to disk
block
number

Each entry
contains a
pointer to
the next
perating System Concepts – 7th Edition, Jan 1, 2005 11.16 block or 0Galvin and Gagne
Silberschatz,
 Indexed Allocation

 Brings all pointers

together into the index
block
 Pros:
 Efficient random
access
 Dynamic access
without external
fragmentation
 Cons:
 Index table storage
overhead

perating System Concepts – 7th Edition, Jan 1, 2005 11.17 Silberschatz, Galvin and Gagne
 Indexed Allocation (Cont.)
 Mapping from logical address LA to physical address
(B,D)
 Assume block size of 512 bytes
 Need one block for index table with 128 pointers
(assuming pointers of 4 bytes each)
 Files have maximum size of 64K bytes

Q
LA/512
R

 4 x Q = displacement into the index table to obtain B

 D = R = displacement into block

perating System Concepts – 7th Edition, Jan 1, 2005 11.18 Silberschatz, Galvin and Gagne
 Indexed Allocation – Linked
Scheme

 Mapping from logical address LA to physical

address (B,D) in a file of unbounded length
 Linked scheme – link blocks of index table (no
limit on size)
 Assume block size is 512 bytes
Q1 and pointer
takes 4 bytes:LA / (512 x 127)
R1
Q1 = block of index table in linked chain of
index blocks Q2
R1 is used as follows:
R / 512 1
R2

4 x Q2 + 4 = displacement into index table (in Q1)

to find B
R2 = displacement into block

perating System Concepts – 7th Edition, Jan 1, 2005 11.19 Silberschatz, Galvin and Gagne
 Indexed Allocation – Two-level
Index
 Assume block size = 512
 Maximum file size is
128x128x512
Q1
LA / (512 x 128)
R1
  4 x Q = displacement
1
into outer-index table
 R is used as follows:
1
outer-index
Q2
index table file R1 / 512
R2
 4 x Q = displacement
2
into block of index
table
 R displacement into
2
block
perating System Concepts – 7th Edition, Jan 1, 2005 11.20 Silberschatz, Galvin and Gagne
 Combined Scheme: UNIX (4K bytes per
block)

perating System Concepts – 7th Edition, Jan 1, 2005 11.21 Silberschatz, Galvin and Gagne
 Free-Space Management

 Linked list of free

blocks
 Pros:
 No waste of space
 Cons:
 Difficult to allocate
contiguous blocks

 Note: FAT free block

management is
automatic and finding
contiguous blocks is
easy

perating System Concepts – 7th Edition, Jan 1, 2005 11.22 Silberschatz, Galvin and Gagne
 Free-Space Management - Bit
Vector

 Bit vector (for n blocks)

0 1 2 n-1
…

0  block[i] free
bit[i] =
1  block[i] occupied

Block number calculation:

(number of bits per word) x (number of 0-value words) +
offset of first 1 bit

perating System Concepts – 7th Edition, Jan 1, 2005 11.23 Silberschatz, Galvin and Gagne
 Bit Vector (cont’d)

 Pro:
 Easy to find space to allocate contiguous
files

 Cons:
 Bit map requires extra space, which can be
huge
 Example:
block size = 29 bytes
disk size = 232 bytes (4G bytes)
n = 232/29 = 223 bits, requiring 220 bytes
(1M bytes)

perating System Concepts – 7th Edition, Jan 1, 2005 11.24 Silberschatz, Galvin and Gagne
 Efficiency and Performance

 Efficiency dependent on:

 Disk allocation and directory algorithms
 Types of data kept in file’s directory entry

 Performance enhancements:
 Disk cache – separate section of main
memory for frequently used blocks
 Free-behind and read-ahead – techniques to
optimize sequential access
 Improve PC performance by dedicating
section of memory as virtual disk, or RAM
disk

perating System Concepts – 7th Edition, Jan 1, 2005 11.25 Silberschatz, Galvin and Gagne
 Page Cache

 A page cache caches

pages rather than disk
blocks using virtual
memory techniques
 Memory-mapped I/O
uses a page cache
 Routine I/O through the
file system uses the
buffer (disk) cache

I/O without a unified buffer cache

perating System Concepts – 7th Edition, Jan 1, 2005 11.26 Silberschatz, Galvin and Gagne
 Unified Buffer Cache

 A unified buffer cache

uses the same page
cache to cache both
memory-mapped pages
and ordinary file system
I/O

perating System Concepts – 7th Edition, Jan 1, 2005 11.27 Silberschatz, Galvin and Gagne
 Recovery

 Consistency checking – compares data in

directory structure with data blocks on disk, and
tries to fix inconsistencies

 Use system programs to back up data from disk

to another storage device (floppy disk, magnetic
tape, other magnetic disk, optical)

 Recover lost file or disk by restoring data from

backup

perating System Concepts – 7th Edition, Jan 1, 2005 11.28 Silberschatz, Galvin and Gagne
 Log Structured File Systems

 Log structured (or journaling) file systems

record each update to the file system as a
transaction
 All transactions are written to a log
 A transaction is considered committed once
it is written to the log
 However, the file system may not yet be
updated
 The transactions in the log are asynchronously
written to the file system
 When the file system is modified, the
transaction is removed from the log
 If the file system crashes, all remaining
transactions in the log must still be performed

perating System Concepts – 7th Edition, Jan 1, 2005 11.29 Silberschatz, Galvin and Gagne
 The Sun Network File System
(NFS)

 NFS is an implementation and a specification of

a software system for accessing remote files
across LANs (or WANs)

 The implementation is part of the Solaris and

SunOS operating systems running on Sun
workstations using an unreliable datagram
protocol UDP/IP protocol e.g. over Ethernet

 NFS is now widely used

perating System Concepts – 7th Edition, Jan 1, 2005 11.30 Silberschatz, Galvin and Gagne
 NFS Design
 NFS is designed to operate in a heterogeneous
environment of different machines, operating
systems, and network architectures
 Allows sharing among file systems on different
machines in a transparent manner
 NFS specifications are independent of these
media
 Independence is achieved through the use of RPC
(Remote Procedure Calling) primitives built on
top of an External Data Representation (XDR)
protocol used between two implementation-
independent interfaces

 The NFS specification distinguishes between the

services provided by a mount mechanism and the
actual remote-file-access services

perating System Concepts – 7th Edition, Jan 1, 2005 11.31 Silberschatz, Galvin and Gagne
 NFS Protocol
 Provides RPC for remote file operations
 The procedures support the following operations:
 Searching for a file within a directory
 Reading a set of directory entries
 Manipulating links and directories
 Accessing file attributes
 Reading and writing files
 NFS servers are stateless; each request has to
provide a full set of arguments (NFS V4 is stateful)
 Modified data must be committed to the server’s disk
before results are returned to the client (lose
advantages of caching)
 The NFS protocol does not provide concurrency-
control mechanisms

perating System Concepts – 7th Edition, Jan 1, 2005 11.32 Silberschatz, Galvin and Gagne
 NFS Remote Operations
 Nearly one-to-one correspondence between
regular UNIX system calls and the NFS protocol
RPCs (except opening and closing files)
 NFS adheres to the remote-service paradigm, but
employs buffering and caching techniques for the
sake of performance
 File-blocks cache – when a file is opened, the
kernel checks with the remote server whether to
fetch or revalidate the cached attributes
 Cached file blocks are used only if the
corresponding cached attributes are up to date
 File-attribute cache – the attribute cache is
updated whenever new attributes arrive from the
server
 Clients do not free delayed-write blocks until the
server confirms that the data have been written to
disk
perating System Concepts – 7th Edition, Jan 1, 2005 11.33 Silberschatz, Galvin and Gagne
 NFS Mounting

 A remote directory is mounted over a local file

system directory
 The mounted directory looks like an integral
subtree of the local file system, replacing the
subtree descending from the local directory
 Specification of the remote directory for the
mount operation is nontransparent
 The host name of the remote directory has
to be provided
 Subject to access-rights accreditation,
potentially any file system (or directory
within a file system), can be mounted
remotely on top of any local directory

perating System Concepts – 7th Edition, Jan 1, 2005 11.34 Silberschatz, Galvin and Gagne
 Three Independent File Systems -
Mounting

User Server1 Server2

After Cascading mounts
NFS mount

perating System Concepts – 7th Edition, Jan 1, 2005 11.35 Silberschatz, Galvin and Gagne
 NFS Mount Protocol
 Establishes initial logical connection between server and
client
 Mount operation includes name of remote directory to be
mounted and name of server machine storing it
 Mount request is mapped to corresponding RPC and
forwarded to mount server running on server machine
 Export list – specifies local file systems that server
exports for mounting, along with names of machines
that are permitted to mount them
 Following a mount request that conforms to its export list,
the server returns a file handle—a key for further accesses
 File handle – a file-system identifier, and an inode number to
identify the mounted directory within the exported file
system
 The mount operation changes only the user’s view and does
not affect the server side

perating System Concepts – 7th Edition, Jan 1, 2005 11.36 Silberschatz, Galvin and Gagne
 Three Major Layers of NFS
Architecture

 UNIX file-system interface (based on the open, read,

write, and close calls, and file descriptors)

 Virtual File System (VFS) layer – distinguishes local

files from remote ones, and local files are further
distinguished according to their file-system types
 The VFS activates file-system-specific operations
to handle local requests according to their file-
system types
 Calls the NFS protocol procedures for remote
requests

 NFS service layer – bottom layer of the architecture

 Implements the NFS protocol

perating System Concepts – 7th Edition, Jan 1, 2005 11.37 Silberschatz, Galvin and Gagne
 Schematic View of NFS
Architecture

perating System Concepts – 7th Edition, Jan 1, 2005 11.38 Silberschatz, Galvin and Gagne
 NFS Path-Name Translation

 Performed by breaking the path into component

names and performing a separate NFS lookup
call for every pair of component name and
directory vnode

 To make lookup faster, a directory name lookup

cache on the client’s side holds the vnodes for
remote directory names

perating System Concepts – 7th Edition, Jan 1, 2005 11.39 Silberschatz, Galvin and Gagne
End of Chapter 11