Topic 7 : File-System Management

Operating System Concepts Essentials – 2nd Edition Silberschatz, Galvin and Gagne ©2013
 Sub-Topic 1 : File-System Interface

 File Concept
 Access Methods
 Disk and Directory Structure
 File-System Mounting
 File Sharing
 Protection

Operating System Concepts Essentials – 2nd Edition 10.2 Silberschatz, Galvin and Gagne ©2013
 File Concept
 Contiguous logical address space
 Types:
 Data
 numeric
 character
 binary
 Program
 Contents defined by file’s creator
 Many types
 Consider text file, source file, executable file

Operating System Concepts Essentials – 2nd Edition 10.3 Silberschatz, Galvin and Gagne ©2013
 File Attributes
 Name – only information kept in human-readable form
 Identifier – unique tag (number) identifies file within file system
 Type – needed for systems that support different types
 Location – pointer to file location on device
 Size – current file size
 Protection – controls who can do reading, writing, executing
 Time, date, and user identification – data for protection, security,
and usage monitoring
 Information about files are kept in the directory structure, which is
maintained on the disk
 Many variations, including extended file attributes such as file
checksum
 Information kept in the directory structure

Operating System Concepts Essentials – 2nd Edition 10.4 Silberschatz, Galvin and Gagne ©2013
 File info Window on Mac OS X

Operating System Concepts Essentials – 2nd Edition 10.5 Silberschatz, Galvin and Gagne ©2013
 File Operations
 File is an abstract data type
 Create
 Write – at write pointer location
 Read – at read pointer location
 Reposition within file - seek
 Delete
 Truncate
 Open(Fi) – search the directory structure on disk for entry Fi,
and move the content of entry to memory
 Close (Fi) – move the content of entry Fi in memory to
directory structure on disk

Operating System Concepts Essentials – 2nd Edition 10.6 Silberschatz, Galvin and Gagne ©2013
 Open Files
 Several pieces of data are needed to manage open files:
 Open-file table: tracks open files
 File pointer: pointer to last read/write location, per
process that has the file open
 File-open count: counter of number of times a file is
open – to allow removal of data from open-file table when
last processes closes it
 Disk location of the file: cache of data access information
 Access rights: per-process access mode information

Operating System Concepts Essentials – 2nd Edition 10.7 Silberschatz, Galvin and Gagne ©2013
 Open File Locking
 Provided by some operating systems and file systems
 Similar to reader-writer locks
 Shared lock similar to reader lock – several processes can
acquire concurrently
 Exclusive lock similar to writer lock
 Mediates access to a file
 Mandatory or advisory:
 Mandatory – access is denied depending on locks held and
requested
 Advisory – processes can find status of locks and decide
what to do

Operating System Concepts Essentials – 2nd Edition 10.8 Silberschatz, Galvin and Gagne ©2013
 File Locking Example – Java API
import java.io.*;
import java.nio.channels.*;
public class LockingExample {
public static final boolean EXCLUSIVE = false;
public static final boolean SHARED = true;
public static void main(String arsg[]) throws IOException {
FileLock sharedLock = null;
FileLock exclusiveLock = null;
try {
RandomAccessFile raf = new RandomAccessFile("file.txt", "rw");
// get the channel for the file
FileChannel ch = raf.getChannel();
// this locks the first half of the file - exclusive
exclusiveLock = ch.lock(0, raf.length()/2, EXCLUSIVE);
/** Now modify the data . . . */
// release the lock
exclusiveLock.release();

Operating System Concepts Essentials – 2nd Edition 10.9 Silberschatz, Galvin and Gagne ©2013
 File Locking Example – Java API (Cont.)

// this locks the second half of the file - shared

sharedLock = ch.lock(raf.length()/2+1, raf.length(),
SHARED);
/** Now read the data . . . */
// release the lock
sharedLock.release();
} catch (java.io.IOException ioe) {
System.err.println(ioe);
}finally {
if (exclusiveLock != null)
exclusiveLock.release();
if (sharedLock != null)
sharedLock.release();
}
}
}

Operating System Concepts Essentials – 2nd Edition 10.10 Silberschatz, Galvin and Gagne ©2013
 File Types – Name, Extension

Operating System Concepts Essentials – 2nd Edition 10.11 Silberschatz, Galvin and Gagne ©2013
 File Structure
 None - sequence of words, bytes
 Simple record structure
 Lines
 Fixed length
 Variable length
 Complex Structures
 Formatted document
 Relocatable load file
 Can simulate last two with first method by inserting
appropriate control characters
 Who decides:
 Operating system
 Program

Operating System Concepts Essentials – 2nd Edition 10.12 Silberschatz, Galvin and Gagne ©2013
 Sequential-access File

Operating System Concepts Essentials – 2nd Edition 10.13 Silberschatz, Galvin and Gagne ©2013
 Access Methods
 Sequential Access
read next
write next
reset
no read after last write
(rewrite)
 Direct Access – file is fixed length logical records
read n
write n
position to n
read next
write next
rewrite n
n = relative block number

 Relative block numbers allow OS to decide where file should be placed

 See allocation problem in Ch 12

Operating System Concepts Essentials – 2nd Edition 10.14 Silberschatz, Galvin and Gagne ©2013
 Simulation of Sequential Access on Direct-access File

Operating System Concepts Essentials – 2nd Edition 10.15 Silberschatz, Galvin and Gagne ©2013
 Other Access Methods

 Can be built on top of base methods

 General involve creation of an index for the file
 Keep index in memory for fast determination of location of
data to be operated on (consider UPC code plus record of
data about that item)
 If too large, index (in memory) of the index (on disk)
 IBM indexed sequential-access method (ISAM)
 Small master index, points to disk blocks of secondary
index
 File kept sorted on a defined key
 All done by the OS
 VMS operating system provides index and relative files as
another example (see next slide)

Operating System Concepts Essentials – 2nd Edition 10.16 Silberschatz, Galvin and Gagne ©2013
 Example of Index and Relative Files

Operating System Concepts Essentials – 2nd Edition 10.17 Silberschatz, Galvin and Gagne ©2013
 Directory Structure

 A collection of nodes containing information about all files

Directory

Files
F1 F2 F4
F3
Fn

Both the directory structure and the files reside on disk

Operating System Concepts Essentials – 2nd Edition 10.18 Silberschatz, Galvin and Gagne ©2013
 Disk Structure
 Disk can be subdivided into partitions
 Disks or partitions can be RAID protected against failure
 Disk or partition can be used raw – without a file system, or
formatted with a file system
 Partitions also known as minidisks, slices
 Entity containing file system known as a volume
 Each volume containing file system also tracks that file system’s
info in device directory or volume table of contents
 As well as general-purpose file systems there are many
special-purpose file systems, frequently all within the same
operating system or computer

Operating System Concepts Essentials – 2nd Edition 10.19 Silberschatz, Galvin and Gagne ©2013
 A Typical File-system Organization

Operating System Concepts Essentials – 2nd Edition 10.20 Silberschatz, Galvin and Gagne ©2013
 Types of File Systems
 We mostly talk of general-purpose file systems
 But systems frequently have may file systems, some general- and
some special- purpose
 Consider Solaris has
 tmpfs – memory-based volatile FS for fast, temporary I/O
 objfs – interface into kernel memory to get kernel symbols for
debugging
 ctfs – contract file system for managing daemons
 lofs – loopback file system allows one FS to be accessed in
place of another
 procfs – kernel interface to process structures
 ufs, zfs – general purpose file systems

Operating System Concepts Essentials – 2nd Edition 10.21 Silberschatz, Galvin and Gagne ©2013
 Operations Performed on Directory
 Search for a file

 Create a file

 Delete a file

 List a directory

 Rename a file

 Traverse the file system

Operating System Concepts Essentials – 2nd Edition 10.22 Silberschatz, Galvin and Gagne ©2013
 Directory Organization

The directory is organized logically to obtain

 Efficiency – locating a file quickly

 Naming – convenient to users
 Two users can have same name for different files
 The same file can have several different names
 Grouping – logical grouping of files by properties, (e.g., all
Java programs, all games, …)

Operating System Concepts Essentials – 2nd Edition 10.23 Silberschatz, Galvin and Gagne ©2013
 Single-Level Directory
 A single directory for all users

 Naming problem
 Grouping problem

Operating System Concepts Essentials – 2nd Edition 10.24 Silberschatz, Galvin and Gagne ©2013
 Two-Level Directory
 Separate directory for each user

 Path name
 Can have the same file name for different user
 Efficient searching
 No grouping capability

Operating System Concepts Essentials – 2nd Edition 10.25 Silberschatz, Galvin and Gagne ©2013
 Tree-Structured Directories

Operating System Concepts Essentials – 2nd Edition 10.26 Silberschatz, Galvin and Gagne ©2013
 Tree-Structured Directories (Cont.)
 Efficient searching

 Grouping Capability

 Current directory (working directory)

 cd /spell/mail/prog
 type list

Operating System Concepts Essentials – 2nd Edition 10.27 Silberschatz, Galvin and Gagne ©2013
 Tree-Structured Directories (Cont)
 Absolute or relative path name
 Creating a new file is done in current directory
 Delete a file
rm <file-name>
 Creating a new subdirectory is done in current directory
mkdir <dir-name>
Example: if in current directory /mail
mkdir count

Deleting “mail”  deleting the entire subtree rooted by “mail”

Operating System Concepts Essentials – 2nd Edition 10.28 Silberschatz, Galvin and Gagne ©2013
 Acyclic-Graph Directories
 Have shared subdirectories and files

Operating System Concepts Essentials – 2nd Edition 10.29 Silberschatz, Galvin and Gagne ©2013
 Acyclic-Graph Directories (Cont.)
 Two different names (aliasing)
 If dict deletes list  dangling pointer
Solutions:
 Backpointers, so we can delete all pointers
Variable size records a problem
 Backpointers using a daisy chain organization
 Entry-hold-count solution
 New directory entry type
 Link – another name (pointer) to an existing file
 Resolve the link – follow pointer to locate the file

Operating System Concepts Essentials – 2nd Edition 10.30 Silberschatz, Galvin and Gagne ©2013
 General Graph Directory

Operating System Concepts Essentials – 2nd Edition 10.31 Silberschatz, Galvin and Gagne ©2013
 General Graph Directory (Cont.)
 How do we guarantee no cycles?
 Allow only links to file not subdirectories
 Garbage collection
 Every time a new link is added use a cycle detection
algorithm to determine whether it is OK

Operating System Concepts Essentials – 2nd Edition 10.32 Silberschatz, Galvin and Gagne ©2013
 File System Mounting
 A file system must be mounted before it can be accessed
 A unmounted file system (i.e., Fig. 11-11(b)) is mounted at a
mount point

Operating System Concepts Essentials – 2nd Edition 10.33 Silberschatz, Galvin and Gagne ©2013
 Mount Point

Operating System Concepts Essentials – 2nd Edition 10.34 Silberschatz, Galvin and Gagne ©2013
 File Sharing
 Sharing of files on multi-user systems is desirable
 Sharing may be done through a protection scheme
 On distributed systems, files may be shared across a network
 Network File System (NFS) is a common distributed file-sharing
method
 If multi-user system
 User IDs identify users, allowing permissions and
protections to be per-user
Group IDs allow users to be in groups, permitting group
access rights
 Owner of a file / directory
 Group of a file / directory

Operating System Concepts Essentials – 2nd Edition 10.35 Silberschatz, Galvin and Gagne ©2013
 File Sharing – Remote File Systems
 Uses networking to allow file system access between systems
 Manually via programs like FTP
 Automatically, seamlessly using distributed file systems
 Semi automatically via the world wide web
 Client-server model allows clients to mount remote file systems from
servers
 Server can serve multiple clients
 Client and user-on-client identification is insecure or complicated
 NFS is standard UNIX client-server file sharing protocol
 CIFS is standard Windows protocol
 Standard operating system file calls are translated into remote calls
 Distributed Information Systems (distributed naming services) such
as LDAP, DNS, NIS, Active Directory implement unified access to
information needed for remote computing

Operating System Concepts Essentials – 2nd Edition 10.36 Silberschatz, Galvin and Gagne ©2013
 File Sharing – Failure Modes

 All file systems have failure modes

 For example corruption of directory structures or other non-
user data, called metadata
 Remote file systems add new failure modes, due to network
failure, server failure
 Recovery from failure can involve state information about
status of each remote request
 Stateless protocols such as NFS v3 include all information in
each request, allowing easy recovery but less security

Operating System Concepts Essentials – 2nd Edition 10.37 Silberschatz, Galvin and Gagne ©2013
 File Sharing – Consistency Semantics
 Specify how multiple users are to access a shared file
simultaneously
 Similar to Ch 5 process synchronization algorithms
 Tend to be less complex due to disk I/O and network
latency (for remote file systems
 Andrew File System (AFS) implemented complex remote file
sharing semantics
 Unix file system (UFS) implements:
 Writes to an open file visible immediately to other users of
the same open file
 Sharing file pointer to allow multiple users to read and write
concurrently
 AFS has session semantics
 Writes only visible to sessions starting after the file is
closed

Operating System Concepts Essentials – 2nd Edition 10.38 Silberschatz, Galvin and Gagne ©2013
 Protection
 File owner/creator should be able to control:
 what can be done
 by whom
 Types of access
 Read
 Write
 Execute
 Append
 Delete
 List

Operating System Concepts Essentials – 2nd Edition 10.39 Silberschatz, Galvin and Gagne ©2013
 Access Lists and Groups
 Mode of access: read, write, execute
 Three classes of users on Unix / Linux
RWX
a) owner access 7  111
RWX
b) group access 6  110
RWX
c) public access 1  001
 Ask manager to create a group (unique name), say G, and add
some users to the group.
 For a particular file (say game) or subdirectory, define an
appropriate access.

Attach a group to a file

chgrp G game

Operating System Concepts Essentials – 2nd Edition 10.40 Silberschatz, Galvin and Gagne ©2013
 Windows 7 Access-Control List Management

Operating System Concepts Essentials – 2nd Edition 10.41 Silberschatz, Galvin and Gagne ©2013
 A Sample UNIX Directory Listing

Operating System Concepts Essentials – 2nd Edition 10.42 Silberschatz, Galvin and Gagne ©2013
 Sub-Topic 2 : File System Implementation
 File-System Structure
 File-System Implementation
 Directory Implementation
 Allocation Methods
 Free-Space Management
 Efficiency and Performance
 Recovery
 NFS
 Example: WAFL File System

Operating System Concepts Essentials – 2nd Edition 10.43 Silberschatz, Galvin and Gagne ©2013
 File-System Structure
 File structure
 Logical storage unit
 Collection of related information
 File system resides on secondary storage (disks)
 Provided user interface to storage, mapping logical to physical
 Provides efficient and convenient access to disk by allowing data
to be stored, located retrieved easily
 Disk provides in-place rewrite and random access
 I/O transfers performed in blocks of sectors (usually 512 bytes)
 File control block – storage structure consisting of information
about a file
 Device driver controls the physical device
 File system organized into layers

Operating System Concepts Essentials – 2nd Edition 10.44 Silberschatz, Galvin and Gagne ©2013
 Layered File System

Operating System Concepts Essentials – 2nd Edition 10.45 Silberschatz, Galvin and Gagne ©2013
 File System Layers
 Device drivers manage I/O devices at the I/O control layer
 Given commands like “read drive1, cylinder 72, track 2, sector
10, into memory location 1060” outputs low-level hardware
specific commands to hardware controller
 Basic file system given command like “retrieve block 123”
translates to device driver
 Also manages memory buffers and caches (allocation, freeing,
replacement)
 Buffers hold data in transit
 Caches hold frequently used data
 File organization module understands files, logical address, and
physical blocks
 Translates logical block # to physical block #
 Manages free space, disk allocation

Operating System Concepts Essentials – 2nd Edition 10.46 Silberschatz, Galvin and Gagne ©2013
 File System Layers (Cont.)
 Logical file system manages metadata information
 Translates file name into file number, file handle, location by
maintaining file control blocks (inodes in UNIX)
 Directory management
 Protection
 Layering useful for reducing complexity and redundancy, but
adds overhead and can decrease performanceTranslates file
name into file number, file handle, location by maintaining file
control blocks (inodes in UNIX)
 Logical layers can be implemented by any coding method
according to OS designer

Operating System Concepts Essentials – 2nd Edition 10.47 Silberschatz, Galvin and Gagne ©2013
 File System Layers (Cont.)
 Many file systems, sometimes many within an operating
system
 Each with its own format (CD-ROM is ISO 9660; Unix has
UFS, FFS; Windows has FAT, FAT32, NTFS as well as
floppy, CD, DVD Blu-ray, Linux has more than 40 types,
with extended file system ext2 and ext3 leading; plus
distributed file systems, etc.)
 New ones still arriving – ZFS, GoogleFS, Oracle ASM,
FUSE

Operating System Concepts Essentials – 2nd Edition 10.48 Silberschatz, Galvin and Gagne ©2013
 File-System Implementation
 We have system calls at the API level, but how do we implement
their functions?
 On-disk and in-memory structures
 Boot control block contains info needed by system to boot OS
from that volume
 Needed if volume contains OS, usually first block of volume
 Volume control block (superblock, master file table) contains
volume details
 Total # of blocks, # of free blocks, block size, free block
pointers or array
 Directory structure organizes the files
 Names and inode numbers, master file table

Operating System Concepts Essentials – 2nd Edition 10.49 Silberschatz, Galvin and Gagne ©2013
 File-System Implementation (Cont.)
 Per-file File Control Block (FCB) contains many details about
the file
 inode number, permissions, size, dates
 NFTS stores into in master file table using relational DB
structures

Operating System Concepts Essentials – 2nd Edition 10.50 Silberschatz, Galvin and Gagne ©2013
 In-Memory File System Structures

 Mount table storing file system mounts, mount points, file

system types
 The following figure illustrates the necessary file system
structures provided by the operating systems
 Figure 12-3(a) refers to opening a file
 Figure 12-3(b) refers to reading a file
 Plus buffers hold data blocks from secondary storage
 Open returns a file handle for subsequent use
 Data from read eventually copied to specified user process
memory address

Operating System Concepts Essentials – 2nd Edition 10.51 Silberschatz, Galvin and Gagne ©2013
 In-Memory File System Structures

Operating System Concepts Essentials – 2nd Edition 10.52 Silberschatz, Galvin and Gagne ©2013
 Partitions and Mounting
 Partition can be a volume containing a file system (“cooked”) or raw
– just a sequence of blocks with no file system
 Boot block can point to boot volume or boot loader set of blocks that
contain enough code to know how to load the kernel from the file
system
 Or a boot management program for multi-os booting
 Root partition contains the OS, other partitions can hold other
Oses, other file systems, or be raw
 Mounted at boot time
 Other partitions can mount automatically or manually
 At mount time, file system consistency checked
 Is all metadata correct?
 If not, fix it, try again
 If yes, add to mount table, allow access

Operating System Concepts Essentials – 2nd Edition 10.53 Silberschatz, Galvin and Gagne ©2013
 Virtual File Systems
 Virtual File Systems (VFS) on Unix provide an object-oriented
way of implementing file systems
 VFS allows the same system call interface (the API) to be used
for different types of file systems
 Separates file-system generic operations from
implementation details
 Implementation can be one of many file systems types, or
network file system
 Implements vnodes which hold inodes or network file
details
 Then dispatches operation to appropriate file system
implementation routines

Operating System Concepts Essentials – 2nd Edition 10.54 Silberschatz, Galvin and Gagne ©2013
 Virtual File Systems (Cont.)
 The API is to the VFS interface, rather than any specific type of
file system

Operating System Concepts Essentials – 2nd Edition 10.55 Silberschatz, Galvin and Gagne ©2013
 Virtual File System Implementation
 For example, Linux has four object types:
 inode, file, superblock, dentry
 VFS defines set of operations on the objects that must be
implemented
 Every object has a pointer to a function table
 Function table has addresses of routines to implement that
function on that object
 For example:
 • int open(. . .)—Open a file
 • int close(. . .)—Close an already-open file
 • ssize t read(. . .)—Read from a file
 • ssize t write(. . .)—Write to a file
 • int mmap(. . .)—Memory-map a file

Operating System Concepts Essentials – 2nd Edition 10.56 Silberschatz, Galvin and Gagne ©2013
 Directory Implementation
 Linear list of file names with pointer to the data blocks
 Simple to program
 Time-consuming to execute
 Linear search time
 Could keep ordered alphabetically via linked list or use
B+ tree
 Hash Table – linear list with hash data structure
 Decreases directory search time
 Collisions – situations where two file names hash to the
same location
 Only good if entries are fixed size, or use chained-overflow
method

Operating System Concepts Essentials – 2nd Edition 10.57 Silberschatz, Galvin and Gagne ©2013
 Allocation Methods - Contiguous

 An allocation method refers to how disk blocks are allocated for

files:
 Contiguous allocation – each file occupies set of contiguous
blocks
 Best performance in most cases
 Simple – only starting location (block #) and length (number
of blocks) are required
 Problems include finding space for file, knowing file size,
external fragmentation, need for compaction off-line
(downtime) or on-line

Operating System Concepts Essentials – 2nd Edition 10.58 Silberschatz, Galvin and Gagne ©2013
 Contiguous Allocation

 Mapping from logical to physical

LA/512

Block to be accessed = Q + starting

address
Displacement into block = R

Operating System Concepts Essentials – 2nd Edition 10.59 Silberschatz, Galvin and Gagne ©2013
 Extent-Based Systems

 Many newer file systems (i.e., Veritas File System) use a

modified contiguous allocation scheme

 Extent-based file systems allocate disk blocks in extents

 An extent is a contiguous block of disks

 Extents are allocated for file allocation
 A file consists of one or more extents

Operating System Concepts Essentials – 2nd Edition 10.60 Silberschatz, Galvin and Gagne ©2013
 Allocation Methods - Linked
 Linked allocation – each file a linked list of blocks
 File ends at nil pointer
 No external fragmentation
 Each block contains pointer to next block
 No compaction, external fragmentation
 Free space management system called when new block
needed
 Improve efficiency by clustering blocks into groups but
increases internal fragmentation
 Reliability can be a problem
 Locating a block can take many I/Os and disk seeks

Operating System Concepts Essentials – 2nd Edition 10.61 Silberschatz, Galvin and Gagne ©2013
 Allocation Methods – Linked (Cont.)
 FAT (File Allocation Table) variation
 Beginning of volume has table, indexed by block number
 Much like a linked list, but faster on disk and cacheable
 New block allocation simple

Operating System Concepts Essentials – 2nd Edition 10.62 Silberschatz, Galvin and Gagne ©2013
 Linked Allocation
 Each file is a linked list of disk blocks: blocks may be scattered
anywhere on the disk
block = pointer

 Mapping
Q
LA/511
R
Block to be accessed is the Qth block in the linked chain of blocks
representing the file.

Displacement into block = R + 1

Operating System Concepts Essentials – 2nd Edition 10.63 Silberschatz, Galvin and Gagne ©2013
 Linked Allocation

Operating System Concepts Essentials – 2nd Edition 10.64 Silberschatz, Galvin and Gagne ©2013
 File-Allocation Table

Operating System Concepts Essentials – 2nd Edition 10.65 Silberschatz, Galvin and Gagne ©2013
 Allocation Methods - Indexed

 Indexed allocation
 Each file has its own index block(s) of pointers to its data blocks

 Logical view

index table

Operating System Concepts Essentials – 2nd Edition 10.66 Silberschatz, Galvin and Gagne ©2013
 Example of Indexed Allocation

Operating System Concepts Essentials – 2nd Edition 10.67 Silberschatz, Galvin and Gagne ©2013
 Indexed Allocation (Cont.)
 Need index table

 Random access

 Dynamic access without external fragmentation, but have overhead

of index block

 Mapping from logical to physical in a file of maximum size of 256K

bytes and block size of 512 bytes. We need only 1 block for index
table
Q
LA/512
R

Q = displacement into index table

R = displacement into block

Operating System Concepts Essentials – 2nd Edition 10.68 Silberschatz, Galvin and Gagne ©2013
 Indexed Allocation – Mapping (Cont.)
 Mapping from logical to physical in a file of unbounded length (block size of
512 words)

 Linked scheme – Link blocks of index table (no limit on size)

Q1
LA / (512 x 511)
R1
Q1 = block of index table
R1 is used as follows:
Q2
R1 / 512
R2

Q2 = displacement into block of index table

R2 displacement into block of file:

Operating System Concepts Essentials – 2nd Edition 10.69 Silberschatz, Galvin and Gagne ©2013
 Indexed Allocation – Mapping (Cont.)
 Two-level index (4K blocks could store 1,024 four-byte pointers in outer index ->
1,048,567 data blocks and file size of up to 4GB)

Q1
LA / (512 x 512)
R1

Q1 = displacement into outer-index

R1 is used as follows:
Q2
R1 / 512
R2

Q2 = displacement into block of index table

R2 displacement into block of file:

Operating System Concepts Essentials – 2nd Edition 10.70 Silberschatz, Galvin and Gagne ©2013
 Indexed Allocation – Mapping (Cont.)

Operating System Concepts Essentials – 2nd Edition 10.71 Silberschatz, Galvin and Gagne ©2013
 Combined Scheme: UNIX UFS
4K bytes per block, 32-bit addresses

More index blocks than can be addressed with 32-bit file pointer

Operating System Concepts Essentials – 2nd Edition 10.72 Silberschatz, Galvin and Gagne ©2013
 Performance
 Best method depends on file access type
 Contiguous great for sequential and random
 Linked good for sequential, not random
 Declare access type at creation -> select either contiguous or
linked
 Indexed more complex
 Single block access could require 2 index block reads then
data block read
 Clustering can help improve throughput, reduce CPU
overhead

Operating System Concepts Essentials – 2nd Edition 10.73 Silberschatz, Galvin and Gagne ©2013
 Performance (Cont.)
 Adding instructions to the execution path to save one disk I/O is
reasonable
 Intel Core i7 Extreme Edition 990x (2011) at 3.46Ghz = 159,000
MIPS
 http://en.wikipedia.org/wiki/Instructions_per_second
 Typical disk drive at 250 I/Os per second
 159,000 MIPS / 250 = 630 million instructions during one disk
I/O
 Fast SSD drives provide 60,000 IOPS
 159,000 MIPS / 60,000 = 2.65 millions instructions during
one disk I/O

Operating System Concepts Essentials – 2nd Edition 10.74 Silberschatz, Galvin and Gagne ©2013
 Free-Space Management
 File system maintains free-space list to track available blocks/clusters
 (Using term “block” for simplicity)
 Bit vector or bit map (n blocks)

0 1 2 n-1
…

1  block[i] free

bit[i] =
0  block[i] occupied

Block number calculation

(number of bits per word) *

(number of 0-value words) +
offset of first 1 bit
CPUs have instructions to return offset within word of first “1” bit

Operating System Concepts Essentials – 2nd Edition 10.75 Silberschatz, Galvin and Gagne ©2013
 Free-Space Management (Cont.)
 Bit map requires extra space
 Example:
block size = 4KB = 212 bytes
disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory

 Easy to get contiguous files

Operating System Concepts Essentials – 2nd Edition 10.76 Silberschatz, Galvin and Gagne ©2013
 Linked Free Space List on Disk
 Linked list (free list)
 Cannot get contiguous
space easily
 No waste of space
 No need to traverse the
entire list (if # free blocks
recorded)

Operating System Concepts Essentials – 2nd Edition 10.77 Silberschatz, Galvin and Gagne ©2013
 Free-Space Management (Cont.)
 Grouping
 Modify linked list to store address of next n-1 free blocks in first
free block, plus a pointer to next block that contains free-block-
pointers (like this one)

 Counting
 Because space is frequently contiguously used and freed, with
contiguous-allocation allocation, extents, or clustering
 Keep address of first free block and count of following free
blocks
 Free space list then has entries containing addresses and
counts

Operating System Concepts Essentials – 2nd Edition 10.78 Silberschatz, Galvin and Gagne ©2013
 Free-Space Management (Cont.)
 Space Maps
 Used in ZFS
 Consider meta-data I/O on very large file systems
 Full data structures like bit maps couldn’t fit in memory ->
thousands of I/Os
 Divides device space into metaslab units and manages metaslabs
 Given volume can contain hundreds of metaslabs
 Each metaslab has associated space map
 Uses counting algorithm
 But records to log file rather than file system
 Log of all block activity, in time order, in counting format
 Metaslab activity -> load space map into memory in balanced-tree
structure, indexed by offset
 Replay log into that structure
 Combine contiguous free blocks into single entry

Operating System Concepts Essentials – 2nd Edition 10.79 Silberschatz, Galvin and Gagne ©2013
 Efficiency and Performance

 Efficiency dependent on:

 Disk allocation and directory algorithms
 Types of data kept in file’s directory entry
 Pre-allocation or as-needed allocation of metadata
structures
 Fixed-size or varying-size data structures

Operating System Concepts Essentials – 2nd Edition 10.80 Silberschatz, Galvin and Gagne ©2013
 Efficiency and Performance (Cont.)
 Performance
 Keeping data and metadata close together
 Buffer cache – separate section of main memory for frequently used
blocks
 Synchronous writes sometimes requested by apps or needed by OS
 No buffering / caching – writes must hit disk before
acknowledgement
 Asynchronous writes more common, buffer-able, faster
 Free-behind and read-ahead – techniques to optimize sequential
access
 Reads frequently slower than writes

Operating System Concepts Essentials – 2nd Edition 10.81 Silberschatz, Galvin and Gagne ©2013
 Page Cache

 A page cache caches pages rather than disk blocks using virtual
memory techniques and addresses

 Memory-mapped I/O uses a page cache

 Routine I/O through the file system uses the buffer (disk) cache

 This leads to the following figure

Operating System Concepts Essentials – 2nd Edition 10.82 Silberschatz, Galvin and Gagne ©2013
 I/O Without a Unified Buffer Cache

Operating System Concepts Essentials – 2nd Edition 10.83 Silberschatz, Galvin and Gagne ©2013
 Unified Buffer Cache
 A unified buffer cache uses the same page cache to cache
both memory-mapped pages and ordinary file system I/O to
avoid double caching

 But which caches get priority, and what replacement algorithms

to use?

Operating System Concepts Essentials – 2nd Edition 10.84 Silberschatz, Galvin and Gagne ©2013
 I/O Using a Unified Buffer Cache

Operating System Concepts Essentials – 2nd Edition 10.85 Silberschatz, Galvin and Gagne ©2013
 Recovery

 Consistency checking – compares data in directory structure

with data blocks on disk, and tries to fix inconsistencies
 Can be slow and sometimes fails

 Use system programs to back up data from disk to another

storage device (magnetic tape, other magnetic disk, optical)

 Recover lost file or disk by restoring data from backup

Operating System Concepts Essentials – 2nd Edition 10.86 Silberschatz, Galvin and Gagne ©2013
 Log Structured File Systems
 Log structured (or journaling) file systems record each metadata
update to the file system as a transaction
 All transactions are written to a log
 A transaction is considered committed once it is written to the
log (sequentially)
 Sometimes to a separate device or section of disk
 However, the file system may not yet be updated
 The transactions in the log are asynchronously written to the file
system structures
 When the file system structures are modified, the transaction is
removed from the log
 If the file system crashes, all remaining transactions in the log must
still be performed
 Faster recovery from crash, removes chance of inconsistency of
metadata

Operating System Concepts Essentials – 2nd Edition 10.87 Silberschatz, Galvin and Gagne ©2013
 The Sun Network File System (NFS)

 An implementation and a specification of a software system

for accessing remote files across LANs (or WANs)

 The implementation is part of the Solaris and SunOS

operating systems running on Sun workstations using an
unreliable datagram protocol (UDP/IP protocol and Ethernet

Operating System Concepts Essentials – 2nd Edition 10.88 Silberschatz, Galvin and Gagne ©2013
 NFS (Cont.)
 Interconnected workstations viewed as a set of independent machines
with independent file systems, which allows sharing among these file
systems in a transparent manner
 A remote directory is mounted over a local file system directory
 The mounted directory looks like an integral subtree of the local
file system, replacing the subtree descending from the local
directory
 Specification of the remote directory for the mount operation is
nontransparent; the host name of the remote directory has to be
provided
 Files in the remote directory can then be accessed in a
transparent manner
 Subject to access-rights accreditation, potentially any file system
(or directory within a file system), can be mounted remotely on top
of any local directory

Operating System Concepts Essentials – 2nd Edition 10.89 Silberschatz, Galvin and Gagne ©2013
 NFS (Cont.)

 NFS is designed to operate in a heterogeneous environment of

different machines, operating systems, and network architectures;
the NFS specifications independent of these media

 This independence is achieved through the use of RPC primitives

built on top of an External Data Representation (XDR) protocol used
between two implementation-independent interfaces

 The NFS specification distinguishes between the services provided

by a mount mechanism and the actual remote-file-access services

Operating System Concepts Essentials – 2nd Edition 10.90 Silberschatz, Galvin and Gagne ©2013
 Three Independent File Systems

Operating System Concepts Essentials – 2nd Edition 10.91 Silberschatz, Galvin and Gagne ©2013
 Mounting in NFS

Mounts Cascading mounts

Operating System Concepts Essentials – 2nd Edition 10.92 Silberschatz, Galvin and Gagne ©2013
 NFS Mount Protocol
 Establishes initial logical connection between server and client
 Mount operation includes name of remote directory to be mounted
and name of server machine storing it
 Mount request is mapped to corresponding RPC and forwarded
to mount server running on server machine
 Export list – specifies local file systems that server exports for
mounting, along with names of machines that are permitted to
mount them
 Following a mount request that conforms to its export list, the
server returns a file handle—a key for further accesses
 File handle – a file-system identifier, and an inode number to
identify the mounted directory within the exported file system
 The mount operation changes only the user’s view and does not
affect the server side

Operating System Concepts Essentials – 2nd Edition 10.93 Silberschatz, Galvin and Gagne ©2013
 NFS Protocol
 Provides a set of remote procedure calls for remote file operations.
The procedures support the following operations:
 searching for a file within a directory
 reading a set of directory entries
 manipulating links and directories
 accessing file attributes
 reading and writing files
 NFS servers are stateless; each request has to provide a full set
of arguments (NFS V4 is just coming available – very different,
stateful)
 Modified data must be committed to the server’s disk before
results are returned to the client (lose advantages of caching)
 The NFS protocol does not provide concurrency-control
mechanisms

Operating System Concepts Essentials – 2nd Edition 10.94 Silberschatz, Galvin and Gagne ©2013
 Three Major Layers of NFS Architecture

 UNIX file-system interface (based on the open, read, write, and

close calls, and file descriptors)

 Virtual File System (VFS) layer – distinguishes local files from

remote ones, and local files are further distinguished according to
their file-system types
 The VFS activates file-system-specific operations to handle
local requests according to their file-system types
 Calls the NFS protocol procedures for remote requests

 NFS service layer – bottom layer of the architecture

 Implements the NFS protocol

Operating System Concepts Essentials – 2nd Edition 10.95 Silberschatz, Galvin and Gagne ©2013
 Schematic View of NFS Architecture

Operating System Concepts Essentials – 2nd Edition 10.96 Silberschatz, Galvin and Gagne ©2013
 NFS Path-Name Translation

 Performed by breaking the path into component names and

performing a separate NFS lookup call for every pair of
component name and directory vnode

 To make lookup faster, a directory name lookup cache on the

client’s side holds the vnodes for remote directory names

Operating System Concepts Essentials – 2nd Edition 10.97 Silberschatz, Galvin and Gagne ©2013
 NFS Remote Operations

 Nearly one-to-one correspondence between regular UNIX system

calls and the NFS protocol RPCs (except opening and closing
files)
 NFS adheres to the remote-service paradigm, but employs
buffering and caching techniques for the sake of performance
 File-blocks cache – when a file is opened, the kernel checks with
the remote server whether to fetch or revalidate the cached
attributes
 Cached file blocks are used only if the corresponding cached
attributes are up to date
 File-attribute cache – the attribute cache is updated whenever new
attributes arrive from the server
 Clients do not free delayed-write blocks until the server confirms
that the data have been written to disk

Operating System Concepts Essentials – 2nd Edition 10.98 Silberschatz, Galvin and Gagne ©2013
 Example: WAFL File System
 Used on Network Appliance “Filers” – distributed file system
appliances
 “Write-anywhere file layout”
 Serves up NFS, CIFS, http, ftp
 Random I/O optimized, write optimized
 NVRAM for write caching
 Similar to Berkeley Fast File System, with extensive
modifications

Operating System Concepts Essentials – 2nd Edition 10.99 Silberschatz, Galvin and Gagne ©2013
 The WAFL File Layout

Operating System Concepts Essentials – 2nd Edition 10.100 Silberschatz, Galvin and Gagne ©2013
 Snapshots in WAFL

Operating System Concepts Essentials – 2nd Edition 10.101 Silberschatz, Galvin and Gagne ©2013