Unit: 5 (I/0 Systems)

Topics

• Mass Storage Structures- Overview

• Disk Scheduling and Management;
• File System Storage-File Concepts,
• Directory and Disk Structure,
• Sharing and Protection;
• File System Implementation- File System Structure,
• Directory Structure
• Allocation Methods
• Free Space Management
 Mass Storage Structures- Overview

Magnetic Disks
• Traditional magnetic disks have the following basic structure:
• One or more platters in the form of disks covered with magnetic media. 
• Hard disk platters are made of rigid metal, while "floppy" disks are made of more flexible plastic.
• Each platter has two working surfaces. Older hard disk drives would sometimes not use the very top or
bottom surface of a stack of platters, as these surfaces were more susceptible to potential damage.
• Each working surface is divided into a number of concentric rings called tracks. 
• The collection of all tracks that are the same distance from the edge of the platter, ( i.e. all tracks
immediately above one another in the following diagram ) is called a cylinder.
• Each track is further divided into sectors, traditionally containing 512 bytes of data each, although some
modern disks occasionally use larger sector sizes.
• The data on a hard drive is read by read-write heads. 
• The storage capacity of a traditional disk drive is equal to the number of heads ( i.e. the number of working
surfaces ), times the number of tracks per surface, times the number of sectors per track, times the
number of bytes per sector.
• A particular physical block of data is specified by providing the head-sector-cylinder number at which it is
located.
Secondary Storage
Disc Structure

Header Data CRC

Sector
 Mass Storage Structures- Overview

Magnetic Disks
• Disk heads "fly" over the surface on a very thin cushion of air. If they should accidentally contact the disk,
then a head crash occurs, which may or may not permanently damage the disk or even destroy it
completely. For this reason it is normal to park the disk heads when turning a computer off, which means
to move the heads off the disk or to an area of the disk where there is no data stored.
• Floppy disks are normally removable. Hard drives can also be removable, and some are even hot-
swappable, meaning they can be removed while the computer is running, and a new hard drive inserted
in their place.
• Disk drives are connected to the computer via a cable known as the I/O Bus. Some of the common
interface formats include Enhanced Integrated Drive Electronics, EIDE; Advanced Technology Attachment,
ATA; Serial ATA, SATA, Universal Serial Bus, USB; Fiber Channel, FC, and Small Computer Systems Interface,
SCSI.
• The host controller is at the computer end of the I/O bus, and the disk controller is built into the disk
itself. The CPU issues commands to the host controller via I/O ports. Data is transferred between the
magnetic surface and onboard cache by the disk controller, and then the data is transferred from that
cache to the host controller and the motherboard memory at electronic speeds.
 Mass Storage Structures- Overview

Magnetic Tape
• Magnetic tapes were once used for common secondary storage before the days of hard disk drives, but today
are used primarily for backups.
• Accessing a particular spot on a magnetic tape can be slow, but once reading or writing commences, access
speeds are comparable to disk drives.
• Capacities of tape drives can range from 20 to 200 GB, and compression can double that capacity.
 Mass Storage Structures- Overview

Solid State Disc

• As technologies improve and economics change, old technologies are often used in different ways. One
example of this is the increasing used of solid state disks, or SSDs.
• SSDs use memory technology as a small fast hard disk. Specific implementations may use either flash
memory or DRAM chips protected by a battery to sustain the information through power cycles.
• Because SSDs have no moving parts they are much faster than traditional hard drives, and certain
problems such as the scheduling of disk accesses simply do not apply.
• However SSDs also have their weaknesses: They are more expensive than hard drives, generally not as
large, and may have shorter life spans.
• SSDs are especially useful as a high-speed cache of hard-disk information that must be accessed quickly.
One example is to store filesystem meta-data, e.g. directory and inode information, that must be accessed
quickly and often. Another variation is a boot disk containing the OS and some application executables,
but no vital user data. SSDs are also used in laptops to make them smaller, faster, and lighter.
• Because SSDs are so much faster than traditional hard disks, the throughput of the bus can become a
limiting factor, causing some SSDs to be connected directly to the system PCI bus for example.
Secondary Storage
Disc Structure
Q. Consider a System which has 16 plotters. Each plotter is divided into 2-Surfaces. Each Surface has 1K
tracks. Each track has 512 sectors. Each Sector can store 2 KB of data.
i) Find the capacity of disc in bytes.
ii) How many bits are required to identify no of sectors.

i) Capacity of Disk = 16*2*1*2^10*512*2*2^10 Bytes

= 2^4 * 2^11*2^9*2^11
= 2^35
= 2^5 GB
= 32 GB

ii) No of Sectors = 16 *2*2^10*2^9

= 2^4*2^11*2^9
= 2^24
24 bits requires to represent no of Sectors.
Secondary Storage
Disc I/O
• Seek Time
• Rotational Time
• Transfer Time
• Transfer Rate
• Seek Time: Amount of Time taken to move Read/Write header from it’s current position to desire track
• Rotational Time: Amount of Time taken to rotate the track when Read/Write header comes to exact
position(i.e. sector). Rotational time is generally taken as ‘half Rotation time’ i.e. (1/2 Rotation)
• Transfer Time: The amount of time taken to transfer the required data. Transfer time depends on the
rotation rate of the disc.
• Transfer Rate: The No of Bytes transferred for unit time is known as Transfer Rate.
 Secondary Storage
Disc Structure
Q. Consider a System where average seek time is 60ns and rotation rate of the disc is 3600rpm. Each track of
the disc has 1K sectors. Each sector is having the size of 512 bytes. What is the time taken to read two
random sectors over the disc?

• Total Seek time = 60ns+60ns = 120ns. Rotation Rate = 3600 rpm. • Transfer Rate
• Rotation Time • RT=1/60 s
3600 rotation  60 sec • 1/60 s  512*2^10B
So, 1 rotation  60/3600 sec = 1/120sec • So, 1s  512*2^10*60B
R.T = 2*1/120 = 1/60 Sec = 2^9*2^10*2*30B
• Now, 2^10*512  1/60 sec = 2^20*30B
1  (1/60)*(1/ 2^10*512) = 30MB
2*512  1/(30*2^10) sec
• Time to read 2 Random Sector = 120ns+ 1/60s + 1/(30*2^10) s
 Mass Storage Structures- Overview

Disc Scheduling
• As mentioned earlier, disk transfer speeds are limited primarily by seek times and rotational latency. When multiple requests
are to be processed there is also some inherent delay in waiting for other requests to be processed.
• Bandwidth is measured by the amount of data transferred divided by the total amount of time from the first request being
made to the last transfer being completed, ( for a series of disk requests. )
• Both bandwidth and access time can be improved by processing requests in a good order.
• Disk requests include the disk address, memory address, number of sectors to transfer, and whether the request is for reading
or writing.
 Mass Storage Structures- Overview
FCFS Scheduling
• First-Come First-Serve is simple and intrinsically fair, but not very efficient. Consider in the following sequence
the wild swing from cylinder 122 to 14 and then back to 124:

FCFS disk scheduling.

• No of Movement made by R/W head
=(98-53)+(183-98)+(183-37)+(122-37)+(124-14)+(129-65)+(67-65) = 640
 Mass Storage Structures- Overview
SSTF Scheduling
• Shortest Seek Time First scheduling is more efficient, but may lead to starvation if a constant stream of requests
arrives for the same general area of the disk.
• SSTF reduces the total head movement to 236 cylinders, down from 640 required for the same set of requests
under FCFS. Note, however that the distance could be reduced still further to 208 by starting with 37 and then
14 first before processing the rest of the requests

SSTF disk scheduling.

• No of Movement made by R/W head
=(65-53)+2+(67-37)+(37-14)+(98-14)+(122-98)+(124-122)+(184-124) = 236
 Mass Storage Structures- Overview
SCAN Scheduling
• SCAN Scheduling: The disk arm starts at one end of the disk, and moves toward the other end, servicing
requests until it gets to the other end of the disk, where the head movement is reversed and servicing
continues.
• SCAN algorithm Sometimes called the elevator algorithm
• Illustration shows total head movement of 208 cylinders
• But note that if requests are uniformly dense, largest density at other end of disk and those wait the longest

• No of Movement made by R/W head = 208

 Mass Storage Structures- Overview
C-SCAN Scheduling
• Provides a more uniform wait time than SCAN
• The head moves from one end of the disk to the other, servicing requests as it goes
• When it reaches the other end, however, it immediately returns to the beginning of the disk, without servicing
any requests on the return trip
• Treats the cylinders as a circular list that wraps around from the last cylinder to the first one

• No of Movement made by R/W head = ?

 Mass Storage Structures- Overview
C-LOOK Scheduling
• C-LOOK a version of C-SCAN
• Arm only goes as far as the last request in each direction, then reverses direction immediately, without first
going all the way to the end of the disk

• No of Movement made by R/W head = 153

 File System Concept
Objective
• To explain the function of file systems
• To describe the interfaces to file systems
• To discuss file-system design tradeoffs, including access methods, file sharing, file locking, and directory structures
• To explore file-system protection
• Contiguous logical address space
Types:
• Data
-numeric
-character
-binary
• Program
• Contents defined by file’s creator
Many types
->Consider text file, source file, executable file
 File System Concept

Attributes
• Name – only information kept in human-readable form
• Identifier – unique tag (number) identifies file within file system
• Type – needed for systems that support different types
• Location – pointer to file location on device
• Size – current file size
• Protection – controls who can do reading, writing, executing
• Time, date, and user identification – data for protection, security, and usage monitoring
• Information about files are kept in the directory structure, which is maintained on the disk
• Many variations, including extended file attributes such as file checksum
• Information kept in the directory structure
 File System Concept

File Operations
• File is an abstract data type
• Create
• Write – at write pointer location
• Read – at read pointer location
• Reposition within file - seek
• Delete
• Truncate
• Open(Fi) – search the directory structure on disk for entry Fi, and move the content of entry to
memory
• Close (Fi) – move the content of entry Fi in memory to directory structure on disk
 File System Concept

Open Files
• Several pieces of data are needed to manage open files: Open-file table: tracks
open files
• File pointer: pointer to last read/write location, per process that has the file open
• File-open count: counter of number of times a file is open – to allow removal of
data from open-file table when last processes closes it
• Disk location of the file: cache of data access information
• Access rights: per-process access mode information
 File System Concept

Open File Locking

• Provided by some operating systems and file systems Similar to reader-writer locks
• Shared lock similar to reader lock – several processes can acquire concurrently
• Exclusive lock similar to writer lock
• Mediates access to a file
• Mandatory or advisory: Mandatory – access is denied depending on locks held and
requested
• Advisory – processes can find status of locks and decide what to do
 File System Concept
Open File Locking

import java.io.*; // this locks the second half of the file - shared
import java.nio.channels.*; sharedLock = ch.lock(raf.length()/2+1, raf.length(),
public class LockingExample { SHARED);
public static final boolean EXCLUSIVE = false; /** Now read the data . . . */
public static final boolean SHARED = true; // release the lock
public static void main(String arsg[]) throws IOException { sharedLock.release();
} catch (java.io.IOException ioe) {
FileLock sharedLock = null;
System.err.println(ioe);
FileLock exclusiveLock = null; }finally {
try { if (exclusiveLock != null)
RandomAccessFile raf = new RandomAccessFile("file.txt", exclusiveLock.release();
"rw");
// get the channel for the file if (sharedLock != null)
FileChannel ch = raf.getChannel(); sharedLock.release();
// this locks the first half of the file - exclusive
}
exclusiveLock = ch.lock(0, raf.length()/2, EXCLUSIVE);
}
/** Now modify the data . . . */
}
// release the lock
exclusiveLock.release();
File System Concept
 File System Concept

Open File Locking

• None - sequence of words, bytes
• Simple record structure
Lines
Fixed length
Variable length
Complex Structures
Formatted document
Relocatable load file
 File System Concept
Sequential Access
Sequential Access File read next
write next
reset
no read after last write
(rewrite)
Direct Access – file is fixed length logical records
read n
write n
position to n
read next
write next
rewrite n
Where n = relative block number
Relative block numbers allow OS to decide
where file should be
 File System Concept
• Can be built on top of base methods
Other Access Methods • General involve creation of an index for the
file
• Keep index in memory for fast determination
of location of data to be operated on
(consider UPC code plus record of data about
that item)
• If too large, index (in memory) of the index
(on disk)
• IBM indexed sequential-access method
(ISAM) Small master index, points to disk
blocks of secondary index
• File kept sorted on a defined key
• All done by the OS
• VMS operating system provides index and
relative files as another example (see next
slide)

Example of Index and Relative Files

File System Implementation
Allocation Methods - Contiguous
An allocation method refers to how disk blocks are allocated for files:
Contiguous allocation – each file occupies set of contiguous blocks
• Best performance in most cases
• Simple – only starting location (block #) and length (number of blocks) are required
• Problems include finding space for file, knowing file size, external fragmentation, need for compaction off-line
(downtime) or on-line
File System Implementation

• Allocation Methods - Extent-Based Systems (modified contiguous

allocation )

• Many newer file systems (i.e., Veritas File System) use a modified contiguous
allocation scheme

• Extent-based file systems allocate disk blocks in extents

• An extent is a contiguous block of disks

Extents are allocated for file allocation
A file consists of one or more extents

Extents can be added to an existing file that needs space to grow. A block can be found
given by the location of the first block in the file and the block count, plus a link to the first
exten
File System Implementation

• Allocation Methods - Linked allocation

Linked allocation – each file a linked list of blocks

• File ends at nil pointer
• No external fragmentation
• Each block contains pointer to next block
• No compaction, external fragmentation
• Free space management system called when new block needed
• Improve efficiency by clustering blocks into groups but increases internal
fragmentation
• Reliability can be a problem
• Locating a block can take many I/Os and disk seeks

FAT (File Allocation Table) variation

• Beginning of volume has table, indexed by block number
• Much like a linked list, but faster on disk and cacheable
• New block allocation simple
File System Implementation

• Allocation Methods - Linked allocation

File System Implementation

• File Allocation Table

File System Implementation

Allocation Methods - Indexed

File System Implementation

Allocation Methods - Indexed Allocation

• Need index table
• Random access
• Dynamic access without external fragmentation, but have overhead of index block
• Mapping from logical to physical in a file of maximum size of 256K bytes and block size of 512
bytes. We need only 1 block for index table

• QR Q = displacement into index table

• R = displacement into block
• Mapping from logical to physical in a file of unbounded length (block size of 512 words)
• Linked scheme – Link blocks of index table (no limit on size)
File System Implementation

Allocation Methods - Indexed Allocation

• Q1 = block of index table
• R1 is used as follows:

• Q2 = displacement into block of index table

• R2 displacement into block of file:

Indexed Allocation Mapping

File System Implementation

Allocation Methods – Combined Scheme

• 4K bytes per block, 32-bit addresses
• If file is small enough, use only direct blocks pointers.

• If number of blocks in file is greater than the number of

direct block pointers, use single, double, or triple indirect. A

• Additional levels of indirection increase the number of

blocks that can be associated with a file.

More index blocks

than can be
addressed with 32-bit
file pointer
File System Implementation

Performance
• Best method depends on file access type Contiguous great for sequential and random
• Linked good for sequential, not random
• Declare access type at creation -> select either contiguous or linked
• Indexed more complex Single block access could require 2 index block reads then data block read
• Clustering can help improve throughput, reduce CPU overhead
• Adding instructions to the execution path to save one disk I/O is reasonable
Intel Core i7 Extreme Edition 990x (2011) at 3.46Ghz = 159,000 MIPS
http://en.wikipedia.org/wiki/Instructions_per_second
Typical disk drive at 250 I/O s per second 159,000 MIPS / 250 = 630 million instructions during one
disk I/O
Fast SSD drives provide 60,000 IOPS 159,000 MIPS / 60,000 = 2.65 millions instructions during one
disk I/O
File System Implementation

Free Space Management

• Bit map requires extra space

Example:
block size = 4KB = 212 bytes
disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory
• Easy to get contiguous files
File System Implementation

Free Space Management:

• Linked Free Space List on Disk
File System Implementation
Free Space Management:
• Grouping Modify linked list to store address of next n-1 free blocks in first free block, plus a pointer to next block that
contains free-block-pointers (like this one)
• Counting Because space is frequently contiguously used and freed, with contiguous-allocation allocation, extents, or
clustering Keep address of first free block and count of following free blocks
• Free space list then has entries containing addresses and counts
• Space Maps Used in ZFS
• Consider meta-data I/O on very large file systems Full data structures like bit maps couldn’t fit in memory -> thousands
of I/Os

• Divides device space into metaslab units and manages metaslabs Given volume can contain hundreds of metaslabs

• Each metaslab has associated space map Uses counting algorithm

• But records to log file rather than file system Log of all block activity, in time order, in counting format

• Metaslab activity -> load space map into memory in balanced-tree structure, indexed by offset Replay log into that
structure
• Combine contiguous free blocks into single entry
File System Implementation

Efficiency and Performance:

• Efficiency dependent on: Disk allocation and directory algorithms
• Types of data kept in file’s directory entry
• Pre-allocation or as-needed allocation of metadata structures
• Fixed-size or varying-size data structures
• Performance Keeping data and metadata close together
• Buffer cache – separate section of main memory for frequently used blocks
• Synchronous writes sometimes requested by apps or needed by OS No buffering / caching – writes
must hit disk before acknowledgement
• Asynchronous writes more common, buffer-able, faster
• Free-behind and read-ahead – techniques to optimize sequential access
• Reads frequently slower than writes
• Buffer cache : where blocks are kept under the assumption that they
will be used again shortly.
• page cache : cache file data using a page cache
• The page cache uses virtual memory techniques to cache file data as
pages rather than as file-system-oriented blocks. as accesses interface
with virtual memory rather than the file system.
• Several systems—including Solaris, Linux, and Windows —use page
caching to cache both process pages and file data. This is known as
unified virtual memory.
 File System Concept

Directory Structure
• A collection of nodes containing information about all files

Both the directory structure and the files reside on disk

 File System Concept

Disk Structure

• Disk can be subdivided into partitions

• Disks or partitions can be RAID protected against failure
• Disk or partition can be used raw – without a file system, or formatted with a file system
• Partitions also known as minidisks, slices
• Entity containing file system known as a volume
• Each volume containing file system also tracks that file system’s info in device directory
or volume table of contents
• As well as general-purpose file systems there are many special-purpose file
systems, frequently all within the same operating system or computer
 File System Concept

A Typical File-system Organization

Operation Perform on Files

• Search for a file
• Create a file
• Delete a file
• List a directory
• Rename a file
• Traverse the file system
 File System Concept

Directory Organization

The Directory is organized logically to obtained

• Efficiency – locating a file quickly

• Naming – convenient to users Two users can have same name for different files
• The same file can have several different names
• Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …)
• A single directory for all users

• Naming problem
• Grouping problem
 File System Concept

Directory Organization

Two Level Directory

• Separate directory for each user

• Path name
• Can have the same file name for different user
• Efficient searching
• No grouping capability
• when the users want to cooperate on some task and to access one another’s files ?
• a user name and a file name define a path name.
• Additional syntax is needed to specify the volume of a file
 File System Concept
Directory Organization
• Tree Structured Directory
• generalization is to extend the directory structure
• A directory is simply another file (One bit in each directory entry defines the entry as a file (0) or as a subdirectory
(1).
• An absolute path name begins at the root and follows a path down to the specified file
• A relative path name defines a
path from the current directory.

• Allowing a user to define her own subdirectories

• Efficient searching

• Grouping Capability

• deletion of a directory?

• users can be allowed to access, in addition to

their files, the files of other users.
 File System Concept
Directory Organization
• Absolute or relative path name
• Creating a new file is done in current directory
• Delete a file
rm <file-name>
• Creating a new subdirectory is done in current directory
mkdir <dir-name>
• Example: if in current directory /mail
• mkdir count

• Deleting “mail” ⇒ deleting the entire subtree rooted by “mail”

 File System Concept
Directory Organization
Acyclic-Graph Directories
• A shared directory or file exists in the file system in two (or more) places at once.
• With a shared file, only one actual file exists, so any changes made by one person are immediately visible to the other
• Link
• duplicate all information about them in both sharing directories
• A file may now have multiple absolute path names
• shared file be deallocated and reused?
• If dict deletes list ⇒ dangling pointer
• Solutions:
• Backpointers, so we can delete all pointers
Variable size records a problem
• Backpointers using a daisy chain organization
• Entry-hold-count solution
• New directory entry type Link –
another name (pointer) to an existing file
• Resolve the link – follow pointer to locate the file
 File System Concept
Directory Organization
General Graph Directories

How do we guarantee no cycles?

• Allow only links to file not subdirectories
• Garbage collection
• Every time a new link is added use a cycle
detection algorithm to determine whether it is
OK
 File System Concept
File Sharing
• Sharing of files on multi-user systems is desirable
• Sharing may be done through a protection scheme
• On distributed systems, files may be shared across a network
• Network File System (NFS) is a common distributed file-sharing method
• If multi-user system User IDs identify users, allowing permissions and protections to be per-user
Group IDs allow users to be in groups, permitting group access rights
• Owner of a file / directory
• Group of a file / directory
 File System Concept
File Sharing
• Uses networking to allow file system access between systems Manually via programs
like FTP
Automatically, seamlessly using distributed file systems
Semi automatically via the world wide web
• Client-server model allows clients to mount remote file systems from servers
Client and user-on-client identification is insecure or complicated
NFS is standard UNIX client-server file sharing protocol
Server can serve multiple clients
• CIFS is standard Windows protocol
• Standard operating system file calls are translated into remote calls
• Distributed Information Systems (distributed naming services) such as LDAP, DNS,
NIS, Active Directory implement unified access to information needed for remote
computing
 File System Concept
File Sharing-Failure Mode
• All file systems have failure modes; For example corruption of directory
structures or other non-user data, called metadata
• Remote file systems add new failure modes, due to network failure,
server failure
• Recovery from failure can involve state information about status of
each remote request
• Stateless protocols such as NFS v3 include all information in each
request, allowing easy recovery but less security
 File System Concept
File Sharing-Failure Mode
• Specify how multiple users are to access a shared file simultaneously
• Similar to process synchronization algorithms
• Tend to be less complex due to disk I/O and network latency (for remote file
systems
• Andrew File System (AFS) implemented complex remote file sharing
semantics
• Unix file system (UFS) implements:
• Writes to an open file visible immediately to other users of the same open file
• Sharing file pointer to allow multiple users to read and write concurrently
• AFS has session semantics
->Writes only visible to sessions starting after the file is closed
 File System Concept
Protection
• File owner/creator should be able to control:
->what can be done
->by whom
Types of access
• Read
• Write
• Execute
• Append
• Delete
• List
File System Implementation

• File-System Structure
• File-System Implementation
• Directory Implementation
• Allocation Methods
• Free-Space Management
File System Implementation

Objective
• To describe the details of implementing local file systems and directory
structures
• To describe the implementation of remote file systems
• To discuss block allocation and free-block algorithms and trade-offs
File System Implementation
Objective
• File structure
Logical storage unit
Collection of related information
• File system resides on secondary storage (disks)
Provided user interface to storage, mapping logical to physical
Provides efficient and convenient access to disk by allowing data to be stored,
located retrieved easily
• Disk provides in-place rewrite and random access
 I/O transfers performed in blocks of sectors (usually 512 bytes)
• File control block – storage structure consisting of information about a file
• Device driver controls the physical device
• File system organized into layers
File System Implementation

Layered File System

Device drivers manage I/O devices at the I/O control layer
Given commands like “read drive1, cylinder 72, track 2,
sector 10, into memory location 1060” outputs low-level
hardware specific commands to hardware controller

Basic file system given command like “retrieve block 123”

translates to device driver
Also manages memory buffers and caches (allocation,
freeing, replacement)
Buffers hold data in transit
Caches hold frequently used data

File organization module understands files, logical address,

and physical blocks
Translates logical block # to physical block #
Manages free space, disk allocation
File System Implementation • Logical file system manages metadata information
Translates file name into file number, file handle, location by
File System Layers maintaining file control blocks (inodes in UNIX)
Directory management
Protection
• Layering useful for reducing complexity and redundancy, but adds
overhead and can decrease performance Translates file name into
file number, file handle, location by maintaining file control blocks
(inodes in UNIX)
Logical layers can be implemented by any coding method
according to OS designer.
Many file systems, sometimes many within an operating system
• Each with its own format (CD-ROM is ISO 9660; Unix has UFS, FFS;
Windows has FAT, FAT32, NTFS as well as floppy, CD, DVD Blu-ray,
Linux has more than 40 types, with extended file system ext2 and
ext3 leading; plus distributed file systems, etc.)
• New ones still arriving – ZFS, GoogleFS, Oracle ASM, FUSE
File System Implementation

• We have system calls at the API level, but how do we implement their
functions?
On-disk and in-memory structures
• Boot control block contains info needed by system to boot OS from that
volume.
 Needed if volume contains OS, usually first block of volume
• Volume control block (superblock, master file table) contains volume
details –
• Total # of blocks, # of free blocks, block size, free block pointers or array
• Directory structure organizes the files
Names and inode numbers, master file table
File System Implementation

• Per-file File Control Block (FCB) contains many details about the file
inode number, permissions, size, dates
NFTS stores into in master file table using relational DB structures
File System Implementation

In-Memory File System Structures

• Mount table storing file system mounts, mount points, file system types
• The following figure illustrates the necessary file system structures
provided by the operating systems
• Figure 12-3(a) refers to opening a file
• Figure 12-3(b) refers to reading a file
• Plus buffers hold data blocks from secondary storage
• Open returns a file handle for subsequent use
• Data from read eventually copied to specified user process memory
address
File System Implementation

In-Memory File System Structures

File System Implementation

Partitions and Mounting

• Partition can be a volume containing a file system (“cooked”) or raw – just a sequence of
blocks with no file system
• Boot block can point to boot volume or boot loader set of blocks that contain enough code to
know how to load the kernel from the file system Or a boot management program for multi-os
booting

• Root partition contains the OS, other partitions can hold other Oses, other file systems, or
be raw Mounted at boot time
• Other partitions can mount automatically or manually

• At mount time, file system consistency checked Is all metadata correct? If not, fix it, try
again
• If yes, add to mount table, allow access
File System Implementation

Virtual File Systems

• Virtual File Systems (VFS) on Unix provide an object-oriented way of implementing file
systems
• VFS allows the same system call interface (the API) to be used for different types of file
systems
• Separates file-system generic operations from implementation details
• Implementation can be one of many file systems types, or network file system
• Implements vnodes which hold inodes or network file details

• Then dispatches operation to appropriate file system implementation routines

File System Implementation

Virtual File Systems

The API is to the VFS interface, rather than any specific type of file system
File System Implementation

Virtual File Systems

For example, Linux has four object types: inode, file, superblock, dentry

VFS defines set of operations on the objects that must be implemented

Every object has a pointer to a function table Function table has addresses
of routines to implement that function on that object
For example:
• int open(. . .)—Open a file
• int close(. . .)—Close an already-open file
• ssize t read(. . .)—Read from a file
• ssize t write(. . .)—Write to a file
• int mmap(. . .)—Memory-map a file
File System Implementation

Directory Implementation
Linear list of file names with pointer to the data blocks
Simple to program
Time-consuming to execute
->Linear search time
->Could keep ordered alphabetically via linked list or use B+ tree

Hash Table – linear list with hash data structure

• Decreases directory search time
• Collisions – situations where two file names hash to the same location
• Only good if entries are fixed size, or use chained-overflow method
 Unified buffer cache.
Opening and accessing a file : One approach is to use memory mapping; the second is to use the
standard system calls read () and write ().

2) double caching, requires caching file-system data twice. Not only does it waste memory but it also
wastes significant CPU and I/O cycles due to the extra data movement within system memory. In addition,
inconsistencies between the two caches can result in corrupt files.

I/O without a unified buffer cache.

• Another issue that can affect the performance of I/O is whether writes to the file
system occur synchronously or asynchronously.
• Synchronous writes occur in the order in which the disk subsystem receives
them, and the writes are not buffered. Thus, the calling routine must wait for the
data to reach the disk drive before it can proceed.
• In an asynchronous write, the data are stored in the cache, and control returns to
the caller. Most writes are asynchronous.
• However, metadata writes, among others, can be synchronous.
• Operating systems frequently include a flag in the open system call to allow a
process to request that writes be performed synchronously. For example, databases
use this feature for atomic transactions, to assure that data reach stable storage in
the required order.
• Any Question???