Operating

Systems:
Internals Chapter 5 Part 2
and Design
Principles
Secondary Storage Mgt.
File Mgt. in Popular OSs
Eighth Edition
By William Stallings
 File Allocation
§ On secondary storage, a file consists of a collection of blocks

§ The operating system or file management system is responsible for

allocating blocks to files

§ The approach taken for file allocation may influence the approach
taken for free space management

§ Space is allocated to a file as one or more portions (contiguous set of

allocated blocks)

§ File allocation table (FAT)

§ data structure used to keep track of the portions assigned to a file
 Preallocation vs
Dynamic Allocation
n A preallocation policy requires that the maximum size of a file be
declared at the time of the file creation request

n For many applications it is difficult to estimate reliably the maximum

potential size of the file
n tends to be wasteful because users and application programmers tend
to overestimate size

n Dynamic allocation allocates space to a file in portions as needed

 Portion Size
n In choosing a portion size there is a trade-off between efficiency from
the point of view of a single file versus overall system efficiency

n Items to be considered:
1) contiguity of space increases performance, especially for
Retrieve_Next operations, and greatly for transactions
running in a transaction-oriented operating system
2) having a large number of small portions increases the size of
tables needed to manage the allocation information
3) having fixed-size portions simplifies the reallocation of space
4) having variable-size or small fixed-size portions minimizes
waste of unused storage due to overallocation
 Alternatives
Two major alternatives:

Variable, large Blocks

contiguous portions • small fixed portions provide
• provides better performance greater flexibility
• the variable size avoids waste • they may require large tables
or complex structures for their
• the file allocation tables are
allocation
small
• contiguity has been abandoned
as a primary goal
• blocks are allocated as needed
 Table 12.2
File Allocation Methods

Contiguous Chained Indexed

Preallocation? Necessary Possible Possible
Fixed or variable Variable Fixed blocks Fixed blocks Variable
size portions?
Portion size Large Small Small Medium
Allocation Once Low to high High Low
frequency
Time to allocate Medium Long Short Medium
File allocation One entry One entry Large Medium
table size
 File Allocation Table
File A File Name Start Block Length
0 1 2 3 4 File A 2 3
File B 9 5
5 6 7 8 9 File C 18 8
File D 30 2
File B
File E 26 3
10 11 12 13 14

15 16 17 18 19
File C
20 21 22 23 24
File E
25 26 27 28 29
File D
30 31 32 33 34

Figure 12.9 Contiguous File Allocation

 File Allocation Table
File A File Name Start Block Length
0 1 2 3 4 File A 0 3
File B File B 3 5
5 6 7 8 9 File C 8 8
File D 19 2
File C
File E 16 3
10 11 12 13 14
File E File D
15 16 17 18 19

20 21 22 23 24

25 26 27 28 29

30 31 32 33 34

Figure 12.10 Contiguous File Allocation (After Compaction)

 File Allocation Table
File B File Name Start Block Length
0 1 2 3 4
File B 1 5
5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24

25 26 27 28 29

30 31 32 33 34

Figure 12.11 Chained Allocation

 File Allocation Table
File B File Name Start Block Length
0 1 2 3 4
File B 0 5
5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24

25 26 27 28 29

30 31 32 33 34

Figure 12.12 Chained Allocation (After Consolidation)

 File Allocation Table
File B File Name Index Block
0 1 2 3 4
File B 24
5 6 7 8 9

10 11 12 13 14

15 16 17 18 19
1
20 21 22 23 24 8
3
25 26 27 28 29 14
28
30 31 32 33 34

Figure 12.13 Indexed Allocation with Block Portions

 File Allocation Table
File B File Name Index Block
0 1 2 3 4
File B 24
5 6 7 8 9

10 11 12 13 14

15 16 17 18 19
Start Block Length
20 21 22 23 24 1 3
28 4
25 26 27 28 29 14 1

30 31 32 33 34

Figure 12.14 Indexed Allocation with Variable-Length Portions

 Free Space Management
n Just as allocated space must be managed, so must the unallocated
space

n To perform file allocation, it is necessary to know which blocks are

available

n A disk allocation table is needed in addition to a file allocation table

Bit Tables
n This method uses a vector containing one bit for each block on the
disk

n Each entry of a 0 corresponds to a free block, and each 1

corresponds to a block in use

Advantages:
• works well with any file
allocation method
• it is as small as possible
 Chained Free Portions
n The free portions may be chained together by using a pointer and
length value in each free portion

n Negligible space overhead because there is no need for a disk

allocation table

n Suited to all file allocation methods

Disadvantages:

• leads to fragmentation
• every time you allocate a block you need to read
the block first to recover the pointer to the new
first free block before writing data to that block
 Indexing
n Treats free space as a file and uses an index table as it would for file
allocation

n For efficiency, the index should be on the basis of variable-size

portions rather than blocks

n This approach provides efficient support for all of the file allocation
methods
Free Block List
There are two effective
Depending on the size of
Each block is assigned a techniques for storing a
the disk, either 24 or 32 small part of the free
number sequentially bits will be needed to store
block list in main
a single block number memory:

the list can be treated as

a push-down stack with
the size of the free the first few thousand
the list of the elements of the stack
block list is 24 or 32
numbers of all free times the size of the kept in main memory
blocks is maintained
corresponding bit
in a reserved portion
table and must be
of the disk stored on disk the list can be treated as
a FIFO queue, with a
few thousand entries
from both the head and
the tail of the queue in
main memory
 Volumes
n A collection of addressable sectors in secondary
memory that an OS or application can use for
data storage
n The sectors in a volume need not be consecutive
on a physical storage device
n they need only appear that way to the OS or application

n A volume may be the result of assembling and

merging smaller volumes
 UNIX File n In the UNIX file system, six
types of files are distinguished:
Management
Regular, or ordinary
• contains arbitrary data in zero or more data blocks

Directory
• contains a list of file names plus pointers to associated inodes

Special
• contains no data but provides a mechanism to map physical devices to file names

Named pipes
• an interprocess communications facility

Links
• an alternative file name for an existing file

Symbolic links
• a data file that contains the name of the file it is linked to
Inodes
n All types of UNIX files are administered by the OS by means of
inodes

n An inode (index node) is a control structure that contains the key

information needed by the operating system for a particular file

n Several file names may be associated with a single inode

n an active inode is associated with exactly one file
n each file is controlled by exactly one inode
 mode
Data Data Data
owners (2)
Data
timestamps (4)
Data Data Data
size

direct(0) Data

direct (1) Data Data

Pointers

Data Data
Pointers
direct(12)

single indirect Pointers

Data
double indirect Pointers

triple indirect
Pointers
block count Data

reference count
Pointers Pointers
flags (2)
Data
Pointers
generation number

blocksize Pointers Pointers

Data
extended attr size

extended
attribute Pointers
blocks Data

Inode

Figure 12.15 Structure of FreeBSD inode and File

 File Allocation
n File allocation is done on a block basis

n Allocation is dynamic, as needed, rather than using preallocation

n An indexed method is used to keep track of each file, with part of

the index stored in the inode for the file

n In all UNIX implementations the inode includes a number of direct

pointers and three indirect pointers (single, double, triple)
 Table 12.3
Capacity of a FreeBSD File with 4 kByte Block Size

Level Number of Blocks Number of Bytes

Direct 12 48K

Single Indirect 512 2M

Double Indirect 512 × 512 = 256K 1G

Triple Indirect 512 × 256K = 128M 512G

Inode table Directory

i1 Name1
i2 Name2
i3 Name3
i4 Name4

Figure 12.16 UNIX Directories and Inodes

 Volume Structure
n A UNIX file Data
system resides Boot block Superblock Inode table
blocks
on a single
logical disk or
disk partition
and is laid out contains
contains
with the code
required to attributes and collection of
storage space
information available for
inodes for
following boot the
about the file each file
data files and
operating subdirectories
elements: system system
 User applications
User
space
GNU C library

System call interface

Inode Virtual File Directory

cache System (VFS) cache

File
system Individual File Kernel
Systems space

Buffer cache

Device drivers

Figure 12.17 Linux Virtual File System Context

 System calls
System calls VFS
using file
using VFS system
system X
user interface Linux calls Mapping Disk I/O
interface
Virtual function File calls
File to file System X
System system X
User
Process

Files on secondary
storage maintained
by file system X

Figure 12.18 Linux Virtual File System Concept

Primary Object Types in VFS
Superblock Dentry Object
Object • represents a specific
• represents a specific directory entry
mounted file system

Inode Object File Object

• represents an open
• represents a file associated with
specific file a process
 Windows File System
n The developers of Windows NT designed a new file system, the New
Technology File System (NTFS) which is intended to meet high-end
requirements for workstations and servers

n Key features of NTFS:

n recoverability
n security
n large disks and large files
n multiple data streams
n journaling
n compression and encryption
n hard and symbolic links
NTFS Volume
and File Structure
n NTFS makes use of the following disk storage concepts:

• the smallest physical storage unit on the disk

Sector • the data size in bytes is a power of 2 and is almost always
512 bytes

Cluster • one or more contiguous sectors

• the cluster size in sectors is a power of 2

• a logical partition on a disk, consisting of one or more

clusters and used by a file system to allocate space
Volume • can be all or a portion of a single disk or it can extend
across multiple disks
• the maximum volume size for NTFS is 264 bytes
 Table 12.4
Windows NTFS Partition and Cluster Sizes

Volume Size Sectors per Cluster Cluster Size

≤ 512 Mbyte 1 512 bytes
512 Mbyte - 1 Gbyte 2 1K
1 Gbyte - 2 Gbyte 4 2K
2 Gbyte - 4 Gbyte 8 4K
4 Gbyte - 8 Gbyte 16 8K
8 Gbyte - 16 Gbyte 32 16K
16 Gbyte - 32 Gbyte 64 32K
> 32 Gbyte 128 64K
partition System
boot Master File Table Files File Area
sector

Figure 12.19 NTFS Volume Layout

 Master File Table (MFT)
n The heart of the Windows file system is the MFT

n The MFT is organized as a table of 1,024-byte rows, called records

n Each row describes a file on this volume, including the MFT itself,
which is treated as a file

n Each record in the MFT consists of a set of attributes that serve to

define the file (or folder) characteristics and the file contents
 Table 12.5
Windows NTFS File and Directory Attribute Types
Attribute Type Description
Standard information Includes access attributes (read-only, read/write, etc.); time
stamps, including when the file was created or last modified;
and how many directories point to the file (link count).
Attribute list A list of attributes that make up the file and the file reference
of the MFT file record in which each attribute is located. Used
when all attributes do not fit into a single MFT file record.
File name A file or directory must have one or more names.
Security descriptor Specifies who owns the file and who can access it.
Data The contents of the file. A file has one default unnamed data
attribute and may have one or more named data attributes.
Index root Used to implement folders.
Index allocation Used to implement folders.
Volume information Includes volume-related information, such as the version and
name of the volume.
Bitmap Provides a map representing records in use on the MFT or
folder.

Note: Colored rows refer to required file attributes; the other attributes are optional.
 I/O Manager
Log the transaction
Log File NTFS Driver Read/write a
Service mirrored or
Read/write
Fault Tolerant striped volume
the file
Flush the Write the Driver
Read/write
log file cache the disk
Disk Driver

Cache Load data from

Manager disk into
memory
Access the mapped
file or flush the cache

Virtual Memory
Manager

Figure 12.20 Windows NTFS Components

/(root)

(ro)
/system bin

etc

lib
/data (rw)
usr

/cache (rw)

/mnt/sdcard removable storage (rw)

ro: mounted as read only

rw: mounted as read and write

Figure 12.21 Typical Directory Tree of Android

 SQLite
n Most widely deployed SQL database engine in the world

n Based on the Structured Query Language (SQL)

n Designed to provide a streamlined SQL-based database management

system suitable for embedded systems and other limited memory systems

n The full SQLite library can be implemented in under 400 KB

n In contrast to other database management systems, SQLite is not a separate

process that is accessed from the client application
n the library is linked in and thus becomes an integral part of the
application program
 Summary
n File structure n Secondary storage management
n File management systems n File allocation
n File organization and access n Free space management
n The pile n Volumes
n The sequential file n Reliability
n The indexed sequential file n UNIX file management
n The indexed file n Inodes
n The direct or hashed file n File allocation
n B-Trees n Directories
n File directories n Volume structure
n Contents n Linux virtual file system
n Structure n Superblock object
n Naming n Inode object
n File sharing n Dentry object
n Access rights n File object
n Simultaneous access n Caches
n Record blocking n Windows file system
n Android file management n Key features of NTFS
n File system n NTFS volume and file structure
n SQLite n Recoverability