File Systems

CSci4061: Introduction to Operating Systems

Kangjie Lu
February 20, 2024
Computer Science & Engineering, University of Minnesota

1
Review: Files

• File I/O operations

• File buffering
• File descriptors
• File redirection
• Pipes and FIFO

2
File systems overview
File Systems

• An important subsystem of OS that controls how data is stored and retrieved

• Why do we need file systems?

3
File Systems

• An important subsystem of OS that controls how data is stored and retrieved

• Why do we need file systems?
1. Usability and maintenance
• Implement an abstraction (files) for secondary storage
• Proper data organization (directories)
• Easy to find data (e.g., indexing)
2. Efficiency
• Efficient data storage, space management
3. Protection
• Protect data from unwanted access (security)
• Prevent data loss from crashes
3
File System Structure

• A file system typically consists of:

• Files: Data objects
• Directories or folders: Collection of files
• Other objects (e.g., links) specific to how data is organized in the file system
• Different file systems implement these entities in different ways

4
Unix File System Structure

• Files: Data objects

• Regular files, device files, pipes
• Directories: Collections of files
• Contain information about files
• Links: Pointers to other files
• Pseudonyms/nicknames

5
Files

• A file is a sequence of bytes with some properties

• Owner, last read/write time, permission, etc.
• A file can also have a type
• Understood by the file system
• Block, character, device, portal, link, etc.
• Understood by other parts of the OS or runtime libraries
• Executable, dll, source, object, text, etc.
• A file’s type can be encoded in its name or contents
• Windows encodes type in name
• .com, .exe, .bat, .dll, .jpg, etc.
• Unix encodes type in contents
• Magic numbers in headers,
• Initial characters (e.g., #! for shell scripts)
6
Directories
Directories

• Directory is a special file

• Directories serve two purposes
• For users, they provide a structured way to organize files
• For the file system, they provide a convenient naming interface that allows the
implementation to separate logical file organization from physical file
placement on the disk
• Most file systems support multi-level directories
• Naming hierarchies (/, /usr, /usr/local/, . . . )
• Most file systems support the notion of a current directory
• Relative names/paths specified with respect to current directory
• Absolute names/paths start from the root of directory tree

7
Unix File-Directory Organization

8
Directory Internals

• A directory structure is a list of entries

• Each entry contains:
• Name of file and file type
• Pointer to file inode (metadata)
• List is usually “unordered” (unspecified ordering)
• Entries usually sorted by program that reads directory
• Directory structures typically stored in files
• Only need to manage one kind of secondary storage unit

9
Directory Structure

10
Directory Operations

• Different operations from the ones used for regular files. Why?
• Cannot use:
• open, read, write, close
• Instead use:
• opendir, readdir, closedir

11
Opening a Directory

DIR *opendir(char *dirname);

• Returns a directory stream of type DIR
• Contains an ordered sequence of directory entries
• Order need not be alphabetical
• Order is implementation-dependent
• Examples
• opendir(".");
• opendir("/home/kjlu");
• opendir("kjlu/");

12
DIR

typedef struct __dirstream DIR;

/* Directory stream type */
struct __dirstream {
int fd; /* File descriptor. */
__libc_lock_define (, lock) /* Mutex lock for this structure. */
size_t allocation; /* Space allocated for the block. */
size_t size; /* Total valid data in the block. */
size_t offset; /* Current offset into the block. */
off_t filepos; /* Position of next entry to read. */
int errcode; /* Delayed error code. */
char data[0] __attribute__ ((aligned (__alignof__ (long double))));
};

13
Reading Directory Entries

struct dirent *readdir(DIR *dirp);

• Returns the next directory entry (within the current directory)
• NULL if end of directory or error
• Directory entry contains information about a file or subdirectory
• Filename and file type
• Pointer to file’s inode
• Pointer to next directory entry

14
Closing and Repositioning a Directory

int closedir(DIR *dirp);

• Closes an open directory
void rewinddir(DIR *dirp);
• Repositions the directory to the beginning

15
Modifying Directory Entries

• Cannot write to a directory entry directly

• Instead directory entries change through operations on files:
• File creation/deletion: creat, remove, open
• Filenames, symbolic links: rename, link, unlink
• Directory creation and removal: mkdir, rmdir
• Some of the operations differ based on whether the argument is a file or a
directory

16
File System Traversal

int chdir(char *path);

• Change working directory to a given path
• cd command

char *getcwd(char *buf, size_t size);

• Get the current working directory
• pwd command

These system calls are useful to implement cd in your second project

17
Path Name Translation

• Let’s say you want to open “/one/two/three”

• What does the file system do?
• Open directory “/” (well known, can always find)
• Search for the entry “one”, get location of “one” (in dir entry)
• Open directory “one”, search for “two”, get location of “two”
• Open directory “two”, search for “three”, get location of “three”
• Open file three
• Systems spend a lot of time walking directory paths
• This is why open is separate from read/write
• OS will cache prefix lookups for performance
• /a/b, /a/bb, /a/bbb, etc., all share /a prefix

18
File and Directory Permissions
File mode bits

• Core permissions are 9 bits, three groups of three bits

• Groups: user, group, other
• In each group, three bits represent read, write, execute
• ls -l format
• drwxr-xr-x 7 klu staff 224 Feb 25 23:03 10-file-system1
• -rw-r--r-- 1 klu staff 52 Jan 20 00:18 Makefile
• Q: what is the first “-’ ’?

19
File mode bits

• Core permissions are 9 bits, three groups of three bits

• Groups: user, group, other
• In each group, three bits represent read, write, execute
• ls -l format
• drwxr-xr-x 7 klu staff 224 Feb 25 23:03 10-file-system1
• -rw-r--r-- 1 klu staff 52 Jan 20 00:18 Makefile
• Q: what is the first “-’ ’?
• A: file type (directory?), actually not part of the permission
• Q: what is octal value of rw-rw-r--?

19
File mode bits

• Core permissions are 9 bits, three groups of three bits

• Groups: user, group, other
• In each group, three bits represent read, write, execute
• ls -l format
• drwxr-xr-x 7 klu staff 224 Feb 25 23:03 10-file-system1
• -rw-r--r-- 1 klu staff 52 Jan 20 00:18 Makefile
• Q: what is the first “-’ ’?
• A: file type (directory?), actually not part of the permission
• Q: what is octal value of rw-rw-r--?
• A: 664

19
Interpretation of mode bits

• Files/dirctories have one user and group ID

• Files/dirctories have one set of bits
• If users match, use user bits
• Else if subject is in the group, use group bits
• Otherwise, use other bits
• Usually, in terms of permissiveness, O ⊆ G ⊆ U

20
Directory mode bits

• Same bits, slightly different interpretation

• Read: read file names, but not attributes
• Write: create, rename, and delete files
• Execute: read file attributes, list files

21
File Sharing

• File sharing is important for getting work done

• Basis for communication between processes and users
• Two key issues when sharing files
• Semantics of concurrent access
• What happens when one process reads while another writes?
• What happens when two processes open a file for writing?
• From week 12
• Protection
• Can a user access a file?
• From week 14

22
File system layout and Inodes
File systems

• File system design: how to allocate and keep track of files and directories
• Does it matter? What is the difference?
• Performance, reliability, limitations on files, overhead, . . .
• Many different file systems have been proposed and continue to be proposed

23
Disk Layout Strategies

• Files span multiple disk blocks

• How do you find all of the blocks for a file?
1. Contiguous allocation
• Like memory
• Fast, simplifies directory access
• Inflexible, causes fragmentation, needs compaction
2. Linked structure
• Each block points to the next, directory points to the first
• Bad for random access patterns
3. Indexed structure (indirection, hierarchy)
• An index block contains pointers to many other blocks
• Handles random better, still good for sequential
• May need multiple index blocks (linked together)
24
File System Layout

How do file systems use the disk to store files?

25
File System Layout

• File systems define a block size (e.g., 4KB)

• Disk space is allocated in granularity of blocks
• A Master Block determines location of root directory
• At fixed disk location, sometimes replicated for reliability
• A free map determines which blocks are free, allocated
• Usually a bitmap, one bit per block on the disk
• Also stored on disk, cached in memory for performance
• Remaining blocks store files (and dirs), and swap!

26
Unix Inodes

• Unix inodes implement an indexed structure for files

• Also store metadata info (protection, timestamps, length, ref count. . . )

struct inode {
umode_t i_mode;
unsigned short i_opflags;
kuid_t i_uid;
kgid_t i_gid;
unsigned int i_flags;
struct posix_acl *i_acl;
const struct inode_operations *i_op;
struct super_block *i_sb;
struct timespec64 i_atime;
...
27
Unix Inodes
• The inode pointer structure lists the addresses of a file’s data blocks
• Each inode contains 15 block pointers
• First 12 are direct blocks, then single, double, and triple indirect. Why?

28
Unix Inodes and Path Search
• Unix Inodes are “not” directories
• Inodes describe where on disk the blocks for a file are placed
• Directories are files, so inodes also describe where the blocks for directories
are placed on the disk
• Directory entries map file names to inodes
• To open “/file”, use Master Block to find inode for “/” on disk
• Open “/”, look for entry for “file”
• This entry gives the disk block number for the inode for “file”
• Read the inode for “file” into memory
• The inode says where first data block is on disk
• Read that block into memory to access the data in the file
• This is why
• we have open in addition to read and write
• we should check potential errors due to the complexity 29
Quiz 1

Path names can be of two types:

a) absolute & relative
b) local & global
c) global & relative
d) relative & local

30
Quiz 1

Path names can be of two types:

a) absolute & relative
b) local & global
c) global & relative
d) relative & local
ans: a

30
Quiz 2

What are the permission list for 755?

a) rw—x–x
b) rw-r–r–
c) rwxr-xr-x
d) rwxrw-rw-

31
Quiz 2

What are the permission list for 755?

a) rw—x–x
b) rw-r–r–
c) rwxr-xr-x
d) rwxrw-rw-
ans: c

31
Quiz 3

Which of the following is not present in the inode?

a. File size
b. File name
c. File access mode
d. All the above

32
Quiz 3

Which of the following is not present in the inode?

a. File size
b. File name
c. File access mode
d. All the above
ans: b

32
Summary

• File systems
• Purposes
• Structure
• Directories
• Structure
• Traversal
• Permissions
• Mode bits
• Inodes
• Inode pointer structure

33
Next lecture

• Links and file-system implementation

• Reading: Robbins Ch. 5.4

34
References

[1] http://www-users.cselabs.umn.edu/classes/Fall-
2018/csci4061/notes/w5_1_post.pdf
[2] http://www-users.cselabs.umn.edu/classes/Spring-
2018/csci4061/notes/file_sys.pdf
[3] https://www.cs.ucr.edu/~csong/cs153/l/fs1.pdf
[4] https://www.cs.ucr.edu/~csong/cs153/l/fs2.pdf

35