Lesson 15 Overview of FAThpfs and NTFS
Lesson 15 Overview of FAThpfs and NTFS
FAT OVERVIEW
FAT Naming Convention
Advantages of FAT
Disadvantages of FAT
HPFS OVERVIEW
Super Block
Spare Block
Advantages of HPFS
Disadvantages of HPFS
NTFS OVERVIEW
Reliability
Added Functionality
POSIX Support
Advantages of NTFS
Disadvantages of NTFS
NTFS Naming Conventions
Win FS
FAT OVERVIEW
The most common file system in the PC world is actually a family of file systems. The
basic name for this file system is FAT; the name comes from one of the main logical
structures that the file system uses: the file allocation table. This file system is the one
that was used by DOS on the first IBM PCs, and it became the standard for the PCs that
followed. Today, most PCs still use a variant of the basic FAT file system created for
those early machines over two decades ago.
The most common name of the file system, "FAT", is problematic, even though it is still
often used. The first FAT file system used 12-bit file allocation tables; this was later
expanded to 16 bits, and became the most common file system implementation for hard
disks from the late 1980s to the late 1990s. To distinguish these versions of FAT from
the 32-bit successor called FAT32, the older FAT variants are now sometimes called
FAT12 or FAT16. However, you will still hear just "FAT" used a lot; if so, you need to find
out what specifically is being referred to, if it matters in that particular context.
While designed specifically for MS-DOS and used by all of the Microsoft operating
systems that followed (in one way or another) FAT's ubiquitousness has caused support
for it to become an important characteristic for other operating systems as well. Virtually
all PC operating systems can support the basic FAT file system, even ones that bear
little resemblance to Microsoft DOS or Windows. FAT12 and FAT16 partitions are the
"least common denominator" of file systems--while they are limited in many ways, they
are the easiest file systems to use when compatibility is a concern. Even non-PC
platforms can in many cases read disks formatted in the FAT file system.
FAT is by far the most simplistic of the file systems supported by Windows NT. The FAT
file system is characterized by the file allocation table (FAT), which is really a table that
resides at the very "top" of the volume. To protect the volume, two copies of the FAT are
kept in case one becomes damaged. In addition, the FAT tables and the root directory
must be stored in a fixed location so that the system's boot files can be correctly located.
A disk formatted with FAT is allocated in clusters, whose size are determined by the size
of the volume. When a file is created, an entry is created in the directory and the first
cluster number containing data is established. This entry in the FAT table either
indicates that this is the last cluster of the file, or points to the next cluster.
Updating the FAT table is very important as well as time consuming. If the FAT table is
not regularly updated, it can lead to data loss. It is time consuming because the disk
read heads must be repositioned to the drive's logical track zero each time the FAT table
is updated.
There is no organization to the FAT directory structure, and files are given the first open
location on the drive. In addition, FAT supports only read-only, hidden, system, and
archive file attributes.
Virtual FAT (VFAT)
Microsoft introduced a new FAT file system from Windows 95. The new variation of FAT
was called Virtual FAT or VFAT for short. VFAT has several key features and
improvements compared to FAT12 and FAT16:
Long File Name Support: One of the most annoying limitations of operating systems
prior to Windows 95 was the eleven-character file name restriction. For most people,
VFAT's most important accomplishment was enabling the use of long file names by the
Windows 95 operating system and applications written for it, while maintaining
compatibility with older software that had been written before VFAT was implemented.
Improved Performance: The disk access and file system management routines for
VFAT were rewritten using 32-bit protected-mode code to improve performance. At the
same time, 16-bit code was maintained, for use when required for compatibility.
Better Management Capabilities: Special support was added for techniques like disk
locking, to allow utilities to access a disk in "exclusive mode" without fear of other
programs using it in the meantime.
Despite the new name and new capabilities, VFAT as a file system is basically the same
as FAT. Most of the new capabilities relate to how the file system is used, and not the
actual structures on the disk The only significant change in terms of actual structures is
the addition of long file names. Even here, VFAT supports these using what is basically
a hack, as opposed to anything really revolutionary.
FAT Naming Convention
FAT uses the traditional 8.3 file naming convention and all filenames must be created
with the ASCII character set. The name of a file or directory can be up to eight
characters long, then a period (.) separator, and up to a three character extension. The
name must start with either a letter or number and can contain any characters except for
the following:
."/\[]:;|=,
If any of these characters are used, unexpected results may occur. The name cannot
contain any spaces.
FAT32 uses space more efficiently. FAT32 uses smaller clusters (that is, 4-KB
clusters for drives up to 8 GB in size), resulting in 10 to 15 percent more efficient use of
disk space relative to large FAT or FAT16 drives.
FAT32 is more robust. FAT32 can relocate the root folder and use the backup
copy of the file allocation table instead of the default copy. In addition, the boot record on
FAT32 drives is expanded to include a backup copy of critical data structures. Therefore,
FAT32 drives are less susceptible to a single point of failure than existing FAT16 drives.
FAT32 is more flexible. The root folder on a FAT32 drive is an ordinary cluster
chain, so it can be located anywhere on the drive. The previous limitations on the
number of root folder entries no longer exist. In addition, file allocation table mirroring
can be disabled, allowing a copy of the file allocation table other than the first one to be
active. These features allow for dynamic resizing of FAT32 partitions.
FAT32 Compatibility Considerations
To maintain the greatest compatibility possible with existing programs, networks, and
device drivers, FAT32 was implemented with as little change as possible to the existing
Windows architecture, internal data structures, Application Programming Interfaces
(APIs), and on-disk format. However, because 4 bytes are now required to store cluster
values, many internal and on-disk data structures and published APIs have been revised
or expanded. In some cases, existing APIs will not work on FAT32 drives. Most
programs will be unaffected by these changes. Existing tools and drivers should
continue to work on FAT32 drives. However, MS-DOS block device drivers (for example,
Aspidisk.sys) and disk tools will need to be revised to support FAT32 drives.
Windows 95 OSR2, Windows 98, Windows 2000, and Windows Me are the only
Microsoft operating systems that can access FAT32 volumes. MS-DOS, the original
version of Windows 95, and Windows NT 4.0 do not recognize FAT32 partitions, and are
unable to boot from a FAT32 volume. Also, FAT32 volumes cannot be accessed
properly if the computer is started by using another operating system.
Advantages of FAT
It is not possible to perform an undelete under Windows NT on any of the supported file
systems. Undelete utilities try to directly access the hardware, which cannot be done
under Windows NT. However, if the file was located on a FAT partition, and the system
is restarted under MS-DOS, the file can be undeleted. The FAT file system is best for
drives and/or partitions under approximately 200 MB, because FAT starts out with very
little overhead.
Disadvantages of FAT
As the size of the volume increases, performance with FAT will quickly decrease.
It is not possible to set permissions on files that are FAT partitions.
HPFS OVERVIEW
The HPFS file system was first introduced with OS/2 1.2 to allow for greater access to
the larger hard drives that were then appearing on the market. Additionally, it was
necessary for a new file system to extend the naming system, organization, and security
for the growing demands of the network server market. HPFS maintains the directory
organization of FAT, but adds automatic sorting of the directory based on filenames.
Filenames are extended to up to 254 double byte characters. HPFS also allows a file to
be composed of "data" and special attributes to allow for increased flexibility in terms of
supporting other naming conventions and security. In addition, the unit of allocation is
changed from clusters to physical sectors (512 bytes), which reduces lost disk space.
Under HPFS, directory entries hold more information than under FAT. As well as the
attribute file, this includes information about the modification, creation, and access date
and times. Instead of pointing to the first cluster of the file, the directory entries under
HPFS point to the FNODE. The FNODE can contain the file's data, or pointers that may
point to the file's data or to other structures that will eventually point to the file's data.
HPFS attempts to allocate as much of a file in contiguous sectors as possible. This is
done in order to increase speed when doing sequential processing of a file.
HPFS organizes a drive into a series of 8 MB bands, and whenever possible a file is
contained within one of these bands. Between each of these bands are 2K allocation
bitmaps, which keep track of which sectors within a band have and have not been
allocated. Banding increases performance because the drive head does not have to
return to the logical top (typically cylinder 0) of the disk, but to the nearest band
allocation bitmap to determine where a file is to be stored.
Additionally, HPFS includes a couple of unique special data objects:
Super Block
The Super Block is located in logical sector 16 and contains a pointer to the FNODE of
the root directory. One of the biggest dangers of using HPFS is that if the Super Block is
lost or corrupted due to a bad sector, so are the contents of the partition, even if the rest
of the drive is fine. It would be possible to recover the data on the drive by copying
everything to another drive with a good sector 16 and rebuilding the Super Block.
However, this is a very complex task.
Spare Block
The Spare Block is located in logical sector 17 and contains a table of "hot fixes" and the
Spare Directory Block. Under HPFS, when a bad sector is detected, the "hot fixes" entry
is used to logically point to an existing good sector in place of the bad sector. This
technique for handling write errors is known as hot fixing.
Hot fixing is a technique where if an error occurs because of a bad sector, the file
system moves the information to a different sector and marks the original sector as bad.
This is all done transparent to any applications that are performing disk I/O (that is, the
application never knows that there were any problems with the hard drive). Using a file
system that supports hot fixing will eliminate error messages such as the FAT "Abort,
Retry, or Fail?" error message that occurs when a bad sector is encountered.
Note: The version of HPFS that is included with Windows NT does not support hot fixing.
Advantages of HPFS
HPFS is best for drives in the 200-400 MB range.
Disadvantages of HPFS
Because of the overhead involved in HPFS, it is not a very efficient choice for a volume
of under approximately 200 MB. In addition, with volumes larger than about 400 MB,
there will be some performance degradation. You cannot set security on HPFS under
Windows NT.
HPFS is only supported under Windows NT versions 3.1, 3.5, and 3.51. Windows NT 4.0
cannot access HPFS partitions.
NTFS OVERVIEW
When Microsoft created Windows NT, it built the operating system pretty much from
scratch--it was based on certain existing concepts, of course, but was totally different
from older Microsoft operating systems. One of the key elements of NT's architecture
was the file system created especially for the operating system, called the New
Technology File System or NTFS. NTFS enables many of the goals of Windows NT to
be realized.
NTFS is a much more complex and capable file system than any of the FAT family of file
systems. It was designed with the corporate and business environment in mind; it is built
for networking and with the goals of security, reliability and efficiency. It includes many
features, including file-by-file compression, full permissions control and attribute settings,
support for very large files, and transaction-based operation. It also does not have the
problems with cluster sizes and hard disk size limitations that FAT does, and has other
performance-enhancing features such as RAID support. Its most significant drawbacks
are increased complexity, and less compatibility with other operating systems compared
to FAT.
The NTFS file system actually has more than one version. The one used by Windows
NT is commonly called either version 1.1 or version 4.0, and has a few less features
than the newer NTFS 5.0 used by Windows 2000.
While originally created for Windows NT and now used primarily by Windows NT and its
successor, Windows 2000, limited NTFS support has also been added to other operating
systems. For example, operating systems like Linux (as well as some other UNIX
variants) and BeOS can read NTFS partitions, which helps with the interoperability of
different operating systems on the same machine. However, non-Microsoft operating
systems usually cannot write to NTFS partitions, generally because of the security
features built into NTFS by Microsoft.
From a user's point of view, NTFS continues to organize files into directories, which, like
HPFS, are sorted. However, unlike FAT or HPFS, there are no "special" objects on the
disk and there is no dependence on the underlying hardware, such as 512 byte sectors.
In addition, there are no special locations on the disk, such as FAT tables or HPFS
Super Blocks.
The goals of NTFS are to provide:
Reliability, which is especially desirable for high end systems and file servers
Reliability
To ensure reliability of NTFS, three major areas were addressed: recoverability, removal
of fatal single sector failures, and hot fixing.
NTFS is a recoverable file system because it keeps track of transactions against the file
system. When a CHKDSK is performed on FAT or HPFS, the consistency of pointers
within the directory, allocation, and file tables is being checked. Under NTFS, a log of
transactions against these components is maintained so that CHKDSK need only roll
back transactions to the last commit point in order to recover consistency within the file
system.
Under FAT or HPFS, if a sector that is the location of one of the file system's special
objects fails, then a single sector failure will occur. NTFS avoids this in two ways: first, by
not using special objects on the disk and tracking and protecting all objects that are on
the disk. Secondly, under NTFS, multiple copies (the number depends on the volume
size) of the Master File Table are kept.
Similar to OS/2 versions of HPFS, NTFS supports hot fixing.
Added Functionality
One of the major design goals of Windows NT at every level is to provide a platform that
can be added to and built upon, and NTFS is no exception. NTFS provides a rich and
flexible platform for other file systems to be able to use. In addition, NTFS fully supports
the Windows NT security model and supports multiple data streams. No longer is a data
file a single stream of data. Finally, under NTFS, a user can add his or her own userdefined attributes to a file.
POSIX Support
NTFS is the most POSIX.1 compliant of the supported file systems because it supports
the following POSIX.1 requirements:
Case Sensitive Naming:
Under POSIX, README.TXT, Readme.txt, and readme.txt are all different files.
Additional Time Stamp:
The additional time stamp supplies the time at which the file was last accessed.
Hard Links:
A hard link is when two different filenames, which can be located in different directories,
point to the same data.
Features
First, NTFS has greatly increased the size of files and volumes, so that they can now be
up to 2^64 bytes (16 exabytes or 18,446,744,073,709,551,616 bytes). NTFS has also
returned to the FAT concept of clusters in order to avoid HPFS problem of a fixed sector
size. This was done because Windows NT is a portable operating system and different
disk technology is likely to be encountered at some point. Therefore, 512 bytes per
sector was viewed as having a large possibility of not always being a good fit for the
WinFS Overview
WinFS is the code name of a Windows storage subsystem, being developed by
Microsoft for use on its future Windows Operating System. WinFS is a relational
database located on NTFS and representing itself to the operating system as a file
storage subsystem. The codename WinFS stands for Windows Future Storage.
WinFS intends to link the worlds of traditional relational databases, objects, XML, and
file systems of unstructured documents with the concept of metadata over files. Instead
of representing a file solely by directory path and filename, WinFS represents individual
domain objects - e.g. images, e-mails, address book entries, and any kind of regular file
- with indexed and searchable context and keyword information.
The underlying system is based on Microsoft SQL Server (c) database engine. WinFS
provides access to data through both traditional file-based APIs, and new object-based
approaches that take advantage of the new features. Applications that are not written to
take advantage of WinFS can access the contents of a WinFS Store through a regular
UNC path.
WinFS
A traditional file system, such as FAT or NTFS, has its contents organized in a hierarchal
directory structure and is relatively slow in searching the content by particular attributes.
If you organize your pictures in folders by Dates there would be no way to access
them by Persons, by Events etc You are to use third party custom software like
Adobe PhotoAlbum (c) to perform this task.
WinFS overcomes the hierarchy and "flattens" the storage of individual files (i.e. there is
no "hierarchy" based on directory and file names), and it enables searching for items by
their attributes (like date the photograph was taken, who or what is in the picture, what
camera was used to take the picture, etc).
WinFS as in Fig. 1 also extends this idea beyond the kinds of information that have been
traditionally stored as files on a file system. WinFS can understand any arbitrary set of
data, such as a "Photo" or an "Email" or a "Calendar". Applications today must store
these kinds of information in their own custom database; if other applications want to
gain access to this information, it's quite a lot of work because there is no applicationneutral storage mechanism for this kind of information.
WinFS Architecture
WinFS is not a physical file
system; it is built on top of NTFS,
and NTFS will remain as the
default file system.
WinFS is based on SQL Server
and its core feature is to provide a
data relations mechanism. This
means that your pictures are
related to dates, events, and
persons and so on. This allows
you to perform a search such as
"All pictures of Joe taken the last
month". This query is not possible
for a traditional file system such as
NTFS, but will be a trivial part of
WinFS. In reality, WinFS stores
are simply SQL Server database
Fig. 1: NTFS vs Win FS file structure
files (.mdf files) with the FILESTREAM feature enabled. These files are stored in
secured folder named "System Volume Information" placed into the volume root, in
folders under the folder "WinFS" with names of GUIDs of these stores.
WinFS Technology
From a technology standpoint, WinFS is made up of five components:
Core WinFS
Core WinFS is made up of the core services that you would expect from a file system.
Think of Core WinFS as the fundamentals, which includes operations and file system
services. Some examples here are security, manageability, Win32 file access support,
import/export, quotas, and so on.
Data Model
Moving beyond the core services, the Data Model provides some of the technical
innovations including the basic item structure, relationships, and the ability to extend
both items and relationships.
Schemas
Without built-in schemas, WinFS would be no better than the existing file system, since
WinFS would not understand your data in richer ways or provide a more structured way
to handle your data's metadata. WinFS schemas include schemas for your everyday
information such as documents, e-mail, appointments, tasks, media, audio video, and
more. WinFS also includes system schemas that include configuration, programs, and
other system-related data.
Services
Synchronization and rules fall into the services
area of WinFS. These technologies "sit on top" of
WinFS to provide you with capabilities that
extend beyond the fundamentals of the system.
Synchronization will enable you to synchronize
WinFS systems across a network, as well as
build synchronization adapters to synchronize
WinFS to other systems. For example, you may
want to synchronize contact information from
your CRM system to WinFS so that you can
relate that data to other data in WinFS or work
with that data offline through WinFS.
Synchronization adapters can be bi-directional,
so any changes made to the data in WinFS can
be synchronized back to the other partner
system.
Fig. 2: Features of WinFS
APIs
WinFS as inFig. 2 includes a rich API that is part of the overall WinFX programming
model in Longhorn or Windows vista. Through the WinFS API, you can program the
different building blocks of the WinFS system including data operations, rules,
synchronization, and the data model. The API that will provide programmatic access is
being coded in C# or Managed C++, so .NET applications can access it, but the actual
base code that will execute the functions of WinFS is coded written in C++. This is to
provide WinFS with access to the Kernel of the operating system which in turn provides
root access to disk management APIs which are part of NTFS.
WinFS Types and Extensibility
WinFS's type system is tied to the .NET Framework's concept of classes and
inheritance.
WinFS Type Extensibility allows developers to extend WinFS by providing a schema that
defines new data types and associations. These schemas are XML files, though they are
not W3C XML Schema Definition, presumably because there are incompatibilities
between base types (e.g. WinFS's "integer" data type is nullable, whereas XSD's integer
type is not). Custom schemas can be installed on a per-application basis (in situations
where an application wants to store its data in WinFS, but not share the structure of that
data with other applications), or they can be made available across the system.
WinFS Synchronization
WinFS includes support for synchronization with external data. WinFS Sync provides
developers with a set of API's that allow for synchronizing with other WinFS stores (i.e.
on other computers), as well as with connected devices like a mobile phone, or with Web
services such as a web-mail server. WinFS Sync handles all the detail-work related to
conflict detection, resolution, and error-reporting. This should allow Sync adapter
developers to focus on mapping their data to WinFS as accurately as possible.
UNIX / Linux File Systems
All file systems perform the same basic functions, based on their fundamental goals: the
intelligent organization of data and efficient control of and access to that data. As a
result, most of them resemble each other to some extent, even if they are also different
from each other in important ways. Even if two PC file systems differ greatly in terms of
their internal architecture and structures, they can be made to resemble each other
closely in their outward appearance. For example, Windows NT will support both FAT
and NTFS partitions, which are totally different in terms of their internal structures, but
have virtually the same interface for the user (and for software applications).
UNIX has been around for many decades, making it the oldest of all file systems used
on PC hardware. UNIX file systems are also probably the most different from the other
file systems used on PC, both internally and externally (referring to how the user
accesses the file system). While most Windows users are accustomed to the Explorertype interface for managing files and folders, UNIX files are usually managed with
discrete text commands--similar to how DOS works (in fact, many principles of the FAT
file system are based on UNIX.). There are graphical UNIX shells as well, of course, but
many UNIX users never use them.
The more important differences, however, are internal. UNIX file systems are designed
not for easy use, but for robustness, security and flexibility. UNIX file systems offer the
following features, and have for many years:
Excellent expandability, and support for large storage devices.
Directory-level and file-level security and access controls, including the ability to control
which users or groups of users can read, write or execute a file.
Very good performance and efficient operation.
The ability to create "flexible" file systems containing many different devices, to combine
devices and present them as a single file system, or to remotely mount other storage
devices for local use.
Facilities for effectively dealing with many users and programs in a multitasking
environment, while requiring a minimum of administration.
Ways to create special constructs such as logically linked files.
Reliability and robustness features such as journaling and support for RAID.
If these features sound similar to those of NTFS, that's because UNIX and Windows
NT/2000 now compete for much of the same market, so NTFS was given most of the
capabilities that UNIX has. There are many other features as well, which differ from one
implementation to another--there is no single "UNIX file system", any more than there is
a single "UNIX operating system". Each UNIX variant (including the popular Linux for the
PC, which itself has many different flavors) has a slightly different file system, though
they are of course very similar to each other, and it is usually possible for different UNIX
implementations to read each other's files.
UNIX file systems were one of the first (if not the first) to use the hierarchical directory
structure, with a root directory and nested subdirectories. (Most of us are familiar with
this from using it with the FAT file system, which works the same way). One of the key
characteristics of UNIX file systems is that virtually everything is defined as being a file-regular text files are of course files, but so are executable programs, directories, and
even hardware devices are mapped to file names. This provides tremendous flexibility to
programmers and users, even if there is a bit of a learning curve at first.
Since few PC users run UNIX on their own machines, it's unlikely that you will ever
actually use a UNIX file system directly. However, you may find yourself using a UNIX
file system if you run a web site, for example, so understanding the basics of how UNIX
works is a good idea. There are literally thousands of other sites and pages about UNIX
on the Internet!
RECAP
In this lesson you had learnt
FAT is the most common file system in the PC world is actually a family of file
systems.
FAT12, FAT 16 and FAT32 are the variations. FAT32 supports drives up to 2
terabytes in size.
Windows NT old versions used HPFS. Under HPFS, directory entries hold more
information than under FAT.
Windows NT 4.0 used NTFS file version 4.0
Windows 2000/2003 use NTFS 5.0 version. NTFS 5.0 includes several new and
useful features like encryption, compression, disk quota etc. compared to the
older NTFS versions used in Windows NT 4.0 and earlier.
WinFS is the code name of a Windows storage subsystem, being developed by
Microsoft for use on its future Windows Operating System.
WinFS's type system is tied to the .NET Framework's concept of classes and
inheritance.