Log-structured file system

A log-structured filesystem is a file system in which data and metadata are written sequentially to a circular buffer, called a log. The design was first proposed in 1988 by John K. Ousterhout and Fred Douglis and first implemented in 1992 by John K. Ousterhout and Mendel Rosenblum.

Rationale

Conventional file systems tend to lay out files with great care for spatial locality and make in-place changes to their data structures in order to perform well on optical and magnetic disks, which tend to seek relatively slowly.

The design of log-structured file systems is based on the hypothesis that this will no longer be effective because ever-increasing memory sizes on modern computers would lead to I/O becoming write-heavy because reads would be almost always satisfied from memory cache. A log-structured file system thus treats its storage as a circular log and writes sequentially to the head of the log.

This has several important side effects:

LogFS

LogFS is a Linux log-structured and scalable flash file system, intended for use on large devices of flash memory. It is written by Jörn Engel and in part sponsored by the CE Linux Forum.

LogFS is included in the mainline Linux kernel and was introduced in version 2.6.34, released on May 16, 2010.

History

As of November 2008, LogFS was mature enough to pass its entire test suite, and was subsequently included in the mainline Linux kernel, marked as 'experimental', in version 2.6.34 released on May 16, 2010.

Operation

LogFS was motivated by the difficulties of JFFS2 with larger flash-memory drives. LogFS stores the inode tree on the drive; JFFS2 does not, which requires it to scan the entire drive at mount and cache the entire tree in RAM. For larger drives, the scan can take dozens of seconds and the tree can take a significant amount of main memory. LogFS avoids these penalties, but it does do more work while the system is running and uses some of the drive's space for holding the inode tree.