Scribd
Upload
249 views21 pages

Trends in Distributed File Systems: Tu Tran

This document summarizes trends in distributed file systems, including new hardware enabling larger in-memory databases, the need for scalability as systems grow larger, challenges of wide area networking and mobile users, increasing demands for fault tolerance, effects of multimedia files, and interesting research projects in areas like serverless network file services, distributed file systems for grid computing, and mobile and distributed file systems.

Uploaded by

kamal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
249 views21 pages

Trends in Distributed File Systems: Tu Tran

This document summarizes trends in distributed file systems, including new hardware enabling larger in-memory databases, the need for scalability as systems grow larger, challenges of wide area networking and mobile users, increasing demands for fault tolerance, effects of multimedia files, and interesting research projects in areas like serverless network file services, distributed file systems for grid computing, and mobile and distributed file systems.

Uploaded by

kamal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 21

Section 5.

Trends In Distributed File Systems


Professor: Dr. Zhang
CSc 8320
Advanced Operating Systems

Presented by:

Tu Tran
Outline

 New Hardware
 Scalability
 Wide Area Networking
 Mobile Users
 Fault Tolerance
 Multimedia
 Interesting Research Papers/Projects
New Hardware

 Costs continue to drop at an amazing rate

 Rapidly dropping memory costs make it possible to


have every larger databases in main memory
 Servers could have entire file system in main
memory

 Advantage:
• Gain in performance
• Simplify file system structure
New Hardware (cont’d)

• Simpler to store each file contiguously in


memory (UNIX – tree, MS-DOS – link listed)

 Disadvantage:
• Power fails, all files stored are lost. However,
current technology allows to back up
continuously files on tapes or other storage
media.

 RAM disk and file system


New Hardware (cont’d)
 Optical Disk
» Write-Once Read Many (WORM)
» CD-ROM/DVD burners
» Excellent backup and archiving method
» Increasingly cheap, but slow.

 Huge capacity networks


» Server with a main memory & a fast fiber optic network.
May not need client’s cache and server’s disk. Files are
backed up by optical disk.
New Hardware (cont’d)

» Problem: memory cache in multiprocessor is similar to


the case in which two clients are caching the same file.

» Solution: design a simple network interface which has a


bit map, one bit per cached file.

 Specialized hardware for sophisticated systems

» Real-time support

» Distributed synchronization and control


Scalability

 Distributed system size strongly affects algorithm


choice

» Working well for 100 machines means nothing for 10,000.

 Centralized algorithms do not scale well

» Opening a file requires to contact the centralized server to


record the fact that the file is open. As the system grows,
the algorithms do not work well.
Scalability (cont’d)

» Solution: partitioning the system into smaller units often


helps

 Broadcasts are a problem

» Consider CPU broadcasting one message per second


» N of these generate N interrupts at N machines
» Not a problem for N=10
» VERY problematic for N=10,000
Scalability (cont’d)

 Data structures become important with scaling

» Linear search easiest and fastest for 10


» Self abuse for even 100

 Strict semantics (Unix) are harder to implement


as systems scale

» Design Principle: use weakest semantics that make


sense
Scalability (cont’d)

» Trade off ease of programming with scalability

 Name space

» A single Unix-like file tree. How long can/should path


names get?

» Partition the tree into smaller trees.


Wide Area Networking

 Virtually all distributed system research has been


done in the context of LANs

» Considerable changes with WAN context


» Latency
» Loss
» Cost
» Interaction
Wide Area Networking (cont’d)

 WAN access of major economic importance

» WWW commerce
» Video on demand
» Distributed Virtual Environments

 Economies of scale

» IP phones
Mobile Users

 Network addressing is a big challenge - Mobile IP

» May be transparent to distributed computing level

 Often seen as highly variable communication


bandwidth

» Isolated
» Wireless
» Wired
Mobile Users (cont’d)

 Interesting effects on caching

» CODA file system claims to support mobility and


intermittent connection

 Rapidly Deployable Radio Network – RDRN[2]

» Wireless end-user and network nodes


» Steerable communication beams
» Self-organizing network structure
Fault Tolerance
 Most systems are not fault tolerant

» But the general population expects things to work


» Phone system  IP phones?

 Requires considerable redundancy

» Hardware
» Communication infrastructure
» Software
» Data – File replication
Fault Tolerance (cont’d)

 Downtimes and periodic crashes will become


less and less acceptable as computers
spread to non-specialists

» ATM machines
» Microwaves
» Phone system (IP mode)
Multimedia

 Current data files are rarely more than a few


megabytes

» Multimedia files can exceed gigabytes


» Compression clearly popular because of this and
has a fundamental affect on network requirements
and economics
Multimedia (cont’d)

 Video-on-demand

» Significant affect on network traffic


» Perhaps also on file systems
» Real-time support is interesting as well
Interesting Research Papers/Projects

 xFS: Serverless Network File Service [3]

 Distributed file systems for grid computing [4]

 Mobile and Distributed file systems [5,6,7]

 A Scalable Distributed File System [8]


References
[1] Andrew S. Tanenbaum. Distributed
Operating Systems. Prentice-Hall,
Englewood Cliffs, NJ 07632, USA, 1995

[2] Rapidly Deployable Radio Network,


http://www.ittc.ku.edu/RDRN/

[3] xFS: Serverless Network File Service,


http://now.cs.berkeley.edu/Xfs/xfs.html

[4] http://www.gridbus.org/papers/gridtech.pdf
References (cont’d)
[5] MFS: an Adaptive Distributed File System
for Mobile Hosts
http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/mfs.pdf

[6] Azzedine Boukerche, Raed Al-Shaikh, Bo Marleau, "Disconnection-


Resilient File System for Mobile Clients," lcn, pp. 608-614, The IEEE
Conference on Local Computer Networks 30th Anniversary (LCN'05)l,
2005.

http://ieeexplore.ieee.org/iel5/10397/33047/01550932.pdf?tp=&arnumb
er=1550932&isnumber=33047

[7] Haddock-FS, http://www.gsd.inesc-id.pt/~jpbarreto/Haddock-FS.html

[8] DiFFS: a Scalable Distributed File System


http://www.hpl.hp.com/techreports/2001/HPL-2001-19.pdf

You might also like