Scribd
Upload
53 views7 pages

The Linux Streams Library Mark Veltzer

The document discusses different options for designing multi-threaded applications that use streams in Linux, noting that each FILE structure is locked to prevent race conditions, and recommending either using different streams for each thread, or manually locking streams between calls to unlocked stream functions if contention is an issue to improve performance. For transactional I/O, it suggests manually locking streams with flockfile and funlockfile and using unlocked functions between locks.

Uploaded by

Mark Veltzer
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
53 views7 pages

The Linux Streams Library Mark Veltzer

The document discusses different options for designing multi-threaded applications that use streams in Linux, noting that each FILE structure is locked to prevent race conditions, and recommending either using different streams for each thread, or manually locking streams between calls to unlocked stream functions if contention is an issue to improve performance. For transactional I/O, it suggests manually locking streams with flockfile and funlockfile and using unlocked functions between locks.

Uploaded by

Mark Veltzer
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
You are on page 1/ 7

The Linux streams library Mark Veltzer mark@veltzer.

net

Streams and performance

Since each FILE structure is held in user space it could be accessed by two threads at the same time. So the streams library locks each FILE to avoid race conditions. You must take these locks into consideration when designing a multi-threaded streams using application.

Options for design best option

Don't use the same stream in different threads If you use the standard stream operation (without the _unlocked suffix) then you get quite good performance. This is due to the fact that uncontended locks in Linux are acquired very fast in user space (futex). Or, you can go a step further and just use the *_unlocked operations and get no locking at all.

Options for design little contention

If your application is not doing lots of IO then you can just go ahead and use the same streams in different threads with little contention. Remember that streams buffer in user space and so most of the time you are causing very slight delays (memcpy). In this case you must not use the _unlocked versions of the functions.

Options for design reducing contention

If you do contend on streams then you can manage the locking yourself and get some more performance. Use flockfile(3) and funlockfile(3) and between them use the *_unlocked operations. If you have multiple IO operations between each lock and unlock then you are saving the need to acquire and release the lock multiple times.

Locking manually for transactional IO

If you have multiple threads that want to access the same files using the streams library and you want each to perform several IO operation without being interrupted then you must resort to manual locking. Use flockfile(3) and funlockfile(3) and between them use the *_unlocked operations. Note that this does not protect you from other processes accessing the same file. If you put the FILE structures and their buffers (using setbuffer) into shared memory you can use the streams library for multi-processed designs.

References

man unlocked_stdio from the Linux manual pages 'I/O on streams' from 'info libc'.

You might also like