Study Material

Department of Computer Science & Engineering

Semester /
Course Code : BCS303 III / II
Year :

Academic
Course Title : Operating Systems 2024-25
Year :
Module 2

Multi threaded Programming: overview , Multithreading

models ;Thread Libraries ; Threading issues

2
 Thread Vs Process
Important Terms in Computer Science

 Process is the program under action

 Thread is the smallest segment of instruction that can be handled independently by

schedular
Overview
 Overview
 Notion of a thread :A fundamental unit of CPU utilization that forms the basis of multithreaded
computer systems
 Thread : A Basic unit of CPU Utilization that comprises of
 Thread ID
 Program counter
 Register set
 Stack
 It shares with other threads belonging to same process its code section , data section and operating
system resources such as open files and signals
 Heavy weight process has a single thread of control (Single-Threaded)
 If process has multiple threads of control ,it can perform more than one task at a time (Multi-threaded)
 Motivation
 Most modern applications are multithreaded
 An application implemented as a separate process with several threads of control
 For example.
 A Web browser might have
 one thread display images or text
 another thread retrieves data from the network
 A word processor may have
 a thread for displaying graphics,
 another thread for responding to keystrokes from the user,
 and a third thread for performing spelling and grammar checking in the background.
 Motivation
 In certain situations, a single application may be required to perform several similar tasks.
 For example, a Web server accepts client requests for Web pages, images, sound, and so forth.
 A busy Web server may have several clients concurrently accessing it. If the Web server ran as a
traditional single-threaded process, it would be able to service only one client at a time, and a client
might have to wait a very long time for its request to be serviced.
 One solution
 The server run as a single process that accepts requests. When the server receives a request, it
creates a separate process to service that request. In fact, this process-creation method was in
common use before threads became popular. Process creation is time consuming and resource
intensive, however. If the new process will perform the same tasks as the existing process, why
incur all that overhead?
 Efficient Solution
 Use one process that contains multiple threads. If the Web-server process is multithreaded, the
server will create a separate thread that listens for client requests. When a request is made, rather
than creating another process, the server will create a new thread to service the request and
resume listening for additional requests.
Single and Multithreaded Processes
Multi-Threaded Server Architecture
 Benefits
 Responsiveness –
 Multithreading an interactive application may allow a program to continue running even if part
of it is blocked or is performing a lengthy operation, thereby increasing responsiveness to the
user.
 For instance a multithreaded Web browser could allow user interaction in one thread while an image
was being loaded in another thread.
 Resource Sharing –
 Processes may only share resources through techniques such as shared memory or message passing.
 Such techniques must be explicitly arranged by the programmer.
 However, threads share the memory and the resources of the process to which they belong by
default.
 The benefit of sharing code and data is that it allows an application to have several different threads
of activity within the same address space.
 Benefits
 Economy –
 Allocating memory and resources for process creation is costly.
 Because threads share the resources of the process to which they belong, it is more economical to
create and context-switch threads.
 It is much more time consuming to create and manage processes than threads.
 In Solaris for example, creating a process is about thirty times slower than is creating a thread, and
context switching is about five times slower.
 Scalability –
 The benefits of multithreading can be greatly increased in a multiprocessor architecture, where
threads may be running in parallel on different processors.
 A single-threaded process can only run on one processor, regardless how many are available.
Multithreading on a multi CPU machine increases parallelism.
 Multicore Programming
 A recent trend in system design has been to place multiple computing cores on a single chip, where
each core appears as a separate processor to the operating system.
 Multithreaded programming provides a mechanism for more efficient use of multiple cores and
improved concurrency.
 Consider an application with four threads.
 On a system with a single computing core, concurrency merely means that the execution of the
threads will be interleaved over time, as the processing core is capable of executing only one
thread at a time.
 On a system with multiple cores, however, concurrency means that the threads can run in
parallel, as the system can assign a separate thread to each core.
 The trend towards multicore systems has placed pressure on system designers as well as application
programmers to make better use of the multiple computing cores.
 Designers of operating systems must write scheduling algorithms that use multiple processing cores to
allow the parallel execution.
 For application programmers, the challenge is to modify existing programs as well as design new
programs that are multithreaded to take advantage of multicore systems.
Multicore Programming
 Multicore Programming
 In general, five areas present challenges in programming for multicore systems:
 Dividing activities
 This involves examining applications to find areas that can be divided into separate,
concurrent tasks and thus can run in parallel on individual cores.
 Balance
 While identifying tasks that can run in parallel, programmers must also ensure that the tasks
perform equal work of equal value.
 In some instances, a certain task may not contribute as much value to the overall process as other
tasks; using a separate execution core to run that task may not be worth the cost.
 Data splitting
 Just as applications are divided into separate tasks, the data accessed and manipulated by the
tasks must be divided to run on separate cores.
 Multicore Programming
 Data dependency
 The data accessed by the tasks must be examined for dependencies between two or more
tasks.
 In instances where one task depends on data from another, programmers must ensure that the
execution of the tasks is synchronized to accommodate the data dependency.
 Testing and debugging
 When a program is running in parallel on multiple cores, there are many different execution
paths.
 Testing and debugging such concurrent programs is inherently more difficult than testing and
debugging single-threaded applications.
 Because of these challenges, many software developers argue that the advent of multicore systems will
require an entirely new approach to designing software systems in the future.
Multithreading Models
 Multithreading Models
 Support for threads may be provided either at the user level, for user thread or by the kernel, for kernel
threads.
 User threads are supported above the kernel and are managed without kernel support, whereas kernel
threads are supported and managed directly by the operating system.
 Ultimately, a relationship must exist between user threads and kernel threads.
 Three common ways of establishing such a relationship.
 Many-to-One
 One-to-One
 Many-to-Many
 Many-to-One Model
 Many user-level threads mapped to single kernel thread
 Thread management is done by the Thread library in user space so it is efficient
 One thread blocking system call causes entire process to block
 Multiple threads may not run in parallel on mutiprocessor system because only
one thread can access the kernel at a time.
 This means that all the user-level threads share the same kernel-level thread, and
the kernel is not aware of the individual user-level threads.
 Example:
 The web browser creates multiple user-level threads to perform tasks such as
rendering web pages, executing JavaScript code, and handling user input.
However, all these user-level threads are mapped to a single kernel-level thread,
which is responsible for managing memory, handling I/O operations, and
scheduling processes.
 Solaris Green Threads, GNU Portable Threads
 One-to-One Model
 Each user-level thread maps to kernel thread. This means that each user-
level thread has its own kernel-level thread, and the kernel is aware of
each individual user-level thread.
 Provide more concurrency than many to one model by allowing another
thread to run in parallel on multiprocessors
 Drawback is
 creating a user-level thread creates a kernel thread can burden the
performance of an application.
 Number of threads per process sometimes restricted due to overhead.
 Example:
 The database server creates multiple user-level threads to handle different
database queries. Each user-level thread is mapped to a separate kernel-
level thread, which is responsible for managing memory, handling I/O
operations, and scheduling processes.
 Windows, Linux
 Many-to-Many Model
 Allows many user level threads to be mapped to smaller or equal number of
kernel threads.
 This means that each user-level thread can be mapped to a different kernel-
level thread, and the kernel is aware of each individual user-level thread.
 The number of kernel threads may be specific to either a particular
application or particular machine (An application may be allocated more
kernel threads on a multiprocessor than on a uniprocessor)
 Developer can create as many user threads as necessary and the
corresponding kernel thread can run in parallel on a multiprocessor.
 When a thread performs a blocking system call ,the kernel can schedule
another thread for execution.
 Example:
 The web server creates multiple user-level threads to handle different HTTP
requests. Each user-level thread is mapped to a different kernel-level thread,
which is responsible for managing memory, handling I/O operations, and
scheduling processes.
 Two-level Model
 Variation to many to many model is Two level model
 Variation to M:M model which multiplexes many user level threads to a
smaller or equal number of kernel threads but also allows a user thread to
be bound to kernel thread
 Examples
 IRIX
 HP-UX
 Tru64 UNIX
 Solaris 8 and earlier
Thread Libraries
 Thread Libraries
 Thread library provides programmer with API for creating and managing threads
 Two primary ways of implementing
 First Way:
 Provide library entirely in user space with no kernel support .
 All code and data structures for library exists in user space
 Invoking a function in library results in a local function call in user space and not a system call
 Second Way
 Implement kernel-level library supported by the OS
 Code and data structures for library exist in kernel space
 Invoking a function in the API for library results in a system call to the kernel
 Thread Libraries
 Three main Thread libraries in use:
 POSIX Pthreads
 the threads extension of the POSIX standard, may be provided as either a user- or kernel-level
library
 Win 32
 a kernel-level library available on Windows systems.
 Java
 allows threads to be created and managed directly in Java programs.
 the Java thread API is generally implemented using a thread library available on the host system.
 Pthreads
 Pthreads refers to POSIX standard (IEEE 1003.1c) defining an API for thread creation and
synchronization
 Specification for thread behaviour, not implementation
 API specifies behavior of the thread library, implementation is up to development of the library
 Numerous system implement the Pthread specification including Solaris, Linux, Mac OS X and
Tru64UNIX
 All Pthreads programs must include the pthread. h header file.
 The statement pthread_t tid declares the identifier for the thread we will create.
 Each thread has a set of attributes, including stack size and scheduling information.
 A separate thread is created with the pthread_create () function call.
 In addition to passing the thread identifier and the attributes for the thread, we also pass the name of
the function where the new thread will begin fexecution. Last, we pass the integer parameter that
was provided on the command line, argv [1].
 Win32 threads
 Technique for creating threads using Win32 thread library is similar to Pthread technique.
 Include the windows . h header file when using the Win32 API.
 Threads are created in Win32 API using createThread() same as process
 A set of attributes for thread is passed to this function
 These attributes include security information ,size of stack and a flag that can be set to indicate if
thread is to start in a suspended state
 Default values used for these attributes which do not initially set to the thread running state by CPU
schedular
 Java Threads
 Threads are the fundamental model of program execution in a Java program, and the Java language and
its API provide a rich set of features for the creation and management of threads.
 All Java programs comprise at least a single thread of control-even a simple Java program consisting of
only a main() method runs as a single thread in the JVM.
 Java threads are managed by the JVM
 There are two techniques for creating threads in a Java program.
 One approach is to create a new class that is derived from the Thread class and to override its run()
method.
 An alternative-and more commonly used technique is to define a class that implements the Runnable
interface.
 Java Threads
 Runnable interface defined as follows :

 Whereas a class implements Runnable ,it must define run() method .The code implementing run()
method is what runs a separate thread.
 Creating a Thread object does not specifically create the new thread; rather, it is the start() method that
creates the new thread. Calling the start() method for the new object does two things:
 It allocates memory and initializes a new thread in the JVM.
 It calls the run() method, making the thread eligible to be run by the JVM.
Threading Issues
 Threading Issues
 The fork() and exec() system calls
 Cancellation
 Signal Handling
 Thread pool
 Thread- Specific Data
 Scheduler Activations
 The fork() and exec() System Calls
 fork() system call is used to create a separate duplicate process
 Semantics of fork() and exec() call changes in a multithreaded program
 If one thread in a program calls fork(), does new process duplicates all threads or is the new process
single threaded?
 Some UNIX system have two versions of fork
 That duplicate all threads
 Duplicate only thread that invoked fork() system call
 exec() usually works as normal – replace the running process including all threads.
 Which of the two versions of fork() to use depends on the application.
 If exec() is called immediately after forking, then duplicating all threads is unnecessary, as the
program specified in the parameters to exec() will replace the process.
 In this instance, duplicating only the calling thread is appropriate.
 If, however, the separate process does not call exec () after forking, the separate process should
duplicate all threads.
 Cancellation
 Task of Terminating a thread before it has completed
 If multiple threads are concurrently searching through a database ,if one thread returns the result
remaining threads might be cancelled
 Thread to be canceled is target thread
 Two general approaches of cancellation
 Asynchronous cancellation one thread immediately terminates the target thread
 Deferred cancellation allows the target thread to periodically check if it should be
cancelled ,allowing it an opportunity to terminate itself
 Cancellation
 Difficulty in cancellation :
 Resources have been allocated to cancelled thread
 Thread cancelled in the middle of updating data it is sharing with other threads .
 This becomes especially troublesome with asynchronous cancellation.
 Often, the operating system will reclaim system resources from a canceled thread but will not
reclaim all resources.
 Therefore, canceling a thread asynchronously may not free a necessary system-wide resource.
 With deferred cancellation, in contrast,
 one thread indicates that a target thread is to be canceled, but cancellation occurs only after the
target thread has checked a flag to determine whether or not it should be canceled.
 The thread can perform this check at a at point which it can be canceled safely
 Signal Handling
 Signals are used in UNIX systems to notify a process that a particular event has occurred.
 Signal may be received either synchronously or Asynchronously depending on source of and the
reason for the event being signaled
 All signals whether synchronous or asynchronous follow the same pattern
1. A Signal is generated by the occurrence of a particular event
2. A generated Signal is delivered to a process
3. Once delivered ,the Signal must be handled.
 Synchronous Signal include Illegal memory access and Division by 0.
 If running program performs these actions a signal is generated
 Synchronous signal delivered to same process that performed operation that caused the signal
 Signals is generated by event external to running process that process receives signal Asynchronously.
 Examples of such signals include terminating a process with specific keystrokes (such as <control>
<C>) and having a timer expire.
 Typically, an asynchronous signal is sent to another process.
 Signal Handling
 Signal is handled by one of two signal handlers:
1. A Default Signal Handler
2. A User-defined Signal Handler
 Every signal has default handler that is run by kernel when handling that signal.
 This default action can be overriden by a user defined handler that is called to handle the signal
 Signals are handled in different ways.
 Some signals (such as changing the size of a window) are simply ignored;
 others (such as an illegal memory access) are handled by terminating the program.
 Signal Handling
 Handling signals in single-threaded programs is straightforward:
 Signals are always delivered to a process.
 However, delivering signals is more complicated in multithreaded programs, where a process may have
several threads. Where should a signal be delivered for multi-threaded?
 Deliver the signal to the thread to which the signal applies
 Deliver the signal to every thread in the process
 Deliver the signal to certain threads in the process
 Assign a specific thread to receive all signals for the process
 Method for delivering a signal depends on type of signal generated
 Thread Pools
 Creating a separate thread is certainly superior to creating a separate process.
 The first issue concerns the amount of time required to create the thread prior to servicing the request,
together with the fact that this thread will be discarded once it has completed its work.
 The second issue is more troublesome: if we allow all concurrent requests to be serviced in a new
thread, we have not placed a bound on the number of threads concurrently active in the system.
Unlimited threads could exhaust system resources, such as CPU time or memory.
 One solution is to use a Thread Pool.
 Thread pool is to create a number of threads at process start up and place them into a pool where they
can sit and wait for work.
 When a server receives a request ,it awakens a thread from this pool-if one is available and passes it the
request for service.
 Once thread completes its service ,it returns to the pool and awaits more work
 If the pool contains no available thread ,the server waits until one becomes free
 Thread Pools
 Thread pool offers these Advantages:
 Servicing a request with an existing thread is usually faster than waiting to create a thread
 Thread pool limits that number of threads that exists at any one point . This is particularly important
on systems that can not support a large number of concurrent threads.
 The number of threads in the pool can be set heuristically based on factors such as
 the number of CPUs in the system,
 the amount of physical memory,
 and the expected number of concurrent client requests.
 More sophisticated thread-pool architectures can dynamically adjust the number of threads in the pool
according to usage patterns. Such architectures provide the further benefit of having a smaller pool-
thereby consuming less memory-when the load on the system is low.
 Thread-Specific Data
 Threads belonging to a process share the data of the process.
 Indeed, this sharing of data provides one of the benefits of multithreaded programming.
 However, in some circumstances, each thread might need its own copy of certain data.
 We will call such data thread-specific data.
 For example, in a transaction-processing system,
 we might service each transaction in a separate thread.
 Furthermore, each transaction might be assigned a unique identifier.
 To associate each thread with its unique identifier, we could use thread-specific data.
 Most thread libraries-including Win32 and Pthreads-provide some form of support for thread-specific
data. Java provides support as well.
 Scheduler Activations
 The final issue to be considered with multithreaded programs concerns
communication between the kernel and the thread library, which may be
required by the many-to-many and two-level models.
 Such coordination allows the number of kernel threads to be dynamically
adjusted to help ensure the best performance.
 Many systems implementing either the many-to-many or the two-level model
place an intermediate data structure between the user and kernel threads. This
data structure-typically known as a lightweight process, or LWP.
 To the user-thread library, the LWP appears to be a virtual processor on
which the application can schedule a user thread to run.
 Each LWP is attached to a kernel thread, and it is kernel threads that the
operating system schedules to run on physical processors.
 If a kernel thread blocks (such as while waiting for an I/0 operation to
complete), the LWP blocks as well.
 Up the chain, the user-level thread attached to the LWP also blocks.
 Scheduler Activations
 An application may require any number of LWPs to run efficiently.
 Consider a CPU-bound application running on a single processor.
 In this scenario, only one thread can run at once, so one LWP is sufficient.
 An application that is I/O intensive may require multiple LWPs to execute, however.
 Typically, an LWP is required for each concurrent blocking system call.
 Suppose, for example, that five different file-read requests occur simultaneously.
 Five LWPs are needed, because all could be waiting for I/0 completion in the kernel.
 If a process has only four LWPs, then the fifth request must wait for one of the LWPs to return from
the kernel.
 One scheme for communication between the user-thread library and the kernel is known as scheduler
activation.
 Scheduler Activations
 It works as follows:
 The kernel provides an application with a set of virtual processors (LWPs), and the application can
schedule user threads onto an available virtual processor.
 Furthermore, the kernel must inform an application about certain events.
 This procedure is known as an Upcalls.
 Upcalls are handled by the thread library with an upcall handler and upcall handlers must run on a
virtual processor.
 One event that triggers an upcall occurs when an application thread is about to block.
 In this scenario, the kernel makes an upcall to the application informing it that a thread is about to
block and identifying the specific thread.
 The kernel then allocates a new virtual processor to the application.
 Scheduler Activations
 It works as follows:
 The application runs an upcall handler on this new virtual processor, which saves the state of the
blocking thread and relinquishes the virtual processor on which the blocking thread is running.
 The upcall handler then schedules another thread that is eligible to run on the new virtual processor.
 When the event that the blocking thread was waiting for occurs, the kernel makes another upcall to
the thread library informing it that the previously blocked thread is now eligible to run.
 The up call handler for this event also requires a virtual processor, and the kernel may allocate a
new virtual processor or preempt one of the user threads and run the upcall handler on its virtual
processor.
 After marking the unblocked thread as eligible to run, the application schedules an eligible thread to
run on an available virtual processor.