THREADS ON MULTI-CORE PROCESSOR

In the previous post, I have discussed about threads in a single core processor. Today we continue
our discussion into threads on multicore processors.

Let us revise an important topic “Concurrency”

When we have multiple processes but only a single CPU, the OS makes it feel like all the processes
are running simultaneously by alternating CPU time between those processes at incredible speed.
It is useful because, if one task takes a much longer time to complete than other which takes a few
milli-seconds, concurrency allows us to write programs in such a way that the shorter task does
not need to wait for the longer task to complete.

If task is waiting for a file read or an I/O response, it can not use the CPU time. So instead of
making the CPU sit idle, the OS switches the CPU time to another task which can use the unused
CPU time.
Therefore, threads are simply a way to tell, that multiple tasks within a same process can run
concurrently. They enable application to perform multiple tasks at the same time.
For example, in an instant messaging and VoIP social platform, we need to display the user
interface on the screen, while listening to the user key strokes while monitoring for any incoming
text or voice messages, while performing a grammar and abuse checking, while supporting any
external services like Spotify, YouTube and Twitch. In order to provide a good user experience,
none of these tasks should wait for each other to complete.

However, implementing concurrency is difficult. If the number of concurrent tasks increases too
much, the system won’t feel smooth anymore. Even if the CPU alternates swiftly between each
task, there comes a point where there are too many tasks and it takes too longer time for each task
to regain the CPU time.

To address this there are three possible solutions:

1. To make the CPU run faster:
If the CPU can handle more work in the same amount of time, tasks can regain the CPU
time more quickly. But this solution is not perfect. Because, if we keep increasing the
number of concurrent tasks, there comes a point again where there are too many tasks and
it takes too longer time for each task to regain the CPU time. Also, increasing the CPU
speed is not as easy as it looks. It again has to deal with lot of rippled complexities within
the CPU.
2. CPU Scheduling:
The second way to schedule the CPU accesses in a cleverer way. This is a very complex
solution that improves the concurrency. It can be done either in software or in hardware.
The main goal of the scheduler is to maximize the throughput, minimize the wait time and
reduce the latency. Mac OS 9 uses a cooperative scheduling for threads, where one process
controls multiple cooperative threads and also provides preemptive scheduling for
multiprocessing tasks.
 3. Add more processors:
This is a brute force approach. If a single processor cannot do all the tasks, just add more
processors.

This can be dine in several ways. By adding more CPU sockets to the same mother board allowing
for multiple physical CPU.

Or by allowing multiple processors within a single packaging or chip, specifically known as Multi-
core Processors.
Or by allowing multiple multi-core chips on a same mother board.

Cores on the same processor share both processor internal resources (L3 cache, system bus,
memory controller, I/O controllers, and interconnects) and processor external resources (main
memory, I/O devices, and networks). Each core appears as a separate processor for the operating
system.
Consider an application with 8 threads. On a system with single core, concurrency merely means
that the execution of the threads will be interleaved over time. Because there is only on processor
or core that can execute the thread and only one thread can be executed at a time.

But on a system with multiple cores, concurrency goes to a next level. Here some threads can truly
run at the same time as there are multiple cores and multiple threads can be executed at the same
time. The system can assign each thread to a separate core. In other words, here we are dealing
with parallelism.
Concurrency Vs Parallelism
Concurrency: It refers to things that appear to happen at the same time. But which may occur
serially. Concurrent operations are arbitrarily interleaved so that they make progress
independently.
• Two or more threads may or may bot be executing the code at the same time, but they are
in the middle of it.
Parallelism: It describes concurrent sequences that proceed simultaneously. True parallelism can
only occur in multiprocessor, but concurrency can occur on both uniprocessor and multiprocessors.
Parallelism requires that a program be able to perform two or more computations at once, while
concurrency requires only that the programmer be able to pretend that more than two things can
happen at once. That is, concurrency is the illusion of parallelism.

• Two or more threads actually run at the same time on different CPUs (see Fig. 1(b)). They
are of course also running concurrently.

Concurrency Parallelism
All modern OS consider threads rather than processes aa a basic unit of executions. With multicore
processors, threads within a task can take full advantage of parallelism.
Here we have to consider 2 important things:

Performance:
If we can truly run more than one task at a time, we could reduce the total execution time of the
process significantly compared to running them sequentially on a system.
Types of Parallelism:

Let us take an example to understand this. Consider a large array of numbers and we have to find
all the prime numbers in that array. This problem requires us to iterate through the entire array
checking whether each number is prime. The key to understand here is, in order to decide a
number is prime or not, we do not need to wait on the result of the previous numbers to be
checked. So, each number can be processed independently.
If we have a quadcore processor, we can divide the array into 4 equal parts and assign each part
of array to each core which perform the same operation on all the numbers. This is an example of
data parallelism.

Now, let’s say we are assigned with four different tasks. Finding the Lowest Value, Highest
Value, Arithmetic Mean and Check if contains 101.
Calculating these operations requires access to the entire data set array of number to compute the
results. In other words, the result of current number is dependent on the result of other numbers.
Therefore, we do need to wait on the result of the previous numbers to be checked.
Here, it doesn’t make sense if we split the array into 4 equal parts. So instead, we assign each
task to each different core which have access to the entire data set. This is an example of task
parallelism.

References:
1. https://www.youtube.com/watch?v=5sw9XJokAqw, Core Dumped.
2. https://www.secs.oakland.edu/~llamocca/Courses/ECE4772/Notes%20-%20Unit%203.pdf,
ECE-4772/5772: High-performance Embedded Programming, Electrical and Computer
Engineering Department, Oakland University, Oakland, USA.
3. https://insights.sei.cmu.edu/blog/multicore-processing/, Donald Firesmith, Carnegie Mellon
University, Pittsburgh, USA.