LEPL1402 - Multithreading in Java

2023-11-30
Sequential programming

Image credits: https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial

Parallel programming

Image credits: https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial

Parallel programming

Breaking down a task into smaller sub-tasks

that can be executed independently and concurrently!

Image credits: https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial

 “P” can be:


Other computers
(distributed computing)

GPUs

CPUs

Image credits: https://levelup.gitconnected.com/basic-parallel-computing-in-go-fda50894241c

Distributed computing

Often done on cluster of computers

Image credits:
https://www.geeksforgeeks.org/an-overview-of-cluster-computing/
https://insights.profitap.com/osi-7-layers-explained-the-easy-way
Distributed computing

Often done on cluster of computers

You’ll need network programming!

Image credits:
https://www.geeksforgeeks.org/an-overview-of-cluster-computing/
https://insights.profitap.com/osi-7-layers-explained-the-easy-way
GPU programming


Graphics rendering

Matrix operations

Numerical simulations

Deep learning! Massive parallelization of simple computations
GPU programming


Graphics rendering

Matrix operations

Numerical simulations

Deep learning! Massive parallelization of simple computations

Much more versatile (I/O, data structures)

and accessible to any developer!
 Why parallel programming on CPUs?

Speed up computations!
 Why parallel programming on CPUs?

Network/cloud applications
Responsive user interfaces

Speed up computations!

Modeling by decomposing into uncoupled tasks

Image credits: https://medium.com/@geminae.stellae/introduction-to-parallel-computing-with-opencl-2ee91c30b8b6
Processes vs. threads

Process Process Process Process

Processes vs. threads

Threads

Process Process Process Process

Processes vs. threads

Threads

Difference 1:

Processes live in isolation

Threads share memory
Process Process Process Process
Processes vs. threads

Threads

Difference 1:

Processes live in isolation

Threads share memory
Process Process Process Process
Difference 2:

Threads are more lightweight
Processes vs. threads

Threads

Difference 1:

Processes live in isolation

Threads share memory
Process Process Process Process
Difference 2:

Threads are more lightweight

Full support for

processes and
threads!
This course: Multithreading
This course: Multithreading

This is the JVM executing your software

This course: Multithreading

This is the JVM executing your software

This is your main() method

This course: Multithreading

This is the JVM executing your software

This is your main() method You can launch “your” threads

Race conditions

Deadlocks

Synchronization
Threads in Java
Threads in Java

Once a concrete class implementing this Runnable interface is available,

it can be executed as a thread by instantiating the Thread class.
Example: Computing the minimum of an array

Sequential algorithm
Running it as a background task
Running it as a background task

(1) Implement Runnable

Running it as a background task

(1) Implement Runnable (2) Launch a Thread

Running it as a background task

Start the thread

(1) Implement Runnable (2) Launch a Thread

Running it as a background task

Start the thread

Wait for its completion

(1) Implement Runnable (2) Launch a Thread

Synchronization
What is the interest?
What is the interest?

User interface is frozen during computation!

What is the interest?

User interface is frozen during computation!

User interface is
still responsive!
Speeding up a computation
Speeding up a computation

Split the array in 2 parts that are processed by 2 threads

Speeding up a computation

Additional information
to be given to the threads

Split the array in 2 parts that are processed by 2 threads

Search is limited to a part of the array
Search is limited to a part of the array

Method to access to the result

Combining the partial results
Combining the partial results
Combining the partial results

Math.min()
What if 0 or 1 element in the array?
What if 0 or 1 element in the array?

This code cannot deal with the “absence of a minimum”:

What if 0 or 1 element in the array?

This code cannot deal with the “absence of a minimum”:

Let us implement a class

to represent the result OR
an absence of result!
A “Result” class for minimum and maximum
A “Result” class for minimum and maximum

Constructor if existing answer

Constructor if missing answer

A “Result” class for minimum and maximum

Constructor if existing answer

Constructor if missing answer

The corresponding Runnable

Before:
Combining partial results
Combining partial results
Frequent pattern of multithreading on arrays

Combine partial results

Frequent pattern of multithreading on arrays

For instance, also

applicable to means:

Combine partial results

Frequent pattern of multithreading on arrays

For instance, also

applicable to means:

We could split into more

than 2 parts to use
more CPU cores!

Combine partial results

Using 4 threads
Using 4 threads

If not divisible by 4
Thread pools
Motivation

We could continue adding more threads in this way (for instance, 8, 16, 32…).
But if we use, say, 100 threads, does that mean that our program will run 100 faster?
Motivation

We could continue adding more threads in this way (for instance, 8, 16, 32…).
But if we use, say, 100 threads, does that mean that our program will run 100 faster?


Obviously, the level of parallelism is limited by
the number of CPU cores. If using a CPU with
4 cores, you cannot expect a speed up of more
than 4.

There is an overhead associated with the
creation and management of a thread. On a
modern computer, creating a simple thread
(without any extra object) takes around 0.05-0.1
ms. That is approximately the time to calculate
the sum from 1 to 100,000.
A thread pool is a group of threads that are ready to work
Creating a thread pool in Java
Creating a thread pool in Java

At the end of the program or when you don’t need the thread pool anymore, you have to
shut it down explicitly to stop all its threads, otherwise the software might not properly exit:
Thread pools handle tasks producing a result

Threads Thread pools

Thread pools are waiting for tasks that produce results of a given type T!

This contrasts with Runnable that does not produce a return value.
Submitting tasks to a thread pool

Submitting one task:

Threads in thread pool are like chefs in the kitchen of a restaurant waiting for
orders. If you submit one task to the pool using the call above, one of the chefs will
take the task and it will immediately start working on it.

You can submit more tasks:

These new tasks might have to wait until one chef is ready!
Accessing the results of a thread pool

If the task is not finished yet, the method get() will wait.
This contrast with the executor.submit() method that always returns immediately.
Back to the minimum/maximum example
Back to the minimum/maximum example
Using the thread pool
Dividing the array into multiple blocks
Dividing the array into multiple blocks

Create a data structure to keep

track of all pending computations
Dividing the array into multiple blocks

Create a data structure to keep

track of all pending computations

Wait for partial results

and combine them
Dividing the array into multiple blocks

Create a data structure to keep

track of all pending computations

Make sure to process all the items

if not divisible by blockSize

Wait for partial results

and combine them
Shared memory
Threads share memory!
 Threads share memory!

Couldn’t the threads directly write to the result variable?

 Threads share memory!

Couldn’t the threads directly write to the result variable?

Using a shared variable to hold the result
Using a shared variable to hold the result
Result is not 200,000!
This is a race condition!

The line counter++ actually corresponds to 3 low-level instructions in your computer:

1) Read the value of variable counter

2) Add 1 to that value

3) Store the result in variable counter

This is a race condition!

The line counter++ actually corresponds to 3 low-level instructions in your computer:

1) Read the value of variable counter

2) Add 1 to that value

3) Store the result in variable counter

Thread 1 Thread 2
With 2 threads, the following sequence can happen: Read counter
Read counter
Add 1
Add 1
Store
Store
This is a race condition!

The line counter++ actually corresponds to 3 low-level instructions in your computer:

1) Read the value of variable counter

2) Add 1 to that value

3) Store the result in variable counter

Thread 1 Thread 2
With 2 threads, the following sequence can happen: Read counter
Read counter
Add 1
Add 1
Store
Store
Thread 1 overwrites change
made by thread 2
Mutual exclusion in Java


We must prevent that a thread changes a variable or an object while another thread
tries to use (or change) it.

Each Java object is associated with a monitor. Only the thread that holds the monitor
can enter the method.

If another thread wants the monitor, it must wait until the monitor is free. This is mutual
exclusion.

Methods are protected by the monitor if they have the synchronized keyword.

The synchronized keyword also ensure visibility of modifications between threads.
Fixing the result data structure
Final warning

Synchronized solves these concurrency issues, but reduce performance!

Image credits: https://medium.com/@kenshencu99/java-multithreading-synchronization-and-concurrency-42249ab33ebf