Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

15.1 Processors, Parallel Processing and Virtual Machines (Part 1 out of 3)

There are four basic computer architectures.

1. Single Instruction Single Data (SISD)

- It refers to the Von Neumann computer architecture.

- It is a computer that does not have the ability for parallel processing.
- All single processor systems are SISD.

- It can process only 1 stream of instruction and only 1 stream of data.

- Instructions are executed sequentially by a single PE.
- ALU is under the control of a single CU.
- It is used in micro-controllers / embedded systems. For example, the microprocessor used to control a washing
machine.

Page 1 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

2. Single Instruction Multiple Data (SIMD)

- A single instruction is executed on multiple data streams.

- The computer has several ALUs. Each ALU executes the same instruction but on different data.

- It can process only 1 stream of instruction and multiple streams of data.

- Multiple PEs work under the control of a single CU.

- The execution of instructions is synchronous. This means that all PEs must finish executing one instruction

before proceeding to the next one. The PEs are 'lock-stepped' together.

- It is used to process 3D graphics in video games.

- It is used in GPUs that contain vector processors / array processors.

Page 2 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

3. Multiple Instruction Single Data (MISD)

- It can process multiple streams of instruction and only 1 stream of data.

- Multiple PEs work under the control of multiple CUs.

- Each CU handles 1 instruction stream which is processed by its corresponding PE.

- It uses a single data stream that passes through each PE.

- This architecture is rarely used because the concept of single data stream executing on multiple PEs is not very

common.

- It is used by real-time computers that need to be fault tolerant. For example, it is used in flight control systems

where fault detection is critical. Several processors execute the same data. All the results must then match.

Otherwise, the unit is replaced (Also known as N-version programming).

Page 3 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

4. Multiple Instruction Multiple Data (MIMD)

- There are several processors. Each processor executes different instructions drawn from a common pool.
- Each processor operates on different data drawn from partitions of a shared memory.

- It can process multiple streams of instruction and multiple streams of data.

- Multiple PEs work under the control of multiple CUs and with multiple memory partitions.

- The execution of instructions is asynchronous. This means that a PE can finish executing one instruction and

proceed to the next one while the other PEs are still executing the first instruction. The PEs are not 'lock-stepped'

together.

- It is used by most parallel computer systems such as desktop computers and smart phones.

Page 4 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

Massively Parallel Computers

- A massively parallel computer consists of a large number of processors working simultaneously on the same
program. The processors communicate via a messaging interface.

- Massive means using a large number (hundreds or even thousands) of processors.

- Parallel means able to perform a set of coordinated computations at the same time.

Example: IBM Blue Gene supercomputer. This computer has 4,098 processors and is used for weather
forecasting and climate change predictions.

There are a number of hardware and software issues that must be overcome for a massively parallel computer is
to function successfully.

Hardware Issue

- There needs to be communication between the different processors.

- Each processor needs to be linked to every other processor so that processed data can be transferred from one
processor to another.

- Hence, it is a challenging topology.

Software Issue

- There needs to be an algorithm which will allow data to be processed by multiple processors at the same time.

- The algorithm must be designed taking data dependencies into account.

- Hence, it is a challenging program to write.

Page 5 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

Parallel Processing

- Parallel processing means using more than 1 CPU simultaneously to execute a program.

- Ideally, parallel processing makes a program run faster because there are more CPUs available.

- Hence, the work can be divided among the number of CPUs available.

- However, in practice, it is often difficult to divide a program in such a way that separate CPUs can execute different
portions without interfering with each other.

Advantages of Parallel Processing over Von Neumann Architecture

• Faster when handling large amounts of data, with each data set requiring the same processing.
• Not limited by the bus transfer rate (a bottleneck in Von Neumann Architecture).
• Can make maximum use of CPU.

Disadvantages of Parallel Processing over Von Neumann Architecture

• Only certain types of data are suitable for parallel processing. Data relying on the result of previous operations
cannot be processed in parallel. For parallel processing, each data set must be independent of each other.
• Several processing blocks needed. Therefore, it is more expensive.

Adapting a Program for Parallel Processing

Programs written for parallel processing are different from programs written for a single processor. The changes
needed to exploit the benefits of massively parallel computers are as follows:

- The program is split into blocks of code that can be processed simultaneously instead of sequentially.

- Each block of code is processed by a different processor.

- This allows each processor to simultaneously process the different blocks of code independently.

- The program requires both parallelism and coordination.

Exercises

Page 6 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

1. (a) Four descriptions and four types of computer architecture are shown below.
Draw a line to connect each description to the appropriate type of computer architecture.

[4 marks]
(b) In a massively parallel computer explain what is meant by:
(i) Massive
___________________________________________________________________________________________

___________________________________________________________________________________________

[1 mark]
(ii) Parallel
___________________________________________________________________________________________

___________________________________________________________________________________________

[1 mark]
(c) There are both hardware and software issues that have to be considered for parallel processing to succeed.
Describe one hardware and one software issue.

Hardware
___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[2 marks]
Software
___________________________________________________________________________________________
Page 7 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[2 marks]
2. (a) The following diagram shows four descriptions and four types of computer architecture.
Draw lines to connect each description to the appropriate computer architecture.

[4 marks]
(b) A computer has a single processor that contains four processing units.
Explain why this is not an example of a massively parallel computer.
___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[2 marks]

(c) An application has previously executed on a single computer. The application will be transferred onto a
massively parallel computer.

Page 8 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

The program code used in the application will need to be updated to ensure that the power of the massively parallel
computer is fully used.

Explain what changes will be required to the program code.

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[3 marks]
3. The table shows four statements about computer architecture.
Put a tick (✓) in each row to identify the computer architecture associated with each statement.

[4 marks]
4. In parallel processing, a computer can have multiple processors running in parallel.
State the four basic computer architectures used in parallel processing.
___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[4 marks]
5. (a) The following incomplete table shows descriptions relating to computer architectures.
Complete the table by inserting the appropriate terms.

Page 9 of 10
 Computer Science Tuition by Mr Chelliah | Maurel Road, Riviere Du Rempart | 59236057

[4 marks]
(b) State three characteristics of massively parallel computers.
___________________________________________________________________________________________

___________________________________________________________________________________________

___________________________________________________________________________________________

[3 marks]
6. There are four basic categories of computer architecture. Identify and describe the four categories of computer
architecture.
Architecture 1: _______________________________________________________________________________

Description: _________________________________________________________________________________

___________________________________________________________________________________________

Architecture 2: _______________________________________________________________________________

Description: _________________________________________________________________________________

___________________________________________________________________________________________

Architecture 3: _______________________________________________________________________________

Description: _________________________________________________________________________________

___________________________________________________________________________________________

Architecture 4: _______________________________________________________________________________

Description: _________________________________________________________________________________

___________________________________________________________________________________________

[8 marks]

Page 10 of 10