Alhussian Dawood – 4th - IE

Distributed systems
Review Mid-term (Lec1-5):
Lec 1:
Multiple Choice Questions (MCQ):
1. What are some of the applications of modern computing?
• A) Text processing
• B) Web browsing
• C) Large-scale simulations
• D) Video editing
2. Which method can help improve application performance by
reducing the time per instruction?
• A) Working harder
• B) Working smarter
• C) Getting help from parallelism
• D) None of the above
3. What is a distributed system?
• A) A single computer system
• B) A collection of independent computers connected via a
network
• C) A system with only one type of processor
• D) A system that doesn't utilize parallelism
4. What is the main reason for the development of distributed
computing systems?
• A) To limit the computational power of single processor systems
• B) To achieve faster processing speeds
 • C) To overcome physical and architectural bounds
• D) To reduce the cost of hardware
5. What is a characteristic of distributed systems according to the
provided text?
• A) Users have full knowledge of the internal organization
• B) Communication between computers is always visible to users
• C) Users interact with the system in a consistent and uniform
way
• D) Components of the system don't need to collaborate
Questions & Answers (Q&A):
1. What is the purpose of high-performance computing (HPC)?
• HPC aims to solve grand challenge applications using computer
modeling, simulation, and analysis.
2. How can application performance be improved?
• Performance can be improved by either working harder (using
faster hardware), working smarter (optimized algorithms and
techniques), or getting help from parallelism (using multiple
computers).
3. Define a distributed system.
• A distributed system is a collection of independent computers
interconnected via a network, capable of collaborating on a task.
4. Why is the development of distributed computing systems
necessary?
• Distributed computing systems are developed to overcome
physical and architectural bounds that limit the computational
power of single processor systems.
5. What are some characteristics of distributed systems?
 • Characteristics include hiding differences between computers and
their communication methods from users, as well as providing
consistent and uniform interaction regardless of location or time.
Difference Between:
Centralized Computing vs. Distributed Computing:
• Centralized computing involves processing on a single processor, while
distributed computing utilizes multiple interconnected computers.
• Centralized computing can be limited by the computational power of a
single processor, whereas distributed computing can overcome these
limits by parallel processing across multiple machines.
• Centralized computing systems may have a single point of failure, while
distributed systems can offer redundancy and fault tolerance through
distributed architecture.
Parallel Computing vs. Sequential Computing:
• Parallel computing involves simultaneous execution of tasks across
multiple processors or cores, while sequential computing executes tasks
one after another.
• Parallel computing can achieve higher performance and efficiency for
certain tasks, especially those that can be divided into independent sub-
tasks, whereas sequential computing may be simpler but less efficient
for complex tasks.
• Parallel computing architectures require coordination and
synchronization mechanisms to ensure correct execution, whereas
sequential computing follows a linear execution flow without such
complexities.
Lec 2:
Multiple Choice Questions (MCQs):
1. What is a program from the programmer's perspective?
• a) A set of instructions executed by the processor
• b) An executable file stored in secondary memory
• c) A sequence of events in a process
• d) A passive entity awaiting execution
2. Which of the following is true about a process?
• a) It consists only of program code
• b) It shares memory space with other processes
• c) It has no temporary variables or file handles
• d) It lacks its own address space
3. What defines a thread?
• a) It is an executable file stored in secondary memory
• b) It shares memory space with other threads
• c) It is a sequential flow of control within a process
• d) It lacks a program counter and register values
4. In the context of parallelism, what does a partial order consist of?
• a) A fully ordered set
• b) A set of events without any order
• c) A binary relation indicating partial ordering
• d) A sequence of independent events
5. What is a reason for implementing parallelism related to reliability?
• a) Physical restrictions
• b) Efficiency in execution
 • c) Synchronization restriction
• d) Accessibility in distributed systems
Questions & Answers (Q&A):
Q: What distinguishes a process from a program?
A: A process is a program in execution, residing in main memory and having
its own address space, program counter, and other attributes.
Q: How does a thread differ from a process?
A: Threads are sequential flows of control within a process, sharing memory
space and resources, whereas processes have their own memory space and
attributes.
Q: What is the significance of partial ordering in parallelism?
A: Partial ordering allows for certain pairs of elements to have a defined
precedence, indicating the order of events in parallel execution.
Q: Why is scalability considered a challenge in distributed systems?
A: Scalability poses challenges in expanding the system and applications
without altering the system structure or application algorithms, particularly
as loads vary.
Differences Between:
1. Program vs. Process:
• Program: A set of instructions awaiting execution by the
processor, stored as an executable file in secondary memory.
• Process: An executing instance of a program in main memory,
possessing its own address space, program counter, and other
attributes.
2. Thread vs. Process:
• Thread: A sequential flow of control within a process, sharing
memory space and resources with other threads.
 • Process: A program in execution, residing in main memory and
having its own memory space and attributes.
3. Parallelism vs. Sequential Computing:
• Parallelism: Involves concurrent execution of multiple tasks to
improve efficiency, scalability, and reliability.
• Sequential Computing: Involves executing tasks one after another
in a predetermined sequence, without parallel execution.
4. Synchronous Circuit vs. Asynchronous Circuit:
• Synchronous Circuit: Operates based on clock signals,
synchronizing operations between components.
• Asynchronous Circuit: Operates without a central clock, allowing
for simultaneous execution at varying speeds.
5. Reliability vs. Performance:
• Reliability: Focuses on ensuring system functionality and service
availability, often through redundancy and fault tolerance.
• Performance: Focuses on optimizing execution speed and
efficiency, particularly in time-critical applications or with large
datasets.

LEC 3:
Multiple Choice Questions (MCQs):
A. Flynn Taxonomy:
1. What does SISD stand for in Flynn Taxonomy?
• a) Single Instruction Single Data
• b) Single Instruction Multiple Data
• c) Multiple Instruction Single Data
• d) Multiple Instruction Multiple Data
 2. Which architecture is characterized by a single control unit and multiple
execution units operating in synchronization?
• a) SISD
• b) SIMD
• c) MISD
• d) MIMD
3. What does MIMD architecture involve?
• a) Single control unit managing multiple data streams
• b) Multiple control units managing a single data stream
• c) Multiple control units and multiple data streams
• d) Single control unit and single data stream
B. Connection Memory - Processor:
4. What is a characteristic of shared memory systems?
• a) Each processor has its own local memory
• b) Processors exchange information through interconnection
network
• c) Global memory is shared by all processors
• d) There is no need for message passing
5. In a message passing system, how is data exchanged between
processors?
• a) Through shared memory
• b) Via direct connection between processors
• c) Using Send/Receive commands
• d) Through a centralized controller
Questions & Answers (Q&A):
Q: What is the difference between SISD and SIMD architectures?
A: SISD (Single Instruction Single Data) involves a single control unit
processing a single data stream, while SIMD (Single Instruction Multiple Data)
has a single control unit managing multiple execution units operating in
synchronization.
Q: How does MISD architecture function?
A: In MISD (Multiple Instruction Single Data) architecture, multiple execution
units perform various operations on the same data stream simultaneously or
sequentially. This architecture is practically unused due to its complexity.
Differences Between:
1. SISD vs. SIMD:
• SISD: Single control unit processing a single data stream
sequentially.
• SIMD: Single control unit managing multiple execution units
operating in parallel on different data.
2. Shared Memory vs. Message Passing:
• Shared Memory: Processors communicate through a global
memory shared by all processors, typically in symmetric
multiprocessor (SMP) systems.
• Message Passing: Processors exchange information through
message passing, without shared memory, typical in distributed
memory systems.
3. MISD vs. MIMD:
• MISD: Multiple execution units perform various operations on the
same data stream simultaneously or sequentially, although it is
rarely used in practice.
• MIMD: Multiple control units manage multiple data streams,
allowing each processor to execute different programs on its own
data.
Lec 4:

Multiple Choice Questions (MCQs):

A. Flynn Taxonomy:
1. What does SISD stand for in Flynn Taxonomy?
• a) Single Instruction Single Data
• b) Single Instruction Multiple Data
• c) Multiple Instruction Single Data
• d) Multiple Instruction Multiple Data
2. Which architecture involves a single control unit managing multiple
execution units operating in synchronization?
• a) SISD
• b) SIMD
• c) MISD
• d) MIMD
B. According to connection memory - processor:
3. What is the primary characteristic of shared memory systems?
• a) Each processor has its own local memory
• b) Processors communicate through a global memory shared by all
processors
• c) Processors exchange information through interconnection network
• d) There is no need for message passing
 4. What determines the access control in shared memory systems?
• a) Cache coherence
• b) Synchronization constraints
• c) Flags in the access control table
• d) Interconnection network
Questions & Answers (Q&A):
Q: What is the key difference between SISD and SIMD architectures?
A: SISD (Single Instruction Single Data) involves a single control unit
processing a single data stream sequentially, while SIMD (Single Instruction
Multiple Data) has a single control unit managing multiple execution units
operating in parallel on different data.
Q: How do shared memory systems handle interprocessor communication?
A: In shared memory systems, all processors share a global memory, and
communication between tasks running on different processors is performed
through reading from and writing to this global memory.
Differences Between:
1. SISD vs. SIMD:
• SISD: Single control unit processing a single data stream
sequentially.
• SIMD: Single control unit managing multiple execution units
operating in parallel on different data.
2. Shared Memory vs. Message System:
• Shared Memory: Processors communicate through a global
memory shared by all processors.
• Message System: Processors exchange information through
message passing without shared memory.
3. UMA vs. NUMA vs. COMA:
 • UMA (Uniform Memory Access): All processors have equal access
to shared memory.
• NUMA (Non-Uniform Memory Access): Each processor has local
memory, and access time depends on the distance to the memory
module.
• COMA (Cache-Only Memory Access): Each processor uses its
cache memory, and data is migrated to the requesting processor.
Lec 5:
Multiple Choice Questions (MCQs):
1. What defines a Dynamic Interconnection System?
• a) Pre-established connections among nodes
• b) Connections established as needed using switching elements
• c) Fixed network structure with shared memory access
• d) Static routing of messages between nodes
2. Which type of interconnection network relies on switch elements to
establish connections?
• a) Static network
• b) Dynamic network
• c) Shared memory network
• d) Hybrid network
Questions & Answers (Q&A):
Q: What characterizes a dynamic interconnection system?
A: Dynamic interconnection systems establish connections among nodes as
needed using switching elements. This allows for flexible routing and efficient
resource utilization.
Q: What are the basic types of interconnection networks discussed in the text?
A: The text outlines two main types of interconnection networks: bus-based
interconnection networks and switch-based interconnection networks.
Differences Between:
1. Static vs. Dynamic Networks:
• Static Networks: Establish fixed connections among nodes when
the system is designed.
• Dynamic Networks: Establish connections on-the-fly as needed,
using switching elements.
2. Single Bus System vs. Multiple Bus System:
• Single Bus System: Uses a single shared bus to connect multiple
processors, leading to low transmission and scalability issues.
• Multiple Bus System: Extends the single bus concept by using
multiple parallel buses, reducing transmission delays but
increasing complexity.
3. Multistage Network vs. Crossbar Network:
• Multistage Network: Provides multiple simultaneous paths
between processors and memory modules, with complexity O(log
N).
• Crossbar Network: Allows simultaneous connections among all
inputs and outputs, with complexity O(N^2) and non-blocking
characteristics.

Note: This Review (Q&A) made by Ai.