Design and Analysis of Algorithms

QUESTION ONE

a) Execution Time of Algorithms

i) Worst Case Analysis (3 Marks)

 Definition: Worst case analysis of an algorithm evaluates its efficiency by considering

the maximum time it takes to complete for any input of size nnn. This scenario examines
the upper bound of the execution time, ensuring the algorithm's performance is
understood under the most challenging conditions.
 Example: For linear search in an unsorted list of nnn elements, the worst case occurs
when the element to be found is at the last position or not present at all, resulting in
O(n)O(n)O(n) time complexity.

ii) Best Case Analysis (3 Marks)

 Definition: Best case analysis evaluates the minimum time an algorithm takes to
complete for any input of size nnn. It represents the scenario where the algorithm
performs the fewest number of operations.
 Example: For linear search, the best case occurs when the element to be found is at the
first position, resulting in O(1)O(1)O(1) time complexity.

iii) Average Case Analysis (3 Marks)

 Definition: Average case analysis evaluates the expected time an algorithm takes to
complete, averaged over all possible inputs of size nnn. This provides a realistic measure
of the algorithm's performance for typical input scenarios.
 Example: For linear search, if the probability of finding the target element at any
position is equal, the average case time complexity is O(n/2)O(n/2)O(n/2), which
simplifies to O(n)O(n)O(n).

b) Decision Control Structures in Programming (6 Marks)

1. If-Else Statement:
o Explanation: Executes a block of code if a specified condition is true. If the
condition is false, an alternative block of code (else part) can be executed.
o Example:

python
Copy code
 if (condition):
// execute this block if condition is true
else:
// execute this block if condition is false

2. Switch-Case Statement:
o Explanation: Allows a variable to be tested against a list of values (cases). When
a match is found, the corresponding block of code is executed.
o Example:

c
Copy code
switch (variable) {
case value1:
// execute this block if variable == value1
break;
case value2:
// execute this block if variable == value2
break;
// other cases...
default:
// execute this block if no case matches
}

c) Difference Between Sorting and Merging (4 Marks)

 Sorting: The process of arranging elements of a list or array in a specific order, typically
ascending or descending. Sorting algorithms include QuickSort, MergeSort, BubbleSort,
etc.
o Example: Sorting the array [3, 1, 4, 1, 5] to [1, 1, 3, 4, 5].
 Merging: The process of combining two or more sorted lists into a single sorted list. This
is often a step within sorting algorithms like MergeSort.
o Example: Merging [1, 3, 5] and [2, 4, 6] to get [1, 2, 3, 4, 5, 6].

d) Algorithm and Pseudo Code to Calculate Sum of Two Variables (8 Marks)

 Definition: An algorithm is a finite set of well-defined instructions to solve a specific

problem or perform a task.
 Algorithm:
1. Start
2. Input the first variable (a)
3. Input the second variable (b)
4. Calculate the sum of the two variables (sum = a + b)
5. Output the result (sum)
6. End
 Pseudo Code:

plaintext
Copy code
START
 INPUT a
INPUT b
sum = a + b
OUTPUT sum
END

e) Memory Usage in Arrays and Objects (3 Marks)

 Arrays: Memory usage for arrays is contiguous, meaning elements are stored in adjacent
memory locations. The total memory usage is determined by the size of the array and the
size of each element.
o Example: An array of 10 integers, where each integer takes 4 bytes, uses
10×4=4010 \times 4 = 4010×4=40 bytes.
 Objects: Memory usage for objects includes space for instance variables and overhead
for object metadata (e.g., type information, garbage collection information). Objects can
reference other objects, leading to non-contiguous memory allocation.

QUESTION TWO

a) Amortized Analysis and Aggregate Method (10 Marks)

Amortized Analysis: A method to analyze the time complexity of algorithms over a sequence of
operations, ensuring that the average time per operation is minimized, even if some operations
are expensive.

Aggregate Method: Calculates the total cost of a sequence of operations and divides it by the
number of operations to find the average cost per operation.

Example: Dynamic array resizing (doubling strategy):

 When the array is full, doubling its size requires copying all elements to a new array.
 Although copying is expensive, it occurs infrequently.
 Over nnn insertions, the total cost of resizing is O(n)O(n)O(n), and the average cost per
insertion is O(1)O(1)O(1).

b) MergeSort and QuickSort Mechanisms (10 Marks)

MergeSort:

 Mechanism: A divide-and-conquer algorithm that divides the array into two halves,
recursively sorts them, and then merges the sorted halves.
 Steps:
1. Divide the array into two halves.
2. Recursively sort each half.
3. Merge the two sorted halves into one sorted array.
 Time Complexity: O(nlog⁡n)O(n \log n)O(nlogn)
QuickSort:

 Mechanism: A divide-and-conquer algorithm that selects a pivot element, partitions the

array around the pivot, and recursively sorts the partitions.
 Steps:
1. Select a pivot element.
2. Partition the array such that elements less than the pivot are on the left and
elements greater than the pivot are on the right.
3. Recursively sort the left and right partitions.
 Time Complexity: Average case O(nlog⁡n)O(n \log n)O(nlogn), worst case
O(n2)O(n^2)O(n2)

QUESTION THREE

a) Breadth First Search (BFS) and Depth First Search (DFS) (10 Marks)

Breadth First Search (BFS):

 Mechanism: Explores all neighbors of a node before moving to the next level.
 Steps:
1. Start from a source node.
2. Visit all its neighbors.
3. Move to the next level of nodes.
4. Repeat until all nodes are visited.
 Use Case: Finding the shortest path in unweighted graphs.
 Data Structure: Queue

Depth First Search (DFS):

 Mechanism: Explores as far as possible along each branch before backtracking.

 Steps:
1. Start from a source node.
2. Visit a neighbor and recursively explore its descendants.
3. Backtrack when no more unvisited neighbors are found.
4. Repeat until all nodes are visited.
 Use Case: Pathfinding and cycle detection.
 Data Structure: Stack (or recursive function call stack)

b) Dijkstra’s Shortest Path Algorithm (10 Marks)

Dijkstra’s Algorithm:

 Goal: Find the shortest path from a source node to all other nodes in a weighted graph.
 Steps:
1. Initialize distances from the source to all nodes as infinite, except the source itself
(distance 0).
 2. Set the source node as the current node.
3. For the current node, update the distance to each of its neighbors.
4. Mark the current node as visited and select the unvisited node with the smallest
distance as the new current node.
5. Repeat steps 3-4 until all nodes are visited or the smallest distance among the
unvisited nodes is infinite.
 Use Case: Network routing protocols and GPS navigation systems.

QUESTION FOUR

a) Upper Bound Notation in Performance Analysis (10 Marks)

Upper Bound Notation (Big-O):

 Definition: Describes the upper bound of an algorithm’s time complexity, providing the
worst-case scenario for the growth rate of an algorithm's running time as the input size
increases.
 Graph Explanation:
o The x-axis represents the input size nnn.
o The y-axis represents the time or number of operations.
o The function f(n)=O(g(n))f(n) = O(g(n))f(n)=O(g(n)) means that f(n)f(n)f(n)
grows at most as fast as g(n)g(n)g(n), up to a constant factor, for sufficiently large
nnn.

b) Computer Memory (1 Mark)

Computer Memory: The physical or virtual storage space in a computer where data and
programs are stored and accessed.

c) Memory Usage in Arrays, Objects, and References (9 Marks)

i) Arrays: Contiguous blocks of memory where elements of the same type are stored
sequentially. Memory usage is the size of the array times the size of each element.

ii) Objects: Non-contiguous memory allocations that include space for instance variables and
metadata. Memory usage depends on the object's

4o
a) Explain the difference between Time Efficiency and Space Efficiency.(2 Marks) b) Explain
the meaning of the following terms: (i) order of an algorithm (ii) brute force (iii) recursion (iv)
binary search. (2 Marks) c) Explain the principle of optimality in relation to the study of
algorithms. (2 Marks) d) Define Big Oh Notation and differentiate it from the Little ‘oh’
notation. (2 Marks) e) Describe the greedy method and give two real examples of how the
method is used in real life situations. . (2 Marks) f) Describe the Tower of Hanoi and its
significance in the study of algorithms. (2 Marks) Library copy Page 2 g) Define the AVL tree.
State two applications of this tree. (2 Marks) h) Describe the heap as an ADT and state two of its
applications. (2 Marks) i) Describe a priority queue. How does it differ from queue ADT? (2
Marks) j) You hear a group of students discussing the term programming, and one of them, who
had seen the words divide and conquer on a white board, on seeing you, asks you, “What is
divide and conquer?” How would you respond to him? (2 Marks) k) Describe Dijkstra’s
Algorithm. Why is it so important in the study of algorithms? (2 Marks) l) Describe the
fractional knapsack problem. How does it differ from 0-1 knapsack problem? (2 Marks) m)
Describe Hamiltonian cycles in a given graph. State one area of its application. (2 Marks) n) One
of the applications of backtracking technique is the 8-queen problem. Describe this problem and
its significance in the study of algorithms. (2 Marks) o) Describe the quick sort algorithm. For
what kind of problem is this method applied? (2 Marks) Question 2 (a) Describe heuristic as a
technique in the design of algorithm. What are its drawbacks? (10 Marks) (b) Distinguish
between DFS and BSF search algorithms. Use an illustration for each. (10 Marks) Question 3 a)
Discuss the difference between backtracking and branch bound. Indicate in which situations each
may be used. (10 Marks) b) What does it mean to say that one node is promising while another is
non promising? Illustrate your answer with a suitable example. (10 Marks) Question 4 a) One of
the methods for solving any recurrence relation problems is called the substitution method.
Explain the meaning of the terms recurrence relation problem and substitution method. State the
recurrence relation of the divide and conquer technique. b) Describe the maximum and
maximum problem. What is the significance of this problem? Write an algorithm to demonstrate
the maximum and maximum problem. (10 Marks) Question 5 (10 Marks) a) Discuss the
difference between the greedy algorithm and dynamic programming. (10 Marks) b) Describe the
features and drawbacks of dynamic programming. (10 Marks)
ChatGPT

QUESTION ONE

a) Difference Between Time Efficiency and Space Efficiency (2 Marks)

 Time Efficiency: Refers to the amount of time an algorithm takes to complete as a

function of the size of its input. It measures the execution time or the number of basic
operations performed by the algorithm.
 Space Efficiency: Refers to the amount of memory an algorithm uses during its
execution as a function of the size of its input. It measures the memory required by the
algorithm, including input, output, and auxiliary space.

b) Terms Explanation (2 Marks)

i) Order of an Algorithm: Refers to the growth rate of the algorithm's time or space
requirements relative to the size of the input. It is often expressed using Big O notation (e.g.,
O(n2)O(n^2)O(n2), O(nlog⁡n)O(n \log n)O(nlogn)).

ii) Brute Force: A straightforward method of solving a problem by trying all possible solutions
and selecting the best one. It is often simple to implement but can be inefficient for large
problem sizes.
iii) Recursion: A technique where a function calls itself to solve a smaller instance of the same
problem. It involves a base case and a recursive case.

iv) Binary Search: An efficient algorithm for finding an item from a sorted list of items by
repeatedly dividing the search interval in half. It has a time complexity of O(log⁡n)O(\log
n)O(logn).

c) Principle of Optimality in Algorithms (2 Marks)

The principle of optimality states that an optimal solution to a problem contains optimal
solutions to its subproblems. This principle is a key concept in dynamic programming and
greedy algorithms, allowing complex problems to be solved by breaking them down into simpler
subproblems.

d) Big Oh Notation and Little ‘oh’ Notation (2 Marks)

 Big O Notation (OOO): Describes an upper bound on the time complexity of an

algorithm. It indicates the maximum time required for the algorithm to complete,
proportional to the input size.
o Example: f(n)=O(g(n))f(n) = O(g(n))f(n)=O(g(n)) means f(n)f(n)f(n) grows at
most as fast as g(n)g(n)g(n) up to a constant factor.
 Little o Notation (ooo): Describes a stricter upper bound on the time complexity,
indicating that the growth rate of the algorithm is strictly less than g(n)g(n)g(n).
o Example: f(n)=o(g(n))f(n) = o(g(n))f(n)=o(g(n)) means f(n)f(n)f(n) grows strictly
slower than g(n)g(n)g(n).

e) Greedy Method and Real-life Examples (2 Marks)

Greedy Method: An algorithmic approach that builds up a solution piece by piece, always
choosing the next piece that offers the most immediate benefit. It doesn't guarantee a globally
optimal solution, but often produces a good approximation.

Examples:

1. Coin Change Problem: Minimizing the number of coins for a given amount using the
largest denominations first.
2. Huffman Coding: Constructing an optimal prefix-free binary code based on the
frequencies of the characters.

f) Tower of Hanoi and Its Significance (2 Marks)

Tower of Hanoi: A mathematical puzzle consisting of three rods and a number of disks of
different sizes. The objective is to move all the disks from the first rod to the third rod, following
these rules:

1. Only one disk can be moved at a time.

 2. A disk can only be placed on top of a larger disk or an empty rod.

Significance: Demonstrates recursive problem-solving and algorithmic thinking.

g) AVL Tree and Its Applications (2 Marks)

AVL Tree: A self-balancing binary search tree where the difference between heights of left and
right subtrees cannot be more than one for all nodes. This ensures O(log⁡n)O(\log n)O(logn) time
complexity for search, insertion, and deletion.

Applications:

1. Databases: Efficient indexing and range queries.

2. Memory Management: Allocation and deallocation of memory blocks.

h) Heap as an ADT and Its Applications (2 Marks)

Heap: A specialized tree-based data structure that satisfies the heap property: in a max heap, for
any given node iii, the value of iii is greater than or equal to the values of its children, and in a
min heap, the value of iii is less than or equal to the values of its children.

Applications:

1. Priority Queues: Efficiently manage and retrieve the highest (or lowest) priority
element.
2. Heap Sort: Sorting algorithm based on the heap data structure.

i) Priority Queue vs. Queue ADT (2 Marks)

Priority Queue: A data structure where each element has a priority, and elements are dequeued
in order of their priority. Higher priority elements are served before lower priority ones,
regardless of their order in the queue.

Queue ADT: A data structure where elements are enqueued at the rear and dequeued from the
front, following the First-In-First-Out (FIFO) principle.

j) Divide and Conquer (2 Marks)

Divide and Conquer: An algorithmic paradigm that breaks a problem into smaller subproblems,
solves the subproblems recursively, and combines their solutions to solve the original problem.

Response: "Divide and conquer is a method where you divide a problem into smaller parts, solve
each part individually, and then combine their solutions to get the final result. This technique is
used in many algorithms like MergeSort and QuickSort."

k) Dijkstra’s Algorithm and Its Importance (2 Marks)

Dijkstra’s Algorithm: A graph algorithm used to find the shortest path from a single source
node to all other nodes in a weighted graph with non-negative weights.

Importance: It is essential for applications in network routing, GPS navigation systems, and
various optimization problems.

l) Fractional Knapsack Problem vs. 0-1 Knapsack Problem (2 Marks)

Fractional Knapsack Problem: Allows breaking items into smaller parts. The goal is to
maximize the total value in the knapsack by taking fractions of items.

0-1 Knapsack Problem: Items cannot be broken into smaller parts; each item can be taken or
left. The goal is to maximize the total value without exceeding the weight capacity.

m) Hamiltonian Cycles in Graphs (2 Marks)

Hamiltonian Cycle: A cycle in a graph that visits each vertex exactly once and returns to the
starting vertex.

Application: Solving problems in routing, scheduling, and network topology.

n) 8-Queen Problem and Its Significance (2 Marks)

8-Queen Problem: Placing eight queens on an 8x8 chessboard so that no two queens threaten
each other. It is a classic example of a constraint satisfaction problem.

Significance: Demonstrates backtracking techniques and constraint satisfaction problem-solving.

o) QuickSort Algorithm and Its Application (2 Marks)

QuickSort: A divide-and-conquer sorting algorithm that selects a pivot element, partitions the
array around the pivot, and recursively sorts the partitions.

Application: Efficiently sorts large datasets and is widely used in practice due to its average-
case O(nlog⁡n)O(n \log n)O(nlogn) time complexity.

QUESTION TWO

a) Heuristic Technique in Algorithm Design (10 Marks)

Heuristic: A problem-solving technique that finds a good enough solution for complex problems
by using practical methods and rules of thumb. It does not guarantee an optimal solution but can
significantly reduce computation time.

Drawbacks:
 1. No Optimality Guarantee: Heuristics do not ensure the best solution.
2. Problem-specific: May not generalize well to different problems or instances.
3. Unpredictable Performance: Can perform poorly on some inputs.

b) DFS vs. BFS with Illustrations (10 Marks)

Depth First Search (DFS):

 Mechanism: Explores as far as possible along each branch before backtracking.

 Illustration:

mathematica
Copy code
Graph: A - B - D
| |
C - E

o DFS Traversal: A, B, D, E, C

Breadth First Search (BFS):

 Mechanism: Explores all neighbors of a node before moving to the next level.
 Illustration:

mathematica
Copy code
Graph: A - B - D
| |
C - E

o BFS Traversal: A, B, C, D, E

QUESTION THREE

a) Backtracking vs. Branch and Bound (10 Marks)

Backtracking:

 Mechanism: Incrementally builds candidates for the solution, abandons a candidate as

soon as it determines that the candidate cannot possibly be completed to a valid solution.
 Usage: Solving constraint satisfaction problems like the n-queens problem, sudoku.

Branch and Bound:

 Mechanism: Systematically explores all possible solutions and eliminates subproblems

that do not satisfy the bounds.
 Usage: Solving optimization problems like the knapsack problem, traveling salesman
problem.
b) Promising vs. Non-promising Nodes (10 Marks)

Promising Node: A node in the search tree that could potentially lead to a solution.

Non-promising Node: A node that cannot lead to a solution and can be pruned.

Example: In the 8-queen problem, a promising node represents a partial solution that does not
violate any constraints (no two queens attack each other). A non-promising node is one where
constraints are violated.

QUESTION FOUR

a) Recurrence Relation Problem and Substitution Method (10 Marks)

Recurrence Relation Problem: Describes the time complexity of an algorithm in terms of its
performance on smaller inputs.

Substitution Method: Solving recurrences by making an educated guess of the form of the
solution and then using mathematical induction to prove it.

Recurrence Relation for Divide and Conquer:

 Example: T(n)=2T(n/2)+O(n)T(n) = 2T(n/2) + O(n)T(n)=2T(n/2)+O(n) for MergeSort.

b) Maximum and Minimum Problem (10 Marks)

Definition: Finding both the maximum and minimum values in an array with the least number of
comparisons.

Significance: Optimizes the number of comparisons to 3n/2−23n/2 - 23n/2−2 for nnn elements.

Algorithm:

1. Compare elements in pairs.

2. Track the maximum and minimum of each pair.
3. Update global maximum and minimum.

python
Copy code
def find_max_min(arr):
if len(arr) % 2 == 0:
global_max = max(arr[0], arr[1])
global_min = min(arr[0], arr[1])
i = 2
else:
global_max = global_min = arr[0]
i = 1
 while i < len(arr) - 1:
local_max = max(arr[i], arr[i + 1])
local_min = min(arr[i], arr[i + 1])

global_max = max(global_max, local_max)

global_min = min(global_min, local_min)

i += 2

return global_max, global_min

QUESTION FIVE

a) Greedy Algorithm vs. Dynamic Programming (10 Marks)

Greedy Algorithm:

 Mechanism: Makes locally optimal choices at each step with the hope of finding a global
optimum.
 Example: Huffman coding, Prim’s algorithm.

Dynamic Programming:

 Mechanism: Solves problems by breaking them down into simpler subproblems and
storing the solutions to avoid redundant computations.
 Example: Fibonacci sequence, knapsack problem.

b) Features and Drawbacks of Dynamic Programming (10 Marks)

Features:

1. Optimal Substructure: Breaks problems into subproblems.

2. Overlapping Subproblems: Reuses solutions to subproblems.

Drawbacks:

1. Space Complexity: Requires extra memory for storing subproblem solutions.

2. Complexity in Implementation: Can be harder to implement than greedy algorithms.

4o