1

Lecture #4

Complexity of Algorithms

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

2 What is an Algorithm?

An algorithm is a well-defined sequential computational technique that accepts a value or a collection of

values as input and produces the output(s) needed to solve a problem.

The criteria of an algorithm

Input: Zero or more inputs are externally supplied to the algorithm.

Output: At least one output is produced by an algorithm.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: In an algorithm, it will be terminated after a finite number of steps for all different cases.
Effectiveness: Each instruction must be very basic, so the purpose of those instructions must be very
clear to us.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

3
Example 1: Write an algorithm to find the maximum of all the elements present in the array.

Follow the algorithm approach as below:

Step 1: Start the Program

Step 2: Declare a variable max with the value of the first element of the array.
Step 3: Compare variable max with other elements using loop.
Step 4: If variable max < array element value, change variable max to new max.
Step 5: If no element is left, return or print max otherwise goto step 3.
Step 6: End of Solution

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

4 There are several reasons why we use algorithms:

 Efficiency: Algorithms can perform tasks quickly and accurately, making them an essential tool for
tasks that require a lot of calculations or data processing.
 Consistency: Algorithms are repeatable and produce consistent results every time they are executed.
This is important when dealing with large amounts of data or complex processes.
 Scalability: Algorithms can be scaled up to handle large datasets or complex problems, which makes
them useful for applications that require processing large volumes of data.
 Automation: Algorithms can automate repetitive tasks, reducing the need for human intervention
and freeing up time for other tasks.
 Standardization: Algorithms can be standardized and shared among different teams or organizations,
making it easier for people to collaborate and share knowledge.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

5
Advantages of Algorithms
 Easy to understand: Since it is a stepwise representation of a solution to a given problem, it is
easy to understand.
 Language Independent: It is not dependent on any programming language, so it can easily be
understood by anyone.
 Debug / Error Finding: Every step is independent / in a flow so it will be easy to spot and fix the
error.
 Sub-Problems: It is written in a flow so now the programmer can divide the tasks which makes
them easier to code.

Disadvantages of Algorithms
 Creating efficient algorithms is time-consuming and requires good logical skills.
 Nasty to show branching and looping in algorithms.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

6 Types of Algorithms:

 Sorting algorithms: Bubble Sort, insertion sort, and many more. These algorithms are used to sort
the data in a particular format.

 Searching algorithms: Linear search, binary search, etc. These algorithms are used in finding a
value or record that the user demands.

 Graph Algorithms: It is used to find solutions to problems like finding the shortest path between
cities, and real-life problems like traveling salesman problems.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

7 Complexities of an Algorithm

The complexity of an algorithm computes the amount of time and spaces required by an
algorithm for an input of size (n). The complexity of an algorithm can be divided into two
types. The time complexity and the space complexity.

1. Time Complexity of an Algorithm :

The time complexity is defined as the process of determining a formula for total time required
towards the execution of that algorithm. This calculation is totally independent of
implementation and programming language.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

8 2. Space Complexity of an Algorithm :

Space complexity is defining as the process of defining a formula for prediction of how much
memory space is required for the successful execution of the algorithm. The memory space is
generally considered as the primary memory.
Some of the reasons for studying space complexity are :
1. If the program is to run on multi-user system, it may be required to specify the amount of
memory to be allocated to the program.
2. It is checked in advanced that whether the sufficient memory is available to run the
program.
3. There may be several possible solutions with different space requirements.
4. It can be used to estimate the size of the largest problem that a program can solve.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

9 Components needed for Space Complexity

 Instruction Space: Space needed to store the executable version of the program and is fixed.
 Data Space: Space needed to store all constants, variables values, and has further two components:
 Space needed by constants and simple variables. This space is fixed.
 Space needed by fixed size, structured variables, such as, Arrays and Structures.
 Dynamically allocated space. This Spaces usually varies.

 Environment stack space: Spaced needed to store the information, needed to resume the
suspended functions. Each time a function is invoked, the following data is stored on the
Environment stack.
a) Return Address that is from where it has to resumed, after completions of the called
functions.
b) Values of all local variables and the values of formal parameters in the function being
invoked.
© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh
10 Measurement of Complexity of an Algorithm

Based on the above three notations of Time Complexity there are three cases to analyze an
algorithm:

1. Best Case Analysis (Very Rarely used) – Omega notation

2. Worst Case Analysis (Mostly used) – Big O notation
3. Average Case Analysis (Rarely used) - Theta notation

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

11 1. Worst Case Analysis (Mostly used)

In the worst-case analysis, we calculate the upper bound on the running time of an algorithm. We must
know the case that causes a maximum number of operations to be executed. For Linear Search, the worst
case happens when the element to be searched (x) is not present in the array. When x is not present, the
search() function compares it with all the elements of arr[] one by one. Therefore, the worst-case time
complexity of the linear search would be O(n).

2. Best Case Analysis (Very Rarely used)

In the best-case analysis, we calculate the lower bound on the running time of an algorithm. We must
know the case that causes a minimum number of operations to be executed. In the linear search problem,
the best case occurs when x is present at the first location. The number of operations in the best case is
constant (not dependent on n). So time complexity in the best case would be Ω(1)
© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh
12 3. Average Case Analysis (Rarely used)

In average case analysis, we take all possible inputs and calculate the computing time for all of the inputs.
Sum all the calculated values and divide the sum by the total number of inputs. We must know (or predict)
the distribution of cases. For the linear search problem, let us assume that all cases are uniformly
distributed (including the case of x not being present in the array). So we sum all the cases and divide the
sum by (n+1). Following is the value of average-case time complexity.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

13 Graphs for Complexity Algorithm

14

12

10

0
Best Case Average Case Worst Case

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

14 Examples with their complexity analysis:

int search(int arr[], int n, int x)

{
int i;
for (i = 0; i < n; i++) {
if (arr[i] == x)
return i;
}
return -1;
}

// Driver's Code
int main()
{
int arr[] = { 1, 10, 30, 15 };
int x = 30;
int n = sizeof(arr) / sizeof(arr[0]);

// Function call
cout << x << " is present at index " << search(arr, n, x);

return 0;
}

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

15 Time Complexity Analysis: (In Big-O notation)

Best Case: O(1), This will take place if the element to be searched is on the first index of the
given list. So, the number of comparisons, in this case, is 1.

Worst Case: O(n), This will take place if:

 The element to be searched is on the last index
 The element to be searched is not present on the list

Average Case: O(n), This will take place if the element to be searched is on the middle index
of the given list.

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

16 Some examples of finding Time Complexity of an algorithm

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

17 Finding Time Complexity of an algorithm

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

18 Finding Time Complexity of an algorithm

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

19 Finding Time Complexity of an algorithm

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

20 Finding Time Complexity of an algorithm

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh

21

Thank You

© LPU :: CAP267 Data Structures :: Dr. Amanpreet Singh