Advanced Data Structures and Algorithms: Learn how to enhance data processing with more complex and advanced data structures (English Edition)

CHAPTER 1

Analysis of Algorithm

Introduction

The chapter emphasizes the basics of algorithmic analysis. Donald Knuth defines the term analysis of algorithms as the common technique for theoretical estimation of required resources, that is used to provide a solution to any specific computational problem. It will discuss the need for analysis of algorithms and help us choose a more suitable algorithm for a given problem statement. In algorithmic design, complexity of an algorithm plays an important aspect in justifying the design decisions. Accordingly, algorithm efficiency is measured in two perspectives, such as time and space complexity. Hence, the major focus of this chapter is on various types of asymptotic notations used for the estimation of time complexity of an algorithm and is discussed with examples.

Structure

In this chapter, we will discuss the following topics:

Analysis of algorithm

Asymptotic Notations

Time Complexity

General Rules for time complexity calculation

Recurrences

Objectives

This chapter discusses the basics of analysis of algorithm, the need for analysis of algorithms as well as the various notations, with examples. Apart from these, time complexity calculation is the major content focused on, in the chapter.

Analysis of algorithm

In this section, we give an overview of a generic framework on analysis of algorithms. It analyzes the efficiency of algorithms, in terms of space efficiency and time efficiency. To represent the complexity of an algorithm, asymptotic notations are a very crucial and important factor in designing algorithms. Therefore, various notations with solved examples are illustrated here.

Space complexity is defined as the amount of memory space required to execute an algorithm. Sometimes, space complexity is ignored as the space used is minimal. But time complexity refers to the amount of time required for the computation of an algorithm. Mostly, execution time depends on the various properties such as disk input/output speed, CPU speed, instructor set and so on. The calculation of time complexity and its rules with solved examples are also described in this chapter. It is followed by the recurrences in algorithm analysis and its three major types are discussed in the later sections of this chapter.

What is analysis of algorithms?

In general terms, analysis of algorithms discusses the efficiency of an algorithm. It tells the estimation of various resources required by an algorithm to crack a specific problem of computation. The resources are the necessary storage or the required time to execute a specific algorithm. The estimated running time of an algorithm is called time complexity and the estimated storage/memory needed for the execution of an algorithm is called space complexity.

Why to analyze algorithms?

Multiple solutions are available for a single problem; analysis will present the best algorithm to solve the given problem out of the multiple solutions. Even though the objective of the algorithms is to generate the expected output, the ways in which the outputs are generated, are different. The algorithm varies in the time and the space complexity. The various cases of analysis such as the worst, best and the average are performed with the help of asymptotic notations, to finalize the best algorithm.

Asymptotic notations

The main idea of asymptotic analysis is to have a measure of efficiency of algorithms, that does not depend on machine specific constants, and neither requires algorithms to be implemented and time taken by programs to be compared. Asymptotic notations are mathematical tools to represent time complexity of algorithms for asymptotic analysis. The following 3 asymptotic notations are mostly used to represent time complexity of algorithms.

Θ Notation

In this theta notation, the function bounds between the upper and the lower bound, and so it is called as an average case of T(n).

Average-case T(n) = average expected time of algorithm over all inputs of size n.

Figure 1.1 features the average case Theta Θ Notation:

Figure 1.1: Average case Theta Θ Notation

Big O Notation

In this Big O notation, the function defines an upper bound, and so it is called the worst case of T(n).

Figure 1.2 features the worst-case Big O Notation:

Figure 1.2: Worst case Big O Notation

Ω (Omega) Notation

In this Ω notation, the function defines a lower bound, and so it is called the best case of T(n).

T(n) = minimum time of algorithm on any input of size n.

It is a slow algorithm that works fast on some input.

Figure 1.3 features the best-case Omega Ώ Notation:

Figure 1.3: Best case Omega Ώ Notation

Table 1.1 features the Asymptotic notations:

Table 1.1: Asymptotic Notations

Example 1

Find Upper Bound, Lower Bound and Tight Bound for the following function 2n+5.

Solution 1:

Consider f(n)=2n+5,

g(n)=n {consider the highest degree of f(n)}

Refer to Table 1.2:

Table 1.2: The constant value consideration for the asymptotic notations

Big O:

f(n)<=c*g(n)

2n+5<=c*n

C=3// as per Table 1.2

2n+5<=3*n

For n=1, 7<=3 → False

For n=2, 9<=6 → False

For n=3, 11<=9 → False

For n=4, 13<=12 → False

For n=5, 15<=15 → True

Therefore f(n)=O(g(n))

2n+5=O(n) for all n>=5, C=3

Omega (Ώ):

f(n)>=c*g(n)

2n+5>=c*n

C=2

2n+5>=2*n

For n=1, 7>=2 → True

Therefore f(n)= Ώ(g(n))

2n+5= Ώ(n) for all n>=1, C=2

Theta (Θ):

C1*g(n) <= f(n) <= c2*g(n)

C1*n <= 2n+5 <= c2*n

C1=2, c2=3

2*n <= 2n+5 <= 3*n

For n=1, 2 <= 7 <=3 →