Comparing Algorithms

In this lesson, we are going to learn how two or more algorithms may be
compared.

We'll cover the following

• Introduction(https://www.educative.io/courses/algorithms-coding-
interviews-cpp/m2M5016E4mG#introduction)
• Important (https://www.educative.io/courses/algorithms-coding-
Criteria: Time interviews-cpp/m2M5016E4mG#important-criteria-time-
and Space and-space)

• Comparing (https://www.educative.io/courses/algorithms-coding-
Execution interviews-cpp/m2M5016E4mG#comparing-execution-time)
Time
• Experimental(https://www.educative.io/courses/algorithms-coding-
Evaluation interviews-cpp/m2M5016E4mG#experimental-evaluation)

• Analytical (https://www.educative.io/courses/algorithms-coding-
Evaluation interviews-cpp/m2M5016E4mG#analytical-evaluation)
• Best Case (https://www.educative.io/courses/algorithms-coding-
Analysis interviews-cpp/m2M5016E4mG#best-case-analysis)
• Worst- (https://www.educative.io/courses/algorithms-coding-
Case interviews-cpp/m2M5016E4mG#worst-case-analysis)
Analysis
• Average (https://www.educative.io/courses/algorithms-coding-
Case interviews-cpp/m2M5016E4mG#average-case-analysis)
Analysis
• The (https://www.educative.io/courses/algorithms-coding-
Verdict interviews-cpp/m2M5016E4mG#the-verdict)

• Analyzing a (https://www.educative.io/courses/algorithms-coding-
Simple C++ interviews-cpp/m2M5016E4mG#analyzing-a-simple-c-
Program program)
 Note: If you have already covered our course Data Structures in C++: An
Interview Refresher
(https://www.educative.io/collection/5642554087309312/564627607912
4480?authorName=Coderust) you may skip this chapter, since a chapter
on Algorithmic Complexity Measures was a part of the said course, too.

 

Introduction #
(https://www.educative.io/courses/algorithms-
coding-interviews-
cpp/m2M5016E4mG#introduction)
There are typically several different algorithms to solve a given computational
problem. It is natural, then, to compare these alternatives. But how do we know
if algorithm A is better than algorithm B?

Important Criteria: Time and Space #

(https://www.educative.io/courses/algorithms-
coding-interviews-
cpp/m2M5016E4mG#important-criteria-time-
and-space)
One important factor that determines the “goodness” of an algorithm is the
amount of time it takes to solve a given problem. If algorithm A takes less time
to solve the same problem than does algorithm B, then algorithm A is
considered better.

Another important factor in comparing two algorithms is the amount of memory

required to solve a given problem. The algorithm that requires less memory is
considered better.

Comparing Execution Time #

(https://www.educative.io/courses/algorithms-
coding-interviews-
cpp/m2M5016E4mG#comparing-execution-time)
For the remainder of this lesson, we will focus on the first factor, i.e., execution
time. How do we compare the execution time of two algorithms?

Experimental Evaluation #
(https://www.educative.io/courses/algorithms-coding-
interviews-cpp/m2M5016E4mG#experimental-evaluation)
Well, we could implement the two algorithms and run them on a computer
while measuring the execution time. The algorithm with less execution time
wins. One thing is for sure; this comparison must be made in a fair manner.
Let’s try to punch holes into this idea:

An algorithm might take longer to run on an input of greater size. Thus,

algorithms being compared must be tested on the same input size. But
that’s not all. Due to the presence of conditional statements, for a given
input size, even the same algorithm’s running time may vary with the
actual input given to it. This means that the algorithms being compared
must be tested on the same input. Since one algorithm may be
disadvantaged over another for a specific input, we must test the
algorithms exhaustively on all possible input values. This is just not
possible.
The algorithms being compared must first be implemented. What if the
programmer comes up with a better implementation of one algorithm than
the other? What if the compiler optimizes one algorithm more than it does
the other? There’s so much that can compromise the fairness at this stage.
The programs implementing the two algorithms must be tested on exactly
the same hardware and software environment. Far fetched as it may be, we
could assign a single machine for all scientists to test their algorithms on.
Even if we did, the task scheduling in modern-day operating systems
involves a lot of randomness. What if the program corresponding to “the
best” algorithm encounters an excessive number of the hardware
interrupts? It is impossible to guarantee the same hardware/software
environment to ensure a fair comparison.
Analytical Evaluation #
(https://www.educative.io/courses/algorithms-coding-
interviews-cpp/m2M5016E4mG#analytical-evaluation)
The above list highlights some of the factors that make fair experimental
evaluation of algorithms impossible. Instead, we are forced to make an
analytical/theoretical comparison. The two key points that we hold on to, from
the previous discussion, is that we must compare algorithms for the same input
size and consider all possible inputs of the given size. Here is how it is done.

We assume a hypothetical computer on which some primitive operations are

executed in a constant amount of time. We consider a specific input size, say, n.
We then count the number of primitive operations executed by an algorithm for
a given input. The algorithm that results in fewer primitive operations is
considered better.

What constitutes a primitive operation, though? You can think of these as

simple operations that are typically implemented as processor instructions.
These operations include assignment to a variable or array index, reading from a
variable or array index, comparing two values, arithmetic operations, a function
call, etc.

What is not considered a primitive operation? Consider a function call, for

instance. When a function is called, all the statements in that function are
executed. So, we can’t consider each function invocation as a single primitive
operation. Similarly, displaying an entire array is not a primitive operation.

The number of times conditional statements run depends on the actual input
values. Sometimes a code block is executed, some times it isn’t. So, how do we
account for conditional statements? We can adopt one of three strategies: best
case analysis, average-case analysis, and worst-case analysis.

Best Case Analysis # (https://www.educative.io/courses/algorithms-coding-

interviews-cpp/m2M5016E4mG#best-case-analysis)

In the best case analysis, we consider the specific input that results in the
execution of the fewest possible primitive operations. This gives us a lower
bound on the execution time of that algorithm for a given input size.
Worst-Case Analysis # (https://www.educative.io/courses/algorithms-coding-
interviews-cpp/m2M5016E4mG#worst-case-analysis)

In the worst-case analysis, we consider the specific input that results in the
execution of the maximum possible primitive operations. This gives us an upper
bound on the execution time of that algorithm for a given input size.

Average Case Analysis # (https://www.educative.io/courses/algorithms-coding-

interviews-cpp/m2M5016E4mG#average-case-analysis)

In the average case analysis, we try to determine the average number of

primitive operations executed for all possible inputs of a given size. This is not
as easy as it may sound to the uninitiated. In order to compute the average-case
running time of an algorithm, we must know the relative frequencies of all
possible inputs of a given size. We compute the weighted average of the number
of primitive operations executed for each input. But how can we accurately
predict the distribution of inputs? If the algorithm encounters a different
distribution of inputs in the field, our analysis is useless.

The Verdict # (https://www.educative.io/courses/algorithms-coding-interviews-

cpp/m2M5016E4mG#the-verdict)

The best-case analysis has limited value because what if you deploy that
algorithm and the best case input rarely occurs. We feel that the worst-case
analysis is more useful because whatever answer it gives you, you can be sure
that no matter what, algorithm A wouldn’t incur more time than that. Unless
otherwise specified, our analysis in this course will be worst-case running time.

The running time of an algorithm computed in the aforementioned way is also

known as its time complexity. Another term that you will often hear is an
algorithm’s space complexity. The space complexity of an algorithm is the
amount of additional or auxiliary memory space that the algorithm requires.
This is memory space other than the actual input itself. We will see examples of
evaluating the space complexity of an algorithm later on in this course.

Analyzing a Simple C++ Program #

(https://www.educative.io/courses/algorithms-
coding-interviews-
cpp/m2M5016E4mG#analyzing-a-simple-c-
program)
Suppose that instead of an algorithm, we were given C++ code, instead. Here’s
how we can analyze the algorithm underlying the given program. Let’s count the
number of primitive operations in the program given below:

#include <iostream>
using namespace std;

int main() {
int x = 0;
x += 1;
cout << x;
return x;
}

Lines 1 and 2 are compiler directives, so we just ignore them. There is a variable
assignment on line number 5, so that’s 1 primitive operation. A variable’s value
is accessed, an addition is performed, and then an assignment takes place on
line 6, so that’s three primitive operations. A variable’s value is accessed and
then displayed on line 7, so that’s two primitive operations. A variable’s value is
accessed and returned on line number 8, so that’s two primitive operations. So,
overall there are 8 primitive operations in the above program. This analysis is
also presented in the following table:

Primitive No. of Primitive

Line No.
operations Operations

5 Variable assignment 1

Variable access,
6 addition, variable 3
assignment
 Primitive No. of Primitive
Line No.
operations Operations

Variable access,
7 2
display value

Variable access, return

8 2
a value

For the above program, the time complexity can be specified as:

Time Complexity = 1 + 3 + 2 + 2 ⇒ 8

Note that there is no notion of input size in this simple example since there is no
input to this program. Algorithms for which the time complexity is independent
of the input size are known as constant-time algorithms.

In the next lesson, we will analyze the running time of an algorithm involving a