 What is an Algorithm?

An algorithm can be defined as a finite set of steps, which has to be

followed while carrying out a particular problem. It is nothing but a
process of executing actions step by step.

An algorithm is a distinct computational procedure that takes input as a

set of values and results in the output as a set of values by solving the
problem. More precisely, an algorithm is correct, if, for each input
instance, it gets the correct output and gets terminated.

An algorithm unravels the computational problems to output the desired

result. An algorithm can be described by incorporating a natural
language such as English, Computer language, or a hardware language.

Algorithms are named for the 9th century Persian mathematician Al-
Khowarizmi. He wrote a treatise in Arabic in 825 AD, On Calculation
with Hindu Numerals. It was translated into Latin in the 12th century as
Algoritmi de numero Indorum, which title was likely intended to mean
"[Book by] Algoritmus on the numbers of the Indians", where
"Algoritmi" was the translator's rendition of the author's name in the
genitive case; but people misunderstanding the title treated Algoritmi as
a Latin plural and this led to the word "algorithm" (Latin algorismus)
coming to mean "calculation method".
 Characteristics of Algorithms

The main Characteristics or features of Algorithms are;

1. Input: It should externally supply zero or more

quantities.

2. Output: It results in at least one quantity.

3. Definiteness: Each instruction should be clear and ambiguous.

4. Finiteness: An algorithm should terminate after executing a

finite number of steps.

5. Effectiveness: Every instruction should be fundamental to be

carried out, in principle, by a person using only pen and paper.

6. Feasible: It must be feasible enough to produce each instruction.

7. Flexibility: It must be flexible enough to carry out desired changes

with no efforts.

8. Efficient: The term efficiency is measured in terms of time and

space required by an algorithm to implement. Thus, an algorithm
must ensure that it takes little time and less memory space
meeting the acceptable limit of development time.
9. Independent: An algorithm must be language independent, which
means that it should mainly focus on the input and the procedure
required to derive the output instead of depending upon the
language.

 Advantages of Algorithms

1. Effective Communication: Since it is written in a natural

language like English, it becomes easy to understand the step-by-
step delineation of a solution to any particular problem.

2. Easy Debugging: A well-designed algorithm facilitates easy

debugging to detect the logical errors that occurred inside the
program.

3. Easy and Efficient Coding: An algorithm is nothing but a

blueprint of a program that helps develop a program.

4. Independent of Programming Language: Since it is a

language-independent, it can be easily coded by incorporating
any high-level language.

 Disadvantages of Algorithms

Developing algorithms for complex problems would be time-

consuming and difficult to understand.

It is a challenging task to understand complex logic through algorithms.

 Pseudocode

Pseudocode refers to an informal high-level description of the operating

principle of a computer program or algorithm. It uses structural
conventions of a standard programming language intended for human
reading rather than the machine reading.

 Advantages of Pseudocode

Since it is similar to a programming language, it can be quickly

transformed into the actual programming language than a flowchart.

1. The layman can easily understand it.

2. Easily modifiable as compared to the flowcharts.

3. Its implementation is beneficial for structured, designed

elements.

4. It can easily detect an error before transforming it into a code.

 Disadvantages of Pseudocode

1. Since it does not incorporate any standardized style or format, it can

vary from one company to another.

2. Error possibility is higher while transforming into a code.

3. It may require a tool for extracting out the Pseudocode and facilitate
drawing flowcharts.

4. It does not depict the design.

 Difference between Algorithm and Pseudocode

An algorithm is simply a problem-solving process, which is used not only

in computer science to write a program but also in our day to day life. It
is nothing but a series of instructions to solve a problem or get to the
problem's solution. It not only helps in simplifying the problem but also
to have a better understanding of it.

However, Pseudocode is a way of writing an algorithm. Programmers

can use informal, simple language to write pseudocode without following
any strict syntax. It encompasses semi-mathematical statements.

 Need of Algorithms (Why do we need Algorithms?)

1. To understand the basic idea of the problem.

2. To find an approach to solve the problem.

3. To improve the efficiency of existing techniques.

4. To understand the basic principles of designing the algorithms.

5. To compare the performance of the algorithm with respect to other
techniques.

6. It is the best method of description without describing the

implementation detail.

7. The Algorithm gives a clear description of requirements and goal of

the problem to the designer.

8. A good design can produce a good solution.

9. To understand the flow of the problem.

 Analysis of Algorithm

The analysis is a process of estimating the efficiency of an algorithm and

that is, trying to know how good or how bad an algorithm could be.
There are two main parameters based on which we can analyze the
algorithm:

Space Complexity: The space complexity can be understood as the

amount of space required by an algorithm to run to completion.

Time Complexity: Time complexity is a function of input size n that

refers to the amount of time needed by an algorithm to run to
completion.
Let's understand it with an example.

Suppose there is a problem to solve in Computer Science, and in general,

we solve a program by writing a program. If you want to write a program
in some programming language like C, then before writing a program, it
is necessary to write a blueprint in an informal language.

Or in other words, you should describe what you want to include in your
code in an English-like language for it to be more readable and
understandable before implementing it, which is nothing but the concept
of Algorithm.

In general, if there is a problem P1, then it may have many solutions,

such that each of these solutions is regarded as an algorithm. So, there
may be many algorithms such as A1, A2, A3, …, An.

Before you implement any algorithm as a program, it is better to find out

which among these algorithms are good in terms of time and memory.

It would be best to analyze every algorithm in terms of Time that relates

to which one could execute faster and Memory or Space corresponding
to which one will take less memory.

So, the Design and Analysis of Algorithm talks about how to design
various algorithms and how to analyze them. After designing and
analyzing, choose the best algorithm that takes the least time and the
least memory and then implement it as a program in C.
We will be looking more on time rather than space because time is
instead a more limiting parameter in terms of the hardware. It is not
easy to take a computer and change its speed. So, if we are running an
algorithm on a particular platform, we are more or less stuck with the
performance that platform can give us in terms of speed.

However, on the other hand, memory is relatively more flexible. We can

increase the memory as when required by simply adding a memory card.
So, we will focus on time than that of the space.

The running time is measured in terms of a particular piece of hardware,

not a robust measure. When we run the same algorithm on a different
computer which might be faster or use different programming languages
which may be designed to compile faster, we will find out that the same
algorithm takes a different time.

 Types of Time Complexity Analysis

We have three types of analysis related to time complexity, which are:

1. Worst-case time complexity: For 'n' input size, the worst-case time
complexity can be defined as the maximum amount of time needed
by an algorithm to complete its execution. Thus, it is nothing but a
function defined by the maximum number of steps performed on an
instance having an input size of n.
2. Average case time complexity: For 'n' input size, the average-case
time complexity can be defined as the average amount of time needed
by an algorithm to complete its execution. Thus, it is nothing but a
function defined by the average number of steps performed on an
instance having an input size of n.

3. Best case time complexity: For 'n' input size, the best-case time
complexity can be defined as the minimum amount of time needed by
an algorithm to complete its execution. Thus, it is nothing but a
function defined by the minimum number of steps performed on an
instance having an input size of n.

 Complexity of Algorithms

The term algorithm complexity measures how many steps are required
by the algorithm to solve the given problem. It evaluates the order of
count of operations executed by an algorithm as a function of input data
size.

To assess the complexity, the order (approximation) of the count of

operation is always considered instead of counting the exact steps.

O(f) notation represents the complexity of an algorithm, which is also

termed as an Asymptotic notation or "Big O" notation. Here the f
corresponds to the function whose size is the same as that of the input
data. The complexity of the asymptotic computation O(f) determines in
which order the resources such as CPU time, memory, etc. are consumed
by the algorithm that is articulated as a function of the size of the input
data.

The complexity can be found in any form such as constant, logarithmic,

linear, n*log(n), quadratic, cubic, exponential, etc. It is nothing but the
order of constant, logarithmic, linear and so on, the number of steps
encountered for the completion of a particular algorithm. To make it
even more precise, we often call the complexity of an algorithm as
"running time".

 Typical Complexities of an Algorithm

We examine the different types of complexities of an algorithm and one

or more of our algorithm or program will fall into any of the following
categories;

 Constant Complexity:

It imposes a complexity of O(1). It undergoes an execution of a

constant number of steps like 1, 5, 10, etc. for solving a given problem.
The count of operations is independent of the input data size.

 Logarithmic Complexity:

It imposes a complexity of O(log(N)). It undergoes the execution of the

order of log(N) steps. To perform operations on N elements, it often
takes the logarithmic base as 2.
For N = 1,000,000, an algorithm that has a complexity of O(log(N))
would undergo 20 steps (with a constant precision). Here, the
logarithmic base does not hold a necessary consequence for the
operation count order, so it is usually omitted.

 Linear Complexity:

It imposes a complexity of O(N). It encompasses the same number of

steps as that of the total number of elements to implement an operation
on N elements. For example, if there exist 500 elements, then it will take
about 500 steps. Basically, in linear complexity, the number of elements
linearly depends on the number of steps. For example, the number of
steps for N elements can be N/2 or 3*N.

It also imposes a run time of O(n*log(n)). It undergoes the execution of

the order N*log(N) on N number of elements to solve the given problem.
For a given 1000 elements, the linear complexity will execute 10,000
steps for solving a given problem.

 Quadratic Complexity:

It imposes a complexity of O(n2). For N input data size, it undergoes the

order of N2 count of operations on N number of elements for solving a
given problem.
If N = 100, it will endure 10,000 steps. In other words, whenever the
order of operation tends to have a quadratic relation with the input data
size, it results in quadratic complexity.

For example, for N number of elements, the steps are found to be in the
order of 3*N2/2.

 Cubic Complexity:

It imposes a complexity of O(n3). For N input data size, it executes the

order of N3 steps on N elements to solve a given problem.

For example, if there exist 100 elements, it is going to execute 1,000,000

steps.

 Exponential Complexity:

It imposes a complexity of O(2n), O(N!), O(n k), …. For N elements, it

will execute the order of count of operations that is exponentially
dependable on the input data size.

For example, if N = 10, then the exponential function 2N will result in

1024. Similarly, if N = 20, it will result in 1048 576, and if N = 100, it will
result in a number having 30 digits.

The exponential function N! grows even faster; for example, if N = 5 will

result in 120. Likewise, if N = 10, it will result in 3,628,800 and so on.
Since the constants do not hold a significant effect on the order of count
of operation, so it is better to ignore them.

Thus, to consider an algorithm to be linear and equally efficient, it must

undergo N, N/2 or 3*N count of operation, respectively, on the same
number of elements to solve a particular problem

A summary of these complexities is given below:

 Algorithm Design Techniques

An algorithm design technique (or “strategy” or “paradigm”) is a general

approach to solving problems algorithmically that is applicable to a
variety of problems from different areas of computing. Learning these
techniques is of utmost importance for the following reasons.

First, they provide guidance for designing algorithms for new problems,
i.e., problems for which there is no known satisfactory algorithm.
Second, algorithms are the cornerstone of computer science. Every
science is interested in classifying its principal subject, and computer
science is no exception. Algorithm design techniques make it possible to
classify algorithms according to an underlying design idea; therefore,
they can serve as a natural way to both categorize and study algorithms.

While the algorithm design techniques do provide a powerful set of

general approaches to algorithmic problem solving, designing an
algorithm for a particular problem may still be a challenging task. Some
design techniques can be simply inapplicable to the problem in question.
Sometimes, several techniques need to be combined, and there are
algorithms that are hard to pinpoint as applications of the known design
techniques.

 Popular Algorithm Design Techniques

The following is a list of several popular design approaches:

1. Divide and Conquer Approach:

The divide-and-conquer paradigm often helps in the discovery of

efficient algorithms. It is a top-down approach. The algorithms which
follow the divide & conquer techniques involve three steps:

Divide the original problem into a set of subproblems. o Solve every

subproblem individually, recursively.
Combine the solution of the subproblems (top level) into a solution of
the whole original problem.

Following are some standard algorithms that are of the Divide and
Conquer algorithms variety.

Binary Search is a searching algorithm. ...

Quicksort is a sorting algorithm. ...

Merge Sort is also a sorting algorithm. ...

Closest Pair of Points The problem is to find the closest pair of points in
a set of points in x-y plane.

2. Greedy Technique:

Greedy method or technique is an algorithmic paradigm that builds up a

solution piece by piece, always choosing the next piece that offers the
most obvious and immediate benefit. So the problems where choosing
locally optimal also leads to global solution are best fit for Greedy. The
Greedy method is used to solve the optimization problem. An
optimization problem is one in which we are given a set of input values,
which are required either to be maximized or minimized (known as
objective), i.e. some constraints or conditions.

Greedy Algorithm always makes the choice (greedy criteria) looks best at
the moment, to optimize a given objective.
The greedy algorithm doesn't always guarantee the optimal solution
however it generally produces a solution that is very close in value to the
optimal.

 Recursion and Recursive Algorithms (Definitions):

The Merriam-Webster describes recursion as:

“a computer programming technique involving the use of a procedure,

subroutine, function, or algorithm that calls itself one or more times
until a specified condition is met at which time the rest of each repetition
is processed from the last one called to the first.”

Recursion is the process of defining something in terms of itself. A

physical world example would be to place two parallel mirrors facing
each other. Any object in between them would be reflected
recursively.

A recursive algorithm is an algorithm which calls itself with "smaller (or

simpler)" input values, and which obtains the result for the current input
by applying simple operations to the returned value for the smaller (or
simpler) input.

There are two main instances of recursion. The first is when recursion is
used as a technique in which a function makes one or more calls to itself.
The second is when a data structure uses smaller instances of the exact
same type of data structure when it represents itself.
 Why use recursion?

Recursion provides an alternative for performing repetitions of the task

in which a loop is not ideal. Most modern programming languages
support recursion. Recursion serves as a great tool for building out
particular data structures.

So now let’s start with an example exercise of creating a factorial

function.

 Factorial Example

The factorial function is denoted with an exclamation point and is

defined as the product of the integers from 1 to n. Formally, we can state
this as:

n! = n ⋅ (n−1) ⋅ (n−2) … 3 ⋅ 2 ⋅ 1

Note, if n = 0, then n! = 1. This is important to take into account,

because it will serve as our base case.

Take this example:

4!=4⋅ 3⋅ 2⋅ 1=24.
So how can we state this in a recursive manner? This is where the
concept of base case comes in.

Base case is a key part of understanding recursion, especially when it

comes to having to solve interview problems dealing with recursion.
Let’s rewrite the above equation of 4! so it looks like this:

4!=4⋅ (3⋅ 2⋅ 1)=24

Notice that this is the same as:

4!=4⋅ 3!=24

Meaning we can rewrite the formal recursion definition in terms of

recursion like so:

n! = n ⋅ (n−1) !

Note, if n = 0, then n! = 1. This means the base case occurs once n=0,

the recursive cases are defined in the equation above. Whenever you are
trying to develop a recursive solution it is very important to think about
the base case, as your solution will need to return the base case once all
the recursive cases have been worked through. Let’s look at how we can
create the factorial function in Python:

def fact(n):

'''
Returns factorial of n (n!).

Note use of recursion

'''

BASE CASE! if n == 0:
return 1

Recursion!
else:

return n * fact(n-1)

 The Three Laws of Recursion

All recursive algorithms must obey three important laws:

1. A recursive algorithm must have a base case.

2. A recursive algorithm must change its state and move toward the
base case.

3. A recursive algorithm must call itself, recursively.

 Recurrence Relations

A recurrence is an equation or inequality that describes a function in

terms of its values on smaller inputs. To solve a Recurrence Relation
means to obtain a function defined on the natural numbers that satisfy
the recurrence.

For Example, the Worst Case Running Time T(n) of the MERGE SORT
Procedures is described by the recurrence.

T (n) = θ (1) if n=1

2T + θ (n) if n>1

 Methods for Resolving Recurrence Relations

Recurrence relations can be resolved with any of the following four

methods:

1. Substitution Method (Guess-and-Verify)

2. Iteration Method.
3. Recursion Tree Method.
4. Master Method.

 Time Complexities:

1. Best Case Complexity: The selection sort algorithm has a best-

case time complexity of O(n2) for the already sorted array.
2. Average Case Complexity: The average-case time complexity for
the selection sort algorithm is O(n2), in which the existing elements
are in jumbled ordered, i.e., neither in the ascending order nor in the
descending order.
3. Worst Case Complexity: The worst-case time complexity is also
O(n2), which occurs when we sort the descending order of an array
into the ascending order.

 Advantages of Selection Sort

1. It is an in-place algorithm. It does not require a lot of space for

sorting.

2. Only one extra space is required for holding the temporal variable.

3. It performs well on items that have already been sorted

 Disadvantage of selection sort

As the input size increases, the performance of selection sort decreases.

 Radix Sort

Radix Sort is a Sorting algorithm that is useful when there is a constant

'd' such that all keys are d digit numbers. To execute Radix Sort, for p =1
towards 'd' sort the numbers with respect to the Pth digits from the right
using any linear time stable sort.

Radix sort is a sorting technique that sorts the elements digit to digit
based on radix. It works on integer numbers. To sort the elements of the
string type, we can use their hash value. This sorting algorithm makes no
comparison.

The Code for Radix Sort is straightforward. The following procedure

assumes that each element in the n-element array A has d digits, where
digit 1 is the lowest order digit and digit d is the highest-order digit.

 Advantages of Radix Sort:

1. Fast when the keys are short i.e. when the range of the array elements
is less.

2. Used in suffix array construction algorithms like Manber's algorithm

and DC3 algorithm.

3. Radix Sort is stable sort as relative order of elements with equal

values is maintained.

 Disadvantages of Radix Sort:

1. Since Radix Sort depends on digits or letters, Radix Sort is much less
flexible than other sorts. ...

2. The constant for Radix sort is greater compared to other sorting

algorithms. It takes more space compared to Quicksort which is in-
place sorting.

 Applications of Radix Sort

Here are a few applications of the radix sort algorithm:

Radix sort can be applied to data that can be sorted lexicographically,

such as words and integers. It is also used for stably sorting strings.

It is a good option when the algorithm runs on parallel machines,

making the sorting faster. To use parallelization, we divide the input into
several buckets, enabling us to sort the buckets in parallel, as they are
independent of each other.

It is used for constructing a suffix array. (An array that contains all the
possible suffixes of a string in sorted order is called a suffix array.

 Stability in Sorting

Stable sort algorithms sort equal elements in the same order that they
appear in the input. For example, in the card sorting example to the right,
the cards are being sorted by their rank, and their suit is being ignored.
This allows the possibility of multiple different correctly sorted versions
of the original list. Stable sorting algorithms choose one of these,
according to the following rule: if two items compare as equal (like the
two 5 cards), then their relative order will be preserved, i.e. if one comes
before the other in the input, it will come before the other in the output.

Stability is important to preserve order over multiple sorts on the same

data set. For example, say that student records consisting of name and
class section are sorted dynamically, first by name, then by class section.
If a stable sorting algorithm is used in both cases, the sort-by-class-
section operation will not change the name order; with an unstable sort,
it could be that sorting by section shuffles the name order, resulting in a
nonalphabetical list of students.

 Why is stable sort useful?

A stable sorting algorithm maintains the relative order of the

items with equal sort keys. An unstable sorting algorithm does not.
In other words, when a collection is sorted with a stable sorting
algorithm, items with the same sort keys preserve their order after the
collection is sorted.

Suppose you need to sort following key-value pairs in the increasing

order of keys:

INPUT: (4,5), (3, 2) (4, 3) (5,4) (6,4)

Now, there is two possible solution for the two pairs where the key is
same i.e.

(4,5) and (4,3) as shown below:

OUTPUT1: (3, 2), (4, 5), (4,3), (5,4), (6,4)

OUTPUT2: (3, 2), (4, 3), (4,5), (5,4), (6,4)

The sorting algorithm which will produce the first output will be known
as stable sorting algorithm because the original order of equal keys are
maintained, you can see that (4, 5) comes before (4,3) in the sorted order,
which was the original order i.e. in the given input, (4, 5) comes before
(4,3) .

On the other hand, the algorithm which produces second output will
know as an unstable sorting algorithm because the order of objects with
the same key is not maintained in the sorted order. You can see that in
the second output, the (4,3) comes before (4,5) which was not the case in
the original input.

 Divide and Conquer Algorithms

Divide and Conquer is an algorithmic pattern. In algorithmic methods,

the design is to take a dispute on a huge input, break the input into
minor pieces, decide the problem on each of the small pieces, and then
merge the piecewise solutions into a global solution. This mechanism of
solving the problem is called the Divide & Conquer Strategy.
Divide and Conquer algorithm consists of a dispute using the following
three steps.

1. Divide the original problem into a set of subproblems.

2. Conquer: Solve every subproblem individually, recursively.

3. Combine: Put together the solutions of the subproblems to get the

solution to the whole problem.

Generally, we can follow the divide-and-conquer approach in a three-

step process.

Examples: The specific computer algorithms are based on the Divide &
Conquer approach:

1. Maximum and Minimum Problem

2. Binary Search

3. Sorting (merge sort, quick sort)

4. Tower of Hanoi.

 Dynamic Programming

Dynamic programming is a technique that breaks the problems into sub-

problems, and saves the result for future purposes so that we do not need
to compute the result again. The subproblems are optimized to optimize
the overall solution is known as optimal substructure property. The main
use of dynamic programming is to solve optimization problems. Here,
optimization problems mean that when we are trying to find out the
minimum or the maximum solution of a problem. The dynamic
programming guarantees to find the optimal solution of a problem if the
solution exists.

From the definition of dynamic programming, it is a technique for

solving a complex problem by first breaking it into a collection of simpler
sub problems, solving each subproblem just once, and then storing their
solutions to avoid repetitive computations.

Let's understand this approach through an example.

Consider an example of the Fibonacci series. The following series is

the Fibonacci series:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ,…

The numbers in the above series are not randomly calculated.

Mathematically, we could write each of the terms using the below
formula:

F(n) = F(n-1) + F(n-2),

With the base values F(0) = 0, and F(1) = 1.

To calculate the other numbers, we follow the above relationship. For
example, F(2) is the sum f(0) and f(1), which is equal to 1.

 How can we calculate F(20)?

The F(20) term will be calculated using the nth formula of the Fibonacci
series.

The below figure shows that how F(20) is calculated.

As we can observe in the above figure that F(20) is calculated as the sum
of F(19) and F(18).

In the dynamic programming approach, we try to divide the problem

into the similar subproblems. We are following this approach in the
above case where F(20) into the similar subproblems, i.e., F(19) and
F(18). If we revisit the definition of dynamic programming that it says
the similar subproblem should not be computed more than once. Still, in
the above case, the subproblem is calculated twice. F(18) is calculated
two times; similarly, F(17) is also calculated twice. However, this
technique is quite useful as it solves the similar subproblems, but we
need to be cautious while storing the results because we are not
particular about storing the result that we have computed once, as it can
lead to a wastage of resources.

 How Dynamic Programming Works

The following are the steps that the dynamic programming follows:

1. It breaks down the complex problem into simpler

subproblems. It finds the optimal solution to these sub-
problems.

2. It stores the results of subproblems (memoization). The process of

storing the results of subproblems is known as memorization.

3. It reuses them so that same sub-problem is calculated more than

once.

Finally, calculate the result of the complex problem.

The above five steps are the basic steps for dynamic programming. The
dynamic programming is applicable that are having properties such as:
Those problems that are having overlapping subproblems and optimal
substructures. Here, optimal substructure means that the solution of
optimization problems can be obtained by simply combining the optimal
solution of all the subproblems.

In the case of dynamic programming, the space complexity would be

increased as we are storing the intermediate results, but the time
complexity would be decreased.

 Approaches of dynamic programming

There are two approaches to dynamic programming:

Top-down approach
Bottom-up approach

 Top-down approach

The top-down approach follows the memorization technique, while

bottom-up approach follows the tabulation method. Here memorization
is equal to the sum of recursion and caching. Recursion means calling
the function itself, while caching means storing the intermediate results.

 Advantages of Top-down aproach

1. It is very easy to understand and implement.

2. It solves the subproblems only when it is required.
It is easy to debug.

 Disadvantages of Top-down approach

1. It uses the recursion technique that occupies more memory in the call
stack. Sometimes when the recursion is too deep, the stack overflow
condition will occur.

2. It occupies more memory that degrades the overall performance.

 Computational Complexity Theory

An algorithm’s performance is always important when you try to solve a

problem. An algorithm won’t do you much good if it takes too long or
requires too much memory or other resources to actually run on a
computer.

Computational complexity theory, or just complexity theory, is the study

of the difficulty of computational problems. Rather than focusing on
specific algorithms, complexity theory focuses on problems.

For example, the mergesort algorithm can sort a list of N numbers in

O(N log N) time. Complexity theory asks what you can learn about the
task of sorting in general, not what you can learn about a specific
algorithm. It turns out that you can show that any sorting algorithm that
sorts by using comparisons must use at least N × log(N) time in the
worst case.
Complexity theory is a large and difficult topic, so there’s no room here
to cover it fully. However, every programmer who studies algorithms
should know at least something about complexity theory in general and
the two sets P and NP in particular. This module introduces complexity
theory and describes what these important classes of problems are.

 Notations Used

The Big O notation describes how an algorithm’s worst-case

performance increases as the problem’s size increases.

For most purposes, that definition is good enough to be useful, but in

complexity theory Big O notation has a more technical definition.

If an algorithm’s run time is f (N) , then the algorithm has Big O

performance of g(N) if f(N) < g(N) x k for some constant k and for N
large enough. In other words, the function g(N) is an upper bound for
the actual run-time function f(N). .

Two other notations similar to Big O notations are sometimes useful

when discussing algorithmic complexity.

Big Omega notation, written Ω(g(N)), means that the run-time

function is bounded below by the function g(N) . For example, as
explained a moment ago, N log(N) is a lower bound for algorithms that
sort by using comparisons, so those algorithms are Ω(N logN) .
Big Theta notation, written ϴ(g(N)) , means that the run-time
function is bounded both above and below by the function g(N) . For
example, the mergesort algorithm’s run time is bounded above by O(N
log N), and the run time of any algorithm that sorts by using
comparisons is bounded below by Ω(N log N), so mergesort has
performance ϴ(N log N).

In summary,

1. Big O notation gives an upper bound,

2. Big Omega gives a lower bound, and

3. Big Theta gives an upper and lower bound.

Some algorithms however, have different upper and lower bounds.

For example, like all algorithms that sort by using comparisons,

quicksort has a lower bound of Ω(N log N).

In the best and expected cases, quicksort’s performance actually is Ω(N

log N).

In the worst case, however, quicksort’s performance is O(N2).

The algorithm’s lower and upper bounds are different, so no function

gives quicksort a Big Theta notation.
 Deterministic Algorithms

A Deterministic algorithm is an algorithm which, given a particular input

will always produce the same output, with the underlying machine
always passing through the same sequence of states.

In other words, Deterministic algorithm will always come up with the

same result given the same inputs.

Deterministic algorithms are by far the most studied and familiar kind of
algorithm as well as one of the most practical, since they can be run on
real machines efficiently.

Formally, a deterministic algorithm computes a mathematics function; a

function has a unique value for any input in its domain, and the
algorithm is a process that produces this particular value as output.

Deterministic algorithms can be defined in terms of a state machine: a

state describes what a machine is doing at a particular instant in time.
State machines pass in a discrete manner from one state to another. Just
after we enter the input, the machine is in its initial state or start state. If
the machine is deterministic, this means that from this point onwards,
its current state determines what its next state will be; its course through
the set of states is predetermined. Note that a machine can be
deterministic and still never stop or finish, and therefore fail to deliver a
result.
Examples of particular abstract machines which are deterministic
include the deterministic Turing machine and deterministic finite
automate.

 Facts about Deterministic Algorithms

Deterministic algorithm is the algorithm which, given a particular input

will always produce the same output, with the underlying machine
always passing through the same sequence of states.

In deterministic algorithm the path of execution for algorithm is same in

every execution.

 What Makes An Algorithm Non-deterministic?

A variety of factors can cause an algorithm to behave in a way which is

not deterministic, or non-deterministic:

1. If it uses external state other than the input, such as user input, a
global variable, a hardware timer value, a random value, or stored
disk data.

2. If it operates in a way that is timing-sensitive, for example if it has

multiple processors writing to the same data at the same time. In this
case, the precise order in which each processor writes its data will
affect the result.

3. If a hardware error causes its state to change in an unexpected way.

 Facts About Non-deterministic Algorithms

A Non-deterministic algorithm is the algorithms in which the result of

every algorithm is not uniquely defined and result could be random.

In a Non-Deterministic algorithm the path of execution is not same for

algorithm in every execution and could take any random path for its
execution.

Non deterministic algorithms are classified as non-reliable algorithms

for a particular input the machine will give different output on different
executions.

In Non-Deterministic Algorithms, the machine executing each operation

is allowed to choose any one of these outcomes subjects to a
determination condition to be defined later.

As outcome is not known and is non-consistent on different executions

so Non-Deterministic algorithm could not get executed in polynomial
time.

 Deterministic versus Non-deterministic Algorithms

The following table gives some vital differences between a

Deterministic and a Non Deterministic algorithm.
BASIS OF NON-DETERMINISTIC
DETERMINISTIC ALGORITHM
COMPARISON ALGORITHM

Deterministic algorithm is the

Non-deterministic algorithm is the
algorithm which, given a particular
algorithms in which the result of
input will always produce the same
Description. every algorithm is not uniquely
output, with the underlying machine
defined and result could be
always passing through the same
random.
sequence of states.

In Non-Deterministic algorithm the

In deterministic algorithm the path of path of execution is not same for
Path Of
execution for algorithm is same in algorithm in every execution and
Execution
every execution. could take any random path for its
execution.

On the basis of execution and

Non deterministic algorithms are
outcome in case of Deterministic
classified as non-reliable
Basis Of algorithm, they are also classified as
algorithms for a particular input the
Comparison reliable algorithms as for a particular
input instructions the machine will machine will give different output
 on different executions.
give always the same output.

In Deterministic Algorithms In Non-Deterministic Algorithms,

execution, the target machine the machine executing each
executes the same instruction and operation is allowed to choose any
Operation
results same outcome which is not one of these outcomes subjects to a
dependent on the way or process in determination condition to be
which instruction get executed. defined later.

As outcome is not known and is

As outcome is known and is
non-consistent on different
consistent on different executions so
Output executions so Non-Deterministic
deterministic algorithm takes
algorithm could not get executed in
polynomial time for their execution.
polynomial time.

 NP (Non-Deterministic Polynomial) Problem

The set of all decision-based problems came into the division of NP

Problems who can't be solved or produced an output within
polynomial time but verified in the polynomial time. NP class
contains P class as a subset. NP problems are very hard to solve.
Note: The term “NP” does not mean “Not Polynomial”. Originally, the term
meant “Non-Deterministic Polynomial. It means according to the one input
number of output will be produced.

 Definition of P Problems

1. Definition of P class Problem: - The set of decision-based problems

come into the division of P Problems who can be solved or produced an
output within polynomial time. P problems being easy to solve

2. Definition of Polynomial time: - If we produce an output according

to the given input within a specific amount of time such as within a
minute, hours. This is known as Polynomial time.

3. Definition of Non-Polynomial time: - If we produce an output

according to the given input but there are no time constraints is known
as Non-Polynomial time. But yes output will produce but time is not
fixed yet.

 Decision Based Problems

A problem is called a decision problem if its output is a simple "yes" or "no"
(or you may need to represent it as true/false, 0/1, accept/reject.) We will
phrase many optimization problems as decision problems.

For example, Greedy method, D.P., given a graph G= (V, E) if there exists
any Hamiltonian cycle.

 NP-hard Problems

A problem is NP-hard if an algorithm for solving it can be translated into

one for solving any NP- problem (nondeterministic polynomial time)
problem. NP-hard therefore means "at least as hard as any NP-problem,"
although it might, in fact, be harder.

A problem must satisfy the following points to be classified as NP-hard

If we can solve this problem in polynomial time, then we can solve all NP
problems in polynomial time

If you convert the issue into one form to another form within the
polynomial time
 NP-complete Problems:

A problem is NP-complete when: it is a problem for which the correctness

of each solution can be verified quickly and a brute-force search algorithm
can find a solution by trying all possible solutions.

A problem is in the class NP-complete if it is in NP and is as hard as any

problem in NP. A problem is NP-hard if all problems in NP are polynomial
time reducible to it, even though it may not be in NP itself. These problems
are called NP-complete.

Many significant computer-science problems belong to this class—e.g., the

traveling salesman problem, satisfiability problems, and graph-covering
problems.

 Pictorial representation of all NP classes

 Tractable and Intractable Problems

1. Tractable Problem:

A problem that is solvable by a polynomial-time algorithm. The upper

bound is polynomial.

Here are examples of tractable problems (ones with known

polynomial-time algorithms):

– Searching an unordered list

– Searching an ordered list

– Sorting a list

– Multiplication of integers (even though there’s a gap)

– Finding a minimum spanning tree in a graph (even though there’s a gap)

 Intractable Problem:

A problem that cannot be solved by a polynomial-time algorithm. The

lower bound is exponential.
From a computational complexity stance, intractable problems are
problems for which there exist no efficient algorithms to solve them. Most
intractable problems have an algorithm that provides a solution, and that
algorithm is the brute-force search.

This algorithm, however, does not provide an efficient solution and is,
therefore, not feasible for computation with anything more than the
smallest input.