Unit –I

Introduction to Algorithm
and Data Structures
BY MRS. MANISHA DESAI
Introduction to Algorithm and Data Structures
Introduction: From Problem to Data Structure (Problem, Logic, Algorithm, and Data Structure).
Data Structures: Data, Information, Knowledge, and Data structure, Abstract Data Types (ADT), Data
Structure Classification (Linear and Non-linear, Static and Dynamic, Persistent and Ephemeral data
structures)
Algorithms: Problem Solving, Introduction to algorithm, Characteristics of algorithm, Algorithm design
tools: Pseudo-code and flowchart
Complexity of algorithm: Space complexity, Time complexity, Asymptotic notation- Big-O, Theta and
Omega, Finding complexity using step count method, Analysis of programming constructs-Linear,
Quadratic, Cubic, Logarithmic.
Algorithmic Strategies- Introduction to algorithm design strategies- Divide and Conquer, and Greedy
strategy.
Case Study: Multiplication technique by the mathematician Carl Friedrich Gauss and Karatsuba
algorithm for fast multiplication.
Introduction: From Problem to Data Structure (Problem, Logic,
Algorithm, and Data Structure).
Data Structures: Data, Information, Knowledge, and Data structure,
Problem
Problem :
“Problem is defined as situation or condition which needs to solve to achieve
goal”
Steps in Problem Solving :
1.Define the problem
2.Data gathering
3.Decide effective solution
4.Implement and evaluate the solution
5.Review the result.
Logic
Logic
A method of human thoughts that involves thinking in linear step by step
manner about how a problem can be solved
Logic is a language for reasoning. It is a collection of rules we use when
doing reasoning.
Example: Mary’s mom has four children
◦ The First child is called April
◦ The second May
◦ The Third June
What is the name of the fourth child..?
Algorithm
Algorithms is set of instructions which are written in simple English language.
An algorithm is defined as a step-by-step procedure or method for solving a
problem by a computer in a finite number of steps.”
From the data structure point of view, following are some important categories
of algorithms −
Search − Algorithm to search an item in a data structure.
Sort − Algorithm to sort items in a certain order.
Insert − Algorithm to insert item in a data structure.
Update − Algorithm to update an existing item in a data structure.
Delete − Algorithm to delete an existing item from a data structure.
Characteristics of Algorithm
1. Unambiguous − Algorithm should be clear and unambiguous. Each of its steps
(or phases), and their inputs/outputs should be clear and must lead to only one
meaning.
2. Input − An algorithm should have 0 or more well-defined inputs.
3. Output − An algorithm should have 1 or more well-defined outputs, and
should match the desired output.
4. Finiteness − Algorithms must terminate after a finite number of steps.
5. Feasibility − Should be feasible with the available resources.
6. Independent − An algorithm should have step-by-step directions, which should
be independent of any programming code.
Example of Algorithm

Let's try to learn algorithm-writing by using an example.

Problem − Design an algorithm to add two numbers and display the result.
Step 1 − START
Step 2 − declare three integers a, b & c
Step 3 − define values of a & b
Step 4 − add values of a & b
Step 5 − store output of step 4 to c
Step 6 − print c
Step 7 − STOP
Data Structure
Data Structure is a way to store and 1, Rajesh, 67, 2, Ankita, 78, 3,Neha,
88,4 , Nitesh, 81
organize data in the memory so that it
can be used efficiently.
Data Organization in .CSV File
It is an organized collection of data which
perform a particular operation.
Roll No. Name Marks
Example: 1 Rajesh 67
2 Ankita 78
Array, Stack, Queue , Tree etc. 3 Neha 88
4 Nitesh 81

Data Organization in .Excel File

 Data
Data :
◦ “Data is nothing but collection raw facts or figures.”
◦ Data is meaningless and in unprocessed form.
◦ It is Derived from Latin word ”Datum” which means “something given”
◦ Data is measured in Bits, Nibble, Byte, KB, MB, GB, TB etc.
◦ Example: 411038

Data Object :
“Data object is a region of storage that contains a value or group of
value”
Stages of Processing

The result and the unprocessed data is stored in the

storage devices
Information

It is processed Data which is accurate, understandable & relevant.

It is meaningful & usually output of any system.
Information = data + Meaning
It is derived from French verb “informare” which means “to inform”
Example:
◦ Grade of student in particular subject.
◦ Percentage calculation of student.
Knowledge
It is the process of applying set of rules to information to help us make
decisions.
Knowledge is accurate as it conveys the true situation.
It provides all the necessary data.
Example
Data
◦ Dog, cat, cow
◦ 165, 175.2, 186.3, 164.3, 169.3
Information
◦ Dog, cat, cow is a list of household pets
◦ 165, 175.2, 186.3, 164.3, 169.3 are the height of 15-year old students.
Knowledge
◦ A tiger is not a household pet as it is not on the list, and it lives in the wild
forest.
◦ The tallest student is 186.3cm.
Data, Information & Knowledge
100 miles is quite a far distance

Knowledge

100 miles

Information

100

Data
NEED OF DATA STRUCTURE

1. Stores huge data

2. Stores data in systematic way
3. Retains logical relationship
4. Provides various structure
5. Static and dynamic formats
6. Better algorithms
Abstract data types
• What does ‘abstract’ mean?
• From Latin: to ‘pull out’—the essentials
– To defer or hide the details
– Abstraction emphasizes essentials and defers the details, making engineering
artifacts easier to use
Abstract data types
• I don’t need a mechanic’s understanding of what’s under a
car’s hood in order to drive it
– What’s the car’s interface?
– What’s the implementation?
Abstract Data Types
• Abstract data type (ADT)
– An ADT is composed of
• A collection of data
• A set of operations on that data
– Specifications of an ADT indicate
• What the ADT operations do, not how to implement
them
– Implementation of an ADT
• Includes choosing a particular data structure
ADT = properties + operations
• An ADT describes a set of objects sharing the same properties
and behaviors
– The properties of an ADT are its data (representing the internal state of
each object
• double d; -- bits representing exponent & mantissa are its data or state
– The behaviors of an ADT are its operations or functions (operations on
each instance)
• sqrt(d) / 2; //operators & functions are its behaviors

• Thus, an ADT couples its data and operations

– OOP emphasizes data abstraction
Formal, language-independent ADTs
• An ADT is a formal description, not code;
independent of any programming language
– Why is code independence a good idea?
• Promotes design by contract:
– Specify responsibilities of suppliers and clients
explicitly, so they can be enforced, if necessary
Figure: A wall of ADT operations isolates a data structure from the program that uses it
Array as an ADT
Objects: A set of pairs <index, value> where for each value of index
there is a value from the set item. Index is a finite ordered set of one or
more dimensions, for example, {0, … , n-1} for one dimension,
{(0,0),(0,1),(0,2),(1,0),(1,1),(1,2),(2,0),(2,1),(2,2)} for two dimensions,
etc.
Methods:
for all A  Array, i  index, x  item, j, size  integer
Array Create(j, list) ::= return an array of j dimensions where list is a
j-tuple whose kth element is the size of the
kth dimension. Items are undefined.
Item Retrieve(A, i) ::= if (i  index) return the item associated with
index value i in array A
else return error
Array Store(A, i, x) ::= if (i in index)
return an array that is identical to array
A except the new pair <i, x> has been
inserted else return error
Primitive Operations on Arrays
1. Create : Creating a new array : This needs the information regarding the
type of data objects and the number of data objects to be stored.
2. Store(array, index, value) : Store a value at particular position : This
needs the name of the array, the position in the array and the value to be
stored at that position.
3. Retrieve ( array, index ) : Retrieve a value at a particular position : This
operation requires the array name and an index and it returns the
corresponding value.
TYPES OF DATA STRUCTURE
Data
Structure

Non-
Primitives
Primitives
data structure
data structure

Files
Integer Float Char Pointer Arrays Lists

Linear Non Linear

Stack Queue Graphs Trees

TYPES OF DATA STRUCTURE
1. Primitives data structure :
“Primitive data structures are those which are predefined Char : 1 byte
way of storing data by the system. ”
e.g. int, char, float etc

2. Non-primitive data structure :

“The data types that are derived from primary data types are
known as non-Primitive data types. These datatype are used
to store group of values.” Char Array of size 4 : 4 Byte

e.g. struct, array, linklist, stack, tree , graph etc.

Linear and Non-Linear Data Structure
1. Linear Data Structure :
“Linear data structure traverses the data elements
sequentially, in which only one data element can
directly be reached”
Ex: Arrays, Linked Lists, stack, queue.

2. Non-Linear Data Structure :

“Every data item is attached to several other data
items in a way that is specific for reflecting
relationships.”
Ex: Graph , Tree
Static and Dynamic Data Structure
1. Static data structure :
“A static data structure is an organization or
collection of data in memory that is fixed in size.”
Ex: Arrays

2. Dynamic Data Structure :

“ In Dynamic data structure the size of the structure
in not fixed and can be modified during the
operations performed on it”
Ex: Linked list
= 1 byte of used memory
Persistent and Ephemeral Data Structure
1. Persistent data structure : 1 2 3 4 5
“A persistent data structure is a data structure that
always preserves the previous version of itself when List 1 List 2
it is modified..”
1 2 3 4 5
Ex: Linked list, tree
New List after concatenation of List 1 & 2
2. Ephemeral Data Structure :
CPU
“ An ephemeral data structure is one of which only
one version is available at a time(it does not preserve
previous version).” Cache
Ex: RAM , Cache memory
Main
Memory
Algorithms: Problem Solving, Introduction to algorithm,
Characteristics of algorithm, Algorithm design tools: Pseudo-code
and flowchart
 ALGORITHM – PROBLEM SOLVING
COMPUTER :
“Computer is multi purpose Electronic Machine which is used for storing , organizing and processing data
by set of program

Problem :
“Problem is defined as situation or condition which needs to solve to achive goal”

Steps in Problem Solving :

1.Define the problem
2.Data gathering
3.Decide effective solution
4.Implement and evaluate the solution
5.Review the result.
PROBLEM SOLVING TECHNIQUES
There are two types :
1.Algorithmic
2.Flowchart

Algorithms is set of instructions which are written in simple English language.

Flowchart is graphical representation of the algorithms.
 INTRODUCTION OF ALGORITHMS

DEFINITION :
“An algorithm is defined as a step-by-step procedure or method for solving a problem by a computer
in a finite number of steps.”

From the data structure point of view, following are some important categories of algorithms −
Search − Algorithm to search an item in a data structure.
Sort − Algorithm to sort items in a certain order.
Insert − Algorithm to insert item in a data structure.
Update − Algorithm to update an existing item in a data structure.
Delete − Algorithm to delete an existing item from a data structure.
CHARACTRISTICS OF ALGORITHM
1.Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or phases), and
their inputs/outputs should be clear and must lead to only one meaning.
2. Input − An algorithm should have 0 or more well-defined inputs.
3. Output − An algorithm should have 1 or more well-defined outputs, and should match the
desired output.
4. Finiteness − Algorithms must terminate after a finite number of steps.
5. Feasibility − Should be feasible with the available resources.
6. Independent − An algorithm should have step-by-step directions, which should be
independent of any programming code.
 ALGORITHM DESIGN TOOLS
There can be two tools :
1. Flowchart
2. Pseudo Code

Flowchart :
“ Flowchart is graphical representation of the algorithms”

Pseudo Code :
“It is simply an implementation of an algorithm in the form of annotations and informative text
written in plain English.
Symbols used in flowchart
Example : flowchart & Pseudocode
Complexity of algorithm: Space complexity, Time
complexity, Asymptotic notation- Big oh, Theta and Omega
Performance Analysis
Is it giving the exact solution for a problem?
How much space it takes?
How much time it takes? Is my
Is it easy to understand algorithm
Best..?
Is it easy to implement
Etc.……..
 Complexity of algorithm
There are two important complexity measures:
Time complexity :
Total time which is required for an algorithm to complete its process.
Independent of programming language or implementation details
 It allows comparing the algorithms to check which one is the efficient one.
Space complexity :
The amount of computer memory required for an algorithm to solve the Problem(successful
execution)
Memory space is generally considered as space of primary memory.
Performance Evolution done in two measure phases
A Priori Estimates (referred as performance Analysis)
Posteriori testing (referred as performance measurement)
Space complexity

Space Space Total space

Required Required required by
for Fixed for Variable an
Part Part Algorithm

simple variables and dynamic memory

constants used, allocation, recursion
program size, etc. stack space, etc.
Space complexity
Algorithm:
SUM(A, B) Space complexity = 1(2 byte) + 3(size of data type of A,B,C)
Step 1 - START
Step 2 - C ← A + B + 10
Step 3 - Stop

Total space
Fixed Part Variable Part required
(depends on
10 A,B,C data type of
A,B,C)
Asymptotic notations

Asymptotic Notations
It is a mathematical representation of algorithm’s complexity.
In asymptotic notation, when we want to represent the complexity of an algorithm, we use only the
most significant terms in the complexity of that algorithm and ignore least significant terms in the
complexity of that algorithm (Here complexity can be Space Complexity or Time Complexity).
Following are the commonly used asymptotic notations to calculate the running time complexity of
an algorithm.
1. Ο (Big Oh) Notation  upper bound
2. Ω (Big Omega) Notation  lower bound of a function
3. θ (Theta) Notation  average bound of a function
Graphical Representation

f(n) = O(g(n)) f(n) = Ω(g(n)) f(n) = Θ(g(n))

1 < log n < √n< nn < n log n < 𝑛2 < 𝑛3 < ⋯ … . < 2𝑛 < 3𝑛 <…..< 𝑛𝑛

Lower Bound Average Bound Upper Bound

Time complexity of any algorithm can be represented with any one of the above.
If it is not among these , it will be multiple of these.
And if not then we require lower bound or upper bound
Big - Oh Notation (O)
Big - Oh notation is used to define the upper bound of an algorithm in terms of Time
Complexity.
That means Big - Oh notation always indicates the maximum time required by an
algorithm for all input values.
That means Big - Oh notation describes the worst case of an algorithm time complexity.
The function f(n) = O(g(n)) if & only if there exist positive constant C , n0
Such that f(n) <= C * g(n) for all n >= n0 where C > 0 and n0 >= 1.
Ex:f(n)=2n+3
Consider g(n)=n Consider g(n)=𝑛2
i.e. 2n+3<= 5 n n>=1 2n+3<= 5 𝑛2 n>=1
f(n)=O(n) f(n)=O(𝑛2 )
Example
Consider the following f(n) and g(n)...
f(n) = 3n + 2
g(n) = n

If we want to represent f(n) =O(g(n)) then it must satisfy

f(n) <= C g(n) for all values of C > 0 and n0>= 1

f(n) <= C g(n)

⇒3n + 2 <= C n
Above condition is always TRUE for all values of C = 4 and n >= 2.

By using Big - Oh notation we can represent the time complexity as follows...

3n + 2 = O(n)
Big – Omega Notation (Ω)
Big - Omega notation is used to define the lower bound of an algorithm in terms of
Time Complexity.
That means Big-Omega notation always indicates the minimum time required by an
algorithm for all input values.
That means Big-Omega notation describes the best case of an algorithm time
complexity.
The function f(n) = Ω(g(n)) if & only if there exist positive constant C , n0
Such that f(n) >= C * g(n) for all n >= n0 where C > 0 and n0 >= 1.
Ex:f(n)=2n+3
Consider g(n)=n consider g(n)=logn
i.e. 2n+3>= 1*n n>=1 2n+3>= 1*log n n>=1

f(n)= Ω(n) f(n)= Ω(log 𝑛)

Example
Consider the following f(n) and g(n)...
f(n) = 3n + 2
g(n) = n

If we want to represent f(n) = Ω(g(n)) then it must satisfy f(n) >= C g(n) for all values of C > 0 and n0>= 1

f(n) >= C g(n)

⇒3n + 2 >= C n
Above condition is always TRUE for all values of C = 1 and n >= 1.

By using Big - Omega notation we can represent the time complexity as follows...
3n + 2 = Ω(n)
Big - Theta Notation (Θ)
Big - Theta notation is used to define the average bound of an algorithm in terms of Time
Complexity.
That means Big - Theta notation always indicates the average time required by an algorithm for
all input values.
That means Big - Theta notation describes the average case of an algorithm time complexity.

The function f(n) = Θ(g(n)) if & only if there exist positive constant C1,C2 , n0
Such that C1 * g(n) <= f(n) <= C2 * g(n) for all n >= n0

Ex:f(n)=2n+3
Consider g(n)=n
i.e. 1*n <= 2n+3 <= 5*n n>=1 f(n)= Θ(n)
Example
Consider the following f(n) and g(n)...
f(n) = 3n + 2
g(n) = n

If we want to represent f(n) = Θ(g(n)) then it must satisfy C1 g(n) <= f(n) <= C2 g(n) for all values
of C1 > 0, C2 > 0 and n0>= 1

C1 g(n) <= f(n) <= C2 g(n)

⇒C1 n <= 3n + 2 <= C2 n
Above condition is always TRUE for all values of C1 = 1, C2 = 4 and n >= 2.

By using Big - Theta notation we can represent the time compexity as follows...
3n + 2 = Θ(n)
Finding complexity using step count method, Analysis of
programming constructs-Linear, Quadratic, Cubic, Logarithmic.
Syntax of for loop in CPP
for (statement 1; statement 2; statement 3)
Example
{ for (int i = 0; i < 5; i++)
{
// code block to be executed cout << i << "\n";
}
}

statement 1 is executed (one time) before the execution of the code block.
statement 2 defines the condition for executing the code block.
statement 3 is executed (every time) after the code block has been executed.
Step Count Method
The step-count method count for all the time spent in all parts of the
program
A program step is loosely defined to be a syntactically or
semantically meaningful segment of a program for which the
execution time is independent of the instance characteristics.
Step count Method
First define steps:
1.int mean(int a[], size_t n)
2.{
3.int sum = 0; // 1 step
4.for (int i = 0; i < n; i++) // 1 step
5.sum += a[i]; // 1 step
6.return sum; // 1 step
7.}
 Next determine the frequency of the
steps based on N:
1.int mean(int a[], size_t n)
2.{ Space Complexity
3.int sum = 0; // 1 step * 1 Variables Used are
a=n
4.for (int i = 0; i < n; i++) // 1 step * (N+1)
Sum=1
5.sum += a[i]; // 1 step * N n=1
6.return sum; // 1 step * 1 i=1
-----------------------
7.}
=n+3
Space Complexity
=O(n)
Add up the steps: 1 + (N+1) + N + 1
Reduce: 2N + 3
Throw away factors that don't grow with N and you're done: O(N)--(order of polynomial is 1 ie of degree 1)
Example
1. seqSearch(arr, n, key) Step Frequency
2. i := 0 1 1
3. while i < n, do 1 n+1
4. if arr[i] = key, then 1 n Space Complexity
5. break 1 0/1 Variables Used are
6. end if
7. done Arr=n
8. return i 1 1 Key=1
n=1
Time complexity =1*1+1*(n+1)+1*n+1*1+1*1 i=1
=n+n+1+3 -----------------------
=n+3
=2n+4 Space Complexity
=O(n) =O(n)
Algorithm for adding two n*n Matrix

Algorithm add(A,B,n) Frequency Steps

{ Space Complexity
Variables Used are
For (i=0; i<n ;i++) n+1 1 A=n*n=𝑛2
{ B=n*n=𝑛2
For (j=0; j<n ;j++) n*n+1 1 C=n*n=𝑛2
{ n=1
i=1
C[ i, j ] = A [ i, j ] + B [ i, j ] ; n*n 1 j=1
} -----------------------
} =3𝑛2 + 3
Space Complexity
} =O(𝑛2 )

Time Complexity= n+1+(𝑛2 +1)+(𝑛2 )=2𝑛2 +n+2 =O(𝑛2 )

 Algorithm for Multiplication of two n*n Matrix
Algorithm multiply(A,B,n) Frequency Steps
{
For (i=0; i<n ;i++) n+1 1
{ Space Complexity
For (j=0; j<n ;j++) n*(n+1) 1 Variables Used are
{ A=n*n=𝑛2
C[ i, j ] =0; n*n 1 B=n*n=𝑛2
for(k=0 ; k<n ; k++) n*n*(n+1) 1 C=n*n=𝑛2
{
n=1
i=1
C[ i, j ] = C[ i, j ] +A [ i, k ] * B [ k, j ] ; n*n*n 1
j=1
}
K=1
}
-----------------------
}
=3𝑛2 + 4
} Space Complexity
=O(𝑛2 )
Time Complexity= n+1+(𝑛2 +n)+(𝑛2 ) + (𝑛3 + 𝑛2 ) + (𝑛3 )=2𝑛3 +3𝑛2 +2n+1 =O(𝑛3 )
Analysis of programming constructs-
Linear, Quadratic, Cubic, Logarithmic
1 < log n < √n< nn < n log n < 𝑛2 < 𝑛3 < ⋯ … . < 2𝑛 < 3𝑛 <…..< 𝑛𝑛
Linear
In this case of algorithm we comes to know that, as the input size gets increment or decrement
linearly the time is also goes on increasing or decreasing
So the controlling factor of for loop is
i++ Add_Array (A, n)
{
i— Sum=0;
for(i=0; i<n; i++)
sum = sum + A[i];
F(n) = O(n) return sum;
}

for(i=n; i>0; i--)

sum = sum + A[i];
Quadratic
Algorithms where one loop is inside the other loop are of two types
Linear Quadratic
for(i=o; i<n ; i++)
The controlling factor of both for loop is i++ or i—
{
Dependent Quadratic
for(j=o; j<n; j++)
Inner for loop is depend on outer for loop
{
for(i=o; i<n ; i++) C[i,j]=A[i,j] + B[i,j]
{ }
for(j=o; j<i; j++) }
{
Print(“Hello”);

}
}
F(n) = O(𝑛2 )
Cubic
In this case of algorithms there are three nested loops
Multiply(A,B,n)
{
for(i=o; i<n ; i++)
{
for(j=o; j<n; j++) F(n) = O(𝑛3 )
{
C[i,j]=0;
for(k=o; k<n; k++)
{
C[i,j]= C[i,j]+A[i,k]*B[k,j];
}
}
}
}
Logarithmic
In this case of algorithm controlling factor is either
For(i=1; i<n; i=i*2)
multiplication (i*2) or division (i/2). {
Print(“hello”);
F(n) = O(log 𝑛) }

for(i=o; i<n ; i++)

Linear Logarithmic {
In this case outer for loop is linear and inner for loop is for(j=1; j<n; j=j*2)
logarithmic means controlling factor of outer for loop is {
i++and inner for loop is i=i*2 or i=i/2
Print (“hello”)
}
F(n) = O(n log 𝑛) }
Lecture 6
Algorithmic Strategies- Introduction to algorithm design strategies- Divide and Conquer, and
Greedy strategy.
ALGORITHMIC STRATEGIES
Algorithm design strategies are the general approaches used to develop efficient solution to
problem.
Algorithm Strategies are :
1. Divide and conquer
2. Greedy Strategy
3. Recursive algorithm
4. Backtracking algorithms
5. Heuristic algorithms
6. Dynamic Programming algorithm
DIVIDE AND CONQUER
It is a top-down approach. The algorithms which follow the divide & conquer techniques involve
three steps:
Divide -Divide the original problem into a set of sub problems.
Conquer -Solve every sub problem individually, recursively.
Combine -Combine the solution of the sub problems (top level) into a solution of the whole
original problem.
Example:
Example
DIVIDE AND CONQUER
In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and then
each problem is solved independently. When we keep on dividing the sub problems into even smaller
sub-problems, we may eventually reach a stage where no more division is possible. Those "atomic"
smallest possible sub-problem (fractions) are solved. The solution of all sub-problems is finally
merged in order to obtain the solution of an original problem.
Following algorithms are based on divide and conquer strategies :
1.Merge sort
2.Binary search
3.Quick sort
4.Closest pair
5.Tower of Hanoi
Merge sort
Example
 GREEDY STRATEGIES
 An algorithm is designed to achieve optimum solution for a given problem.
 In greedy algorithm approach, decisions are made from the given solution domain.
 As being greedy, the closest solution that seems to provide an optimum solution is chosen.
Example of greedy strategy :
1. Travelling Salesman Problem
2. Prim's Minimal Spanning Tree Algorithm
3. Kruskal's Minimal Spanning Tree Algorithm
4. Dijkstra's Minimal Spanning Tree Algorithm
5. Knapsack Problem
6. Job Scheduling Problem
Example
Krushkal’s Minimum Spanning tree
Example