BENSA DAYE CAMPUS

DEPARTMENT OF COMPUTER SCIENCE

COURSE TITLE: DATA STRICTURE AND
ALGORITHM
1
Course Code:- CoSc2041
Prerequisite(s): Fundamental of Programming in C++
 CHAPTER ONE

INTRODUCTION TO DATA
STRUCTURES AND
ALGORITHMS
INTRODUCTION
A program is written in order to solve a problem.
A solution to a problem actually consists of two
things:
 A way to organize the data
 Sequence of steps to solve the problem

The way data are organized in a computer’s

memory is said to be Data Structure and
The sequence of computational steps to solve a
problem is said to be an algorithm.
Therefore, a program is nothing but data
structures plus algorithms. 3
What are Data Structures?
Data structure is a storage that is used to store and organize data.
It is a way of arranging data on a computer so that it can be
accessed and updated efficiently.
Depending on your requirement and project, it is important to
choose the right data structure for your project.
For example, if you want to store data sequentially in the memory,
then you can go for the Array data structure.

Array data Structure Representation

Note: Data structure and data types are slightly different.
Data structure is the collection of data types arranged in a specific
order.
CON’T…
 An entity with the properties just described is called
an abstract data type (ADT).
 Abstract data type (ADT) is a set of objects together
with a set of operations.
 Objects such as lists, sets, and graphs, along with
their operations, can be viewed as ADTs
 The ADT specifies:
1. What can be stored in the Abstract Data Type?
2. What operations can be done on/by the Abstract Data Type?
 Abstraction is a process of classifying characteristics
as relevant and irrelevant for the particular purpose at
hand and ignoring the irrelevant ones.

5
DATA TYPES VS DATA STRUCTURE
 What is a data type?
 The two things we must know about the data
types are:
 It defines a certain domain of values.
 It defines operations allowed on those values.
 For example:
 If the data is of int (integer) type, Then it takes
only integer values
 Data type is one of the forms of a variable to which
the value can be assigned of a given type only. This
value can be used throughout the program.
 Data structure is a collection of data of different
data types. This collection of data can be represented
using an object and can be used throughout the
6
program.
CON’T…
 Based on the type of data store, there are two different
kinds of data structures.
 Primitive Data Structures are basic data structures
provided by programming languages to represent single
values, such as integers, floating-point numbers,
characters, and Booleans.
 Abstract Data Structures are higher-level data
structures that are built using primitive data types and
provide more complex and specialized operations.
 Some common examples of abstract data structures
include arrays, linked lists, stacks, queues, trees, and
graphs.
 Basically, based on organize and process a very large amount of data, data
structures are divided into two categories:
 Linear data structure 7
 Non-linear data structure
CON’T…

8
Linear data structures
In linear data structures, the elements are arranged in sequence one
after the other.
Since elements are arranged in particular order, they are easy to
implement.
However, when the complexity of the program increases, the linear
data structures might not be the best choice because of operational
complexities.
Popular linear data structures are:
i. Array Data Structure
In an array, elements in memory are arranged in continuous memory.
All the elements of an array are of the same type.
And, the type of elements that can be stored in the form of arrays is
determined by the programming language.

An array with each element represented by an index

ii. Stack Data Structure
In stack data structure, elements are works in the LIFO principle. That is, the last
element stored in a stack will be removed first.
It works just like a pile of plates where the last plate kept on the pile will be
removed first.

In a stack, operations can be performing only from one end (top here).
iii. Queue Data Structure
Unlike stack, the queue data structure works in the FIFO principle where first
element stored in the queue will be removed first.
It works just like a queue of people in the ticket counter where first person on the
queue will get the ticket first.
In a queue, addition and removal are performed from separate ends.

iv. Linked List Data Structure

In linked list data structure, data elements are connected through a series of nodes.
And, each node contains the data items and address to the next node.

Linked list
Nonlinear data structures
Unlike linear data structures, elements in non-linear data structures are not in any sequence.

Instead they are arranged in a hierarchical manner where one element will be connected to one or more elements.

Non-linear data structures are further divided into graph and tree based data structures.

i. Graph Data Structure

In graph data structure, each node is called vertex and each vertex is connected to other vertices through edges.

Graph data structure example

Popular Graph Based Data Structures:

 Spanning Tree and Minimum Spanning Tree

 Strongly Connected Components

 Adjacency Matrix

 Adjacency List

ii. Trees Data Structure

Similar to a graph, a tree is also a collection of vertices and edges. However, in tree data structure, there can only be one edge
between two vertices.

Tree data Tree structure example

Popular Tree based Data Structure
 Binary Tree
 Binary Search Tree
 AVL Tree
 B-Tree
 B+ Tree
 Red-Black Tree
Linear Vs Non-linear Data Structures
Now that we know about linear and non-linear data structures, let's see the major
differences between them.
Linear Data Structures Non Linear Data Structures

The data items are arranged in sequential The data items are arranged in non-
order, one after the other. sequential order (hierarchical manner).

All the items are present on the single layer. The data items are present at different layers.

It can be traversed on a single run. That is, if It requires multiple runs. That is, if we start
we start from the first element, we can from the first element it might not be
traverse all the elements sequentially in a possible to traverse all the elements in a
single pass. single pass.

Different structures utilize memory in

The memory utilization is not efficient. different efficient ways depending on the
need.

The time complexity increase with the data

Time complexity remains the same.
size.

Example: Arrays, Stack, Queue Example: Tree, Graph, Map

CON’T…

13
ALGORITHM
 An algorithm is a well-defined computational procedure that takes some
value or a set of values as input and produces some value or a set of values as
output.
 Data structures model the static part of the world. They are unchanging while
the world is changing.
 Algorithms are the dynamic part of a program’s world model.
 An algorithm transforms data structures from one state to
another state in two ways:
 An algorithm may change the value held by a data
structure
 An algorithm may change the data structure itself
 The quality of a data structure is related to its ability to
successfully model the characteristics of the world.
 Similarly, the quality of an algorithm is related to its ability to
successfully simulate the changes in the world.
14
 Generally speaking, correct data structures lead to simple and
efficient algorithms and correct algorithms lead to accurate
and efficient data structures.
PROPERTIES OF AN ALGORITHM
 Finiteness:
 Definiteness:

 Sequence:

 Feasibility

 Correctness:

 Language Independence

 Completeness

 Effectiveness:

 Efficiency:

 Generality:
15
 Input/Output:
 GOOD ALGORITHMS?
 Run in less time
 Consume less memory

But computational resources (time complexity) is usually more

important
Measuring Algorithms Efficiency
 The efficiency of an algorithm is a measure of the amount of

resources consumed in solving a problem of size n.

 The resource we are most interested in is time
 We can use the same techniques to analyze the consumption of other
resources, such as memory space.
 It would seem that the most obvious way to measure the
efficiency of an algorithm is to run it and measure how much
processor time is needed
16
 FACTORS
 Hardware
 Operating System
 Compiler
 Size of input
 Nature of Input
 Algorithm

Which should be improved?

RUNNING TIME OF AN ALGORITHM
 Depends upon
 Input Size
 Nature of Input
 Generally time grows with size of input, so running time
of an algorithm is usually measured as function of input
size.
 Running time is measured in terms of number of
steps/primitive operations performed 17

 Independent from machine, OS

Measuring complexity
 Algorithm analysis refers to the process of determining how
much computing time and storage that algorithms will
require.
 In other words, it’s a process of predicting the resource
requirement of algorithms in a given environment.
 In order to solve a problem, there are many possible algorithms.
One has to be able to choose the best algorithm for the problem
at hand using some scientific method.
 To classify some data structures and algorithms as good, we
need precise ways of analyzing them in terms of resource
requirement.
 The main resources are:
 Running Time
 Memory Usage
 Running time is usually treated as the most important since
computational time is the most precious resource in most
problem domains. 18
 There are two approaches to measure the efficiency of algorithms:
 Empirical
1. Empirical Algorithm Analysis
 It works based on the total running time of the program.
It uses actual system clock time.
Example:
t1(Initial time before the program starts)
for(int i=0; i<=10; i++)
cout<<i;
t2 (final time after the execution of the program is finished)
Running time taken by the above algorithm (TotalTime) =
t2-t1;
 It is difficult to determine efficiency of algorithms using this
approach, because clock-time can vary based on many
factors. For example:
a) Processor speed of the computer
b) Current processor load
c) Specific data for a particular run of the program
d) Operating System
 Multitasking Vs Single tasking 19

 Internal structure
2. Theoretical Algorithm Analysis
 Determining the quantity of resources required
using mathematical concept.
 Analyze an algorithm according to the number

of basic operations (time units) required,

rather than according to an absolute amount of
time involved.
 We use theoretical approach to determine the

efficiency of algorithm because:

 The number of operation will not vary under
different conditions.
 It helps us to have a meaningful measure that
permits comparison of algorithms independent
of operating platform.
 It helps to determine the complexity of20
algorithm
Complexity Analysis
 Complexity Analysis is the systematic study of the cost of
computation, measured either in time units or in
operations performed, or in the amount of storage
space required.
 Two important ways to characterize the effectiveness of an
algorithm are its Space Complexity and Time Complexity.
 Time Complexity: Determine the approximate number of
operations required to solve a problem of size n. The limiting
behavior of time complexity as size increases is called
the Asymptotic Time Complexity.
 Space Complexity: Determine the approximate
memory required to solve a problem of size n.
The limiting behavior of space complexity as size
increases is called the Asymptotic Space
Complexity.
o Asymptotic Complexity of an algorithm
21
determines the size of problems that can be
solved by the algorithm.
o There is no generally accepted set of rules for
algorithm analysis. However, an exact count of
operations is commonly used.
Analysis Rules:
1. We assume an arbitrary time unit.
2. Execution of one of the following operations takes
time 1:
 Assignment Operation
 Single Input/Output Operation
 Single Boolean Operations
 Single Arithmetic Operations
 Function Return
3. Running time of a selection statement (if, switch) is
the time for the condition evaluation + the
maximum of the running times for the individual 22
clauses in the selection.
Example:
int x;
int sum=0;
if(a>b)
{
sum= a+b;
cout<<sum;
}
else
{
cout<<b;
}
T(n) = 1 +1+max(3,1) = 5
23
4. Loops: Running time for a loop is equal to the
running time for the statements inside the loop *
number of iterations.
o The total running time of a statement inside a
group of nested loops is the running time of the
statements multiplied by the product of the sizes
of all the loops.
 For nested loops, analyze inside out.
 Always assume that the loop executes the

maximum number of iterations possible.

5. Running time of a function call is 1 for
setup + the time for any parameter
calculations + the time required for the
execution of the function body.
24
Examples:
1. int count(){
int k=0;
cout<< “Enter an integer”;
cin>>n;
for (i=0;i<n;i++)
k=k+1;
return 0;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int k=0
1 for the output statement.
1 for the input statement.
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n) 25
2. int total(int n)
{
int sum=0;
for (int i=1;i<=n;i++)
sum=sum+1;
return sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int sum=0
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
-------------------------------------------------------------------
26
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
3. void func() Time Units to Compute
{ ---------------------------------------------
int x=0; 1 for the first assignment statement:
int i=0; x=0;
int j=1; 1 for the second assignment statement:
cout<< “Enter an Integer value”; i=0;
cin>>n; 1 for the third assignment statement:
while (i<n){ j=1;
x++;
1 for the output statement.
i++;
1 for the input statement.
}
In the first while loop:
while (j<n)
{
n+1 tests
j++;
n loops of 2 units for the two
} increment (addition) operations
} In the second while loop:
n tests
n-1 increments
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 27

= 5n+5 = O(n)
4. int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment.
1 assignment, n+1 tests, and n increments.
n loops of 4 units for an assignment, an addition, and two
multiplications.
1 for the return statement.
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)

28
Formal Approach to Analysis
 In the above examples we have seen that analyzing
Loop statements is so complex. However, it can be simplified by
using some formal approach in which case we can ignore
initializations, loop control, and updates.
For Loops: Formally
 In general, a for loop translates to a summation. The index and

bounds of the summation are the same as the index and bounds of the
for loop.

 Suppose we count the number of additions that are done. There is 1

addition per iteration of the loop, hence N additions in total.

29
Consecutive Statements: Formally
 Add the running times of the separate blocks of your code.

 Nested Loops: Formally, nested for loops translate into multiple

summations, one for each For loop.

o Consecutive Statements: Formally, add the running times of

the separate blocks of your code.

30
Conditionals: Formally
If (test) s1 else s2: Compute the maximum of the running

time for s1 and s2.

31
 Big-oh notation and others
Definition of Big O notation
In plain words, Big O notation describes the
complexity of your code using algebraic terms.
To understand what Big O notation is, we can

take a look at a typical example, O(n²), which is

usually pronounced “Big O squared”. The letter “n”
here represents the input size, and the function
“g(n) = n²” inside the “O()” gives us an idea of
how complex the algorithm is with respect to the
input size.
A typical algorithm that has the complexity of

O(n²) would be the selection sort algorithm.

Selection sort is a sorting algorithm that iterates
through the list to ensure every element at index i32
is the ith smallest/largest element of the list.
 The CODE below gives a visual example of it.
 The algorithm can be described by the following code. In order

to make sure the ith element is the ith smallest element in the
list, this algorithm first iterates through the list with a for loop.
Then for every element it uses another for loop to find the
smallest element in the remaining part of the list.
SelectionSort(List) {
for(i from 0 to List.Length) {
SmallestElement = List[i]
for(j from i to List.Length) {
if(SmallestElement > List[j]) {
SmallestElement = List[j]
}
}
Swap(List[i], SmallestElement)
33
}
}
 In this scenario, we consider the variable List as the input, thus input size n is
the number of elements inside List. Assume the if statement, and the value
assignment bounded by the if statement, takes constant time. Then we can find the
big O notation for the SelectionSort function by analyzing how many times the
statements are executed.
 First the inner for loop runs the statements inside n times. And then after i is
incremented, the inner for loop runs for n-1 times… …until it runs once, then both
of the for loops reach their terminating conditions.

34
Big O, Little O, Omega & Theta
Big O: “f(n) is O(g(n))” iff for some constants c and N₀, f(N) ≤

cg(N) for all N > N₀

Omega: “f(n) is Ω(g(n))” iff for some constants c and N₀, f(N) ≥
cg(N) for all N > N₀
Theta: “f(n) is Θ(g(n))” iff f(n) is O(g(n)) and f(n) is Ω(g(n))
Little O: “f(n) is o(g(n))” iff f(n) is O(g(n)) and f(n) is not Θ(g(n))
Formal Definition of Big O, Omega, Theta and Little O
In plain words:

Big O (O()) describes the upper bound of the complexity.

Omega (Ω()) describes the lower bound of the complexity.

Theta (Θ()) describes the exact bound of the complexity.

Little O (o()) describes the upper bound excluding the exact
35
bound.
 36

Relationships between Big O, Little O, Omega & Theta Illustrated