ECEG4171 Data Structure & Algorithm

Chapter-1
Introduction to data structure and algorithm
basics

06/14/2024 UoG, IoT, ECE 1

Abstract data type (ADT)
• An ADT is composed of
• A collection of data

• A set of operations on that data

• Specifications of an ADT indicate

• What the ADT operations do, not how to implement them

• Implementation of an ADT
• Includes choosing a particular data structure

An ADT is a formal description, not code; independent of any programming language

06/14/2024 UoG, IoT, ECE 2
Abstract Data Types

Figure 1
A wall of ADT operations isolates a data structure from the program that uses it
06/14/2024 UoG, IoT, ECE 3
The ADT List
• ADT List operations
• Create an empty list
• Determine whether a list is empty
• Determine the number of items in a list
• Add an item at a given position in the list
• Remove the item at a given position in the list
• Remove all the items from the list
• Retrieve (get) item at a given position in the list

06/14/2024 UoG, IoT, ECE 4

Introduction to DATA STRUCTURE
 A data structure is a scheme for organizing data in the memory of a computer.
 Data structure mainly specifies the structured organization of data.
 Data structure and the operations on organized data items can integrally solve
the problem using a computer .
Data structure = Organized data + Operations
 Some of the more commonly used data structures include lists, arrays, stacks,
queues, heaps, trees, and graphs.
 The way in which the data is organized affects the performance of a program
for different tasks.

06/14/2024 UoG, IoT, ECE 5

Classification of data structure
Data structures are broadly divided into two :

1. Primitive data structures :

 Basic data structures and are directly operated upon by the machine
instructions, which is in a primitive level.
 They are integers, floating point numbers, characters, string constants,
pointers etc.
 These primitive data structures are the basis for the discussion of more
sophisticated (non-primitive) data structures.

06/14/2024 UoG, IoT, ECE 6

2. Non-primitive data structures :
 It is a more sophisticated data structure emphasizing on structuring of a group
of homogeneous (same type) or heterogeneous (different type) data items.
 Array, list, files, linked list, trees and graphs fall in this category.

06/14/2024 UoG, IoT, ECE 7

06/14/2024 UoG, IoT, ECE 8
ALGORITHM

 Algorithm is a step-by-step finite sequence of instruction, to solve

a well-defined computational problem.
 An algorithm is a step-by-step procedure for solving a problem in
a finite amount of time.

Input Algorithm Output

06/14/2024 UoG, IoT, ECE 9
An algorithm is “a finite set of precise instructions for performing a

computation or for solving a problem”

• A program is one type of algorithm
• All programs are algorithms

• Not all algorithms are programs!

• Directions to somebody’s house is an algorithm

• A recipe for cooking a cake is an algorithm

• The steps to compute the cosine of 90° is an algorithm

06/14/2024 UoG, IoT, ECE 10

Expressing algorithms
An algorithm may be expressed in a number of ways, including:

1. Natural language: usually verbose and ambiguous.

2. Flow charts: using diagrams.
3. Pseudo-code: Informal language used to present algorithms. There is no
particular agreement on syntax.
4. Programming language: Tend to require expressing low-level details
that are not necessary for a high-level understanding.

06/14/2024 UoG, IoT, ECE 11

 What is a Flowchart? START

Display message “How

many hours did you
work?”

• A flowchart is a diagram that depicts the Read Hours

“flow” of a program.
Display message “How

• The figure shown here is a flowchart for much do you get paid per
hour?”

the pay-calculating program .

Read Pay Rate

Multiply Hours by Pay

Rate. Store result in
Gross Pay.

Display Gross Pay

END
06/14/2024 UoG, IoT, ECE 12
 Terminal

Basic Flowchart Symbols

START

Display message
“How many hours
did you work?”

• Terminals Read Hours

• represented by rounded Display message “How much do

rectangles you get paid per hour?”

• indicate a starting or ending

point
Read Pay Rate

Multiply Hours by
Pay Rate. Store
START result in Gross
Pay.

Display Gross Pay

END Terminal
END

06/14/2024 UoG, IoT, ECE 13

Basic Flowchart Symbols
START

Display
message “How
many hours did
you work?”

• Input / Output Operations Read Hours

• represented by parallelograms Display message

• indicate an input or output “How much do
you get paid per
Input/Output
Operation
operation hour?”

Read Pay Rate

Multiply Hours by
Pay Rate. Store
Display result in Gross
Pay.
message “How
Read Hours
many hours Display Gross Pay

did you work?”

END

06/14/2024 UoG, IoT, ECE 14

Basic Flowchart Symbols
START

Display
message “How
many hours did
you work?”

• Processes Read Hours

• represented by rectangles Display message

• indicates a process such as a “How much do
you get paid per
mathematical computation or variable hour?”

assignment
Read Pay Rate

Multiply Hours
by Pay Rate.
Process Store result in
Multiply Hours Gross Pay.
by Pay Rate.
Store result in Display Gross
Pay
Gross Pay.
END

06/14/2024 UoG, IoT, ECE 15

Basic Flowchart Symbols
START

Display message “How many hours

did you work?”

Worked
decision
• Decision
Hours>=1hrs?
false
True
• represented by Diamond Display message
• A diamond indicates a decision “How much do
you get paid per
hour?”

Read Pay Rate

Multiply Hours
by Pay Rate.
Store result in
Gross Pay.

Display Gross
Pay

END

06/14/2024 UoG, IoT, ECE 16

 Four Flowchart Structures
1. Sequence
2. Decision
3. Repetition
4. Case

06/14/2024 UoG, IoT, ECE 17

1. Sequence Structure
• A series of actions are performed in sequence.
• The pay-calculating example was a sequence flowchart.

06/14/2024 UoG, IoT, ECE 18

2. Decision Structure
• One of two possible actions is taken, depending on a condition.

06/14/2024 UoG, IoT, ECE 19

Decision Structure
• The diamond symbol indicates a yes/no question. If the answer to the
question is yes, the flow follows one path. If the answer is no, the flow
follows another path

NO YES

06/14/2024 UoG, IoT, ECE 20

Decision Structure
• In the flowchart segment below, the question “is x < y?” is
asked. If the answer is no, then process A is performed. If
the answer is yes, then process B is performed.

NO YES
x < y?

Process Process
A B

06/14/2024 UoG, IoT, ECE 21

Decision Structure
• The flowchart segment below shows how a decision structure is expressed
in C++ as an if/else statement.

Flowchart C++ Code

NO YES if (x < y)
x < y? a = x * 2;
else

Calculate a Calculate a a = x + y;
as x plus y. as x times
2.

06/14/2024 UoG, IoT, ECE 22

Decision Structure
• The flowchart segment below shows a decision structure with only one
action to perform. It is expressed as an if statement in C++ code.

Flowchart C++ Code

NO YES if (x < y)
x < y? a = x * 2;

Calculate a
as x times 2.

06/14/2024 UoG, IoT, ECE 23

3. Repetition Structure
• A repetition structure represents part of the program that repeats. This type
of structure is commonly known as a loop.

06/14/2024 UoG, IoT, ECE 24

Repetition Structure
• Notice the use of the diamond symbol. A loop tests a condition, and if the
condition exists, it performs an action. Then it tests the condition again. If
the condition still exists, the action is repeated. This continues until the
condition no longer exists.

06/14/2024 UoG, IoT, ECE 25

Repetition Structure
• In the flowchart segment, the question “is x < y?” is
asked. If the answer is yes, then Process A is performed.
The question “is x < y?” is asked again. Process A is
repeated as long as x is less than y. When x is no longer
less than y, the repetition stops and the structure is
exited.

YES
x < y? Process A

06/14/2024 UoG, IoT, ECE 26

Repetition Structure
• The flowchart segment below shows a repetition structure
expressed in C++ as a while loop.

Flowchart C++ Code

while (x < y)

YES x++;
x < y? Add 1 to x

06/14/2024 UoG, IoT, ECE 27

Controlling a Repetition Structure
• The action performed by a repetition structure must eventually cause the
loop to terminate. Otherwise, an infinite loop is created.
• In this flowchart segment, x is never changed. Once the loop starts, it will
never end.
• QUESTION: How can this
flowchart be modified so
it is no longer an infinite YES
loop? x < y? Display x

06/14/2024 UoG, IoT, ECE 28

Controlling a Repetition Structure
• ANSWER: By adding an action within the repetition that changes the value
of x.

YES
x < y? Display x Add 1 to x

06/14/2024 UoG, IoT, ECE 29

A Pre-Test Repetition Structure
• This type of structure is known as a pre-test repetition structure. The
condition is tested BEFORE any actions are performed.

YES
x < y? Display x Add 1 to x

06/14/2024 UoG, IoT, ECE 30

A Pre-Test Repetition Structure
• In a pre-test repetition structure, if the condition does not exist, the loop will
never begin.

YES
x < y? Display x Add 1 to x

06/14/2024 UoG, IoT, ECE 31

A Post-Test Repetition Structure
• This flowchart segment shows a post-test
repetition structure.
• The condition is tested AFTER the actions
Display x
are performed.
• A post-test repetition structure always
performs its actions at least once. Add 1 to x

YES
x < y?

06/14/2024 UoG, IoT, ECE 32

A Post-Test Repetition Structure
• The flowchart segment below shows a post-test repetition structure
expressed in C++ as a do-while loop.

C++ Code
Display x
do
{
Flowchart cout << x << endl;
Add 1 to x
x++;
} while (x < y);

YES
x < y?

06/14/2024 UoG, IoT, ECE 33

4. Case Structure
• One of several possible actions is taken,
depending on the contents of a variable.

06/14/2024 UoG, IoT, ECE 34

Case Structure
• The structure below indicates actions to perform
depending on the value in years_employed.

CASE
years_employed

1 2 3 Other

bonus = bonus = bonus = bonus =

100 200 400 800

06/14/2024 UoG, IoT, ECE 35

Case Structure

If years_employed = If years_employed =
2, bonus is set to 200 3, bonus is set to 400
If years_employed = If years_employed is
CASE
1, bonus is set to 100 years_employed any other value,
bonus is set to 800

1 2 3 Other

bonus = bonus = bonus = bonus =

100 200 400 800

06/14/2024 UoG, IoT, ECE 36

Combining Structures
• Structures are commonly combined to create more complex algorithms.
• The flowchart segment below combines a decision structure with a
sequence structure.

YES
x < y? Display x Add 1 to x

06/14/2024 UoG, IoT, ECE 37

Combining Structures
• This flowchart segment
shows two decision
structures combined. NO YES
x > min?

Display “x is NO YES
outside the limits.”
x < max?

Display “x is Display “x is
outside the limits.” within limits.”

06/14/2024 UoG, IoT, ECE 38

Pseudocode

• Informal language used to present algorithms.

• Pretty close to English but precise enough for a computing agent to carry out.

• High-level description of an algorithm

• More structured than English prose

• Less detailed than a program

• Preferred notation for describing algorithms

• Hides program design issues

06/14/2024 UoG, IoT, ECE 39

 Example: find max element of an array

Algorithm arrayMax(A, n)
Input array A of n integers
Output maximum element of A
currentMax  A[0]
for i  1 to n  1 do
if A[i]  currentMax then
currentMax  A[i]
return currentMax

06/14/2024 UoG, IoT, ECE 40

Pseudocode Details
• Control flow • Method/Function call
• if … then … [else …] var.method (arg [, arg…])
• while … do … • Return value
• repeat … until … return expression
• for … do …
• Indentation replaces braces • Expressions
¬Assignment
• Method declaration
(like  in C++)
Algorithm method (arg [, arg…])
= Equality testing
Input …
(like  in C++)
Output …
n2 Superscripts and other
mathematical formatting allowed

06/14/2024 UoG, IoT, ECE 41

Properties of algorithms
Algorithms generally share a set of properties:
• Input: what the algorithm takes in as input

• Output: what the algorithm produces as output

• Definiteness: the steps are defined precisely

• Correctness: should produce the correct output

• Finiteness: the steps required should be finite

• Effectiveness: each step must be able to be performed in a finite amount of

time
• Generality: the algorithm should be applicable to all problems of a similar form

06/14/2024 UoG, IoT, ECE 42

Algorithm Examples?
• Problem 1: What is the largest integer
INPUT: All the integers { … -2, -1, 0, 1, 2, … }
OUTPUT: The largest integer
Algorithm:
• Arrange all the integers in a list in decreasing order;
• MAX = first number in the list;
• Print out MAX;
• WHY is the above NOT an Algorithm?
• (Hint: How many integers are there?)

• Problem 2: Who is the tallest women in the world?

• Algorithm:
06/14/2024 UoG, IoT, ECE 43
Problems vs Algorithms vs Programs
For each problem or class of problems, there may be many different
algorithms.
For each algorithm, there may be many different implementations (programs).

06/14/2024 UoG, IoT, ECE 44

Algorithmic Performance
There are two aspects of algorithmic performance:

1. Time
• Instructions take time.

• How fast does the algorithm perform?

• What affects its runtime?

2. Space
• Data structures take space

• What kind of data structures can be used?

• How does choice of data structure affect the runtime?

06/14/2024 UoG, IoT, ECE 45
SPACE COMPLEXITY
Analysis of space complexity of an algorithm or program is the amount of memory
it needs to run to completion.
Some of the reasons for studying space complexity are:
1. If the program is to run on multi user system, it may be required to specify the
amount of memory to be allocated to the program.
2. We may be interested to know in advance that whether sufficient memory is
available to run the program.
3. There may be several possible solutions with different space requirements.
4. Can be used to estimate the size of the largest problem that a program can
solve
06/14/2024 UoG, IoT, ECE 46
TIME COMPLEXITY
 The time complexity of an algorithm or a program is the amount of time it
needs to run to completion.
 The exact time will depend on the implementation of the algorithm, programming
language, optimizing the capabilities of the compiler used, the CPU speed,
other hardware characteristics/specifications and so on.

 To measure the time complexity accurately, we have to count all sorts of

operations performed in an algorithm. If we know the time for each one of the
primitive operations performed in a given computer, we can easily compute the
time taken by an algorithm to complete its execution.

06/14/2024 UoG, IoT, ECE 47

 Our intention is to estimate the execution time of an algorithm irrespective
of the computer machine on which it will be used.

 The more sophisticated method is to identify the key operations and count
such operations performed till the program completes its execution.
 A key operation in our algorithm is an operation that takes maximum time
among all possible operations in the algorithm.
 The time complexity can now be expressed as function of number of key
operations performed.

06/14/2024 UoG, IoT, ECE 48

 Worst-Case/ Best-Case/ Average-Case Analysis
 Worst-Case Analysis –The maximum amount of time that an algorithm require
to solve a problem of size n. This gives an upper bound for the time complexity of
an algorithm.
 Best-Case Analysis –The minimum amount of time that an algorithm require to
solve a problem of size n. The best case behavior of an algorithm is NOT so
useful.
 Average-Case Analysis –The average amount of time that an algorithm require
to solve a problem of size n. Sometimes, it is difficult to find the average-case
behavior of an algorithm.
Worst-case analysis is more common than average-case analysis.
06/14/2024 UoG, IoT, ECE 49
TIME-SPACE TRADE OFF
There may be more than one approach (or algorithm) to solve a problem. The best
algorithm (or program) to solve a given problem is one that requires less space in
memory and takes less time to complete its execution. But in practice, it is not always
possible to achieve both of these objectives. One algorithm may require more space but
less time to complete its execution while the other algorithm requires less space but
takes more time to complete its execution. Thus, we may have to sacrifice one at the
cost of the other. If the space is our constraint, then we have to choose a program that
requires less space at the cost of more execution time. On the other hand, if time is our
constraint such as in real time system, we have to choose a program that takes less time
to complete its execution at the cost of more space.

06/14/2024 UoG, IoT, ECE 50

ANALYSIS OF ALGORITHM
After designing an algorithm we have to make analysis for two reasons,

1) It has to be checked and its correctness needs to be predicted; this is done by

 Analyzing the algorithm.
 Tracing all step-by-step instructions,
 Reading the algorithm for logical correctness, and
 Testing it on some data using mathematical techniques to prove it correct.

2 ) To know the simplicity of the algorithm (performance analysis in terms of time and space)
 Characterizes running time as a function of the input size, n.
 Takes into account all possible inputs
 Allows us to evaluate the speed of an algorithm independent of the hardware/software
environment
06/14/2024 UoG, IoT, ECE 51
 When we analyze algorithms, we should employ mathematical techniques
that analyze algorithms independently of specific implementations,
computers, or data.

To analyze algorithms:
• First, we start to count the number of significant /basic / primitive
operations in a particular solution to assess its efficiency.
• Then, we will express the efficiency of algorithms using growth
functions.

06/14/2024 UoG, IoT, ECE 52

Primitive Operations
• Examples:
• Basic computations performed by an algorithm
• Evaluating an expression
• Identifiable in pseudocode
• Assigning a value to a
• Largely independent from the programming
variable
language
• Indexing into an array
• Exact definition not important
• Calling a method
• Assumed to take a constant amount of time in • Returning from a method
the RAM model

06/14/2024 UoG, IoT, ECE 53

General Rules for Estimation
1.We assume an arbitrary time unit.

2.Execution of one of the following operations takes time 1:

1.Assignment operation

2.Single I/O operations

3.Single Boolean operations, numeric comparisons

4.Single arithmetic operations

5.Function return

6.Array index operations, pointer dereferences

06/14/2024 UoG, IoT, ECE 54
More Rules
3. Running time of a selection statement (if, switch) is the time for the
condition evaluation + the maximum of the running times for the
individual clauses in the selection.

4. Loop execution time is the sum, over the number of times the loop is
executed, of the body time + time for the loop check and update
operations + time for the loop setup.
• Always assume that the loop executes the maximum number of iterations possible

5. Running time of a function call is 1 for setup + the time for any parameter
calculations + the time required for the execution of the function body.
06/14/2024 UoG, IoT, ECE 55
 Counting Primitive Operations
• By inspecting the pseudocode, we can determine the maximum number of
primitive operations executed by an algorithm, as a function of the input size
Time Units to Compute
-------------------------------------------------
Examples:
1 for the assignment statement: int k=0
int count()
{ 1 for the output statement.
int k=0; 1 for the input statement.
cout<< “Enter an integer”;
In the for loop:
cin>>n;
for (i=0;i<n;i++) 1 assignment, n+1 tests and n increments.
k=k+1; n loops of 2 units for an assignment and an addition.
return 0;
1 for the return statement.
}
-------------------------------------------------------------------
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
06/14/2024 UoG, IoT, ECE 56
 Time Units to Compute
-------------------------------------------------
int total(int n)
1 for the assignment statement: int sum=0
{ In the for loop:
int sum=0; 1 assignment, n+1 tests, and n increments.

for (int i=1;i<=n;i++) n loops of 2 units for an assignment, and an addition.

1 for the return statement.
sum=sum+1;
-------------------------------------------------------------------
return sum;
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
}

06/14/2024 UoG, IoT, ECE 57

 Time Units to Compute
-------------------------------------------------
void func()
{ 1 for the first assignment statement: x=0;
int x=0; 1 for the second assignment statement: i=0;
int i=0;
int j=1; 1 for the third assignment statement: j=1;
cout<< “Enter an Integer value”; 1 for the output statement.
cin>>n;
while (i<n) 1 for the input statement.
{ In the first while loop:
x++;
i++; n+1 tests
} n loops of 2 units for the two increment (addition) operations
while (j<n)
{ In the second while loop:
j++; n tests
}
} n-1 increments
-------------------------------------------------------------------
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5 = O(n)
06/14/2024 UoG, IoT, ECE 58
 int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}

Time Units to Compute

-------------------------------------------------
1 for the assignment.
1 assignment, n+1 tests, and n increments.
n loops of 4 units for an assignment, an addition, and two
multiplications.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)

06/14/2024 UoG, IoT, ECE 59

 • Each operation in an algorithm (or a program) has a cost.
 Each operation takes a certain amount of time.

count = count + 1;  take a certain amount of time, but it is constant

A sequence of operations:
count = count + 1; Cost: c1
sum = sum + count; Cost: c2

 Total Cost = c1 + c2

06/14/2024 UoG, IoT, ECE 60

 Example: Simple If-Statement
Cost Times
if (n < 0) c1 1
absval = -n c2 1
else
absval = n; c3 1

Total Cost <= c1 + max(c2,c3)

06/14/2024 UoG, IoT, ECE 61

Example: Simple Loop
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}

Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

 The time required for this algorithm is proportional to n

06/14/2024 UoG, IoT, ECE 62

 Example: Nested Loop
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
06/14/2024  The time required for this algorithm is proportional to n2
UoG, IoT, ECE 63
Growth-Rate Functions
O(1) Time requirement is constant, and it is independent of the problem’s size.
O(log2n) Time requirement for a logarithmic algorithm increases slowly as the
problem size increases.
O(n) Time requirement for a linear algorithm increases directly with the size of
the problem.
O(n*log2n) Time requirement for a n*log2n algorithm increases more rapidly than a
linear algorithm.
O(n2) Time requirement for a quadratic algorithm increases rapidly with the
size of the problem.
O(n3) Time requirement for a cubic algorithm increases more rapidly with the
size of the problem than the time requirement for a quadratic algorithm.
O(2n) As the size of the problem increases, the time requirement for an
exponential algorithm increases too rapidly to be practical.
06/14/2024 UoG, IoT, ECE 64
 Growth-Rate Functions
• If an algorithm takes 1 second to run with the problem size 8, what is the time requirement

(approximately) for that algorithm with the problem size 16?

• If its order is:

O(1)  T(n) = 1 second

O(log2n)  T(n) = (1*log216) / log28 = 4/3 seconds

O(n)  T(n) = (1*16) / 8 = 2 seconds

O(n*log2n)  T(n) = (1*16*log216) / 8*log28 = 8/3 seconds

O(n2)  T(n) = (1*162) / 82 = 4 seconds

O(n3)  T(n) = (1*163) / 83 = 8 seconds

O(2 ) 
n
06/14/2024 T(n) = (1*216) / 28 = 28 seconds = 256 seconds
UoG, IoT, ECE 65
A Comparison of Growth-Rate Functions

06/14/2024 UoG, IoT, ECE 66

A Comparison of Growth-Rate Functions (cont.)

06/14/2024 UoG, IoT, ECE 67

06/14/2024 UoG, IoT, ECE 68
 Properties of Growth-Rate Functions

1. We can ignore low-order terms in an algorithm’s growth-rate function.

• If an algorithm is O(n3+4n2+3n), it is also O(n3).

• We only use the higher-order term as algorithm’s growth-rate function.

2. We can ignore a multiplicative constant in the higher-order term of an algorithm’s

growth-rate function.
• If an algorithm is O(5n3), it is also O(n3).

3. O(f(n)) + O(g(n)) = O(f(n)+g(n))

• We can combine growth-rate functions.

• If an algorithm is O(n3) + O(4n2), it is also O(n3 +4n2)  So, it is O(n3).

• Similar rules hold for multiplication.

06/14/2024 UoG, IoT, ECE 69
Some Mathematical Facts
• Some mathematical equalities are:

n
n * (n  1) n 2

i 1
i  1  2  ...  n 
2

2

n
n * ( n  1) * ( 2 n  1) n 3


i 1
i 2
 1  4  ...  n 2

6

3

n 1

 2 i

i 0
 1  2  8 ...  2 n 1
 2 n
1

06/14/2024 UoG, IoT, ECE 70

Growth-Rate Functions – Example1

Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}

T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

= (c3+c4+c5)*n + (c1+c2+c3)
= a*n + b
 So, the growth-rate function for this algorithm is O(n)
06/14/2024 UoG, IoT, ECE 71
Growth-Rate Functions – Example2
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
= (c5+c6+c7)*n2 + (c3+c4+c5+c8)*n + (c1+c2+c3)
= a*n2 + b*n + c
 So, the growth-rate function for this algorithm is O(n2)

06/14/2024 UoG, IoT, ECE 72

Growth-Rate Functions – Example3

Cost Times
for (i=1; i<=n; i++) c1 n+1
n

 ( j  1)
for (j=1; j<=i; j++) c2 j 1
n j

for (k=1; k<=j; k++) c3

 (k  1)
j 1 k 1
n j

 k
x=x+1; c4 j 1 k 1

n n j n j

 ( j  1)
j 1
 (k  1)
j 1 k 1
 k
j 1 k 1
T(n) = c1*(n+1) + c2*( ) + c3* ( ) + c4*( )

= a*n3 + b*n2 + c*n + d

 So, the growth-rate function for this algorithm is O(n3)
06/14/2024 UoG, IoT, ECE 73
 Asymptotic Notation

The notation we use to describe the asymptotic running time of an algorithm

are defined in terms of functions whose domains are the set of natural
numbers
N  0, 1, 2, ...

Three types
 “Big O” Notation O()
 “Big Omega” Notation ()
 “Big Theta” Notation ()
06/14/2024 UoG, IoT, ECE 74
 Big-O notation
• For a given function g (n), we denote by O( g (n)) the set
of functions
 f (n) : there exist positive constants c and n0 s.t.
O( g (n))   
 0  f ( n )  cg ( n ) for all n  n 0 

• We use O-notation to give an asymptotic upper bound of

a function, to within a constant factor.
• f (n)  O( g (n)) means that there existes some constant c
s.t. f (n) is always  cg (n) for large enough n.

06/14/2024 UoG, IoT, ECE 75

 Big-O notation
Big O: Constant Runtime
 Suppose an algorithm simply tests whether the first element of an array is equal
to the second element.
 If the array has 10 elements, this algorithm requires only one comparison.
 If the array has 1000 elements, the algorithm still requires only one comparison.
 In fact, the algorithm is independent of the number of array elements.
 This algorithm is said to have a constant runtime, which is represented in Big O
notation as O(1).
 An algorithm that is O(1) does not necessarily require only one comparison.
 O(1) just means that the number of comparisons is constant—it does not grow
as the size of the array increases.
 An algorithm that tests whether the first element of an array is equal to any of
the next three elements will always require three comparisons, but in Big O
notation it’s still considered O(1).
 O(1) is often pronounced “on the order of 1” or more simply “order 1.”

06/14/2024 UoG, IoT, ECE 76

 Big-O notation
Big O: Linear Runtime
 An algorithm that tests whether the first element of an array is equal to
any of the other elements of the array requires at most n – 1
comparisons, where n is the number of elements in the array.
 If the array has 10 elements, the algorithm requires up to nine
comparisons.
 If the array has 1000 elements, the algorithm requires up to 999
comparisons.
 As n grows larger, the n part of the expression n – 1 “dominates,” and
subtracting one becomes inconsequential.
 Big O is designed to highlight these dominant terms and ignore terms
that become unimportant as n grows.
 An algorithm that requires a total of n – 1 comparisons is said to be O(n)

and is referred to as having a linear runtime.

 O(n) is often pronounced “on the order of n” or more simply “order n.”
06/14/2024 UoG, IoT, ECE 77
 Big-O notation
Big O: Quadratic Runtime
 Now suppose you have an algorithm that tests whether any element of an
array is duplicated elsewhere in the array.
 The first element must be compared with every other element in the array.
 The second element must be compared with every other element except the
first (it was already compared to the first).
 The third element then must be compared with every other element except
the first two.
 In the end, this algorithm will end up making (n – 1) + (n – 2) + … + 2 + 1 or
n2/2 – n/2 comparisons.
 As n increases, the n2 term dominates and the n term becomes
inconsequential.
 Again, Big O notation highlights the n2 term, leaving n2/2.

06/14/2024 UoG, IoT, ECE 78

 Big-O notation
 Big O is concerned with how an algorithm’s runtime grows in relation to
the number of items processed.
 Suppose an algorithm requires n2 comparisons.
 With four elements, the algorithm will require 16 comparisons; with eight
elements, 64 comparisons.
 With this algorithm, doubling the number of elements quadruples the
number of comparisons.
 Consider a similar algorithm requiring n2/2 comparisons.
 With four elements, the algorithm will require eight comparisons; with
eight elements, 32 comparisons.
 Again, doubling the number of elements quadruples the number of
comparisons.
 Both of these algorithms grow as the square of n, so Big O ignores the
constant, and both algorithms are considered to be O(n2), which is
referred to as quadratic runtime and pronounced “on the order of n-
squared” or more simply “order n-squared.”
06/14/2024 UoG, IoT, ECE 79
 Big-O notation
O(n2) Performance
 When n is small, O(n2) algorithms will not noticeably affect
performance.
 As n grows, you’ll start to notice the performance degradation.
 An O(n2) algorithm running on a million-element array would require a
trillion “operations” (where each could actually require several
machine instructions to execute).
◦ This could require hours to execute.
 A billion-element array would require a quintillion operations, a
number so large that the algorithm could take decades! Unfortunately,
O(n2) algorithms tend to be easy to write.

06/14/2024 UoG, IoT, ECE 80

 Big- Ω Omega notation
• For a given function g (n) , we denote by ( g (n)) the
set of functions

 f (n) : there exist positive constants c and n0 s.t.

( g (n))   
 0  cg (n)  f (n) for all n  n0 
• We use Ω-notation to give an asymptotic lower bound on
a function, to within a constant factor.
• f (n)  ( g (n)) means that there exists some constant c s.t.
f (n)  cg (n)
is always for large enough n.

06/14/2024 UoG, IoT, ECE 81

 Big- Θ Theta notation

• For a given function g (n), we denote by ( g (n)) the

set of functions

 f (n) : there exist positive constants c1 , c2 , and n0 s.t.

( g (n))   
 0  c1 g (n)  f (n)  c2 g (n) for all n  n0 
• A function f (n) belongs to the set ( g (n)) if there exist
positive constants c1 and c2 such that it can be “sand-
wiched” between c1 g (n) and c2 g (n) or sufficienly large n.
• f (n)  ( g (n)) means that there exists some constant c1
and c2 s.t. c1 g (n)  f (n)  c 2 g (n) for large enough n.

06/14/2024 UoG, IoT, ECE 82

 Asymptotic notation

Graphic examples of , O, and  .

06/14/2024 UoG, IoT, ECE 83

1. f(n)=10n+5 and g(n)=n. Show that f(n) is O(g(n)).
To show that f(n) is O(g(n)) we must show that constants c and no such that
f(n) <=c.g(n) for all n>=no
Or 10n+5<=c.n for all n>=no
Try c=15. Then we need to show that 10n+5<=15n
Solving for n we get: 5<=5n or 1<=n.
So f(n) =10n+5 <=15.g(n) for all n>=1.
(c=15,no=1).

06/14/2024 UoG, IoT, ECE 84

 2. f(n) = +4n+1. Show that f(n)=O().
+4n+1<=3+4+ for all n>=1
+4n+1<=8 for all n>=1
So we have shown that f(n)<=8 for all n>=1
Therefore, f (n) is O() (c=8,k=1)

06/14/2024 UoG, IoT, ECE 85

 Example 1.
1 2 2
Show that f ( n )  n  3n   ( n )
2

We must find c1 and c2 such that

1 2
c1n  n  3n  c2 n 2
2
2
Dividing bothsides by n2 yields
1 3
c1    c2
2 n
For n  7 , 1 n 2  3n  (n 2 )
0
2
06/14/2024 UoG, IoT, ECE 86
 Theorem

• For any two functions f (n) and g (n), we have

f (n)  ( g (n))

if and only if

f (n)  O( g (n)) and f (n)  ( g (n)).

06/14/2024 UoG, IoT, ECE 87

 Example 2.

f (n)  3n 2  2n  5  (n 2 )

Because :

3n 2  2n  5  (n 2 )

3n 2  2n  5  O(n 2 )

06/14/2024 UoG, IoT, ECE 88

 Example 3.

3n 2  100n  6  O(n 2 ) since for c  3, 3n 2  3n 2  100n  6

3n 2  100n  6  O(n 3 ) since for c  1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6  O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6  (n 2 ) since for c  2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6  (n 3 ) since for c  3, 3n 2  100n  6  n 3 when n  3
3n 2  100n  6  (n) since for any c, cn  3n 2  100n  6 when n  100
3n 2  100n  6  (n 2 ) since both O and  apply.
3n 2  100n  6  (n 3 ) since only O applies.
3n 2  100n  6  (n) since only  applies.

06/14/2024 UoG, IoT, ECE 89

 THE END

06/14/2024 UoG, IoT, ECE 90