CHAPTER 1

BASIC CONCEPT

All the programs in this file are selected from

Ellis Horowitz, Sartaj Sahni, and Susan Anderson-Freed
“Fundamentals of Data Structures in C”,
Computer Science Press, 1992.

CHAPTER 1 1
 Algorithm

Definition
An algorithm is a finite set of instructions that
accomplishes a particular task.

CHAPTER 1 2
Characteristics of Algorithms
Criteria
– input
– output
– definiteness: Each instruction should be clear and unambiguous
– Uniqueness- well defined and ordered procedure that consists of a set of instructions in specific order
– finiteness: terminate after a finite number of steps
– effectiveness: implies that it should be traceable manually
– Correctness- Algorithm should be correct

CHAPTER 1 3
Characteristics of Algorithms
Generality- an algorithm should be generic, independent of any
programming language or OS and able to handle all ranges of inputs
( not written for specific instance )

-an algorithm that is definite and effective is called a computational

procedure

CHAPTER 1 4
Fundamental Stages of Problem Solving

The problem solving process starts with the understanding of a given

problem and ends with the programing code of the given problem.

[Link] the problem

[Link] an algorithm

[Link] an algorithm

[Link] and verifying an algorithm

[Link] an algorithm

[Link] an algorithm

[Link] Empirical Analysis

CHAPTER 1 5
 Understanding the problem – Is the problem solvable? A
proper problem statement is required
guideline for problem-solving is necessary to understand the problem

 Planning an algorithm: Model of Computation and Data

organization
The nature of data and their organization have an impact on the efficiency of
algorithms

 Designing an algorithm: how to design an algorithm,

how to express an algorithm- algorithmic specification

After the algorithm is designed it should be communicated to the programmer so
that the algorithm can be coded as a program
Communication – algorithm specifications – pseudo code

CHAPTER 1 6
 Validating and Verifying :
Algorithm validation is a process of checking whether the algorithm gives correct
outputs for valid inputs or not.
Algorithm verification is a process of providing mathematical proof that the given
algorithm works correctly for all instances of data
Creating set of assertions that are expressed using mathematical logic.
Assertions are the statements that indicate the conditions of the algorithm variables at
various point of the program
Variable status – before and after execution
A proof of an algorithm is said to exist if preconditions can sown to imply postconditions
logically

 Analyzing an algorithm :
To decide the efficiency of the given algorithms
To compare algorithms for deciding the effective solutions for a given problem

CHAPTER 1 7
Classification of Algorithms
 Based on Implementation
– Recursive and Nonrecursive
– Exact and Approximation algorithms
– Deterministic and nondeterministic

 Based on design
 Based on Area of Specialization
String algorithms
graph algorithms

 Based on tractability
 If a problem is solvable – tractable
 Polynomial time or exponential time

CHAPTER 1 8
CHAPTER 1 9
Design Approaches

 Top-Down Design

 Bottom-up Design

CHAPTER 1 10
Algorithm Specifications

CHAPTER 1 11
CHAPTER 1 12
CHAPTER 1 13
CHAPTER 1 14
CHAPTER 1 15
Analysis of Iterative Algorithms

 Measure the input size, that is the length of input data

 Measure the running time using either step count or
operation count
 In case of operation count, find the worst, best and average
case efficiencies of algorithm
 Identify the rate of growth. This step determines the
performance of algorithm when the input size is scaled
higher ( Increasing the input size and assessing the
behavior of algorithm)

CHAPTER 1 16
Measuring Running Time

Algorithm Analysis
 Step Count
 Operation Count
 Asymptotic Analysis

CHAPTER 1 17
Step Count
 The idea is to count the number of
instructions or steps that are used by a given
algorithm to perform a given task
 The first step is to find steps per execution

and frequency of statements

 The number of steps per execution is zero

for non-executable statements and 1 for

executable statements
 Frequency is the number of times the

statement is executed
CHAPTER 1 18
STEP COUNT

 Declarative statement – 0
 Assignment Statement-1
 Comments/ brackets -0
 Expression -1
 Function invocation, return statement/
break/ continue - 1

CHAPTER 1 19
CHAPTER 1 20
Operation Count
The idea is to count the number of operations.
Operations –

Elementary

(Assignment, Comparison, Arithmetic, Logical)

Non- elementary

CHAPTER 1 21
CHAPTER 1 22
CHAPTER 1 23
CHAPTER 1 24
Worst case Complexity and Upper Bound: The worst case complexity is a
complex function of the worst case input for which the algorithm takes
maximum time and causes a computer to run slower

Best case complexity and lower bound: The best case analysis is given as
the minimum computation time of the algorithmm for all the instances of
the domain

Average case Complexity: The average case analysis assumes tat the
input is random and provides a prediction about the running time of an
algorithmm for a random input.

CHAPTER 1 25
 Space Complexity
S(P)=C+SP(I)
 Fixed Space Requirements (C)
Independent of the characteristics of the inputs and outputs
– instruction space
– space for simple variables, fixed-size structured variable, constants
 Variable Space Requirements (SP(I))
depend on the instance characteristic I
– number, size, values of inputs and outputs associated with I
– recursive stack space, formal parameters, local variables, return address

CHAPTER 1 26
CHAPTER 1 27
Analysis of Recursive Algorithms

CHAPTER 1 28
CHAPTER 1 29
 Time Complexity
T(P)=C+TP(I)
 Compile time (C)
independent of instance characteristics
 run (execution) time TP
 Definition
A program step is a syntactically or semantically meaningful program segment whose execution time is
independent of the instance characteristics.
 Example
– abc = a + b + b * c + (a + b - c) / (a + b) + 4.0
– abc = a + b + c

Regard as the same unit

machine independent
CHAPTER 1 30
 Methods to compute the step count

 Introduce variable count into programs

 Tabular method
– Determine the total number of steps contributed by each statement
step per execution  frequency
– add up the contribution of all statements

CHAPTER 1 31
 Iterative summing of a list of numbers

float sum(float list[ ], int n)

{
float tempsum = 0; count++; /* for assignment */
int i;
for (i = 0; i < n; i++) {
count++; /*for the for loop */
tempsum += list[i]; count++; /* for assignment */
}
count++; /* last execution of for */
return tempsum;
count++; /* for return */
}
2n + 3 steps
CHAPTER 1 32
float sum(float list[ ], int n)
{
float tempsum = 0;
int i;
for (i = 0; i < n; i++)
count += 2;
2n + 3 steps
count += 3;
return 0;
}

CHAPTER 1 33
 Recursive summing of a list of numbers

float rsum(float list[ ], int n)

{
count++; /*for if conditional */
if (n) {
count++; /* for return and rsum invocation */
return rsum(list, n-1) + list[n-1];
}
count++;
return list[0];
}

2n+2

CHAPTER 1 34
 Matrix addition

void add( int a[ ] [MAX_SIZE], int b[ ] [MAX_SIZE],

int c [ ] [MAX_SIZE], int rows, int cols)
{
int i, j;
for (i = 0; i < rows; i++)
for (j= 0; j < cols; j++)
c[i][j] = a[i][j] +b[i][j];
}

CHAPTER 1 35
void add(int a[ ][MAX_SIZE], int b[ ][MAX_SIZE],
int c[ ][MAX_SIZE], int row, int cols )
{
int i, j;
for (i = 0; i < rows; i++){ 2rows * cols + 2 rows + 1
count++; /* for i for loop */
for (j = 0; j < cols; j++) {
count++; /* for j for loop */
c[i][j] = a[i][j] + b[i][j];
count++; /* for assignment statement */
}
count++; /* last time of j for loop */
}
count++; /* last time of i for loop */
}
CHAPTER 1 36
 Tabular Method

Iterative function to sum a list of numbers

steps/execution
Statement s/e Frequency Total steps
float sum(float list[ ], int n) 0 0 0
{ 0 0 0
float tempsum = 0; 1 1 1
int i; 0 0 0
for(i=0; i <n; i++) 1 n+1 n+1
tempsum += list[i]; 1 n n
return tempsum; 1 1 1
} 0 0 0
Total 2n+3

CHAPTER 1 37
 Recursive Function to sum of a list of numbers

Statement s/e Frequency Total steps

float rsum(float list[ ], int n) 0 0 0
{ 0 0 0
if (n) 1 n+1 n+1
return rsum(list, n-1)+list[n-1]; 1 n n
return list[0]; 1 1 1
} 0 0 0
Total 2n+2

CHAPTER 1 38
 Matrix Addition

Statement s/e Frequency Total steps

Void add (int a[ ][MAX_SIZE]‧ ‧ ‧ ) 0 0 0

{ 0 0 0
int i, j; 0 0 0
for (i = 0; i < row; i++) 1 rows+1 rows+1
for (j=0; j< cols; j++) 1 rows‧ (cols+1) rows‧ cols+rows
c[i][j] = a[i][j] + b[i][j]; 1 rows‧ cols rows‧ cols
} 0 0 0

Total 2rows‧ cols+2rows+1

CHAPTER 1 39
CHAPTER 1 40
 Asymptotic Notation (O)
 Definition
f(n) = O(g(n)) iff there exist positive constants c and n0 such that f(n)  cg(n) for all n, n  n0.
 Examples
– 3n+2=O(n) /* 3n+24n for n2 */
– 3n+3=O(n) /* 3n+34n for n3 */
– 100n+6=O(n) /* 100n+6101n for n10 */
– 10n2+4n+2=O(n2) /* 10n2+4n+211n2 for n5 */
– 6*2n+n2=O(2n) /* 6*2n+n2 7*2n for n4 */

CHAPTER 1 41
CHAPTER 1 42
– 3n+2=Omega(n) /* 3n+2>=3n for n1 */
– 100n+6=Omega(n) /* 100n+6>=100n for n1
*/
– 10n2+4n+2=Omega(n2) /* 10n2+4n+2>=n2 for n1
*/
– 6*2n+n2=Omega(2n) /* 6*2n+n2 >=2n for n1 */

CHAPTER 1 43
CHAPTER 1 44
CHAPTER 1 45
 Example
 Complexity of c1n2+c2n and c3n
– for sufficiently large of value, c3n is faster than
c1n2+c2n
– for small values of n, either could be faster
• c1=1, c2=2, c3=100 --> c1n2+c2n  c3n for n  98
• c1=1, c2=2, c3=1000 --> c1n2+c2n  c3n for n  998
– break even point
• no matter what the values of c1, c2, and c3, the n beyond
which c3n is always faster than c1n2+c2n
CHAPTER 1 46
 O(1): constant
 O(n): linear
 O(n2): quadratic
 O(n3): cubic
 O(2n): exponential
 O(logn)
 O(nlogn)

CHAPTER 1 47
 nlogn

logn

CHAPTER 1 48
*Figure 1.9:Times on a 1 billion instruction per second computer(p.40)

CHAPTER 1 49