Algorithms, Performance

Analysis, Space Complexity,

Time Complexity
 Algorithms

• Algorithm is a step-by-step finite sequence of instruction, to

solve a well-defined computational problem.

2
 Algorithms

• Input : All algorithms can have zero or more inputs. The logic of
the algorithm should work on this input to give the desired result.
• Output : At least one output should be produced from the
algorithm based on the input given.
• Definiteness : Every step of algorithm should be clear and not
have any ambiguity.
• Finiteness : Every algorithm should have a proper end. The
algorithm can’t lead to an infinite condition.
• Effectiveness : Every step in the algorithm should be easy to
understand and can be implemented using any programming
language.
3
 Performance Analysis

• Performance analysis focuses on obtaining estimates of time

and space that are machine independent.
• Performance measurement obtains machine-dependent running
times.
• These times are used to identify inefficient code segments.

Definitions: The space complexity of a program is the amount of

memory that it needs to run to completion.
The time complexity of a program is the amount of computer
time that it needs to run to completion.

4
 Space complexity

The space needed by a program consists of the following

components:
(1)Fixed space requirements:
•This component refers to space requirements that do not depend on
the number and size of the program's inputs and outputs.
•The fixed requirements include the instruction space (space needed
to store the code), space for simple variables, fixed-size structured
variables (such as structs) and constants.

5
 Space complexity

(2) Variable space requirements:

•This component consists of the space needed by structured
variables whose size depends on the particular instance I of the
problem being solved.
•It also includes the additional space required when a function uses
recursion.
•The variable space requirement of a program P working on an
instance I is denoted Sp(I).
•Sp(I) is usually given as a function of some characteristics of the
instance I.
• Commonly used characteristics include the number, size, and
values of the inputs and outputs associated with I. 6
 Space complexity
(2) Variable space requirements:
•For example, if our input is an array containing n numbers then n is
an instance characteristic.
•If n is the only instance characteristic we wish to use when
computing Sp(I), we will use Sp(n) to represent Sp(I).
•We can express the total space requirement S(P) of any program as:
S(P) = c + Sp(I)
•where c is a constant representing the fixed space requirements.
•When analyzing the space complexity of a program we are
usually concerned with only the variable space requirements.
•This is particularly true when we want to compare the space
complexity of several programs.
7
 Space complexity
Example1 :

8
 Space complexity
Example1 :

•Sabc(I) = 0

9
 Space complexity
Example2 :

10
 Space complexity
Example2 :
We want to add a list of numbers (Program 1.10).
•Although the output is a simple value, the input includes an
array.
•Therefore, the variable space requirement depends on how the
array is passed into the function.
• Programming languages like Pascal may pass arrays by value.
This means that the entire array is copied into tem-porary storage
before the function is executed. In these languages the variable
space requirement for this program is Ssum(I) = Ssum(n) = n
•where n is the size of the array.

11
 Space complexity
Example2 :
•C passes all parameters by value.
•When an array is passed as an argument to a function, C
interprets it as passing the address of the first element of the array.
•C does not copy the array.
•Therefore, Ssum(n) = 0

12
 Space complexity
Example3 :

13
 Space complexity
Example3 : Program 1.11 also adds a list of numbers, but this
time the summation is handled recursively.
•This means that the compiler must save the parameters, the local
variables, and the return address for each recursive call.
•In this example, the space needed for one recursive call is the
number of bytes required for the two parameters and the return
address.
•We can use the sizeof function to find the number of bytes
required by each type.
•On an 80386 computer, integers and pointers require 2 bytes of
storage and floats need 4 bytes.
•Figure 1.1 shows the number of bytes required for one recursive
call. 14
Space complexity

15
 Space complexity
• If the array has n = MAX-SIZE numbers, the total variable space
needed for the recursive version is
Srsum(MAX-SIZE) = 6*MAX-SIZE.
• If MAX-SIZE = 1000, the variable space needed by the recursive
version is 6*1000 = 6,000 bytes.
• The iterative version has no variable space requirement.
• As you can see, the recursive version has a far greater overhead
than its iterative counterpart.

16
 Time complexity
•The time T(P) taken by a program P is the sum of its compile time and its
run (or execution) time.
•The compile time is similar to the fixed space component since it does not
depend on the instance characteristics.
•In addition, once we have verified that the program runs correctly, we may
run it many times without recompilation.
•Consequently, we are really concerned only with the program's execution
time Tp .
•Determining Tp is not an easy task because it requires a detailed knowledge
of the compiler's attributes.
• That is, we must know how the compiler translates our source program into
object code.

17
 Time complexity
• For example, suppose we have a simple program that adds and
subtracts numbers.
• Letting n denote the instance characteristic, we might express
Tp(n)
as:
Tp(n) = ca ADD(n) + cs SUB(n) + cl LDA(n) + cst STA(n)
• where ca , cs, cl and cst are constants that refer to the time needed
to perform each operation
and
• ADD, SUB, LDA, STA are the number of additions,
subtractions, loads, and stores that are performed when the
program is run with instance characteristic n.
18
 Time complexity
• Obtaining such a detailed estimate of running time is rarely
worth the effort.
• If we must know the running time, the best approach is to use
the system clock to time the program.
• Alternately, we could count the number of operations the
program performs.
• This gives us a machine-independent estimate, but we must
know how to divide the program into distinct steps.

Definition: A program step is a syntactically or semantically

meaningful program segment whose execution time is
independent of the instance characteristics.
19
 Time complexity
• Note that the amount of computing represented by one program
step may be different from that represented by another step.
• So, for example, we may count a simple assignment statement
of the form a = 2 as one step and also count a more complex
statement such as a = 2*b+3*c/d-e +f /g/a /b/c as one step.
• The only requirement is that the time required to execute each
statement that is counted as one step be independent of the
instance characteristics.
• We can determine the number of steps that a program or a
function needs to solve a particular problem instance by creating
a global variable, count, which has an initial value of 0 and then
inserting statements that increment count by the number of
program steps required by each executable statement.
20
 Time complexity
Iterative summing of a list of numbers :
•

Note that the step count only tells how many steps are executed, it
does not tell us how much time each step takes. 21
 Time complexity

2n+3 steps
22
 Time complexity
Recursive summing of a list of numbers :

23
 Time complexity

2n+2
24
 Time complexity

•To determine the step count for this function, we first need to figure out the step
count for the boundary condition of n = 0. Looking at Program 1.14, we can see
that when n = 0 only the if conditional and the second return statement are
executed. So, the total step count for n = 0 is 2. For n > 0, the if conditional and
the first return statement are executed. So each recursive call with n > 0 adds
two to the step count. Since there are n such function calls and these are
followed by one with n = 0, the step count for the function is 2n + 2.
25
 Time complexity
• By physically placing count statements within our
functions we can run the functions and obtain precise
counts for various instance characteristics.
• Another way to obtain step counts is to use a tabular
method.
• To construct a step count table we first determine the step
count for each statement. We call this the steps/execution,
or s/e for shorts.
• Next we figure out the number of times that each
statement is executed. We call this the frequency.
 Time complexity
• The frequency of a nonexecutable statement is zero.
• Multiplying s/e by the frequency, gives us the total steps
for each statement.
• Summing these totals, gives us the step count for the entire
function.
• Although this seems like a very complicated process, in
fact, it is quite easy.
Time complexity
Time complexity
 Time complexity
• The examples we have looked at so far were sufficiently simple that the time
complexities were nice functions of fairly simple characteristics like the
number of elements, and the number of rows and columns.
• For many programs, the time complexity is not dependent solely on the
number of inputs or outputs or some other easily specified characteristic.
• Consider the function binarysearch . This function searches an ordered list.
A natural parameter with respect to which you might wish to determine the
step count is the number of elements(n) in the list. That is, we would like to
know how the computing time changes as we change the number of elements
n. The parameter n is inadequate. For the same n, the step count varies with
the position of the element searchnum that is being searched for. We can
extricate ourselves from the difficulties resulting from situations when the
chosen parameters are not adequate to determine the step count uniquely by
defining 'three kinds of steps counts: best case, worst case and average.
 Time complexity
• The best case step count is the minimum number of steps that
can be executed for the given paramenters.

• The worst case step count is the maximum number of steps that
can be executed for the given paramenters.

• The average step count is the average number of steps executed

on instances with the given parameters.
 Time complexity
Matrix addition:

32
 Time complexity
Matrix addition:
•We want to determine the step count for a function that adds two-
dimensional arrays (Program 1.15).
•The arrays a and b are added and the result is returned in array c.
• All of the arrays are of size rows x cols.
•Program 1.16 shows the add function with the step counts
introduced.
•As in the previous examples, we want to express the total count in
terms of the size of the inputs, in this case rows and cols.

33
Time complexity

34
 Time complexity
Matrix addition:

35
Thank you