Notes DAA
Notes DAA
Algorithm: The word algorithm comes from the name of a Persian author „Abu Jafar
Mohammed ibn Musa al Khowarizmi‟, who wrote a text book on mathematics. According to
him an algorithm is a set of rules used to perform some calculations either by hand (or) more
usually on a machine.
(or)
Design of Algorithm: The study of algorithm includes many important and active areas
of researches. They are:
Understanding the problem: This is the very first step in designing of an algorithm.
In this step first of all need to understand the problem statement completely by
reading the problem description carefully. After that, find out what are the necessary
inputs for solving that problem. The input to the algorithm is called instance of the
problem. It is very important to decide the range of inputs so that the boundary
values of algorithm get fixed. The algorithm should work correctly for all valid
inputs.
Decision Making: After finding the required input set for the given problem we have
to analyze the input and need to decide certain issues such as
Capabilities of computational devices: It is necessary to know the
computational capabilities of devices on which the algorithm will be running
ie sequential or parallel algorithm. Complex problems require huge amount of
memory and more execution time. For solving such problems it is essential to
have proper choice of a computational device which is space and time
efficient.
Choice for either exact or approximate problem solving method: The next
important decision is to decide whether the problem is to be solved exactly or
approximately. If the problem needs to be solved correctly then we need exact
algorithm. Otherwise, if the problem is so complex then we need
approximation algorithm.(Example: Salesperson Problem)
Data Structures: Data Structure and algorithm work together and these are
interdependent. Hence choice of proper data structure is required before
designing the actual algorithm.
Algorithmic Strategies: It is a general approach by which many problems can
be solved algorithmically. Algorithmic strategies are also called as algorithmic
techniques or algorithmic paradigm.
Algorithm Design Techniques:
Brute Force: This is straight forward technique with naïve approach.
Divide-and-Conquer: The problem is divided into smaller instances.
Dynamic Programming: The results of smaller, reoccurring instances
are obtained to solve the problem.
Greedy Technique: To solve the problem locally optimal decisions are
made.
Back tracking: This method is based on the trial and error.
Specification of Algorithm: There are various ways to specify an algorithm
Using natural language: It is very simple to specify an algorithm using natural
language
For Example: Write an algorithm to perform addition of two numbers.
Step1: Read the first number say a.
Step2: Read the second number say b.
Step3: Add the two numbers and store the result in a variable c.
Step 4: Display the result.
Pseudo code: It is a combination of natural language and programming
language
Algorithm sum(a,b):
// Addition of two numbers
//Input: Two numbers a and b
//Output: sum of numbers
{
c=a+b;
Write(c)
}
Flowchart: Flowchart is a graphical representation of an algorithm
Start/ Stop state:
Transition:
Conditional Statement:
Algorithm Verification: Algorithm verification means checking correctness of an
algorithm that is to check whether the algorithm gives correct output in finite amount
of time for a valid set of input.
else
{
Statement 1;
Statement 2;
}
Performance Analysis: Algorithm evaluation can be done in two ways either before
execution of a program or after execution of a program
Where C is constant which is fixed space and Sp is variable space which varies
depend on the problem
Second Method
StatementNum Statement Steps per execution Frequency Total steps
1 Algorithm sum(a,n) 0 - 0
2 { 0 - 0
3 sum:=0; 1 1 1
4 for i:=1 to n 1 n+1 n+1
5 sum:=sum+a[i] 1 n n
6 return sum; 1 1 1
7 } 0 - 0
Total Steps for algorithm 2n+3
First Method:
Algorithm Rsum(a,n):
// Addition of elements using recursion
{
count:=count+1; // for if condition
if(n<=0) then
count:=count+1; // for return stmt
return 0;
else
return Rsum(a,n)+a[n]; // for addition, function invocation and return
}
Time Complexity: 2(for n=0)+ TRsum(n-1)
2+TRsum(n-1) => 2+2+TRsum(n-2) …….. n(2)+TRsum(0) => 2n+2 n>0
Second Method:
StatementNum Statement Steps per execution Frequency Total steps
n=0 n>0 n=0 n>0
1 Algorithm Rsum(a,n): 0 - - 0 0
2 { 0 - - 0 0
3 if(n<=0) then 1 1 1 1 1
4 return 0; 1 1 0 1 0
5 else 0 - - 0 0
6 return Rsum(a,n)+a[n]; 1+x 0 1 0 1+x
7 } 0 - - 0 0
Consider f(n)=2n+2, g(n)=n2 so that f(n) < c*g(n) by using mathematical induction
Show that the time complexity of 3n2+4n-2 is O(n2)
Remaining problems see running notes
Big Omega Notation: It is denoted by Ώ. This notation is used to represent the lower
bound of algorithm runtime that is best case time complexity.
Definition: A function f(n) is said to be Ώ(g(n)) if f(n) is bounded below by some
positive constant multiple of g(n) such that f(n)≥ c*g(n)
If f(n)≥ c*g(n) then f(n)= Ώ(g(n))
Consider f(n)=2n2+5, g(n)=7n show that f(n)= Ώ(g(n))
Practical Complexities:
Time complexity of an algorithm is generally some function of the instance
characteristics.
This function is very useful in determining how the time requirements vary as the
instance characteristics change.
The complexity function can also be used to compare two algorithms P and Q that
perform the same task.
Assume that algorithm P has complexity Ɵ(n) and algorithm Q has complexity Ɵ(n2).
We can assert that algorithm P is faster than algorithm Q for sufficiently large n by
seeing polynomial of high degree.
How the various functions grow with „n‟. See the below table and figure.
By seeing the above figure we can say that the function 2n grows very rapidly with n.
Amortized Analysis:
Amortized Analysis means finding average running time per operation over a worst
case sequence of operations.
An Amortized analysis indicates that average cost of a single operation is small if
average of sequence of operations is obtained.
There is a difference between amortized and average case analysis. In average case
analysis, averaging over all possible inputs but in amortized analysis, averaging over a
sequence of operations.
Suppose that a sequence of operations I1, I2, D1, I3, I4, I5, I6, D2, I7 of insert and
delete operations are performed on set. Assume the actual cost of each seven inserts
takes 1 unit of time and delete operations D1 and D2 takes 8 and 10 respectively.
Total Actual Cost=7+8+10=25
In amortization scheme, charge some actual cost of operations to other operations.
This reduces cost of one operation and reduces cost of other operations.
If we charge cost of one unit for each insertion (I1 to I6) from delete operation then
D1=6, D2=6.
The two units of D1 is transferred to I1, I2 that is I1=2, I2=2 and 4 units of D2 is
transferred that is I3=2, I4=2, I5=2, I6=2, lastly I7=1.
Total Amortized Cost of I1, I2, D1, I3, I4, I5, I6, D2, I7 = 2+2+6+2+2+2+2++6+1=25
that is amortized cost equal to actual cost. In general amortized cost greater than or
equal to sum of their actual cost
There are three commonly used techniques used in amortized analysis:
Aggregate Analysis
Accounting Method
Potential Method
Aggregate Analysis: It is similar to average case analysis but we consider the
average for worst case sequence.
In aggregate analysis, if sequence of „n‟ operations takes worst case time T(n)
in total. In that worst case, the average cost or amortized cost per operation is T(n)/n
Amortized cost > Actual cost
Potential function p(i)=Amortization cost (i) – Actual cost(i) + p(i-1)
P(n) - P(0) > 0
Example 1: In Jan, you buy a new car from a dealer who offers you the following
maintenance contract $50 each month other than March, June, September and December,
$100 every March, June and September and $200 every December.
Worst Case Method: $50- Jan,Feb,Apr,May,Jul,Aug,Oct,Nov
$100- Mar,Jun,Sep
$200- Dec
Aggregate Method: To use aggregate method for amortized complexity, we first determine
an upper bound on the sum of the costs for the first „n‟ months. As tight a bound as is
possible is desired.
The sum of the actual monthly costs of the contract for the first „n‟ months is
(50*8)+(100*3)+(200)=400+300+200=900/12=75
MONTH 1 2 3 4 5 6 7 8 9 10 11 12
Actual Cost 50 50 100 50 50 100 50 50 100 50 50 200
Amortized Cost 75 75 75 75 75 75 75 75 75 75 75 75
P(i) 25 50 25 50 75 50 75 100 75 100 125 0
Potential Method: This method is similar to accounting method in which the concept
the prepay is used. In this method there will be no credit but there will be some
potential difference or energy which can be used to pay for future operations. Instead
of associating potential with specific object, it is associated with whole data structure.
Working of Potential Method: Let D0 be the initial data structure for „n‟ operations.
We have data structure D0 to Dn and then actual cost has C1…Cn. Potential function is
denoted by Ø then
Amortization cost = actual cost + potential difference
n n
|
∑ Ci = ∑ Ci+[Ø(Dn) – Ø(D0)]
i=1 i=1
Note: Amortized analysis is used for algorithms where an occasional operation is very
slow but most of the other operations are faster.
In Amortized analysis, we analyze a sequence of operations and guarantee a
worst case average time which is lower than the worst case time of a particular
expensive operation.
Performance Measurement:
Performance evaluation can be done in two ways. Before the execution of a program
called Performance Analysis and after the execution of a program called Performance
Measurement.
Performance Measurement is concerned with obtaining the space and time
requirements of a particular algorithm.
These quantities depend on the compiler and options used as well as on a computer on
which the algorithm is run.
To obtain computing or run time of a program we need clocking procedure that
returns the current time in milliseconds.
Time complexity can be calculated with the help of any programming language like
C, C++, Java and Python.
In C language, the time events are stores in standard library #include<time.h> and use
GetTime() to get current time in milliseconds.
In Java language, the time events are stores in package java.util.Date and use
getTime() to get current time in milliseconds.
In Python language, the time events are stores in package time and use time.time() to
get current time in milliseconds.
Performance Measurement can be calculated by subtracting start time from current
time that is performance measurement=current time - start time
Suppose we wish to measure the worst case performance of the sequential search
algorithm. First, we need to decide the values of n for which the times are to be
obtained and then determine for each of the above values of n, the data that exhibit the
worst case behavior.
Algorithm SeqSearch(a,x,n)
{
i:=n;
a[0]:=x;
while(a[i]!=x) do
i:=i-1;
return i;
}
In the worst case, to search for x, given the size n of a. An asymptotic analysis reveals
that this time is ϴ(n). So, we expect a plot of the times to be a straight line.
To measure the computing time of this algorithm we need to write a TimeSeqSearch
algorithm, in that GetTime() is used to get the current time and start time of algorithm
in milliseconds.
Finally, performance measurement of an algorithm can be done in milliseconds.
Algorithm TimeSeqSearch(a,x,n)
{
h:=GetTime();
i:=n;
a[0]:=x;
while(a[i]!=x) do
i:=i-1;
return i;
h1:=GetTime();
t:=h1-h;
write(t);
}
Now we are writing an algorithm to measure the time in milliseconds for „n‟ different
values.
Algorithm TimeSearch()
{
for j:=1 to 1000 do
{
a[j]:=j;
}
for j:=1 to 10 do
{
n[j]:=10*(j-1);
n[j+10]:=100*j;
}
for j:=1 to 20 do
{
h:=GetTime();
k:=SeqSearch(a,0,n[j]);
h1:=GetTime();
t:=h1-h;
write(n[j],t);
}
}
Timing results of above algorithm in milliseconds are
The times obtained are too small to be of any use to us. Most of the times are zero,
this indicates that precision of our clock is inadequate. The nonzero times are just
noise and are not representative of the time taken.
To time a short event, it is necessary to repeat it several times and divide the total time
for the event by the number of repetitions
Algorithm TimeSearch()
{
// Repetition factors
r[21]:={0, 200000, 200000, 150000, 100000, 100000, 100000, 50000,
50000, 50000, 50000, 50000, 50000, 50000, 50000, 50000, 50000,
25000, 25000, 25000, 25000}
for j:=1 to 1000 do
{
a[j]:=j;
}
for j:=1 to 10 do
{
n[j]:=10*(j-1);
n[j+10]:=100*j;
}
for j:=1 to 20 do
{
h:=GetTime();
for i:=1 to r[j]do
{
k:=SeqSearch(a,0,n[j]);
}
h1:=GetTime();
t1:=h1-h;
t:=t1;
t:=t/r[j];
write(n[j],t1,t);
}
}
Timing results and graph of above algorithm in milliseconds are
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner