Algorithms and complexity analysis

• An algorithm is a step-by-step procedure used to

solve a problem (often includes repetition of steps)
Application of the search algorithm - Example
The city of Grand Rapids has 500,000 records in its
engineering database including:

Storm Inlets, Sanitary manholes, Water system valves.

Each element has a unique key code for example:

361255 is the water valves in Monroe St. near McKAY tower.

We would like the design an algorithm to retrieve

records as fast as possible (in a flash).
Advanced algorithms make the software more expensive
For example:
MS Access is a low level database system which cost ~$130
Oracle software has advanced algorithms for database
operations and cost much more
 Pseudo code
• Describing algorithms requires a notation for
expressing a sequence of steps to be performed.
The three most common options are:
• (1) English
• (2) pseudo code, or
• (3) programming language like Visual Basic, C++, Java
• There is a natural tradeoff between greater ease of
expression and precision.
• Pseudocode is a structured English version of the
program which uses English nouns and verbs and
VB/C# control structure syntax to express the way the
program should work (the algorithm).
 LOOPS OR Iterations
• We would like to repeat a certain operation:
Print a column of 4 stars on a form
simple solution
Print " "
*
Print " "
*
Or use a loop
Print " "
*
Print " "
*
True Statements
i<4?
print “x”

False
 Searching - Sequential

• Sequential Search method - checks every record in the list

• We would like to look for the phone number of Mr. Thompson
using the Sequential Search method
Pseudo code
Input: list of names (phone book)
Do While not End of list
Read line into memory, KEY = first word in the line
If (Key = “Thompson”) Then
Display record
End
End If
Loop
Output: phone number of Mr. Thompson
 Complexity and algorithms analysis

• In assessing how efficient an algorithm is, computer scientists

concentrate on what is called algorithm complexity - how the
performance of the algorithm is affected by the amount of data
to be processed.
• The O notation denote the general behavior of an algorithm as
a function of the problem size (n=number of points) .
• Time of the Sequential Search method is in the Order of n
( Number of records = 500000)
• The theoretical analysis of complexity is done by highlighting
which components are dependent on the number of points and
therefore play an important part in the algorithm.
 Binary search -Pseudo code
• Input ordered file
• Output record of element no 361255
• Low=1: High=500000
• Do While high>=low
• Mid=(low+high) / 2
• Read line Mid into memory, KEY = first word in the
line
• If (KEY > 361255) then high = mid-1
Elseif (KEY < 361255) then low = mid+1
else
Display record
End
• End If
• Loop
 Binary search

• Each step of the

algorithm divides
the block of
items being
searched in half.
• Divide and
conquer strategy
Binary Search at:
http://www.cs.armstrong.edu/liang/animation/web/BinarySearch.html
 Searching - Binary search
1. How many steps will it take to find the “ Surveying Eng. Dep.
2. Describe the procedure
No. Name Phone Address
1 Automobile repair shop 5224569 321 Parkway Dr.
2 Bank Two 5913232 123 State St.
3 GIS consultants 5228844 542 Main St.
4 Hospital, Mecosta 5913311 44 Michigan St.
5 Real Estate services 5226666 54 Division St.
6 Surveying Eng. Dep. 5912845 915 Campus Dr.
7 Tire Center 5226660 1027 14 Mile Rd.
 Searching - Binary search
No. Name Phone Address
1 Automobile repair shop 5224569 321 Parkway Dr.
2 Bank Two 5913232 123 State St.
3 GIS consultants 5228844 542 Main St.
4 Hospital, Mecosta 5913311 44 Michigan St.
5 Real Estate services 5226666 54 Division St.
6 Surveying Eng. Dep. 5912845 915 Campus Dr.
7 Tire Center 5226660 1027 14 Mile Rd.
Search for Surveying Eng. Dep.
1. Low=1: High=7 =>
2. Mid=4 =>
3. Key=”Hospital Mecosta” =>
4. H<S =>
5. Low =Mid+1=5
6. Low=5: High=7 =>
7. Mid=6 =>
8. Key=” Surveying Eng. Dep.” => Bingo !!
 Complexity - What's the big deal ?
• O(1) Constant growth –can’t beat it!
• O( log (n)) Logarithmic growth
• O(n) Linear Growth
• O(n2) Quadratic growth
• O(2n), O(3n) Exponential growth,
• O(nn), O(n!) acceptable only for really small inputs (e.g. n<20)
 Complexity and algorithms analysis
• We say that O(g(n)) if it is
the set of all functions which
grow no faster than g(n).

• Worst-case time: It is the

maximum running time over
all (legal) inputs of size n.

• Average-case time: It is the

average running time over all
legal inputs of size n.
 Complexity and algorithms analysis
• Simple list – each new item is placed at the end of the
file. It takes an average of O(n+1)/2 operations to find
the item (when we have n points in a list)
• Ordered sequential list – dictionary or phone book.
Binary search requires O(log2(n)) operations.

• Example: imagine that each operation takes a second

than with 100000 points database and
• Using linear search the operation should take
• (10000+1)/2= 50000 seconds
• Using binary search the operation should take
• Log2100000=log10100000/log102=5/0.3=15 seconds
 Can we improve binary search?
• Binary search is notoriously tricky algorithm; for
less then 100 elements sequential search may be even
better.
• Are certain items accessed more often then other
ones?
• For Example
The hospital phone number is used more often than
…
• By putting the most popular words at the top of the
list and the least ones at the bottom we can reduce
the number of comparison.

• Remember: In order to use the advantage of Binary

search tree we need to have an ordered or sorted file.
 Sorting
• Sorting is one of the most important operations
performed by computers
• Card players all know how to sort …
• First card is already sorted
• With all the rest,
Scan back from the end until you find the first card
larger than the new one,
Move all the lower ones up one slot
insert it


         
A K 10 2 J 2 2 Q 9

9 
 Bubble Sorting
• Bubble, Selection, Insertion Sorts
• Uses the same strategy that you use for sorting your
bridge hand. You pick up a card, start at the beginning
of your hand and find the place to insert the new card,
insert it and move all the others up one place.
• From the first element
• Exchange pairs if they’re out of order
• Last one must now be the largest
• Repeat from the first to n-1
• Stop when you have only one element to check

2 8 5 7 6 1 3 4
 Bubble Sorting
http://math.hws.edu/eck/js/sorting/xSortLab.html
/* Bubble sort for integers */

For i=0 to n
for j=1 to n-i
if( a[j-1]>a[j] ) then
temp=a[j-1]
a[j-1]=a[j]
a[j]=temp
End if
Next j
Next I

The complexity of the algorithm is (arithmetic

series):

(n-1)+..7+6+5+4+3+2+1=
n( n  1)
 O(n 2 )
2
 Bubble Sorting
GIS Bank Bank Bank Bank
Bank GIS GIS GIS GIS
Hospital Hospital Hospital Hospital Hospital
Tire Tire Tire Tire Automobile
Automobile Automobile Automobile Automobile Tire
Surveying Surveying Surveying Surveying Surveying
Real Real Real Real Real

Bank Bank
GIS GIS
Hospital Hospital
Automobile Automobile
Surveying Surveying
Tire Real
Real Tire

http://math.hws.edu/TMCM/java/xSortLab/
 Quicksort
• Efficient sorting algorithm
• Discovered by C.A.R. Hoare
• C. A. R. Hoare, Quicksort, Comp. J.
5(1) p10-15 1962.

• Example of Divide and Conquer

algorithm
• Two phases
• Partition phase
• Divides the work into half
• Sort phase
• Conquers the halves!

http://cg.scs.carleton.ca/~morin/misc/sortalg/
 Quicksort
• Partition
• Choose a pivot
• Find the position for the pivot so that
• all elements to the left are less
• all elements to the right are greater

< pivot pivot > pivot

http://www.cs.auckland.ac.nz/software/AlgAnim/qsort.html
Quicksort

• Conquer
• Apply the same algorithm to each half

< pivot > pivot

< p’ p’ > p’ pivot < p” p” > p”

Class Exercise
Class Exercise
 Recursion
A routine's calling of itself in order to repeat a process in
computing. For example the factorial function:
Function factorial( n As Integer) As Integer
if ( n = 0 ) then factorial =1
else
factorial = n * factorial( n-1 )
End if
End Function
Note how this function calls itself to evaluate the next term.
Eventually it will reach the termination condition and exit.
The termination condition is obviously extremely important when
dealing with recursive functions. If it is omitted, then the function
will continue to call itself until the program runs out of stack
space - usually with moderately unpleasant results!
 Quick Sort
Function quicksort (a, low, high )
if ( high > low )
pivot = partition( a, low, high )
quicksort ( a, low, pivot-1 )
quicksort ( a, pivot+1, high )
End Function
• a is the list of records,
• The partition subroutine examines every item in the array
and determine if it is bigger or smaller than the pivot by
that divide the given list into two
• The pivot is taken as the lowest element in the list.
Quicksort is a O(n logn) algorithm. However, in the worst
case, quicksort is an O(n2) algorithm! A better method is the
HEAP SORT
GISC225 student using an input device

110 A-6
Exam review - Binary numbers questions

• (101101)2 = = (45)10

• (75)10 = = (1001011)2

• (101101)2 + (1110)2 = = (111011)2

• (111010)2 - (11001)2 = =(100001)2

• (1011)2 ×(101)2 = =(110111)2

• (13)8 = =(11)

• (4D3)16 =3+208+1024=1235
 Exam review DBMS – Example

What are the results?

CREATE TABLE Projects ([J#] TEXT, [JNAME] TEXT, PRIMARY KEY ([J#]) );
ALTER TABLE Projects ADD CITY TEXT
INSERT INTO Projects VALUES("J1", "SERVCO" , "Big Rapids" );
 DBMS – Example

The three tables Part, Suppliers, and Projects have many to many
relationships through table SPJ (the Bridges table)
 DBMS – Example

What are the results?

SELECT * FROM Parts

WHERE Parts.CITY <> 'Los Angeles‘ And (Parts.Price<1);
 DBMS – Example

What are the results?

SELECT AVG(Parts.Price) FROM Parts; 0.816

 DBMS –
Example

SELECT Projects.JNAME, Parts.PNAME, SPJ.QTY

FROM Projects INNER JOIN (Parts INNER JOIN SPJ ON Parts.
[P#] = SPJ.[P#]) ON Projects.[J#] = SPJ.[J#]
WHERE (((Projects.JNAME)="LakeView"));