Sunday, 30 March 2014

! of 7 1 !

Algorithms revision
Algorithms and Articial Intelligence AC21007 Introduction
An algorithm is a nite set of instructions that, if followed, accomplish a particular task. They have the following criteria:

"

" "

Input: zero or more. Output: at least one quantity produced (but not necessarily output) Deniteness: clear and unambiguous. Finiteness: always nishes after a given number of steps. Effectiveness: every instruction must be basic such that it can be carried out by a person.

Creating an algorithm can never be fully automated because it requires an understanding of the problem. Algorithms can also be validated/veried, analysed for space/time complexity, and tested (debugging and proling). Recursion is just the calling of a function from within itself. This builds a stack of functions, and then each one returns a value to the previous function in the stack. Eventually, the bottom function in the stack returns a value to the caller program.

Space complexity
Space complexity is the measure of approximately how much space the function needs in order to return a value to the caller program. This includes any recursive calls and variables inside the function.

"

If something doubles in size, repeatedly at every run, we have a O(2n) function. If something halves in size at every run, we have a O(log2n) function. Remember the two comes from the fact that we are either multiplying or dividing by 2. If we are multiplying or dividing by a different number, the 2 will change in our complexity.

"

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 2 !

How the algorithms return statements are structured will affect the maximum size of our recursive stack, and therefore our space complexity. For example in a Fibonacci algorithm, we only ever store the one branch we are traversing, and therefore the space complexity is O(n). If we store the whole tree (which if we are only trying to compute values isnt necessary), then the space complexity would be the predicted O(2n).

" "

We always use the complexity of the highest section of the polynomial: if we have a function that has space complexity of 2n3 + n2 + 5, the complexity is O(n3).
Algorithm Iterative Factorial Iterative Fibonacci Iterative Euclidean Iterative Sum / Odd numbers, etc Space complexity O(1) O(1) O(1) O(1)

"
Algorithm Recursive Sum Recursive Factorial Recursive Fibonacci / Tribonacci Recursive Permutation Generator Space complexity O(n) O(n) O(n) O(n)

"
Algorithm Recursive Euclidean Space complexity O(log

"

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 3 !

Time complexity
Time complexity is more or less the CPU usage, or the amount of time it takes for the algorithm to complete.

"

Operation Comment Assignment of variable Loops Loops (frequency)

Number of steps 1 1 1 Equal to the number of times the loop condition is executed (usually n+1, think of for i=1 <= n)

Remember that if there is a condition, the end complexity may differ if the condition is true or false. In these instances, separate complexities must be calculated for each condition.

" "

Sorting algorithms are worst-case O(n2), except MergeSort which is worst-case O(n.log2n). Below, Paper No. 4, Question 1.

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 4 !

Recursion
If we have a recursive algorithm which calls itself, then the time complexity of said algorithm is equal to the complexity of one function call plus the complexity of all recursive calls inside the algorithm.

"

" "

Algorithm RSum (a,n) { if (n!0) then return 0.0 else return RSum(a,n-1)+a[n]; }

Here the complexity for RSum(a, n) is always 2+x, where x is RSum(a, n-1). Therefore if we call RSum with parameters a={1, 3, 5, 6} and n=4, the algorithm stack trace is as follows: RSum({1, RSum({1, RSum({1, RSum({1, RSum({1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 6}, 4) 3) 2) 1) 0) 4) 3) 2) 1) 0) 4) 3) 2) 1) 0) 4) 3) 2) 1) 0) => => => => => 0 => => => => 1 => 0 => => => 4 => 1 => 0 => => => => =>

"

RSum({1, RSum({1, RSum({1, RSum({1, RSum({1,

"

RSum({1, RSum({1, RSum({1, RSum({1, RSum({1,

"

RSum({1, RSum({1, RSum({1, RSum({1, RSum({1,

"

9 4 1 0

RSum({1, 3, 5, 6}, 4) => 15

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 5 !

"

RSum({1, RSum({1, RSum({1, RSum({1,

3, 3, 3, 3,

5, 5, 5, 5,

6}, 6}, 6}, 6},

3) 2) 1) 0)

=> => => =>

9 4 1 0

So the algorithm runs n+1 times, and the complexity of each run of the algorithm is 2. (n+1) multiplied by 2 gives us 2n+2, which is the specic complexity of this algorithm.

" Therefore the complexity is O(n). "

So RSum(a, n) = RSum(a, n-1) + 1. This is equal to T(n) = T(n-1) + O(1). This recurrence relation gives us a complexity of O(n).

" Other recurrence relations to remember are: "

Recurrence relation T(n) = T(n-1) + O(1) T(n) = 2T(n/2) + O(1) T(n) = T(n/2) + O(1) T(n) = T(n-1) + O(n) T(n) = 2T(n/2) + O(n) T(n) = 2T(n-1) + O(1) Big-Oh solution O(n) O(n) O(log O(n2 O(n.log O(2n O(n2

Algorithm Sequential search Tree traversal Binary search Sorting algorithms except MergeSort MergeSort (special because of its divide-and-conquer method) Binary tree construction

"

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 6 !

" "

Below, Paper No. 5, Questions 3 and 4.!

Jose Salvatierra (http://jslvtr.com)

Sunday, 30 March 2014

! of 7 7 !

Complexity
What is the notation of best, worst, and average case for a function?

"

O is the upper bound when multiplied by a constant. ! is the lower bound when multiplied by a constant. " can be the lower or upper bounds, and it only exists if there are constants C1 and C2 such that multiplying them by the function gives us values always higher or lower than the target function.

"

For these three we only care about values beyond which the target function is surpassed: we dont care about small values.

Jose Salvatierra (http://jslvtr.com)