EEE 13 LECTURE 9

INTRODUCTION TO ALGORITHM
COMPLEXITY
MOTIVATION

REAL WORLD SCENARIO

It’s 8:00PM of February 13. You want to give chocolates to
your valentine tomorrow. How would get chocolates?



A. Make them yourself

B. Buy from a store

C. Buy from your org friends
MOTIVATION

ANOTHER REAL WORLD SCENARIO

You have to buy components for your EEE 34 (or EEE 100) project
so that you can work tomorrow. It is currently 2:30PM, and you
have EEE 13 at 5PM. The components that you need can only be
found in the Megamall Branch of Alexan. How do you propose to
travel to Megamall and get back on time?



A. Pantranco -> MRT

B. Taxi

C. Uber

D. Grab

E. Call a friend
MOTIVATION

HOW DO WE COMPARE OUR OPTIONS

1. Time

2. Distance

3. Price
MOTIVATION

WHAT ARE DATA STRUCTURES AND ALGORITHMS?

▸ Abstract Data Type - a mathematical model for data types,

specifically in terms of possible values, possible operations
on data of this type, and the behavior of these operations

▸ Data Structure - a way of organizing data (Recall: Structs),

implementation of an abstract data type

▸ Algorithm - a process or set of rules to be followed in

calculations or other problem-solving operations,
especially by a computer
MOTIVATION

WHY STUDY DATA STRUCTURES AND ALGORITHMS?

▸ Solve problems efficiently

▸ Model complex systems

▸ Image and Video Processing
▸ Speech Processing
▸ Search
▸ Networks
▸ Electricity Grids
▸ Genomes
▸ Stocks
▸ Economic Models
▸ …and many more!
MOTIVATION

PERFORMANCE GAINS BECAUSE OF

ALGORITHMS > PERFORMANCE GAINS
BECAUSE OF HARDWARE
IMPROVEMENTS

2010 Report to the US President and US Congress

MOTIVATION

WHY STUDY DATA STRUCTURES AND ALGORITHMS?

▸ Provide novel solutions to unsolved problems

▸ Monte Carlo Simulations

▸ Fulfill your desire of solving problems in general (Fun!)

MOTIVATION

CASE: MULTIPLICATION

▸ Let’s multiply 4761 and 5041 using the method we

learned in grade school. (Quick Answer: 24000201)

▸ How many times did we do single digit addition and

single-digit multiplication? (Quick Answer: ~4n2 where n is
the number of digits of each of the two inputs)
MOTIVATION

INTRODUCING KARATSUBA MULTIPLICATION

Using the previous example, let us do the following

▸ Let x = 4761 and y = 5041

▸ Let a = 47, b = 61, c = 50, d = 41

▸ Calculate ac = 2350 -> [A]

▸ Calculate bd = 2501 -> [B]

▸ Calculate (a + b)*(c + d) = 9828 -> [C]

▸ Calculate [C] - [A] - [B] = 4977 -> [D]

▸ Finally, calculate ([A] * 10^4) + [B] + ([D] * 10^2) = 24000201 (SAME!)

MOTIVATION

INTRODUCING KARATSUBA MULTIPLICATION

Why did that work?



xy = (10(n/2)a + b)*(10(n/2)c + d)

= 10nac + 10(n/2)(ad + bc) + bd

With the formula above, we can create a recursive code that

reduces the number of single-digit operations required to do
the complete multiplication process!



How fast is it? ~nlog2(3) (approx. n1.585)
ALGORITHM ANALYSIS

HOW DO WE ASSESS THE PERFORMANCE OF AN ALGORITHM?

Determine the effect of the input in terms of:



1. Processing Time

2. Memory Usage

3. Other resources (CPU, I/O, etc.)

ALGORITHM ANALYSIS

HOW?

▸ Actual - measure using tools, time.h, or similar during use

▸ Random Data - plug-in random data sets, trial-and-error

▸ Perverse Data - plug-in worst possible input

ALGORITHM ANALYSIS

THEORETICAL APPROCHES

▸ Worst case analysis

▸ Average case analysis

▸ Best case analysis

BIG-OH NOTATION

BIG-OH NOTATION: RELATIVE REPRESENTATION OF THE COMPLEXITY OF AN ALGORITHM

▸ A notation to indicate the worst-case performance of an

algorithm
▸ It looks like O(<something>)

▸ Essentially a notation that classifies algorithms according

to their complexity

▸ Mathematical approximation of the performance of an

algorithm using the smallest number of terms possible
BIG-OH NOTATION

EXAMPLE: O(1) - EXECUTES AT CONSTANT TIME REGARDLESS OF INPUT

void printFirstElementInt(int iArr[])


{

printf("First element is: %d\n", iArr[0]);

}
BIG-OH NOTATION

EXAMPLE: O(1) - EXECUTES AT CONSTANT TIME REGARDLESS OF INPUT

void doSomethingTrivial()

{

int a = 0, b = 1, c = 2;



c = a + b;



printf("%d + %d = %d\n", a, b, c);

}
BIG-OH NOTATION

EXAMPLE: O(N) - TIME TO COMPUTE SCALES LINEARLY WITH INPUT SIZE

void printElements(int iArr[], int n)

{
int ctr;



for (ctr = 0; ctr < n; ctr++) {

printf("iArr[%d] = %d\n", ctr, iArr[ctr]);

}

}
BIG-OH NOTATION

EXAMPLE: O(N) - TIME TO COMPUTE SCALES LINEARLY WITH INPUT SIZE

void printElementsTwice(int iArr[], int n)

{
int ctr;



for (ctr = 0; ctr < n; ctr++) {

printf("iArr[%d] = %d\n", ctr, iArr[ctr]);

}


for (ctr = 0; ctr < n; ctr++) {


printf("iArr[%d] = %d\n", ctr, iArr[ctr]);

}

}
BIG-OH NOTATION

2
EXAMPLE: O(N ) - TIME TO COMPUTE SCALES QUADRATICALLY WITH INPUT SIZE

void printMatrix(int **iArr, int rows, int cols)

{
int ctr1, ctr2;

for (ctr1 = 0; ctr1 < rows; ctr1++) {

for (ctr2 = 0; ctr2 < cols; ctr2++) {
printf("iArr[%d][%d] = %d\n", ctr1, ctr2, iArr[ctr1][ctr2]);
}
}

}
BIG-OH NOTATION

2
EXAMPLE: O(N ) - TIME TO COMPUTE SCALES QUADRATICALLY WITH INPUT SIZE

void printMatrix(int **iArr, int rows, int cols)

{
int ctr1, ctr2;

for (ctr1 = 0; ctr1 < rows; ctr1++) {

for (ctr2 = 0; ctr2 < cols; ctr2++) {
printf("iArr[%d][%d] = %d\n", ctr1, ctr2, iArr[ctr1][ctr2]);
}
}

//Print first row only

for (ctr1 = 0; ctr1 < cols; ctr1++) {

printf("iArr[0][%d] = %d\n", ctr1, iArr[0][ctr1]);

}

}
BIG-OH NOTATION

N
EXAMPLE: O(2 ) - TIME TO CALCULATE DOUBLES PER INCREASE IN INPUT SIZE

int Fibonacci(int number)

{
if (number <= 1) return number;

return Fibonacci(number - 2) + Fibonacci(number - 1);

}
BIG-OH NOTATION

EXAMPLE: O(LOG(N)) - DIVIDE A LINEAR DATA SET INTO TWO EACH ITERATION

int bsearch (int *a, int n, int x) {

int i = 0, j = n - 1;
while (i <= j) {
int k = (i + j) / 2;
if (a[k] == x) {
return k;
}
else if (a[k] < x) {
i = k + 1;
}
else {
j = k - 1;
}
}
return -1;
}
BIG-OH NOTATION

EXAMPLE: O(LOG(N)) - DIVIDE A LINEAR DATA SET INTO TWO EACH ITERATION

int bsearch_r (int *a, int x, int i, int j) {

if (j < i) {
return -1;
}
int k = (i + j) / 2;
if (a[k] == x) {
return k;
}
else if (a[k] < x) {
return bsearch_r(a, x, k + 1, j);
}
else {
return bsearch_r(a, x, i, k - 1);
}
}
BIG-OH NOTATION

EXAMPLE: O(N*LOG(N))

▸ Merge Sort

▸ Heap Sort
GENERALIZATION

GROWTH OF FUNCTIONS

▸ Generally, we can measure the run time T of a code

depending on the input size N

▸ Time is usually defined as the number of operations

▸ T = f(N)
GENERALIZATION

GROWTH OF FUNCTIONS
int func1(int aList[], int N) {
int iSum;

int i;
iSum = 0; /* 1 */

i = 0; /* 1 */

while( i < N) { /* N+1 */
iSum += aList[i]; /* 3N */

i++; /* N */
}

return iSum; /* 1 */
} /* f(N) = 5N + 4 */
GENERALIZATION

GROWTH OF FUNCTIONS
void func2(int aList[], int N) {

int iSum;
int i;
int j;
for (i = 0; i < N; i++) { /*1 + N + 1 + N = 2N + 2*/

for (j = 0; j < N; j++) { /*N*(2N+2)*/

printf("%d\n", aList[j]*aList[i]);/*N*4N*/

}

/*f(N) = 6N2 + 4N + 2*/
GENERALIZATION

BIG-OH NOTATION - MATHEMATICAL DEFINITION

▸ We use the Big-Oh notation to allow us to classify

algorithms according to upper bounds on their total
running times.

▸ A function g(N) is said to be O(f(N)) if there exists a

constants C0 and N0 such that:

▸ g(N) < C0 f(N)


for all N > N0. We write

▸ g(N) = O(f(N))
GENERALIZATION

BIG-OH NOTATION - MATHEMATICAL DEFINITION

▸ T(N) = 5N + 4 = O(N)



Let there be constants C0 and N0 such that:



C0 N > 5N + 4, when N > N0

(Example: C0 = 6, N0 = 4)
GENERALIZATION

BIG-OH NOTATION - PROPERTIES

▸ If T(N) = O(f(N)) and S(N) = O(g(N)), then:




1. T(N) + S(N) = max((O(f(N)), (O(g(N)))

2. T(N)*S(N) = O(f(N) * g(N))



In simpler terms:

1. Get the fastest growing expression

2. You can multiply Big-Oh expressions
GENERALIZATION

BIG-OH NOTATION - THINGS TO TAKE NOTE

▸ The greatest running time (greatest power/degree) is the

only one that counts

▸ For loops, note the maximum number of iterations and if

the "calculation space" is being divided by two every
single run
BEYOND BIG-OH

OTHER NOTATIONS FOR MEASURING PERFORMANCE

▸ Omega Notation

▸ Theta Notation

▸ Little-oh notation



Why? Worst-case does not always happen
READINGS

SOME LINKS TO HELP YOU OUT

▸ http://stackoverflow.com/questions/487258/what-is-a-plain-
english-explanation-of-big-o-notation

▸ https://www.interviewcake.com/article/java/big-o-notation-time-
and-space-complexity

▸ https://rob-bell.net/2009/06/a-beginners-guide-to-big-o-
notation/

▸ https://justin.abrah.ms/computer-science/big-o-notation-
explained.html

▸ http://bigocheatsheet.com/
SEE YOU NEXT
WEEK :D