Advanced Algorithms

(Lecture Note 2)

Introduction to
Algorithm (2)

2025. 3. 7

Prof. Wonjun Kim

School of Electrical and Electronics Engineering

Deep Computer Vision Lab.

Review of Last Class
Content
 Introduction to algorithm
• Definition of algorithm
• Warming-up with some examples using C

 Practice to be familiar with C programming !

 Today class gives …

• Computation of algorithms’ complexity
• Check it with the Euclidean examples

Deep Computer Vision Lab.

1
Algorithm Complexity (1/8)
Content
 Analysis of algorithm
• How can we measure the “complexity” ?
 Three main cases : worst case / average case / best case

User Face Fingerprint Vein Not

features verified ? verified ? verified ? valid

Score > 95 Score > 95 Score > 95

Pass Pass Pass

[ User verification system ]

* What is the general criterion ?

* Compare three cases and consider the above scenario
Deep Computer Vision Lab.
2
Algorithm Complexity (2/8)
Content
 Analysis of algorithm – cont’d
• Step of analyzing your algorithms

1. Apply the vast amount of data

: 1,000,000 images as the test dataset for face recognition
* Distribution of three cases can be obtained ( refine algorithm)

2. Decompose the algorithm into basic components

: For example, sorting algorithms  compare + exchange
* We can expect the processing time via CPU’s burdens by
the combination of those components

Deep Computer Vision Lab.

3
Algorithm Complexity (3/8)
Content
 Big-O notation

• Definition

For all the N satisfying N ≥ N0, if T(N) ≤ C0f(N), then T(N) = O(f(N))
* T(N) : function for the processing time

 Think about the meaning of this sentence

For example,
T(N) = 3*N*N + 1, f(N) = N*N, C0 = 4  O(N2)

Deep Computer Vision Lab.

4
Algorithm Complexity (4/8)
Content
 Big-O notation – cont’d
• Analysis based on the big-O notation
: It represents the worst case of the algorithm performance
cf) Best performance / average performance / worst performance

void main()
{
int sum = 0; 1 operation
int j; 1 operation
for( j=1; j<=30; j++) 30 operations 63 operations
{
sum += j; 30 operations  O(63)
}

printf(“Sum (1-30) : %d\n”, sum); 1 operation

}

Constant is ignored in Big-O. i.e., O(200) = O(63) = O(1) (Why ?)

Deep Computer Vision Lab.

5
Algorithm Complexity (5/8)
Content
 Big-O notation – cont’d
• Estimation of complexity via O(▪)
: Return the minimum value (consider # of for loops)

int find_min(int *a, int N)

{ int find_min(int *a, int N)
bool find_flag; {
int min = a[0];
for(int i=N-1; i>0; i--){
for(int j=0; j<N; j++){ for(int i=1; i<N; i++)
if(a[ j] > a[i]) {
fing_flag = false; vs. if(min > a[i])
} min = a[i];
if(find_flag) }
break;
} return min;
}
return a[i];
}

Complexity : O(?) Complexity : O(?)

Deep Computer Vision Lab.
6
Algorithm Complexity (6/8)
Content
 Big-O notation – cont’d
• One more example (practice) !
- Compute the complexity of the following algorithm

int test_sample(int *a, int N)

{
int sum = 0;

for(int i=0; i<N; i++) Problems :

{
for(int j=i+1; j<N; j++)
{ 1. Total # of “for” operations ?
sum += a[i]*a[ j];
} 2. What is the big-O notation ?
}

return sum;
}

Deep Computer Vision Lab.

7
Algorithm Complexity (7/8)
Content
 Big-O notation– cont’d
• Other types of O(▪) complexities
출처 : http://media.photobucket.com/image/amit_9b/BIGO_GRAPH.jpg

- O(log(N)) : smoothly increase

when input data

- O(Nlog(N)) : slightly fast increase

c.p. linear (N)
when input data
- O(N3 ) : e.g., for-for-for

- O(2N) : worst case

Try to build-up your algorithm with the complexity of

O(logN), O(NlogN), and O(N)

Deep Computer Vision Lab.

8
Algorithm Complexity (8/8)
Content
 Big-O notation – summary
• Complexity according to different O(▪) types

Function 10 100 1,000 10,000 100,000 1,000,000

1 1 1 1 1 1 1

log2N 3 6 9 13 16 19

N 10 102 103 104 105 106

Nlog2N 30 664 9,965 105 106 107

N2 102 104 106 108 1010 1012

N3 103 106 109 1012 1015 1018

2N 103 1030 10301 103,010 1030,103 10301,030

※ Input 1,000,000 has become common in computer vision !

(Think about 1000 x 1000 pixels image)

Deep Computer Vision Lab.

9
Euclidean Examples (1/10)
Content
 Great common divisor (GCD)
• Example 1 : find GCD between 280 and 30

Subtraction & replacement

2 280 30
A = 280, B = 30
5 140 15
28 3 A = aG, B = bG
A-B = (a-b)G, B = bG
2 x 5 = 10 (GCD)
① GCD of A and B =
GCD of A-B and B
How can CPU conduct this
operator ? ② GCD of A and B = B and A

③ GCD of A and 0 = A

Deep Computer Vision Lab.

10
Euclidean Examples (2/10)
Content
 Great common divisor (GCD) – cont’d
• Example 1 : find GCD between 280 and 30

GCD(280, 30) = GCD(250, 30) = GCD(20, 10)

= GCD(220, 30) = GCD(10, 10)
= GCD(190, 30) = GCD(0, 10)
= GCD(160, 30) = GCD(10, 0)
= GCD(130, 30) = 10
= GCD(100, 30)
= GCD(70, 30) - We require a lot of lines !
= GCD(40, 30) - How about CPU ?
= GCD(10, 30)
= GCD(30, 10)

Deep Computer Vision Lab.

11
Euclidean Examples (3/10)
Content
 Great common divisor (GCD) – cont’d
• Example 2 : find GCD between 280 and 30 (division-based)

Practice
④ GCD of A and B =
GCD of mod(A,B) and B Make C programs for GCD !

GCD(280, 30) = GCD(10, 30) void main()

= GCD(30, 10) {
// input two values
= GCD(0, 10)
= GCD(10, 0) val1 = GCD_minus(a,b);
val2 = GCD_mod(a,b);
= 10
// compare ouputs
# of operations are significantly
}
reduced

Deep Computer Vision Lab.

12
Euclidean Examples (4/10)
Content
 Great common divisor (GCD) – cont’d
• Compare two algorithms from small to large numbers
: Ex) GCD(5332, 24), GCD(568978, 1), etc.

• Think about one more algorithm via recursion

: Using the mod-based GCD concept !

void GCD_recursion(int a, int b)

{
if (b == 0)
return a;
else
return GCD_recursion(b, a%b);
}

Check the processing time of algorithms

Deep Computer Vision Lab.
13
Euclidean Examples (5/10)
Content
 Appendix : time checker
• Compute the processing time as follows :

#include <Windows.h>

LARGE_INTEGER freq, start, stop;

double diff;

QueryPerformanceFrequency(&freq); // computer frequency

QueryPerformanceCounter(&start); // starting point

Algorithm
Measuring
time

QueryPerformanceCounter(&stop); // stopping point

diff = (double)(stop.QuadPart - start.QuadPart)/ freq.QuadPart;

Deep Computer Vision Lab.

14
Euclidean Examples (6/10)
Content
 Practice : Compare GCD algorithms

Input two positive integers :

Minus method :
Modulus method :
Recursion method :

GCD value Processing

time

* Processing time can be estimated using “for loop”

ex) for (int k = 0; k < 1000; k++) {Your algorithm;}

Deep Computer Vision Lab.

15
Euclidean Examples (7/10)
Content
 Prime number (소수)
• Determine whether a given number is prime or not
: No positive divisors other than 1 and itself

is_prime(int N) void main()

{ {
int j; int N;

for( j=2; j<N; j++) printf(“%d” is “%s” prime,

{ N, is_prime(N) ? “ “ : “not”);
if(N%j == 0) }
return FALSE;
}
Efficiently print out results
return TRUE; without if conditions
}

Deep Computer Vision Lab.

16
Euclidean Examples (8/10)
Content
 Prime number (소수) – cont’d
• Determine whether a given number is prime or not
: How to efficiently reduce the search range ?

 Ex) 8 = 1x8, 2x4, 4x2, 8x1

(symmetric characteristic)
39 = 1x39, 3x13, 13x3, 39x1

Thus, the search range can be reduced to √N

for( j=2; j<=sqrt(N); j++)

{
int a = (int)sqrt(N); Check the processing time
if(a%j == 0)
return FALSE; when the large number is input !
}

return TRUE;

Deep Computer Vision Lab.

17
Euclidean Examples (9/10)
Content
 Prime number (소수) – cont’d
• Find all the prime numbers
: Simple algorithm – Eratosthenes’s sieve
 From 2 to the input number, remove the multiplier of it !

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Implement this algorithm using C programming

Deep Computer Vision Lab.

18
Euclidean Examples (10/10)
Content
 Prime number (소수) – cont’d
• Pseudo code

1. Input N (compute the prime number smaller than N)

2. Define array[N] and initialization as 0 (using “calloc”)
3. For i=2 to N-1,

3. 1. If array[i] != 0, go to Step 3
3. 2. For( j=i+i; j<=N; j+=i)
3. 3. Assign 1 to array[ j]
4. For i=2 to N, print out if array[i] = 0

* array[-] can be regarded as the flag array

 That is, if array[x] = 0, the x-th value is the prime number

Deep Computer Vision Lab.

19
Optimization
Content
 Optimization
• Select the optimal algorithm

• Consider the best way to use functions

: Overhead effect vs. size of the executable file

• Do your best to be “integer coding”

: It is very important skill in the industry field !

• Simplify statements in the loop (for, while, etc.)

• Minimize the recursive functions

: Be careful for the stack overflow

Deep Computer Vision Lab.

20
Summary
Content
 Algorithm : find the best way to the solution
 Consider the complexity to be optimized
• Algorithm complexity can be measured by Big-O notation

 Practice on Euclidean examples

Deep Computer Vision Lab.

21