Data Structure

Fall 2024
Instructor: Prof. John Yun
Lecture: 9-12P, Wednesdays: S2-401

Department of AI  Big Data

Woosong University
Grading Policy (VERY IMPORTANT!!)

Evaluation # of Each Weight Total Weight

Occurrence
Attendance 13 Lectures ~1.5% 20.0%
Homework 12 Weeks 0.87% 10.4%
Quizzes 2 4.8% 9.6%
Mid-term 1 16.0% 16.0%
Final Exam 1 28.0% 28.0%
Class Project 2 8.0% 16.0%

Total 100.0%

Two tardiness (10 mins. after class begins) will be counted as 1 absent.
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
Hanoi Tower Problem and
Mathematical Induction
T(n): Total number of ring moves for n number of rings in the
Hanoi Tower problem
T(n): 2n – 1

Proof: Mathematical Induction

• (Base Condition Satisfaction)
First number of move for 0 number of ring is 0.  T(0) = 20 – 1 = 0

• (Inductive Assumption)
Let’s assume that the number of required moves for moving k
number of rings  T(k) = 2k – 1

• (Inductive Analysis: It is assumed that the number of required

moves for moving k+1 rings is T(k+1) = 2k+1 -1
Hanoi Tower
To move k+1 rings, you need 2
moves to move k number of rings,
and 1 move to move the last ring.

• move(k+1, a, b, c)
move(k, a, c, b)
move the only ring on a to b
move(k, c, b, a)
Inductive Proof
Therefore,

T(k+1)= 2T(k) + 1
= 2(2k – 1) + 1
= (2ㆍ2k) - 2 + 1
= 2k+1 - 1
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
What is the Algorithm Execution Time?
Algorithm Execution Time
• How long it takes to process the input size n.
• Time to run a program
• Time to store n data into the storage
• Time to send n data through the network

• The input size n is mostly trivial.

• Sorting: # of elements to sort
• Index: # of elements contained in the index
Examples of Algorithm Execution Time
sample1(A[], n):
k ← n/2 Constant time
return A[k]

sample2(A[], n):
sum ← 0
for i ← 0 to n-1 Time proportional to n
sum ← sum + A[i]
return sum

sample3(A[], n):
sum ← 0
for i ← 0 to n-1
for j ← 0 to n-1 Time proportional to n2
sum ← sum + A[i] * A[j]
return sum
sample4(A[], n):
sum ← 0
for i ← 0 to n-1
for j ← 0 to n-1
k ← A[0...n-1] 에서 임의로
The biggest number n/2n/2개를
from among 뽑은picked
numbers 것들from
중A[0…n-1]
최댓값
sum ← sum + k
return sum
Time proportional to n3
sample5(A[], n):
sum ← 0
for i ← 0 to n-2 Time proportional to n2
for j ← i+1 to n-1
sum ← sum + A[i] * A[j]
return sum

factorial(n):
if (n = 1) return 1 Time proportional to n
return n* factorial(n- 1)
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
Asymptotic Complexity

In the case where the highest-order

term is quadratic, as n grows larger,
the quadratic term ultimately
dominates.
Growth rates of multiple functions and asymptotic concepts
Algorithmic Asymptotic Complexity Expression
• What is Asymptotic Complexity?
→ Complexity when the input size is sufficiently large.

• Intuitive Definition (input size: n)

• The algorithm takes at most time proportional to n2. → O(n2)

• The algorithm takes at least time proportional to n2. → Ω(n2)

• The algorithm always takes time proportional to n2. → Θ(n2)

Asymptotic Complexity

O- notation
O(n2): The set of all functions whose highest-order term has a degree
no greater than n2

Ω- notation
Ω(n2): The set of all functions whose highest-order term has a degree
no less than n2

Θ- notation
Θ(n2): The set of all functions whose highest-order term has a degree
of exactly n2
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
 O - notation
O(f (n)): Big-O
• All functions whose highest-order term has a degree less than or equal to f (n)
• Asymptotic Upper Bound
• Example:
O(n), O(n log n), O(n2), O(2n), …
O(n2) = {n2, 5n2 +7n+12, 100n2 -2n+9, n log n+5n, 3n+9, 150, … }
All functions with a highest-order term of degree at most 2
• It should be noted that 5n2 +7n+12 ϵ O(n2), but for convenience, we write it as
5n2 +7n+12 = O(n2)
 Ω - notation
Ω(f (n)): Big-Omega
• All functions whose highest-order term has a degree greater than or equal to f (n)
• Asymptotic Lower Bound
• Example:
Ω(n), Ω(n log n), Ω(n2), Ω(2n), …
n
Ω(n2) = {n2, 5n2 +7n+12, 100n2 -2n+9, n3+5n, 7n5+n3+9, 2 , n!, … }
All functions with a highest-order term of degree 2 or higher
• It should be noted that 5n2 +7n+12 ϵ Ω(n2), but for convenience, we write it as
5n3 +12 = Ω(n2)
 Θ - notation
Θ(f (n)): Big-Theta
• All functions whose highest-order term has a degree equal to f (n)
• Example:
Θ(n), Θ(n log n), Θ(n2), Θ(2n), …
Θ(n2) = {n2, 5n2 +7n+12, 100n2 -2n+9, … }
All functions whose highest-order term has a degree of 2
• It should be noted that 5n2 +12 ϵ Θ(n2) , but for convenience, we write it as 5n2
+12 = Θ(n2)

Θ(n2) = O(n2) ∩ Ω(n2)

Example of Notation Usage
sample4(A[], n):
sum ← 0
for i ← 0 to n-1
for j ← 0 to n-1
k ← A[0...n-1]
The biggest 에서 임의로
element amongn/2
n/2 개를 뽑은
number 것들 중
of elements 최댓값
random selected from A[0…n-1].
sum ← sum + k
return sum

The runtime of the function sample4() is Θ(n3).

 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
Mathematical Definition
O(g(n)) = { f(n) | ∃c > 0, n0 > 1 s.t. ∀n ≥ n0, f(n) ≤ cg(n) }

To explain what this means…

O(g(n)) = { f(n)| there exists a constant number c for all

large number n such that f(n) ≤ cg(n)

While a formal definition is necessary when mathematically proving asymptotic

complexity, the intuitive definition provided earlier is sufficient for the level of a
data structures course.
In asymptotic notation, = is used in place of ∈

If the runtime of an algorithm is T(n) = O(nlogn),

it would mean T(n) ∈ O(nlogn).

In other words, T(n) is a function that belongs to O(nlogn)

Therefore, the expression O(nlogn) = T(n) is not appropriate.

Note: Visual Meaning
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
Algorithm • Mathematical Definition
Performance • Application

Week 2 – Lecture 1
Fall 2024
Let’s try using Ο, Ω, Θ notations

alg1(A[], n): • It takes constant time regardless of n.

k ← n/2 • It is both Ω(1) and O(1)
return A[k]
• Θ(1): the most accurate expression

alg2(A[], n): • It always takes time proportional to n.

sum ← 0
for i ← 0 to n-1 • It is both Ω(n) and O(n).
sum ← sum + A[i]
return sum • Θ(n): the most accurate expression.
Let’s try using Ο, Ω, Θ notations
alg3(A[], n) : • In the worst case, it takes time proportional to n.
sum ← 0
if (n(n이is 홀수 ) number)
an odd
• In the best case, it takes constant time.
for i ← 0 to n-1 • It is Ω(1) and O(n).
sum ← sum + A[i]
• We say it is Θ(1) in the best case and Θ(n) in the
return sum
worst case.

alg4(A[], n):
sum ← 0 • It always takes time proportional to n2.
for i ← 0 to n-2
for j ← i+1 to n-1 • It is both Ω(n2) and O(n2).
sum ← sum + A[i] * A[j] • Θ(n2): the most accurate expression.
return sum
Example of complexity analysis
for a recursive algorithm After comparing with the
element in the middle of the
array, you encounter a problem
binarySearch (A[], x, low, high) : that is the same but with half
// A: array, x: search key, low, high: array bounds
if (low > high) return “Not found” the size.
mid ← (low + high)/2
if (A[mid] < x) return binarySearch(A, x, mid+1, high) In other words, T(n) = c + T(n/2)
else if (A[mid] > x) return binarySearch(A, x, low, mid-1)
else return mid By reducing the size by half in
this manner, you will encounter
a problem of size 1 in
Same problem but smaller in size approximately log2n steps.
binarySearch binarySearch
Since the work required to
halve the problem size is
constant time, it will take at
most time proportional to log2n.
binarySearch
Runtime:
• Worst Case: Θ(log n)
• Best Case: Θ(1)
• When stated without any qualifier, it is simply O(log n) 29
 • Mathematical Induction for
Hanoi Problem
• What is the Execution
Time?
• Algorithm Complexity
• Notation
• Mathematical Definition
Class Lab • Application
• Class Lab
Week 2 – Lecture 1
Fall 2024