CS 253

Data Structures and Algorithms

Lecture 3: Efficiency & Complexity
Spring 2025
Department of Computer Science,
Emory University
Joon-Seok Kim, PhD.
Recap
• Problems and instances • Basic Data Structures
• Algorithms • Arrays
• Definition • Lists
• Linked Lists
• Notation (natural language,
• Stacks
pseudocode, program code)
• Queues
• Properties
• Trees
• Generality
• Definitions
• Determinism
• Properties
• Termination
• Binary Trees
• (Partial) Correctness
• Traversals
• Efficiency

2
Tree Traversal
• Q: Which algorithm is efficient? class TreeNode<E> {
E value;
• Depth-first traversal TreeNode<E> left;
• Breadth-first traversal TreeNode<E> right;
}
• How would you measure the
efficiency of an algorithm?

* *

+ - + -

5 7 3 1 5 7 3 1
3
Efficiency of Algorithms
• Minimizing resources needed:
• Run-time (number of operations)
• Memory usage
• Number of accesses to a secondary storage (e.g., hard disk, cloud)
• Communication cost (e.g., network)

• Actual resource-usage of an algorithm depend on many factors:

• CPU clock rate (Number of operations the CPU can execute per second)
• Length of the input
• Implementation of operations (programming language dependent)

4
Complexity
• The complexity of an algorithm
• Is a function 𝑇(𝑛) that describes the resource cost depending on input 𝑛
• Usually considered asymptotically (for 𝑛→∞)
Problem 𝑻(𝒏)
Searching among a set of 𝑛 elements # comparisons,
# nodes (of a data structure) accessed
Sorting a list of 𝑛 elements # comparisons
# swaps (of value)
Execution of a recursive function 𝑓(𝑛) # functions calls
Finding all prime number up to 𝑛 # arithmetic operation
Matrix multiplication 𝑚×𝑛 ∗ (𝑛×𝑚) # scalar multiplications

5
Complexity Example: sumTo
• What is 𝑇(𝑛)? How many operations will be executed?
int sumTo(int n) {
int sum = 0; // 1 operation
for(int i = 1; i <= n; i++) // ? iterations
sum += i; // ? operations
return sum; // 1 operation
}
int sumTo(int n) {
int sum = 0; // 1 assignment operation
int i = 1; // 1 assignment operation
while(i <= n) { // (n+1) * 1 comparison operation
sum = sum + i; // n * 1 addition assignment operations
i++; // n * 1 increment operation
}
return sum; // 1 return operation
}
Do we need this detail? If so, when do we need it?
6
Order of Growth: sumTo
• Example: sumTo(int n)
• Declare and initialize variable sum
• Add 1 to sum
• Add 2 to sum
• ...
• Add n to sum
• Return sum
• Each operation takes constant time (e.g., 𝑐* , 𝑐+ ).
• The number of operations grows linearly with the input size 𝑛.
• When 𝑛 is doubled, the number of operations also doubles.
• sumTo: 𝑇 𝑛 = 𝑐!𝒏 + 𝑐"
7
Complexity Example: Matrix Operations
Matrix addition Algorithm AddMatrices(A, B, n):
Input: 𝐴 ∈ 𝑛×𝑛, 𝐵 ∈ 𝑛×𝑛 // A and B are n x n matrices
Create a new matrix C of size n x n
Algorithm:
Declare matrix C ∈ 𝑛×𝑛 // Iterate over each element in the matrices
For each element 𝑐"# : for i = 0 to n-1:
𝑐"# = 𝑎"# + 𝑏"# for j = 0 to n-1:
Return C C[i][j] = A[i][j] + B[i][j]
• Addition
• For 𝑛! matrix cells we have to return C
compute one addition
Ø Complexity for addition grows
quadratically in 𝑛

8
Complexity Example: Matrix Operations
Matrix multiplication Algorithm MultiplyMatrices(A, B, n):
Input: 𝐴 ∈ 𝑛×𝑛, 𝐵 ∈ 𝑛×𝑛 // A and B are n x n matrices
Create a new matrix C of size n x n
Algorithm:
Declare matrix C ∈ 𝑛×𝑛
For all 𝑐"# : // Iterate over each element in the matrices
𝑐"# = ∑'"$% 𝑎"( ⋅ 𝑏(#
for i = 0 to n-1:
Return C for j = 0 to n-1:
C[i][j] = 0
• Multiplication for k = 0 to n-1:
• For 𝑛! matrix cells we have to C[i][j] = C[i][j] + (A[i][k] * B[k][j])
compute 𝑛 additions and 𝑛
multiplication
Ø Complexity for multiplication return C
grows cubically in 𝑛

9
Asymptotic Efficiency of Algorithms
• How does the resource cost 𝑇(𝑛) scale for very large input 𝑛?
• sumTo: 𝑇 𝑛 = 𝑐!𝑛 + 𝑐"
• Matrix addition: 𝑇 𝑛 = 𝑐#𝑛" + 𝑐$
• Matrix multiplication: 𝑇 𝑛 = 𝑐%𝑛# + 𝑐&𝑛" + 𝑐'
• Assume that 𝑐* … 𝑐, are a constant we can obtain from simplification.
• Can we approximate the order of growth?
• Measure of complexity, independent of constant summands and
factors
• Disregards summands such as initialization
• Disregards factors such as slower/faster machines

10
Asymptotic Complexity
• Purpose
• Interested in running time of a function of 𝑛, where 𝑛 is input size of
algorithms no matter what the input (E.g., Sorting [1, 2, 3, 4] or [3, 4, 1, 2]).
• Abstract away some details of functions when 𝑛 is very large (like infinity).
• Find a simple function of similar order of growth.
• Example:
• 𝑇 𝑛 = 𝑐%𝑛# + 𝑐&𝑛" + 𝑐'
• What is the dominant term or highest-order term?
• 𝑐! 𝑛" + 𝑐# 𝑛$ + 𝑐%
• As 𝑛 grows large, the 𝑐%𝑛# term will come to dominate, so that all other
(lower-order) terms (i.e., 𝑐&𝑛" + 𝑐') can be neglected.

11
Asymptotic Complexity: O-Notation
• How can we define a function in such a way that we focus only on 𝑛! in
𝑇 𝑛 = 𝑐" 𝑛! + 𝑐# 𝑛$ + 𝑐% ?
• Can we write 𝑇 𝑛 ∈ 𝑂(𝑛! )?
• Consider the following examples
• 𝑇% 𝑛 = 3𝑛) + 5𝑛!
• 𝑇! 𝑛 = 2𝑛) + 7𝑛! + 𝑛
• 𝑇) 𝑛 = 𝑛) + 𝑛! + 𝑛 + 50
• …
• To achieve this, we will explore two equivalent definitions of the O-
Notation to bound the run-time complexity of algorithms:
• The classic mathematical definition, as typically found in textbooks, and
• A simplified definition using the limit notation (analytical approach).

12
Upper and Lower Bounds
• An upper or lower bound of a subset 𝑆 of a preordered set (𝐾, ≤) is an
element of 𝐾 that is greater than or equal to, or less than or equal to, every
element of 𝑆, respectively.
• Upper bound: An element 𝑢 ∈ 𝐾 is an upper bound for 𝑆 if 𝑢 ≥ 𝑠 for all 𝑠 ∈ 𝑆.
• Lower bound: An element 𝑙 ∈ 𝐾 is a lower bound for 𝑆 for 𝑙 ≤ 𝑠 for all 𝑠 ∈ 𝑆.
• Example (Set-based):
• Let 𝐾 = {1,2,3, … }
lower bounds upper bounds
• Let 𝑆 = {4,5,7}, a subset of 𝐾
• Any 𝑥 where 𝑥 ≤ 4 is a lower bound for 𝑆.
• Any 𝑥 where 𝑥 ≥ 7 is an upper bound for 𝑆. 2 3 4 5 6 7 8 9 10 11
• Similarily, for functions
• If 𝑇 𝑥 ≤ 𝑓 𝑥 for all 𝑥, then 𝑓 𝑥 is an upper bound for 𝑇 𝑥 , because 𝑓 𝑥 is
greater than or equal to 𝑇 𝑥 for all values of 𝑥.

13
O-Notation: Limit Definition
• Let 𝑔 𝑛 = 𝑛/ and 𝑓: ℕ → ℝ0 .
• E.g., 𝑓! 𝑛 = 3𝑛# + 5𝑛", 𝑓" 𝑛 = 7𝑛" + 𝑛, 𝑓# 𝑛 = 𝑛 + 50
• Pop Quiz: What is the blank?
• Hint: 𝑔(𝑛) is an asymptotic upper bound for 𝑓(𝑛).

0 𝑓 𝑛
𝑂 𝑔 𝑛 = 𝑓: ℕ → ℝ lim <∞
1→3 𝑔 𝑛

+ ( + ( + (
1. lim =0 2. lim , ( =1 3. lim >0
(→* , ( (→* (→* , (

+ ( + ( + (
4. lim <∞ 5. lim =∞ 6. 0 < lim <∞
(→* , ( (→* , ( (→* , (
14
Ω-Notation: Limit Definition
• Let 𝑔 𝑛 = 𝑛 and 𝑓: ℕ → ℝ0 .
• E.g., 𝑓! 𝑛 = 3𝑛# + 5𝑛", 𝑓" 𝑛 = 7𝑛" + 𝑛, 𝑓# 𝑛 = 𝑛 + 50
• Pop Quiz: What is the blank?
• Hint: 𝑔(𝑛) is an asymptotic lower bound for 𝑓(𝑛).

0 𝑓 𝑛
Ω 𝑔 𝑛 = 𝑓: ℕ → ℝ lim >0
1→3 𝑔 𝑛

+ ( + ( + (
1. lim =0 2. lim , ( =1 3. lim >0
(→* , ( (→* (→* , (

+ ( + ( + (
4. lim <∞ 5. lim =∞ 6. 0 < lim <∞
(→* , ( (→* , ( (→* , (
15
Θ-Notation: Limit Definition
• Let 𝑔 𝑛 = 𝑛/ and 𝑓: ℕ → ℝ0 .
• E.g., 𝑓! 𝑛 = 3𝑛# + 5𝑛", 𝑓" 𝑛 = 2𝑛# + 7𝑛" + 𝑛, 𝑓# 𝑛 = 𝑛# + 𝑛" + 𝑛 + 50
• Pop Quiz: What is the blank?
• Hint: 𝑔(𝑛) is an asymptotically tight bound for 𝑓(𝑛).

0 𝑓 𝑛
Θ 𝑔 𝑛 = 𝑓: ℕ → ℝ 0 < lim <∞
1→3 𝑔 𝑛

+ ( + ( + (
1. lim =0 2. lim , ( =1 3. lim >0
(→* , ( (→* (→* , (

+ ( + ( + (
4. lim <∞ 5. lim =∞ 6. 0 < lim <∞
(→* , ( (→* , ( (→* , (
16
O-Notation: Limit Definition
Limit definition:
0 𝑓 𝑛
𝑂 𝑔 𝑛 = 𝑓: ℕ → ℝ lim <∞
1→3 𝑔 𝑛
We say:
• 𝑔 𝑛 is an asymptotic upper bound for 𝑓 𝑛 .
• 𝑓 𝑛 grows at most as fast as 𝑔 𝑛 .
• 𝑓 𝑛 is in 𝑂 𝑔 𝑛 .
•𝑓 𝑛 ∈𝑂 𝑔 𝑛 .
• In some textbooks, 𝑓 𝑛 = 𝑂 𝑔 𝑛 .

17
Example: Complexity of sumTo(n)
• What is the complexity of the sumTo function?
int sumTo(int n) {
int sum = 0; // 1 operation
for(int i = 1; i <= n; i++) // n iterations
sum += i; // 1 operation
return sum; // 1 operation
}
• Total number of operations:𝑓 𝑛 = 1 + 𝑛 + 1 ⋅ 1 + 1 = 𝑛 + 3
• Using the limit definition of O-notation
• 𝑔 𝑛 =𝑛
𝑓(𝑛) 𝑛+3
lim = lim =1
'→+ 𝑔(ℎ) '→+ 𝑛

• Since 1 < ∞: 𝑓 𝑛 ∈ 𝑂 𝑔 𝑛 = 𝑂(𝑛)

18
O-Notation: Example
Example: 𝑓 𝑛 = 31 , 𝑔 𝑛 = 21 ⇒ 𝑓 𝑛 ∈ 𝑂 𝑔 𝑛 , is it true?
4 1 /! / 1
• lim = lim = lim =∞
1→3 5 1 1→3 +! 1→3 +

0 4 1
ØContradiction to definition: 𝑂 𝑔 = {𝑓: ℕ → ℝ | lim < ∞}
1→3 5 1

∴𝑓 𝑛 ∉𝑂 𝑔 𝑛

19
O-Notation: Properties
• Constants
𝑓 ∈ 𝑂(1)
• Scalar Multiplication
𝑎⋅𝑓 ∈𝑂 𝑔
• Addition
𝑓* + 𝑓+ ∈ 𝑂(max(𝑔* , 𝑔+ ))
• Multiplication
𝑓* ⋅ 𝑓+ ∈ 𝑂(𝑔* ⋅ 𝑔+ )

20
O-Notation: Constants
O-Notation: Constants
If 𝑓 𝑥 = 𝑎 ∈ ℝ is a constant function, then 𝑓 ∈ 𝑂(1)

Examples:
•3∈𝑂 1
• 1000000 ∈ 𝑂 1
•𝑂 42 = 𝑂 1 = 𝑂(𝜋)

21
O-Notation: Scalar Multiplication
O-Notation: Scalar Multiplication
Let 𝑓 ∈ 𝑂 𝑔 and 𝑎 ∈ ℝ, then:
𝑎⋅𝑓 ∈𝑂 𝑔

Examples:
• 1000𝑛 ∈ 𝑂 𝑛
• 23𝑛6 ∈ 𝑂 23𝑛6 = 𝑂(𝑛6 )
• 2107 = 21 ⋅ 27 ∈ 𝑂(21 )

22
O-Notation: Addition
O-Notation: Addition
Let 𝑓* ∈ 𝑂 𝑔* and 𝑓+ ∈ 𝑂(𝑔+ ), then:
𝑓* + 𝑓+ ∈ 𝑂(max(𝑔* , 𝑔+ ))

Examples:
• 2𝑛6 + 25𝑛/ + 8𝑛+ + 42𝑛 + 8 ∈ 𝑂(𝑛6 )
• log 𝑛 + 𝑛 ∈ 𝑂(𝑛)

23
O-Notation: Multiplication
O-Notation: Multiplication
Let 𝑓* ∈ 𝑂 𝑔* and 𝑓+ ∈ 𝑂(𝑔+ ), then:
𝑓* ⋅ 𝑓+ ∈ 𝑂(𝑔* ⋅ 𝑔+ )

Examples:
• log 𝑛 + 7𝑛 + 𝑛+ ⋅ 𝑛 ∈ 𝑂 𝑛+ 𝑛 = 𝑂(𝑛+.6 )

24
 Orders of Common Functions
Order Description Example N=10 N=100 N=1000
𝑂(1) constant A single operation 1 1 1
𝑂(log 𝑛) logarithmic Binary search 4 7 11
𝑂(𝑛) linear sequential search 10 100 1000
𝑂(𝑛 ⋅ log ! 𝑛 ) log-linear sorting of an array 40 700 11000
quasi-linear
𝑂(𝑛" ) quadratic matrix addition 100 10000 1000000
𝑂(𝑛# ) cubic matrix multiplication 1000 1000000 1000000000
𝑂(𝑛! ) polynomial
𝑂(2$ ) exponential combinations 1024 10#% 10#%&
𝑂(𝑛!) factorial permutations 10' 10&() 10"(*)
𝑂(𝑛$ ) 10&% 10"%% 10#%%%
*Let n be the size of the input Note: The number of atoms in the entire universe is estimated at 10)"

25
Relationship between Classes of Complexity
• Complexity defines an order on real-valued functions:
𝑂 1 ⊂ 𝑂 log 𝑛 ⊂ 𝑂 𝑛 ⊂ 𝑂 𝑛 log 𝑛 ⊂ 𝑂 𝑛+ ⊂ 𝑂(𝑛/ )

• In particular: Order of polynomial functions by their exponent:

∀𝑎 ≤ 𝑏: 𝑂 𝑛7 ⊂ 𝑂(𝑛9 )
• Exponential functions grow superpolynomial:
∀𝑎, 𝑏 > 1: 𝑂(𝑛7 ) ⊂ 𝑂 𝑏 1
• Logarithms grow slower than any polynomial:
∀𝑎 ≥ 1: 𝑂 log 𝑛 ⊂ 𝑂(𝑛7 )

26
Asymptotically Tight Bound
• The O-Notation provides an upper-bound of the complexity of a
function.
• For example: If we say that the complexity of an algorithm is in 𝑂 𝑛" , then
the algorithm may yet lie in 𝑂 𝑛 since 𝑂 𝑛 ⊂ 𝑂(𝑛")
• So 𝑂(𝑛") can be interpreted a “no worse (but possibly better) than quadratic”
• When we use the O-Notation to describe the complexity of an
algorithm, what we often (implicitly) mean is the tightest complexity
class that an algorithm lies in.
• For example: If I’d ask you: “What is the complexity class of the function 𝑛" +
3𝑛 + 5?” you could respond “𝑂(𝑛&:)” and that’d be correct.
• Because 𝑛" + 3𝑛 + 5 ∈ 𝑂 𝑛" ⊂ 𝑂(𝑛*+ )
• But if I’d ask you, what is the tightest complexity class, then the only correct
answer would be 𝑂 𝑛"

27
Big Theta: Θ-Notation
• Formally: The Θ(𝑛) (“big Theta”) notation is used to denote an
asymptotic tight bound.
• In most cases, when computer scientist says, “Big Oh”, they usually mean “Big
Theta”.

Limit definition:
0 𝑓 𝑛
Θ 𝑔 𝑛 = 𝑓: ℕ → ℝ 0 < lim <∞
1→3 𝑔 𝑛

Unless specified otherwise, we will also use 𝑂(𝑔) to refer to Θ(𝑔)

28
Applied to Iterative Code
• Let’s apply our O-Notation to code.
• Elementary operations are in 𝑂 1
• Variable declaration: String name;
• Initialization and variable assignment: int i = j + 3;
• Comparison: if (name == “bob”) ...
• Sequences of operations are added

• Loops are multiplied

• If the number of iterations of a loop is bound by 𝑂(𝑓) and the body of the
loop has a complexity of 𝑂 𝑔 , then the complexity of the loop is in 𝑂(𝑓𝑔)

29
Applied to Iterative Code
boolean containsDuplicates(int[] values) {
for(int i = 0; i < values.length; i++) {
for (int j = 0; j < values.length; j++) {
if (i == j) { continue; }
if (values[i] == values[j]) { return true; }
}
}
return false;
}

• Let 𝑛 be the length of input array values.

• The first loop is iterated 𝑛 ∈ 𝑂 𝑛 times.
• Total complexity of this loop is 𝑂 𝑛 ⋅ 𝑂(𝑏𝑜𝑑𝑦1) where 𝑏𝑜𝑑𝑦1 is the
complexity of the body the first loop.

30
Applied to Iterative Code
boolean containsDuplicates(int[] values) {
for(int i = 0; i < values.length; i++) {
for (int j = 0; j < values.length; j++) {
if (i == j) { continue; }
if (values[i] == values[j]) { return true; }
}
}
return false;
}

• In the body of the first loop, we have a second loop which is iterated
𝑛 ∈ 𝑂 𝑛 time.
• Total complexity of 𝑏𝑜𝑑𝑦1 is in 𝑂 𝑛 ⋅ 𝑂(𝑏𝑜𝑑𝑦2) where 𝑏𝑜𝑑𝑦2 is the
body of the second loop.

31
Applied to Iterative Code
boolean containsDuplicates(int[] values) {
for(int i = 0; i < values.length; i++) {
for (int j = 0; j < values.length; j++) {
if (i == j) { continue; }
if (values[i] == values[j]) { return true; }
}
}
return false;
}

• The body the second loop has two instructions, which are both in
𝑂(1).
• 𝑂 1 + 𝑂 1 = 𝑂(1)
• So, we have: 𝑏𝑜𝑑𝑦2 ∈ 𝑂 1

32
Applied to Iterative Code
boolean containsDuplicates(int[] values) {
for(int i = 0; i < values.length; i++) {
for (int j = 0; j < values.length; j++) {
if (i == j) { continue; }
if (values[i] == values[j]) { return true; }
}
}
return false;
}

• 𝑏𝑜𝑑𝑦1 ∈ 𝑂 𝑛 ⋅ 𝑂 𝑏𝑜𝑑𝑦2 = 𝑂 𝑛 ⋅ 𝑂(1) = 𝑂 𝑛

• Total complexity of the loop: 𝑂 𝑛 ⋅ 𝑂 𝑏𝑜𝑑𝑦1 = 𝑂 𝑛 ⋅ 𝑂 𝑛 =
𝑂(𝑛+ )

33
Example (What is “n”)
Quiz: What is the complexity of this algorithm? int myAddition(int summand1, summand2) {
A. O 1 int sum = summand1;
B. O(n) for(int i = 1; i <= summand2; i++)
C. O(n!) sum++;
D. None of the above return sum;
}

34
Example (What is “n”)
Quiz: What is the complexity of this algorithm? int myAddition(int summand1, summand2) {
A. O 1 int sum = summand1;
B. O(n) for(int i = 1; i <= summand2; i++)
C. O(n!) sum++;
D. None of the above return sum;
}
• Variable n is not defined here!
• 𝑂(𝑛) would mean that “The cost of this algorithm is at most linear in n.” But
what is n? There is no n here!
• In textbooks or other classes, you may have seen variables n and x used.
• Variables must be defined somewhere. For example, for sorting algorithms, 𝑛
is usually defined as the length of the input array.
• Correct Answer here: 𝑂(summand2)
• Alternative answer: “𝑂(𝑛), where 𝑛 is the value of summand2 ”

35
Applied to Recursive Code
• For recursive functions (using functions calls from the function to itself) instead
of iteration (loops), the complexity can be computed using the Master Theorem
for divide-and-conquer recurrences.
• Details omitted here.

36
For More Advanced Topics
• CS 326 - Analysis of Algorithms
• Description of CS 326
• This course explores the formal underpinnings of computational complexity,
and studies how to mathematically characterize the efficiency and running
times of different computer algorithms.

37
Complexity of an Algorithm vs.
Complexity of a Problem
• We will mostly use the O-Notation to describe the complexity of different
algorithms.
• For example: BubbleSort ∈ 𝑂 𝑛" , MergeSort ∈ 𝑂(𝑛 log 𝑛)
• The O-Notation can also be used to describe the complexity of a problem.
• Sorting ∈ 𝑂(𝑛 log 𝑛)
• Complexity of a problem is the tightest complexity of any algorithm to solve
the problem.
• More formally: Sorting ⊆ Ω(𝑛 log 𝑛)
• Ω(𝑛 log 𝑛) is a lower bound like O(𝑛 log 𝑛) in an upper bound
• We say: “Sorting is a 𝑛 log 𝑛 hard problem”
• For some problems, their complexity may not be known
• There might be yet unknown algorithms to solve problems faster
• For other problems, a lower bound can be theoretically proven

40
 Upper Bound for sumTo
• sumTo: 𝑇 𝑥 = 𝑎𝑥 + 𝑏
• Can you find any function 𝑇′(𝑥)
that is an upper bound for a 𝑇′ 𝑥 = 𝑐 ⋅ 𝑓(𝑥)
function sumTo? 𝑡
𝑇 𝑥 = 𝑎𝑥 + 𝑏
• i.e., 𝑇 𝑥 ≤ 𝑇′ 𝑥
• E.g., 𝑇 5 (𝑥) = 𝑐 ⋅ 𝑓 𝑥 , 𝑓(𝑥) = 𝑥
• where 𝑓 𝑥 = 𝑥+𝑑, 𝑐 > 𝑎
• How about 𝑇′(𝑥) with the
following conditions?
• After some point (𝑥6 ), 𝑇 𝑥 ≤ 𝑇′ 𝑥 .
• E.g., 𝑇′ 𝑥 = 𝑐 ⋅ 𝑓 𝑥 𝑏
"
• where 𝑓 𝑥 = 𝑥, 𝑐 > 𝑎, 𝑥! =
#$%
𝑥% 𝑥

41
Yellow-framed slides are not part of exams. So, don’t worry too much about it.
Asymptotic Complexity: O-Notation
Formal O-Notation:
𝑂 𝑔 𝑥 = 𝑓: ℝ → ℝ ∃𝑐 > 0 ∃𝑥? > 0 ∀ 𝑥 ≥ 𝑥? : 𝑓 𝑥 ≤ 𝑐 ⋅ |𝑔(𝑥)|}

In computer science we mostly have discrete inputs and non-negative cost functions:
𝑂 𝑔 𝑛 = 𝑓: ℕ → ℝ0 ∃𝑐 > 0 ∃𝑛? > 0 ∀ 𝑛 ≥ 𝑛? : 𝑓 𝑛 ≤ 𝑐 ⋅ 𝑔(𝑛)}

42
O-Notation: Visualization
𝑂 𝑔 𝑛 = 𝑓: ℕ → ℝ0 ∃𝒄 > 𝟎 ∃𝑛? > 0 ∀ 𝑛 ≥ 𝑛? : 𝑓(𝑛) ≤ 𝒄 ⋅ 𝑔(𝑛)}

Example: 𝑓 𝑛 = 2𝑛, 𝑔 𝑛 = 𝑛 ⇒ 𝑓 𝑛 ∈ 𝑂(𝑛)

• For 𝑐 = 3, 𝑛? = 0: 𝑓 𝑛 = 2𝑛 ≤ 3𝑛 = 𝑐 ⋅ 𝑔 𝑛

• Omits constant factors

43
O-Notation: Visualization
𝑂 𝑔 𝑛 = 𝑓: ℕ → ℝ0 ∃𝑐 > 0 ∃𝒏𝟎 > 𝟎 ∀ 𝒏 ≥ 𝒏𝟎 : 𝑓(𝑛) ≤ 𝑐 ⋅ 𝑔(𝑛)}

Example: 𝑓 𝑛 = 2𝑛 + 30, 𝑔 𝑛 = 𝑛+ ⇒ 𝑓 𝑛 ∈ 𝑂(𝑛+ )

• For 𝑐 = 1, 𝑛? = 7: 𝑓 𝑛 ≤ 1 ⋅ 𝑓 𝑛 for all 𝑛 > 𝑛?

• For small input, the runtime of 𝑔(𝑛)

may be lower than that of 𝑓(𝑛)
• But asymptotically, 𝑓(𝑛) grows slower

44
Example: Complexity of sumTo(n)
• What is the complexity of the sumTo function?
int sumTo(int n) {
int sum = 0; // 1 operation
for(int i = 1; i <= n; i++) // n iterations
sum += i; // 1 operation
return sum; // 1 operation
}
• Total number of operations:𝑓 𝑛 = 1 + 𝑛 + 1 ⋅ 1 + 1 = 𝑛 + 3
• Using the classical definition of O-notation
• 𝑓 𝑛 ≤ 𝑐 ⋅ 𝑔 𝑛 for 𝑔 𝑛 = 𝑛, 𝑐 = 2 , 𝑛 ≥ 𝑛6 = 3
Ø𝑓 ∈ 𝑂(𝒏)
• Infinite number of alternatives:
%
• 𝑓 𝑛 ≤ 𝑐 ⋅ 𝑔 𝑛 for 𝑔 𝑛 = 𝑛, 𝑐 = 2000, 𝑛 ≥ 𝑛6 =
766

45
O-Notation: Two Variables
• We can extend our notation to the case of two parameters 𝑛 and 𝑚
that can go to infinity independently at different rates. For a given
function 𝑓 𝑛, 𝑚 , we denote by 𝑂 𝑓 𝑛, 𝑚 the set of functions

𝑂 𝑓 𝑛, 𝑚 = 𝑔: ℕ → ℝ0 ∃𝑐 > 0 ∃𝑛? > 0∃𝑚? > 0∀ 𝑛 ≥ 𝑛? ∀ 𝑚

≥ 𝑚? : 𝑔(𝑛, 𝑚) ≤ 𝑐 ⋅ 𝑓(𝑛, 𝑚)}

46
Quiz: Other Notations
+ (
𝑓 𝑛 =𝑜 𝑔 𝑛 : : lim , ( =0
(→*

+ (
𝑓 𝑛 =𝑂 𝑔 𝑛 : : lim =1
(→* , (

+ (
𝑓 𝑛 =Θ 𝑔 𝑛 : : lim >0
(→* , (

+ (
𝑓 𝑛 ~ 𝑔 𝑛 : : lim <∞
(→* , (

+ (
𝑓 𝑛 =𝜔 𝑔 𝑛 : : lim =∞
(→* , (

+ (
𝑓 𝑛 =Ω 𝑔 𝑛 : :0< lim <∞
(→* , (
50
Quiz: Other Notations
+ (
𝑓 𝑛 = 𝑜 𝑔 𝑛 : Small O; Little O; : lim , ( =0
(→*

+ (
𝑓 𝑛 = 𝑂 𝑔 𝑛 : Big O : lim <∞
(→* , (

+ (
𝑓 𝑛 = Θ 𝑔 𝑛 : Big Theta :0< lim <∞
(→* , (

+ (
𝑓 𝑛 ~ 𝑔 𝑛 : Asymptotic equivalence : lim =1
(→* , (

+ (
𝑓 𝑛 = 𝜔 𝑔 𝑛 : Small Omega; Little Omega; : lim , ( =∞
(→*
+ (
𝑓 𝑛 = Ω 𝑔 𝑛 : Big Omega : lim , ( >0
(→*
51
Summary: Complexity
• Measuring Complexity
• Need a measure that is independent of machine and programming language
• O-Notation
• Asymptotic Notation (for large input)
• Formal Definition
• Limit Definition
• O-Notation: Properties
• Constants, Scalar Multiplication, Addition, Multiplication
• Common Complexity Classes
• Application to Programs
• Iterative Programs: “Loop Counting”
• Recursive Programs: Master’s Theorem (details omitted)

52