Solutions to Big Oh Tutorial

IMPORTANT - Each question in this notebook provides both annotated solutions

of the proofs and a visualisation for sanity checking the values of and makec n0

sense. The important part is the proof and methodology to reach the solution.
You will not have access to the plots in your exams and in-class tests.
Furthermore the plots themselves are not proofs.
Note: you need to run the below two code cells before you can use any of the plots to
sanity check your answer(s).
In [1]: import math
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

In [2]: def plot_oh(f, g, c, n_0, fn, cgn, min_x, max_x):

xs = np.arange(min_x,max_x,1, dtype='int64')
fn_ys = f(xs)
gn_ys = c*g(xs)
plt.axvline(x = n_0, color = 'k', linestyle='--')
plt.scatter(x=xs, y=fn_ys, marker='x')
plt.scatter(x=xs, y=gn_ys, marker='.')
plt.xlabel('n')
plt.ylabel('f(n)')
plt.legend(['n_0 = '+str(n_0),'f(n): '+fn, 'cg(n):'+cgn])

Big-Oh Definition
Given positive functions and , we can say that
f (n) g(n) f (n) is O(g(n)) if and only
if there exists positive constants and such that
c n0 f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
Q1. Prove that is 5 O(1)

Solution
Picking apart the definition, we have that:
f (n) = 5

g(n) = 1

We need to show that there exists positive constants and that satisfies:
c n0

5 ≤ c ⋅ 1, ∀n ≥ n0

This is trivial since we just need to choose a value of c ≥ 5 and any positive value for
n0. This gives us:
5 ≤ 5 ⋅ 1, ∀n ≥ 1
 Notice that we chose a value for despite our inequality not depending on . This is
n0 n

required from the definition:∃c ≥ 0, ∃n0 ≥ 0 ...

This reduces to the following inequality which is trivially true:
5 ≤ 5, ∀n ≥ 1

∴ 5 is O(1)using c = 5 and $n_0=1

Sanity check
In [3]: c = 5
n_0 = 1
f = lambda n: 0*n+5
g = lambda n: 0*n+1

plot_oh(f, g, c, n_0, '$5$', str(c)+'$\cdot 1$', 0, 50)

#plt.rcParams["figure.figsize"] = (4,4)

Q2. Prove that 2n + 1 is O(3n)

Solution
To recall the definition of Big-Oh: Given positive functions and , we can say
f (n) g(n)

that f (n) is O(g(n)) if and only if there exists positive constants and such that
c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
As before, to prove that 2n + 1 is O(3n) , we need to substitute the definition which
gives:
 2n + 1 is O(3n) if and only if there exists positive constants c and n0 such that 2

If we choose c = 1, we are left to show that there exists an such that

n > 0

2n + 1 ≤ 3n, ∀n ≥ n0 .
But first we can simplify the equality by subtracting from both sides to give
2n

1 ≤ n, ∀n ≥ n0 .
We can trivially pick n0 = 1 to satisfy the equation
1 ≤ n, ∀n ≥ 1

Step-wise:
2n + 1 is O(3n) if and only if there exists positive constants c and n0 such that 2

choosing c = 1 gives
2n + 1 ≤ 3n, ∀n ≥ n0

simplifying by subtracting gives

2n 1 ≤ n, ∀n ≥ n0

pickingn0 = 1 gives1 ≤ n, ∀n ≥ 1

1 ≤ n, ∀n ≥ 1 is trivially true
∴ 2n + 1 is O(3n) usingc = 1 and
n0 = 1

Sanity check
In [4]: c = 1
n_0 = 1
f = lambda n: 2*n+1
g = lambda n: 3*n

plot_oh(f, g, c, n_0, '$2n+1$', str(c)+'$\cdot 3n$', 0, 50)

#plt.rcParams["figure.figsize"] = (4,4)
Basic Questions
Q3. Prove that is 4 O(2)

Solution
To recall the definition of Big-Oh: Given positive functions and , we can say
f (n) g(n)

that f (n) is O(g(n)) if and only if there exists positive constants and such that
c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
To prove that is , we need to show that there exists positive constants and
4 O(2) c

n0 such that
4 ≤ c ⋅ 2, ∀n ≥ n0

Again this one is trivial, pick anyc ≥ 2 and any value for . Note however that when
n0

doing proofs, we should pick sensible values for and . Yes,

c n0

n0 = 258972570257249805764 would work but is bad style.

Choosing c = 2 and n0 = 1 , we need to show that:
4 ≤ 2 ⋅ 2, ∀n ≥ 1

4 ≤ 4, ∀n ≥ 1 which is trivially true.

∴ 4 is O(2) using and
c = 2 n0 = 1 .
 Sanity check
In [5]: c = 2
n_0 = 1
f = lambda n: 0*n+4
g = lambda n: 0*n+2

plot_oh(f, g, c, n_0, '$4$', str(c)+'$\cdot 2$', 0, 50)

#plt.rcParams["figure.figsize"] = (4,4)

Q4. Prove that 2n + 1 is O(n)

Solution(s)
To recall the definition of Big-Oh: Given positive functions and , we can say
f (n) g(n)

that f (n) is O(g(n)) if and only if there exists positive constants and such that
c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
We can try the same as we did before, but we find that we are using a trial-and-error
process to find a value of that works. Another approach is to attempt to "derive"
c

the value for .

c

2n + 1 is O(n) if and only if there exists positive constants c and n0 such that 2n

We can first try and simplify the inequality by subtracting to give:

2n

1 ≤ cn − 2n, ∀n ≥ n0
 which further simplifies to:
1 ≤ (c − 2)n, ∀n ≥ n0

It is more obvious to us now what value of should be chosen as this would never
c

work for any value of . We could choose

c ≤ 2 c = 2.01 but then we need to work out
the value for . We can avoid complicating things by choosing .
n0 c = 3

choosingc = 3 gives1 ≤ (3 − 2)n, ∀n ≥ n0

which simplifies to1 ≤ n, ∀n ≥ n0

From this we can see that n0 ≥ 1

which gives 1 ≤ n, ∀n ≥ 1 which is trivially true

∴ 2n + 1 is O(n) usingc = 3 and n0 = 1 .
We are allowed to use fractional values of . Below is an example of the solution
c

using c = 2.01 .
choosingc = 2.01 gives1 ≤ (2.01 − 2)n, ∀n ≥ n0

which simplifies to1 ≤ 0.01n, ∀n ≥ n0

multiplying by 100 to remove fractional gives

n 100 ≤ n, ∀n ≥ n0

From this we can see that n0 ≥ 100

which gives 100 ≤ n, ∀n ≥ 100 which is trivially true

∴ 2n + 1 is O(n) usingc = 2.01 and n0 = 100 .
Sanity check using c = 3

In [6]: c = 3
n_0 = 1
f = lambda n: 2*n+1
g = lambda n: n

plot_oh(f, g, c, n_0, '$2n+1$', str(c)+'$\cdot n$', 0,100)

 Sanity check using c = 2.01

In [7]: c = 2.01
n_0 = 100
f = lambda n: 2*n+1
g = lambda n: n

plot_oh(f, g, c, n_0, '$2n+1$', str(c)+'$\cdot n$', 90, 110)

plt.rcParams["figure.figsize"] = (8,8)
Medium Difficulty Questions
To recall the definition of Big-Oh: Given positive functions and , we can say f (n) g(n)

that f (n) is O(g(n)) if and only if there exists positive constants and such that c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
Q5. Prove that is n
2
O(2n )
2

Solution
We can say that n
2
is O(2n )
2
if and only if there exists positive constants and
c n0

such that n
2 2
≤ c ⋅ 2n , ∀n ≥ n0 .
2 2
n ≤ c ⋅ 2n , ∀n ≥ n0

1 ≤ c ⋅ 2, ∀n ≥ n0(divide by ) n
2

0.5 ≤ c, ∀n ≥ n0 (divide by 2)
Here we can choose any not less than 0.5. Here we will choose
c but suggest c = 1

you try the proof on your own with ; what differs (if anything).
c = 0.5

0.5 ≤ 1, ∀n ≥ n0 (choose ) c = 1

0.5 ≤ 1, ∀n ≥ 1 (choose ) n0 = 1

∴ n
2
using
is O(2n )
2
and .
c = 1 n0 = 1

Sanity check
In [8]: c = 1
n_0 = 1
f = lambda n: (n**2)
g = lambda n: (2*(n**2))

plot_oh(f, g, c, n_0, '$n^2$', str(c)+'$\cdot 2n^2$', 1, 10)

#plt.rcParams["figure.figsize"] = (4,4)

Q6. Prove that n

2
− 3 is O(n )
2

Solution
We can say that n
2
− 3 is O(n )
2
if and only if there exists positive constants and
c

n0such that n
2
− 3 ≤ c ⋅ n , ∀n ≥ n0
2
.
2 2
n − 3 ≤ c ⋅ n , ∀n ≥ n0
2 2
n − c ⋅ n ≤ 3, ∀n ≥ n0

2
n (1 − c) ≤ 3, ∀n ≥ n0

Need term to be zero (or negative) to satisfy the inequality.

n
2

2
(choose )
n (1 − 1) ≤ 3, ∀n ≥ n0 c = 1
 0 ≤ 3, ∀n ≥ n0

0 ≤ 3, ∀n ≥ 1 (choose )
n0 = 1

∴ n
2
− 3 is O(n )
2
using and
c = 1 n0 = 1

Sanity check
In [9]: c = 1
n_0 = 1
f = lambda n: (n**2 - 3)
g = lambda n: (n**2)

plot_oh(f, g, c, n_0, '$n^2-3$', str(c)+'$\cdot n^2$', 1, 10)

# plt.rcParams["figure.figsize"] = (8,8)

Q7. Prove that n

2
− 5n is O(n )
2

Solution
We can say that n
2
− 5n is O(n )
2
if and only if there exists positive constants and
c

n0such that n
2 2
− 5n ≤ c ⋅ n , ∀n ≥ n0 .
 2 2
n − 5n ≤ c ⋅ n , ∀n ≥ n0

0 ≤ c ⋅ n
2
− n
2
(rearranging)
+ 5n, ∀n ≥ n0

0 ≤ n(c ⋅ n − n + 5), ∀n ≥ n0 (factorising)

0 ≤ n(n(c − 1) + 5), ∀n ≥ n0 (factorising)
0 ≤ 5n, ∀n ≥ n0 (choose ) c = 1

Which is trivially true for ; hence,

n ≥ 0.2 . n0 = 1

0 ≤ 5, ∀n ≥ 1 (choose ) n0 = 1

∴ n
2
− 5n is O(n ) using
2
and c = 1 n0 = 1

Sanity check
In [10]: c = 1
n_0 = 1
f = lambda n: (n**2 - 5*n)
g = lambda n: (n**2)

plot_oh(f, g, c, n_0, '$n^2-5n$', str(c) + '$ \cdot n^2$', 1, 10)

#plt.rcParams["figure.figsize"] = (8,8)
 Q8. Prove that n
2
+ 1 is O(n )
2

Solution
We can say that n
2
+ 1 is O(n )
2
if and only if there exists positive constants and
c

n0such that n
2
+ 1 ≤ c ⋅ n , ∀n ≥ n0
2
.
2 2
n + 1 ≤ c ⋅ n , ∀n ≥ n0

1 ≤ (c − 1) ⋅ n , ∀n ≥ n0 (subtract )
2
n
2

c must be greater than 1 for the inequality to be true!

1 ≤ (2 − 1) ⋅ n , ∀n ≥ n0 (choose )
2
c = 2
2
1 ≤ n , ∀n ≥ n0 (simplification)
2
1 ≤ n , ∀n ≥ 1 (pick ) n0 = 1

∴ n
2
+ 1 is O(n )using 2
and .
c = 2 n0 = 1

Sanity check
In [11]: c = 2
n_0 = 1
f = lambda n: (n**2 + 1)
g = lambda n: (n**2)

plot_oh(f, g, c, n_0, '$n^2+1$', str(c)+'$\cdot n^2$', 1, 10)

#plt.rcParams["figure.figsize"] = (8,8)
Conceptually
(Answers) Challenging Questions
To recall the definition of Big-Oh: Given positive functions and , we can say
f (n) g(n)

thatf (n) is O(g(n)) if and only if there exists positive constants and such that
c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
Q9. From the definitions, prove or disprove that is 1 O(n)

Solution
Provably true:
1 ≤ c ⋅ n, ∀n ≥ n0

1 ≤ n, ∀n ≥ n0 (choose )
c = 1

1 ≤ n, ∀n ≥ 1 (pick )
n0 = 1

∴ 1 is O(n)

Sanity check
In [12]: c = 1
n_0 = 1
f = lambda n: (n*0+1)
g = lambda n: (n)

plot_oh(f, g, c, n_0, '$1$', str(c)+'$n$', 1, 100)

Q10. From the definitions, prove or disprove that is n O(1)

Solution
From the definition,
∃n0 ≥ 0, ∃c ≥ 0 such that
n ≤ c ⋅ 1, ∀n ≥ n0 and and are
c n0

constants.
For this inequality to hold true, would need to depend on however from the
c n0

definition both are constants hence we can always choose a value of

n = c + 1

which invalidates the inequality. That is,

∀c ≥ 0, ∀n0 ≥ 0, ∃n ≥ n0 such that
n ≰ c ⋅ 1 whereby
n = c + 1 .

Q11. From the definitions, prove or disprove that is n

2
O(n)
 Solution
Similarly with Q10 this can be disproved.
Would need to prove that n
2
≤ c ⋅ n, ∀n ≥ n0

Simplifying the inequality by dividing my gives us n n ≤ c ⋅ 1, ∀n ≥ n0

Proof: "goto Q10"

Q12. Given that

state 2

the Big-Oh Behaviour and prove it from the definition

f (n) = IF even(n) THEN n + 3 ELSE n + 5

Any natural number "n" is either even or not even. If the number is even, then we get
feven (n) = n + 3, otherwise we get .
fodd (n) = n
2
+ 5

feven = O(n) whereas . But as a single function ,

fodd = O(n )
2
. f
2
f (n) = O(n )

We can prove that is f (n) by choosing fixed values of and for both even
O(n )
2
c n0

and odd cases; that is, and should be the same for both the odd case and the
c n0

even case.
Solution
Even case Odd case
2 2 2
n + 3 ≤ c ⋅ n n + 5 ≤ c ⋅ n ∀n ≥ n0

2
3 ≤ n(cn − 1) 5 ≤ n (c − 1) ∀n ≥ n0

Need , choose
c > 1 c = 2 .
2 2
3 ≤ 2n − n 5 ≤ n ∀n ≥ n0

n ≥ 1.5 n ≥ √5 ∀n ≥ n0

Choose ceiling of 1.5 and . √5

n ≥ 1.5 n ≥ √5 ∀n ≥ 3

∴ f (n) is O(n )
2
using c = 2 and n0 = 3 .
Sanity check
In [13]: c = 2
n_0 = 3

# max value of n in plot

max_n = 20

# f(n)
 xs = np.arange(1,max_n+1,1)
fns = np.ones(max_n)
for n in range(1,max_n+1):
fns[n-1] = ((n + 3) if (n % 2 == 0) else (n**2 + 5))

# g(n)
gns = (np.arange(1,max_n+1,1)**2)*c

# plot
plt.axvline(x = n_0, color = 'k', linestyle='--')
plt.scatter(x=xs, y=fns, marker='x')
plt.scatter(x=xs, y=gns, marker='.')
plt.xlabel('n')
plt.ylabel('f(n)')
plt.legend(['$n_0 = $'+str(n_0),'$f(n)$', '$cg(n):$' + str(c) + '$\cdot n

Out[13]: <matplotlib.legend.Legend at 0x1193bbac0>

Algebraically Challenging (Answers)

Work out the Big-Oh of the following functions and prove them using the definitions.
Q13. 3n
3
+ 10000n
 Solution
(Big-Oh Derivation using Rules)
3
3n + 10000n

3 10000
n ⋅ (3 + 2
)
n

10000

n
2
→ 0 as n → ∞

3
∴ f (n) = O(n )

(Proof)
To recall the definition of Big-Oh: Given positive functions and , we can say f (n) g(n)

that if and only if there exists positive constants and such that
f (n) is O(g(n)) c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
3n
3
is
+ 10000n if and only if there exists positive constants and such
O(n )
3
c n0

that 3n
3
+ 10000n ≤ c ⋅ n , ∀n ≥ n0
3

3 3
3n + 10000n ≤ c ⋅ n , ∀n ≥ n0

3n
3
+ 10000n ≤ 4n , ∀n ≥ n0
3
(choose c = 4 )
10000n ≤ n , ∀n ≥ n0
3
(subtract ) 3n
3

10000 ≤ n , ∀n ≥ n0
2
(divide by ) n

100 ≤ n, ∀n ≥ n0 ( ) √.

Hence, n0 ≥ 100 so pick n0 = 100

100 ≤ n, ∀n ≥ 100 -- trivial

∴ 3n
3
+ 10000n is for
O(n )
3
and
c = 4 n0 = 100 .

Sanity check
In [14]: c = 4
n_0 = 100
f = lambda n: 3*n**3+10000*n
g = lambda n: n**3

plot_oh(f, g, c, n_0, '$3n^3+10000n$', str(c) + '$\cdot n^3$', 75, 125)

Q14. n log(n) + 2n

Solution
(Big-Oh Derivation using Rules)
n log(n) + 2n

n (log(n) + 2) (factor out n)

n (log n) (drop smaller terms)
∴ n log(n) + 2n = O(n log(n))

(Proof)
To recall the definition of Big-Oh: Given positive functions and , we can say
f (n) g(n)

thatf (n) is O(g(n)) if and only if there exists positive constants and such that
c n0

f (n) ≤ c ⋅ g(n), ∀n ≥ n0 .
n log(n) + 2n is O(n log(n)) if and only if there exists positive constants and
c n0

such that n log(n) + 2n ≤ c ⋅ n log(n), ∀n ≥ n0

 n log(n) + 2n ≤ c ⋅ n log(n), ∀n ≥ n0

2n ≤ (c − 1) ⋅ n log(n), ∀n ≥ n0

2n ≤ n log(n), ∀n ≥ n0 (choose c = 2 )
2 ≤ log(n)

2 log2 (n)
2 ≤ 2

4 ≤ n (using log base 2)

∴ n log(n) + 2n = O(n log(n)) for c = 2 and n0 = 4

Sanity check
In [15]: c = 2
n_0 = 4
f = lambda n: (n*np.log2(n)+2*n)
g = lambda n: (n*np.log2(n))

plot_oh(f, g, c, n_0, '$n\;log(n)+2n$', str(c) + '$\cdot n\;log(n)$', 1,

 Q15. 2
n
+ n

Solution
Exponents beat powers (from the rules) so is just:
n
= O(2 )

n n
2 + n ≤ c ⋅ 2 , ∀n ≥ n0

n
n ≤ 2 (c − 1), ∀n ≥ n0 (choose ) c = 2

n
n ≤ 2 , ∀n ≥ n0 which is true for all integers.
n
n ≤ 2 , ∀n ≥ 1

∴ 2
n
+ n is O(2 )
n
for c = 2 and n0 = 1.

Sanity check
In [16]: c = 2
n_0 = 1
f = lambda n: (2**n + n)
g = lambda n: (2**n)

plot_oh(f, g, c, n_0, '$2^n + n$', str(c) + '$\cdot 2^n$', 1, 10)

Summary: Venn Diagram
Draw a Venn Diagram of the sets ,
O(1) O(n) and 2
O(n ) , and place the following
functions on the diagram:
f 1(n) = 1

f 2(n) = 42

f 3(n) = n

f 4(n) = 3n + 5
2
f 5(n) = n
2
f 6(n) = n + log(n)

Solution: