LECTURE 13

Review
What Did We Learn This Semester?
CS1010E Road Map
Memoization Object-Oriented
Big-O Exception ADVANCED
Visualisation Programming

Higher-Order Multi-Dimensional
Searching Array INTERMEDIATE
Functions
& Sorting Deepmap
Functional Iteration Sequence
Wishful BASIC
Selection Abstraction
Thinking Recursion Collection

Fundamental concepts of computer programming

Nuts and Bolts of Programming
• Va r i a b l e
• A name that represents something in a program
• A storage for something

• Ty p e s
• The values that the something can take
• int, float, bool, str

• Operators
• How to manipulate that something to produce something else
• +, -, *, /, //, %, **
• >, >=, <, <=, ==, !=, in
• and, or, not
Operations in Python
• Modular Arithmetic

• Short-Circuit Operations
Operations in Python
• Sequence Operations
Syntax Sequence String
seq1 + seq2 Concatenate Append
num * seq Repetition Repetition
seq[num] Indexing Indexing
seq[start:stop:step] Slicing Slicing
len(seq) Length Length
min(seq) Minimum Minimum
max(seq) Maximum Maximum
elem in seq Check Substring (str1 in str2)
Order of Operations
Order of Operations
Functional Abstraction
• Designer cares about implementation
• User cares about Inputs
• Purpose of the function (value)
• Input/output relationship

• Function
is a way to manage
complexity
Outputs
(value)
 Functional Abstraction
(1) create this (2) called square (3) that takes in
function one input called x

def square(x):
res = x ** 2 (4) then perform
return res this operation

(5) and return the

result from variable
called res
Functional Abstraction
• Function as parameter
def apply(f, x):
return f(x)
apply(abs, -2)

• Function as return value

def make(x):
def f():
return x
return f
x = make(5)
x()
Selection
• Choose what to do based on a condition

if <condition>:
<statement> # then-body
elif <condition>:
<statement> # elif-body only one of these
optional
else: will be executed
<statement> # else-body
 Iteration
• Repeatedly execute a block of code

don’t forget to until the condition

while <condition>:
update the condition <statement> # loop-body is False
in loop-body for each values in
for <var> in <iterable>:
<statement> # loop-body the iterable
Recursion
•A function that calls itself
• Defining something in terms of itself
1 if 𝑛𝑛 ≤ 1
• 𝑛𝑛 ! = �
𝑛𝑛 × 𝑛𝑛 − 1 ! otherwise
 Recursion

def factorial(n):
base case if n <= 1:
return 1 base value
recursive case else:
return n * factorial(n-1) recursive call

continuation next value

Quick Tracing Technique
Tr a c e Code
Iteration res i res=res*i def factorial(n):
pre 1 - - res = 1
#1 1 1 1 for i in range(1,n+1):
#2 1 2 2 res = res*i
#3 2 3 6 return res
#4 6 4 24
post 24 ??? -
factorial(4)
Quick Tracing Technique
Tr a c e Code
24 def factorial(n):
factorial(4) if n <= 1:
24
return 1
4 * factorial(3) else:
6
return n*factorial(n-1)
3 * factorial(2)
2
2 * factorial(1)
factorial(4)
1
1
Iteration vs Recursion
• Different ways of problem solving
Iteration vs Recursion
How to Find Your Row?
• Yo u r row number is 1 more than the row in front of you
• Ask the person in front of you for their row number and
ask 1 to it
• The person in front uses the same strategy
• E v e n t u a l l y, the person in the front row simply replies 1

this is recursive thinking

Iteration vs Recursion
How to Find Your Row?
• Get out of your seat, start counting from row 1
• Step d o w n t o t h e n e x t r o w, a d d 1 t o t h e c o u n t
• Repeat until you reach the first row

this is iterative thinking

Iteration vs Recursion
Iteration Recursion
def factorial(n): def factorial(n):
res = 1 if n <= 1:
while n > 1: return 1
res = res * n else:
n = n-1 return n * factorial(n-1)
return res

initial value ≡ base value computation ≡ continuation

loop condition ≡ not (base case) next value ≡ next value argument
Multi-Dimensional Array
1-D Array 2-D Array
d1arr = [1,2,3,4,5] d2arr = [[1,2,3,4,5],
[1,2,3,4,5],
[1,2,3,4,5]]
Not an Array
not_arr = [[1,2,3],
[4,5],
[7,8,9,10]]
Deepmap
deepmap
def map(f, lst):
if not lst:
return []

else:
return [f(lst[0])] +
map(f, lst[1:])
Deepmap
deepmap flatten
def deepmap(f, lst): def flatten(f, lst):
if not lst: if not lst:
return [] return []
elif type(lst[0]) == list: elif type(lst[0]) == list:
return [deepmap(f, lst[0])] + return [flatten(f, lst[0])] +
deepmap(f, lst[1:]) flatten(f, lst[1:])
else: else:
return [f(lst[0])] + return [f(lst[0])] +
deepmap(f, lst[1:]) flatten(f, lst[1:])
Deepmap
deepmap flatten
def deepmap(f, lst): def flatten(f, lst):
if not lst: if not lst:
return [] return []
elif type(lst[0]) == list: elif type(lst[0]) == list:
return [deepmap(f, lst[0])] + return flatten(f, lst[0]) +
deepmap(f, lst[1:]) flatten(f, lst[1:])
else: else:
return [f(lst[0])] + return [f(lst[0])] +
deepmap(f, lst[1:]) flatten(f, lst[1:])
Searching and Sorting
•A look into algorithms and complexity
• Search
• Linear Search and Binary Search
• Sort
• SelectionSort, Merge Sort, Bubble Sort, etc
• Python has sorting methods too
• lst.sort()
• sorted(seq)
Object-Oriented Programming
•A way of looking at the world
•A program is made up of objects of a particular class
• Each object manages its own data (attributes)
• The state (a set of attribute values) of an object may
be changed or retrieved through methods
• Changed via mutators
• Retrieved via accessors
• Object may interact with one another through method
calls
 Object-Oriented Programming
class name superclass name

class ClassName(SuperClass):
def __init__(self, attr): constructor
calling superclass
super().__init__(attr)
method
self.attr = attr
def accessor(self): retrieve attribute
attribute return self.attr
def mutator(self, val): change attribute
self.attr = val
Errors and Exceptions
• Types of Error
• SyntaxError
• Runtime errors
• ZeroDivisionError 1/0
• TypeError 1+"2"
• ValueError int("one")
• NameError undefined
• IndexError [1,2,3][4]
• KeyError {'a':0}['b']
Errors and Exceptions
try: try: except ErrorType1:
# may cause error
# handle error
except ErrorType1:
# what to do on error
except ErrorType2: except ErrorType2:
# what to do on error else: # handle error
except:
# what to do on error
except:
else:
# what to do when no error finally:
# handle generic error
finally:
# always executed
Order of Growth
• In Physics, we consider
• Ti m e

• Space
Order of Growth
• In CS, we consider
• Ti m e
• How long a program runs

• Space
• How much memory
needed
Order of Growth
• Analogy
• Suppose you want to buy
a Blu-ray movie from Amazon
• Two options
1. Download
2. 2-day Prime Shipping
 Which is
faster?
Order of Growth
• Analogy
• More movies?

 Which is
faster?
Order of Growth
• Download vs Delivery
Time
Download

2 days Delivery

1 hour
# of movies
Order of Growth
• Ultimate question
• If the “volume” increases
• How much more resources (time and space)
are needed?
Order of Growth
• Example
• Will they grow in the same manner?
• From • From
factorial(10) fib(10)
• To • To
factorial(20) fib(20)
• To • To
factorial(100) fib(100)
• To • To
factorial(10000) fib(10000)
Order of Growth
• Is NOT...
• The absolutetime/space a program needed
to complete its computation
• Is...
• The
proportion of growth of the time/space of
a program with respect to the growth of the
input
Order of Growth
• Let’s test on something we know
ctx = {'fact': 0, 'fib': 0}

def factorial(n):
ctx['fact'] += 1
if n <= 1:
return 1
else:
return n * factorial(n-1)

def fib(n):
ctx['fib'] += 1
if n <= 1:
return n
else:
return fib(n-1) + fib(n-2)
Order of Growth
• Compare
>>> factorial(5) >>> fib(5)
120 5
>>> ctx['fact'] >>> ctx['fib']
5 15
>>> ctx['fact'] = 0 >>> ctx['fib'] = 0
>>> factorial(10) >>> fib(10)
3628800 55
>>> ctx['fact'] >>> ctx['fib']
10 177
>>> ctx['fact'] = 0 >>> ctx['fib'] = 0
>>> factorial(20) >>> fib(20)
2432902008176640000 6765
>>> ctx['fact'] >>> ctx['fib']
20 21891
Order of Growth
• Factorial
>>> factorial(5)
120
• The order of growth is simple
>>> ctx['fact'] • If input is n, then the function
5 is called n times
>>> ctx['fact'] = 0
>>> factorial(10) • Because each time n is
3628800 reduced by 1
>>> ctx['fact']
10 • So the number of times the
>>> ctx['fact'] = 0 function is called is proportional
>>> factorial(20)
2432902008176640000 to n
>>> ctx['fact']
20
Order of Growth
• Fibonacci
• More complicated >>> fib(5)
5
• fib(n) calls: >>> ctx['fib']
• fib(n-1) 15
>>> ctx['fib'] = 0
• fib(n-2) >>> fib(10)
• We need to draw the 55
>>> ctx['fib']
recursion tree 177
>>> ctx['fib'] = 0
>>> fib(20)
6765
>>> ctx['fib']
21891
Recursion Tree root

fib(5)

fib(4) fib(3)

fib(3) fib(2) fib(2) fib(1)

fib(2) fib(1) fib(1) fib(0) fib(1) fib(0) 1

fib(1) fib(0) 1 1 0 1 0

1 0
• Roughly half of a full
tree
leaves • Full tree has 2n-1
nodes
Order of Growth
• Compare
>>> factorial(5) >>> fib(5)
120 5
>>> ctx['fact'] >>> ctx['fib']
5 15
>>> ctx['fact'] = 0 >>> ctx['fib'] = 0
>>> factorial(10) >>> fib(10)
3628800 55
>>> ctx['fact'] >>> ctx['fib']
10 177
>>> ctx['fact'] = 0 >>> ctx['fib'] = 0
>>> factorial(20) >>> fib(20)
2432902008176640000 6765
>>> ctx['fact'] >>> ctx['fib']
20 21891
proportional to n proportional to 2n
Order of Growth
• Compare

proportional to n proportional to 2n
Order of Growth
• Other algorithms
Linear Search Binary Search
• #comparisons • #comparisons
proportional to n proportional to log(n)
• Because on average, the • Because we divide the
expected number of list into half for at most
search is n/2 l o g 2( n ) t i m e s
Order of Growth
• Other algorithms
Selection/Bubble/Insertion Merge Sort
• #comparisons • #comparisons
proportional to n2 proportional to n×log(n)
• Because we looped n • Because we divide the
times list into half for at most
l o g 2( n ) t i m e s
• And each time, we
rearrange 1 to n items • Andeach time, we
arrange n items
Experiment
from random import randint
from time import time
sizes = [2000, 4000, 6000, 8000, 10000, 12000, 14000, 16000]
stats = {'bubble': [], 'merge': []}
for size in sizes:
print(size)
seq = [randint(1, 10*size) for _ in range(size)]
st = time()
bubble_sort(seq.copy())
stats['bubble'].append(time()-st)
st = time()
merge_sort(seq.copy())
stats['merge'].append(time()-st)
Experiment
ALGORITHM
anyone can give some algorithms
Algorithm
• Bogo Sort
• bogo_sort(seq)
• Repeat:
• Choose a random permutation
of the sequence seq
• If seq is sorted, return seq
• Ifyou wait long enough,
s e q will be sorted
 ALGORITHM
anyone can give some algorithms

but how fast is your algorithm?

Algorithm
• Quantum Bogo Sort
• q_bogo_sort(seq)
1. Choose a random permutation
of seq
2. If it is sorted, return seq
3. Otherwise, destroy universe

 Remember q_bogo_sort when you learn

about non-deterministic Turing Machines
Algorithm
• Remove Sort
• remove_sort(seq)
1. Iterate through the sequence
2. If an element is smaller than
the element before it, then
remove the element

 Obviously this produces different

sequence but still very efficient
HOW TO IMPROVE?
Let’s try on fib(n)
 Easy Way: Memoization
We save the result when fib(5) Instead of computing fib(3) here
it was computed here
fib(4) fib(3)

fib(3) fib(2) fib(2) fib(1)

fib(2) fib(1) fib(1) fib(0) fib(1) fib(0) 1

fib(1) fib(0) 1 1 0 1 0

1 0
 Easy Way: Memoization
We save the result when fib(5)
it was computed here
fib(4) fib(3)

fib(3) fib(2) 2

fib(2) fib(1) fib(1) fib(0) Instead of computing fib(2) here

fib(1) fib(0) 1 1 0

1 0
Easy Way: Memoization
fib(5)

fib(4) fib(3)

fib(3) fib(2) 2

fib(2) fib(1) 1

fib(1) fib(0) 1

1 0

Now it looks more like

linear proportion!
Memoization
• Steps fibmemo = {}
1. Create a dictionary def fibm(n):
to remember the if n in fibmemo:
answer if fibm(n) return fibmemo[n]
has been computed
before if n <= 1:
2. If it was computed, ans = n
immediately retrieve else:
the answer ans = fibm(n-1) + fibm(n-2)
3. Otherwise, compute
and put into dictionary fibmemo[n] = ans
return ans
Memoization
• Steps fibmemo = {}
1. Create a dictionary def fibm(n):
to remember the if n in fibmemo:
answer if fibm(n) return fibmemo[n]
has been computed
before if n <= 1:
2. If it was computed, ans = n
immediately retrieve else:
the answer ans = fibm(n-1) + fibm(n-2)
3. Otherwise, compute
and put into dictionary fibmemo[n] = ans
return ans
Memoization
• Steps def memoize(f):
1. Create a dictionary memo = {}
to remember the def memo_f(*args):
answer if fibm(n) if args in memo:
has been computed return memo[args]
before else:
2. If it was computed, memo[args] = f(*args)
immediately retrieve return memo[args]
the answer return memo_f
3. Otherwise, compute
and put into dictionary
Next Step: Recursion Removal
fib(5)

fib(4) fib(3)

fib(3) fib(2) 2

fib(2) fib(1) 1 This top fib(3) depends on the

result generated by fib(3) below
fib(1) fib(0) 1

1 0
Next Step: Recursion Removal
fib(5)

fib(4) fib(3)

fib(3) fib(2) 2

fib(2) fib(1) 1 This top fib(2) depends on the

result generated by fib(2) below
fib(1) fib(0) 1

1 0
Next Step: Recursion Removal
fib(5)

fib(4) fib(3)

fib(3) fib(2) 2

fib(2) fib(1) 1 So why not we just compute from

bottom-up instead of top-down?
fib(1) fib(0) 1

1 0
Recursion Removal
• Steps def fibrr(n):
1. Store all answers in an ans = [0, 1]
n-dimensional array
• In fib, it’s 1-D for i in range(2, n+1):
2. Computing from bottom
ans.append( ans[i-1] +
is computing from smallest ans[i-2] )
argument
3. Work our way up to larger return ans[n]
argument
• In fib, we compute fib(i) by fib(i-1) + fib(i-2)
Recursion Removal Wait a minute, do
we need all past
numbers if we only
• Steps def fibrr(n): need fib(i-1)
1. Store all answers in an ans = [0, 1] and fib(i-2)?
n-dimensional array
• In fib, it’s 1-D for i in range(2, n+1):
2. Computing from bottom
ans.append( ans[i-1] +
is computing from smallest ans[i-2] )
argument
3. Work our way up to larger return ans[n]
argument
• In fib, we compute fib(i) by fib(i-1) + fib(i-2)
Iterative Fibonacci
• Key Idea def fibi(n):
• Only need to keep if n <= 1:
return n
fib(i-1) and fib(i-2)
ans = [0, 1]
• This is the last two for i in range(2, n+1):
values in the array ans.append( ans[i-1] +
ans[i-2] )
ans = ans[-2:]
return ans[-1]
Iterative Fibonacci
• Key Idea def fibi(n):
• Only need to keep if n <= 1:
return n
fib(i-1) and fib(i-2)
fn1,fn2 = 0,1
• This is the last two for i in range(2, n+1):
values in the array fn1, fn2 = fn2, fn1+fn2
• Make it into two return fn2
variables!
Further Improvement?
• For CS1010E, you should know how to compute fib(n)
with time proportional to n
• The best algorithm to compute arbitrary fib(n)
without loss of precision can be done with time
proportional to log(n)
• To k n o w m o r e a b o u t t h i s
• CS1232, CS2040, CS3230, etc
• What you learned today is called the Big-O notation
• 𝑂𝑂 l o g 𝑛𝑛 , 𝑂𝑂 𝑛𝑛 , 𝑂𝑂 ( 𝑛𝑛 l o g 𝑛𝑛 ) , 𝑂𝑂 𝑛𝑛 2 , 𝑂𝑂 2 𝑛𝑛 , e t c