Algorithms

for Everyone

F r e e m a n , P h D
Eric
1
 eric freeman
computer science phd, yale
adjunct faculty, UT austin
oreilly 5x author
advisor, head first series
former disney cto
MIT tr35

Your instructor
2
3
Take a deep breath, this class
takes an applied approach
not a theoretical one.

4
How class works

• We're here 1 day, 3 hours

• We'll take 10 minute breaks on the hour
• There will be some light code, but it is optional.
• Class interaction highly encouraged!

5
What will you get out of this course?
• Solid introduction to algorithms and data structures
• Understand the crucial Big O notation
• Practical knowledge of essential algorithms
• Insight into fundamental data structures
• Improved coding practices to write more efficient
and scalable code
• Prepare you to further explore algorithms and data
structures on your own

6
The MYTH of the
Tower of Hanoi

7
The MYTH of the
Tower of Hanoi

8
Tower of Hanoi Rules
Objective

• The goal is to move the entire stack of disks from the initial peg
(A) to the destination peg (C), following the rules.

Rules

• Move One Disk at a Time: Only one disk can be moved at a time.
• Top Disk Only: Only the top disk of any peg can be moved.
• No Larger Disk on Smaller Disk: A larger disk cannot be placed
on top of a smaller disk.

9
Tower of Hanoi — Give it a try

https://www.mathplayground.com/logic_tower_of_hanoi.html
10
According to the Tower of Hanoi Myth, how much
time do we have?

In this class we'll acquire the skills to:

• Learn to use divide and conquer techniques to develop and algorithm

and implement the solution
• Learn about recursive techniques needed to solve the puzzle
• Figure out the time complexity of our algorithm
• Compute how long it is going to take the monks to complete their 64-
disk tower, and the end of the world

11
Doctor Who: The Celestial Toymaker (1966) - The Doctor faces
Tower of Hanoi Rules
the enigmatic Celestial Toymaker in a series of deadly games.

Objective

• The goal is to move the entire stack of disks from the initial peg
(A) to the destination peg (C), following the rules.

Rules

• Move One Disk at a Time: Only one disk can be moved at a time.
• Top Disk Only: Only the top disk of any peg can be moved.
• No Larger Disk on Smaller Disk: A larger disk cannot be placed
on top of a smaller disk.

12
 Making a Peanut Butter and Jelly Sandwich

1. Get ingredients and tools:

◦ 2 slices of bread
◦ Butter knife
We are all
◦ Peanut butter
◦ Plate
◦ Jelly
3. Spread peanut butter:
taught an ◦ Use the knife to spread peanut butter on one slice of bread.

algorithm is 3. Spread jelly:

◦ Use the knife to spread jelly on the other slice of bread.

like a recipe 4. Combine slices:

◦ Place the two slices of bread together, with the peanut butter and
jelly sides facing each other.
5. Serve:

◦ Place the sandwich on a plate and enjoy.

13
What is an algorithm?

14
Donald Knuth says…

An algorithm:
• Is finite
• Is precisely defined
• Has input
• Has output
• Must be effective

credit: quanta magazine

Group Question:
How is an algorithm
different from a recipe?
We're interested in good algorithms, but
what makes an algorithm good? Extensibility

Correctness
Efficiency Scalability

ality Simplic
ity
Optim Clarity

Robustness Reliability

Adapta
bility
ability
Maintain
Modularity
17
We're going to be concerned with
two of these Extensibility

Correctness
Efficiency Scalability

ality Simplic
ity
Optim Clarity

Robustness Reliability

Adapta
bility
ability
Maintain
Modularity
18
Two ways of measuring efficiency

• Time Complexity
Measures the amount of time an
algorithm takes to complete as a
function of the size of its input.

• Space Complexity
Measures the amount of memory an
algorithm uses as a function of the size
of its input.

19
Let's play a guessing game…

I'm going to pick a number between

1 and 100, and you are going to see
how many guesses it takes to
identify the number.
After each guess I will tell you if you
the guess was too high, too low, or
you got the answer.

20
Let's play a guessing game…

Player 1: Jill

Player 2: Melissa

We're going to play with two players,

the secret number is going to be 40.
21
Let's play a guessing game…

Jill: 1 Jill: 6 and so

Too low Too low on until
Jill: 2 Jill: 7 40…
Too low Too low

Jill: 3 Jill: 8
Too low Too low

Jill: 4 Jill: 9
Too low Too low

Jill: 5 Jill: 10
Too low Too low

22
Let's play a guessing game…
Melissa: 50
Too high
Melissa: 25
Too low
Melissa: 37
Too low
Melissa: 43
Too high
Melissa: 40
You got it!

23
We have a winner!

40 guesses Algorithm 1: Jill

5 guesses Algorithm 2: Melissa

24
Question:
But what were Jill and
Melissa's algorithms?
Algorithm 1

def algorithm1(number):
for i in range(1, 101):
print(f"Guessing {i}")
if i == number:
print(f"You guessed the number {number} in {i} guesses")
break

num = 40 Guessing 1
Guessing 2
algorithm1(num) Guessing 3
Guessing 4
Guessing 5
Guessing 6
Guessing 7
.
.
.
.
Guessing 40
You guessed the number 40 in 40 guesses
26
Algorithm 1
def algorithm1(number):
for i in range(1, 101):
print(f"Guessing {i}")
if i == number:
print(f"You guessed the number {number} in {i} guesses")
break

num = 40
algorithm1(num)

Best Case Worst Case

1 100
27
Algorithm 2
def algorithm2(number):
low = 1
high = 100
found = False
guesses = 0;

while not found:

guesses = guesses + 1
mid = (low + high) // 2
print(f"Guessing {mid}")

if mid == number:
print(f"You guessed the number {number} in {guesses} guesses")
found = True
elif mid < number:
low = mid + 1 Guessing 50
else: Guessing 25
high = mid - 1 Guessing 37
Guessing 43
num = 40 Guessing 40
algorithm2(num) You guessed the number 40 in 5 guesses
28
How long does it take each of these algorithms?
Let's say 1 comparison takes a millisecond

Algorithm 1 Algorithm 2
Size Time (seconds) Time (seconds)
100 .1 0.00664
1000 1.0 0.00997
10,000 10.0 0.01329
100,000 100.0 0.01661
1,000,000 1000.0 0.01993

So can we say algorithm 2 is 50,000 times

faster than algorithm 1? 29
Let's try a billion and see…

Algorithm 1 Algorithm 2
Size Time (seconds) Time (seconds)
100 .1 0.00664
1000 1.0 0.00997
10,000 10.0 0.01329
100,000 100.0 0.01661
1,000,000 1000.0 0.01993
1,000,000,000 1000000.0 0.02990
NO! When we have a billion numbers, the
difference is over 34 million times faster!
30
We need a better way
that timing to know how
good an algorithm is.
Big O Notation

32
Big O Notation

The big "O"

The time complexity

33
Big O Notation

The big "O"

The time complexity

34
Comparison of n and n2

Size
100 100 10,000
1000 1000 1,000,000
10,000 10,000 100,000,000
100,000 100,000 10,000,000,000
1,000,000 1,000,000 1,000,000,000,000

35
So what is the time
complexity of Jill's
Algorithm 1 and
Melissa's Algorithm 2?
Algorithm 1

What is the worst

case for Jill?

37
Algorithm 1's time complexity

The upper bound with Algorithm 1 is

the number of possible numbers,
you have to go through each one.
38
Algorithm 2

What is the worst

case for Melissa?

39
Algorithm 2's time complexity

I could just tell you it is

40
Algorithm 2

What is the worst

case for Melissa?

41
For each guess, how many
possibilities does Jill rule
out? What about Melissa?

42
Each iteration of algorithm 2 removes half the
possibilities from consideration

{
100
50
7 guesses
25
for 100 13
numbers 7
4
2
1
43
Let's consider 512 possibilities

{
512
256
9 guesses 128
for 512
64
32
numbers
16
8
4
2
1
44
Notice how these are powers of 2?

{
512
256
9 guesses 128
for 512
64
32
numbers
16
8
4
2
1
45
A little handy math review

46
So we can compute the upper bound with log

{
512
256
9 guesses 128
for 512
64
32
numbers
16
8
4
2
1
47
 Worst
Algorithm 2 worst case Problem case
size guesses

If the size of our guessing game

is n, then the worst case is:

rop
We usually d nd
the base 2 ag"
just say "lo

48
Remembering Logarithms

Think of them as the inverse of exponentials:

x
2 = 16 x=4
Then

log 216 = x x=4

Algorithm 2
def algorithm2(number):
low = 1 Best Case Worst Case
high = 100

1
found = False
guesses = 0;

while not found:

guesses = guesses + 1
mid = (low + high) // 2
print(f"Guessing {mid}")

if mid == number:
print(f"You guessed the number {number} in {guesses} guesses")
found = True
elif mid < number:
low = mid + 1
else:
high = mid - 1

num = 40
algorithm2(num)
50
Algorithm 2's time complexity

The upper bound with Algorithm 2 is

the log (base 2) of n.
51
More formally comparing algorithms

Algorithm 1: guesses

Algorithm 2: guesses

52
Algorithm 2

What is the worst

case for Melissa?

53
Group Question:
What is the worst case
for guessing over 1
billion numbers?
Group Question:
What is the worst case for
a range of 1 billion?

log(1,000,000,000) is
only 30!
Comparing algorithms,
if a guess takes a microsecond…

1 Billion guesses takes

16.67 minutes

1 Billion guesses takes

?
29.8974 microseconds

56
Comparing algorithms,
if a guess takes a microsecond…

1 Billion guesses takes

16.67 minutes

1 Billion guesses takes

29.8974 microseconds

57
 Common Time Complexities

58
Common Time Complexities

59
Simplifying Big O Notation
We study the algorithm and
come up with this expression

Choose the highest order term

Drop the lower order

terms and constant.

We even drop the constant factor

in front of the highest order term

Leaving us with just n2

60
Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

61
Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

62
 Ride Wait Times

VIP Pass Express O(1)

Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)

Logarithmic Spiral
Factorial Frenzy O(n!)

63
 Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

Linear Loop 64
 Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

Quadratic Twist 65
 Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

Exponential Drop 66
 Ride Wait Times
VIP Pass Express O(1)
Logarithmic Spiral O(log n)
Linear Loop O(n)
Quadratic Twist O(n2)
Exponential Drop O(2n)
Factorial Frenzy O(n!)

Factorial Frenzy 67
Searching
Algorithms

68
Group Question:
How hard is finding
phone number by
last name in a
phone book?
Group Question:
How hard is finding
a last name by
phone number in a
phone book?
Group Question:
Finding all people
with a given last
name, say,
"Johnson".
Group Question:
Finding all people
with a given first
name, say, "John".
Arrays
● A fixed-size sequence of elements of the
same type (generally)
● Contiguous memory locations
● Each element can be accessed directly by
index
● First element has an index of 0
● Storage and retrieval by index are O(1)

0 1 2 3 4 5

But what about finding an element?

73
Let's search for something in the array

Movie releases by Disney animation, sorted by year.

[1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970, 1989,

1991, 1992, 1994, 1995, 1998, 1999, 2000, 2001, 2002,
2003, 2004, 2005, 2007, 2008, 2009, 2010, 2011, 2012,
2013, 2014, 2016, 2018, 2019, 2020, 2021, 2022]

Did they release a movie in 1967? Let's see…

74
Linear Search

def linear_search(arr, target):

for i in range(len(arr)):
if arr[i] == target:
return i
return -1 # If the target is not found

arr = [1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970, 1989, 1991, 1992, 1994, 1995, 1998,
1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014,
2016, 2018, 2019, 2020, 2021, 2022]

target = 1967
result = linear_search(arr, target)

print(f"Element {target} found at index {result}")

75
Linear Search Time Complexity

def linear_search(arr, target):

for i in range(len(arr)):
if arr[i] == target:
return i
return -1 # If the target is not found

arr = [1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970, 1989, 1991, 1992, 1994, 1995, 1998,
1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014,
2016, 2018, 2019, 2020, 2021, 2022]

target = 1967
result = linear_search(arr, target)

print(f"Element {target} found at index {result}")

76
Binary Search
def binary_search(arr, target): # The given array
low = 0 arr = [1937, 1950, 1953, 1955, 1959, 1961, 1967,
high = len(arr) - 1 1970, 1989, 1991, 1992, 1994, 1995, 1998, 1999,
2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008,
while low <= high: 2009, 2010, 2011, 2012, 2013, 2014, 2016, 2018,
mid = (low + high) // 2 2019, 2020, 2021, 2022]
mid_value = arr[mid]

if mid_value == target:
return mid
elif mid_value < target:
low = mid + 1
else:
high = mid - 1

return -1 # If the target is not found

target = 1967
result = binary_search(arr, target)
print(f"Element {target} found at index {result}")

77
[1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970, 1989, 1991, 1992, 1994, 1995, 1998,
1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
2014, 2016, 2018, 2019, 2020, 2021, 2022]

[1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970, 1989, 1991, 1992, 1994, 1995, 1998,
1999, 2000, 2001]

[1937, 1950, 1953, 1955, 1959, 1961, 1967, 1970]

[1959, 1961, 1967, 1970]

[1967, 1970]

[1967] 78
Binary Search Time Complexity
def binary_search(arr, target): # The given array
low = 0 arr = [1937, 1950, 1953, 1955, 1959, 1961, 1967,
high = len(arr) - 1 1970, 1989, 1991, 1992, 1994, 1995, 1998, 1999,
2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008,
while low <= high: 2009, 2010, 2011, 2012, 2013, 2014, 2016, 2018,
mid = (low + high) // 2 2019, 2020, 2021, 2022]
mid_value = arr[mid]

if mid_value == target:
return mid
elif mid_value < target:
low = mid + 1
else:
high = mid - 1

return -1 # If the target is not found Does this

target = 1967 seem familiar?
result = binary_search(arr, target)
print(f"Element {target} found at index {result}")

79
Divide and Conquer
A method for solving problems by breaking them into smaller
subproblems, solving each independently, and combining the results.

• Built on two main principles:

- Principle of Problem Decomposition
- Principle of Recursive Problem Solving

• Examples:
- Merge Sort
- Quick Sort
- Binary Search
80
Best, worst, average case?

• Worst Case: Maximum time/space required, ensuring

upper limit performance.
• Best Case: Minimum time/space required, representing the
fastest scenario.
• Average Case: Expected time/space, averaged over all
possible inputs for typical performance.

Big O notation typically refers to the worst-case scenario,

providing a reliable upper bound for algorithm performance.

81
We can also talk about the complexity of array operations

82
Dynamic Arrays
● We allocated more locations than needed
● We expand if we use all locations
● Contiguous memory locations
● Each element can be accessed directly by
index
● First element has an index of 0
● Storage and retrieval by index are O(1)

0 1 2 3 4 5

How does the run/space time complexity differ?

83
We can also talk about the complexity of array operations

84
Linked Lists
● Dynamic memory allocation without
contiguous storage
● Efficient insertion and deletion operations
● Nodes contain data and a link/reference
● Use in implementation of other data
structures (queues, hash tables, etc.)
● Supports:
- insert()
- delete()
- update()
- search()

85
The data structure

Pointer Pointer Pointer

to next to next to next
element element element

Start
of list 537-8129 739-4851 359-2784 682-3475

First element Last element

86
Doubly-linked list

Pointer Pointer Pointer

to next to next to next
element element element

Start End
of list 537-8129 739-4851 359-2784 682-3475 of list

Pointer to Pointer to Pointer to

First element Last element
previous previous previous
element element element

87
We can also talk about the complexity of array operations

88
Group Question:
How do arrays and
linked lists
compare?
For search, can we
do better in search
than O(log n)?
Hash tables
A hash table, also known as a hash map,
is a fundamental data structure that
provides efficient access to data through
the use of a key.

- Use a hash function to compute

an index based on a key
- Provides O(1) insertions,
lookups, and deletion
- Powerful tool, balancing speed
and efficiency for many practical
applications

91
Hash table uses

• Spell checkers
• Caching
• Compilers
• Network routing
• Web servers
• Dictionaries, objects

92
How hash tables work

(key, value) pairs key hash function hash value

(Emma Johnson, 834-2109) Emma Johnson 3

(Liam Smith, 658-2934) Liam Smith 1

(Olivia Brown, 903-1748) Olivia Brown 7

(Noah Davis, 273-4816) Noah Davis 0

93
How hash tables work
value

0
key
1
Emma Johnson
2
Liam Smith
3
Olivia Brown 4
Noah Davis 5
6
7
94
Putting values in
hash table hash function value

0
key
1
Emma Johnson
2
Liam Smith
3 834-2109
Olivia Brown 4
Noah Davis 5
6
7
95
Putting values in
hash table hash function value

0 273-4816
key
1 658-2934
Emma Johnson
2
Liam Smith
3 834-2109
Olivia Brown 4
Noah Davis 5
6
7 903-1748
96
Getting a value
from hash table hash function value

0 273-4816
key
1 658-2934
Emma Johnson
2
3 834-2109
834-2109 4
5
6
7 903-1748
97
Collisions and
chaining hash function value

0 273-4816
key
1 658-2934
Emma Johnson
2
3 834-2109 511-9871
4
collision

5
6
Thomas Kay
7 903-1748
98
hash function
value

0 273-4816 924-7153 847-5963

1 658-2934 163-5892
2
3 834-2109 511-9871 537-8129 739-4851 359-2784 682-3475
4
5 483-9271 472-3581
6
903-1748 593-7482
99
Hash table complexity

With a decent hash

function we can expect this.

100
Hashing in Python
my_dict = {}
my_dict["key"] = "value"
print(my_dict["key"]) # Output: value

# See the hash value of the key

key = "key"
hash_value = hash(key)
print(f"The hash value of the key '{key}' is: {hash_value}")

value
The hash value of the key 'key'
is: -4123338472047041157

101
 Superman's powers: Super
Hashtables in Python strength, flight, x-ray vision
hero_powers = {} Wonder Woman's powers: Super
strength, agility, lasso of truth
hero_powers["Superman"] = "Super strength, flight, x-ray vision" Batman's powers: Intellect,
hero_powers["Wonder Woman"] = "Super strength, agility, lasso of truth" martial arts, high-tech gadgets
hero_powers["Batman"] = "Intellect, martial arts, high-tech gadgets" Spider-Man is not in the hash
table.
print("Superman's powers:", hero_powers["Superman"]) Updated Batman's powers:
print("Wonder Woman's powers:", hero_powers["Wonder Woman"]) Intellect, martial arts,
print("Batman's powers:", hero_powers["Batman"]) detective skills
Wonder Woman has been removed
if "Spider-Man" in hero_powers: from the hash table.
print("Spider-Man's powers:", hero_powers["Spider-Man"])
else: Final state of the hash table:
print("Spider-Man is not in the hash table.") Superman: Super strength, flight,
x-ray vision
hero_powers["Batman"] = "Intellect, martial arts, detective skills" Batman: Intellect, martial arts,
print("Updated Batman's powers:", hero_powers["Batman"]) detective skills
del hero_powers["Wonder Woman"]
print("Wonder Woman has been removed from the hash table.")

print("\nFinal state of the hash table:")

for hero, powers in hero_powers.items():
print(f"{hero}: {powers}") 102
Even more data structures to know
● Stacks:
- LIFO (Last In, First Out): The last element added is the first one to be removed.
- Common Operations: push, pop, peek.
- Use Cases: Expression evaluation, backtracking algorithms, function call management.
- Complexity: push and pop operations: O(1) O(1)

● Queues:
- FIFO (First In, First Out): The first element added is the first one to be removed.
- Common Operations: enqueue, dequeue, front.
- Use Cases: Scheduling tasks, managing requests in web servers, BFS.
- Complexity: enqueue and dequeue operations: O(1) O(1)

● Heaps:
- Binary Heaps: Complete binary tree used to implement priority queues.
- Types: Min-heap (smallest element at root) and max-heap (largest element at root).
- Use Cases: Priority queues, Dijkstra’s shortest path algorithm, heap sort.
- Complexity: insert: O(log n) O(logn), extract-min or extract-max: O(log n) O(logn), find-min or
find-max: O(1)O(1)
103
Stacks
● Last In, First Out (LIFO) data structure.
● Like restaurant plate warmer, the last plate
added is the first to be retrieved.
● Supports:
- push()
- pop()
- peek()
- isEmpty()
- size()
- All operations are O(1)

104
 Superman arrived and joined the
Stack in Python stack.
stack = [] Wonder Woman arrived and joined
the stack.
stack.append("Superman")
print("Superman arrived and joined the stack.")
Wonder Woman is performing a task
stack.append("Wonder Woman") and leaves the stack.
print("Wonder Woman arrived and joined the stack.")
Batman arrived and joined the
last_hero = stack.pop() stack.
print(f"\n{last_hero} is performing a task and leaves the stack.")
Spider-Man arrived and joined the
stack.append("Batman") stack.
print("Batman arrived and joined the stack.")
stack.append("Spider-Man") Spider-Man is performing a task
print("Spider-Man arrived and joined the stack.") and leaves the stack.
last_hero = stack.pop() Batman is performing a task and
print(f"\n{last_hero} is performing a task and leaves the stack.") leaves the stack.
last_hero = stack.pop() Iron Man arrived and joined the
print(f"{last_hero} is performing a task and leaves the stack.") stack.
stack.append("Iron Man")
print("Iron Man arrived and joined the stack.") Iron Man is performing a task and
last_hero = stack.pop() leaves the stack.
print(f"\n{last_hero} is performing a task and leaves the stack.")
print("\nCurrent stack of superheroes:") Current stack of superheroes:
print(stack) ['Superman']
105
Queues
● First In, First Out (FIFO) data structure.
● Like a move theater line, the first person in
line is the first to be given service.
● Supports:

- enqueue()
- dequeue()
- peek()
- isEmpty()
- size()

- All operations are O(1)

106
 Superman arrived and joined the
Queues in Python queue.
Wonder Woman arrived and joined
from collections import deque the queue.
queue = deque() Batman arrived and joined the
queue.append("Superman") queue.
print("Superman arrived and joined the queue.")
queue.append("Wonder Woman")
Current queue of superheroes:
print("Wonder Woman arrived and joined the queue.") deque(['Superman', 'Wonder
Woman', 'Batman'])
queue.append("Batman")
print("Batman arrived and joined the queue.")
Superman is performing a task and
print("\nCurrent queue of superheroes:") leaves the queue.
print(queue)
Wonder Woman is performing a task
first_hero = queue.popleft() and leaves the queue.
print(f"\n{first_hero} is performing a task and leaves the queue.")
second_hero = queue.popleft() Current queue of superheroes:
print(f"{second_hero} is performing a task and leaves the queue.")
deque(['Batman'])
print("\nCurrent queue of superheroes:") Spider-Man arrived and joined the
print(queue)
queue.
queue.append("Spider-Man")
print("Spider-Man arrived and joined the queue.") Final queue of superheroes:
print("\nFinal queue of superheroes:") deque(['Batman', 'Spider-Man'])
print(queue)
107
Recursion

108
Understanding Recursion

• Recursion: a method where the solution

to a problem depends on solutions to
smaller instances of the same problem.

• Key Concept: A function calls itself to

solve sub-problems.

• Relies and a base and recursive case.

• Can provides a clear and elegant way to

solve complex problems.

109
Introducing factorial

1! = 1 = 1
2! = 2 * 1 = 2
3! = 3 * 2 * 1 = 6
4! = 4 * 3 * 2 * 1 = 24
5! = 5 * 4 * 3 * 2 * 1 = 120

99! = 99 * 98 * 97 * … =

110
Thinking recursively

0! = 1 = 1
1! = 1 * 0! = 1
2! = 2 * 1! = 2
3! = 3 * 2! = 6
4! = 4 * 3! = 24
5! = 5 * 4! = 120

99! = 99 * 98!
111
Factorial gets large fast

99! = 9332621544394415268169923885626670
0490715968264381621468592963895217
5999932299156089414639761565182862
5369792082722375825118521091686400
0000000000000000000000

112
Implementing Factorial with iteration

def factorial(n):
result = 1
for i in range(2, n + 1):
result *= i
return result

number = 5
print("The factorial of {number} is
{factorial(number)}")

113
Implementing Factorial with Recursion

Recursive definition, but

we're not finished yet…
def factorial(n):
return n * factorial(n - 1)

number = 5
print("The factorial of {number} is
{factorial(number)}")

114
Implementing Factorial with Recursion

def factorial(n): This is the BASE case

if n == 0:
return 1
else:
This is the
return n * factorial(n - 1) recursive case

number = 5
print(f"The factorial of {number} is
{factorial(number)}")

115
 Another recursive algorithm: Fibonacci

The Fibonacci Sequence

Source: wikipedia 116

Implementing Fibonacci

def fibonacci(n):
if n <= 1:
return n
return fibonacci(n - 1) + fibonacci(n - 2)

117
 fibonacci(6)
├─ fibonacci(5)
The recursive runtime │ ├─ fibonacci(4)
│ │ ├─ fibonacci(3)
for fibonacci(6) │
│
│
│
│
│
├─ fibonacci(2)
│ ├─ fibonacci(1)
│ │ │ │ └─ fibonacci(0)
│ │ │ └─ fibonacci(1)
│ │ └─ fibonacci(2)
│ │ ├─ fibonacci(1)
│ │ └─ fibonacci(0)
│ └─ fibonacci(3)
│ ├─ fibonacci(2)
│ │ ├─ fibonacci(1)
│ │ └─ fibonacci(0)
│ └─ fibonacci(1)
└─ fibonacci(4)
├─ fibonacci(3)
│ ├─ fibonacci(2)
│ │ ├─ fibonacci(1)
│ │ └─ fibonacci(0)
│ └─ fibonacci(1)
└─ fibonacci(2)
├─ fibonacci(1)
└─ fibonacci(0)
118
 fibonacci(6)
├─ fibonacci(5)
The recursive runtime │ ├─ fibonacci(4)
│ │ ├─ fibonacci(3)
for fibonacci(6) │
│
│
│
│
│
├─ fibonacci(2)
│ ├─ fibonacci(1)
│ │ │ │ └─ fibonacci(0)
│ │ │ └─ fibonacci(1)
│ │ └─ fibonacci(2)
│ │ ├─ fibonacci(1)
│ │ └─ fibonacci(0)
│ └─ fibonacci(3)
│ ├─ fibonacci(2)
│ │ ├─ fibonacci(1)
│ │ └─ fibonacci(0)
│ └─ fibonacci(1)
└─ fibonacci(4)
├─ fibonacci(3)
│ ├─ fibonacci(2)
│ │ ├─ fibonacci(1)
Yikes!
│ │ └─ fibonacci(0)
│ └─ fibonacci(1)
└─ fibonacci(2)
├─ fibonacci(1)
└─ fibonacci(0)
119
 Naive Fibonacci Runtime

.
.
.

120
RecursionError: maximum
recursion depth exceeded
Understanding the call stack

● The call stack is a data structure that tracks function calls and their execution order.
● Each function call adds a new frame to the call stack, tracking execution context.
● Recursive calls are just function calls, so each recursion adds a new stack frame; stack grows
with calls, shrinks on returns.
● Beware of stack overflow from excessive recursion.

def fact(n):
if n == 0 or n == 1:
return 1
else:
return n * fact(n - 1)
Call stack
fact(5)

4 * fact(3)

122
 fact(1) = 1 fact(1) = 1
1
2 * fact(1) 2 * fact(1) 2 * fact(1)

3 * fact(2) 3 * fact(2) 3 * fact(2) 3 * fact(2)

4 * fact(3) 4 * fact(3) 4 * fact(3) 4 * fact(3) 4 * fact(3)

def fact(n):
if n == 0 or n == 1:
return 1
else:
2 * fact(1) return n * fact(n - 1)
2 fact(5)
3 * fact(2) 3 * fact(2)
6
4 * fact(3) 4 * fact(3) 4 * 6 = 24
Dynamic programming
A method for solving complex problems by breaking them down into
simpler subproblems

• Built on two main principles:

- Principle of Overlapping Subproblems
- Principle of Optimal Substructure

• Memoization vs. Tabulation

• Right now our Fibonacci using a naive recursive approach
• Let's try a dynamic programming approach using something called
Memoization

124
Fibonacci with dynamic programming
def fibonacci(n, cache={}):
if n <= 1: Memoizing, dynamic
return n
programming technique,
if n in cache: significantly optimizes
return cache[n]
performance by storing
cache[n] = fibonacci(n - 1, cache)
+ fibonacci(n - 2, cache) and reusing previously
return cache[n]
computed values.

for i in range(40):
print(f"Fibonacci({i}) = {fibonacci(i)}")

125
Memoized Fibonacci Runtime

.
.
.
.
.
.

126
Sorting
Algorithms

127
bubblesort in action

https://www.hackerearth.com/practice/algorithms/sorting/bubble-sort/visualize/
128
bubblesort implementation

def bubble_sort(arr):
n = len(arr)
for i in range(n):
for j in range(0, n-i-1):
if arr[j] > arr[j+1]:
arr[j], arr[j+1] = arr[j+1], arr[j]
return arr

arr = [100, 110, 98, 11, 33, 99, 105, 110, 200, 0, 1, 54]
result = bubble_sort(arr)
print(f"{arr}")

129
Can we guess what its time
complexity is?

130
bubblesort time complexity

def bubble_sort(arr):
n = len(arr)
for i in range(n):
for j in range(0, n-i-1):
if arr[j] > arr[j+1]:
arr[j], arr[j+1] = arr[j+1], arr[j]
return arr

arr = [100, 110, 98, 11, 33, 99, 105, 110, 200, 0, 1, 54]
result = bubble_sort(arr)
print(f"{arr}")

131
merge sort

https://www.hackerearth.com/practice/algorithms/sorting/merge-sort/visualize/ 132
merge sort

def merge_sort(arr):
if len(arr) <= 1:
return arr

mid = len(arr) // 2
left_half = arr[:mid]
right_half = arr[mid:]

left_sorted = merge_sort(left_half)
right_sorted = merge_sort(right_half)

return merge(left_sorted, right_sorted)

133
 # Example usage
def merge(left, right): arr = [38, 27, 43, 3, 9, 82, 10]
sorted_array = [] sorted_arr = merge_sort(arr)
left_index = 0 print("Original array:", arr)
right_index = 0 print("Sorted array:", sorted_arr)

while left_index < len(left) and right_index < len(right):

if left[left_index] < right[right_index]:
sorted_array.append(left[left_index])
left_index += 1
else:
sorted_array.append(right[right_index])
right_index += 1

while left_index < len(left):

sorted_array.append(left[left_index])
left_index += 1

while right_index < len(right):

sorted_array.append(right[right_index])
right_index += 1

return sorted_array
134
merge sort time complexity

def merge_sort(arr):
if len(arr) <= 1:
return arr

mid = len(arr) // 2
left_half = arr[:mid]
right_half = arr[mid:]

left_sorted = merge_sort(left_half)
right_sorted = merge_sort(right_half)

return merge(left_sorted, right_sorted) Divide and conquer at

work, giving is O(log n),
and then we have to
merge, which is O(n)

135
what about space complexity?

• Bubblesort is an in place sort,

requiring no memory outside of
the array.

• Mergesort requires an extra bit of

memory to merge the sort.

136
Other sorts to know

• Quick Sort: Efficient, divides and

conquers, uses pivot.
• Heap Sort: Utilizes a heap, sorts in-
place, good worst-case.
• Insertion Sort: Simple, efficient for
small or nearly sorted data.
• Selection Sort: Selects minimum
element, swaps into position.

137
Graph
Algorithms

138
Graphs - Undirected
A

An undirected graph B
has edges that
C
connect vertices
without any direction,
D
E
allowing bidirectional
G
traversal.
F

139
Graphs - Weights
A
2

An undirected B 34
42
weighted graph has
23 C
edges with no 15
10
direction, and each
D 22
E 17
edge has an 11
G
associated weight. 3

140
Graphs - Directed
A

A directed graph has B

edges with specified
C
directions, allowing
traversal in one or both
D
E
ways, without weights.
G

141
Graphs - Cycle
A

A cycle is a path in a B
graph that starts and
C
ends at the same
vertex, forming a loop.
D
E

142
Graphs - Directed with Weights
2 A
34
A directed weighted B
9
graph has edges with 23 42
C
specified directions 15
10
and associated
D 22
E 17
weights, indicating the 11
17 3
G
cost of traversal.
F

143
Trees
A
A tree is a connected
acyclic graph with a B C D

hierarchical structure,
having one root node E F G H I J
and no cycles.
K L M

144
Graph representations
● Use adjacency lists
A
graph = { 'A': ['B', 'C'],
'B': ['A', 'D', 'E'],
'C': ['A', 'F'], B
'D': ['B'],
'E': ['B', 'F'],
'F': ['C', 'E'] }
C
● Or, an adjacency matrix

adj_matrix = [ D
[0, 1, 1, 0, 0, 0], E
[1, 0, 0, 1, 1, 0],
[1, 0, 0, 0, 0, 1],
[0, 1, 0, 0, 0, 0], F
[0, 1, 0, 0, 0, 1],
[0, 0, 1, 0, 1, 0]
]

145
Graph time and space complexity

146
Graph Traversal
The process of visiting all the vertices and
edges in a graph.

● Types of Traversals:
- Depth-First Search (DFS): Explores as far
down a branch as possible before
backtracking.
- Breadth-First Search (BFS): Explores all
neighbors at the present depth before
moving on to nodes at the next depth level.
● Applications:
- Finding connected components
- Pathfinding algorithms
- Solving puzzles and games

147
Depth first search (DFS)
DFS explores as far down a branch as
possible before backtracking, making it ideal
for scenarios requiring exhaustive
pathfinding like maze-solving.

● DFS can be implemented using a explicit or

recursion.
● DFS has a time complexity of O(V+E), where
V is the number of vertices and E is the
number of edges in the graph.
● Used in applications like topological sorting,
cycle detection in graphs, and solving puzzles
with only one solution path, such as mazes.

148
Depth first search (DFS)

B
Visited: []
Stack: [B] C

D
E
F

149
Depth first search (DFS)

B
Visited: [B]
Stack: [D E A] C

D
E
F

150
Depth first search (DFS)

B
Visited: [B D]
Stack: [E A] C

D
E
F

151
Depth first search (DFS)

B
Visited: [B D E]
Stack: [F A] C

D
E
F

152
Depth first search (DFS)

B
Visited: [B D E F]
Stack: [A] C

D
E
F

153
Depth first search (DFS)

B
Visited: [B D E F A]
Stack: [C] C

D
E
F

154
Depth first search (DFS)

B
Visited: [B D E F A C]
Stack: [] C

D
E
F

155
Depth first implementation

def dfs(graph, start, visited=None):

if visited is None:
visited = set()

visited.add(start)
for neighbor in graph.adj_list.get(start, []):
if neighbor not in visited:
dfs(graph, neighbor, visited)

return visited

156
Depth first implementation A

B
graph = {
'A': ['B', 'C'], C

'B': ['D', 'E', 'A'], D

E
'C': ['A'], F
'D': ['B', 'E'],
'E': ['D', 'F'], B D E F A C
'F': ['E']
}

def dfs(graph, start, visited=set()):

visited.add(start)
print(start, end=' ')

for neighbor in graph[start]:

if neighbor not in visited:
dfs(graph, neighbor, visited)

dfs(graph, 'B')
157
Breadth first search (BFS)
BFS explores all neighbors of a node before
moving to the next level, making it ideal for
finding the shortest path in unweighted graphs.

● BFS can be implemented using an explicit

queue to manage the order of node exploration.
● BFS has a time complexity of O(V+E), where V
is the number of vertices and E is the number of
edges in the graph.
● BFS is used in applications like shortest path
finding, level-order traversal of trees, and
solving puzzles with multiple possible paths,
such as word ladders.

158
Breadth first search (BFS)

B
Visited: []
Queue: [B] C

D
E
F

159
Breadth first search (BFS)

B
Visited: [B]
Queue: [A E D] C

D
E
F

160
Breadth first search (BFS)

B
Visited: [B D]
Queue: [A E] C

D
E
F

161
Breadth first search (BFS)

B
Visited: [B D E]
Queue: [F A] C

D
E
F

162
Breadth first search (BFS)

B
Visited: [B D E A]
Queue: [C F] C

D
E
F

163
Breadth first search (BFS)

B
Visited: [B D E A F]
Queue: [C] C

D
E
F

164
Breadth first search (BFS)

B
Visited: [B D E A F C]
Queue: [] C

D
E
F

165
Breadth first implementation A

from collections import deque B

C
def bfs(graph, start):
visited = set() D
E
queue = deque([start]) F

bfs_order = []
graph = {
'A': ['B', 'C'],
while queue: 'B': ['D', 'E', 'A'],
vertex = queue.popleft() 'C': ['A'],
if vertex not in visited: 'D': ['B', 'E'],
visited.add(vertex) 'E': ['D', 'F'],
bfs_order.append(vertex) 'F': ['E']
}
for neighbor in graph[vertex]:
bfs_result = bfs(graph, 'B')
if neighbor not in visited: print("BFS Order:", bfs_result)
queue.append(neighbor)

return bfs_order BFS Order: ['B', 'D', 'E', 'A', 'F', 'C']

166
Dijkstra's Algorithm
● Optimal Path Finding: Efficiently determines
the shortest path in weighted graphs.
● Real-World Applications: Crucial for GPS
navigation, network routing, and logistics.
● Weighted Graphs: Handles varying edge
weights, unlike BFS which only works for
unweighted graphs.
● Efficiency: Uses a greedy approach to build
the shortest path tree incrementally.
● Scalability: Suitable for large graphs with
numerous nodes and edges.
● Algorithm Foundation: Builds understanding
for more complex algorithms like A* and
Bellman-Ford.

167
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A infinity 2 6 D
B infinity
C
8
infinity B
D infinity C
5
pizza infinity 4

168
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A 3 home 2 6 D
B 2 home
C
8
6 home B
D infinity C
5
pizza infinity 4

169
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A 3 home 2 6 D
B 2 home
C
8
6 home B
D infinity C
5
pizza infinity 4

170
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A 3 home 2 6 D
B 2 home
C
8
6 home B
D infinity C
5
pizza 4 C 4

171
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A 3 home 2 6 D
B 2 home
C
8
6 home B
D 13 A C
5
pizza 4 C 4

172
Shortest path to pizza

3 A
Vertices Distance Previous 10
home 0
A 3 home 2 6 D
B 2 home
C
8
6 home B
D 13 A C
5
pizza 4 C 4

173
Dijkstra's Algorithm
import heapq
def dijkstra(graph, start):
priority_queue = [(0, start)]
distances = {vertex: float('inf') for vertex in graph.adj_list}
distances[start] = 0
visited = set()
while priority_queue:
current_distance, current_vertex = heapq.heappop(priority_queue)
if current_vertex in visited:
continue
visited.add(current_vertex)
for neighbor, weight in graph.adj_list.get(current_vertex, []):
distance = current_distance + weight
if distance < distances[neighbor]:
distances[neighbor] = distance
heapq.heappush(priority_queue, (distance, neighbor))
174
return distances
First, let's understand a heap data structure
● Acts as a priority queue
● Like an airport checkin with priority for
class levels (first, business, economy)
● Supports:

- insert() O(log n)
- extractmin and max() O(log n)
- peek() O(1)
- delete() O(1)

175
 Superman arrived and joined the
Heaps in Python priority queue.
Wonder Woman arrived and joined the
import heapq
priority queue.
priority_queue = [] Batman arrived and joined the
priority queue.
heapq.heappush(priority_queue, (2, 'Superman'))
print("Superman arrived and joined the priority queue.")
Priority Queue after adding
heapq.heappush(priority_queue, (1, 'Wonder Woman')) superheroes:
print("Wonder Woman arrived and joined the priority queue.") [(1, 'Wonder Woman'), (2,
heapq.heappush(priority_queue, (3, 'Batman')) 'Superman'), (3, 'Batman')]
print("Batman arrived and joined the priority queue.")
Wonder Woman is performing a task
print("\nPriority Queue after adding superheroes:") and leaves the queue.
print(priority_queue) Superman is performing a task and
highest_priority_hero = heapq.heappop(priority_queue) leaves the queue.
print(f"\n{highest_priority_hero[1]} is performing a task and leaves the queue.")
Spider-Man arrived and joined the
next_priority_hero = heapq.heappop(priority_queue) priority queue.
print(f"{next_priority_hero[1]} is performing a task and leaves the queue.")
heapq.heappush(priority_queue, (4, 'Spider-Man')) Priority Queue after adding another
print("\nSpider-Man arrived and joined the priority queue.") superhero:
print("\nPriority Queue after adding another superhero:") [(3, 'Batman'), (4, 'Spider-Man')]
print(priority_queue) Batman is performing a task and
leaves the queue.
while priority_queue: Spider-Man is performing a task and
hero = heapq.heappop(priority_queue)
print(f"{hero[1]} is performing a task and leaves the queue.")leaves the queue.
print("\nPriority Queue is now empty.") 176
Dijkstra's Algorithm
import heapq
def dijkstra(graph, start):
priority_queue = [(0, start)]
distances = {vertex: float('inf') for vertex in graph.adj_list}
distances[start] = 0
visited = set()
while priority_queue:
current_distance, current_vertex = heapq.heappop(priority_queue)
if current_vertex in visited:
continue
visited.add(current_vertex)
for neighbor, weight in graph.adj_list.get(current_vertex, []):
distance = current_distance + weight
if distance < distances[neighbor]:
distances[neighbor] = distance
heapq.heappush(priority_queue, (distance, neighbor))
return distances
177
Greedy Algorithms

● Build solutions step-by-step, always choosing the

immediate local optimum with the hope of finding a
global optimum.
● Simple, efficient, suitable for optimization problems
where locally optimal choices lead to a global optimum.
● Work best with greedy-choice property and optimal
substructure; not always guaranteed to find the global
optimum.
● Dijkstra’s Algorithm finds shortest paths in weighted
graphs efficiently, essential for applications like GPS
navigation and network routing protocols.
● Greedy algorithms, especially Dijkstra’s, are crucial for
solving various optimization problems efficiently in
computer science and real-world scenarios.

178
The return of the MYTH
of the
Tower of Hanoi

179
Tower of Hanoi Rules
Objective

• The goal is to move the entire stack of disks from the initial peg
(A) to the destination peg (C), following the rules.

Rules

• Move One Disk at a Time: Only one disk can be moved at a time.
• Top Disk Only: Only the top disk of any peg can be moved.
• No Larger Disk on Smaller Disk: A larger disk cannot be placed
on top of a smaller disk.

180
181
182
183
184
185
186
187
188
Implementing Tower of Hanoi
def tower_of_hanoi(n, source, target, auxiliary):
if n == 1:
print(f"Move disk 1 from {source} to {target}")
return

tower_of_hanoi(n - 1, source, auxiliary, target)

print(f"Move disk {n} from {source} to {target}")

tower_of_hanoi(n - 1, auxiliary, target, source)

n = 3 # Number of disks
tower_of_hanoi(n, 'A', 'C', 'B')

Move disk 1 from A to C

Move disk 2 from A to B
Move disk 1 from C to B
Move disk 3 from A to C
Move disk 1 from B to A
Move disk 2 from B to C
Move disk 1 from A to C

189
What is its time complexity?
Every call to tower_of_hanoi doubles the
computation until we reach the base case

def tower_of_hanoi(n, source, target, auxiliary):

if n == 1:
print(f"Move disk 1 from {source} to {target}")
return

tower_of_hanoi(n - 1, source, auxiliary, target)

print(f"Move disk {n} from {source} to {target}")

tower_of_hanoi(n - 1, auxiliary, target, source)

c a ll d o u b le s our work by
Every
alling
recursively c
noi again
tower_of_ha
190
How long do we have?

191
 How long do we have?

In 5 billion years the Sun will exhaust the

hydrogen in its core, causing the core to contract
and heat up. The outer layers will expand and
cool, turning the Sun into a red giant, which will
consume the earth.
192
Wrap up

193
Wrapping things up, a cheat sheet
e
Constant tim
No loops
hms
Code with one loop Linear algorit

Code with nested loops Bubble sort

h
Each loop halves work Binary searc

s
Each loop halves work nqu er style algo
Divide and co
with combined results

Combinatorial, or ial, Tower of Hanoi,

Fact
permutations , Shortest pat
h
194
There's so much more…

● String Algorithms - Linear Programming (Simplex,

● Sorting Algorithms Interior-Point Methods)
- Merge Sort ● Advanced Data Structures
- Quicksort - Binary Trees
- Many More - AVL Trees, Red-Black Trees
● More Graph Algorithms - Suffix Trees, Tries
- A* Algorithm ● Mastering Recursion
- Bellman-Ford Algorithm - Tail Recursion
- Network Flow Algorithms - Recursion vs. Iteration
● Algorithm Design Paradigms - Memoization Techniques
- Advanced Dynamic ● Intractable Problems
Programming - NP-Completeness

195
Where to go next

196
Where to go next

197
 Where to go next

Coming in 2025

198
Thank you
199