1.

1 The Different Types of Data

Intro
- Data type – a way of categorizing data
- Data type conveys:
o The allowed values for a piece of data
o The allowed operations we can perform on a piece of data

Numeric Data
- A natural number is a value from the set {0, 1, 2, …}
o Denoted by symbol ℕ
o In computer science, 0 is a natural number
- An integer is a value from the set {…, -2, -1, 0, 1, 2, …}
o Denoted by symbol ℤ
𝑝
- A rational number is a value from the set {𝑞 | 𝑝, 𝑞 ∈ ℤ 𝑎𝑛𝑑 𝑞 ≠ 0}
o Denoted by symbol ℚ
- An irrational number is a number with an infinite and non-repeating decimal expansion
o i.e. pi, e
o Denoted by symbol ℚ
- A real number is either a rational or irrational number
- Denoted by symbol ℝ

Operations on Numeric Data

- Standard arithmetic operations
- Standard comparisons for equality and inequality
- Modulo operator that produces the remainder when one integer is divided by another
o Denoted by symbol %
o a%b means “the remainder when a is divided by b”

Boolean Data
- A Boolean is value from the set {True, False}
o Yes/No question

Operations on Boolean Data

- Booleans can be combined using logical operators
- Three most common logical operators:
o Not – reverses the value of a Boolean
i.e. “not True” is False
 o And – takes two Boolean values and produces True when both of the values are
True, and False otherwise
i.e. “True and False” is False, “True and True” is True
o Or – takes two Boolean values and produces True when at least one of the
values is True, and False otherwise
i.e. “True or False” is True, “False or False” is False

Textual Data
- A string is an ordered sequence of characters, it’s used to represent text

Writing Textual Data

- Strings are surrounded by single quotes to differentiate them from surrounding text
o e.g. ‘David’
- Empty string – a string that has zero characters
o Denoted by ‘’ or 𝜖

Operations on Textual Data

- |s|
o String length/size
o Returns the number of characters in s
- s1 = s2
o String equality
o Returns whether s1 and s2 have the same characters, in the same order
- s+t
o String concatenation
o Returns a new string consisting of the characters of s followed by the characters
of t
If s1 represents the string ‘Hello’ and s2 represents the string ‘Goodbye’,
then s1 + s2 is the string ‘HelloGoodbye’
- s[i]
o String indexing
o Returns the i-th character of s, where indexing starts at 0
s[0] returns the first character of s, s[1] returns the second, etc.

Set Data (Unordered Distinct Values)

- Set – an unordered collection of zero or more distinct values, called its elements
o i.e. the set of all people in Toronto; the set of words of the English language; etc.
Writing Sets
- Two different ways:
o Writing each element of the set within the braces, separated by commas
i.e. {1, 2, 3} or {‘hi’, ‘bye’}
o Using set builder notation, in which we define the form of elements of a set using
variables
𝑝
i.e. {𝑞 | 𝑝, 𝑞 ∈ ℤ 𝑎𝑛𝑑 𝑞 ≠ 0}
- Empty set – a set having zero elements
o Denoted by {} or ∅

Operations on Set Data

- |A|
o Returns the size of set A
i.e. the number of elements in A
- 𝑥∈𝐴
o Returns True when x is an element of A
- 𝑦∉𝐴
o Returns True when y is not an element of A
- 𝐴⊆𝐵
o Returns True when every element of A is also in B
o A is a subset of B
o A set A is a subset of itself, and the empty set is a subset of every set
- A=B
o Returns True when A and B contain the exact same elements
- 𝐴∪𝐵
o The union of A and B
o Returns the set consisting of all elements that occur in A, in B, or in both
o 𝐴 ∪ 𝐵 = {𝑥 | 𝑥 ∈ 𝐴 𝑜𝑟 𝑥 ∈ 𝐵}
- 𝐴∩𝐵
o The intersection of A and B
o Returns the set consisting of all elements that occur in both A and B
o 𝐴 ∩ 𝐵 = {𝑥 | 𝑥 ∈ 𝐴 𝑎𝑛𝑑 𝑥 ∈ 𝐵}
- A\B
o The difference of A and B
o Returns the set consisting of all elements that are in A but that are not in B
o 𝐴 \ 𝐵 = {𝑥 | 𝑥 ∈ 𝐴 𝑎𝑛𝑑 𝑥 ∉ 𝐵}
- 𝐴×𝐵
o The (Cartesian) product of A and B
 o Returns the set consisting of all pairs (a, b) where a is an element of A and b is an
element of B
o 𝐴 × 𝐵 = {(𝑥, 𝑦) | 𝑥 ∈ 𝐴 𝑎𝑛𝑑 𝑦 ∈ 𝐵}
- P(A)
o The power set of A
o Returns the set consisting of all subsets of A
o i.e. if A = {1, 2, 3}, then P(A) = {∅, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}
o 𝑃(𝐴) = {𝑆 | 𝑆 ⊆ 𝐴}

List Data (Ordered Values)

- List – an ordered collection of zero or more (possibly duplicated) values, called its
elements
- Used instead of set when the elements of the collection should be in a specified order,
or if it may contain duplicates
o i.e. the list of all people in Toronto, ordered by age; the list of words of the
English language, ordered alphabetically; etc.

Writing Lists
- Written with square brackets enclosing zero or more values separated by commas
o i.e. [1, 2, 3]
- Empty list – a list having zero elements
o Denoted by []

Operations on List Data

- |A|
o Returns the size of A
i.e. the number of elements in A (counting all duplicates)
- 𝑥∈𝐴
o same meaning as for sets
- A=B
o A and B have the same elements in the same order
- A[i]
o List indexing
o Returns the i-th element of A, where the indexing starts at 0
i.e. A[0] returns the first element of A, A[1] returns the second, etc.
- A+B
o List concatenation
o Returns a new list consisting of the elements of A followed by the elements of B
 i.e. [1, 2, 3] + [2, 4, 6] = [1, 2, 3, 2, 4, 6]
o Similar to set union, but duplicates are kept, and order is preserved

Mapping Data
- Mapping – an unordered collection of pairs of values
- Each pair consists of a key and an associated value
- The keys must be unique, but the values can be duplicated
- A key cannot exist without a corresponding value
- Used to represent associations between two collections of data
o i.e. a mapping from the name of a country to its GDP; a mapping from student
number to name; etc.

Writing Mappings
- Curly braces are used to represent a mapping
o Similar to sets as both mappings and sets are unordered and both have a
uniqueness constraint (a set’s elements; a mapping’s keys)
- Each key-value pair in a mapping is written using a colon, with the key on the left side of
the colon and its associated value on the right
- i.e. {‘fries’ : 5.99, ‘steak’ : 25.99, ‘soup’ : 8.99}

Operations on Mappings
- |M|
o Returns the size of the mapping M
i.e. the number of key-value pairs in M
- M=N
o Returns whether two mappings are equal
i.e. when they contain exactly the same key-value pairs
- 𝑘∈𝑀
o Returns whether k is a key contained in the mapping M
- M[k]
o When k is a key in M, this operation returns the value that corresponds to k in
the mapping M

Extra
- Images can be represented as a list of integers
o Each element in the list corresponds to a dot called pixel on the screen
o For each dot, three integer values are used to represent three colour channels:
red, green, and blue
 o We can add these channels together to get a very wide range of colours
Called the RGB colour model

1.2 Introducing the Python Programming Language

What is a Programming Language?

- A programming language is a way of communicating a set of instructions to a computer
- A program is the text of instructions we wish to instruct the computer to execute
o We call this text program code to differentiate it from other forms of text
- Two key properties of the language:
o Syntax – rules governing what constitutes a valid program in the language
Specifies format of instructions
o Semantics – rules governing the meaning of different instructions in the
language
Specifies what the computer should do for each instruction

The Python Programming Language

- Python interpreter takes programs written in the Python language and execute the
instructions
- Two ways of writing code in Python to be understood by the interpreter
o Write full programs in Python, saving them as text files
Python programs use the .py file extension
Writing the instructions, and then run them with the interpreter
o Run the Python interpreter in an interactive mode, which we call the Python
console or Python shell
We can write small fragments of Python code and ask the Python
interpreter to execute each fragment one at a time
Useful for experimenting and exploring the language, as we get feedback
after every single instruction
Drawback: interactions with the interpreter in the Python console are
ephemeral, lost every time you restart the console
- In the course, we will use the Python console to learn about and experiment with the
Python language, and write full programs in .py files

1.3 Representing Data in Python

The Python Console

- When we start the Python console, we see <<<
 o The text <<< is the Python prompt
The console is “prompting” us to type in some Python code to execute
- Expression – a piece of Python code that produces a value
- Evaluation – the act of calculating the value of an expression
- Expression 4 + 5 is formed from 2 smaller expressions – numbers 4 and 5
- Literal – simplest kind of Python expression
o The piece of code that represents the exact value as written
o i.e. 4 is an integer literal literally representing the number 4

Numeric Data in Python (int, float)

- Two data types for representing numeric data: int and float
- int – integer
o int literal is simply an integer
o i.e. 110; -3421
- 2 ** 5: 2 to the power of 5
- Comment – code ignored by the Python interpreter, begin comment with “#”
- Python follows the “BEDMAS” rule
- Python always produce an int value for addition, subtraction, multiplication, and
exponentiation
- Python has 2 different forms of division
- Floor division/Integer division – operator //, x / y rounded down to the nearest integer
o i.e. 6 // 2 outputs 3; 15 // 2 outputs 7; -15 // 2 outputs -8
- Division – operator /, returns a float type
- float – represents arbitrary real numbers
o float literal is written as a sequence of digits followed by a decimal point (.), then
another sequence of digits
o i.e. 2.5; .123; 1000.00000001
- 2 ** 0.5 calculates the square root of 2, however, it is only an approximation of sqrt2,
which does not equal it
o Since sqrt2 is an irrational number, its decimal expansion cannot be represented
in any finite amount of computer memory
- float cannot always represent real numbers exactly
- 3 vs. 3.0
o 6 // 2 outputs 3; 6 / 2 outputs 3.0
o When x and y are ints,
x // y always evaluates to an int
x / y always evaluates to a float
- Mixing ints and floats
 o For two ints x and y, x + y, x – y, x * y, x // y, and x ** y all produce ints; x / y
always produces a float
o For two floats, all six of the operations above produce a float
o An arithmetic operation that is given one int and one float always produces a
float
o Even in long expressions where only 1 value is a float, the whole expression will
evaluate to a float
- Operation description
a + b Returns the sum of a and b
a – b Returns the result of subtraction of b from a
a * b Returns the result of multiplying a by b
a / b Returns the result of dividing a by b
a % b Returns the remainder when a is divided by b
a ** b Returns the result of a being raised to the power of b
a // b Returns the floored division a / b
- Comparison operators
a == b Returns whether a and b are equal
a != b Returns whether a and b are not equal (opposite of ==)
a > b Returns whether a is less than the value of b
a < b Returns whether a is less than the value of b
a >= b Returns whether a is greater than or equal to b
a <= b Returns whether a is less than or equal to the value of b
- Python can recognize when the values of ints and floats represent the same number
o i.e. 4 == 4.0 outputs True

Boolean Data in Python (bool)

- Two literal values of type bool: True and False
- 3 boolean operators we can perform on boolean values:
o not
o and
o or
o i.e. not True outputs False; True and True outputs True; True and False outputs
False; False or True outputs True; False or False operates False
- The or operator in Python is inclusive, which produces True when both of its operand
expressions are True
- Boolean operator expressions can be combined with itself or the arithmetic comparison
operators
o i.e. True and (False or True) outputs True; (3 == 4) or (5 < 10) outputs False
Textual Data in Python (str)
- str – short for “string”
- A str literal is a sequence of characters surrounded by single-quotes (‘)
o i.e. ‘Foundations of Computer Science I’
- We can compare string using ==
o i.e. ‘David’ == ‘David’ outputs True; ‘David’ == ‘david’ outputs False
- String indexing – extracting a single character from a string
o Starts at 0 – s[0] is the first character in s
o i.e. ‘David’[0] outputs ‘D’; ‘David’[3] outputs ‘I’
- Concatenation – operator +, combines strings
o i.e. ‘One string’ + ‘to rule them all.’ outputs ‘One stringto rule them all’; “One
string’ + ‘ to rule them all.’ outputs ‘One string to rule them all’
- String repetition – operator *, repeats string
o i.e. ‘David’ * 3 outputs ‘DavidDavidDavid’
- The above string operations can be nested within each other
o i.e. (‘David’ + ‘Mario’) * 3 outputs ‘DavidMarioDavidMarioDavidMario’

Set Data in Python (set)

- Python uses the set data type to store set data
- set literal begins with a { and ends with a }, and each element of the list is written inside
the braces, separated from each other by commas
o i.e. {1, 2, 3} is a set of ints; {1, 2.0, ‘three’} is a set of elements of mixed types
- Sets can be compared for equality using ==
o Order does not matter in sets
o i.e. {1, 2, 3} == {3, 1, 2} outputs True
- in – equivalent to ∈
o i.e. 1 in {1, 2, 3} outputs True; 10 in {1, 2, 3} outputs False
- not in – equivalent to ∉
o i.e. 1 not in {1, 2, 3} outputs False; 10 not in {1, 2, 3} outputs True

List Data in Python (list, tuple)

- Two different data types to store list data: list and tuple
- list literals are written the same way as set literals, except using square brackets []
instead of curly braces {}
- Lists support the same operations for strings and sets
o [1, 2, 3] == [1, 2, 3] outputs True; [1, 2, 3] == [3, 2, 1] outputs False
o ([‘David’, ‘Mario’])[0] outputs ‘David’
 o [‘David’, ‘Mario’] + [‘Jacqueline’, ‘Diane’] outputs [‘David’, ‘Mario’, ‘Jacqueline’,
‘Diane’]
o 1 in [1, 2, 3] outputs True
- tuple literals are written using regular parentheses ()
o The above examples for list work the same way for tuples
o () instead of []

Mapping Data in Python (dict)

- dict – short for “dictionary”, data type for mapping data
- dict literals are similar to sets, with each key-value pair separated by a colon
o i.e. {‘fries’: 5.99, ‘steak’: 25.99, ‘soup’: 8.99}
In this dictionary, the keys are strings, the values are floats
- The literal {} represents an empty dictionary (not an empty set)
o Python has no literal to represent an empty set
- Dictionaries support equality comparison using ==
- They also support key lookup using the same syntax as string and list indexing
o i.e. ({‘fries’: 5.99, ‘steak’: 25.99, ‘soup’: 8.99})[‘fries’] outputs 5.99
- in – checking whether a key is present in dictionary
o i.e. ‘fries’ in {‘fries’: 5.99, ‘steak’: 25.99, ‘soup’: 8.99} outputs True

1.4 Storing Data in Variables

Variables
- Variable – a piece of code that refers to a value
- We create variables in Python using the syntax:
o <variable> = <expression>
Called an assignment statement
When we execute an assignment statement, it doesn’t produce a value –
it instead defines a variable
- Expression – a piece of Python code that is evaluated to produce a value
- Python executes an assignment statement in 2 steps:
1. The expression on the right side of the = is evaluated, producing a value
2. That value is bound to the variable on the left side
- After the assignment statement is executed, the variable may be used to refer to the
value

Choosing Good Variable Names

- To remember the purpose for each variable
 - Rules for choosing variable names:
o Use only lowercase letters, digits, and underscores
i.e. distance1 (not Distance1)
o Write each word in lowercase and separate them with an underscore
i.e. total_distance = distance1 _ distance2 (not totaldistance or
totalDistance)
o Avoid single-letter variable names and non-standard acronyms/abbreviations
i.e. total_distance (not td)
A second person might not understand what td stands for

The Value-Based Python Memory Model

- Memory model – a structured way of representing variables and data in a program
o Memory refers to computer memory
o Uses a table to represent the associations between variables and their
associated values
o i.e.
Variable Value
distance1 2.265409645965010650498645056
distance2 10.0

1.5 Build Up Data with Comprehensions

From Set Builder Notation to Set Comprehensions

- Set builder notation – a concise way of defining a mathematical set
o S = {1, 2, 3, 4, 5}
o A set of squares of the elements of S: {𝑥 2 | 𝑥 ∈ 𝑆}
- Set comprehension – set builder notation translated into Python
o Syntax: {<expr> for <variable> in <collection>}
expr – expression
- Set comprehension is evaluated by taking the <expr> and evaluating it once for each
value in <collection> assigned to the <variable>
o Analogous to the set builder notation, using for instead of | and in instead of ∈
- Example
o numbers = {1, 2, 3, 4, 5}
o {x ** 2 for x in numbers}
o Output: {1, 4, 9, 16, 25}
{x ** 2 for x in numbers} == {1 ** 2, 2 ** 2, 3 ** 2, 4 ** 2, 5 ** 2}
• Replacing x with 1, 2, 3, 4, and 5
 - We can use set comprehensions with a Python list as well
o i.e. {x ** 2 for x in [1, 2, 3, 4, 5]} outputs {1, 4, 9, 16, 25}
- Set comprehensions can be used with any “collection” data type in Python

List and Dictionary Comprehensions

- List comprehension – similar to a set comprehension, but used square brackets []
instead of curly braces {}
o Syntax: [<expr> for <variable> in <collection>]
<collection> can be a set or a list
o i.e. [x + 4 for x in {10, 20, 30}] outputs [14, 24, 34]; [x * 3 for x in [100, 200, 300]]
outputs [300, 600, 900]
- Do not assume a particular ordering of the elements when a list comprehension
generates elements from a set
- Dictionary comprehension – similar to a set comprehension, but specifies both an
expression to generate keys and an expression to generate their associated values
o Syntax: {<key_expr> : <value_expr> for <variable> in <collections>}
o Example of a dictionary comprehension that creates a “table of values” for the
function f(x) = x2 + 1:
{x : x ** 2 + 1 for x in {1, 2, 3, 4, 5}} outputs {1: 2, 2: 5, 3: 10, 4: 17, 5: 26}

Comprehensions with Multiple Variables

- Cartesian product of two sets:
o 𝐴 × 𝐵 = {(𝑥, 𝑦) | 𝑥 ∈ 𝐴 𝑎𝑛𝑑 𝑦 ∈ 𝐵}
The expression (x, y) is evaluated once for every possible combination of
elements x of A and elements y of B
- We can specify additional variables in a comprehension by adding extra for <variable> in
<collection> clauses to the comprehension
o nums1 = {1, 2, 3}
o nums2= {10, 20, 30}
o {(x, y) for x in nums1 for y in nums2}
o Output: {(3, 30), (2, 20), (2, 10), (1, 30), (3, 20), (1, 20), (3, 10), (1, 10), (2, 30)}
Sets are unordered
- If we have a comprehension with clauses for v1 in collection1, for v2 in collection2, etc.,
then the comprehension’s inner expression is evaluated once for each combination of
values for the variables

2.1 Python’s Built-In Functions

Functions in Mathematics
- Let A and B be sets, a function f: A B is a mapping from elements in A to elements in B
o A is called the domain of the function
o B is called the codomain of the function
- Functions can have more than 1 input
o 𝑓: 𝐴1 × 𝐴2 × … × 𝐴𝑘 → 𝐵
o Functions that take one, two, and three inputs are unary, binary, and ternary
functions, respectively
i.e. the addition operator + is a binary function (ℝ × ℝ → ℝ) that takes 2
real numbers and returns their sum

Python’s Built-In Functions in Python

- Built-in functions – functions we can use to perform additional operations defined by
Python
o They are made automatically available to us anywhere in a Python program
- Function call – a Python expression that uses a funciton to operate on a given input
o Syntax: <function_name> (<argument>, <argument>, …)
o i.e. abs(-10) outputs 10
- Arguments – the input expresssions in a function call expression
o i.e. the -10 above
- When we evaluate a function call, the arguments are passed to the function
o i.e. -10 is passed to abs
- When a function call produces a value, we say that the functin call returns a value, and
refer to this value as the return value of the function call expression
o i.e. 10
- Examples of built-in functions
o len – takes a string or a collection data type (e.g. set, list) and returns the size of
its input
i.e. len({10, 20, 30}) outputs 3
o sum – takes a collection of numbers (e.g. a set or list whose elements are all
numbers) and returns the sum of the numbers
i.e. sum({10, 20, 30}) outputs 60
o sorted – takes a collection and returns a listy that contains the same elemenmts
as the input collection, sorted in ascending order
i.e. sorted({10, 3, 20, -4}) outputs [-4, 3, 10, 20]
o max – when it is called with two or more numeric inputs, returns the largest one;
when it is called with a collection of numbers, returns the largest number in the
collection
 i.e. max(2, 3) outputs 3; max({2, 3}) outputs 3
o range – takes 2 integers start and stop and returns a value representing a range
of consecutive numbers between start and stop – 1, inclusive
i.e. range(5, 10) represents 5, 6, 7, 8, 9; range(10, 5) represents an empty
sequence
- One special function: type
o Takes any Python value and returns its type
o type(3) outputs <class ‘int’>
o type(3.0) outputs <class ‘float’>
o type(‘David’) outputs <class ‘str’>
o type([1, 2, 3]) outputs <class ‘list’>
o type({‘a’: 1, ‘b’: 2}) outputs <class ‘dict’>
- Nested function calls
o We can write function calls within each other
i.e. max(abs(-100), 15, 3*20) outputs 100
o Too much nesting can make Python expressions difficult to read and understand
o It is better to break down a complex series of function calls into intermediate
steps using variables

Methods: Functions Belonging to Data Types

- The built-in functions above can all be given arguments of at least two different data
types
- Method – a function that is defined as part of a data type
o All methods are functions but not vice versa
i.e. the built-in functions are not methods
- Top-level functions – functions that are not methods
- str.lower – takes a string and returns a new string with all uppercase letters turned into
lowercase
o i.e. str.lower(‘David’) outputs ‘david’
- To call a method, we refer to it by first specifying the name of the data type it belongs to
(i.e. str), followed by a period (.), and then the name of the method (lower)
- str.split – splits a string into words
o i.e. str.split(‘David wuz hear’) outputs [‘David’, ‘wuz’, hear’]
- set.union – performs the setr union operation
o i.e. set.union({1, 2, 3}, {2, 10, 20}) outputs {1, 2, 3, 20, 10}
- list.count – counts the number of times a value appears in a list
o i.e. list.count([1, 2, 3, 1, 2, 4, 2], 2) outputs 3
2.2 Defining Our Own Functions

Defining a Python Function

- A “squaring” function in Python
o def square(x: float) -> float:
“””Return x squared.

>>> square(3.0)
9.0
>>> square(2.5)
6.25
“””
return x ** 2
- Function header – the first line, def square(x: float) -> float:
o Conveys the following pieces of information:
The function’s name (square)
The number and type of arguments the function expects
• Parameter – a variable in a function definition that refers to an
argument when the function is called
• The function has one parameter with name x and type float
The function’s return type
• The type following the -> (float)
o Syntax for a function header for a unary function:
def <function_name> (<parameter_name>: <parameter_type>) ->
<return_type>:
We choose the name square rather than f as the function name
We use data types to specify the function domain and codomain
• x: float specifies that the parameter x must be a float value
• -> float specifies that this function always return a float value
Type contract – the domain-codomain restriction in an analogous way to
𝑓: ℝ → ℝ
• float -> float
- Function docstring – the next lines that start and end with triple-quotes (“””)
o Another way of writing a comment in Python
o Text that is meant to be read by humans, but not executed as Python code
o Goal: to communicate what the function does
o First part of the docstring, Return x squared, is an English description of the
function
 o Second part of the docstring looks like Python code
The first example: “when you type square(3.0) into the Python console,
9.0 is returned
The second example : “when you type square(2.5) into the Python
console, 6.25 is returned
Doctest examples – refers to the above examples
o The function docstring is indented inside the function header, as a visual
indicator that it is part of the overall function definition
- Body – the final line, return x ** 2
o Code that is executed when the function is called
o Also indented like the function docstring
o Uses keyword return, which signals the return statement
Form of return statement: return <expression>
o When a return statement is executed,
1. The <expression> is evaluated, producing a value
2. That value is then returned to wherever the function was called
• No more code in the function body is executed after this point

What Happens When a Function is Called?

- Suppose we’ve defined square as above, and then call it in the Python console:
>>> square(2.5)
- When we press Enter, the Python interpreter evaluates the function call by doing the
following:
1. Evaluate the argument 2.5, and then assign 2.5 to the function parameter x
2. Evaluate the body of the square function, by doing:
a. First evaluate x ** 2, which is 6.25 (since x refers to the value 2.5)
b. Then stop executing the function body, and return the value 6.25 back to
the Python console
3. The function call square(2.5) evaluates to 6.25, and this is displayed on the
screen
- When we call square twice in the same expression, i.e.
>>> square(2.5) + square(-1.0)
1. Python evaluates the operands to + in left-to-right order, so evaluate square(2.5)
first
a. Evaluate 2.5, and then assign 2.5 to the function parameter x
b. Evaluate the body of the square function, by doing:
i. First evaluate x ** 2, which is 6.25 (since x refers to 2.5)
 ii. Then stop executing the function body, and return the value 6.25
back to the Python console
2. Nothing is displayed yet. There’s still square(-1.0) to be evaluated
a. Evaluate -1.0, and then assign -1.0 to the function parameter x
b. Evaluate the body of the square function, by doing:
i. First evaluate x ** 2, which is 1.0 (since x refers to -1.0)
ii. Then stop executing the function body, and return the value 1.0
back to the Python console
3. Now the expression to evaluate has been simplified to 6.25 + 1.0, which
evaluates to 7.25. This value is displayed on the screen

Defining Functions in Files

- We save functions in files so that we can reuse them across multiple sessions in the
Python console (and in other files)

Defining Functions with Multiple Parameters

- Distance formula: 𝑑 = √(𝑥1 − 𝑥2 )2 + (𝑦1 − 𝑦2 )2
- When we write the function above in Python, the function will take 2 inputs:
o Each input is a tuple of two floats, representing the x- and y-coordinates of each
point
- Function header and docstring:
o def calculate_distance(p1: tuple, p2: tuple) -> float:
“””Return the distance between points p1 and p2

p1 and p2 are tuples of the form (x, y), where the x- and y-coordinates
are points.

>>> calculate_distance((0, 0), (3.0, 4.0))

5.0
“””
- In order to use the above formula, we need to extract the coordinates from each point
x1 = p1[0]
y1 = p1[1]
x2 = p2[0]
y2 = p2[1]
- After having the four coordinates, we can apply the formula and return the result
return ((x1 – x2) ** 2 + (y1 – y2) ** 2) ** 0.5
Function Reuse
- For the above function body, we can reuse the square function defined above instead of
** 2
o def calculate_distance(p1: tuple, p2: tuple) -> float:
“””Return the distance between points p1 and p2.

p1 and p2 are tuples of the form (x, y), where the x- and y-coordinates
are points.

>>> calculate_distance((0, 0), (3.0, 4.0))

5.0
“””
x1 = p1[0]
y1 = p1[1]
x2 = p2[0]
y2 = p2[1]
return (square(x1 – x2) + square(y1 – y2)) ** 0.5
- It is essential to organize our code into different functions as our programs grow larger

2.3 Local Variables and Function Scope

Intro
- A function call can only access its own variables, but not variables defined within other
functions

Example 1: Introducing Local Variable Scope

- Example:

def square(x: float) -> float:

“””Return x squared.

>>> square(3.0)
9.0
>>> square(2.5)
6.25
“””
return x ** 2
 - The parameter x is a variable that is assigned a value based on when the function was
called
o This variable cannot be accessed from outside the body
- Local variable – variable limited to the function body
o i.e. x
- Scope – places in the code where a variable can be accessed
o A local variable of a function is a variable whose scope is the body of that
function
- Example:
>>> n = 10.0
>>> result = square(n + 3.5)
o 13.5 is assigned to the parameter x
o Incorrect memory model diagram:
Variable Value
n 10.0
x 13.5
o We group the variables together based on whether they are introduced in the
Python console or inside a function:
_main_ (console) square
Variable Value Variable Value
n 10.0 x 13.5

o We use the name _main_ to label the table for variables defined in the Python
console
o Inside the body of square, the only variable that can be used is x
At the point that the body of square is evaluated, only the “square” table
in the memory model is active
o Outside in the Python console, the only variable that can be used is n
After square returns and we’re back to the Python console, the “square”
table is no longer accessible, and only the _main_ table is active

Example 2: Duplicate Variable Names

- Suppose we modify our example above to use x instead of n in the Python console:
>>> x = 10.0
>>> result = square(x + 3.5)
o This does not modify the x variable in the Python console
They are different variables though they shared the same name
_main_ (console) square
 Variable Value Variable Value
x 10.0 x 13.5

Example 3: (Not) Accessing Another Function’s Variables

- Example consisting two functions:

def square(x: float) -> float:

“””Return x squared.

>>> square(3.0)
9.0
>>> square(2.5)
6.25
“””
return x ** 2

def square_of_sum(numbers: list) -> float:

“””Return the square of the sum of the given numbers.”””
total = sum(numbers)
return square(total)

o Calling the function:

>>> nums = [1.5, 2.5]
>>> result = square_of_sum(nums)
>>> result
16.0

Right before Right before square is Right before square

square_of_sum is called called (from returns
(from console) square_of_sum)
_main_ _main_ _main_
Variable Value Variable Value Variable Value
nums [1.5, 2.5] nums [1.5, 2.5] nums [1.5, 2.5]
square_of_sum square_of_sum
Variable Value Variable Value
numbers [1.5, 2.5] numbers [1.5, 2.5]
total 4.0 total 4.0
 square
Variable Value
x 4.0
o The list [1.5, 2.5] is passed from the console to square_of_sum, and the number
4.0 is passed from square_of_sum to square
o Though the value 4.0 is passed from total to x, it would not work calling total in
square
o Even if square of sum is still active, total cannot be accessed from square
- Python’s rule for local scope: a local variable can only be accessed in the function body it
is defined
o Prevents us from accidentally using a variable from a completely different
function when working on a function

2.4 Importing Modules

Intro
- Modules – Python code files
- The modules are not automatically loaded

The Import Statement

- Import statement – what we use to load a Python module
o Syntax: import <module_name>
- i.e. to load the math module, we type
>>> import math
- To access the function, we use dot notation
>>> math.log2(1024)
10.0
- Call the built-in function dir on the module to see a list of functions and other variables
defined in the module
o i.e. dir(math)

The Datetime Module

- Provides not just functions but new data types for representing time-based data
- date – a data type that represents a specific date
o i.e.
>>> import datetime
>>> canada_day = datetime.date(1867, 7, 1) # Create a new date
>>> type(canada_day)
 <clad ‘datetime.date’>
>>> term_start = datetime.date(2020, 9, 10)
>>> datetime.date.weekday(term_start) # Return the day of the week of
the date
3 # 0 = Monday, 1 = Tuesday, etc.
o Note the double use of dot notation
datetime.date is the date type being accessed, and .weekday accesses a
method of the data type
- We can compare dates for equality using == and chronological order (e.g. < for
comparing one date comes before another)
- We can also subtract dates
o i.e.
>>> term_start - canada_day
datetime.timedelta(days=55954)
o The difference between two dates is an instance of the datetime.timedelta data
type, which is used to represent an interval of time

2.5 The Function Design Recipe

The Function Design Recipe by Example

1. Writing example uses
a. A good name for a function is a short answer to the question “What does the
function do?”
b. Write 1 or 2 examples of calls to the function and the expected return values
i. Include an example of a standard case (as opposed to a tricky case)
c. Put the examples inside a triple-quoted string that is indented
d. i.e.
“””
>>> is_even(2)
True
>>> is_even(17)
False
“””
2. Write the function header
a. Above the docstring
b. Choose a meaningful name for each parameter
c. Include the type contract (the types of the parameters and return value)
d. i.e.
 def is_even(value: int) -> bool:
“””
>>> is_even(2)
True
>>> is_even(17)
False
“””
3. Write the function description
a. Add a description of what the function does before the examples
b. Make sure the purpose of each parameter is clear
c. Describe the return value
d. i.e.
def is_even(value: int) -> bool:
“””Return whether value is even.

>>> is_even(2)
True
>>> is_even(17)
False
“””
4. Implement the function body
a. Indent it to match the docstring
b. Review the examples and consider how we determined the return values
c. i.e.
def is_even(value: int) -> bool:
“””Return whether value is even.

>>> is_even(2)
True
>>> is_even(17)
False
“””
return value % 2 == 0
5. Test the function
a. Test all examples
i. Try with some tricky cases as well
b. Test by calling it in the Python console
 c. If we encounter any errors/incorrect return values, make sure that our tests are
correct
d. Go back to Step 4 and try to identify and fix any possible errors in the code
i. Called debugging

The Importance of Documenting our Functions

- The Function Design Recipe places a large emphasis on developing a precise and
detailed function header and docstring before writing any code for the function body
- Forces us to write out the function header, docstring, and examples reinforces our
understanding about what we need to do
- The function header and docstring serve as documentation for the function
o Communicating to others and our future self what the function is supposed to do

2.6 Testing Functions I: Doctest and Pytest

Doctests: Basic Examples in Docstrings

- Manual testing – requires human interaction to complete
o i.e. importing the function into the Python console, and then manually evaluate
each doctest example one at a time and compare the output with the expected
output in the docstring
- Use the Python library doctest, which can automatically extract doctest examples from
docstrings and convert them into runnable tests
- To use doctest, add the following code to the bottom of any Python file
if __name__ == ‘__main__’:
import doctest # import the doctest library
doctest.testmod() # run the tests
o We will receive a report about which tests failed
- In order to use doctest, our docstring examples must be correctly formatted and valid
Python code

Creating Test Suites with Pytest

- We use pytest to write our tests in a separate file, thus including an exhaustive set of
tests without cluttering our code files
- i.e. suppose we have defined the following function in a files trues.py

# In files trues.py

def has_more_trues(booleans: list) -> bool:

 “””Return whether booleans contain more True

>>> has_more_trues([True, False, True])

True
>>> has_more_trues([True, False, False])
False
“””
# Function body omitted
- Write tests in a new file and name it test_trues.py
- Unit test – a block of code that checks for the correct behaviour of a function for one
specific input
- Test suite – a collection of tests that check the behaviour of a function or (usually small)
set of functions
- Every test file contains a test suite
- In Python, we express a unit test as a function whose name starts with the prefix:
test_
- The body of the function contains an assert statement, which is used to check whether
some boolean expression is True or False
- i.e.

# In file test_trues.py

from trues import has_more_trues

def test_mixture_one_more_true() -> None:

“””Test has_more_trues on a list with a mixture of True and False, with one
more True than False.
“””
assert has_more_trues([True, False, True])

def test_mixture_one_more_false() -> None:

“””Test has_more_trues on a list with a mixture of True and False, with one
more False than True.
“””
assert not has_more_trues([True, False, False])
- The unit test functions are similar to functions we’ve defined previously, with a few
differences:
 o Each test name and docstring documents what the test is by describing the test
input
o The return type of the test function is None, which is a special type that indicates
that no value at all is returned by the function.
There is no return statement – instead, there’s an assert
- An assert statement with the form assert <expression> does the following when
executed:
1. First, it evaluates <expression>, which should produce a boolean value
2. If the value is True, nothing else happens, and the program continues onto the
next statement
3. If the value is False, an AssertionError is raised. This sygnals to Pytest that the
test has failed.
- When pytest “runs” a unit test, it calls a test function like test_mixture_one_more_true
o If the function call ends without raising an AssertionError, the test passes
o If the function call does raise an AssertionError, the test fails
- A single unit test function can contain multiple assert statements
- To run our unit test functions with pytest, we need to first import pytest and then call a
specific test function

# At the bottom of test_trues.py

if __name__ == ‘__main__’:
import pytest
pytest.main([‘test_trues.py’])

2.7 Type Conversion Functions

Intro
- We can convert values between different data types
- i.e.
>>> int(‘10’)
10
>>> float(‘10’)
10.0
>>> bool(1000)
True
>>> bool(0)
False
 >>> list({1, 2, 3})
[1, 2, 3]
>>> set([1, 2, 3])
{1, 2, 3}
>>> dict([(‘a’, 1), (‘b’, 2), (‘c’, 3’)])
{‘a’: 1, ‘b’: 2, ‘c’: 3}
- Every value of the data types we’ve studied so far has a string representation
o i.e.
>>> str(10)
‘10’
>>> str(True)
‘True’
>>> str([1, 2, 3])
‘[1, 2, 3]’

Warning: Conversion Errors

- Not all values of one type can be converted into another
- i.e.
>>> int(‘David’)
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
ValueError: invalid literal for int() with base 10: ‘David’
>>> list(1000)
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
TypeError: ‘int’ object is not iterable

Preview: Creating Values of Arbitrary Data Types

- range
o We could call range to create a sequence of numbers
o When calling range by itself, this happens
>>> range(5, 10)
range(5, 10) # rather than [5, 6, 7, 8, 9]
o range is a type conversion function
Python has a range data type that is distinct from lists
o i.e.
>>> five_to_nine = range(5, 10)
>>> type(five_to_nine)
 <class ‘range’>
>>> five_to_nine == [5, 6, 7, 8, 9]
False
- datetime.date
o Example:
>>> import datetime
>>> canada_day = datetime.date(1867, 7, 1) # Create a new date
>>> type(canada_day)
<clad ‘datetime.date’>
o The data type datetime.date is called on 3 arguments instead of 1
o datetime.date is called to create a new date value given three arguments (the
year, month, and day)
- We’ll be able to take any data type – even ones we defined ourselves – and create
values of that type by calling the data type as a function

2.8 Application: Representing Text

Intro
- Numbers represent textual data via functions
- Bit – what we call each 0 or 1
- ASCII – a function with domain {0, 1, …, 127} whose codomain is the set of all possible
characters
o Length-7 sequences of bits
Can represent 27 = 128 different characters
o The function is one-to-one, meaning no two numbers map to the same character
o This standard covered all English letters (lowercase and uppercase), digits,
punctuation, and various others (e.g. to communicate a new line)
i.e. 65 is mapped to ‘A’ and 126 is mapped to ‘~’, etcl
- Computer scientists extended ASCII from length-7 to length-8 sequences of bits, and
hence its domain increased to size 256 ({0, 1, …, 255}
o Allowed “extended ASCII” to support some other characters used in similar Latin-
based languages
i.e. ‘é’ (233), ‘ō’ (246), ‘©’ (169), etc.
- The latest standard, Unicode, uses up to 32 bits that gives us a domain of {0, 1, …, 232-1},
over 4 billion different numbers
o This number is larger than the number of distinct characters in use across all
different languages
o There are several unused numbers in the domain of Unicode
 o The unused numbers are being used to map to emojis
An emoji may appear as a different emoji on another device
o The process involves submitting a proposal for a new emoji and computer
scientists supporting newly approved emojis by updating their software

Python’s Unicode Conversion Functions

- Python has 2 built-in functions that implement the (partial) mapping between
characters and their Unicode number
- ord – takes a single-character string and returns its Unicode number as an int
o i.e.
>>> ord(‘A’)
65
>>> ord(‘é’)
233
>>> ord(‘猿’)
29503
- chr – computes the inverse of ord
o Given an integer representing a Unicode number, chr returns a string containing
the corresponding character
o i.e.
>>> chr(65)
‘A’
>>> chr(233)
‘é’
>>> chr(29503)
‘猿’
- String ordering comparisons (<, >) are based on Unicode numeric values
o i.e. the Unicode value of ‘Z’ is 90 and the Unicode value of ‘a’ is 97
>>> ‘Z’ < ‘a’
True
>>> ‘Zebra’ < ‘animal’
True
- Sorting a collection of strings can seem alphabetical, but treats lowercase and
uppercase letters differently
o i.e.
>>> sorted({‘David’, ‘Mario’, ‘Jacqueline’})
[‘David’, ‘Jacqueline’, ‘Mario’]
>>> sorted({‘david’, ‘Mario’, ‘Jacqueline’})
 [‘Jacqueline’, ‘Mario’, ‘david’]

3.1 Propositional Logic

Propositions
- Propositional logic – an elementary system of logic that is a crucial building block
underlying other
- Proposition – a statement that is either True or False
o i.e. 2 + 4 = 6; 3 – 5 > 0; Python’s implementation of list.sort is correct on every
input list; etc.
- Propositional variables – variables that represent propositions
o By conventions, propositional variable names are lowercase letters starting at p
- Propositional/Logical operator – an operator whose arguments must all be either True
or False
- Propositional formula – an expression that is built up from propositional variables by
applying the propositional operators

The Basic Operators NOT, AND, OR

- NOT – unary operator, also called “negation”, denoted by the symbol ¬
o Negates the truth value of its input
o i.e. if p is True, then ¬p is False, and vice versa
- AND – binary operator, also called “conjunction”, denoted by the symbol ⋀
o Returns True when both its arguments are True
- OR – binary operator, also called “disjunction”, denoted by the symbol ⋁
o Returns True if one or both of its arguments are True
o The inclusive or

The Implication Operator

- Implication – represented by the symbol ⇒
o The implication p ⇒ q asserts that whenever p is True, q must also be True
Hypothesis – p
Conclusion – q
o Vacuous truth – p is False but p ⇒ q is True
o Formula p ⇒ q has 2 equivalent formulas
The formula is only False when p is True and q is False
Equivalent – the two formulas have the same truth values
• i.e. the formulas will either both be True or both be False
¬𝑝 ⋁ 𝑞
 • If p is False then p ⇒ q is True
• If p is True then q must be True as well
• Only False when p is True and q is False
¬𝑞 ⇒ ¬𝑝
• If q doesn’t occur, then p cannot have occurred either
• Only False when p is True and q is False
• Contrapositive for p ⇒ q
o Converse – the implication q ⇒ p
Obtained by switching the hypothesis and conclusion
Not logically equivalent to the original implication
- In Python, there is no operator or keyword that represents implication directly
o To express an implication in Python, use the first equivalent form from above
Write p ⇒ q as ¬𝑝 ⋁ 𝑞

Biconditional (“If and Only If”)

- Biconditional – denoted by 𝑝 ⟺ 𝑞
o Returns True when p ⇒ q and its converse q ⇒ p are both True
o Abbreviation for p ⇒ q ⋀ q ⇒ p
o “if p then q, and if q the p”
o “p if and only if q”
o “p iff q”
- In Python, we use == to determine whether two boolean values are the same

Summary
Operator Notation English Python Operation
NOT ¬𝑝 p is not true not p
AND 𝑝⋀𝑞 p and q p and q
OR 𝑝⋁𝑞 p or q (or both) p or q
Implication 𝑝⇒𝑞 if p, then q not p or q
Biconditional 𝑝⟺𝑞 p if and only if q p == q

3.2 Predicate Logic

Intro
- Predicate – a statement whose truth value depends on 1 or more variables from any set
o Codomain of the function: {True, False}
o Use uppercase letters starting from P to represent predicates
o i.e. P(x) is defined to be the statement “x is a power of 2”
 P(8) is True and P(7) is False
- Predicates can depend on more than 1 variable
o i.e. Q(x, y) means “x2 = y”
Q(5, 25) is True since 52 = 25, but Q(5, 24) is False
- We must always give the domain of a predicate as part of its definition
o i.e. P(x): “x is a power of 2,”, where 𝑥 ∈ ℕ

Quantification of Variables
- A predicate by itself does not have a truth value
o i.e. “x is a power of 2” is neither True nor False, since we don’t know that value
of x
o Setting x = 8 in the statement, the statement is now True
- Most of the times, we care about some aggregation of the predicate’s truth values over
all elements of its domain
o i.e. “Every real number x satisfies the inequality 𝑥 2 − 2𝑥 + 1 ≥ 0” makes a claim
about all possible values of x
- Quantifier – modifies a predicate by specifying how a certain variable should be
interpreted
- Existential quantifier
o Existential quantifier – written as ∃
“There exists an element in the domain that satisfies the given predicate”
i.e. ∃𝑥 ∈ ℕ, 𝑥 ≥ 0
“There exists a natural number x that is greater than or equal to 0.”
• True because when x = 1, x ≥ 0
o There has to be at least 1 element of the domain satisfying the predicate
Doesn’t say exactly how many elements do
o ∃𝑥 ∈ 𝑆 – a big OR that runs through all possible values for x from the domain S
i.e. for the previous example,
(0 ≥ 0) ∨ (1 ≥ 0) ∨ (2 ≥ 0) ∨ …
- Universal quantifier
o Universal quantifier – written as ∀
“Every element in the domain satisfies the given predicate”
i.e. ∀𝑥 ∈ ℕ, 𝑥 ≥ 0
“Every natural number x is greater than or equal to 0”
• True because the smallest natural number is 0 itself
i.e. ∀𝑥 ∈ ℕ, 𝑥 ≥ 10 is False
o ∀𝑥 ∈ 𝑆 – a big AND that runs through all possible values of x from S
i.e. for the first example,
 (0 ≥ 0) ∧ (1 ≥ 0) ∧ (2 ≥ 0) ∧ …
- Example: We define Loves(a, b) is a binary predicate that is True whenever person a
loves person b
o A = {Ella, Patrick, Malena, Breanna}
o B = {Laura, Stanley, Thelonious, Sophia}
o A line between 2 people indicates that the person on the left loves the person on
the right
Brenna Sophia
Malena Thelonious
Patrick Stanley
Ella Laura
o Consider the following statements:
∃𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, Thelonious) means “there exists someone in A who
loves Thelonious”
• True (Malena loves Thelonious)
∃𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, Sophia) means “there exists someone in A who loves
Sophia”
• False (no one loves Sophia)
∀𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, Stanley) means “every person in A loves Stanley”
• True (all 4 people in A loves Stanley)
∀𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, Thelonious) means “every person in A loves
Thelonious”
• False (Ella does not love Thelonious)

Python Built-ins: Any and All

- Any – built-in function that represent logical statements using the existential quantifier
o Takes a collection of boolean values and returns True when there exists a True
value in the collection
o i.e.
>>> any([False, False, True])
True
>>> any([]) # An empty collection has no True values
False
o i.e. suppose we’re given a set of strings S and wish to determine whether any of
them start with the letter ‘D’,
>>> strings = [‘Hello’, ‘Goodbye’, ‘David’]
>>> any([s[0] == ‘D’ for s in strings])
True
 o Parallels between mathematical statements and Python expressions:
∃ corresponds to calling the any function
𝑠 ∈ 𝑆 corresponds to for s in strings
s[0] = ‘D’ corresponds to s[0] == ‘D’
- All – built-in function that can be used as a universal quantifier
o Takes a collection values and evaluates to True when every element has the
value True
o i.e. to express ∀𝑠 ∈ 𝑆, s[0] = ‘D’ in Python,
>>> strings = [‘Hello’, ‘Goodbye’, ‘David’]
>>> all([s[0] == ‘D’ for s in strings])
False
- Since Python is limited on the size of collections, we cannot easily express existential
statement quantified over infinite domains like ℕ or ℝ

Writing Sentences in Predicate Logic

- A general formula in predicate logic is built up using the existential and universal
quantifiers, the propositional operators, and arbitrary predicates
- Sentence – a formula with no unquantified variables
o i.e.
the formula ∀𝑥 ∈ ℕ, 𝑥 2 > 𝑦 is not a sentence as y is not quantified
the formula ∀𝑥, 𝑦 ∈ ℕ, 𝑥 2 > 𝑦 is a sentence
- Avoid using commas
o “Does the comma mean ‘and’ or ‘then’?”
i.e. P(x), Q(x)
o Never use the comma to connect propositions
Use ∧ and ⟹
o English is too ambiguous for fields of computer science
i.e. “or” can be inclusive or exclusive
o Where to use comma:
Immediately after a variable quantification, or separating two variables
with the same quantification
Separating arguments to a predicate
i.e. ∀𝑥, 𝑦 ∈ ℕ, ∀𝑧 ∈ ℝ, 𝑃(𝑥, 𝑦) ⟹ 𝑄(𝑥, 𝑦, 𝑧)

Manipulating Negation
- Given any formula, we can state its negation by preceding it by a ¬ symbol
o i.e. ¬(∀𝑥, 𝑦 ∈ ℕ, ∀𝑧 ∈ ℝ, 𝑃(𝑥, 𝑦) ⟹ 𝑄(𝑥, 𝑦, 𝑧))
o Hard to understand if we try to transliterate each part separately
 - Given a formula using negations, we apply some simplification rules to “push” the
negation symbol closer to the individual predicates:
o ¬(¬𝑝) becomes 𝑝
o ¬(𝑝 ∨ 𝑞) becomes (¬𝑝) ∨ (¬𝑞)
o ¬(𝑝 ∧ 𝑞) becomes (¬𝑝) ∧ (¬𝑞)
o ¬(𝑝 ⟹ 𝑞) becomes 𝑝 ∧ (¬𝑞)
o ¬(𝑝 ⟺ 𝑞) becomes (𝑝 ∧ (¬𝑞)) ∨ ((¬𝑝) ∧ 𝑞))
o ¬(∃𝑥 ∈ 𝑆, 𝑃(𝑥)) becomes ∀𝑥 ∈ 𝑆, ¬𝑃(𝑥)
o ¬(∀𝑥 ∈ 𝑆, 𝑃(𝑥)) becomes ∃𝑥 ∈ 𝑆, ¬𝑃(𝑥)

3.3 Filtering Collections

Expressing Conditions in Predicate Logic

- Consider the statement: “Every natural number n greater than 3 satisfies the inequality
n2 + n ≥ 20.”
o “Greater than 3” is a condition that modifies the statement, limiting the original
domain of n (the natural numbers) to a smaller subset (the natural numbers
greater then 3)
o Two ways to represent such conditions in predicate logic:
Define a new set
• i.e. 𝑆1 = {𝑛 | 𝑛 ∈ ℕ 𝑎𝑛𝑑 𝑛 > 3}, ∀𝑛 ∈ 𝑆1 , 𝑛 2 + 𝑛 ≥ 10
Use an implication to express the condition
• Rewrite the original statement using an “if … then …” structure:
“For every natural number n, if n is greater than 3 then n satisfies
the inequality n2 + n ≥ 20.
• Translating the above into predicate logic:
∀𝑛 ∈ ℕ, 𝑛 > 3 ⟹ 𝑛 2 + 𝑛 ≥ 20
• 𝑛 > 3 ⟹ has a filtering effect, due to the vacuous truth case of
implication
o i.e. For the values n ∈ {0, 1, 2}, the hypothesis of the
implication is False, and so for these values the implication
itself is True
• Since the overall statement is universally quantified, these
vacuous truth cases don’t affect the truth value of the statement
- The “forall-implies” structure arises naturally when a statement is universally quantified

Filtering Collections in Python

 - Filter operation – an operation that takes a collection of data and returns a new
collection consisting of the elements in the original collection that satisfy some
predicate
o {<expression> for <variable> in <collection> if <condition>}
o This form of set comprehension behave the same way as the ones we studied
last chapter, except that <expression> only gets evaluated for the values of the
variable that make the condition evaluate to True
o i.e.
>>> numbers = {1, 2, 3, 4, 5} # Initial collection
>>> {n for n in numbers if n > 3} # Pure filtering: only keep elements > 3
{4, 5}
>>> {n * n for n in numbers if n > 3} # Filtering with a data transformation
{16, 25}
- By combining filtering comprehensions with aggregation functions, we can limit the
scope of an aggregation
o i.e.
>>> numbers = {1, 2, 3, 4, 5}
>>> sum({n for n in numbers if n % 2 == 0}) # Sum of only the even
numbers
6
- The keyword if used in this syntax is connected to our use of implication
o i.e. ∀𝑛 ∈ 𝑁, 𝑛 > 3 ⟹ 𝑛^2 + 𝑛 ≥ 20 in Python:
>>> numbers = {1, 2, 3, 4, 5, 6, 7, 8}
>>> all({n ** 2 + n >= 20 for n in numbers if n > 3})
True

3.4 Conditional Execution

The if Statement
- If statement – compound statement that expresses conditional execution of code
o Compound statement – contains other statements within it
o Syntax:
if <condition>:
<statement>
…
else:
<statement>
…
o If condition – the <condition> following if that evaluates to a boolean
Analogous to the hypothesis of an implication
o If branch – statements under the if
o Else branch – statements under the else
o When an if statement is executed,
The if condition is evaluated, producing a boolean value
If the condition evaluates to True, then the statements in the if branch
are executed.
If the condition evaluates to False, then the statements in the else branch
are executed instead
o i.e.
def get_status(scheduled: int, estimated: int) -> str:
“””Return the flight status for the given scheduled and estimated
departure times

The times are given as integers between 0 and 23 inclusive,

representing the hour of the day.

The status is either ‘On time’ or ‘Delayed’.

>>> get_status(10, 10)

‘On time’
>>> get_status(10, 12)
‘Delayed’
“””
o If we only need to calculate a bool for whether the flight is delayed, simply
return estimated <= scheduled
o We use if statements to execute different code based on these cases:
i.e.
def get_status(scheduled: int, estimated: int) -> str:
“””…”””
if estimated <= scheduled:
return ‘On time’
else:
return ‘Delayed’
This uses the boolean expression we identified earlier to trigger different
return statements to return the correct string
Code with More Than Two Cases
- i.e. whenever a flight is delayed by more than 4 hours, the airline cancels the flight
def get_status_v2(scheduled: int, estimated: int) -> str:
“””Return the flight status for the given scheduled and estimated
departure times

The times are given as integers between 0 and 23 inclusive, representing

the hour of the day.

The status is either ‘On time’, ‘Delayed’, or ‘Cancelled’

>>> get_status_v2(10, 10)

‘On time’
>>> get_status_v2(10, 12)
‘Delayed’
>>> get_status_v2(10, 15)
‘Cancelled”
“””
- We can express subcases using nested if statements
o i.e.
def get_status_v2(scheduled: int, estimated: int) -> str:
“””…”””
if estimated <= scheduled:
return ‘On time’
else:
if estimated – scheduled <= 4:
return ‘Delayed’
else:
return ‘Cancelled’
- Excessive nesting of statements can make code difficult to read and understand
- elif – short for “else if”
o Syntax:
if <condition1>:
<statement>
…
elif <condition2>:
<statement>
…
 … # [any number of elif conditions and branches]
else:
<statement>
…
o When executed,
The if condition (<condition1>) is evaluated, producing a boolean value
If the condition evaluates to True, then the statements in the if branch
are executed. If the condition evaluates to False, the next elif condition is
evaluated, producing a boolean
If that condition evaluates to True, then the statements in that elif branch
are executed. If that condition evaluates to False, then the next elif
condition is evaluated. This step repeats until either one of the elif
conditions evaluate to True, or all of the elif conditions have evaluated to
False
If neither the if condition nor any of the elif conditions evaluate to True,
then the else branch executes
o i.e.
def get_status_v3(scheduled: int, estimated: int) -> str:
“””…”””
if estimated <= scheduled:
return ‘On time’
elif estimated – scheduled <= 4:
return ‘Delayed’
else:
return ‘Cancelled’
Equivalent to the nested version but easier to read

Testing All the Branches

- We need to design our unit tests so that each possible execution path is used at least
once
- White box testing – testing where we “see through the box” and therefore can design
tests based on the source code itself
- Black box testing – tests created without any knowledge of the source code
o No knowledge of the different paths the code can take
- In our doctests for get_status_v3, we chose three different examples, each
corresponding to a different possible case of the if statement
- Code coverage – the percentage of lines of program code that are executed when a set
of tests are used on that program
 o Metric used to access the quality of tests

Building on Our Example

- i.e. write a function that determines how many flights are cancelled in a day. We are
provided with a dictionary in the form {flight_number: [scheduled, estimated]}
>>> flights = {‘AC110’: [10, 12], ‘AC321”: [12, 19], ‘AC999’: [1, 1]}
>>> flights[‘AC110’]
[10, 12]
o We can call our get_status_v3 function for flights
i.e.
>>> flight_ac110 = flights[‘AC110’]
>>> get_status_v3(flight_ac110[0], flight_ac110[1])
‘Delayed’
o Instead of specifying the flight number ourselves, we could substitute in
different flight numbers based on our data using comprehensions
i.e.
>>> {k for k in flights}
{‘AC299’, ‘AC110’, AC321’}
>>> {get_status_v3(flights[k][0], flights[k][1] == ‘Cancelled’ for k in
flights)}
{False, True}
>>> [get_status_v3(flights[k][0], flights[k][1] == ‘Cancelled’ for k in
flights)]
[False, True, False]
The first set comprehension does not tell us if the flight was cancelled
For the second set comprehension we could see that there was at least 1
flight cancelled
The list comprehension tells us that exactly 1 out of 3 flights were
cancelled
o We could combine the first set comprehension with the second
i.e.
>>> {k for k in flights if get_status_v3(flights[k][0], flights[k][1] ==
‘Cancelled’)}
{‘AC321’}
>>> [k for k in flights if get_status_v3(flights[k][0], flights[k][1] ==
‘Cancelled’)]
[‘AC321’]
o To convert the set into an integer, use the built-in len function on the set
 i.e.
def count_cancelled(flights: dict) -> int:
“””Return the number of cancelled flights for the given
flight data.

flights is a dictionary where each key is a flight ID, and

whose corresponding value is a list of 2 numbers, where
the first is the scheduled departure time and the second is
the estimated departure time.

>>> count_cancelled({‘AC110’: [10, 12], ‘AC321”: [12, 19],

‘AC999’: [1, 1]})
1
“””
cancelled_flights = {k for k in flights if
get_status_v3(flights[k][0], flights[k][1] == ‘Cancelled’)}
return len(cancelled_flights)

3.5 Simplifying If Statements

Computing Booleans: When if Statements Aren’t Necessary

- In cases where we need a boolean value, it is often simpler to write an expression to
calculate the value directly, rather than using if statements
- i.e.
def is_even(n: int) -> bool:
“””Return whether n is even (divisible by 2).”””
if n % 2 == 0:
return True
else:
return False
can be simplified to
def is_even(n: int) -> bool:
“””…”””
return n % 2 == 0
- i.e. a more complex example
def mystery(x: lst, y: lst) -> bool:
if x == []:
if y == []:
 return True
else:
return False
else:
if y = []:
return False
else:
return True
simplify the nested if statements:
def mystery(x: lst, y: lst) -> bool:
if x == []:
return y == []
else:
return y != []
combining the cases,
def mystery(x: lst, y: lst) -> bool:
return (x == [] and y == []) or (x != [] and y != [])
for readability,
def mystery(x: lst, y: lst) -> bool:
both_empty = x == [] and y == []
both_non_empty = x != [] and y != []
return both_empty or both_non_empty

Using if Statements
- Prefer using a sequence of elifs rather than nested if statements
- Write conditions from most specific to most general
o Order matters for conditions, since they are checked one at a time in top-down
order

3.6 if __name__ == ‘__main__’

Import Statements
- Allows the program that executes the import statement to access the functions and
data types defined within that module
- By default, all statements in the imported module are executed, not just function and
data type definitions
- Without the if statement, all the doctests inside the imported modules will be run, even
though they are not relevant for a program that just wants to use the imported modules
Enter __name__
- __name__ - a special variable for each module when a program is run
o Double underscore denotes special variable or function names
o i.e. the __name__ attribute of math is ‘math’
>>> import math
>>> math.__name__
‘math’
- When we run a module, the Python interpreter overrides the default module __name__
and instead sets it to the special string ‘__main__’
- Checking the __name__ variable is a way to determine if the current module is being
run, or whether it’s being imported by another module
- if __name__ == ‘__main__’ – “Execute the following code if this module is being run,
and ignore the following code if this module is being imported by another module”

Organizing a Python File

- Main block – all the code under if __name__ == ‘__main__’
- The only code that goes outside of the main block are:
o Import statements (import …)
o Function definitions (def …)
o Data type definitions (class …)
- Other code, like code for running doctest or pytest, goes inside the main block so that it
is only executed when the module is run, and not when it is imported
- The main block goes at the bottom of the module

3.7 Function Specifications

Function Specifications and Correctness

- A specification for a function consists of 2 parts:
o A description of what values the function takes as valid inputs
We can represent this description as a set of predicates, where a valid
input must satisfy all these predicates
Preconditions – how we call these predicates
o A description of what the function returns/does, in terms of its inputs
We can represent this description as a set of predicates that must all be
satisfied by the return value of the function
Postconditions – how we call these predicates
- A function implementation is correct when the following holds:
 o For all inputs that satisfy the specification’s preconditions, the function
implementation’s return value satisfies the specification’s postconditions
- The person implementing the function needs to make sure that the code correctly
returns or does what the specification says
o They do not need to worry about exactly how the function will called, and can
assume that the function’s input is always valid
- The person calling the function needs to make sure that they call the function on valid
inputs
o They do not need to worry about exactly how the function is implemented, and
can assume that the function works correctly

Simple Specifications
- Example:
def is_even(n: int) -> bool:
“””Return whether n is even.

>>> is_even(1)
False
>>> is_even(2)
True
“””
# Body omitted
- The function’s type contract and description forms a complete specification of this
function’s behaviour:
o The type annotation of the parameter n tells us that the valid inputs to is_even
are int values
This type annotation int specifies a precondition of the function
o The type annotation for the return value tells us that the function will always
return a bool
The function description Return whether n is even completes the
specification by indicating how the return value is based on the input
These specify postconditions of the function
- is_even is implemented correctly when for all ints n, is_even(n) returns a bool that is
True when n is even, and False when n is not even
- If this happens:
>>> is_even(4)
False
o It’s the implementer’s fault
 - If this happens and an error occurs
>>> is_even([1, 2, 3])
o It’s the caller’s fault

Preconditions in General
- Consider this function:
def max_length(strings: set) -> int:
“””Return the maximum length of a string in the set of strings.

>>> max_length({‘Hello’, ‘Mario, ‘David Liu’})

9
“””
return max({len(s) for s in strings})
- When the set is empty, there would be an error
o The implementer is at fault
The only description of “valid inputs” given is the type annotation set
• An empty set is a set
o We can add additional precondition to the function docstring:
def max_length(strings: set) -> int:
“””Return the maximum length of a string in the set of strings.

Preconditions:
len(strings) > 0
“””
return max({len(s) for s in strings})
Now, when we call max_length(empty_set), and receive an error, it is our
fault for violating the precondition
- Checking preconditions automatically with python_ta
o Preconditions can be turned into executable Python code
Use an assert statement as follows:
def max_length(strings: set) -> int:
“””Return the maximum length of a string in the set of
strings.

Preconditions:
len(strings) > 0
“””
 assert len(strings) > 0, ‘Precondition violated: max_length
called on an empty set’
return max({len(s) for s in strings})
• Now, precondition is checked every time the function is called,
with a meaningful error message when the precondition is
violated
o We can also use the python_ta library to check preconditions
if __name__ == ‘__main__’:
import python_ta.contracts
python_ta.contracts.DEBUG_CONTRACTS = False # Disable
contract debug messages
python_ta.contracts.check_all_contracts()

max_length(set())
The function we import, check_all_contracts, takes the function type
contract and any preconditions it finds in the function docstring, and
causes the function to check the preconditions every time the function is
called
o check_all_contracts also checks the return type of each function

Preconditions as Assumptions and Restrictions

- Preconditions place restrictions on the user of the function
o The onus is on them to respect these preconditions
- Besides ruling out the invalid input with a precondition, we can explicitly define some
alternate function behaviour for this input:
def max_length(strings: set) -> int:
“””Return the maximum length of a string in the set of strings.

Return 0 if strings is empty

“””
if strings == set():
return 0
else:
return max({len(s) for s in strings})
o Result:
>>> empty_set = set()
>>> max_length(empty_set)
0
3.8 The typing Module and Richer Type Annotations

Intro
- Recall the function max_length
def max_length(strings: set) -> int:
“””Return the maximum length of a string in the set of strings.

Preconditions:
len(strings) > 0
“””
return max({len(s) for s in strings})
o >>> max_length ({1, 2, 3}) outputs an error despite the fact that our inputs are
valid

Typing’s Collection Types

- There are 4 collection types that we can import from the typing module
Type Description
Set[T] A set whose elements all have type T
List[T] A list whose elements all have type T
Tuple[T1, A tuple whose first element has type T1, second element has type
T2, …] T2, etc.
A dictionary whose keys are of type T1 and whose values are of type
Dict[T1, T2]
T2
- i.e.
o {‘hi’, bye’} has type Set[str]
o [1, 2, 3] has type List[int]
o (‘hello’, True, 3.4) has type Tuple[str, bool, float]
o {‘a’: 1, ‘b’: 2, ‘c’: 3} has type Dict[str, int]
- Improving max_length with the typing module
from typing import Set

def max_length(strings: Set[str]) -> int:

“””Return the maximum length of a string in the set of strings.

Preconditions:
len(strings) > 0
 “””
return max({len(s) for s in strings})
- General collections
o The above typing module’s collection types are not needed when:
We don’t care what’s in the list
We want a list with elements of different types

3.9 Working with Definitions

Intro
- Definitions – what we use to express a long idea using a single term
- Let 𝑛, 𝑑 ∈ ℤ. We say that d divides n, or n is divisible by d, when there exists a 𝑘 ∈ ℤ
such that 𝑛 = 𝑑𝑘
o In this case, we use the notation 𝑑 | 𝑛 to represent “d divides n”
o | is a binary predicate
i.e. 3 | 6 is True; 4 | 10 is false
o This definition permits 𝑑 = 0
When 𝑑 = 0, 𝑑 | 𝑛 if and only if 𝑛 = 0
- Expressing the statement “For every integer x, if x divides 10, then it also divides 100”
o With the predicate:
∀𝑥 ∈ ℤ, 𝑥 | 10 ⟹ 𝑥 | 100
o Without the predicate
Replace every instance of 𝑑 | 𝑛 with ∃𝑘 ∈ ℤ, 𝑛 = 𝑑𝑘
∀𝑥 ∈ ℤ, (∃𝑘 ∈ ℤ, 10 = 𝑘𝑥) ⟹ (∃𝑘 ∈ ℤ, 100 = 𝑘𝑥)
Each subformula in the parentheses has its own 𝑘 variable, whose scope
is limited by the parentheses
To emphasize their distinctness,
• ∀𝑥 ∈ ℤ, (∃𝑘1 ∈ ℤ, 10 = 𝑘1 𝑥) ⟹ (∃𝑘2 ∈ ℤ, 100 = 𝑘2 𝑥)
- Let 𝑝 ∈ ℤ. We say 𝑝 is prime when it is greater than 1 and the only natural numbers that
divide it are 1 and itself
- Define a predicate 𝐼𝑠𝑃𝑟𝑖𝑚𝑒(𝑝) to express the statement that “𝑝 is a prime number”
o First part: “greater than 1”
o Second part: “if a number 𝑑 divides 𝑝, then 𝑑 = 1 or 𝑑 = 𝑝”
o 𝐼𝑠𝑃𝑟𝑖𝑚𝑒(𝑝) ∶ 𝑝 > 1 ∧ (∀𝑑 ∈ ℕ, 𝑑 | 𝑝 ⇒ 𝑑 = 1 ∨ 𝑑 = 𝑝), where 𝑝 ∈ ℤ
o To express the idea without using divisibility predicate,
𝐼𝑠𝑃𝑟𝑖𝑚𝑒(𝑝) ∶ 𝑝 > 1 ∧ (∀𝑑 ∈ ℕ, (∃𝑘 ∈ ℤ, 𝑝 = 𝑘𝑑) ⇒ 𝑑 = 1 ∨ 𝑑 = 𝑝),
where 𝑝 ∈ ℤ
Expressing Definitions in Programs
- Consider the divisibility predicate |, where 𝑑 | 𝑛 means ∃𝑘 ∈ ℤ, 𝑛 = 𝑘𝑑 (for 𝑑, 𝑛 ∈ ℤ)
- Without using the modulo operator %, it is challenging to translate the mathematical
definition of divisibility precisely into a Python function
o We cannot represent infinite sets in a computer program
- We can use a property of divisibility to restrict the set of numbers to quantify over:
o When 𝑛 ≠ 0, every number that divides 𝑛 must lie in the range {−|𝑛|, −|𝑛| +
1, … , |𝑛| − 1, |𝑛|}
- In Python, we represent the above set using the range data type:
o possible_divisors = range(- abs(n), abs(n) + 1)
- After we replace ℤ with possible_divisors, we can now translate the definition into
Python code
def divides(d: int, n: int) -> bool:
“””Return whether d divides n.”””
possible_divisors = range(- abs(n), abs(n) + 1)
return any({n == k * d for k in possible_divisors}))
- We can also translate the prime number definition in Python:
def is_prime(p: int) -> bool:
“””Return whether p is prime.”””
possible_divisors = range(1, p+1)
return (
p > 1 and
all({d == 1 or d == p for d in possible_divisors if divides(d, p)})
)

Divisibility and the Remainder Operation

- We can check whether n is divisible by 2 by checking whether then remainder when n is
divided by 2 is 0 or not
def is_even(n: int) -> bool:
“””Return whether n is even.”””
return n % 2 == 0
- ∀𝑛, 𝑑 ∈ ℤ, 𝑑 ≠ 0 ⇒ (𝑑 | 𝑛 ⇔ 𝑛 % 𝑑 = 0)
o When 𝑑 ≠ 0, the remainder 𝑛 % 𝑑 is undefined
- To account for 𝑑 = 0,
def divides2(d: int, n: int) -> bool:
“””Return whether d divides n.”””
if d == 0:
return n == 0
 else:
return n % d == 0
- divides2 is more efficient than divides though they are logically equivalent
o Performs fewer calculations
o No comprehensions
- To speed up our implementation of is_prime, we can call divides3 instead of divides

3.10 Testing Functions II: hypothesis

Property-Based Testing
- Property-based testing – a single test that consists of a large set of possible inputs that is
generated in a programmatic way
- Property-based tests use assert statements to check for properties that the function
being tested should satisfy
- Possible properties that every output of the function should satisfy:
o The type of the output
The function str should always return a string
o Allowed values of the output
The function len should always return an integer that is greater than or
equal to zero
o Relationships between input and output
The function max(x, y) should return something that is greater than or
equal to both x and y
o Relationships between two (or more) input-output pairs
“For any two lists of numbers nums1 and nums2, we know that
sum(nums1 + nums2) == sum(nums1) + sum(nums2)

Using hypothesis
- Consider the function is_even, which we define in a file called my_functions.py
# Suppose we’ve saved this in my_functions.py

def is_even(value: int) -> bool:

“””Return whether value is divisible by 2.

>>> is_even(2)
True
>>> is_even(17)
False
 “””
return value % 2 == 0
- Rather than choosing specific inputs to test is_even on, we’re going to test the following
two properties:
o is_even always return True when given an int of the form 2 * x (where x is an int)
o is_even always return False when given an int of the form 2 * x + 1 (where x is an
int)
- Using symbolic notation:
o ∀𝑥 ∈ ℤ, is_even(2𝑥)
o ∀𝑥 ∈ ℤ, ¬is_even(2𝑥 + 1)
- To test the function, we first create a new file called test_my_functions.py and include
the following “test” function
# In file test_my_functions.py
from my_functions import is_even

def test_is_even_2x(x: int) -> None:

“””Test that is_even returns True when given a number of the form 2 *
x”””
assert is_even(2 * x)
- The hypothesis module offers a set of strategies that we can use
o These strategies are able to generate several values of a specific type of input
o i.e. use the integers strategy to generate int data types
- We add the following 2 lines to the top of our test file:
from hypothesis import given
from hypothesis.strategies import integers
- Decorator – specified by using the @ symbol with an expression in the line immediately
before a function definition
o Used to attach given and integer to our test function
# In file test_my_functions.py
from hypothesis import given
from hypothesis.strategies import integers

from my_functions import is_even

@given(x=integers())
def test_is_even_2x(x: int) -> None:
“””Test that is_even returns True when given a number of the form 2 *
x”””
 assert is_even(2 * x)
- Integers – a hypothesis function that returns a special data type called a strategy
o Strategy – what hypothesis uses to generate a range of possible inputs
o Calling integers() returns a strategy that generates ints
- Given – a hypothesis function that takes in arguments in the form <param>=<strategy>,
which acts as a mapping for the test parameter name to a strategy that hypothesis
should use for generating arguments for that parameter
- The line @given(x=integers()) decorates the test function, so that when we run the test
function, hypothesis will call the test several times, using int values for x as specified by
the strategy integers()
o @given helps automate the process of “run the test on different int values”
- To actually run the test, we use pytest
if __name__ == ‘__main__’:
import pytest
pytest.main([‘test_my_functions.py’, ‘-v’])
- Testing odd values
o We can write multiple property-based tests in the same file and have pytest run
each of them
o This version of test_my_function.py adds a second test for numbers of the form
2𝑥 + 1
# In file test_my_functions.py
from hypothesis import given
from hypothesis.strategies import integers

from my_functions import is_even

@given(x=integers())
def test_is_even_2x(x: int) -> None:
“””Test that is_even returns True when given a number of the
form 2 * x”””
assert is_even(2 * x)

@given(x=integers())
def test_is_even_2x_plus_1(x: int) -> None:
“””Test that is_even returns Fals when a given number of the
form 2 * x + 1.”””
assert is_even(2 * x + 1)
 if __name__ == ‘__main__’:
import pytest
pytest.main([‘test_my_function.py’, ‘-v’])

Using hypothesis with Collections

- We added the following function into my_functions.py:
# In my_functions.py
from typing import List

def num_evens(nums: List[int]) -> int:

“””Return the number of even elements in nums.”””
return len([n for n in nums if is_even(n)])
- Let 𝐿𝑖𝑛𝑡 be the set of lists of integers, the property we’ll express is:
o ∀𝑛𝑢𝑚𝑠 ∈ 𝐿𝑖𝑛𝑡 , ∀𝑥 ∈ ℤ, nums_evens(nums + [2𝑥]) = num_evens(nums) + 1
o “For any list of integers nums and any integer x, the number of even elements of
nums + [2 * x] is one more than the number of even elements of nums.”
- Writing a test function,
# In test_my_functions.py
def test_num_evens_one_more_even(nums: List[int], x: int) -> None:
“””Test num_evens when you add one more even element.”””
assert num_evens(nums + [2 * x]) == num_evens(nums) + 1
- We need to use @given to tell hypothesis to generate inputs for this test function
o i.e. @given(nums=…, x=…)
o We can use the integers() strategy for x
o We can import the lists function from hypothesis.strategies to create strategies
for generating lists
o We use lists(integers()) to return a strategy for generating lists of integers
- The full test file:
# In file test_my_functions.py
from hypothesis import given
from hypothesis.strategies import integers, lists

from my_functions import is_even, num_evens

@given(nums=lists(integers()), x=integers())
def test_num_evens_one_more_even(nums: List[int], x: int) -> None:
“””Test num_evens when you add one more even element.”””
assert num_evens(nums + [2 * x]) == num_evens(nums) + 1
 if __name__ == ‘__main__’:
import pytest
pytest.main([‘test_my_function.py’, ‘-v’])
- Choosing “enough” properties
o A single property alone does not guarantee that a function is correct
o The ideal goal of property-based testing is choosing properties to verify
If all of the properties are verified, then the function must be correct
o An implementation for num_evens is correct (i.e. returns the number of even
elements for any list of numbers) if and only if it satisfies all 3 of the following:
nums_evens([]) = 0
∀nums ∈ 𝐿𝑖𝑛𝑡 , ∀𝑥 ∈ ℤ, nums_evens(nums + [2𝑥]) =
nums_evens(nums) + 1
∀nums ∈ 𝐿𝑖𝑛𝑡 , ∀𝑥 ∈ ℤ, nums_evens(nums + [2𝑥 + 1]) =
nums_evens(nums)
o This means that we can be certain that our num_evens function is correct with
just 1 unit test and 2 property tests

3.11 Working with Multiple Quantifiers

Intro
- Recall the love table
Sophia Thelonious Stanley Laura
Breanna False True True False
Malena False True True True
Patrick False False True False
Ella False False True True
- Since the 𝐿𝑜𝑣𝑒𝑠 predicate is binary, we can quantify both of its inputs
- ∀𝑎 ∈ 𝐴, ∀𝑏 ∈ 𝐵, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
o “For every person 𝑎 in 𝐴, for every person 𝑏 in 𝐵, 𝑎 loves 𝑏”
o The order in which we quantified 𝑎 and 𝑏 doesn’t matter
o “For every person 𝑏 in 𝐵, for every person 𝑎 in 𝐴, 𝑎 loves 𝑏” means the same
thing
- The following two formulas are equivalent:
o ∀𝑥 ∈ 𝑆1 , ∀𝑦 ∈ 𝑆2 , 𝑃(𝑥, 𝑦)
o ∀𝑦 ∈ 𝑆2 , ∀𝑥 ∈ 𝑆1 , 𝑃(𝑥, 𝑦)
- The following two formulas are also equivalent:
o ∃𝑥 ∈ 𝑆1 , ∃𝑦 ∈ 𝑆2 , 𝑃(𝑥, 𝑦)
 o ∃𝑦 ∈ 𝑆2 , ∃𝑥 ∈ 𝑆1 , 𝑃(𝑥, 𝑦)
- The above would not be the case for a pair of alternating quantifiers
- ∀𝑎 ∈ 𝐴, ∃𝑏 ∈ 𝐵, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
o “For every person 𝑎 in 𝐴, there exists a person 𝑏 in 𝐵, such that 𝑎 loves 𝑏”
“Everyone in 𝐴 loves someone in 𝐵”
o This is true: every person in 𝐴 loves at least one person
𝑎 (from 𝐴) 𝑏 (a person in 𝐵 who 𝑎 loves)
Breanna Thelonious
Malena Laura
Patrick Stanley
Ella Stanley
o Since the quantifier ∃𝑏 ∈ 𝐵 occurs after 𝑎, the choice of 𝑏 is allowed to depend
on the choice of 𝑎
- ∃𝑏 ∈ 𝐵, ∀𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
o “There exists a person 𝑏 in 𝐵, where for every person 𝑎 in 𝐴, 𝑎 loves 𝑏”
“Someone in 𝐵 is loved by everyone in 𝐴”
o This is true because everyone in 𝐴 loves Stanley
𝑏 (from 𝐵) Loved by everyone in 𝐴?
Sophia No
Thelonious No
Stanley Yes
Laura No
o Since the quantifier ∃𝑏 ∈ 𝐵 occurs before 𝑎, the choice of 𝑏 must be
independent of the choice of 𝑎
- When reading a nested quantified expression, always read from left to right and pay
attention to the order of the quantifiers

Translating Multiple Quantifiers into Python Code

- We can represent the love table as a list of lists (List[List[bool]])
# In loves.py
LOVES_TABLE = [
[False, True, True, False],
[False, True, True, True],
[False, False, True, False],
[False, False, True, True]
]
o Global constants – variables that are not defined in a function definition
i.e. LOVES_TABLE
 Called “global” because their scope is the entire Python module in which
they are defined
Can be accessed anywhere in the file, including all function bodies
Can be imported and used in other Python modules
Available when we run the file in the Python console
- Exploring LOVES_TABLE
o We can access LOVES_TABLE by running loves.py in the Python console
>>> LOVES_TABLE[0] # The first row of the table
[False, True, True, False]
>>> LOVES_TABLE[0][1] # The (0, 1) entry in the table
True
o LOVES_TABLE[i][j] evaluates to the entry in row i and column j of the table
o To access colomn j, we can use a list comprehension to access the j-th element in
each row
>>> [LOVES_TABLE[i][0] for i in range(0, 4)]
[False, False, False, False]
o We can add two more constants to represent the sets 𝐴 and 𝐵
A={
‘Breanna’: 0,
‘Malena’: 1,
‘Patrick’: 2,
‘Ella’: 3
}

B={
‘Sophia’: 0,
‘Thelonious’: 1,
‘Stanley’: 2,
‘Laura’: 3
}
- We can define a loves predicate, which takes in two strings and return whether person a
loves person b
def loves(a: str, b: str) -> bool:
“””Return whether the person at index a loves the person at index b.

Preconditions:
- a in A
- b in B
 >>> loves(‘Breanna’, ‘Sophia’)
False
“””
a_index = A[a]
b_index = B[b]
return LOVES_TABLE[a_index][b_index]
- We can represent the statements in predicate logic we’ve written
o ∀𝑎 ∈ 𝐴, ∀𝑏 ∈ 𝐵, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
>>> all ({loves(a, b) for a in A for b in B})
False
o ∃𝑎 ∈ 𝐴, ∃𝑏 ∈ 𝐵, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
>>> any ({loves(a, b) for a in A for b in B})
True
o ∀𝑎 ∈ 𝐴, ∃𝑏 ∈ 𝐵, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
>>> all({any({loves(a, b) for b in B}) for a in A})
True
It is very hard to read the above statement
Never nest all/any calls
We can pull out the inner any into its own function
def loves_someone(a: str) -> bool:
“””Return whether a loves at least one person in B.

Preconditions:
- a in A
“””
return any({loves(a, b) for b in B})

>>> all({loves_someone(a) for a in A})

True
o ∃𝑏 ∈ 𝐵, ∀𝑎 ∈ 𝐴, 𝐿𝑜𝑣𝑒𝑠(𝑎, 𝑏)
>>> any({all({loves(a, b)} for a in A) for b in B})
After pulling out the inner all expression into a named function,
def loved_by_everyone(b: str) -> bool:
“””Return whether b is loved by everyone in A.

Preconditions:
- b in B
 “””
return all({loves(a, b)} for a in A)

>>> any({loved_by_everyone(b) for b in B})

True

A Famous Logical Statement

- “There are infinitely many primes.”
- Because primes are natural numbers, if there are infinitely many of them, then they
have to keep growing bigger and bigger
o “Every natural number has a prime number larger than it”
o ∀𝑛 ∈ ℕ, ∃𝑝 ∈ ℕ, 𝑝 > 𝑛 ∧ 𝐼𝑠𝑃𝑟𝑖𝑚𝑒(𝑝)
- If we want to express this statement without either the 𝐼𝑠𝑃𝑟𝑖𝑚𝑒 or divisibility
predicates, we would end up with an extremely cumbersome statement

4.1 Tabular Data

Toronto Getting Married

ID Civic Centre Marriage Licenses Issued Time Period
1657 ET 80 Jan 2011
1658 NY 136 Jan 2011
1659 SC 159 Jan 2011
1660 TO 367 Jan 2011
1661 ET 109 Feb 2011
1662 NY 150 Feb 2011
1663 SC 154 Feb 2011
1664 TO 383 Feb 2011
- Question: what is the average number of marriage licenses issued by each civic centre?
- We can store this table as a list of lists, where each inner list represents one row of the
table
- Based on the sample data we have:
o The ids and number of marriage licenses are natural numbers
Use int data type
o The civic centre is a two-letter code
Use str data type
o The time periods a year-month combination
Use the datetime module
- In Python,
 import datetime
marriage_data = [
[1657, ‘ET’, 80, datetime.date(2011, 1, 1)],
[1658, ‘NY’, 136, datetime.date(2011, 1, 1)],
[1659, ‘SC’, 159, datetime.date(2011, 1, 1)],
[1660, ‘TO’, 367, datetime.date(2011, 1, 1)],
[1661, ‘ET’, 109, datetime.date(2011, 2, 1)],
[1662, ‘NY’, 150, datetime.date(2011, 2, 1)],
[1663, ‘SC’, 154, datetime.date(2011, 2, 1)],
[1664, ‘TO’, 383, datetime.date(2011, 2, 1)]
]
- We can play around with the data
>>> len(marriage_data) # There are 8 rows of data
8
>>> len(marriage_data[0] # The first rown has 4 elements
4
>>> [len(row) for row in marriage_data] # Every row has 4 elements
[4, 4, 4, 4, 4, 4, 4, 4]
>>> marriage_data[0]
[1657, ‘ET’, 80, datetime.date(2011, 1, 1)]
>>> marriage_data[1]
[1658, ‘NY’, 136, datetime.date(2011, 1, 1)]
>>> marriage_data[0][0]
1657
>>> marriage_data[0][1]
‘ET’
>>> marriage_data[0][2]
80
>>> marriage_data[0][3]
datetime.date(2011, 1, 1)

Accessing Columns and Filtering Rows

- We can retrieve a column by using a list comprehension
>>> [row[1] for row in marriage_data] # The civic centre column
[‘ET’, ‘NY’, ‘SC’, ‘TO’, ‘ET’, ‘NY’, ‘SC’, ‘TO’]
>>> {row[1] for row in marriage_data}
{‘NY’, ‘TO’, ‘ET’, ‘SC’}
 - We can retrieve all rows corresponding to a specific civic centre using if conditions in
comprehensions
>>> [row for row in marriage_data if row[1] == ‘TO’]
[[1660, ‘TO’, 367, datetime.date(2011, 1, 1)], [1664, ‘TO’, 383,
datetime.date(2011, 2, 1)]]
- We can also filter rows based on a threshold for the number of marriage licenses issued
>>> [row for row in marriage_data if row[2] > 380]
[[1664, ‘TO’, 383, datetime.date(2011, 2, 1)]]

A Worked Example
- Question: what is the average number of marriage licenses issued by each civic centre?
- To solve the question, we need to create a dictionary comprehension
>>> names = {row[1] for row in marriage_data}
>>> names
{‘NY’, ‘TO’, ‘ET’, ‘SC’}
>>> {key: 0 for key in names}
{‘NY’: 0, ‘TO’: 0, ‘ET’: 0, ‘SC’: 0}
- Calculate the average number of marriage licenses issued per month for the ‘TO’ civic
centre
>>> [row for row in marriage_data if row[1] == ‘TO’] # The ‘TO’ rows
[[1660, ‘TO’, 367, datetime.date(2011, 1, 1)], [1664, ‘TO’, 383,
datetime.date(2011, 2, 1)]]
>>> [row[2] for row in marriage_data if row[1] == ‘TO’] # The ‘TO’ marriages
issued
[367, 383]
>>> issued_by_TO = [row[2] for row in marriage_data if row[1] == ‘TO’]
- Now issued_by_TO is a list containing the number of marriage licenses issued by the
‘TO’ civic centre
- We can calculate its average by dividing its length
>>> sum(issued_by_TO) / len(issued_by_TO)
375.0
- To merge the above code with the dictionary comprehension outline, first design a
function that calculates the average for only 1 civic centre
from typing import List

def average_licenses_issued(data: List[list], civic_centre: str) -> float:

“””Return the average number of marriage licenses issued by civic_centre
in data.
 Return 0.0 if civic_centre does not appear in the given data.

Preconditions:
- all({len(row) == 4 for row in data})
- data is in the format described in Section 4.1
“””
issued_by_civic_centre = [row[2] for row in data if row[1] == civic_centre]

if issued_by_civic_centre == []:
return 0.0
else:
total = sum(issued_by_civic_centre)
count = len(issued_by_civic_centre)

return total / count

>>> average_licenses_issued(marriage_data, ‘TO’)

375.0
- Finally, we can combine it with our dictionary comprehension (‘TO’ can be replaced with
the key that is changing)
>>> {key: 0 for key in names}
{‘NY’: 0, ‘TO’: 0, ‘ET’: 0, ‘SC’: 0}
>>> {key: average_licences_issued(marriage_data, key) for key in names}
{‘NY’: 143.0, ‘TO’: 375.0, ‘ET’: 94.5, ‘SC’: 156.5}
- We will save our work by writing the entirety of above as a function:
def average_licenses_by_centre(marriage_data: List[list]) -> Dict[str, float]:
“””Return a mapping of the average number of marriage licenses issued
at each civic centre.

In the returned mapping:

- Each key is the name of a civic centre
- Each corresponding value is the average number of marriage licences
issued at that centre.

Preconditions:
- marriage_data is in the format described in Section 4.1
“””
 names = {‘TO’, ‘NY’, ‘ET’, ‘SC’}
return {key: average_licenses_issued(marriage_data, key) for key in
names}

4.2 Defining Our Own Data Types, Part 1

Defining a Data Class

- Class – another term for data type
o type(3) evaluates to <class ‘int’>
- Data class – the simplest kind of data type
o Purpose: to bundle individual pieces of data into a single Python object
- If we want to represent a “person” consisting of a given name, family name, age, and
home address, we could define our own data class to create a new data type consisting
of these four values:
from dataclasses import dataclass

@dataclass
class Person:
“””A custom data type that represents data for a person.”””
given_name: str
family_name: str
age: int
address: str
o from dataclasses import dataclass is a Python import statement that lets us use
dataclass below
o @dataclass is a Python decorator. It tells Python that the data type we’re
defining is a data class
o class Person: signals the start of a class definition
The name of the class is Person
The rest of the code is indented to put it inside of the class body
o The next line is a docstring that describes the purpose of the class
o Each remaining line defines a piece of data associated with the class
Each piece of data is called an instance attribute of the class
For each instance attribute, we write a name and a type annotation
- General data class definition syntax
@dataclass
class <ClassName>:
“””Description of data class.
 “””
<attribute1>: <type1>
<attribute2>: <type2>
…

Using Data Classes

- To create an instance of our Person data class, we write a Python expression that calls
the data class, passing in as arguments of the values for each instance attribute
>>> david = Person(‘David’, ‘Liu’, 100, ‘40 St. George Street’)
>>> type(david)
<class Person>
- To access the individual values, we use the names of the instance attributes together
with dot notation
>>> david.given_name
‘David’
>>> david.family_name
‘Liu”
>>> david.age
100
>>> david.address
’40 St. George Street’
- Much more readable tan list indexing

Tip: Naming Attributes when Creating Data Class Instances

- In the expression Person(‘David’, ‘Liu’, 100, ‘40 St. George Street’), the order of the
arguments must match the order the instance attributes are listed in the definition of
the data class
- Python enables us to create data class instances using keyword arguments to explicitly
name which argument corresponds to which instance attribute:
david = Person(given_name = ‘David’, family_name = ‘Liu’, age = 100, address =
‘40 St. George Street’)
- Using keyword arguments allows us to pass the values in any order we want:
david = Person(family_name = ‘Liu’, given_name = ‘David’, address = ‘40 St.
George Street’, age = 100)
- Improves readability of code
- Representing data classes in the memory model
Variable Value
david
 Person(
family_name = ‘Liu’,
given_name = ‘David’,
address = ‘40 St. George Street’
age = 100
)

4.3 Defining Our Own Data Types, Part 2

Constraining Data Class Values: Representation Invariants

- We don’t always want to allow every possible value of a given type for an attribute
value
o i.e. we don’t want negative numbers as age
- We can record a second piece of information about the age attribute: age >= 0
o This kind of constraint is called a representation invariant
Representation invariant – a predicate describing a condition on how we
represent something that must always be true
- All type annotations, like age: int, are representation invariants
- To add non-type-annotation representation invariants,
@dataclass
class Person:
“””A custom data type that represents data for a person.

Representation Invariants:
- self.age >= 0
”””
given_name: str
family_name: str
age: int
address: str
- In the class docstring, we use the variable name self to refer to a generic instance of the
data class
o This use of self is Python convention
- Checking representation invariants automatically with python_ta
o Like preconditions, representation invariants are assumptions that we make
about values of a data type
i.e. we can assume that every Person instance has an age that’s greater
than or equal to zero
 o Representation invariants are also constraints on how we can create a data class
instance
o python_ta.contracts supports checking all representation invariants:
# class Person above

if __name__ == ‘__main__’:
import python_ta.contracts
python_ta.contracts.DEBUG_CONTRACTS = False
python_ta.contracts.check_all_contracts()
o Then, a negative age would output an assertion error

The Data Class Design Recipe

- 1. Write the class header
@dataclass
class Person:
o The class header consists of 3 parts:
The @dataclass decorator
The keyword class
The name of the data class
o Remember to import from dataclasses
o The name of the class should use the “CamelCase” naming convention
i.e. capitalize every word, not separate the words with underscores
- 2. Write the instance attributes for the data class
@dataclass
class Person:
given_name: str
family_name: str
age: int
address: str
o Every instance of the data class will have all of these attributes
o The name of the attributes should use the “snake_case” naming convention
Like functions and variable names
o Use the typing module whenever possible
- 3. Write the data class docstring
@dataclass
class Person:
“””A data class representing a person.
 Instance Attributes:
- given_name: the person’s given name
- family_name: the person’s family name
- age: the person’s age
- address: the person’s address
“””
given_name: str
family_name: str
age: int
address: str
o Write a description of the class and a description for every instance attribute
o Use the header “Instance Attributes:” to mark the beginning of the attribute
descriptions
- 4. Write an example instance (optional)
@dataclass
class Person:
“””A data class representing a person.

Instance Attributes:
- given_name: the person’s given name
- family_name: the person’s family name
- age: the person’s age
- address: the person’s address

>>> david = Person(‘David’, ‘Liu’, 40, ’40 St. George Street’)

“””
given_name: str
family_name: str
age: int
address: str
o Used to illustrate all of the instance attributes
- 5. Document any additional representation invariants
@dataclass
class Person:
“””A data class representing a person.

Instance Attributes:
- given_name: the person’s given name
 - family_name: the person’s family name
- age: the person’s age
- address: the person’s address

Representation Invariants:
- self.age >= 0

>>> david = Person(‘David’, ‘Liu’, 40, ’40 St. George Street’)

“””
given_name: str
family_name: str
age: int
address: str
o Each representation invariant should be a boolean expression in Python
o Use self.<attribute> to refer to an instance attribute within a representation
invariant

A Worked Example
- Recall the marriage license data set
marriage_data = [
[1657, ‘ET’, 80, datetime.date(2011, 1, 1)],
[1658, ‘NY’, 136, datetime.date(2011, 1, 1)],
[1659, ‘SC’, 159, datetime.date(2011, 1, 1)],
[1660, ‘TO’, 367, datetime.date(2011, 1, 1)],
[1661, ‘ET’, 109, datetime.date(2011, 2, 1)],
[1662, ‘NY’, 150, datetime.date(2011, 2, 1)],
[1663, ‘SC’, 154, datetime.date(2011, 2, 1)],
[1664, ‘TO’, 383, datetime.date(2011, 2, 1)]
]
- Rather than storing each row in the table as a list, we can introduce a new data class to
store this information
from dataclasses import dataclass
from datetime import date

@dataclass
class MarriageData
“””A record of the number of marriage licenses issued in a civic centre in
a given month.
 Instance Attributes:
- id: a unique identifier for the record
- civic_centre: the name of the civic centre
- num_licenses: the number of licenses issued
- month: the month these licenses were issued
“””
id: int
civic_centre: str
num_licenses: int
month: date
- Using the above data class, we can represent tabular data as a list of MarriageData
instances rather than a list of lists
- The values representing each entry in the table are the same, but how we “bundle” each
row of data into a single entity is different:
>>> marriage_data = [
MarriageData(1657, ‘ET’, 80, datetime.date(2011, 1, 1)),
MarriageData(1658, ‘NY’, 136, datetime.date(2011, 1, 1)),
MarriageData(1659, ‘SC’, 159, datetime.date(2011, 1, 1)),
MarriageData(1660, ‘TO’, 367, datetime.date(2011, 1, 1)),
MarriageData(1661, ‘ET’, 109, datetime.date(2011, 2, 1)),
MarriageData(1662, ‘NY’, 150, datetime.date(2011, 2, 1)),
MarriageData(1663, ‘SC’, 154, datetime.date(2011, 2, 1)),
MarriageData(1664, ‘TO’, 383, datetime.date(2011, 2, 1))
]
- Instead of writing row[1] and row[2] in a comprehension, we now write
row.civic_centre and row.num_licenses
o This is more explicit in what attributes of the data are accessed
o “Explicit is better than implicit”

4.4 Repeated Execution: For Loops

Introducing the Problem: Repeating Code

- Variable reassignment statement – assigning a value to a variable that has already been
given a value
o i.e. sum_so_far = sum_so_far + numbers[0]
 o First, the right-hand side of the assignment (sum_so_far + numbers[0])
statement is evaluated; then, the value is stored in the variable on the left-hand
side (sum_so_far)

The For Loop

- For loop – a compound statement that repeats a block of code once for element in a
collection
- Syntax of a for loop:
for <loop_variable> in <collection>:
<body>
- Parts of a for loop:
o <collection> is an expression for a Python collection (e.g. a list or set)
o <loop_variable> is the name for the loop variable that will refer to an element in
the collection
o <body> is a sequence of one or more statements that will be repeatedly
executed.
Called the body of the for loop
The statements within the loop body may refer to the loop variable to
access the “current” element in the collection
o The body of a for loop must be indented relative to the for keyword
- When a for loop is executed,
o 1. The loop variable is assigned to the first element in the collection
o 2. The loop body is executed, using the current value of the loop variable
o 3. Steps 1 and 2 repeat for the second element of the collection, then the third,
etc. until all elements of the collection have been assigned to the loop variable
exactly once

The Accumulator Pattern and my_sum

- Implementation of my_sum
def my_sum(numbers: List[int]) -> int:
“””Return the sum of the given numbers.

>>> my_sum([10, 20, 30])

60
“””
sum_so_far = 0

for number in numbers:

 sum_so_far = sum_so_far + number

return sum_so_far
- We no longer need to use list indexing (numbers[n]) to access individual list elements
o The for loop in Python handles the extracting of individual elements for us
- Accumulators and tracing through loops
o The frequent reassignment of a variable can make loops hard to reason about
o We call the variable sum_so_far the loop accumulator
o Loop accumulator – stores an aggregated result based on the elements of the
collection that have been previously visited by the loop
o We can keep track of the execution of the different iterations of the loop in the
loop accumulation table consisting of three columns:
How many iterations have occurred so far
the value of the loop variable for that iteration
The value of the loop accumulator at the end of that iteration
Iteration Loop variable (number) Loop accumulator (sum_so_far)
0 N/A 0
1 10 10
2 20 30
3 30 60
o Use the _so_far suffix in the variable name of accumulator variables and add a
comment explaining the purpose of the variable
def my_sum(numbers: List[int]) -> int:
“””Return the sum of the given numbers.

>>> my_sum([10, 20, 30])

60
“””
# ACCUMULATOR sum_so_far: keep track of the running sum of the
elements in numbers
sum_so_far = 0

for number in numbers:

sum_so_far = sum_so_far + number

return sum_so_far
- When the collection is empty
o When we call my_sum on an empty list,
 >>> my_sum([])
0
o This happens because sum_so_far is assigned to 0, and then the for loop does
not execute any code, and do 0 is returned
o When the collection is empty, the initial value of sum_so_far is returned

Designing Loops Using the Accumulator Pattern

- The accumulator pattern
o 1. Choose a meaningful name for an accumulator variable based on what we’re
computing
Use the suffix _so_far to remind ourselves that this is an accumulator
o 2. Pick an initial value for the accumulator
This value is usually what should be returned if the collection is empty
o 3. In the loop body, update the accumulator variable based on the current value
of the loop variable
o 4. After the loop ends, return the accumulator
<x>_so_far = <default_value>

for element in <collection>:

<x>_so_far = … <x>_so_far … element … # Somehow combine loop
variable and accumulator

return <x>_so_far
- Accumulating the product
from typing import List

def product(numbers: List[int]) -> int:

“””Return the product of the given numbers.

>>> product([10, 20])

200
>>> product([-5, 4])
-20
# ACCUMULATOR product_so_far: keep track of the product of the
# elements in numbers seen so far in the loop
product_so_far = 1

for number in numbers:

 product_so_far = product_so_far * number

return product_so_far

Looping Over Sets

- Because sets are unordered, we cannot assume a particular order that the for loop will
visit the elements in
- For loops over sets should be used when the same result would be obtained regardless
of the order of the elements
o i.e. sum

Looping Over Strings

- Python treats a string as an ordered collection of characters (strings of length one)
- We can use for loops with strings to iterate over its characters one at a time
- Example:
def my_len(s: str) -> int:
“””Return the number of characters in s

>>> my_len(‘David’)
5
“””
# ACCUMULATOR len_so_far: keep track of the number of
# characters in s seen so far in the loop
len_so_far = 0

for _ in s:
len_so_far = len_so_far + 1

return len_so_far
o We used an underscore here because we don’t care what the actual value the
character is – we are only counting iterations
The loop variable is not used in the body of the for loop

Looping Over Dictionaries

- When we iterate over a dictionary, the loop variable refers to the key of each key-value
pair
- For example, we are given a dictionary mapping restaurant menu items (as strings) to
their prices (as floats) and we want to calculate the sum of all the prices on the menu:
 def total_menu_price(menu: Dict[str, float]) -> float:
“””Return the total price of the given menu items.

>>> total_menu_price({‘fries’: 3.5, ‘hamburger’: 6.5})

10.0
“””
# ACCUMULATOR total_so_far: keep track of the total cost of
# all items in the menu seen so far in the loop.
total_so_far = 0.0

for item in menu:

total_so_far = total_so_far + menu[item]

return total_so_far
o Loop accumulation table:
Iteration Loop variable (item) Loop accumulator (total_so_far)
0 0.0
1 ‘fries’ 6.5
2 ‘hamburger’ 10.0
- Like sets, dictionaries are unordered

A New Type Annotation: Iterable

- my_len also works on lists, sets, and other collections
o The type annotation does not have to be restricted to strings
- A Python data type is iterable when its values can be used as the “collection” of a for
loop
o A Python object is iterable when it is an instance of an iterable data type
- Equivalent to when a value can be used as the “collection” of a comprehension
- Import the Iterable type from typing
- A more general my_len:
from typing import Iterable

def my_len(collection: Iterable) -> int:

“””Return the number of characters in collection

>>> my_len(‘David’)
5
>>> my_len([1, 2, 3])
 3
>>> my_len({‘a’: 1000})
1
“””
# ACCUMULATOR len_so_far: keep track of the number of
# characters in s seen so far in the loop
len_so_far = 0

for _ in collection:
len_so_far = len_so_far + 1

return len_so_far
- Accumulators can work with any iterable object
- Alternatives to for loops
o Many of the above examples can be solved using comprehensions rather than
loops
o Comprehensions are often shorter and more direct translations of a computation
than for loops
o For loops allow us to customize exactly how filtering and aggregation occurs

4.5 For Loop Variations

Multiple Accumulators
- For example, given a dictionary mapping menu items to prices, we can get the average
price using 2 accumulators:
def average_menu_price(menu: Dict[str, float]) -> float:
“””Return the average price of an item from the menu.

>>> average_menu_price({‘fries’: 3.5, ‘hamburger’: 6.5})

5.0
“””
# ACCUMULATOR len_so_far: keep track of the number of
# items in the menu seen so far in the loop
len_so_far = 0
# ACCUMULATOR total_so_far: keep track of the cost of
# all items in the menu seen so far in the loop
total_so_far = 0.0
 for item in menu:
len_so_far = len_so_far + 1
total_so_far = total_so_far + menu[item]

return total_so_far / len_so_far

o Loop accumulation table:
Iteration Loop variable Accumulator Accumulator
(item) len_so_far total_so_far
0 0 0.0
1 ‘fries’ 1 6.5
2 ‘hamburger’ 2 10.0

Conditional Execution of the Accumulator

- Consider the following problem: given a string, count the number of vowels in the string
- By nesting an if statement inside a for loop, we can adapt our accumulator pattern to
only update the accumulator when certain conditions are met
def count_vowels(s: str) -> int:
“””Return the number of vowels in s.

>>> count_vowels(‘aeiou’)
5
>>> count_vowels(‘David’)
2
“””
# ACCUMULATOR vowels_so_far: keep track of the number of vowels
# seen so far in the loop.
vowels_so_far = 0

for letter in s:
if letter in ‘aeiou’:
vowels_so_far = vowels_so_far + 1

return vowels_so_far
o If the word is the empty string, the for loop will not iterate once and the value 0
is returned
o Two cases for the loop body:
1. When letter is a vowel, the reassignment vowels_so_far =
vowels_so_far + 1 increases the number of vowels seen so far by 1
 2. When letter is not a vowel, nothing else happens in the current
iteration because this if statement has no else branch.
• The vowel count remains the same
o Loop accumulation table for count_vowels(‘David’)
Loop Iteration Loop Variable letter Accumulator vowels_so_far
0 0
1 ‘D’ 0
2 ‘a’ 1
3 ‘v’ 1
4 ‘i’ 2
5 ‘d’ 2
o This function can be compared to an equivalent implementation using a filtering
comprehension
- Implementing max
from typing import List

def my_max(numbers: List[int]) -> int:

“””Return the maximum value of the numbers in numbers.

Preconditions:
- numbers != []

>>> my_max([10, 20])

20
>>> my_max([-5, -4])
-4
“””
# ACCUMULATOR max_so_far: keep track of the maximum value
# of the elements in numbers seen so far in the loop.
max_so_far = numbers[0]

for number in numbers:

if number > max_so_far:
max_so_far = number

return max_so_far
o The accumulator max_so_far is updated only when a larger number is
encountered (if number > max_so_far)
- Existential search
def starts_with(strings: Iterable[str], char: str) -> bool:
“””Return whether one of the given strings starts with the character char.

Precondition:
- all({s != ‘’ for s in strings})
- len(char) == 1

>>> starts_with([‘Hello’, ‘Goodbye’, ‘David’, ‘Mario’], ‘D’)

True
>>> starts_with([‘Hello’, ‘Goodbye’, ‘David’, ‘Mario’], ‘A’)
False
“””
return any({s[0] == char for s in words})
o To implement this function without using the any function, consider the
following:
The syntax for s in words can be used to create a for loop
The expression s[0] == char can be used as a condition for an if statement
def starts_with_accumulator(words: Iterable[str], char: str) -> bool:
“””…”””
# ACCUMULATOR starts_with_so_far: keep track of whether
# any of the words seen by the loop so far starts with char
starts_with_so_far = False

for s in strings:
if s[0] == char:
starts_with_so_far = True

return starts_with_so_far
o To update the accumulator, we set it to True when the current string s starts
with char
o Loop accumulation table:
Iteration Loop variable s Accumulator starts_with_so_far
0 False
1 ‘Hello’ False
2 ‘Goodbye’ False
3 ‘David’ True
4 ‘Mario’ True
- Early returns
o The function starts_with_v2 performs unnecessary work because it must loop
through every element of the collection before returning a result
o As soon as condition s[0] == char evaluates to True, we know that the answer is
yes without checking any of the remaining strings
o We can use the return statement inside the body of the loop
o No code execute after the return statement
def starts_with_early_return(strings: Iterable[str], char: str) -> bool:
“””…”””
for s in strings:
if s[0] == char:
return True

return False
We no longer have the accumulator variable
- One common error
def starts_with_wrong(strings: Iterable[str], char: str) -> bool:
“””…”””
for s in strings:
if s[0] == char:
return True
else:
return False
o The loop will only ever perform 1 iteration
o Existential searches are asymmetric:
The function can return True early as soon as it has found an element of
the collection meeting the desired criterion
To return False, it must check every element of the collection
- Universal search
def all_start_with(strings: Iterable[str], char: str) -> bool:
“””Return whether all of the given strings starts with the character char.

Precondition:
- all({s != ‘’ for s in strings})
- len(char) == 1

>>> all_starts_with([‘Hello’, ‘Goodbye’, ‘David’, ‘Mario’], ‘D’)

False
 >>> all_starts_with([‘Drip’, ‘Drop’, ‘Dangle’], ‘D’)
True
“””
return all({s[0] == char for s in strings})
o We can use the accumulator pattern from starts_with_accumulator
o A better way is to write this function using an early return, since we can return
False as soon as a counterexample is found:
def all_starts_with_early_return(strings: Iterable[str], char: str) -> bool:
“””…”””
for s in words:
if s[0] != char:
return False

return True

4.6 Index-Based For Loops

Repeating Code
- Recall the my_sum function
def my_sum(numbers: List[int]) -> int:
“””Return the sum of the given numbers.

>>> my_sum([10, 20, 30])

60
“””
# ACCUMULATOR sum_so_far: keep track of the running sum of the
elements in numbers
sum_so_far = 0

for number in numbers:

sum_so_far = sum_so_far + number

return sum_so_far
- For the my_sum function, we know that the index starts at 0 and ends at the length – 1
def my_sum_v2(numbers: List[int]) -> int:
“””…”””
# ACCUMULATOR sum_so_far: keep track of the running sum of the
# elements in numbers.
 sum_so_far = 0

for i in range(0, len(numbers)):

sum_so_far = sum_so_far + numbers[i]

return sum_so_far
- Differences between my_sum and my_sum_v2:
o Loop variable number vs. i:
number refers to an element of lthe list numbers (starting with the first
element)
i refers to an integer (starting at 0)
o Looping over a list vs. a range:
for number in numbers causes the loop body to execute once for each
element in numbers
for i in range(0, len(numbers)) causes the loop body to execute once for
each integer in range(0, len(numbers))
o Updating the accumulator:
Since number refers to a list element, we can add it directly to the
accumulator
Since i refers to where we are in the list, we access the corresponding list
element using list indexing to add it to the accumulator
- Both our element-based and index-based implementations are correct here

When Location Matters

- Example
def count_adjacent_repeats(string: str) -> int:
“””Return the number of times in the given string that two adjacent characters
are equal

>>> count_adjacent_repeats(‘look’)
1
>>> count_adjacent_repeats(‘David’)
0
“””
- We want to use an accumulator variable that starts at 0 and increases by 1 every time
two adjacent repeated characters are found
- Comparisons:
o string[0] == string[1]
 o string[1] == string[2]
o etc.
def count_adjacent_repeats(string: str) -> int:
“””…”””
# ACCUMULATOR repeats_so_far: keep track of the number of adjacent
# characters that are identical
repeats_so_far = 0

for i in range(0, len(string) - 1):

if string[i] == string[i + 1]:
repeats_so_far = repeats_so_far + 1

return repeats_so_far
o Since we are indexing string[i + 1], our loop variable i only needs to go up to n – 2
rather than n – 1
- We could not have implemented the above function using an element-based for loop
o We would not be able to access the character adjacent the current one

Two Lists, One Loop

- Index-based for loops can be used to iterate over two collections in parallel using a
single for loop
- Example
def count_money(counts: List[int], denoms: List[float]) -> float:
“””Return the total amount of money for the given coin counts and
denominations.

counts stores the number of coins of each type, and denominations stores the
value of each coin type. Each element in counts corresponds to the element at
the same index in denoms.

Preconditions:
- len(counts) == len(values)

>>> count_money([2, 4, 3], [0.05, 0.10, 0.25])

1.25
“””
- We need to multiply each corresponding element of counts and denoms, and add the
results:
 o (counts[0] * denoms[0]) + (counts[1] * denoms[1]) + …
- Using range,
o [counts[i] * denoms[i] for i in range(0, len(counts))]
- Then, we can compute the sum sing the built-in Python function
def count_money(counts: List[int], denoms: List[float]) -> float:
“””…”””
return sum([counts[i] * denoms[i] for i in range(0, len(counts))])
- The above works well, however, we can also implement the function using a for loop
def count_money(counts: List[int], denoms: List[float]) -> float:
“””…”””
# ACCUMULATOR money_so_far: keep track of the total money so far.
money_so_far = 0.0

for i in range(0, len(counts)):

money_so_far = money_so_far + counts[i] * values[i]

return money_so_far

Choosing the Right For Loop

- Element-based for loops (for <loop_variable> in <collection>) are useful when:
o We want to process each element in the collection without knowing about its
position in the collection
- Index-based for loops (for <loop_variable> in <range>) are useful when:
o The location of elements in the collection matters
As in count_adjacent_repeats
o We want to loop through more than 1 list at a time using the same index for
both lists
As in count_money
- Though index-based loops are more powerful than element-based loops
o Not all collections can be indexed (i.e. set and dict)
o Index-based for loops are more indirect than element-based for loops
- Use element-based for loops if possible

4.7 Nested For Loops

Nested Loops and Nested Data

- Suppose we have a list of lists of integers
o >>> lists_of_numbers = [[1, 2, 3], [10, -5], [100]]
- Our goal is to compute the sum of all the elements of the list
def sum_all(lists_of_numbers: List[List[int]]) -> int:
“””Return the sum of all the numbers in the given lists_of_numbers.

>>> sum_all([[1, 2, 3], [10, -5], [100]])

111
# ACCUMULATOR sum_so_are: keep track of the running sum of the numbers.
sum_so_far = 0

for numbers in lists_of_numbers: # numbers is a list of numbers, not 1 number

sum_so_far = sum_so_far + sum(numbers)

return sum_so_far
- Without using sum, we need another for loop:
def sum_all(lists_of_numbers: List[List[int]]) -> int:
“””…”””
# ACCUMULATOR sum_so_far: keep track of the running sum of the numbers.
sum_so_far = 0

for numbers in lists_of_numbers: # numbers is a list of numbers, not 1 number

for number in numbers: # number is a single number
sum_so_far = sum_so_far + numbers

return sum_so_far
o for number in numbers loops is nested within the for numbers in
lists_of_numbers
o If we call our doctest example, sum_all([[1, 2, 3], [10, -5], [100]]), the following
happens:
1. The assignment statement sum_so_far = 0 execute, creating our
accumulator variable
2. The outer loop is reached
• The loop variable list_of_numbers is assigned the first element in
lists_of_numbers, which is [1, 2, 3]
• Then, the body of the outer loop is executed. Its body is just 1
statement: the inner for loop, for number in numbers
o The inner loop variable number is assigned the first value
in numbers, which is 1
 o The inner loop body gets executed, updating the
accumulator. sum_so_far is reassigned to 1 (since 0 + 1 ==
1)
o The inner loop iterates twice more, for number = 2 and
number 3. At each iteration, the accumulator is updated,
first by adding 2 and then 3. At this point, sum_so_far = 6
(which is 0 + 1 + 2 + 3)
o After all 3 iterations of the inner loop occur, the inner loop
stops. The Python interpreter is done executing this
statement.
• The next iteration of the outer loop occurs; numbers is assigned to
the list [10, -5]
• Again, the body of the outer loop occurs
o The inner loop now iterates twice: for number = 10 and
number = -5. sum_so_far is reassigned twice more, with a
final value of 11 (which is 6 + 10 + -5)
• The outer loop iterates one more time, for numbers = [100]
• Again, the body of the outer loop occurs
o The inner loop iterates once, for number = 100.
sum_so_far is reassigned to 111 (which is 11 + 100)
• At last, there are no more iterations of the outer loop, and so it
stops
3. After the outer loop is done, the return statement executes, returning
the value of sum_so_far, which is 111
o The above behaviour can be summarized by a loop accumulation table
Inner
Outer Outer Loop Inner
Loop Accumulator
Loop Variable Loop
Variable (sum_so_far)
Iteration (list_of_numbers) Iteration
(number)
0 0
1 [1, 2, 3] 0 1 0
1 [1, 2, 3] 1 2 1
1 [1, 2, 3] 2 3 3
1 [1, 2, 3] 3 6
2 [10, -5] 0 10 6
2 [10, -5] 1 -5 16
2 [10, -5] 2 11
 3 [100] 0 100 11
3 [100] 1 111

The Cartesian Product

- We want to use nested loops on two different collections, obtaining all pairs of possible
values from each collection
def product(set1: set, set2: set) -> Set[tuple]:
“””Return the Cartesian product of set1 and set2.

>>> result = product({10, 11}, {5, 6, 7})

>>> result == {(10, 5), (10, 6), (10, 7), (11, 5), (11, 6), (11, 7)}
True
“””
# ACCUMULATOR product_so_far: keep track of the tuples from the pairs
# of elements visited so far.
product_so_far = set()

for x in set1:
for y in set2:
product_so_far = set.union(product_so_far, {(x, y)}_

return product_so_far
o Loop accumulation table
Outer Inner
Outer Loop Inner Loop Accumulator
Loop Loop
Iteration Iteration (product_so_far)
Var (x) Var (y)
0 set()
1 10 0 set()
1 10 1 5 {(10, 5)}
1 10 2 6 {(10, 5), (10, 6)}
1 10 3 7 {(10, 5), (10, 6), (10, 7)}
2 11 0 {(10, 5), (10, 6), (10, 7)}
{(10, 5), (10, 6), (10, 7),
2 11 1 5
(11, 5)}
{(10, 5), (10, 6), (10, 7),
2 11 2 6
(11, 5), (11, 6)}
{(10, 5), (10, 6), (10, 7),
2 11 3 7
(11, 5), (11, 6), (11, 7)}
Outer and Inner Accumulators
- Each loop can have its own accumulator
- Example: suppose we have a list of lists of integers called grades
grades = [
[70, 75, 80], # ENG196
[70, 80, 90, 100], # CSC110
[80, 100] # MAT137
]
o Each element of grades corresponds to a course and contains a list of grades
obtained in that course
o The list of grades for course ENG196 does not have the same length as CSC110 or
MAT137
- We have defined a function average that calculates the average to a list of int
- Goal: return a new list containing the average grade of each course
- We can calculate a list of averages for each course using a comprehension:
def course_averages_comprehension(grades: List[List[int]]) -> List[float]
“””Return a new list for which each element is the average of the grades in the
inner list at the corresponding position of grades.

>>> course_averages_comprehension([[70, 75, 80], [70, 80, 90, 100], [80, 100]])
[75.0, 85.0, 90.0]
“””
return [average(course_grades) for course_grades in grades]
- We can translate the above function into a for loop using a list accumulator variable and
list concatenation for the update
def course_averages_loop(grades: List[List[int]]) -> List[float]
“””…”””
# ACCUMULATOR averages_so_far: keep track of the averages of the lists
# visited so far in grades
average_so_far = []

for course_grades in grades:

course_average = average(course_grades)
averages_so_far = averages_so_far + [course_average]

return averages_so_far
- We can also implement the function without using the average function by expanding
the definition of average directly in the loop body
def course_averages(grades: List[List[int]]) -> List[float]
“””…”””
# ACCUMULATOR averages_so_far: keep track of the averages of the lists
# visited so far in grades
average_so_far = []

for course_grades in grades:

# ACCUMULATOR len_so_far: keep track of the number of elements
# seen so far in course_grades
len_so_far = 0
# ACCUMULATOR total_so_far: keep track of the total of the elements
# seen so far in course_grades
total_so_far = 0

for grade in course_grades:

len_so_far = len_so_far + 1
total_so_far = total_so_far + grade

course_average = total_so_far / len_so_far

average_so_far = averages_so_far + [course_average]

return averages_so_far
- The inner loop accumulators are assigned to inside the body of the outer loop rather
than at the top of the function body
o This is because len_so_far and total_so_far are specific to course_grades, which
changes at each iteration of the outer loop
o The statements len_so_far = 0 and total_so_far = 0 act to “reset” these
accumulators for each new course_grades list
- Loop accumulation table
Inner
Outer Outer Loop Inner Inner Inner Outer
Loop
Loop Variable Loop Accumulator Accumulator Accumulator
Variable
Iteration (course_grades) Iteration (len_so_far) (total_so_far) (averages_so_far)
(grade)
0 []
1 [70, 75, 80] 0 0 0 []
1 [70, 75, 80] 1 70 1 70 []
 1 [70, 75, 80] 2 75 2 145 []
1 [70, 75, 80] 3 80 3 225 [75.0]
2 [70, 80, 90, 100] 0 0 0 [75.0]
2 [70, 80, 90, 100] 1 70 1 70 [75.0]
2 [70, 80, 90, 100] 2 80 2 150 [75.0]
2 [70, 80, 90, 100] 3 90 3 240 [75.0]
2 [70, 80, 90, 100] 4 100 4 340 [75.0, 85.0]
3 [80, 100] 0 0 0 [75.0, 85.0]
3 [80, 100] 1 80 1 80 [75.0, 85.0]
3 [80, 100] 2 100 2 180 [75.0, 85.0, 90.0]

Summary: Understanding and Simplifying Nested For Loops

- Follow the following guidelines when using nested loops
o Use nested loops when we have a single accumulator that can be initialized just
once before the nested loop (e.g. sum_all and cartesian_product)
o If we have a nested loop where the inner loop can be replaced by a built-in
aggregation function (e.g. sum or len), use the built-in function instead
o If we have a nested loop where the inner loop has a separate accumulator that is
assigned inside the outer loop (e.g. course_averages), move the accumulator and
inner loop into a new function, and call that function from within the original
outer loop

5.1 Variable Reassignment and Object Mutation

Variable Reassignment
- Assignment statement – __ = __
o Takes a variable name on the left side and an expression on the right side, and
assigns the value of the expression to the variable
- Variable reassignment – assigns a value to a variable that already refers to a value
>>> x = 1
>>> x = 5 # The variable x is reassigned on this line
- A variable reassignment changes which object a variable refers to
o x changes from referring to an object representing 1 to an object representing 5
- Used to update the accumulator variable inside the loop

Reassignment is Independent of Prior Uses

- Variable reassignment only changes the immediate variable being reassigned, and does
not change any other variables or objects, even ones that were defined using the
variable being reassigned
 >>> x = 1
>>> y = x + 2
>>> x = 7
>>> y
3

Object Mutation
- Object mutation – an operation that changes the value of an existing object
o Python’s list data type contains several methods that mutate the given list object
rather than create a new one
- Function squares without mutation
def squares(nums: List[int]) -> List[int]:
“””Return a list of the squares of the given numbers.”””
squares_so_far = []

for num in nums:

squares_so_far = squares_so_far + [num * num]
return squares_so_far
- Function squares with mutation (using list.append)
def squares(nums: List[int]) -> List[int]:
“””…”””
squares_so_far = []

for num in nums:

list.append(squares_so_far, num * num)
return squares_so_far
o squares run by assigning squares_so_far to a single list object before the loop,
and then mutating that list object at each loop iteration
o This code is more efficient that the one without mutation
The old one creates a bunch of new list objects
o squares_so_far is not reassigned; instead, the object that it refers to gets
mutated
- list.append returns None and has a side effect of mutating its list argument

Mutable and Immutable Data Types

- A Python data type is mutable when it supports at least one kind of mutating operation
o It is immutable if it does not
- Mutable data types: sets, lists, dictionaries
 - Immutable data types: int, float, bool, str
- If we have an object representing the number 3 in Python, that object’s value will
always be 3
o The variable that refers to this object might be reassigned to a different object
later

List vs. Tuple, and What’s In a Set

- A list is mutable; a tuple is immutable
o i.e. we can modify a list value by adding an element with list.append; but there is
no equivalent tuple.append, nor any other mutating method on tuples
- sets may only contain immutable objects; dicts may only contain immutable keys
o We cannot have a set of sets or set of lists in Python
Yes it does not match what we have in mathematics
o But we can have a list of lists

Reasoning About Code with Changing Values

- Reassignment will change which object a variable refers to, sometimes creating a brand
new object
- Object mutation changes the object itself, independent of what variable(s) refer to that
object
- Introducing reassignment and mutation makes our code harder to reason about, as we
need to track all changes to variable values line by line
- Avoid using variable reassignment and object mutation when possible
o Use them in structured code patterns (i.e. the loop accumulator pattern)

5.2 Operations on Mutable Data Types

list.append, list.insert, and list.extend

- list.insert – takes a list, an index, and an object, and inserts the object at the given index
into the list at the given index
>>> strings = [‘a’, ‘b’, ‘c’, ‘d’]
>>> list.insert(strings, 2, ‘hello’) # Insert ‘hello’ into strings at index 2
>>> strings
[‘a’, ‘b’, ‘hello’, ‘c’, ‘d’]
- list.extend – takes two lists and add all items from the second list at the end of the first
list
>>> strings = [‘a’, ‘b’, ‘c’, ‘d’]
>>> list.extend(strings, [‘CSC110’, ‘CSC111’])
 >>> strings
[‘a’, ‘b’, ‘c’, ‘d’, ‘CSC110’, ‘CSC111’]

Assigning to a Specific List Index

- We can overwrite the element stored at a specific index
>>> strings = [‘a’, ‘b’, ‘c’, ‘d’]
>>> strings[2] = ‘Hello’
>>> strings
[‘a’, ‘b’, ‘Hello’, ‘d’]
- Unlike list.insert, assigning to an index removes the element previously stored at the
index from the list

Mutating sets
- Two main mutating methods:
o set.add
o set.remove
- Re-implement our squares function with set instead of list
def squares(numbers: Set[int]) -> Set[int]:
“””…”””
squares_so_far = set()
for n in numbers:
set.add(squares_so_far, n * n)

return squares_so_far
o set.add will only add the element if the set does not already contain it
o Sets are unordered whereas list.append will add the element to the end of the
sequence

Mutating Dictionaries
- To mutate a dictionary,
o Add a new key-value pair
>>> items = {‘a’: 1, ‘b’: 2}
>>> items[‘c’] = 3
>>> items
{‘a’: 1, ‘b’: 2, ‘c’: 3}
The left side of the assignment is not a variable but instead an expression
representing a component of items (i.e. the key ‘c’ in the dictionary)
 When this assignment statement is evaluated, the right side value 3 is
stored in the dictionary items as the corresponding value for ‘c’
o Change the associated value for a key-value pair
>>> items[‘a’] = 100
>>> items
{‘a’: 100, ‘b’: 2, ‘c’: 3}
The assignment statement takes an existing key-value pair and replaces
the value with a different one

Mutating Data Classes

- Python data classes are mutable by default
@dataclass
class Person:
“””A person with some basic demographic information

Representation Invariants:
- self.age >= 0

“””
given_name: str
family_name: str
age: int
address: str
- We mutate instances of data classes by modifying their attributes
o We do this by assigning to their attributes directly, using dot notation on the left
side of an assignment statement
>>> p = Person(‘David’, ‘Liu’, 100, ’40 St. George Street’)
>>> p.age = 200
>>> p
Person(given_name=’David’, family_name=’Liu’, age=200, address=’40 St.
George Street’)
- Respect the representation invariants when mutating data class instances

5.3 The Python Memory Model: Introduction

Representing Objects
- Every piece of data is stored in a Python program in an object
- We cannot control which memory addresses are used to store objects, but we can
access a representation of this memory address using the built-in id function
>>> id(3)
1635361280
>> id(‘words’)
4297547872
- Id – a unique int identifier to refer to the object
- Every object in Python has three important properties:
o Id
The only one among the three guaranteed to be unique
o Value
o Type
- A variable is not an object and so does not actually store data
o Variables store an id hat refers to an object that stores data
i.e. variables contain the id of an object
- With a full object-based Python memory model, we draw one table-like structure on the
left showing the mapping between variables and object ids, and the the objects on the
right
o Each object is represented as a box, with its id in the upper-left corner, type in
the upper-right corner, and value in the middle
o The actual object id reported by the id function is unimportant
We just need to know that each object has a unique identifier
>>> x = 3
>>> word = ‘bonjour’
id92 int
__main__
3
x id92
id5 str
word id5
‘bonjour’
o There is no 3 inside the box for variable x; instead, there is the id of an object
whose value is 3
- Assignment statements and evaluating expressions
o Evaluating an expression
Produces and id of an object representing the value of the expression
o Assignment statements
1. Evaluate the expression on the right side, yielding the id of an object
2. If the variable on the left side doesn’t already exist, create it
3. Store the id from the expression on the right side in the variable on the
left side
Representing Compound Data
- An instance of a compound data type does not store values directly
o Instead, it stores the ids of other objects
- Lists
o lst = [1, 2, 3]
id4 list
0 1 2
__main__
id10 id11 id12
lst id4
id10 int id11 int id12 int
1 2 3
4 separate objects on the diagram:
• One for each of the ints 1, 2, 3
• One for the list
- Sets
o my_set = {1, 2, 3}
id4 set
0 1 2
__main__
id10 id11 id12
my_set id4
id10 int id11 int id12 int
1 2 3
- Dictionaries
o my_dict = {‘a’: 1, ‘b’: 2}
id10 str id11 str
id2 dict
__main__ ‘a’ ‘b’
id10 : id12
my_dict id2 id12 int id13 int
id11 : id13
1 2
5 objects in total
- Data classes
o For the Person object
id7 Person id11 str id13 str
given_name id11 ‘David’ ‘Liu’
__main__
family_name id13
david id7 id72 int id8 str
age id72
100 ‘123 Fake St’
address id8
- Use the convention of drawing a double box around objects that are immutable
Visualizing Variable Reassignment and Object Mutation
- Consider this case of variable reassignment
>>> s = [1, 2]
>>> s = [‘a’, b’]
o Before reassignment
id4 list
__main__ id10 int id11 int
0 1
s id4 1 2
id10 id11
o After reassignment
id4 list
id10 int id11 int
0 1
1 2
__main__ id10 id11
s id40 id40 list
id70 str id71 str
0 1
‘a’ ‘b’
id70 id71
o The original list object [1, 2] is not mutated
Variable reassignment does not mutate any objects
What the variable refers to is changed
- Consider this case of object mutation
>>> s = [1, 2]
>>> list.append(s, 3)
o Before mutation
id4 list
__main__ id10 int id11 int
0 1
s id4 1 2
id10 id11
o After mutation
id10 int id11 int
id4 list
__main__ 1 2
0 1 2
s id4 id80 int
id10 id11 id80
3
o No new list object is created
The list object [1, 2] is mutated, and a third id is added at its end
o The id of s is not changed despite changing in size
- Consider this case of assigning to part of a compound data type
>>> s = [1, 2]
>>> s[1] = 300
 o Before mutation
id4 list
__main__ id10 int id11 int
0 1
s id4 1 2
id10 id11
o After mutation
id10 int id11 int
id4 list
__main__ 1 2
0 1
s id4 id80 int
id10 id80
300
o Rather than reassigning a variable, it reassigns an id that is part of an object
o This statement does mutate an object, and doesn’t reassign any variables

5.4 Aliasing and “Mutation at a Distance”

Aliasing
- Let v1 and v2 be Python variables. v1 and v2 are aliases when they refer to the same
object
- Consider the following
>>> x = [1, 2, 3]
>>> y = [1, 2, 3]
>>> z = x # make z refer to the object that x refers to
o x and z are aliases, as they both reference the same object
They have the same id
o x and y are two different list objects stored separately in the computer’s memory

Aliasing and Mutation

- Aliasing allows “mutation at a distance”
>>> x = [1, 2, 3]
>>> z = x
>>> z[0] = -999
>>> x
[-999, 2, 3]
o The third line mutates the object that z refers to
- Another example
>>> x = [1, 2, 3]
>>> z = x
>>> y[0] = -999
 >>> x
[1, 2, 3]

Variable Reassignment
- Example of variable reassignment
>>> x = (1, 2, 3)
>>> z = x
>>> z = (1, 2, 3, 40)
>>> x
(1, 2, 3)
o When we change z on the third line, x does not change this time
o We reassigned z to a new object, which has no effect on the object that x refers
to
- Reassigning breaks the aliasing
o Afterwards the two variables would no longer refer to the same object

Aliasing and Loop Variables

- In element-based for loops, the loop variable is an alias to one of the objects within the
collection
- Example
>>> numbers = [5, 6, 7]
>>> for number in numbers:
… number = number + 1
…
>>> numbers
[5, 6, 7]
o The assignment statement inside the for loop simply changes what the loop
variable refers to, but does not change what the contents of the list numbers
refers to
- Assignment statements of the form <name> = ______ reassign the variable <name> to a
new value
- Assignment statements of the form <name>[<index>] = ______ mutate the list object
that <name> currently refers to

Two Types of Equality

- Value equality – whether the two objects have the same value
o x == y
- Identity equality – whether two objects have the same ids
 o x is y
- Identity equality is a stronger property than value equality
o For all objects a and b, if a is b then a == b
The converse is not true

Aliasing with Immutable Data Types

- Aliasing exists for immutable data types, but there is never any “action at a distance”
o Because immutable values can never change
- Automatic aliasing of (some) immutable objects
o If two variables have the same immutable value, the program’s behaviour does
not depend on whether the two variables are aliases or not
o The Python interpreter saves some computer memory by not creating new
objects for some immutable values
i.e. every occurrence of the boolean value True refers to the same object
“Small” integers are automatically aliased, while “large” integers are not
o The Python interpreter takes the object creation “shortcut” with some string
values
- Rules for object comparisons:
o For boolean values, use is to compare for equality
o For non-boolean immutable values, use == to compare for equality
Using is can lead to surprising results
o For mutable values, use == to compare value equality
o For mutable values, use is to check for aliasing

5.5 The Full Python Memory Model: Function Calls

Stack Frames
- Suppose we define the following function, and then call it in the Python console
def repeat(n: int, s:str) -> str:
message = s * n
return message

# In the Python console

>>> count = 3
>>> word = ‘abc’
>>> result = repeat(count, word)
 o When repeat(count, word) is called, immediately before the return message
statement executes, the full Python memory model diagram would look like the
following
__main__
id5 int
count id5
3
word id2
repeat id2 str
n id5 ‘abc’
s id2 id19 str
message id19 ‘abcabcabc’
The variables are separated into 2 separate boxes
• One for the Python console and one for the function call for
repeat
All variables, regardless of which box they’re in, store only ids that refer
to objects on the right side
count and n are aliases
word and s are aliases
- Stack frame – the boxes on the left side of the diagram
o Also known as frame
o Keep track of the functions that have been called in a program, and the variables
defined within each function
- Function call stack – the collection of stack frames
- Every time we call a function, the Python interpreter does the following:
o 1. Create a new stack frame and add it to the call stack
o 2. Evaluate the arguments in the function call, yielding the ids of objects (one per
argument).
Each of these ids is assigned to the corresponding parameter, as an entry
in the new stack frame
o 3. Execute the body of the function
o 4. When a return statement is executed in the function body, the id of the
returned object is saved and the stack frame for the function call is removed
from the call stack

Argument Passing and Aliasing

- When we called repeat(count, word), it is as if we wrote
n = count
s = word
 before executing the body of the function
- This aliasing allows us to define functions that mutate their argument values
- Example
def emphasize(words: List[str]) -> None:
“””Add emphasis to the end of a list of words.”””
new_words = [‘believe’, ‘me!’]
list.extend(words, new_words)

# In the Python console

>>> sentence = [‘winter’, ‘is’, ‘coming’]
>>> emphasize(sentence)
>>> sentence
[‘winter’, ‘is’, ‘coming’, ‘believe’, ‘me!’]
o words and sentence are aliases, and so mutating words within the function
causes a change to occur in __main__ as well
- Another example
def emphasize_v2(words: List[str]) -> None:
“””…”””
new_words = [‘believe’, ‘me!’]
words = words + new_words

# In the Python console

>>> sentence = [‘winter’, ‘is’, ‘coming’]
>>> emphasize_v2(sentence)
>>> sentence
[‘winter’, ‘is’, ‘coming’]
o List concatenation with + creates a new list object
o words and sentence are no longer alias after words is reassigned

5.6 Testing Functions III: Testing Mutation

Intro
- If a function’s documentation does not specify that an object will be mutated, then it
must not be mutated

Identifying Mutable Parameters

- Consider the squares function
def squares(nums: List[int]) -> List[int]:
 “””Return a list of the squares of the given numbers.”””
squares_so_far = []

for num in nums:

list.append(squares_so_far, num * num)
return squares_so_far
o Because squares_so_far is created by the function squares, it is okay that it is
mutated (i.e. the call to list.append inside the loop)
o The nums list is passed as an argument to squares
Because the docstring does not indicate that nums will be mutated, it is
expected that the squares function will not mutate the list object
referred to by nums
- Contrast the above program with the following where the function does mutate its input
def square_all(nums: List[int]) -> None:
“””Modify nums by squaring each of its elements.”””
for i in range(0, len(nums)):
nums[i] = nums[i] * nums[i]

Testing for No Mutation

- Write a test that ensures the squares function does not mutate the list referred to by
nums
def test_squares_no_mutation() -> None:
“””Test that squares does not mutate the list it is given.”””
lst = [1, 2, 3]
squares(lst)

assert lst == [1, 2, 3]

- In order to test that a list is not mutated,
o We first create a list lst
o We then call the squares function on lst
We do not assign the result to a variable because we don’t care about the
returned value for the purpose of this test
o We add an assertion that ensures lst has not been mutated
- Our assertion checks that after the call to squares, lst still has value [1, 2, 3]
- Another way to accomplish this, without re-typing the list value, is by creating a copy of
lst before the call to squares
def test_squares_no_mutation() -> None:
“””Test that squares does not mutate the list it is given.”””
 lst = [1, 2, 3]
lst_copy = list.copy(lst) # Create a copy of lst (not an alias)
squares(lst)

assert lst == lst_copy

- We create the list and its copy before the call to squares
- We test for mutation (i.e. the assertion) after the call to squares
- Generalizing this test
o For the above test, if we replaced lst’s value with another list, the test would
behave in the exact same way
o Our test is suitable to be generalized into a property-based test, representing the
following property:
For all lists of integer lst, calling squares(lst) does not mutate lst
from typing import List

from hypothesis import given

from hypothesis.strategies import lists, integers

@given(lst=lists(integers()))
def test_squares_no_mutation_general(lst: List[int]) -> None:
“””Test that squares does not mutate the list it is given.”””
lst_copy = list.copy(lst) # Create a copy of lst (not an alias)
squares(lst)

assert lst == lst_copy

Testing for Mutation

- Consider testing the square_all function
def test_square_all() -> None:
“””Test that square_all mutates the list it is given correctly.”””
lst = [1, 2, 3]
result = squares_all(lst)

assert result == [1, 4, 9]

o This test fails because square_all returns None
None == [1, 4, 9] is False
- We test if the value of lst has changed
def test_square_all_mutation() -> None:
 “””Test that square_all mutates the list it is given correctly. “””
lst = [1, 2, 3]
square_all(lst)

assert lst == [1, 4, 9]

- The above test can be generalized into a property-based test by storing a copy of the
original list and verifying the relationship between corresponding elements
@given(lst=lists(integers()))
def test_square_all_mutation_general(lst: List[int]) -> None:
“””Test that square_all mutates the list it is given correctly.”””
lst_copy = list.copy(lst)
square_all(lst)

assert all({lst[i] == lst_copy[i] ** 2 for i in range(0, len(lst))})

6.1 An Introduction to Number Theory

Divisibility, Primality, and the Greatest Common Divisor

- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑛, 𝑑 ∈ ℤ. We say that d divides n, when there exists a 𝑘 ∈ ℤ such that
𝑛 = 𝑑𝑘. We use the notation 𝑑 | 𝑛 to represent the statement “𝑑 divides 𝑛”
o The following phrases are synonymous with “𝑑 divides 𝑛”:
𝑛 is divisible by 𝑑
𝑑 is a factor of 𝑛
𝑛 is a multiple of 𝑑
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑝 ∈ ℕ. We say 𝑝 is prime when it is greater than 1 and the only natural
numbers that divide it are 1 and itself
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑥, 𝑦, 𝑑 ∈ ℤ. We say that 𝑑 is a common divisor of 𝑥 and 𝑦 when 𝑑
divides 𝑥 and 𝑑 divides 𝑦
- We say that 𝑑 is the greatest common divisor of 𝑥 and 𝑦 when it is the largest number
that is a common divisor of 𝑥 and 𝑦, or – when 𝑥 and 𝑦 are both 0.
o We can define the function gcd ∶ ℤ × ℤ → ℕ as the function which takes
numbers 𝑥 and 𝑦, and returns their greatest common divisor
- 1 divides every natural number, so 1 is a common divisor between any two natural
numbers
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑚, 𝑛 ∈ ℤ. We say that 𝑚 and 𝑛 are coprime when gcd(𝑚, 𝑛) = 1

Quotients and Remainders

 - For all 𝑛 ∈ ℤ and 𝑑 ∈ ℤ+ , there exist 𝑞 ∈ ℤ and 𝑟 ∈ ℕ such that 𝑛 = 𝑞𝑑 + 𝑟 and 0 ≤
𝑟 < 𝑑. Moreover, these 𝑞 and 𝑟 are unique for a given 𝑛 and 𝑑
o We say that 𝑞 is the quotient when 𝑛 is divided by 𝑑, and that 𝑟 is the remainder
when 𝑛 is divided by 𝑑
- In Python, for given integers n and d, we can compute their quotient using //, their
remainder using %, and both at the same time using the built-in function divmod
>>> 9 // 2
4
>>> 9 % 2
1
>>> divmod(9, 2)
(4, 1)

Modular Arithmetic
- We often care about the remainder when we divide a number by another
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑎, 𝑏, 𝑛 ∈ ℤ with 𝑛 ≠ 0. We say that 𝑎 is equivalent to 𝑏 modulo 𝑛
when 𝑛 | 𝑎 − 𝑏. In this case, we write 𝑎 ≡ 𝑏 (mod 𝑛)
- Modular equivalence can be used to divide up numbers based on their remainders when
divided by 𝑛
o Let 𝑎, 𝑏, 𝑛 ∈ ℤ with 𝑛 ≠ 0. Then 𝑎 ≡ 𝑏 (mod 𝑛) if and only if 𝑎 and 𝑏 have the
same remainder when divided by 𝑛
- Let 𝑎, 𝑏, 𝑐, 𝑛 ∈ ℤ with 𝑛 ≠ 0. Then the following hold:
o 𝑎 ≡ 𝑎 (mod 𝑛)
o If 𝑎 ≡ 𝑏 (mod 𝑛) then 𝑏 ≡ 𝑎 (mod 𝑛)
o If 𝑎 ≡ 𝑏 (mod 𝑛) and 𝑏 ≡ 𝑐 (mod 𝑛) the 𝑎 ≡ 𝑐 (mod 𝑛)
- Let 𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ with 𝑛 ≠ 0. If 𝑎 ≡ 𝑐 (mod 𝑛) and 𝑏 ≡ 𝑑 (mod 𝑛), then the following
hold:
o 𝑎 + 𝑏 ≡ 𝑐 + 𝑑 (mod 𝑛)
o 𝑎 − 𝑏 ≡ 𝑐 − 𝑑 (mod 𝑛)
o 𝑎𝑏 ≡ 𝑐𝑑 (mod 𝑛)
- Addition, subtraction, and multiplication operations preserve modular equivalence
relationships
o However, this is not the case with division

6.2 Proofs with Number Theory

Intro
 - Mathematical proof – how we communicate ideas about the truth or falsehood of a
statement to others
- A proof is made of communication, from the person creating the proof to the person
digesting it
- Audience of our proof: an average computer science student
o Formal
o No assuming much background knowledge

First Examples
- Four parts that leads to a completed proof:
o 1. The statement that we want to prove or disprove
o 2. A translation of the statement into predicate logic
Provides insight into the logical structure of the statement
o 3. A discussion to try to gain some intuition about why the statement is true
Informal
Usually reveals the mathematical insight that forms the content of a
proof
The hardest part of developing a proof
o 4. A formal proof
The “final product” of our earlier work
- Ex. Prove that 23 | 115
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. We will expand the definition of divisibility to rewrite this
statement in terms of simpler operations:
∃k ∈ Z, 115 = 23k
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We just need to divide 115 by 23
o 𝑃𝑟𝑜𝑜𝑓. let 𝑘 = 5
o Then 115 = 23 ⋅ 5 = 23 ⋅ 𝑘 QED
- A typical proof of an existential
o Given statement to prove: ∃𝑥 ∈ 𝑆, 𝑃(𝑥)
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑥 = ______
o [Proof that P(____) is True.] QED
o The two blanks represent the same element of 𝑆, which we get to choose as a
prover
- Ex. Prove that there exists an integer that divides 104
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. We could write ∃𝑎 ∈ ℤ, 𝑎 ∣ 104. Expanding the definition of
divisibility,
∃𝑎, 𝑘 ∈ ℤ, 104 = 𝑎𝑘
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We get to pick both 𝑎 and 𝑘. Any pair of divisors will work
 o 𝑃𝑟𝑜𝑜𝑓. Let 𝑎 = −2 and let 𝑘 = −52
o Then 104 = 𝑎𝑘 QED
- A mathematical proof must introduce all variables contained in the sentence being
proven

Alternating Quantifiers
- Ex. Prove that all integers are divisible by 1
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. The statement contains a universal quantification: ∀𝑛 ∈ ℤ, 1 ∣ 𝑛.
Unpacking the definition of divisibility,
∀𝑛 ∈ ℤ, ∃ 𝑘 ∈ ℤ, 𝑛 = 1 ⋅ 𝑘
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. The statement is valid when 𝑘 equals 𝑛. Introduce the variables in
the same order they are quantified in the statement
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑛 ∈ ℤ. Let 𝑘 = 𝑛
o Then 𝑛 = 1 ⋅ 𝑛 = 1 ⋅ 𝑘 QED
- A typical proof of a universal
o Given statement to prove: ∀𝑥 ∈ 𝑆, 𝑃(𝑥)
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑥 ∈ 𝑆. (i.e. let 𝑥 be an arbitrary element of 𝑆)
o [Proof that 𝑃(𝑥) is True]. QED
- Any existentially-quantified variable can be assigned a value that depends on the
variables defined before it
o In programming, we first initialize a variable n, and then define a new variable k
that is assigned the value of n
- The order of variables in the statement determines the order in which the variables
must be introduced in the proof
o And hence which variables can depend on which other variables
- We cannot use a variable before it’s defined

Proofs Involving Implications

- Ex. Prove that for all integers 𝑥, if 𝑥 divides (𝑥 + 5), then 𝑥 also divides 5
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. There is both a universal quantification and implication in this
statement
∀𝑥 ∈ ℤ, 𝑥 ∣ (𝑥 + 5) ⇒ 𝑥 ∣ 5
o Unpacking the definition of divisibility,
∀𝑥 ∈ ℤ, (∃𝑘1 ∈ ℤ, 𝑥 + 5 = 𝑘1 𝑥) ⇒ (∃𝑘2 ∈ ℤ, 5 = 𝑘2 𝑥)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We are going to assume that 𝑥 divides 𝑥 + 5, and we need to prove
that 𝑥 divides 5
o Since 𝑥 id divisible by x, we should be able to subtract it from 𝑥 + 5 and keep the
result a multiple of x
 o We need to “turn” the equation 𝑥 + 5 = 𝑘1 𝑥 into the equation 5 = 𝑘2 𝑥
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑥 be an arbitrary integer. Assume that 𝑥 ∣ (𝑥 + 5), i.e., that there
exists 𝑘1 ∈ ℤ such that 𝑥 + 5 = 𝑘1 𝑥. We want to prove that there exists 𝑘2 ∈ ℤ
such that 5 = 𝑘2 𝑥
o Let 𝑘2 = 𝑘1 − 1
𝑘2 𝑥 = (𝑘1 − 1)𝑥
= 𝑘1 𝑥 − 𝑥
(𝑥
= + 5) − 𝑥 (we assumed 𝑥 + 5 = 𝑘1 𝑥)
=5
- To prove an implication, we need to assume that the hypothesis is True, and then prove
that the conclusion is also True
- The previous example is not a claim that 𝑥 ∣ (𝑥 + 5) is True; rather, it is a way to
consider what would happen if 𝑥 ∣ (𝑥 + 5) is True
- A typical proof of an implication (direct)
o Given statement to prove: 𝑝 ⇒ 𝑞
o 𝑃𝑟𝑜𝑜𝑓. Assume 𝑝
o [Proof that 𝑞 is True.] QED

Generalizing our Example

- Generalization – taking a true statement (and a proof of the statement) and then
replacing a concrete value in the statement with a universally quantified variable
- Ex. Prove that for all 𝑑 ∈ ℤ, and for all 𝑥 ∈ ℤ, if 𝑥 divides (𝑥 + 𝑑), then 𝑥 also divides 𝑑
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. Same translation as the previous example, except now we have an
extra variable:
∀𝑑, 𝑥 ∈ ℤ, (∃𝑘1 ∈ ℤ, 𝑥 + 𝑑 = 𝑘1 𝑥) ⇒ (∃𝑘2 ∈ ℤ, 𝑑 = 𝑘2 𝑥)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We should be able to use the same set of calculations as the
previous example
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑑 and 𝑥 be arbitrary integers. Assume that 𝑥 ∣ (𝑥 + 𝑑), i.e., there
exists 𝑘1 ∈ ℤ such that 𝑥 + 𝑑 = 𝑘1 𝑥. We want to prove that there exists 𝑘2 ∈ ℤ
such that 𝑑 = 𝑘2 𝑥.
o Let 𝑘2 = 𝑘1 − 1
𝑘2 𝑥 = (𝑘1 − 1)𝑥
= 𝑘1 𝑥 − 𝑥
= (𝑥 + 𝑑) − 𝑥
=𝑑
QED
o The above proof is basically the same as the previous one
We have simply swapped out all the 5’s with 𝑑’s
 The proof did not depend on the value 5
The original statement and proof generalize to this second version
- The more general the statement, the more useful it becomes
o i.e. the exponent laws
They apply to every possible number, regardless of what our concrete
calculation is
We know we can use them in our calculations

6.3 Proofs and Algorithms I: Primality Testing

Intro
- Function that determines whether p is prime
def is_prime(p: int) -> bool:
“””Return whether p is prime.”””
possible_divisors = range(1, p+1)
return (
p > 1 and
all({d == 1 or d == p for d in possible_divisors if divides(d, p)})
)
o It is a direct translation of the mathematical definition of prime numbers, with
the only difference being our restriction of the range of possible divisors
o This algorithm is inefficient because it checks more numbers than necessary
o The range of possible divisors extends only to the square root of the input p
from math import floor, sqrt

def is_prime(p: int) -> bool:

“””Return whether p is prime.”””
possible_divisors = range(2, floor(sqrt(p)) + 1)
return (
p > 1 and
all({not divides(d, p) for d in possible_divisors})
)
o This version is faster as the range of possible divisors to check is smaller
o We need proofs to know that this version of is_prime is correct

A Property of Prime Numbers

- Theorem. Let 𝑝 ∈ ℤ, 𝑃𝑟𝑖𝑚𝑒(𝑝) ⟺ (𝑝 > 1 ∧ (∀𝑑 ∈ ℕ, 2 ≤ 𝑑 ≤ √𝑝 ⟹ 𝑑 ∤ 𝑝))
- 𝑝 ⟺ 𝑞 is equivalent to (𝑝 ⟹ 𝑞) ∧ (𝑞 ⟹ 𝑝)
 o To argue that a biconditional is True, we do so by proving the two different
implications
- A typical proof of a biconditional
o Given statement to prove: 𝑝 ⟺ 𝑞
o 𝑃𝑟𝑜𝑜𝑓. This proof is divided into two parts.
o Part 1 (𝑝 ⟹ 𝑞): Assume 𝑝
o [Proof that 𝑞 is True]
o Part 2 (𝑞 ⟹ 𝑝): Assume 𝑞
o [Proof that 𝑝 is True]
- Proving the first implication
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We assume that 𝑝 is prime, and will need to prove:
𝑝>1
∀𝑑 ∈ ℕ, 2 ≤ 𝑑 ≤ √𝑝 ⟹ 𝑑 ∤ 𝑝
o Definition of prime in predicate logic:
𝑃𝑟𝑖𝑚𝑒(𝑝) ∶ 𝑝 > 1 ∧ ( ∀𝑑 ∈ ℕ, 𝑑 ∣ 𝑝 ⟹ 𝑑 = 1 ∨ 𝑑 = 𝑝 )
o If 𝑑 is between 2 and √𝑝, then it can’t equal 1 or 𝑝, which are the only possible
divisors of 𝑝
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑝 ∈ ℤ and assume that 𝑝 is prime. We need to prove that 𝑝 > 1 and
for all 𝑑 ∈ ℕ, if 2 ≤ 𝑑 ≤ √𝑝 then 𝑑 does not divide 𝑝
o Part 1: proving that 𝑝 > 1
o By the definition of prime, we know that 𝑝 > 1
o Part 2: proving that for all 𝑑 ∈ ℕ, if 2 ≤ 𝑑 ≤ √𝑝 then 𝑑 does not divide 𝑝
o First, since 2 ≤ 𝑑, we know 𝑑 > 1, and so 𝑑 ≠ 1. Second, since 𝑝 > 1, we know
that √𝑝 < 𝑝, and so 𝑑 ≤ √𝑝 < 𝑝
o This means that 𝑑 ≠ 1 and 𝑑 ≠ 𝑝. By the definition of prime, we can conclude
that 𝑑 ∤ 𝑝 QED
o When the input p is a prime number, we know that the expressions p > 1 and
all(not divides(d, p) for d in possible_divisors) will both evaluate to True, and so
the function will return True
We’ve proven that is_prime returns the correct value for every prime
number, without a single test case
- Proving the second implication
o At this point, we’ve said nothing about how is_prime behaves when given a non-
prime number
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We now need to prove the converse of the first implication: if 𝑝 >
1 and ∀𝑑 ∈ ℕ, 2 ≤ 𝑑 ≤ √𝑝 ⟹ 𝑑 ∤ 𝑝, then 𝑝 must be prime
 o Expanding the definition of prime, we need to prove that 𝑝 > 1 (which we’ve
assumed) and for all 𝑑1 ∈ ℕ, 𝑑1 ∣ 𝑝 ⟹ 𝑑1 = 1 ∨ 𝑑1 = 𝑝
o Idea: Let 𝑑1 ∈ ℕ and assume 𝑑1 ∣ 𝑝, and use the condition that ∀𝑑 ∈ ℕ, 2 ≤ 𝑑 ≤
√𝑝 ⟹ 𝑑 ∤ 𝑝 to prove that 𝑑1 is 1 or 𝑝
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑝 ∈ ℕ, and assume 𝑝 > 1 and that ∀𝑑 ∈ ℕ, 2 ≤ 𝑑 ≤ √𝑝 ⟹ 𝑑 ∤ 𝑝.
We want to prove that 𝑝 is prime, i.e., that 𝑝 > 1 and that 𝑑1 ∈ ℕ, 𝑑1 ∣ 𝑝 ⟹
𝑑1 = 1 ∨ 𝑑1 = 𝑝
o For the first part, 𝑝 > 1 is true because it’s one of our assumptions. For the
second part, first let 𝑑1 ∈ ℕ, and assume 𝑑1 ∣ 𝑝. We’ll prove that 𝑑1 = 1 ∨ 𝑑1 =
𝑝
o From our second assumption, we know that since 𝑑1 ∣ 𝑝, it is not between 2 and
√𝑝. So then either 𝑑1 < 2 or 𝑑1 > √𝑝. We divide our proof into two cases based
on these possibilities
o Case 1: assume 𝑑1 < 2
o Since 𝑑1 ∈ ℕ, it must be 0 or 1 in this case. We know 0 ∤ 𝑝 because 𝑝 > 1, and
so 𝑑1 = 1
o Case 2: assume 𝑑1 > √𝑝
o Since we assumed 𝑑1 ∣ 𝑝, we expand the definition of divisibility to conclude that
𝑝 𝑝
∃𝑘 ∈ ℤ, 𝑝 = 𝑑1 𝑘. Since 𝑑1 > √𝑝 in this case, we know that 𝑘 = 𝑑 < = √𝑝
1 √𝑝
o Since 𝑝 = 𝑑1 𝑘, we know that 𝑘 ∣ 𝑝 as well, and so our second assumption
applied to 𝑘 tells us that 𝑘 is not between 2 and √𝑝
𝑝
o So 𝑘 < √𝑝 and is not between 2 and √𝑝. Therefore 𝑘 = 1, and so 𝑑1 = 𝑘 = 𝑝
o QED
o What we’ve proved is that if is_prime(p) returns True, then p must be prime.
The contrapositive: if p is not prime, then is_prime(p) returns False
- Putting the two implications together, we have:
o For all integers p, if p is prime then is_prime(p) returns True
o For all integers p, if is_prime(p) returns True the p is prime
- Since every integer p is either prime or not prime, we can conclude that this
implementation of is_prime is correct according to its specification

Algorithm Correctness and Theoretical Properties

- The correctness of our algorithm is derived from the theoretical properties of prime
numbers that we expressed in formal predicate logic

6.4 Proof by Cases and Disproofs

Proof by Cases
- When different arguments are required for different elements, we divide the domain
into different parts, and then write a separate argument for each part
- We pick a set of unary predicates 𝑃1 , 𝑃2 , … , 𝑃𝑘 (for some positive integer 𝑘), such that
for every element 𝑥 in the domain, 𝑥 satisfies at least one of the predicates (we say that
these predicates are exhaustive)
- In our previous example, we started with a domain “𝑑1 ∈ ℕ” and then narrowed this to
“𝑑1 ∈ ℕ 𝑎𝑛𝑑 (𝑑1 < 2 ∨ 𝑑1 > √𝑝)“, leading to the following predicates for our cases:
𝑃1 (𝑑1) ∶ 𝑑1 < 2, 𝑃2 (𝑑1 ) ∶ 𝑑1 > √𝑝
- Then, we divide the proof body into cases, where in each case we assume that one of
the predicates is True, and use that assumption to construct a proof that specifically
works under that assumption
- A typical proof by cases
o Given statement to prove: ∀𝑥 ∈ 𝑆, 𝑃(𝑥). Pick a set of exhaustive predicates
𝑃1 , … , 𝑃𝑘 of 𝑆
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑥 ∈ 𝑆. We will use proof by cases
o Case 1. Assume 𝑃1 (𝑥) is True
o [Proof that 𝑃(𝑥) is True, assuming 𝑃1 (𝑥)]
o Case 2. Assume 𝑃2 (𝑥) is True
o [Proof that 𝑃(𝑥) is True, assuming 𝑃2 (𝑥)]
o ⋮
o Case 𝒌. Assume 𝑃𝑘 (𝑥) is True
o [Proof that 𝑃(𝑥) is True, assuming 𝑃𝑘 (𝑥)] QED
- A simple proof which works for all elements of the domain is preferable than a proof by
cases

Cases and the Quotient-Remainder Theorem

- Theorem. (Quotient-Remainder Theorem) For all 𝑛 ∈ ℤ and 𝑑 ∈ ℤ+ , there exist 𝑞 ∈
ℤ and 𝑟 ∈ ℕ such that 𝑛 = 𝑞𝑑 + 𝑟 and 0 ≤ 𝑟 < 𝑑. Moreover, these 𝑞 and 𝑟 are unique
for a given 𝑛 and 𝑑
- We say that 𝑞 is the quotient when 𝑛 is divided by 𝑑, and that 𝑟 is the remainder when 𝑛
is divided by 𝑑
- This theorem tells us that for any non-zero divisor 𝑑 ∈ ℤ+ , we can separate all possible
integers into 𝑑 different groups, corresponding to their possible remainders (between 0
and 𝑑 − 1) when divided by 𝑑
- Ex. Prove that for all integers 𝑥, 2 ∣ 𝑥 2 + 3𝑥
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. Expanding the definition of divisibility,
∀𝑥 ∈ ℤ, ∃𝑘 ∈ ℤ, 𝑥 2 + 3𝑥 = 2𝑘
 o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We want to “factor out a 2” from the expression 𝑥 2 + 3𝑥, but this
only works if 𝑥 is even. If 𝑥 is odd, then both 𝑥 2 and 3𝑥 will be odd, and adding 2
odd numbers together produces an even number.
o With the Quotient-Remainder Theorem, we can:
“Know” that every number has to be either even or odd
Formalize the algebraic operations of “factoring out a 2” and “adding 2
odd numbers together”
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑥 ∈ ℤ. By the Quotient-Remainder Theorem, we know that when 𝑥 is
divided by 2, the two possible remainders are 0 and 1.
o Case 1: assume the remainder when 𝑥 is divided by 2 is 0, i.e., we assume there
exists 𝑞 ∈ ℤ such that 𝑥 = 2𝑞 + 0. We will show that there exists 𝑘 ∈ ℤ such
that 𝑥 2 + 3𝑥 = 2𝑘
o We have:
𝑥 2 + 3𝑥 = (2𝑞)2 + 3(2𝑞)
= 4𝑞2 + 6𝑞
= 2(2𝑞2 + 3𝑞)
o So let 𝑘 = 2𝑞2 + 3𝑞. Then 𝑥 2 + 3𝑥 = 2𝑘
o Case 2: assume the remainder when 𝑥 is divided by 2 is 1, i.e., we assume there
exists 𝑞 ∈ ℤ such that 𝑥 = 2𝑞 + 1. We will show that there exists 𝑘 ∈ ℤ such
that 𝑥 2 + 3𝑥 = 2𝑘
o We have:
𝑥 2 + 3𝑥 = (2𝑞 + 1)2 + 3(2𝑞 + 1)
= 4𝑞2 + 4𝑞 + 1 + 6𝑞 + 3
= 2(2𝑞2 + 5𝑞 + 2)
o So let 𝑘 = 2𝑞2 + 5𝑞 + 2. Then 𝑥 2 + 3𝑥 = 2𝑘 QED

False Statements and Disproofs

- An absence of proof is not enough to convince us that the statement is False
- Disproof – a proof that the negation of the statement is True
- Ex. Disprove the following statement: every natural number divides 360
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. The statement can be written as ∀𝑛 ∈ ℕ, 𝑛 ∣ 360
o To prove it False, we need its negation
¬( ∀𝑛 ∈ ℕ, 𝑛 ∣ 360 )
∃𝑛 ∈ ℕ, 𝑛 ∤ 360
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. The number 7 doesn’t divide 360
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑛 = 7
360
o Then 𝑛 ∤ 360, 𝑠𝑖𝑛𝑐𝑒 7
= 51.428 … is not an integer. QED
 - Counterexample – the value that causes the predicate to be False (or causes the
negation of the predicate to be True)
o The value 7 is the counterexample for the previous example
- A typical disproof of a universal (counterexample)
o Given statement to disprove: ∀𝑥 ∈ 𝑆, 𝑃(𝑥)
o 𝑃𝑟𝑜𝑜𝑓. We prove the negation, ∃𝑥 ∈ 𝑆, ¬𝑃(𝑥). Let 𝑥 = ______
o [Proof that ¬𝑃(______) is True] QED

6.5 Greatest Common Divisor

Intro
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑥, 𝑦, 𝑑 ∈ ℤ. We say that 𝑑 is a common divisor of 𝑥 and 𝑦 when 𝑑
divides 𝑥 and 𝑑 divides 𝑦
- We say that 𝑑 is the greatest common divisor of 𝑥 and 𝑦 when it is the largest number
that is a common divisor or 𝑥 and 𝑦, or 0 when 𝑥 and 𝑦 are both 0
- We can define the function gcd∶ ℤ × ℤ → ℕ as the function which takese numbers 𝑥 and
𝑦, and returns their greatest common divisor
- If 𝑒 is any number which divides 𝑚 and 𝑛, then 𝑒 ≤ 𝑑
- Let 𝑚, 𝑛, 𝑑 ∈ ℤ, and suppose 𝑑 = gcd(𝑚, 𝑛), then 𝑑 satisfies the following:
(𝑚 = 0 ∧ 𝑛 = 0 ⟹ 𝑑 = 0) ∧
(𝑚 ≠ 0 ∨ 𝑛 ≠ 0 ⟹ 𝑑 ∣ 𝑚 ∧ 𝑑 ∣ 𝑛 ∧ ( ∀𝑒 ∈ ℕ, 𝑒 ∣ 𝑚 ∧ 𝑒 ∣ 𝑛 ⟹ 𝑒 ≤ 𝑑 ))
- Ex. Prove that for all integers 𝑝 and 𝑞, if 𝑝 and 𝑞 are distinct primes, then 𝑝 and 𝑞 are
coprime, meaning gcd(𝑝, 𝑞) = 1
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. Structure of the above statement:
∀𝑝, 𝑞 ∈ ℤ, (𝑃𝑟𝑖𝑚𝑒(𝑝) ∧ 𝑃𝑟𝑖𝑚𝑒(𝑞) ∧ 𝑝 ≠ 𝑞) ⟹ gcd(𝑝, 𝑞) = 1
o We could unpack the definitions of 𝑃𝑟𝑖𝑚𝑒 and gcd, but it would not be
necessary
To show that gcd(𝑝, 𝑞) = 1, we just need to make sure that neither 𝑝 nor
𝑞 divides the other
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑝, 𝑞 ∈ ℤ. Assume that 𝑝 and 𝑞 are both prime, and that 𝑝 ≠ 𝑞. We
want to prove that gcd(𝑝, 𝑞) = 1
o By the definition of prime, 𝑝 ≠ 1 (since 𝑝 > 1)
o The only positive divisors of 𝑞 are 1 and 𝑞 itself
o Since we assumed 𝑝 ≠ 𝑞 and concluded 𝑝 ≠ 1, we know that 𝑝 ∤ 𝑞
o Since we know that 1 divides every number, 1 is the only positive common
divisor of 𝑝 and 𝑞, so gcd(𝑝, 𝑞) = 1 QED

Linear Combinations and the Greatest Common Divisor

 - 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑚, 𝑛, 𝑎 ∈ ℤ. We say that 𝑎 is a linear combination of 𝑚 and 𝑛 when
there exist 𝑝, 𝑞 ∈ ℤ such that 𝑎 = 𝑝𝑚 + 𝑞𝑛
o i.e. 101 is a linear combination of 5 and 3, since 101 = 10 ⋅ 5 + 17 ⋅ 3
- Theorem. (Divisibility of Linear Combinations) Let 𝑚, 𝑛, 𝑑 ∈ ℤ. If 𝑑 divides 𝑚 and 𝑑
divides 𝑛, then 𝑑 divides every linear combination of 𝑚 and 𝑛
- Theorem. (GCD Characterization) Let 𝑚, 𝑛, 𝑑 ∈ ℤ, and assume at least one of them is
non-zero. Then gcd(𝑚, 𝑛) is the smallest positive integer that is a linear combination of
𝑚 and 𝑛
- Ex. For all 𝑚, 𝑛, 𝑑 ∈ ℤ, if 𝑑 divides both 𝑚 and 𝑛 then 𝑑 also divides gcd(𝑚, 𝑛)
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. We can translate this statement as
∀𝑚, 𝑛, 𝑑 ∈ ℤ, 𝑑 ∣ 𝑚 ∧ 𝑑 ∣ 𝑛 ⟹ 𝑑 ∣ gcd(𝑚, 𝑛)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. By the GCD Characterization Theorem, we can write
gcd(𝑚, 𝑛) as 𝑝𝑚 + 𝑞𝑛
Any number that divides 𝑚 and 𝑛 will divide 𝑝𝑚 + 𝑔𝑛 as well
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑚, 𝑛, 𝑑 ∈ ℤ. Assume that 𝑑 ∣ 𝑚 and 𝑑 ∣ 𝑛. We want to prove that 𝑑 ∣
gcd(𝑚, 𝑛). We’ll divide our proof into two cases.
o Case 1: assume 𝑚 = 0 and 𝑛 = 0
o By the definition of gcd we know that gcd(𝑚, 𝑛) = 0. So 𝑑 ∣ gcd(𝑚, 𝑛), since we
assumed that 𝑑 divides 0
o Case 2: assume 𝑚 ≠ 0 or 𝑛 ≠ 0
o By the GCD Characterization Theorem, there exists integers 𝑝, 𝑞 ∈ ℤ such that
gcd(𝑚, 𝑛 ) = 𝑝𝑚 + 𝑞𝑛
o By the Divisibility of Linear Combinations Theorem, since 𝑑 ∣ 𝑎 and 𝑑 ∣ 𝑏 (by
assumption), we know that 𝑑 ∣ 𝑝𝑚 + 𝑞𝑛
o Therefore, 𝑑 ∣ gcd(𝑚, 𝑛) QED

6.6 Proofs and Algorithms II: Computing the Greatest Common Divisor

Naively Searching for the GCD

- We had an implementation of the predicate as a function called divides
def divides(d: int, n: int) -> bool:
“””Return whether d divides n.”””
if d == 0:
return n == 0
else:
return n % d == 0
- With the above function, we can implement a gcd function
def naive_gcd(m: int, n: int) -> int:
 “””Return the gcd of m and n.”””
if m == 0 and n == 0:
return 0
else:
possible_divisors = range(1, min(abs(m), abs(n) + 1))
return max({d for d in possible_divisors if divides(d, m) and
divides(d, n)

GCD and Remainders

- Theorem. (Quotient-Remainder Theorem) For all 𝑛 ∈ ℤ and 𝑑 ∈ ℤ, if 𝑑 ≠ 0 then there
exist 𝑞 ∈ ℤ and 𝑟 ∈ ℕ such that 𝑛 = 𝑞𝑑 + 𝑟 and 0 ≤ 𝑟 < |𝑑|. Moreover, these 𝑞 and 𝑟
are unique for a given 𝑛 and 𝑑
- We say that 𝑞 is the quotient when 𝑛 is divided by 𝑑, and that 𝑟 is the remainder when
𝑛 is divided by 𝑑, and write 𝑟 = 𝑛 % 𝑑
- Theorem. For all 𝑎, 𝑏 ∈ ℤ where 𝑏 ≠ 0, gcd(𝑎, 𝑏) = gcd(𝑏, 𝑎 % 𝑏)
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. ∀𝑎, 𝑏 ∈ ℤ, 𝑏 ≠ 0 ⟹ gcd(𝑎, 𝑏) = gcd(𝑏, 𝑎 % 𝑏)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We’ll define the variable 𝑑 = gcd(𝑏, 𝑎 % 𝑏), and prove that 𝑑 =
gcd(𝑎, 𝑏) as well. To do so, we’ll need to prove that 𝑑 divides both 𝑎 and 𝑏, and
that it is greater than every other common divisor of 𝑎 and 𝑏
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑎, 𝑏 ∈ ℤ and assume 𝑏 ≠ 0. Let 𝑟 = 𝑎 % 𝑏 (the remainder when 𝑎 is
divided by 𝑏). We need to prove that gcd(𝑎, 𝑏) = gcd(𝑏, 𝑟)
o Let 𝑑 = gcd(𝑏, 𝑟). We’ll prove that 𝑑 = gcd(𝑎, 𝑏) as well, by proving three
things: that 𝑑 ∣ 𝑎, that 𝑑 ∣ 𝑏, and that every common divisor of 𝑎 and 𝑏 is ≤ 𝑑
o Part 1: proving that 𝑑 ∣ 𝑎
o By our definition of 𝑟 and the Quotient-Remainder Theorem, there exists 𝑞 ∈ ℤ
such that 𝑎 = 𝑞𝑏 + 𝑟. Since 𝑑 = gcd(𝑏, 𝑟), 𝑑 divides both 𝑏 and 𝑟. By the
Divisibility of Linear combinations Theorem, 𝑑 ∣ 𝑞𝑏 + 𝑟, and so 𝑑 ∣ 𝑎
o Part 2: proving that 𝑑 ∣ 𝑏
o By the definition of gcd, 𝑑 ∣ 𝑏
o Part 3: proving that every common divisor of 𝑎 and 𝑏 is ≤ 𝑑
o Let 𝑑1 ∈ ℤ and assume that 𝑑1 ∣ 𝑎 and 𝑑1 ∣ 𝑏. We will prove that 𝑑1 ≤ 𝑑
o First, we’ll prove that 𝑑1 ∣ 𝑟. By the Quotient-Remainder Theorem, the equation
𝑎 = 𝑞𝑏 + 𝑟 can be rewritten as 𝑟 = 𝑎 − 𝑞𝑏. Then using our assumption that 𝑑1
is a common divisor of 𝑎 and 𝑏, by the Divisibility of Linear Combinations
Theorem, 𝑑1 ∣ 𝑟
o So then 𝑑1 ∣ 𝑏 (by our assumption), and 𝑑1 ∣ 𝑟, and so it is a common divisor of 𝑏
and 𝑟. Therefore by the definition of gcd, 𝑑1 ≤ gcd(𝑏, 𝑟) = 𝑑 QED
GCD, Remainders, and a New Algorithm
- The theorem above suggests a possible way of computing the gcd of 2 numbers in an
iterative (repeated) fashion
o i.e. 24 and 16
Since 24 % 16 = 8, gcd(24, 16) = gcd(16, 8)
Since 16 % 8 = 0, gcd(16, 8) = gcd(8, 0)
Since the gcd of any positive integer 𝑛 and 0 is simply 𝑛 itself, gcd(8, 0) =
8
gcd(24, 16) = 8
- The above algorithm of computing the gcd of 2 numbers is known as the Euclidean
algorithm
- Euclidean Algorithm
o Given: integers a and b. Returns: gcd(a, b)
o 1. Initialize 2 variables x, y to the given numbers a and b
o 2. Let r be the remainder when x is divided by y
o 3. Reassign x and y to y and r, respectively
o 4. Repeat steps 2 and 3 until y is 0
o 5. At this point, x refers to the gcd of a and b
- Visualizing the changing values of x and y
Iteration x y
0 24 16
1 16 8
2 8 0

The While Loop

- A while loop looks similar to an if statement
while <condition>:
<statement>
…
o Unlike an if statement, after executing its body the while loop will check the
condition again. If the condition still evaluates to True, then the body is repeated
- Example
>>> numbers = []
>>> number = 1
>>> while number < 100:
... numbers.append(number)
... number = number * 2
...
 >>> numbers
[1, 2, 4, 8, 16, 32, 64]
o number appear in both the while loop’s body and its condition
o number increases at each iteration
o Eventually, number refers to the value 128 and the while loop is done because
128 < 100 evaluates to False
o The number of iterations is dependent on the initial value of number

Implementing the Euclidean Algorithm

def euclidean_gcd(a: int, b: int) -> int:
“””Return the gcd of a and b.”””
# Step 1: initialize x and y
x=a
y=b
while y != 0: # Step 4: repeat Steps 2 and 3 until y is 0
# Step 2: calculate the remainder of x divided by y
r=x%y

# Step 3: reassign x and y

x=y
y=r

# Step 5: x now refers to the gcd of a and b

return x
o Step 1, initializing x and y, occurs in the code before the while loop begins
o Steps 2 and 3 are performed inside the loop body
o Step 4, the repetition, is achieved by the while loop. Rather than specifying a
stopping condition, we must write a continuing condition (the negation of the
stopping condition)
“until y = 0” becomes “while y != 0”
o Step 5, the return value, is exactly what is specified by the algorithm
Iteration x y
0 24 16
1 16 8
2 8 0
o We don’t have a typical accumulator pattern
Both x and y are loop variables for the while loop
- In a for loop, the loop variable is initialized and reassigned automatically by the Python
interpreter to each element of the collection being looped over
- In a while loop, the loop variable(s) must be initialized and reassigned explicitlyin code
that we write
- Use for loops where possible (when we have an explicit collection to loop over) and
reserve while loops for situations that can’t be easily implemented with a for loop
- Parallel assignment
o Suppose we had swapped the last two lines of the loop body
while y != 0:
r=x%y
y=r
x=y
x = y assigns x to the new value of y rather than its old one
o Parallel assignment – a feature in which multiple variables can be assigned in the
same statement
while y != 0:
r=x%y
x, y = y, r
o The above assignment statement is evaluated as follows:
First, the right side y, r is evaluated, producing two objects
Then, each object is assigned to the corresponding variable on the left
side
o The right side is fully evaluated before any variable reassignment occurs
Order does not matter
o Rewriting euclidean_gcd using parallel assignment:
def euclidean_gcd(a: int, b: int) -> int:
“””Return the gcd of a and b.”””
x, y = a, b

while y != 0:
r=x%y
x, y = y, r

return x
- Documenting loop properties: loop invariants
o The Euclidean Algorithm relies on a key property – gcd(x, y) == gcd(y, x % y)
Even though x and y change, their gcd doesn’t
o gcd(x, y) == gcd(a, b)
 This statement is called a loop invariant
Loop invariant – a property about loop variables that must be true at the
start and end of each loop iteration
o By convention, we document loop invariants at the top of a loop body using an
assert statement
def euclidean_gcd(a: int, b: int) -> int:
“””Return the gcd of a and b.”””
x, y = a, b

while y != 0:
# assert naive_gcd(x, y) == naive_gcd(a, b) # loop invariant
r=x%y
x, y = y, r

return x
o After the loop stops, the loop invariant should tell us that gcd(x, 0) == gcd(a, b),
and so we know that x == gcd(a, b), which is why x is returned
o To know for sure whether a loop invariant is correct, we need a proof

6.7 Modular Arithmetic

Intro
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑎, 𝑏, 𝑛 ∈ ℤ, and assume 𝑛 ≠ 0. We say that 𝑎 is equivalent to 𝑏 modulo
𝑛 when 𝑛 ∣ 𝑎 − 𝑏. In this case, we write 𝑎 ≡ 𝑏 (mod 𝑛)
o 𝑎 and 𝑏 have the same remainder when divided by 𝑛
- Theorem. For all 𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ, if 𝑛 ≠ 0, if 𝑎 ≡ 𝑐 (mod 𝑛) and 𝑏 ≡ 𝑑 (mod 𝑛), then:
1. 𝑎 + 𝑏 ≡ 𝑐 + 𝑑 (mod 𝑛)
2. 𝑎 − 𝑏 ≡ 𝑐 − 𝑑 (mod 𝑛)
3. 𝑎𝑏 ≡ 𝑐𝑑(mod 𝑛)
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛 1.
∀𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ, (𝑛 ≠ 0 ∧ ( 𝑛 ∣ 𝑎 − 𝑐 ) ∧ ( 𝑛 ∣ 𝑏 − 𝑑 )) ⟹
𝑛 ∣ (𝑎 + 𝑏) − (𝑐 + 𝑑)
o 𝑃𝑟𝑜𝑜𝑓 1. Let 𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ. Assume that 𝑛 ≠ 0, 𝑛 ∣ 𝑎 − 𝑐, and 𝑛 ∣ 𝑏 − 𝑑. This
means we want to prove that 𝑛 ∣ (𝑎 + 𝑏) − (𝑐 + 𝑑).
o By the Divisibility of Linear Combinations Theorem, since 𝑛 ∣ (𝑎 − 𝑐) and
𝑛 ∣ (𝑏 − 𝑑), it divides their sum:
𝑛 ∣ (𝑎 − 𝑐) + (𝑏 − 𝑑)
𝑛 ∣ (𝑎 + 𝑏) − (𝑐 + 𝑑)
 QED
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛 2.
∀𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ, (𝑛 ≠ 0 ∧ ( 𝑛 ∣ 𝑎 − 𝑐 ) ∧ ( 𝑛 ∣ 𝑏 − 𝑑 )) ⟹
𝑛 ∣ (𝑎 − 𝑏) − (𝑐 − 𝑑)
o 𝑃𝑟𝑜𝑜𝑓 2. Let 𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ. Assume that 𝑛 ≠ 0, 𝑛 ∣ 𝑎 − 𝑐, and 𝑛 ∣ 𝑏 − 𝑑. This
means we want to prove that 𝑛 ∣ (𝑎 − 𝑏) − (𝑐 − 𝑑).
o By the Divisibility of Linear Combinations Theorem, since 𝑛 ∣ (𝑎 − 𝑐) and
𝑛 ∣ (𝑏 − 𝑑), it divides their difference:
𝑛 ∣ (𝑎 − 𝑐) − (𝑏 − 𝑑)
𝑛 ∣ (𝑎 − 𝑏) − (𝑐 − 𝑑)
QED
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛 3.
∀𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ, (𝑛 ≠ 0 ∧ ( 𝑛 ∣ 𝑎 − 𝑐 ) ∧ ( 𝑛 ∣ 𝑏 − 𝑑 )) ⟹
𝑛 ∣ 𝑎𝑏 − 𝑐𝑑
o 𝑃𝑟𝑜𝑜𝑓 3. Let 𝑎, 𝑏, 𝑐, 𝑑, 𝑛 ∈ ℤ. Assume that 𝑛 ≠ 0, 𝑛 ∣ 𝑎 − 𝑐, and 𝑛 ∣ 𝑏 − 𝑑. This
means we want to prove that 𝑛 ∣ 𝑎𝑏 − 𝑐𝑑.
o Expanding the definition of division, we want to show that
∃𝑘 such that 𝑎𝑏 − 𝑐𝑑 = 𝑘𝑛
o Take 𝑘 = 𝑛(𝑞𝑎 𝑞𝑏 − 𝑞𝑐 𝑞𝑑 ) + 𝑟1 (𝑞𝑏 − 𝑞𝑑 ) + 𝑟2 (𝑞𝑎 − 𝑞𝑐 )
o By the Quotient-Remainder Theorem,
𝑎 = 𝑛𝑞𝑎 + 𝑟1
𝑐 = 𝑛𝑞𝑐 + 𝑟1
𝑏 = 𝑛𝑞𝑏 + 𝑟2
𝑑 = 𝑛𝑞𝑑 + 𝑟2
2
𝑎𝑏 = 𝑛 𝑞𝑎 𝑞𝑏 + 𝑛𝑟1 𝑞𝑏 + 𝑛𝑟2 𝑞𝑎 + 𝑟1 𝑟2
𝑐𝑑 = 𝑛 2 𝑞𝑐 𝑞𝑑 + 𝑛𝑟1 𝑞𝑑 + 𝑛𝑟2 𝑞𝑐 + 𝑟1 𝑟2
𝑎𝑏 − 𝑐𝑑 = 𝑛 2 (𝑞𝑎 𝑞𝑏 − 𝑞𝑐 𝑞𝑑 ) + 𝑛𝑟1 (𝑞𝑏 − 𝑞𝑑 ) + 𝑛𝑟2 (𝑞𝑎 − 𝑞𝑐 ) = 𝑘𝑛
QED

Modular Division
- Division does not preserve modular equivalence
- Theorem. (Modular inverse) Let 𝑛 ∈ ℤ+ and 𝑎 ∈ ℤ. If gcd(𝑎, 𝑛) = 1, then there exists
𝑝 ∈ ℤ such that 𝑎𝑝 ≡ 1 (mod 𝑛).
We call this 𝑝 a modular inverse of 𝑎 modulo 𝑛
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. ∀𝑛 ∈ ℤ+ , ∀𝑎 ∈ ℤ, gcd(𝑎, 𝑛) = 1 ⟹ (∃𝑝 ∈ ℤ, 𝑎𝑝 ≡ 1 (mod 𝑛))
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑛 ∈ ℤ+ and 𝑎 ∈ ℤ. Assume gcd(𝑎, 𝑛) = 1
o Since gcd(𝑎, 𝑛) = 1, by the GCD Characterization Theorem we know that there
exist integers 𝑝 and 𝑞 such that 𝑝𝑎 + 𝑞𝑛 = gcd(𝑎, 𝑛) = 1
 o Rearranging the equation, we get that 𝑝𝑎 − 1 = −𝑞𝑛, and so (by the definition
of divisibility, taking 𝑘 = −𝑞), 𝑛 ∣ 𝑝𝑎 − 1
o Then by the definition of modular equivalence, 𝑝𝑎 ≡ 1 (mod 𝑛) QED
- Ex. Let 𝑎 ∈ ℤ and 𝑛 ∈ ℤ+ . If gcd(𝑎, 𝑛) = 1, then for all 𝑏 ∈ ℤ, there exists 𝑘 ∈ ℤ such
that 𝑎𝑘 ≡ 𝑏 (mod 𝑛)
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. ∀𝑎, 𝑛 ∈ ℤ, gcd(𝑎, 𝑛) = 1 ⟹ (∀𝑏 ∈ ℤ, ∃𝑘 ∈ ℤ, 𝑎𝑘 ≡ 𝑏 (mod 𝑛))
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. This is saying that under the given assumptions, 𝑏 is “divisible” by 𝑎
modulo 𝑝.
o Since it is assumed that gcd(𝑎, 𝑛) = 1, we can use the modular inverses
theorem, which gives us a 𝑝 ∈ ℤ such that 𝑎𝑝 ≡ 1 (mod 𝑛)
Looks like we can multiply both sides by 𝑏
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑎 ∈ ℤ and 𝑛 ∈ ℤ+. Assume gcd(𝑎, 𝑛) = 1, and let 𝑏 ∈ ℤ. We want to
prove that there exists 𝑘 ∈ ℤ such that 𝑎𝑘 ≡ 𝑏 (mod 𝑛)
o First, using the Modular Inverses theorem, since we assumed gcd(𝑎, 𝑛) = 1,
there exists 𝑝 ∈ ℤ such that 𝑎𝑝 ≡ 1 (mod 𝑛)
o Second, since modular equivalence preserves multiplication, 𝑎𝑝𝑏 ≡ 𝑏 (mod 𝑛)
o Let 𝑘 = 𝑝𝑏, we have that 𝑎𝑘 ≡ 𝑏 (mod 𝑛) QED

Exponentiation and Order

- Powers of positive integers increase without bound
- Because there are only a finite number of remainders for any given 𝑛 ∈ ℤ+ , for any
𝑎 ∈ ℤ the infinite sequence of remainders of 𝑎0 , 𝑎1 , 𝑎2 , 𝑎3 , … must repeat at some point
o For example, let’s see what happens for each of the possible bases modulo 7:
0: 01 ≡ 0 (mod 7), 02 ≡ 0 (mod 7)
1: 11 ≡ 1 (mod 7), 12 ≡ 1 (mod 7)
2: 21 ≡ 2 (mod 7), 22 ≡ 4 (mod 7), 23 ≡ 1 (mod 7), 24 ≡ 2 (mod 7)
3: 31 ≡ 3 (mod 7), 32 ≡ 2 (mod 7), 33 ≡ 6 (mod 7),
34 ≡ 4 (mod 7), 35 ≡ 5 (mod 7), 36 ≡ 1 (mod 7), 37 ≡ 3 (mod 7)
4: 41 ≡ 4 (mod 7), 42 ≡ 2 (mod 7), 43 ≡ 1 (mod 7), 44 ≡ 4 (mod 7)
o No matter which number we start with, we enter a cycle
i.e. the cycle starting with 2 is [2, 4, 1, 2, …]
• This cycle has length 3, since it takes 3 elements in the sequence
for the 2 to repeat
o Cycle lengths for each possible 𝑎 ∈ {0, 1, … , 6}:
𝑎 Cycle length
0 1
1 1
2 3
 3 6
4 3
5 6
6 2
o For each base other than 0, the cycle length for base 𝑎 is the smallest positive
integer 𝑘 such that 𝑎𝑘 ≡ 1 (mod 7)
i.e. 23 ≡ 1 (mod 7), and the cycle repeats at 24 ≡ 23 ⋅ 2 ≡ 2 (mod 7)
o Cycle length for the remainders modulo 7 always divide 6
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑎 ∈ ℤ and 𝑛 ∈ ℤ+ . We define the order of 𝑎 modulo 𝑛 to be the
smallest positive integer 𝑘 such that 𝑎𝑘 ≡ 1 (mod 𝑛), when such a number exists
o We denote the order of 𝑎 modulo 𝑛 as ord𝑛 (𝑎)
- The table for modulo 17
𝑎 Cycle length
0 1
1 1
2 8
3 16
4 4
5 16
6 16
7 16
8 8
9 8
10 16
11 16
12 16
13 4
14 16
15 8
16 2
o The cycle length for these bases always divides 16
o Again, for each base 𝑎 other than 1, the cycle length corresponding to 𝑎 is the
least positive integer 𝑘 such that 𝑎𝑘 ≡ 1 (mod 17)
- For 17, every base 𝑎 other than 0 satisfies 𝑎16 (mod 17)
o This generalizes to every prime number
- Theorem. (Fermat’s Little Theorem) Let 𝑝, 𝑎 ∈ ℤ and assume 𝑝 is prime and that 𝑝 ∤ 𝑎.
Then 𝑎𝑝−1 ≡ 1 (mod 𝑝)
Euler’s Theorem
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. We define the function 𝜑: ℤ+ → ℕ, called the Euler totient function (or
Euler phi function), as follows:
𝜑(𝑛) = |{ 𝑎 ∣ 𝑎 ∈ {1, … , 𝑛 − 1}, and gcd(𝑎, 𝑛) = 1 }|
o Examples of the Euler totient function:
𝜑(5) = 4, since {1, 2, 3, 4} are all coprime to 5
𝜑(6) = 2, since only {1, 5} are coprime to 6
In general, for any prime number 𝑝, 𝜑(𝑝) = 𝑝 − 1, since all the numbers
{1, 2, …, 𝑝 − 1} are coprime to 𝑝
𝜑(15) = 10, since the numbers {1, 2, 4, 7, 8, 11, 13, 14} are all coprime
to 15.
• Note that “removed” numbers are all multiples of 3 or 5, the
prime factors of 15
In general, for any two distinct primes 𝑝 and 𝑞, 𝜑(𝑝𝑞) = (𝑝 − 1)(𝑞 − 1),
although this is not obvious, and requires a proof
- With the Euler totient function in hand, we can now state the generalization of Fermat’s
Little Theorem
- Theorem. (Euler’s Theorem). For all 𝑎 ∈ ℤ and 𝑛 ∈ ℤ+ , if gcd(𝑎, 𝑛) = 1,
then 𝑎𝜑(𝑛) ≡ 1 (mod 𝑛)

7.1 An Introduction to Cryptography

Cryptography
- Cryptography – study of theoretical and practical techniques for keeping data secure
- Encryption involves turning coherent messages into seemingly-random nonsensical
strings, and then back again

Setting the Stage: Alice and Bob

- Two-party confidential communication – simplest setup in cryptography
o We have two people, Alice and Bob, who wish to send messages to each other
that only they can read
o We also have Eve, who has access to all of the communications between Alice
and Bob, and wants to discover what they’re saying
- Through some shared piece of information called a secret key, Alice and Bob need to
encrypt their messages in such a way that they will each be able to decrypt each other’s
messages
o Eve won’t be able to decrypt the messages without knowing their secret key
 - We define a secure symmetric-key cryptosystem as a system with the following parts:
o A set 𝑃 of possible original messages, called the plaintext messages
E.g. a set of strings
o A set 𝐶 of possible encrypted messages, called the ciphertext, messages
E.g. another set of strings
o A set 𝐾 of possible shared secret keys
Known by both Alice and Bob, but no one else
o Two functions 𝐸𝑛𝑐𝑟𝑦𝑝𝑡 ∶ 𝐾 × 𝑃 → 𝐶 and 𝐷𝑒𝑐𝑟𝑦𝑝𝑡 ∶ 𝐾 × 𝑃 → 𝐶 that satisfies
the following two properties:
Correctness – For all 𝑘 ∈ 𝐾 and 𝑚 ∈ 𝑃, 𝐷𝑒𝑐𝑟𝑦𝑝𝑡(𝑘, 𝐸𝑛𝑐𝑟𝑦𝑝𝑡(𝑘, 𝑚)) = 𝑚
• i.e. if we encrypt and encrypt the same message with the same
key, we get back the original message
Security – For all 𝑘 ∈ 𝐾 and 𝑚 ∈ 𝑃, if an eavesdropper only knows the
value of 𝑐 = 𝐸𝑛𝑐𝑟𝑦𝑝𝑡(𝑘, 𝑚) but does not know 𝑘, it is computationally
infeasible to find 𝑚

Example: Caesar’s Substitution Cipher

- The plaintext and ciphertext sets are strings, and the secret key is some positive integer
𝑘
- Consider messages that consist of uppercase letters and spaces, and associate each
letter with a number as follows:
Character Value Character Value Character Value Character Value
‘A’ 0 ‘H’ 7 ‘O’ 14 ‘V’ 21
‘B’ 1 ‘I’ 8 ‘P’ 15 ‘W’ 22
‘C’ 2 ‘J’ 9 ‘Q’ 16 ‘X’ 23
‘D’ 3 ‘K’ 10 ‘R’ 17 ‘Y’ 24
‘E’ 4 ‘L’ 11 ‘S’ 18 ‘Z’ 25
‘F’ 5 ‘M’ 12 ‘T’ 19 ‘’ 26
‘G’ 6 ‘N’ 13 ‘U’ 20
o In Python, we can implement this conversion as follows:
LETTERS = ‘ABCDEFGHIJKLMNOPQRSTUVWXYZ ‘

def letter_to_num(c: str) -> int:

“””Return the number that corresponds to the given letter.

Preconditions:
- len(c) == 1 and c in LETTERS
“””
 return str.index(LETTERS, c)

def num_to_letter(n: int) -> str:

“””Return the letter that corresponds to the given number.

Preconditions:
- 0 <= n < len(LETTERS)
“””
return LETTERS[n]
- In the Caesar cipher, the secret key 𝑘 is an integer from the set {1, 2, …, 26}
- Before sending any messages, Alice and Bob meet and decide on a secret key from the
set
- When Alice wants to send a string message 𝑚 to Bob, encrypts her message as follows:
o For each letter of 𝑚, Alice shifts it by adding the secret key 𝑘 to its
corresponding numbers, taking remainders modulo 27, the length of LETTERS
o i.e. if 𝑘 = 3, and the plaintext message is ‘HAPPY’, encryption happens as
follows:
Plaintext Corresponding Shifted Ciphertext
Character Integer Integer Character
‘H’ 7 10 ‘K’
‘A’ 0 3 ‘D’
‘P’ 15 18 ‘S’
‘P’ 15 18 ‘S’
‘Y’ 24 0 ‘A’
o When Bob receives the ciphertext ‘KDSSA’, he decrypts the ciphertext by
applying the corresponding shift in reverse
i.e. subtracting the secret key 𝑘 instread of adding it
o We can implement the above example in Python
def encrypt_caesar(k: int, plaintext: str) -> str:
“””Return the encrypted message using the Caesar cipher with key k.

Preconditions:
- all({x in LETTERS for x in plaintext})
- 1 <= k <= 26
“””
l = len(LETTERS)
ciphertext = ‘’
 for letter in plaintext:
ciphertext += num_to_letter((letter_to_num(letter) + k) % l))

return ciphertext

def decrypt_caesar(k: int, ciphertext: str) -> str:

“””Return the decrypted message using the Caesar cipher with key k.

Preconditions:
- all({x in LETTERS for x in plaintext})
- 1 <= k <= 26
“””
l = len(LETTERS)
plaintext = ‘’

for letter in ciphertext:

plaintext += num_to_letter((letter_to_num(letter) – k) % l))

return plaintext

Expanding the Set of Letters

- The key mathematical idea of Caesar generalizes to larger sets of letters
- Recall the following two built-in Python functions
>>> ord(‘A’) # Convert a character into an integer
65
>>> chr(33) # Convert an integer into a character
‘!’
- Using the above two functions, we can modify our encrypt and decrypt functions to
operate on arbitrary Python strings
o Our secret key now takes on values from the set {1, 2, …, 127}
def encrypt_ascii(k: int, plaintext: str) -> str:
“””Return the encrypted message using the Caesar cipher with key k.

Preconditions:
- all({ord(c) < 128 for c in plaintext})
- 1 <= k <= 127
 “””
ciphertext = ‘’

for letter in plaintext:

ciphertext += chr((ord(letter) + k) % 128)

return ciphertext

def decrypt_caesar(k: int, ciphertext: str) -> str:

“””Return the decrypted message using the Caesar cipher with key k.

Preconditions:
- all({ord(c) < 128 for c in plaintext})
- 1 <= k <= 127
“””
plaintext = ‘’

for letter in ciphertext:

plaintext += chr((ord(letter) - k) % 128)

return plaintext
- The Caesar cipher is not secure
o An eavesdropper can try all possible secret keys to decrypt a ciphertext

7.2 The One-Time Pad and Perfect Secrecy

Intro
- The Caesar cipher should never be used in practice
o Consider the ciphertext ‘0LaT0+T^+NZZW’
The 1st and the 5th letters in the plaintext must be the same
The 1st and 10th characters of the plaintext must be consecutive ASCII
characters
o Vulnerable to a brute-force exhaustive key search attack
Given a ciphertext, it is possible to try out every secret key and see which
key yields a meaningful plaintext message

The One-Time Pad

 - Works by shifting each character in the plaintext message, but the shift is not the same
for each character
- Uses a string of length greater than or equal to the length of the plaintext message we
wish to encrypt
- To encrypt a plaintext ASCII message 𝑚 with a secret key 𝑘, for each index 𝑖 between 0
and |𝑚| − 1, we compute:
o (𝑚[𝑖] + 𝑘[𝑖]) % 128, where 𝑚[𝑖] and 𝑘[𝑖] are converted to their numeric
representations to do the arithmetic
- Ex. encrypting the plaintext ‘HELLO’ with the secret key ‘david’
Plaintext Key Ciphertext
H 72 d 100 , 44
E 69 a 97 & 38
+ →
L 76 v 118 B 66
L 76 i 105 5 53
O 79 d 100 3 51
- For decryption, we take the ciphertext 𝑐 and recover the plaintext by subtracting each
letter of the secret key:
o (𝑐[𝑖] − 𝑘[𝑖]) % 128

Perfect Secrecy and Its Costs

- The one-time pad cryptosystem has the property of perfect secrecy
o Perfect secrecy – a ciphertext reveals no information about its corresponding
plaintext other than its length
o i.e. any five-letter plaintext message can be encrypted to obtain ‘,&B53’
- Because of perfect secrecy, an eavesdropper cannot gain any information about the
original plaintext message
- Cost: the secret key must have at least the same length as the message being sent, and
cannot be reused from one message to another

Stream Ciphers
- Stream cipher – a type of symmetric-key cryptosystem that emulate a one-time pad but
share a much smaller secret key
- The shared secret key is small, and both parties use an algorithm to generate an
arbitrary number of new random characters, based on both the secret key and any
previously-generated characters
- Do not have perfect secrecy, since the characters used in encryption aren’t truly
random, though it can appear “random” if the algorithm is good enough
7.3. Computing Shared Secret Keys

Alice and Bob are Mixing Paint

- Suppose that Alice and Bob would like to establish a secret paint colour that only the
two of them know, they would use the following procedure
- Step 1. They both agree on a random, not-secret colour of paint to start with
o i.e. yellow
o Eavesdroppers also know this colour
- Step 2. They each choose their own secret colour, which they will never share with each
other or anyone else
o i.e. Alice chooses red and Bob chooses teal
- Step 3. They each mix their secret colours with their shared colour yellow
o i.e. Alice gets light orange, Bob gets blue
o This is done in secret
- Step 4. They exchange these colours with each other
o This is done publicly
o At this point, there are 3 not-secret colours: yellow and the two mixtures
o There are 2 secret colours: Alice’s red and Bob’s teal
- Step 5. Alice mixes Bob’s blue colour with her original secret red to produce a brown.
Bob mixes Alice’s light orange with his original secret teal to produce the same brown.
o They are the same brown because they both consist of the same mixture of
three colours:
Yellow (shared)
Red (Alice’s secret)
Teal (Bob’s secret)
- Any eavesdropper has access to 3 colours: the original shared yellow, and the two
mixtures orange and blue
o Assume colour mixtures are not easily separated, the eavesdropper cannot
determine what Alice and Bob’s secret colours are, and therefore can’t mix them
together with the yellow to produce the right shade of brown

The Diffie-Hellman Key Exchange

- Setting: two parties, Alice and Bob
- Result: Alice and Bob share a secret key 𝑘
- Step 1. Alice chooses a prime number 𝑝 greater than 2 and an integer 𝑔 which satisfies
2 ≤ 𝑔 ≤ 𝑝 − 1, and sends both to Bob
- Step 2. Alice chooses a secret number 𝑎 ∈ {1, 2, . . , 𝑝 − 1} and sends Bob 𝐴 = 𝑔𝑎 % 𝑝
- Step 3. Bob chooses a secret number 𝑏 ∈ {1, 2, … , 𝑝 − 1} and sends Alice 𝐵 = 𝑔𝑏 % 𝑝
 - Step 4. Alice computes 𝑘𝐴 = 𝐵 𝑎 % 𝑝. Bob computes 𝑘𝐵 = 𝐴𝑏 % 𝑝
o It turns out that 𝑘𝐴 = 𝑘𝐵 , and so this value is chosen as the secret key 𝑘 that
Alice and Bob share
- Example
o 1. Alice starts by choosing 𝑝 = 23 and 𝑔 = 2. She sends both 𝑝 and 𝑔 to Bob
o 2. Alice chooses a secret number 𝑎 = 5. She sends 𝐴 = 𝑔𝑎 % 𝑝 = 25 % 23 = 9
to Bob
o 3. Bob chooses a secret number 𝑏 = 14. He sends 𝐵 = 𝑔𝑏 % 𝑝 = 214 % 23 = 8
to Alice
o 4. Alice computes 𝑘𝐴 = 𝐵 𝑎 % 𝑝 = 85 % 23 = 16. Bob computes 𝑘𝐵 = 𝐴𝑏 % 𝑝 =
914 % 23 = 16. 𝑘𝐴 = 𝑘𝐵 , and so these form the secret key 𝑘

Correctness: Are 𝑘𝐴 and 𝑘𝐵 Always Equal?

- Theorem. (Correctness of Diffie-Hellman key exchange)
∀𝑝, 𝑔, 𝑎, 𝑏 ∈ ℤ+ , (𝑔𝑏 % 𝑝)𝑎 % 𝑝 = (𝑔𝑎 % 𝑝)𝑏 % 𝑝
- 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We can analyze the numbers using modular arithmetic modulo 𝑝. In this
case the calculation involves just switching around exponents in 𝑔𝑎𝑏
- 𝑃𝑟𝑜𝑜𝑓. Let 𝑝, 𝑔, 𝑎, 𝑏 ∈ ℤ+. Let 𝐴 = 𝑔𝑎 % 𝑝 and 𝐵 = 𝑔𝑏 % 𝑝. We will prove that
𝐵 𝑎 % 𝑝 = 𝐴𝑏 % 𝑝
- First, we have that 𝐴 ≡ 𝑔𝑎 (mod 𝑝) and 𝐵 ≡ 𝑔𝑏 (mod 𝑝). So then 𝐴𝑏 ≡ (𝑔𝑎 )𝑏 ≡
𝑔𝑎𝑏 (mod 𝑝), and 𝐵 𝑎 ≡ (𝑔𝑏 )𝑎 ≡ 𝑔𝑎𝑏 (mod 𝑝). Since 𝑔𝑎𝑏 = 𝑔𝑏𝑎 , we can conclude that
𝐴𝑏 ≡ 𝐵 𝑎 (mod 𝑝)
- So then 𝐴𝑏 and 𝐵 𝑎 must have the same remainder when divided by 𝑝, and so 𝐵 𝑎 % 𝑝 =
𝐴𝑏 % 𝑝

Security: How Secret is the Key?

- Over the course of the algorithm, the eavesdropper has access to 𝑝, 𝑔, 𝑔𝑎 % 𝑝, 𝑔𝑏 % 𝑝
o Question: from the information, can the eavesdropper determine the secret key
𝑘?
- The eavesdropper can try to compute 𝑎 and 𝑏 directly. This is an instance of the discrete
logarithm problem
o Discrete logarithm problem – given 𝑝, 𝑔, 𝑦 ∈ ℤ+ , find an 𝑥 ∈ ℤ+ such that
𝑔 𝑥 ≡ 𝑦 (mod 𝑝)
- While we could implement a brute-force algorithm for solving this problem that simply
tries all possible exponents 𝑥 ∈ {0, 1, … , 𝑝 − 1}, this is computationally inefficient in
practice when 𝑝 is chosen to be extremely large
- There is no known efficient algorithm for solving the discrete logarithm problem.
Therefore, the Diffie-Hellman key exchange is computationally secure
 o While there are known algorithms that eavesdroppers could use for determining
the shared secret key, all known algorithms are computationally infeasible for
standard primes chosen
o i.e. Diffie-Hellman key exchanges tend to use primes on the order of 22048 ≈
10617

7.4 The RSA Cryptosystem

Intro
- Limitation of symmetric-key encryption: a secret key needs to be established for every
pair of people who want to communicate
o If there are 𝑛 people who each want to communicate securely with each other,
𝑛(𝑛−1)
there are keys needed
2
- Public-key cryptosystem – each person has two keys:
o A private key known only to them
o A public key known to everyone

Public-Key Cryptography
- Suppose Alice want to send Bob a message
o Alice uses Bob’s public key to encrypt the message
o Bob uses his private key to decrypt the message
- A secure public-key cryptosystem has the following parts:
o A set 𝑃 of possible original messages, called plaintext messages (e.g. a set of
strings)
o A set 𝐶 of possible encrypted messages, called ciphertext messages (e.g. another
set of strings)
o A set 𝐾1 of possible public keys and a set 𝐾2 of possible private keys
o A subset 𝐾 ⊆ 𝐾1 × 𝐾2 of possible public-private key pairs
Not every public key can be paired with every private key
o Two functions 𝐸𝑛𝑐𝑟𝑦𝑝𝑡 ∶ 𝐾1 × 𝑃 → 𝐶 and 𝐷𝑒𝑐𝑟𝑦𝑝𝑡 ∶ 𝐾2 × 𝐶 → 𝑃 that satisfy
the following two properties:
Correctness – For all (𝑘1 , 𝑘2 ) ∈ 𝐾 and 𝑚 ∈ 𝑃,
𝐷𝑒𝑐𝑟𝑦𝑝𝑡(𝑘2 , 𝐸𝑛𝑐𝑟𝑦𝑝𝑡(𝑘1 , 𝑚)) = 𝑚
• i.e. if we encrypt and then decrypt the same message with a
public-private key pair, we get back the original message
Security – For all (𝑘1 , 𝑘2 ) ∈ 𝐾 and 𝑚 ∈ 𝑃, if an eavesdropper only knows
the values of the public key 𝑘1 and the ciphertext 𝑐 = 𝐸𝑛𝑐𝑟𝑦𝑝𝑡(𝑘1 , 𝑚)
 but does not know 𝑘2 , it is computationally infeasible to find the
plaintext message 𝑚

The RSA Cryptosystem

- The Diffie-Hellman key exchange algorithm relies on the hardness of the discrete
logarithm problem
- The Rivest-Shamir-Adleman (RSA) cryptosystem relies on the hardness of factoring large
integers (i.e. with hundreds of digits)
- Phase 1: Key Generation
o Step 1. Alice picks two distinct prime numbers 𝑝 and 𝑞
o Step 2. Alice computes the product 𝑛 = 𝑝𝑞
o Step 3. Alice chooses an integer 𝑒 ∈ {2, 3, … , 𝜑(𝑛) − 1} such that
gcd(𝑒, 𝜑(𝑛)) = 1
o Step 4. Alice chooses an integer 𝑑 ∈ {2, 3, … , 𝜑(𝑛) − 1} that is the modular
inverse of 𝑒 modulo 𝜑(𝑛)
i.e. 𝑑𝑒 ≡ 1 (mod 𝜑(𝑛))
o Alice’s private key is the tuple (𝑝, 𝑞, 𝑑), and her public key is tuple (𝑛, 𝑒)
- Phase 2: Message Encryption
o Bob wants to send Alice a plaintext message 𝑚
o Bob computes the ciphertext 𝑐 = 𝑚𝑒 % 𝑛 and sends it to Alice
- Phase 3: Message Decryption
o Alice computes 𝑚′ = 𝑐 𝑑 % 𝑛
- Example
o 1. Alice chooses the primes numbers 𝑝 = 23 and 𝑞 = 31
o 2. The product is 𝑛 = 𝑝 ⋅ 𝑞 = 23 ⋅ 31 = 713
o 3. Alice chooses an 𝑒 where gcd(𝑒, 𝜑(𝑛)) = 1
Alice calculates that 𝜑(713) = 660, and chooses 𝑒 = 547 to satisfy the
constraints on 𝑒
o 4. Alice calculates the modular inverse to find out the last part of the private key
𝑑 ⋅ 547 = 1 (mod 660)), so 𝑑 = 403
o Private key: (𝑝 = 23, 𝑞 = 31, 𝑑 = 403), public key: (𝑛 = 713, 𝑒 = 547)
o Bob wants to send the number 42 to Alice.
He computes the encrypted number to be 𝑐 = 42𝑒 % 𝑛 =
42547 % 713 = 106
o Alice receives the number 106 from Bob. She computes the decrypted number
to be 𝑚 = 106𝑑 % 713 = 106403 % 713 = 42

Proving the Correctness of RSA

 - Theorem. Let (𝑝, 𝑞, 𝑑) ∈ ℤ+ × ℤ+ × ℤ+ be a private key and (𝑛, 𝑒) ∈ ℤ+ × ℤ+ be its
corresponding public key as generated by “RSA Phase 1”. Let 𝑚, 𝑐, 𝑚′ ∈ {1, … , 𝑛 − 1} be
the original plaintext message, ciphertext, and decrypted message, respectively.
- Then 𝑚′ = 𝑚
- 𝑃𝑟𝑜𝑜𝑓. Let 𝑝, 𝑞, 𝑛, 𝑑, 𝑒, 𝑚, 𝑐, 𝑚′ ∈ ℕ be defined as in the above definition of the RSA
algorithm. We need to prove that 𝑚′ = 𝑚
- From the definition of 𝑚′, we know 𝑚′ ≡ 𝑐 𝑑 (mod 𝑛). From the definition of 𝑐, we
know 𝑐 ≡ 𝑚𝑒 (mod 𝑛). Putting these together, we have:
𝑚′ ≡ (𝑚𝑒 )𝑑 ≡ 𝑚𝑒𝑑 (mod 𝑛)
- We know that 𝑑𝑒 ≡ 1 (mod 𝜑(𝑛)), i.e. there exists a 𝑘 ∈ ℤ such that 𝑑𝑒 = 𝑘𝜑(𝑛) + 1
- We also know that since gcd(𝑚, 𝑛) = 1, by Euler’s Theorem 𝑚𝜑(𝑛) ≡ 1 (mod 𝑛)
- Putting the above all together, we have
𝑚′ ≡ 𝑚𝑒𝑑 (mod 𝑛)
≡ 𝑚𝑘𝜑(𝑛)+1 (mod 𝑛)
𝑘
≡ (𝑚𝜑(𝑛) ) ⋅ 𝑚 (mod 𝑛)
≡ 1𝑘 ⋅ 𝑚 (mod 𝑛) (by Euler's Theorem)
≡ 𝑚 (mod 𝑛)
′
- So 𝑚 ≡ 𝑚 (mod 𝑛). Since we also know 𝑚 and 𝑚′ are between 1 and 𝑛 − 1, we can
conclude that 𝑚′ = 𝑚 QED

The Security of RSA

- We know from the RSA encryption phase that 𝑐 ≡ 𝑚𝑒 (mod 𝑛), so if we know all 3 of
𝑐, 𝑒, 𝑛, can we determine the value of 𝑚?
o No! We don’t have an efficient way of computing “𝑒-th roots” in modular
arithmetic
- We can attempt to discover Alice’s private key. Since 𝑑𝑒 ≡ 1 (mod 𝜑(𝑛)), 𝑑 is the
inverse of 𝑒 modulo 𝜑(𝑛)
o We can compute the modular inverse of 𝑒 modulo 𝜑(𝑛) when we know both 𝑒
and 𝜑(𝑛), but right now we only know 𝑛, not 𝜑(𝑛)
o If 𝑛 = 𝑝 ⋅ 𝑞 where 𝑝 and 𝑞 are distinct primes, then 𝜑(𝑛) = (𝑝 − 1)(𝑞 − 1),
however, it is not computationally feasible to factor 𝑛 when it is extremely large
This is known as the Integer Factorization Problem
There is no known efficient general algorithm for factoring integers

7.5 Implementing RSA in Python

Key Generation
- Assume that prime numbers 𝑝 and 𝑞 are given
 def rsa_generate_key(p: int, q: int) -> Tuple[Tuple[int, int, int], Tuple[int, int]]:
“””Return an RSA key pair generated using primes p and q.

The return value is a tuple containing two tuples:

1. The first tuple is the private key, containing (p, q, d).
2. The second tuple is the public key, containing (n, e).

Preconditions:
- p and q are prime
- p != q
“””
# Compute the product of p and q
n=p*q

# Choose e such that gcd(e, phi_n) == 1

phi_n = (p – 1) * (q – 1)

# Since e is chosen randomly, we repeat the random choice

# until e is coprime to phi_n.
e = random.randint(2, phi_n – 1)
while math.gcd(e, phi_n – 1)
e = random.randint(2, phi_n – 1)

# Choose d such that e * d % phi_n = 1

# Notice that we’re using our modular_inverse from our work in the last chapter
d = modular_inverse(e, phi_n)

return ((p, q, d), (n, e))

Encrypting and Decrypting a Number

- The plaintext is a number 𝑚 between 1 and 𝑛 − 1
- The ciphertext is another number 𝑐 = 𝑚𝑒 % 𝑛
def rsa_encrypt(public_key: Tuple[int, int], plaintext: int) -> int:
“””Encrypt the given plaintext using the recipient’s public key.

Preconditions:
- public_key is a valid RSA public key (n, e)
- 0 < plaintext < public_key[0]
 “””
n, e = public_key

encrypted = (plaintext ** e) % n

return encrypted

def rsa_decrypt(private_key: Tuple[int, int, int], ciphertext: int) -> int:

“””Decrypt the given ciphertext using the recipient’s private key.

Preconditions:
- private_key is a valid RSA private key (p, q, d)
- 0 < ciphertext < private_key[0] * private_key[1]
“””
p, q, d = private_key
n=p*q

decrypted = (ciphertext ** d) % n

return decrypted

Encrypting and Decrypting Text Using RSA

- The above implementation of RSA is unsatisfying because it encrypts numbers instead of
strings
- One strategy is to take a string and break it up into individual characters and encrypt
each character
- We can use ord/chr to convert between characters and numbers
- We can use a string accumulator to keep track of the encrypted/decrypted results
def rsa_encrypt_text(public_key: Tuple[int, int], plaintext: str) -> str:
“””Encrypt the given plaintext using the recipient’s public key.

Preconditions:
- public_key is a valid RSA public key (n, e)
- all({0 < ord(c) < public_key[0] for c in plaintext})
“””
n, e = public_key
 encrypted = ‘’
for letter in plaintext:
# Note: we could have also used our rsa_encrypt function here instead
encrypted = encrypted + chr((ord(letter) ** e) % n)

return encrypted

def rsa_decrypt_text(private_key: Tuple[int, int, int], ciphertext: int) -> int:

“””Decrypt the given ciphertext using the recipient’s private key.

Preconditions:
- private_key is a valid RSA private key (p, q, d)
- all({0 < ord(c) < private_key[0] * private_key[1] for c in ciphertext})
“””
p, q, d = private_key
n=p*q

decrypted = ‘’
for letter in ciphertext:
# Note: we could have also used our rsa_decrypt function here instead
decrypt = decrypted + chr((ord(letter) ** d) % n)

return decrypted

7.6 Application: Securing Online Communications

HTTPS and the Transport Layer Security Protocol

- The HTTPS protocol consists of two parts:
o HTTP (Hypertext Transfer Protocol) – governs the format of data being
exchanged between our computer and the server
o TLS (Transport Layer Security) – governs how the data formatted by HTTP is
encrypted during the communication process
- HTTP allows our computer to communicate with servers around the world
- When combined with TLS, those communications are secure and cannot be “snooped”
by an eavesdropper

TLS: An Overview (Simplified)

 - Client – refers to our computer
- Server – refers to the website we are communicating with
- When the client initiates a request to the server (e.g. when we type in a URL into our
web browser and press “Enter”)
o Step 1. When the client initiates the request, the server sends a “proof of
identity” that the client has connected with the intended server, which the client
verifies
Not encrypted
o Step 2. Then, the client and server perform the Diffie-Hellman key exchange
algorithm to establish a shared secret key
Not encrypted
o Step 3. All remaining communication (e.g. the actual website data) is encrypted
using an agreed-upon symmetric-key cryptosystem, like a stream cipher
- Two key questions
o 1. Why is symmetric-key encryption (rather than public-key encryption) used to
encrypt the communication in step 3?
o 2. Given that the first two steps of TLS are unencrypted, how can the client be
sure it is actually communicating with the intended server the whole time?

Why Symmetric-Key Encryption?

- Public-key cryptosystems, like RSA, are significantly slower than their symmetric-key
counterparts
o RSA relies on modular exponentiation as the key encryption

Who Am I Connected To?

- Question: how do we know we are communicating with the right server?
o i.e. are we connecting to the real Google server, not some fake server that’s
simply pretending to be Google?
- We need some way to verify that the server (e.g. Google) we intend to speak with is
actually who they say they are
- Every public-key cryptosystem (including RSA) can implement two additional algorithms
to:
o Sign message using the private key
o Verify a signature using the public key
- These algorithms allow a server to sign every message it sends with its private key, and
then have the client verify each message signature using the server’s public key
o We call these digital signatures
o They help us identify exactly who we are speaking with
 - Alice can add her signature, which is a function of her private key, to a message. Bob can
verify that Alice is the sender with Alice’s public key
- Digital signatures are used in each of the first two steps in the TLS protocol
- Establishing identity: digital certificates
o In the first step of TLS, Tthe “proof of identity” the server sends is called a digital
certificate
Digital certificate – contains identifying information for the server,
including:
• Its domain (e.g. www.google.com)
• Its organization name (e.g. “Google LLC”)
• Its public key
o The certificate also includes the digital signature of a certificate authority
Certificate authority – an organization whose purpose is to issue digital
certificates to website domains and verify the identities of the operators
of each of those domains
o When the client “verifies” the digital certificate provided by the server, the client
is actually verifying the digital signature provided by the certificate authority,
using the certificate authority’s public key
- Maintaining identity during Diffie-Hellman
o During the Diffie-Hellman algorithm, the server signs all messages it sends, so
that at every step the client can verify that the message came from the intended
server
This relies on the client knowing the server’s public key, which it gets
from the digital certificate in the previous step
o It is this digital signature from the server that allows the client to consistently
verify that it is communicating with the server, and that messages haven’t been
tampered with
o At the end of Step 2, the client and server have a shared secret key, and can now
communicate safely using symmetric-key encryption

(In)effectiveness of Cryptography
- Diffie-Hellman and RSA are secure because it is very difficult to extract the private part
of the data from what is being public communicated
- Unfortunately, many servers use the same group of prime numbers
- Some steps of the Diffie-Hellman algorithm can be precomputed for a specific group of
prime numbers
- 512-bit and 1024-bit keys are prone to the Logjam attack
o 2048-bit keys are used to avoid it
8.1 An Introduction to Running Time

How Do We Measure Running Time?

- Consider the following function, which prints out the first n natural numbers:
def print_integers(n: int) -> None:
for i in range(0, n):
print(i)
- We expect the function to take longer as n gets larger
- Rather than use an empirical measurement of runtime, we use an abstract
representation of runtime: the number of “basic operations” an algorithm executes
o This means that we can analyze functions without needing a computer, and our
analysis theoretically applies to any computer system
- “Basic operation” is a vague term:
o What counts as a “basic operation”?
o How do we tell which “basic operations” are used by an algorithm?
o Do all “basic operations” take the same amount of time?

Linear Running Time

- For the function print_integers, we know that:
o The for loop will call print once per iteration
o The loop iterates n times (i.e. with i taking on the values 0, 1, 2, …, n – 1)
- For an input n, there are n calls to print
- We say that the running time of print_integers on input n is n basic operations
- If we plot n against this measure running time, we obtain a line
- We say that print_integers has a linear running time, as the number of basic operations
is a linear function of the input n

Quadratic Running Time

- Consider a function that prints all combinations of pairs of integers:
def print_pairs(n: int) -> None:
“””Print all combinations of pairs of the first n natural numbers.”””
for i in range(0, n):
for j in range(0, n):
print(i, j)
- The outer loop repeats its body n times, and its body is another loop, which repeats its
body n times
o print is called n^2 times in total
 - We say that print_pairs has a quadratic running time, as the number of basic operations
is a quadratic function of the input n

Logarithmic Running Time

- Consider the following function, which prints out the powers of two that are less than a
positive integer n
def print_powers_of_two(n: int) -> None:
“””Print the powers of two that are less than n.

Preconditions:
-n>0
“””
for i in range(0, math.ceil(math.log2(n))):
print(2 ** i)
- The number of calls to print is log 2 𝑛
- The running time of print_powers_of_two is approximately, but not exactly log 2 𝑛
- We say that print_powers_of_two has a logarithmic running time

Constant Running Time

- Consider this function
def print_ten(n: int) -> None:
“””Print n ten times.”””
for i in range(0, 10):
print(n)
- The loop iterates 10 times regardless of what n is
- We say that print_ten has a constant running time
o The number of basic operations is independent to the input size

Basic Operations
- From fastest to slowest:
o Constant running time logarithmic running time linear running time
quadratic running time
- There are different ways of interpreting “basic operations”, i.e.
o We assign a variable at every loop iteration
o print calls take longer than variable assignment
o Calling the function is also an operation
o This can get extremely complicated
 - No matter how we interpret “basic operations” we know for sure that linear is faster
than quadratic

8.2 Comparing Asymptotic Function Growth with Big-O

Four Kinds of Dominance

- We will mainly be concerned about functions mapping the natural numbers to the
nonnegative real numbers
o i.e. functions 𝑓 ∶ ℕ → ℝ≥0
- We will look at the property of the function that describes the long-term (i.e.
asymptotic) growth of a function
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → ℝ≥0. We say that 𝑔 is absolutely dominated by 𝑓 when for
all 𝑛 ∈ ℕ, 𝑔(𝑛) ≤ 𝑓(𝑛)
o Ex. Let 𝑓(𝑛) = 𝑛 2 and 𝑔(𝑛) = 𝑛. Prove that 𝑔 is absolutely dominated by 𝑓.
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. Unpacking the definition, ∀𝑛 ∈ ℕ, 𝑔(𝑛) ≤ 𝑓(𝑛)
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑛 ∈ ℕ. We want to show that 𝑛 ≤ 𝑛 2.
o Case 1: Assume 𝑛 = 0. In this case, 𝑛 2 = 𝑛 = 0, so the inequality holds.
o Case 2: Assume 𝑛 ≥ 1. In this case, we take the inequality 𝑛 ≥ 1 and multiply
both sides by 𝑛 to get 𝑛 2 ≥ 𝑛, or equivalently 𝑛 ≤ 𝑛 2 QED
o Absolute dominance is too strict for our purposes
i.e. if 𝑔(𝑛) ≤ 𝑓(𝑛) for every natural number except 5, then we can’t say
that 𝑔 is absolutely dominated by 𝑓
𝑔(𝑛) = 2𝑛 is not absolutely dominated by 𝑓(𝑛) = 𝑛2
• 𝑔(𝑛) ≤ 𝑓(𝑛) everywhere except 𝑛 − 1
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → ℝ≥0. We say that 𝑔 is dominated by 𝑓 up to a constant
factor when there exists a positive real number 𝑐 such that
for all 𝑛 ∈ ℕ, 𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛)
o Ex. Let 𝑓(𝑛) = 𝑛 2 and 𝑔(𝑛) = 2𝑛. Prove that 𝑔 is dominated by 𝑓 up to a
constant factor
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. Unpacking the definition, ∃𝑐 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We already saw that 𝑛 is absolutely dominated by 𝑛 2, so if the 𝑛 is
multiplied by 2, then we should be able to multiply 𝑛 2 by 2 as well to get the
calculation to work out
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑐 = 2, and let 𝑛 ∈ ℕ. We want to prove that 𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛), or in
other words, 2𝑛 ≤ 2𝑛 2
o Case 1: Assume 𝑛 = 0. In this case, 2𝑛 = 0 and 2𝑛 2 = 0, so the inequality holds
o Case 2: Assume 𝑛 ≥ 1. Taking the assumed inequality 𝑛 ≥ 1 and multiplying
both sides by 2𝑛 yields 2𝑛 2 ≥ 2𝑛, or equivalently 2𝑛 ≤ 2𝑛2 QED
 o “Dominated by up to a constant factor” allows us to ignore multiplicative
constants in our functions
i.e. 𝑛, 2𝑛, 10000𝑛 are all equivalent in the sense that each one dominates
the other two up to a constant factor
o Consider the functions 𝑓(𝑛) = 𝑛 2 and 𝑔(𝑛) = 𝑛 + 90. No matter how much we
scale up 𝑓 by multiplying it by a constant, 𝑓(0) will always be less than 𝑔(0), so
we cannot say that 𝑔 is dominated by 𝑓 up to a constant factor
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → ℝ≥0 . We say that 𝑔 is eventually dominated by 𝑓 when
there exists 𝑛0 ∈ ℝ+ such that ∀𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛0, then 𝑔(𝑛) ≤ 𝑓(𝑛)
o Ex. Let 𝑓(𝑛) = 𝑛 2 and 𝑔(𝑛) = 𝑛 + 90. Prove that 𝑔 is eventually dominated by
𝑓
o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. ∃𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ 𝑔(𝑛) ≤ 𝑓(𝑛)
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. We need to argue that for “large enough” values of 𝑛, 𝑛 + 90 ≤ 𝑛 2.
We could solve it using factoring or the quadratic formula, but we can also
choose a large enough value of 𝑛0
o 𝑃𝑟𝑜𝑜𝑓. Let 𝑛0 = 90, let 𝑛 ∈ ℕ. We want to prove that 𝑛 + 90 ≤ 𝑛 2
𝑛 + 90 ≤ 𝑛 + 𝑛 since 𝑛 ≥ 90
= 2𝑛
≤ 𝑛 ⋅ 𝑛 since 𝑛 ≥ 2
= 𝑛2
QED
o This definition allows us to ignore “small” values of 𝑛
Important for ignoring the influence of slow-growing terms in a function,
which may affect the function values for “small” 𝑛, but eventually are
overshadowed by the faster-growing terms
• i.e. it took a while for the faster growth rate of 𝑛 2 to “catch up” to
𝑛 + 90
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → ℝ≥0. We say that 𝑔 is eventually dominated by 𝑓 up to a
constant factor when there exist 𝑐, 𝑛0 ∈ ℝ+ , such that for all 𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛0, then
𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛)
o In this case, we also say that 𝑔 is Big-O of 𝑓, and write 𝑔 ∈ 𝒪(𝑓)
o We formally define 𝒪(𝑓) to be the set of functions that are eventually
dominated by 𝑓 up to a constant factor:
𝒪(𝑓) = {𝑔 | 𝑔 ∶ ℕ → ℝ≥0 , and ∃𝑐, 𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0
⟹ 𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛)}
o Ex. Let 𝑓(𝑛) = 𝑛 and 𝑔(𝑛) = 𝑛 3 + 100𝑛 + 5000. Prove that 𝑔 ∈ 𝒪(𝑓).
3

o 𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛. ∃𝑐, 𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ 𝑛 3 + 100𝑛 + 5000 ≤ 𝑐𝑛 3

o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. 𝑔 is neither eventually dominated by 𝑓 nor dominated by 𝑓 up to a
constant factor.
Splitting up the inequality 𝑛 3 + 100𝑛 + 5000 ≤ 𝑐𝑛 3,
• 𝑛 3 ≤ 𝑐1 𝑛3
• 100𝑛 ≤ 𝑐2 𝑛 3
• 5000 ≤ 𝑐3 𝑛 3
If we can make these three inequalities true, adding them together will
give us our desired result (setting 𝑐 = 𝑐1 + 𝑐2 + 𝑐3)
There are multiple approaches to satisfy these inequalities
Approach 1: focus on choosing 𝑛0
• We can satisfy the three inequalities even if 𝑐1 = 𝑐2 = 𝑐3 = 1
• 𝑛 3 ≤ 𝑛 3 is always true (so for all 𝑛 ≥ 0)
• 100𝑛 ≤ 𝑛 3 when 𝑛 ≥ 10
•
3
5000 ≤ 𝑛 3 when 𝑛 ≥ √5000 ≈ 17.1
• We can pick 𝑛0 to be the largest of the lower bounds on 𝑛,
3
√5000, and then these inequalities will be satisfied
Approach 2: Pick 𝑐1 , 𝑐2 , 𝑐3 to make the right sides large enough to satisfy
the inequalities
• 𝑛 3 ≤ 𝑐1 𝑛3 when 𝑐1 = 1
• 100𝑛 ≤ 𝑐2 𝑛 3 when 𝑐2 = 100
• 5000 ≤ 𝑐3 𝑛 3 when 𝑐3 = 5000, as long as 𝑛 > 1
3
o 𝑃𝑟𝑜𝑜𝑓. (using approach 1) Let 𝑐 = 3 and 𝑛0 = √5000. Let 𝑛 ∈ ℕ, and assume
that 𝑛 ≥ 𝑛0. We want to show that 𝑛 3 + 100𝑛 + 5000 ≤ 𝑐𝑛3
o First, we prove the below inequalities:
𝑛 3 ≤ 𝑛 3 (since the two quantities are equal)
Since 𝑛 ≥ 𝑛0 ≥ 10, we know that 𝑛 2 ≥ 100, and so 𝑛3 ≥ 100𝑛
Since 𝑛 ≥ 𝑛0, we know that 𝑛 3 ≥ 𝑛03 = 5000
o Adding the above inequalities,
𝑛 3 + 100𝑛 + 5000 ≤ 𝑛 3 + 𝑛 3 + 𝑛 3 = 𝑐𝑛3
QED
o 𝑃𝑟𝑜𝑜𝑓. (using approach 2) Let 𝑐 = 5101 and 𝑛0 = 1. Let 𝑛 ∈ ℕ, and assume that
𝑛 ≥ 𝑛0. We want to show that 𝑛 3 + 100𝑛 + 5000 ≤ 𝑐𝑛3
o First, we prove the below inequalities:
𝑛 3 ≤ 𝑛 3 (since the two quantities are equal)
Since 𝑛 ∈ ℕ, we know that 𝑛 ≤ 𝑛 3, and so 100𝑛 ≤ 100𝑛3
Since 1 ≤ 𝑛, we know that 1 ≤ 𝑛 3, and then multiplying both sides by
5000 gives us 5000 ≤ 5000𝑛3
 o Adding the above inequalities,
𝑛 3 + 100𝑛 + 5000 ≤ 𝑛 3 + 100𝑛3 + 5000𝑛 3 = 5101𝑛 3 = 𝑐𝑛 3
QED

8.3 Big-O, Omega, and Theta

Intro
- Big-O is not necessarily an exact description of growth
o i.e. 𝑛 + 10 ∈ 𝒪(𝑛100 ) is not necessarily informative
- Big-O allows us to express upper bounds on the growth of a function
o It does not allow us to distinguish between an upper bound that is tight and one
that vastly overestimates the rate of growth

Omega and Theta

- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → 𝑅≥0. We say that 𝑔 is Omega of 𝑓 when there exist
constants 𝑐, 𝑛0 ∈ ℝ+ such that for all 𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛0, then 𝑔(𝑛) ≥ 𝑐 ⋅ 𝑓(𝑛). In this
case, we can also write 𝑔 ∈ Ω(𝑓)
o When 𝑔 ∈ Ω(𝑓), then 𝑓 is a lower bound on the growth rate of 𝑔
i.e. 𝑛 2 − 𝑛 ∈ Ω(𝑛)
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → 𝑅≥0 . We say that 𝑔 is Theta of 𝑓 when 𝑔 is both Big-O of 𝑓
and Omega of 𝑓. In this case, we can write 𝑔 ∈ Θ(𝑓), and say that 𝑓 is on 𝑔
o Equivalently, 𝑔 is Theta of 𝑓 when there exist constants 𝑐1 , 𝑐2 , 𝑛0 ∈ ℝ+ such that
for all 𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛0 then 𝑐1 𝑓(𝑛) ≤ 𝑔(𝑛) ≤ 𝑐2 𝑓(𝑛)
o The “Theta bound” means that the two functions have the same approximate
rate of growth, not that one is larger than the other
i.e. 10𝑛 + 5 ∈ Θ(𝑛), 10𝑛 + 5 ∉ Θ(𝑛 2 )

A Special Case: 𝒪(1), Ω(1), and Θ(1)

- Consider the constant function 𝑓(𝑛) = 1
o If a function 𝑔 is Big-O of this 𝑓, i.e. 𝑔 ∈ 𝒪(𝑓),
∃𝑐, 𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ 𝑔(𝑛) ≤ 𝑐 ⋅ 𝑓(𝑛)
∃𝑐, 𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ 𝑔(𝑛) ≤ 𝑐 (since 𝑓(𝑛) = 1)
o There exists a constant 𝑐 such that 𝑔(𝑛) is eventually always less than or equal
to 𝑐
o We say that such functions 𝑔 are asymptotically bounded with respect to their
input, and write 𝑔 ∈ 𝒪(1) to represent this
- Similarly, we use 𝑔 ∈ Ω(1) to express that functions are greater than or equal to some
constant 𝑐
 1
o 𝑔(𝑛) = 𝑛+1 is 𝒪(1), but not Ω(1)
o 𝑔(𝑛) = 𝑛 2 is Ω(1), but not 𝒪(1)
- Θ(1) refers to functions that are both 𝒪(1) and Ω(1)

Properties of Big-O, Omega, and Theta

- Elementary functions
o Theorem. For all 𝑎, 𝑏 ∈ ℝ+, the following statements are true
If 𝑎 > 1 and 𝑏 > 1, then log 𝑎 𝑛 ∈ Θ(log 𝑏 𝑛)
If 𝑎 < 𝑏, then 𝑛 𝑎 ∈ 𝒪(𝑛 𝑏 ) and 𝑛 𝑎 ∉ Ω(𝑛 𝑏 )
If 𝑎 < 𝑏, then 𝑎𝑛 ∈ 𝒪(𝑏𝑛 ) and 𝑎𝑛 ∉ Ω(𝑏𝑛 )
If 𝑎 > 1, then 1 ∈ 𝒪(log 𝑎 𝑛) and 1 ∉ Ω(log 𝑎 𝑛)
log 𝑎 𝑛 ∈ 𝒪(𝑛 𝑏 ) and log 𝑎 𝑛 ∉ Ω(𝑛 𝑏 )
If 𝑏 > 1, then 𝑛 𝑎 ∈ 𝒪(𝑏𝑛 ) and 𝑛 𝑎 ∉ Ω(𝑏𝑛 )
o Progression of functions toward longer running times
1 Constant
log 2 𝑛 , log 3 𝑛 , … , log100 𝑛 Logarithms
𝑛 0.000000000000001
𝑛 0.5
𝑛 Powers of 𝑛
2
𝑛
10000000000000
𝑛
1.0000000001𝑛
2𝑛 Exponentials
𝑛
100
- Basic properties
o Theorem. For all 𝑓 ∶ ℕ → ℝ≥0 , 𝑓 ∈ Θ(𝑓)
o Theorem. For all 𝑓, 𝑔 ∶ ℕ → ℝ≥0 , 𝑔 ∈ 𝒪(𝑓) if and only if 𝑓 ∈ Ω(𝑔)
o Theorem. For all 𝑓, 𝑔, ℎ ∶ ℕ → ℝ≥0 :
If 𝑓 ∈ 𝒪(𝑔) and 𝑔 ∈ 𝒪(ℎ), then 𝑓 ∈ 𝒪(ℎ)
If 𝑓 ∈ Ω(𝑔) and 𝑔 ∈ Ω(ℎ), then 𝑓 ∈ Ω(ℎ)
If 𝑓 ∈ Θ(𝑔) and 𝑔 ∈ Θ(ℎ), then 𝑓 ∈ Θ(ℎ)
- Operations on functions
o 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let 𝑓, 𝑔 ∶ ℕ → ℝ≥0. We can define the sum of 𝑓 and 𝑔 as the
function 𝑓 + 𝑔 ∶ ℕ → ℝ≥0 such that
∀𝑛 ∈ ℕ, (𝑓 + 𝑔)(𝑛) = 𝑓(𝑛) + 𝑔(𝑛)
o Theorem. For all 𝑓, 𝑔, ℎ ∶ ℕ → ℝ≥0, the following hold
If 𝑓 ∈ 𝒪(ℎ) and 𝑔 ∈ 𝒪(ℎ), then 𝑓 + 𝑔 ∈ 𝒪(ℎ)
If 𝑓 ∈ Ω(ℎ), then 𝑓 + 𝑔 ∈ Ω(ℎ)
 If 𝑓 ∈ Θ(ℎ) and 𝑔 ∈ 𝒪(ℎ), then 𝑓 + 𝑔 ∈ Θ(ℎ)
o Proof for the first statement
𝑇𝑟𝑎𝑛𝑠𝑙𝑎𝑡𝑖𝑜𝑛.
∀𝑓, 𝑔, ℎ ∶ ℕ → ℝ≥0 , (𝑓 ∈ 𝒪(ℎ) ∧ 𝑔 ∈ 𝒪(ℎ)) ⟹ 𝑓 + 𝑔 ∈ 𝒪(ℎ)
𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. Assuming 𝑓 ∈ 𝒪(ℎ) tells us that there exist positive real
numbers 𝑐1 and 𝑛1 such that for all 𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛1 then
𝑓(𝑛) ≤ 𝑐1 ⋅ ℎ(𝑛). There similarly exist 𝑐2 and 𝑛2 such that
𝑔(𝑛) ≤ 𝑐2 ⋅ ℎ(𝑛) whenever 𝑛 ≥ 𝑛2.
• Warning: we can’t assume that 𝑐1 = 𝑐2 or 𝑛1 = 𝑛2, or any other
relationship between these two sets of variables
We want to prove that there exist 𝑐, 𝑛0 ∈ ℝ+ such that for all 𝑛 ∈ ℕ, if
𝑛 ≥ 𝑛0 then 𝑓(𝑛) + 𝑔(𝑛) ≤ 𝑐 ⋅ ℎ(𝑛)
We can assume the following inequalities:
• 𝑓(𝑛) ≤ 𝑐1 ℎ(𝑛)
• 𝑔(𝑛) ≤ 𝑐2 ℎ(𝑛)
• Adding the two given inequalities gives the third
• We need to make sure that both given inequalities hold by
choosing 𝑛0 to be large enough, and let 𝑐 be large enough to take
into account both 𝑐1 and 𝑐2
𝑃𝑟𝑜𝑜𝑓. Let 𝑓, 𝑔, ℎ ∶ ℕ → ℝ≥0, and assume 𝑓 ∈ 𝒪(ℎ) and 𝑔 ∈ 𝒪(ℎ). By
these assumptions, there exist 𝑐1 , 𝑐2 , 𝑛1 , 𝑛2 ∈ ℝ+ such that for all 𝑛 ∈ ℕ,
• if 𝑛 ≥ 𝑛1 , then 𝑓(𝑛) ≤ 𝑐1 ⋅ ℎ(𝑛), and
• if 𝑛 ≥ 𝑛2 , then 𝑔(𝑛) ≤ 𝑐2 ⋅ ℎ(𝑛)
We want to prove that 𝑓 + 𝑔 ∈ 𝒪(ℎ), i.e., that there exist 𝑐, 𝑛0 ∈ ℝ+
such that for all 𝑛 ∈ ℕ, if 𝑛 ≥ 𝑛0 then 𝑓(𝑛) + 𝑔(𝑛) ≤ 𝑐 ⋅ ℎ(𝑛)
Let 𝑛0 = max {𝑛1 , 𝑛2 } and 𝑐 = 𝑐1 + 𝑐2. Let 𝑛 ∈ ℕ, and assume that
𝑛 ≥ 𝑛0. We now want to prove that 𝑓(𝑛) + 𝑔(𝑛) ≤ 𝑐 ⋅ ℎ(𝑛)
Since 𝑛0 ≥ 𝑛1 and 𝑛0 ≥ 𝑛2, we know that 𝑛 is greater than or equal to 𝑛1
and 𝑛2 as well. Then using the Big-O assumptions,
• 𝑓(𝑛) ≤ 𝑐1 ⋅ ℎ(𝑛)
• 𝑔(𝑛) ≤ 𝑐2 ⋅ ℎ(𝑛)
Adding these two inequalities,
𝑓(𝑛) + 𝑔(𝑛) ≤ 𝑐1 ℎ(𝑛) + 𝑐2 ℎ(𝑛) = (𝑐1 + 𝑐2 )ℎ(𝑛) = 𝑐 ⋅ ℎ(𝑛) QED
≥0 +
o Theorem. For all 𝑓 ∶ ℕ → ℝ and for all 𝑎 ∈ ℝ , 𝑎 ⋅ 𝑓 ∈ Θ(𝑓)
o Theorem. For all 𝑓1 , 𝑓2 , 𝑔1 , 𝑔2 ∶ ℕ → ℝ≥0 , if 𝑔1 ∈ 𝒪(𝑓1 ) and 𝑔2 ∈ 𝒪(𝑓2 ), then
𝑔1 ⋅ 𝑔2 ∈ 𝒪(𝑓1 ⋅ 𝑓2 )
The statement is still true if we replace Big-O with Omega or Theta
 o Theorem. For all 𝑓 ∶ ℕ → ℝ≥0, if 𝑓(𝑛) is eventually greater than or equal to 1,
then ⌊𝑓⌋ ∈ Θ(𝑓) and ⌈𝑓⌉ ∈ Θ(𝑓)
- Properties from calculus
o Our asymptotic notation of 𝒪, Ω, and Θ are concerned with comparing the long-
term behaviour of two functions
i.e. the limit of the function as its input approaches infinity
o Let 𝑓 ∶ ℕ → ℝ and 𝐿 ∈ ℝ. We have the following definitions:
lim 𝑓(𝑛) = 𝐿 ∶ ∀𝜖 ∈ ℝ+ , ∃𝑛0 ∈ ℕ, ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ |𝑓(𝑛) − 𝐿| < 𝜖
𝑛→∞
lim 𝑓(𝑛) = ∞ ∶ ∀𝑀 ∈ ℝ+ , ∃𝑛0 ∈ ℕ, ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0 ⟹ 𝑓(𝑛) > 𝑀
𝑛→∞
o Theorem. for all 𝑓, 𝑔 ∶ ℕ → ℝ≥0 , if 𝑔(𝑛) ≠ 0 for all 𝑛 ∈ ℕ, then the following
statement hold:
𝑓(𝑛)
If there exists 𝐿 ∈ ℝ+ such that lim = 𝐿, then 𝑔 ∈ Ω(𝑓) and
𝑛→∞ 𝑔(𝑛)
𝑔 ∈ 𝒪(𝑓).
• In other words, 𝑔 ∈ Θ(𝑓)
𝑓(𝑛)
If lim = 0, then 𝑓 ∈ 𝒪(𝑔) and 𝑔 ∉ 𝒪(𝑓)
𝑛→∞ 𝑔(𝑛)
𝑓(𝑛)
If lim = ∞, then 𝑔 ∈ 𝒪(𝑓) and 𝑓 ∉ 𝒪(𝑔)
𝑛→∞ 𝑔(𝑛)
o The converse of the above statements is not true

8.4 Analyzing Algorithm Running Time

Intro
- Consider the following function
def print_items(lst: list) -> None:
for item in lst:
print(item)
- We will concentrate on how the size of the input influences the running time of a
program
- We measure running time using asymptotic notation, and not exact expressions
- Basic operation – any block of code whose running time does not depend on the size of
the input
o Including assignment statements, arithmetic calculations, list/string indexing

The Runtime Function

- We expect print_items to take the same amount time on every list of length 100
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let func be an algorithm. For every 𝑛 ∈ ℕ, we define the set 𝐼𝑓𝑢𝑛𝑐,𝑛 to be
the set of allowed inputs to func of size 𝑛
 o i.e. 𝐼𝑝𝑟𝑖𝑛𝑡_𝑖𝑡𝑒𝑚𝑠,100 is the set of all lists of length 100, 𝐼𝑝𝑟𝑖𝑛𝑡_𝑖𝑡𝑒𝑚𝑠,0 is the set
containing just one input: the empty set
- For all 𝑛 ∈ ℕ, every element of 𝐼𝑝𝑟𝑖𝑛𝑡_𝑖𝑡𝑒𝑚𝑠,𝑛 has the same runtime when passed to
print_items
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let func be an algorithm whose runtime depends only on its input size. We
define the running time function of func as 𝑅𝑇𝑓𝑢𝑛𝑐 ∶ ℕ → ℝ≥0 , where 𝑅𝑇𝑓𝑢𝑛𝑐 (𝑛) is
equal to the running time of func when given an input of size 𝑛
- Goal of a running time analysis for func is to find a function 𝑓 such that 𝑅𝑇𝑓𝑢𝑛𝑐 𝑖𝑛 Θ(𝑓)
- Runtime analysis
o 1. Identify the blocks of code which can be counted as a single basic operation
They don’t depend on the input size
o 2. Identify any loops in the code, which cause basic operations to repeat
We need to figure out how many times those loops run, based on the size
of the input
o 3. Use our observations from the previous two steps to come up with an
expression for the number of basic operations used in the algorithm
i.e. find an exact expression for 𝑅𝑇𝑓𝑢𝑛𝑐 (𝑛)
o 4. Use the properties of asymptotic notation to find an elementary function 𝑓
such that 𝑅𝑇𝑓𝑢𝑛𝑐 ∈ Θ(𝑓(𝑛))
- Ex. Consider the function print_items. We define input size to be the number of items of
the input list. Perform a runtime analysis of print_items
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list lst
o Each iteration of the loop can be counted as a single operation, because nothing
in it depends on the size of the input list
o The running time depends on the number of loop iterations
o Since this is a for loop over the lst argument, thus the total number of basic
operations performed is 𝑛, and so the running time is 𝑅𝑇𝑝𝑟𝑖𝑛𝑡_𝑖𝑡𝑒𝑚𝑠 (𝑛) = 𝑛,
which is Θ(𝑛) QED
- Ex. Analyse the running time of the following function.
def my_sum(numbers: List[int]) -> int:
sum_so_far = 0

for number in numbers:

sum_so_far += number

return sum_so_far
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list
o This function body consists of three statements
 The assignment statement counts as 1 step
The for loop takes 𝑛 steps: it has 𝑛 iterations, and each iteration takes 1
step
The return statement counts as 1 step
o The total running time is the sum of these three parts: 1 + 𝑛 + 1 = 𝑛 + 2, which
is Θ(𝑛) QED

Nested Loops
- Count the number of repeated basic operations in a loop starting with the innermost
loop and working our way out
- Ex. Consider the following function:
def print_sums(lst: list) -> None:
for item1 in lst:
for item2 in lst:
print(item1 + item2)
Perform a runtime analysis of print_sums.
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of lst
o The inner loop runs 𝑛 times, and each iteration is just a single basic operation
o The outer loop runs 𝑛 times, and each of its iterations take 𝑛 operations
o The total number of basic operations is 𝑅𝑇𝑝𝑟𝑖𝑛𝑡_𝑠𝑢𝑚𝑠 (𝑛) =
steps for the inner loop × number of times inner loop is repeated
=𝑛×𝑛
= 𝑛2
o So the running time of this algorithm is Θ(𝑛 2 ) QED
- Ex. Consider the following function:
def f(lst: List[int]) -> None:
for item in lst:
for i in range(0, 10):
print(item + i)
Perform a runtime analysis of this function.
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list lst
o The inner loop repeats 10 times, and each iteration is a single basic operation,
for a total of 10 basic operations
o The outer loop repeats 𝑛 times, and each iteration takes 10 steps, for a total of
10𝑛 steps.
o The running time of this function is Θ(𝑛)
o Alternatively, the inner loop’s running time does not depend on the number of
items in the input list, so we can count it as a single basic operation
 o The outer loop runs 𝑛 times, and each iteration takes 1 step, for a total of 𝑛
steps, which is Θ(𝑛) QED
- Ex. Analyze the running time of the following function.
def combined(lst: List[int]) -> None:
# Loop 1
for item in lst:
for i in range(0, 10):
print(item + i)

# Loop 2
for item1 in lst:
for item2 in lst:
print(item1 + item2)
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of lst. We have already seen that
the first loop runs in time Θ(𝑛), while the second loop runs in time Θ(𝑛 2 )
o By the Sum of functions theorem from the previous section, we can conclude
that combined runs in time Θ(𝑛 2 ). QED

Loop Iterations With Changing Costs

- Ex. Analyze the running time of the following function.
def all_pairs(lst: List[int]) -> None:
for i in range(0, len(lst)):
for j in range(0, i):
print(lst[i] + lst[j])
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. The inner loop’s running time depends on the current value of i
o We need to manually add up the cost of each iteration of the outer loop, which
depends on the number of iterations of the inner loop
o Since j goes from 0 to i – 1, the number of iterations of the inner loop is i, and
each iteration of the inner loop counts as one basic operation
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list.
o We start by analysing the running time of the inner loop for a fixed iteration of
the outer loop, and a fixed value of 𝑖
The inner loop iterates 𝑖 times and each iteration takes one step.
Therefore the cost of the inner loop is 𝑖 steps, for one iteration of the
outer loop.
o The outer loop iterates 𝑛 times for 𝑖 going from 0 to 𝑛 − 1.
o The cost of iteration 𝑖 is 𝑖 steps, and so the total cost of the outer loop is
 𝑛−1
𝑛(𝑛 − 1)
∑𝑖 =
2
𝑖=0
𝑛(𝑛−1)
o And so the total number of steps taken by all pairs is , which is Θ(𝑛 2 )
2
QED

8.5 Analyzing Comprehensions and While Loops

Comprehensions
- Consider the following function
def square_all(numbers: List[int]) -> List[int]:
“””Return a new list containing the squares of the given numbers”””
return [x ** 2 for x in numbers]
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. We analyze it in the same way as a for loop
1. We determine the number of steps required to evaluate the leftmost
expression in the comprehension. In this case, evaluating x ** 2 takes 1
step
2. The collection that acts as the source of the comprehension (i.e.
numbers) determines how many times the leftmost expression is
evaluated
o Let 𝑛 be the length of the input list numbers. The comprehension expression
takes 𝑛 steps
1 step per element of numbers
o The running time of square_all is 𝑛 steps, which is Θ(𝑛) QED
- The same analysis would hold in the above function if we had used a set or dictionary
comprehension instead

While Loops
- Analysing the running time of code involving while loops follows the same principle as
for loops
o We calculate the sum of the different loop iterations (by
multiplication/summation)
- Ex. Analyse the running time of the following function:
def my_sum_v2(numbers: List[int]) -> int:
“””Return the sum of the given numbers.”””
sum_so_far = 0
i=0
 while i < len(numbers):
sum_so_far += numbers[i]
i += 1

return sum_so_far
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input numbers
o We can divide up the function into 3 parts
1. The cost of the assignment statements sum_so_far = 0 and i = 0 is
constant time
2. The while loop
• Each iteration is constant time
• There are 𝑛 iterations, since i starts at 0 and increases by 1 until it
reaches 𝑛
3. The return statement takes constant time
o The total running time is 1 + 𝑛 + 1 = 𝑛 + 2, which is Θ(𝑛) QED
- Ex. Analyse the running time of the following function:
def my_sum_powers_of_two(numbers: List[int]) -> int:
“””Return the sum of the given numbers whose indexes are powers of 2.

That is return numbers[1] + numbers[2] + numbers[4] + numbers[8] + …

“””
sum_so_far = 0
i=1

while i < len(numbers):

sum_so_far += numbers[i]
i *= 2

return sum_so_far
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list numbers
o We count the initial assignment statements as 1 step, and the return statement
as 1 step
o Each iteration takes constant time
o To determine the number of loop iterations, we follow these steps:
1. Find a pattern for how i changes at each loop iteration, and a general
formula for 𝑖𝑘 , the value of i after 𝑘 iterations
Iteration Value of i
0 1
 1 2
2 4
3 8
4 16
So we find that after 𝑘 iterations, 𝑖𝑘 = 2𝑘
2. We know the while loop continues while i < len(numbers)
• i.e. the while loop continues until i >= len(numbers)
To find the number of iterations, we need to find the smallest value of 𝑘
such that 𝑖𝑘 ≥ 𝑛, which makes the loop condition False
𝑖𝑘 ≥ 𝑛
2𝑘 ≥ 𝑛
𝑘 ≥ log 2 𝑛
So we need to find the smallest value of 𝑘 such that 𝑘 ≥ log 2 𝑛, which is
⌈log 2 𝑛⌉
o The while loop iterates ⌈log 2 𝑛⌉ times, with 1 step per iteration, for a total of
⌈log 2 𝑛⌉ steps
o The function my_sum_powers_of_two has a running time of 1 + ⌈log 2 𝑛⌉ + 1 =
⌈log 2 𝑛⌉ + 2, which is Θ(log 𝑛) QED

A Trickier Example
- Example of a standard loop, with a twist in how the loop variable changes at each
iteration
def twisty(n: int) -> int:
“””Return the number of iterations it takes for this special loop to stop
for the given n.
“””
iterations_so_far = 0
x=n
while x > 1:
if x % 2 == 0:
x=x/2
else:
x=2*x–2
iterations_so_far += 1

return iterations_so_far
- The loop variable x does not always get closer to the loop stopping condition
o i.e. sometimes it increases
- We will perform an analysis based on multiple iterations
- 𝐶𝑙𝑎𝑖𝑚. For any integer value of x greater than 2, after two iterations of the loop in
twisty the value of x decreases by at least one.
- 𝑃𝑟𝑜𝑜𝑓. Let 𝑥0 be the value of variable x at some iteration of the loop, and assume
𝑥0 > 2. Let 𝑥1 be the value of 𝑥 after one loop iteration, and 𝑥2 be the value of 𝑥 after
two loop iterations. We want to prove that 𝑥2 ≤ 𝑥0 − 1
o We divide up this proof into four cases, based on the remainder of 𝑥0 when
dividing by 4
o Case 1: Assume 4 ∣ 𝑥0, i.e. ∃𝑘 ∈ ℤ, 𝑥0 = 4𝑘
In this case, 𝑥0 is even, so the if branch executes in the first loop
𝑥0
iteration, and so 𝑥1 = = 2𝑘. Then 𝑥1 is also even, and so the if branch
2
𝑥1
executes again: 𝑥2 = =𝑘
2
1
So then 𝑥2 = 4 𝑥0 ≤ 𝑥0 − 1 (since 𝑥0 ≥ 4), as required.
o Case 2: Assume 4 ∣ 𝑥0 − 1, i.e. ∃𝑘 ∈ ℤ, 𝑥0 = 4𝑘 + 1
In this case, 𝑥0 is odd, so the else branch executes in the first loop
iteration, and so 𝑥1 = 2𝑥0 − 2 = 8𝑘. Then 𝑥1 is even, and so
𝑥1
𝑥2 = = 4𝑘.
2
So then 𝑥2 = 4𝑘 = 𝑥0 − 1, as required.
o Case 3: Assume 4 ∣ 𝑥0 − 2, i.e. ∃𝑘 ∈ ℤ, 𝑥0 = 4𝑘 + 2
In this case, 𝑥0 is even, so the if branch executes in the first loop
𝑥0
iteration, and so 𝑥1 = = 2𝑘 + 1. The 𝑥1 is odd, and so the else branch
2
executes: 𝑥2 = 2𝑥1 − 2 = 4𝑘.
So then 𝑥2 = 4𝑘 ≤ 𝑥0 − 1, as required.
o Case 4: Assume 4 ∣ 𝑥0 − 3, i.e. ∃𝑘 ∈ ℤ, 𝑥0 = 4𝑘 + 3
In this case, 𝑥0 is odd, so the else branch executes in the first loop
iteration, and so 𝑥1 = 2𝑥0 − 2 = 8𝑘 + 4. Then 𝑥1 is even, and so
𝑥1
𝑥2 = = 4𝑘 + 2.
2
So the 𝑥2 = 4𝑘 + 2 = 𝑥0 − 1, as required. QED
- 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. (Analysis of twisty)
o We count the variable initialization before the while loop as 1 step, and the
return statement as 1 step
o For the while loop:
The loop body takes 1 step
To count the number of loop iterations, we first observe that 𝑥 starts at 𝑛
and the loop terminates when 𝑥 reaches 1 or less. The Claim tells us that
after every 2 iterations, the value of 𝑥 decreases by at least 1
 So the after 2 iterations, 𝑥 ≤ 𝑛 − 1; after 4 iterations, 𝑥 ≤ 𝑛 − 2, and in
general, after 2𝑘 iterations, 𝑥 ≤ 𝑛 − 𝑘
This tells us that after 2(𝑛 − 1) loop iterations, 𝑛 ≤ 𝑛 − (𝑛 − 1) = 1,
and so the loop must stop.
o This analysis tells us that the loop iterates at most 2(𝑛 − 1) times, and so takes
at most 2(𝑛 − 1) steps
o So the total running time of twisty is at most 1 + 2(𝑛 − 1) + 1 = 2𝑛 steps,
which is 𝒪(𝑛) QED
- We did not compute the exact number of steps the function twisty takes, only an upper
bound on the number of steps
- We were only able to conclude a Big-O bound, and not a Theta bound
o We don’t know whether this bound is tight
- It is possible to prove something remarkable about what happens to the variable x after
three iterations of the twisty loop
- 𝐶𝑙𝑎𝑖𝑚 (Improved). For any integer value of x greater than 2, let 𝑥0 be the initial value of
1 1
x and let 𝑥3 be the value of x after three loop iterations. Then 8 𝑥0 ≤ 𝑥3 ≤ 2 𝑥0
- The running time of twisty is both 𝒪(log 𝑛) and Ω(log 𝑛), and hence conclude that its
running time is Θ(log 𝑛)

8.6 Analyzing Built-In Data Type Operations

Timing Operations
- Python provides a module called timeit that can tell us how long Python code takes to
execute
>>> from timeit import timeit
>>> timeit(‘5 + 15’, number=1000)
1.97999133245455784654e-05
o The above call to timeit will perform the operation 5 + 15 one thousand times
o The function returns the total time elapsed

How Python Lists are Stored in Memory

- All data used by a program are stored in blocks of computer memory, which are labeled
by numbers called memory addresses
- For every Python list object, the references to its elements are stored in a contiguous
block of memory
o i.e. [8, 7, 100, -3]
Address Memory
⋮
 400 601 (list index 0)
401 512
402 699
403 650
404
⋮
512 7
⋮
601 8
⋮
650 -3
⋮
699 100
⋮
o Each list elements are always stored consecutively
- This type of list implementation is called an array-based list implementation

Fast List Indexing

- Array-based list implementation makes list indexing fast
- List indexing is a constant-time operation
- True for both evaluating a list indexing expression or assigning to a list index

Mutating Contiguous Memory

- The references must always be stored in a contiguous block of memory
o There can’t be any “gaps”
- When a list element is deleted, all items after it have to be moved back one memory
block to fill the gap
- Similarly, when a list element is inserted somewhere in the list, all items after it moves
forward by one block
- When we remove the element at index 𝑖 in the list, where 0 ≤ 𝑖 < 𝑛, then 𝑛 − 𝑖 − 1
elements must be moved, and so the running time of this operation is Θ(𝑛 − 𝑖)
- Inserting/deleting at the front of a Python list (𝑖 = 0) takes Θ(𝑛) time
- Inserting/deleting at the back of a Python list (𝑖 = 𝑛 − 1) is a constant time operation

Summary of List Operation Asymptotic Running Times (𝑛 is the list size)

Operation Running time
List indexing (lst[i]) Θ(1)
 List index assignment (lst[i] = …) Θ(1)
List insertion at end (list.append(lst, …)) Θ(1)
List deletion at end (list.pop(lst)) Θ(1)
List insertion at index (list.insert(lst, i, …)) Θ(𝑛 − 𝑖)
List deletion at index (list.pop(lst, i)) Θ(𝑛 − 𝑖)

When Space Runs Out

- We assume that there will always be free memory blocks at the end of the list for the
list to expand into

Running-Time Analysis with List Operations

- Ex. Analyse the running time of the following function
def squares(number: List[int]) -> int:
“””Return a list containing the square of the given numbers.”””
squares_so_far = 0

for number in numbers:

list.append(squares_so_far, number ** 2)

return squares_so_far
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list (i.e. numbers)
o The assignment statement counts as 1 step
o The for loop:
Takes 𝑛 iterations
list.append takes constant time, and so the entire loop body counts as 1
step
This means the for loop takes 𝑛 ⋅ 1 = 𝑛 steps total
o The return statement counts as 1 step
o The total running time is 1 + 𝑛 + 1 = 𝑛 + 2, which is Θ(𝑛) QED
- Ex. Analyse the running time of the following function
def squares_reversed(numbers: List[int]) -> int:
“””Return a list containing the squares of the given numbers, in reverse
order.”””
squares_so_far = 0

for number in numbers:

# Now, insert number ** 2 at the START of squares_so_far
list.insert(squares_so_far, 0, number ** 2)
 return squares_so_far
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Let 𝑛 be the length of the input list (i.e., numbers)
o The assignment statement counts as 1 step
o The for loop:
Takes 𝑛 iterations
Inserting at the front of a Python list causes all of its current elements to
be shifted over, taking time proportional to the size of the list. Therefore
this call takes Θ(𝑘) time, where 𝑘 is the current length of squares_so_far
We know that squares_so_far starts as empty, and then increases in
length by 1 at each iteration. So then 𝑘 takes on the values 0, 1, 2, …,
𝑛 − 1, and we can calculate the total running time of the for loop using a
summation:
𝑛−1
(𝑛 − 1)𝑛
∑𝑘 =
2
𝑘=0
o The return statement counts as 1 step
(𝑛−1)𝑛
o The total running time is 1 + + 1 = 𝑛 + 2, which is Θ(𝑛2 ) QED
2

Sets and Dictionaries

- Both are implemented using a data structure called a hash table
o Which allows constant-time lookup, insertion, and removal of elements (for a
set) and key-value pairs (for a dictionary)
- Downside: not mutable

Data Classes
- Data classes store their instance attributes using a dictionary that maps attribute names
to their corresponding values
- Data classes benefit from the constant-time dictionary operations above
- The two operations that we can perform on a dataclass instance: looking up an attribute
value (i.e. david.age), and mutating the instance by assigning to an attribute (i.e.
david.age = 99) both take constant time

Summary of Set, Dictionary, and Data Class Operations

Operation Running time
Set/dict search (in) Θ(1)
set.add/set.remove Θ(1)
Dictionary key lookup (d[k]) Θ(1)
 Dictionary key assignment (d[k] = …) Θ(1)
Data class attribute access (obj.attr) Θ(1)
Data class attribute assignment (obj.attr = …) Θ(1)

Aggregation Functions
- sum, max, min have a linear running time (Θ(𝑛)), proportional to the size of the input
collection
o Each element of the collection must be processed in order to calculate each of
these values
- len has a constant running time (Θ(1)), independent of the size of input collection
o The Python interpreter does not need to process each element of a collection
when calculating the collection’s size
o Each of these collection data types stores a special attribute referring to the size
of that collection
- any and all need to check every element of their input collection, but they can short-
circuit (stopping before checking every element)
o Similar to the logical or and and operators
o Their running time isn’t a fixed function of the input size, but rather a possible
range of values, depending on whether this short-circuiting happens or not

8.7 Worst-Case Running Time Analysis

Intro
- Algorithms often depend on the actual value of the input, not just its size
- Consider the following function
def has_even(numbers: List[int]) -> bool:
“””Return whether numbers contain an even element.”””
for number in numbers:
if number % 2 == 0:
return True
return False
o Because this function returns as soon as it finds an even number in the list, its
running time is not necessarily proportional to the length of the input list
- The running time of a function can vary even when the input size is fixed
o 𝐼ℎ𝑎𝑠_𝑒𝑣𝑒𝑛,10 do not all have the same runtime
- Because our asymptotic notation is used to describe the growth rate of functions, we
cannot use it to describe the growth of a whole range of values with respect to
increasing input sizes
 - We focus on the maximum of this range, which corresponds to the slowest the
algorithm could run for a given input size
- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let func be a program. We define the function 𝑊𝐶𝑓𝑢𝑛𝑐 ∶ ℕ → ℕ, called the
worst-case running time function of func, as follows:
𝑊𝐶𝑓𝑢𝑛𝑐 (𝑛) = max{running time of executing 𝑓𝑢𝑛𝑐(𝑥) | 𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 }
- 𝑊𝐶𝑓𝑢𝑛𝑐 is a function, not a constant number: it returns the maximum possible running
time for an input of size 𝑛, for every natural number 𝑛
o And so we can use asymptotic notation to describe it
- The goal of a worst-case runtime analysis for func is to find an elementary function 𝑓
such that 𝑊𝐶𝑓𝑢𝑛𝑐 ∈ Θ(𝑓)
- We take a two-pronged approach: proving matching upper and lower bounds on the
worst-case running time of our algorithm

Upper Bounds on the Worst-Case Runtime

- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let func be a program, and 𝑊𝐶𝑓𝑢𝑛𝑐 its worst-case runtime function. We say
that a function 𝑓 ∶ ℕ → 𝑅≥0 is an upper bound on the worst-case runtime when
𝑊𝐶𝑓𝑢𝑛𝑐 ∈ 𝒪(𝑓)
- Suppose we use absolute dominance rather than Big-O, expanding the phrase “𝑊𝐶𝑓𝑢𝑛𝑐
is absolutely dominated by 𝑓”:
∀𝑛 ∈ ℕ, 𝑊𝐶𝑓𝑢𝑛𝑐 (𝑛) ≤ 𝑓(𝑛)
⟺ ∀𝑛 ∈ ℕ, max{running time of executing 𝑓𝑢𝑛𝑐(𝑥) | 𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 } ≤ 𝑓(𝑛)
⟺ ∀𝑛 ∈ ℕ, ∀𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 , running time of executing 𝑓𝑢𝑛𝑐(𝑥) ≤ 𝑓(𝑛)
o An upper bound on the worst-case runtime is equivalent to an upper bound on
the runtimes of all inputs
- Translation of 𝑊𝐶𝑓𝑢𝑛𝑐 ∈ 𝒪(𝑓):
∃𝑐, 𝑛0 ∈ ℝ+ , ∀𝑛 ∈ ℕ, 𝑛 ≥ 𝑛0
⟹ (∀𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 , running time of executing 𝑓𝑢𝑛𝑐(𝑥) ≤ 𝑐 ⋅ 𝑓(𝑛))
- To approach an analysis of an upper bound on the worst-case, we typically find a
function 𝑔 such that 𝑊𝐶𝑓𝑢𝑛𝑐 is absolutely dominated by 𝑔, and then find a simple
function 𝑓 such that 𝑔 ∈ 𝒪(𝑓)
- Ex. Find an asymptotic upper bound on the worst-case running time of has_even
o 𝐷𝑖𝑠𝑐𝑢𝑠𝑠𝑖𝑜𝑛. The intuitive translation using absolute dominance is usually
enough for an upper bound analysis
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. (Upper bound on worst-case)
Let 𝑛 ∈ ℕ, let numbers be an arbitrary list of length 𝑛
The loop iterates at most 𝑛 times. Each loop iteration counts as a single
step, so the loop takes at most 𝑛 ⋅ 1 = 𝑛 steps in total.
 The return False statement (if it is executed) counts as 1 basic operation.
Therefore the running time is at most 𝑛 + 1, and 𝑛 + 1 ∈ 𝒪(𝑛). So we
can conclude that the worst-case running time of has_even is 𝒪(𝑛).
QED
- Because we calculated an upper bound rather than an exact number of steps, we can
only conclude a Big-O
o We don’t yet know that this upper bound is tight

Lower Bounds on the Worst-Case Runtime

- 𝐷𝑒𝑓𝑖𝑛𝑖𝑡𝑖𝑜𝑛. Let func be a program, and 𝑊𝐶𝑓𝑢𝑛𝑐 is the worst-case runtime function. We
say that a function 𝑓 ∶ ℕ → ℝ≥0 is a lower bound on the worst-case runtime when
𝑊𝐶𝑓𝑢𝑛𝑐 ∈ Ω(𝑓)
- Using absolute dominance,
∀𝑛 ∈ 𝑁, 𝑊𝐶𝑓𝑢𝑛𝑐 (𝑛) ≥ 𝑓(𝑛)
⟺ ∀𝑛 ∈ ℕ, max{running time of executing 𝑓𝑢𝑛𝑐(𝑥) | 𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 } ≥ 𝑓(𝑛)
⟺ ∀𝑛 ∈ 𝑁, ∃𝑥 ∈ 𝐼𝑓𝑢𝑛𝑐,𝑛 , running time of executing 𝑓𝑢𝑛𝑐(𝑥) ≥ 𝑓(𝑛)
- Our goal is to find an input family whose runtime is asymptotically larger than our target
lower bound
o Input family – a set of inputs, one per input size 𝑛
- In has_even we want to prove that the worst-case running time is Ω(𝑛) to match the
𝒪(𝑛) upper bound
- Ex. Find an asymptotic lower bound on the worst-case running time of has_even
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. (Lower bound on worst-case)
Let 𝑛 ∈ ℕ. Let numbers be the list of length 𝑛 consisting of all 1’s.
In this case, the if condition in the loop is always false, so the loop never
stops early. Therefore it iterates exactly 𝑛 times, with each iteration
taking 1 step.
Finally, the return False statement executes, which is one step. So the
total number of steps for this input is 𝑛 + 1, which is Ω(𝑛) QED

Putting It All Together

- We can combine our upper and lower bounds on 𝑊𝐶ℎ𝑎𝑠_𝑒𝑣𝑒𝑛 to obtain a tight
asymptotic bound
- Ex. Find a tight bound on the worst-case running time of has_even
o 𝑅𝑢𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠. Since we’ve proved that 𝑊𝐶ℎ𝑎𝑠_𝑒𝑣𝑒𝑛 is 𝒪(𝑛) and Ω(𝑛),
it is Θ(𝑛). QED
- To obtain a tight bound on the worst-case running time of a function, we need to do 2
things:
 o Use the properties of the code to obtain an asymptotic upper bound on the
worst-case running time. We would say something like 𝑊𝐶𝑓𝑢𝑛𝑐 ∈ 𝒪(𝑓)
o Find a family of inputs whose running time is Ω(𝑓). This will prove that
𝑊𝐶𝑓𝑢𝑛𝑐 ∈ Ω(𝑓)
o After showing that 𝑊𝐶𝑓𝑢𝑛𝑐 ∈ 𝒪(𝑓) and 𝑊𝐶𝑓𝑢𝑛𝑐 ∈ Ω(𝑓), we can conclude that
𝑊𝐶𝑓 ∈ Θ(𝑓)

A Note About Best-Case Runtime

- It is possible to define one
- Would not be useful to know

Early Returning in Python Built-Ins

- The worst-case running time of the in operation for lists is Θ(𝑛), where 𝑛 is the length of
the list
- any and all have a worst-case running time of Θ(𝑛), where 𝑛 is the size of th input
collection

Any, All, and Comprehensions

- The following two lines are slow despite the fact that the function could early-returned
on the first element
>>> any([x == 0 for x in range(0, 1000000)])
>>> any({x == 0 for x in range(0, 1000000)})
- The full comprehension is evaluated before any is called
- We can write the comprehension expression in the function call without any
surrounding square brackets or curly braces
>>> any(x == 0 for x in range(0, 1000000))
- The above example is called a generator comprehension, and is used to produce a
special Python collection data type called a generator
- Generator comprehensions do not evaluate their elements all at once, but instead only
when they are needed by the function being called
- In the previous example, only the x = 0 value from the generator comprehension gets
evaluated

Don’t Assume Bounds are Tight!

- Ex. We say that a string is a palindrome when it can be read the same forwards and
backwards (i.e. ‘abba’, ‘racecar’, ‘z’). We say that a string 𝑠1 is a prefix of another string
𝑠2 when 𝑠1 is a substring of 𝑠2 that starts at index 0 of 𝑠2 (i.e. ‘abc’ is a prefix of ‘abcde’).
- The algorithm below takes a non-empty string as input, and returns the length of the
longest prefix of that string that is a palindrome.
o For example, the string “attack” has 2 non-empty prefixes that are palindromes,
“a” and “atta”, and so our algorithm will return 4

def palindrome_prefix(s: str) -> int:

n = len(s)
for prefix_length in range(n, 0, -1): # goes from n down to 1
# Check whether s[0:prefix_length] is a palindrome
is_palindrome = all(s[i] == s[prefix_length – 1 – i] for i in range(0,
prefix_length)

# If a palindrome prefix is found, return the current length.

if is_palindrome:
return prefix_length
o The for loop iterable is range(n, 0, -1)
The third argument -1 causes the loop variable to start at n and decrease
by 1 at each iteration
• The loop is checking the possible prefixes starting with the longest
prefix (length n) and working its way to the shortest prefix
o The call to all checks pairs of characters starting at either end of the current
prefix
It uses a generator comprehension so that it can stop early as soon as it
encounters a mismatch (i.e. when s[i] != s[prefix_length – 1 – i])
o The algorithm is guaranteed to find a palindrome prefix despite the only return
statement is inside the for loop, since the first letter of s by itself is a palindrome
- It is not too hard to show that the worst-case runtime of this function is 𝒪(𝑛 2 ), but it is
hard to show that the worst-case runtime is Ω(𝑛 2)
o The call to all can stop as soon as the algorithm detects that a prefix is not a
palindrome
o The return statement occurs when the algorithm has determined that a prefix is
a palindrome
- We can consider two extreme input families
o The entire string 𝑠 is a palindrome of length 𝑛. In this case, in the first iteration of
the loop, the entire string is checked. The all call checks all pairs of characters,
but this means that is_palindrome = True, and the loop returns during its very
firsts iterations
Since the all call takes 𝑛 steps, this input family takes Θ(𝑛) time to run
 o The entire string 𝑠 consists of 𝑛 different letters. In this case, the only
palindrome prefix is just the first letter of 𝑠 itself. This means that the loop will
run for all 𝑛 iterations, only returning in its last iteration (i.e. when prefix_length
== 1).
However, the all call will always stop after just 1 step, since it starts by
comparing the first letter of 𝑠 with another letter, which is guaranteed to
be different by our choice of input family.
Leads to a Θ(𝑛) running time
- We want to choose an input family that doesn’t contain a long palindrome (so the loop
runs for many iterations), but whose prefixes are close to being palindromes (so the all
call checks many pairs of letters)
- Let 𝑛 ∈ ℤ+ . We define the input 𝑠𝑛 as follows:
𝑛
o 𝑠𝑛 [⌈ 2⌉] = 𝑏
o Every other character in 𝑠𝑛 is equal to 𝑎
For example, 𝑠4 = 𝑎𝑎𝑏𝑎 and 𝑠11 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑏𝑎𝑎𝑎𝑎
o 𝑠𝑛 is very close to being a palindrome
𝑛
o The longest palindrome of 𝑠𝑛 has length roughly 2
𝑛
The loop iterates roughly 2 times
𝑛
o The “outer” characters of each prefix of 𝑠𝑛 containing more than 2 characters are
all the same
The all call checks many pairs to find the mismatch between 𝑎 and 𝑏
o This input family has an Ω(𝑛 2 ) runtime

8.8 Testing Functions IV: Efficiency

An Efficiency Test
- Consider the following example
from math import floor, sqrt
from timeit import timeit

def is_prime(p: int) -> bool:

“””Return whether p is prime.”””
possible_divisors = range(2, floor(sqrt(p)) + 1)
return p > 1 and all(not p % d == 0 for d in possible_divisors))

def test_is_prime_performance() -> None:

“””Test the efficiency of is_prime.”””
 numbers_to_test = range(2, 1000)
for number in numbers_to_test:
time = timeit(f’is_prime({number})’, number=100,
globals=globals())
assert time < 0.001, ‘Failed performance constraint of 0.001s.’
- Where did number=100 come from?
o Since there are many external factors that can impact the results, several
samples of an experiment (i.e. measurements of time) need to be taken
- Where did 0.001 seconds come from?
o This number is arbitrary as computer systems are different from one another
o This part may be tuned over time in the testing suite
o May help identify the minimum hardware requirements for running a piece of
software
- It is challenging to come up with the actual parameters
o i.e. number of function calls, inputs to the function, total acceptable runtime
- When a code change causes an efficiency test to fail, the programmers can decide
whether to the change efficiency constraint or explore alternative code changes
o Without efficiency tests in place, the change in performance might not have
been found until it impacted a real user of the software

9.1 An Introduction to Abstraction

Intro
- We can think of abstraction as allowing for the separation of two groups of people with
different goals:
o The creators of an entity
Responsible for designing, building, and implementing an entity
o The users (or clients) of that entity
Responsible for using it
- The interface of an entity is the boundary between creator and user
o Interface – the set of rules (implicit or explicit) governing how users can interact
with that entity
The public side of an entity
The part of the creator’s work that everyone can interact it

Abstraction in Computer Science

 - We are users of the Python programming language, which provides an interface that
hides the details of our computer hardware, processor instructions, memory, storage,
and graphics
- We are users of built-in Python functions, data types, and modules
- We are creators of new Python functions, data types and modules
- For a function, its interface is its header and docstring
o These specify how to call the function, the arguments it expects, and what the
function does
o The function body is not part of the interface
- For a data class, its interface is the name of the data class and the names and types of
its attributes, and the class docstring
o Every part of what we write to define a new data class is part of its interface
- For a Python module, the interface is the collection of interfaces of the functions and
data types defined in that module, plus any additional documentation in the module
- Every time we write a proof, we act as a creator of knowledge, providing airtight
evidence that a statement is True
- Every time we use an “external statement” in a proof (i.e. Quotient-Remainder
Theorem, Fermat’s Little Theorem), we are acting as users of these statements, and do
not worry about how they have been proved

Interfaces are Contracts

- Every interface is a contract between creator and user
o Creators have the responsibility to make the interface easy and intuitive for
users
- When we act as the creators of a function or module, we are free to modify their
implementations in any way we wish, as long as we do not change the public interface

9.2 Defining Our Own Data Types, Part 3

What If We Just Remove the @dataclass?

- Recall the Person data class example
# @dataclass
class Person:
“””A custom data type that represents data for a person.”””
given_name: str
family_name: str
age: int
address: str
 - If we remove the @dataclass decorator from our class definition (it is commented out),
we get an unexpected consequence
>>> david = Person(‘David’, ‘Liu’, 100, ’40 St. George Street’)
TypeError: Person() takes no arguments
- We can create an instance of the Person class passing in zero arguments
>>> david = Person()
>>> type(david)
<class ‘Person’>

Defining an Initializer
- A Person object has been created, but it has no attributes
- We need to define a new method for Person called the initializer
o The initializer method of a class is called when an instance of the class is created
in Python
o Purpose: to initialize all of the instance attributes for the new object
o Python always use the name __init__ for the initializer method
- When we use the @dataclass decorator the Python interpreter automatically creates an
initializer method for the class, like what is shown below:
class Person:
“””A custom data type that represents data for a person.”””
given_name: str
family_name: str
age: int
address: str

def __init__(self, given_name: str, family_name: str, age: int, address: str)
-> None:
“””Initialize a new Person object.”””
self.given_name = given_name
self.family_name = family_name
self.age = age
self.address = address
- This method is indented so that it is inside the body of the class Person definition
- Every initializer has a first parameter that refers to the instance that has just been
created and is to be initialized
o By convention, we always call it self
o We could have written self: Person, but it is redundant because the type for self
should always be the class that the initializer belongs to
 - We use the initializer by calling the data class as usual:
>>> david – Person(‘david’, ‘Liu’, 100, ’40 St. George Street’)
o The initializer is called automatically
o We never have to pass a value for self
Python automatically sets it to the instance that is to be initialized
- Memory at the beginning of the initializer:
__main__ id60 Person id11 str
“David”
Person.__init__ id12 str id13 int
self id60 “Liu” 100
given_name id11
family_name id12 id14 str
age id13 “40 St. George Street”
address id14
- The initializer’s job is to create and initialize the instance attributes
o To do this, we use one assignment statement per instance attribute
Uses the same dot notation syntax for assigning to instance attributes
o given_name is a parameter of the initialize
o self.given_name is an instance attribute
- Memory immediately before the initializer returns:
id60 Person
given_name id11
__main__ id11 str
family_name id12
“David”
age id13
address id14
Person.__init__ id12 str id13 int
self id60 “Liu” 100
given_name id11
family_name id12 id14 str
age id13 “40 St. George Street”
address id14

What Really Happens When We Create a New Object

- Person doesn’t just cause __init__ to be called, instead, it does three things:
o Create a new Person object behind the scenes
o Call __init__ with the new object passed to the parameter self, along with the
other arguments
 o Return the new object. This step is where the object is returned, not directly
from the call to __init__ in step 2
- __init__ is a helper function in the object creation process
o Its task is only to initialize attributes for an object
o Python handles both creating the object beforehand, and returning the new
object after __init__ has been called

Methods as Part of a Data Type Interface

- We can define methods for a data type, which become part of the interface of that data
type
- When we define a class with methods, those methods are always bundled with the
class, and so any instance of the class can use those methods, without needing to
import them separately
- One example of a method definition is the initializer, __init__
- Any function that operates on an instance of a class can be converted into a method by
doing the following:
o Indent the function so that it is part of the clss body, underneath the instance
attributes
o Ensure that the first parameter of the function is an instance of the class, and
name this parameter self
- For example, suppose we had the following function to increase a person’s age:
def increase_age(person: Person, years: int) -> None:
“””Add the given number of years to the given person’s age.”””
person.age = person.age + years
- We can turn increase_age into a Person method as follows:
class Person:
“””A custom data type that represents data for a person.”””
given_name: str
family_name: str
age: int
address: str

def __init__(self, given_name: str, family_name: str, age: int, address: str)
-> None:
“””Initialize a new Person object.”””
self.given_name = given_name
self.family_name = family_name
self.age = age
 self.address = address

def increase_age(self, years: int) -> None:

“””Add the given number of years to this person’s age”””

self.age = self.age + years

o We use parameter self (without a type annotation) to access instance attributes,
just like what we did in the initializer

Shortcut Syntax for Method Calls

- We can call increase_age like this:
>>> Person.increase_age(david, 10)
- The alternate form for calling the increase_age method is to use dot notation with the
Person instance directly:
>>> david.increase_age(10)
- When we call david.increase_age(10), the Python interpreter does the following:
o It looks up the class of david, which is Person
o It looks up the increase_age method of the Person class
o It looks up the increase_age on david and 10
The interpreter automatically passes the value to the left of the dot (i.e.
david) as the method’s first parameter self
- This works for all built-in data types as well
o list.append(lst, 10) can be written as lst.append(10)
o str.lower(s) can be written as s.lower()
o obj.method(x1, x2, …, xn) is equivalent to type(obj).method(obj, x1, x2, …, xn)
- The “object dot notation” style (i.e. david.increase_age) is the more common one in
Python programming
o It matches other languages with an object-oriented style of programming, where
the object being operated own is of central importance
david.increase_age(10) implies that david is the most important object in
the code expression
o Only the “object dot notation” style supports inheritance

9.3 Data Types, Abstract and Concrete

Intro
- Concrete data types – synonymous to Python class
o Have concrete implementations in Python code
 - Abstract data type (ADT) – defines an entity that stores some kind of data and the
operations that can be performed on it
o Language-independent
o Pure interface concerned only with the what (i.e. what data is stored, what we
can do with the data) and not the how (i.e. how a computer actually stores this
data or implements these operations)

Familiar Abstract Data Types

- Set
o Data: a collection of unique elements
o Operations: get size, insert a value (without introducing duplicates), remove a
specified value, check membership in the set
- List
o Data: an ordered sequence of elements (which may or may not be unique)
o Operations: get size, access element by index, insert a value at a given index,
remove a value at a given index
- Iterable
o Data: a collection of values (may or may not be unique)
o Operations: iterate through the elements of the collection one at a time
- ADT’s are abstract enough to transcend any individual program or even programming
languages

Abstract vs. Concrete Data Types

- dict is not an abstract data type, but it is an obvious implementation of the Mapping
ADT
- There is not a one-to-one correspondence between abstract data types and concrete
data types, in Python or any other programming language
- A single abstract data type can be implemented by many different concrete data types
o {0: ‘hello’, 1: 42, 2: ‘goodbye’} # A Map using a Python dict
o [(0, ‘hello’), (1, 42), (2, ‘goodbye’)] # A Map using a Python list
- Every concrete data type can be used to implement multiple ADTs
- i.e. it is possible to implement a Map using list, but this choice is worse than using dict
o Because of efficiency

9.4 Stacks

The Stack ADT

- A stack contains 0 or more items
- When we add an item, it goes “on the top” of the stack
o “Pushing” onto the stack
- When we remove an item, it is removed from the top
o “Popping” from the stack
- Last-In-First-Out (LIFO) behaviour
- Stack
o Data: a collection of items
o Operations: determine whether the stack is empty, add an item (push), remove
the most recently-added item (pop)
class Stack
“””A last-in-first-out (LIFO) stack of items.
Stores data in last-in, first-out order. When removing an item from the
stack, the most recently-added item is the one that is removed.
Sample Usage:
>>> s = Stack()
>>> s.is_empty()
True
>>> s.push(‘hello’)
>>> s.is_empty()
False
>>> s.push(‘goodbye’)
>>> s.pop()
‘goodbye’
“””
def __init__(self) -> None:
“””Initialize a new empty stack.”””

def is_empty(self) -> bool:

“””Return whether this stack contains no items.”””

def push(self, item: Any) -> None:

“””Add a new element to the top of this stack.”””

def pop(self) -> Any:

“””Remove and return the element at the top of this stack.

Preconditions:
- not self.is_empty()
 “””

Applications of Stacks
- “Undo” feature
o We want to undo the most recent action

Implementing the Stack ADT Using Lists

- We can implement the Stack ADT using list
- We’ve chosen to use the end of the list to represent the top of the stack
class Stack1
“””…
Instance Attributes:
- items: The items stored in the stack. the end of the list
represents the top of the stack.
…”””
items: list

def __init__(self) -> None:

“””…”””
self.items = []

def is_empty(self) -> bool:

“””…”””
return self.items == []

def push(self, item: Any) -> None:

“””…”””
self.items.append(item)

def pop(self) -> Any:

“””…”””
return self.items.pop()

Attributes and the Class Interface

- Users can also access the items instance attribute
- To make an instance attribute that isn’t part of a class’ interface, we prefix its name with
an underscore _
- Private instance attributes – attributes whose names begin with an underscore
 - Public instance attributes – attributes whose names do not begin with an underscore
- All public instance attributes are part of the interface, all private ones aren’t
class Stack1:
“””…”””
# Private Instance Attributes:
# - _items: The items stored in the stack. The end of the list
# represents the top of the stack.
_items: list

def __init__(self) -> None:

“””…”””
self._items = []

def is_empty(self) -> bool:

“””…”””
return self._items == []

def push(self, item: Any) -> None:

“””…”””
self._items.append(item)

def pop(self) -> Any:

“””…”””
return self._items.pop()
- There is no mention of the attribute _items when we call help on our class

Warning: Private Attributes Can Be Accessed!

- Private instance attributes can still be accessed from outside the class
- The Python programming language prefers flexibility over restriction when it comes to
accessing attributes
- By making an instance attribute private, we are communicating that client code should
not access this attribute
o We reduce the cognitive load on the client
o We also give flexibility to the designer of the class to change or even remove a
private attribute if they want to update their implementation of the class,
without affecting the class’ public interface

Analyzing Efficiency
 - We could have implemented Stack1 using the front of _items to represent the top of the
stack
class Stack2:
# Duplicated code from Stack1 omitted. Only push and pop are different.

def push(self, item: Any) -> None:

“””…”””
self._items.insert(0, item)

def pop(self0 -> Any:

“””…”””
return self._items.pop(0)
- Key difference between Stack1 and Stack2: efficiency
o Running time of Stack1.push: Θ(1)
o Running time of Stack2.push: Θ(𝑛)
o Running time of Stack1.pop: Θ(1)
o Running time of Stack2.pop: Θ(𝑛)

9.5 Exceptions As a Part of the Public Interface

Letting an Error Happen

- Consider this version of Stack.pop, which removes the precondition but keeps the same
implementation
def pop(self) -> Any:
“””…”””
return self._items.pop()
o When we call pop on an empty stack, we encounter an IndexError
o From the perspective of the client code, it is bad that the exception report refers
to a list (IndexError: pop from empty list) and a private attribute (self._items)
that the client code should have no knowledge of

Custom Exceptions
- A better solution is to raise a custom exception that is descriptive, yet does not reveal
any implementation details
- We can define our own type of error by defining a new class
class EmptyStackError(Exception):
“””Exception raised when calling pop on an empty stack.”””
- To use EmptyStackError in our pop method,
 def pop(self) -> Any:
“””Remove and return the element at the top of this stack.
Raise a EmptyStackError if this stack is empty.
“””
if self.is_empty():
raise EmptyStackError
else:
return self._items.pop()
- The exception is now part of the public interface because the docstring names both the
type of exception and the scenario that will cause that exception to be raised
- The Python keyword raise will raise an exception
- When we call pop on an empty stack, it will display EmptyStackError rather than
mentioning any implementation details

Custom Exception Messages

- To provide a custom exception message, we can define a new special method with the
name __str__ in our exception class:
class EmptyStackError(Exception):
“””…”””
def __str__(self) -> str:
“””Return a string representation of this error.”””
return ‘pop may not be called on an empty stack’
- Now it will display EmptyStackError: pop may not be called on an empty stack

Testing Exceptions
- We cannot simply call pop on an empty stack and check the return value or the state of
stack after pop returns
o Raising an error interrupts the regular control flow of a Python program
- The pytest module allows us to write tests that expects an exception to occur using a
function pytest.raises together with the with keyword
# Assuming our stack implementation is contained in a file stack.py.
from stack import Stack, EmptyStackError
import pytest

def test_empty_stack_error():
“””Test that popping from an empty stack raises an exception.”””
s = Stack()
 with pytest.raises(EmptyStackError):
s.pop()
o The test passes when that exception is raised, and fails when that exception is
not raised
Also fails when a different exception is raised

Handling Exceptions
- Python provides the try-except statement to execute a block of code and handle3 a case
where one or more pre-specified exceptions are raised in that block
o The simplest form of a try-except statement:
try:
<statement>
…
except <ExceptionClass>:
<statement>
…
- When a try-except statement is executed:
o The block of code indented within the try is executed
o If no exception occurs when executing this block, the except part is skipped, and
the Python interpreter continues to the next statement after the try-except
o If an exception occurs when executing this block:
If the exception has type <ExceptionClass>, the block under the except is
executed, and then after that the Python interpreter continues executing
the next statement after the try-except
• In this case the problem does not immediately half
If the exception is a different type, this does stop the normal program
execution
- Try-except statements shield users from seeing errors that they should never see, and
allows the rest of the program to continue
- Example: a function that takes a stack and returns the second item from the top of the
stack
def second_from_top(s: Stack) -> Optional[str]:
“””Return the item that is second from the top of s.
If there is no such item in the Stack, returns None.
“””
try:
hold1 = s.pop()
except EmptyStackError:
 # In this case, s is empty. We can return None.
return None

try:
hold2 = s.pop()
except EmptyStackError:
# In this case, s had only 1 element
# We restore s to its original state and return None
s.push(hold1)
return None

# If we reach this point, both of the previous s.pop() calls succeeded.

# In this case, we restore s to its original state and return the second
item.
s.push(hold2)
s.push(hold1)

return hold2

9.6 Queues

The Queue ADT

- Unlike a stack, items come out of a queue in the order in which they entered
o First-In-First-Out (FIFO)
- Queue
o Data: a collection of items
o Operations: determine whether the queue is empty, add an item (enqueue),
remove the least recently-added item (dequeue)
class Queue:
“””A first-in-first-out (FIFO) queue of items.
Stores data in a first-in, first-out order. When removing an item from the
queue, the most recently-added item is the one that is removed
>>> q = Queue()
>>> q.is_empty()
True
>>> q.enqueue(‘hello’)
>>> q.is_empty()
False
 >>> q.enqueue(‘goodbye’)
>>> q.dequeue()
‘hello’
>>> q.dequeue()
‘goodbye’
>>> q.is_empty()
True

def __init__(self) -> None:

“””Initialize a new empty queue.”””

def is_empty(self) -> bool:

“””Return whether this queue contains no items.”””

def enqueue(self, item: Any) -> None:

“””Add <item> to the back of this queue.”””

def dequeue(self) -> Any:

“””Remove and return the item at the front of this queue.

Raise an EmptyQueueError if this queue is empty.

“””

class EmptyQueueError(Exception):
“””Exception raised when calling dequeue on an empty queue.”””

def __str__(self) -> str:

“””Return a string representation of this error.”””
return ‘dequeue may not be called on an empty queue’

List-Based Implementation of the Queue ADT

- We have decided that the beginning of the list (i.e. index 0) is the front of the queue
class Queue:
“””…”””
# Private Instance Attributes:
# - _items: The items stored in this queue. The front of the list
# represents the front of the queue.
_items: list
 def __init__(self) -> None:
“””…”””
self._items = []

def is_empty(self) -> bool:

“””…”””
return self._items == []

def enqueue(self, item: Any) -> None:

“””…”””
self._items.append(item)

def dequeue(self) -> Optional[Any]:

“””…”””
if self.is_empty():
raise EmptyQueueerror
else:
return self._items.pop(0)

Implementation Efficiency
- Our Queue.enqueue calls list.append, which takes constant time
- Our Queue.dequeue calls self._items.pop(0), which takes Θ(𝑛) time
- If we change things around so that the front of the queue is the end of the list (rather
than the beginning), we simply swap these running times
- Using an array-based list, we can either have an efficient enqueue or an efficient
dequeue operation

9.7 Priority Queues

The Priority Queue ADT

- Items are removed from a Priority Queue in order of their priority
- Priority Queue
o Data: a collection of items and their priorities
o Operations: determine whether the priority queue is empty, add an item with a
priority (enqueue), remove the highest priority item (dequeue)
- We will represent priorities as integers, with larger integers representing higher
priorities
class PriorityQueue
“””A collection items that are be removed in priority order.
When removing an item from the queue, the highest-priority item is the
one that is removed.
>>> pq = PriorityQueue()
>>> pq.is_empty()
True
>>> pq.enqueue(1, ‘hello’)
>>> pq.is_empty()
False
>>> pq.enqueue(5, ‘goodbye’)
>>> pq.enqueue(2, ‘hi’)
>>> pq.dequeue()
‘goodbye’
“””

def __init__(self) -> None:

“””Initialize a new and empty priority queue.”””

def is_empty(self) -> bool:

“””Return whether this priority queue contains no items.”””

def enqueue(self, priority: int, item: Any) -> None:

“””Add the given item with the given priority to this priority
queue.
“””

def dequeue(self) -> Any:

“””Remove and return the item with the highest priority.
Raise an EmptyPriorityQueueError when the priority queue is
empty.
“””

class EmptyPriorityQueueError(Exception):
“””Exception raised when calling dequeue on an empty priority queue.”””

def __str__(self) -> str:

“””Return a string representation of this error.”””
 return ‘You called dequeue on an empty priority queue.’

List-Based Implementation of the Priority Queue ADT

- Our implementation idea here is to use a private attribute that is a list of tuples, where
each tuple is a (priority, item) pair
- Our list will also be sorted with respect to priority (breaking ties by insertion order), so
that the last element in the list is always the next item to be removed from the priority
queue
class PriorityQueue:
“””…”””
# Private Instance Attributes:
# - _items: a list of the items in this priority queue
_items: List[Tuple[int, Any]]

def __init__(self) -> None:

“””…”””
self._items = []

def if_empty(self) -> bool:

“””…”””
return self._items == []

def dequeue(self) -> Any:

“””…”””
if self.is_empty():
raise EmptyPriorityQueueError
else:
_priority, item = self._items.pop()
return item
o For PriorityQueue.enqueue, we can insert the new priority and item into the list,
and then sort the list by priority, however, this is inefficient
o Our enqueue implementation will search for the right index in the list to add the
new item
def enqueue(self, priority: int, item: Any) -> None:
“””…”””
i=0
while i < len(self._items) and self._items[i][0] < priority:
# Loop invariant: all items in self._items[0:i]
 # have a lower priority than <priority>
i=i+1

self._items.insert(i, (priority, item))

o It should be < rather than <= because if there already exist items with the same
priority, the new one should be the least-prioritized (more in front than other
items with the same priority but behind items with lower priority)
- Analysis
o The while loop takes at most 𝑛 iterations, since i starts at 0 and increases by 1 at
each iteration, and the loop must stop when i reaches 𝑛 (if it hasn’t stopped
earlier)
Since each loop iteration takes 1 step, in total the while loop takes at
most 𝑛 steps
o list.insert takes at most 𝑛 steps, where 𝑛 is the length of the list being inserted
into
o Adding up these two quantities, the total running time of this algorithm is at
most 𝑛 + 𝑛 = 2𝑛 steps, which is 𝒪(𝑛)
- We can incorporate the value of variable i in our calculation. Let 𝐼 be the value of
variable i after the loop finishes. Then:
o We now know that the while loop takes exactly 𝐼 iterations, for a total of 𝐼 steps
(1 step per iterations)
o We know that calling list.insert on a list of length 𝑛 to insert an item at index 𝐼
takes 𝑛 − 𝐼 steps
o So the total running time is actually 𝐼 + (𝑛 − 𝐼) = 𝑛 steps, which is Θ(𝑛)
- We’ve shown that every call to this implementation of PriorityQueue.enqueue will take
Θ(𝑛) time, regardless of the priority being inserted.

Using an Unsorted List

- If we used an unsorted list of tuples instead, we would have Θ(1) enqueue operations,
simply by appending a new (priority, item) tuple to the end of self._items
- However, we must search for the highest priority item in a list of unsorted items, which
would take Θ(𝑛) time

Looking Ahead: Heaps

- Heap – a data structure commonly used to implement the Priority Queue ADT in
practice
- We can use this data structure to implement both PriorityQueue.enqueue and
PriorityQueue.dequeue with a worst-case running time of Θ(log 𝑛)
9.8 Defining a Shared Public Interface with Inheritance

Intro
- Recall that the Stack ADT can be implemented using a Python list in 2 ways:
o Storing the top of the stack at the end of the list (Stack1)
o Storing the top of the stack at the front of the list (Stack2)
- They share the same public interface of the Stack ADT

The Stack Abstract Class

- Defining a Stack class that consists only of the public interface of the Stack ADT
class Stack:
“””A last-in-first-out (LIFO) stack of items.

This is an abstract class. Only subclasses should be initiated.

“””

def is_empty(self) -> bool:

“””…”””
raise NotImplementedError

def push(self, item: Any) -> None:

“””…”””
raise NotImplementedError

def pop(self) -> Any:

“””…”””
raise NotImplementedError

class EmptyStackError(Exception):
“””Exception raised when calling pop on an empty stack.”””

- In Python, we mark a method as unimplemented by having its body raise a special

exception, NotImplementedError
o We say that a method is abstract when it is not implemented and raises this
error
o We say that a class is abstract when at least one of its methods is abstract (i.e.
not implemented)
 o A concrete class is a class that is not abstract
- The Stack class we have defined is a direct translation of the Stack ADT: an interface that
describes the methods that a concrete class that wants to implement the Stack ADT
must define

Inheriting the Stack Abstract Class

- Earlier, we defined Stack1 and Stack2, however, the code did not indicate that the types
were related in any way
- With the abstract class Stack, we can indicate this relationship in the code through
inheritance
class Stack1(Stack):
“””…”””

class Stack2(Stack):
“””…”””
o The syntax (Stack) indicates that Stack1 and Stack2 inherit from Stack
o Stack: base class, superclass, parent class
o Stack1, Stack2: subclass, child class, derived class
- When one class in Python inherits from another,
o The Python interpreter treats every instance of the subclass as an instance of the
superclass as well
>>> s1 = Stack()
>>> isinstance(s1, Stack1)
True
>>> isinstance(s1, Stack)
True
>>> isinstance(s1, Stack2)
False
o When the superclass is abstract, the subclass must implement all abstract
methods from the superclass, without changing the public interface of those
methods
- Inheritance serves as a form of contract:
o The implementer of the subclass must implement the methods from the abstract
superclass
o Any user of the subclass may assume that they can call the superclass methods
on instances of the subclass
- Because Stack1 and Stack2 are both subclasses of Stack, we expect them to implement
all the stack methods
 o They might also implement additional methods that are unique to each subclass
(i.e. not shared)

Writing Polymorphic Code Using Inheritance

- Consider the following code that operates on a stack:
def push_and_pop(s: Stack, item: Any) -> None:
“””Push and pop the given item onto the stack s.”””
s.push(item)
s.pop()
o We use the abstract data class Stack as the type annotation, to indicate that our
function push_and_pop can be called with any instance of any Stack subclass
o We can pass a Stack1 or Stack2 object to the push_and_pop function because
they both inherit from Stack
>>> s1 = Stack1()
>>> push_and_pop(s1) # This works
>>> s2 = Stack2()
>>> push_and_pop(s2) # This also works
- There are 3 versions of push: Stack.push, Stack1.push, Stack2.push
o When the Python interpreter evaluates s.push(item), it first computes type(s).
i.e. type(s1) is Stack1, type(s2) is Stack2
o The Python interpreter then looks in that class for a push method and calls it,
passing in s for the self argument
- We say that the Python interpreter dynamically looks up (or resolves) the s.push/.pop
method, because the actualmethodf called by s.push/.pop changes depending on the
argument passed to push_and_pop
- We say that the push_and_pop function is polymorphic
o Polymorphic – being able to take as inputs values of different concrete data type
and select a specific method based on the type of input
- Inside the push_and_pop function, neither one of below would work
o Stack.push(s, item)
Raises a NotImplementedError
o Stack1.push(s, item)
Only works on the Stack1 instances, but not any other Stack subclasses
o Both make push_and_pop no longer polymorphic

Application: Running Timing Experiments on Stack Implementation

- We can use polymorphism to help us measure the performance of each implementation
(i.e. Stack1 and Stack2)
 o We are calling the same function for both timing experiments (of Stack1 and of
Stack2)

9.9 The Object Superclass

Intro
- object is an ancestor class of every other class
o Ancestor class – parent class, or parent of a parent class
- Whenever we define a new class (including data classes), if we do not specify a
superclass in parentheses, object is the implicit superclass

The Object Special Methods

- The object class defines several special methods as part of its shared public interface,
including:
o __init__(self, …)
The initializer
o __str__(self)
Returns a str representation of the object

Method Inheritance
- The object class is not abstract and implements each of the special methods
- Here, where the superclass is a concrete class, inheritance is used not just to define a
shared public interface, but also to provide default implementations for each method in
the interface
- Suppose we create a dummy class with a completely empty body:
class Donut:
“””A donut.”””
- This class inherits the object.__init__ method, which allows us to create new Donut
instances
>>> donut = Donut()
>>> type(donut)
<class ‘__main__.Donut’>
- Similarly, this class inherits the object.__str__ method, which returns a string that states
the class name and memory location of the object
>>> d = Donut()
>>> d.__str__()
‘<__main__.donut object at 0x7fc299d7b588>’
 - We can use the built-in dir function to see all of the special methods that Donut has
inherited form object
>>> dir(Donut)
[‘__class__’, ‘__delattr__’, ‘__dict__’, (the rest is omitted by me)]
- The special methods are often called by other functions or parts of Python syntax
o We have already seen how the __init__ method is called when a new object is
initialized
o The __str__ method is called when we attempt to convert an object to a string
by calling str on it
>>> d = Donut()
>>> d.__str__()
‘<__main__.Donut object at 0x7fc299d7b588>’
>>> str(d)
‘<__main__.Donut object at 0x7fc299d7b588>’
o The built-in print function first converts its arguments into strings using their
__str__ methods, and then prints out the resulting text

Method Overriding
- Every time we’ve defined our own __init__ in a class, we have overridden the
object.__init__ method
- We say that a class C overrides a method m when the method m is defined in the
superclass of C, and is also given a concrete implementation in the body of C
- When we defined a custom exception class
class EmptyStackError(Exception):
“””…”””
def __str__(self) -> str:
“””…”””
return ‘pop may not be called on an empty stack’
o This class overrode the __str__ method to use its own string representation,
which is displayed when this exception is raised.

10.1 The Problem Domain: Food Delivery Networks

What Is a Problem Domain?

- Problem domain – collection of knowledge about a specific field, phenomenon, or
discipline, and an understanding of the goals, problems, deficiencies, and/or desired
improvements within that area
o We’ve touched on a wide array of problem domains, such as:
 Tracking marriage records in Toronto
Modelling the spread of infectious diseases
Generating course timetables as U of T students
Cryptography
- As we dove into cryptography, we learned about:
o Terminology and definitions
Symmetric-key and public-key cryptosystems
Encryption and decryption
Various existing cryptosystems
o Concepts and skills
Proving that a cryptosystem is correct
Justifying the security of a cryptosystem based on the presumed
hardness of mathematical problems like Integer Factorization
o The context and history
Ancient cryptosystems
How cryptography is applied to Internet communications
- It was the domain-specific knowledge we learned that explained how we came up with
the cryptography algorithms and why they are correct

Introducing Hercules
- We want to launch a Hercules app that allows people to order groceries and meals from
grocery stores and restaurants, and arrange for couriers to make deliveries right to their
front doors
- When designing and implementing this app, we need to consider:
o How restaurants will register with the app and post menus
o How customers will register with the app to browse restaurants and place orders
o How couriers will register with the app to claim orders and deliver them from
restaurants to customers
o … and more

Food Delivery As a System

- We can view food delivery in Toronto as a system
o System – a group of entities (or agents) that interact with each other over time
- The first part of creating a computational model for such a system is to design and
implement the various entities in the system
o In the case of the Hercules Ltd., these are entities like couriers and customers
placing orders
- The entities in a system change over time
 o i.e. new people sign up and place food orders
- The second part of our computational model is a simulation that uses randomness to
generate events that cause the system to change over time
o i.e. our food delivery simulation will specify how often customers place an order,
taking into account that some times of day are busier than others
- A well-designed simulation allows the programmer to start with a simple model and
extend and tweak it in response to new domain-specific knowledge

10.2 Object-Oriented Modelling of Our Problem Domain

Entities in a Food Delivery System

- We use two strategies for picking our relevant entities:
o Identify different roles that people/group play in the domain
Each “role” is likely an entity
• i.e. customer, courier, restaurant
o Identify a bundle of data that makes sense as a logical unit
Each “bundle” is likely an entity
• i.e. an order is a bundle of related information about a user’s food
request
- A standard approach is to create a class to represent each of these entities
o We can start with a data class and turn it into a general class if we need a more
complex design (i.e. to add methods, including the initializer, or mark attributes
as private)
@dataclass
class Restaurant:
“””A place that servers food.”””

@dataclass
class Customer:
“””A person who orders food.”””

@dataclass
class Courier:
“””A person who delivers food orders from restaurants to customers.”””

@dataclass
class Order:
“””A food order from a customer.”””
Designing the Restaurant Data Class
- We need a way to identify each restaurant: its name
o We’ll use a str to represent it
- A user needs to see what food is available to order, so we need to store a food menu for
each restaurant
o Since it has a few different options, we’ll use a dict that maps the names of
dishes (strs) to their price (floats)
- Couriers need to know where restaurants are in order to pick up food orders, and so we
need to store a location for each restaurant
o We could store its address as a str
o We could also store the latitude and longitude (a tuple of floats)
- Each of these three pieces of information (restaurant name, food menu, location) are
appropriate attributes for the restaurant
@dataclass
class Restaurant:
“””A place that serves food.

Instance Attributes:
- name: the name of the restaurant
- address: the address of the restaurant
- menu: the menu of the restaurant with the name of the dish
mapping to the price
- location: the location of the restaurant as (latitude, longitude)

Representation Invariants:
- all(self.menu[item] >= 0 for item in self.menu)
- -90 <= self.location[0] <= 90
- -180 <= self.location[1] <= 180
“””
name: str
address: str
menu: Dict[str, float]
location: Tuple[float, float]
- Since the menu is a compound data type, we could have created a completely separate
Menu data class
- Each new class we create introduces a little more complexity into our program, and for a
relatively simple class for a menu, this additional complexity does not worth it
 - We could have used a dictionary to represent a restaurant instead of a Restaurant data
class
o This would have reduced on area of complexity, but introduced another
i.e. the “valid” keys of a dictionary used to represent a restaurant

Designing the Order Data Class

- An order must track the customer who placed the order, the restaurant where the food
is being ordered from, and the food items that are being ordered
- An order should have an associated courier who has been assigned to deliver the order
- We’ll also keep track of when the order was created, and when the order is completed
- The associated courier and the time when the order is completed might only be
assigned values after the order has been created
o We use a default value None to assign to these two instance attributes when an
Order is first created
o We could implement this by converting the data class to a general class and
writing our own __init__ method
o We’ll take advantage of a new feature with data classes: the ability to specify
default values for an instance attribute after the type annotation
@dataclass
class Order
“””A food order from a customer.

Attributes:
- customer: the name of the customer who placed this order
- restaurant: the name of the restaurant the order is place for
- food_items: Dict[str, int]
- start_time: datetime.datetime
- courier: Optional[Courier] = None
- end_time: Optional[datetime.datetime] = None
- The line courier: Optional[Courier] = None is how we define an instance attribute
Courier with a default value of None
o The type annotation Optional[Courier] means tha this attribute can either bge
None or a Courier instance
o Similarly, the end_time attribute must be either None (its initial value) or a
datetime.datetime value
- Here is how we could use this class
o Note: Customer is currently an empty data class, and so is instantiated simply as
Customer()
 >>> david = Customer()
>>> mcdonalds = Restaurant(name=’McDonalds’, address=’160 Spadina Ave’,
menu={‘fries’: 4.5}, location=(43.649, -79.397))
>>> order = Order(customer=david, restaurant=mcdonalds, food_items={‘fries’:
10}, start_time=datetime.datetime(2020, 11, 5, 11, 30))

>>> order.courier is None # Illustrating default values

True
>>> order.end_time is None
True

Class Composition
- Classes can be “nested” within each other through their instance attributes
o i.e. our Order data class has attributes which are instances of other classes we
have defined (Customer, Restaurant, and Courier)
- The relationship between Order and these other classes is called class composition, and
is fundamental to object-oriented design
- We use class composition to represent a “has a” relationship between two classes
o i.e. “an Order has a Customer”

10.3 A “Manager” Class

Intro
- We can create a new manager class whose role is to keep track of all the entities in the
system and to mediate the interactions between them (like a customer placing a new
order)
- The FoodDeliverySystem will store (and have access to) every customer, courier, and
restaurant represented in our system
class FoodDeliverySystem
“””A system that maintains all entities (restaurants, customers, couriers,
and orders).

Public Attributes:
- name: the name of this food delivery system

Representation Invariants:
- self.name != ‘’
- all(r == self._restaurants[r].name for r in self._restaurants)
 - all(c == self._customers[c].name for c in self._customers)
- all(c == self._couriers[c].name for c in self._couriers)
“””
name: str

# Private Instance Attributes:

# - _restaurants: a mapping from restaurant name to Restaurant
# object. This represents all the restaurants in the system.
# - _customers: a mapping from customer name to Customer
# object. This represents all the customers in the system.
# - _couriers: a mapping from courier name to Courier object.
# This represents all the couriers in the system.
# - _orders: a list of all orders (both open and completed orders).

_restaurants: Dict[str, Restaurant]

_customers: Dict[str, Customer]
_couriers: Dict[str, Courier]
_orders: List[Order]

def __init__(self, name: str) -> None:

“””Initialize a new food delivery system with the given company
name.

The system starts with no entities.

“”"
self.name = name
self._restaurants = {}
self._customers = {}
self._couriers = {}
self._orders = []

Changing State
- So far, we have modelled the static properties of our food delivery system, that is, the
attributes that are necessary to capture a particular snapshot of the state of the sytem
at a specific moment in time
- Adding entities
o We can define simple methods to add entities to the system
class FoodDeliverySystem:
 …
def add_restaurant(self, restaurant: Restaurant) -> bool:
“””Add the given restaurant to this system.
Do NOT add the restaurant if one with the same name already
exists.
Return whether the restaurant was successfully added to this
system.
“””
if restaurant.name in self._restaurants:
return False
else:
self._restaurants[restaurant.name] = restaurant
return True

def add_customer(self, customer: Customer) -> bool”

“””Add the given customer to this system.
Do NOT add the customer if one with the same name already
exists.
Return whether the courier was successfully added to this system.
# Similar implementation to add_restaurant

def add_courier(self, courier: Courier) -> bool:

“””Add the given courier to this system.
Do NOT add the courier if one with the same name already exists.
Return whether the courier was successfully added to this system.
# Similar implementation to add_restaurant
- Placing orders
o When a customer places an order, a chain of events is triggered:
1. The order is sent to the restaurant and to the assigned courier.
2. The courier travels to the restaurant and picks up the food, and then
brings it to the customer.
3. Once the courier has reached their destination, they indicate that the
delivery has been made.
class FoodDeliverySystem:
…
def place_order(self, order: Order) -> None:
“””Record the new given order.
Assign a courier to this new order (if a courier is available).
 Preconditions:
- order not in self.orders
“””

def complete_order(self, order: Order) -> None:

“””Mark the given order as complete.
Make the courier who was assigned this order available to take a
new order.

Preconditions:
- order in self.orders
“””
o FoodDeliverySystem.place_order would be responsible for both recording the
order and assigning a courier to that order
o FoodDeliverySystem.complete_order marks the order as complete and un-
assigning the courier so that they are free to take a new order

10.4 Food Delivery Events

The Event Interface

- We’ll define abstract Event class with subclasses NewOrderEvent and
CompleteOrderEvent to represent different kinds of events
class Event:
“””An abstract class representing an event in a food delivery
simulation.”””

def handle_event(self, system: FoodDeliverySystem) -> None:

“””Mutate the given food delivery system to process this
event.”””
raise NotImplementedError
o The abstract method handle_event is how we connect each event to a change in
a food delivery system
o Each Event subclass is responsible for implementing handle_event based on the
type of change the subclass represents
The NewOrderEvent.handle_event method should add a new order to
the system
Common Instance Attributes
- A superclass can declare public instance attributes that its subclasses must have in
common
- For our Event class, we can establish that all event subclasses will have a timestamp
indicating when the event took place
class Event:
“””…

Instance Attributes:
- timestamp: the start time of the event
“””
timestamp: datetime.datetime

def __init__(self, timestamp: datetime.datetime) -> None:

“””Initialize this event with the given timestamp.”””
self.timestamp = timestamp

class NewOrderEvent(Event):
“””An event where a customer places an order for a restaurant.”””
o Since subclasses inherit all the methods from their superclass, we must provide a
datetime.datetime object as the first argument when creating a new
NewOrderEvent object
>>> e = NewOrderEvent(datetime.datetime(2020, 9, 8))
>>> e.timestamp
datetime.datetime(2020, 9, 8, 0, 0)

Subclass-Specific Attributes
- We often make the subclass-specific attributes private, to avoid changing the public
interface declared by the abstract superclass
- We do not need to repeat the documentation for the timestamp attribute
class NewOrderEvent(Event):
“””An event representing when a customer places an order at a
restaurant.”””
# Private Instance Attributes:
# - _order: the new order to be added to the FoodDeliverySystem
_order: Order

def __init__(self, order: Order) -> None:

 Event.__init__(self, order.start_time)
self._order = order
o Now, whenever we call NewOrderEvent.__init__, Python also calls
Event.__init__
This classes all shared instance attributes from Event to be “inherited” by
the NewOrderEvent subclass
- When inheriting from a class that defines its own initializer:
o 1. The initializer of a subclass must call the initializer of its super calss to initialize
all common attributes
o 2. The initializer of a subclass is responsible for initializing any additional
attributes that are specific to that subclass

Implementing NewOrderEvent.handle_event
class NewOrderEvent(Event):
“””…”””
…
def handle_event(self, system: FoodDeliverySystem) -> None:
“Mutate system by placing an order.”””
system.place_order(self._order)

Implementing Other Event Subclass

- Similar to NewOrderEvent, but the initializer takes an explicit datetime.datetime
argument to represent when the given order is completed
class CompleteOrderEvent(Event):
“””When an order is delivered to a customer by a courier.”””
# Private Instance Attributes:
# - _order: the order to be completed by this event
_order: Order

def __init__(self, timestamp: datetime.datetime, order: Order) -> None:

Event.__init__(self, timestamp)
self._order = order

def handle_event(self, system: FoodDeliverySystem) -> None:

“””Mutate the system by recording that the order has been
delivered to the customer.”””
system.complete_order(self._order, self.timestamp)
Event Generation
- Processing one event can cause other events to occur
o i.e. when we process a NewOrderEvent, we expect that at some point in the
future, a corresponding CompleteOrderEvent will occur
- We can change the return type of handle_event from None to List[Event], where the
return value is a list of the events caused by the current event
o NewOrderEvent would return a list containing a CompleteOrderEvent
If there are no available couriers, we would not want it to return a
CompleteOrderEvent
class NewOrderEvent(Event):
…
def handle_event(self, system: FoodDeliverySystem) -> List[Event]:
“””…”””
success = system.place_order(self._order)
if success:
completion_time = self.timestamp +
datetime.timedelta(minutes=10)
return [CompleteOrderEvent(completion_time,
self._order)]
else:
self._order.start_time = self.timestamp +
datetime.timedelta(minutes=5)
return [NewOrderEvent(self._order)]

Returning No Events
- Our CompleteOrderEvent does not cause any new events to happen
o Returns an empty list

A New Event Type

- This new event type will represent a random generation of new orders over a given time
period, which we’ll use to drive our simulation
class GenerateOrdersEvent(Event):
“””An event that causes a random generation of new orders.

Private Representation Invariants:

- self.._duration > 0
“””
# Private Instance Attributes:
 # - _duration: the number of hours to generate orders for
_duration: int

def __init__(self, timestamp: datetime.datetime, duration: int) -> None:

“””Initialize this event with timestamp and the duration in hours.

Preconditions:
- duration > 0
“””

def handle_event(self, system: FoodDeliverySystem) -> List[Event]:

“””Generate new orders for this event’s timestamp and
duration.”””
events = []
while…:
# Create a randomly-generated NewOrderEvent
new_order_event = …
events.append(new_order_event)

return events

10.5 Creating a Discrete-Event Simulation

The Main Simulation Loop

- Discrete-event simulation – simulation driven by individual events occurring at specified
periods of time
- A discrete-event simulation runs as follows:
o 1. It keeps track of a collection of events, which begins with some initial events.
The collection is a priority queue, where an event’s priority is its timestamp
(earlier timestamps mean higher priority)
o 2. The highest-priority event (i.e. the one with the earliest timestamp) is
removed and processed. Any new events it generates are added to the priority
queue
o Step 2 repeats until there are no events left
- Assuming we have a priority queue implementation called EventQueueList, here is how
we could write a simple function that runs this simulation loop:
def run_simulation(initial_events: List[Event], system: FoodDeliverySystem) ->
None:
 # Initialize an empty priority queue of events
events = EventQueueList()
for event in initial_events:
events.enqueue(event)

# Repeatedly remove and process the next event

while not events.is_empty():
event = events.dequeue()

new_events = event.handle_event(system)
for new_event in new_events:
events.enqueue(new_event)
- Our run_simulation function is polymorphic
o It works regardless of what Event instances it’s given in its initial_events
parameter, or what new events are generated and stored in new_events
o Our function needs to be able to call the handle_event method on each event
object

A Simulation Class
class FoodDeliverySimulation
“””A simulation of the food delivery system.”””
# Private Instance Attributes:
# - _system: The FoodDeliverySystem instance that this simulation uses
# - _events: A collection of the events to process during the simulation
_system: FoodDeliverySystem
_events: EventQueue

def __init__(self, start_time: datetime.datetime, num_days: int,

num_couriers: int, num_customers: int, num_restaurants: int) -> None:
“””Initialize a new simulation with the given simulation parameters.

start_time: the starting time of the simulation

num_days: the number of days that the simulation runs
num_couriers: the number of couriers in the system
num_customers: the number of customers in the system
num_restaurants: the number of restaurants in the system
“””
self._events = EventQueueList()
 self._system = FoodDeliverySystem()

self._populate_initial_events(start_time, num_days)
self._generate_system(num_couriers, num_customers, num_restaurants)

def _populate_initial_events(self, start_time: datetime.datetime, num_days: int)

-> None:
“””Populate this simulation’s Event priority queue with
GenerateOrdersEvents
One new GenerateOrderEvent is generated per day, starting with
start_time and repeating num_days times.
“””

def _generate_system(self, num_couriers: int, num_customers: int,

num_restaurants: int) -> None:
“””Populate this simulation’s FoodDeliverySystem with the specified
number of entities.
“””

def run(self) -> None:

“””run this simulation.
“””
while not self._events.is_empty():
event = self._events.dequeue()

new_events = event.handle_event(self._system)
for new_event in new_events:
self._events.enqueue(new_event)
- Key items to note in this (incomplete) implementation:
o The run_simulation method has been renamed to simply run, since it’s a method
in the FoodDeliverySimulation class
o The local variable events and parameter system from the function are now
instance attributes for the FoodDeliverySimulation class, and have been moved
out of the run method entirely. It’s the job of the
FoodDeliverySimulation.__init__ to initialize these objects
o The initializer takes in several parameters representing configuration values for
the simulation. It then uses these values in two helper methods to initialize the
_system and _events objects. These methods are marked private (named with a
 leading underscore) because they’re only meant to be called by the initializer,
and not code outside of the class
- To use the FoodDeliverySimulation class:
>>> simulation = foodDeliverySimulation(datetime.datetime(2020, 11, 30), 7, 4,
100, 50)
>>> simulation.run()

Populating Initial Events

- The key idea for our first helper method is that given a start time and a number of days,
our initial events will be a series of GenerateOrderEvents that will generate
NewOrderEvents when they are processed
def _populate_initial_events(self, start_time: datetime.datetime, num_days: int)
-> None:
“””…”””
for day in range(0, num_days):
# 1. Create a GenerateOrderEvent for the given day after the start
# time
# 2. Enqueue the new event

Populating the System Entities

- Currently, only orders are dynamic in our system; the restaurants, customers, and
couriers do not change over time
- The easiest way to populate these 3 entity times is to randomly generate new instances
of each of these classes:
def _generate_system(self, num_couriers: int, num_customers: int,
num_restaurants: int) -> None:
“””…”””
for i in rtange(0, num_customers):
location = _generate_location()
customer = Customer(f’Customer {i}’, location)
self._system.add_customer(customer)

# Couriers and Restaurants are similar

…
# Outside the class: helper for generating random locations in Toronto
TORONTO_COORDS = (43.747743, 43.691170, -79.644951, -79.176646)

def _generate_location() -> Tuple[float, float]:

“””Return a randomly-generated location (latitude, longitude) within the
Toronto bounds.
“””
return (random.uniform(TORONTO_COORDS[0], TORONTO_COORDS[1]),
random.uniform(TORONTO_COORDS[2], TORONTO_COORDS[3]))