m y

d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
 Course philosophy
m y
the Python language
d e
c a
à canonical CPython 3.6+ implementation

A
the standard library

becoming an expert Python developer

y te
idiomatic Python
th B
M a
obtaining a deeper understanding of the Python language

t ©
i g h and the standard library

yr
this is NOT an introductory course

o p
à refer to prerequisites video or course description

C
this is NOT cookbook course
 Course Topics
m y
d e
c a
focused on Object Oriented Programming (OOP) concepts in Python

à classes (types) and objects

te A
B y
t
à functions, methods and properties

a h
M
à polymorphism and special dunder methods

©
h t
à single inheritance

yr i g
à descriptors

o p
C à enumerations

à exceptions
Included Course Materials
m y
d e
a
lecture videos

coding videos
Ac
y te
Jupyter notebooks

th B
PDFs of all lecture slides

M a
©
projects, exercises and solutions

t
i g h
github repository for all code

p yr https://github.com/fbaptiste/python-deepdive
C o
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Python 3: Deep Dive (Part 4) - Prerequisites

m y
à assumes you are not a beginner programmer
d e
c a
à solid understanding of basic Computer Science concepts

te A
y
à must have practical experience programming in Python already

th B
à real projects, not just a bootcamp course and some isolated exercises

M a
t
à this is not a cookbook course! ©
i g h
yr
à does not explain how to solve specific problems

o p
à does explain Python fundamentals

C à how various concepts work and can be used

Python 3: Deep Dive (Part 4) - Prerequisites

m y
This course assumes that you have in-depth knowledge of functional programming in Python:

d e
a
scopes and namespaces

Ac
te
Boolean truth values id == vs is

B y
functions and function arguments lambdas

a th
packing and unpacking iterables
© M my_func(*my_list) f, *_, l = (1, 2, 3, 4, 5)

h t
closures

yr i g
nested scopes free variables

o p
C
decorators

zip sorted any all à itertools module

Python 3: Deep Dive (Part 4) - Prerequisites

This course assumes that you have in-depth knowledge of:

m y
d e
sequences iterables iterators à yield __iter__
c a
__next__ __getitem__

te A
y
comprehensions à list, dictionary, set, generator exp à relation to functions and closures

th B
a
generators

context managers
© M
h t
mapping types

yr i g à dictionaries sets collections module

o p
object equality and hashing

C
importing modules and symbols
Python 3: Deep Dive (Part 4) - Prerequisites

You should have some basic exposure to creating and using classes in Python
m y
d e
class Person:
def __init__(self, name, age):
c a
self.name = name
self._age = age
te A
B y
@property
def age(self):
a th
M
return self._age

def __eq__(self, other):

t ©
g h
return isinstance(other, Person) and self.name == other.name

i
yr
def __hash__(self):

p
o
return hash(self.name)

C
def __lt__(self, other):
…
Python 3: Deep Dive (Part 4) - Prerequisites

m y
à know how to work with Python virtual environments
d e
c a
à pip install

te A
Most code examples are provided using Jupyter Notebooks
B y
Freely available https://jupyter.org/
a th
© M
GitHub and git

h t
yr i g
https://github.com/fbaptiste/python-deepdive

o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
à what are objects in general

m y
à what are classes
d e
c a
à instantiating and initializing objects from a class

te A
à class and instance attributes
B y
à data
a th
à functions

© M
h t
g
à function bindings and methods

yr i
à instance methods

o p
à class methods

C à static methods

à properties
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
What is an object?

a container

m y
d e
a
contains data à state à attributes

contains functionality à behavior à methods

Ac
y te
my_car
th B
dot notation

brand = Ferrari
M amy_car.brand à Ferrari

model = 599XX
t ©
state my_car.purchase_price = 1_600_000
year = 2010
i g h
accelerate
p yr my_car.accelerate(10)

C
brake

steer
o behavior my_car.steer(-15)
Creating objects

How do we create the "container"?

m y
How do we define and set state?
d e
c a
A
How do we define and implement behavior?

y te
Many languages use a class-based approach

th B
à C++, Java, Python, etc

M a
A class is like a template used to create objects

t ©
h
à also called a type

yr i g
à objects created from that class are called instances

o p
of that class or type

C
Classes

Classes are themselves objects

m y
They have attributes (state)
d e
c a
à e.g. class name (or type name)

te A
They have behavior
B y
a th
à e.g. how to create an instance of the class

© M
t
if a class is an object, and objects are created from classes, how are classes created?

i g h
yr
Python à classes are created from the type metaclass

o p
C
Instances
Classes have behavior à they are callable MyClass()

m y
à this returns an instance of the class

d e
a
à often called objects, differentiating from class

c
A
(even though a class is technically an object as well)

te
Instances are created from classes
B y
their type is the class they were created from
a th
if MyClass is a class in Python
© M
and my_obj is an instance of that class my_obj = MyClass()

h t
yr i
type(my_obj) à MyClass
g this is an object
(classes are objects)

o p
C
isinstance(my_obj, MyClass) à True
Creating Classes

use the class keyword

m y
class MyClass:

d e
pass

c a
à Python creates an object

te A
à called MyClass
B y
à of type type
a th
M
à automatically provides us certain attributes (state) and methods (behavior)

©
t
string
MyClass.__name__

i g h
à 'MyClass' (state)

p yr
MyClass() à returns an instance of MyClass (behavior)

C o
type(MyClass) à type

isinstance(MyClass, type) à True

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Defining Attributes in Classes

class MyClass:

m y
e
language = 'Python'
version = '3.6'

c a d
te A
MyClass is a class à it is an object (of type type)

B y
a th
in addition to whatever attributes Python automatically creates for us

©
e.g. __name__ with a state of 'MyClass'M
h t
i g
à it also has language and version attributes

yr
p
with a state of 'Python' and '3.6' respectively

o
C
Retrieving Attribute Values from Objects

class MyClass:
m y
language = 'Python'

d e
a
version = '3.6'

Ac
à getattr function
y te
getattr(object_symbol, attribute_name, optional_default)

th B
a
getattr(MyClass, 'language') à 'Python'

getattr(MyClass, 'x')

© M à AttributeError exception

h
getattr(MyClass, 'x', 'N/A')
t à 'N/A'

yr i g
p
à dot notation (shorthand)

o
C
MyClass.language

MyClass.x
à 'Python'

à AttributeError exception
Setting Attribute Values in Objects

class MyClass:
language = 'Python'
m y
version = '3.6' object symbol

d e
a
string

à setattr function
Ac
setattr(object_symbol, attribute_name, attribute_value)

y te
B
setattr(MyClass, 'version', '3.7')
MyClass.version = '3.7'
a th
© M
this has modified the state of MyClass à MyClass was mutated

h t
yr i g
getattr(MyClass, 'version')
à '3.7'

p
MyClass.version

C o
Setting Attribute Values in Objects

class MyClass:
language = 'Python'
m y
version = '3.6'

d e
c a
A
What happens if we call setattr for an attribute we did not define in our class?

te
B y
Python is a dynamic language à can modify our classes at runtime (usually)

a th
M
setattr(MyClass, 'x', 100) or MyClass.x = 100

t ©
à MyClass now has a new attribute named x with a state of 100

i g h
p yr
getattr(MyClass, 'x')

o
à 100

C
MyClass.x
Where is the state stored?

à in a dictionary

m y
class MyClass:
d e
language = 'Python'
c a
version = '3.6'

te A
MyClass.__dict__
B y class namespace

a th
©
'language': 'Python', M
mappingproxy({'__module__': '__main__',

t
'version': '3.6',

h
g
'__dict__': <attribute '__dict__' of 'MyClass' objects>,

yr i
'__weakref__': <attribute '__weakref__' of 'MyClass' objects>,

p
'__doc__': None})

C o
à not directly mutable dictionary (but setattr can)
à ensures keys are strings (helps speed things up for Python)
Mutating Attributes

y
We saw we can modify the sate or create a brand new attribute using setattr or the dot notation

m
class MyClass:
d e
MyClass.__dict__ à mappingproxy({'language': 'Python',
language = 'Python'

c a 'version': '3.6', …})

A
version = '3.6'

We can then mutate MyClass:

y te
th B
setattr(MyClass, 'x', 100) or

M a
MyClass.x = 100

t ©
And this is reflected in the namespace:

i g h
yr
MyClass.__dict__ à mappingproxy({'language': 'Python',

p
o
'version': '3.6',

C
'x': 100, …})
Deleting Attributes

m y
So if we can mutate the namespace at runtime by using setattr (or equivalent dot notation)

Can we remove an attribute at runtime?

d e
c a
Yes! (usually) à delattr(obj_symbol, attribute_name)

te A or del keyword

B y
h
class MyClass:

t
MyClass.__dict__ à mappingproxy({'language': 'Python',

a
language = 'Python' 'version': '3.6', …})
version = '3.6'

© M
delattr(MyClass, 'version')

h t or del MyClass.version

yr i g
p
MyClass.__dict__ à mappingproxy({'language': 'Python',

C o …})

à version has been removed from namespace

Accessing the Namespace Directly

As we saw the class namespace uses a dictionary, which we can request using the __dict__
m y
attribute of the class

d e
The __dict__ attribute of a class returns a mappingproxy object
c a
te A
Although this is not a dict, it is still a hash map (dictionary), so we can at least read access the
class namespace directly – not common practice!!

B y
class MyClass: th
MyClass.language

a
language = 'Python'
version = '3.6'
© M getattr(MyClass, 'language') à 'Python'

h t MyClass.__dict__['language']

yr i g
o p
Be careful with this – sometimes classes have attributes that don't show up in that dictionary!

C
(we'll come back to that later)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Setting an Attribute Value to a Callable

Attribute values can be any object à other classes à any callable

y
à anything…

m
d e
a
So we can do this: class MyClass:
language = 'Python'

Ac
def say_hello():

y te
B
print('Hello World!')

say_hello is also an attribute of the class

a th
à its value happens to be a callable

MyClass.__dict__ à
© M
h t
g
mappingproxy({'language': 'Python',

yr i
'say_hello': <function __main__.MyClass.say_hello()>, ...})

o p
C
How do we call it?

We could get it straight from the namespace dictionary:

m y
my_func = MyClass.__dict__['say_hello']
d e
my_func()
c a
A
à 'Hello World!'

y te
B
MyClass.__dict__['say_hello']() à 'Hello World!'

or we could use getattr:

a th
© M
t
getattr(MyClass, 'say_hello')() à 'Hello World!'

i g h
yr
or we can use dot notation:

p
C o
MyClass.say_hello() à 'Hello World!'
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Classes are Callable

When we create a class using the class keyword

m y
Python automatically adds behaviors to the class
d e
in particular:
c a
te A
y
à it adds something to make the class callable

h
à the return value of that callable is an object

t B we say the object is an instance of

a
à the type of that object is the class object

M
the class

©
also called class instantiation

my_obj = MyClass()
h t or instantiating the class

yr i g
p
type(my_obj) à MyClass

o
C
isinstance(my_obj, MyClass) à True
Instantiating Classes

When we call a class, a class instance object is created

m y
This class instance object has its own namespace
d e
c a
à distinct from the namespace of the class that was used to create the object

te A
B y
This object has some attributes Python automatically implements for us:

a th
M
à __dict__ is the object's local namespace

t ©
à __class__ tells us which class was used to instantiate the object

i g h
yr
à prefer using type(obj) instead of obj.__class__

o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Data Attributes, Classes and Instances

Let's focus on data attributes first (i.e. not functions) class MyClass:

m y
e
language = 'Python'

a
my_obj = MyClass()

c d
MyClass.__dict__ à {'language': 'Python'}

te A
my_obj.__dict__ à {}

B y
MyClass.language
a th
à Python starts looking for language attribute in MyClass namespace

M
à MyClass.language à 'Python'

©
my_obj.language

h t
à Python starts looking in my_obj namespace

yr i g à if it finds it, returns it

o p à if not, it looks in the type (class) of my_obj, i.e. MyClass

C à 'Python'
 class MyClass:
Data Attributes, Classes and Instances language = 'Python'

MyClass.language à 'Python'
class attribute
m y
my_obj = MyClass()
my_obj.__dict__ à {}
d e
c a
instance attribute
my_obj.language = 'java'
my_obj.__dict__ à {'language': 'java'}
te A
B y
my_obj.language à 'java'

a th
M
MyClass.language à 'Python'

other_obj = MyClass()
t ©
i
other_obj.__dict__ = {}
g h
yr
other_obj.language à 'Python'

p
C o
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
 class MyClass:
Function Attributes, Classes and Instances def say_hello():

y
print('Hello World!')
What happens when attributes are functions is different!

e m
my_obj = MyClass()

MyClass.say_hello <function __main__.MyClass.say_hello()>

c a d
my_obj.say_hello
te A
<bound method MyClass.say_hello of <__main__.MyClass
object at 0x10383f860>>

B y
Same with getattr:

a th
M
getattr(my_obj, 'say_hello') <bound method MyClass.say_hello of <__main__.MyClass
object at 0x10383f860>>

t ©
g h
MyClass.say_hello() à 'Hello World!'

i
p yr
my_obj.say_hello() à TypeError

C o say_hello() takes 0 positional

arguments but 1 was given

bound? method?
Methods

y
method is an actual object type in Python

like a function, it is callable

e m
but unlike a function if is bound to some object

c a d
and that object is passed to the method as its first parameter

te A
B y
h
my_obj.say_hello() à say_hello is a method object

a t
à it is bound to my_obj

M
à when my_obj.say_hello is called, the bound object my_obj is

©
t
injected as the first parameter to the method say_hello

i g h
yr
so it's essentially calling this: MyClass.say_hello(my_obj)

p
à but there's more to it than just calling the function this way – method object

o
C
One advantage of this is that say_hello now has a handle to the object's
namespace! à the object it is bound to
Methods

Methods are objects that combine:

m y
à instance (of some class)

d e
à function

c a
à like any object it has attributes

te A
__self__ à the instance the method is bound to

B y
__func__
a th
à the original function (defined in the class)

calling obj.method(args)
© M
à method.__func__(method.__self__, args)

h t
class Person:

yr
def hello(self): i g p = Person()

pass
o p
C p.hello.__func__
p.hello.__self__
Instance Methods

This means we have to account for that "extra" argument when we define functions in our
classes – otherwise we cannot use them as methods bound to our instances
m y
d e
a
These functions are usually called instance methods

Ac
first param will receive instance object
class MyClass:
y te
B
we often call this an instance method
def say_hello(obj):
print('Hello World!')

a th but not actually a method object yet!

© M at this point it's just a regular function

my_obj = MyClass()

h t now it's a method

yr
my_obj.say_hello
i g and is bound to my_obj, an instance of MyClass

o p
C
instance method

my_obj.say_hello() à 'Hello World!'

à MyClass.say_hello(my_obj)
Instance Methods

Of course functions in our classes can have their own parameters

m y
e
When we call the corresponding instance method with arguments à passed to the method as well

d
c a
And the method still receives the instance object reference as the first argument

class MyClass:
te A
à we have access to the instance (and class) attributes!
language = 'Python'

B y
def say_hello(obj, name):

a th
return f'Hello {name}! I am {obj.language}. '

© M
python = MyClass()
h t
yr i g
python.say_hello('John') à MyClass.say_hello(python, 'John')

o p à 'Hello John! I am Python'

C
java = MyClass()
java.language = 'Java'
java.say_hello('John') à MyClass.say_hello(java, 'John')
à 'Hello John! I am Java'
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Initializing Class Instances

When we instantiate a class, by default Python does two separate things:

m y
à creates a new instance of the class
d e
c a
A
à initializes the namespace of the class

class MyClass:
y te
language = 'Python'

th
obj = MyClass()
B obj.__dict__ à {}

M a
We can provide a custom initializer method that Python will use instead of its own:

t © notice that __init__ is defined to

h
class MyClass:

i
language = 'Python'

yr g work as a bound instance method

p
def __init__(obj, version):

C o
obj.version = version
Deconstructing this…
self (convention)
class MyClass:
language = 'Python'
m y
language is a class attribute à in class namespace

d e
__init__ is a class attribute à in class namespace
def __init__(obj, version):

c a (as a function)

A
obj.version = version

when we call MyClass('3.7')

y te
th B
à Python creates a new instance of the object with an empty namespace

M a
à if we have defined an __init__ function in the class

t ©
à it calls obj.__init__('3.7') à bound method à MyClass.__init__(obj, '3.7')

i g h
yr
à our function runs and adds version to obj's namespace

o p
à version is an instance attribute

C obj.__dict__ à {'version': '3.7'}

a standard convention is to use an argument named self

Important!

By the time __init__ is called

m y
d e
a
Python has already created the object and a namespace for it (like a __dict__ in most cases)

c
te A
then __init__ is called as a method bound to the newly created instance

B y
a th
We can actually also specify a custom function to create the object

__new__
© M
h t
yr i g
we'll come back to this later

o p
C
But __init__ is not creating the object, it is only running some code after the
instance has been created
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Instances and Setting Function Attributes at Runtime

y
We saw that we can add to an instance namespace directly at runtime by using setattr or the dot

m
e
notation

class MyClass:
c a d
A
language = 'Python'

obj = MyClass() obj.__dict__ à {}

y te
obj.version = '3.7'
th B
obj.__dict__ à {'version': '3.7'}

M a
What happens if we create a new attribute whose value is a function?

t ©
obj.say_hello = lambda: 'Hello World!'

i g h
yr
then

o p
obj.say_hello à function not a bound method!

C
obj.say_hello() à 'Hello World!'

but say_hello does not have access to the instance namespace

Can we create and bind a method to an instance at runtime?

Yes

m y
à just need to define a method that binds the function to the instance
d e
c a
A
class MyClass:

te
language = 'Python'

y
function we want to bind
obj = MyClass()

th B
from types import MethodType

M a
MethodType(function, object)
the object to bind to

t ©
i g h
obj.say_hello = MethodType(lambda self: f'Hello {self.language}!', obj)

p yr
say_hello is now a method bound to obj à 'Hello Python!'

C o
à only obj has been affected

à no other instances have that method

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Properties

We saw that we can define "bare" attributes in classes and instances

m y
e
class MyClass: obj = MyClass('Python')
def __init__(self, language):
self.language = language
a
print(obj.language)

c ddirect access to
language

te A
obj.language = 'Java'
attribute

y
In many languages direct access to attributes is highly discouraged

B
th
Instead the convention is to make the attribute private, and create public getter and setter methods

a
M
Although we don't have private attributes in Python, we could write it this way:

©
class MyClass:

h t What code would you rather write?

g
def __init__(self, language):

yr i
self._language = language print(obj.language)

p
obj.language = 'Java'

o
def get_language(self):

C
return self._language or
print(obj.get_language())
def set_language(self, value): obj.set_language('Java')
self._language = value
 class MyClass:
Properties
def __init__(self, language):
In this case, language is considered an instance property
self._language = language

m y
But is only accessible via the get_language and
d e
def get_language(self):

a
return self._language

c
set_language methods

te A
def set_language(self, value):
self._language = value

B y
h
There are some good reasons why we might want to approach attributes using this programming style

a t
M
à provides control on how an attribute's value is set and returned

©
h t
If you start with a class that provides direct access to the language attribute, and later need to

yr i g
change it to use accessor methods, you will change the interface of the class

o p
any code that uses the attribute directly: obj.language = 'Java'

C
will need to be refactored to use
the accessor methods instead:
obj.set_language('Java')
Python has a Solution!

We can use the property class to define properties in a class:

m y
d e
class MyClass:

c a
m = MyClass('Python')

A
def __init__(self, language): m.language = 'Java'

te
self.language = language print(m.language)

B y
class MyClass:
def __init__(self, language):
a th
self._language = language

© M m = MyClass('Python')

t
m.language = 'Java'

h
def get_language(self): print(m.language)

i g
return self._language

yr
o p
def set_language(self, value):
self._language = value

C
language = property(fget=get_language, fset=set_language)

à changed an attribute to a property without changing the class interface!

The property Class

property is a class (type)

m y
d e
à constructor has a few parameters:

c a
fget

te A
specifies the function to use to get instance property value

fset
B y
specifies the function to use to set the instance property value

fdel
a th
specifies the function to call when deleting the instance property

doc

© M
a string representing the docstring for the property

h t
yr i g
In general we start with plain attributes, and if later we need to change to a

p
property we can easily do so using the property class without changing the
interface

C o
class MyClass:
def __init__(self, language):
self._language = language

def get_language(self):

m y
e
return self._language

def set_language(self, value):

self._language = value

c a d
language = property(fget=get_language, fset=set_language)

te A
m = MyClass('Python')
B y
m.__dict__ à {'_language': 'Python'}

m.language = 'Java'
a th
m.__dict__ à {'_language': 'Java'}

© M
'language' is not in m.__dict__

h t
i g
Remember how Python looks for attributes:

yr
p
à searches instance namespace first

C o
à but also looks in class namespace
à finds language which is a property object that has get and set accessors

à uses the accessor methods (how? à later…)

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
The property Class

The property class can be instantiated in different ways:

m y
x = property(fget=get_x, fset=set_x)
d e
c a
te A
The class defines methods (getter, setter, deleter) that can take a callable as an argument

y
and returns the instance with the appropriate method now set

Could create it this way instead:

th
x = property() B
M a
x = x.getter(get_x)

t ©x = x.setter(set_x)
or

i g h
yr
x = property(get_x)

p
o
x = x.setter(set_x)

C
 we now have a property language

y
def MyClass:
with a getter method defined

m
def __init__(self, language):
self._language = language

d e
remind you of decorators?
def language(self):
c a
return self._language

te A
language = property(language)

B y
Instead we can write:
a th
def MyClass:
© M
t
def __init__(self, language):

h
g
self._language = language

yr i
p
@property

C o
def language(self):
return self._language

Next, we may want to define a setter method as well

def MyClass:
def __init__(self, language):
self._language = language

m y
@property
d e
def language(self):
c a
return self._language

te A
at this point language is now a property instance

B y
h
def set_language(self, value): this is a setter method
self._language = value
a t which we need to assign to the

© M language property

h t
g
language = language.setter(set_language)

yr i
p
But again, we can rewrite this using the @ decorator syntax:

C o
@language.setter
def language(self, value):
self._language = value
If you find this a bit confusing, think of doing it this way first:

def MyClass:
m y
def __init__(self, language):

d e
a
self._language = language

@property
Ac
store a reference to the language object

te
def language(self): (of type property)
return self._language

B y
lang_prop = language
a th redefine the symbol language as a method

def language(self, value):

© M assign the setter method to the property object
self._language = value

h t (setter returns the property object itself)

yr i g
language = lang_prop.setter(language)

o p make the language symbol refer

C to the property object again

del lang_prop delete lang_prop symbol

from namespace
 To summarize, we can use decorators to create property type objects as well:

m y
e
def MyClass:

d
def __init__(self, language): function name defines the property instance name (symbol)
self._language = language
c a
@property

te A
def language(self):
return self._language
B y
language is now a property instance (an object)

a th
M
we use the setter method of the language property object
@language.setter
def language(self, value)

t ©
h
self._language = value

yr i g
p
important to use the same name,

C o
otherwise we end up with a new
symbol for our property!
(we'll see this in the code video)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
"Read-Only" Properties

y
To create a read-only property, we just need to create a property with only the get accessor defined

e
à not truly read-only since underlying storage variable could be accessed directly
m
à useful for computed properties
c a d
te A
y
class Circle: c = Circle(1)

B
def __init__(self, r):

h
c.area() à 3.14…

t
self.r = r

def area(self):
M a
©
return math.pi * self.r * self.r

h t
class Circle:

yr i g
def __init__(self, r):

p
c = Circle(1)

o
self.r = r
c.area à 3.14…

C
@property
def area(self): feels more natural
since area is really a
return math.pi * self.r * self.r
property of a circle
Application: Caching Computed Properties

y
Using property setters is sometimes useful for controlling how other computed properties are cached

m
à Circle

d e
à area is a computed property
c a
à lazy computation – only calculate area if requested

te A
y
à cache value – so if re-requested we save the computation

B
a th
à but what if someone changes the radius?

M
à need to invalidate the cache

©
h t
yr i g
à control setting the radius using a property

o p
C
à we are now aware when the property has been changed
Application: Caching Computed Properties
setting _area cache to None
class Circle:
def __init__(self, r):
m y
self._r = r

d e
a
self._area = None

Ac
te
@property

y
def radius(self):

B
return self._r

@radius.setter
a th
M
def radius(self, r):
if r < 0:

t ©
raise ValueError('Radius must be non-negative')
self._r = r

i g h
yr
self._area = None invalidate cache

o
@property
p
C
def area(self): calculate and cache area if not
if self._area is None: already cached
self._area = math.pi * (self.radius ** 2)
return self._area
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Deleting Properties

Just like we can delete attributes from an object:

m y
c = Circle()
d e
c.color = 'yellow' c.color à yellow
c a
del c.color
c.color à AttributeError
te A
or delattr(c, 'color')
B y
a th
We can also support deleting properties from an instance object:

© M
deleter argument of the property initializer @prop_name.deleter

h t
yr i g
à generally used to perform some cleanup activity upon deletion of the property

p
à not used very often

o
C
Important: calling the deleter runs code contained in the deleter method
à does not remove property from class itself
à it just calls the deleter method
Deleting Properties
class Circle:
def __init__(self, color):
self._color = color
m y
d e
def get_color(self):
return self._color
c a
te A
y
def set_color(self, value): when this method is invoked, it will remove _color

B
self._color = value from the instance namespace (dictionary)

def del_color(self):
a th
del self._color

© M
t
color = property(get_color, set_color, del_color)

h
c = Color('yellow')
yr i gc.__dict__ à {'_color': 'yellow'}

o p
c.color à 'yellow'

c.color
C
del c.color c.__dict__ à {}

à AttributeError
c._color
Deleting Properties

We can also use the decorator syntax:

c = UnitCircle('red')

m y
class UnitCircle:
def __init__(self, color):
d e
c.__dict__ à {'_color': 'red'}
self._color = color
c a
A
del c.color

te
@property c.__dict__ à {}
def color(self):

B y
h
return self._color c.color à AttributeError

@color.setter
a t
M
à because getter tries to read self._color
def color(self, value):
self._color = value

t ©
@color.deleter
i g h But the property still exists à defined on class

yr
def color(self):

o p
del self._color c.color = 'blue'

C c.__dict__ à {'_color': 'blue'}

c.color à 'blue'
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
The property Class

Can we define properties without using the property class? Yes!

m y
d e
The property class is a convenience class, but we don't have to use it
c a
But it is really convenient and works just fine most of the time!
te A
B y
Where might it fall short?

a th
Reusability
© M
If we want to have the same property (with same functionality) in

h t
many different classes, having to redefine the accessor methods
in each of the classes can become tedious

yr i g
More control

o p may want more control over a property's behaviors

C
We'll come back to this when we study data descriptors
Another Question

y
We saw how to use the property class (and corresponding decorators) to define instance properties

m
d e
Can we create class properties (that are bound to the class)
c a
à like a class attribute, but using accessor methods
te A
B y
à Yes!
a th
© M
t
à metaclasses

i g h
p yr
C o
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Class Methods

When we define a function in a class à how we call it will alter behavior

m y
class MyClass:
d e
MyClass.hello à just a plain function defined in MyClass
def hello(): MyClass.hello() à Hello
c a
A
return 'Hello'

m = MyClass()

y te
B
m.hello à method bound to object m à instance method

th
m.hello() à TypeError (missing argument)

a
© M
t
Can we create a function in a class that will always be bound to the class, and never the instance?

h
yr i g
MyClass.fn à method bound to MyClass

o p
m.fn à method bound to MyClass

C à @classmethod
Class Methods class MyClass:
def hello():
print('hello…')

m y
def inst_hello(self):
d e
print(f'hello from {self}')

c a
@classmethod

te A by default, any

y
def cls_hello(cls):

B
print(f'hello from {cls}') function defined in a

a th class will be handled

as a bound method

M
when called from an
Instance

©
MyClass instance

h t
g
hello regular function method bound to instance à call will fail!

yr i
inst_hello
o p regular function method bound to instance

C
cls_hello method bound to class method bound to class
Static Methods

y
So can we define a function in a class that will never be bound to any object when called?

m
à Yes!
d e
c a
à in Python, those are called static methods à @staticmethod

te A
class Circle:

B y
h
@staticmethod
def help():
a t
M
return 'help available'

t ©
i g
type(Circle.help) à function
h Circle.help() à help available

p yr
c = Circle()

C o
type(c.help) à function c.help() à help available
Recap

m y
e
function bound to instance when called from

d
class MyClass: instance - will receive instance as first
parameter
c a
A
def inst_hello(self):

te
print(f'hello from {self}')
function bound to class when called from
@classmethod
B y
either the class or the instance - will receive the
def cls_hello(cls):
print(f'hello from {cls}')
a th
class (MyClass) as first parameter

M
static method is never bound to anything –
receives no extra argument no matter how it is

©
@staticmethod
def static_hello():

h t called

g
print('static hello')

yr i
o p
C
Why use static methods?

à cases where it makes sense for a function to live in a class

m y
à but does not need access to either the instance or the class state

d e
c a
Timer

te A
B y
h
start(self) à instance method

a t
end(self)

© M
à instance method

h t
g
timezone à class attribute à allows us to modify time zone for all instances

yr i
p
current_time_utc()

o
à static method

C
current_time(cls) à class method (needs class time zone)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Not all types are in the builtin module…

Certain types that we use commonly in Python int str list tuple
m…y
d e
a
list is a type

c
are part of the builtins

l = [1, 2, 3]
type(l) is list
à True
te A
isinstance(l, list)
B y
a th
But some types are not available directly in the builtins

© M
t
functions modules generators …

i g h
yr
def my_func():
this is a string

p
pass

C o
print(type(my_func)) à 'function'

but type(my_func) actually returns a type (think class)

The types Module

The function type can actually be found in the types module

m y
d e
a
import types

def my_func():
Ac
te
pass

B y
h
type(my_func) is types.FunctionType

a t à True

M
isinstance(my_func, types.FunctionType)

t ©
g h
Nothing fancy, just that some standard types are not

i
yr
available directly in the builtins

o p
C
types.ModuleType types.FunctionType

types.GeneratorType …
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Nested scopes in Class definitions à module has its own (global) scope
contains Python, p

m y
e
# module1.py

d
à class body has its own scope
class Python:
c a
contains kingdom, phylum , family , __init__

kingdom = 'animalia'
, say_hello

te A
phylum = 'chordata'

B y
à what about the scope of functions
defined in the body of a class?

h
family = 'pythonidae'

def __init__(self, species):

a t
M
à turns out they are NOT nested inside the
self.species = species class body scope

t ©
h
def say_hello(self):

g
à symbols __init__, say_hello are in the class

i
return 'ssss…'

yr
body scope

o p
p = Python('monty')
à but functions themselves are nested in the

C class's containing scope

(module1 in this example)
Think of it this way

m y
# module1.py

d e
class Python:
c a
kingdom = 'animalia'
te A
phylum = 'chordata'
y
def callable_1(self, species):

B
h
family = 'pythonidae' self.species = species

a t
M
__init__ = callable_1

say_hello = callable_2

t © def callable_2(self):

p = Python('monty')
i g h return 'ssss…'

p yr
C o
à when Python looks for a symbol in a function, it will therefore not use the class body scope!
In practical terms…

y
class Account: this works because APR and

m
COMP_FREQ = 12 COMP_FREQ are symbols in the
APR = 0.02 # 2%
e
same (class body) namespace

d
a
APY = (1 + APR/COMP_FREQ) ** COMP_FREQ - 1

def __init__(self, balance):

Ac
this works because we used self.APY

te
self.balance = balance

def monthly_interest(self):
B y
return self.balance * self.APY

a th
@classmethod

© M
def monthly_interest_2(cls, amount):
this works because we used cls.APY

h t
return amount * cls.APY

yr
@staticmethod
i g this works because we used Account.APY

o p
def monthly_interest_3(amount):
return amount * Account.APY

C
def monthly_interest_3(self):
this will fail if APY is not defined in this function's
scope or in any enclosing scope
return self.amount * APY BEWARE: This can produce subtle bugs!
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Recap
c = Circle() à c in an instance of Circle
class Circle:
origin = (0, 0) class attribute – Circle.origin

m y
e
custom initializer à instance method (bound to instance)
def __init__(self, r):
self._r = r

@staticmethod
a d
_r is an instance attribute (private by convention only)

c à can still access using c._r

def create_unit_circle():
return Circle(1)

te A
static method – not bound to anything

y
c2 = Circle.create_unit_circle()
@classmethod
def set_origin(cls, x, y):

th B
class method – bound to class
cls.origin = (x, y)

M a
Circle.set_origin(10, 10)

©
à can also be called using c.set_origin(10, 10)
def double_radius(self):

h t instance method – bound to instance

g
self._r *= 2

i
c.double_radius()
@property

p yr
o
def radius(self):

C
return self._r
instance property (radius) with getter/setter methods
@radius.setter
def radius(self, value):
self._r = value
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Polymorphism

applied to different types

m y
the ability to define a generic type of behavior that will (potentially) behave differently when

d e
Python is very polymorphic in nature
c a
à duck typing

te A
y
if it walks like a duck and quacks like a duck then it is a duck

B
à e.g. when we iterate over a collection
a th
M
à object just needs to supports the iterable protocol

©
h t
does it matter if the collection is a list, a tuple, a dictionary, a generator?

yr i g
No - only thing we care about is that it supports the iterable protocol, so we can call

o p
iter()

C
to get an iterator, that itself can be anything as long as it
implements the iterator protocol
Polymorphism

Similarly, operators such as +, -, *, / are polymorphic

m y
à integer, floats, decimals, complex numbers
d e
c a
A
à lists, tuples

à custom objects
y te
th B
a
We can add support to our own classes for the + operator by implementing __add__ method

© M
t
We'll come back to polymorphism in the context of inheritance later

i g h
p yr
C o
Special Methods

We can add support in our classes for many of Python's functionality using special methods

m y
d e
a
These are methods that start with a double underscore and end with a double underscore

c
A
à dunder methods

te
B y
Never use this naming standard for your own methods or attributes

a th
Although you may not clash with Python's current special methods, it may in the future

© M
t
No need to use __my_func__ à just use something else!

i g h
p yr
C o
Special Methods

We've already seen many such special methods

m y
d e
a
__init__ à used during class instantiation

__enter__ context managers

Ac
__exit__ with ctx() as obj:
y te
th Bin this section we'll study a few more

a
__getitem__

__setitem__

© M
sequence types
a[i], a[i:j], del a[i]
__delitem__

h t
__iter__
yr i g
iterables and iterators
__next__
o p iter() and next()

C
__len__ à implements len()
__contains__ à implements in
 m y
d e
c a
__str__ e A
__repr__
t
B y
a th
© M
h t
yr i g
o p
C
__str__ vs __repr__

à both used for creating a string representation of an object

m y
à typically __repr__ is used by developers
d e
c a
à try to make it so that the string could be used to recreated the object

à otherwise make it as descriptive as possible

te A
B y
h
à useful for debugging

à called when using the repr() function

a t
© M
à __str__ is used by str() and print() functions, as well as various formatting functions

h t
i g
à typically used for display purposes to end user, logging, etc

yr
p
à if __str__ is not implemented, Python will look for __repr__ instead

C o
à if neither is implemented and since all objects inherit from
Object, will use __repr__ defined there instead

à we'll come back to this after we discuss inheritance

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Special Methods: Arithmetic Operators

__add__ +
m y
e
__matmul__ @

__sub__ -

c a d
New in 3.5: Currently Python does not have

__mul__ * A
this operator implemented in any type, but

te
this was added for better numpy support,

B y
which does implement this for matrix
__truediv__ /

a th multiplication. We can use too of course.

__floordiv__ //

© M
__mod__ %
h t
yr i g
p
__pow__ **

C o
à to indicate the operation is not supported, implement method
and return NotImplemented
Special Methods: Reflected Operators

Consider a + b

m y
d e
a
à Python will attempt to call a.__add__(b)

if this returns NotImplemented AND

Ac
operands are not of the same type

Python will swap the operands and try this instead:

y te
b.__radd__(a)

th B
__radd__ __rsub__
M a
__rmul__

t ©
h
__rtruediv__ __rfloordiv__ __rmod__

__rpow__
yr i g
o p
C
à we'll look at some code examples where this might be useful
Special Methods: In-Place Operators

__iadd__ +=
m y
d e
__isub__ -=

c a
__imul__ *=
te A
B y
__itruediv__ /=

a th
__ifloordiv__ //=

© M
__imod__ %=
h t
yr i g
p
__ipow__ **=

C o
Special Methods: Unary Operators, Functions

__neg__ -a
m y
d e
__pos__ +a

c a
__abs__ abs(a)
te A
B y
a th
© M
h t
yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Special Methods: Rich Comparisons

Reflections
m y
__lt__ < __gt__ à return NotImplemented
d e
c a
__le__ <= __ge__

te A
à Python automatically uses the reflection

__eq__ == __ne__
B y
h
if a < b returns NotImplemented

__ne__ != __eq__
a t
M
à Python will try b > a

__gt__ > __lt__

t ©
i g h
yr
__ge__ >= __le__

o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Special Methods: Hashing and Equality

Recall that for an object to be usable in a mapping type

m y
d e
a
à key in a dictionary
à element of a set

Ac
it must be hashable
y te
à implement __hash__
th B
à should also implement __eq__

M a
If __eq__ is implemented, __hash__ is implicitly set to None unless __hash__ is implemented

t © (but only do so for immutable objects)

i g h
yr
By default, when an override is not specified:

p
__hash__ uses id of object

o
C__eq__ uses identity comparison (is)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Special Methods: Boolean

Every object in Python has an associated truth (boolean) value

m y
any non-zero number à True False otherwise (i.e if equal to 0)
d e
c a
A
an empty collection (len() is 0) à False True otherwise

By default, any custom object also has a truth value

y te
th
à can override this by defining the __bool__ methodB à must return True / False

à if __bool__ is not defined

M a
t ©
h
à Python looks for __len__ 0 à False, anything else will be True

yr i g
à if neither present, always returns True

o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Special Methods: Callables

Any object can be made to emulate a callable by implementing a __call__ method

m y
d e
class Person:
c a
A
def __call__(self, name):

te
return f'Hello {name}'

B y
h
p = Person()

a t
p('Eric') à p.__call__('Eric')

© M à Eric

h t
à useful for creating function-like objects that need to maintain state

yr i g
à useful for creating decorator classes

o p
C
 m y
d e
c a
te A
B y
__del__
a th
© M
h t
yr i g
o p
C
A Class Finalizer

The garbage collector destroys objects that are no longer referenced anywhere
m y
d e
à hook into that lifecycle event
c a
te A
y
à use the __del__ method

th B
M a
The __del__ method will get called right before the object is destroyed by the GC

t ©
g h
à so the GC determines when this method is called

i
p yr
o
à __del__ is sometimes called the class finalizer

C
(sometimes called the destructor, but not entirely accurate, since
GC destroys the object)
When does __del__ get called?

m y
e
à that's the basic issue with the __del__ method

à we do not control when it will get called!

c a d
te A
called only when all references to the object are gone

B y
a th
M
à have to be extremely careful with our code

©
h t
à easy to inadvertently create additional references, or circular references

yr i g
o p
C
Additional Issues

If __del__ contains references to global variables, or other objects

m y
d e
à those objects may be gone by the time __del__ is called
c a
te A
If an exception occurs in the __del__ method
B y
à exception is not raised – it is silenced
a th
© M
à exception description is sent to stderr

h t
à main program will not be aware something went wrong during finalization

yr i g
o p
à prefer using context managers to clean up resources

C
à personally I do not use __del__
 m y
d e
c a
te A
B y
th
__format__
a
© M
h t
yr i g
o p
C
Yet another representation…

m y
e
We know we can use the format() function to precisely format certain types

à numbers, dates, etc

c a d
te
à see https://docs.python.org/3/library/string.html#formatspecA
B y
th
We can support this in our custom classes by implementing the __format__ method

a
format(value, format_spec)
© M
h t
i g
à if format_spec is not supplied, it defaults to an empty string

yr
p
à and it will instead use str(value)

o
C (which in turn may fall back to repr)
Implementation

Implementing our own format specification is difficult!

m y
d e
à beyond scope of this course

c a
te A
y
Frequently we delegate formatting back to another type that already supports it

B
a th
© M
h t
yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Modular Arithmetic

m y
For this project we are going to implement a special Mod class to implement some concepts in
modular arithmetic

d e
c a
A
You can look at these resources for more information on modular arithmetic

te
y
https://www.khanacademy.org/computing/computer-science/cryptography#modarithmetic

B
https://en.wikipedia.org/wiki/Modular_arithmetic
a th
© M
h t
g
but here's what we'll need for this project…

yr i
o p
C
Modular Arithmetic

à assume n is a positive integer à assume a and b are integers

m y
d e
à the residue of a number a modulo n, is simply a % n

c a
te A
B y
Two numbers, a and b, are said to be congruent modulo n: a = b (mod n)

if their residues are equal

a th
i.e. a % n == b % n

© M
Example:
h t
yr i g
7 % 12 à 7
à 7 is congruent to 19 modulo 12

o p 19 % 12 à 7
(think of a twelve hour clock for example)

C
Project

à create a class, called Mod

m y
d e
a
à initialize with value and modulus arguments

à ensure modulus and value are both integers

Ac should be read-only

à moreover, modulus should be positive

y te
th B
à store the value as the residue

M a
©
i.e. if value = 8 and modulus = 3, store value as 2 (8 % 3)

h t
yr i g
à implement congruence for the == operator

o p
à allow comparison of a Mod object to an int (in which case use the residue of the int)

C
à allow comparison of two Mod objects only if they have the same modulus

à ensure objects remain hashable

Project

à provide an implementation so that int(mod_object) will return the residue

m y
d e
a
à provide a proper representation (repr)

à implement the operators: +, - *, **

Ac
y te
à support other operand to be Mod (with same modulus only)

th B
à support other operand to be an integer (and use the same modulus)

M
à always return a Mod instance a
t ©
perform the +, - *, ** operations on the values (so there's nothing complicated here)

i g h
i.e. Mod(2, 3) + 16 à Mod(2 + 16, 3) à Mod(0, 3)

p yr
C o
à implement the corresponding in-place arithmetic operators

à implement ordering (makes sense since we are comparing residues)

à support other operand to be a Mod (with same modulus), or an integer
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Mutiple vs Single Inheritance

Python supports multiple inheritance

m y
d e
à can inherit from more than one class
c a
àmultiple inheritance can get a bit tricky

te A
B y
We're going to study single inheritance first

a th
à inheriting from a single class

© M
à single inheritance chain

h t
yr i g
à technically all classes inherit from the object class

o p
C
à later we'll look at multiple inheritance, and some useful use cases
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Inheritance

à fundamental concept in object oriented programming

y
à classes define properties and methods

m
e
à they can form a natural hierarchy
Shape

c a d
te A
Polygon Line
B y Ellipse

a th IS-A relationships
Quadrilateral Triangle

© M
h t Circle

g
Rectangle Isosceles

yr i
Square
o p Equilateral

C
characteristics (state and behavior)
à inherit (e.g. translate center pt)
à extend (e.g. # sides in polygon)
à override (e.g. # sides in triangle)
Terminology

y
Person
Parent
- name

e m
d
- eat()

c a
single inheritance à single parent
Teacher Student
Children
te A
- teach() - study()

B y
a th
Student inherits from Person

© M
t
Student is a child of Person

h
i g
Person is a parent of Student

yr
p
Student is a subclass of Person

C o
Student subclasses Person
Student extends Person
Student derives from Person
Instances

Objects that are created from a class are called instances of that class

m y
Person

d e
c a
Teacher Student

te A
p1 = Person() s1 = Student()
B y t1 = Teacher()

a th
s1 is a Student

© M à Student inherits from Person

à s1 is a Person

t
à s1 is an instance of Person
à s1 is an instance of Student

i g h à s1 is not a Teacher

p yr
C o
p1 is a Person à but p1 is not a Student
à but p1 is not a Teacher
isinstance() vs type() Person

Teacher Student

m y
d e
p1 = Person() s1 = Student()

c a t1 = Teacher()

te A
y
à s1 is an instance of Student isinstance(s1, Student) à True

à s1 is an instance of Person
th B
isinstance(s1, Person) à True

à s1 is not a Teacher
M a
©
isinstance(s1, Teacher) à False

h t
g
à p1 is not a Student isinstance(p1, Student) à False

yr i
o
type(instance)
p à returns the class the instance was created from

C
type(s1) à Student type(t1) à Teacher type(p1) à Person
 isinstance() vs type()

y
à more often use isinstance() rather than type() why?

e m
d
à often more concerned whether an object has certain behaviors

c a
A
Let's say we want to call the eat() method of an object if it has one

te
we could do this: if type(obj) is Person:

B y
h
obj.eat()

a t
M
à but if obj is a Student (or Teacher), this won't call its eat() method

Instead we could write:

t ©
if type(obj) in [Person, Student, Teacher]:

i g hobj.eat()

p yr
Much simpler to use isinstance():

C o if isinstance(obj, Person):
obj.eat()

tip: when using isinstance, try to use the least restrictive parent class you actually need
The issubclass() function

y
Used to inspect inheritance relationships between classes (not instances)

Note:
e m
Person

a d
à Person is a parent of Student

c
A
à Person is not a parent of CollegeStudent

te
Teacher Student à Person is an ancestor of CollegeStudent

B y à parent is a direct relationship

HighSchoolStudent
th
CollegeStudent

a
à subclass is not necessarily direct

issubclass(Student, Person)
© M à True

h t
g
issubclass(CollegeStudent, Student) à True

yr i
issubclass(CollegeStudent, Person) à True

o p
C
issubclass(Student, Teacher)

issubclass(Person, Student)
à False

à False
Defining Subclasses what about this one?
is it inheriting from nothing?

class Person: à the object class

m y
pass
d e
a
(we'll come back to this)

Ac
te
class Teacher(Person): class Student(Person):
pass pass

B y
a th
pass
© M
class CollegeStudent(Student): class HighSchoolStudent(Student):
pass

h t
yr i g
o p
C
 m y
d e
c a
te A
B y
t
object
a h
© M
h t
yr i g
o p
C
The object Class
class Person:

when we define a class that does not inherit from another class explicitly
pass

m y
à Python makes the class inherit from object implicitly
d esame as

c a class Person(object):

te A
à object is a class (even though it does not use a camel case) pass

type(object) à type

B y not needed
type(Person) à type

a th
M
object

©
à this means every class we create is a subclass of object

h t
à every object, even functions, modules, … Person

yr i g
p
issubclass(Student, object) à True Teacher Student

C o
isinstance(s1, object) à True
Implications of inheriting from object

y
à any class we create automatically inherits behaviors and attributes from the object class

m
à __name__

d e
à __new__
c a
à __init__

te A
à __repr__

B y
à __hash__

a th
M
à __eq__ and many more…

t ©
So even if we create an "empty" class: class Person: p = Person()

i g h pass

p.__repr__

p yr
à <__main__.Person object at 0x106cb79e8>

C
p == p
o
p.__hash__ à -9223372036579215458

à True

it will use whatever default implementation object has defined

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Overriding Functionality

y
When we inherit from another a class, we inherit its attributes, including all callables

à we can choose to redefine an existing callable in the sub class

e m
à this is called overriding

c a d
class Person:
te
class Student(Person):A
def say_hello(self):
return 'Hello!'
B y
def say_hello(self):
return 'Yo!'

def say_bye(self):
a th
return 'Bye!'

© M
h t
g
p = Person() s = Student()

yr i uses override

p
<obj>.say_hello() Hello! Yo!

C o
<obj>.say_bye() Bye! Bye!
uses inherited
Overriding Functionality

inherited ones
m y
When we create any class, we can override any method defined in the parent class, including

à including those defined in object

d e
c a
A
class Person: overrides __init__ in object

te
def __init__(self, name):

y
self.name = name

def __repr__(self):
th B overrides __repr__ in object

return f'Person(name={self.name})'

M a
t © overrides __repr__ in Person

h
class Student(Person):

i
def __repr__(self):

yr g
return f'Student(name={self.name})'

p
inherits __init__ from Person

C o
p = Person('john') str(p) à Person(name=john)
s = Student('eric'') str(s) à Student(name=eric)
Tip
Objects have a property: __class__ à returns the class the object was created from
Classes have a property: __name__

m y
à returns a string containing the name of the class

To get the name (string) of the class used to create an object

d e
à object.__class__.__name__

c a
class Person:
def __init__(self, name):
te A
class Student(Person):
def __repr__(self):
self.name = name

B y
return f'Student(name={self.name})'

def __repr__(self):
a th
M
return f'Person(name={self.name})'

©
instead we can do this, and get the same effect:

t
class Person:

i g h
yr
def __init__(self, name):
self.name = name

o p
C
def __repr__(self):
return f'{self.__class__.__name__}(name={self.name})'

class Student(Person):
pass
Overriding and the inheritance chain à there are some subtle points here to note.

Suppose we have this type of hierarchy:

Person
m y
- eat() à "Person eats" p = Person()

d e
- sleep() à "Person sleeps" p.routine() à

c aPerson eats

A
- work() à "Person works" Person works

te
- routine() à eat() work() sleep() Person sleeps

B y
h
Now we create a Student class that overrides the work() method only:
Student:
a t
M
- work() à "Student studies" what happens when we call routine() on a Student instance?
s = Student() à s.routine()

t ©
i g h
à runs routine as defined in Person class – but bound to s

p yr
à routine calls eat() à eat() in Person class bound to s à Person eats

C o
à … calls work() à finds the override in Student!!!
à uses the override in Student à Student studies

à… calls sleep() à sleep() in Person class bound to s à Person sleeps

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Extending Functionality

We can also extend functionality à creating a more specialized class

m y
d e
a
class Person:
pass

Ac
class Student(Person):
y te
def study():
return "study…study…study"
th B
M a
©
Student now has "extra" functionality à study()

h t
yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Delegating to parent

Often when overriding methods, we need to delegate back to the parent class

m y
d e
à calling a method specifically in the ancestry hierarchy (or a sibling class as we'll see later)

c a
A
The most common example is the __init__ method

class Person:
y te
class Student(Person):
def __init__(self, name, age):
self.name = name
th B
def __init__(self, name, age, major):
self.major = major
self.age = age

M a self.name = name
self.age = age
there has to be a
better way!

t ©
g h
We can explicitly call a method from the parent class

i
à delegating to parent

à super()
p yr
C o
super().method()

à will call method in the parent, but bound to the instance it is called from
Example

class Person:
def sing(self):
m y
return "I'm a lumberjack and I'm OK"

d e
c a
A
class Student(Person):

te
def sing(self):

y
return super().sing() + '\n' + "I sleep all night and I work all day"

th B
s = Student()
print(s.sing()) à
M a
I'm a lumberjack and I'm OK

t ©
I sleep all night and I work all day

i g h
p yr
C o
Careful! If you forget super() in super().sing() and
use self.sing() you'll end up with infinite recursion!
Example
à delegation works its way up the inheritance hierarchy until it finds what it needs

class Person:
m y
def sing(self):
d e
return "I'm a lumberjack and I'm OK"

c a
class Student(Person):

te A
y
pass

class MusicStudent(Student):
th B
def sing(self):

M a
return super().sing() + '\n' + "I sleep all night and I work all day"

t ©
i g h
yr
m = MusicStudent()

p
print(m.sing()) à I'm a lumberjack and I'm OK

C o
s = Student()
I sleep all night and I work all day

print(s.sing()) à I'm a lumberjack and I'm OK

Example __init__

class Person:
m y
def __init__(self, name, age):

d e
a
self.name = name
self.age = age

Ac
y te
B
class Student:
delegate back to parent
def __init__(self, name, age, major):
super().__init__(name, age)
a th
M
self.major = major
and do some additional work

t ©
i g h
When delegating, you don't have to delegate first

p yr
o
class Student:

C
def __init__(self, name, age, major):
self.major = major
super().__init__(name, age)

but usually safer to do so

Why?

y
à executing the delegate method may modify something you've already set in the instance

m
d e
a
class Person:
def __init__(self, name, age):
self.name = name
Ac
self.age = age

y te
B
self.major = 'N/A'

class Student(Person):
a th
M
def __init__(self, name, age, major):
self.major = major

t ©
super().__init__(name, age)

i g h
yr
s = Student('douglas', 42, 'literature')

p
C o
self.name à douglas
self.age à 42
self.major à N/A
 Delegation and Method Binding

when we call a method from an instance à method is bound to the instance

m y
when we delegate from an instance to parent method
d e
c a
A
à method is also bound to the instance it was called from

y te
B
class Person: class Student(Person):
def hello(self):
print('In Person class:', self)
a th def hello(self):
print('In Student class:', self)

p = Person()
© M super().hello()

s = Student()
h t
yr i g
p.hello() à In Person class: <__main__.Person object at 0x106cc3128>

o p
C
s.hello() à In Student class: <__main__.Student object at 0x106cc3da0>
In Person class <__main__.Student object at 0x106cc3da0>
Where things get really weird…

y
Since delegated method are bound to the calling instance

m
à any method called from the parent class will use the calling instance's "version" of the method

e
class Person:
def wake_up(self):
a
class Student(Person):

c
def do_work(self): d
print('Person awakes')
A
print('Student studies')

te
def do_work(self):
y
def routine(self):

B
h
print('Person works') super().routine()

def sleep(self):
a t print('but not before a quick game!')

print('Person sleeps')

© M s = Student()
s.routine()

t
def routine(self):
self.wake_up()

i g h à Person awakes

yr
self.do_work()
self.sleep() Student studies

o p Person sleeps

C
p = Person() Person awakes
p.routine() but not before a quick game!
à Person works
Person sleeps
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Slots

y
Remember that instance attributes are normally stored in a local dictionary of class instances

m
class Point: p = Point(0,0)
d e
def __init__(self, x, y):
c a
self.x = x
self.y = y
te A
p.__dict__ à {'x': 0, 'y': 0}

B y
th
As we know there is a certain memory overhead with dictionaries

a
M
What happens if we have thousands of instances of Point?

©
à a lot of overhead!

h t
yr i g
Python 3.3 introduced key sharing dictionaries to help alleviate this problem

o p
C
à but we can do even better

à slots
Slots

We can tell Python that a class will contain only certain pre-determined attributes

m y
à Python will then use a more compact data structure to store attribute values
d e
c a
A
an iterable containing the attribute names we will
class Point:

te
use in our class
__slots__ = ('x', 'y')
def __init__(self, x, y):

B y
h
self.x = x
self.y = y
a t
p = Point(0,0)

© M
t
p.__dict__ à Attribute Error: Point object has not attribute __dict__

h
vars(p)
yr i g
à TypeError: vars() argument must have __dict__ attribute

o p
C
dir(p) à […, 'x', 'y']

p.x à 0 p.x = 100 p.x à 100

Slots

y
memory savings, even compared to key sharing dictionaries, can be substantial

class Point: class Point:

e m
def __init__(self, x, y):
self.x = x
__slots__ = ('x', 'y')

c a
def __init__(self, x, y):d
self.y = y
A
self.x = x

te
self.y = y
10,000 instances

B y
1,729 KB
a th 635 KB

© M
Isn’t creating that many instances of an object rare?
à depends on your program
h t
yr i g
à example: returning multiple rows from a database and putting each row into an object

o p
C
à use slots in cases where you know you will benefit substantially
Slots

y
using slots results in generally faster operations (on average)

e m
d
class PersonDict: class PersonSlots:
pass __slots__ = ('name', )
c a
te A
def manipulate_dict():
B
def manipulate_slots(): y
p = PersonDict()
p.name = 'John' th
p = PersonSlots()

a
p.name = 'John'
p.name
del p.name
© Mp.name
del p.name

h t
yr
timeit(manipulate_dict)
i g timeit(manipulate_slots)

o p à about 30% faster

C
Slots

So why not use slots all the time then?

m y
d e
a
à if we use slots, then we cannot add attributes to our objects that are not defined in slots

c
class Point:
te A
__slots__ = ('x', 'y')
B y
def __init__(self, x, y):
self.x = x
a th
self.y = y

© M
p = Point(0,0)
h t
yr i g
p.z = 100

o p à AttributeError: 'Point' object

C
setattr(p, 'z', 100) has no attribute 'z'

à can cause difficulties in multiple inheritance

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Slots and Single Inheritance

So what happens if we create a base class with slots and extend it?
m y
d e
class Person:
c a
__slots__ = 'name',

te A
y
we don't specify slots in subclass
class Student(Person):
pass

th B
s = Student()
M a
t © so there is an instance dict
s.name = 'Alex'

i g h s.__dict__ à {}

p yr
in fact:

C o à works just fine

s.age = 18
s.__dict__ à {'age': 18}
Slots and Single Inheritance

m y
So subclasses will use slots from the parents (if present), and will also use an instance dictionary

d e
What if we want our subclass to also just use slots?
c a
te A
y
à specify __slots__ in the subclass

à but only specify the additional ones

th B
M a
à don't re-specify slots from up the inheritance chain

t ©
h
class Person:

i
__slots__ = 'name',

yr g
o p
class Student(Person):

C
__slots__ = 'age',

à Students will now use slots for both name and age
Slots and Single Inheritance

What happens if we do re-specify some slots? class Person:

m y
__slots__ = 'name',

d e
c a
A
class Student(Person):

te
__slots__ = 'name', 'age'

y
à works (kind of)

à memory usage has increased

th B
M a
à hides the attribute defined in the parent class

t ©
h
à may actually break in the future

yr i g
p
Python docs: In the future, a check may be added to prevent this.

C o
What happens if we redefine a slot attribute in a subclass?
à hides the attribute defined in the parent class
class Person:
m y
def __init__(self, name):

d e
a
self.name = name

c
p = Person('Alex')
@property
def name(self):
te A
p.name à ALEX
return self._name.upper()

B y
@name.setter
a th
M
def name(self, value):
self._name = value

t ©
i g h
p yr
class Student(Person): s = Student('Alex', 18)

C o
__slots__ = 'name', 'age'

def __init__(self, name, age):

s.name à Alex

self.name = name
self.age = age
What happens when a subclass defines slots but inherits from a parent that does not?

m y
class Person:

d e
pass

c a
te A
y
class Student(Person):

B
__slots__ = 'name', 'age'

a th
© M
Student will use slots for name and age

h t
yr i g
à but an instance dictionary will always be present too

o p
C
How are slotted attributes different from properties?

à a slotted attribute is not stored in an instance dictionary

m y
d e
a
à properties are also not in an instance dictionary

Ac
class Useless:

y te
obj = Useless()

B
@property

h
def useless(self):

t
obj.useless à 'useless!'

a
return 'useless!'

M
obj.useless = 'something else'
@useless.setter

©
obj.__dict__ à {}

t
def useless(self, value):
pass

i g h
p yr
they are not different – in fact they both use data descriptors

C o
à in both cases, the attributes are present in the class dictionary
à slots essentially create properties (getters, setters, deleters, and storage) for us
à we'll cover data descriptors later
The best of both worlds

m y
e
slots à faster attribute access, less memory

instance dictionary à can add attributes arbitrarily at run-time

c a d
à can we do both? à yes!
te A
B y
h
à specify __dict__ as a slot

a t
class Person:

© M
t
__slots__ = 'name', '__dict__'

i g h
yr
p.name = 'Alex'

o p
C
p.age = 18 p.__dict__ à {'age': 18}

à uses slots for name, and an instance dict for age

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
what are descriptors?

m y
d e
a
the underpinning mechanism for properties, methods, slots, and even functions!

Ac
non-data vs data descriptors
y te
th B
writing custom data descriptors

M a
t ©
i g h
avoiding common storage pitfalls

p yr
o
weak references and weak dictionaries

C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Descriptors

Let's look at a problem we want to solve:

m y
Suppose we want a Point2D class whose coordinates must always be integers
d e
c a
à plain attributes for x and y cannot guarantee this

te A
y
à instead we can use a property with getter and setter methods

B
a th
Let's implement x first:
M
class Point2D:

©
@property

h tdef x(self):

yr i g return self._x

p
@x.setter

C o def x(self, value):

self._x = int(value)

def __init__(self, x):

self.x = x
OK, so now we do the same thing for y:

y
class Point2D:

m
@property
def x(self):

d e
a
return self._x

@x.setter
Ac
def x(self, value):
self._x = int(value) te
this is tedious, repetitive boiler plate code!

y
th B better way needed!

a
@property

M
def y(self):
return self._y

t ©
@y.setter

i g h
yr
def y(self, value):
self._y = int(value)

o p
C
def __init__(self, x, y):
self.x = x
self.y = y
Descriptors

What if we could write a separate class like this: class IntegerValue:

m y
e
def get(self):

c a d
return self._value

te A
def set(self, value):
self._value = int(value)

B y
def __init__(self, value=None):

a th if value:
self.set(value)

M
and somehow use that in our class in this way:

©
h t
g
class Point2D:

yr
x = IntegerValue()
i
p
y = IntegerValue()

C o
and have code like this work by actually calling the get and set methods:

p = Point2D() p.x = 100.1 p.x à 100

Descriptors

Of course the previous code will not work as expected:

m y
d e
a
class Point2D: these are bound to the class, not the instances
x = IntegerValue()
y = IntegerValue()
Ac
y te
p.x = 100.1

th B
à will just create a new instance variable x with the float that will shadow the
class attribute

M a
©
à even if we use the get and set methods ourselves, we are still dealing with

t
h
IntegerValue instances bound to the class Point2D, not instances of the class

yr i g
p
à need to be able to tell Python two things:

o
C
x = IntegerValue
p.x
à this is meant to be bound to instances at run-time
à use these get and set methods of the IntegerValue instance
Descriptors

y
And this is where the descriptor protocol comes in

e m
d
There are 4 main methods that make up the descriptor protocol – they are not all required.

c a
A
à __get__ used to get an attribute value p.x

à __set__ used to set an attribute value

y te
p.x = 100

à __delete__
th
used to delete an attribute
B del p.x

à __set_name__
M a
new in Python 3.6 – I'll come back to this later

t ©
i g h
yr
Two categories of descriptors:

o p
those that implement __get__ only à non-data descriptors

C
those that implement __set__ and/or __delete__ à data descriptors

the distinction is really important as we'll see later

Using a Descriptor Class

We first define a class that implements the __get__ method only

y
à non-data descriptor

m
from datetime import datetime
d e
we'll come back to
class TimeUTC:
c awhat those are
def __get__(self, instance, owner_class):

te A
y
return datetime.utcnow().isoformat()

th B
a
Next we use it in our class by specifying it as a class attribute

class Logger:

© M
t
current_time = TimeUTC()

i g h
yr
And we can use it this way:
l = Logger()

o p
C
l.current_time à '2019-03-13T18:59:49.435411'

à as you can see it called __get__

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
The __get__ method class TimeUTC:
def __get__(self, instance, owner_class):

y
return datetime.utcnow().isoformat()
Important to understand how __get__ is being called
class Logger:

e m
d
current_time = TimeUTC()

c a
A
à Logger defines a single instance of TimeUTC as a class attribute

y te
à because TimeUTC implements __get__ Python will use that method when retrieving the

B
instance attribute value

But we can access the attribute from:

a th
à the Logger class itself

© M Logger.current_time

h t
à an instance of the Logger class l.current_time

yr i g
So, when __get__ is called, we may want to know:

o p
à which instance was used (if any) à None if called from class

C
à what class owns the TimeUTC (descriptor) instance à Logger in this case

à this is why we have the signature: __get__(self, instance, owner_class)

The __get__ method

So we can return different values from __get__ depending on:

m y
à called from class
d e
c a
A
à called from instance

very often, we choose to:

y te
th B
à return the descriptor (TimeUTC) instance when called from class itself (Logger class)

M a
à gives us an easy handle to the descriptor instance

t ©
à return the attribute value when called from an instance of the class (Logger instance)

i g h
yr
class TimeUTC:

o p
def __get__(self, instance, owner_class):
if not instance:

C return self
return datetime.utcnow().isoformat()
The __set__ method

The __set__ signature is as follows: self, instance, value

m y
d e
c a
à self: just like for the __get__, this references the descriptor instance, like any regular method

à instance: the instance the __set__ method was called from

te A
B y
h
à value: the value we want to assign to the attribute

a t
M
You'll notice there is no owner_class like we have in the __get__ method

©
h t
à setters (and deleters) are always called from instances

yr i g
à descriptors are meant to be used for instance properties

o p
C
Caveat with Set and Delete (and Get)

There is one really important caveat with __set__ and __delete__

m y
d e
a
notice that we have only created a single instance
class Logger:
current_time = TimeUTC() of the descriptor

Ac
So what happens when we do this? l1 = Logger()
y te
h
l2 = Logger()

t B
M a
à any instance of Logger will be referencing the same instance of TimeUTC

t ©
g h
à the same instance of TimeUTC is shared by all instances of Logger

i
p yr
o
à in this case it does not matter

C __get__ just returns the current UTC time

Caveat with Set and Delete (and Get)

But what happens when we have to "store" and "retrieve" data from the instances?

m y
Suppose IntegerValue is a data descriptor
d e
à implements __get__ and __set__ methods

class Point2D:
c a
x = IntegerValue()
y = IntegerValue()
te A
two separate instances of IntegerValue

y
assigned to the class attributes x and y

p1 = Point2D()
th B
p2 = Point2D()
M a
two separate instances of Point2D

t
but what object does p1.x reference?
© à the class attribute x

i g h
yr
what about p2.x? à the same class attribute x

o p à the same instance of IntegerValue

C
à we have to be mindful of which instance we are "storing" the data for

à this is one of the reasons both __get__ and __set__ need to know the instance
à one of the arguments
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Where to store the attribute value?

à we know the instance we are dealing with in both __get__ and __set__

m y
à could maybe store it in the instance dictionary?

d e
c a
that might work…

te A
but remember __slots__?
B y
a th
M
à we're not guaranteed to have an instance dictionary available

©
h t
à even if we were, what symbol to use? Might overwrite an existing attribute…

yr i g
à so, maybe we use a dictionary that's local to the data descriptor instance

o p
à key = object à problem if object is not hashable!

C à value = attribute value

Assuming our objects are hashable…

à create a dictionary in the data descriptor instance

m y
(e.g. in IntegerValue instance)

à when using __set__ save the value in the dictionary using instance as a key
d e
c a
A
à when using __get__ lookup the instance in the dictionary and return the value

te
class IntegerValue:
B y
def __init__(self):
self.data = {}
a th class Point2D:

M
x = IntegerValue()
y = IntegerValue()

©
def __set__(self, instance, value):

t
h
self.data[instance] = int(value)

yr i g
def __get__(self, instance, owner_class):
à works, but…

p
if not instance:

C oreturn self
return self.data.get(instance)
class IntegerValue: class Point2D:
def __init__(self): x = IntegerValue()

y
self.data = {} y = IntegerValue()

def __set__(self, instance, value):

e m
self.data[instance] = int(value)

c a d
def __get__(self, instance, owner_class):
if not instance:
te A
return self

B y
h
return self.data.get(instance)

a t
p = Point2D()

© M
à reference count is 1

p.x = 100.1
h t
à p is now a key in self.data

yr i g
à have a second reference to that point object

o p à reference count is 2

del p
C à our local (or global) reference to point is gone
à but reference count is still 1
à object is not garbage collected!
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Strong References

p1 object

m y
e
p1 = Person()

p2 = p1
p2
c a d
these are called strong references

te A
y
del p1 à there is still a strong reference (p2) to the object

h B
à object is still "alive", so Python does not garbage collect it

t
M a
©
del p2 à no more strong references to object

h t
à object will be garbage collected by Python

yr i g
o p
à that's the problem we faced in our data descriptor

C
Weak References

There is another type of object reference in Python à weak reference

m y
d e
a
think of it as a reference to an object that does not affect the reference count

c
A
as far as the memory manager is concerned

y te
p1
strong

h
object

t B
M
weak
a
tp2
©
i g h
yr
del p1 à no more (strong) references to object

o p
à object is garbage collected

C
à p2 is "dead"

à so for our data descriptor instead of storing the object as key

à store a weak reference to the object
Weak References

à use the weakref module

m y
d e
p1 = Person()
c
à p1 has a strong reference to the object
a
te A
p2 = weakref.ref(p1) à p2 is an (other) object that contains a weak reference to the object

B y
à p2 is a callable
a th
p2()
©
à returns the original objectM
h t
à or None if the object has been garbage collected

yr i g
Careful:
p
p3 = p2()

o
C à you just created a strong reference to the object!
Dictionaries of Weak References

m y
So, we'll want to create a dictionary of weak references (for our keys) for our data descriptor

d e
à weakref has a WeakKeyDictionary to do just that!
c a
p1 = Person()
te A
à p1 is a strong reference to the Person instance

B y
h
d = WeakKeyDictionary()

a t
à a weak reference is used for the Person instance

M
d[p1] = 'some value'

t ©
del p1

i g h à no more strong references à garbage collected

p yr
o
à item is automatically removed from weak key dictionary

C
(so be careful if you’re iterating over the dictionary views if that
happens during the iteration!

We won’t need to, so won't be a problem for our use-case)

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Outstanding Problem

I will show you in the code section how to use the WeakKeyDictionary approach
m y
d e
à but technique only works for hashable objects

c a
à cannot use the object as the key in a dictionary

te A
B y
h
Instead we'll try this approach first:

à use id(instance) as the key

a t
à still has a drawback
© M
h t
g
if an object is finalized, the corresponding entry still remains in dictionary

yr i
(that was a big advantage of the WeakKeyDictionary)

o p
à unnecessary clutter

C
à potential risks if id is re-used

à but at least we don't maintain strong reference to object

Final Approach!!
weakref.ref à callback functionality

m
à automatically calls a custom function when the object is being finalizedy
d e
à use regular data dictionary

c a
à use id(instance) as key

te A
y
à use (weak_ref, value) as corresponding dictionary value

B
th
à for each weak_ref register a callback function

a
M
à callback function will remove dead entry from dictionary

©
h t
g
We can now implement data descriptors that:

yr i
à have instance specific storage

o p
à do not use the instance itself for storage (__slots__ problem)

C
à handle non-hashable objects
à keep the data storage mechanism clean
 m y
d e
c a
te A
B y
th
__set_name__
a
© M
h t
yr i g
o p
C
The __set_name__ Method

à New in Python 3.6

m y
d e
c a
A
This is a very handy method that gets called (once) when the descriptor is first instantiated

y te
B
à that opens up some new possibilities

a th
M
à better error messages

t ©
à include name of attribute that raised the exception

i g h
yr
à useful application in descriptors used for validation

o p
C
Application

Here's a pretty typical application of using custom descriptors

m y
à again, key here is re-usability
d e
c a
te A
Suppose we have some attributes in a non-slotted class that need to be validated each time they

y
are set

th B
à get property name from __set_name__

M a
à __set__
t ©
i g h
yr
à validate data

p
à if OK, store data in instance dictionary, under the same name

o
C à wait a minute! does instance dictionary not shadow class attribute?

à not always with descriptors!

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Property Value Lookup Resolution

Interesting thing:
m y
d e
à class can have a property (descriptor) called x

c a
à it can have an instance dictionary __dict__

te A
B y
à that dictionary can contain a key, also called x

a th
what happens when we do this:

© M
h t
or i
obj.x

yr g
o p obj.x = value

Cà does it use __dict__ entry, or the descriptor?

It depends…

on whether the descriptor is a data or non-data descriptor

m y
d e
data descriptors (both __get__ and __set__ are defined)

c a
te A
à always override the instance dictionary (by default – can override this behavior)

B y
class MyClass:
prop = DataDescriptor()
a th
m = MyClass()
© M
h t
m.prop = value_1

yr i g m.prop à value_1

o p
m.__dict__['prop'] = value_2 m.prop à value_1

C
m.prop = value_3 à modifies the property value, not the
dictionary entry
It depends…

non-data descriptors (only __get__ is defined, and potentially __set_name__)

m y
d e
à looks in the instance dictionary first

c a
à if not present, uses the data descriptor

te A
B y
class MyClass:
prop = NonDataDescriptor()
a th
© M
m = MyClass()

h t
m.prop

yr i g
à 'prop' is not in m.__dict__, so calls __get__

o p
m.prop = 100

C
à prop is a non-data descriptor
à m.__dict__ is now {'prop': 100}

m.prop à 'prop' is in m.__dict__, so uses that value

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Back to properties…

property objects are data descriptors

m y
d e
à they have __get__, __set__ and __delete__ methods
c a
te A
y
age = property(fget=get_age, fset=set_age)

th B
p.age à calls __get__

M a
à in turn calls get_age(p)

p.age = 10
t ©
à calls __set__ à in turn calls set_age(p, 10)

i g h
yr
if fset was not defined

o p
à calls __set__

C à __set__ sees fset is not defined

à raises an AttributeError (can't set attribute)

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Functions and Descriptors

one more thing I want to discuss regarding descriptors

m y
e
à no real practical use, but good to understand how Python does some of its "magic"

d
When we write a class with an class Person:
c a
instance methods like this: def __init__(self, name):
self.name = name
te A
B y
h
def say_hello(self):

a t
return f'{self.name} says hello'

Then we can write code like this:

© M
p = Person('Alex')

h t p.say_hello()

yr i g
How does the function end up being bound to the instance?

o p
C
Does Python have C code that says "oh, this function is defined inside a class, and
being called from an instance, do something different"?

No! à functions are objects that implement the (non-data) descriptor protocol!
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Project

We have a project where we need to define classes that have fields that we want validated

m y
e
before we can set their value.

a d
This might be because these objects will later be serialized into a database, and we need to

c
A
ensure the data is valid before we write to the database.

Part 1
y te
Write two data descriptors:

th B
IntegerField
M a
à only allows integral numbers, between minimum and maximum value

CharField
t ©
à only allows strings with a minimum and maximum length

i g h
p yr
So we want to be able use the descriptors like this:

C o
class Person:
name = CharField(1, 50)
age = IntegerField(0, 200)
Part 2

You probably wrote two unrelated classes to do this.

m y
d e
But you'll notice there's quite a bit of code duplication, with only the actual validation being different

c a
te A
Refactor your code and create a BaseValidator class that will handle the common functionality

B y
h
Then change your IntegerField and CharField descriptors to inherit from BaseValidator

a t
© M
t
If you haven't coded you descriptors that way already, make sure you can also omit one or both

h
g
of the minimum and maximum values where it makes sense.

yr i
p
For example we may want to specify a string field that has no maximum limit.

C o
Or we may want an integer field that has an upper bound, but no lower bound.
Or maybe no bounds at all.
don't forget unit tests!

m y
d e
c a
te A
B y
Good
a thluck!
© M
h t
yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
In this section we'll cover:
STATUS_STARTED = 'started'

m y
e
STATUS_PENDING = 'pending'
how to deal with multiple related constants

c a
STATUS_OK = 'ok'
d
STATUS_ERROR = 'error'

what are enumerations and why have them?

A
STATUSES = [STATUS_STARTED, …, STATUS_OK]

te
B y
Python's support for enumerations

a th
©
Python enumerations are versatile M
h t
i g
à lots of associated functionality!

yr
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
How to deal with collections of related constants?

We could try this approach: RED = 1

m y
GREEN = 2
d e
BLUE = 3

c a
COLORS = (RED, GREEN, BLUE)

te A
and then we could use it in our code this way:
B y
a th
pixel_color = {RED: 255, BLUE: 0, GREEN: 200}

M
pixel_color[RED] à 255

©
h t
yr i g
o p
C
But there's some downsides:

RED in COLORS à True but: 1 in COLORS à True

m y
is 1 really a color?

RED < GREEN à True à meaningless!

d e
c a
does the string 'RED' correspond to a valid color name?
te A
B y
h
à maybe do this instead: RED = 'red'
GREEN = 'green'
BLUE = 'blue'
a t
© M
t
COLORS = (RED, GREEN, BLUE)

à repetitive code!
i g h
p yr
à later can change associated values: RED = 'violet'

C o
à could run into bugs RED = 'red'
GREEN = 'red'
BLUE = 'blue'
à non-unique values by mistake!

à RED * 2 à this is meaningless!

What we really want…

an immutable collection of related constant members:

m y
à have unique names (that may have meaning)
d e
c a
à have an associated constant value

te A
à unique associated values

B y
th
à operations such as RED * 2 or RED < GREEN are not even allowed

a
M
à supports enumerating members by name

©
t
à 'RED', 'GREEN', 'BLUE'

h
yr i g
à lookup member by name

o p
à lookup member by value

C
Maybe use a class?

class Colors: Colors.RED

m y
e
RED = 1
GREEN = 2
BLUE = 3
hasattr(Colors, 'RED')

c a d
A
getattr(Colors, 'RED')

y te
h
à how do we look up a member based on it's value?

t B
à how do we iterate the members?
M a
t
à we can, but is order preserved?
©
i g h
yr
à still cannot guarantee uniqueness of values

o p
C
Aliases

y
Sometimes we want multiple symbols to refer to the same "thing"

POLY_4 = 4
e m
RECTANGLE = 4
SQUARE = 4
c a d
Consider RECTANGLE, SQUARE and RHOMBUS to be aliases for POLY_4
RHOMBUS = 4

te A
POLY_3 = 3
y
Consider TRIANGLE, EQUILATERAL and ISOCELES to be aliases for POLY_3

B
h
TRIANGLE = 3
EQUILATERAL = 3

a t
M
ISOSCELES = 3

à lookup value 3
©
à return POLY_3

t à values are still unique, names are just

à lookup name SQUARE

i g h à return POLY_4 aliases to the first one

p yr
o
RECTANGLE

C POLY_4
(4) SQUARE

RHOMBUS
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Python Enumerations

à PEP 435 à Python 3.4

m y
d e
à enum module

c a
à Enum type

te A
B y
à specialized enumerations: IntEnum, Flag, and IntFlag

a th
Python 3.6

© M
h t
à enumerations are created by subclassing Enum

yr i g
o p
C
Terminology

class Color(Enum):

m y
e
RED = 1
GREEN = 2
BLUE = 3
c a d
te A
y
à Color is called an enumeration

à Color.RED is called an enumeration member

th B
à members have associated values
M a
t ©
g h
à the type of a member is the enumeration it belongs to

i
p yr
C o
 class Color(Enum):
The basics… RED = 1

y
GREEN = 2
BLUE = 3

e m
type(Color.RED) à Color

c a d
isinstance(Color.RED, Color) à True

te A
B y
h
str(Color.RED) à 'Color.RED'

a t
repr(Color.RED)
M
à '<Color.RED: 1>'

©
Color.RED.name
h t
à 'RED'

yr i g
o p
Color.RED.value à 1

C
 class Color(Enum):
Equality and Membership RED = 1

y
GREEN = 2
BLUE = 3
member equality is done using identity à is (but == works too)

e m
c a d
A
membership uses in

y te
B
Color.GREEN in Color à True

Color.RED is Color.RED à True

a th
Color.RED is Color.BLUE
M
à False

©
h t
yr i g
à note that member and it's associated value are not equal!

o p
Color.RED == 1 à False

C
 class Color(Enum):
Members are hashable RED = 1

y
GREEN = 2
BLUE = 3
Enumeration members are always hashable
e m
à can be used as keys in dictionaries
c a d
à can be used as elements of a set

te A
B y
a th
M
pixel_color = {

©
Color.RED: 100,
Color.GREEN: 25,
Color.BLUE: 255
h t
}

yr i g
o p
C
 class Color(Enum):
Programmatic Access to Members
RED = 1

y
GREEN = 2
can reference a member this way: BLUE = 3

m
Color.RED

how to "get" a member using:

d e
c a
A
à value (e.g. 1)
à string of the name (e.g. 'RED')

y te
th B
à to get a member by value

enumerations are callable

M a
Color(2)
t
à Color.GREEN
©
i g h
yr
à to get a member by name

o p
enumerations implement __getitem__ method

C
Color['GREEN'] à Color.GREEN

Color(2) is Color['GREEN'] à True

 class Color(Enum):
Enumerating Members RED = 1

y
GREEN = 2
à enumerations are iterables BLUE = 3

e m
list(Color) à [Color.RED, Color.GREEN, Color.BLUE]

c a d
à definition order is preserved

te A
class Color(Enum):

B y
h
GREEN = 2
BLUE = 3
RED = 1
a t
© M
t
list(Color) à [Color.GREEN, Color.BLUE, Color.RED]

i g h
yr
à has nothing to do with member value

o p
C
à Enumerations have a __members__ property

à returns a mapping proxy (immutable dict)

à keys are the names (as strings), and values are the values
Constant Members and Constant Values

Once an enumeration has been declared:

m y
d e
à member list is immutable
c
(cannot add or remove members)
a
à member values are immutable
te A
B y
à cannot be subclassed (extended via inheritance)

a th
M
à unless it contains no members!

©
h t
(we'll come back to that later)

yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Members are unique

Recall that members are guaranteed to be unique

m y
d e
So this should not work:

c a
class Color(Enum):
te A
red = 1
B y
crimson = 1
carmine = 1
a th
blue = 2
aquamarine = 2
© M
h t
yr i g
o p but it works!!!

C
Aliases class Color(Enum):
red = 1
we still have unique members but we now also have aliases

m y
crimson = 1
carmine = 1
à in fact our enumeration contains only two members:
d e
blue = 2

c a aquamarine = 2
à Color.red
list(Color) à Color.red, Color.blue
te A
à Color.blue

B y
a th
The remaining "members" point to these two members:

Color.crimson à Color.red
© M
h t
g
Color.carmine à Color.red

yr i
Color.aquamarine à Color.blue

o p
C
Color.crimson is Color.red à True

Color.carmine is Color.crimson à True

 class Color(Enum):
Lookups red = 1

y
crimson = 1

m
Lookups with aliases will always return the "master" member carmine = 1

e
blue = 2

d
a
aquamarine = 2
Color(1) à Color.red
Ac
Color['crimson'] à Color.red
y te
th B
Containment
M a
t
Color.crimson in Color à True
©
i g h
p yr
C o
 class Color(Enum):
Iterating Aliases
red = 1

y
crimson = 1

m
list(Color) à Color.red, Color.blue carmine = 1

d e
blue = 2

a
aquamarine = 2

à the only way to see the aliases is to use __members__

Ac
y te
Color.__members__
th B
à mappingproxy
M a
t ©
h
{

i g
'red': <Color.red: 1>,

yr
'crimson': <Color.red: 1>,

o p 'carmine': <Color.red: 1>,

'blue': <Color.blue: 2>,

C }
'aquamarine': <Color.aquamarine: 3>

à note how keys are different, but point to the same member
Ensuring Unique Values

m y
e
à we may want to guarantee that our enumerations do not contain aliases (unique values)

à we could just be careful writing our code!

c a d
te A
à or use the @enum.unique decorator

B y
a th
@enum.unique
class Color(Enum):
© M
red = 1
h t
crimson = 1
carmine = 1
yr i g
blue = 2

o p
aquamarine = 2

C
à ValueError: duplicate values found
(when class is compiled)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Customizing

m y
e
Enums are classes

class attributes become instances of that class à members

c a d
te A
We can define functions in the enumeration class
B y
a th
à become bound methods when called from a member (instance of the class)

© M
h t
à custom methods

yr i g
o p
à implement dunder methods

C
__str__ __repr__ __eq__ __lt__ etc…
Member Truthyness

by default, every member of an enum is truthy

m y
d e
à irrespective of the member value
c a
te A
class State(Enum): bool(State.READY) à True
B y
h
READY = 1
BUSY = 0

a t
bool(State.BUSY) à True

© M
So we can implement the __bool__ method to override this behavior:

h t
class State(Enum):
READY = 1
yr i g
o
BUSY = 0
p bool(State.READY) à True

C
def __bool__(self):
return bool(self.value)
bool(State.BUSY) à False
Extending Enums

m y
e
enumerations are classes à they can be extended (subclassed)

BUT… only if they do not contain any members

c a d
te A
B y
à cannot create a partial enum with some members and extended it with more members

a th
M
à might seem limiting, but not really

©
h t
à create a base enum with functionality (methods)

yr i g
p
à use it as a base class for other enumerations that define their members

o
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Auto generating associated values

à Python 3.6
m y
d e
We can use enum.auto() as member values
c a
te A
Behind the scenes, uses a method in the Enum class:

B y
_generate_next_value_

a th
© M
à default implementation results in sequential integer numbers

h t
yr i
class Number(enum.Enum):
g Number.ONE.value à 1

p
ONE = enum.auto()

o
TWO = enum.auto() Number.TWO.value à 2

C
THREE = enum.auto() NumberTHREE.value à 3
_generate_next_value_(name, start, count, last_values)

à static method à called by enum.auto()

m y
d e
arguments:
c a
name the name of the member
te A
start
B y
(only actually used in functional creation – not covered in this course)

count
a th
the number of members already created (be careful with aliases!)

last_values
© M
a list of all preceding values

h t
yr i g
returns:
o p
C
value to be assigned to member
Overriding

m y
e
The default implementation of _generate_next_value_ generates sequential integer numbers

d
c a
A
à be careful mixing auto() and your own values!

y te
B
à safer not do do it

a th
© M
à override the default implementation by implementing _generate_next_value_ in our enum

h t
yr i g
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Application Exceptions

m y
e
In many cases, especially in larger projects, we want to have an easy way to generate exceptions

d
c a
à raise consistent exception types

te A
à associated exception code

B y
à associated default exception message
a th
© M
t
à ability for our user to easily list out all the possible exceptions in our app

i g h
yr
This is very common when writing REST APIs for example

p
C o
à many ways to do this, but here we want to practice using enumerations
Functionality

à single enumeration AppException

m y
d e
à exceptions have a name (key) and three associated values

c a
•
•
name (e.g. NotAnInteger)
code (e.g. 100)
te A
•

B y
default message (e.g. 'Value is not an integer.')

h
associated exception type (e.g. ValueError)

t
•

M a
à lookup by exception name (key) or code (value)

t ©
AppException['NotAnInteger'] AppException(100)

i g h
yr
à method to raise an exception

p
AppException.Timeout.throw()

o
C
à ability to override default message when throwing exception
AppException.Timeout.throw('Timeout connecting to DB.)
Tips

m y
e
enumeration members will be defined using a tuple containing three values

à code, exception type (class name), default message

c a d
te A
NotAnInteger = 100, ValueError, 'Value is not an integer'

B y
h
Use the __new__ approach I showed you earlier

a t
Make the value the error code

© M
h t
g
Provide an additional property for message

yr i
o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
In this section

m y
what are exceptions? à not necessarily errors

d e
c a
A
exception propagation

Python's exception class hierarchy

y te
th B
handling exceptions

M a
raising exceptions

t ©
i g h
yr
creating our own exception classes

o p
à custom hierarchy

C à extending functionality
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
What are exceptions?

m y
e
à exceptions are objects à they are instances of some class

c a d
à when an exception is raised

te A
y
à trigger a special execution propagation workflow

B
à exception handling
a th
© M
à if current call does not handle exception it is propagated up to caller

h t
i g
à call stack trace is maintained

yr
o p à documents origin of exception

C à every call in the stack

Call Stack
Call Stack

m y
à call function_1 function_1
d e
c a
à calls function_2 function_2
te A
B y
function_1

a th
à calls function_3

© M function_3

h t function_2

yr i g function_1

o p
C
Exception Propagation

m y
d e
a
exception is raised in function_3
function_3

Ac
y te
if unhandled, propagates to function_2

function_2
th B
M a
t ©
if unhandled, propagates to function_1

function_1
i g h
p yr
C o if unhandled, program terminates

à stack trace will contain information describing this call stack

Exception Handling

m y
exceptions are not necessarily fatal, i.e. do not necessarily result in program termination

d e
à we can handle exceptions as they occur

c a
te
à do something and let program continue running "normally" A
B y
h
à do something and let original exception propagate

a t
M
à do something and raise a different exception

t ©
à try
g h
(compound statement)

i
p
à except
yr
C o
à finally

à else
What are exceptions used for?

m y
e
à exceptions are not necessarily errors

c a d
à indicate some sort of anomalous behavior

te A
à sometimes not even that
B y
a th
consider StopIteration exception raised by iterators

© M
after all we would expect this to happen à not really anomalous!

h t
yr i g
o p
C
Two main categories of exceptions

m y
à compilation exceptions (e.g. SyntaxError)

d e
c a
à execution exceptions

te A
(e.g. ValueError, KeyError, StopIteration)

B y
a th
Python's built-in exception classes use inheritance to form a class hierarchy

© M
h t
base exception for every exception in Python

yr i g
à BaseException

o p
C à but do not inherit from this one
Python's Exception Hierarchy

à see https://docs.python.org/3/library/exceptions.html#bltin-exceptions for full list

m y
d e
BaseException
c a
SystemExit
te A
(raised on sys.exit())

B y
KeyboardInterrupt
th
(raised on Ctrl-C for example)

a
GeneratorExit

© M (raised when generator or coroutine is closed)

Exception
h t à everything else!

yr i g
o p
C
Python's Exception Hierarchy

à most of the time any exception we work with inherits from Exception

m y
direct subclasses of Exception include:
d e
c a
• ArithmeticError A
• FloatingPointError

te
• ZeroDivisionError
• AttributeError

B y
h
• LookupError • IndexError
• SyntaxError
a t
• Key Error

M
• RuntimeError
• TypeError
• ValueError
t ©
i g h
yr
and more…

o p
C
exception objects are instances of these classes
Python's Exception Hierarchy

when exceptions inherit from other exceptions: Exception

m y
LookupError
d e
c a
IndexError

te A
y
an IndexError exception IS-A LookupError exception

a LookupError exception IS-A Exception exception

th B
M
an IndexError exception IS-A Exception exception
a
t ©
i g h
à if we catch a LookupError exception

yr
à it will also catch an IndexError exception

o p
C
à if we catch an Exception exception

à it will also catch any subclass of Exception

à broad catch – usually bad practice
Basic Exception Handling

the try statement is used to for exception handling

m y
d e
à multi-part statement
c a
te A
y
à basic: try:
…
except ValueError:
th B
…

M a
t ©
à get a handle to exception object in except clause:

i
try:
g h
yr
…

o p except ValueError as ex:

…

C
à we'll come back to the try statement
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Handling exceptions using try

the full try statement has these clauses:

m y
d e
à keep this guarded code as short as possible
try:

c a
A
code that we want to protect from à often just a single statement!

te
some potential exception(s)

y
à only guard code where you can do
except <ExceptionType> as ex:

th B
something about the exception

code that will run if that specified

M
<ExceptionType> occurs (or any subclass) a
finally:
t ©
i g h
code that always executes –

yr
whether exception occurred or not

else:
o p
C
code that executes if try terminates normally
(an except clause must be present)
Handling exceptions using try
the try statement clauses are quite flexible

m y
d e
à this is required
a
try:

Ac
except <ExceptionType> as ex:
te
à may appear 0 or more times

y
h B
à allows handling different exception types in different ways

t
M a
à as ex is optional
à <ExceptionType> is optional

t ©
h
à broad exception handler - be careful!

finally:

yr i g
à appears 0 or 1 times

o p
C
else: à appears 0 or 1 times
à only allowed if an except clause is present

à and of course they can be nested

Handling Multiple Exception Types

m y
try:

d e
a
…

Ac
te
except AttributeError: always go from most specific to least specific
…

B y
h
(in terms of class hierarchy)
except TypeError:
…
a t
except IndexError:
© M
first except match will run

h t
yr i g
o p
C
except order matters à IndexError
Exception à LookupError
à KeyError

m y
l = [1, 2, 3]
d e
try:
c a
A
l[3]

te
except IndexError:
print('invalid index')
except LookupError: y
à 'invalid index'

B
print('lookup error')

a th
© M
l = [1, 2, 3]

h t
i g
try:

yr
l[3]

p
except LookupError: à 'lookup error'

C o
print(lookup error')
except IndexError:
print(index error')
Grouping Exception Handlers

à sometimes we want to handle multiple exception types in the same way

m y
d e
try:

c a
A
…

te
except ValueError as ex:

y
log(ex)
except TypeError as ex:
log(ex)
th B
M a
à in that case group the exception types in a single except clause

t ©
try:
i g h
yr
…

p
except (ValueError, TypeError) as ex:

C o
log(ex)
Bare Exception Handlers

à exception handlers that do not specify an exception type

m y
à catch any exception
d e
c a
à overly broad for general use

te A
à but good in certain circumstances

B y
a th
à do some cleanup work, logging, … and re-raise exception

© M à we'll come back to that

try:

h t
how do we know what the exception is?

g
…
except:

yr i
sys.exc_info()

p
…

C o à returns info about current exception

à tuple

exc_type, exc_value, exc_traceback

Exception Objects

What properties and methods an exception object has depends on the exception

m y
d e
a
standard exceptions have at least these two properties:

args
Ac
à arguments used to create exception object
à often error message

y te
__traceback__

h B
à the traceback object

t
M a
à use the traceback module for easier visualization

t © à print_tb

i g h à print_exception

p yr
à tb is same object as last item returned by sys.exc_info()

C o
à not needed unless you are writing really advanced
Python code (like frameworks)

à may be good for logging

try… finally…

à finally code block is guaranteed to run, exception or no exception

m y
d e
à can actually do this

c a
try:
te A
…

B y
h
finally:
…
a t
à no except clauses
© M
h t
à the exception is not handled – so it will propagate up

yr i g
à but finally block runs before propagation

o p
à useful for cleanup code

C
What happens if an exception handler itself raise an exception?

m y
e
works as normal – if the exception is not handled, it is propagated up

c a d
à exception handling can be nested inside an except handler

te A
à not just inside the try clause
B y
a th
M
à in fact can be nested inside else, or even finally

©
h t
yr i g
o p
C
Handling Exceptions vs Avoiding Exceptions

It's easier to ask forgiveness than it is to get permission.

m y
Grace Hopper

d e
c a
Sometimes referred to as the EAFP principle in Python
A
(Easier to Ask For Permission)

te
B y
h
Consider these two alternatives:

a t this does not even begin to

M
try: address possible permission
with open(fname, 'r') as f:

©
issues!

t
…
except OSError:

i g h Also things could go wrong while

writing to the file, and we won't close

yr
# do something
it (hence the context manager)

o p
if os.path.exists(fname) and not os.path.isdir(fname):

…. C
f = open(fname, 'r')

f.close()
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Raising an Exception

m y
à use the raise statement

d e
c a
à raised object must be an instance of BaseException

te A
B y
h
à i.e. class must be a subclass of BaseException

a t
(does not have to be a direct subclass)

© M
h t
i g
raise ValueError()

yr
o p
C
BaseException

m y
e
à __init__ can handle *args

à accessible via args attribute of exception object (instance)

c a d
à used for str() and repr() representations
te A
B y
à subclasses inherit this behavior

a th
ex = ValueError('a', 'b', 'c')
© M
h t
i g
ex.args à ('a', 'b', 'c')

p yr
str(ex) à "('a', 'b', 'c')"

C o
repr(ex) à "ValueError('a', 'b', 'c')"

à typically use first argument for the main "message"

Re-Raising current exception being handled

m y
When are handling an exception

d e
à inside an except block
c a
te A
y
à we can re-raise the current exception raise (no exception object specified)

à this will resume exception propagation

th B
M a
t ©
h
try:
…
except:

yr i g à very handy to catch any exception

o p
# bare except!
log('…')
à log it

Craise à resume as if we had not interrupted it

à perfectly acceptable use of bare except

Exception Traceback

m y
e
à we have seen: exception handlers that themselves raise exceptions (nested exceptions)

d
à "final" exception traceback shows us a history of this
c a
à traceback

te A
try:

B y
h
raise ValueError()
except ValueError:
a t ValueError

M
try:
TypeError
raise TypeError()
except TypeError:
t © KeyError
raise KeyError()

i g h
p yr
o
à sometimes this is too much information for our user

C
à our internal implementations should maybe remain opaque
Using raise… from…

We can control (to some extent) what traceback is included

m y
d e
a
try:
raise ValueError()
except ValueError:
Ac
try:
raise TypeError()
y te
except TypeError:

th BKeyError

a
raise KeyError() from None

© M
try:

h t
g
raise ValueError()

yr i
except ValueError as ex_1: ValueError

p
try:

C o
raise TypeError()
except TypeError:
raise KeyError() from ex_1 KeyError

à useful to hide exception stacks that are just implementation details

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Creating New Exception Classes

à create a new class that inherits from Exception (or one of its subclasses)
m y
d e
à recall BaseException is the base class for all exceptions
c a
te A
BaseException

B y
Exception

a th
M
SystemExit

©
KeyboardInterrupt very specialized

t
GeneratorExit

i g h
p yr
The standard is that most exceptions, including custom exceptions, inherit from Exception

C o
à can still broadly trap using Exception

à without trapping SystemExit, KeyboardInterrupt, etc

Creating New Exception Classes

m y
class WidgetException(Exception):

d e
a
"""base custom exception for Widget library"""

Ac
class WidgetValueError(ValueError):
y te
"""custom ValueError exception"""

th B
M a
class OutOfStockException(WidgetException):

t ©
"""out of stock exception"""

i g h
p yr
C o
Creating an Exception Hierarchy

m y
e
à often we have an entire set of custom exceptions

à keep the exceptions organized à just like Python

c a d
à create a hierarchy of custom exceptions

te A
à allows trapping exceptions at multiple levels

B y
h
class WidgetException(Exception)

t
à except OutOfStock

M
class StockException(WidgetException)
a à except StockException

t ©
class OutOfStock(StockException) à except WidgetException

i g h
yr
class OverInventoryLimit(StockException)

o p
class SalesException(WidgetException)

C
class OverOrderLimit(SalesException)

class CouponCodeInvalid(SalesException)
Extending Functionality

à exceptions are classes

m y
d e
à add functionality (properties, methods, attributes)

c a
à even implement special methods (__str__, __repr__, etc)

te A
à for example, add auto-logging to your exceptions
B y
a th
à use for consistent output mechanisms (e.g. json representation of the exception)

© M
h t
yr i g
o p
C
Multiple Inheritance

Exceptions, like any class, can inherit from multiple classes

m y
d e
Keeping it simple

c a
à custom exception can inherit from multiple exceptions

te A
B y
class WidgetException(Exception)

a th
M
class SalesException(WidgetException)

©
t
class InvalidSalePrice(SalesException, ValueError)

i g h
p yr
o
à if an InvalidSalePrice exception is raised

C
à can be trapped with either InvalidSalePrice or ValueError
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
What is metaprogramming?

m y
e
Wikipedia: Metaprogramming is a programming technique in which computer programs
have the ability to treat other programs as their data.

c a d
te A
It means that a program can be designed to read, generate, analyze or
transform other programs, and even modify itself while running.

B y
a th
© M
à basic idea is we can use code to modify code

h t
à keeps code DRY
yr i g
o p
C
à Don't Repeat Yourself

à some functionality requires Python 3.6 or above

You already know some metaprogramming techniques…

m y
à decorators

d e
use code to modify the behavior of another piece of code
c a
te A
à we'll explore class decorators a bit, where we decorate classes, not just functions

B y
à as well as decorator classes where we use classes to decorate

a th
M
à descriptors

t ©
use code to essentially modify the behavior of the dot (.) operator

i g h
p yr
C o
Metaclasses

m y
We'll look at what metaclasses are
d e
c a
à how they are used in the creation of classes (types)

te A
y
à think of a metaclass as a class(type) factory

B
th
à how we can use them to hook into the class creation cycle

a
M
à we'll still stick to single inheritance

©
t
à metaclasses don't always play nicely with multiple inheritance

h
yr i g
o p
C
Word of caution

“Metaclasses are deeper magic than 99% of users should ever worry about. If you
wonder whether you need them, you don’t (the people who actually need them know
m y
d e
with certainty that they need them, and don’t need an explanation about why).”
— Tim Peters
c a
te A
B
à superficially, metaclasses are not difficult to understand
y
à but the details can get complicated

a th
M
à knowing when to use a metaclass is not easy

©
h t
à unless you come across a problem where the use of a metaclass is obvious à don't use them!

yr i g
à makes code harder to read

p
à just because you have a new hammer, does not mean everything is a nail

o
C
à beware of the "solution in search of a problem" syndrome!

à if you're writing a library/framework à maybe use metaclasses

à if you're writing application code à probably not!
Word of caution

m y
so you are unlikely to ever need to use metaclasses

d e
à unless you’re writing a library or framework
c a
te A
B y
à still good to understand how they work

a th
M
à provides deeper insight into Python internal mechanics and how things work

©
à it's actually fun!

h t
yr i g
à worth the time to understand, even if you don't use

o p
C
Word of caution

m y
this is going to be a wild ride!

d e
c
à some of the concepts just take time to absorb and remember
a
à you will get confused (I still do on occasion)
te A
B y
a th
à when you do, go back to previous lectures and code videos

M
à you will probably need to watch some videos multiple times

©
à don't rush
h t
yr i g
à take time to experiment with code

o p
C
 m y
d e
c a
te A
B y
__new__
a th
© M
h t
yr i g
o p
C
Constructing Instances of a Class

m y
à call the class ex: Person('Guido')
e
à classes are callable

d
c a
à how? back to that later

te A
à the new class instance is created

B y
a th
(and initialized in some ways)

© M
à the __init__ method is called (bound to the new object)

h t
yr i g
à after the instance has been created

o p à gives us a "hook" to customize the initialization

C à but how is the new instance actually created?

The __new__ Method

à object implements the __new__ method

m y
d e
à it is a default implementation of __new__

c a
à used in the creation of instances of any class

te A
à can be called directly
B y
class Person:
a th
def __init__(self, name):
self.name = name
© M
h t
i g
p = object.__new__(Person)

yr
o p
à p is a new object, an instance of Person

C à __init__ is not called

à do it ourselves p.__init__('Raymond')
The __new__ Method

object.__new__(class, *args, **kwargs)

m y
d e
à it is a static method

c a
à not bound to object

te A
à class is the symbol for the class we want to instantiate

B y
h
à accepts *args, **kwargs

a t
à signature must match __init__ of class

© M
à but it just ignores these arguments

h t
g
à returns a new object of type class

yr i
p
à can override __new__ in our custom classes

C o
à should return a new object
à should be an instance of the class

à but does not have to be

Overriding the __new__ Method

m y
e
à typically we do something before/after creating the new instance

à delegating actual creation to object.__new__

c a d
à in practice we use super().__new__

te A
à ensures inheritance works properly
B y super().__new__(cls)

class Person:
a th
def __new__(cls, name, age):
# can do things here
© M
h t
i g
# create the object we want to return

yr
instance = object.__new__(cls)

o p
# more code here

C # and finally return the object we want

return instance
How is __new__ Called?

à it is called by Python when we call the class

m
à we'll come back to how later
y
d e
Person('Guido')

c a
te A
à Python calls __new__(Person, 'Guido')

B y
à __new__ returns an object
a th
© M
à if that object is of the same type as the one "requested"

h t
i g
à new_object.__init__('Guido') is called

yr
p
à new object is returned from call

o
C p = Person('Guido')
Remember

__new__ is a static method

m y
d e
à done implicitly for us by Python
c a
te A
class Person:
B y
@staticmethod
def __new__(cls, name):
a th
M
return super().__new__(cls)

©
h t
i g
à don't need to use @staticmethod

yr
o p
class Person:

C def __new__(cls, name):

return super().__new__(cls)
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Classes are created somehow…

When Python encounters a class as it compiles (executes) our code

m y
class Person:

d e
pass

c a
à a symbol Person is created in the namespace

te A
B y
à that symbol is a reference to the class Person (it is an object)

a th
M
à how does Python create that class?

©
h t
i g
à a class is an instance of type

yr
o p
à that's why a class is also called a type

C à type is a callable

(and is in fact a type (class) itself, and inherits from object)

The inner mechanics of class creation
class name

class Person:
class body

m y
planet = 'Earth'
name = property(fget=lambda self: self._name)
d e
c a
def __init__(self, name):
self._name = name
te A
B y
th
Step 1: The class body is extracted (basically just a blob of text, but it is valid code)

a
M
Step 2: A new dictionary is created – this will be the namespace of the new class

©
t
Step 3: the body code is executed inside that namespace, thereby populating the namespace

h
yr i g
(Think of this code directly inside a module: after the code has been executed, the
module namespace will contain planet, name, __init__)

o p
C
Step 4: A new type instance is created using the name of the class, the base
classes and the populated dictionary

à type(class_name, class_bases, class_dict)

Calling type

m y
what happens when we call a custom class (e.g. Person('Alex'))?

d e
c
à creates an instance of that custom class (e.g. Person instance)
a
te A
à type is itself a class

B y
what happens when we call type?
a th
© M
t
à type(class_name, class_bases, class_dict)

h
yr i g
à creates an instance of type

o p
à which will be a type (or class)

C à named class_name
à inherits from class_bases
à class dictionary will be class_dict
 m y
d e
c a
te A
B y
a th type
© M
h t
yr i g
o p
C
Using type to create new types

we saw how to use type to create new types (classes)

m y
type(class_name, class_bases, class_dict)
d e
c a
à type is an object

te A
à like any object it inherits from object

B
à it has __new__, and __init__
y
à type is a class à is is callable
a th
© M
à calling it creates a new instance of type

h t
g
à just like calling any class

yr i à calls __new__

o p
C
type(name, bases, cls_dict)
à type.__new__(type, name, bases, cls_dict)
à returns an instance of type
type is a class, so…

à can be used as a base class for a custom class

m y
d e
class MyType(type)

c a
te A
à we can override __new__
y
à tweak things, but delegate to type for actual type creation

B
à essentially we can create custom types
a th
© M
t
MyClass = MyType(name, bases, cls_dict)

i g h
yr
à __new__(MyType, name, bases, cls_dict)

o p
C
à returns an instance of MyType
à but MyType subclasses type
à instance is still a type instance
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Finally…

we have reached the point where metaclasses becomes easy!

m y
d e
à we saw how type is used to create classes
c a
type(name, bases, dict_)

te A
B y
a th
à we saw how we can customize type by subclassing it and overriding __new__
MyType(name, bases, dict_)

© M
h t
g
à tedious! get the class code as text

yr i
create the class dictionary

o p exec the code within that dictionary

C determine the bases

call type (or MyType)
Creating classes normally

à to create a new type (class), we usually don't call type

m y
class Student(Person):
d e
a
instead of this
pass

Ac
y
somehow Python does all the manual steps we were doing
te
h B
à class name, bases, class dictionary, class body code

t
M a
à and calls type(name, bases, dict_)
use this

t ©
à so if we have
h
class MyType(type):

i g
yr
def __new__(cls, name, bases, dict_):
…

o p
C
à we just want to tell Python

à "this" has a technical name… à metaclass

Metaclass

à to create a class, another class is used

m y
d e
a
à typically type

Ac
à the class used to create a class, is called the metaclass of that class

y te
à by default Python uses type as the metaclass

th B
à but we can override this
M a
t ©
class Person(metaclass=MyType):
…

i g h
p yr
o
à so default is actually:

C class Person(metaclass=type):
…
Putting it together first argument is the metaclass (MyType)

y
second argument is the name of the class being created

m
class MyType(type):

d e
a
def __new__(mcls, name, bases, cls_dict):
# tweak things

Ac
# create the class itself via delegation

y te
B
new_class = super().__new__(mcls, name, bases, cls_dict)

# tweak some more

a th
# and return the new class

© M does all the manual steps: name, code, class dict,

bases

t
return new_class

i g h then calls MyType(name, bases, cls_dict)

p yr
class Person(metaclass=MyType): à MyType.__new__(MyType, name, bases, cls_dict)

C o
def __init__(self, name):
self.name = name
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Metaclasses aren't always the best approach…

à metaclasses are very powerful

m y
d e
à but they can be hard to understand when reading code
c a
te A
à sometimes decorators can work just as well

B y
a th
à generally easier to understand

© M
h t
à more functional in nature

yr i g
o p
C
Decorating Classes

@my_dec
m y
class MyClass:
d e
…

c a
recall what the decorator @ syntax actually does with a function:

te A
B y
h
def my_func():

t
@my_dec

a
pass
def my_func(): à
pass

© M my_func = my_dec(my_func)

h t
same with class

yr i g
p
class MyClass:
@my_dec

C o
class MyClass:
pass
à
pass

MyClass = my_dec(MyClass)
Decorating Classes

à write a decorator that expects a class as the input

m y
à and returns the tweaked class
d e
c a
A
def savings_account(cls): @savings_account

te
cls.account_type = 'Savings' class BankAccount:

y
return cls def __init__(self, account_number, balance):

th B
self.account_number = account_number
self.balance = balance

M a
same as doing this:
t ©
class BankAccount:

i g h
def __init__(self, account_number, balance):

yr
self.account_number = account_number
self.balance = balance

o p
C
BankAccount = savings_account(BankAccount)

à class is created first

à then it's decorated
Can even make the decorator parametrized

def apr(rate):
def inner(cls): à account_type is a decorator factory
m y
cls.apr = apr
d e
cls.apy = … à it returns a decorator

c a
A
return cls

te
return inner

@apr(0.02)
B y
class SavingsAccount():
pass
a th
© M
t
@apr(0.0)
class CheckingAccount():

i g h
yr
pass

o p
à could use a metaclass instead à overkill!

C
à would need to pass the rate parameter to the metaclass

à possible, we haven't learned that yet

Class decorators can be used to…

m y
e
create, delete or modify class attributes

c a d
à plain attributes

te A
B y
h
à modify methods

a t
M
à maybe even apply decorators to class methods

©
h t
yr i g
à this is also metaprogramming

o p
C
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
 Classes can be used as decorators too

à don't confuse with class decorators

m y
à here we want to use a class to decorate functions

d e
à Class instances can be callable
c
à implement __call__ method
a
à parallel to decorator functions
te A
B y fn is captured by __init__

a th
M
def decorator(fn): class Decorator:
def wrapper(*args, **kwargs): def __init__(self, fn):
return fn(*args, **kwargs)
t © self.fn = fn
return wrapper

i g h
yr
def __call__(self, *args, **kwargs):
return self.fn(*args, **kwargs)

o p fn is captured by closure

C
fn is called by calling wrapper
fn is called by calling __call__
Function vs Class approach

y
def decorator(fn):
def wrapper(*args, **kwargs):
return fn(*args, **kwargs)
e m
return wrapper

c a d
my_func = decorator(my_func)

te A
y
à calling my_func actually calls the wrapped function returned from calling decorator

th B
class Decorator:
def __init__(self, fn):
M a
self.fn = fn

t ©
i g h
def __call__(self, *args, **kwargs):

yr
return self.fn(*args, **kwargs)

o p
my_func = Decorator(my_func)

C
à my_func is an instance of Decorator
à calling my_func calls the __call__ method of the instance
Major Difference

à both approaches work fine to decorate functions

m y
d e
à one major difference
c a
te A
à function decorator, decorated function is still a function

B y
h
à class decorator, decorated function is now an instance of a class

a t
M
à it is not a function

t ©
i g h
p yr
C o
Major Difference

à this affects our ability to decorate class methods using the decorator class
m y
d e
c a
A
How do functions defined in a class become methods when called from instance?

à functions are non-data descriptors

y te
th B
a
à Python calls the __get__ method of the function

M
t ©
à if called from instance it returns a bound method

i g h
yr
à if not, it just returns itself

o p
C
à we can implement similar functionality in our decorator

à decorator class will now work with instance, class and static methods
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Pros and Cons

m y
Class Decorator Metaclass

d e
c a
easier to understand
A
harder to understand

te
B y
h
can stack decorators can only specify a single metaclass

a t
© M
subclasses do not "inherit" decorator subclasses "inherit" parent metaclass

h t
g
can decorate classes in an only a single custom metaclass (or

yr i
inheritance chain with different subclasses thereof, using multiple

p
decorators inheritance) can be in play in an

C o inheritance chain
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
Metaclass Usage

m y
e
class Metaclass(type):
def __new__(mcls, name, bases, cls_dict):
return super().__new__(mcls, name, bases, cls_dict)
c a d
class MyClass(metaclass=Metaclass):
te A
pass

B y
a th
© M
à can we pass extra parameters to Metaclass.__new__?

h t
yr i g
à yes, starting in Python 3.6

o p
C
Passing Additional Parameters to the Metaclass

m y
e
class Metaclass(type):

d
def __new__(mcls, name, bases, cls_dict, arg1, arg2, arg3=None):
return super().__new__(mcls, name, bases, cls_dict)

c a
te A
y
à specify these extra arguments when we define the custom class

B
à they must be passed as named arguments

a th
© M
h t
class MyClass(metaclass=Metaclass, arg1=val1, arg2=val2):

i g
pass

p yr
why named arguments?

C o
à positional arguments are used to specify parent classes
 m y
d e
c a
te A
B y
th
__prepare__
a
© M
h t
yr i g
o p
C
Recall…
à in a metaclass

m y
à __new__(mcls, name, bases, cls_dict)
d e
c a
A
When the metaclass is called (via class MyClass(metaclass=MyMeta))

te
y
à Python determines and sends to the __new__ method of the metaclass

B
th
à the metaclass used to create the class (mcls)

a
M
à name of the class we are creating (name)

©
t
à the classes we are inheriting from (bases)

h
yr i g
à a dictionary used as the class namespace (cls_dict)

o p
à where does cls_dict come from?

Cà __prepare__ method of the metaclass

à type implements it
The __prepare__ Method

m y
e
à type implements a default __prepare__ method

à it is also a static method (like __new__)

c a d
te A
à if additional (named) args are passed to metaclass, they are also passed to __prepare__

B y
h
à Python calls __prepare__ before it calls __new__

a t
M
à the return value of __prepare__ must be a mapping type (e.g. a dict)

t ©
à Python manipulates that dictionary a bit

i g h
yr
à then calls __new__, passing that return value as the class dictionary argument

o p
C
 m y
d e
c a
te A
B y
a th
__call__
M
h t©
y rig
o p
C
Declarative Creation of Classes

when Python executes this code:

m
class Student(Person, metaclass=MyMeta):y
class_attr = 100

d e
def my_func(self):
c a
pass

te A
it is essentially doing the following steps for us:

B y
à name = 'Student'
a th
à bases = (Person, )

© M
t
à creates a class dictionary, by calling __prepare__

i g h
à executes the class body within that class dictionary

p yr
à calls the metaclass to create the Student class

C o
MyMeta(name, bases, class_dict)

à assigns the created class to the symbol Student in our scope

à plus a few other things (__qualname__, __doc__)
Programmatic Creation of Classes

We can reproduce those same steps ourselves

m y
à define name, bases
d e
c a
A
à create an initial class dictionary (so like calling __prepare__, but there is no __prepare__

te
here, we're on our own)

à calculate and add __qualname__ to class dictionary

B y
à add __doc__ to class dictionary
a th
© M
à execute the code in the context of the class dictionary

h t
g
à fully populated class dictionary

yr i
o p
à call metaclass

C type(name, bases, class_dict)

à there's a lot going on à at the end though, the metaclass is called

How do we make instances of our custom class callable?

à implement the __call__ method in our class

m y
class Person:

d e
def __call__(self):
print('__call__')
c a
à instances of the class Person are now callable
te A
p = Person()
B y
à call the class to create an instance of it
p() à '__call__'
a th
M
à __call__ was called as a method bound to p

©
h t
yr i g
à so an object is callable if the class used to create it implements __call__

o p
à but the Person class is itself callable

C
à so the class used to create Person must implement __call__

à the metaclass of Person

What does the __call__ Method in type do?

à type is a class that implements a __call__ method

m y
d e
à it is called when we create instances of our Person class

c a
p = Person(…)

te
à it will get called as a method bound to the Person class A
B y
h
à calls Person.__new__ to create a new instance of the Person class

a t
M
à calls __init__ bound to that new instance returned by __new__

t ©
à returns the new, initialized instance of Person

i g h
p yr
C o
Instance Creation

instance creation
m y
type.__call__
d e
c a
A
type/custom p = Person()

te
__call__

B y
a
instance th
M
Person.__new__ Person
instance of

t ©
h
Person
__new__

yr i g
__init__

o p
C initialized instance of Person
The metaclass of type

à a class is callable because it's metaclass implements __call__

m y
à every class has a metaclass à the class we use to create the class
d e
c a
à type is a class à it also has a metaclass à itself

te A
y
à type(type) is type

B
à so it has a __call__ method

a th
à that's why type is callable

© M
h t
g
à it behaves the same way

yr i
p
à calls type.__new__ à returns the newly created object (the class)

C o
à calls __init__ bound to the newly created object (the class)

à returns the newly created object (the class)

Class Creation

instance (of type) creation

m y
type.__call__
d e
c a
type(type/custom)
__call__
te A
Person = type(name, bases, cls_dict)

B y
a
instance th
M
type.__new__ Person
class of

t ©
h
type/custom
__new__

yr i g
__init__

o p
C initialized Person class
Declarative Approach

y
à when we use class declaration, process is almost the same, just tweaked a bit

instance (of type) creation

e m
Python process

c a d
__qualname__
__docs__

te Aclass Person(metaclass=type):
…

B y
th
type(type/custom)

a
__call__

© M new Person class

h t
yr i g
o p type/custom

C __prepare__ __new__ __init__

 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
The Attribute Accessors

Whenever we write code like this:

m y
print(person.name)
d e
c a
A
or
getattr(person, 'name')

y te
th B
a
à we are getting an attribute value from the object

M
à Python is doing that for us
t ©
i g h
yr
à as we have seen, that attribute's value could be living in any number of places

p
C o
à instance dictionary

à class attribute
à descriptor

à in a parent class
The __getattribute__ Method

à getting an attribute value will call __getattribute__ on our object

m y
d e
à Python provides a default implementation
c a
à it does a lot of work!

te A
B y
à but we can override it

a th
M
à implement __getattribute__(self, name)

©
h t
yr i g
à if we do, we often use super().__getattribute__ to do the actual work

o p
C
à remember even calling a method first needs to get the method from the object

à special __ methods are looked up differently than our own attributes

The __getattr__ Method

If the __getattribute__ method cannot find the requested attribute

m y
à raises AttributeError
d e
c a
A
à Python then tries to call __getattr__

y te
th B
à think of it as a "last resort" attempt to find an attribute

M a
à but only called if __getattribute__ fails to find attribute

t ©
h
à default implementation does nothing but re-raise AttributeError

yr i g
à we can also override this method

o p
à more common than overriding __getattribute__

C
à attribute access is a combination of __getattribute__
and, possibly __getattr__
 Default attribute lookup flow obj.attr
can override

m y
e
obj.__getattribute__('attr') (often using

yes
c a
no d delegation to
super)
in class (incl parents) dict?

te A
yes
data descriptor?
no

B y yes
in instance dict?
no

a th
M
yes no
__get__ in instance dict? return it AttributeError

t ©
i g hyes

yr
non-data descriptor?
__getattr__

o p no

C
__get__
can override AttributeError
return class attr
return it
Caution

m y
e
à very easy to get into infinite recursion bugs!

c a d
A
à just use super().__getattribute__ to bypass your own overrides

te
B y
à we'll see examples in the coding section

a th
© M
h t
yr i g
o p
C
Overriding Access for Class Attributes

m y
à __getattribute__ and __getattr__ are instance methods

d e
à how do we override class attribute access?
c a
te A
y
à override __getattribute__ and __getattr__ in the metaclass

B
a th
M
à since classes are instances of a metaclass

t ©
i g h
p yr
C o
 m y
d e
c a
te A
B y
a th
© M
h t
yr i g
o p
C
The __setattr__ Method

We saw how __getattribute__ and __getattr__ get used

m
à getting an attributey
d e
What about setting attributes?
c a
te A
à __setattr__(self, name, value)

B y
a th
M
à there is no __setattribute__ like with the getters

©
h t
yr i g
à if setting an attribute that does not exist, it is created in the object __dict__

o p à unless there is no __dict__

C
Default attribute setter flow obj.attr = value
can override

m y
e
obj.__setattr__('attr', value) (often using

yes
c a
no d delegation to
super)
in class (incl parents) dict?

te A
yes
data descriptor?
no

B y
yes
a th no
__set__

© M obj.__dict__ available?

t
AttributeError

i g h
yr
insert / update it

o p
C
à the same caveats regarding infinite recursion apply here
m y
à use super().__setattr__
d e
c a
A
à especially careful when overriding both getters and setters

te
B y
h
à for class attributes

a t
M
à use metaclass

t ©
i g h
p yr
C o