Object Relational Tutorial - SQLAlchemy 0.9 Documentation
Object Relational Tutorial - SQLAlchemy 0.9 Documentation
9 Documentation
SQLAlchemy ORM
Object Relational Tutorial
Version Check
Connecting
Declare a Mapping
Create a Schema
Create an Instance of the
Mapped Class
Creating a Session
Adding and Updating Objects
Rolling Back
Querying
Common Filter Operators
Returning Lists and Scalars
Using Textual SQL
6/27/16, 4:11 PM
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 1 of 37
6/27/16, 4:11 PM
Version Check
A quick check to verify that we are on at least version 0.9 of
SQLAlchemy:
>>> import sqlalchemy
>>> sqlalchemy.__version__
0.9.0
Connecting
For this tutorial we will use an in-memory-only SQLite database.
To connect we use create_engine() :
>>> from sqlalchemy import create_engine
>>> engine = create_engine('sqlite:///:memory:', echo=True)
Lazy Connecting
The Engine , when first
returned by
shortly.
6/27/16, 4:11 PM
database.
See also
Database Urls - includes examples of create_engine() connecting to
several kinds of databases with links to more information.
Declare a Mapping
When using the ORM, the configurational process starts by
describing the database tables well be dealing with, and then by
defining our own classes which will be mapped to those tables. In
modern SQLAlchemy, these two tasks are usually performed
together, using a system known as Declarative, which allows us to
create classes that include directives to describe the actual
database table they will be mapped to.
Classes mapped using the Declarative system are defined in
terms of a base class which maintains a catalog of classes and
tables relative to that base - this is known as the declarative
base class. Our application will usually have just one instance of
this base in a commonly imported module. We create the base
class using the declarative_base() function, as follows:
>>> from sqlalchemy.ext.declarative import declarative_base
>>> Base = declarative_base()
Page 3 of 37
...
...
...
...
...
...
...
6/27/16, 4:11 PM
name = Column(String)
fullname = Column(String)
password = Column(String)
def __repr__(self):
return "<User(name='%s', fullname='%s', password='
self.name, self.fullname, self
Create a Schema
With our User class constructed via the Declarative system, we
have defined information about our table, known as table
metadata. The object used by SQLAlchemy to represent this
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 4 of 37
6/27/16, 4:11 PM
>>> User.__table__
Table('users', MetaData(bind=None),
Column('id', Integer(), table=<users>, primary_key=
Column('name', String(), table=<users>),
Column('fullname', String(), table=<users>),
Column('password', String(), table=<users>), schema
Classical Mappings
The Declarative system,
though highly recommended,
is not required in order to use
SQLAlchemys ORM. Outside of
Declarative, any plain Python
class can be mapped to any
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 5 of 37
6/27/16, 4:11 PM
PRAGMA table_info("users")
()
CREATE TABLE users (
id INTEGER NOT NULL,
name VARCHAR,
fullname VARCHAR,
password VARCHAR,
PRIMARY KEY (id)
)
()
COMMIT
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 6 of 37
6/27/16, 4:11 PM
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>"
self.name, self.fullname, self
We include this more verbose table definition separately to highlight the
difference between a minimal construct geared primarily towards inPython usage only, versus one that will be used to emit CREATE TABLE
statements on a particular set of backends with more stringent
requirements.
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 7 of 37
6/27/16, 4:11 PM
Creating a Session
Were now ready to start talking to the database. The ORMs
handle to the database is the Session . When we first set up
the application, at the same level as our create_engine()
statement, we define a Session class which will serve as a
factory for new Session objects:
>>> from sqlalchemy.orm import sessionmaker
>>> Session = sessionmaker(bind=engine)
In the case where your application does not yet have an Engine
when you define your module-level objects, just set it up like
this:
>>> Session = sessionmaker()
6/27/16, 4:11 PM
close it?.
In fact, the Session has identified that the row returned is the
same row as one already represented within its internal map of
objects, so we actually got back the identical instance as that
which we just added:
>>> ed_user is our_user
True
Page 9 of 37
6/27/16, 4:11 PM
Also, weve decided the password for Ed isnt too secure, so lets
change it:
>>> ed_user.password = 'f8s7ccs'
We tell the Session that wed like to issue all remaining changes
to the database and commit the transaction, which has been in
progress throughout. We do this via commit() . The Session
emits the UPDATE statement for the password change on ed, as
well as INSERT statements for the three new User objects weve
added:
>>> session.commit()
SQL
Page 10 of 37
6/27/16, 4:11 PM
SQL
After the Session inserts new rows in the database, all newly
generated identifiers and database-generated defaults become
available on the instance, either immediately or via load-on-firstaccess. In this case, the entire row was re-loaded on access
because a new transaction was begun after we issued commit() .
SQLAlchemy by default refreshes data from a previous
transaction the first time its accessed within a new transaction,
so that the most recent state is available. The level of reloading is
configurable as is described in Using the Session.
Rolling Back
Since the Session works within a transaction, we can roll back
changes made too. Lets make two changes that well revert;
ed_user s user name gets set to Edwardo :
>>> ed_user.name = 'Edwardo'
Page 11 of 37
6/27/16, 4:11 PM
>>> session.add(fake_user)
Querying the session, we can see that theyre flushed into the
current transaction:
>>> session.query(User).filter(User.name.in_(['Edwardo',
SQL 'fakeu
[<User(name='Edwardo', fullname='Ed Jones', password='f8s7ccs'
SQL
SQL
Querying
A Query object is created using the query() method on
Session . This function takes a variable number of arguments,
which can be any combination of classes and class-instrumented
descriptors. Below, we indicate a Query which loads User
instances. When evaluated in an iterative context, the list of
User objects present is returned:
>>> for instance in session.query(User).order_by(User.id):
SQL
...
print instance.name, instance.fullname
ed Ed Jones
wendy Wendy Williams
mary Mary Contrary
fred Fred Flinstone
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 12 of 37
6/27/16, 4:11 PM
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 13 of 37
6/27/16, 4:11 PM
Page 14 of 37
6/27/16, 4:11 PM
equals :
query.filter(User.name == 'ed')
not equals :
query.filter(User.name != 'ed')
LIKE :
query.filter(User.name.like('%ed%'))
IN :
query.filter(User.name.in_(['ed', 'wendy', 'jack']))
# works with query objects too:
query.filter(User.name.in_(
session.query(User.name).filter(User.name.like
))
NOT IN :
query.filter(~User.name.in_(['ed', 'wendy', 'jack']))
IS NULL :
query.filter(User.name == None)
# alternatively, if pep8/linters are a concern
query.filter(User.name.is_(None))
IS NOT NULL :
query.filter(User.name != None)
# alternatively, if pep8/linters are a concern
query.filter(User.name.isnot(None))
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 15 of 37
6/27/16, 4:11 PM
AND :
# use and_()
from sqlalchemy import and_
query.filter(and_(User.name == 'ed', User.fullname ==
Note
Make sure you use and_() and not the Python and
operator!
OR :
from sqlalchemy import or_
query.filter(or_(User.name == 'ed', User.name == 'wendy'
Note
Make sure you use or_() and not the Python or operator!
MATCH :
query.filter(User.name.match('wendy'))
Note
Page 16 of 37
6/27/16, 4:11 PM
one() , fully fetches all rows, and if not exactly one object
identity or composite row is present in the result, raises an
error. With multiple rows found:
>>> from sqlalchemy.orm.exc import MultipleResultsFound
SQL
>>> try: #doctest: +NORMALIZE_WHITESPACE
...
user = query.one()
... except MultipleResultsFound, e:
...
print e
Multiple rows were found for one()
Page 17 of 37
6/27/16, 4:11 PM
>>> session.query(User).from_statement(
SQL
...
text("SELECT * FROM users where name=:n
...
params(name='ed').all()
[<User(name='ed', fullname='Ed Jones', password='f8s7ccs')>
See also
Using Textual SQL - The text() construct explained from the
perspective of Core-only queries.
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 18 of 37
6/27/16, 4:11 PM
Counting
Query includes a convenience method for counting called
count() :
>>> session.query(User).filter(User.name.like('%ed')).count
SQL
2
Counting on count()
Query.count() used to be a
very complicated method
when it would try to guess
whether or not a subquery
was needed around the
existing query, and in some
exotic cases it wouldnt do the
right thing. Now that it uses a
simple subquery every time,
its only two lines long and
always returns the right
answer. Use func.count() if
a particular statement
Page 19 of 37
6/27/16, 4:11 PM
Building a Relationship
Lets consider how a second table, related to User , can be
mapped and queried. Users in our system can store any number
of email addresses associated with their username. This implies a
basic one to many association from the users to a new table
which stores email addresses, which we will call addresses .
Using declarative, we define this table along with its mapped
class, Address :
>>> from sqlalchemy import ForeignKey
>>> from sqlalchemy.orm import relationship, backref
Page 20 of 37
6/27/16, 4:11 PM
Page 21 of 37
6/27/16, 4:11 PM
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 22 of 37
6/27/16, 4:11 PM
Lets add and commit Jack Bean to the database. jack as well
as the two Address members in the corresponding addresses
collection are both added to the session at once, using a process
known as cascading:
>>> session.add(jack)
>>> session.commit()
SQL
Querying for Jack, we get just Jack back. No SQL is yet issued for
Jacks addresses:
SQL
Page 23 of 37
6/27/16, 4:11 PM
...
filter(User.id==Address.user_id).\
...
filter(Address.email_address=='jack@goo
...
all():
# doctest: +NORMALIZE_WHITESPA
...
print u
...
print a
<User(name='jack', fullname='Jack Bean', password='gjffdd')
<Address(email_address='[email protected]')>
The actual SQL JOIN syntax, on the other hand, is most easily
achieved using the Query.join() method:
>>> session.query(User).join(Address).\
SQL
...
filter(Address.email_address=='[email protected]'
...
all() #doctest: +NORMALIZE_WHITESPACE
[<User(name='jack', fullname='Jack Bean', password='gjffdd'
#
#
#
#
explicit con
specify rela
same, with e
same, using
As you would expect, the same idea is used for outer joins,
using the outerjoin() function:
query.outerjoin(User.addresses)
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 24 of 37
6/27/16, 4:11 PM
Using Aliases
When querying across multiple tables, if the same table needs to
be referenced more than once, SQL typically requires that the
table be aliased with another name, so that it can be
distinguished against other occurrences of that table. The Query
supports this most explicitly using the aliased construct. Below
we join to the Address entity twice, to locate a user who has two
distinct email addresses at the same time:
Using Subqueries
The Query is suitable for generating statements which can be
used as subqueries. Suppose we wanted to load User objects
along with a count of how many Address records each user has.
The best way to generate SQL like this is to get the count of
addresses grouped by user ids, and JOIN to the parent. In this
case we use a LEFT OUTER JOIN so that we get rows back for
those users who dont have any addresses, e.g.:
Using the Query , we build a statement like this from the inside
out. The statement accessor returns a SQL expression
representing the statement generated by a particular Query http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 25 of 37
6/27/16, 4:11 PM
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 26 of 37
6/27/16, 4:11 PM
Using EXISTS
The EXISTS keyword in SQL is a boolean operator which returns
True if the given expression contains any rows. It may be used in
many scenarios in place of joins, and is also useful for locating
rows which do not have a corresponding row in a related table.
There is an explicit EXISTS construct, which looks like this:
>>> from sqlalchemy.sql import exists
>>> stmt = exists().where(Address.user_id==User.id)
>>> for name, in session.query(User.name).filter(stmt):
SQL
...
print name
jack
# doc
Page 27 of 37
6/27/16, 4:11 PM
Eager Loading
Recall earlier that we illustrated a lazy loading operation, when
we accessed the User.addresses collection of a User and SQL
was emitted. If you want to reduce the number of queries
(dramatically, in many cases), we can apply an eager load to the
query operation. SQLAlchemy offers three types of eager loading,
two of which are automatic, and a third which involves custom
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 28 of 37
6/27/16, 4:11 PM
Subquery Load
In this case wed like to indicate that User.addresses should
load eagerly. A good choice for loading a set of objects as well as
their related collections is the orm.subqueryload() option,
which emits a second SELECT statement that fully loads the
collections associated with the results just loaded. The name
subquery originates from the fact that the SELECT statement
constructed directly via the Query is re-used, embedded as a
subquery into a SELECT against the related table. This is a little
elaborate but very easy to use:
>>> jack.addresses
[<Address(email_address='[email protected]')>, <Address(email_add
Note
Joined Load
The other automatic eager loading function is more well known
and is called orm.joinedload() . This style of loading emits a
JOIN, by default a LEFT OUTER JOIN, so that the lead object as
well as the related object or collection is loaded in one step. We
illustrate loading the same addresses collection in this way note that even though the User.addresses collection on jack
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 29 of 37
6/27/16, 4:11 PM
is actually populated right now, the query will emit the extra join
regardless:
>>> from sqlalchemy.orm import joinedload
>>> jack.addresses
[<Address(email_address='[email protected]')>, <Address(email_add
Note that even though the OUTER JOIN resulted in two rows, we
still only got one instance of User back. This is because Query
applies a uniquing strategy, based on object identity, to the
returned entities. This is specifically so that joined eager loading
can be applied without affecting the query results.
While joinedload() has been around for a long time,
subqueryload() is a newer form of eager loading.
subqueryload() tends to be more appropriate for loading
related collections while joinedload() tends to be better suited
for many-to-one relationships, due to the fact that only one row
is loaded for both the lead and the related object.
Page 30 of 37
6/27/16, 4:11 PM
Deleting
Lets try to delete jack and see how that goes. Well mark as
deleted in the session, then well issue a count query to see that
no rows remain:
>>> session.delete(jack)
>>> session.query(User).filter_by(name='jack').count()
SQL# doctes
0
>>> session.query(Address).filter(
SQL
...
Address.email_address.in_(['[email protected]', 'j25@yaho
... ).count() # doctest: +NORMALIZE_WHITESPACE
2
Uh oh, theyre still there ! Analyzing the flush SQL, we can see
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 31 of 37
6/27/16, 4:11 PM
that the user_id column of each address was set to NULL, but
the rows werent deleted. SQLAlchemy doesnt assume that
deletes cascade, you have to tell it to do so.
Page 32 of 37
...
...
...
...
...
6/27/16, 4:11 PM
def __repr__(self):
return "<Address(email_address='%s')>" % self.email
Now when we load the user jack (below using get() , which
loads by primary key), removing an address from the
corresponding addresses collection will result in that Address
being deleted:
# load Jack by primary key
>>> jack = session.query(User).get(5)
#doctest: +NORMALIZE_W
SQL
Deleting Jack will delete both Jack and the remaining Address
associated with the user:
>>> session.delete(jack)
>>> session.query(User).filter_by(name='jack').count()
SQL# doctes
0
>>> session.query(Address).filter(
SQL
...
Address.email_address.in_(['[email protected]', 'j25@yahoo
... ).count() # doctest: +NORMALIZE_WHITESPACE
0
More on Cascades
Further detail on configuration of cascades is at Cascades. The cascade
functionality can also integrate smoothly with the ON DELETE CASCADE
functionality of the relational database. See Using Passive Deletes for
details.
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 33 of 37
6/27/16, 4:11 PM
Page 34 of 37
...
6/27/16, 4:11 PM
Note
The above class declarations illustrate explicit __init__() methods.
Remember, when using Declarative, its optional!
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 35 of 37
6/27/16, 4:11 PM
SQL
Usage is not too different from what weve been doing. Lets give
Wendy some blog posts:
>>> wendy = session.query(User).\
SQL
...
filter_by(name='wendy').\
...
one() #doctest: +NORMALIZE_WHITESPACE
>>> post = BlogPost("Wendy's Blog Post", "This is a test",
>>> session.add(post)
We can now look up all blog posts with the keyword firstpost.
Well use the any operator to locate blog posts where any of its
keywords has the keyword string firstpost:
>>> session.query(BlogPost).\
SQL
...
filter(BlogPost.keywords.any(keyword='firstpost
...
all() #doctest: +NORMALIZE_WHITESPACE
[BlogPost("Wendy's Blog Post", 'This is a test', <User(name
>>> session.query(BlogPost).\
SQL
...
filter(BlogPost.author==wendy).\
...
filter(BlogPost.keywords.any(keyword='firstpost
...
all() #doctest: +NORMALIZE_WHITESPACE
[BlogPost("Wendy's Blog Post", 'This is a test', <User(name
Page 36 of 37
6/27/16, 4:11 PM
Further Reference
Query Reference: query_api_toplevel
Mapper Reference: Mapper Configuration
Relationship Reference: Relationship Configuration
Session Reference: Using the Session
Website content copyright by SQLAlchemy authors and contributors. SQLAlchemy and its documentation are
licensed under the MIT license.
SQLAlchemy is a trademark of Michael Bayer. mike(&)zzzcomputing.com All rights reserved.
Website generation by Blogofile and Mako Templates for Python.
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Page 37 of 37