Week 10: ORM and SQLAlchemy

1. ORM Fundamentals

What is ORM?
Object-Relational Mapping (ORM) is a programming technique that allows
developers to interact with relational databases using object-oriented programming
languages. It acts as a bridge between the object-oriented world of application code
and the relational world of databases.
In traditional database interactions, developers write SQL queries to create, read,
update, and delete data. With ORM, these operations are abstracted away, and
developers can work with database records as if they were regular objects in their
programming language of choice.
Key concepts of ORM include:
1. Mapping: ORM tools provide a way to map database tables to classes in your
programming language. Each row in a table corresponds to an instance of the class,
and each column maps to an attribute of that class.
2. Abstraction: ORM abstracts away the underlying database structure, allowing
developers to work with familiar programming constructs rather than raw SQL.
3. Database Agnostic: Many ORM tools support multiple database systems, allowing
developers to switch between different databases with minimal code changes.
4. Query Language: ORMs often provide their own query language or API that
translates object-oriented operations into SQL queries behind the scenes.

Why use ORM?

There are several compelling reasons to use ORM in your applications:
1. Productivity: ORM significantly reduces the amount of code developers need to
write. Instead of crafting SQL statements manually, developers can use high-level
methods and attributes to interact with the database.
2. Maintainability: ORM promotes cleaner, more organized code. Database
operations are centralized in model classes, making it easier to manage and update
data access logic.
3. Database Independence: ORMs often support multiple database systems. This
allows developers to switch between different databases (e.g., from SQLite during
development to PostgreSQL in production) with minimal code changes.
4. Security: Many ORM tools automatically handle SQL injection prevention by
properly escaping and quoting input data.
5. Performance Optimization: ORMs often include built-in optimization techniques
like lazy loading and caching, which can improve application performance.
6. Consistency: ORM enforces a consistent way of interacting with the database
across your application, reducing the likelihood of errors and inconsistencies.
7. Object-Oriented Paradigm: ORM allows developers to work with familiar object-
oriented concepts, making it easier to integrate database operations with the rest of the
application logic.
8. Relationships: ORMs make it easier to work with complex relationships between
tables, such as one-to-many or many-to-many relationships.
9. Validation: Many ORM tools provide built-in validation capabilities, ensuring data
integrity at the application level before it reaches the database.
10. Database Migrations: Some ORM tools offer migration capabilities, making it
easier to evolve your database schema over time as your application grows and
changes.

While ORM offers many advantages, it's important to note that it may not be the best
solution for every scenario. In cases where you need fine-grained control over SQL
queries or are working with extremely large datasets, writing raw SQL might be more
appropriate. However, for many applications, the benefits of ORM far outweigh the
drawbacks.

2. SQLAlchemy Overview
SQLAlchemy is a powerful and flexible ORM library for Python. It provides a full
suite of well-known enterprise-level persistence patterns, designed for efficient and
high-performing database access. SQLAlchemy is known for its ability to handle both
simple and complex database operations with equal ease.
Key features of SQLAlchemy include:
1. SQL Expression Language: A SQL construction API that allows for fluent,
Pythonic construction of SQL statements.
2. ORM: An object-relational mapper that provides data mapper patterns for domain
modeling.
3. Schema/Types: Extensible systems for describing database schemas and custom
types.
4. Connection Pooling: Efficient management of database connections.
5. Dialects: Support for multiple database backends through dialects.

Declarative vs. Core

SQLAlchemy offers two main modes of usage: Declarative and Core. Understanding
the difference between these two approaches is crucial for effectively using
SQLAlchemy.

SQLAlchemy Core
SQLAlchemy Core is a full-featured SQL abstraction toolkit. It provides a Pythonic
way of representing tables, columns, and SQL expressions. Core is closer to SQL and
gives you more control over the generated queries.
Key aspects of Core:
1. Table and Column Constructs: Define database schema using Python objects.
2. SQL Expression Language: Write SQL-like expressions in Python.
3. Connection and Execution: Direct connection to the database for executing
queries.
4. Result Sets: Fetch and process query results.

Example of using Core

python
from sqlalchemy import Table, Column, Integer, String, MetaData, create_engine
metadata = MetaData()
users = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
Column('fullname', String),
)
engine = create_engine('sqlite:///example.db')
metadata.create_all(engine)
# Inserting data
with engine.connect() as conn:
 conn.execute(users.insert().values(name='john', fullname='John Doe'))

# Querying data
with engine.connect() as conn:
result = conn.execute(users.select().where(users.c.name == 'john'))
for row in result:
print(row)

SQLAlchemy ORM (Declarative)

The ORM layer of SQLAlchemy provides a higher level of abstraction. It allows you
to map Python classes to database tables and work with Python objects instead of SQL
statements. The Declarative system is the most common way to use the ORM.

Key aspects of Declarative ORM:

1. Declarative Base: A base class for your models.
2. Mapped Classes: Python classes that represent database tables.
3. Session: Manages persistence operations for ORM-mapped objects.
4. Query API: ORM-level query construction.

Example of using Declarative ORM:

python
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = 'users'

id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)

engine = create_engine('sqlite:///example.db')
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
# Inserting data
new_user = User(name='john', fullname='John Doe')
session.add(new_user)
session.commit()

# Querying data
users = session.query(User).filter_by(name='john').all()
for user in users:
print(user.name, user.fullname)

Choosing between Core and ORM

The choice between Core and ORM depends on your specific needs:

- Use Core when you need fine-grained control over SQL generation, are working
with existing schemas, or need to optimize for performance.
- Use ORM when you want to work with Python objects, need to manage complex
object relationships, or prefer a higher level of abstraction.
In practice, many applications use a combination of both approaches, leveraging the
strengths of each where appropriate.

3. Setting up SQLAlchemy
Setting up SQLAlchemy involves installing the library and configuring it to work with
your chosen database. We'll cover setup for three popular databases: SQLite,
PostgreSQL, and MySQL.

Installing SQLAlchemy
First, install SQLAlchemy using pip:
pip install sqlalchemy

With SQLite
SQLite is a lightweight, file-based database that's great for development and small
applications. It's included in Python's standard library, so no additional driver is
needed.
python
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

# Create an engine that stores data in the local directory's

# sqlalchemy_example.db file.
engine = create_engine('sqlite:///sqlalchemy_example.db')

# Create all tables in the engine. This is equivalent to "Create Table"

# statements in raw SQL.
Base = declarative_base()
Base.metadata.create_all(engine)

# Create a configured "Session" class

Session = sessionmaker(bind=engine)
# Create a Session
session = Session()
# Here you can start using the session to query the database

With PostgreSQL
PostgreSQL is a powerful, open-source object-relational database system. To use
SQLAlchemy with PostgreSQL, you'll need to install the psycopg2 driver:

pip install psycopg2

Then, set up SQLAlchemy:
python
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
# Replace 'username', 'password', 'hostname', and 'database' with your PostgreSQL
credentials
engine = create_engine('postgresql://username:password@hostname/database')
Base = declarative_base()
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()

With MySQL
MySQL is another popular open-source relational database. To use SQLAlchemy with
MySQL, you'll need to install the mysqlclient driver:

pip install mysqlclient

Then, set up SQLAlchemy:

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

# Replace 'username', 'password', 'hostname', and 'database' with your MySQL

credentials
engine = create_engine('mysql://username:password@hostname/database')

Base = declarative_base()
Base.metadata.create_all(engine)

Session = sessionmaker(bind=engine)
session = Session()
In each case, the `create_engine()` function is used to create an `Engine` instance. The
`Engine` is the starting point for any SQLAlchemy application. It's the home base for
the actual database and its DBAPI, delivered to the SQLAlchemy application through
a connection pool and a `Dialect`, which describes how to talk to a specific kind of
database/DBAPI combination.
The `declarative_base()` function is used to create a base class for declarative class
definitions. This base class will serve as a foundation for creating your model classes.
The `sessionmaker()` function creates a custom `Session` class which is bound to our
database. The `Session` establishes all conversations with the database and represents
a "holding zone" for all the objects which you've loaded or associated with it during
its lifespan.
After setting up these components, you're ready to start defining your models and
interacting with the database using SQLAlchemy.

4. Defining Models and Tables using SQLAlchemy

In SQLAlchemy ORM, models are Python classes that represent tables in the
database. These classes are defined using declarative syntax, which allows you to
define tables and models in one go.

Here's a step-by-step guide to defining models and tables:

1. Import necessary modules

from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
2. Define your models
Let's create two related models: `User` and `Address`.
class User(Base):
__tablename__ = 'users'

id = Column(Integer, primary_key=True)
name = Column(String(50))
fullname = Column(String(50))
email = Column(String(50))
addresses = relationship("Address", back_populates="user")
def __repr__(self):
return f"<User(name='{self.name}', fullname='{self.fullname}',
email='{self.email}')>"
class Address(Base):
__tablename__ = 'addresses'

id = Column(Integer, primary_key=True)
email_address = Column(String(50), nullable=False)
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship("User", back_populates="addresses")
def __repr__(self):
return f"<Address(email_address='{self.email_address}')>"

Let's break down these model definitions:

- `__tablename__`: This specifies the name of the table in the database.
- `Column`: This defines a column in the table. The first argument is the column type.
- `primary_key=True`: This indicates that this column is the primary key.
- `ForeignKey`: This establishes a foreign key relationship to another table.
- `relationship`: This sets up the relationship between the two models.
- `__repr__`: This method provides a string representation of the object, useful for
debugging.

3. Create the tables in the database

After defining your models, you need to create the corresponding tables in the
database. You can do this using the `create_all()` method:

from sqlalchemy import create_engine

engine = create_engine('sqlite:///example.db')
Base.metadata.create_all(engine)
This will create all tables that don't already exist in the database.

4. Using the models

Now that you've defined your models, you can use them to interact with the database.
Here's a quick example:
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session()
# Create a new user
new_user = User(name='john', fullname='John Doe', email='[email protected]')
session.add(new_user)
session.commit()
# Add an address for the user
new_address = Address(email_address='[email protected]', user=new_user)
session.add(new_address)
session.commit()
# Query the database
user = session.query(User).filter_by(name='john').first()
print(user)
print(user.addresses)
This example demonstrates how to create new records, establish relationships between
them, and query the database using the models we defined.
Some additional points to consider when defining models:

- Column Types: SQLAlchemy provides many column types like `Integer`, `String`,
`Boolean`, `DateTime`, `Float`, etc. Choose the appropriate type for your data.
- Constraints: You can add constraints to your columns, such as `nullable=False` for
required fields, or `unique=True` for unique values.
- Indexes: You can create indexes on your tables for better query performance using
the `Index` class.
- Table Arguments: You can pass additional arguments to the table constructor using
the `__table_args__` attribute.
By properly defining your models, you create a powerful abstraction layer that allows
you to work with your database using Python objects, greatly simplifying database
operations in your application.

5. Performing CRUD Operations using SQLAlchemy ORM

CRUD stands for Create, Read, Update, and Delete - the four basic operations
performed on database records. SQLAlchemy ORM provides a high-level, Pythonic
way to perform these operations. Let's go through each operation using the `User`
model we defined earlier.
Create
To create a new record, you instantiate your model class and add it to the session:
# Create a new session
Session = sessionmaker(bind=engine)
session = Session()
# Create a new user
new_user = User(name='alice', fullname='Alice Wonderland',
email='[email protected]')
# Add the new user to the session
session.add(new_user)
# Commit the transaction
session.commit()
You can also add multiple objects at once using `session.add_all([obj1, obj2, ...])`.

Read
SQLAlchemy provides several ways to query the database:

1. Query all records:

python
users = session.query(User).all()
for user in users:
print(user)
2. Query with filters:
python
user = session.query(User).filter_by(name='alice').first()
print(user)
3. More complex queries:
python
users = session.query(User).filter(User.name.like('a%')).order_by(User.id).all()
for user in users:
print(user)

Update
To update a record, you first query for the object, modify its attributes, and then
commit the session:
user = session.query(User).filter_by(name='alice').first()
user.email = '[email protected]'
session.commit()
You can also update multiple records at once:
session.query(User).filter(User.name.like('a%')).update({"fullname": "Updated
Name"}, synchronize_session=False)
session.commit()

Delete
To delete a record, you can either delete a specific object or delete based on a query:
1. Delete a specific object:
python
user = session.query(User).filter_by(name='alice').first()
session.delete(user)
session.commit()

2. Delete based on a query:

python
session.query(User).filter_by(name='bob').delete()
session.commit()
Remember to always commit your changes to persist them to the database. If you
want to roll back changes, you can use `session.rollback()`.

6. Relationships in SQLAlchemy
Relationships are a key feature of relational databases, and SQLAlchemy provides
powerful tools for working with them. There are three main types of relationships:
One-to-One, One-to-Many, and Many-to-Many. Let's explore each type using
examples.

One-to-One Relationships
A one-to-one relationship exists when one record in a table is associated with one and
only one record in another table. For example, let's consider a `User` and a `Profile`
where each user has exactly one profile.

python
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
profile = relationship("Profile", uselist=False, back_populates="user")
class Profile(Base):
__tablename__ = 'profiles'

id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('users.id'), unique=True)
bio = Column(String(200))
user = relationship("User", back_populates="profile")
In this example:
- `uselist=False` in the `User` class ensures that `profile` is treated as a scalar (single
object) rather than a list.
- `unique=True` on `Profile.user_id` ensures that each user can have only one profile.

Usage:
python
user = User(name="John Doe")
profile = Profile(bio="A software developer")
user.profile = profile
session.add(user)
session.commit()
# Accessing the relationship
print(user.profile.bio) # Output: A software developer
print(profile.user.name) # Output: John Doe
One-to-Many Relationships
A one-to-many relationship exists when a record in one table can be associated with
multiple records in another table. For example, a `User` can have multiple `Post`s.
python
class User(Base):
__tablename__ = 'users'

id = Column(Integer, primary_key=True)
name = Column(String(50))
posts = relationship("Post", back_populates="author")
class Post(Base):
__tablename__ = 'posts'

id = Column(Integer, primary_key=True)
title = Column(String(100))
content = Column(String(500))
author_id = Column(Integer, ForeignKey('users.id'))
author = relationship("User", back_populates="posts")

Usage:
python
user = User(name="Jane Smith")
post1 = Post(title="First Post", content="Hello, world!")
post2 = Post(title="Second Post", content="Another post")

user.posts.extend([post1, post2])
session.add(user)
session.commit()
# Accessing the relationship
for post in user.posts:
print(f"{post.title} by {post.author.name}")

Many-to-Many Relationships
A many-to-many relationship exists when multiple records in one table are associated
with multiple records in another table. This typically involves a third, intermediate
table. For example, let's consider `Student`s and `Course`s, where a student can enroll
in multiple courses and a course can have multiple students.

python
student_course = Table('student_course', Base.metadata,
Column('student_id', Integer, ForeignKey('students.id')),
Column('course_id', Integer, ForeignKey('courses.id'))
)
class Student(Base):
__tablename__ = 'students'

id = Column(Integer, primary_key=True)
name = Column(String(50))
courses = relationship("Course", secondary=student_course,
back_populates="students")
class Course(Base):
__tablename__ = 'courses'

id = Column(Integer, primary_key=True)
name = Column(String(50))
students = relationship("Student", secondary=student_course,
back_populates="courses")

In this example, `student_course` is the association table that links students and
courses.
Usage:
python
student1 = Student(name="Alice")
student2 = Student(name="Bob")
course1 = Course(name="Mathematics")
course2 = Course(name="Physics")
student1.courses.extend([course1, course2])
student2.courses.append(course1)
session.add_all([student1, student2, course1, course2])
session.commit()
# Accessing the relationship
for student in course1.students:
print(f"{student.name} is enrolled in {course1.name}")

for course in student1.courses:

print(f"{student1.name} is taking {course.name}")

Eager Loading
When working with relationships, it's important to be aware of the "N+1 problem,"
where accessing related objects can result in multiple database queries. SQLAlchemy
provides eager loading strategies to mitigate this:

1. Joined Loading: Loads the related objects in the same query using JOIN.
users = session.query(User).options(joinedload(User.posts)).all()
2. Subquery Loading: Loads the related objects using a separate query.
users = session.query(User).options(subqueryload(User.posts)).all()
3. Selectin Loading: Similar to subquery loading but uses IN clauses.
users = session.query(User).options(selectinload(User.posts)).all()
Choose the appropriate loading strategy based on your specific use case and
performance requirements.