Django

Introduction
App Overview
By the end of this course, you will have built a RESTful API, using Test-Driven Development.
The API itself will follow RESTful design principles, using the basic HTTP verbs: GET, POST,
PUT, and DELETE.

Endpoint HTTP Method CRUD Method Result

/api/movies GET READ get all movies
/api/movies/:id GET READ get a single movie
/api/movies POST CREATE add a movie
/api/movies/:id PUT UPDATE update a movie
/api/movies/:id DELETE DELETE delete a movie

Along with Python and Django, we'll use Docker to quickly set up our local development
environment and simplify deployment and Django REST Framework (DRF) to develop a
RESTful API. We'll use pytest instead of unittest for writing unit and integration tests to test
the Django API. Finally, we'll store the code on a GitLab repository and utilize the Continuous
Integration (CI) features in GitLab to run tests before deploying to Heroku.
Before diving in, let's take a minute to go over why some of the above tools are being used.

Django
Django and Flask are the two most popular Python web frameworks designed for building
web applications. While they are both free and open-source, Django is older and more mature
and it has a larger community supporting it. It's also more opinionated than Flask, so it
requires fewer decisions to be made by you or your team. You can probably move faster that
way as long as you don't have any strong disagreements with one of the choices that Django
makes for you or you have unique application requirements that limit the number of features
you can take advantage of.

Info

For more, review Django vs. Flask: Which Framework to Choose.

Django REST Framework

Django REST Framework (DRF) is a widely-used, full-featured API framework designed for
building RESTful APIs with Django. At its core, DRF integrates with Django's main features -
- models, views, and URLs -- making it simple and seamless to create RESTful HTTP
resources.

Docker
Docker is a container platform used to streamline application development and deployment
workflows across various environments. It's used to package application code, dependencies,
and system tools into lightweight containers that can be moved from development machines
to production servers quickly and easily.

Pytest
pytest is a test framework for Python that makes it easy (and fun!) to write, organize, and run
tests. When compared to unittest, from the Python standard library, pytest:

1. Requires less boilerplate code so your test suites will be more readable.
2. Supports the plain assert statement, which is far more readable and easier to
remember compared to the assertSomething methods --
like assertEquals , assertTrue , and assertContains -- in unittest.
3. Is updated more frequently since it's not part of the Python standard library.
4. Simplifies setting up and tearing down test state with its fixture system.
5. Uses a functional approach.

GitLab
GitLab is a web-based solution for managing the full software development lifecycle. Along
with source code management, GitLab provides a number of project management and
DevOps-related services, like Kanban boards, package management, logging and monitoring,
continuous integration and delivery, secrets management, and container orchestration.
For more, review GitLab's DevOps Tools Landscape.

Getting Started
In this chapter, we'll set up the base project structure.
Start by creating a new working directory, installing Django and Django REST Framework,
and creating a new Django project and app:

$ mkdir django-tdd-docker && cd django-tdd-docker

$ mkdir app && cd app
$ python3.10 -m venv env
$ source env/bin/activate

(env)$ pip install django==4.0 djangorestframework==3.13.1

(env)$ django-admin startproject drf_project .
(env)$ python manage.py startapp movies

Info

Feel free to swap out virtualenv and Pip for Poetry or Pipenv. For more, review Modern
Python Environments.

Add the apps to the INSTALLED_APPS setting in the settings.py file:

# app/drf_project/settings.py

INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
 'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework', # new
'movies', # new
]

Next, before applying the first migration, let's create a custom User model.

Info

It's a good idea to configure a custom User model when starting a new Django project.
For more, review Creating a Custom User Model in Django.

Add the following to the bottom of settings.py to make Django reference a User model of our
design:

# app/drf_project/settings.py

AUTH_USER_MODEL = 'movies.CustomUser'

Next, define the custom User model in the movies/models.py file:

# app/movies/models.py

from django.contrib.auth.models import AbstractUser

class CustomUser(AbstractUser):
pass

Here, we created a new class called CustomUser that subclasses AbstractUser .

That's it for now. Since we're not making any changes just yet to the User model, we won't
make any updates to the Django admin app.
Go ahead and create the migration file and apply all the migrations to the database:

(env)$ python manage.py makemigrations

(env)$ python manage.py migrate

To verify that CustomUser is being used in place of User , you can view the schema within the
SQLite shell:

$ sqlite3 db.sqlite3

SQLite version 3.28.0 2019-04-15 14:49:49

Enter ".help" for usage hints.

sqlite> .tables

auth_group django_migrations
auth_group_permissions django_session
auth_permission movies_customuser
django_admin_log movies_customuser_groups
django_content_type movies_customuser_user_permissions
 sqlite> .schema movies_customuser

CREATE TABLE IF NOT EXISTS "movies_customuser" (

"id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
"password" varchar(128) NOT NULL,
"last_login" datetime NULL,
"is_superuser" bool NOT NULL,
"username" varchar(150) NOT NULL UNIQUE,
"first_name" varchar(30) NOT NULL,
"last_name" varchar(150) NOT NULL,
"email" varchar(254) NOT NULL,
"is_staff" bool NOT NULL,
"is_active" bool NOT NULL,
"date_joined" datetime NOT NULL
);

sqlite> .exit

Create a new superuser and run the Django development server:

(env)$ python manage.py createsuperuser

(env)$ python manage.py runserver

Navigate to http://localhost:8000/ within your browser of choice to view the Django welcome
screen. Make sure you can also log in to the Django admin
at http://localhost:8000/admin/ with your superuser credentials. Kill the server once done.
Exit then remove the virtual environment as well.
Then, add a requirements.txt file to the "app" directory:

Django==4.0
djangorestframework==3.13.1

Add a .gitignore to the project root:

__pycache__
env
*.sqlite3

Your directory structure should look like this:

├── .gitignore
└── app
├── db.sqlite3
├── drf_project
│ ├── __init__.py
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── manage.py
├── movies
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
 │ │ ├── 0001_initial.py
│ │ └── __init__.py
│ ├── models.py
│ ├── tests.py
│ └── views.py
└── requirements.txt

Init a git repo and commit your code.

Docker Config
Let's containerize the Django app.
Start by ensuring that you have Docker and Docker Compose:

$ docker -v
Docker version 20.10.11, build dea9396

$ docker-compose -v
Docker Compose version v2.2.1

Info

Make sure to install or upgrade them if necessary.

Add a Dockerfile to the "app" directory, making sure to review the code comments:

# pull official base image

FROM python:3.10.1-slim-buster

# set working directory

WORKDIR /usr/src/app

# set environment variables

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# add app
COPY . .

Here, we started with a slim-buster-based Docker image for Python 3.10.1. We then set a
working directory along with two environment variables:

1. PYTHONDONTWRITEBYTECODE : Prevents Python from writing pyc files to disc (equivalent

to python -B option)
2. PYTHONUNBUFFERED : Prevents Python from buffering stdout and stderr (equivalent
to python -u option)

Finally, we updated Pip, copied over the requirements.txt file, installed the dependencies, and
copied over the Django project itself.
 Info

Depending on your environment, you may need to add RUN mkdir -p

/usr/src/app just before you set the working directory:

RUN mkdir -p /usr/src/app

WORKDIR /usr/src/app

Add a .dockerignore file to the "app" directory as well:

env
.dockerignore
Dockerfile
Dockerfile.prod

Like the .gitignore file, the .dockerignore file lets you exclude specific files and folders from
being copied over to the image.

Info

Review Docker Best Practices for Python Developers for more on structuring Dockerfiles
as well as some best practices for configuring Docker for Python-based development.

Then add a docker-compose.yml file to the project root:

version: '3.8'

services:
movies:
build: ./app
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./app/:/usr/src/app/
ports:
- 8009:8000
env_file:
- ./app/.env.dev

This config will create a service called movies from the Dockerfile.

The volume is used to mount the code into the container. This is a must for a development
environment in order to update the container whenever a change to the source code is made.
Without this, you would have to re-build the image each time you make a change to the code.

Take note of the Docker compose file version used -- 3.8 . Keep in mind that this version
does not directly relate back to the version of Docker Compose installed; it simply specifies the
file format that you want to use.

Info

Review the Compose file reference for info on how this file works.
Update the SECRET_KEY , DEBUG , and ALLOWED_HOSTS variables in settings.py:

# app/drf_project/settings.py

SECRET_KEY = os.environ.get("SECRET_KEY")

DEBUG = int(os.environ.get("DEBUG", default=0))

# 'DJANGO_ALLOWED_HOSTS' should be a single string of hosts with a space

between each.
# For example: 'DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]'
ALLOWED_HOSTS = os.environ.get("DJANGO_ALLOWED_HOSTS").split(" ")

Add the import:

import os

Then, create a .env.dev file in the "app" directory to store environment variables for
development:

DEBUG=1
SECRET_KEY=foo
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]

Build the image:

$ docker-compose build

This will take a few minutes the first time. Subsequent builds will be much faster since Docker
caches the results. If you'd like to learn more about Docker caching, review the Order
Dockerfile Commands Appropriately section.

Once the build is done, fire up the container in detached mode:

$ docker-compose up -d

Navigate to http://localhost:8009/ to again view the welcome screen. Check for errors in the
logs if this doesn't work via docker-compose logs -f .

Info

If you run into problems with the volume mounting correctly, you may want to remove it
altogether by deleting the volume config from the Docker Compose file. You can still go
through the course without it; you'll just have to re-build the image after you make
changes to the source code.

Windows Users: Having problems getting the volume to work properly? Review the
following resources:

1. Docker on Windows—Mounting Host Directories

2. Configuring Docker for Windows Shared Drives

You also may need to add COMPOSE_CONVERT_WINDOWS_PATHS=1 to

the environment portion of your Docker Compose file. Review Declare default
 environment variables in file for more info.

Bring down the development containers (and the associated volumes with the -v flag):

$ docker-compose down -v

Since we'll be moving to Postgres, go ahead and remove the db.sqlite3 file from the "app"
directory.

├── .gitignore
├── app
│ ├── .dockerignore
│ ├── .env.dev
│ ├── Dockerfile
│ ├── drf_project
│ │ ├── __init__.py
│ │ ├── asgi.py
│ │ ├── settings.py
│ │ ├── urls.py
│ │ └── wsgi.py
│ ├── manage.py
│ ├── movies
│ │ ├── __init__.py
│ │ ├── admin.py
│ │ ├── apps.py
│ │ ├── migrations
│ │ │ ├── 0001_initial.py
│ │ │ └── __init__.py
│ │ ├── models.py
│ │ ├── tests.py
│ │ └── views.py
│ └── requirements.txt
└── docker-compose.yml

Postgres Setup
In this chapter, we'll configure Postgres, get it up and running in another container, and link it
to the movies service.
To configure Postgres, we'll need to add a new service to the docker-compose.yml file, update
the Django settings, and install Psycopg2.

First, add a new service called movies-db to docker-compose.yml:

version: '3.8'

services:
movies:
build: ./app
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./app/:/usr/src/app/
ports:
- 8009:8000
env_file:
- ./app/.env.dev
depends_on:
 - movies-db
movies-db:
image: postgres:14-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=movies
- POSTGRES_PASSWORD=movies
- POSTGRES_DB=movies_dev

volumes:
postgres_data:

To persist the data beyond the life of the container we configured a volume. This config will
bind postgres_data to the "/var/lib/postgresql/data/" directory in the container.
We also added an environment key to define a name for the default database and set a
username and password.

Info

Review the "Environment Variables" section of the Postgres Docker Hub page for more
info.

Once spun up, Postgres will be available on port 5432 for services running in other
containers.
We'll need some new environment variables for the movies service as well, so
update .env.dev like so:

DEBUG=1
SECRET_KEY=foo
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
SQL_ENGINE=django.db.backends.postgresql
SQL_DATABASE=movies_dev
SQL_USER=movies
SQL_PASSWORD=movies
SQL_HOST=movies-db
SQL_PORT=5432

Update the DATABASES dict in settings.py:

# app/drf_project/settings.py

DATABASES = {
"default": {
"ENGINE": os.environ.get("SQL_ENGINE",
"django.db.backends.sqlite3"),
"NAME": os.environ.get("SQL_DATABASE", os.path.join(BASE_DIR,
"db.sqlite3")),
"USER": os.environ.get("SQL_USER", "user"),
"PASSWORD": os.environ.get("SQL_PASSWORD", "password"),
"HOST": os.environ.get("SQL_HOST", "localhost"),
"PORT": os.environ.get("SQL_PORT", "5432"),
}
}
Here, the database is configured based on the environment variables that we just defined.
Take note of the default values.

Update the Dockerfile to install the appropriate packages required for Psycopg2:

# pull official base image

FROM python:3.10.1-slim-buster

# set working directory

WORKDIR /usr/src/app

# set environment variables

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# new
# install system dependencies
RUN apt-get update \
&& apt-get -y install gcc postgresql \
&& apt-get clean

# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# add app
COPY . .

Add Psycopg2 to the requirements.txt file:

Django==4.0
djangorestframework==3.13.1
psycopg2-binary==2.9.2

Build the new image and spin up the two containers:

$ docker-compose up -d --build

Run the migrations:

$ docker-compose exec movies python manage.py migrate --noinput

Info

Get the following error?

django.db.utils.OperationalError: FATAL: database "movies_dev" does not exist

Run docker-compose down -v to remove the volumes along with the containers. Then,
re-build the images, run the containers, and apply the migrations.

Ensure the default Django tables were created:

 $ docker-compose exec movies-db psql --username=movies --dbname=movies_dev

psql (14.1)
Type "help" for help.

movies_dev=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access
privileges
------------+--------+----------+------------+------------+-----------------
--
movies_dev | movies | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | movies | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | movies | UTF8 | en_US.utf8 | en_US.utf8 | =c/movies
+
| | | | |
movies=CTc/movies
template1 | movies | UTF8 | en_US.utf8 | en_US.utf8 | =c/movies
+
| | | | |
movies=CTc/movies
(4 rows)

movies_dev=# \c movies_dev
You are now connected to database "movies_dev" as user "movies".

movies_dev=# \dt
List of relations
Schema | Name | Type | Owner
--------+------------------------------------+-------+--------
public | auth_group | table | movies
public | auth_group_permissions | table | movies
public | auth_permission | table | movies
public | django_admin_log | table | movies
public | django_content_type | table | movies
public | django_migrations | table | movies
public | django_session | table | movies
public | movies_customuser | table | movies
public | movies_customuser_groups | table | movies
public | movies_customuser_user_permissions | table | movies
(10 rows)

movies_dev=# \q

You can check that the volume was created as well by running:

$ docker volume inspect django-tdd-docker_postgres_data

You should see something similar to:

[
{
"CreatedAt": "2021-12-16T16:14:14Z",
"Driver": "local",
"Labels": {
"com.docker.compose.project": "django-tdd-docker",
"com.docker.compose.version": "2.2.1",
"com.docker.compose.volume": "postgres_data"
 },
"Mountpoint": "/var/lib/docker/volumes/django-tdd-
docker_postgres_data/_data",
"Name": "django-tdd-docker_postgres_data",
"Options": null,
"Scope": "local"
}
]

Since the movies service is dependent not only on the container being up and running but
also the actual Postgres instance being up and healthy, let's add an entrypoint.sh file to the
"app" directory:

#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
echo "Waiting for postgres..."

while ! nc -z $SQL_HOST $SQL_PORT; do

sleep 0.1
done

echo "PostgreSQL started"

fi

python manage.py flush --no-input

python manage.py migrate

exec "$@"

So, we referenced the Postgres container using the name of the service, movies-db , which is
defined by the SQL_HOST environment variable. The loop continues until something
like Connection to movies-db port 5432 [tcp/postgresql] succeeded! is returned.

Update the file permissions locally:

$ chmod +x app/entrypoint.sh

Depending on your environment, you may need to chmod 755 or 777 instead of +x. If you
still get a "permission denied", review the docker entrypoint running bash script gets
"permission denied" Stack Overflow question.

Then, update the Dockerfile to install the required dependency, copy over
the entrypoint.sh file, and run it as the Docker entrypoint command:

# pull official base image

FROM python:3.10.1-slim-buster

# set working directory

WORKDIR /usr/src/app

# set environment variables

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# updated
 # install system dependencies
RUN apt-get update \
&& apt-get -y install netcat gcc postgresql \
&& apt-get clean

# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# new
# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh

# add app
COPY . .

# new
# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"

Add the DATABASE environment variable to .env.dev:

DEBUG=1
SECRET_KEY=foo
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
SQL_ENGINE=django.db.backends.postgresql
SQL_DATABASE=movies_dev
SQL_USER=movies
SQL_PASSWORD=movies
SQL_HOST=movies-db
SQL_PORT=5432
DATABASE=postgres

Sanity check:

$ docker-compose up -d --build

Ensure http://localhost:8009/ still works.

Info

Despite adding Postgres, we can still create an independent Docker image for Django as
long as the DATABASE environment variable is not set to postgres . To test, build a new
image and then run a new container:

$ docker build -f ./app/Dockerfile -t hello_django:latest ./app

$ docker run -p 8001:8000 \
-e "SECRET_KEY=please_change_me" -e "DEBUG=1" -e
"DJANGO_ALLOWED_HOSTS=*" \
hello_django python /usr/src/app/manage.py runserver 0.0.0.0:8000

You should be able to view the welcome page at http://localhost:8001.

Pytest Setup
Let's get our tests up and running with pytest.

Setup
Start by adding pytest and pytest-django, which provides a number of useful tools for using
pytest with Django, to the requirements file:

Django==4.0
djangorestframework==3.13.1
psycopg2-binary==2.9.2
pytest-django==4.5.2
pytest==6.2.5

For this project, let's keep all of our tests together in a single "tests" directory divided by
Django app. Create a new directory in "app" called "tests", and then create the following files
and directories inside "tests":

└── tests
├── __init__.py
└── movies
└── __init__.py

By default, pytest will autodiscover test files that start or end with test --
e.g., test_*.py or *_test.py . Test functions must begin with test_ , and if you want to use
classes they must also begin with Test .

Example:

# if a class is used, it must begin with Test

class TestFoo:

# test functions must begin with test_

def test_bar(self):
assert "foo" != "bar"

Info

If this is your first time with pytest be sure to review the Installation and Getting
Started guide.

Next, add a pytest.ini file, to the "app" directory, to define

the DJANGO_SETTINGS_MODULE environment variable to point to the Django settings file
and the standard test discovery rules (for explicitness):

[pytest]
DJANGO_SETTINGS_MODULE = drf_project.settings

# -- recommended but optional:

python_files = tests.py test_*.py *_tests.py
First Tests
Create a new test file in "tests" called test_foo.py:

# app/tests/test_foo.py

def test_hello_world():
assert "hello_world" == "hello_world"
assert "foo" != "bar"

While unittest requires test classes, pytest just requires functions to get up and running. In
other words, pytest tests are just functions that either start or end with test .

You can still use classes to organize tests in pytest if that's your preferred pattern.

To run the test, we first need to re-build the Docker images since requirements are installed at
build time rather than run time:

$ docker-compose up -d --build

With the containers up and running, run the tests:

$ docker-compose exec movies pytest

You should see:

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 1 item

tests/test_foo.py .
[100%]

========================================= 1 passed in 0.23s

=========================================

Next, let's create a quick Django view that we can easily test.

Add a views.py file to the "drf_project" directory:

# app/drf_project/views.py

from django.http import JsonResponse

def ping(request):
data = {"ping": "pong!"}
return JsonResponse(data)

Next, update urls.py in "drf_project":

 # app/drf_project/urls.py

from django.contrib import admin

from django.urls import path

from .views import ping

urlpatterns = [
path('admin/', admin.site.urls),
path('ping/', ping, name="ping"),
]

Verify http://localhost:8009/ping/ works as expected in your browser:

{
"ping": "pong!"
}

Update the tests:

# app/tests/test_foo.py

import json

from django.urls import reverse

def test_hello_world():
assert "hello_world" == "hello_world"
assert "foo" != "bar"

def test_ping(client):
url = reverse("ping")
response = client.get(url)
content = json.loads(response.content)
assert response.status_code == 200
assert content["ping"] == "pong!"

Info

client is a pytest-django helper fixture that provides an instance of django.test.Client.

Does it pass?

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 2 items
 tests/test_foo.py ..

Fixtures
Although, we won't be using them just yet, it's important to understand pytest fixtures.
For example, if you wanted to use a different database during test runs, you could add
a conftest.py file to the "app" directory with the following code:

import os
from django.conf import settings
import pytest

DEFAULT_ENGINE = "django.db.backends.postgresql_psycopg2"

@pytest.fixture(scope="session")
def django_db_setup():
settings.DATABASES["default"] = {
"ENGINE": os.environ.get("DB_TEST_ENGINE", DEFAULT_ENGINE),
"HOST": os.environ["DB_TEST_HOST"],
"NAME": os.environ["DB_TEST_NAME"],
"PORT": os.environ["DB_TEST_PORT"],
"USER": os.environ["DB_TEST_USER"],
"PASSWORD": os.environ["DB_TEST_PASSWORD"],
}

Fixtures are reusable objects for tests. They have a scope associated with them, which
indicates how often the fixture is invoked:

1. function - once per test function (default)

2. class - once per test class
3. module - once per test module
4. session - once per test session

Info
Check out All You Need to Know to Start Using Fixtures in Your pytest Code for
more on pytest fixtures.
Fixtures are perfect for setting up and tearing down resources:

@pytest.fixture(scope="module")
def foo():
# set up code
yield "bar"
# tear down code

In essence, all code before the yield statement serves as setup code while everything after
serves as the teardown.

Info

For more on this review Fixture finalization / executing teardown code.

Given-When-Then
When writing tests, try to follow the Given-When-Then framework to help make the process
of writing tests easier and faster. It also helps communicate the purpose of your tests better so
it should be easier to read by your future self and others.

State Explanation Code

Given the state of the application before the test setup code, fixtures, database
runs state
When the behavior/logic being tested code under test
Then the expected changes based on the behavior asserts

Example:

def test_ping(client):
# Given
# client

# When
url = reverse("ping")
response = client.get(url)
content = json.loads(response.content)

# Then
assert response.status_code == 200
assert content["ping"] == "pong!"

Django REST Framework

With pytest in place, let's add Django REST Framework and start building out the endpoints.

Architecture
Django REST Framework (DRF) is a widely-used, full-featured API framework designed for
building RESTful APIs. If you need to build a RESTful API with Django, it's the package to
use.

DRF is composed of the following components:

1. Serializers are used to convert Django querysets and model instances to (serialization)
and from (deserialization) JSON (and a number of other data rendering formats like
XML and YAML).
2. Views (along with ViewSets), which are similar to traditional Django views, handle
HTTP requests and return the serialized data. The view itself uses the serializers to
validate incoming payloads and contains the necessary logic to return the response.
Views are coupled with routers, which map the views back to the exposed URLs.

Info

Unfamiliar with Django REST Framework? Check out the official Django REST
Framework tutorial
Model
First, we need to create a model that represents the concept of a movie. Before adding the
actual model, since we're following Test-Driven Development, create a new test file in
"app/tests/movies" called test_models.py:

# app/tests/movies/test_models.py

import pytest

from movies.models import Movie

@pytest.mark.django_db
def test_movie_model():
movie = Movie(title="Raising Arizona", genre="comedy", year="1987")
movie.save()
assert movie.title == "Raising Arizona"
assert movie.genre == "comedy"
assert movie.year == "1987"
assert movie.created_date
assert movie.updated_date
assert str(movie) == movie.title

By default, pytest-django takes a conservative approach to enabling database access in your

tests. In other words, you must explicitly request database access via
the @pytest.mark.django_db decorator.

Ensure the test fails:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 2 items / 1 error / 1 selected

============================================== ERRORS
===============================================
___________________________ ERROR collecting tests/movies/test_models.py
____________________________
ImportError while importing test module
'/usr/src/app/tests/movies/test_models.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.10/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/movies/test_models.py:3: in <module>
from movies.models import Movie
E ImportError: cannot import name 'Movie' from 'movies.models'
(/usr/src/app/movies/models.py)
========================================= 1 error in 0.48s
==========================================
Add the model:

# app/movies/models.py

from django.contrib.auth.models import AbstractUser

from django.db import models

class CustomUser(AbstractUser):
pass

class Movie(models.Model):
title = models.CharField(max_length=255)
genre = models.CharField(max_length=255)
year = models.CharField(max_length=4)
created_date = models.DateTimeField(auto_now_add=True)
updated_date = models.DateTimeField(auto_now=True)

def __str__(self):
return f"{self.title}"

Create the migration:

$ docker-compose exec movies python manage.py makemigrations

Ensure the tests pass:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 3 items

tests/movies/test_models.py .
[ 33%]
tests/test_foo.py ..
[100%]

========================================= 3 passed in 0.93s

=========================================

Apply the migrations:

$ docker-compose exec movies python manage.py migrate

Admin
With the model configured, let's set up the corresponding admin page:

# app/movies/admin.py
 from django.contrib import admin
from django.contrib.auth.admin import UserAdmin as DefaultUserAdmin

from .models import Movie, CustomUser

@admin.register(CustomUser)
class UserAdmin(DefaultUserAdmin):
pass

@admin.register(Movie)
class MovieAdmin(admin.ModelAdmin):
fields = (
"title", "genre", "year", "created_date", "updated_date",
)
list_display = (
"title", "genre", "year", "created_date", "updated_date",
)
readonly_fields = (
"created_date", "updated_date",
)

Create a superuser account:

$ docker-compose exec movies python manage.py createsuperuser

Log in to the admin and add a few new movies.

django admin

Serializer
Again, serializers translate model data to and from JSON. In other words, the serializer
deserializes data from a payload sent with an HTTP request, validates it, and converts it into
something Django can work with. When a user requests data via an HTTP GET request, the
serializer then serializes data from the database into JSON to be sent back via the HTTP
response.

Starting with a test, create a new file called test_serializers.py and add the following code:
 # app/tests/movies/test_serializers.py

from movies.serializers import MovieSerializer

def test_valid_movie_serializer():
valid_serializer_data = {
"title": "Raising Arizona",
"genre": "comedy",
"year": "1987"
}
serializer = MovieSerializer(data=valid_serializer_data)
assert serializer.is_valid()
assert serializer.validated_data == valid_serializer_data
assert serializer.data == valid_serializer_data
assert serializer.errors == {}

def test_invalid_movie_serializer():
invalid_serializer_data = {
"title": "Raising Arizona",
"genre": "comedy"
}
serializer = MovieSerializer(data=invalid_serializer_data)
assert not serializer.is_valid()
assert serializer.validated_data == {}
assert serializer.data == invalid_serializer_data
assert serializer.errors == {"year": ["This field is required."]}

Run the tests:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 3 items / 1 error / 2 selected

============================================== ERRORS
===============================================
_________________________ ERROR collecting tests/movies/test_serializers.py
_________________________
ImportError while importing test module
'/usr/src/app/tests/movies/test_serializers.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.10/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/movies/test_serializers.py:1: in <module>
from movies.serializers import MovieSerializer
E ModuleNotFoundError: No module named 'movies.serializers'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.45s
==========================================
Create a serializers.py file:

# app/movies/serializers.py

from rest_framework import serializers

from .models import Movie

class MovieSerializer(serializers.ModelSerializer):

class Meta:
model = Movie
fields = '__all__'
read_only_fields = ('id', 'created_date', 'updated_date',)

Here, we created a new class called MovieSerializer from a ModelSerializer which outputs
all of the fields from the model. By identifying certain fields as "read only", we can ensure that
they will never be created or updated via the serializer.

Info

DRF serializers are most often coupled with a Django model via a ModelSerializer . In
other words, serializers tend to be closely related to the models they represent. That said,
serializers are used to convert any Python data structure into a format consumable by a
RESTful API, so they do not have to be tied to a model.

Ensure the test now passes:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 5 items

tests/movies/test_models.py .
[ 20%]
tests/test_foo.py ..
[ 60%]
tests/movies/test_serializers.py ..
[100%]

========================================= 5 passed in 0.98s

=========================================

Info

For more on DRF serializers, check out the Effectively Using Django REST Framework
Serializers article.
Selecting Tests
You can select specific tests to run using substring matching.
For example, to run all tests that have models in their names:

$ docker-compose exec movies pytest -k models

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 5 items / 4 deselected / 1 selected

tests/movies/test_models.py .
[100%]

================================== 1 passed, 4 deselected in 0.94s

==================================

So, you can see that the one test in test_models.py ran (and passed) while the remaining tests
from test_serializers.py and test_foo.py were skipped.
Which test(s) will run when you run this command:
$ docker-compose exec movies pytest -k hello_world
Now that we have the model and serializer set up, we can combine them together with URLs
and DRF views to create our endpoints in order to handle HTTP requests.

RESTful Routes
Next, let's set up three endpoints, following RESTful best practices, with TDD:

Endpoint HTTP Method CRUD Method Result

/api/movies GET READ get all movies
/api/movies/:id GET READ get a single movie
/api/movies POST CREATE add a movie

For each, we'll:

1. write a test
2. run the test, to ensure it fails (red)
3. write just enough code to get the test to pass (green)
4. refactor (if necessary)

Add a Movie
Test

First, add a new file called test_views.py to "app/tests/movies":

# app/tests/movies/test_views.py

import json
 import pytest
from movies.models import Movie

@pytest.mark.django_db
def test_add_movie(client):
movies = Movie.objects.all()
assert len(movies) == 0

resp = client.post(
"/api/movies/",
{
"title": "The Big Lebowski",
"genre": "comedy",
"year": "1998",
},
content_type="application/json"
)
assert resp.status_code == 201
assert resp.data["title"] == "The Big Lebowski"

movies = Movie.objects.all()
assert len(movies) == 1

The test should fail:

E assert 404 == 201

E + where 404 = <HttpResponseNotFound status_code=404,
"text/html">.status_code

View

Like traditional Django views, DRF views are Python functions or classes that take HTTP
requests and return HTTP responses.
DRF has three types of views:

1. Views, which subclasses Django's View class, are the most basic (and most explicit) DRF
view type. They can be function (implemented via the api_view decorator) or class
(implemented via the APIView class) based.
2. ViewSets provide a layer of abstraction above DRF views. They are often used to combine
the create, read, update, and destroy (CRUD) logic into a single view. A ViewSet "helps
ensure that URL conventions will be consistent across your API, minimizes the amount of
code you need to write, and allows you to concentrate on the interactions and
representations your API provides rather than the specifics of the URL conf". They are
perfect for implementing the basic CRUD operations for your API. They can be limiting
though if your API goes beyond the basics or the API endpoints don't cleanly map back
to the models.
3. Generic Views typically take the abstraction further by inferring the response format,
allowed methods, and payload shape based on the serializer.

Info
We'll be using the APIView class view in this course. Feel free to check your
understanding by implementing your API with a different type of view.

Add the following to views.py to create a MovieList class from the APIView class:
 # app/movies/views.py

from django.http import Http404

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status

from .models import Movie

from .serializers import MovieSerializer

class MovieList(APIView):
def post(self, request, format=None):
serializer = MovieSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)

URL

Add a urls.py file to the movies app:

# app/movies/urls.py

from django.urls import path

from .views import MovieList

urlpatterns = [
path("api/movies/", MovieList.as_view()),
]

Next, we need to roll those app-specific URLs up to the project-level by updating

the app/drf_project/urls.py file like so:

# app/drf_project/urls.py

from django.contrib import admin

from django.urls import include, path

from .views import ping

urlpatterns = [
path("admin/", admin.site.urls),
path("ping/", ping, name="ping"),
path("", include("movies.urls")),
]

The tests should now pass:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
 rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 6 items

tests/movies/test_models.py .
[ 16%]
tests/movies/test_views.py .
[ 33%]
tests/test_foo.py ..
[ 66%]
tests/movies/test_serializers.py ..
[100%]

========================================= 6 passed in 1.21s

=========================================

Try adding a new movie with HTTPie as well:

$ http --json POST http://localhost:8009/api/movies/ title=Fargo

genre=comedy year=1996

You should see:

HTTP/1.1 201 Created

Allow: POST, OPTIONS
Content-Length: 145
Content-Type: application/json
Cross-Origin-Opener-Policy: same-origin
Date: Thu, 16 Dec 2021 20:45:58 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.10.1
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
"created_date": "2021-12-16T20:45:58.468329Z",
"genre": "comedy",
"id": 4,
"title": "Fargo",
"updated_date": "2021-12-16T20:45:58.468353Z",
"year": "1996"
}

This covers the happy path. Let's add some tests to ensure the following scenarios are being
handled correctly:

1. A payload is not sent

2. The payload is invalid -- i.e., the JSON object is empty or it contains the wrong keys

@pytest.mark.django_db
def test_add_movie_invalid_json(client):
movies = Movie.objects.all()
assert len(movies) == 0

resp = client.post(
"/api/movies/",
 {},
content_type="application/json"
)
assert resp.status_code == 400

movies = Movie.objects.all()
assert len(movies) == 0

@pytest.mark.django_db
def test_add_movie_invalid_json_keys(client):
movies = Movie.objects.all()
assert len(movies) == 0

resp = client.post(
"/api/movies/",
{
"title": "The Big Lebowski",
"genre": "comedy",
},
content_type="application/json"
)
assert resp.status_code == 400

movies = Movie.objects.all()
assert len(movies) == 0

Ensure the tests pass:

$ docker-compose exec movies pytest

======================================== test session starts

========================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-4.5.2
collected 8 items

tests/movies/test_models.py .
[ 12%]
tests/movies/test_views.py ...
[ 50%]
tests/test_foo.py ..
[ 75%]
tests/movies/test_serializers.py ..
[100%]

========================================= 8 passed in 1.08s

=========================================

GET a Single Movie

Test

Start with some tests:

 @pytest.mark.django_db
def test_get_single_movie(client):
movie = Movie.objects.create(title="The Big Lebowski", genre="comedy",
year="1998")
resp = client.get(f"/api/movies/{movie.id}/")
assert resp.status_code == 200
assert resp.data["title"] == "The Big Lebowski"

def test_get_single_movie_incorrect_id(client):
resp = client.get(f"/api/movies/foo/")
assert resp.status_code == 404

Ensure the new tests fail.

View

# app/movies/views.py

from django.http import Http404

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status

from .models import Movie

from .serializers import MovieSerializer

class MovieList(APIView):
def post(self, request, format=None):
serializer = MovieSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)

class MovieDetail(APIView):
def get_object(self, pk):
try:
return Movie.objects.get(pk=pk)
except Movie.DoesNotExist:
raise Http404

def get(self, request, pk, format=None):

movie = self.get_object(pk)
serializer = MovieSerializer(movie)
return Response(serializer.data)

URL

Update app/movies/urls.py:

# app/movies/urls.py

from django.urls import path

from .views import MovieList, MovieDetail
 urlpatterns = [
path("api/movies/", MovieList.as_view()),
path("api/movies/<int:pk>/", MovieDetail.as_view()),
]

The tests should pass:

$ docker-compose exec movies pytest

Let's manually test this as well:

$ http --json http://localhost:8009/api/movies/1/

You should see:

HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Content-Length: 153
Content-Type: application/json
Cross-Origin-Opener-Policy: same-origin
Date: Thu, 16 Dec 2021 20:53:08 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.10.1
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
"created_date": "2021-12-16T16:53:25.829566Z",
"genre": "comedy",
"id": 1,
"title": "A Serious Man",
"updated_date": "2021-12-16T16:53:25.829588Z",
"year": "2009"
}

Ensure the Browsable API works as well -- http://localhost:8009/api/movies/1/.

GET All Movies

Test

Again, let's start with a test.

Since we'll have to add a few movies first, let's add a fixture that uses the "factory as fixture"
pattern to a new file called conftest.py in "app/tests/movies":

# app/tests/movies/conftest.py

import pytest

from movies.models import Movie

@pytest.fixture(scope='function')
def add_movie():
 def _add_movie(title, genre, year):
movie = Movie.objects.create(title=title, genre=genre, year=year)
return movie
return _add_movie

Now, refactor the test_get_single_movie test, like so:

@pytest.mark.django_db
def test_get_single_movie(client, add_movie):
movie = add_movie(title="The Big Lebowski", genre="comedy", year="1998")
resp = client.get(f"/api/movies/{movie.id}/")
assert resp.status_code == 200
assert resp.data["title"] == "The Big Lebowski"

Make sure the tests still pass.

With that, let's add the new test:

@pytest.mark.django_db
def test_get_all_movies(client, add_movie):
movie_one = add_movie(title="The Big Lebowski", genre="comedy",
year="1998")
movie_two = add_movie("No Country for Old Men", "thriller", "2007")
resp = client.get(f"/api/movies/")
assert resp.status_code == 200
assert resp.data[0]["title"] == movie_one.title
assert resp.data[1]["title"] == movie_two.title

Make sure it fails:

E assert 405 == 200

E + where 405 = <Response status_code=405,
"application/json">.status_code

View

# app/movies/views.py

from django.http import Http404

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status

from .models import Movie

from .serializers import MovieSerializer

class MovieList(APIView):
def get(self, request, format=None):
movies = Movie.objects.all()
serializer = MovieSerializer(movies, many=True)
return Response(serializer.data)

def post(self, request, format=None):

serializer = MovieSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
 return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)

class MovieDetail(APIView):
def get_object(self, pk):
try:
return Movie.objects.get(pk=pk)
except Movie.DoesNotExist:
raise Http404

def get(self, request, pk, format=None):

movie = self.get_object(pk)
serializer = MovieSerializer(movie)
return Response(serializer.data)

The tests should pass:

$ docker-compose exec movies pytest

Manually test via HTTPie and the Browsable API before moving on.

Database Seed
Add a movies.json file to "app":

[
{
"model": "movies.movie",
"pk": 1,
"fields": {
"title": "Fargo",
"genre": "comedy",
"year": "1996",
"created_date": "2021-01-07T14:07:13.540Z",
"updated_date": "2021-01-07T14:07:13.540Z"
}
},
{
"model": "movies.movie",
"pk": 2,
"fields": {
"title": "No Country for Old Men",
"genre": "thriller",
"year": "2007",
"created_date": "2021-01-07T14:06:59.408Z",
"updated_date": "2021-01-07T14:06:59.408Z"
}
},
{
"model": "movies.movie",
"pk": 3,
"fields": {
"title": "A Serious Man",
"genre": "comedy",
"year": "2009",
"created_date": "2021-01-07T14:06:51.542Z",
"updated_date": "2021-01-07T14:06:51.542Z"
 }
}
]

To seed the database with some initial data, run:

$ docker-compose exec movies python manage.py flush

$ docker-compose exec movies python manage.py loaddata movies.json

Make sure you can view the movies:

$ http --json http://localhost:8009/api/movies/

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 466
Content-Type: application/json
Cross-Origin-Opener-Policy: same-origin
Date: Thu, 16 Dec 2021 21:00:27 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.10.1
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

[
{
"created_date": "2021-01-07T14:07:13.540000Z",
"genre": "comedy",
"id": 1,
"title": "Fargo",
"updated_date": "2021-01-07T14:07:13.540000Z",
"year": "1996"
},
{
"created_date": "2021-01-07T14:06:59.408000Z",
"genre": "thriller",
"id": 2,
"title": "No Country for Old Men",
"updated_date": "2021-01-07T14:06:59.408000Z",
"year": "2007"
},
{
"created_date": "2021-01-07T14:06:51.542000Z",
"genre": "comedy",
"id": 3,
"title": "A Serious Man",
"updated_date": "2021-01-07T14:06:51.542000Z",
"year": "2009"
}
]

ViewSets and Routers

Refactor the views into a single ViewSet on your own. If you want to take it a step further, try
combining the ViewSet with a DRF Router component to automatically create the URL
endpoints. Compare this to the more explicit views and URL endpoints that we set up. You
can obviously move faster with ViewSets and Routers -- when your API endpoints map back
to the model -- but you sacrifice readability since the logic is buried. This can make it more
difficult to onboard a new developer onto your project.

Pytest Commands
Before moving on, let's review some useful pytest commands:

# normal run
$ docker-compose exec movies pytest

# disable warnings
$ docker-compose exec movies pytest -p no:warnings

# run only the last failed tests

$ docker-compose exec movies pytest --lf

# run only the tests with names that match the string expression
$ docker-compose exec movies pytest -k "movie and not all_movies"

# stop the test session after the first failure

$ docker-compose exec movies pytest -x

# enter PDB after first failure then end the test session
$ docker-compose exec movies pytest -x --pdb

# stop the test run after two failures

$ docker-compose exec movies pytest --maxfail=2

# show local variables in tracebacks

$ docker-compose exec movies pytest -l

# list the 2 slowest tests

$ docker-compose exec movies pytest --durations=2