2.

2: Use Pip and (Virtualenv or venv)

WARNING: Don’t Use SQLite3 with Django in Production

For any web project with more than one user, or requiring anything but light con-
currency, SQLite3 is a nightmare in the making. In the simplest terms possible,
SQLite3 works great in production until it doesn’t. We’ve experienced it ourselves,
and heard horror stories from others.

This issue compounds itself with the difficulty and complexity involved in migrat-
ing data out of SQLite3 and into something designed for concurrency (e.g., Post-
greSQL) when problems eventually arise.

While we’re aware that there are plenty of articles advocating the use of SQLite3 in
production, the fact that a tiny group of SQLite3 power users can get away with it
for particular edge cases is not justification for using it in production Django.

2.2 Use Pip and (Virtualenv or venv)

If you are not doing so already, we strongly urge you to familiarize yourself with both pip
and virtualenv. They are the de facto standard for Django projects, and most companies that
use Django rely on these tools.

Pip is a tool that fetches Python packages from the Python Package Index and its mirrors.
It is used to manage and install Python packages and it comes with most non-Linux Python
installations.

Virtualenv is a tool for creating isolated Python environments for maintaining package
dependencies. It’s great for situations where you’re working on more than one project at a
time, and where there are clashes between the version numbers of different libraries that
your projects use.

For example, imagine that you’re working on one project that requires Django 3.0 and an-
other that requires Django 3.2.

ä Without virtualenv (or an alternative tool to manage dependencies), you have to re-
install Django every time you switch projects.
ä If that sounds tedious, keep in mind that most real Django projects have at least a
dozen dependencies to maintain.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 13

Chapter 2: The Optimal Django Environment Setup

Pip is already included in Python 3.4 and higher. Further reading and installation instruc-
tions can be found at:

ä pip: pip.pypa.io
ä virtualenv: virtualenv.pypa.io

PACKAGE TIP: Virtualenv Alternative: Conda

Conda is an open source package management system and environment manage-
ment system that runs on Windows, Mac, and Linux. Originating from the data
science community, of all the packaging environments, it is the easiest for users
on Windows platforms who want to install compiled binaries. Conda also makes
the handling of multiple Python versions trivial. At the moment, this is Daniel’s
preferred development, he uses it with pip to manage his dependencies.
More information: docs.conda.io/

PACKAGE TIP: Pip+Virtualenv Alternatives: Poetry and Pipenv

Poetry helps you declare, manage, and install dependencies of Python projects, en-
suring you have the right stack everywhere. We appreciate how it elegantly encap-
sulates so much functionality in an intuitive CLI. It is popular and stable enough to
be a dependency management platform we can recommend.
More information: python-poetry.org
Pipenv is a tool that wraps pip and virtualenv into a single interface. It automates
processes like creating environments and introduces Pipfile.lock, a file that allows
for deterministic builds.
More information: pipenv.pypa.io/

WARNING: Pip is Not Necessarily Installed by Default on Linux

Distros
For many Linux distributions the maintainers remove some modules from the stan-
dard Python library and keep them in separated installable packages. For exam-
ple, on ubuntu you must install python3-pip, python3-setuptools, python3-wheel, and
python3-distutils.
Since Windows 10 will in the near future come with stable versions of WSL2 (Linux
on Windows), this pattern may also affect those users in a similar manner.

14 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 2.2: Use Pip and (Virtualenv or venv)

2.2.1 virtualenvwrapper

We also highly recommend virtualenvwrapper for Mac and Linux or

virtualenvwrapper-win for Windows. The project was started by Doug Hellman.

Personally, we think virtualenv without virtualenvwrapper can be a pain to use, because every
time you want to activate a virtual environment, you have to type something long like:

Example 2.1: Activating virtualenv

$ source ~/.virtualenvs/twoscoops/bin/activate

With virtualenvwrapper, you’d only have to type:

Example 2.2: Activating virtualenv

$ workon twoscoops

Virtualenvwrapper is a popular companion tool to pip and virtualenv and makes our lives
easier, but it’s not an absolute necessity.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 15

Chapter 2: The Optimal Django Environment Setup

Figure 2.1: Pip, virtualenv, and virtualenvwrapper in ice cream bar form.

2.3 Install Django and Other Dependencies via Pip

The official Django documentation describes several ways of installing Django. Our recom-
mended installation method is with pip and requirements files.

To summarize how this works: a requirements file is like a grocery list of Python packages
that you want to install. It contains the name and optionally suitable version range of each
package. You use pip to install packages from this list into your virtual environment.

We cover the setup of and installation from requirements files in Chapter 5: Settings and
Requirements Files.

TIP: Setting PYTHONPATH

If you have a firm grasp of the command line and environment variables, you can
set your virtualenv PYTHONPATH so that the django-admin command can be used
to serve your site and perform other tasks.

You can also set your virtualenv’s PYTHONPATH to include the current directory with
the latest version of pip. Running “pip install -e .” from your project’s root
directory will do the trick, installing the current directory as a package that can be

16 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 2.4: Use Git For Version Control

edited in place.

If you don’t know how to set this or it seems complicated, don’t worry about it and
stick with manage.py.

Additional reading:
ä hope.simons-rock.edu/~pshields/cs/python/pythonpath.html
ä docs.djangoproject.com/en/3.2/ref/django-admin/

2.4 Use Git For Version Control

Version control systems are also known as revision control or source control. Whenever you
work on any Django project, you should use a version control system to keep track of your
code changes.

Git is the industry standard for developers of all languages and tools. Git makes it easy to
create branches and merge changes. When using a version control system, it’s important
to not only have a local copy of your code repository but also to use a code hosting service
for backups. Fortunately, there are a number of services and tools who host repositories. Of
them, we recommend using GitHub (github.com) or GitLab (gitlab.com).

2.5 Optional: Identical Environments

What works on a programmer’s laptop might not work in production. But what if your
local development environment was identical to your project’s staging, test, and production
environments?

That said, if the production infrastructure consists of 10,000 servers, it’s completely unrealis-
tic to have another 10,000 local servers for development purposes. So when we say identical,
we mean “as identical as realistically possible.”

These are the environment differences that you can eliminate:

Operating system differences. If we’re developing on a Mac or on Windows, and if our

site is deployed on Ubuntu Linux, then there are huge differences between how our
Django project works locally and how it works in production.
Python setup differences. Let’s face it, many developers and sysadmins don’t even know
which version of Python they have running locally, although no one will admit it.
Why? Because setting up Python properly and understanding your setup completely
is hard.
Developer-to-developer differences. On large development teams, a lot of time can be
wasted trying to debug differences between one developer’s setup and another’s.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 17

Chapter 2: The Optimal Django Environment Setup

The most common way to set up identical development environments is with Docker.

2.5.1 Docker
At the time of this writing, Docker is the industry standard for containerization of environ-
ments. It has excellent support across all operating systems, including Microsoft Windows.
Working with Docker is sort of like developing inside of a VM, except more lightweight.
Docker containers share the host OS but have their own isolated process and memory space.
Furthermore, since Docker uses a union-capablefilesystem, containers can be built
quickly off of a snapshot plus deltas rather than building from scratch.

For the purposes of local development, its main benefit is that it makes setting up environ-
ments that closely match development and production much easier.

For example, if our development laptops run Mac (or Windows, or Centos, etc) but a
project’s configuration is Ubuntu-specific, we can use Docker via Docker Compose to
quickly get a virtual Ubuntu development environment set up locally, complete with all
the packages and setup configurations needed for the project. We can:

ä Set up identical local development environments for everyone on our project’s dev
team.
ä Configure these local development environments in a way similar to our staging, test,
and production servers.

The potential downsides are:

ä Extra complexity that is not needed in many situations. For simpler projects where
we’re not too worried about OS-level differences, it’s easier to skip this.
ä On older development machines, running even lightweight containers can slow per-
formance to a crawl. Even on newer machines, small but noticeable overhead is added.

References for developing with Docker:

ä cookiecutter-django.readthedocs.io/en/latest/
developing-locally-docker.html
ä http://bit.ly/1dWnzVW Real Python article on Django and Docker Compose
ä dockerbook.com

2.6 Summary
This chapter covered using the same database in development as in production, pip, vir-
tualenv, venv, conda, poetry, pipenv, version control, and Docker. These are good to have
in your tool chest, since they are commonly used not just in Django, but in the majority of
Python software development.

18 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

3 | How to Lay Out Django Projects

Project layout is one of those areas where core Django developers have differing opinions
about what they consider best practice. In this chapter, we present our approach, which is
one of the most commonly used ones.

PACKAGE TIP: Django Project Templates

There are a number of project templates that really kickstart a Django project and
follow the patterns described in this chapter. Here are several links that may be of
use when we bootstrap a project:
ä github.com/pydanny/cookiecutter-django
Featured in this chapter.
ä github.com/grantmcconnaughey/cookiecutter-django-vue-graphql-aws
Also featured in this chapter.
ä djangopackages.org/grids/g/cookiecutters/
A list of alternative cookiecutter templates.

3.1 Django 3’s Default Project Layout

Let’s examine the default project layout that gets created when you run startproject and
startapp:

Example 3.1: Default startproject and startapp

django-admin startproject mysite

cd mysite
django-admin startapp my_app

Here’s the resulting project layout:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 19

Chapter 3: How to Lay Out Django Projects

Example 3.2: Default Project Layout

mysite/
├── manage.py
├── my_app
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
│ │ └── __init__.py
│ ├── models.py
│ ├── tests.py
│ └── views.py
└── mysite
├── __init__.py
├── asgi.py
├── settings.py
├── urls.py
└── wsgi.py

There are a number of problems with Django’s default project layout. While useful for the
tutorial, it’s not quite as useful once you are trying to put together a real project. The rest of
this chapter will explain why.

3.2 Our Preferred Project Layout

We rely on a modified two-tier approach that builds on what is generated by the
django-admin startproject management command. Our layouts at the highest level
are:

Example 3.3: Project Root Levels

<repository_root>/
├── <configuration_root>/
├── <django_project_root>/

Let’s go over each level in detail:

3.2.1 Top Level: Repository Root

The <repository_root> directory is the absolute root directory of the project. In addition to the
<django_project_root> and <configuration_root>, we also include other critical components

20 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 3.2: Our Preferred Project Layout

like the README.md, docs/ directory, manage.py, .gitignore, requirements.txt files, and
other high-level files that are required for deployment and running the project.

Figure 3.1: Yet another reason why repositories are important.

TIP: Common Practice Varies Here

Some developers like to combine the <django_project_root> into the <reposi-
tory_root> of the project.

3.2.2 Second Level: Django Project Root

The <django_project_root>/ directory is the root of the actual Django project. Non-
configuration Python code files are inside this directory, its subdirectories, or below.

If using django-admin startproject, you would run the command from within the
repository root. The Django project that it generates would then be the project root.

3.2.3 Second Level: Configuration Root

The <configuration_root> directory is where the settings module and base URLConf (urls.py)
are placed. This must be a valid Python package containing an __init__.py module. Even in
Python 3, if the __init__.py is not included then the <configuration_root> won’t be recog-
nized as a python package.

If using django-admin startproject, the configuration root is initially inside of the

Django project root. It should be moved to the repository root.

The files in the configuration root are part of what is generated by the
django-admin startproject command.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 21

Chapter 3: How to Lay Out Django Projects

Figure 3.2: Three-tiered scoop layout.

3.3 Sample Project Layout

Let’s take a common example: a simple rating site. Imagine that we are creating Ice Cream
Ratings, a web application for rating different brands and flavors of ice cream.

This is how we would lay out such a project:

Example 3.4: Layout for icecreamratings

icecreamratings_project
├── config/
│ ├── settings/
│ ├── __init__.py
│ ├── asgi.py
│ ├── urls.py
│ └── wsgi.py
├── docs/
├── icecreamratings/
│ ├── media/ # Development only!
│ ├── products/
│ ├── profiles/
│ ├── ratings/
│ ├── static/
│ └── templates/
├── .gitignore

22 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 3.3: Sample Project Layout

├── Makefile
├── README.md
├── manage.py
└── requirements.txt

Let’s do an in-depth review of this layout. As you can see, in the icecreamratings_project/
directory, which is the <repository_root> , we have the following files and directories. We
describe them in the table below:

File or Directory Purpose

.gitignore Lists the files and directories that Git
should ignore.
config/ The <configuration_root> of the project,
where project-wide settings, urls.py, and
wsgi.py modules are placed (We’ll cover
settings layout later in Chapter 5: Settings
and Requirements Files).
Makefile Contains simple deployment tasks and
macros. For more complex deployments
you may want to rely on tools like Invoke,
Paver, or Fabric.
manage.py If you leave this in, don’t modify its con-
tents. Refer to Chapter 5: Settings and Re-
quirements Files for more details.
README.md and docs/ Developer-facing project documentation.
We cover this in Chapter 25: Documenta-
tion: Be Obsessed.
requirements.txt A list of Python packages required by your
project, including the Django 3.x package.
You’ll read more about this in Chapter 23:
Django’s Secret Sauce: Third-Party Pack-
ages.
icecreamratings/ The <django_project_root> of the project.

Table 3.1: Repository Root Files and Directories

When anyone visits this project, they are provided with a high-level view of the project.
We’ve found that this allows us to work easily with other developers and even non-
developers. For example, it’s not uncommon for designer-focused directories to be created
in the root directory.

Inside the icecreamratings_project/icecreamratings directory, at the <django_project_root>, we

place the following files/directories:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 23

Chapter 3: How to Lay Out Django Projects

File or Directory Purpose

media/ For use in development only: user-
generated static media assets such as
photos uploaded by users. For larger
projects, this will be hosted on separate
static media server(s).
products/ App for managing and displaying ice
cream brands.
profiles/ App for managing and displaying user pro-
files.
ratings/ App for managing user ratings.
static/ Non-user-generated static media assets in-
cluding CSS, JavaScript, and images. For
larger projects, this will be hosted on sepa-
rate static media server(s).
templates/ Where you put your site-wide Django tem-
plates.

Table 3.2: Django Project Files and Directories

TIP: Conventions for Static Media Directory Names

In the example above, we follow the official Django documentation’s convention of
using static/ for the (non-user-generated) static media directory.

If you find this confusing, there’s no harm in calling it assets/ or site_assets/ instead.
Just remember to update your STATICFILES_DIRS setting appropriately.

3.4 What About the Virtualenv?

Notice how there is no virtualenv directory anywhere in the project directory or its subdi-
rectories? That is completely intentional.

A good place to create the virtualenv for this project would be a separate directory where
you keep all of your virtualenvs for all of your Python projects. We like to put all our envi-
ronments in one directory and all our projects in another.

24 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 3.4: What About the Virtualenv?

Figure 3.3: An isolated environment, allowing your ice cream to swim freely.

For example, on Mac or Linux:

Example 3.5: On Mac or Linux

~/projects/icecreamratings_project/
~/.envs/icecreamratings/

On Windows:

Example 3.6: On Windows

c:\projects\icecreamratings_project\
c:\envs\icecreamratings\

If you’re using virtualenvwrapper (Mac or Linux) or virtualenvwrapper-win (Windows),

that directory defaults to ~/.virtualenvs/ and the virtualenv would be located at:

Example 3.7: virtualenvwrapper

~/.virtualenvs/icecreamratings/

Also, remember, there’s no need to keep the contents of your virtualenv in version control
since it already has all the dependencies captured in requirements.txt, and since you won’t be
editing any of the source code files in your virtualenv directly. Just remember that require-
ments.txt does need to remain in version control!

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 25

Chapter 3: How to Lay Out Django Projects

WARNING: Don’t Upload Environment Directories To Public

Repos
A common mistake by less experienced coders trying to build a GitHub portfolio is
to include a venv or icecreamratings_env directory in public GitHub repos. This is
a red flag to people looking over such projects, demonstrating in the clearest terms
the inexperience of the person.
The same goes for .node_modules.
The way to avoid making this mistake is for all projects to have a .gitignore file in
the root directory. This ensures that files and directories that shouldn’t be added to
git and never included in a commit. Useful references:
ä help.github.com/en/github/using-git/ignoring-files -
GitHub’s instructions on the topic.
ä github.com/github/gitignore/blob/master/Python.gitignore
- A good basic .gitignore for python projects.

3.4.1 Listing Current Dependencies

If you have trouble determining which versions of dependencies you are using in your vir-
tualenv, at the command line you can list your dependencies by typing:

Example 3.8: Listing Current Dependencies

$ pip freeze

With Mac or Linux, you can pipe this into a requirements.txt file:

Example 3.9: Saving Current Dependencies to a File

$ pip freeze > requirements.txt

3.5 Going Beyond startproject

Django’s startproject command allows you to create and use simple Django project
templates. However, over time the controls (deployment, front end tooling, etc) around a
project grow more and more complex. Most of us hit the limitations of startproject
quickly and need a more powerful project templating tool. Hence the use of Cookiecutter
(cookiecutter.readthedocs.io), an advanced project templating tool that can be used
for generating Django project boilerplate.

26 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 3.5: Going Beyond startproject

TIP: Audrey on Cookiecutter

I originally created Cookiecutter in 2013 to meet my own Python package boiler-
plate creation needs. It was the first project to template file paths and file contents
identically, an idea I thought was silly but decided to implement anyway.

There are now thousands of Cookiecutter templates for Python, Django, FastAPI,
C, C++, JavaScript, Ruby, LaTeX/XeTeX, Berkshelf-Vagrant, HTML, Scala, 6502
Assembly, and more. Numerous companies use Cookiecutter internally to power a
variety of tools.

Cookiecutter isn’t just a command-line tool, it’s a library used by a host of orga-
nizations. You can also find it integrated into IDEs such as PyCharm and Visual
Studio.

In this section, we present a popular Django project template, rendered by Cookiecutter.

3.5.1 Generating Project Boilerplate With Cookiecutter

Here’s how Cookiecutter works:

1 First, it asks you to enter a series of values (e.g. the value for project_name).
2 Then it generates all your boilerplate project files based on the values you entered.

First, install Cookiecutter as per the instructions in the official Cookiecutter documentation.

3.5.2 Generating a Starting Project With Cookiecutter Django

Here’s how you would use Cookiecutter to generate your Django 3 boilerplate from
Cookiecutter Django:

Example 3.10: Using Cookiecutter and Cookiecutter Django

cookiecutter https://github.com/pydanny/cookiecutter-django
You've downloaded /home/quique/.cookiecutters/cookiecutter-django
,→ before. Is it okay to delete and re-download it? [yes]: no
Do you want to re-use the existing version? [yes]: yes
project_name [My Awesome Project]: icecreamratings
project_slug [icecreamratings]:
description [Behold My Awesome Project!]: Support your Ice Cream
,→ Flavour!
author_name [Daniel Feldroy]:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 27

Chapter 3: How to Lay Out Django Projects

<snip for brevity>

After filling out all the values, in the directory where you ran Cookiecutter, it will create
a directory for your project. In this case with the values entered above, the name of this
directory will be icecreamratings.

The resulting project files will be roughly similar to the layout example we provided. The
project will include settings, requirements, starter documentation, a starter test suite, and
more.

TIP: What Are All the Other Files?

Keep in mind that Cookiecutter Django goes much further than the basic project
layout components that we outlined earlier in this chapter. It’s our ultimate Django
project template that we use for our projects, so it has a lot of other bells and whistles.

It’s a lot fancier than the default startproject template provided by Django.
We’d rather have you see our actual, real-life template that we use for our projects
than a stripped-down, beginner-oriented template that we don’t use.

You are welcome to fork Cookiecutter Django and customize it to fit your own Django
project needs.

3.6 Other Alternatives to startproject

People can get very opinionated about their project layout being the “right” way, but as we
mentioned, there’s no one right way.

It’s okay if a project differs from our layout, just so long as things are either done in a hier-
archical fashion or the locations of elements of the project (docs, templates, apps, settings,
etc) are documented in the root README.md.

We encourage you to explore the forks of Cookiecutter Django, and to search for other
Cookiecutter-powered Django project templates online. You’ll learn all kinds of interesting
tricks by studying other people’s project templates.

TIP: cookiecutter-django-vue-graphql-aws
Another good open source Cookiecutter-powered template is Grant Mc-
Connaughey’s cookiecutter-django-vue-graphql-aws. It knits together
technology we enjoy (Django, GraphQL, Vue, AWS Lambda + Zappa,

28 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 3.7: Summary

etc.) in a concise manner. What isn’t documented is named explicitly.

More information can be found at github.com/grantmcconnaughey/
cookiecutter-django-vue-graphql-aws

Figure 3.4: Project layout differences of opinion can cause ice cream fights.

3.7 Summary
In this chapter, we covered our approach to basic Django project layout. We provided a
detailed example to give you as much insight as possible into our practice. Cookiecutter and
two templates are introduced.

Project layout is one of those areas of Django where practices differ widely from developer
to developer and group to group. What works for a small team may not work for a large
team with distributed resources. Whatever layout is chosen it should be documented clearly.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 29

Chapter 3: How to Lay Out Django Projects

30 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

4 | Fundamentals of Django App
Design

It’s not uncommon for new Django developers to become understandably confused by
Django’s usage of the word “app”. So before we get into Django app design, it’s very impor-
tant that we go over some definitions.

A Django project is a web application powered by the Django web framework.

Django apps are small libraries designed to represent a single aspect of a project. A Django
project is made up of many Django apps. Some of those apps are internal to the project
and will never be reused; others are third-party Django packages.
INSTALLED_APPS is the list of Django apps used by a given project available in its
INSTALLED_APPS setting.
Third-party Django packages are simply pluggable, reusable Django apps that have been
packaged with the Python packaging tools. We’ll begin coverage of them in Chap-
ter 23: Django’s Secret Sauce: Third-Party Packages.

Figure 4.1: It’ll make more sense when you see the next figure.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 31

Chapter 4: Fundamentals of Django App Design

Figure 4.2: Did that make sense? If not, read it again.

4.1 The Golden Rule of Django App Design

James Bennett is a Django core developer. He taught us everything that we know about
good Django app design. We quote him:

“The art of creating and maintaining a good Django app is that it should
follow the truncated Unix philosophy according to Douglas McIlroy: ‘Write
programs that do one thing and do it well.”’

In essence, each app should be tightly focused on its task. If an app can’t be explained in
a single sentence of moderate length, or you need to say ‘and’ more than once, it probably
means the app is too big and should be broken up.

4.1.1 A Practical Example of Apps in a Project

Imagine that we’re creating a web application for our fictional ice cream shop called “Two
Scoops”. Picture us getting ready to open the shop: polishing the countertops, making the
first batches of ice cream, and building the website for our shop.

We’d call the Django project for our shop’s website twoscoops_project. The apps within our
Django project might be something like:

32 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 4.1: The Golden Rule of Django App Design

ä A flavors app to track all of our ice cream flavors and list them on our website.
ä A blog app for the official Two Scoops blog.
ä An events app to display listings of our shop’s events on our website: events such as
Strawberry Sundae Sundays and Fudgy First Fridays.

Each one of these apps does one particular thing. Yes, the apps relate to each other, and you
could imagine events or blog posts that are centered around certain ice cream flavors, but it’s
much better to have three specialized apps than one app that does everything.

In the future, we might extend the site with apps like:

ä A shop app to allow us to sell pints by mail order.

ä A tickets app, which would handle ticket sales for premium all-you-can-eat ice cream
fests.

Notice how events are kept separate from ticket sales. Rather than expanding the events app
to sell tickets, we create a separate tickets app because most events don’t require tickets, and
because event calendars and ticket sales have the potential to contain complex logic as the
site grows.

Eventually, we hope to use the tickets app to sell tickets to Icecreamlandia, the ice cream
theme park filled with thrill rides that we’ve always wanted to open.

Did we say that this was a fictional example? Ahem...well, here’s an early concept map of
what we envision for Icecreamlandia:

Figure 4.3: Our vision for Icecreamlandia.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 33

Chapter 4: Fundamentals of Django App Design

4.2 What to Name Your Django Apps

Everyone has their own conventions, and some people like to use really colorful names. We
like to use naming systems that are dull, boring, and obvious. In fact, we advocate doing the
following:

When possible keep to single word names like flavors, animals, blog, polls, dreams, estimates,
and finances. A good, easy-to-remember app name makes the project easier to maintain.

As a general rule, the app’s name should be a plural version of the app’s main model, but
there are many good exceptions to this rule, blog being one of the most common ones.

Don’t just consider the app’s main model, though. You should also consider how you want
your URLs to appear when choosing a name. If you want your site’s blog to appear at
http://www.example.com/weblog/, then consider naming your app weblog rather than
blog, posts, or blogposts, even if the main model is Post, to make it easier for you to see
which app corresponds with which part of the site.

Use valid, PEP 8-compliant, importable Python package names: short, all-lowercase names
without numbers, dashes, periods, spaces, or special characters. If needed for readability, you
can use underscores to separate words, although the use of underscores is discouraged.

4.3 When in Doubt, Keep Apps Small

Don’t worry too hard about getting app design perfect. It’s an art, not a science. Sometimes
you have to rewrite them or break them up. That’s okay.

Try and keep your apps small. Remember, it’s better to have many small apps than to have
a few giant apps.

34 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 4.4: What Modules Belong in an App?

Figure 4.4: Two small, single-flavor pints are better than a giant, 100-flavor container.

4.4 What Modules Belong in an App?

In this section, we cover both the common and uncommon Python modules that belong in
an app. For those with even a modicum of experience with Django, skipping to Section 4.4.2:
Uncommon App Modules may be in order.

4.4.1 Common App Modules

Here are common modules seen in 99% of Django apps. These will prove very familiar to
most readers, but we’re placing this here for those just coming into the world of Django.
For reference, any module ending with a slash (‘/’) represents a Python package, which can
contain one or more modules.

Example 4.1: Common App Modules

# Common modules
scoops/
├── __init__.py
├── admin.py
├── forms.py
├── management/
├── migrations/
├── models.py
├── templatetags/
├── tests/

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 35

Chapter 4: Fundamentals of Django App Design

├── urls.py
├── views.py

Over time a convention of module names has emerged for building Django apps. By fol-
lowing this convention across the building of apps we set behaviors for ourselves and others,
making examining each other’s code easier. While Python and Django are flexible enough
that most of these don’t need to be named according to this convention, not doing so will
cause problems. Probably not from an immediate technical perspective, but when you or
others look at nonstandard module names later, it will prove to be a frustrating experience.

4.4.2 Uncommon App Modules

Here are less common modules, which may or may not be familiar to many readers:

Example 4.2: Uncommon Django Modules

# uncommon modules
scoops/
├── api/
├── behaviors.py
├── constants.py
├── context_processors.py
├── decorators.py
├── db/
├── exceptions.py
├── fields.py
├── factories.py
├── helpers.py
├── managers.py
├── middleware.py
├── schema.py
├── signals.py
├── utils.py
├── viewmixins.py

What is the purpose of each module? Most of these have clear English names, but we’ll go
over a few that might not be so clear.

api/ This is the package we create for isolating the various modules needed when creating
an api. See Section 17.3.1: Use Consistent API Module Naming.

36 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 4.5: Alternative: Ruby on Rails-Style Approaches

behaviors.py An option for locating model mixins per Section 6.7.1: Model Behaviors
a.k.a Mixins.
constants.py A good name for placement of app-level settings. If there are enough of them
involved in an app, breaking them out into their own module can add clarity to a
project.
decorators.py Where we like to locate our decorators. For more information on decorators,
see Section 9.3: Decorators Are Sweet.
db/ A package used in many projects for any custom model fields or components.
fields.py is commonly used for form fields but is sometimes used for model fields when
there isn’t enough field code to justify creating a db/ package.
factories.py Where we like to place our test data factories. Described in brief in Sec-
tion 24.3.5: Don’t Rely on Fixtures
helpers.py What we call helper functions. These are where we put code extracted from
views (Section 8.5: Try to Keep Business Logic Out of Views) and models (Sec-
tion 6.7: Understanding Fat Models) to make them lighter. Synonymous with utils.py
managers.py When models.py grows too large, a common remedy is to move any custom
model managers to this module.
schema.py This is where code behind GraphQL APIs is typically placed.
signals.py While we argue against providing custom signals (see ??: ??), this can be a useful
place to put them.
utils.py Synonymous with helpers.py
viewmixins.py View modules and packages can be thinned by moving any view mixins to
this module. See Section 10.2: Using Mixins With CBVs.

For all of the modules listed in this section, their focus should be at the ‘app-level’, not
global tools. Global-level modules are described in Section 31.1: Create a Core App for
Your Utilities.

4.5 Alternative: Ruby on Rails-Style Approaches

Some people have found success using more Ruby on Rails-style approaches. Ruby on Rails
or Rails for short, is a successful Ruby-powered application framework approximately the
same age as Django. Notable Rails projects include GitHub, GitLab, Coinbase, Stripe, and
Square. There are enough similarities between Django and Rails as well as Python and Ruby
to make their approach worth examining.

4.5.1 Service Layers

When coding in Django, it’s common for newcomers to struggle determining where busi-
ness logic should go. The classic example is the process of creating user objects and related
data across multiple models and apps. In our case a ticket to enter the Icecreamlandia theme

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 37

Chapter 4: Fundamentals of Django App Design

park:

1 System creates a user record in a manager method called create_user()

2 create_user() uploads a photo of the user to a file hosting system
3 create_user() also creates a ticket object by calling create_ticket() , which is on
a manager method called create_ticket()
4 create_ticket() makes an API call to the guest Icecreamlandia check-in app, which
is a third-party service

The typical placement for these actions is spread across methods on the managers assigned
to the User and Ticket. Others prefer to use class methods instead. So we might see:

Example 4.3: Typical Django Business Logic Placement

class UserManager(BaseUserManager):
"""In users/managers.py"""
def create_user(self, email=None, password=None,
,→ avatar_url=None):
user = self.model(
email=email,
is_active=True,
last_login=timezone.now(),
registered_at=timezone.now(),
avatar_url=avatar_url
)
resize_avatar(avatar_url)
Ticket.objects.create_ticket(user)
return user

class TicketManager(models.manager):
"""In tasks/managers.py"""
def create_ticket(self, user: User):
ticket = self.model(user=user)
send_ticket_to_guest_checkin(ticket)
return ticket

While this works, the service layer approach argues that there should be separation of con-
cerns between the user object and tickets. Specifically, embedding logic in the User code
to call the Ticket code tightly couples the two domains. Thus, a new layer is added to keep
concerns separate, and this is called the service layer. Code for this architecture is placed in
service.py and selectors.py.

38 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 4.5: Alternative: Ruby on Rails-Style Approaches

Example 4.4: Service Layers

# Service layer example

scoops/
├── api/
├── models.py
├── services.py # Service layer location for business logic
├── selectors.py # Service layer location for queries
├── tests/

Under the service layer design, the above code would be executed thus:

Example 4.5: Service Layer Business Logic Placement

# In users/services.py
from .models import User
from tickets.models import Ticket, send_ticket_to_guest_checkin

def create_user(email: str, password: str, avatar_url: str) ->

,→ User:
user = User(
email=email,
password=password,
avatar_url=avatar_url
)
user.full_clean()
user.resize_avatar()
user.save()

ticket = Ticket(user=user)
send_ticket_to_guest_checkin(ticket)
return user

A popular explanation for this technique can be found at github.com/HackSoftware/

Django-Styleguide#services

Advantages to this approach:

ä In this simplistic example, 12 lines of code versus 17

ä Loose coupling means the user or ticketing code can be more readily replaced
ä Separation of concerns makes it easier to do functional tests of individual components

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 39

Chapter 4: Fundamentals of Django App Design

ä Adding type annotations of returned objects is easier with this approach than typical
Django

Disadvantages to this approach:

ä Very rare for projects to replace entire modules without entire rewrites
ä For small projects often adds unnecessary complexity
ä For complex projects the service layer typically grows to thousands of lines of code,
forcing the development of new architecture to support it
ä Use of selectors.py everywhere forces an extra step any time a simple ORM query
could be made
ä Django persistence is based on importable business logic placed on the model itself.
Service layers remove that capability, removing some advanced techniques

Further refutation of service layers or explanations of how to organize business logic in

traditional Django tools can be read in long-time Django maintainer James Bennett’s blog
and an article by Tom Christie, founder of Django REST Framework:

ä b-list.org/weblog/2020/mar/16/no-service/
ä b-list.org/weblog/2020/mar/23/still-no-service/
ä dabapps.com/blog/django-models-and-encapsulation/

For the authors of Two Scoops of Django the use of service layers isn’t new - we’ve seen
it for years. As with any technique that diverges from core Django practices, our anecdotal
experience is we’ve seen projects fail a bit more frequently using service layers. That isn’t to
say that the approach doesn’t have merit, but rather than the additional abstraction may not
be worth the effort.

4.5.2 The Large Single App Project

This is where a project puts all code into a single app. While common for small Django
projects, larger projects typically don’t follow this pattern.

There are positives to this approach. Specifically, migrations are easier and table names fol-
low a simplified pattern.

It has to be said that other frameworks such as Rails embrace this technique and are quite
successful. However, while Rails and other tools are designed to follow this pattern, Django
isn’t optimized for this design. Using this technique with Django requires experience and
expertise with patterns rarely described in documentation on Django.

40 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 4.6: Summary

Typically, everything is lumped together in giant models.py, views.py, tests.py, and urls.py
modules. Eventually either the project collapses under its own growing complexity or the
files are broken up into sub-modules based on the domains covered by models. Using our
icecreamlandia project, anything about “Users” is placed into models/users.py and “Tickets”
is placed into models/tickets.py.

Please note this can work if done with careful foresight. Indeed, Audrey has done it with
a project to great success, it did involve a bit more juggling of code than expected. It’s a
technique that should only be explored after a Djangonaut has completed several successful
Django projects.

WARNING: Be Careful with Too Divergent Django Architectures

James Beith is a senior engineer whom we respect. He has an interesting take on how
to structure a Django project. In his design, James overtly rewrites common patterns
for Django. While it appears to work for him, it makes for a harder-to-maintain
project in that any new developer on the project has to learn a new paradigm. The
charm of Django is heavily based on convention over configuration, and his design
explicitly breaks it. While we salute his desire to try something new, we feel he’s
gone too far in re-architecting Django projects.
James Beith’s article: jamesbeith.co.uk/blog/
how-to-structure-django-projects/

4.6 Summary
This chapter covered the art of Django app design. Specifically, each Django app should
be tightly-focused on its own task and possess a simple, easy-to-remember name. If an app
seems too complex, it should be broken up into smaller apps. Getting app design right takes
practice and effort, but it’s well worth the effort.

We also covered alternative approaches to architecting apps and how logic is called within
them.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 41

Chapter 4: Fundamentals of Django App Design

42 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

5 | Settings and Requirements Files

Django 3 has over 150 settings that can be controlled in the settings module, most of which
come with default values. Settings are loaded when your server starts up, and experienced
Django developers stay away from trying to change settings in production since they require
a server restart.

Figure 5.1: As your project grows, your Django settings can get pretty complex.

Some best practices we like to follow:

ä All settings files need to be version-controlled. This is especially true in production

environments, where dates, times, and explanations for settings changes absolutely
must be tracked.
ä Don’t Repeat Yourself. You should inherit from a base settings file rather than
cutting-and-pasting from one file to another.
ä Keep secret keys safe. They should be kept out of version control.

5.1 Avoid Non-Versioned Local Settings

We used to advocate the non-versioned local_settings anti-pattern. Now we know better.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 43

Chapter 5: Settings and Requirements Files

As developers, we have our own necessary settings for development, such as settings for
debugging tools that should be disabled (and often not installed to) staging or production
servers.

Furthermore, there are often good reasons to keep specific settings out of public or private
code repositories. The SECRET_KEY setting is the first thing that comes to mind, but API
key settings to services like Amazon, Stripe, and other password-type variables need to be
protected.

WARNING: Protect Your Secrets!

The SECRET_KEY setting is used in Django’s cryptographic signing functionality
and needs to be set to a unique, unpredictable setting best kept out of version control.
Running Django with a known SECRET_KEY defeats many of Django’s security
protections, which can lead to serious security vulnerabilities. For more details, read
docs.djangoproject.com/en/3.2/topics/signing/.

The same warning for SECRET_KEY also applies to production database passwords,
AWS keys, OAuth tokens, or any other sensitive data that your project needs in
order to operate.

Later in this chapter, we’ll show how to handle the SECRET_KEY issue in the “Keep
Secret Keys Out With Environment Settings” section.

A common solution is to create local_settings.py modules that are created locally per server
or development machine, and are purposefully kept out of version control. Developers now
make development-specific settings changes, including the incorporation of business logic
without the code being tracked in version control. Staging and deployment servers can have
location-specific settings and logic without them being tracked in version control.

What could possibly go wrong?!?

Ahem...

ä Every machine has untracked code.

ä How much hair will you pull out, when after hours of failing to duplicate a production
bug locally, you discover that the problem was custom logic in a production-only
setting?
ä How fast will you run from everyone when the ‘bug’ you discovered locally, fixed, and
pushed to production was actually caused by customizations you made in your own
local_settings.py module and is now crashing the site?

44 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.2: Using Multiple Settings Files

ä Everyone copy/pastes the same local_settings.py module everywhere. Isn’t this a vio-
lation of Don’t Repeat Yourself but on a larger scale?

Let’s take a different approach. Let’s break up development, staging, test, and production
settings into separate components that inherit from a common base object in a settings file
tracked by version control. Plus, we’ll make sure we do it in such a way that server secrets
will remain secret.

Read on and see how it’s done!

5.2 Using Multiple Settings Files

TIP: History of This Setup Pattern
The setup described here is based on the so-called “The One True Way”, from Ja-
cob Kaplan-Moss’ The Best (and Worst) of Django talk at OSCON 2011. See
slideshare.net/jacobian/the-best-and-worst-of-django.

Instead of having one settings.py file, with this setup you have a settings/ directory containing
your settings files. This directory will typically contain something like the following:

Example 5.1: Settings Directory

settings/
├── __init__.py
├── base.py
├── local.py
├── staging.py
├── test.py
├── production.py

WARNING: Requirements + Settings

Each settings module should have its own corresponding requirements file. We’ll
cover this at the end of this chapter in Section 5.5: Using Multiple Requirements
Files.

TIP: Multiple Files With Continuous Integration Servers

You’ll also want to have a ci.py module containing that server’s settings. Similarly,
if it’s a large project and you have other special-purpose servers, you might have

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 45

Chapter 5: Settings and Requirements Files

Settings file Purpose

base.py Settings common to all instances of the
project.
local.py This is the settings file that you use when
you’re working on the project locally. Lo-
cal development-specific settings include
DEBUG mode, log level, and activation of
developer tools like django-debug-toolbar.

staging.py Staging version for running a semi-private

version of the site on a production server.
This is where managers and clients should
be looking before your work is moved to
production.
test.py Settings for running tests including test
runners, in-memory database definitions,
and log settings.
production.py This is the settings file used by your live
production server(s). That is, the server(s)
that host the real live website. This file con-
tains production-level settings only. It is
sometimes called prod.py.

Table 5.1: Settings files and their purpose

custom settings files for each of them.

Let’s take a look at how to use the shell and runserver management commands with this
setup. You’ll have to use the --settings command line option, so you’ll be entering the
following at the command-line.

To start the Python interactive interpreter with Django, using your settings/local.py settings
file:

Example 5.2: Local Settings Shell

python manage.py shell --settings=config.settings.local

To run the local development server with your settings/local.py settings file:

Example 5.3: Local Settings Runserver

python manage.py runserver --settings=config.settings.local

46 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.2: Using Multiple Settings Files

TIP: DJANGO_SETTINGS_MODULE and PYTHONPATH

A great alternative to using the --settings command line option everywhere
is to set the DJANGO_SETTINGS_MODULE and PYTHONPATH environment vari-
able to your desired settings module path. To accomplish this, you’ll need to set
DJANGO_SETTINGS_MODULE to the corresponding settings module for each envi-
ronment.

For those with a more comprehensive understanding of virtualenvwrapper, an-

other alternative is to set DJANGO_SETTINGS_MODULE and PYTHONPATH in the
postactivate script and unset them in the postdeactivate script. Then, once
the virtualenv is activated, you can just type python from anywhere and import
those values into your project. This also means that typing django-admin at the
command-line works without the --settings option.

For the settings setup that we just described, here are the values to use with the --settings
command line option or the DJANGO_SETTINGS_MODULE environment variable:

Environment Option To Use With --settings (or

DJANGO_SETTINGS_MODULE value)
Your local development server twoscoops.settings.local
Your staging server twoscoops.settings.staging
Your test server twoscoops.settings.test
Your production server twoscoops.settings.production

Table 5.2: Setting DJANGO_SETTINGS_MODULE per location

5.2.1 A Development Settings Example

As mentioned earlier, we need settings configured for development, such as selecting the
console email backend, setting the project to run in DEBUG mode, and setting other con-
figuration options that are used solely for development purposes. We place development
settings like the following into settings/local.py:

Example 5.4: settings/local.py

from .base import *

DEBUG = True

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 47

Chapter 5: Settings and Requirements Files

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'twoscoops',
'HOST': 'localhost',
}
}

INSTALLED_APPS += ['debug_toolbar', ]

Now try it out at the command line with:

Example 5.5: runserver with local settings

python manage.py runserver --settings=config.settings.local

Open http://127.0.0.1:8000 and enjoy your development settings, ready to go into

version control! You and other developers will be sharing the same development settings
files, which for shared projects, is awesome.

Yet there’s another advantage: No more ‘if DEBUG’ or ‘if not DEBUG’ logic to copy/paste
around between projects. Settings just got a whole lot simpler!

At this point we want to take a moment to note that Django settings files are the single,
solitary place we advocate using import *. The reason is that for the singular case of Django
setting modules we want to override all the namespace.

5.2.2 Multiple Development Settings

Sometimes we’re working on a large project where different developers need different set-
tings, and sharing the same local.py settings module with teammates won’t do.

Well, it’s still better tracking these settings in version control than relying on everyone cus-
tomizing the same local.py module to their own tastes. A nice way to do this is with multiple
dev settings files, e.g. local_audrey.py and local_pydanny.py:

Example 5.6: settings/local_pydanny.py

# settings/local_pydanny.py
from .local import *

48 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.3: Separate Configuration From Code

# Set short cache timeout

CACHE_TIMEOUT = 30

Why? It’s not only good to keep all your own settings files in version control, but it’s also
good to be able to see your teammates’ dev settings files. That way, you can tell if someone’s
missing a vital or helpful setting in their local development setup, and you can make sure
that everyone’s local settings files are synchronized. Here is what our projects frequently use
for settings layout:

Example 5.7: Custom Settings

settings/
__init__.py
base.py
local_audreyr.py
local_pydanny.py
local.py
staging.py
test.py
production.py

5.3 Separate Configuration From Code

One of the causes of the local_settings anti-pattern is that putting SECRET_KEY, AWS keys,
API keys, or server-specific values into settings files has problems:

ä Config varies substantially across deploys, code does not.

ä Secret keys are configuration values, not code.
ä Secrets often should be just that: secret! Keeping them in version control means that
everyone with repository access has access to them.
ä Platforms-as-a-service usually don’t give you the ability to edit code on individual
servers. Even if they allow it, it’s a terribly dangerous practice.

To resolve this, our answer is to use environment variables in a pattern we like to call, well,
The Environment Variables Pattern.

Every operating system supported by Django (and Python) provides the easy capability to
create environment variables.

Here are the benefits of using environment variables for secret keys:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 49

Chapter 5: Settings and Requirements Files

ä Keeping secrets out of settings allows you to store every settings file in version control
without hesitation. All of your Python code really should be stored in version control,
including your settings.
ä Instead of each developer maintaining an easily-outdated, copy-and-pasted version of
the local_settings.py.example file for their own development purposes, everyone shares
the same version-controlled settings/local.py .
ä System administrators can rapidly deploy the project without having to modify files
containing Python code.
ä Most platforms-as-a-service recommend the use of environment variables for config-
uration and have built-in features for setting and managing them.

TIP: 12 Factor App: Store Config in the Environment

If you’ve read the 12 Factor App’s article on configuration you’ll recognize this pat-
tern. For reference, see 12factor.net/config. Some developers even advocate
combining the use of environment variables with a single settings modules. We
cover this practice in Chapter 37: Appendix E: Settings Alternatives.

5.3.1 A Caution Before Using Environment Variables for Secrets

Before you begin setting environment variables, you should have the following:

ä A way to manage the secret information you are going to store.

ä A good understanding of how bash works with environment variables on servers, or
a willingness to have your project hosted by a platform-as-a-service.

For more information, see en.wikipedia.org/wiki/Environment_variable.

WARNING: Environment Variables Do Not Work With Apache

If your target production environment uses Apache (outside of Elastic Beanstalk),
then you will discover that setting operating system environment variables as de-
scribed below doesn’t work. Confusing the issue is that Apache has its own environ-
ment variable system, which is almost but not quite what you’ll need.
If you are using Apache and want to avoid the local_settings anti-pattern, we rec-
ommend reading Section 5.4: When You Can’t Use Environment Variables later in
this chapter.

50 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.3: Separate Configuration From Code

5.3.2 How to Set Environment Variables Locally

On pre-Catalina Mac and many Linux distributions that use bash for the shell, one can add
lines like the following to the end of a .bashrc, .bash_profile, or .profile. Macs on Catalina
and afterwards use .zshrc. When dealing with multiple projects using the same API but
with different keys, you can also place these at the end of your virtualenv’s bin/postactivate
script:

Example 5.8: Setting Environment Variables on Linux/OSX

export SOME_SECRET_KEY=1c3-cr3am-15-yummy
export AUDREY_FREEZER_KEY=y34h-r1ght-d0nt-t0uch-my-1c3-cr34m

On Windows systems, it’s a bit trickier. You can set them one-by-one at the command line
(cmd.exe) in a persistent way with the setx command, but you’ll have to close and reopen
your command prompt for them to go into effect. A better way is to place these commands
at the end of the virtualenv’s bin/postactivate.bat script so they are available upon activation:

Example 5.9: Setting Environment Variables on Windows

> setx SOME_SECRET_KEY 1c3-cr3am-15-yummy

PowerShell is much more powerful than the default Windows shell and comes with Win-
dows Vista and above. Setting environment variables while using PowerShell:

For the current Windows user only:

Example 5.10: Setting Environment Variables on Powershell

[Environment]::SetEnvironmentVariable('SOME_SECRET_KEY',
'1c3-cr3am-15-yummy', 'User')
[Environment]::SetEnvironmentVariable('AUDREY_FREEZER_KEY',
'y34h-r1ght-d0nt-t0uch-my-1c3-cr34m', 'User')

Machine-wide:

Example 5.11: Globally Setting Environment Variables on Powershell

[Environment]::SetEnvironmentVariable('SOME_SECRET_KEY',
'1c3-cr3am-15-yummy', 'Machine')

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 51

Chapter 5: Settings and Requirements Files

[Environment]::SetEnvironmentVariable('AUDREY_FREEZER_KEY',
'y34h-r1ght-d0nt-t0uch-my-1c3-cr34m', 'Machine')

For more information on Powershell, see en.wikipedia.org/wiki/PowerShell

TIP: virtualenvwrapper Makes This Easier

Mentioned earlier in this book, virtualenvwrapper, simplifies per-virtualenv envi-
ronment variables. It’s a great tool. However, setting it up requires a more-than-
basic understanding of the shell and Mac, Linux, or Windows.

5.3.3 How to Unset Environment Variables Locally

When you set an environment variable via the commands listed above it will remain in
existence within that terminal shell until it is unset or the shell is ended. This means that
even if you deactivate a virtualenv, the environment variable remains. In our experience,
this is fine 99% of the time. However, there are occasions when we want to tightly control
environment variables. To do this, we execute the appropriate command for the operating
system or shell variant:

Example 5.12: Unsetting Environment Variables on Linux/OSX/Windows

unset SOME_SECRET_KEY
unset AUDREY_FREEZER_KEY

Example 5.13: Unsetting Environment Variables on Powershell

[Environment]::UnsetEnvironmentVariable('SOME_SECRET_KEY', 'User')
[Environment]::UnsetEnvironmentVariable('AUDREY_FREEZER_KEY',
,→ 'User')

If you are using virtualenvwrapper and want to unset environment variables whenever a
virtualenv is deactivated, place these commands in the postdeactivate script.

5.3.4 How to Set Environment Variables in Production

If you’re using your own servers, your exact practices will differ depending on the tools you’re
using and the complexity of your setup. For the simplest 1-server setup for test projects, you

52 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.3: Separate Configuration From Code

can set the environment variables manually. But if you’re using scripts or tools for auto-
mated server provisioning and deployment, your approach may be more complex. Check
the documentation for your deployment tools for more information.

If your Django project is deployed via a platform-as-a-service (PaaS), such as Heroku,

Python Anywhere, platform.sh, etc., check the documentation for specific instructions.

To see how you access environment variables from the Python side, open up a new Python
prompt and type:

Example 5.14: Accessing Environment Variables in Python’s REPL

>>> import os
>>> os.environ['SOME_SECRET_KEY']
'1c3-cr3am-15-yummy'

To access environment variables from one of your settings files, you can do something like
this:

Example 5.15: Accessing Environment Variables in Python

# Top of settings/production.py
import os
SOME_SECRET_KEY = os.environ['SOME_SECRET_KEY']

This snippet simply gets the value of the SOME_SECRET_KEY environment variable from
the operating system and saves it to a Python variable called SOME_SECRET_KEY.

Following this pattern means all code can remain in version control, and all secrets remain
safe.

5.3.5 Handling Missing Secret Key Exceptions

In the above implementation, if the SECRET_KEY isn’t available, it will throw a KeyError ,
making it impossible to start the project. That’s great, but a KeyError doesn’t tell you that
much about what’s actually wrong. Without a more helpful error message, this can be hard
to debug, especially under the pressure of deploying to servers while users are waiting and
your ice cream is melting.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 53

Chapter 5: Settings and Requirements Files

Here’s a useful code snippet that makes it easier to troubleshoot those missing environment
variables. If you’re using our recommended environment variable secrets approach, you’ll
want to add this to your settings/base.py file:

Example 5.16: The get_env_variable() Function

# settings/base.py
import os

# Normally you should not import ANYTHING from Django directly

# into your settings, but ImproperlyConfigured is an exception.
from django.core.exceptions import ImproperlyConfigured

def get_env_variable(var_name):
"""Get the environment variable or return exception."""
try:
return os.environ[var_name]
except KeyError:
error_msg = 'Set the {} environment
,→ variable'.format(var_name)
raise ImproperlyConfigured(error_msg)

Then, in any of your settings files, you can load secret keys from environment variables as
follows:

Example 5.17: Using get_env_variable()

SOME_SECRET_KEY = get_env_variable('SOME_SECRET_KEY')

Now, if you don’t have SOME_SECRET_KEY set as an environment variable, you get
a traceback that ends with a useful error message like this:

Example 5.18: Error Generated by get_env_variable()

django.core.exceptions.ImproperlyConfigured: Set the

,→ SOME_SECRET_KEY
environment variable.

54 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.4: When You Can’t Use Environment Variables

WARNING: Don’t Import Django Components Into Settings

Modules
This can have many unpredictable side effects, so avoid any sort of import of Django
components into your settings. ImproperlyConfigured is the exception because
it’s the official Django exception for...well...improperly configured projects. And just
to be helpful we add the name of the problem setting to the error message.

PACKAGE TIP: Packages for Settings Management

A number of third-party packages take the idea of our get_env_variable() func-
tion and expand on it, including features like defaults and types and supporting .env
files. The downside is the same you get with any complex packages: sometimes the
edge cases cause problems. Nevertheless, most of them are quite useful and we’ve
listed some of our favorites:
ä github.com/joke2k/django-environ (Used in Cookiecutter Django)
ä github.com/jazzband/django-configurations

TIP: Using django-admin Instead of manage.py

The official Django documentation says that you should use django-admin rather
than manage.py when working with multiple settings files:
docs.djangoproject.com/en/3.2/ref/django-admin/

That being said, if you’re struggling with getting django-admin to work, it’s perfectly
okay to develop and launch your site running it with manage.py.

5.4 When You Can’t Use Environment Variables

The problem with using environment variables to store secrets is that it doesn’t always work.
The most common scenario for this is when using Apache for serving HTTP, but this
also happens even in Nginx-based environments where operations wants to do things in a
particular way. When this occurs, rather than going back to the local_settings anti-pattern,
we advocate using non-executable files kept out of version control in a method we like to
call the secrets file pattern.

To implement the secrets file pattern, follow these three steps:

1 Create a secrets file using the configuration format of choice, be it JSON, .env, Config,
YAML, or even XML.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 55

Chapter 5: Settings and Requirements Files

2 Add a secrets loader ( JSON-powered example below) to manage the secrets in a

cohesive, explicit manner.
3 Add the secrets file name to the project’s .gitignore file.

5.4.1 Using JSON Files

Our preference is to use shallow JSON files. The JSON format has the advantage of being a
format of choice for both Python and non-Python tools. To do this, first create a secrets.json
file:

Example 5.19: secrets.json

{
"FILENAME": "secrets.json",
"SECRET_KEY": "I've got a secret!",
"DATABASES_HOST": "127.0.0.1",
"PORT": "5432"
}

To use the secrets.json file, add the following code to your base settings module.

Example 5.20: The get_settings() Function

# settings/base.py
import json

# Normally you should not import ANYTHING from Django directly

# into your settings, but ImproperlyConfigured is an exception.
from django.core.exceptions import ImproperlyConfigured

# JSON-based secrets module

with open('secrets.json') as f:
secrets = json.load(f)

def get_secret(setting, secrets=secrets):

'''Get the secret variable or return explicit exception.'''
try:
return secrets[setting]
except KeyError:
error_msg = 'Set the {0} environment
,→ variable'.format(setting)
raise ImproperlyConfigured(error_msg)

56 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.5: Using Multiple Requirements Files

SECRET_KEY = get_secret('SECRET_KEY')

Now we are loading secrets from non-executable JSON files instead of from unversioned
executable code. Hooray!

PACKAGE TIP: Zappa Provides Powerful Options

One of the reasons we prefer Zappa for deploying to AWS Lambda is the different
and powerful options it provides for environment variable management. Definitely
worth a look and something we want to see other tools and hosting platforms adopt.
Reference: github.com/Miserlou/Zappa#setting-environment-variables

5.4.2 Using .env, Config, YAML, and XML File Formats

While we prefer the forced simplicity of shallow JSON, others might prefer other file for-
mats. We’ll leave it up to the reader to create additional get_secret() alternatives that
work with these formats. Just remember to be familiar with things like yaml.safe_load()
and XML bombs. See Section 28.10: Defend Against Python Code Injection Attacks.

5.5 Using Multiple Requirements Files

Finally, there’s one more thing you need to know about multiple settings files setup. It’s
good practice for each settings file to have its own corresponding requirements file. This
means we’re only installing what is required on each server.

To follow this pattern, recommended to us by Jeff Triplett, first create a requirements/ di-
rectory in the <repository_root>. Then create ‘.txt’ files that match the contents of your
settings directory. The results should look something like:

Example 5.21: Segmented Requirements

requirements/
├── base.txt
├── local.txt
├── staging.txt
├── production.txt

In the base.txt file, place the dependencies used in all environments. For example, you might
have something like the following in there:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 57

Chapter 5: Settings and Requirements Files

Example 5.22: requirements/base.txt

Django==3.2.0
psycopg2-binary==2.8.8
djangorestframework==3.11.0

Your local.txt file should have dependencies used for local development, such as:

Example 5.23: requirements/local.txt

-r base.txt # includes the base.txt requirements file

coverage==5.1
django-debug-toolbar==2.2

The needs of a continuous integration server might prompt the following for a ci.txt file:

Example 5.24: requirements/ci.txt

-r base.txt # includes the base.txt requirements file

coverage==5.1

Production installations should be close to what is used in other locations, so production.txt

commonly just calls base.txt:

Example 5.25: requirements/production.txt

-r base.txt # includes the base.txt requirements file

5.5.1 Installing From Multiple Requirements Files

For local development:

Example 5.26: Installing Local Requirements

pip install -r requirements/local.txt

For production:

58 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.6: Handling File Paths in Settings

Example 5.27: Installing Production Requirements

pip install -r requirements/production.txt

TIP: Pin Requirements Exactly

All the pip requirements.txt examples in this chapter are explicitly set to a package
version. This ensures a more stable project. We cover this at length in Section 23.7.2:
Step 2: Add Package and Version Number to Your Requirements.

TIP: Using Multiple Requirements Files With PaaS

We cover this in ??: ??

5.6 Handling File Paths in Settings

If you switch to the multiple settings setup and get new file path errors to things like tem-
plates and media, don’t be alarmed. This section will help you resolve these errors.

We humbly beseech the reader to never hardcode file paths in Django settings files. This is
really bad:

Example 5.28: Never Hardcode File Paths

# settings/base.py

# Configuring MEDIA_ROOT
# DON’T DO THIS! Hardcoded to just one user's preferences
MEDIA_ROOT = '/Users/pydanny/twoscoops_project/media'

# Configuring STATIC_ROOT
# DON’T DO THIS! Hardcoded to just one user's preferences
STATIC_ROOT = '/Users/pydanny/twoscoops_project/collected_static'

# Configuring STATICFILES_DIRS
# DON’T DO THIS! Hardcoded to just one user's preferences
STATICFILES_DIRS = ['/Users/pydanny/twoscoops_project/static']

# Configuring TEMPLATES
# DON’T DO THIS! Hardcoded to just one user's preferences
TEMPLATES = [
{

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 59

Chapter 5: Settings and Requirements Files

'BACKEND':
,→ 'django.template.backends.django.DjangoTemplates',
DIRS: ['/Users/pydanny/twoscoops_project/templates',]
},
]

The above code represents a common pitfall called hardcoding. The above code, called a
fixed path, is bad because as far as you know, pydanny (Daniel Feldroy) is the only person
who has set up their computer to match this path structure. Anyone else trying to use this
example will see their project break, forcing them to either change their directory structure
(unlikely) or change the settings module to match their preference (causing problems for
everyone else including pydanny).

Don’t hardcode your paths!

To fix the path issue, we dynamically set a project root variable intuitively named BASE_DIR
at the top of the base settings module. Since BASE_DIR is determined in relation to the
location of base.py, your project can be run from any location on any development computer
or server.

Figure 5.2: While we’re at it, let’s go down this path.

We find the cleanest way to set a BASE_DIR-like setting is with Pathlib, part of Python
since 3.4 that does elegant, clean path calculations:

Example 5.29: Using Pathlib to discover project root

# At the top of settings/base.py

from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent.parent
MEDIA_ROOT = BASE_DIR / 'media'
STATIC_ROOT = BASE_DIR / 'static_root'

60 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

 5.6: Handling File Paths in Settings

STATICFILES_DIRS = [BASE_DIR / 'static']

TEMPLATES = [
{
'BACKEND':
,→ 'django.template.backends.django.DjangoTemplates',
'DIRS': [BASE_DIR / 'templates']
},
]

If you really want to set your BASE_DIR with the Python standard library’s os.path library,
though, this is one way to do it in a way that will account for paths:

Example 5.30: Using os.path to discover project root

# At the top of settings/base.py

from os.path import abspath, dirname, join

def root(*dirs):
base_dir = join(dirname(__file__), '..', '..')
return abspath(join(base_dir, *dirs))

BASE_DIR = root()
MEDIA_ROOT = root('media')
STATIC_ROOT = root('static_root')
STATICFILES_DIRS = [root('static')]
TEMPLATES = [
{
'BACKEND':
,→ 'django.template.backends.django.DjangoTemplates',
'DIRS': [root('templates')],
},
]

With your various path settings dependent on BASE_DIR, your file path settings should
work, which means your templates and media should be loading without error.

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 61

Chapter 5: Settings and Requirements Files

TIP: How Different Are Your Settings From the Django

Defaults?
If you want to know how things in your project differ from Django’s defaults, use
the
diffsettings management command.

5.7 Summary
Remember, everything except for passwords and API keys ought to be tracked in version
control.

Any project that’s destined for a real live production server is bound to need multiple set-
tings and requirements files. Even beginners to Django need this kind of settings/require-
ments file setup once their projects are ready to leave the original development machine.
We provide our solution, as well as an Apache-friendly solution since it works well for both
beginning and advanced developers.

Also, if you prefer a different shell than the ones provided, environment variables still work.
You’ll just need to know the syntax for defining them.

The same thing applies to requirements files. Working with untracked dependency differ-
ences increases risk as much as untracked settings.

62 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues