Real Python Part 3: Advanced Web Development

with Django
Jeremy Johnson

Contents

Preface

Thank you . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11

Course Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

Errata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

14

2 Introduction

15

Welcome to Advanced Web Programming! . . . . . . . . . . . . . . . . . . . . .

15

Why this course? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

What is Software Craftsmanship? . . . . . . . . . . . . . . . . . . . . . . . . . .

18

What will you learn? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19

What will you build? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

20

3 Software Craftsmanship

23

Testing Routes and Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29

Refactoring our Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43

Testing Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

48

Testing Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

Testing Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60

4 Test Driven Development

63

The TDD Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

64

Implementing TDD with Existing Code . . . . . . . . . . . . . . . . . . . . . . .

66

Django Ecommerce Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

67

Offense is only as good as your worst defense . . . . . . . . . . . . . . . . . . . .

77

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

84

5 Git Branching at a Glance

85

Git Branching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

86

Branching Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

87

Enough about git branching . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

98

6 Upgrade, Upgrade, and Upgrade some more

Django 1.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

99

99

The Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

DiscoverRunner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Update Project Structure

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Upgrading to Python 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

110

Python 3 Changes Things Slightly . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Upgrading to PostgreSQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

125

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7 Graceful Degradation and Database Transactions with Django 1.8

What is Graceful Degradation? . . . . . . . . . . . . . . . . . . . . . . . . . . .

131
132

133

User Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Handling Unpaid Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Improved Transaction Management

. . . . . . . . . . . . . . . . . . . . . . . . 142
2

What is a transaction? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Whats wrong with transaction management prior to Django 1.6? . . . . . . . . .

144

Whats right about transaction management in Django 1.6 and above? . . . . . . .

146

SavePoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

152

Nested Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155

Completing the Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

157

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

159

Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

161

8 Building a Membership Site

163

Its time to make something cool . . . . . . . . . . . . . . . . . . . . . . . . . . 164

A Membership Site

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

165

The User Stories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

9 Bootstrap 3 and Best Effort Design

Start with the User Story

168

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Overview of what we have . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Installing Bootstrap 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

171

Making Bootstrap your own . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

176

HTML5 Sections and the Semantic Web

. . . . . . . . . . . . . . . . . . . . . . 183

More Bootstrap Customizations . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Custom template tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
A Note on Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
10 Building the Members Page

208

User Story . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Update Main Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Template 1: Showing User Info for the Current User . . . . . . . . . . . . . . . .
3

212

Gravatar Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

217

Template 2: Status Updating and Reporting . . . . . . . . . . . . . . . . . . . . 220

Template 3: Displaying Status Updates . . . . . . . . . . . . . . . . . . . . . . . 226
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
11 REST

230

Structuring a REST API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

231

REST for MEC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

Django REST Framework (DRF)
Now Serving JSON

. . . . . . . . . . . . . . . . . . . . . . . . . . 235

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Using Class-Based Views

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
12 Django Migrations

264

The problems that Migrations Solve . . . . . . . . . . . . . . . . . . . . . . . . . 265

Getting Started with Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
The Migration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
When Migrations Dont Work . . . . . . . . . . . . . . . . . . . . . . . . . . . .

271

Data Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
13 AngularJS Primer

283

What we are covering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

Angular for Pony Fliers (aka Django Devs) . . . . . . . . . . . . . . . . . . . . . 286
Angular vs Django - re: template tags . . . . . . . . . . . . . . . . . . . . . . . . 289
Angular Data Binding Explained
Building User Polls

. . . . . . . . . . . . . . . . . . . . . . . . . . 290

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
4

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
14 Djangular: Integrating Django and Angular

308

The User Polls Backend Data Model . . . . . . . . . . . . . . . . . . . . . . . . . 309

A REST Interface for User Polls . . . . . . . . . . . . . . . . . . . . . . . . . . .

311

Structural Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

Building the Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
Loading Poll Items Dynamically . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Refactoring - progress bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
Refactoring - multiple users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
Using Angular Factories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
15 Angular Forms

343

Field Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

Display Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

351

Form submission with Angular . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

How I taught an old view some new JSON tricks. . . . . . . . . . . . . . . . . . . 361
Fixing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
Breaking Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
16 MongoDB Time!

375

Building the front-end first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

MongoDB vs SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
Installing Mongo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
Configuring Django for MongoDB . . . . . . . . . . . . . . . . . . . . . . . . . . 384
5

Django Models and MongoDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386

A Geospatial primer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
Mongoengine Support for Geospatial Features . . . . . . . . . . . . . . . . . . . 389
Showing a Dot on a Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
Connecting the Dots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
Getting User Locations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
17 One Admin to Rule Them All

402

Basic Admin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

All your model are belong to us . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
Editing Stuff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
Making things Puuurdy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
Adminstrating Your Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
Resetting passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
18 Testing, Testing, and More Testing

Why do we need more testing?

442

. . . . . . . . . . . . . . . . . . . . . . . . . . . 442

What needs to be GUI tested? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

Ensuring Appropriate Testing Coverage . . . . . . . . . . . . . . . . . . . . . . . 445
GUI Testing with Django

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

Our First GUI Test Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448

The Selenium API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
Acting on elements you have located . . . . . . . . . . . . . . . . . . . . . . . . 454
Waiting for things to happen . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
Page Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

When Selenium Doesnt Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

Limitations of Djangos LiveServerTestCase

. . . . . . . . . . . . . . . . . . . . 473

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
19 Deploy

479

Where to Deploy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480

What to Deploy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
Firing up the VPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
Configuring the OS
Django Setup

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493

Configuration as Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496

Lets script it . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
Continuous Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
20 Conclusion

506

21 Appendix A - Solutions to Exercises

508

Software Craftsmanship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508

Test Driven Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

514

Bootstrap 3 and Best Effort Design . . . . . . . . . . . . . . . . . . . . . . . . .

521

Building the Members Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

REST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546
Django Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
AngularJS Primer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

561

Djangular: Integrating Django and Angular . . . . . . . . . . . . . . . . . . . . . 564

Angular Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
MongoDB Time! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
7

One Admin to Rule Them All . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580

Testing, Testing, and More Testing . . . . . . . . . . . . . . . . . . . . . . . . . 586
Deploy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590

Chapter 1
Preface
Thank you
I commend you for taking the time out of your busy schedule to improve your craft of software
development. My promise to you is that I will hold nothing back and do my best to impart
the most important and useful things I have learned from my software development career
in hopes that you can benefit from it and grow as a software developer yourself.
Thank you so much for purchasing this course, and a special thanks goes out to all the early
Kickstarter supporters that had faith in the entire Real Python team before a single line of
code had been written.

License
This e-course is copyrighted and licensed under a Creative Commons AttributionNonCommercial-NoDerivs 3.0 Unported License. This means that you are welcome to
share this course and use it for any non-commercial purposes so long as the entire course
remains intact and unaltered. That being said, if you have received this copy for free and
have found it helpful, I would very much appreciate if you purchased a copy of your own.
The example Python scripts associated with this course should be considered open content.
This means that anyone is welcome to use any portion of the code for any purpose.

10

Conventions
NOTE: Since this is the Beta release, we do not have all the conventions in place.
We are working on it. Patience, please.

Formatting
1. Code blocks will be used to present example code.
1

print Hello world!

2. Terminal commands follow the Unix format:

1

$ python hello-world.py

(dollar signs are not part of the command)

3. Italic text will be used to denote a file name:
hello-world.py.

Bold text will be used to denote a new or important term:

Important term: This is an example of what an important term should look like.
NOTES, WARNINGS, and SEE ALSO boxes appear as follows:
NOTE: This is a note filled in with bacon impsum text. Bacon ipsum dolor
sit amet t-bone flank sirloin, shankle salami swine drumstick capicola doner
porchetta bresaola short loin. Rump ham hock bresaola chuck flank. Prosciutto beef ribs kielbasa pork belly chicken tri-tip pork t-bone hamburger bresaola
meatball. Prosciutto pork belly tri-tip pancetta spare ribs salami, porchetta strip
steak rump beef filet mignon turducken tail pork chop. Shankle turducken spare
ribs jerky ribeye.
WARNING: This is a warning also filled in with bacon impsum. Bacon ipsum
dolor sit amet t-bone flank sirloin, shankle salami swine drumstick capicola
doner porchetta bresaola short loin. Rump ham hock bresaola chuck flank.
Prosciutto beef ribs kielbasa pork belly chicken tri-tip pork t-bone hamburger

11

bresaola meatball. Prosciutto pork belly tri-tip pancetta spare ribs salami,
porchetta strip steak rump beef filet mignon turducken tail pork chop. Shankle
turducken spare ribs jerky ribeye.
SEE ALSO: This is a see also box with more tasty impsum. Bacon ipsum dolor sit amet t-bone flank sirloin, shankle salami swine drumstick capicola doner
porchetta bresaola short loin. Rump ham hock bresaola chuck flank. Prosciutto beef ribs kielbasa pork belly chicken tri-tip pork t-bone hamburger bresaola
meatball. Prosciutto pork belly tri-tip pancetta spare ribs salami, porchetta strip
steak rump beef filet mignon turducken tail pork chop. Shankle turducken spare
ribs jerky ribeye.

12

Course Repository
Like the other two courses, this course has an accompanying repository. Broken up by chapter, in the _chapters folder, you can follow along with each chapter, then check your answers in the repository. Before you start each chapter, please make sure your local code
aligns with the code from the previous chapter. For example, before starting chapter 6 make
sure youre code matches the code from the chp5 folder from the repository.
You can download the course files directly from the repository. Press the Download ZIP button which is located at the bottom right of the page. This allows
you to download the most recent version of the code as a zip archive. Be sure
to download the updated code for each release.
That said, its recommended that you work on one project throughout this course, rather than
breaking it up.
NOTE: Since this is the Beta release, we do not have the structure completely
updated in the repository. We will eventually change to a Git tagging model. If
you have any suggestions or feedback, please contact us.

13

Errata
I welcome ideas, suggestions, feedback, and the occasional rant. Did you find a topic confusing? Or did you find an error in the text or code? Did I omit a topic you would love to know
more about. Whatever the reason, good or bad, please send in your feedback.
You can find my contact information on the Real Python website. Or submit an issue on the
Real Python official support repository. Thank you!
NOTE: The code found in this course has been tested on Mac OS X v. 10.8.5,
Windows XP, Windows 7, Linux Mint 17, and Ubuntu 14.04 LTS.

14

Chapter 2
Introduction
Welcome to Advanced Web Programming!
Dear Reader:
Let me first congratulate you for taking the time to advance your skills and become a better
developer. Its never easy in todays hectic world to carve out the time necessary to spend on
something like improving your Python development skills. So Im not going to waste your
time, lets just jump right in.
1

# tinyp2p.py 1.0 (documentation at

http://freedom-to-tinker.com/tinyp2p.html)
import sys, os, SimpleXMLRPCServer, xmlrpclib, re, hmac # (C) 2004,
E.W. Felten
ar,pw,res = (sys.argv,lambda
u:hmac.new(sys.argv[1],u).hexdigest(),re.search)
pxy,xs =
(xmlrpclib.ServerProxy,SimpleXMLRPCServer.SimpleXMLRPCServer)
def ls(p=""):return filter(lambda n:(p=="")or
res(p,n),os.listdir(os.getcwd()))
if ar[2]!="client": # license:
http://creativecommons.org/licenses/by-nc-sa/2.0
myU,prs,srv = ("http://"+ar[3]+":"+ar[4], ar[5:],lambda
x:x.serve_forever())
def pr(x=[]): return ([(y in prs) or prs.append(y) for y in x] or
1) and prs
def c(n): return ((lambda f: (f.read(), f.close()))(file(n)))[0]
15

10

11

12
13

14
15

16

f=lambda p,n,a:(p==pw(myU))and(((n==0)and pr(a))or((n==1)and

[ls(a)])or c(a))
def aug(u): return ((u==myU) and pr()) or
pr(pxy(u).f(pw(u),0,pr([myU])))
pr() and [aug(s) for s in aug(pr()[0])]
(lambda sv:sv.register_function(f,"f") or
srv(sv))(xs((ar[3],int(ar[4]))))
for url in pxy(ar[3]).f(pw(ar[3]),0,[]):
for fn in filter(lambda n:not n in ls(),
(pxy(url).f(pw(url),1,ar[4]))[0]):
(lambda fi:fi.write(pxy(url).f(pw(url),2,fn)) or
fi.close())(file(fn,"wc"))

OK, let me explain this line by line no seriously - this is NOT what we are going to be doing.
While this script is a cool example of the power of Python, its painful to try to understand
whats happening with the code and nearly impossible to maintain without pulling your hair
out. Its a bit unfair to Ed Felten, the author of this code, because he wrote this intentionally
to fit the entire program in 15 lines. Its actually a pretty cool code example.
The point Im trying to make is this:
There are many ways to write Python code. It can be a thing of beauty; it can be something
very compact and powerful; it can even be a little bit scary. The inherent dynamic nature of
Python, plus its extreme flexibility, makes it both awesome and dangerous at the same time.
Thats why this book is not only about writing cool web apps with Django and Python; its
about harnessing the power of Python and Django in a way that not only benefits from the
immense power of the two - but in a way that is simple to understand, easy to maintain, and
above all fun to work with

16

Why this course?

In todays market theres an abundance of programmers, and because of the huge push from
the everybody must learn to code movement, that number is only going to increase.
In fact, enrollment in Computer Science degree programs for U.S. Universities rose by 29% in
2013. With everybody jumping in on the game, it may seem like the market will soon become
saturated with would-be employees all vying for the same number of limited jobs. However,
most high-tech companies tell a different story - the story of way too many resumes and not
enough talent. The truth is: Anybody can write code that looks like the sample from above,
but far fewer developers can write truly maintainable, well structured, easy to understand,
reliable code.
That is exactly why the major theme of this course is Software Craftsmanship.

17

What is Software Craftsmanship?

Software Craftsmanship is the idea that writing software is about more than hammering on
the keyboard until your program does what its supposed to do. Its about focusing on the
overall cost of ownership of the product - e.g., not only the costs of developing the software,
but the cost of fixing all the defects and maintaining the software for years to come.
This is precisely the reason that Django was developed. Billed as the framework for perfectionists with deadlines, if you understand its intricacies, its enjoyable to write beautiful,
well-factored code, and to write it with a deadline.

18

What will you learn?

Quality and maintainability
We will dive into this idea of well-factored code and quality right from the start.
Its important to place an emphasis on quality and maintainability because those are the hallmarks of modern day software development, and they dont just happen by chance. They
come about through the process of deliberately using a number of proven techniques and
practices, all of which you will learn.

Advanced Django
Although Software Craftsmanship should underlie all that we do, it is of course important
to understand the inner workings of the Django Framework so you can use it efficiently and
effectively. So, we will cover specific Django topics in nearly every chapter in this course. We
will cover all the usual suspects - like views, models and, templates as well as more advanced
topics such as database transactions, class-based views, mixins, the Django Rest Framework,
forms, permissions, custom fields, and so on.
This course is here to teach you all you need to know to develop great web applications that
could actually be used in a production environment, not just simple tutorials that gloss over
all the difficult stuff.

Full-stack Python
In todays world of HTML5 and mobile browsers, JavaScript is everywhere. Python and
Django just are not enough. To develop a website from end to end, you must know
the JavaScript frameworks, REST-based APIs, Bootstrap and responsive design, NoSQL
databases and all the other technologies that make it possible to compete at the global
level. Thats why this course does not just cover Django exclusively. Instead, it covers all
the pieces that you need to know to develop modern web applications from the client to the
server - developing, testing, tooling, profiling, and deploying.
By the end of this course you will have developed a REAL web based application ready to be
used to run an online business. Dont worry, we are not going to solely focus on buzzwords
and hype. Were going to have fun along the way and build something awesome.
Throughout the entire course we focus on a single application, so you can see how applications
evolve and understand what goes into making a REAL application.

19

What will you build?

We will start with the Django sample application introduced in the second course of this
series, Web Development with Python. Although its highly recommend to complete course
two before starting this one, there was only one chapter dedicated to that specific Django
application, and all the code is in the associated Github repository for this course.
To give you an idea of where well build, the current application looks like this:

Figure 2.1: Initial look

Its a nice start, but admittedly not something that you could take to Y Combinator to get
funded. However, by chapter 10 that same application will look like this:
And by the end of the course well, youre just going to have to work your way through the
course and find out.
20

Figure 2.2:21Chapter 10

Now lets build something REAL!

22

Chapter 3
Software Craftsmanship
Somewhere along the path to software development mastery every engineer comes to the
realization that there is more to writing code than just producing the required functionality.
Novice engineers will generally stop as soon as the requirements are met, but that leaves a
lot to be desired. Software development is a craft, and like any craft there are many facets to
learn in order to master it. True craftsman insist on quality throughout all aspects of a product development process; it is never good enough to just deliver requirements. McDonalds
delivers requirements, but a charcoal-grilled, juicy burger cooked to perfection on a Sunday
afternoon and shared with friends thats craftsmanship.
In this course we will talk about craftsmanship from the get go, because its not something
you can add in at the end. Its a way of thinking about software development that values:
Not only working software, but well-crafted software;
Moving beyond responding to change and focusing on steadily adding value; and,
Creating maintainable code that is well-tested and simple to understand.
This chapter is about the value of software craftsmanship, and writing code that is elegant,
easily maintainable and dare I say beautiful. And while this may seem high-browed, fluteytooty stuff, the path to achieving well-crafted software can be broken down into a number of
simple techniques and practices.
We will start off with the most important of these practices and the first that any engineer
looking to improve his or her craft should learn: Automated Unit Testing.

23

Automated unit testing

Automated unit testing is the foundation upon which good Software Craftsmanship practices
are built. Since this is a course on Django development, and the third course in the Real
Python series, we will start by taking a look at the Django MVP app from the last chapter of
the previous course, Web Development with Python by Michael Herman.
One of the first things you should do when you start work on an existing project is to run
the tests (if there are any of course). Further, since we have taken ownership of this project
its now our responsibility to ensure that the code is free from bugs, inconsistencies, or any
other problems that could cause issues down the road. This is achieved primarily through
unit testings. By adding unit tests to this application we will start to see how testing should
be done and, more importantly, how it can make you a better software developer.
Before we get our hands dirty, lets go over the testing tools we have at our disposable with
Django.

Behind the Scenes: Running unit tests with Django

There are several classes we can inherit from to make our testing easier in Django.
Provided test case classes

1. unittest.TestCase() - The default Python test case, which:

Has no association with Django, so theres nothing extra to load; because of this,
it produces the fastest test case execution times.
Provides basic assertion functionality.
2. django.test.SimpleTestCase() - A very thin wrapper around unittest.TestCase(),
which:
Asserts that a particular exception was raised.
Asserts on Form Fields - i.e., to check field validations.
Provides various HTML asserts, like assert.contains - which checks for an
HTML fragment.
Loads Django settings and allows for loading custom Django settings.
3. django.test.TestCase() or django.test.TransactionTestCase() - which
differ only by how they deal with the database layer.

24

Figure 3.1: Django Unittest Classes Hierarchy

25

 Both:
Automatically load fixtures to load data prior to running tests.
Create a Test Client instance, self.client, which is useful for things like
resolving URLS.
Provide a ton of Django-specific asserts.

TransactionTestCase() allows for testing database transactions and automatically resets the database at the end of the test run by truncating all tables.
TestCase() meanwhile wraps each test case in a transaction, which is then
rolled back after each test.
Thus, the major difference is when the data is cleared out - after each test for
TestCase() versus after the entire test run for TransactionTestCase().
4. django.test.LiveServerTest() - Used mainly for GUI Testing with something
like Selenium, which
Does basically the same thing as TransactionTestCase().
Also fires up an actual web server so you can perform GUI tests.
Which test base class should you use?

Keep in mind that, with each test base class, theres always a trade off between speed and test
complexity. As a general rule testing becomes easier and less complex the more you rely on
Djangos internal functionality. However, there is a cost- speed.
So, which test class is best? Unfortunately, there is no easy answer to that. It all depends
on the situation.

From a purely performance-oriented point of view, you should always use unittest.TestCase()
and not load any Django functionality. However, that may make it very difficult/cumbersome
to test certain parts of your application.
Any time you use one of the django.test.* classes you will be loading all the Django infrastructure and thus your test might be doing a lot more than what you bargained for. Take the
following example:
1
2
3

def test_uses_index_html_template(self):
index = self.client.get('/')
self.assertTemplateUsed(index, "index.html")

Its only two lines of code, right? True. But behind the scenes theres a lot going on:
26

The Django Test Client is fired up.

The Django Router is then called from self.client.get.
A Django request object is automatically created and passed to self.client.get.
A Django response object is returned.
Third party middleware is called.
Application middleware is also called.
Context Managers are called.

As you can see there is a lot going on here, we are testing multiple parts of the system. Test
that cover multiple parts of a system and verify the result when all these parts act in concert
are generally referred to as an ** Integration Test **.
By contrast a unit test by definition should test a small defined area of code (i.e. a unit of
code), in isolation. Which in most cases translates to a single function.
To rewrite this test as a unit test we could inherit from django.test.SimpleTestCase() and
then our test would look like this:
1

from django.test import RequestFactory

2
3
4
5
6
7

def test_uses_index_html_template_2(self):
#create a dummy request
request_factory = RequestFactory()
request = request_factory.get('/')
request.session = {} #make sure it has a session associated

8
9
10

#call the view function

resp = index(request)

11
12
13

#check the response

self.assertContains(resp, "<title>Your MVP</title>")

Obviously this test is a bit longer than two lines, but since we are creating a dummy request
object and calling our view function directly, we no longer have so much going on behind
the scenes. There is no router, there is no middleware; basically all the Django magic isnt
happening. This means we are testing our code in isolation (its a unit test), which has the
following consequences:
Tests will likely run faster, because there is less going on (again, behind the scenes).
Tracking down errors will be significantly easier because only one thing is being tested.

27

 Writing tests can be harder because you may have to create dummy objects or stub out
various parts of the Django system to get your tests to work.
NOTE: A Note on assertTemplateUsed Notice that in the above test we
have changed the assertTemplateUsed to assertContains. This is because
assertTemplateUsed does not work with the RequestFactory(). In fact, if
you use the two together the assertTemplateUsed assertion will always pass,
no matter what template name you pass in! So be careful not to use the two
together!
Integration vs unit testing

Both integration and unit tests have their merits, and you can use both types together in the
same test suite - so its not all-or-nothing. But it is important to understand the differences of
each approach so you can choose which option is best for you. There are some other considerations, specifically designing testable code and mocking, which we will touch upon later.
For now the important thing to understand is the difference between the two types and the
amount of work that the Django testing framework is doing for you behind the scenes.

28

Testing Routes and Views

Setup
If you havent gone through the second Real Python course, Web Development with Python,
yet and you want to get a broad overview of several of the most popular web frameworks out
there - like Flask, Django, and web2py - you really should have a look at it here. If you ask
nicely, by emailing [email protected], you can probably get a discount.
However, if you just want to just get started on this course, you can grab a copy of the Django
MVP application by using the supplied repo for this course. Click here to download the compressed file directly to your file system. Once downloaded, extract the files.

Unit Testing in Django

Django makes unit testing pretty straight-forward. Unit testing is generally accomplished by
using the Django helper classes defined in django.test.testcases. These helper classes
extend the standard Python unittest framework by providing a number of helper functions
that make it much simpler to test Django applications.
If you think about how most Django applications are structured, they loosely follow the MVC
(Model-View-Controller) model. But to make things confusing, a view in Django is actually
a controller in MVC and a Django template is an MVC view. See the Django docs for more
details. Either way, Django applications generally follow the standard activity patterns (as
do most web frameworks):
1. Receive a web request.
2. Route that request to the appropriate Django view.
3. Perform some business logic, which could include accessing a datastore through a
Django model.
4. Return an HTTP Response.
Thus, we want to test:
1.
2.
3.
4.

That requests are routed properly.

Appropriate responses are returned.
The underlying business logic.
Datastore access through a Django model.

29

Out of the box, Django gives us two standard ways to write unit tests - using the standard
unittest style or the DocTest style.
SEE ALSO: Please consult the Python documentation for more info in both standard unittest and doctests. Jump to this StackOverflow question to get more info
on when to use unittest vs. doctest.

Both are valid and the choice is more a matter of preference than anything else. Since the
standard unittest style is the most common, we will focus on that.

Built-in Django tests

Regardless of which you choose, the first thing you should do before you start creating your
own tests is to verify that your Django environment is setup correctly. This can be done by
running Djangos built-in tests. If you have worked through the previous course, you should
have a virtual environment setup and ready to go for the MVP app. If not, set one up. Or you
can use a more powerful tool called virtualenvwrapper, which you will find instructions for
below.
WARNING: Stop here if you either dont know how to set up a basic virtual
environment with virtualenv or if you dont know what its used for. These are
strong indicators that you need to go through the second Real Python course.
Baby steps.

If youre continuing from the previous course, make sure you have your virtualenv activated
and ready to go by running:
1

$ source myenv/bin/activate

If youre starting from the downloaded application:

1. Ensure you have virtualenvwrapper installed, which is a wrapper around virtualenv
that makes it easier to use:
1

$ pip install virtualenvwrapper

2. Then set up the virtual environment with virtualenvwrapper:

1
2
3

$ cd django_project
$ mkvirtualenv chp4
$ pip install -r ./requirements.txt
30

This will read the requirements.txt file in the root directory of the django_ecommerce
project and use pip to install all the dependencies listed in the requirements.txt file,
which at this point should just be Django and Stripe.
NOTE: Using a requirements.txt file as part of a Python project is a great
way to manage dependencies. Its a common convention that you need to
adhere to for any Python project, not just Django. A requirements.txt file is
a simple text file that lists one dependency per line of your project; pip will
read this file and all the dependencies will then be downloaded and installed
automatically.

After you have your virtual environment set up and all your dependencies installed, type pip freeze > requirements.txt to automatically create
the requirements.txt file that can later be used to quickly install the project
dependencies. Then check the file into git and share it with your team, and
everybody will be using the same set of dependencies.
More information about requirements.txt files can be found on the official
docs page.
3. OPTIONAL: If for whatever reason, the requirements.txt is not found, then we need to
set one up. No worries. Its good practice.
Start by installing the following dependencies:
1
2

$ pip install django==1.5.5

$ pip install stripe==1.9.2

Now just create the requirements.txt file in the root directory (repo):
1

$ pip freeze > requirements.txt

Thats it.
4. Take a quick glance at the project structure, paying particular attention to the tests.py
files. See anything that stands out. Well, there are no tests, except for the generic 2+2
test:
1

from django.test import TestCase

2
3
4
5
6
7

class SimpleTest(TestCase):
def test_basic_addition(self):
"""
Tests that 1 + 1 always equals 2.
31

8
9

"""
self.assertEqual(1 + 1, 2)

So, we need to first add tests to ensure that this current app works as expected without
any bugs, otherwise we cannot confidently implement new features.
5. Once you have all your dependencies in place, lets run a quick set of tests to ensure
Django is up and running correctly:
1
2

$ cd django_ecommerce
$ python manage.py test

I generally just chmod +x manage.py. Once you do that, you can just run:
1

$ ./manage.py test

Not a huge difference, but over the long-run that can save you hundreds of keystrokes.
NOTE: You could also add the alias alias p="python" to your ~/.bashrc
file so you can just run p instead of python: p manage.py test.

Nevertheless, after running the tests you should get some output like this:
1
2
3
4
5
6
7
8
9
10
11
12
13

Creating test database for alias 'default'...

.....................................................
.....................................................
........................................s............
.....................................................
.....................................................
...............x.....................................
.....................................................
.....................................................
.....................................................
....................................................
----------------------------------------------------Ran 530 tests in 15.226s

14
15
16

OK (skipped=1, expected failures=1)

Destroying test database for alias 'default'...

Notice that with Django 1.5 there will always be one failure - (expected failures=1);
thats expected, so just ignore it. If you get an output like this, than you know your
system is correctly configured for Django. Now its time to test our own app.
32

However, if you get an error about ValueError: unknown local e: UTF-8 this is the
dreaded Mac UTF-8 hassle. Depending on who you ask, its a minor annoyance in
Macs default terminal to the worst thing that could ever happen. In truth, though, its
easy to fix. From the terminal simply type the following:
1
2

$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8

Feel free to use another language if you prefer. With that out of the way, your
tests should pass just fine. Before moving on, though, add these two lines to your
.bash_profile file so you dont have to retype them each time you start your terminal.
NOTE With virtualenvwrapper, to deactivate you still use the deactivate
command. When youre ready to reactivate the virtualenv to work on your
project again simply type workon django_mvp_app. You can also just
type workon to view your current working virtual environments. Simple.

Testing Django Routing

Looking back at our Django MVP app, we can see it provides routing for a main page, admin
page, sign_in, sign_out, register, contact and edit. These can be seen in django_ecommerce/urls.py:
1
2
3
4
5

urlpatterns = patterns('',
url(r'^admin/', include(admin.site.urls)),
url(r'^$', 'main.views.index', name='home'),
url(r'^pages/', include('django.contrib.flatpages.urls')),
url(r'^contact/', 'contact.views.contact', name='contact'),

# user registration/authentication
url(r'^sign_in$', views.sign_in, name='sign_in'),
url(r'^sign_out$', views.sign_out, name='sign_out'),
url(r'^register$', views.register, name='register'),
url(r'^edit$', views.edit, name='edit'),

7
8
9
10
11
12

Our first set of tests will ensure that this routing works correctly and that the appropriate
views are called for the corresponding URLs. In effect we are answering the question, Is my
Django application wired up correctly?
This may seem like something too simple to test, but keep in mind that Djangos routing
functionality is based upon regular expressions, which are notoriously tricky and hard to
understand.
33

NOTE: For more on regular expressions, be sure to check out the excellent chapter in the first Real Python course, Introduction to Python.

Furthermore, Djangos routing functionality follows a first-come-first-serve model, so its not

at all uncommon to include a new url pattern that accidentally overwrites an old one. Once
we have these tests in place and we run them after each change to the application, then we
will be quickly notified if we make such a mistake, allowing us to quickly resolve the problem
and move on with our work, confidently.
First test: resolve

Here is what our first test might look like:

main/tests.py
1
2
3

from django.test import TestCase

from django.core.urlresolvers import resolve
from .views import index

4
5
6

class MainPageTests(TestCase):

7
8
9
10

def test_root_resolves_to_main_view(self):
main_page = resolve('/')
self.assertEqual(main_page.func, index)

Here we are using Djangos own urlresolver (docs) to test that the / URL resolves, or
maps, to the main.views.index function. This way we know that our routing is working
correctly.
Want to test this in the shell to see exactly what happens?
1
2
3
4
5

$ ./manage.py shell
>>> from django.core.urlresolvers import resolve
>>> main_page = resolve('/')
>>> print main_page.func
<function index at 0x102270848>

This just shows that the / URL does in fact map back to the main.views.index
function. You could also test the url name, which is called home - url(r'^$',
'main.views.index', name='home'),:

34

1
2
3
4
5

$ ./manage.py shell
>>> from django.core.urlresolvers import resolve
>>> main_page = resolve('/')
>>> print main_page.url_name
home

Perfect.
Now, run the tests:
1

$ ./manage.py test main

We should see the test pass

1
2
3
4

Creating test database for alias 'default'...

.
---------------------------------------------------------------------Ran 1 test in 0.633s

5
6

OK

Great!
You could test this in a slightly different way, using reverse(), which is also a urlresolver:
1
2
3
4
5

$ ./manage.py shell
>>> from django.core.urlresolvers import reverse
>>> url = reverse('home')
>>> print url
/

Can you tell what the difference is, though, between resolve() and reverse()? Check out
the Django docs for more info.
Second test: status code

Now that we know our routing is working correctly lets ensure that we are returning the appropriate view. We can do this by using the client.get() function from
django.tests.TestCase (docs. In this scenario, we will send a request from a dummy
web browser known as the Test Client then assert that the returned response is correct. More
on the Test Client later.
A basic test for this might look like:

35

1
2
3

def test_returns_appropriate_html(self):
index = self.client.get('/')
self.assertEquals(index.status_code, 200)
NOTE: Note on Test Structure

In Django 1.5 the default place to store tests is in a file called tests.py inside the
each applications directory - which is why these tests are in main/tests.py.
For the remainder of this chapter we are going to stick to that standard and put
all tests in app_name/tests.py files. In a later chapter we will discuss more on
the advantages/disadvantages of this test structure. For now lets just focus on
testing.
Test again:
1
2
3
4

Creating test database for alias 'default'...

..
---------------------------------------------------------------------Ran 2 tests in 0.759s

5
6

OK

Testing Templates and Views

The issue with the last test is that even if youre returning the incorrect HTML, it will still
pass.
First test: template

Its better to verify the actual template used:

1
2
3

def test_uses_index_html_template(self):
index = self.client.get('/')
self.assertTemplateUsed(index, "index.html")

So, assertTemplateUsed() is one of those helper function provided by Django, which just
checks to see if a HTTPResponse object gets generated by a specific template. Check out the
Django docs for more info.

36

Second test: expected HTML

We can further increase our test coverage to verify that not only are we using the appropriate template but that the HTML being returned is correct as well. In other words, after the
template engine has done its thing, do we get the expected HTML?
The test looks like this:
1

from django.shortcuts import render_to_response

2
3
4
5
6
7
8

def test_returns_exact_html(self):
index = self.client.get("/")
self.assertEquals(
index.content,
render_to_response("index.html").content
)

In the above test, the render_to_response() shortcut function is used to run our index.html template through the Django template engine and ensure that the response is the
same had an end user called the / url.
Third test: expected HTML redux

The final thing to test for the index view is that it returns the appropriate html for a loggedin user. This is actually a bit trickier than it sounds, because the index() function actually
performs three separate functions:
1. It checks the session to see if a user is logged in.
2. If the user is logged in, it queries the database to get that users information.
3. Finally, it returns the appropriate HTTP Response.
Now is the time that we may want to argue for refactoring the code into multiple functions to
make it easier to test. But for now lets assume we arent going to refactor and we want to test
the functionality as is. This means that really we are executing a Integration Test instead
of a Unit Test, because we are now testing several parts of the application as opposed to just
one individual unit.
NOTE: Notice how during this process we found an issue in our code - e.g., one
function having too many responsibilities. This is a side effect of testing: You
learn to write better code.

37

Forgetting about the distinction for the time being, lets look at how we might test this functionality:
1. Create a dummy session with the user entry that we need.
2. Ensure there is a user in the database, so our lookup works.
3. Verify that we get the appropriate response back.
Lets work backwards:

def test_index_handles_logged_in_user(self):

2
3

# test logic will go here

4
5

self.assertTemplateUsed(resp, 'user.html')

We have seen this before: See the first test in this section, test_uses_index_html_template().
Above we are just trying to verify that when there is a logged in user, we return the user.html
template instead of the normal index.html template.

1
2
3
4
5
6
7
8

def test_index_handles_logged_in_user(self):
# create the user needed for user lookup from index page
from payments.models import User
user = User(
name='jj',
email='[email protected]',
)
user.save()

9
10

...snipped code...

11
12
13

# verify the response returns the page for the logged in user
self.assertTemplateUsed(resp, 'user.html')

Here we are creating a user and storing that user in the database. This is more or less the
same thing we do when a new user registers.

38

Take a breather Before continuing, be sure you understand exactly what happens when
you run a Django unittest. Flip back to the last chapter if you need a quick refresher.

Do you understand why we need to create a user to test this function correctly if our
main.views.index() function is working? Yes? Move on. No? Flip back and read about
the django.test.TestCase(), focusing specifically on how it handles the database layer.
Ready?
Move on Want the answer? Its simple: Its because the django.test.TestCase class
that we are inheriting from will clear out the database prior to each run. Make sense?

Also, you may not have noticed, but we are testing against the actual database. This is problematic, but there is a solution. Sort of. We will get to this later. But first, think about the
request object that is passed into the view. Here is the view code we are testing:
1
2
3
4
5
6
7
8
9

def index(request):
uid = request.session.get('user')
if uid is None:
return render_to_response('index.html')
else:
return render_to_response(
'user.html',
{'user': User.objects.get(pk=uid)}
)

As you can see, the view depends upon a request object and a session. Usually these are
managed by the web browser and will just be there, but in the context of unit testing they
wont just be there. We need a way to make the test think there is a request object with a
session attached.
And there is a technique in unit testing for doing this.
Mocks, fakes, test doubles, dummy objects, stubs There are actually many names
for this technique: mocking, faking, using dummy objects, test doubling, stubbing,
etc.

There are subtle difference between each of these, but the terminology just adds confusion.
For the most part, they are all referring to the same thing - which we will define as: Using
a temporary, in-memory object to simulate a real object for the purposes of decreasing test
complexity and increasing test execution speed.
Visually you can think of it as:

39

Figure 3.2: mocking example

40

As you can see from the illustration, a mock (or whatever you want to call it) simply takes
the place of a real object allowing you to create tests that dont rely on external dependencies.
This is advantageous for two main reasons:
1. Without external dependencies your tests often run much faster; this is important because unit tests should be ran frequently (ideally after every code change).
2. Without external dependencies its much simpler to manage the necessary state for a
test, keeping the test simple and easy to understand.
Coming back to the request and its state - the user value stored in the session - we can mock
out this external dependency by using Djangos built-in RequestFactory():
1
2
3
4
5

# create a dummy request

from django.test import RequestFactory
request_factory = RequestFactory()
request = request_factory.get('/')
request.session = {} # make sure it has an associated session

By creating the request mock, we can set the state (i.e. our session values) to whatever we want
and simply write and execute unit tests to ensure that our view function correctly responds
to the session state.
Step 1: Verify that we get the appropriate response back And with that we can complete our earlier unit test:
1
2

from payment.models import User

from django.test import RequestFactory

3
4
5
6
7
8
9
10
11

def test_index_handles_logged_in_user(self):
#create the user needed for user lookup from index page
from payments.models import User
user = User(
name='jj',
email='[email protected]',
)
user.save()

12
13
14
15

#create a Mock request object, so we can manipulate the session

request_factory = RequestFactory()
request = request_factory.get('/')
41

#create a session that appears to have a logged in user

request.session = {"user": "1"}

16
17
18

#request the index page

resp = index(request)

19
20
21

#verify it returns the page for the logged in user

self.assertEquals(
resp.content,
render_to_response('user.html', {'user': user}).content
)

22
23
24
25
26

Run the tests:

1
2
3
4

Creating test database for alias 'default'...

....
---------------------------------------------------------------------Ran 4 tests in 0.964s

5
6
7

OK
Destroying test database for alias 'default'...

Good.
Lets recap

Our test_index_handles_logged_in_user() test:

1. First creates a user in the database.
2. Then we mock the request object and set the session to have a user with an id of 1
(the user we just created). In other words, we are setting the (initial) state so we can
test that the index view responds correctly when we have a logged-in user.
3. Then we call the index view function, passing in our mock request then verifying that
it returns the appropriate HTML.
At this point we have the test completed for our main app. You may want to review the tests.
Would you do anything different? Experiment. Challenge yourself. Theres always better,
more efficient ways of accomplishing a task. Learn how to fail. Struggle. Ask questions.
Experience a few wins. Then learn how to succeed.
Before we moving to testing the models, we should have a look at refactoring some of our
tests.
42

Refactoring our Tests

Refactoring, part 1
Looking back at the tests, there are some discrepancies, as some of the tests use the
RequestFactory() mock while others dont. So, lets clean up/refactor the tests
a bit so that all of them will use request mocks. Why? Simple: This should make
them run faster and help keep them isolated (more on this below). However, the
test_root_resolves_to_main_view() tests the routing so we should not mock out
anything for that test. We really do want it to go through all the heavy Django machinery to
ensure that its properly wired up.
To clean up the tests, the first thing we should do is create a setUp method which will run
prior to test execution. This method will create the mock so we dont have to recreate it for
each of our tests:
1
2
3
4
5

@classmethod
def setUpClass(cls):
request_factory = RequestFactory()
cls.request = request_factory.get('/')
cls.request.session = {}

Notice the above method, setUpClass(), is a @classmethod. This means it only gets run
once when the MainPageTest class is initially created, which is exactly what we want for our
case. In other words, the mock is created when the class is initialized, and it gets used for
each of the tests. See the Python docs for more info.
If we instead wanted the setup code to run before each test, we could write it as:
1
2
3
4

def setUp(self):
request_factory = RequestFactory()
self.request = request_factory.get('/')
self.request.session = {}

This will run prior to each test running. Use this when you need to isolate each test run.
After we have created our setUp method, we can then update our remaining test cases as
follows:
1
2
3
4

from
from
from
from

django.test import TestCase

django.core.urlresolvers import resolve
.views import index
django.shortcuts import render_to_response
43

5
6

from payments.models import User

from django.test import RequestFactory

7
8
9

class MainPageTests(TestCase):

10
11
12
13

###############
#### Setup ####
###############

14
15
16
17
18
19

@classmethod
def setUpClass(cls):
request_factory = RequestFactory()
cls.request = request_factory.get('/')
cls.request.session = {}

20
21
22
23

##########################
##### Testing routes #####
##########################

24
25
26
27

def test_root_resolves_to_main_view(self):
main_page = resolve('/')
self.assertEqual(main_page.func, index)

28
29
30
31

def test_returns_appropriate_html_response_code(self):
resp = index(self.request)
self.assertEquals(resp.status_code, 200)

32
33
34
35

#####################################
#### Testing templates and views ####
#####################################

36
37
38
39
40
41
42

def test_returns_exact_html(self):
resp = index(self.request)
self.assertEquals(
resp.content,
render_to_response("index.html").content
)

43
44

def test_index_handles_logged_in_user(self):
44

# Create the user needed for user lookup from index page
user = User(
name='jj',
email='[email protected]',
)
user.save()

45
46
47
48
49
50
51

# Create a session that appears to have a logged in user

self.request.session = {"user": "1"}

52
53
54

# Request the index page

resp = index(self.request)

55
56
57

#ensure we return the state of the session back to normal so

#we don't affect other tests
self.request.session = {}

58
59
60
61

#verify it returns the page for the logged in user

expectedHtml = render_to_response('user.html', {'user':
user}).content
self.assertEquals(resp.content, expectedHtml)

62
63

64

Looks pretty much the same.

But all the tests, with the notable exception of test_root_resolves_to_main_view(),
are now isolated from any Django URL routing or middleware, so the tests should run faster
be more robust and easier to debug should there be any problems.

Sanity check
Run the test!
1
2
3
4

Creating test database for alias 'default'...

....
---------------------------------------------------------------------Ran 4 tests in 0.667s

5
6
7

OK
Destroying test database for alias 'default'...

45

Refactoring, part 2
We still have one test, though, test_index_handles_logged_in_user() that is dependent on the database and, thus, needs to be refactored. Why? As a general rule. you do not
want to test the database with a unit test. That would be an integration test. Mocks are perfect
for handling calls to the database.
First, install the mock library:
1

$ pip install mock

Our test then might look like this:

1
2

import mock
from payments.models import User

3
4
5
6
7
8
9
10

def test_index_handles_logged_in_user(self):
# Create the user needed for user lookup from index page
# Note that we are not saving to the database
user = User(
name='jj',
email='[email protected]',
)

11
12
13

# Create a session that appears to have a logged in user

self.request.session = {"user": "1"}

14
15

with mock.patch('main.views.User') as user_mock:

16
17
18
19

# Tell the mock what to do when called

config = {'get.return_value': user}
user_mock.objects.configure_mock(**config)

20
21
22

# Run the test

resp = index(self.request)

23
24

25

# Ensure that we return the state of the session back to

normal
self.request.session = {}

26
27
28

expectedHtml = render_to_response(
'user.html', {'user': user}).content
46

29

self.assertEquals(resp.content, expectedHtml)

Test it again. All four should still pass.

Whats happening here?

1. with mock.path('main.views.User')as user_mock says, When we come

across a User object in the main.views module, call our mock object instead of the
actual user object.
2. Next we call user_mock.objects.configure_mock and pass in our config dictionary, which says, when get is called on our User.objects mock just return our
dummy user that we created at the top of the function.
3. Finally we just ran our test as normal and asserted that it did the correct thing.
Now our view method gets the value it needs from the database without ever actually touching
the database.
Since we now have no reliance on the Django testing framework, we actually could inherit directly from unittest.TestCase(). This would further isolate our tests from Django, which
should speed them up.
Because of this, many Python developers believe you should mock out everything. Is that the
best choice, though? Probably not. But well talk more about why in the next chapter when
we test our models. Regardless, the mock framework is extremely powerful, and it can be a
very useful tool to have in your Django unit testing tool belt. You can learn more about the
mock library here.
Now, lets move on to testing models.

47

Testing Models
In the previous section we used the Mock library to mock out the Django ORM so we could
test a Model without actually hitting the database. This can make tests faster, but its actually
NOT a good approach to testing models. Lets face it, ORM or not, your models are tied to
a database - and are intended to work with a database. So testing them without a database
can often lead to a false sense of security.
The vast majority of model-related errors happen within the actual database. Generally
database errors fall into one of the four following categories:

Migration: Working off an out of date database.

Datatype: Trying to store a string in a number column.
Referential / integrity Constraints: Storing two rows with the same id.
Retrieval: Using incorrect table JOINs.

All of these issues wont occur if youre mocking out your database, because you are effectively
cutting out the database from your tests where all these issues originate from. Hence, testing
models with mocks provides a false sense of security, letting these issues slip through the
cracks.
This does not mean that you should never use mocks when testing models; just be very careful
youre not convincing yourself that you have fully tested your model when you havent really
tested it at all.
That said, lets look at several techniques for safely testing Django Models.
WARNING: Its worth noting that many developers do not run their tests in development with the same database engine used in their production environment.
This can also be a source of errors slipping through the cracks. In a later chapter,
well look at using Travis CI to automate testing after each commit including how
to run the tests on multiple databases.

Technique #1 - Just dont test them

Dont laugh. Its not a joke. After all, models are simply just a collection of fields that rely on
standard Django functionality. We shouldnt test standard Django functionality because we
can always run manage.py test and all the tests for standard Django functionality is tested
for us. If the code is built-in to Django do not test it. This includes the fields on a model.
Specifically, look at the payments.models.User model:
48

1
2

from django.db import models

from django.contrib.auth.models import AbstractBaseUser

3
4
5
6
7
8
9

10
11
12

class User(AbstractBaseUser):
name = models.CharField(max_length=255)
email = models.CharField(max_length=255, unique=True)
# password field defined in base class
last_4_digits = models.CharField(max_length=4, blank=True,
null=True)
stripe_id = models.CharField(max_length=255)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)

13
14

USERNAME_FIELD = 'email'

15
16
17

def __str__(self):
return self.email

What do we really need to test here?

Many tutorials have you writing tests like this:
1
2

from django.test import TestCase

from .models import User

3
4
5

class UserModelTest(TestCase):

6
7
8

def test_user_creation(self):
User(email = "[email protected]", name='test user').save()

9
10
11

users_in_db = User.objects.all()
self.assertEquals(users_in_db.count(), 1)

12
13
14
15

user_from_db = users_in_db[0]
self.assertEquals(user_from_db.email, "[email protected]")
self.assertEquals(user_from_db.name, "test user")

Congratulations. You have verified that the Django ORM can indeed store a model correctly.
But we already knew that, didnt we? Whats the point then? Not much, actually. Remember:
We do not need to test Djangos inherent functionality.
49

What about custom functionality?

Technique #2 - Create data on demand

Rather than spend a bunch of time testing stuff we dont really need to test, try to follow this
rule: Only test custom functionality that you created for models.
Is there any custom functionality in our User model? Not much.
Arguably, we may want to test that our model uses an email address for the USERNAME_FIELD
just to make sure that another developer doesnt change that. We could also test that the user
prints out with the associated email.
To make things a bit more interesting, lets add a custom function to our model.
We can add a find_user() function. This will replace calls to User.objects.get(pk)
like in the main.views.index() function:
1
2
3
4
5
6
7
8
9

def index(request):
uid = request.session.get('user')
if uid is None:
return render_to_response('index.html')
else:
return render_to_response(
'user.html',
{'user': User.objects.get(pk=uid)}
)

Isnt that the same thing? Not exactly. In the above code the view would have to know about
the implementation details in our model, which breaks encapsulation. In reality, for small
projects this doesnt really matter, but in large projects it can make the code difficult to maintain.
Imagine you had 40 different views, and each one uses your User model in a slightly different
way, calling the ORM directly. Do you see the problem? In that situation its very difficult to
make changes to your User model, because you dont know how other objects in the system
are using it.
Thus, its a much better practice to encapsulate all that functionally within the User model
and as a rule everything should only interact with the User model through its API, making it
much easier to maintain.

50

Update the model

Without further ado, here is the change to our payments models.py file:
1
2

from django.db import models

from django.contrib.auth.models import AbstractBaseUser

3
4
5
6
7
8
9

10
11
12

class User(AbstractBaseUser):
name = models.CharField(max_length=255)
email = models.CharField(max_length=255, unique=True)
#password field defined in base class
last_4_digits = models.CharField(max_length=4, blank=True,
null=True)
stripe_id = models.CharField(max_length=255)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)

13
14

USERNAME_FIELD = 'email'

15
16
17
18

@classmethod
def get_by_id(cls, uid):
return User.objects.get(pk=uid)

19
20
21

def __str__(self):
return self.email

Notice the @classmethod at the bottom of the code listing. We use a @classmethod because
the method itself is stateless, and when we are calling it we dont have to create the User object,
rather we can just call User.get_by_id.
Test the model

Now we can write some tests for our model. Create a tests.py file in the payments folder
and add the following code:
1
2

from django.test import TestCase

from payments.models import User

3
4
5

class UserModelTest(TestCase):
51

6
7
8
9
10

@classmethod
def setUpClass(cls):
cls.test_user = User(email="[email protected]", name='test user')
cls.test_user.save()

11
12
13

def test_user_to_string_print_email(self):
self.assertEquals(str(self.test_user), "[email protected]")

14
15
16

def test_get_by_id(self):
self.assertEquals(User.get_by_id(1), self.test_user)

Test!
1
2
3
4

Creating test database for alias 'default'...

..
---------------------------------------------------------------------Ran 2 tests in 0.003s

5
6
7

OK
Destroying test database for alias 'default'...

Whats happening here? In these tests, we dont use any mocks; we just create the User
object in our setUpClass() function and save it to the database. This is our second technique for testing models in action: Just create the data as needed..
WARNING: Be careful with the setUpClass(). While using it in the above
example can speed up our test execution - because we only have to create one
user - it does cause us to share state between our tests. In other words, all of our
tests are using the same User object, and if one test were to modify the User
object, it could cause the other tests to fail in unexpected ways. This can lead
to VERY difficult debugging sessions because tests could seemingly fail for no
apparent reason. A safer strategy, albeit a slower one, would be to create the
user in the setUp() method, which is run before each test run. You may also
want to look at the tearDown() method and use it to delete the User model after
each test is run if you decide to go the safer, latter route. As with most things in
development its a trade-off. If in doubt though I would go with the safer option
and just be a little more patient.

52

So, these tests provide some reassurance that our database is in working order because we
are round-tripping - writing to and reading from - the database. If we have some silly
configuration issue with our database, this will catch it, in other words.

NOTE: In effect, testing that the database appears sane is a side effect of our
test here - which again means, by definition, these are not unit tests - they are
integration tests. But for testing models the majority of your tests should really
be integration tests.
Update the view and the associated test

Now that we have added the get_by_id() function update the main.views.index() function accordingly.
1
2
3
4
5
6
7
8

def index(request):
uid = request.session.get('user')
if uid is None:
return render_to_response('index.html')
else:
return render_to_response(
'user.html', {'user': User.get_by_id(uid)}
)

Along with the test:

1

import mock

2
3
4
5

def test_index_handles_logged_in_user(self):
# Create a session that appears to have a logged in user
self.request.session = {"user": "1"}

6
7

with mock.patch('main.views.User') as user_mock:

8
9
10
11

# Tell the mock what to do when called

config = {'get_by_id.return_value': mock.Mock()}
user_mock.configure_mock(**config)

12
13
14

# Run the test

resp = index(self.request)

15
16

# Ensure we return the state of the session back to normal

53

17

self.request.session = {}

18
19
20
21
22

expected_html = render_to_response(
'user.html', {'user': user_mock.get_by_id(1)}
)
self.assertEquals(resp.content, expected_html.content)

This small change actually simplifies our tests. Before we had to mock out the ORM - i.e.
User.objects, and now we just mock the model.
This makes a lot more sense as well, because previously we were mocking the ORM when we
were testing a view. That should raise a red flag. We shouldnt have to care about the ORM
when we are testing views; after all, thats why we have models.

Technique #3 - Use fixtures

It should be mentioned that Django provides built-in functionality for automatically loading
and populating model data: fixtures.
The idea is you can create a data file in such formats as XML, YAML or JSON and then ask
Django to load that data into your models. This process is described in the official documentation and is sometimes used to load test data for unit testing.
Ill come right out and say it: I dont like this approach. There are several reasons why:
It requires you to store data in a separate file, creating an additional place to look if
youre debugging test errors.
It can be difficult to keep the file up-to-date as your model changes.
Its not always obvious what data in your test fixture applies to which test case.
Slow performance is often given as another reason why developers dont like it as well, but
that can generally be rectified by using a SQLite database.
Our example MVP application is already using a SQLite database, but if your application is
using some other SQL database engine, and you want your tests to run in SQLite so they will
run faster, add the following line to your settings.py file. Make sure it comes after your
actual database definition:
1
2
3
4

import sys
# Covers regular testing and django-coverage
if 'test' in sys.argv or 'test_coverage' in sys.argv:
DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'
54

Now anytime you run Django tests, Django will use a SQLite database so you shouldnt notice
much of a performance hit at all. Awesome!
NOTE: Do keep in mind the earlier warning about testing against a different
database engine in development than production. Well come back to this and
show you how you can have your cake and eat it to! :)

Even though out of the box fixtures are difficult, there are libraries that make using fixtures
much more straight-forward.
Fixture libraries

Model Mommy and django-dynamic-fixtures are both popular libraries.

These fixture libraries are actually closer to the second technique, Create data on Demand,
than they are to the current technique. Why? Because you dont use a separate data file.
Rather, these type of libraries create models for you with random data.
The school of thought that most of these libraries subscribe to is that testing with static data
is bad. However, these arguments are far from convincing. That said, these libraries do
make it super easy to generate test data, and if youre willing to add another library to your
requirements.txt file then you can pull them in and use them.
Check out these write-ups on Django fixture libraries for more info:
ModelMommy writeup
Django-dynamic-fixtures writeup
Have a look at those articles to get started if you so choose. I prefer just using the models
directly to create the data that I need for two reasons:
1. I dont really buy into the whole static test data is bad.
2. If its cumbersome to create test data with your models, maybe its a code smell telling
you to make your models easier to use. Or in more general terms, one of the most
valuable aspects of unit testing is its ability to drive good design. Think of unit tests
not only as functional tests of your code, but also as usabilities tests of your API. If its
hard to use, its time to refactor.
Of course youre free to use these libraries if you like. I wont give you too much of a hard
time about it.
55

Testing Forms
We have a number of forms in our application, so lets go ahead and test those as well.

payments.forms has several forms in there. Lets start simple with the SigninForm. To
refresh you memory, it looks like this:
1
2

from django import forms

from django.core.exceptions import NON_FIELD_ERRORS

3
4
5
6
7

class SigninForm(PaymentForm):
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
widget=forms.PasswordInput(render_value=False)
)

When it comes to forms in Django, they generally have the following lifecycle:
1.
2.
3.
4.

A form is generally instantiated by a view.

Some data is populated in the form.
Data is validated against form rules.
Data is then cleaned and passed on.

We want to test by first populating data in the form and then ensuring that it validates correctly.

Mixins
Because most form validation is very similar, lets create a mixin to help us out. If youre
unfamiliar with mixins StackOverflow can help.
Add the following code to payments.tests:
1

class FormTesterMixin():

2
3
4

def assertFormError(self, form_cls, expected_error_name,

expected_error_msg, data):

5
6
7

from pprint import pformat

test_form = form_cls(data=data)
56

8
9

#if we get an error then the form should not be valid

self.assertFalse(test_form.is_valid())

10
11
12
13
14
15
16
17
18

self.assertEquals(
test_form.errors[expected_error_name],
expected_error_msg,
msg="Expected {} : Actual {} : using data {}".format(
test_form.errors[expected_error_name],
expected_error_msg, pformat(data)
)
)

This guy makes it super simple to validate a form. Just pass in the class of the form, the name
of the field expected to have an error, the expected error message, and the data to initialize
the form. Then, this little function will do all the appropriate validation and provide a helpful
error message that not only tells you the failure - but what data triggered the failure.
Its important to know what data triggered the failure because we are going to use a lot of data
combinations to validate our forms. This way it becomes easier to debug the test failures.
Test

Here is an example of how you would use the mixin:

1
2

from payments.forms import SigninForm

import unittest

3
4
5

class FormTests(unittest.TestCase, FormTesterMixin):

6
7
8
9
10
11
12
13

def test_signin_form_data_validation_for_invalid_data(self):
invalid_data_list = [
{'data': {'email': '[email protected]'},
'error': ('password', [u'This field is required.'])},
{'data': {'password': '1234'},
'error': ('email', [u'This field is required.'])}
]

14
15
16
17

for invalid_data in invalid_data_list:

self.assertFormError(SigninForm,
invalid_data['error'][0],
57

invalid_data['error'][1],
invalid_data["data"])

18
19

Lets test: ./manage.py test payments

1
2
3
4

Creating test database for alias 'default'...

...
---------------------------------------------------------------------Ran 3 tests in 0.004s

5
6

OK

1. We first use multiple inheritance to inherit from the Python standard unittest.TestCase
as well as the helpful FormTesterMixin.
2. Then in the test_signin_form_data_validation_for_invalid_data() test,
we create an array of test data called invalid_data_list. Each invalid_data
item in invalid_data_list is a dictionary with two items: the data item, which is
the data used to load the form, and the error item, which is a set of field_names
and then the associated error_messages.
With this setup we can quickly loop through many different data combinations and check
that each field in our form is validating correctly.

Why is form validation so important?

Validation

Form validations are one of those things that developers tend to breeze through or ignore
because they are so simple to do. But for many applications they are quite critical (Im looking
at you banking and insurance industries). QA teams seek out form validation errors like a
politician looking for donations.
Whats more, since the purpose of validation is to keep corrupt data out of the database, its
especially important to include a number of detailed tests. Think about edge cases.
Cleaning

The other thing that forms do is clean data. If you have custom data cleaning code, you
should test that as well. Our UserForm does, so lets test that, starting with the validation
then moving on to cleaning.
58

Please note: youll have to inherit from SimpleTestCase to get the assertRaisesMessage function
Test Form Validation
1
2

from payments.forms import UserForm

from django import forms

3
4
5
6
7
8
9
10
11
12
13
14
15

def test_user_form_passwords_match(self):
form = UserForm(
{
'name': 'jj',
'email': '[email protected]',
'password': '1234',
'ver_password': '1234',
'last_4_digits': '3333',
'stripe_token': '1'}
)
# Is the data valid? -- if not print out the errors
self.assertTrue(form.is_valid(), form.errors)

16
17

18

#this will throw an error if the form doesn't clean

correctly
self.assertIsNotNone(form.clean())

19
20
21
22
23
24
25
26
27
28
29
30

def test_user_form_passwords_dont_match_throws_error(self):
form = UserForm(
{
'name': 'jj',
'email': '[email protected]',
'password': '234',
'ver_password': '1234', # bad password
'last_4_digits': '3333',
'stripe_token': '1'
}
)

31
32
33

# Is the data valid?

self.assertFalse(form.is_valid())

34

59

self.assertRaisesMessage(forms.ValidationError,
"Passwords do not match",
form.clean)

35
36
37

Run the tests!

1
2
3
4

Creating test database for alias 'default'...

.....
---------------------------------------------------------------------Ran 5 tests in 0.006s

5
6
7

OK
Destroying test database for alias 'default'...

Thats it for now.

Testing Summary
This chapter provided a solid overview of unit testing for Django applications.
1.
2.
3.
4.

Part 1: Tested our Django url routing, templates and views using unit tests.
Part 2: Refactored those tests with Mocks to simply and further isolate the tests.
Part 3: Tested our models using the Django ORM by creating data on demand
Part 4: Tested our forms using mixins.

Summary
We covered quite a bit in this chapter, so hopefully this table will help summarize everything:
Testing Technique

When to use

Unit Testing
Integration Testing
GUI Testing
Mocking
Fixtures

Early and often to test code you write

Test integration between components
Test end to end from the website
to make unit tests more independent
load necessary data for a test

Testing Technique

Avantages

Unit Testing

test individual pieces of code, fast

60

Testing Technique

Avantages

Integration Testing
GUI Testing
Mocking
Fixtures

simple to write, test many things at one

testing final product soup to nuts, test browser behavior
separate concerns and test one thing at a time
quick and easy data management

Testing Technique

Disadvantages

Unit Testing
Integration Testing
GUI Testing
Mocking
Fixtures

may require mocking

test many things at once, slow
can be slow and tests easily break
complicated to write
sometimes difficult to know where data is coming from

This should provide you with the background you need to start unit testing your own applications. While it may seem strange at first to write unit tests alongside the code, keep at it and
before long you should be reaping the benefits of unit testing in terms of less defects, ease of
re-factoring, and having confidence in your code. If youre still not sure what to test at this
point, I will leave you with this quote:
In god we trust. Everything else we test.
-anonymous

Exercises
Most chapters have exercises at the end. They are here to give you extra practice on the
concepts covered in the chapter. Working through them will improve your mastery of the
subjects. If you get stuck, answers can be found in Appendix A.
1. Our URL routing testing example only tested one route. Write tests to test the other
routes. Where would you put the test to verify the pages/ route? Do you need to do
anything special to test the admin routes?
2. Write a simple test to verify the functionality of the sign_out view. Do you recall how
to handle the session?
3. Write a test for contact.models. What do you really need to test? Do you need to
use the database backend?

61

4. QA teams are particularly keen on boundary checking. Research what it is, if you are
not familiar with it, then write some unit tests for the CardForm from the payments
app to ensure that boundary checking is working correctly.

62

Chapter 4
Test Driven Development
Test Driven Development (TDD) is a software development practice that puts a serious emphasis on writing automated tests as, or immediately before, writing code. The tests not only
serve to verify the correctness of the code being written but they also help to evolve the design
and overall architecture. This generally leads to high quality, simpler, easier to understand
code, which are all goals of Software Craftsmanship.
Programmer (noun): An organism that converts caffeine or alcohol into
code.

~Standard Definition
While the above definition from Uncyclopedia is somewhat funny and true it doesnt tell the
whole story. In particular its missing the part about half-working, buggy code that takes a
bit of hammering to make it work right. In other words, programmers are humans, and we
often make mistakes. We dont fully think through the various use cases a problem presents,
or we forget some silly detail which can lead to some strange edge case bug that only turns
up eight years later and costs the company $465 million dollars in trading losses in about 45
minutes. It happened.
Wouldnt it be nice if there were some sort of tool that would alert a programmer when the
code isnt going to do what you think its going to do? Well, there is.
Its called TDD.
How can TDD do that? It all comes down to timing. If you write a test to ensure a certain
functionality is working, then immediately write the code to make the test pass, you have a
pretty good idea that the code does what you think it does. TDD is about rapid feedback. And
that rapid feedback helps developers who are not just machines (e.g., all of us) deliver great
code.
63

The TDD Workflow

Now that we have covered how to write unit tests for Django apps, we can talk about TDD,
which tells us WHEN to write unit tests. TDD describes a particular workflow for a developer
to follow when writing code. The workflow is depicted below:

Figure 4.1: TDD workflow

1. Red Light
The above workflow starts with writing a test before you write any code. The test will of
course fail because the code that its supposed to verify doesnt exist yet. This may seem like
a waste of time, writing a test before you even have code to test, but remember TDD is as
much about design as it is about testing and writing a test before you write any code is a good
way to think about how you expect your code to be used.
You could think about this in any number of ways, but a common checklist of simple considerations might look like this:
Should it be a class method or an instance method?
What parameters should it take?
64

 How do I know the function was successful?

What should the function return?
Do I need to manage any state?
These are just a few examples, but the idea is to think through how the code is going to be
used before you start writing the code.

2. Green Light
Once you have a basic test written, write just enough code to make it pass. Just enough is
important here. You may have heard of the term YAGNI - You Aint Going to Need It. Words
to live by when practicing TDD.
Since we are writing tests for everything, its trivial to go back and update/extend some functionality later if we decide that we indeed are going to need it. But often times you wont. So
dont write anything unless you really need it.

3. Yellow Light
Getting the test to pass isnt enough. After the test passes, we should review our code and
look at ways we can improve it. There are entire books devoted to re-factoring, so I wont go
into too much detail here, but just think about ways you can make the code easier to maintain,
more reusable, cleaner, and simpler.
Focus on those things and you cant go wrong. So re-factor the code, make it beautiful, and
then write another test to further verify the functionality or to describe new functionality.

4. Wash, Rinse, Repeat

Thats the basic process; we just follow it over and over and every few minutes this process
will produce code thats well tested, thoughtfully designed and re-factored. This should avoid
costing your company $465 million dollars in trading losses.

65

Implementing TDD with Existing Code

If you are starting a new project with TDD, thats the simple case. Just Do It! For most
people reading this, though, odds are youre somewhere half-way through a project with everlooming deadlines, possibly some large quality issues, and a desire to get started with this
whole TDD thing. Well fear not, you dont have to wait for you next project to get started
with TDD. You can start applying TDD to existing code, and in this section youre going to
learn just how to do that.
Before we jump into the code lets first talk briefly about where to start with TDD. There are
three places where it really makes sense to kickstart TDD in an existing project:
1. New Functionality - Anytime you are writing new functionality for an ongoing
project, write it using TDD.
2. Defects - When defects are raised, write at least two unit tests. One that passes and
verifies the existing desired functionality and one that reproduces the bug, and thus
fails. Now re-factor the code until both tests pass and continue on with the TDD cycle.
3. Re-factoring - There comes a time in every project when it just becomes apparent
that the 600 line method that no one wants to own up to needs re-factoring. Start with
some unit tests to guide your re-factoring, and then re-factor. We will explore this case
in more detail using our existing system.
With that in mind, lets look at our running code example.

66

Django Ecommerce Project

To refresh your memory the application we have been working on is a simple Django application that provides a registration function, which is integrated with Stripe to take payments,
as well as the basic login/logout functionality.
If youve been following along up to this point (and doing all the exercises) your application
should be the same as what we are about to dissect in further detail. If you havent or you just
want to make sure you are looking at the same application you can get the correct version of
the code from the git repo.

Registration Functionality
For this application, when a user registers, his or her credit card information will be passed
to Stripe, a third party payment processor, for processing. This logic is handled in the payments/views.py file. The register() function basically grabs all the information from the
user, checks it against Stripe and stores it in the database.
Lets look at that function now. Its called payments.views.register:
1
2
3
4
5

def register(request):
user = None
if request.method == 'POST':
form = UserForm(request.POST)
if form.is_valid():

6
7

8
9
10
11
12
13
14
15
16
17
18
19

#update based on your billing method (subscription vs

one time)
customer = stripe.Customer.create(
email=form.cleaned_data['email'],
description=form.cleaned_data['name'],
card=form.cleaned_data['stripe_token'],
plan="gold",
)
# customer = stripe.Charge.create(
#
description=form.cleaned_data['email'],
#
card=form.cleaned_data['stripe_token'],
#
amount="5000",
#
currency="usd"
# )

20

67

21
22
23
24
25
26

user = User(
name=form.cleaned_data['name'],
email=form.cleaned_data['email'],
last_4_digits=form.cleaned_data['last_4_digits'],
stripe_id=customer.id,
)

27
28
29

#ensure encrypted password

user.set_password(form.cleaned_data['password'])

30
31
32
33
34
35
36
37

try:
user.save()
except IntegrityError:
form.addError(user.email + ' is already a member')
else:
request.session['user'] = user.pk
return HttpResponseRedirect('/')

38
39
40

else:
form = UserForm()

41
42
43
44
45
46
47
48
49
50
51
52
53

return render_to_response(
'register.html',
{
'form': form,
'months': range(1, 12),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': user,
'years': range(2011, 2036),
},
context_instance=RequestContext(request)
)

It is by no means a horrible function (this application is small enough that there arent many
bad functions), but it is the largest function in the application and we can probably come up
with a few ways to make it simpler so it will serve as a reasonable example for re-factoring.
The first thing to do here is to write some simple test cases to verify (or in the case of the 600
line function, determine) what the function actually does.

68

So lets start in payments.tests.

In

the

previous

chapters exercises, the first question asked, "Our URL

routing testing example only tested one route. Write tests to test
the other routes. Where would you put the test to verify thepages/route?
Do you need to do anything special to test the admin routes?"And in
*Appendix A* we proposed the answer of using aViewTesterMixin().
If you were to implement that solution here you would probably come up with a class, like
RegisterPageTests(), that tests that the registration page returns the appropriate HTML/template data.
The code for those tests are shown below:
1
2
3

from django.shortcuts import render_to_response

import django_ecommerce.settings as settings
from payments.views import soon, register

4
5
6

class RegisterPageTests(TestCase, ViewTesterMixin):

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

@classmethod
def setUpClass(cls):
html = render_to_response(
'register.html',
{
'form': UserForm(),
'months': range(1, 12),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': None,
'years': range(2011, 2036),
}
)
ViewTesterMixin.setupViewTester(
'/register',
register,
html.content,
)

26
27
28

def setUp(self):
request_factory = RequestFactory()
69

self.request = request_factory.get(self.url)

29

If you dont have that code you can grab the appropriate tag from git by typing the following
from your code directory:

git checkout tags/tdd_start

Notice there are a few paths through the view that we need to test:
Condition

1.
2.
3.
4.

request.method
request.method
request.method
request.method

!=
==
==
==

POST
POST and not form.is_valid()
POST and user.save() is OK
POST and user.save() throws an IntegretyError:

Expected Results

1.
2.
3.
4.

Return basic register.html template

Return register.html (possibly with an error msg)
Set session['user'] == user.pk and return HTTPResponseRedirect('/')
Return register.html with a user is already a member error.

Condition 1

Our existing tests covers the first case, by using the ViewTesterMixin(). Lets write tests
for the rest.
Condition 2

Condition: request.method == POST and not form.is_valid()

Expected Results: Return register.html (possibly with an error msg)

1
2

from django.test import RequestFactory

import mock

3
4
5
6

def setUp(self):
request_factory = RequestFactory()
self.request = request_factory.get(self.url)
70

7
8

def test_invalid_form_returns_registration_page(self):

9
10

with mock.patch('payments.forms.UserForm.is_valid') as
user_mock:

11

user_mock.return_value = False

12
13

self.request.method = 'POST'
self.request.POST = None
resp = register(self.request)
self.assertEquals(resp.content, self.expected_html)

14
15
16
17
18

#make sure that we did indeed call our is_valid function

self.assertEquals(user_mock.call_count, 1)

19
20

Make sure to also update the imports:

1
2

from django.test import RequestFactory

import mock

To be clear, in the above example we are only mocking out the call to is_valid(). Normally
we might need to mock out the entire object or more functions of the object, but since it will
return False, no other functions on from the UserForm will be called.
NOTE: Also take note that we are now re-creating the mock request before each
test is executed in our setUp() method. This is because this test is changing
properties of the request object and we want to make sure that each test is independent.
Condition 3

Condition: request.method == POST and user.save() is OK

Expected Results: Set session['user'] == user.pk and return HTTPResponseRedirect('/')
For condition 3, we might start with something like this:
1

def test_registering_new_user_returns_successfully(self):

2
3
4

self.request.session = {}
self.request.method = 'POST'
71

5
6
7
8
9
10
11
12

self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '4242424242424242',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

13
14
15
16

resp = register(self.request)
self.assertEquals(resp.status_code, 200)
self.assertEquals(self.request.session, {})

The problem with this condition is that the test will actually attempt to make a request to the
Stripe server - which will reject the call because the stripe_token is invalid.
Lets mock out the Stripe server to get around that.
1

def test_registering_new_user_returns_succesfully(self):

2
3
4
5
6
7
8
9
10
11
12

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

13
14

with mock.patch('stripe.Customer') as stripe_mock:

15
16
17

config = {'create.return_value': mock.Mock()}

stripe_mock.configure_mock(**config)

18
19
20
21
22

resp = register(self.request)
self.assertEquals(resp.content, "")
self.assertEquals(resp.status_code, 302)
self.assertEquals(self.request.session['user'], 1)

23

72

# verify the user was actually stored in the database.

# if the user is not there this will throw an error
User.objects.get(email='[email protected]')

24
25
26

So now we dont have to actual call Stripe to run our test. Also, we added a line at the end to
verify that our user was indeed stored correctly in the database.
NOTE: A side note on syntax: In the above code example we used the following
syntax to configure the mock:

1
2

config = {'create.return_value': mock.Mock()}

stripe_mock.configure_mock(**config)

In this example we created a python dictionary called config and then

passed it to configure_mock with two stars ** in front of it. This is referred to as argument unpacking. In effect its the same thing as calling
configure_mock(create.return_value=mock.Mock(). However, that
syntax isnt allowed in python so we got around it with argument unpacking.
More details about this technique can be found here.
Coming back to testing, you may have noticed that these tests are getting large and testing
several different things. To me thats a code smell indicating that we have a function that is
doing too many things.
We should really be able to test most functions in three or four lines of code. Functions that
require mocks generally need a few more, though. That said, as a rule of thumb, if your test
methods start getting longer than about 10 lines of code, look for places to re-factor.
Before we do that, though, and for the sake of completeness, lets do the last test for Condition
4. This is a BIG ONE.
Condition 4

Condition: request.method == POST and user.save() throws an IntegretyError:

Expected Results: Return register.html with a user is already a member error.

def test_registering_user_twice_cause_error_msg(self):

2
3
4

#create a user with same email so we get an integrity error

user = User(name='pyRock', email='[email protected]')
73

user.save()

6
7
8
9
10
11
12
13
14
15
16
17

#now create the request used to test the view

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

18
19
20
21
22

#create our expected form

expected_form = UserForm(self.request.POST)
expected_form.is_valid()
expected_form.addError('[email protected] is already a
member')

23
24
25
26
27
28
29
30
31
32
33
34
35

#create the expected html

html = render_to_response(
'register.html',
{
'form': expected_form,
'months': range(1, 12),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': None,
'years': range(2011, 2036),
}
)

36
37
38

#mock out stripe so we don't hit their server

with mock.patch('stripe.Customer') as stripe_mock:

39
40
41

config = {'create.return_value': mock.Mock()}

stripe_mock.configure_mock(**config)

42
43

#run the test

74

44

resp = register(self.request)

45
46
47
48

#verify that we did things correctly

self.assertEquals(resp.status_code, 200)
self.assertEquals(self.request.session, {})

49
50
51
52

#assert there is only one record in the database.

users = User.objects.filter(email="[email protected]")
self.assertEquals(len(users), 1)

As you can see we are testing pretty much the entire system, and we have to set up a bunch
of expected data and mocks so the test works correctly.
This is sometimes required when testing view functions that touch several parts of the system.
But its also a warning sign that the function may be doing too much. One other thing: If you
look closely at the above test, you will notice that it doesnt actually verify the html being
returned.
We can do that by adding this line:
1

self.assertEquals(resp.content, html.content)

Now if we run it BOOM! Run it. Test Fails. Examining the results we can see the following
error:
Expected: <a href="/sign_in">Login</a>
Actual: <a href="/sign_out">Logout</a>
It looks like even though a user didnt register correctly the system thinks s user is logged into
the system.We can quickly verify this with some manual testing. Sure enough, after we try to
register the same email the second time this is what we see:
Further, clicking on Logout will give a nasty KeyError as the logout() function tries to
remove the value in session["user"], which is of course not there.
There you go: We found a defect. :D
While this one wont cost us $465 million dollars, its still much better that we find it before
our customers do. This is a type of subtle error that easily slips through the cracks, but that
unit tests are good at catching. The fix is easy, though. Just set user=None in the except
IntegretyError: handler within payments/views:
1
2

try:
user.save()
75

Figure 4.2: Registration Error

3
4
5
6
7
8

except IntegrityError:
form.addError(user.email + ' is already a member')
user = None
else:
request.session['user'] = user.pk
return HttpResponseRedirect('/')

Run the test.

1
2
3
4
5
6

Creating test database for alias 'default'...

...........
.
......
---------------------------------------------------------------------Ran 18 tests in 0.244s

7
8
9

OK
Destroying test database for alias 'default'...

Perfect.

76

Offense is only as good as your worst defense

Now with some good defense in place, in the form of our unit tests, we can move on to refactoring our register view, TDD style. The important thing to keep in mind is to make one
small change at a time, rerun your tests, add any new tests that are necessary, and then
repeat.
Start by pulling out the section of code that creates the new user and place it in the User()
model class. This way, we leave all the User processing stuff to the User() class.
To be clear, take this section:
1
2
3
4
5
6

user = User(
name = form.cleaned_data['name'],
email = form.cleaned_data['email'],
last_4_digits = form.cleaned_data['last_4_digits'],
stripe_id = customer.id,
)

7
8
9

#ensure encrypted password

user.set_password(form.cleaned_data['password'])

10
11
12

try:
user.save()

And put it in the User() model class, within payments/models.py so it looks like this:
1
2
3
4
5

@classmethod
def create(cls, name, email, password, last_4_digits, stripe_id):
new_user = cls(name=name, email=email,
last_4_digits=last_4_digits, stripe_id=stripe_id)
new_user.set_password(password)

6
7
8

new_user.save()
return new_user

Using classmethods as alternative to constructors is a idiomatic way to do things in

Python. We are just following acceptable patterns here. The call to this function from the
registration() view function is as follows:
1
2
3

cd = form.cleaned_data
try:
user = User.create(
77

4
5
6
7
8
9
10
11
12
13
14
15

cd['name'],
cd['email'],
cd['password'],
cd['last_4_digits'],
customer.id
)
except IntegrityError:
form.addError(cd['email'] + ' is already a member')
user = None
else:
request.session['user'] = user.pk
return HttpResponseRedirect('/')

No big surprises here.

Note: the line cd = form.cleaned_data is there as a placeholder. Also note that we dont
actually need the previous fix of user = None in the IntegrityError handler anymore
because our new create() function will just return None if there is an integrity error.
So after making the change, we can rerun all of our tests and everything should still pass!

Updating Tests
Taking a defensive approach, lets update some of our tests before moving on with the refactoring.
The first thing is to move the testing of the user creation to the user model and test the create
class method. So, in the UserModelTest class we can add the following two methods:
1
2
3

def test_create_user_function_stores_in_database(self):
user = User.create("test", "[email protected]", "tt", "1234", "22")
self.assertEquals(User.objects.get(email="[email protected]"), user)

4
5
6
7
8
9
10
11
12
13

def test_create_user_allready_exists_throws_IntegrityError(self):
from django.db import IntegrityError
self.assertRaises(
IntegrityError,
User.create,
"test user",
"[email protected]",
"jj",
"1234",
78

89

14
15

With these tests we know that user creation is working correctly in our model and therefore
if we do get an error about storing the user from our view test, we know it must be something
in the view and not the model.
At this point we could actually leave all the existing register view tests as is. Or we could mock
out the calls to the database since we dont strictly need to do them any more, as we already
tested that the User.create method functions work as intended.
Lets go ahead and mock out the test_registering_new_user_returns_succesfully()
function to show how it works:
1
2
3
4
5

@mock.patch('stripe.Customer.create')
@mock.patch.object(User, 'create')
def test_registering_new_user_returns_succesfully(
self, create_mock, stripe_mock
):

6
7
8
9
10
11
12
13
14
15
16

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

17
18
19
20

#get the return values of the mocks, for our checks later
new_user = create_mock.return_value
new_cust = stripe_mock.return_value

21
22

resp = register(self.request)

23
24
25
26
27
28

self.assertEquals(resp.content, "")
self.assertEquals(resp.status_code, 302)
self.assertEquals(self.request.session['user'], new_user.pk)
#verify the user was actually stored in the database.
create_mock.assert_called_with(
79

'pyRock', '[email protected]', 'bad_password', '4242',

new_cust.id
)

29

30

Lets quickly explain how it works:

There are two @mock.patch decorators that mock out the specified method call. The return
value of the mocked method call can be accessed using the return_value() function.
You can see that here:
1
2
3

#get the return values of the mocks, for our checks later
new_user = create_mock.return_value
new_cust = stripe_mock.return_value

We store those return values so we can use them for our assertions later on, which are used
in these two assertions:
1
2
3
4

self.assertEquals(self.request.session['user'], new_user.pk)
#verify the user was actually stored in the database.
create_mock.assert_called_with(
'pyRock', '[email protected]', 'bad_password', '4242',
new_cust.id
)

1. The first assertion just verifies that we are setting the session value to the id of the
object returned from our create_mock.
2. The second assertion, meanwhile, is an assertion that is built into the mock library,
allowing us to check and see if a mock was called - and with what parameters. So
basically we are asserting that the User.create() function was called as expected.
Run the tests.
1
2
3
4
5
6

Creating test database for alias 'default'...

...........
.
........
---------------------------------------------------------------------Ran 20 tests in 0.301s

7
8
9

OK
Destroying test database for alias 'default'...
80

A few words about mocking

While mocks can be handy at cutting out part of the system or isolating a test, in this example
they are probably more trouble then they are worth. It is of course necessary to mock out the
stripe.Customer.create call, as otherwise we would always be trying to hit their servers
- NOT GOOD.
But in terms of the database, dont bother mocking it. We already have tests in the User()
class for create() so we know it works. This means technically we dont need to test
create() again in this function. But its probably easier to go ahead and test it again, so we
can avoid messing around with the mocks too much. And further, since we do already have
the tests in place, and they pass, we can be fairly certain that any failure in this test is not
related to user creation.
As with all things in development, there are trade offs with mocking. In terms of guidelines
I generally think about it like this:
When should you Mock?

When a function is horribly slow and you dont want to run it each time.
Similarly, when a function needs to call something not on your development machine
(i.e., web APIs).
When a function requires setup, or state, that is more difficult to manage than setting
up the mock (i.e., unique data generation).
When a function is somebody elses code and you really dont trust it or dont want to
be responsible for it (but see When not to Mock below).
Time-dependent functionality is good to mock as its often difficult to recreate the dependency in a unit test.
When should you not Mock?

Not to Mock should be the default choice, when in doubt dont mock
When its easier or faster or simpler not to mock.
If you are going to mock out web APIs or code you dont want to be responsible for, its
a good idea to include some separate integration tests that you can run from time to
time to ensure the expected functionality is still working, or hasnt been changed. As
changing APIs can be a huge headache when developing against third party code.
Some people swear that the speed of your unit test is paramount to all else. While it is true
mocking will generally speed up the execution time of your unit test suite, unless its a significant speed increase (like in the case for stripe.Customer.create) my advice is: Mocking
81

is a great way to kill brain cycles; and actually improving your code is a much better use of
those brain cycles.

82

Conclusion
The final thing you might want to re-factor out of the register() function is the actual
Stripe customer creation.
This makes a lot of sense if you have a use case where you may want to choose between subscription vs one-time billing. You could just write a customer manager function that takes in
the billing type as an argument and calls Stripe appropriately returning the customer.id.
Well leave this one as an exercise for you, dear reader.
We have discussed the general approach to TDD and how following the three steps, and making one small change at a time is very important to getting TDD right.
As a review the three steps of TDD are:
1.
2.
3.
4.

RED LIGHT - write a test that fails.

GREEN LIGHT - write just enough code to make the test pass.
YELLOW LIGHT - re-factor your code/tests until you have something beautiful.
RINSE and REPEAT

Follow those steps and they should serve you well in your path to Elite Software Craftsmanship.
We also touched on applying TDD and unit testing to existing/legacy code and how to think
defensively so you dont break existing functionality while trying to improve the system.
With the two testing chapters now completed we can turn our focus on some of the cool and
exciting features offered in Django 1.6, 1.7, and 1.8, but first here are a couple of exercises to
cement all this TDD stuff in your brain.

83

Exercises
1. Try mocking out the test_registering_user_twice_cause_error_msg() test if
you really want to get your head around mocks. Start by mocking out the User.create
function so that it throws the appropriate errors.
HINT: this documentation is a great resource and the best place to start. In particular,
search for side_effect.
Want more? Mock out the UserForm as well. Good luck.
2. As alluded to in the conclusion, remove the customer creation logic from register()
and place it into a separate CustomerManager() class. Re-read the first paragraph of
the conclusion before you start, and dont forget to update the tests accordingly.

84

Chapter 5
Git Branching at a Glance
We talked about testing and re-factoring in the first few chapters of this book because they
are so important to the art of Software Craftsmanship. And we will continue to use them
throughout this book. But there are other tools which are important to master in the neverending quest to become a better craftsman. In this chapter we will briefly introduce Git
Branching so we can use it in the next chapter, which is about the art of the upgrade - i.e.,
Upgrading to Django 1.8.
It is assumed that the reader has some basic familiarity with how to push, pull and setup a
repo with git prior to reading this chapter. If you dont, you can revisit the Getting Started
chapter in the second Real Python course. Or you can go through this quick and to the point
git-guide.

85

Git Branching
When developing applications we are often faced with situations where we want to experiment or try new things. It would be nice if we could do this in a way that wouldnt affect our
main line of code and that we could easily roll back from if our experiment didnt quite work
out, or save it off and work on it later. This is exactly what Git Branching provides. Oftentimes
referred to as Gits killer feature, the branching model in Git is incredibly useful.
There are four main reasons that Gits branching model is so much more useful than branching in traditional Version Control Systems (Im looking at you SVN):
1. Git Branching is done in the same directory as you main line code. No need to switch
directories. No need to take up a lot of additional disk space. Just do a git checkout
<branchname> and Git switches all the files for you automatically.
2. Git Branching is lightning fast. Since it doesnt copy all the files but rather just applies
the appropriate diffs, its really fast.
3. Git Branching is local. Since your entire Git repository is stored locally, there is no need
for network operations when creating a branch, allowing for branching and merging
to be done completely offline (and later pushed back up to the origin server if desired).
This is also another reason why Git branching is so fast; in the time it takes SVN to
connect to the remote server to perform the branch operation, Git will have already
completed.
4. But the most important and useful feature of Git Branching is not Branching at all - its
merging. Git is fantastic at merging (in fact, every time you do a git pull you are
merging) so developers dont need to be afraid of merging branches. Its more or less
automatic and Git has fantastic tools to clean up those cases where the merge needs
some manual intervention.
5. (Yeah I know I said only 4, count this one as a bonus) Merging can be undone. Trying
to merge and everything goes bad? No problem, just git reset --hard.
The main advantage to all of these features is that it makes it simple - in fact, advisable - to
include branching as part of your daily workflow (something you generally wouldnt do in
SVN). With SVN we generally branch for big things, like upgrading to Django 1.8 or another
new release. With Git we can branch for everything.
One common branching pattern in Git is the feature branch pattern where a new branch
is created for every requirement and only merged back into the main line branch after the
feature is completed.

86

Branching Models
Thats a good segue into talking about the various branching models that are often employed
with Git.
A branching model is just an agreed upon way to manage your branches. It usually talks
about when and for what reasons a branch is created, when it is merged (and to what branch
it is merged into) and when the branch is destroyed. Its helpful to follow a branching model
so you dont have to keep asking yourself, Should I create a branch for this?. As developers,
we have enough to think about, so following a model to a certain degree takes the thinking
out of it - just do what the model says. That way you can focus on writing great software and
not focus so much on how your branching model is supposed to work.

Lets look at some of the more common branching models used in Git.

The I just switched from SVN and I dont want to really learn Git
Branching Model
Otherwise known as the no-branching model. This is most often employed by Git noobs,
especially if they have a background in something like CVS or SVN where branching is hard.
I admit when I started using Git this is exactly the model that I followed. For me, and I think
for most people who follow this model, its because branching seems scary and there is code
to write, and tests to run, and who has time to learn about the intricacies of branching yadda,
yadda, yadda.
The problem with this model is there is such a huge amount of useful features baked into Git
that you simply are not getting access to them if youre not branching. How do you deal with
production issues? How do you support multiple versions of your code base? What do you
do if you have accidentally gone down the wrong track and have screwed up all your code?
How do you get back to a known good state? The list goes on.
There are a ton of reasons why this isnt a good way to use Git. So do yourself a favor, take
some time and at least learn the basics of git branching. Your future self will thank you. :)

Git-Flow
Vincent Driessen first documented this excellent branching model here. In his article (which
you should really read) entitled A Successful Git Branching Model, he goes through a somewhat complex branching model that handles just about any situation you could think of when
working on a large software project. His model eloquently handles:

87

Managing multiple versions of a code base simultaneously;

Always having a known good, deployable version of the code;
Handling production issues and hot fixes;
Feature branches;
Latest/development branches; and,
Easily handling UAT and last minute release type of work, before a branch of code
makes it into production.

A quick snapshot of what the branching model looks like can be seen below:
For large software projects, when working with a team, this is the recommended model. Since
Vincent does such a good job of describing it on his website, were not going to go into detail
about it here, but if you are working on a large project this is a good branching model to use.
For smaller projects with less team members you may want to look at the model below. Also
I have reports from a number highly skilled developers use combinations of git-flow and the
Github pull-request model. So choose what fits your style best - or use them all. :)
Also to make git-flow even easier to use, Vincent has created an open source tool called
git-flow that is a collection of scripts that make using this branching model pretty straightforward, which can save a lot of keystrokes. Git-Flow can be found on Github here.
Thank you Vincent!

The Github Pull-Request Model

Somewhere in between the no-branching model and Git-Flow is something we like to call The
Github Pull-Request Model. We call it the Github model because the folks at GitHub are
the ones that really made the pull-request popular and easy to use. Git out of the box doesnt
provide pull-request capabilities but many git front ends now do. For example, gitorious now
offers pull-request functionality so you dont strictly have to be using Github - but it helps.
Lets first look at a picture of how the model works:
As you can see from the model above there are only two branches to deal with in the GitHub
Pull-Request Model, as opposed to the five in the git-flow model. This makes for a very
straight-forward model.
Basically all you do is create a feature branch for each requirement you are working on, then
issue a pull request to merge the feature branch back into master after you are done. So really
you only have one permanent branch Master and a bunch of short-lived feature branches.
Then of course there is the pull-request which allows for an easy way to review code / alert
others that the code is ready to merge back into master. Dont be fooled by its simplicity, this
model works pretty effectively.
88

Figure 5.1: git-flow

89branching model

Figure 5.2: Github Pull-Request Model

90

Lets go though the typical workflow:

1. Create a feature branch from Master.
2. Write your code and tests on the feature branch.
3. Issue a pull request to merge your feature branch back to master when it is completed
and ready to merge back into master.
4. Pull requests are used to perform code reviews and potentially make any changes necessary so the feature branch can be merged into Master.
5. Once approved, merge the feature branch back into Master.
6. Tag releases.
Thats the basic outline. Pretty straight-forward. Lets look each of the steps in a bit more
detail.
Create a feature branch from Master

For this model to work effectively the Master branch must always be deployable. It must be
kept clean at all times, and you should never commit directly to master. The point of the Pull
Request is to ensure that somebody else reviews your code before it gets merged into Master
to try to keep Master clean. There is even a tool called git-reflow that will enforce a LGTM
(Looks Good to Me) for each pull request before it is merged into master.
Write your code and tests on the feature branch

All work should be done on a feature branch to keep it separate from Master and to allow
each to go through a code review before being merged back into master.
Create the branch by simply typing:
1

$ git checkout -b feature_branch_name

Then do whatever work it is you want to do. Keep in mind that commits should be incremental and atomic. Basically that means that each commit should be small and affect only one
aspect of the system. This makes things easy to understand when looking through history.
Wikipedia has a detailed article on atomic commit conventions.
Once your done with your commits but before you issue the pull request, its important to
rebase against Master and tidy things up, with the goal of keeping the history useful.
Rebasing in git is a very powerful tool and you can do all kinds of things with it. Basically it
allows you to take the commits from one branch and replay them on another branch. Not only
91

can you replay the commits but you can modify the commits as they are replayed as well. For
example you can squash several commits into a single commit. You can completely remove
or skip commits, you can change the commit message on commits, or you can even edit what
was included in the commit.
Now that is a pretty powerful tool. A great explanation of rebase can be found at the git-scm
book. For our purposes though we just want to use the -i or interactive mode so we can clean
up our commit history so our fellow developers dont have to bear witness to all the madness
that is our day to day development. :)
Start with the following command:
1

$ git rebase -i origin/master

NOTE: the -i flag is optional but that gives you an interactive menu where you
can reorder commits, squash them or fix them up all nice and neat before you
share your code with the rest of the team.

Issuing the above command will bring up your default text editor (mine is vim) showing a list
of commits with some commands that you can use:
1
2
3
4

pick
pick
pick
pick

7850c5d
6999b75
6598edc
8779d40

tag: Ch3_before_exercises
Ch3_ex1
Ch3_ex1_extra_credit
Ch3_ex2

5
6
7
8
9
10
11
12
13
14
15
16

17
18
19

#
#
#
#
#
#
#
#
#
#
#

Rebase 7850c5d..8779d40 onto dd535a5

Commands:
p, pick = use commit
r, reword = use commit, but edit the commit message
e, edit = use commit, but stop for amending
s, squash = use commit, but meld into previous commit
f, fixup = like "squash", but discard this commit's log message
x, exec = run command (the rest of the line) using shell
These lines can be re-ordered; they are executed from top to
bottom.

#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
92

20
21
22

# However, if you remove everything, the rebase will be aborted.

#
# Note that empty commits are commented out

Above we can see there are five commits that are to be rebased. Each commit has the word
pick in front of it followed by the SHA ID of the commit and the commit message. You can
modify the word in front of the commit pick to any of the following commands:
1
2
3
4
5
6
7

#
#
#
#
#
#
#

Commands:
p, pick = use commit
r, reword = use commit, but edit the commit message
e, edit = use commit, but stop for amending
s, squash = use commit, but meld into previous commit
f, fixup = like "squash", but discard this commit's log message
x, exec = run command (the rest of the line) using shell

The commands are all self explanatory. Note that both squash and fixup come in handy.
edit is also nice. If you were to change the commits to look like the following:
1
2
3
4

pick
pick
edit
pick

7850c5d
6999b75
6598edc
8779d40

Ch3_before_exercises
Ch3_ex1
Ch3_ex1_extra_credit
Ch3_ex2

Then after you saved and closed the file, rebase would run and it would play back from the top
of the file, applying commit 7850c5d then commit 6999b75 and then the changes for commit
6598edc will be applied and rebase will drop you back to the shell allowing you to make any
changes you want. Then just git add your changes and once you issue the command1

$ git rebase --continue

-the changes that you git add-ed will be replayed as part of the 6598edc commit. Then
finally the last commit 8779d40 will be applied. This is really useful in situations where you
forgot to add a file to git or you just need to make some small changes.
Have a play with git rebase; its a really useful tool and worth spending some time learning.
Once youre done with the rebase and you have your commit history looking just how you
want it to look then its time to move on to the next step.
Does this make sense? Would you like to see a video depicting an example? If
so, please let us know! [email protected]

93

Issue a pull request to merge your feature branch back to master

Technically you dont have to use GitHubs pull request feature. You could simply push your
branch up to origin and have others check it like this:
1

$ git push origin feature_branch_name

But Githubs pull requests are cool and provide threaded comments, auto-merging and other
good stuff as well. If you are going to use a pull request you dont need to fork the entire
repository, as GitHub now allows you to issue a pull request from a branch. So after you have
pushed up your branch you can just create the pull request on it through the GitHub website.
Instructions for this are here.
There is also an excellent set of command line tools called hub that make working with Github
from the command line a breeze. You can find hub here. If youre on OSX and you use
homebrew (which you should) just brew install hub. Then you dont have to push your
feature_branch at all just do something like this:
1
2

$ git checkout feature_branch_name

$ git pull-request

This will open a text editor allowing you to put in comments about the pull request. Be aware
though that this will create a fork of the repo. If you dont want to create the fork (which you
probably dont) you can do a slightly longer command such as:
1

$ git pull-request "pull request message" -b repo_owner:master -h

repo_owner:feature_branch_name

This will create a pull request from your feature branch to the master branch for the repo
owned by repo_owner. If youre also tracking issues in Github you can link the pull request
to an issue by adding -i ### where ### is the issue to link to. Awesome! :)
Perform the Code Review

This is where the GitHub pull request feature really shines. Your team can review the pull
request, make comments or even checkout the pull-request and add some commits. Once
the review is done the only thing left to do is merge the pull request into Master.
Merge Pull Request into Master

Process:

94

1
2
3

$ git checkout master

$ git pull origin master
$ git merge --no-ff feature_branch_name

The --no-ff flag is important for keeping the history clean. Basically this groups all the
commits from your feature_branch and breaks them out into a merge bubble so they are
easy to see in the history. An example would look like this:
1
2
3
4
5
6
7

* 5340d4b Merge branch 'new_feature_branch'

|\
| * 4273443 file2
| * af348dd file1.py
|/
* 9b8326c updated dummy
* e278c5c my first commit

While some people really dont like having that extra merge commit - i.e., commit number
5340d4b - around, its actually quite helpful to have. Lets say for example you accidentally
merged and you want to undo. Then because you have that merge commit, all you have to do
is:
1

git revert -m 1 5340d4b

2
3
4

# and then the entire branch merge will be undone

# and your history will look like

5
6

git log --graph --oneline --all

7
8
9
10
11
12
13
14
15

* 5d4a11c Revert "Merge branch 'new_feature_branch'"

* 5340d4b Merge branch 'new_feature_branch'
|\
| * 4273443 file2
| * af348dd file1.py
|/
* 9b8326c updated dummy
* e278c5c my first commit

So the merge bubbles are a good defensive way to cover your ass (CYA) in case you make any
mistakes.

95

Tag your releases

The only other thing you do in this branching model is periodically tag your code when its
ready to release.
Something like:
1

$ git tag -a v.0.0.1 -m "super alpha release"

This way you can always checkout the tag, if for example you have production issues, and
work with the code from that specific release.
Thats all there is to it.

96

Enough about git branching

That should be enough to make you dangerous. :) But more importantly you should have a
good idea as to why branching strategies are useful and how to go about using them. We will
try it out for real in the next chapter on upgrading to Django 1.8.

97

Exercises
The exercises for this chapter are pretty fun. Enjoy!
1. Peter Cottle has created a great online resource to help out people wanting to really
understand git branching. Your task is to play around with this awesome interactive
tutorial. It can be found here: http://pcottle.github.io/learnGitBranching/. What are
your thoughts? Write them down. Blog about them. Email us about it.
2. There is a great talk about the GitHub Pull Request Model by Zach Holman of GitHub
entitled, How GitHub uses GitHub to Build GitHub. Its worth watching the entire
video, but the branching part starts here.

98

Chapter 6
Upgrade, Upgrade, and Upgrade some
more
Django 1.8
Up until now we have been working on Django 1.5, but its worthwhile to look at some of the
new features offered by Django 1.6, 1.7, and 1.8 as well. In addition, it will give us a chance
to look at the upgrade process and what is required when updating a system. Lets face it: if
youre lucky enough to build a system that lasts for a good amount of time, youll likely have
to upgrade. This will also give us a chance to try out our git branching strategy and just have
some good clean code wrangling fun.

99

The Upgrade
Lets start off with creating a feature branch for our work.
1
2

$ git checkout -b django1.8

Switched to a new branch 'django1.8'

Then we also need to create a separate virtualenv and upgrade the version of Django in that
virtualenv to 1.8. Before you do this, make sure to deactivate the main virtualenv.
1
2
3
4
5

$
$
$
$
$

pip freeze > requirements.txt

mkvirtualenv django1.8
pip install -r requirements.txt
pip install django==1.8.2
pip freeze > requirements.txt

Want to check the Django version? Enter the shell

1
2
3

>>> import django

>>> django.VERSION
(1, 8, 2, 'final', 0)

Want to look at the branches currently available?

1
2
3

$ git branch
* django1.8
master

Test the Install

We have created a new virtualenv called django1.8, installed all of our dependencies in that
environment, and upgraded to Django 1.8. Now to see if everything worked, we want to run
all of our tests. There is a nice option you can use when running your tests, --traceback,
that will show all warnings in addition to errors; its a good idea to turn that on now as well
so we can spot any other potential gotchas. Cross your fingers as you type
1

$ ./manage.py test --traceback

Lets look at the error

1
2
3

======================================================================
ERROR: test_signin_form_data_validation_for_invalid_data
(payments.tests.FormTests)
100

4
5
6
7
8
9
10
11
12
13

---------------------------------------------------------------------Traceback (most recent call last):

File "tests.py", line 65, in
test_signin_form_data_validation_for_invalid_data
invalid_data["data"])
File "/site-packages/django/test/testcases.py", line 398, in
assertFormError
contexts = to_list(response.context)
AttributeError: type object 'SigninForm' has no attribute 'context'
----------------------------------------------------------------------

Would you look at that. Django decided to take the name of our function assertFormError.
Guess the Django folks also thought we needed something like that. Okay. Lets just change
the name of our function to something like should_have_form_error() so it wont get
confused with internal Django stuff.
You should also see the same error thrown from each one of your test cases. It looks like this:
1
2
3
4
5

6
7

======================================================================
ERROR: tearDownClass (tests.contact.testContactModels.UserModelTest)
---------------------------------------------------------------------Traceback (most recent call last):
File "/site-packages/django/test/testcases.py", line 962, in
tearDownClass
cls._rollback_atomics(cls.cls_atomics)
AttributeError: type object 'UserModelTest' has no attribute
'cls_atomics'

Whats that? This is actually due to a cool new feature in Django 1.8 called setUpTestData
meant to make it easier to load data for a test case. The documentation is here. To achieve
this little bit of awesomeness the django.test.TestCase in 1.8 uses setUpClass to wrap
the entire test class in an atomic block (see migrations chapter). Anyways the solution for
this is to change our setUpClass function (in all of our test classes) to call the setUpClass
function in django.test.TestCase. We do this with the super function. Here is an example for the MainPageTetss:
1

class MainPageTests(TestCase):

2
3
4
5
6

@classmethod
def setUpClass(cls):
super(MainPageTests, cls).setUpClass()
... snipped the rest of the function ...
101

Basically the line starting with super can be read as, Call the setUpClass method on the
parent class of UserModelTest and pass in one argument cls. You then have to do the
same change to the setUpClass function for each TestCase class subsituting the name of
the class for UserModelTest.
NOTE: Since we will later in this chapter upgrade to Python 3 its
worth noting that the syntax to call super is a bit cleaner in Python
3. Instead of super(UserModelTest, cls).setUpClass() with
Python 3 you can just call super().setUpClass() and it will figure
out the correct arguments to use!

While it will certainly work to update all of our test cases with the called to super().setUpClass()
Django 1.8 introduces a new test case feature which may be more appropriate for some of
our test cases. The new feature called setUpTestData and its a class function that is
actually called from setUpClass is meant to be used for creaing test data. From a functional persepective there are no differences between creating data for your TestCase in
setUpClass vs setUpTestData. However due to the name of the latter, it is arguably a
cleaner way to express intent, making your test cases that much easier to understand. To
show an example, lets update our payments.testUserModel.UserModelTest class. To
do this we would complete remove the setUpClass function and replace it with this:
1
2
3
4

@classmethod
def setUpTestData(cls):
cls.test_user = User(email="[email protected]", name='test user')
cls.test_user.save()

Doing this will create the test_user only one time, just before all the test functions in the
class run. Also note because this is created inside an atomic block, there is no need to call
tearDownTest to delete the data.
After updating all your test classes, re-run the tests and now they are all at least passing. Nice.
But we still have a few warnings to deal with
First warning
1

../Users/michaelherman/Documents/repos/realpython/book3-exercises/_chapters/_chp06
RemovedInDjango18Warning: Creating a ModelForm without either
the 'fields' attribute or the 'exclude' attribute is deprecated
- form ContactView needs updating
class ContactView(ModelForm):
102

This is Django being helpful. RemovedInDjango18Warning refers to functionality that you

will have to change for the next version. But since we are upgrading anyway, lets go ahead
and get rid of the warning.
This particular issue (explained in detail in the Django 1.6 release notes) is an attempt by the
Django folks to make things more secure. With ModelForms you can automatically inherit
the fields from the model, but this can lead to all fields in the model being editable by accident.
To prevent this issue and to make things more explicit, you will now have to specify the fields
you want to show in your Meta class.
The fix is simple: just add one line to the Meta class for ContactView():
1
2

class ContactView(ModelForm):
message = forms.CharField(widget=forms.Textarea)

3
4
5
6

class Meta:
fields = ['name', 'email', 'topic', 'message']
model = ContactForm

Its only one line of extra work, and it prevents a potential security issue. If we now re-run
our unit tests, you should no longer see that warning.
Lets just add one more test, to contact.tests to make sure that we are displaying the
expected fields for our ContactForm:
1
2

from django.test import TestCase, SimpleTestCase

from contact.forms import ContactView

3
4

class ContactViewTests(SimpleTestCase):

5
6
7
8

def test_displayed_fields(self):
expected_fields = ['name', 'email', 'topic', 'message']
self.assertEquals(ContactView.Meta.fields, expected_fields)

Why build such a simple, seemingly worthless test? The main reason for a test like that is to
communicate to other developers on our team that we intentionally have set the fields and
want only those fields set. This means if someone decided to add or remove a field they will
be alerted that they have broken the intended functionality which should hopefully prompt
them to at least think twice and discuss with us as to why they may need to make this change.
Run tests again. They should still pass. Now lets address the other warning.
Second warning

103

1
2

?: (1_6.W001) Some project unit tests may not execute as expected.

HINT: Django 1.6 introduced a new default test runner. It looks
like this project was generated using Django 1.5 or
earlier. You should ensure your tests are all running &
behaving as expected. See
https://docs.djangoproject.com/en/dev/releases/1.6/#discovery-of-tests-in-a
for more information.

This is basically telling us there is a new test runner in town and we should be aware of and
make sure all of our tests are still running. We already know that all of our tests are running
so we are okay there. However, this new django.test.runner.DiscoverRunner() test
runner is pretty cool as it allows us to structure our tests in a much more meaningful way.
Before we get into the DiscoverRunner though lets get rid of the warning message.
Basically Django looks at the settings.py file to try to determine if your original Project was
generated with something previous to 1.6. Which means you will have certain default settings like SITE_ID, MANAGERS, TEMPLATE_LOADERS or ADMINS. Django also checks to see if
you have any of the Django 1.6 default settings that werent present in previous versions and
then attempts to guess if your project was created with 1.6 or something older. If Django
determines that you project was generated prior to 1.6 django warms you about the new
DiscoverRunner. Conversely if django thinks your project was created with 1.6 or something newer it assume you know what your doing and doesnt give you the warning message.
The simplest way to remove the error is to specify a TEST_RUNNER. Lets use the new one. :)
1

TEST_RUNNER = 'django.test.runner.DiscoverRunner'

The DiscoverRunner is the default test runner starting in Django 1.6 and now since we
explicitly set it in our settings.py, Django wont give us the warning anymore. Dont believe
me? Run the tests again.
1
2
3
4
5
6

Creating test database for alias 'default'...

...............
.
.............
---------------------------------------------------------------------Ran 29 tests in 0.329s

7
8
9

OK
Destroying test database for alias 'default'...

Boom.
104

NOTE: Its also worth noting that the the check management command
- ./manage.py check would have highlighted this warning as well. This
command checks our configuration and ensure that it is compatible with the
installed version.

With the warning out of the way, lets talk a bit about the DiscoverRunner.

105

DiscoverRunner
In Django 1.5 you were strongly encouraged to put you test code in the same folder as your
production code in a module called tests.py. While this is okay for simple projects, it can
make it hard to find tests for larger projects, and it breaks down when you want to test global
settings or integrations that are outside of Django. For example, it might be nice to have a
single file for all routing tests to verify that every route in the project points to the correct
view. Starting in Django 1.6 you can do just that because you can more or less structure your
tests any way that you want.
Ultimately the structure of your tests is up to you, but a common and useful structure (in many
different languages) is to have your source directory (django_ecommerce, in our case) as
well as a test top level directory. The test directory mirrors the source directory and only
contains tests. That way you can easily determine which test applies to which model in the
project.
Graphically that structure would look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14

django_ecommerce
contact
django_ecommerce
main
payments
tests
contact
userModelTests.py
contactViewTests.py
django_ecommerce
main
mainPageTests.py
payments

Ive omitted a number of the tests, but you get the idea. With the test directory tree separate
and identical to the code directory, we get the following benefits:
Separation of tests and production code.
Better test code structure (like separate files for helper test functions; for example,
the ViewTesterMixin() from could be refactored out into a utility module, like
tests\testutils.py, so each utility can be easily shared amongst the various tests
classes).
106

 Simple to see the linkage between the test cases and the code its testing.
Helps ensure that test code isnt accidentally copied over to a production environment.
NOTE Users of Nose or other third party testing tools for Django wont think
this is a big deal because they have had this functionality for quite some time.
The difference now is that its built into Django so you dont need a third party
tool.

Lets go ahead and make the changes.

107

Update Project Structure

You can find these changes in the chp06 directory in the repository. Or you can move the
tests to the new directory structure yourself. If you move them yourself keep in mind that
each folder must have a init.py or else the DiscoverRunner wont find the tests.
Once you have all the tests separated in the test folder, to run them all you need to do is type:
1

$ ./manage.py test ../tests

This will find all the tests in the ../tests directory and all sub-directories. Run them now
and make sure everything passes:
1
2
3
4
5
6

Creating test database for alias 'default'...

.......................
.
.....
---------------------------------------------------------------------Ran 29 tests in 0.349s

7
8
9

OK
Destroying test database for alias 'default'...

Once everything is working, remember we need to issue a pull request so the code can be
reviewed and eventually merged back into our main line. You have been making atomic commits as described in the previous chapter correct?
To show the commits we have made we could do the following from the command line:
1
2
3

4
5

$ git log --oneline

a9b81a1 moved all tests out to a separate tests directory
f5dab51 updated ContactView to explicitly list fields to be
displayed
7cda748 wouldn't you know it, Django took my assertFormError name...
58f1503 yes we are upgrading to Django 1.8

Now all we need to do is issue a pull request.

Pull Request with Hub

If you installed hub from the previous chapter, you can just type:
1

$ git pull-request "upgraded to django 1.8 fixed all errors" -b

repo_owner:master -h repo_owner:django_1.8
108

NOTE: Keep in mind that hub must also be aliased to git by adding the line
alias git=hub to your .bash_profile for the previous line to work.

Pull Request without Hub

If you did not install hub, we can just issue a pull request by pushing the code up to Github:
1

$ git push origin django1.8

And then manually creating the pull request through the GitHub web UI.

Merge
This will issue the merge request which can be discussed and reviewed from the GitHub website. If youre unfamiliar with this process, please check out the official Github docs. After
the merge request has been reviewed/approved then it can be merged from the command
line with:
1
2
3

$ git checkout master

$ git pull origin master
$ git merge --no-ff django1.8

That will merge everything back into master and leave us a nice little merge bubble (as described in the previous chapter) in the git history.

Summary
Thats it for the upgrade to Django 1.8. Pretty straight-forward as we have a small system
here. With larger systems its just a matter of repeating the process for each error/warning
and looking up any warning or errors you are not sure about in the Django 1.6 release notes as
well as the Django 1.7 and Django 1.8 release notes. Its also a good idea to check the release
notes and make sure there arent any new changes or gotchas that may cause bugs, or change
the way your site functions in unexpected ways. And of course (dare I say) manual testing
shouldnt be skipped either, as automated testing wont catch everything.

109

Upgrading to Python 3
Django officially started supporting Python 3 in version 1.6. In the Django docs they highly
recommend Python 3.3 or greater, so lets go ahead and upgrade to 3.4.1, which is the latest
version as of writing.
NOTE: If you do decide to upgrade to a different version greater than 3.3, the
instructions below should remain the same. Please let us know if this is not the
case.

To Upgrade or Not to Upgrade - a Philosophical Debate

The first question to consider is: Why upgrade to Python 3 in the first place? It would seem
pretty obvious, but lately there has been a lot of grumbling in the Python community about
Python 3, so lets look at both sides of the debate.
The negative

Armin Ronacher has been one of the most popular anti-Python 3 posts. Its worth a read,
but to sum it up, it basically comes down to: Porting to Python 3 is hard, and unicode/byte
string support is worse than it is in Python 2. To be fair, he isnt wrong and some of the issues
still exist, but a good amount of the issues he refers to are now fixed in Python 3.3.
More recently Alex Gaynor wrote a similar post to Armins with the main point that upgrading
is hard, and the Python Core Devs should make it easier.
While I dont want to discount what Armin and Alex have said, as they are both quite intelligent guys, I dont see any value in keeping 2.x around any longer than necessary. I agree
that the upgrade process has been kind of confusing and there havent seemed to be a ton
of incentives to upgrade, but right now we have this scenario where we have to support two
non-compatible versions, and that is never good. Look, we should all just switch to Python
3 as fast as possible (weve had 5 years already) and never look back. The sooner we put this
behind us, the sooner we can get back to focusing on the joy of Python and writing awesome
code.
Of course the elephant in the room is third party library support. While the support of Python
3 is getting better, there are still a number of third party libraries that dont support Python
3. So before you upgrade, make sure the libraries that you use support Python 3, otherwise
you may be in for some headaches.
With that said, lets look at some of the features that are in Python 3 that would make upgrading worthwhile.
110

The positive

Heres a short list: Unicode everywhere, virtualenv built-in, Absolute import, Set literals,
New division, Function Signature Objects, print() function, Set comprehension, Dict Comprehension, multiple context managers, C-based IO library, memory views, numbers modules, better exceptions, Dict Views, Type safe comparisons, standard library cleanup, logging dict config, WSGI 1.0.1, super() no args, Unified Integers, Better OS exception hierarchy, Metaclass class arg, lzma module, ipaddress module, faulthandler module, email packages rewritten, key-sharing dicts, nonlocal keyword, extended iterable unpacking, stable ABI,
qualified names, yield from, import implemented in python, __pycache__, fixed import
deadlock, namespace packages, keyword only arguments, function annotations, and much,
much more!
While we wont cover each of these in detail, there are two really good places to find information on all the new features:
The Python docs whats new page.
Brett Cannons talk (core python developer) on why 3.3 is better than 2.7.
There are actually a lot of really cool features, and throughout the rest of this book well introduce several of them to you.
So now you know both sides of the story. Lets look at the upgrade process.

Final checks before the upgrade

The first thing to do is to make sure that all of the third party Django applications and tools
as well as any Python libraries that you are using support Python 3.
You can find a list of packages hosted by PyPI that support Python 3 here. Some developers
dont update PyPI so frequently, so its worth checking on the project page or GitHub repo
for a project if you dont find it in the list, as you may often find a fork that supports Python
3, or even a version that has the support that just hasnt been pushed to PyPI yet. Of course
if the support is not there, you could always do the Python 3 upgrade yourself (Im about to
show you how ) and submit a patch. :)
NOTE: You could also check whether your project is ready for Python 3 based
on its dependencies here.

111

What type of upgrade is best?

There are several ways to perform an upgrade to Python 3. It all depends upon what end
result you want. Lets look at a couple of the more common ways.
I want to support Python 2 and 3 at the same time

If youre writing a reusable Django app that you expect others to use, or some sort of library
that you want to give to the community, then this is a valid option and what you most likely
want to do. If youre writing your own web app (as we are doing in this course) it makes no
sense to support both 2 and 3. Just upgrade to 3 and move on. Still, lets examine how you
might support both 2 and 3 and the same time.
1. Drop support for python 2.5 - while it is possible to support all versions of Python from
2.5 to 3.4 on the same code base, its probably more trouble than its worth. Python 2.6
was released in 2008, and given that its a trivial upgrade from 2.5 to 2.6, it shouldnt
take anybody more than 6 years to do the upgrade. So just drop the support for 2.5 and
make your life easier. :)
2. Set up a Python 3 virtualenv assuming you already have an existing Python 2 virtualenv that youve been developing on, set up a new one with Python 3 so you can test
your changes against both Python 2 and Python 3 and make sure they work in both
cases. You can set up a virtualenv using Python 3 with the following command:
1

$ virtualenv -p /usr/local/bin/python3 <path/to/new/virtualenv/>

where /usr/local/bin/python3 is the path to wherever you installed Python 3 on

your system.
3. Use 2to3 - 2to3 is a program that will scan your Python project and report on what
needs to change to support Python 3. It can be used with the -w option asking it to
change the files automatically, but you wouldnt want to do that if you want to continue
to offer Python 2 support. However, the report it generates can be invaluable in helping
you make the necessary changes.
4. Make the changes - be sure to test the code on your Python 2 virtualenv as well as your
Python 3 virtualenv. When you have everything working, youre done. Pat yourself on
the back.
Finally, heres a few resources that may help if you get stuck:

112

 Pythons official documentation on Porting to Python 3

Also the six compatibility layer is a wonderful library that takes care of a lot of the
work of supporting both Python 2 and 3 at the same time. You can find it here. Also
note that six is what Django uses internally to support both 2 and 3 on the same code
base, so you cant go wrong with six.
As an example of a project upgraded to support both Python 2 and 3 on the same code
base, have a look at the following pull request. (Recognize the name of the contributor?)
I just want to upgrade to Python 3 and never look back

This is what we are going to do for this project. Lets look at it step by step.
1. Create a virtualenv for Python 3:
1

$ mkvirtualenv -p /usr/local/bin/python3 py3

2. Install all our dependencies in the upgraded environment:

1

$ pip install -r requirements.txt

The previous commands creates a new virtualenv with Python 3, switches
you into that environment, and then installs all the project dependencies.

3. Run 2to3 - now lets see what we need to change to support Python 3:
1

$ 2to3 django_ecommerce

It will return a diff listing of what needs to change, which should look like:
1
2
3
4
5
6
7
8
9

10

RefactoringTool: Skipping implicit fixer: buffer

RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: No changes to django_ecommerce/manage.py
RefactoringTool: No changes to django_ecommerce/contact/forms.py
RefactoringTool: No changes to django_ecommerce/contact/models.py
RefactoringTool: No changes to django_ecommerce/contact/views.py
RefactoringTool: No changes to
django_ecommerce/django_ecommerce/settings.py
RefactoringTool: No changes to
django_ecommerce/django_ecommerce/urls.py
113

11

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

RefactoringTool: No changes to
django_ecommerce/django_ecommerce/wsgi.py
RefactoringTool: No changes to django_ecommerce/main/views.py
RefactoringTool: Refactored django_ecommerce/payments/forms.py
--- django_ecommerce/payments/forms.py (original)
+++ django_ecommerce/payments/forms.py (refactored)
@@ -29,12 +29,12 @@
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
label=(u'Password'),
+
label=('Password'),
widget=forms.PasswordInput(render_value=False)
)
ver_password = forms.CharField(
required=True,
label=(u' Verify Password'),
+
label=(' Verify Password'),
widget=forms.PasswordInput(render_value=False)
)

30
31
32
33
34
35
36
37

RefactoringTool: No changes to django_ecommerce/payments/models.py

RefactoringTool: Refactored django_ecommerce/payments/views.py
--- django_ecommerce/payments/views.py (original)
+++ django_ecommerce/payments/views.py (refactored)
@@ -33,7 +33,7 @@
else:
form = SigninForm()

38
39
40

print form.non_field_errors()
print(form.non_field_errors())

41
42
43
44
45
46
47
48
49

return render_to_response(
'sign_in.html',
@@ -92,11 +92,11 @@
'register.html',
{
'form': form,
'months': range(1, 12),
+
'months': list(range(1, 12)),
114

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': user,
'years': range(2011, 2036),
'years': list(range(2011, 2036)),
},
context_instance=RequestContext(request)

)
@@ -133,8 +133,8 @@
'form': form,
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'months': range(1, 12),
'years': range(2011, 2036)
+
'months': list(range(1, 12)),
+
'years': list(range(2011, 2036))
},
context_instance=RequestContext(request)
)
RefactoringTool: Files that need to be modified:
RefactoringTool: django_ecommerce/manage.py
RefactoringTool: django_ecommerce/contact/forms.py
RefactoringTool: django_ecommerce/contact/models.py
RefactoringTool: django_ecommerce/contact/views.py
RefactoringTool: django_ecommerce/django_ecommerce/settings.py
RefactoringTool: django_ecommerce/django_ecommerce/urls.py
RefactoringTool: django_ecommerce/django_ecommerce/wsgi.py
RefactoringTool: django_ecommerce/main/views.py
RefactoringTool: django_ecommerce/payments/forms.py
RefactoringTool: django_ecommerce/payments/models.py
RefactoringTool: django_ecommerce/payments/views.py

The diff listing shows us a number of things that need to change. We could just run
2to3 -w and let it handle these changes for us, but that doesnt teach us much. So
lets look at the errors and see where we need to make changes.

2to3
1
2

RefactoringTool: Refactored django_ecommerce/payments/forms.py

--- django_ecommerce/payments/forms.py (original)
115

3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

+++ django_ecommerce/payments/forms.py (refactored)

@@ -29,12 +29,12 @@
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
label=(u'Password'),
+
label=('Password'),
widget=forms.PasswordInput(render_value=False)
)
ver_password = forms.CharField(
required=True,
label=(u'Verify Password'),
+
label=('Verify Password'),
widget=forms.PasswordInput(render_value=False)
)

In the first change, we just need to add u in front of the strings. Remember in Python 3
everything is unicode. There are no more pure ansi strings (there could however be byte
strings, denoted with a b), so this change is just denoting the strings as unicode. Strictly
speaking in Python 3 this is not required, in fact its redundant, since strings are always treated
as unicode. 2to3 puts it in so you can support both Python 2 and 3 at the same time, which
we are not doing. So we can skip this one.
On to the next issues:
1
2

print form.non_field_errors()
print(form.non_field_errors())

In Python 3 print is a function, and like any function, you have to surround the arguments
with (). This has been changed just to standardize things. So we need to make that change.
The next couple of issues are the same:
1
2

'years': range(2011, 2036),

'years': list(range(2011, 2036)),

+
+

'months': range(1, 12),

'years': range(2011, 2036)
'months': list(range(1, 12)),
'years': list(range(2011, 2036))

and
1
2
3
4

116

Above 2to3 is telling us that we need to convert the return value from a range() to a list.
This is because in Python 3, range() effectively behaves the same as xrange() in Python
2. The switch to returning a generator - range() in Python 3 and xrange() in Python 2 - is
for efficiency purposes. With generators there is no point in building an entire list if we are
not going to use it. In our case here we are going to use all the values, always, so we just go
ahead and covert the generator to a list.
Thats about it for code.
You can go ahead and make the changes manually or let 2to3 do it for you:
1

$ 2to3 django_ecommerce -w

Test 2to3 changes

If we try to run our tests we will get a bunch of errors because the tests also need to be converted to Python 3. Running 2to3 on our tests directory - 2to3 tests - will give us the
following errors.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

RefactoringTool: Skipping implicit fixer: buffer

RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: No changes to tests/contact/testContactModels.py
RefactoringTool: No changes to tests/main/testMainPageView.py
RefactoringTool: No changes to tests/payments/testCustomer.py
RefactoringTool: Refactored tests/payments/testForms.py
--- tests/payments/testForms.py (original)
+++ tests/payments/testForms.py (refactored)
@@ -29,9 +29,9 @@
def test_signin_form_data_validation_for_invalid_data(self):
invalid_data_list = [
{'data': {'email': '[email protected]'},
'error': ('password', [u'This field is required.'])},
+
'error': ('password', ['This field is required.'])},
{'data': {'password': '1234'},
'error': ('email', [u'This field is required.'])}
+
'error': ('email', ['This field is required.'])}
]

21
22
23

for invalid_data in invalid_data_list:

@@ -78,14 +78,14 @@
117

'data': {'last_4_digits': '123'},

'error': (
'last_4_digits',
[u'Ensure this value has at least 4 characters

24
25
26
27

28

(it has 3).']

['Ensure this value has at least 4 characters
(it has 3).']
)

29

},
{

30
31

'data': {'last_4_digits': '12345'},

'error': (
'last_4_digits',
[u'Ensure this value has at most 4 characters

32
33
34
35

36

(it has 5).']

['Ensure this value has at most 4 characters
(it has 5).']
)

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

}
]
RefactoringTool: No changes to tests/payments/testUserModel.py
RefactoringTool: Refactored tests/payments/testviews.py
--- tests/payments/testviews.py (original)
+++ tests/payments/testviews.py (refactored)
@@ -80,11 +80,11 @@
'register.html',
{
'form': UserForm(),
'months': range(1, 12),
+
'months': list(range(1, 12)),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': None,
'years': range(2011, 2036),
+
'years': list(range(2011, 2036)),
}
)
ViewTesterMixin.setupViewTester(
RefactoringTool: Files that need to be modified:
RefactoringTool: tests/contact/testContactModels.py
118

60
61
62
63
64

RefactoringTool:
RefactoringTool:
RefactoringTool:
RefactoringTool:
RefactoringTool:

tests/main/testMainPageView.py
tests/payments/testCustomer.py
tests/payments/testForms.py
tests/payments/testUserModel.py
tests/payments/testviews.py

Looking through the output we can see the same basic errors that we had in our main code:
1. Include u before each string literal,
2. Convert the generator returned by range() to a list, and
3. Call print as a function.
Applying similar fixes as we did above will fix these, and then we should be able to run our
tests and have them pass. To make sure after you have completed all your changes, type the
following from the terminal:
1
2

$ cd django_ecommerce
$ ./manage.py test ../tests

Whoa! Errors all over the place.

1
2
3
4
5
6

7
8
9

10
11
12
13

Creating test database for alias 'default'...

F................F.....
.
...F.
======================================================================
FAIL: test_contactform_str_returns_email
(tests.contact.testContactModels.UserModelTest)
---------------------------------------------------------------------Traceback (most recent call last):
File "../testContactModels.py", line 23, in
test_contactform_str_returns_email
self.assertEquals("[email protected]", str(self.firstUser))
AssertionError: '[email protected]' != 'ContactForm object'
- [email protected]
+ ContactForm object

14
15
16
17

======================================================================
FAIL: test_registering_new_user_returns_succesfully
(tests.payments.testviews.RegisterPageTests)
119

18
19
20

21
22

23
24

---------------------------------------------------------------------Traceback (most recent call last):

File "../py3/lib/python3.4/site-packages/mock.py", line 1201, in
patched
return func(*args, **keywargs)
File "../testviews.py", line 137, in
test_registering_new_user_returns_succesfully
self.assertEquals(resp.content, "")
AssertionError: b'' != ''

25
26
27

28
29
30
31
32

======================================================================
FAIL: test_returns_correct_html
(tests.payments.testviews.SignOutPageTests)
---------------------------------------------------------------------Traceback (most recent call last):
File "../testviews.py", line 36, in test_returns_correct_html
self.assertEquals(resp.content, self.expected_html)
AssertionError: b'' != ''

33
34
35

---------------------------------------------------------------------Ran 29 tests in 0.297s

36
37
38

FAILED (failures=3)
Destroying test database for alias 'default'...

As it turns out, 2to3 cant catch everything, so we need to go in and fix the issues. Lets deal
with them one at a time.
First error
1
2

3
4
5

6
7

======================================================================
FAIL: test_contactform_str_returns_email
(tests.contact.testContactModels.UserModelTest)
---------------------------------------------------------------------Traceback (most recent call last):
File
"/Users/michaelherman/Documents/repos/realpython/book3-exercises/_chapters/ch
line 23, in test_contactform_str_returns_email
self.assertEquals("[email protected]", str(self.firstUser))
AssertionError: '[email protected]' != 'ContactForm object'
120

8
9

- [email protected]
+ ContactForm object

It appears that calling str and passing our ContactForm no longer returns the users name.
If we look at contact/models.py/ContactForm we can see the following function:
1
2

def __unicode__(self):
return self.email

In Python 3 the unicode() built-in function has gone away and str() always returns a
unicode string. Therefore the __unicode__() function is just ignored.
Change the name of the function to fix this issue:
1
2

def __str__(self):
return self.email

This and various other Python 3-related errata for Django can be found here.
Run the tests again. A few left, but lucky for us they all have the same solution
Unicode vs Bytestring errors

They basically all look like the following:

1

2
3
4

5
6

7
8

FAIL: test_registering_new_user_returns_succesfully
(tests.payments.testviews.RegisterPageTests)
---------------------------------------------------------------------Traceback (most recent call last):
File "/py3/lib/python3.4/site-packages/mock.py", line 1201, in
patched
return func(*args, **keywargs)
File "../testviews.py", line 137, in
test_registering_new_user_returns_succesfully
self.assertEquals(resp.content, "")
AssertionError: b'' != ''

In Python 2 a bytestring ( b'some_string' ) is effectively the same as a unicode string (

u'some_string' ) as long as some_string only contains ASCII data. However in Python 3
bytestrings should be used only for binary data or when you actually want to get at the bits
and bytes. In Python 3 b'some_string' !=usome_string.
BUT, and the big but here, is that according to PEP 3333, input and output streams are
always byte objects. What is response.content? Its a stream. Thus, it should be a byte
object.

121

Djangos recommended solution for this is to use assertContains() or assertNotContains().

Unfortunately assertContains doesnt handle redirects - e.g., a status_code of 302. And
from our errors we know its the redirects that are causing the problems. The solution then is to change the setUpClass method for the classes that test for redirects in
test/payments/testViews in our SignOutPageTest class:
1

class SignOutPageTests(TestCase, ViewTesterMixin):

@classmethod
def setUpClass(cls):
ViewTesterMixin.setupViewTester(
'/sign_out',
sign_out,
b"", # a redirect will return an empty bytestring
status_code=302,
session={"user": "dummy"},
)

3
4
5
6
7
8
9
10
11

Here, we just changed the third argument from "" to b"", because response.context
is now a bytestring, so our expected_html must also be a bytestring. The last error in
tests/payments/testviews/ in the test_registering_new_user_returns_succesfully
test is the same type of bytestring error, with the same type of fix:
Update:

python self.assertEqual(resp.content, b)
To:
1

self.assertEqual(resp.content, b"")

Same story as before: We need to make sure our string types match. And with that, we can
rerun our tests and they all pass.
1
2
3
4
5
6

Creating test database for alias 'default'...

.......................
.
.....
---------------------------------------------------------------------Ran 29 tests in 0.430s

7
8
9

OK
Destroying test database for alias 'default'...

122

Success!
So now our upgrade to Python 3 is complete. We can commit our changes, and merge back
into the master branch. We are now ready to move forward with Django 1.8 and Python 3.
Awesome.

123

Python 3 Changes Things Slightly

Before moving on with the rest of the chapter, have a look around the directory tree for the
project. Notice anything different? You should now have a bunch of __pycache__ directories. Lets look at the contents of one of them:
1
2
3
4
5

$ ls -al django_ecommerce/django_ecommerce/__pycache__
total 24
drwxr-xr-x 5 michaelherman staff 170 Jul 25 19:12 .
drwxr-xr-x 11 michaelherman staff 374 Jul 25 19:12 ..
-rw-r--r-- 1 michaelherman staff 208 Jul 25 19:12
__init__.cpython-34.pyc
-rw-r--r-- 1 michaelherman staff 2690 Jul 25 19:12
settings.cpython-34.pyc
-rw-r--r-- 1 michaelherman staff 852 Jul 25 19:12
urls.cpython-34.pyc

There are a few things to note about the directory structure:

1. Each of these files is a .pyc. No longer are pyc files littered throughout your project;
with Python 3 they are all kept in the appropriate __pycache__ directory. This is nice
as it cleans things up a bit and un-clutters your code directory.
2. Further, .pyc files are now in the format <file_name>.<vm_name>-<python_version>.pyc.
This allow for storing multiple .pyc files; if youre testing against Python 3.4 and 2.7,
for example, you dont have to regenerate your .pyc files each time you test against a
different environment. Also each VM (jython, pypy, etc.) can store its own .pyc files
so you can run against multiple VMs without regenerating .pyc files as well. This will
come in handy later when we look at running multiple test configurations with Travis
CI.
All in all the __pycache__ directory provides a cleaner, more efficient way of handling multiple versions of Python and multiple VMs for projects that need to do that. For projects that
dont well, at least it gets the .pyc files out of your code directories.
As said before, there are a ton of new features in Python 3 and we will see several of them
as we progress throughout the course. Weve seen a few of the features already during our
upgrade. Some may seem strange at first; most of them can be thought of as cleaning up
the Python API and making things clearer. While it may take a bit to get used to the subtle
changes in Python 3, its worth it. So stick with it.

124

Upgrading to PostgreSQL
Since we are on the topic of upgrading, lets see if we can fit one more quick one in.
Up until now we have been using SQLite, which is OK for testing and prototyping purposes,
but you would probably wouldnt want to use SQLite in production. (At least not for a high
traffic site. Take a look at SQLites own when to use page for more info). Since we are trying
to make this application production-ready, lets go ahead and upgrade to PostgreSQL. The
process is pretty straightforward, especially since Postgres fully supports Python 3.
By now it should go without saying that we are going to start working from a new branch..so
I wont say it. :). Also as with all upgrades, its a good idea to back things up first, so lets pull
all the data out of our database like this:
1

$ ./manage.py dumpdata > db_backup.json

This is a built-in Django command that exports all the data to JSON. The nice thing about
this is that JSON is database-independent, so we can easily pass this data to other databases.
The downside of this approach is that metadata like primary and foreign keys are not saved.
For our example that isnt necessary, but if you are trying to migrate a large database with
lots of tables and relationships, this may not be the best approach. Lets first get Postgres set
up and running, then we can come back and look at the specifics of the migration.

Step 1: Install PostgreSQL

There are many ways to install PostgreSQL depending upon the system that you are running,
including the prepackaged binaries, apt-get for debian-based systems, brew for Mac, compiling from source and others. Were not going to go into the nitty-gritty details of every single
means of installation; youll have to figure that out on your own. Use your Google searching
skills. Check out the official docs for help - PostgreSQL Installation Page
Windows Users

The easiest way to install Postgres on Windows is with a product called EnterpriseDB. Download it from here. Make sure to choose the newest version available for your operating system.
Download the installer, run it and then follow these instructions.
Mac OS Users

Brew it baby. We talked about homebrew before so now that you have it installed from the
command line just type:
1

$ brew install postgresql

125

Debian Users

1. Install Postgres
1

$ sudo apt-get install postgresql

2. Verify installation is correct:

1

$ psql --version

And you should get something like the following:

1

psql (PostgreSQL) 9.2.4

3. Now that Postgres is installed, you need to set up a database user and create an account
for Django to use. When Postgres is installed, the system will create a user named
postgres. Lets switch to that user so we can create an account for Django to use:
1

$ sudo su postgres

4. Creates a Django user in the database:

1

$ createuser -P djangousr

Enter the password twice and remember it; you will use it in your settings.py later.
5. Now using the postgres shell, create a new database to use in Django. Note: dont type
the lines starting with a #. These are comments for your benefit.
1

$ psql

2
3
4

# Once in the shell type the following to create the database

CREATE DATABASE django_db OWNER djangousr ENCODING 'UTF8';

5
6
7

# Then to quit the shell type

$ \q

6. Then we can set up permissions for Postgres to use by editing the file /etc/postgresql/9.1/main/pg_
Just add the following line to the end of the file:
1

local

django_db

djangousr

md5

Then save the file and exit the text editor. The above line basically says djangousr
user can access the django_db database if they are initiating a local connection and
using an md5-encrypted password.
126

7. Finally restart the postgres service.

1

$ sudo /etc/init.d/postgresql restart

This should restart postgres and you should now be able to access the database. Check
that your newly created user can access the database with the following command:
1

$ psql django_db djangousr

This will prompt you for the password; type it in and you should get to the database
prompt. You can execute any SQL statements that you want from here, but at this point
we just want to make sure we can access the database, so just do a \q and exit out of
the database shell. Youre all set- Postgres is working! You probably want to do a final
exit from the command line to get back to the shell of your normal user.
If you do encounter any problems installing PostgreSQL, check the wiki. It has a lot of good
troubleshooting tips.
Now that Postgres is installed, you will want to install the Python bindings.

Step 2: Install Python bindings

Ensure that you are in the virtualenv that we created earlier, and then install the PostgreSQL
binding, psycopg2 with the following command:
1

$ pip install psycopg2

This will require a valid build environment and is prone to failure, so if it doesnt work you can
try one of the several pre-packaged installations. For example, if youre on a Debian-based
system, youll probably need to install a few dependencies first:
1

$ sudo apt-get install libpq-dev python-dev libpython3.3-dev

Once done, try re-running the pip install psycopg2 command.

Alternatively, if you want to install psycopg2 globally you can use apt-get for that:
1

$ sudo apt-get install install postgresql-plpython-9.1

WARNING: To be clear installing Postgres globally is not an issue as far as
this course is concerned. It can however become an issue if you are working on
multiple projects each using a different version of Postgres. If you have Postgres
installed globally then it makes it much more difficult to have multiple version
installed for different applications, which is exactly the problem that virtualenv
is meant to fix. If you dont plan on running multiple versions of Postgres then
by all means install it globally and dont worry about it.

127

Since we are adding another Python dependency to our project, we should update the
requirements.txt file in our project root directory to reflect this.
1

$ pip freeze > requirements.txt

The should now look something like this:

1
2
3
4
5

Django==1.8.2
mock==1.0.1
psycopg2==2.5.4
requests==2.3.0
stripe==1.9.2

Thats all the dependencies we need for now.

OSX Homebrew Users: If you installed Python and/or Postgres through homebrew and
the pip install psycopg2 command fails, ensure that youre using a version of Python
installed by homebrew and that you installed Postgres via homebrew. If you did, then everything should work properly. If however you have some combination of the OSX system
Python and a Homebrew Postgres - or vice versa - things are likely to fail.
Windows users: Probably the easiest user experience is to use the pre-packaged installers,
however these will install Postgres globally and not to your specific virtualenv. The installer
is available here. Be sure to get the Python 3 version!

Step 3: Configure Django

Youve got postgres set up and working, and youve got the Python bindings all installed. We
are almost there. Now lets configure Django to use Postgres instead of SQLite. This can be
done by modifying settings.py. Remove the SQLite entry and replace it with the following
entry for Postgres:
1
2
3
4
5
6
7
8
9
10

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db',
'USER': 'djangousr',
'PASSWORD': 'your_password_here',
'HOST': 'localhost',
'PORT': '5432',
}
}

128

Please note that 5432 is the default postgres port, but you can double-check your system
setup with the following command:
1
2
3

$ grep postgres /etc/services

postgresql
5432/udp
# PostgreSQL Database
postgresql
5432/tcp
# PostgreSQL Database

Once thats done just sync the new database:

1

$ ./manage.py syncdb
NOTE You will get a very different output from running syncdb than you are
used to after you upgrade to Django 1.8 This is because of the new key feature
of Django 1.8: migrations. We have an entire chapter coming up dedicated to
migrations, so for now just ignore the output, and we will come back to syncdb
and migrations in an upcoming chapter.

Step 4: Import backup data

The final thing is to import the db_backup.json file we created earlier:
1
2

$ ./manage.py loaddata db_backup.json

Installed 51 object(s) from 1 fixture(s)

Of course this isnt essential if youre following this course strictly as you wont yet have much,
if any, data to load. But this is the general idea of how you might go about migrating data from
one database to the next.
Dont forget to make sure all of our tests still run correctly:
1

$ ./manage.py test ../tests

Did you get this error?

1

Got an error creating the test database: permission denied to

create database

If so, check out this StackOverflow answer. Then enter the Postgres shell and run the following command:
1

ALTER USER djangousr CREATEDB;

Test again. Everything should pass. Congratulations! Youve upgraded to PostgreSQL. Be

sure to commit all your changes and merge your branch back to master - and then youre
ready to move on.
129

Conclusion
Whoa - thats a lot of upgrading. Get used to it. Software moves so fast these days, that
software engineers cant escape the reality of the upgrade. If it hasnt happened to you yet, it
will. Hopefully this chapter has provided you with some food for thought on how to approach
an upgrade and maybe even given you a few upgrade tools to keep in your tool-belt for the
next time you have to upgrade.

130

Exercises
There are some really good videos out there on upgrading to Python 3 and Django 1.6/1.7/1.8.
So for this chapters exercises, watch the following videos to gain a better understanding of
why you might want to upgrade:
1. Porting Django apps to Python 3 talk by the creator of Django Jacob Kaplan-Moss
2. Python 3.3 Trust Me, Its Better than 2.7 by Brett Cannon (core python developer) video

131

Chapter 7
Graceful Degradation and Database
Transactions with Django 1.8
Graceful Degradation used to be a hot topic. Today you dont hear about it too much, at least
not in the startup world where everyone is rushing to get their MVP out the door and get signups. Well fix it later, right? Starting in Django 1.6 database transactions were drastically
simplified. So we can actually apply a little bit of Graceful Degradation to our sign-in process
without having to spend too much time on it.

132

What is Graceful Degradation?

Lets start with a quick definition of Graceful Degradation: Put simple, its the property that
enables a system to continue operating properly in the event of a failure to some of its components. If something breaks, we just keep on ticking, in other words.
A common example of this in web development is support for Internet Explorer. Although
Internet Explorer is getting better, a lot of the cool functionality (WebGL for example) that
makes web sites fun simply is not supported. To compensate, developers must write the front
end code with a bunch of IF-type statements - IF browser == IE THEN do something basic
- because the cool stuff is broken.
Lets try to apply this thinking to our registration function.

133

User Registration
If Stripe is down, you still want your users to be able to register, right? We just want to hold
their info and then re-verify it when Stripe is back up, otherwise we will probably lose that
user to a competitor if we dont allow them to register until Stripe comes back up. Lets look
at how we can do that.
Following what we learned before, lets first create a branch for this feature:
1

$ git checkout -b transactions

Now that we are working in a clean environment, lets write a unit test in tests/payments/testViewsin theRegisterPageTests() class:
1

import socket

2
3

def test_registering_user_when_stripe_is_down(self):

4
5
6
7
8
9
10
11
12
13
14
15

#create the request used to test the view

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

16
17
18
19
20
21

#mock out Stripe and ask it to throw a connection error

with mock.patch(
'stripe.Customer.create',
side_effect=socket.error("Can't connect to Stripe")
) as stripe_mock:

22
23
24

#run the test

register(self.request)

25
26
27

#assert there is a record in the database without Stripe id.

users = User.objects.filter(email="[email protected]")
134

self.assertEquals(len(users), 1)
self.assertEquals(users[0].stripe_id, '')

28
29

That should do it. We have just more or less copied the test for test_registering_new_user_returns_succ
but removed all the databases mocks and added a stripe_mock that throws a socket.error
every time its called. This should simulate what would happen if Stripe goes down.
Of course running this test is going to fail - OSError: Can't connect to Stripe. But
thats exactly what we want (remember the TDD Fail, Pass, Refactor loop). Dont forget to
commit that change.
Okay. So, how can we get it to work?
Well, we want to hold their info and then re-verify it when Stripe is back up.
First thing is to update the Customer.create() in payments/views/ method to handle
that pesky socket.error so we dont have to deal with it.
1

import socket

2
3

class Customer(object):

@classmethod
def create(cls, billing_method="subscription", **kwargs):
try:
if billing_method == "subscription":
return stripe.Customer.create(**kwargs)
elif billing_method == "one_time":
return stripe.Charge.create(**kwargs)
except socket.error:
return None

5
6
7
8
9
10
11
12
13

This way when Stripe is down, our call to Customer.create() will just return None. This
design is preferable so we dont have to put try-except blocks everywhere.
Rerun the test. It should still fail:
1

AttributeError: 'NoneType' object has no attribute 'id'

So now we need to modify the register() function. Basically all we have to do is change
the part that saves the user.
Change:
1
2

try:
user = User.create(
135

3
4
5
6
7
8
9

cd['name'],
cd['email'],
cd['password'],
cd['last_4_digits'],
customer.id
)
except IntegrityError:

To:
1
2
3
4
5
6
7
8

try:
user = User.create(
cd['name'],
cd['email'],
cd['password'],
cd['last_4_digits'],
stripe_id=''
)

9
10
11
12

if customer:
user.stripe_id = customer.id
user.save()

13
14

except IntegrityError:

Here, we broke up the single insert on the database into an insert and then an update.
Running the test still fails, though.
Since we are not passing in the stripe_id when we initially create the user, we have to
change the test_registering_new_user_returns_succesfully() test to not expect
that. In fact, lets remove all the database mocks from that test, because in a minute we are
going to start adding some transaction management.
As a generally rule, its best not to use database mocks when testing code that
directly manages or uses transactions.

Why? This is because the mocks will effectively ignore all transaction management, and thus
subtle defects can often slide into play with the developer thinking that the code is well tested
and free of defects.
After we take the database mocks out of test_registering_new_user_returns_succesfully()
the test looks like this:

136

def get_mock_cust():

2
3

class mock_cust():

4
5
6
7

@property
def id(self):
return 1234

8
9

return mock_cust()

10
11

12

@mock.patch('payments.views.Customer.create',
return_value=get_mock_cust())
def test_registering_new_user_returns_succesfully(self,
stripe_mock):

13
14
15
16
17
18
19
20
21
22
23

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

24
25

resp = register(self.request)

26
27
28

self.assertEqual(resp.content, b"")
self.assertEqual(resp.status_code, 302)

29
30
31
32

users = User.objects.filter(email="[email protected]")
self.assertEqual(len(users), 1)
self.assertEqual(users[0].stripe_id, '1234')

33
34

def get_MockUserForm(self):

35
36

from django import forms

37
38

class MockUserForm(forms.Form):
137

39

def is_valid(self):
return True

40
41
42

@property
def cleaned_data(self):
return {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

43
44
45
46
47
48
49
50
51
52
53

def addError(self, error):

pass

54
55
56

return MockUserForm()

57

In the above test we want to explicitly make sure that the stripe_id IS being set, so we
mocked the Customer.create() function and had it return a dummy class that always
provides 1234 for its id. That way we can assert that the new user in the database has the
stripe_id of 1234.
All good:
1
2
3
4
5
6

Creating test database for alias 'default'...

........................
.
.....
---------------------------------------------------------------------Ran 30 tests in 0.505s

7
8
9

OK
Destroying test database for alias 'default'...

Time for a commit.

138

Handling Unpaid Users

At this point we are now letting users register even if Stripe is down.
In effect this means we are letting the user in for free. Obviously if everybody starts getting
in for free, it wont be long before our site tanks, so lets fix that.
WARNING: The solution were about to propose isnt the most elegant, but we
need an example to use for database transactions that fits into the our current
Project! Plus, well give you a chance to fix it later in the exercises.

The proposed solution to this endeavor is to create a table of unpaid users so we can harass
those users until they cough up their credit cards. Or, if you want to be a bit more politically
correct: So our account management system can help the customers with any credit card
billing issues they may have had.
To do that, we will create a new table called unpaid_users with two columns: the user email
and a timestamp used to keep track of when we last contacted this user to update billing
information.
First lets create a new model in payments/models.py:
1

from django.utils import timezone

2
3
4
5

class UnpaidUsers(models.Model):
email = models.CharField(max_length=255, unique=True)
last_notification = models.DateTimeField(default=timezone.now())
NOTE: Were intentionally leaving off foreign key constraints for now. We may
come back to it later.
Further NOTE: django.utils.timezone.now() functions the same as
datetime.now() except that timezone is always timezone aware. This will
prevent Django from complaining about naive timezone.
Further Further NOTE: After creating the new model, youll want to run
./manage.py syncdb. This will show some different output than youre used
to as Django 1.7 now has migration support. For the time being we will stick with
syncdb, however there is an upcoming chapter on Migrations which will explain
Django 1.7s migrations.

139

Now create the test. We want to ensure that the UnpaidUsers() table is populated if/when
Stripe is down. Lets modify our payments/testViews/test_registering_user_when_strip_is_down
test. All we need to do is add a couple of asserts at the end of that test:
1
2
3
4

# check the associated table got updated.

unpaid = UnpaidUsers.objects.filter(email="[email protected]")
self.assertEquals(len(unpaid), 1)
self.assertIsNotNone(unpaid[0].last_notification)

Make sure to update the imports:

1

from payments.models import User, UnpaidUsers

This test asserts that we got a new row in the UnpaidUsers() table and it has a
last_notification timestamp. Run the test watch it fail.
Now lets fix the code.
Of course we have to populate the table during registration if a user fails to validate their card
through Stripe. So lets adjust payments.views.registration as is shown below:
1

def register(request):

2
3

... snip ...

4
5
6
7

cd = form.cleaned_data
try:
user = User.create(cd['name'], cd['email'],
cd['password'],
cd['last_4_digits'],
stripe_id='')

9
10
11
12
13
14

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

15
16
17
18
19
20

except IntegrityError:
import traceback
form.addError(cd['email'] + ' is already a member' +
traceback.format_exc())
user = None
140

else:
request.session['user'] = user.pk
return HttpResponseRedirect('/')

21
22
23
24
25

... snip ...

Again, make sure to update the imports:

1

from payments.models import User, UnpaidUsers

Now re-run the tests and they should all pass. Were golden.
Right? Not exactly.

141

Improved Transaction Management

If youve ever devoted much time to Django database transaction management, you know
how confusing it can get. In the past (prior to Django 1.6), the documentation provided quite
a bit of depth, but understanding only came through building and experimenting.

For example, there was a plethora of decorators to work with, like: commit_on_success,
commit_manually, commit_unless_managed, rollback_unless_managed, enter_transaction_manag
leave_transaction_management, just to name a few. Fortunately, with Django 1.6 (or
greater, of course) that all goes out the door. You only really need to know about a few
functions for now, which well get to in just a few seconds. First, well address these topics:
What is transaction management?
Whats wrong with transaction management prior to Django 1.6?
Before jumping into:
Whats right about transaction management in Django 1.6?
And then dealing with a detailed example:
Stripe Example
Transactions
The recommended way
Using a decorator
Transaction per HTTP Request

SavePoints
Nested Transactions

142

What is a transaction?
According to SQL-92, An SQL-transaction (sometimes simply called atransaction) is a sequence of executions of SQL-statements that is atomic with respect to recovery. In other
words, all the SQL statements are executed and committed together. Likewise, when rolled
back, all the statements get rolled back together.
For example:
1
2
3
4
5
6

# START
note1 = Note(title="my first note", text="Yay!")
note2 = Note(title="my second note", text="Whee!")
note1.save()
Note2.save()
# COMMIT

A transaction is a single unit of work in a database, and that single unit of work is demarcated
by a start transaction and then a commit or an explicit rollback.

143

Whats wrong with transaction management prior to

Django 1.6?
In order to fully answer this question, we must address how transactions are dealt with in the
database, client libraries, and within Django.

Databases
Every statement in a database has to run in a transaction, even if the transaction includes
only one statement.
Most databases have an AUTOCOMMIT setting, which is usually set to True as a default.
This AUTOCOMMIT wraps every statement in a transaction that is immediately committed if
the statement succeeds. You can also manually call something like START_TRANSACTION
which will temporarily suspend the AUTOCOMMIT until you call COMMIT_TRANSACTION or
ROLLBACK.
However, the takeaway here is that the AUTOCOMMIT setting applies an implicit commit after
each statement.

Client Libraries
Then there are the Python client libraries like sqlite3 and mysqldb, which allow Python
programs to interface with the databases themselves. Such libraries follow a set of standards
for how to access and query databases. That standard, DB API 2.0, is described in PEP 249.
While it may make for some slightly dry reading, an important takeaway is that PEP 249
states that the database AUTOCOMMIT should be OFF by default.
This clearly conflicts with whats happening within the database:
SQL statements always have to run in a transaction, which the database generally
opens for you via AUTOCOMMIT.
However, according to PEP 249, this should not happen.
Client libraries must mirror what happens within the database, but since they are not
allowed to turn AUTOCOMMIT on by default, they simply wrap your SQL statements in
a transaction, just like the database.
Okay. Stay with me a little longer

144

Django
Enter Django. Django also has something to say about transaction management. In Django
1.5 and earlier, Django basically ran with an open transaction and auto-committed that
transaction when you wrote data to the database. So every time you called something like
model.save() or model.update(), Django generated the appropriate SQL statements
and committed the transaction.
Also in Django 1.5 and earlier, it was recommended that you used the TransactionMiddleware
to bind transactions to HTTP requests. Each request was given a transaction. If the response
returned with no exceptions, Django would commit the transaction, but if your view function
threw an error, ROLLBACK would be called. In effect, this turned off AUTOCOMMIT. If you
wanted standard, database-level autocommit style transaction management, you had to manage the transactions yourself - usually by using a transaction decorator on your view function
such as @transaction.commit_manually, or @transaction.commit_on_success.
Take a breath. Or two.

What does this mean?

Yes, there is a lot going on there, and it turns out most developers just want the standard
database level autocommits - meaning transactions stay behind the scenes, doing their thing,
until you need to manually adjust them.

145

Whats right about transaction management in Django

1.6 and above?
Now, welcome to Django 1.6 and above. Simply remember that in Django 1.6 (or greater),
you use database AUTOCOMMIT and manage transactions manually when needed. Essentially,
we have a much simpler model that basically does what the database was designed to do in
the first place!
Coming back to our earlier question: Is our registration function really Golden? We just want
to hold users info and then re-verify it with Stripe.
We wrote a failing test, then made it pass. And now its time for the refactor portion of TDD.
Thinking about transactions and keeping in mind that by default Django gives us AUTOCOMMIT behavior for our database, lets stare at the code a little longer.
1
2
3
4

cd = form.cleaned_data
try:
user = User.create(cd['name'], cd['email'], cd['password'],
cd['last_4_digits'], stripe="")

5
6
7
8
9
10

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

11
12
13
14
15

except IntegrityError:
import traceback
form.addError(cd['email'] + ' is already a member' +
traceback.format_exc())

Did you spot the issue? We would just want to hold their info and then re-verify it
What happens if the UnpaidUsers(email=cd['email']).save() line fails?
Well, then you have a user registered in the system who never verified their credit card while
the system assumes they have. In other words, somebody got in for free. Not good. So this
is the perfect case for when to use a transaction, because we want it all or nothing here. In
other words, we only want one of two outcomes:
1. User is created (in the database) and has a stripe_id.
146

2. User is created (in the database), doesnt have a stripe_id and has an associated row
in the UnpaidUsers table with the same email address as the User.
This means we want the two separate database statements to either both commit
or both rollback. A perfect case for the humble transaction. There are many ways we can
achieve this.

First, lets write some tests to verify things behave the way we want them to:
1
2
3

@mock.patch('payments.models.UnpaidUsers.save',
side_effect=IntegrityError)
def test_registering_user_when_strip_is_down_all_or_nothing(self,
save_mock):

4
5
6
7
8
9
10
11
12
13
14
15

#create the request used to test the view

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

16
17
18
19
20
21

#mock out stripe and ask it to throw a connection error

with mock.patch(
'stripe.Customer.create',
side_effect=socket.error("can't connect to stripe")
) as stripe_mock:

22
23
24

#run the test

resp = register(self.request)

25
26
27
28

#assert there is no new record in the database

users = User.objects.filter(email="[email protected]")
self.assertEquals(len(users), 0)

29
30

#check the associated table has no updated data

147

unpaid =
UnpaidUsers.objects.filter(email="[email protected]")
self.assertEquals(len(unpaid), 0)

31

32

This test is more or less a copy of test_register_user_when_stripe_is_down(), except

we added the @mock.patch decorator that throws an IntegrityError when save() is
called on UnpaidUsers.
Run the test:
1
2
3
4
5
6
7

8
9
10

11
12

13
14

Creating test database for alias 'default'...

...................F
.....
.
.....
======================================================================
FAIL: test_registering_user_when_strip_is_down_all_or_nothing
(tests.payments.testviews.RegisterPageTests)
---------------------------------------------------------------------Traceback (most recent call last):
File
"/Users/michaelherman/Documents/repos/realpython/book3-exercises/_chapters/ch
line 1201, in patched
return func(*args, **keywargs)
File
"/Users/michaelherman/Documents/repos/realpython/book3-exercises/_chapters/ch
line 273, in
test_registering_user_when_strip_is_down_all_or_nothing
self.assertEquals(len(users), 0)
AssertionError: 1 != 0

15
16
17

---------------------------------------------------------------------Ran 31 tests in 1.381s

18
19
20

FAILED (failures=1)
Destroying test database for alias 'default'...

Nice. It failed. Seems funny to say that, but its exactly what we wanted. And the error
message tells us that the User is indeed being stored in the database; we dont want that.
Have no fear, transactions to the rescue

148

Creating the transaction in Django

There are actually several ways to create the transaction in Django 1.6. Lets go through a
couple.
The recommended way

According to Django 1.6 documentation, atomic can be used as both a decorator or as a

context_manager. So if we use it as a context manager, the code in our register function
would look like this:
1

def register(request):

2
3

... snip ...

cd = form.cleaned_data
try:
with transaction.atomic():
user = User.create(cd['name'], cd['email'],
cd['password'],
cd['last_4_digits'],
stripe_id="")

5
6
7
8

10

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

11
12
13
14
15
16

except IntegrityError:
import traceback
form.addError(cd['email'] + ' is already a member' +
traceback.format_exc())
user = None
else:
request.session['user'] = user.pk
return HttpResponseRedirect('/')

17
18
19
20
21
22
23
24
25
26

... snip ...

Add the import:

149

from django.db import transaction

Note the line with transaction.atomic():. All code inside that block will be executed
inside a transaction. Re-run our tests, and they all pass!
Using a decorator

We can also try adding atomic as a decorator. But if we do and rerun our tests they fail
with the same error we had before putting any transactions in at all!
Why is that?
Why didnt the transaction roll back correctly? The reason is because transaction.atomic
is looking for some sort of DatabaseError and, well, we caught that error (e.g., the
IntegrityError in our try/except block), so transaction.atomic never saw it and thus
the standard AUTOCOMMIT functionality took over.
But removing the try/except will cause the exception to just be thrown up the call chain and
most likely blow up somewhere else, so we cant do that either.
The trick is to put the atomic context manager inside of the try/except block, which is what
we did in our first solution.
Looking at the correct code again:
1

from django.db import transaction

2
3
4
5
6

try:
with transaction.atomic():
user = User.create(cd['name'], cd['email'], cd['password'],
cd['last_4_digits'], stripe_id="")

7
8
9
10
11
12

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

13
14
15
16
17

except IntegrityError:
import traceback
form.addError(cd['email'] + ' is already a member' +
traceback.format_exc())

150

When UnpaidUsers fires the IntegrityError, the transaction.atomic() context_manager will catch it and perform the rollback. By the time our code executes in
the exception handler (e.g., the form.addError line), the rollback will be complete and
we can safely make database calls if necessary. Also note any database calls before or after
the transaction.atomic() context manager will be unaffected regardless of the final
outcome of the context_manager.
Transaction per HTTP Request

Django < 1.6 (like 1.5) also allows you to operate in a Transaction per request mode. In
this mode, Django will automatically wrap your view function in a transaction. If the function throws an exception, Django will roll back the transaction, otherwise it will commit the
transaction.
To get it set up you have to set ATOMIC_REQUEST to True in the database configuration for
each database that you want to have this behavior. In our settings.py we make the change
like this:
1
2
3
4
5
6
7

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(SITE_ROOT, 'test.db'),
'ATOMIC_REQUEST': True,
}
}

But in practice this just behaves exactly as if you put the decorator on your view function
yourself, so it doesnt serve our purposes here. It is however worthwhile to note that with
both AUTOMIC_REQUESTS and the @transaction.atomic decorator it is possible to still
catch and then handle those errors after they are thrown from the view. In order to catch
those errors you would have to implement some custom middleware, or you could override
urls.handler or make a 500.html template.

151

SavePoints
We can also further break down transactions into savepoints. Think of savepoints as partial
transactions. If you have a transaction that takes four database statements to complete, you
could create a savepoint after the second statement. Once that savepoint is created, then if
the 3rd or 4th statements fails you can do a partial rollback, getting rid of the 3rd and 4th
statement but keeping the first two.
Its basically like splitting a transaction into smaller lightweight transactions, allowing you
to do partial rollbacks or commits. But do keep in mind if the main transaction were to get
rolled back (perhaps because of an IntegrityError that gets raised but not caught), then
all savepoints will get rolled back as well.
Lets look at an example of how savepoints work:
1
2

@transaction.atomic()
def save_points(self,save=True):

3
4
5

user = User.create('jj','inception','jj','1234')
sp1 = transaction.savepoint()

6
7
8
9

user.name = 'staring down the rabbit hole'

user.stripe_id = 4
user.save()

10
11
12
13
14

if save:
transaction.savepoint_commit(sp1)
else:
transaction.savepoint_rollback(sp1)

Here the entire function is in a transaction. After creating a new user we create a savepoint
and get a reference to the savepoint. The next three statements:
1
2
3

user.name = 'staring down the rabbit hole'

user.stripe_id = 4
user.save()

Are not part of the existing savepoint, so they stand the potential of being part of the next
savepoint_rollback, or savepoint_commit. In the case of a savepoint_rollback.
The line user = User.create('jj','inception','jj','1234') will still be committed to the database even though the rest of the updates wont.
Put in another way, these following two tests describe how the savepoints work:
152

def test_savepoint_rollbacks(self):

2
3

self.save_points(False)

4
5
6
7

#verify that everything was stored

users = User.objects.filter(email="inception")
self.assertEquals(len(users), 1)

8
9
10
11

#note the values here are from the original create call
self.assertEquals(users[0].stripe_id, '')
self.assertEquals(users[0].name, 'jj')

12
13
14
15

def test_savepoint_commit(self):
self.save_points(True)

16
17
18
19

#verify that everything was stored

users = User.objects.filter(email="inception")
self.assertEquals(len(users), 1)

20
21
22
23

#note the values here are from the update calls

self.assertEquals(users[0].stripe_id, '4')
self.assertEquals(users[0].name, 'staring down the rabbit hole')

After we commit or rollback a savepoint, we can continue to do work in the same transaction,
and that work will be unaffected by the outcome of the previous savepoint.
For example, if we update our save_points function as such:
1
2

@transaction.atomic()
def save_points(self,save=True):

3
4
5

user = User.create('jj','inception','jj','1234')
sp1 = transaction.savepoint()

6
7
8

user.name = 'staring down the rabbit hole'

user.save()

9
10
11

user.stripe_id = 4
user.save()

12

153

13
14
15
16

if save:
transaction.savepoint_commit(sp1)
else:
transaction.savepoint_rollback(sp1)

17
18
19

user.create('limbo','illbehere@forever','mind blown',
'1111')

Now regardless of whether savepoint_commit or savepoint_rollback was called, the

limbo user will still be created successfully, unless something else causes the entire transaction to be rolled-back.

154

Nested Transactions
In addition to manually specifying savepoints with savepoint(), savepoint_commit, and
savepoint_rollback, creating a nested Transaction will automatically create a savepoint
for us, and roll it back if we get an error.
Extending our example a bit further we get:
1
2

@transaction.atomic()
def save_points(self,save=True):

3
4
5

user = User.create('jj','inception','jj','1234')
sp1 = transaction.savepoint()

6
7
8

user.name = 'staring down the rabbit hole'

user.save()

9
10
11

user.stripe_id = 4
user.save()

12
13
14
15
16

if save:
transaction.savepoint_commit(sp1)
else:
transaction.savepoint_rollback(sp1)

17
18
19
20
21
22
23
24

try:
with transaction.atomic():
user.create('limbo','illbehere@forever','mind blown',
'1111')
if not save: raise DatabaseError
except DatabaseError:
pass

Here we can see that after we deal with our savepoints, we are using the transaction.atomic
context manager to encase our creation of the limbo user. When that context manager is
called, it is in effect creating a savepoint (because we are already in a transaction) and that
savepoint will be committed or rolled-back upon exiting the context manager.
Thus the following two tests describe their behavior here:
1

def test_savepoint_rollbacks(self):

155

self.save_points(False)

4
5
6
7

#verify that everything was stored

users = User.objects.filter(email="inception")
self.assertEquals(len(users), 1)

8
9
10
11

#savepoint was rolled back so we should have original values

self.assertEquals(users[0].stripe_id, '')
self.assertEquals(users[0].name, 'jj')

12
13
14
15

#this save point was rolled back because of DatabaseError

limbo = User.objects.filter(email="illbehere@forever")
self.assertEquals(len(limbo),0)

16
17
18
19

def test_savepoint_commit(self):
self.save_points(True)

20
21
22
23

#verify that everything was stored

users = User.objects.filter(email="inception")
self.assertEquals(len(users), 1)

24
25
26
27

#savepoint was committed

self.assertEquals(users[0].stripe_id, '4')
self.assertEquals(users[0].name, 'staring down the rabbit hole')

28
29

30
31

#save point was committed by exiting the context_manager

without an exception
limbo = User.objects.filter(email="illbehere@forever")
self.assertEquals(len(limbo),1)

So in reality you can use either atomic or savepoint to create savepoints inside a transaction, but with atomic you dont have to worry explicitly about the commit/rollback, where
as with savepoint you have full control over when that happens.

156

Completing the Front-end

Fire up the application, disconnect your Internet, and then try to register a user. What happens? You should still get a failure (or a very strange error), because we need to update the
form on the front-end for handling errors.
In static/application.js there is an existing function Stripe.createToken() that grabs the
form inputs, including the stripe token so we dont have to store credit card numbers in the
back-end:
1
2
3
4
5
6
7
8
9
10
11
12
13

Stripe.createToken(card, function(status, response) {

if (status === 200) {
console.log(status, response);
$("#credit-card-errors").hide();
$("#last_4_digits").val(response.card.last4);
$("#stripe_token").val(response.id);
form.submit();
} else {
$("#stripe-error-message").text(response.error.message);
$("#credit-card-errors").show();
$("#user_submit").attr("disabled", false);
}
});

Lets update this to allow for signup even if Stripe is down:

1
2
3
4
5
6
7
8
9
10

Stripe.createToken(card, function(status, response) {

if (status === 200) {
console.log(status, response);
$("#credit-card-errors").hide();
$("#last_4_digits").val(response.card.last4);
$("#stripe_token").val(response.id);
}
//always submit form even with errors
form.submit();
});

This will get then send the POST method to the payments.views.register() function,
which will in turn call our recently updated Customer.create() function. However now
we are likely to get a different error, in fact we will probably get two different errors:
Connection Errors - socket.error or stripe.error.APIConnectionError
157

 Invalid Request Errors - stripe.error.InvalidRequestError

Thus we can change the payments.views.create() function to catch each of the exceptions like:
1
2
3

except (socket.error, stripe.APIConnectionError,

stripe.InvalidRequestError):
return None

The updated function:

1

class Customer(object):

2
3
4
5
6
7
8
9
10
11
12

@classmethod
def create(cls, billing_method="subscription", **kwargs):
try:
if billing_method == "subscription":
return stripe.Customer.create(**kwargs)
elif billing_method == "one_time":
return stripe.Charge.create(**kwargs)
except (socket.error, stripe.APIConnectionError,
stripe.InvalidRequestError):
return None

Test it out now. It should work.

158

Conclusion
If you have had any previous experience with earlier versions of Django, you can see how
much simpler the transaction model is. Also having auto-commit on by default is a great
example of sane defaults that Django and Python both pride themselves in delivering. For
many systems you wont need to deal directly with transactions, just let auto-commit do its
work. But when you do, hopefully this chapter will have given you the information you need
to manage transactions in Django like a pro.
Further here is a quick list of reminders to help you remember the important stuff:

Important Transaction Concepts

AUTOCOMMIT

Functions at the database level; implicitly commit after each SQL statement..
1
2
3

START TRANSACTION
SELECT * FROM DEV_JOBS WHERE PRIMARY_SKILL = 'PYTHON'
END TRANSACTION

Atomic Decorator

Django >= 1.6 based transaction management has one main API which is atomic. Using

atomic wraps a code block in a db transaction.

1
2
3

with transaction.atomic():
user1.save()
unpaidUser1.save()

Transaction per http Request

This causes Django to automatically create a transaction to wrap each view function call. To
activate add ATOMIC_REQUEST to your database config in settings.py
1
2
3
4
5

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(SITE_ROOT, 'test.db'),
'ATOMIC_REQUEST': True,
159

6
7

Savepoints

Savepoints can be thought of as partial transactions. They allow you to save/rollback part of
a transaction instead of the entire transaction. For example:
1
2

@transaction.atomic()
def save_points(self,save=True):

3
4
5

user = User.create('jj','inception','jj','1234')
sp1 = transaction.savepoint()

6
7
8
9

user.name = 'staring down the rabbit hole'

user.stripe_id = 4
user.save()

10
11
12
13
14

if save:
transaction.savepoint_commit(sp1)
else:
transaction.savepoint_rollback(sp1)

160

Exercises.
1. For the final example of savepoints(), what would happen if you removed the try/except altogether?
For reference, the code were referring to is below (minus the try/except):
1
2

@transaction.atomic()
def save_points(self,save=True):

3
4
5

user = User.create('jj','inception','jj','1234')
sp1 = transaction.savepoint()

6
7
8

user.name = 'staring down the rabbit hole'

user.save()

9
10
11

user.stripe_id = 4
user.save()

12
13
14
15
16

if save:
transaction.savepoint_commit(sp1)
else:
transaction.savepoint_rollback(sp1)

17
18
19
20
21

with transaction.atomic():
user.create('limbo','illbehere@forever','mind blown',
'1111')
if not save: raise DatabaseError

Verify your expectation with a test or two. Can you explain why that happened? Perhaps these lines from the Django Documentation may help you understand it more
clearly.
Under the hood, Djangos transaction management code:

opens a transaction when entering the outermost atomic block;

creates a savepoint when entering an inner atomic block;
releases or rolls back to the savepoint when exiting an inner block;
commits or rolls back the transaction when exiting the outermost block.

2. Build your own transaction management system. Just joking. Theres no need to reinvent the wheel. Instead, you could read through the complete Django documentation
161

on the new transaction management features here if you really, really wanted to.

162

Chapter 8
Building a Membership Site
Up until now we have covered the nitty gritty of unit testing; how to do TDD; git branching
for teams; upgrading to Django 1.8, Python 3 and PostgreSQL as well as the latest in Django
Database Transactions. While these are all necessary tools and techniques that modern day
web developers should have in their tool belt, you could say we havent built much yet.

163

Its time to make something cool

Thats all about to change. Modern software development is roughly split between tools and
techniques, front and back-end development. So we have covered quite a bit in terms of tools
and techniques and some back-end development strategies as well. At this point its time
to switch gears a bit and focus on front-end development. While many companies still have
separate front and back-end developers, the practice of full-stack development is becoming
more and more the norm.
Full-stack developers can develop complete features from the back to the front, from database
design to Javascript animation. About the only thing you wouldnt include in the repertoire
of a full-stack developer is design.
The ability to develop both in a back-end language such as Python as well as THE front-end
language of Javascript (vanilla JS, jQuery, AJAX, and even a front-end framework) is extremely beneficial. Even if you end up working for a team where you are solely a back-end
developer, having a solid understanding of front-end concerns can help you develop a better
API that makes your front-end counterparts more productive and efficient.
In order to better illustrate the need to be a full-stack developer we are going to take our
Django app to the next level and build a Membership Site. We will still focus on a MVP
(Minimum Viable Product) but we will discuss how to create the complete feature set. That
is to say, the Python/Django back-end, the Javascript front-end and even just enough CSS to
get by.
Before we jump into building the features, though, its helpful to take a step back and document what features we want.
Were not going to go into high vs low fidelity prototyping or the benefits of developing agile
and utilizing user stories, because there are plenty of excellent books devoted to that. In fact,
check out the previous Real Python course, Web Development with Python, to learn more
about the how and why you should utilize user stories in the development process, specifically
in the Flask: Behavior-Driven Development with Behave chapter.
Well keep it practical. Lets look at a few of the user stories that were going to implement in
the coming chapters to paint a picture of where we are headed.

164

A Membership Site
Lets look at what we have before diving into what were going to add. Currently we have the
following implemented:

Static homepage
User login/logout (session management)
User Registration
Stripe integration supporting one-time and subscription payments
A Contact Us Page
About Us Page (using Django flatpages)

Not too shabby, but also not something thats going to make the front page of Hacker News
either. Lets see what we can do to flesh this out to into something a bit more interesting.
Lets start out with an overview of what our MVP is and why we are creating it. To make it
more interesting lets pretend we are building a membership site for Star Wars lovers. Lets
call our wonderful little membership site Mos Eisleys Cantina or just MEC for short.

Product Vision
Mos Eisleys Cantina (MEC) aims to be the premiere online membership site for Star Wars
Enthusiasts. Being a paid site, MEC will attract only the most loyal Star Wars fans and will
encourage highly entertaining debate and discussion about the entirety of the Star Wars Universe. A unique polling system will allow us to once and for all decide on important questions
such as who is truly the most powerful Jedi, are the separatists good or evil, and seriously
what is the deal with Jar Jar Binks? In addition, MEC will provide news and videos of current Star Wars-related happenings and in general be the best place to discuss Star Wars in
the entire galaxy.

Thats the vision; were building a real geek site. It probably wont make a dime, but thats
not the point. The techniques we are using here will be applicable to any membership site.
Now that we know what we are building, lets list out a few user stories so we have something
to focus on.

165

The User Stories

US1: Main Page
Upon arriving at the site the youngling (or unregistered user) will be greeted with a beautiful overview page describing the great features of MEC with a large sign up button rightsmack-dab in the middle of the page. Toward the bottom of the page there should be about
and contact us links so youglings can find out more information about the site. Finally there
should be a login button to allow returning padwans (or registered users) to log into the
site.

US2: Registration
After clicking the sign up button the applicant padwan (user wishing to sign up but not yet
verified by credit card) will be presented with a screen to collect basic registration information
including credit card information. Credit card information should be immediately processed
through Stripe, after which the applicant padwan should be upgraded to padwan status
and redirected to the Members Home Page.

US3: Members Home Page

The main page where returning padwans are greeted. This members page is a place for
announcements and to list current happenings. It should be the single place a user needs to
go to know all of the latest news at MEC.

US4: User Polls

Determining the truth of the galaxy and balancing the force are both very important goals
at MEC. As such, MEC should provide the functionality to poll padwans and correlate the
results in order to best determine or predict the answers to important topics of the Star Wars
galaxy. This includes questions like Kit Fisto vs Aayla Secura, who has the best dreadlocks? Or
who would win in a fight, C3PO or R2-D2? Results should also be displayed to the padwans
so that all shall know the truth.

US5: Galactic Map

A map displaying the location of each of the registered padwans. Useful for physical meetups, for the purpose of real life re-enactments of course. By Galactic here we mean global.

166

This map should provide a graphic view of who is where and allow for zooming in on certain
locations.

We could go on for days, but thats enough to fill a course. In the coming chapters we are
going to look at each of these user stories and try to implement them. US2 (Registration) is
pretty much done, but the others all need to be implemented. Weve arranged them in order
of increasing difficulty so we can build on the knowledge learned by implementing each of
the user stories.
Without further ado the next chapter will cover US1. See you on the next page.

167

Chapter 9
Bootstrap 3 and Best Effort Design
As we mentioned in the last chapter, traditional web design is usually not expected of most
full-stack developers. If you can design, then more power to you, but for the rest of us artistically challenged developers there is Bootstrap. This fantastic library, originally released by
the amazing design team at Twitter, makes it so easy to style a website that even somebody
with absolutely zero artistic ability can make a site look presentable. Granted, youre site is
probably not going to win any awards for design excellence, but it wont be embarrassing
either. With that in mind, lets look at what we can do to make our site look a little bit better.

168

Start with the User Story

Last chapter we laid out several User Stories that we will be working through in the coming
chapters to develop MEC, our awesome Star Wars membership site. To refresh your memory,
heres the first user story that we will work through in this chapter:

US1: Main Page

Upon arriving at the site the youngling (or unregistered user) will be greeted with a beautiful overview page describing the great features of MEC with a large sign up button rightsmack-dab in the middle of the page. Toward the bottom of the page there should be about
and contact us links so youglings can find out more information about the site. Finally there
should be a login button to allow returning padwans (or registered users) to log into the
site.

In other words, we need something that looks cool and gets the youngling to click on the
signup button (convert).
If we had to do that in straight HTML and CSS, it might take ages. But with Bootstrap we
can do it pretty quickly. The plan then is to create a main page with a nice logo, a couple of
pictures and descriptions of what the site is about and a nice big signup button to grab the
users (err younglings) attention.
Lets get started.

169

Overview of what we have

Although we havent talked about it much in this course, we are using the simple application
that you created in the second course, Web Development with Python. You created the application with Bootstrap version 2. Currently Bootstrap is on version 3 (3.2.0 as of writing) and
quite a bit has changed between 2 and 3. The most significant change is the Mobile First
ideology of Bootstrap 3. Basically responsive design is enabled from the start and the idea is
that everything should work on a mobile device first and foremost. This is a reflection of the
current and growing popularity of mobile.
While we could upgrade to Bootstrap 3, however since we are going to significantly redesign
the site, lets start over (more or less) with the front end. If you have a large site with a lot of
pages, its not recommended to go this route, but since we only have a few pages, we can get
away with it.

170

Installing Bootstrap 3
The first thing to do is grab Bootstrap file. So lets download it locally and chuck it in our static
directory. Before that, go ahead and remove all files from the django_ecommerce/static
directory except for application.js.
Keep in mind that we could serve the two main Bootstrap files, bootstrap.min.css
and bootstrap.min.js from a Content Delivery Network (CDN). When you are
developing locally, its best to download the files on your file system, though,
in case youre trying to develop/design and you lose Internet access. There are
various benefits to utilizing a CDN when you deploy your app to a production
server, which we will detail in a later chapter.
1. Download the Bootstrap distribution files from here.
2. Unzip the files and add the css, fonts, and js directories to the django_ecommerce/static
file directory. Add the application.js file to the js directory. When its all said and
done, your directory structure should look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

.
css
bootstrap-theme.css
bootstrap-theme.css.map
bootstrap-theme.min.css
bootstrap.css
bootstrap.css.map
bootstrap.min.css
fonts
glyphicons-halflings-regular.eot
glyphicons-halflings-regular.svg
glyphicons-halflings-regular.ttf
glyphicons-halflings-regular.woff
js
application.js
bootstrap.js
bootstrap.min.js

3. Another critical file that we must not forget to add is jQuery. As of this writing 1.11.1
is the most recent version. Download the minified version - e.g., jquery-1.11.1.min.js from the jQuery download page site, and add it to the js folder.
171

NOTE: On a side note, do keep in mind that if you have a real site thats
currently using a version of jQuery earlier than 1.9, and youd like to upgrade
to the latest version, you should take the time to look through the release
notes and also look at the jQuery Migrate plugin. In our case we are actually
upgrading from version 1.6x; however, since our example site only makes
very little use of jQuery (at the moment), its not much of an issue. But in
a production environment ALWAYS make sure you approach the upgrade
carefully and test thoroughly.

4. Say goodbye to the current design. Take one last look at it if youd like, because we are
going to wipe it out.

Figure 9.1: Initial look

5. With Bootstrap in the static folder, lets start with the basic Bootstrap template found
172

on the same page that you downloaded Bootstrap from. Take that template and overwrite your base.html file:
1
2
3
4
5
6

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width,
initial-scale=1">
<title>Bootstrap 101 Template</title>

8
9
10

<!-- Bootstrap -->

<link href="css/bootstrap.min.css" rel="stylesheet">

11
12

13

14
15

16

17
18
19

<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements

and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via
file:// -->
<!--[if lt IE 9]>
<script
src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></scri
<script
src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>

20
21

<h1>Hello, world!</h1>

22
23

24
25
26
27
28
29
30

<script src="https://js.stripe.com/v2/"
type="text/javascript"></script>
<script type="text/javascript">
//<![CDATA[
Stripe.publishableKey = '{{ publishable }}';
//]]>
</script>
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<script
src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js
173

31

32
33
34
35

<!-- Include all compiled plugins (below), or include

individual files as needed -->
<script src="js/bootstrap.min.js"></script>
<script src="js/application.js"></script>
</body>
</html>
SEE ALSO: While youre on the Bootstrap Getting Started page, check
out some of the other examples. These are some basic starter templates
that provide the basic Bootstrap components, allowing you to get started
faster.

6. Finally, from the django_ecommerce directory run ./manage.py runserver and

navigate to the main URL shown - [http://localhost:8000/](http://localhost:8000/.
You should see Hello, world!
7. Some of the paths in the above template are not necessarily correct, so lets fix that,
then we will put our Star Wars theme on the main page!
The first thing to do is use the Django {% load static %} template tag, which will
allow us to reference static files by the STATIC_DIR entry in our settings.py file. After
adding {% load static %} as the first line in our base.html file, we can change the
src lines that load the CSS and JavaScript files to:
1
2
3
4
5

<link href= "{%

...snip...
<script src="{%
<script src="{%
<script src="{%

static "css/bootstrap.min.css" %}" rel="stylesheet">

static "js/jquery-1.11.1.min.js" %}"></script>
static "js/bootstrap.min.js" %}"></script>
static "js/application.js" %}"></script>

If you refresh your page, you will see that there is a change in the font, indicating the
the CSS file is now loading correctly.
Installation complete. Your final base.html file should look like this:
1

{% load static %}

2
3
4
5
6
7

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
174

<meta name="viewport" content="width=device-width,

initial-scale=1">
<title>Bootstrap 101 Template</title>

10
11
12

<!-- Bootstrap -->

<link href= "{% static "css/bootstrap.min.css" %}"
rel="stylesheet">

13
14

15

16
17

18

19
20
21

<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements

and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via
file:// -->
<!--[if lt IE 9]>
<script
src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script
src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>

22
23

<h1>Hello, world!</h1>

24
25
26

27
28
29
30
31

<script src="https://js.stripe.com/v2/"
type="text/javascript"></script>
<script type="text/javascript">
//<![CDATA[
Stripe.publishableKey = '{{ publishable }}';
//]]>
</script>

32
33
34
35

36
37
38
39

<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->

<script src="{% static "js/jquery-1.11.1.min.js" %}"></script>
<!-- Include all compiled plugins (below), or include
individual files as needed -->
<script src="{% static "js/bootstrap.min.js" %}"></script>
<script src="{% static "js/application.js" %}"></script>
</body>
</html>
175

Making Bootstrap your own

With the installation out of the way, our base.html now has the basic setup we need (assuming
all went well with the Bootstrap installation, of course). Now we can start customizing.

Page Title
We probably want to change the title since the current title, Bootstrap 101 Template, has
little to do with Star Wars. You can customize this how you want, but well be using Mos
Eisleys Cantina our example.
1

<title>Mos Eisley's Cantina</title>

Custom Fonts
Lets also use a custom Star Wars font called Star Jedi.
Custom fonts can be a bit tricky on the web because there are so many different formats
you need to support because of cross browser compatibility issues. Basically, you need four
different formats of the same font. If you have a TrueType font you can convert it into the
four different fonts that you will need with an online font conversion tool. Then youll have
to create the correct CSS entries.
You can do this conversion on your own if youd like to practice. Simply download the font
from the above URL and then convert it. Or you can grab the fonts already converted in the
chp08 folder on the exercise repo.
1. Add a new file called mec.css in the static/css directory. Add the follow styles to start
with:
1
2
3
4
5
6
7
8
9
10

@font-face {
font-family: 'starjedi';
/*IE9 */
src: url('../fonts/Starjedi.eot');
/* Chrome, FF, Opera */
src: url('../fonts/Starjedi.woff') format('woff'),
/* android, safari, iOS */
url('../fonts/Starjedi.ttf') format('truetype'),
/* legacy safari */
url('../fonts/Starjedi.svg') format('svg');
176

font-weight: normal;
font-style: normal;

11
12
13

14
15
16
17
18

body {
padding-bottom: 40px;
background-color: #eee;
}

19
20
21
22

h1 {
font-family: 'starjedi', sans-serif;
}

23
24
25
26

.center-text {
text-align: center;
}

27
28
29
30

.featurette-divider {
margin: 80px 0;
}

The main thing going on in the CSS file above is the @font-face directive. This defines
a font-family called starjedi (you can call it whatever you want) and specifies the
various font files to use based upon the format requested from the browser.
Its important to note that the path to the font is the relative path from the location of
the CSS file. Since this is a CSS file, we dont have our Django template tag like static
available to us.
2. Make sure to add the new mec.css to the base.html template:
1
2
3
4

<!-- Bootstrap -->

<link href= "{% static "css/bootstrap.min.css" %}" rel="stylesheet">
<!-- custom styles -->
<link href= "{% static "css/mec.css" %}" rel="stylesheet">

3. Test! Fire up the server. Check out the changes. Our Hello, world! is now using the
cool startjedi font so our site can look a bit more authentic.

Layout
Now lets work on a nice layout. This is where we are going:
177

Figure 9.2: main page

178

Not too shabby for somebody with zero design ability.

Ok, there is a lot going on in the page, so lets break it down a piece at a time (sound familiar?)
and look at the implementation for each.
Navbar

Navbars are pretty common these days, and they are a great way to provide quick links for
navigation. We already had a navbar on our existing template, but lets put a slightly different
one in for the sake of another example. The basic structure of the navbar is the same in
Bootstrap 3 as it is in Bootstrap 2, but with a few different classes. The structure should look
like this:
1
2

<!-- NAVBAR ==========================================

<div class="navbar-wrapper">

-->

3
4

5
6
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<div class="navbar navbar-inverse navbar-static-top"

role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle"
data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="#">Mos Eisley's Cantina</a>
</div>
<div class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="active"><a href="{% url 'home' %}">Home</a></li>
<li><a href="/pages/about">About</a></li>
<li><a href="{% url 'contact' %}">Contact</a></li>
{% if user %}
<li><a href="{% url 'sign_out' %}">Logout</a></li>
{% else %}
<li><a href="{% url 'sign_in' %}">Login</a></li>
<li><a href="{% url 'register' %}">Register</a></li>
{% endif %}
</ul>
179

27
28
29

</div> <!-- end navbar links -->

</div> <!-- end container -->
</div> <!-- end navbar -->

30
31

</div><!-- end navbar-wrapper -->

Add this to your base.html template just below the opening body tag.
Comparing the above to our old navbar, you can see that we still have the same set of list
items. However, with the new navbar we have a navbar header and some extra divs wrapping
it. Most important are the <div class="container"> (aka container divs) because they
are vital for the responsive design in Bootstrap 3.
Since Bootstrap 3 is based on the mobile-first philosophy, the site is responsive from the
start. For example, if you re-size your browser to make the window smaller, you will see the
navbar links will disappear and a drop down button will be shown instead of all the links
(although we do have to insert the correct Javascript for this to happen, which we havent
done yet). Clicking that drop-down button will show all the links. An image of this is shown
below (ignore the colors for now).

Figure 9.3: responsive dropdown

If you want to test out the mobile-first-ness of Bootstrap on your mobile phone, you can tell
180

Django to bind to your network adapter when you start it so that the development server will
accept connections from other machines. To do this, just type the following:
1

$ ./manage.py runserver 0.0.0.0:8000

Then from your mobile phone, put in the IP address of your computer (and port 8000), and
you can see the site on your mobile phone or tablet. On unix machine or Macs you can do an
ifconfig from terminal to get your IP address. On windows machines its ipconfig from the
command prompt. We tested it on 192.168.1.12:8000.
Theres always certain sense of satisfaction that comes out of seeing your websites on your
own phone. Right?
There are also simpler ways to test out the responsiveness of bootstrap. The simplest being
simply resizing your browser and what bootstrap automatically adjust. Also you can use an
online tool like viewpoint-resizer.

Colors
To make the navbar use our cool starjedi font, update mec.css by replacing the current h1
style with:
1
2
3

h1, .nav li, .navbar-brand {

font-family: 'starjedi', sans-serif;
}

This just applies our font selection to the links in the navbar as well as the h1 tag. But as
is often the case with web development, doing this will make the navbar look a bit off when
viewed from the iPhone in portrait mode. To fix that, we have to adjust the sizing of the
.navbar-brand to accommodate the larger size of the startjedi font. This can be done by
adding the following to mec.css:
1
2
3

.navbar-brand {
height: 40px;
}

Now your navbar should look great on pretty much any device. With the navbar more or less
taken care of, lets add a footer as well, just below - <h1>Hello, world!</h1>:
1

<hr class="featurette-divider">

2
3
4

<footer>
<p class="pull-right"><a href="/#">Back to top</a></p>
181

<p class="pull-left">&copy; 2014 <a

href="http://realpython.com">The Jedi Council</a></p> <p
class="text-center"> &middot; Powered by about 37 AT-AT
Walkers, Python 3 and Django 1.8 &middot; </p>
</footer>

The actual HTML is more or less the same, right? But you may be saying to yourself, What
is the <footer> tag? Why shouldnt we just use a <div>?
Well Im glad you asked, because that brings us to our next topic.

182

HTML5 Sections and the Semantic Web

Have you heard the term Semantic Web before? Its an idea that has been around for a
long time, but hasnt really become commonplace yet. One of the goals behind HTML5 is to
change that. Before we get into the HTML5 part though, lets define Semantic Web straight
from the Wikipedia page:
The Semantic Web is a collaborative movement led by international standards
body the World Wide Web Consortium (W3C). The standard promotes common
data formats on the World Wide Web. By encouraging the inclusion of semantic
content in web pages, the Semantic Web aims at converting the current web,
dominated by unstructured and semi-structured documents into a web of data.
The Semantic Web stack builds on the W3Cs Resource Description Framework
(RDF).
According to the W3C, The Semantic Web provides a common framework that
allows data to be shared and reused across application, enterprise, and community boundaries. The term was coined by Tim Berners-Lee for a web of data that
can be processed by machines.
So in a nutshell, the Semantic Web is an idea, started by the creator of the web, that aims
to make DATA on the web more accessible, specifically more accessible programmatically.
There are a number of techniques used to achieve this such as Microformats, Resource Description Frameworks, Web Ontology Languages, and others, that when combined together
can make the web much more accessible programmatically. Its a huge topic, but well just
focus on one small aspect: the new section tags in HTML5.
To preface the usefulness of section tags in HTML5, lets revisit our navigation bar:
1
2

<!-- NAVBAR ==========================================

<div class="navbar-wrapper">

-->

3
4

5
6
7

8
9

<div class="navbar navbar-inverse navbar-static-top"

role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle"
data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
183

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="#">Mos Eisley's Cantina</a>
</div>
<div class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="active"><a href="{% url 'home' %}">Home</a></li>
<li><a href="/pages/about">About</a></li>
<li><a href="{% url 'contact' %}">Contact</a></li>
{% if user %}
<li><a href="{% url 'sign_out' %}">Logout</a></li>
{% else %}
<li><a href="{% url 'sign_in' %}">Login</a></li>
<li><a href="{% url 'register' %}">Register</a></li>
{% endif %}
</ul>
</div> <!-- end navbar links -->
</div> <!-- end container -->
</div> <!-- end navbar -->

30
31

</div><!-- end navbar-wrapper -->

Now imagine for a second that you are a computer program trying to determine the meaning
of that section of HTML. Its not very hard to tell what is going on there, right? You have a
few clues like the classes that are used, but they are not standard across the web, and the best
you could do is guess.
NOTE: The astute reader may point out the role attribute of the third div from
the top, which is actually part of the HTML5 semantic web specification. Its used
to provide a means for assistive technology to better understand the interface. So
its really just a high-level overview. What were after is more of a granular look
at the HTML itself.

The problem is that HTML is really a language to describe structure; it tells us how this data
should look, not what this data is. HTML5 section tags are the first baby steps used to start
to make data more accessible. An HTML5 section tag can be used in place of a div to provide
some meaning as to what type of data is contained in the div.

184

Inheritance
Before moving on, lets update the parent template, base.html, to add template internee.
Remove:
1

<h1>Hello, world!</h1>

Then update the body like so:

1

<div class="container">

2
3
4

{% block content %}
{% endblock %}

5
6
7

<hr class="featurette-divider">

8
9
10
11

12

<footer>
<p class="pull-right"><a href="/#">Back to top</a></p>
<p class="pull-left">&copy; 2014 <a
href="http://realpython.com">The Jedi Council</a></p> <p
class="text-center"> &middot; Powered by about 37 AT-AT
Walkers, Python 3 and Django 1.8 &middot; </p>
</footer>

13
14

</div>

HTML5 Section Tags

HTML5 defines the following section (or section-like) tags:
1. section Used for grouping together similar content. Its like a div with semantic
meaning. So all the data in the section tag should be related. An authors bio could
be sectioned off from the blog post in a section tag, for example.
2. article - Same as a section tag, but specifically for content that should be syndicated;
a blog post is probably the best example.
3. aside Used for somewhat related content that isnt necessarily critical to the rest of
the content. There are several places in this course where asides are used. For example,
there is an aside (NOTE) just a few paragraphs up that starts with The astute reader.
185

4. header Not to be confused with the head tag (which is the head of the document),
the header is the header of a section. It is common that headers have h1 tags. But
like with all the tags in this section, header describes the data, so its not the same as
h1 which describes the appearance. Also note that there can be several headers in an
HTML document, but only one per section tag.
5. nav Used mainly for site navigation, like our navbar above.
6. footer Despite the name, footer doesnt need to be at the bottom of the page.
Footer elements contain information about the containing section they are in. Oftentimes (like we did above) they contain copyright info, information about the author /
company. They can of course also contain footnotes. Also like headers, there can be
multiple footer tags per HTML document, but one per section tag.

Section Tags in action

So with our new understanding of some of the HTML section elements, lets rewrite our
navbar taking advantage of them:
1

2
3
4

5
6
7
8
9
10

11
12
13
14
15
16
17
18
19

<nav class="navbar navbar-inverse navbar-static-top"

role="navigation">
<div class="container">
<header class="navbar-header">
<button type="button" class="navbar-toggle"
data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="{% url 'home' %}">Mos Eisley's
Cantina</a>
</header>
<div class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="active"><a href="{% url 'home' %}">Home</a></li>
<li><a href="/pages/about">About</a></li>
<li><a href="{% url 'contact' %}">Contact</a></li>
{% if user %}
<li><a href="{% url 'sign_out' %}">Logout</a></li>
{% else %}
186

20
21
22
23
24
25
26

<li><a href="{% url 'sign_in' %}">Login</a></li>

<li><a href="{% url 'register' %}">Register</a></li>
{% endif %}
</ul>
</div>
</div>
</nav>

Notice the difference? Basically we replaced two of the divs with more meaningful section
elements.
1. The second line and the second to last line define the nav or navigation section, which
lets a program know: This is the main navigation section for the site.
2. Inside the nav element there is a header element, which encompasses the brand part
of the navbar, which is intended to show the name of the site with a link back to the
main page.
Those two changes may not look like much, but depending on the scenario, they can have a
huge impact. Lets look at a few examples real quick.
1. Web scraping: Pretend you are trying to scrape the website. Before, you would have
to follow every link on the page, make sure the destination is still in the same domain,
detect circular links, and finally jump through a whole bunch of loops and special cases
just to visit all the pages on the site. Now you can basically achieve the same thing in
just a few steps. Take a look at the following pseudo-code:
1

access site

2
3
4
5
6

for <a> in <nav>:

click <a>
check for interesting sections
extract data

7
8
9

#n ow return to main landing page

<nav>.<header>.<a>.click()

2. Specialized interfaces: Imagine tying to read a web page on something tiny like google
glass, or a smart watch. Lots of pinching and zooming and pretty tough to do. Now
imagine if that smart watch could determine the content and respond to voice commands like Show navigation, Show Main Content, Next Section. This could happen across all sites today if they used Semantic markup. But there is no way to make
that work if your site is just a bunch of unstructured divs.
187

3. Accessibility Devices: Several accessibility devices rely on semantic markup to make it

possible for those with accessibility needs to access the web. There are a number or
write ups about this on the web including here and here and here
But its not just about making your site more accessible in a programatic way. Its also about
making the code for your site clearer and thus easier to maintain.
As we have talked about before, writing maintainable code is a huge part of Software Craftsmanship. If you go back and look at our previous example with all the divs vs the semantic
markup you should find it much easier to see whats going on at a glance by just looking at
the section tags. This may not seem like much, but these tiny change add up quickly. Every
little bit counts.
Try this: Use your browser to View Page Source of your favorite web site. Chances are if
its not using semantic markup youll be drowning in divs, making it tough to dig through
the HTML and see what corresponds to what. Adding semantic markup make this process
much simpler and thus makes things more maintainable.
Thats basically it. In our case, we now have a navigation structure that not only looks cool
(hopefully), but is accessible programmatically - which is a really good thing. HTML5 section
tags are probably the simplest thing you can do to make your page more semantic. To get
started, simply replace div tags where appropriate with HTML5 sections tag that correspond
to the type of data youre showing.
We will cover some of the other new tags in HTML5 that apply to semantics as we go further
into the chapter. Covering the entirety of the Semantic Web is way beyond the scope of this
course, but if you want to find out more information, start at the Wikipedia page and watch
the TED talk by the creator of the web about it.

188

More Bootstrap Customizations

A Carousel of Star Wars Awesomeness
Coming back to our new site design, the next thing well do is add in some cool Star Wars
pictures that auto-rotate (like a slideshow) across the main page. This is done with the Bootstrap carousel control. Since we have everything we need at this point in the base.html file,
lets put the carousel in our index.html file.
Heres some example code:
1

{% extends 'base.html' %}

2
3

{% block content %}

4
5

{% load static %}

6
7
8

9
10
11

12
13
14
15
16
17

18
19
20

21

22
23
24

<center>
<section id="myCarousel" class="carousel slide"
data-ride="carousel" style="max-width: 960px;">
<!-- Indicators -->
<ol class="carousel-indicators">
<li data-target="#myCarousel" data-slide-to="0"
class="active"></li>
<li data-target="#myCarousel" data-slide-to="1"></li>
<li data-target="#myCarousel" data-slide-to="2"></li>
</ol>
<div class="carousel-inner">
<figure class="item active">
<img src="{% static 'img/darth.jpg' %}" alt="Join the
Dark Side" style="max-height: 540px;">
<figcaption class="carousel-caption">
<h1>Join the Dark Side</h1>
<p>Or the light side. Doesn't matter. If your into Star
Wars then this is the place for you.</p>
<p><a class="btn btn-lg btn-primary" href="{% url
'register' %}" role="button">Sign up today</a></p>
</figcaption>
</figure>
<figure class="item">
189

25

26
27
28

29

30
31
32
33

34
35
36

37

38
39
40
41

42

43
44

<img src="{% static 'img/star-wars-battle.jpg' %}"

alt="Wage War" style="max-height: 540px;">
<figcaption class="carousel-caption">
<h1>Wage War</h1>
<p>Daily chats, Weekly Gaming sessions, Monthly
real-life battle re-inactments.</p>
<p><a class="btn btn-lg btn-primary" href="#"
role="button">I'm ready to fight</a></p>
</figcaption>
</figure>
<figure class="item">
<img src="{% static 'img/star-wars-cast.jpg' %}"
alt="Meet People" style="max-height: 540px;">
<figcaption class="carousel-caption">
<h1>Meet fellow Star Wars Fans</h1>
<p>Join forces with fellow padwans and Jedi who lives
near you.</p>
<p><a class="btn btn-lg btn-primary" href="#"
role="button">Check the Members Map</a></p>
</figcaption>
</figure>
</div>
<a class="left carousel-control" href="#myCarousel"
data-slide="prev"><span class="glyphicon
glyphicon-chevron-left"></span></a>
<a class="right carousel-control" href="#myCarousel"
data-slide="next"><span class="glyphicon
glyphicon-chevron-right"></span></a>
</section>
</center>

45
46

{% endblock %}

Thats a fair amount of code there, so lets break it down into sections.
1

2
3
4

<section id="myCarousel" class="carousel slide"

data-ride="carousel">
<!-- Indicators -->
<ol class="carousel-indicators">
<li data-target="#myCarousel" data-slide-to="0"
class="active"></li>
190

5
6
7

<li data-target="#myCarousel" data-slide-to="1"></li>

<li data-target="#myCarousel" data-slide-to="2"></li>
</ol>

1. The first line is the HTML5 section tag, which is just separating the carousel as a
separate section of the document. The attribute data-ride="carousel" starts the
carousel animation on page load.
2. The ordered list ol displays the three dots near the bottom of the carousel that indicate
which page of the carousel is being displayed. Clicking on the list will advance to the
associated image in the carousel.
The next section has three items that each correspond to an item in the carousel. They all
behave the same, so lets just describe the first one, and the same will apply to all three.
1
2
3
4
5

7
8

<figure class="item active">

<img src="{% static 'img/darth.jpg' %}" alt="Join the Dark Side">
<figcaption class="carousel-caption">
<h1>Join the Dark Side</h1>
<p>Or the light side. Doesn't matter. If you're into Star
Wars then this is the place for you.</p>
<p><a class="btn btn-lg btn-primary" href="{% url 'register'
%}" role="button">Sign up today</a></p>
</figcaption>
</figure>

1. figure is another HTML5 element that we havent talked about yet. Like the section
elements, it is intended to provide some semantic meaning to the page:
NOTE: The figure element represents a unit of content, optionally with a
caption, which is self-contained, that is typically referenced as a single unit
from the main flow of the document, and that can be moved away from the
main flow of the document without affecting the documents meaning.

2. Just like with the HTML5 section tags, we are replacing the overused div element with
an element figure that provides some meaning.
3. Also as a sub-element of the figure we have the <figcaption> element which represents a caption for the figure. In our case we put the join now message an a link to
the registration page in our caption.
191

The final part of the carousel are two links on the left and right of the carousel that look like
> and <. These advance to the next or previous slide in the carousel, and the code for these
links look like:
1

<a class="left carousel-control" href="#myCarousel"

data-slide="prev"><span class="glyphicon
glyphicon-chevron-left"></span></a>
<a class="right carousel-control" href="#myCarousel"
data-slide="next"><span class="glyphicon
glyphicon-chevron-right"></span></a>

Finally, create an img folder within the static folder, and then grab the star-warsbattle.jpg, star-wars-cast.jpg, and darth.jpg images from the chp08 folder in the exercise
repo and them to that newly created img folder.
And thats it. You now have a cool carousel telling visiting younglings about the benefits of
joining the site.

Some additional content

We can add some additional content below the carousel to talk about some of the benefits
and features of the membership site. Starting with the most straightforward way, we could
just add a bunch of extra HTML after the carousel like so:
1

<br><br>

2
3
4
5
6
7

8
9

10

11
12
13

<section class="container marketing">

<!-- Three columns of text below the carousel -->
<div class="row center-text">
<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/yoda.jpg' %}"
width="140" height="140" alt="Generic placeholder image">
<h2>Hone your Jedi Skills</h2>
<p>All members have access to our unique training and
achievements ladders. Progress through the levels and
show everyone who the top Jedi Master is! </p>
<p><a class="btn btn-default" href="#" role="button">Sign Up
Now &raquo;</a></p>
</div><!-- /.col-lg-4 -->
<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/clone_army.jpg'
%}" width=140 height=140 alt="Clone Army">
192

14
15

16

17
18
19

20
21

22

23
24
25

<h2>Build your Clan</h2>

<p>Engage in meaningful conversation, or bloodthirsty battle!
If it's related to Star Wars, in any way, you better
believe we do it here.</p>
<p><a class="btn btn-default" href="#" role="button">Sign Up
Now &raquo;</a></p>
</div><!-- /.col-lg-4 -->
<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/leia.jpg' %}"
width=140 height=140 alt="Leia">
<h2>Find Love</h2>
<p>Everybody knows Star Wars fans are the best mates for Star
Wars fans. Find your Princess Leia or Han Solo and
explore the stars together.</p>
<p><a class="btn btn-default" href="#" role="button">Sign Up
Now &raquo;</a></p>
</div><!-- /.col-lg-4 -->
</div> <!-- /.row -->
</section>

Be sure to grab the images - yoda.jpg, clone_army.jpg, leia.jpg - from the repo again.
This gives us three sections each with an image in a circular border (because circles are hot
these days right?), some text and a view details button. This is all pretty straight-forward
Bootstrap stuff; Bootstrap uses a grid system, which breaks up the page into columns and
rows.
By putting content into the same row (that is to say, all elements that are a child of the
same <div class="row center-text"> ), it will appear lined up side by side. And as
you might expect, a second row will appear underneath the previous row. Likewise with
columns, add content and a column will appear to the right of the previous column (i.e. <div
class="column">) or to the left of the subsequent column.
For our marketing info, we create one row div with three child divs <div class="col-lg-4">.
col-lg-4 is interesting. With Bootstrap each row has a total of 12 columns. You can
break up the page how you want as long as the sum of all the columns is 12. In other
words, <div class='col-6'> will take up half of the width of the page, whereas <div
class='col-4'> will take up a third of the width of the page. For a more in-depth
discussion on this, take a look at this blog post.
Did you notice the other identifier? lg. This stands for large. Along with lg, theres xs, sm,
and md for extra-small, small, and medium, respectively.

193

Bootstrap 3, being responsive by default, has the notion of extra-small, small, medium, and
large screen sizes. So by using the lg identifier, you are can saying, I want columns only if
the screen size is large,. Likewise, you can do the same with other sizes.
You can see this by looking at our Marketing Items using a large screen size (where it will
have three columns because we are using the col-lg-4 class:

Figure 9.4: large screen size

And if we view the page on a medium or small screen size it will show only one column:
The grid system is really powerful and completely responsive. With the above code, try resizing your browser to fill your entire monitor and you will see three columns. Make the width
of your browser smaller and you will see the columns stack on top of each other. Pretty cool,
huh?
The best place to get all the information about the Bootstrap grid system is directly from the
Bootstrap docs. They are really good and worth a read.
So we have some basic marketing material there, but we had to type a lot of boilerplate HTML
code. Plus we are not really taking advantage of Djangos templating system. Lets do that.

194

195

Figure 9.5: small screen size

Custom template tags

Built-in tags
Django provides you with a lot of built-in template tags like the {% static %} tag that can
make dealing with dynamic data easier. Several of them you are already familiar with:
{% static %} provides a way to reference the location of your static folder without
hardcoding it in.
{% extend 'base.html' %} allows for the extending of a parent template.
{% csrf %} provides protection against Cross Site Request Forgeries.
Django also provides a number of built in filters which make working with data in your templates easier. Some examples include:
{{ value | capfirst }} - if value = jack this will produce Jack
{{ value | add:"10" }} - if value is 2 this will produce 12
{{ value | default:"Not logged in"}} - if value evaluates to False, use the
given default - i.e., "Not logged in"
There are tons more built-in filters and template tags. Again, check out the Django documentation for more information.

Custom template tags

Cant find what your looking for in the docs? You can also create your own custom template
tags. This is great if you have certain content/html structures that youre repeating over and
over.
For example, this section of code1
2

3
4

<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/yoda.jpg' %}"
width="140" height="140" alt="Generic placeholder image">
<h2>Hone your Jedi Skills</h2>
<p>All members have access to our unique training and
achievements ladders. Progress through the levels and show
everyone who the top Jedi Master is! </p>
<p><a class="btn btn-default" href="#" role="button">View details
&raquo;</a></p>
</div><!-- /.col-lg-4 -->
196

-could be easily re-factored into a template tag so you wouldnt have to type so much. Lets
do it.
Directory structure

First, create a directory in your main app called templatetags and add the following files to
ti:
1
2
3
4
5
6
7

.
__init__.py
models.py
templatetags
__init__.py
marketing.py
views.py

Define the tag

In marketing.py add the following code to define the custom template tag:
1
2

from django import template

register = template.Library()

3
4
5
6

8
9
10
11
12
13
14

@register.inclusion_tag('circle_item.html')
def circle_header_item(img_name="yoda.jpg", heading="yoda",
caption="yoda",
button_link="register", button_title="View
details"):
return {
'img': img_name,
'heading': heading,
'caption': caption,
'button_link': button_link,
'button_title': button_title
}

The first two lines just register circle_header_item() as a template tag so you can use it
in a template using the following syntax:
1

{% circle_header_item img_name='img.jpg' heading='nice image' %}

197

Just as if you were calling a regular Python function, when calling the template tag you can
use keywords or positional arguments - but not both. Arguments not specified will take on
the default value.
The @register function simply creates a context for the template to use. It does this by
creating a dictionary of variable names mapped to values. Each of the variable names in the
dictionary will become available as template variables.
The remainder of the line1

@register.inclusion_tag('circle_item.html')

-declares an HTML fragment that is rendered by the template tag. Django will look for the
HTML file everywhere that is specified in the TEMPLATE_LOADERS list, which is specified in
the settings.py file. In our case, this is under the template subdirectory.
HTML Fragment

The circle_item.html file looks like this:

1

{% load staticfiles %}

2
3
4

5
6
7

<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/'|add:img %}"
width="140" height="140" alt="{{img}}">
<h2>{{ heading }}</h2>
<p>{{ caption }}</p>
<p><a class="btn btn-default" href="{% url button_link %}"
role="button">{{button_title}}</a></p>
</div>

Add this to the templates directory.

We took our original HTML to define the structure and then used several template variables
(the ones we returned from main.templatetags.marketing.circle_header_item),
which enables us to dynamically populate the structure with data. This way we can repeat the
structure several times without having to repeat the HTML. Everything is pretty standard
here, but there is one template tag/filter you may not be familiar with:
1

src="{% static 'img/'|add:img %}

Looking at the src attribute you can see it is using the standard {% static %} tag. However
there is a funny looking bit - 'img/'|add:img - which is used for concatenation.
You can use it for addition:
198

{{ num|add:'1' }}

If you passed 5 in for num, then this would render 6 in your HTML.

add can also concatenate two lists as well as strings (like in our case). Its a pretty handy filter
to have.
Adding the tag

In terms of our inclusion tag, we call it directly from index.html:

1
2
3
4

<section class="container marketing">

<!-- Three columns of text below the carousel -->
<div class="row center-text">
{% circle_header_item img_name='yoda.jpg' heading='Hone your
Jedi Skills'
caption='All members have access to our unique training and
achievements ladders.
Progress through the levels and show everyone who the top
Jedi Master is!' %}

7
8

10

{% circle_header_item img_name='clone_army.jp' heading='Build

your Clan'
caption='Engage in meaningful conversation, or bloodthirsty
battle! If it's
related to Star Wars, in any way, you better believe we do it.
:)' %}

11
12
13

14
15
16
17

{% circle_header_item img_name="leia.jpg" heading="Find Love"

caption="Everybody knows Star Wars fans are the best mates
for Star Wars
fans. Find your Princess Leia or Han Solo and explore the stars
together." button_title="Sign Up Now" %}
</div>
</section>
NOTE: Django template tags are not allowed to span multiple lines, so you actually have to string everything together in one super long line. We broke it apart
for readability in this course. Be sure to turn word-wrap on in your text editor
so you can view all of the text without having to scroll.

199

There you have it!

Just three calls to our new template tag and nowhere near as much HTML all over the place.
This is a bit of a cheat though since we hard coded the text directly in the HTML. Youd actually
want to pass it in through the view to make it truly dynamic. Well save this for an exercise at
the end of the chapter.

200

A Note on Structure
Pairing Bootstrap with Django templates can not only make design a breeze, but also greatly
reduce the amount of code you have to write.
If you do start to use a number of custom template tags and/or use multiple templates to
render a page, it can make it more difficult to debug issues within the template. The hardest
part of the process is often determining which file is responsible for the markup you see on
the screen. This becomes more prevalent the larger an application gets. In order to combat
this issue you should choose a meaningful structure to organize your templates and stick to it
religiously. It will make debugging and maintenance much easier.
Currently, in our code, we are just throwing everything in the templates directory. This
approach doesnt scale; it makes it very difficult to know which templates belong to which
views and how the templates relate to one other. Lets fix that.

Restructure
First, lets organize the templates by subfolder, where the name of each subfolder corresponds
with the app that is using the template. Reorganizing our template folder as such produces:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

.
base.html
contact
contact.html
flatpages
default.html
main
index.html
templatetags
circle_item.html
user.html
payments
cardform.html
edit.html
errors.html
field.html
register.html
sign_in.html

201

With this setup we can easily see which templates belong to which application. This should
make it easier to find the appropriate templates. We do have to change the way in which we
refer to templates in both our views and our templates by appending the app name before the
template when calling it so that Django can find the template correctly. This is just a minor
change and also adds readability.
For example, in a template you change this:
1

{% include "cardform.html" %}

{% include "payments/_cardform.html" %}

To:

The only thing we are doing differently here is including the folder where the template
is stored. And since the folder is the same name as the app that uses the template, we
also know where to find the corresponding views.py file, which in this case would be
payments/views.py.
In a views.py file its the same sort of update:
1
2
3
4
5
6
7
8

return render_to_response(
'payments/sign_in.html',
{
'form': form,
'user': user
},
context_instance=RequestContext(request)
)

Again, just like in the template, we add the name of the directory so that Django can locate
the template correctly.
Dont make these changes just yet.
This is a good first step to organizing our templates, but we can take it a bit further and
communicate the purpose of the template by adhering to a naming convention.

Naming Convention
Basically you have three types of templates:

1. Templates that were meant to be extended, like our base.html template.

2. Templates that are meant to be included, like our payments/cardform.html template.
202

3. Template tags and filters, including inclusion tags.

If we further extend our organization structure to differentiate between those three types
of templates, then we can quickly understand the purpose of the template with a simple ls
command (in Unix), without even having to open up the template. You can use any naming
convention you like as long as it works for you.
Case

Description

Example

1
2
3

start name with __

start name with _
put in templatetag sub directory

__base.html
payments/_cardform.html
@register.inclusion_tag(main/templatetags/circle_item.html)

Doing this will let us quickly identify what the purpose of each of our templates is. This will
make things easier, especially when your application grows and you start including a number
of different types of templates and referencing them from all over your application.
Go ahead and make the restructuring and renaming changes now. Once done, run the sever
to make sure you caught everything.
Your templates directory should now look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

.
__base.html
contact
contact.html
flatpages
default.html
main
index.html
templatetags
circle_item.html
user.html
payments
_cardform.html
_field.html
edit.html
errors.html
register.html
sign_in.html

203

As you can see, the basic structure stays the same, but now if you look at the payments folder,
for example, you can quickly tell that *_cardform.html* and *_field.html* are templates that
are meant to be included in other templates, while the three other templates represent a page
in the application. And we know all of that without even looking at their code.

Update template tag

The final thing we should change is the name of our template tag, which is currently
marketing. Remember when we load the tags to be used in a template, our load looks like
this:
1

{% load marketing %}

The first question you should have after reading that is, Well where are the marketing template tags, and whats in there?
Lets first rename the marketing.py to main_marketing.py so at a glance we can tell where
the template tags are located (within the templates/main/templatetags folder). Also in
the main_marketing.py file we have a tag called circle_header_item. While this may
describe what it is, it doesnt tell us where it came from. Larger Django projects could have
templates that include ten other templatetag libraries. In such a case, its pretty difficult to
tell which tags belong to which library. The solution is to name the tag after the library.
One convention is to use the taglibname__tagname. In this case circle_header_item
becomes marketing__circle_item. This way, if we find it in a template, we know it
comes from a library called marketing, and if we just jump to the top of the HTML file,
well see the {% load main_marketing %} tag and thus well know to look for the code:
main.templatetags.main_marketing.marketing__circle_item.
This may not seem like much but its a life saver when the application becomes large and/or
you are revisiting the app six months after you wrote it. So take the time and add a little
structure to your templates. Youll thank yourself later.
Make all the changes and run your tests. Youll probably pick up anything you missed just by
running the test suite since moving around templates will cause your tests to fail. Still having
problems? Check the Project in the chp09 folder in the exercise repo.

204

Conclusion
We have talked about several important topics in this chapter with regards to Bootstrap:
1. First, we went through the basics of Bootstrap 3, its grid system and some of the cool
tools it has for us to use.
2. Then, we talked about using custom fonts, putting in our own styling and imagery, and
making Bootstrap and Django templates play nicely together.
This all represents the core of what you need to know to successfully work with Bootstrap and
Django. That being said, Bootstrap is a giant library, and we barely scratched the surface.
Read more about Bootstrap on the official website. There are a number of things that you
can do with Bootstrap, and the more you use it, the better your Bootstrap skills will become.
With that in mind, the exercises will go through a number of examples designed to give you
more practice with Bootstrap.
Finally, if you need additional help with Bootstrap, check out this blog post, which touches
on a number of features and components of the framework, outside the context of Django which may be easier to understand.
We also talked about two important concepts that go hand and hand with Bootstrap and front
end work:
1. HTML5 Semantic Tags - these help make your web site more accessible programmatically, while also making your HTML easier to understand and maintain by fellow web
developers. The web is about data and making the data of your website more accessible
is generally a good thing.
2. Template Tags: We talked a lot about custom template tags and how they can reduce
your typing, and make your templates much easier to use. We will explore custom tags
further in the exercises with a look at how to data drive your website using custom
template tags.
A lot of ground was covered so be sure to go through the exercises to help ensure you understand everything that we covered in this chapter.

205

Exercises
1. Bootstrap is a front-end framework, and although we didnt touch much on it in this
chapter, it uses a number of CSS classes to insert things on the page, making it look
nice and provide the responsive nature of the page. It does this by providing a large
number of classes that can be attached to any HTML element to help with placement.
All of these capabilities are described on the Bootstraps CSS page. Have a look through
it, and then lets put some of those classes to use.
In the main carousel, the text, Join the Dark Side on the Darth Vader image,
blocks the image of Darth himself. Using the Bootstrap / carousel CSS, can you
move the text and sign up button to the left of the image so as to not cover Lord
Vader?
If we do the above change, everything looks fine until we view things on a phone
(or make our browser really small). Once we do that, the text covers up Darth
Vader completely. Can you make it so on small screens the text is in the normal
position (centered / lower portion of the image) and for larger images its on the
left.
2. In this chapter, we updated the Home Page but we havent done anything about the
Contact Page, the Login Page, or the Register Page. Bootstrapify them. Try to make
them look awesome. The Bootstrap examples page is a good place to go to get some
simple ideas to implement. Remember: try to make the pages semantic, reuse the
Django templates that you already wrote where possible, and most of all have fun.
3. Previously in the chapter we introduced the marketing__circle_item template tag.
The one issue we had with it was that it required a whole lot of data to be passed into
it. Lets see if we can fix that. Inclusion tags dont have to have data passed in. Instead, they can inherit context from the parent template. This is done by passing in
takes_context=True to the inclusion tag decorator like so:
1

@register.inclusion_tag('main/templatetags/circle_item.html',
takes_context=True)

If we did this for our marketing__circle_item tag, we wouldnt have to pass in all
that data; we could just read it from the context.
Go ahead and make that change, then you will need to update the main.views.index
function to add the appropriate data to the context when you call render_to_respnose.
Once that is all done, you can stop hard-coding all the data in the HTML template and
instead pass it to the template from the view function.

206

For bonus points, create a marketing_info model. Read all the necessary data from
the model in the index view function and pass it into the template.

207

Chapter 10
Building the Members Page
Now that we have finished getting the public facing aspects of our site looking nice and
purdy, its time to turn our attention to our paying customers and give them something fun
to use, that will keep them coming back for more (at least in theory).

208

User Story
As always, lets start with the user story we defined in Chapter 7

US3: Members Home Page

The main page where returning padwans are greeted. This members page is a place for
announcements and to list current happenings. It should be the single place a user needs to
go to know all of the latest news at MEC.

And that is exactly what we are going to do. To give you an idea of where we should end up
with by the end of this chapter, have a look at the screenshot below:

Figure 10.1: Members Page

209

Update Main Template

This is what a registered user will see after login. If you recall, the registered user page is
in templates/main/user.html. If we add three boxes to the user.html page - Report Back to
Base, Jedi Badge, and Recent Status Reports boxes - then the template should look like
this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

{% extends "__base.html" %}
{% load staticfiles %}
{% block content %}
<div class="row member-page">
<div class="col-sm-8">
<div class="row">
{% include "main/_statusupdate.html" %}
{% include "main/_lateststatus.html" %}
</div>
</div>
<div class="col-sm-4">
<div class="row">
{% include "main/_jedibadge.html" %}
</div>
</div>
</div>
{% endblock %}

Looks pretty simple, right? Here we added the basic Bootstrap scaffolding. Remember that
Bootstrap uses a grid system, and we can access that grid system by creating CSS classes that
use the row and col syntax. In this case, we have used special column classes col-sm-8 and
col-sm-4. Since the total columns available is 12, this tells Bootstrap that our first column
should be 2/3 of the screen (8 of 12 columns) and our second column should be 1/3 of the
screen width.
The sm part of the class denotes that these columns should appear on tablets and larger devices. On anything smaller than a tablet there will only be one column. You have four options
for column size with Bootstrap:
class name

width in pixels

device type

.col-xs.col-sm.col-md.col-lg-

< 768px
>= 768px
>= 992 px
>= 1200 px

Phones
Tablets
Desktops
Large Desktops

210

class name

width in pixels

device type

Keep in mind that choosing a column size will ensure that the column is available for that
size and all sizes larger (i.e., specifying .col-md will show the column for desktops and large
desktops but not phones or tablets).
After setting up the grid, we used three includes:
1
2
3

{% include "main/_statusupdate.html" %}
{% include "main/_lateststatus.html" %}
{% include "main/_jedibadge.html" %}

After the last chapter you should be familiar with includes; they just let us include a separate template in our current template. Each of these represents an info box - e.g, the Report
Back to Base box - which is a separate reusable piece of logic kept in a separate template file.
This ensures that your templates stay relatively small and readable, and it makes it easier to
understand things.

Lets set up these sub-templates now.

211

Template 1: Showing User Info for the Current User

The *main/_jedibadge.html* template displays information about the current logged in user:
1
2
3
4
5

6
7
8
9
10
11
12

13

<!-- The jedi badge info box, shows user info -->
{% load staticfiles %}
<section class="info-box" id="user_info">
<h1>Jedi Badge</h1>
<img class="img-circle" src="{% static 'img/yoda.jpg' %}"
width="140" height="140" alt="user.avatar"/>
<ul>
<li>Rank: {{user.rank}}</li>
<li>Name: {{user.name}}</li>
<li>Email: {{user.email}}</li>
<li><a id="show-achieve" href="#">Show Achievements</a></li>
</ul>
<p>Click <a href="{% url 'edit' %}">here</a> to make changes to
your credit card.</p>
</section>

We start with a section tag that wraps the whole section and gets its styling from a class called
info-box. Add the following CSS styles to mec.css:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

/* Member Page */
.info-box {
border: 2px solid #000000;
margin-bottom: 20px;
padding-left: 10px;
padding-right: 10px;
padding-bottom: 5px;
background-color: #eee;
}
#user_info {
text-align: center;
margin-left: 20px;
}
#user_info ul {
list-style-type: none;
text-align: left;
}
.member-page {
212

padding-top: 20px;
padding-bottom: 40px;
background-color: white;
margin-left: 0px;
margin-right: 0px;
margin-top: -20px;

19
20
21
22
23
24
25

Mainly we are just setting some spacing and background colors. Nothing too fancy here.
Coming back to the jedi badge info box, there are two things we have changed about the user:
1. The user now has an avatar (which for the time being we are hard-coding as the
yoda image) - <img class="img-circle" src="{% static 'img/yoda.jpg'

%}"width="140" height="140" alt="user.avatar"/>

2. Users now have a rank attribute. If you remember from our user stories we
talked about users having a rank of either a youngling or padwan - <li>Rank:

{{user.rank}}</li>

Add Rank to Models

Adding the rank attribute to out models is simple: Its just another column in payments.models.User
object:
1

rank = models.CharField(max_length=50, default="Padwan")

We default the value to Padwan because all registered users start with that value, and unregistered users, who technically have a rank of youngling, wont see the rank displayed
anywhere on the system.
Since we are making changes to the User model we need to re-sync our database to add the
new rank column to the payments_user table. However, if you run ./manage.py syncdb,
Django will essentially do nothing because it cannot add a column to a table that has users
in it. It is worth nothing that in Django versions prior to 1.7 you would see an error populate.
To resolve this, you need to first remove all the users and then the syncdb can continue error
free.
For right now that is okay: We can delete the users. But if you are updating a system where
you need to keep the data (say, a system that is in production), this is not a feasible option.
Thankfully, in Django 1.7 the concept of migrations has been introduced. Migrations actually come from a popular Django framework called South, which allows you to update a

213

table/schema without losing data. An upcoming chapter will cover migrations in more detail.
For now we will just drop the database, and then re-sync the tables.
Before you do that, lets talk quickly about users. Since we are developing the membership
page (which requires a login), we will need registered users so we can log into the system.
Having to delete all the users anytime we make a change to the User model can be a bit
annoying, but we can get around this issue by using fixtures.

Fixtures
We talked about fixtures in the Software Craftsmanship chapter as a means of loading data
for unit testing. Well, you can also use fixtures to load initial data into a database. This can
make things harder to debug, but in this case since we are doing a lot of work on how the
system responds to registered users, we can save ourselves a lot of time by pre-registering
users.
Since users are in the payments application, we should put our fixture there. Lets create a
fixture that runs each time ./manage.py syncdb is run (which also runs every time our
unit test are run):
1. Create a directory called fixtures in the payments directory.
2. Make sure you have some registered users in your database. Manually add
some if necessary.
Then run ./manage.py dumpdata payments.User >
payments/fixtures/initial_data.json.

manage.py dumpdata spits out JSON for all the data in the database, or in the case above for
a particular table. Then we just redirect that output to the file payments/fixtures/initial_data.json.
Now when you run ./manage.py syncdb, it will search all of the applications registered
in settings.py for a fixtures/initial_data.json file and load that data into the database that is
stored in that file.
Here is an example of what the data might look like, formatted to make it more humanreadable:
1
2
3
4
5
6
7

[
{
"pk": 1,
"fields": {
"last_login": "2014-03-11T08:58:20.136",
"rank": "Padwan",
"name": "jj",
214

"password":
"pbkdf2_sha256$12000$c8TnAstAXuo4$agxS589FflHZf+C14EHpzr5+EzFtS1V1t
"email": "[email protected]",
"stripe_id": "cus_3e8fBA8rIUEg5X",
"last_4_digits": "4242",
"updated_at": "2014-03-11T08:58:20.239",
"created_at": "2014-03-11T08:58:20.235"

9
10
11
12
13

},
"model": "payments.user"

14
15
16
17

},
{

"pk": 2,
"fields": {
"last_login": "2014-03-11T08:59:19.464",
"rank": "Jedi Knight",
"name": "kk",
"password":
"pbkdf2_sha256$12000$bEnyOYJkIYWS$jqwLJ4iijmVgPHu9na/Jncli5nJnxbl47
"email": "[email protected]",
"stripe_id": "cus_3e8gyBJlWAu8u6",
"last_4_digits": "4242",
"updated_at": "2014-03-11T08:59:19.579",
"created_at": "2014-03-11T08:59:19.577"
},
"model": "payments.user"

18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44

},
{

"pk": 3,
"fields": {
"last_login": "2014-03-11T09:12:09.802",
"rank": "Jedi Master",
"name": "ll",
"password":
"pbkdf2_sha256$12000$QE2hn0nj0IWm$Ea+IoZMzv6KYV2ycpe+g7afFWi2wPSSya
"email": "[email protected]",
"stripe_id": "cus_3e8tB7EaspoOiJ",
"last_4_digits": "4242",
"updated_at": "2014-03-11T09:12:10.033",
"created_at": "2014-03-11T09:12:10.029"
},
215

"model": "payments.user"

45

46
47

With that, you wont have to worry about re-registering users every time you run a unit test
or you resync your database. But if you do use the above data exactly, it will break your unit
tests, because our unit tests assume there is no data in the database. In particular, the test
test_get_by_id in tests.payments.testUserModel.UserModelTest should now be
failing (among others). Lets fix it really quick:
1
2

def test_get_by_id(self):
self.assertEqual(User.get_by_id(self.test_user.id),
self.test_user)

Before we hard-coded the id to 1, which is okay if you know what the state of the database is
but its still hard-coding, and it has come back to bite us in the rear. Never again! Now we
just use the id of the test_user (that we created in the setUpClass method), so it doesnt
matter how much data we have in the database; this test should continue to pass, time after
time.

Update Database
With the fixtures setup, lets update the database.
1. First drop the database from the Postgres Shell:
1

DROP DATABASE django_db;

2. Then recreate it:

1

CREATE DATABASE django_db;

3. Finally, run syncdb:

1

$ ./manage.py syncdb

Make sure to run your tests. Right now you should have three errors since we have not created
the *main/_statusupdate.html* template yet.

216

Gravatar Support
Most users are going to want to be able to pick their own avatar as opposed to everybody
being Yoda. We could give the user a way to upload an image and store a reference to it in the
user table, then just look up the image and display it in the jedi badge info box. But somebody
has already done that for us
Gravatar or Globally Recognized Avatars, is a site that stores an avatar for a user based upon
their email address and provides APIs for all of us developers to access that Avatar so we can
display it on our site. This is a nice way to go because it keeps you from having to reinvent the
wheel and it keeps the user from having to upload yet another image to yet another service.
To use it, lets create a custom tag that will do the work of looking up the gravatar for us.
Once the tag is created it will be trivial to insert the gravatar into our *main/_jedibadge.html*
template.
Create main/templatetags/main_gravatar.py and fill it with the following code:
1
2
3

from django import template

from urllib.parse import urlencode
import hashlib

4
5

register = template.Library()

6
7
8
9
10
11

12

@register.simple_tag
def gravatar_img(email, size=140):
url = get_url(email, size)
return '''<img class="img-circle" src="%s" height="%s"
width="%s"
alt="user.avatar" />''' % (url, size, size)

13
14
15
16
17

def get_url(email, size=140):

default = ('http://upload.wikimedia.org/wikipedia/en/9/9b/'
'Yoda_Empire_Strikes_Back.png')

18
19
20

query_params = urlencode([('s', str(size)),

('d', default)])

21
22
23

return ('http://www.gravatar.com/avatar/' +
hashlib.md5(email.lower().encode('utf-8')).hexdigest() +
217

24

'?' + query_params)

Whats happening here?

Lets go through this code a bit.
Lines 1-8: provide the needed imports and register a tag called gravatar_img.
Lines 9-12: get the gravatar url and then create a nice circular image tag and return it.
The image is sized based upon the passed-in value.
The rest: this is the code that actually constructs the url to use when calling gravatar
to get the users gravatar.
To construct the url we have a few steps to cover. Lets work backwards.
1
2
3

return ('http://www.gravatar.com/avatar/' +
hashlib.md5(email.lower().encode('utf-8')).hexdigest() +
'?' + query_params)

The base url is http://www.gravatar.com/avatar/. To that we add the users email address hashed with md5 (which is required by gravatar) along with the query_params.
We pass in two query parameters:
s: the size of the image to return
d: a default image that is returned if the email address doesnt have a gravatar account
The code to do that is here:
1
2

default = ('http://upload.wikimedia.org/wikipedia/en/9/9b/'
'Yoda_Empire_Strikes_Back.png')

3
4
5

query_params = urlencode([('s', str(size)),

('d', default)])

urlencode is important as it provides any necessary escaping/character mangling to ensure

you have a proper query string. For our default image, it must be available on the web somewhere, so weve just picked a random Star Wars image. There are several other query string
parameters that gravatar accepts, and they are all explained in detail here.
With that we should now have a functioning tag called gravatar_img that takes in an email
address and an optional size and returns the appropriate img markup for us to use. Lets now
use this in our _jedibadge template.
218

1
2
3
4
5
6
7
8
9
10
11
12
13

14

<!-- The jedi badge info box, shows user info -->
{% load staticfiles %}
{% load main_gravatar %}
<section class="info-box" id="user_info">
<h1>Jedi Badge</h1>
{% gravatar_img user.email %}
<ul>
<li>Rank: {{user.rank}}</li>
<li>Name: {{user.name }}</li>
<li>Email: {{user.email }}</li>
<li><a id="show-achieve" href="#">Show Achievements</a></li>
</ul>
<p>Click <a href="{% url 'edit' %}">here</a> to make changes to
your credit card.</p>
</section>

Notice we have changed line 3 and line 6 from our earlier template: Line 3 loads our custom
tag library and line 6 calls it, passing in user.email. Now we have gravatar support - and it
only took a handful of lines!
NOTE: There are a number of gravatar plugins available on GitHub, most of
which are basically the same thing we just implemented. While you shouldnt reinvent the wheel, theres not much point in utilizing an external dependency for
something that is this straight-forward. However if/when we need more than
basic gravatar support, it may be worth looking into some of the pre-existing
packages.

To finalize the gravatar support, we better re-run our tests and make sure nothing fails (aside
for the three previous errors, of course), as well as add some new tests for the gravatar tag.
Since this is review, add this on your own - its always good to get some extra practice.

219

Template 2: Status Updating and Reporting

Its pretty common these days for membership sites to have some sort of status update functionality, where users can post their status or whatever is on their minds and others can see
a list of the most recent status updates. The screenshot below shows the two info boxes that
will participate in the status updating / reporting functionality.

Figure 10.2: Status Reporting

The top info box allows users to submit status updates and the bottom info box shows the
history of status updates. Lets see how these are implemented.
First for the status updater. The HTML template templates/main/_statusupdate.html
is shown below:
1
2
3
4

<!-- represents the status update info box -->

<section class="info-box" id="updates">
<h1>Report back to base</h1>
<form accept-charset="UTF-8" action="{% url 'report' %}"
220

5
6
7

8
9
10

11
12
13
14

role="form" method="post">{% csrf_token %}

<div class="input-group">
<input id="status" type="text" class="form-control"
name="status"
placeholder="{{user.name}} whats your status?">
<span class="input-group-btn">
<button class="btn btn-primary"
type="submit">Report</button>
</span>
</div>
</form>
</section>

Its just a simple form that POSTS the report URL; be sure to add the URL to urls.py as:
1

url(r'^report$', 'main.views.report', name="report"),

To fully understand the view function, main.views.report, we need to first have a look at
the model, which the view function relies on. The listing for user.models.StatusReport
is below:
1
2
3
4

class StatusReport(models.Model):
user = models.ForeignKey('payments.User')
when = models.DateTimeField(auto_now_add=True)
status = models.CharField(max_length=200)

Update the model, and then run syncdb.

Just three columns here:
1. user- a foreign key into the User table in payments.models
2. when - a time stamp defaulted to now
3. status - the actual status message itself
For the view functionality we could have created a Django form like we did with the sign-in
or registration function, but thats not strictly required, especially since we arent doing
any validation. So lets use a straight-forward view function which is implemented in
main.views.report:
1
2
3

def report(request):
if request.method == "POST":
status = request.POST.get("status", "")
221

4
5
6
7
8

#update the database with the status

if status:
uid = request.session.get('user')
user = User.get_by_id(uid)
StatusReport(user=user, status=status).save()

9
10
11

#always return something

return index(request)

Update the imports:

1

from main.models import MarketingItem, StatusReport

A brief description of the important lines:

Line 2: only respond to POST requests
Line 3: pull the status out of the request, which corresponds to name of status on our
form:
1
2

<input id="status" type="text" class="form-control" name="status"

placeholder="{{user.name}} what's your status?">
Line 7-8: grab the current logged-in user from the request
Line 9: update the StatusReport table with the update the user just submitted, then
return the index page, which will in turn return user.html since we have a logged-in
user.

The last line will in effect cause the Recent Status Reports info box to update with the newly
posted status, with the following updates to the main.views.index function:
1
2
3
4
5
6
7
8
9
10

def index(request):
uid = request.session.get('user')
if uid is None:
#main landing page
market_items = MarketingItem.objects.all()
return render_to_response(
'main/index.html',
{'marketing_items': market_items}
)
else:
222

11
12
13
14
15
16
17

#membership page
status = StatusReport.objects.all().order_by('-when')[:20]
return render_to_response(
'main/user.html',
{'user': User.get_by_id(uid), 'reports': status},
context_instance=RequestContext(request),
)

Be sure to update the imports as well:

1
2
3

from django.shortcuts import render_to_response, RequestContext

from payments.models import User
from main.models import MarketingItem, StatusReport

The main difference from our previous version of this function is:
1

status = StatusReport.objects.all().order_by('-when')[:20]

This line grabs a list of twenty status reports ordered by posted date in reverse order. Please
keep in mind that even though we are calling objects.all() we are never actually retrieving all records from the database for this table. Djangos ORM by default uses Lazy Loading
for its query sets, meaning Django wont actually query the database until you access the data
from the query set - which, in this case, is the at the point of slicing, [:20]. Thus we will only
pull at most 20 rows from the database.

Sanity Check - a brief aside

Dont believe me? Test it out.
We can prove that we are only pulling up to twenty results by turning on logging and running
the query. One way to do this is to fire up the Django shell by typing ./manage.py shell,
then running the following code:
1
2
3
4

>>> from main.models import StatusReport

>>> q = StatusReport.objects.all().order_by('-when')[:20]
>>> print(q.query)
SELECT "main_statusreport"."id", "main_statusreport"."user_id",
"main_statusreport"."when", "main_statusreport"."status" FROM
"main_statusreport" ORDER BY "main_statusreport"."when" DESC
LIMIT 20

As you can see from line 4, the exact SQL that Django executes is outputted for you to read
and it does include a LIMIT 20 at the end. In general, dropping down into the shell and
223

outputting the exact query is a good sanity check to quickly verify that the ORM is executing
what you think it is executing.

Django 1.8 QuerySets - another brief aside

In Django you have always been able to create a custom QuerySet that can make it easier to understand the intent of your code. For example lets create a latest query for the
StatusReports object. To do that in Django 1.8 we first create a StatusReportQuerySet
class like so:
1
2
3

class StatusReportQuerySet(models.QuerySet):
def latest(self):
return self.all().order_by('-when')[:20]

Then we hook it up to our StatusReport model by adding one line to our StatusReport
class:
1

objects = StatusReportQuerySet.as_manager()

Once that is setup we can use our new query set in place of our main.views.index function by
changing this line:
1

status = StatusReport.objects.all().order_by('-when')[:20]

status = StatusReport.objects.latest()

to:

Whats the difference?

Well, prior to Django 1.7 we had to write:

1

status = StatusReport.objects.get_query_set().latest()

At the very least Django 1.7 saves us a few keystrokes. In general though, there is some value
in using custom query sets because it can make the intent of the code clearer. Its pretty easy
to guess that latest() returns the latest StatusReports. This is especially advantageous
when you have to write some custom SQL or you have a complex query that isnt easy to understand what it does. By wrapping it in a query_set youre creating a kind of self-documenting
code that makes it easier to maintain, and now in 1.7 takes less key strokes as well.
Dont forget we can also verify that our new query is doing what we expect by using the same
technique we used previously, e.g., run ./manage.py shell import the model and then
run:

224

print (StatusReport.objects.latest().query)

And this should give the same output as our original query.

Back to the task a hand

Coming back to our view function:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def index(request):
uid = request.session.get('user')
if uid is None:
#main landing page
market_items = MarketingItem.objects.all()
return render_to_response(
'main/index.html',
{'marketing_items': market_items}
)
else:
#membership page
status = StatusReport.objects.all().order_by('-when')[:20]
return render_to_response(
'main/user.html',
{'user': User.get_by_id(uid), 'reports': status},
context_instance=RequestContext(request),
)

Lines 10-14:
1
2
3
4
5

return render_to_response(
'main/user.html',
{'user': User.get_by_id(uid), 'reports': status},
context_instance=RequestContext(request),
)

The difference here is that we are now returning context_instance=RequestContext(request).

This is required because in our template we added the {% csrf_token %} to avoid crosssite scripting vulnerabilities. Thus we need the view to return the RequestContext which
will be picked up by Djangos CsrfViewMiddleware and used to prevent against cross-site
scripting attacks.
With all that, we have the functionality to allow a user to submit a status update. Now we
need to display the status updates.
225

Template 3: Displaying Status Updates

If you glance back up at the code listing for main.views.index you will notice that we already included the code to return the list of Status Reports. We just need to display it with a
template: templates/main/_lateststatus.html:
1

2
3
4
5
6
7
8
9

10
11
12
13
14
15
16
17

<!-- list of latest status messages sent out by all users of the
site -->
{% load staticfiles %}
{% load main_gravatar %}
<section class="info-box" id="latest_happenings">
<h1>Recent Status Reports</h1>
{% for report in reports %}
<div class="media">
<div class="media-object pull-left">
<img src="{% gravatar_url report.user.email 32 %}"
width="32" height="32"
alt="{{ report.user.name }}"/>
</div>
<div class="media-body">
<p>{{ report.status }}</p>
</div>
</div>
{% endfor %}
</section>

Take note of:

Line 6: loops through the list of status reports returned from the database (no more
than twenty).
Line 9: new template tag gravatar_url that we will explain in a minute.
Line 13: displays the status update.
Line 6, Line 7, and Line 11: the classes media, media-object, and media-body are
all supplied by Bootstrap and designed to provide a list of items with images next to
them.
That gives us the list of the most recent status updates. The only thing left to do is to explain
the new gravatar_url template tag.
If you recall from our earlier gravatar_img tag found in main/templatetags/main_gravatar.py,
internally it called a function get_url to get the gravatar URL. What we have done is simply
226

exposed that get_url function by adding the @register.simple_tag decorator, and

changed the name of the function to gravatar_url to fit in with our template tag naming
conventions discussed in the last chapter.
So the code now looks like this:
1
2
3
4

@register.simple_tag
def gravatar_url(email, size=140):
default = ('http://upload.wikimedia.org/wikipedia/en/9/9b/'
'Yoda_Empire_Strikes_Back.png')

5
6
7
8

#mainly for unit testing with a mock object

if not(isinstance(email, str)):
return default

9
10
11

query_params = urlencode([('s', str(size)),

('d', default)])

12
13
14
15

return ('http://www.gravatar.com/avatar/' +
hashlib.md5(email.lower().encode('utf-8')).hexdigest() +
'?' + query_params)

And the final templatetag:

1
2
3

from django import template

from urllib.parse import urlencode
import hashlib

4
5

register = template.Library()

6
7
8
9
10
11

12

@register.simple_tag
def gravatar_img(email, size=140):
url = gravatar_url(email, size)
return '''<img class="img-circle" src="%s" height="%s"
width="%s"
alt="user.avatar" />''' % (url, size, size)

13
14
15
16
17

@register.simple_tag
def gravatar_url(email, size=140):
default = ('http://upload.wikimedia.org/wikipedia/en/9/9b/'
227

'Yoda_Empire_Strikes_Back.png')

18
19
20
21
22

#mainly for unit testing with a mock object

if not(isinstance(email, str)):
return default

23
24
25

query_params = urlencode([('s', str(size)),

('d', default)])

26
27
28
29

return ('http://www.gravatar.com/avatar/' +
hashlib.md5(email.lower().encode('utf-8')).hexdigest() +
'?' + query_params)

That should give us the basic functionality for our members page. Youll be adding some more
functionality in the exercises to give you a bit of practice, and then in the next chapter well
look at switching to a REST-based architecture.
Run your automated tests:
1
2
3

$ ./manage.py test ../tests

Creating test database for alias 'default'...
....................

4
5
6
7
8
9

.....
.
.....
---------------------------------------------------------------------Ran 31 tests in 1.434s

10
11
12

OK
Destroying test database for alias 'default'...

Then manually test:

1.
2.
3.
4.
5.

Register a new user

Ensure that the user can update their status
Logout
Login as a different user
Ensure you can see the status update posted by the previous user

228

Exercises
1. Our User Story US3 Main Page says that the members page is a place for announcements and to list current happenings. We have implemented user announcements in
the form of status reports, but we should also have a section for system announcements/current events. Using the architecture described in this chapter, create an Announcements info_box to display system-wide announcements?
2. You may have noticed that in the Jedi Badge box there is a list achievements link. What
if the user could get achievements for posting status reports, attending events, and any
other arbitrary action that we create in the future? This may be a nice way to increase
participation, because everybody likes badges, right? Go ahead and implement this
achievements feature. Youll need a model to represent the badges and a link between
each user and the badges they own (maybe a user_badges table). Then youll want
your template to loop through and display all the badges that the given user has.

229

Chapter 11
REST
Remember the status updates features that we implemented in the last chapter? It works but
we can do better.
The main issue is that when you submit a status, the entire page must reload before you see
your updated status. This is so web 1.0. We cant have that, can we? The way to improve this
and remove the screen refresh is by using AJAX, which is a client-side technology for making
asynchronous requests that dont cause an entire page refresh. Often AJAX is coupled with
a server-side API to make it much easier to get the data you need from JavaScript.
One of the most popular server-side API styles in modern web programming is REST.
REST stands for Representational State Transfer, which to most people means absolutely
nothing. Lets hazard a definition: REST is a stateless architectural style generally run
over HTTP that relies on consistent URL names and HTTP verbs (GET, POST, DELETE, etc.)
to make it easy for various client programs to simply and consistently access and manipulate
resources from a server in a standard way.
REST doesnt actually specify what format should be used for data exchange, but most new
REST APIs are implemented with JSON. This is great for us since JSON is extremely simple
to work with in Python, as a Python dictionary is basically JSON out of the box. So we will
also use JSON in our examples here.
When implementing a REST API, there are a number of ways you could choose to implement
it, and a lot of debate about which is the best way. Well focus on a standard method otherwise
we might never finish this chapter!

230

Structuring a REST API

There are three key points one must adhere to when implementing a nice REST-based API

Resources should be the main concern of your REST architecture

and URLs should be structured accordingly.
In REST your URL structure defines your API and thus how clients interact with your API
server. The URL structure should be based around the resources your server provides. Generally there are two ways to access a resources in REST, Through the Collection URI (Uniform Resource Identifiers) and through the Member URI (also commonly referred to as the
element URI).
For example, to represent our status updates as a REST API, we would use the following
URIs:
For a collection:
1

http://<site-name>/status_reports/

And for the member (i.e. an individual status_report):

1

http://<site-name>/status_reports/2

(where 2 is the id of the status_report object.)

It is also common to prefix the RESTful URLs with an api/ directory - i.e:
1

http://<site-name>/api/status_reports/

This helps to differentiate between the REST API and URLs that just return HTML.
Finally, to be good web citizens, its a good idea to put the version of your API into your URL
structure so you can change it in future versions without breaking everybodys code. Doing
so would have your URLs looking like:
1

http://<site-name>/api/v1/status_reports/

REST is built on top of HTTP; thus, it should use the appropriate

HTTP verbs.
The following tables describes the HTTP Verbs and how they map to a typical REST API:
For a Collection URI - i.e., http://<site-name>/api/v1/status_reports/

231

HTTP Verb

Typical Use

GET
PUT
POST
DELETE

Returns the entire collection as a list, potentially with related information

Replaces the entire collection with the passed-in collection
Creates a new entry in the collection, generally with an auto-assigned ID
Blows away the entire collection Bye Bye

For a Member URI - i.e., http://<site-name>/api/v1/status_reports/2

HTTP Verb

Typical Use

GET
PUT
POST
DELETE

Returns the member and any related data - status report with an id of 2
Replaces the addressed member with the one passed in
USUALLY NOT USED: POST to the Collections URI instead
Deletes the member with corresponding id

A couple of notes are worth mentioning. PUT is meant to be idempotent, which means you
can expect the same result every time you call it. That is why it implements an insert. It either
updates an existing member or inserts a new one if the member does not exist; the end result
is that the appropriate member will exist and have data equal to the data passed in.
POST on the other hand is not idempotent and is thus used to create things.
There is also an HTTP verb called PATCH which allows for partial updates; there is a lot of
debate about if it should be used and how to use it. Many (err most) developers ignore it
since you can create a new item with POST and update with PUT. Well ignore it as well since
it does over-complicate things.

Use HTTP return codes appropriately

HTTP has a rich set of status codes, and they should be used to help convey the result of an
API call. For example, successful calls can return a status code of 200 and errors/failures can
use the 4xx or the 5xx error codes. If you want to provide more specific information, you can
include it in the details portion of your response.
Those are three main concerns with regards to designing a RESTful API. Authentication is a
fourth concern, but lets come back to authentication a bit later. Lets start by first designing
what our API should look like, and then implementing it.
It is important to note that we dont necessarily need to expose all the resources of our system
to the REST API; just the ones we care to share. Put another way, we should only expose the
resources that other developers care about, who would use the data in some meaningful way.
232

Conversely, oftentimes when thinking through how to design a REST API, developers are
stuck with the idea that they need more verbs to provide the access they want to provide.
While this is sometimes true, it can usually be solved by exposing more resources. The canonical example of this is login. Rather than implementing something like:
1

GET api/v1/users/1/login

Consider login as a separate resource. Or better yet, call it a session:

1
2

POST api/v1/session - login and return a session ID

DELETE api/v1/session/sessionID - log out

This sticks more strictly to the REST definition. Of course with REST there are only suggestions/conventions, and nothing will stop you from implementing whatever URL structure
you wish. This can get ugly if you dont stick with the conventions outlined. Only deviate
from them if you have a truly compelling reason to do so.

233

REST for MEC

For Mos Eisleys Cantina, lets start off with the simplest REST framework we can get away
with and add to it as we go. The one issue we are trying to fix now is the posting of status
updates without having to reload the page, so lets just create the status API, and then we will
expose other resources as necessary.
Following the design considerations discussed above, we will use the following URL structure:

Collection
GET - api/v1/status_reports - returns ALL status reports
POST - api/v1/status_reports - creates a status update and returns the id

Member
GET - api/v1/status_reports/<id> - returns a particular report status by id
Thats all we need for now. You can see that we could simply add a number of URL query
strings - i.e., user, date, etc. - to provide further query functionality, and maybe a DELETE
as well if you wanted to add additional functionality, but we dont need those yet.

234

Django REST Framework (DRF)

Before jumping in, its always worth weighing the cost of learning a new framework versus
the cost of implementing things yourself. If you had a very simple REST API that you needed
to implement, you could develop it in a few lines of code:
1
2

from django.http import HttpResponse

from django.core import serializers

3
4
5
6

def report(request):
if request.method = "GET":
status = StatusReport.objects.all().order_by('-when')[:20]

7
8
9
10
11
12

return HttpResponse(
serializers.serialize("json", status),
content_type='application/json',
context_instance=RequestContext(request)
)

There you go: You now have a simple and extremely naive REST API with one method.
Of course that isnt going to get you very far. We could abstract the JSON functionality into
a mixin class, and then by using class-based views make it simple to use JSON on all of our
views. This technique is actually described in the Django documentation.
1
2

from django.core import serializers

from django.http import HttpResponse

3
4
5
6
7
8
9
10

11
12
13
14
15
16

class JSONResponseMixin(object):
"""
A mixin that can be used to render a JSON response.
"""
def render_to_json_response(self, context, **response_kwargs):
"""
Returns a JSON response, transforming 'context' to make the
payload.
"""
return HttpResponse(
serializers.serialize("json",context),
content_type='application/json',
**response_kwargs
)
235

While this is a bit better, it still isnt going to help a whole lot with authentication, API discovery, handling POST parameters, etc. To that end, we are going to use a framework to help us
implement our API. The two most popular ones as of writing are Django REST Framework
and Tastypie. Which should you use? They really are about the same in terms of functionality,
so its up to you.
Its worth nothing (or maybe not) that we chose Django REST Framework since it has a
cooler logo.
If youre unsure (and not sold by the logo argument), you can look at the popularity of the
package, how active it is, and when its last release date to help you decide. There are two
great places to find this information: Django Packages and Github. Look for the number of
watchers, stars, forks, and contributors. Check out the issues page as well to see if there are
any reported show stopper issues.
Using this information, especially when you are unfamiliar with the project can greatly aid in
the decision making process of which project to use.
That said, lets jump right into the meat and potatoes.

Installation
The first thing to do is install DRF:
1

$ pip install djangorestframework

Also, lets update our requirements.txt file to include our new dependency:
1

$ pip freeze > requirements.txt

Your file should look like this:

1
2
3
4
5
6
7

Django==1.8.2
django-embed-video==0.11
djangorestframework==3.1.1
mock==1.0.1
psycopg2==2.5.3
requests==2.3.0
stripe==1.9.2

Serializers
DRF provides a number of tools you can use. Lets start with the most fundamental: the
serializer.
236

Serializers provide the capability to translate a Model or QuerySet to/from JSON. This is
what we just saw with the django.core.serializers above. DRF serializers do more-orless the same thing, but they also hook directly into all the other DRF goodness.
Lets create a serializer for the StatusReport object. Create a new file called main/serializers.py with the following content:
1
2

from rest_framework import serializers

from main.models import StatusReport

3
4
5
6
7
8
9

class StatusReportSerializer(serializers.Serializer):
id = serializers.ReadOnlyField()
user = serializers.StringRelatedField()
when = serializers.DateTimeField()
status = serializers.CharField(max_length=200)

10
11
12

def create(self, validated_data):

return StatusReport(**validated_data)

13
14
15
16
17

18
19

def update(self, instance, validated_data):

instance.user = validated_data.get('user', instance.user)
instance.when = validated_data.get('when', instance.when)
instance.status = validated_data.get('status',
instance.status)
instance.save()
return instance

If youve worked with Django Forms before, the StatusReportSerializer should look
somewhat familiar. It functions like a Django Form. You first declare the fields to include in
the serializer, just like you do in a form (or a model, for that matter). The fields you declare
here are the fields that will be included when serializing/deserializing the object.
Two quick notes about the fields we declared for our serializer:
1. We declared an id (primary key) field as type serializers.ReadOnlyField(). This
is a read-only field that cannot be changed, but it needs to be there since it maps to our
id field, which we will need on the client side for updates.
2. Our user field is of type serializers.StringRelatedField(). This represents a
many-to-one relationship and says that we should serialize the user object by using its
__str__ function. In our case this is the users email address.

237

There are two functions, create() and update(). They do pretty much exactly what you
would think: Create a new model instance from serialized data, or update an existing model
instance. Any custom creation logic you may need can be put in these functions. By creation
logic were not referring to the code that would normally be put into your __init__ function,
because that is already going to be called. Creation logic is simply any logic you may need to
include when creating the object from a deserialized JSON string (i.e., a python dictionary).
For a single object serializer like the one shown above, there isnt likely to be much extra logic,
but there is nothing preventing you from writing a serializer that works on an entire object
chain. This is sometimes helpful when dealing with nested or related objects.

Tests
Lets write some tests for the serializer to ensure it does what we want it to do. You do remember the Test Driven Development chapter, right?
Implementing a new framework/reusable app is another great example of where Test Driven
Development shines. It gives you an easy way to get at the underpinnings of a new framework,
find out how it works, and ensure that it does what you need it to. It also gives you an easy
way to try out different techniques in the framework and quickly see the results (as we will
see throughout the rest of this chapter).
Serialization

The DRF serializers actually work in two steps: It first converts the object into a Python
dictionary and then converts the dictionary into JSON. Lets test that those two steps work
as we expect.
Create a file called ../tests/main/testSerializers.py:
1
2
3
4
5
6
7
8

from
from
from
from
from
from
from
from

django.test import TestCase

main.models import StatusReport
payments.models import User
main.serializers import StatusReportSerializer
rest_framework.renderers import JSONRenderer
collections import OrderedDict
rest_framework.parsers import JSONParser
django.utils.six import BytesIO

9
10
11

class StatusReportSerializer_Tests(TestCase):
238

12
13
14
15
16

@classmethod
def setUpTestData(cls):
cls.u = User(name="test", email="[email protected]")
cls.u.save()

17

cls.new_status = StatusReport(user=cls.u, status="hello

world")
cls.new_status.save()

18

19
20

cls.expected_dict = OrderedDict([
('id', cls.new_status.id),
('user', cls.u.email),
('when', cls.new_status.when.isoformat()),
('status', 'hello world'),
])

21
22
23
24
25
26
27
28
29
30

def test_model_to_dictionary(self):
serializer = StatusReportSerializer(self.new_status)
self.assertEquals(self.expected_dict, serializer.data)

There is a fair amount of code here, so lets take it a piece at a time. The first part of this file1
2
3
4
5
6
7

from
from
from
from
from
from
from

django.test import TestCase

main.models import StatusReport
payments.models import User
main.serializers import StatusReportSerializer
rest_framework.renderers import JSONRenderer
collections import OrderedDict
rest_framework.parsers import JSONParser

8
9
10

class StatusReportSerializer_Tests(TestCase):

11
12
13
14
15

@classmethod
def setUpTestData(cls):
cls.u = User(name="test", email="[email protected]")
cls.u.save()

16
17

cls.new_status = StatusReport(user=cls.u, status="hello

world")
239

cls.new_status.save()

18
19

cls.expected_dict = OrderedDict([
('id', cls.new_status.id),
('user', cls.u.email),
('when', cls.new_status.when.isoformat()),
('status', 'hello world'),
])

20
21
22
23
24
25

-is responsible for all the necessary imports and for setting up the user/status report that we
will be working with in our tests.
The first test1
2
3

def test_model_to_dictionary(self):
serializer = StatusReportSerializer(self.new_status)
self.assertEquals(self.expected_dict, serializer.data)

-verifies that we can take our newly created object self.new_status and serialize it to a
dictionary. This is what our serializer class does. We just create our serializer by passing in
our object to serialize and then call serialize.data, and out comes the dictionary we want.
Run the test boom! Hashtag FAIL. If you look at the results you should see an error about
the two dictionaries not being equal. You should also see a warning about naive datetime.
A Brief Aside about Timezone support

JSON, by convention, should use a datetime format called iso-8601. This is a universal
datetime format that includes timezone information. In python you can output this format
by calling isoformat on your string. The specification format will output the date then the
time then the timezone but if the timezone is UTC it will output Z instead of the timezone so
its something like: 2013-01-29T12:34:56.123Z however pythons isoformat will output
2013-01-29T12:34:56.123+00:00 for the same date. This makes our test fail if we are
using UTC, which is the default timzone. For a quick fix, lets just change our setUpClass
method as such, to use the properly formated timezone in the expected_dict so the function now will look like:
1
2
3
4

@classmethod
def setUpTestData(cls):
cls.u = User(name="test", email="[email protected]")
cls.u.save()

5
6
7

cls.new_status = StatusReport(user=cls.u, status="hello world")

cls.new_status.save()
240

8
9
10
11

when = cls.new_status.when.isoformat()
if when.endswith('+00:00'):
when = when[:-6] + 'Z'

12
13
14
15
16
17
18

cls.expected_dict = OrderedDict([
('id', cls.new_status.id),
('user', cls.u.email),
('when', when),
('status', 'hello world'),
])

Notice in lines 9-11 that we grab the time (when) and convert it to the format that DRF expects.
Then we use that value to populate our expected_dict. After making this change our test
should pass.
Dictionary to JSON

The next step in the object to JSON conversion process is converting the dictionary to JSON:
1
2
3
4
5

def test_dictionary_to_json(self):
serializer = StatusReportSerializer(self.new_status)
content = JSONRenderer().render(serializer.data)
expected_json = JSONRenderer().render(self.expected_dict)
self.assertEquals(expected_json, content)

To convert to JSON you must first call the serializer to convert to the dictionary, and then
call JSONRenderer().render(serializer.data). This instantiates the JSONRenderer
object and passes it a dictionary to render as JSON. The render function calls json.dumps
and ensures the output is in the proper unicode format. Now we have an option of how we
want to verify the results. We could build the expected JSON string and compare the two
strings.
One drawback here is that you often have to play around with formatting the string exactly
right, especially when dealing with date formats that get converted to the JavaScript date
format.
Another option (which we used) is to create the dict that we should get from the serializer
(and we know what the dict is because we just ran that test), then convert that dict to JSON
and ensure the results are the same as converting our serializer.data to JSON. This also
has its issues, as the order in which the attributes are placed in the results JSON string is
important, and dictionaries dont guarantee order. So we have to use OrderedDict, which
241

will ensure our dictionary preserves the order of which the keys were inserted. After all that,
we can verify that we are indeed converting to JSON correctly.
Run the tests:
1

$ ./manage.py test ../tests

So serialization seems to work correctly.

How about deserializaiton?
Deserialization

We need to run the opposite test:

1

def test_json_to_StatusReport(self):

2
3
4
5

json = JSONRenderer().render(self.expected_dict)
stream = BytesIO(json)
data = JSONParser().parse(stream)

6
7

8
9

#where calling update to pass in existing object, plus data to

update
serializer = StatusReportSerializer(self.new_status, data=data)
self.assertTrue(serializer.is_valid())

10
11
12
13
14
15

status = serializer.save()
self.assertEqual(self.new_status.id, status.id)
self.assertEqual(self.new_status.status, status.status)
self.assertEqual(self.new_status.when, status.when)
self.assertEqual(self.new_status.user, status.user)

All should pass.

Of course we could replace all the asserts at the end with:
1

self.asesrtEqual(self.new_status, status)

But I just wanted to be explicit and show that each field was infact being deserialized correctly.
Now what about creating a new serializer instance from json? Lets write another test:
1
2
3

def test_json_to_new_StatusReport(self):
json = JSONRenderer().render(self.expected_dict)
stream = BytesIO(json)
242

data = JSONParser().parse(stream)

5
6
7

serializer = StatusReportSerializer(data=data)
self.assertTrue(serializer.is_valid())

8
9
10
11
12

status = serializer.save()
self.assertEqual(self.new_status.status, status.status)
self.assertIsNotNone(status.when)
self.assertEqual(self.new_status.user, status.user)

You probably already guessed it, but this test is going to fail.
1
2
3
4

5
6

Traceback (most recent call last):

File "/testSerializers.py", line 61, in test_json_to_StatusReport
self.assertEqual(self.new_status.user, status.user)
File "/site-packages/django/db/models/fields/related.py", line
6088, in __get__
"%s has no %s." % (self.field.model.__name__, self.field.name)
django.db.models.fields.related.RelatedObjectDoesNotExist:
StatusReport has no user.

The important part is RelatedObjectDoesNotExist; thats the error you get when you try
to look up a model object from the db and it doesnt exist. Why dont we have a user object
associated with our StatusReport? Remember in our serializer, we used this line:
1

user = serializers.StringRelatedField()

This means that we serialize the user field by calling its __str__ function, which just returns an email. Then when we deserialize the object, our create function is called, but since
StringRelatedField is by read only then the user wont get passed in. Nor will the id for
that matter (because its set to be a ReadOnlyField). Well come back to the solution for this
in just a second.
First, lets talk about ModelSerializers.
ModelSerializers

Our initial StatusReportSerializer contains a ton of boilerplate code. Were really just
copying the field from our model. Fortunately, there is a better way. Enter ModelSerializers.
If we rewrite our StatusReportSerializer using DRFs ModelSerializer, it looks like
this:

243

1
2

from rest_framework import serializers

from main.models import StatusReport

3
4
5

class StatusReportSerializer(serializers.ModelSerializer):

6
7
8
9

class Meta:
model = StatusReport
fields = ('id', 'user', 'when', 'status')

Wow! Thats a lot less code. Just like Django gives you a Form and a ModelForm, DRF
gives you a Serializer and a ModelSerializer. And just like Djangos ModelForm, the
ModelSerializer will get all the information it needs from the model. You just have to
point it to the model and tell it what fields you want to use.
The only difference between these four lines of code in the ModelSerializer and the twelve
lines of code in our Serializer is that the Serializer serialized our user field using the
id instead of the email address. This is not exactly what we want, but it does mean that when
we deserialize the object from JSON, we get our user relationship back! To verify that, and to
update our tests to account for the user being serialized by id instead of email, we only have
to change our cls.expected_dict to look like this in testSerializers.py:
1
2
3
4
5
6

cls.expected_dict = OrderedDict([
('id', cls.new_status.id),
('user', cls.u.id),
('when', cls.new_status.when),
('status', 'hello world'),
])

Almost there. If you recall, our main.models.index uses the users email address so we can
do the gravatar lookup. How do we get that email address?
We create a custom relationship field in serializers.py:
1

from payments.models import User

2
3

class RelatedUserField(serializers.RelatedField):

4
5

read_only = False

6
7
8

def to_representation(self, value):

return value.email

244

10
11

def to_internal_value(self, data):

return User.objects.get(email=data)
Line 3: We inherit from the serializers.RelatedField which is the base relationship field for DRF. To fully implement this field type we need two function
to_representation - for serialization, and to_internal_value - for deserialization
Line 5: If you want deserialization to occur, you have to set the read_only = False
attribute, in effect saying this field should be read / write
Line 7-8: The function to_representation(self, value) will receive the target
of the field (in our case the User), and its our job to return the serialized representation
we want (in our case the email address).
Line 10-11: The function to_internal_value(self, data) will receive the JSON
value from the field (as the data parameter), and its our job to return the object we
want. We do this by looking it up in the users table.

Full update:
1
2
3

from rest_framework import serializers

from main.models import StatusReport
from payments.models import User

4
5

class RelatedUserField(serializers.RelatedField):

6
7

read_only = False

8
9
10

def to_representation(self, value):

return value.email

11
12
13

def to_internal_value(self, data):

return User.objects.get(email=data)

14
15
16

class StatusReportSerializer(serializers.ModelSerializer):
user = RelatedUserField(queryset=User.objects.all())

17
18
19
20

class Meta:
model = StatusReport
fields = ('id', 'user', 'when', 'status')

245

Take note of line 16. When declaring a RelatedField in our serailizer queryset is a required parameter. The intent it to make it explicit where the data is comming from for this
field.
Update the cls.expected_dict again, back to using the users email:
1
2
3
4
5
6

cls.expected_dict = OrderedDict([
('id', cls.new_status.id),
('user', cls.u.email),
('when', cls.new_status.when),
('status', 'hello world'),
])

Now run the tests. They should pass. Perfect.

There are several other ways to manage/display relationships in DRF; for more information
on these, check out the docs.
Also have a look at the SlugRelatedField as it basically does the same thing we just implemented.

246

Now Serving JSON

Since we spent all that time getting our serializer to work just right, lets make a view for it
and get our REST API up and running. Start by creating a separate file for our REST API
views, main/json_views.py. This isnt required, but its useful to keep them separate.
The distinction is not just that one set of views returns JSON and other returns HTML, but
that the API views (which return JSON) are defining the REST framework for our application e.g., how we want other clients or programs to be able to access our information. On the other
hand, the standard views are involved with the web view of our application. We are in effect
splitting our application into two distinct parts here:
1. a core application which exposes our resources in a RESTful way.
2. our web app which focuses on displaying web content and is also a client of our core
application.
This is a key design principle of REST, as building the application this way will allow us to
more easily build other clients (say, mobile apps) without having to re-implement the backend. In fact, taken to its natural conclusion we could eventually have one Django app that is
the core application that exposes the REST API to our web app, Mos Eisleys Cantina. We
arent quite there yet, but its a good metaphor to keep in your head when thinking about the
separation of the REST API from your web app.

And A View
Our new view function in main/json_views.py should look like:
1
2
3
4
5

from
from
from
from
from

rest_framework import status

rest_framework.decorators import api_view
rest_framework.response import Response
main.serializers import StatusReportSerializer
main.models import StatusReport

6
7
8
9
10
11

@api_view(['GET', 'POST'])
def status_collection(request):
"""Get the collection of all status_reports
or create a new one"""

12
13

if request.method == 'GET':
247

14
15

16
17
18
19
20
21

22

status_report = StatusReport.objects.all()
serializer = StatusReportSerializer(status_report,
many=True)
return Response(serializer.data)
elif request.method == 'POST':
serializer = StatusReportSerializer(data=request.DATA)
if serializer.is_valid():
serializer.save()
return Response(serializer.data,
status=status.HTTP_201_CREATED)
return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)
Line 1-5: Import what we need
Line 8: The @api_view is a decorator provided by DRF, which:

checks that the appropriate request is passed into the view function
adds context to the Response so we can deal with stuff like CSRF tokens
provides authentication functionality (which we will discuss later)
handles ParseErrors

Line 8: The arguments to the @api_view are a list of the HTTP verbs to support.
Line 13-16: A GET request on the collection view should return the who list. Grab the
list, then serialize to JSON and return it.

Line 16: DRF also provides the Response object which inherits django.template.response.Simple
It takes in unrendered content (for example, JSON) and renders it based upon a
Content-Type specified in the request.header.
Line 17-20: For POST requests just create a new object based upon passed-in data.
Line 17: Notice the use of request.DATA DRF provides the Request class that
extends Djangos HTTPRequest and provides a few enhancements: request.DATA,
which works similar to HTTPRequest.POST but handles POST, PUT and PATCH
methods.
Line 21: On successfully saving, return a response with HTTP return code of 201 (created). Notice the use of status.HHTP_201_CREATED. You could simply put in 201,

248

but using the DRF status identifiers makes it more explicit as to what code youre returning so that people reading your code dont have to remember all the HTTP return
codes.
Line 22: If the deserialization process didnt work (i.e., serializer.is_valid()
returns False) then return HTTP_400_BAD_REQUEST. This basically means dont call
me again with the same data because it doesnt work.
Thats a lot of functionality and not very much code. Also if you recall from the section on
Structuring a REST API, this produces a REST API that uses the correct HTTP Verbs and
returns the appropriate response codes. If you further recall from the discussion on Structuring a REST API, resources have a collection URL and a member URL. To finish the example
we need to flesh out the member URL below by updating main/json_views.py:
1
2
3

@api_view(['GET', 'PUT', 'DELETE'])

def status_member(request, id):
"""Get, update or delete a status_report instance"""

4
5
6
7
8

try:
status_report = StatusReport.objects.get(id=id)
except StatusReport.DoesNotExist:
return Response(status=status.HTTP_404_NOT_FOUND)

9
10
11
12
13
14

15
16
17
18

19
20
21

if request.method == 'GET':
serializer = StatusReportSerializer(status_report)
return Response(serializer.data)
elif request.method == 'PUT':
serializer = StatusReportSerializer(status_report,
data=request.DATA)
if serializer.is_valid():
serializer.save()
return Response(serializer.data)
return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)
elif request.method == 'DELETE':
status_report.delete()
return Response(status=status.HTTP_204_NO_CONTENT)

This is nearly the same as the collection class, but we support different HTTP verbs and are
dealing with one object instead of an entire collection of objects. With that, we now have the
entire API for our StatusReport
249

NOTE: In the code above, the PUT request is not idempotent. Do you know why?
What happens if we call a PUT request with an id that is not in the database? For
extra credit go ahead and implement the fix now or just read on; we will fix it
later.

Now were getting somewhere.

Lets not forget to test it.

Test
Create a new file called ../tests/main/testJSONViews.py:
First the GET functionality:
1
2
3
4

from
from
from
from

django.test import TestCase

main.json_views import status_collection
main.models import StatusReport
main.serializers import StatusReportSerializer

5
6
7

class dummyRequest(object):

8
9
10
11
12
13
14

def __init__(self, method):

self.method = method
self.encoding = 'utf8'
self.user = "root"
self.QUERY_PARAMS = {}
self.META = {}

15
16
17

class JsonViewTests(TestCase):

18
19
20
21

22

def test_get_collection(self):
status = StatusReport.objects.all()
expected_json = StatusReportSerializer(status,
many=True).data
response = status_collection(dummyRequest('GET'))

23
24

self.assertEqual(expected_json, response.data)

Above we create a dummyRequest that has the information that DRF expects.
250

NOTE: We cant use the RequestFactory yet because we havent setup the
URLs.

Then in our JsonViewTests we call our status_collection function, passing in the GET
parameter.
This should return all the StatusReport objects as JSON. We manually query all the
StatusReport, convert them to JSON, and then compare that to the return from
our view call. Notice the returned response we call response.data as opposed to
response.content which we are used to, because this response hasnt actually been
rendered yet.
Otherwise the test is the same as any other view test. To be complete, we should check the
case where we have data and where there is no data to return as well, and we should also test
the POST with and without valid data. Well leave that as an exercise for you, dear reader.
Dont forget to run the test:
1

$ ./manage.py test ../tests

Now that we have tested our view, lets go ahead and wire up the URLs. We are going to create
a separate urls file specifically for the JSON URLs in our main application as opposed to using
our default django_ecommerce/urls.py file. This creates better separation of concerns and
allows our REST API to be more independent.
Lets create a main/urls.py file that contains:
1

from django.conf.urls import patterns, url

2
3
4
5
6
7

urlpatterns = patterns(
'main.json_views',
url(r'^status_reports/$', 'status_collection'),
url(r'^status_reports/(?P<id>[0-9]+)$', 'status_member'),
)

We need our django_ecommerce/urls.py to point to this new urls.py. So add the following
URL entry to the end of the list:
1

url(r'^api/v1/', include('main.urls')),

Dont forget to actually add rest_framework to the list of INSTALLED_APPS in your settings.py. This should now look like:
1
2

INSTALLED_APPS = (
'django.contrib.auth',
251

'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'main',
'django.contrib.admin',
'django.contrib.flatpages',
'contact',
'payments',
'embed_video',
'rest_framework',

3
4
5
6
7
8
9
10
11
12
13
14
15

Now if your start the development server and navigate to http://127.0.0.1:8000/api/v1/

status_reports/ you should see something like:

Figure 11.1: DRF browsable API for Status Collection

Wow! Thats DRFs Browsable API and its a real usability win. With no extra work you
automatically get this nice browsable API, which shows you, in human-readable format, what
function is being called and its return value.
Also if you scroll down on the page a little bit, you will see that it gives you a way to easily
call the API. Perfect for a quick manual test/sanity check to make sure things are working
correctly. (But obviously not a replacement for unit testing, so dont get any ideas.)
252

Of course, when you call your API from your program you dont want to see that
page; we just want the JSON. Dont worry: DRF has you covered. By default the
@api_view wrapper, which gives us the cool browsable API amongst other things, listens to the Accept header to determine how to render the template (remember the
rest_framework.response.Response is just a TemplateResponse object).
Try this from the command line (with the development server running):
1

$ curl http://127.0.0.1:8000/api/v1/status_reports/
NOTE: Windows Users Sorry. You most likely dont have curl installed by
default like the rest of us do. You can download it here. Just scroll ALL the way
down to the bottom and select the appropriate download for you system.

Or, to be more explicit:

1

$ curl http://127.0.0.1:8000/api/v1/status_reports/ -H 'Accept:

application/json'

And you will get back raw JSON (as long as you have a status update in the table, of course).
1

[{"id": 1, "user": "[email protected]", "when":

"2014-08-22T01:35:03.965Z", "status": "test"}]

Thus returning JSON is the default action the DRF Response will take. However, by default
your browser will set the Accept header to text/html, which you can also do from curl like
this:
1

$ curl http://127.0.0.1:8000/api/v1/status_reports -H 'Accept:

text/html'

And then youll get back a whole mess of HTML. Hats off to the DRF folks. Very nicely done.

253

Using Class-Based Views

Up until now we have been using function-based views. Functions are cool and easy, but
there are some things that become easier if you use a class-based view - mainly, reusing functionality. DRF provides a number of mixins that can be used with your class-based views to
make your life easier.
Lets refactor our function-based views to class-based views.

By using some of the mixins that DRF provides, we can do more with a lot less code. Update
json_views.py:
1
2
3

from rest_framework import mixins, generics

from main.serializers import StatusReportSerializer
from main.models import StatusReport

4
5
6
7
8

class StatusCollection(mixins.ListModelMixin,
mixins.CreateModelMixin,
generics.GenericAPIView):

9
10
11

queryset = StatusReport.objects.all()
serializer_class = StatusReportSerializer

12
13
14

def get(self, request):

return self.list(request)

15
16
17

def post(self, request):

return self.create(request)

Where did all the code go? Thats exactly what the mixins are for:
Line 7: mixins.ListModelMixin provides the list(request) function that allows you to serialize a collection to JSON and return it.
Line 7: mixins.CreateModelMixin provides the create(request) function that
allows for the POST method call - e.g., creating a new object of the collection type.
Line 7: generics.GenericAPIView - this mixin provides the core functionality
plus the Browsable API we talked about in the previous section.
Line 10: defining a class-level queryset member is required so the ListModelMixin
can works its magic.

254

 Line 11: defining a class-level serializer_class member is also required for all the
Mixins to work.
Remaining Lines: we implement GET and POST by passing the call to the respective
Mixin.
Using the class-based view in this way with the DRF mixins saves a lot of boilerplate code
while still keeping things pretty easy to understand. Also, we can clearly see what happens
with a GET vs a POST request without having a number of if statements, so there is better
separation of concerns.
NOTE: It would help even more if the mixin would have been called
generics.GetPostCollectionAPIView so that you know its for GET and
POST on a collection as opposed to having to learn DRF. ListCreateAPIView
doesnt really tell us anything about the REST API that this view function is
creating unless we are already familiar with DRF. In general, the folks at Real
Python like to be a bit more explicit even if it means just a bit more code.
Fortunately, there is nothing preventing you from putting in a nice docstring to
explain what the function does - which is a good compromise. Ultimately its up
to you to decide which one you prefer.

To complete the example, here is the status_member function after being refactored into a
class view:
1
2
3
4

class StatusMember(mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
generics.GenericAPIView):

5
6
7

queryset = StatusReport.objects.all()
serializer_class = StatusReportSerializer

8
9
10

def get(self, request, *args, **kwargs):

return self.retrieve(request, *args, **kwargs)

11
12
13

def put(self, request, *args, **kwargs):

return self.update(request, *args, **kwargs)

14
15
16

def delete(self, request, *args, **kwargs):

return self.destroy(request, *args, **kwargs)
255

In fact we can even simplify the class inheritance further using:

1

class StatusMember(mixins.RetrieveUpdateDestroyAPIView):

This is just a combination of the four mixins we inherited from above. The choice is yours.
We also need to change our main/urls.py function slightly to account for the class-based
views:
1
2

from django.conf.urls import patterns, url

from main import json_views

3
4
5
6

urlpatterns = patterns(
'main.json_views',
url(r'^status_reports/$',
json_views.StatusCollection.as_view()),
url(r'^status_reports/(?P<pk>[0-9]+)/$',
json_views.StatusMember.as_view())
)

There are two things to take note of here:

1. We are using the class method as_view() that provides a function-like interface into
the class.
2. The second URL for the StatusMember class must use the pk variable (before we were
using id), as this is a requirement of DRF.
And finally, we need to modify our test a little bit to account for the class-based views:
1

from main.json_views import StatusCollection

2
3
4
5
6

def test_get_collection(self):
status = StatusReport.objects.all()
expected_json = StatusReportSerializer(status, many=True).data
response = StatusCollection.as_view()(dummyRequest("GET"))

7
8

self.assertEqual(expected_json, response.data)

Notice in line 5 that we need to call the as_view() function of our StatusCollection class
just like we do in main/urls.py. We cant just call StatusCollection().get(dummyRequest("GET"))
directly. Why? Because as_view() is magic. It sets up several instance variables such as
self.request, self.args, and self.kwargs; without these member variables set up,
your test will fail.
Make sure to run your tests before moving on.
256

Authentication
There is one final topic that needs to be covered so that a complete REST API can be implemented: Authentication.
Since we are charging a membership fee for MEC, we dont want unpaid users to have access to our members-only data. In this section we will look at how to use authentication so
that only authorized users can access your REST API. In particular, we want to enforce the
following constraints:
Unauthenticated requests should be denied access
Only authenticated users can post status reports
Only the creator of a status report can update and/or delete that status report

Unauthenticated requests should be denied access

We can use DRFs built-in permissions - rest_framework.permissions.IsAuthenticated
- to take care of this. All we need to do is add that class as a property of our class views
(StatusCollection and StatusMember) by adding the property like so:
1

permission_classes = (permissions.IsAuthenticated,)

Pay attention to that comma at the end of the line! We need to pass a tuple, not a single item.
Also, dont forget the proper import:
1

from rest_framework import permissions

Testing Authentication
We can verify that this is working by running our unit tests which should now fail, as they try
to check if the user is authenticated:
1

AttributeError: 'str' object has no attribute 'is_authenticated'

We need an authenticated user. Again, DRF comes through with some helpful test functions.
Previously we were using a dummyRequest class to provide the functionality that we needed.
Lets drop that and use DRFs APIRequestFactory:
1
2

from django.test import TestCase

from rest_framework.test import APIRequestFactory,
force_authenticate
257

from payments.models import User

4
5

class JsonViewTests(TestCase):

6
7
8
9
10

@classmethod
def setUpClass(cls):
super().setUpClass()
cls.factory = APIRequestFactory()

11
12
13
14

@classmethod
def setUpTestData(cls)
cls.test_user = User(id=2222, email="[email protected]")

15
16
17
18
19
20

def get_request(self, method='GET', authed=True):

request_method = getattr(self.factory, method.lower())
request = request_method("")
if authed:
force_authenticate(request, self.test_user)

21
22

return request

OK so what did we do in the above section of code?

Line 1: - import all the DRF testing goodies that we need.
Lines 7-10: - using the setUpClass function that will only run once for the test suite,
create the APIReqeustFactory that we will use in our tests.
Lines 12-14 - create a dummy user that we will use to authenticate. Note: we do not
need to actually save the user to the database, we just need a user object.
Line 16: - get_request is a factory that will create a request using the HTTP verb
passed in as method (the default value of which is GET).
Line 17: - the actual factory pattern that will get a method such as APIRequestFactory.get
or APIRequestFactory.post depending on the value we pass in for the method
parameter.
Line 18: - create the request that we got from the factory in line 11 - e.g., create a get
or post or delete or whatever request.
Lines 19-20: - if we passed in a truthy value for the method parameter auth ensure
the request is authenticated with our self.test_user otherwise the request will be
unauthenticated.
Line 22: - return it and we are done.
258

In a nutshell we have coded a way to create a request for any type of HTTP verb. Further, we
can decide if that request is authenticated or not. This gives us the flexibility to test as many
permutations as we might need to so we can properly exercise our REST API. For now we
put the code in the JSONViewTests because thats all we need. However you might consider
creating your own DRFTestCase, perhaps putting it in ../test/base_test_case.py, for
example. Then you could easily share it amongst whatever tests you create that need the
functionality.

Using Our New Request Factory

Now that we have the new fangled get_request factory function lets update our
test_get_collection so it will pass.
1
2
3

def test_get_collection(self):
status = StatusReport.objects.all()
expected_json = StatusReportSerializer(status, many=True).data

4
5
6

response = StatusCollection.as_view()(self.get_request())
self.assertEqual(expected_json, response.data)

Line 5 is the only line that changed, as it now calls our newly created sef.get_request()
factory method.
Lets add one more test, test_get_collection_requires_logged_in_user, to verify
that our authentication is working correctly:
1

from rest_framework import status

2
3

def test_get_collection_requires_logged_in_user(self):

4
5
6

anon_request = self.get_request(method='GET', authed=False)

response = StatusCollection.as_view()(anon_request)

7
8

self.assertEqual(response.status_code,
status.HTTP_403_FORBIDDEN)
Line 1: we use DRFs statuses in our test as they are more descriptive, so we will need
to add this import line to the top of the module
Line 5: we pass in authed=False to our get_request factory method, which sets
the user to be unauthorized

259

 Line 8: we verify that we return status.HTTP_403_FORBIDDEN, which is the correct

HTTP status code to return in the case of unauthorized access
Now we have our permission setup to ensure unauthorized users get a good ole Heisman.
This also takes care of our first authorization requirement: Only authenticated users can post
status reports. Now for our final requirement: Only the creator of a status report can update
or delete that status report.
To implement this type of permission, we need to create our own custom permission class.
Create a main/permissions.py file and then add the following code:
1

from rest_framework import permissions

2
3
4

class IsOwnerOrReadOnly(permissions.BasePermission):

5
6

def has_object_permission(self, request, view, obj):

7
8
9
10

#Allow all read type requests

if request.method in ('GET', 'HEAD', 'OPTIONS'):
return True

11
12

13

#this leaves us with write requests (i.e. POST / PUT /

DELETE)
return obj.user == request.user

Once the class is created, we need to update our StatusMember class in json_views.py as
well:
1

from main.permissions import IsOwnerOrReadOnly

2
3
4
5
6

class StatusMember(mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
generics.GenericAPIView):

7
8
9
10

queryset = StatusReport.objects.all()
serializer_class = StatusReportSerializer
permission_classes = (permissions.IsAuthenticated,
IsOwnerOrReadOnly)

11

260

12
13

def get(self, request, *args, **kwargs):

return self.retrieve(request, *args, **kwargs)

14
15
16

def put(self, request, *args, **kwargs):

return self.update(request, *args, **kwargs)

17
18
19

def delete(self, request, *args, **kwargs):

return self.destroy(request, *args, **kwargs)

And thats it. Now only owners can update / delete their status reports.
It is important to note that all of this authentication is using the default Authentication classes
which are SessionAuthentication and BasicAuthentication. If your main client is
going to be an AJAX web-based application then the default authentication classes will work
fine, but DRF does provide several other authentication classes if you need something like
OAuthAuthentication, TokenAuthentication or something custom.
The official DRF documentation does a pretty good job of going over these if you want more
info.

261

Conclusion
We started this chapter talking about the desire to not refresh the page when a user submitted
a status report. And, well,we didnt even get to that solution yet. Think of it as a cliff hanger
to be continued in the next chapter.
We did however go over one of the key ingredients to making that no-refresh happen: a good
REST API. REST is increasingly popular because of its simplicity to consume resources and
because it can be accessed from any client that can access the web.
Django REST Framework makes implementing the REST API relatively straight-forward and
helps to ensure that we follow good conventions. We learned how to serialize and deserialize
our data and structure our views appropriately along with the browsable API and some of the
important features of DRF. There are even more features in DRF that are worth exploring
such as ViewSet and Routers. While these powerful classes can greatly reduce the code you
have to write, you sacrifice readability. But that doesnt mean you shouldnt check them out
and use them if you like.
In fact, its worth going through the DRF site and browsing through the API Guide. Weve
covered the most common uses when creating REST APIs, but with all the different use-cases
out there, some readers will surely need some other part of the framework that isnt covered
here.
Either way: REST is everywhere on the web today. If youre going to do much web
development, you will surely have to work with REST APIs - so make sure you
understand the concepts presented in this chapter.

262

Exercises
1. Flesh out the unit tests. In the JsonViewTests, check the case where there is no data
to return at all, and test a POST request with and without valid data.
2. Extend the REST API to cover the user.models.Badge.
3. Did you know that the browsable API uses Bootstrap for the look and feel? Since we
just learned Bootstrap, update the browsable API Template to fit with our overall site
template.
4. We dont have permissions on the browsable API. Add them in.

263

Chapter 12
Django Migrations
Whats new in Django 1.7? Basically migrations. While there are some other nice features,
the new migrations system is the big one.
In the past you probably used South to handle database changes. However, in Django 1.7,
migrations are now integrated into the Django Core thanks to Andrew Godwin, who ran this
Kickstarter. He is also the original creator of South.
Lets begin

264

The problems that Migrations Solve

Migrations:
1. Speed up the notoriously slow process of changing a database schema.
2. Make it easy to use git to track your database schema and its associated changes.
Databases simply arent aware of git or other version control systems. Git is awesome
for code, but not so much for database schemas.
3. Provide an easy way to maintain fixture data linked to the appropriate schema.
4. Keep the code and schema in sync.
Have you ever had to make a change to an existing table (like re-naming a field) and you
didnt want to mess with dropping the table and re-adding it?
Migrations solve that.
Or perhaps you needed to update the schema on a live application with millions of rows of
data that simply cannot be lost.
Migrations make this much easier.
In general, migrations allow you to manage and work with your database schema in the same
way you would with your Django code. You can store versions of it in git, you can update
it from the command line, and you dont have to worry about creating large complex SQL
queries to keep everything up to date - although you still can if you love pain I mean SQL.

265

Getting Started with Migrations

The basic process for using migrations is simple:
1. Create and/or update a model.
2. Run ./manage.py makemigrations <app_name>.
3. Run ./manage.py migrate to migrate everything or ./manage.py migrate
<app_name> to migrate an individual app.
4. Repeat as necessary.
Thats it! Pretty straight-forward for the basic use-case, and this will work the majority of the
time. But when it doesnt work it can be hard to figure out why, so lets dig a bit deeper to get
a better understanding of how migrations work.

266

The Migration File

Start by running the following migration
1

$ ./manage.py makemigrations contact

You should see something like:

1
2
3

Migrations for 'contact':

0001_initial.py:
- Create model ContactForm

What exactly happened?

Django migrations actually created a migration file that describes how to create (or update)
the appropriate tables in the database. In fact, you can look at the migration file that was
created. Dont be afraid: Its just python :). The file is located within the migration directory
in the contact app. Open contact/migrations/0001_initial.py.
It should look something like this:
1
2

# -*- coding: utf-8 -*from __future__ import unicode_literals

3
4
5

from django.db import models, migrations

import datetime

6
7
8

class Migration(migrations.Migration):

9
10
11

dependencies = [
]

12
13
14
15
16
17
18
19
20
21
22

operations = [
migrations.CreateModel(
name='ContactForm',
fields=[
('id', models.AutoField(
verbose_name='ID', primary_key=True,
serialize=False, auto_created=True)),
('name', models.CharField(max_length=150)),
('email', models.EmailField(max_length=250)),
('topic', models.CharField(max_length=200)),
267

('message', models.CharField(max_length=1000)),
('timestamp',
models.DateTimeField(auto_now_add=True)),

23
24

],
options={
'ordering': ['-timestamp'],
},
bases=(models.Model,),

25
26
27
28
29

),

30
31

For a migration to work, the migration file must contain a class called Migration() that
inherits from django.db.migrations.Migration. This is the class that the migration
framework looks for and executes when you ask it to run migrations - which we will do later.
The Migration() class contains two main lists, dependencies and operations.

Migration dependencies
Dependencies is a list of migrations that must be run prior to the current migration being
run. In the case above, nothing has to run prior so there are no dependencies. But if you
have foreign key relationships then you will have to ensure a model is created before you can
add a foreign key to it.
To see that lets create migrations for our main app:
1

$ ./manage.py makemigrations main

You should see something like this:

1
2
3
4
5
6

Migrations for 'main':

0001_initial.py:
- Create model Announcement
- Create model Badge
- Create model MarketingItem
- Create model StatusReport

After running that have a look at main/migrations/0001_initial.py. In the dependency

list you will see:
1
2
3

dependencies = [
('payments', '__first__'),
]
268

The dependency above says that the migrations for the payments app must be run before the
current migration. You might be wondering, How does Django know I have a dependency
on payments when I only ran makemigrations for main?:
Short Answer: Magic.
Slightly Longer Answer: makmigrations look at things like ForeignKey Fields to
determine dependencies (more on this later).
You can also have a dependency on a specific file.
To see an example, lets initialize another migration:
1

$ ./manage.py makemigrations payments

You should see something like:

1
2
3
4

Migrations for 'payments':

0001_initial.py:
- Create model UnpaidUsers
- Create model User

Check out the dependency in the payments/migrations/0001_initial.py migration

file.
1
2
3

dependencies = [
('main', '0001_initial'),
]

This essentially means that it depends on the 0001_inital.py file in the main app to run first.
This functionality provides a lot of flexibility, as you can accommodate foreign keys that depend upon models from different apps.
Dependencies can also be combined (its just a list after all) so you can have multiple dependencies - which means that the numbering of the migrations (usually 0001, 0002, 0003, )
doesnt strictly have to be in the order they are applied. You can add any dependency you
want and thus control the order without having to re-number all the migrations.

Migration operations
The second list in the Migration() class is the operations list. This is a list of operations
to be applied as part of the migration. Generally the operations fall under one of the following
types:
269

 CreateModel: You guessed it: This creates a new model. See the migration above for
an example.
DeleteModel: removes a table from the database; just pass in the name of the model.
RenameModel: Given the old_name and new_name, this renames the model.
AlterModelTable: changes the name of the table associated with a model. Same as
the db_table option.
AlterUniqueTogether: changes unique constraints.
AlteIndexTogether: changes the set of custom indexes for the model.
AddField: Just like it sounds. Here is an example (and a preview of things to come
dun dun dun dun):
1
2
3
4
5

migrations.AddField(
model_name='user',
name='badges',
field=models.ManyToManyField(to='main.Badge')
),

RemoveField: We dont want that field anymore so just drop it.

RenameField: Given a model_name, an old_name and a new_name, this changes
the field with old_name to new_name.
There are also a few special operations:
RunSQL: This allows you to pass in raw SQL and execute it as part of your model.
RunPython: passes in a callable to be executed; useful for things like data loading as
part of the migration.
You can even write your own operations. Generally when you run makemigrations, Django
will create the necessary migrations with the appropriate dependencies and operations that
you need. However, understanding the migration files themselves and how they work give
you more flexibility.

270

When Migrations Dont Work

One of the most frequent causes for migrations not working correctly is a circular dependency
error.
Lets now try applying the migrations:
1

$ ./manage.py migrate

You should see the follow error:

1
2

raise CircularDependencyError(path[path.index(start):] + [start])

django.db.migrations.graph.CircularDependencyError: [('main',
'0001_initial'), ('payments', '0001_initial'), ('main',
'0001_initial')]

And, yes: This is a circular dependency error.

Lets look at whats happening.

Models
Our payments.models.User looks like this:
1
2
3
4
5

6
7
8
9
10

class User(AbstractBaseUser):
name = models.CharField(max_length=255)
email = models.CharField(max_length=255, unique=True)
#password field defined in base class
last_4_digits = models.CharField(max_length=4, blank=True,
null=True)
stripe_id = models.CharField(max_length=255)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
rank = models.CharField(max_length=50, default="Padwan")
badges = models.ManyToManyField(Badge)

Notice the ManyToMayField called badges at the end, which references main.models.Badge:
1
2
3
4

class Badge(models.Model):
img = models.CharField(max_length=255)
name = models.CharField(max_length=100)
desc = models.TextField()

Okay. So far there are no issues, but we have another model to deal with:
271

1
2
3
4

class StatusReport(models.Model):
user = models.ForeignKey('payments.User')
when = models.DateTimeField(blank=True)
status = models.CharField(max_length=200)

Oops! We now have payments.models depending on main.models and main.models depending on payments.models. Thats a problem. In the code, we solved this already by not
importing payments.models and instead using the line:
1

user = models.ForeignKey('payments.User')

While that trick works at the application-level, it doesnt work when we try to apply migrations to the database.

Migrations
How about the migration files? Again, take a look at the dependencies:

main.migrations.001_initial:
1
2
3

dependencies = [
('payments', '__first__'),
]

payments.migrations.001_initial:
1
2
3

dependencies = [
('main', '0001_initial'),
]

So, the latter migration depends upon the main migration running first, and thus - we
have a circular reference. Remember how we talked about makemigrations looking at
ForeignKey fields to create dependencies? Thats exactly what happened to us here.

The fix
When I was an intern in college (my first real development job) my dev manager said something to me which I will never forget: You cant really understand code unless you can write
it yourself.
This was after a copy and paste job I did crashed the system.
So lets write a migration from scratch.
272

We are going to remove the StatusReport model from main/migrations/0001_inital.py

and add it to a new migration.
Create a new file called main/migrations/0002_statusreport.py:
1
2

# -*- coding: utf-8 -*from __future__ import unicode_literals

3
4

from django.db import models, migrations

5
6
7

class Migration(migrations.Migration):

8
9
10
11
12

dependencies = [
('payments', '__first__'),
('main', '0001_initial'),
]

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

operations = [
migrations.CreateModel(
name='StatusReport',
fields=[
('id', models.AutoField(
primary_key=True, auto_created=True,
verbose_name='ID', serialize=False)),
('when', models.DateTimeField(blank=True)),
('status', models.CharField(max_length=200)),
('user', models.ForeignKey(to='payments.User')),
],
options={
},
bases=(models.Model,),
),
]

Notice how this migration depends upon both main.0001_initial and payments.__first__.
This means that the payments.user model will already be created before this migration
runs and thus the user foreign key will be created successfully.
Dont forget to:

1. Modify main/migrations/0001_inital.py to remove the payments dependency

273

on payments (so it has no dependencies):

2. Remove the CreateModel() for StatusReports.
And now we have resolved the circular dependency by splitting up the migration into two
parts. This is a common pattern to use when fixing circular dependencies, so keep it in mind
if you run into similar issues.
Go ahead and run the migrations now to ensure everything is working correctly:
1

$ ./manage.py migrate

You should now have your database in sync with your migration files!

Timezone support
If you have timezone support enabled in your settings.py file, you will likely get an error,
when running migrations that says something to the effect of: native datetime
while time zone support is active. The fix is to use django.utils.timezone
instead of datetime.
Youll have to update contact.models.ContactForm and
payments.models.UnpaidUsers. Just replace datetime.datetime with django.utils.timezone.
You should update both the models and the migrations files.
More info on timezone support can be found in the Django docs.

Warning Message on Migrate

There is another datetime related issue that you may run into with migrations. You may see
the following warning message when you run migrate:
1

Your models have changes that are not yet reflected in a migration,
and so won't be applied.

This has to do with the same datetime fields we just discussed. Since we are defaulting the
value of the datetime field to now() Django will see the default value as always being different
and thus it will give you the warning above. While this is supposed to be a helpful warning
message to remind you to keep your migrations up-to-date with your models.py in this case
it is a minor annoyance. There is nothing to be done here, just ignore the warning.

274

Data Migrations
Migrations are mainly for keeping the data model of you database up-to-date, but a database
is more than just a data model. Most notably, its also a large collection of data. So any discussion of database migrations wouldnt be complete without also talking about data migrations.
Data migrations are used in a number of scenarios. Two very popular ones are:
1. Loading system data: When you would like to load system data that your application depends upon being present to operate successfully.
2. Migrating existing data: When a change to a data model forces the need to change
the existing data.
Do note that loading dummy data for testing is not in the above list. You could
use migrations to do that, but migrations are often run on production servers,
so you probably dont want to be creating a bunch of dummy test data on your
production server. (More on this later)
Lets look at examples of each

Loading System Data

As an example of creating some system data, lets define a system user that always needs to
exist. In payments/migrations, add a new file called 0003_initial_data.py.
Start by adding the following code:
1
2

# -*- coding: utf-8 -*from __future__ import unicode_literals

3
4

from django.db import migrations

5
6
7

class Migration(migrations.Migration):

8
9
10
11

dependencies = [
('payments', '0001_initial'),
]

12
13

operations = [
275

migrations.RunPython(create_default_user)

14
15

Like any other migration, we create a class called Migration(), set its dependency, and then
define the operation.
For Data Migrations we can use the RunPython() operation, which just accepts a callable
and calls that function. So, we need to add in the create_default_user() function:
1
2
3
4
5

# -*- coding: utf-8 -*from __future__ import unicode_literals

from django.db import migrations
from payments.models import User
from django.contrib.auth.hashers import make_password

6
7
8
9
10
11
12
13
14

def create_default_user(apps, schema_editor):

new_user = apps.get_model("payments", "User")
try:
vader = new_user.objects.get(email="[email protected]")
vader.delete()
except new_user.DoesNotExist:
pass

15
16
17
18
19

u = new_user(
name='vader', email="[email protected]",
last_4_digits="1234", password= make_password("darkside")
).save()

20
21
22

class Migration(migrations.Migration):

23
24
25
26

dependencies = [
('payments', '0001_initial'),
]

27
28
29
30

operations = [
migrations.RunPython(create_default_user)
]

This function just adds the new user. A couple of things are worth noting:

276

1. We dont use the User object but rather grab the model from the App Repository. By
doing this, migrations will return us the historic version of the model. This is important as the fields in the model may have changed. Grabbing the model from the App
Repository will ensure we get the correct version of the model.
2. Since we are grabbing from a historic app repository it is likely we dont have access
to out custom defined functions such as user.create. So we saved the user without
using the user.create function.
3. There are cases where you may want to rerun all your migrations or perhaps there is
existing data in the database before you run migrations, so weve added a check to clear
out any conflicting data before we create the new user:
1
2
3
4
5

try:
vader = new_user.objects.get(email="[email protected]")
vader.delete()
except new_user.DoesNotExist:
pass

This will prevent the annoying duplicate primary key error that you would get if you somehow
ran this migration twice.
Go ahead an apply the migration:
1

$ ./manage.py migrate

You should see something like:

1
2
3

4
5
6
7
8
9

Operations to perform:
Synchronize unmigrated apps: contact, rest_framework
Apply all migrations: admin, sessions, payments, sites,
flatpages, contenttypes, auth, main
Synchronizing apps without migrations:
Creating tables...
Installing custom SQL...
Installing indexes...
Running migrations:
Applying payments.0003_initial_data... OK

So we have two main use cases for loading data:

1. To load system data, that is data that needs to be in the database for the application to
work correctly.
2. To load data that is necessary / helpful for our tests to run.
277

While both use cases can be accomplished with migrations the second use case, loading test
data, should be though of as something separate from migrations. You can continue to use
fixtures for loading test data, or better yet just create the data you need in the testcase itself.

Aside: How do migrations know what to migrate?

Lets digress for a bit and look at what happens when you run migrations multiple times. The
standard behavior is that Django will never run a migration more than once on the same
database. This is managed by a table called django_migrations that is created in your
database the first time you apply migrations. For each migration that is ran or faked, a new
row is inserted into the table.
Lets look at our table. Open the Postgres Shell, connect to the database, and then run:
1

SELECT * FROM django_migrations;

You should see something similar to:

1
2
3
4
5
6

id |
app
|
name
|
date applied
----+-----------+--------------------+-----------------------------1 | main
| 0001_initial
| 2014-09-20 23:51:38.499414-05
2 | payments | 0001_initial
| 2014-09-20 23:51:38.600185-05
4 | main
| 0002_statusreport | 2014-09-20 23:52:33.808006-05
5 | payments | 0003_initial_data | 2014-09-21 11:36:12.702975-05

The next time migrations are run, it will skip the migration files listed in the database here.
This means that even if you change the migration file manually, it will be skipped if there is an
entry for it in the database. This makes sense as you generally dont want to run migrations
twice. But if for whatever reason you do, one way to get it to run again is to first delete the
corresponding row from the database. In the case of schema migrations, though, Django will
first check the database structure, and if it is the same as the migration (i.e. the migration
doesnt apply any new changes) then the migration will be faked meaning not really ran.
Conversely, if you want to undo all the migrations for a particular app, you can migrate to
a special migration called zero. For example if you type:
1

$ ./manage.py migrate payments zero

It will undo (reverse) all the migrations for the payments app. You dont have to use zero; you
can use any arbitrary migration (like ./manage.py migrate payments 0001_initial),
and if that migration is in the past then the database will be rolled back to the state of that
migration, or rolled forward if the migration hasnt yet been run. Pretty powerful stuff!
Note: This doesnt apply to data migrations.

278

Migrating existing data

Coming back to data migrations, the other reason why you might use data migrations is when
you actually need to migrate the data - e.g., change how the data is stored.
The Django docs have a good example of this called combine_names, which uses a data migration to combine the first and last name into one column, name. Most likely this migration
would come just after the migration that created the new name column, but before the migration that deleted the first_name and last_name columns. The migration is called the
same way as in our previous example using create_default_user.
Lets look at the actual combine_names function that is demonstrated in the documentation:
1
2

# -*- coding: utf-8 -*from django.db import models, migrations

3
4
5
6

7
8
9

10

def combine_names(apps, schema_editor):

# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical
version.
Person = apps.get_model("yourappname", "Person")
for person in Person.objects.all():
person.name = "%s %s" % (person.first_name,
person.last_name)
person.save()

11
12

class Migration(migrations.Migration):

13
14
15
16

dependencies = [
('yourappname', '0001_initial'),
]

17
18
19
20

operations = [
migrations.RunPython(combine_names),
]

When you create a Python function to be called by the RunPython migration, it must accept
two arguments. The first is apps, which is of type django.apps.registry.Apps and gives
you access to the historical models/migrations. In other words, this is a model that has the
state as defined in the previous migration (which could be vastly different to the current state
of the model). By state we are mainly referring to the fields associated with the model.
The second argument is the schema_editor for changing the schema, which should not be
279

necessary very often when migrating data, because youre not changing the schema, just the
data.
In the example we call apps.get_model, which gives us that historical model. Then we loop
through all rows in the model and combine the first_name and last_name into a single
name and save the row. Thats it. our migration is done. Its actually pretty straight-forward
to write the migration once you get the hang of it.
The hardest part is remembering the structure of a migrations file, but Django Migrations
has got that covered!. From the command line, if you run1

$ ./manage.py makemigrations --empty yourappname

-this will create an empty migration file in the appropriate app. Django will also suggest a
name for the migration (which you are free to change), and it will add your dependencies
automatically, so you can just start writing your operations.

Migrations and Fixtures

If you recall from earlier chapters we created a few fixtures to load some initial data:
main/fixtures/inital_data.json
payments/fixtures/initial_data.json
If you need to load system data you should load it in a migration. This is probably a good use
case for the MarketingItems data (from the exercises at the end of the Bootstrap chapter)
because we want that to display our web page.
That said, do not use any fixtures named inital_data.json, as it will cause problems. So, lets
rename the fixtures to:
main/fixtures/system_data.json
payments/fixtures/system_data.json
To load them you need to use the django command loaddata:

$ ./manage.py loaddata main/fixtures/system_data.json

$ ./manage.py loaddata payments/fixtures/system_data.json

$ ./manage.py loaddata dummy_data

Or:

This will load the data into your database, so you can call it as needed.
280

Conclusion
Weve covered the most common scenarios youll encounter when using migrations. There
are plenty more, and if youre curious and really want to dive into migrations, the best place
to go (other than the code itself) is the official docs. Its the most up-to-date and does a pretty
good job of explaining how things work.
Remember that in the general case, you are dealing with either:
1. Schema Migrations - a change to the structure of the database or tables with no
change to the data. This is the most common type, and Django can generally create
these migrations for you automatically.
2. Data Migrations - a change to the data, or loading new data. Django cannot generate
these for you. They must be created manually using the RunPython migration.
So pick the migration that is correct for you, run makemigrations and then just be sure to
update your migration files every time you update your model - and thats more or less it.
That will allow you to keep your migrations stored with your code in git and ensure that you
can update your database structure without having to lose data.
Happy migrating!
NOTE: We will be building on the migrations that we created in this chapter,
and problems will arise if your migration files do not match exactly (including
the name of the files) with the migration files from from the repo. Compare your
code/migration files with code/migration files from the repo. Fix any discrepancies. Thanks!

281

Exercises
1. At this point if you drop your database, run migrations and then run the tests you will
have a failing test because there are no MarketItems in the database. For testing you
have two options:
Load the data in the test (or use a fixture).
Load the data by using a datamigration.
The preferred option for this case is to create a data migration to load the MarketingItems.
Can you explain why?. Create the migration.
NOTE: For some fun (and a somewhat ugly hack) we can add a line to create
the data to test.main.testMainPageView.setUpClass. See if you can
figure out what this line does and why adding it will fix your test:
1
2

from main.models import MarketingItems

[MarketingItem(**m.__dict__).save() for m in market_items]

2. We have a new requirement for two-factor authentication. Add a new field to the
user model called second_factor. Run ./manage.py makemigration payments.
What did it create? Can you explain what is going on in each line of the migration?
Now run ./manage.py migrate and check the database to see the change that has
been made. What do you see in the database? Now assume management comes back
and says two-factor is too complex for users; we dont want to add it after all. List two
different ways you can remove the newly added field using migrations.
3. Lets pretend that MEC has been bought by a big corporation - well call it BIGCO.
BIGCO loves making things complicated. They say that all users must have a
bigCoID, and that ID has to follow a certain formula. The ID should look like this:

<first_two_digits_in_name><1-digit-Rank_code><sign-up-date><runningNumber>
1-digit-Rank_code = Y for youngling, P for padwan, J for Jedi
sign-up-date is in the format mmddyyyy
Now create the new field and a migration for the field, then manually write a data
migration to populate the new field with the data from the pre-existing users.

282

Chapter 13
AngularJS Primer

Figure 13.1: Angular

In this chapter, we will dive into client-side programming and specifically talk about the popular client-side framework AngularJS. Referred to as a Superheroic JavaScript MVW Framework and as what HTML would have been, had it been designed for building web-apps, Angular is perfect for creating dynamic web apps that function like desktop apps. It also plays
well with Django and the Django templating system, and it even shares some design principles with Django. Thus, of all the JavaScript frameworks out there, Angular is probably the
best suited to use in conjunction with Django.
Django strongly encourages separation of concerns by following an MVC (Model-ViewController) architecture. Angular follows a similar MVC pattern, although the Angular folks
call it MVW (Model View whatever works for you), which means that you have some freedom
to use it in a way that makes sense to you. Since youre already familiar with Djangos MVC
pattern, you can keep the same pattern in mind when working with Angular since MVW is a
subset of MVC.
This is one of the main reasons we have chosen to use Angular as the JavaScript framework in
this course: Since its overall design is very similar to Django, it allows developers to keep the

283

same conceptual architecture in mind when working on either the front-end with Angular or
the back-end with Django.

284

What we are covering

Angular is a large framework. It has many many capabilities, and we cannot cover all of it in
a chapter. The focus of this chapter is to expose you to enough of Angular that you can build
some useful front-end magic for your application. Well also detail how best to integrate
Angular and Django and the trade-offs that should be considered when doing so.
In order to achieve this, lets start first with Angular and discuss some of the basic components
of the framework before adding Django into the mix.
In particular, well cover:
1.
2.
3.
4.

Directives
Angular Models
Data/Expression Bindings
Angular Controllers

Lets start off with user story 4 from the Membership Site Chapter:
US4: User Polls
Determining the truth of the galaxy and balancing the force are both very important goals
at MEC. As such, MEC should provide the functionality to poll padwans and correlate the
results in order to best determine or predict the answers to important topics of the Star Wars
galaxy. This includes questions like Kit Fisto vs Aayla Secura, who has the best dreadlocks? Or
who would win in a fight, C3PO or R2-D2? Results should also be displayed to the padwans
so that all shall know the truth.

Again, lets implement this first in Angular before integrating it into our Django app.
SEE ALSO: Since we cannot possibly cover all of the fundamentals in this chapter, please review this introductory Angular tutorial before moving on.

285

Angular for Pony Fliers (aka Django Devs)

Hello World just doesnt make much sense in the Star Wars Galaxy, so lets start with Hello
Universe (didnt see that one coming, did you?). So a minimal Angular app would look like
this:

index.html
1
2
3
4
5
6
7
8
9
10
11
12
13

14
15
16
17

18
19
20

21
22
23

<!doctype html>
<html lang="en" ng-app=''>
<head>
<meta charset="UTF-8">
<title>{{ msg }} Universe</title>
<!-- styles -->
<link href="http://netdna.bootstrapcdn.com/bootswatch/3.1.1/
yeti/bootstrap.min.css" rel="stylesheet" media="screen">
</head>
<body>
<div class="container">
<br><br>
<p>What say you, padwan: <input type="text" ng-model="msg"
ng-init="msg='Hello'"></p>
<p>{{ msg }} Universe!</p>
</div>
<!-- scripts -->
<script
src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://netdna.bootstrapcdn.com/bootstrap/3.1.1/js/
bootstrap.min.js"></script>
<script
src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.16/
angular.min.js" type="text/javascript"></script>
</body>
</html>

Save this file as index.html in a new directory called angular_test outside of the Django
Project.
If you open the above web page in your browser then you will see that whatever you type in
the text box is displayed in the paragraph below it. This is an example of data binding in
286

Angular.
Before we look at the code in greater detail, lets look at Angular directives, which power
Angular.

Angular Directives
When a HTML page is rendered, Angular searches through the DOM looking for directives
and then executes the associated JavaScript. In general, all of these directives start with ng
(which is supposed to stand for Angular). You can also prefix them with data- if you are
concerned about strict w3c validation. Use data-ng- in place of ng-, in other words.
There are a number of built-in directives, like:
ng-click handles user actions, specifying a Javascript function to call when a user
clicks something.
ng-hide controls when an HTML element is visible or hidden, based on a conditional.
These are just a few examples. As we go through this chapter well see many uses of directives.

Back to the code

Lets look at the example code:
Line 18: You must include the AngularJS file. This is step one for any Angular app and
is of course required. You can either downloaded the file directly from Angularjs.org
or use a CDN.
Line 2 ng-app is a directive that is generally added to the main HTML tag or any div
tag and tells Angular to startup (also known as bootstrapping, but to avoid confusion
with the CSS framework Bootstrap well refer to it as Angular initialization). Only Angular code that is a child of the element containing ng-app will be evaluated by the
framework. In our case, since ng-app is on the HTML tag, all elements on the page
are evaluated by Angular.
Line 13: ng-model and ng-init are both directives. In this case the directives create
a model called msg and initialize the initial value (or state) of msg to Hello. Thus, when
the page initially loads, the msg model will have the value of Hello. So what exactly is
a model? Lets digress

287

Angular Models
A model in Angular plays a similar role to a context variable in a Django template. If you
think about a simple Django template1

<p> {{ msg }} Universe! </p>

-Django allows you to pass in the value of msg from your view function. Assuming the above
template was called index.html, your view function might look like this:
1
2

def index():
return render_to_response(index.html, {"msg": Hello})

This will create the index template with a context variable msg that has the value of Hello,
which is exactly what Angular is doing directly in the template, on the client-side. The
difference is that with Django (in most cases) you can only change the value of your context
variable by requesting a new value from the server, whereas with Angular you can change
the value of the model anytime on the client-side by executing some JavaScript code. This is
ultimately how we are going to be able to update the page without a refresh (our main goal),
because we can just update the model directly on the client-side.

Return of the code

Line 5 and 14: An Angular model isnt much use if you cant update the HTML code
as well. Just like Django, Angular uses templates. The Angular template consists of
Angular directives and data/expression bindings (as well as filters and form controls,
which we havent covered yet). Again, when you initialize Angular with the ng-app
directive, the Angular template is evaluated. We know that directives are evaluated
during Angular initialization. Expression bindings are also handled at that time. Expression Bindings use the same markup. In the case of line 5 and 13, the expression
bindings {{ msg }} are asking Angular to substitute the value of the msg model in
place of the binding. Again this conceptually works the same way as templates do in
Django - it just happens on the client-side. That said, Angular expression bindings are
not limited to models (i.e., data binding); you can also use actual expressions:
{{ 1 + 2 }}: any mathematical operation will work here
{{ model.property }}: you can access properties of the model
{{ 2 * 4 | currency }}: an example of an Angular filter, which outputs the results of the expression as a currency

288

Angular vs Django - re: template tags

The following table below summarizes the similarities between Angular and Django with regards to the topics we have covered thus far:
Features

Angular

Django

Expression Binding {{ 1+2 }}

Data Binding
{{ msg }}
Filtering
{{ 1+2 | currency }}
Directives
ng-app
Models
ng-model="msg"

Doesnt evaluate expressions

{{ msg }}
{{ val | currency }}
{% marketing__circle_item %}
{'msg': 'hello'}

So 1. Filters in Angular can be applied to either expressions (1 + 2) or data (variable_name),

while in Django they can only be applied to context variables (variable_name).
2. Directives are defined with HTML attributes in Angular and with template tags in
Django.
3. Models are passed into the template from view functions: render_to_response(index.html

{'msg': 'hello'})
Hopefully this table will serve as a quick guide to remember what the various functionality in
Angular does.
There are some rather sizable differences between the two frameworks as well. The main
difference is the client-side nature of Angular versus the server-side nature of Django. The
client-side nature of Angular leads to one of its most prominent features, which is marketed
as two-way data binding.

289

Angular Data Binding Explained

To understand Angulars two way data-binding, its helpful to first look at one-way data binding. Well use a Django specific example but most one-way data binding systems work in a
similar way.

One-way data binding

Above is an image depicting one-way data binding as it happens in Django. The process
begins with an HTTP request. In the case of a GET request, sent from the end user to the
server, Django handles the request by first routing it to the appropriate controller with urls.py
and *views.py, respectively. The view then handles the request, possibly grabbing some data
from the database (and perhaps manipulating it), and sends a response that generally looks
something like this:
1

return render_to_response('index.html',
{'marketing_items':market_items})

Here, we pass in two arguments to the render_to_response function - the first being the
template name. Meanwhile, the second is technically the template context ; however, since it
generally consists of a model or set of models, well refer to it as a model for this discussion.
Djangos template processor parses the template substituting the data binding expressions
and template tags with the appropriate data, then produces a view, which is finally returned
to the user to see.
At that point, the data binding process is complete. If the user changes data on the web page
(e.g., the view), those changes wont be reflected in the model unless the change is sent back
to the server (usually with a POST request) and the data binding process starts all over again.
Since data is bound only once (in the response), this technique is referred to as one-way data
binding, which works well for a number of use cases.

Two-way data binding

Working within the Django paradigm, to keep the data on the view in sync with the data in
the model (when you have an interactive page) can be difficult and would require a number
of requests back and forth to and from the server.
However, if we remove the combining of the model and template from the server-side and
place it on the client-side, then we can significantly decrease the number of required calls to

290

Figure 13.2: one way data binding

291

and from the server in order to keep the model and view in sync. This results in a site that
appears much more responsive and functional to the end user.
Angulars two way data-binding is depicted in the image above. The process starts with a user
action (rather than a request) - i.e., typing a value into a text box - then the JavaScript event
handler updates the associated model and triggers Angulars template compiler to rebuild
the view with the newly updated data from the model.
In a similar fashion, if you write JavaScript code that changes the model directly, Angular
will again fire off the template compiler to ensure the view stays up-to-date with the model.
Regardless of whether you make a change to the view or the model, Angular ensures the two are kept in sync.

This is two-way data binding in action, which results in keeping everything (model and view)
in sync. Since its all done on the client-side, its also done very quickly, without requiring
multiple trips to the server or a page refresh.
To be completely fair, you dont need to use Angular to keep the model and view up-to-date.
Since Angular uses vanilla JavaScript behind the scenes, you could use AJAX to achieve similar results. Angular makes the process much, much easier, allowing you to do a lot with little
code.

292

Figure 13.3: two way data binding

293

Building User Polls

With that brief introduction done, theres quite a lot you can already accomplish in Angular.
Lets take it one step further and build out the User Polls functionality. Polls? That should
sound familiar, as you either have gone through the official Django tutorial or at least heard
of it. Lets do the same type of poll application in Angular.

Quick start
Well start with just the poll in a standalone HTML file. Again, save this file in the angular_test directory as polls.html.
First, lets create some basic markup for the poll:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21

22
23
24

<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Poll: Who's the most powerful Jedi?</title>
<!-- styles -->
<link href="http://netdna.bootstrapcdn.com/bootswatch/3.1.1/yeti/
bootstrap.min.css" rel="stylesheet" media="screen">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-md-8">
<h1>Poll: Who's the most powerful Jedi?</h1>
<br>
<span class="glyphicon glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">40%</span>
<div class="progress">
<div class="progress-bar progress-bar-danger"
role="progressbar" aria-value="40"
aria-valuemin="0" aria-valuemax="100" style="width:
40%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"></span>
294

25
26
27
28

29

30
31
32
33
34
35
36

37

38
39
40
41
42
43
44

45

46
47
48
49
50
51
52

53

54
55
56

<strong>Qui-Gon Jinn</strong>
<span class="pull-right">30%</span>
<div class="progress">
<div class="progress-bar progress-bar-info"
role="progressbar" aria-value="30"
aria-valuemin="0" aria-valuemax="100" style="width:
30%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"></span>
<strong>Obi-Wan Kenobi</strong>
<span class="pull-right">10%</span>
<div class="progress">
<div class="progress-bar progress-bar-warning"
role="progressbar" aria-value="10"
aria-valuemin="0" aria-valuemax="100" style="width:
10%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"></span>
<strong>Luke Sykwalker</strong>
<span class="pull-right">5%</span>
<div class="progress">
<div class="progress-bar progress-bar-success"
role="progressbar" aria-value="5"
aria-valuemin="0" aria-valuemax="100" style="width:
5%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"></span>
<strong>Me... of course</strong>
<span class="pull-right">15%</span>
<div class="progress progress-striped active">
<div class="progress-bar" role="progressbar"
aria-valuenow="15"
aria-valuemin="0" aria-valuemax="100" style="width:
15%;">
<span class="sr-only">15% Complete</span>
</div>
</div>
295

57
58
59
60
61

62
63
64

65
66
67

</div>
</div>
</div>
<!-- scripts -->
<script
src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://netdna.bootstrapcdn.com/bootstrap/3.1.1/js/
bootstrap.min.js"></script>
<script
src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.16/
angular.min.js" type="text/javascript"></script>
</body>
</html>

This snippet above relies on Bootstrap to create a simple list of choices each with a + next to
them and a progress bar below. Weve put in some default values just so you can see what it
might look like after people have voted. The screenshot is below.

Figure 13.4: User Poll

As for the user functionality, when the + is clicked, the progress bar below the Jedi should
increment. Lets add that now. In other words, lets add Angular!

Bootstrap Angular
How about we start with Yoda. Update the top of the file like so:
296

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

17
18
19
20

21

22
23

<!doctype html>
<html lang="en" ng-app=''>
<head>
<meta charset="UTF-8">
<title>Poll: Who's the most powerful Jedi?</title>
<!-- styles -->
<link href="http://netdna.bootstrapcdn.com/bootswatch/3.1.1/yeti/
bootstrap.min.css" rel="stylesheet" media="screen">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-md-8">
<h1>Poll: Who's the most powerful Jedi?</h1>
<br>
<span ng-click='votes_for_yoda = votes_for_yoda + 1'
ng-init="votes_for_yoda=0" class="glyphicon
glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">{{ votes_for_yoda }}</span>
<div class="progress">
<div class="progress-bar progress-bar-danger"
role="progressbar" aria-value="{{ votes_for_yoda }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_yoda }}%;">
</div>
</div>
Line 2: ng-app will bootstrap in Angular and get everything working.
Line 16: ng-click='votes_for_yoda = votes_for_yoda + 1' - this Angular
directive creates a click handler that increments the model votes_for_yoda by 1. It
will be called each time the user clicks on the + span.
Line 16: ng-init="votes_for_yoda=0" - this Angular directive initializes the
value of votes_for_yoda to 0.
Line 18, 20, and 21: the expression {{ votes_for_yoda }} uses two-way data
binding to keep the values in sync with the votes_for_yoda model. Since these values
control the progress bar, the bar will now grow each time we click on the plus.

Once this is working, update the code for all the remaining Jedis.
297

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

17
18
19
20

21

22
23
24

25
26
27
28

29

30
31

<!doctype html>
<html lang="en" ng-app=''>
<head>
<meta charset="UTF-8">
<title>Poll: Who's the most powerful Jedi?</title>
<!-- styles -->
<link href="http://netdna.bootstrapcdn.com/bootswatch/3.1.1/yeti/
bootstrap.min.css" rel="stylesheet" media="screen">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-md-8">
<h1>Poll: Who's the most powerful Jedi?</h1>
<br>
<span ng-click='votes_for_yoda = votes_for_yoda + 1'
ng-init="votes_for_yoda=0" class="glyphicon
glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">{{ votes_for_yoda }}</span>
<div class="progress">
<div class="progress-bar progress-bar-danger"
role="progressbar" aria-value="{{ votes_for_yoda }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_yoda }}%;">
</div>
</div>
<span ng-click='votes_for_qui = votes_for_qui + 1'
ng-init="votes_for_qui=0" class="glyphicon
glyphicon-plus"></span>
<strong>Qui-Gon Jinn</strong>
<span class="pull-right">{{ votes_for_qui }}</span>
<div class="progress">
<div class="progress-bar progress-bar-info"
role="progressbar" aria-value="{{ votes_for_qui }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_qui }}%;">
</div>
</div>

298

32

33
34
35
36

37

38
39
40

41
42
43
44

45

46
47
48

49
50
51
52

53

54
55
56
57
58
59

<span ng-click='votes_for_obi = votes_for_obi + 1'

ng-init="votes_for_obi=0" class="glyphicon
glyphicon-plus"></span>
<strong>Obi-Wan Kenobi</strong>
<span class="pull-right">{{ votes_for_obi }}</span>
<div class="progress">
<div class="progress-bar progress-bar-warning"
role="progressbar" aria-value="{{ votes_for_obi }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_obi }}%;">
</div>
</div>
<span ng-click='votes_for_luke = votes_for_luke + 1'
ng-init="votes_for_luke=0" class="glyphicon
glyphicon-plus"></span>
<strong>Luke Sykwalker</strong>
<span class="pull-right">{{ votes_for_luke }}</span>
<div class="progress">
<div class="progress-bar progress-bar-success"
role="progressbar" aria-value="{{ votes_for_luke }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_luke }}%;">
</div>
</div>
<span ng-click='votes_for_me = votes_for_me + 1'
ng-init="votes_for_me=0" class="glyphicon
glyphicon-plus"></span>
<strong>Me... of course</strong>
<span class="pull-right">{{ votes_for_me }}</span>
<div class="progress progress-striped active">
<div class="progress-bar" role="progressbar"
aria-value="{{ votes_for_me }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_me }}%;">
</div>
</div>
</div>
</div>
</div>
<!-- scripts -->
299

60

61
62
63

64
65
66

<script
src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://netdna.bootstrapcdn.com/bootstrap/3.1.1/js/
bootstrap.min.js"></script>
<script
src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.16/
angular.min.js" type="text/javascript"></script>
</body>
</html>

With that, you have a simple user poll that uses progress bars to show which option has the
most votes. This will update automatically without making any calls to the back-end. This
however puts a lot of logic into our actual HTML and can thus make maintenance a bit difficult. Ideally we would like a separate place to keep our logic.
We can tackle this problem by using an Angular controller.

Angular Controller
We havent talked about controllers in Angular yet, but for now you can think of an Angular
controller as your views.py file that handles the HTTP requests from the browser. Of course
with Angular the requests are coming from individual client actions, but the two more or less
serve the same purpose.
If we modify the above code to use a controller, it might look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<!doctype html>
<html lang="en" ng-app="mecApp">
<head>
<meta charset="UTF-8">
<title>Poll: Who's the most powerful Jedi?</title>
<!-- styles -->
<link href="http://netdna.bootstrapcdn.com/bootswatch/3.1.1/yeti/
bootstrap.min.css" rel="stylesheet" media="screen">
</head>
<body>
<div class="container" ng-controller="UserPollCtrl">
<div class="row">
<div class="col-md-8">
<h1>Poll: Who's the most powerful Jedi?</h1>
<br>
300

16

17
18
19
20

21

22
23
24

25
26
27
28

29

30
31
32

33
34
35
36

37

38
39
40

41
42
43
44

<span ng-click="vote('votes_for_yoda')" class="glyphicon

glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">{{ votes_for_yoda }}</span>
<div class="progress">
<div class="progress-bar progress-bar-danger"
role="progressbar" aria-value="{{ votes_for_yoda }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_yoda }}%;">
</div>
</div>
<span ng-click="vote('votes_for_qui')" class="glyphicon
glyphicon-plus"></span>
<strong>Qui-Gon Jinn</strong>
<span class="pull-right">{{ votes_for_qui }}</span>
<div class="progress">
<div class="progress-bar progress-bar-info"
role="progressbar" aria-value="{{ votes_for_qui }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_qui }}%;">
</div>
</div>
<span ng-click="vote('votes_for_obi')" class="glyphicon
glyphicon-plus"></span>
<strong>Obi-Wan Kenobi</strong>
<span class="pull-right">{{ votes_for_obi }}</span>
<div class="progress">
<div class="progress-bar progress-bar-warning"
role="progressbar" aria-value="{{ votes_for_obi }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_obi }}%;">
</div>
</div>
<span ng-click="vote('votes_for_luke')" class="glyphicon
glyphicon-plus"></span>
<strong>Luke Sykwalker</strong>
<span class="pull-right">{{ votes_for_luke }}</span>
<div class="progress">
<div class="progress-bar progress-bar-success"
role="progressbar" aria-value="{{ votes_for_luke }}"
301

45

46
47
48

49
50
51
52

53

54
55
56
57
58
59
60

61
62
63

64
65
66

aria-valuemin="0" aria-valuemax="100" style="width:

{{ votes_for_luke }}%;">
</div>
</div>
<span ng-click="vote('votes_for_me')" class="glyphicon
glyphicon-plus"></span>
<strong>Me... of course</strong>
<span class="pull-right">{{ votes_for_me }}</span>
<div class="progress progress-striped active">
<div class="progress-bar" role="progressbar"
aria-value="{{ votes_for_me }}"
aria-valuemin="0" aria-valuemax="100" style="width:
{{ votes_for_me }}%;">
</div>
</div>
</div>
</div>
</div>
<!-- scripts -->
<script
src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://netdna.bootstrapcdn.com/bootstrap/3.1.1/js/
bootstrap.min.js"></script>
<script
src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.16/
angular.min.js" type="text/javascript"></script>
<script type="text/javascript">
var mecApp = angular.module('mecApp',[]);

67
68
69
70
71
72
73

mecApp.controller('UserPollCtrl', function($scope) {
$scope.votes_for_yoda = 80;
$scope.votes_for_qui = 30;
$scope.votes_for_obi = 20;
$scope.votes_for_luke = 10;
$scope.votes_for_me = 30;

74
75
76
77
78

$scope.vote = function(voteModel) {
$scope[voteModel] = $scope[voteModel] + 1;
};
});
302

79
80
81

</script>
</body>
</html>

Lets go through this one line at a time:

Line 2: <html lang="en" ng-app="mecApp">. Notice on this line we passed a
name into ng-app. Normally when we pass no name we are using the default Angular module. When we pass a name we can create a module with that name. This is
important because you can declare multiple ng-apps on the same page and have a different module for each. Also, this helps to prevent name classes in large applications
accidentally overriding each other.
Line 11: An Angular module, which in our case is called mecApp, can contain multiple controllers. When you define a controller on a div, it will apply to
all Angular directives / bindings declared inside that div. By using the directive:
ng-controller="UserPollCtrl" we are declaring a controller with the name
UserPollCtrl and assigning it to handle everything inside the div.
Line 16: ng-click="vote('votes_for_yoda')". Here we are using the
ng-click directive to set up a click handler like we did in the previous example.
The difference here is instead of putting the code to execute in-line, we are telling
the click handler to call a function called vote() and passing it the string value
votes_for_yoda. As you might have guessed, the vote() function will be defined
in the UserPollCtrl controller. Also note that we are passing in a string to vote and
not the actual votes_for_yoda model (this will become important later).
Lines 16, 18, 20: Just like in the previous example we are using the model
votes_for_yoda to update the values in the progress bar so that the bar length
increases as we click on the +.
Lines 65-79: Here we are inserting the actual JavaScript that defines the controller
and its behavior. Oftentimes in Angular apps, controllers are put in a separate file
called controllers.js, but to keep things simple we will just put everything in one
file.
Lets look at the controller in more detail. Here is the code again:
1
2

<script type="text/javascript">
var mecApp = angular.module('mecApp',[]);

3
4
5
6

mecApp.controller('UserPollCtrl', function($scope) {
$scope.votes_for_yoda = 80;
$scope.votes_for_qui = 30;
303

7
8
9

$scope.votes_for_obi = 20;
$scope.votes_for_luke = 10;
$scope.votes_for_me = 30;

10
11
12
13
14
15

$scope.vote = function(voteModel) {
$scope[voteModel] = $scope[voteModel] + 1;
};
});
</script>
Line 2: We declare an Angular module called mecApp, which is what we named the
module in our ng-app directive in the earlier example. If we dont declare a module
with the name used in ng-app then things wont work. We created our module with
two arguments - the first is the name of the module and the second is an array of the
names of all the modules this module depends on. Since we dont depend on any other
modules we just pass an empty array.
Line 4: Here we create our controller by calling the controller method on our module. We pass in two arguments - the first is the name of the controller and the second is a constructor function which will be called when Angular parses the DOM and
finds the ng-controller directive. The constructor method is always passed the
$scope variable which Angular uses as context when evaluating expressions - i.e., ({{
votes_for_yoda }}) - or for propagating events.

What is Scope in Angular?

In Angular you can think of Scope or more specifically $scope as a data structure used to
share information between the controller and the view. So any value you set on $scope in
the controller will be reflected in the view during expression evaluation. An example may
help understanding here.
Given the following HTML1
2
3

<div ng-controller='DaysCtrl'>
<p>Today is {{ day_of_week }} </p>
</div>

-and the following JS1

2
3

app.controller('DaysCtrl', function($scope) {
$scope.day_of_week = "Monday";
});
304

-will cause <p>Today is {{ day_of_week }}</p> to be rendered as <p>Today is

Monday</p> because day_of_week has its value set to Monday in the controller. In some
ways this is similar to a template context that is created in a Django view. The difference is
that scope works two ways (two-way binding). So with the following code1
2

3
4

<div ng-controller='DaysCtrl'>
<input id="day-of-week" ng-model="day_of_week" placeholder="what
day is it?">
<p>Today is {{ day_of_week }} </p>
</div>

When the user types in a value in the day-of-week input then that value is stored in
$scope.day_of_week and can be accessed from the controller as well as in the next line of
the html - <p>Today is {{ day_of_week }} </p>.
For our purposes that is really all you need to know about scope.

Back to the code

Here we define five variables in the $scope (votes_for_yoda,
votes_for_qui, and so forth) and set their initial values. By defining the variables
in the $scope data structure, we make them available to the view so that they can be
used in expressions. In other words, after line 5 is run, {{ votes_for_yoda }} in
the view will evaluate to 80.
Lines 11-13: In the HTML part of our example we used the following ng-click directive: ng-click="vote('votes_for_yoda')". Lines 11-13 define the vote() function that is called by the ng-click directive. If no such function were defined, Angular
Lines 5-9:

would silently fail and give no indication the function wasnt defined.
Line 12: When voteModel is a string (in our case, something like the string
votes_for_yoda), then this expression is just looking up the value by name in the
$scope data structure. This works much like a dictionary in Python. By using this
syntax, we can use the same vote() function for whatever we want to vote for; we just
pass in the name of the model we want to increment, and it will be incremented.
Thats it. We probably want to set the default values to 0 for everybody to be fair; we set nonzero values in the example above to show that the default values would actually be reflected
in the view.

305

Conclusion
That certainly isnt all there is to Angular. Again, Angular is a massive framework with a
lot to offer, but this is enough to get us started building some features in Angular. As we
go through the process of developing these user stories, we will most surely stumble upon
some more features of Angular, but for now this should get your feet wet. The key here is to
understand the similarities between Angular and Django (e.g., a templating language that is
very similar to Djangos templating language) and its differences (e.g., client-side vs serverside, two-way data binding vs one-way data binding). If you can grasp these core Angular
concepts, you will be well on your way to developing some cool features with Angular.
In the next chapter we will look at integrating Angular with Django, talk about how to structure things so the two frameworks play nice together, and also talk about how to use Angular
to update the back-end.

306

Exercises
1. Our User Poll example uses progress bars, which are showing a percentage of votes.
But our vote function just shows the raw number of votes, so the two dont actually
match up. Can you update the vote() function to return the current percentage of
votes. (HINT: You will also need to keep track of the total number of votes.)
2. Spend a bit more time familiarizing yourself with Angular. Here are a few excellent
resources to go through:
The Angular PhoneCat Tutorial - https://docs.angularjs.org/tutorial.
The Angular team is very good at releasing educational talks and videos about
Angular; check those out on the Angular JS YouTube Channel.
There is a really good beginner to expert series from the ng-newsletter, which is
highly recommended.

307

Chapter 14
Djangular: Integrating Django and
Angular
In the last chapter we went through a brief introduction to Angular and how to use it to deliver
the User Polls functionality from our User Stories. However, there are a few issues with that
implementation:
1. We implemented it stand-alone in a separate HTML page, so we still need to integrate
it into our Django Project.
2. We hard coded a number of values, so it will only work for a single type of poll; we need
a more general solution.
3. As implemented, the user poll will not share poll data between users (because the
model only exists on the client side); we need to send the votes back to the server so all
user votes can be counted and shared.
This chapter will focus on how to accomplish those three things. We will look at the Django
backend as well as the Django and Angular templates. Lets start first with the Django backend.

308

The User Polls Backend Data Model

In order to store the voting data and thus share it amongst multiple users, we first need to
create the appropriate models. Lets create this as a completely separate Django App, as we
may want to use it in other projects.
To do that, start with the command line:
1

$ ./manage.py startapp djangular_polls

NOTE: Well skip talking about unit testing for this chapter to focus on the task
at hand - Angular. Thats no excuse for you to not continut writing tests on your
own, though.

With the new app created, we can update the djangular_polls/models.py file and add
our new model:
1

from django.db import models

2
3
4
5
6

class Poll(models.Model):
title = models.CharField(max_length=255)
publish_date = models.DateTimeField(auto_now=True)

7
8
9

def poll_items(self):
return self.pollitem_set.all()

10
11
12
13
14
15
16
17
18

class PollItem(models.Model):
poll = models.ForeignKey(Poll, related_name='items')
name = models.CharField(max_length=30)
text = models.CharField(max_length=300)
votes = models.IntegerField(default=0)
percentage = models.DecimalField(
max_digits=5, decimal_places=2, default=0.0)

19
20
21

class Meta:
ordering = ['-text']

We just created two models:

309

1. Poll(): represents the overall poll - i.e., who is the best Jedi.
2. PollItem(): the individul items to vote on in the poll.
Notice within PollItem() we also added a default ordering so that the returned JSON will
always be in the same order. (This will become important later.) Dont forget to add the app
to INSTALLED_APPS in the settings.py file and migrate the changes to your database:
1
2

$ ./manage.py makemigrations djangular_polls

$ ./manage.py migrate djangular_polls

Now you have a nicely updated database and are ready to go.

310

A REST Interface for User Polls

The next step is to make a nice RESTful interface where we can consume the Poll() and the
associated PollItem(). We can, of course, use the Django REST Framework for this.

Serializer
First create the serializers:
1
2

from rest_framework import serializers

from djangular_polls.models import Poll, PollItem

3
4
5
6
7
8

class PollItemSerializer(serializers.ModelSerializer):
class Meta:
model = PollItem
fields = ('id', 'name', 'text', 'votes', 'percentage')

9
10
11
12

class PollSerializer(serializers.ModelSerializer):
items = PollItemSerializer(many=True)

13

class Meta:
model = Poll
fields = ('title', 'publish_date', 'items')

14
15
16

Save this code in a file called serializers.py.

Since we covered DRF in a previous chapter, we wont go into details on the serializers. That
said, the intent is to create the following JSON for each Poll:
1
2
3
4
5

{
'title': 'Who is the best jedi',
'publish_date': datetime.datetime(2014, 5, 20, 5, 47, 56, 630638),
'items': [
{'id': 4, 'name': 'Yoda', 'text': 'Yoda', 'percentage':
Decimal('0.00'), 'votes': 0},
{'id': 5, 'name': 'Vader', 'text': 'Vader', 'percentage':
Decimal('0.00'), 'votes': 0},
{'id': 6, 'name': 'Luke', 'text': 'Luke', 'percentage':
Decimal('0.00'), 'votes': 0}
311

8
9

This makes it easy to get a Poll() and its associated PollItem() in one JSON call. To
do this, we gave the relationship, between a poll and its items, a name in the PollItem()
model:
1

poll = models.ForeignKey(Poll, related_name='items')

The related_name provides a name to use for the reverse relationship from Poll.
Next, in the PollSerializer() class we included the items property:
1

items = PollItemSerializer(many=True)

Why? Well, without it the serializer will just return the primary keys.
Then we also have to include the reverse relationship in the fields list:
1

fields = ('title', 'publish_date', 'items')

Those three things will give us the JSON that we are looking for. This would be a good thing
to write some unit tests for to make sure its working appropriately. Yes, try it on your own.

Endpoints
Next, we need to create the REST endpoints in djangular_polls/json_views.py.
1

2
3
4

from djangular_polls.serializers import PollSerializer,

PollItemSerializer
from djangular_polls.models import Poll, PollItem
from rest_framework import mixins
from rest_framework import generics

5
6
7
8
9

class PollCollection(mixins.ListModelMixin,
mixins.CreateModelMixin,
generics.GenericAPIView):

10
11
12

queryset = Poll.objects.all()
serializer_class = PollSerializer

13
14
15

def get(self, request):

return self.list(request)
312

16
17
18

def post(self, request):

return self.create(request)

19
20
21
22
23
24

class PollMember(mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
generics.GenericAPIView):

25
26
27

queryset = Poll.objects.all()
serializer_class = PollSerializer

28
29
30

def get(self, request, *args, **kwargs):

return self.retrieve(request, *args, **kwargs)

31
32
33

def put(self, request, *args, **kwargs):

return self.update(request, *args, **kwargs)

34
35
36

def delete(self, request, *args, **kwargs):

return self.destroy(request, *args, **kwargs)

37
38
39
40
41

class PollItemCollection(mixins.ListModelMixin,
mixins.CreateModelMixin,
generics.GenericAPIView):

42
43
44

queryset = PollItem.objects.all()
serializer_class = PollItemSerializer

45
46
47

def get(self, request):

return self.list(request)

48
49
50

def post(self, request):

return self.create(request)

51
52
53
54
55

class PollItemMember(mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
313

generics.GenericAPIView):

56
57
58
59

queryset = PollItem.objects.all()
serializer_class = PollItemSerializer

60
61
62

def get(self, request, *args, **kwargs):

return self.retrieve(request, *args, **kwargs)

63
64
65

def put(self, request, *args, **kwargs):

return self.update(request, *args, **kwargs)

66
67
68

def delete(self, request, *args, **kwargs):

return self.destroy(request, *args, **kwargs)

Again, this should all be review from the Django REST Framework chapter. Basically we are
just creating the endpoints for pPoll() and the associated PollItem() with basic CRUD
functionality.

Routing
Dont forget to update our Django routing in djangular_polls/urls.py:
1
2

from django.conf.urls import patterns, url

from djangular_polls import json_views

3
4
5
6
7
8
9
10
11

12

urlpatterns = patterns(
'djangular_polls.main_views',
url(r'^polls/$', json_views.PollCollection.as_view(),
name='polls_collection'),
url(r'^polls/(?P<pk>[0-9]+)$', json_views.PollMember.as_view()),
url(r'^poll_items/$', json_views.PollItemCollection.as_view(),
name="poll_items_collection"),
url(r'^poll_items/(?P<pk>[0-9]+)$',
json_views.PollItemMember.as_view()),
)

Here, we are creating the standard endpoint addresses.

NOTE: Notice in the URL patterns listed above that there are no trailing /.
This is important because some parts of Angular dont like trailing slashes. To

314

complicate things, Django has a default setting of APPEND_SLASH=True, which

means if you (or Angular) request polls/1 and your routing includes trailing
/, then Django will automatically redirect by returning HTTP status code 301
to polls/1/. It works, but it is not ideal since (a) there is an extra request and
(b) for PUT and POST requests, your browser will generally pop up an annoying
message about the page being redirected. This isnt what we want. So the best
answer when using Angular with Django is to remove the trailing / from your
routes.
Now add the following code to django_ecommerce/urls.py:
1
2

from main.urls import urlpatterns as main_json_urls

from djangular_polls.urls import urlpatterns as
djangular_polls_json_urls

3
4

main_json_urls.extend(djangular_polls_json_urls)

Above, we are combining the json_urls from our main application with the json_urls
from our djangular_polls application. Then they are added to the overall django_ecommerce.urls
with the same line we had before:
1

url(r'^api/v1/', include(main_json_urls)),

This way our entire REST API stays in one place, even if it spans multiple Django apps. Now
we should have two new URLs in our REST API:
1
2

/api/vi/polls/
/api/vi/poll_items/

Test! Fire up the server and navigate to the browseable API:

1. http://localhost:8000/api/v1/polls/
2. http://localhost:8000/api/v1/poll_items/
These endpoints are what we will use to get our Polls functionality working.

315

Structural Concerns
Now that we have the backend all sorted, we have some choices to make about how to best
integrate our frontend framework, Angular, with our backend framework, Django. There are
many ways you can do this, and it really depends on what your design goals are and, as always,
personal preference. Lets discuss a couple of integration options.

The Single Page Application

The much touted Single Page Application (or SPA) is a web page that acts like a desktop app,
which basically means it doesnt ever do a full page refresh. You land on the main page, and
from there everything is handled via AJAX and REST calls, and the user basically forgets that
s/he is using a web page. To achieve this, you would have oly one single standard Django
View that sends back your entire page (something similar to our main.views.index), and
from there you let Angular take over and do all the view work, client-side. You would then
build a number of JSON views (using DRF) so that Angular could asynchronously fetch all
the data it needs and never have to refresh the page again.

The Multi-Single Page App

This is really just a funny way of saying that you want multiple Single Page Apps working
together. For example, with our MEC app, we might have one App for the non-logged in
users that shows the marketing info and allows a user to register and/or access the sign-in
page. Then wed have another App for the logged in user, where there would be a screen
refresh after registration/sign-in as the user would be switching apps. Everything after the
user logged in (which would translate to our main.views.user) would be asynchronous so no more page refreshes.
In reality, you can choose to have a page refresh whenever you like. Each page refresh could
correlate to calling a standard Django view function. When you dont want a page refresh and
you need to get or update data, then use Angular to make a REST call to one of your JSON
vies. This is a common practice when you want to logically split up a large web application
into multiple apps or sections.

The I Just Need a Bit of Angular Love Here App

Perhaps you have a Django application all ready to go, but you just want to use Angular for
a single piece of the application - like a poll or form validation - then we can make a view,

316

or even an inclusion tag, that uses Angular to accomplish that task and use standard Django
views for the remainder of the application.
Those three choices should basically cover every possiblity, and the choice really just comes
down to how much Angular you want to include. Thus, there is no wrong or right choice; just
do what best suits your needs.

317

Building the Template

For our purposes, we are going to follow the I Just Need a Bit of Angular Love Here idea,
since we already have an application built. But how much love do we need? Lets start off
with the simplest case. If we just wanted to take the Angular code that we did in the previous
chapter and cram it into our existing app, we could do the following
Lets add the poll along the right hand side of our users page just below where the status
updates are:

user.html template
First, update our main/user.html template to include our djangular_polls/_polls.html
template in a new info box:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

{% extends "__base.html" %}
{% load staticfiles %}
{% block content %}
<div id="achievements" class="row member-page hide">
{% include "main/_badges.html" %}
</div>
<div class="row member-page">
<div class="col-sm-8">
<div class="row">
{% include "main/_announcements.html" %}
{% include "main/_lateststatus.html" %}
</div>
</div>
<div class="col-sm-4">
<div class="row">
{% include "main/_jedibadge.html" %}
{% include "main/_statusupdate.html" %}
{% include "djangular_polls/_polls.html" %}
</div>
</div>
</div>
{% endblock %}
{% block extrajs %}
<script src="{% static "js/angular.min.js" %}"
type="text/javascript"></script>
318

Figure 14.1: Users Page

319

25

26

<script src="{% static "js/userPollCtrl.js" %}"

type="text/javascript"></script>
{% endblock %}
Line 18 - This is the line that imports the new _polls.html template that will display
the Polls info box shown in the above picture.
Line 23-26 - Were using a new block called extrajs which were using here to load
both Angular and the code for our userPollCtrl.js - which will include the Angular module.

Now, lets make a few updates to wire up our Polls

First, lets add angular.min.js to our static files. Just save the following file to the static/js
directory - https://code.angularjs.org/1.2.16/angular.min.js. Make sure to add https://code.
angularjs.org/1.2.16/angular.min.js.map as well.
Next, since we added a new block, extrajs, we need to include this in the __base.html
template as well. Add the following line just before the closing body tag:
1

{% block extrajs %}{% endblock %}

Then add a new file to the same directory (static/js) called userPollCtrl.js and add the following code:
1

var pollsApp = angular.module('pollsApp',[]);

2
3
4
5

pollsApp.controller('UserPollCtrl', function($scope) {
$scope.total_votes = 0;
$scope.vote_data = {}

6
7
8
9
10
11
12

13
14
15
16

$scope.vote = function(voteModel) {
if (!$scope.vote_data.hasOwnProperty(voteModel)) {
$scope.vote_data[voteModel] = {"votes": 0, "percent": 0};
$scope[voteModel]=$scope.vote_data[voteModel];
}
$scope.vote_data[voteModel]["votes"] =
$scope.vote_data[voteModel]["votes"] + 1;
$scope.total_votes = $scope.total_votes + 1;
for (var key in $scope.vote_data) {
var item = $scope.vote_data[key];
item["percent"] = item["votes"] / $scope.total_votes * 100;
320

17
18

}
};

19
20

});

_polls.html template
Now lets add the _polls.html template within templates/djangular_polls:
1
2
3
4

5
6

7
8

10
11
12

13
14

15
16

17

18
19
20

<section class="info-box" id="polls" ng-app="pollsApp">

<div ng-controller='UserPollCtrl'>
<h1>Poll: Who's the most powerful Jedi?</h1>
<span ng-click='vote("votes_for_yoda")' class="glyphicon
glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">[[ votes_for_yoda.percent | number:0
]]%</span>
<div class="progress">
<div class="progress-bar progress-bar-danger"
role="progressbar" aria-value="[[ votes_for_yoda.percent
]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
votes_for_yoda.percent ]]%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"
ng-click="vote('votes_for_qui')" ></span>
<strong>Qui-Gon Jinn</strong>
<span class="pull-right">[[ votes_for_qui.percent | number:0
]]%</span>
<div class="progress">
<div class="progress-bar progress-bar-info"
role="progressbar" aria-value="[[ votes_for_qui.percent
]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
votes_for_qui.percent ]]%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"
ng-click="vote('votes_for_obi')"></span>
321

21
22

23
24

25

26
27
28

29
30

31
32

33

34
35
36

37
38

39
40

41

42
43
44
45

<strong>Obi-Wan Kenobi</strong>
<span class="pull-right">[[ votes_for_obi.percent | number:0
]]%</span>
<div class="progress">
<div class="progress-bar progress-bar-warning"
role="progressbar" aria-value="[[ vote_for_obi.percent ]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
votes_for_obi.percent ]]%;">
</div>
</div>
<span class="glyphicon
glyphicon-plus"ng-click="vote('votes_for_luke')"></span>
<strong>Luke Sykwalker</strong>
<span class="pull-right">[[ votes_for_luke.percent | number:0
]]%</span>
<div class="progress">
<div class="progress-bar progress-bar-success"
role="progressbar" aria-value="[[ vote_for_luke.percent
]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
votes_for_luke.percent ]]%;">
</div>
</div>
<span class="glyphicon glyphicon-plus"
ng-click="vote('votes_for_me')"></span>
<strong>Me... of course</strong>
<span class="pull-right">[[ votes_for_me.percent | number:0
]]%</span>
<div class="progress">
<div class="progress-bar" role="progressbar" aria-value="[[
vote_for_me.percent ]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
votes_for_me.percent ]]%;">
</div>
</div>
</div>
</section>

This should look familiar, as its pretty much the same code we had in the last chapter, except
now we are shoving it all into one of our info-boxes, which we introduced way back in the
322

Bootstrap chapter.
There is one important difference here though, can you spot it?
Notice that we are using different template tags - e.g., [[ ]] instead of {{ }}. Why? Because
Django and Angular both use {{ }} by default. Therefore we will instruct angular to use
the delimeters [[ ]] and let Django continue to use the delemiters {{ }} - so we can be
clear who is doing the interpolation. Luckily in Angular this is very easy to do. Just add the
following lines to the top of the userPollCtrl.js file:
1
2
3
4

pollsApp.config(function($interpolateProvider){
$interpolateProvider.startSymbol('[[')
.endSymbol(']]');
});

The Angular Module should now look like this:

1

var pollsApp = angular.module('pollsApp',[]);

2
3
4
5
6

pollsApp.config(function($interpolateProvider){
$interpolateProvider.startSymbol('[[')
.endSymbol(']]');
});

7
8
9
10

pollsApp.controller('UserPollCtrl', function($scope) {
$scope.total_votes = 0;
$scope.vote_data = {}

11
12
13
14
15
16
17

18
19
20
21
22
23

$scope.vote = function(voteModel) {
if (!$scope.vote_data.hasOwnProperty(voteModel)) {
$scope.vote_data[voteModel] = {"votes": 0, "percent": 0};
$scope[voteModel]=$scope.vote_data[voteModel];
}
$scope.vote_data[voteModel]["votes"] =
$scope.vote_data[voteModel]["votes"] + 1;
$scope.total_votes = $scope.total_votes + 1;
for (var key in $scope.vote_data) {
var item = $scope.vote_data[key];
item["percent"] = item["votes"] / $scope.total_votes * 100;
}
};

24
25

});
323

So, the config function of an Angular Module allows you to take control of how Angular will
behave. In our case, we are using the config function to tell angular to use [[ ]] for demarcation, which Django ignores.
With that, you should have a functioning User Poll info-box on the main users page. Make
sure to manually test this out. Once its working, then add unit tests.
This app still isnt talking to the backend. Lets rectify that.

324

Loading Poll Items Dynamically

We have two options here:
1. Return the poll item as a template context variable from our main.views.user function and let the Django templates do their magic.
2. Use Angular to call the api/v1/polls REST API and have Angular build the HTML
template for User Polls.
Since this is a chapter on Angular and we have the proper endpoints setup, lets go with option
2. To do that, we need to talk about how to make REST calls with Angular.

Making REST calls with Angular

The simpilest way to make a rest call in Angular is by using the $http service (documentation). $http is a core Angular service used to make Asynchronous Requests. If we wanted to
call our polls API to get a poll via id, we can update our UserPollCtrl.js by adding the following
code:
1
2

// Get the Poll

$scope.poll = ""

3
4
5
6
7
8
9

10

$http.get('/api/v1/polls/1').
success(function(data){
$scope.poll = data;
}).
error(function(data,status) {
console.log("calling /api/v1/polls/1 returned status " +
status);
});
Line 2 - Initializes the poll scope variable as an empty string to avoid undefined errors
later.
Line 4 - $http.get makes a GET request to the specified URL.
Line 5 - The return value from $http.get is a promise. Promises allow us to register
callback methods that receive the response object. Here, we are registering a method,
success, to be called on a success. It receives the data object - e.g, the data returned
in the response - and we set our $scope.poll variable to this, which should be the
JSON representation of our poll, where id is 1.
325

Lets digress for a few minutes to talk about promises

A brief aside on HttpPromise

The $http service has several shortcut methods, corresponding to HTTP verbs, such as get,
put, post, etc. When you call these methods they all return a promise object. A promise
object is an object that represents a value that is not yet known because JavaScript executes
asynchronously. While it takes time for the AJAX request to actually execute, your JavaScript
code doesnt wait around for the result - it continues executing.
This is not what we want. What would happen if you have another function thats dependent
on the results from the $http service and that function fires before the results are returned.
Thats a problem, and its exactly what promises solve.
So a promise has states such as waiting, success, or error, which allows you to
attach handlers (or methods that will be called) to each of those states. For example, $http.get().success(function() specifies the handler to be called when the
HttpPromise returned by $http.get() reaches the success state. The same logic applies
to an error handler.
The key advantage here is that we can write asynchronous code like synchronous code (which
is much easier) and not have to worry too much about the asynchronous part.
The other important thing to understand about the HttpPromise is that it always passes the
following parameters into your handler:
Parameter

Data Type

Explanation

data
status
headers
config
statusText

String or Object
Number
Function
Object
String

Response body
HTTP status code - 200, 404, 500, etc.
Header getter function to be called to get the header item
Config object used to generate the request
HTTP Status text of the response

NOTE: In JavaScript you can always just ignore extra parameters that you do
not need like we have done in our success function above.

Back to the code

Line 8 - This is the handler for the error state. In response to an error we just log the
HTTP status code to the console.
326

So that little bit of code allows us to retrieve the information about a Poll to be displayed from
our Django backend. Make sure to inject the $http service into the Angular Controller:
1

pollsApp.controller('UserPollCtrl', function($scope, $http)

Now we need to display that information. Lets modify our djangular_polls/_polls.html

template:
1
2
3
4
5
6

7
8

9
10
11
12

13
14
15
16
17
18

<section class="info-box" id="polls" ng-app="pollsApp">

{% verbatim %}
<div ng-controller='UserPollCtrl'>
<h1>Poll: [[ poll.title ]]</h1>
<div ng-repeat="item in poll.items">
<span ng-click='vote(item)' class="glyphicon
glyphicon-plus"></span>
<strong>[[ item.text ]]</strong>
<span class="pull-right">[[ item.percentage | number:0
]]%</span>
<div class="progress">
<div class="progress-bar" role="progressbar"
aria-value="[[ item.percentage ]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
item.percentage ]]%;">
</div>
</div>
</div>
</div>
{% endverbatim %}
</section>

Thats a whole lot less HTML!

This is mainly because of the ng-repeat on line 5 (documentation).

ng-repeat functions much like {% for item in poll.items %} would in a Django template. The main difference is that ng-repeat is generally used as an attribute of an HTML
tag, and it repeats that tag.
In our case, the <div> is repeated, which includes all of the HTML inside the <div>. We
only have to create the HTML for one of the voting items and use ng-repeat to generate a
new item for every poll item returned from our $http.get('/api/v1/polls/1') call.
With ng-repeat we specify item in poll.items; this exposes item as an variable that
we can use in Angular expressions:
327

 Line 6 - ng-click='vote(item)' passes in the current item to the vote function in

our controller.
Line 7 - We can also access properties of the item object like we do here with [[
item.text ]].
The result of the ng-repeat function is to create all of the vote items with their corresponding
progress bars. Then we can change the vote function in the controller to work with our polls
model that we are getting from our JSON call. The new vote function looks like this:
1

$scope.total_votes = 0;

2
3
4
5

$scope.vote = function(item) {
item.votes = item.votes + 1;
$scope.total_votes = $scope.total_votes + 1;

6
7
8
9

10
11

for (i in $scope.poll.items){
var temp_item = $scope.poll.items[i];
temp_item.percentage = temp_item.votes / $scope.total_votes *
100;
}
};

Update this in the userPollCtrl.js file.

The vote function is now also simpler than it was before, as we can rely on the post model
holding all the information so we dont have to jump through the hoops that we did before.
Lets look at this a bit closer:
Line 1 - Initialize total_votes to 0 on page load. We will need to change this eventually to support multiple users.
Line 4 - Remember from the HTML above we are calling vote(item) where item is
the poll item returned from our original query. Thus we can just add one to the item
to account for the new vote.
Line 5 - Increment the total number of votes so we can calculate percentages.
Lines 7-10 - Loop through all poll.items and update the percentages. Note:
poll.items is an array. So for (i in $scope.poll.itmes) is similar to
for(var i = 0; i < $scope.poll.items.length; i++), just with a lot less
code.
Now we have a functioning user poll. Right? Not yet. We still need to add some data. We
can do that directly from the browseable API.
328

Navigate to http://localhost:8000/api/v1/polls/ in your browser and add the following

JSON to the Raw Data form.
1

{
"title": "Who is the best Jedi?",
"publish_date": "2014-10-21T04:05:24.107Z",
"items": [
{
"id": 1,
"name": "Yoda",
"text": "Yoda",
"votes": 0,
"percentage": "0.00"
},
{
"id": 2,
"name": "Vader",
"text": "Vader",
"votes": 0,
"percentage": "0.00"
},
{
"id": 3,
"name": "Luke",
"text": "Luke",
"votes": 0,
"percentage": "0.00"
}
]

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

Be sure to test it out! Navigate to http://localhost:8000/. After you login make sure the Poll
is at the bottom right of the page. Test the functionality.

329

Refactoring - progress bars

One issue we have is that all the progress bars are the same color. Lets rectify that. To do so,
we use a feature of ng-repeat. When you are looping through items in ng-repeat, $index
represents the current iteration number - i.e., (0, 1, 2, etc.). So we can use that counter to
pick a different class to control the color for each progress bar.
First, update djangular_polls/_polls.html:
1
2
3
4
5

6
7

8
9

10
11

12
13
14
15
16

<section class="info-box" id="polls" ng-app="pollsApp">

<div ng-controller='UserPollCtrl'>
<h1>Poll: [[ poll.title ]]</h1>
<div ng-repeat="item in poll.items">
<span ng-click='vote(item)' class="glyphicon
glyphicon-plus"></span>
<strong>[[ item.text ]]</strong>
<span class="pull-right">[[ item.percentage | number:0
]]%</span>
<div class="progress">
<div class="progress-bar [[ barcolor($index) ]]"
role="progressbar"
aria-value="[[ item.percentage ]]"
aria-valuemin="0" aria-valuemax="100" style="width: [[
item.percentage ]]%;">
</div>
</div>
</div>
</div>
</section>

So, we added in [[ barcolor($index) ]] as one of the classes. Now when Angular comes
across this, it calls $scope.barcolor() from the controller and passes in the current iteration number of ng-repeat.
Lets update the controller as well in static/js/userPollCtrl.js:
1
2
3
4
5
6

$scope.barcolor = function(i) {
colors = ['progress-bar-success', 'progress-bar-info',
'progress-bar-warning', 'progress-bar-danger', '']
idx = i % colors.length;
return colors[idx];
}
330

This function uses a list of various Bootstrap classes to control the color of the progress bar.
Using % (modulus) ensures that no matter how many vote items we get, we will always get an
index that is in the bounds of our colors list.
Now all we need to do is make this thing work for multiple users.

331

Refactoring - multiple users

To do that, we need to keep track of total_votes and the votes per poll item on the backend
so they can be shared across all users.
Lets first add the total_votes as an attribute of our Poll() class and make sure it is sent
across in the /api/v1/polls GET request. To do this, we can add a @property to the
Poll() class like so:
1
2
3
4

@property
def total_votes(self):
return self.poll_items().aggregate(
models.Sum('votes')).get('votes__sum', 0)

Before we jump into the explanation of what this code does lets talk about using a @property
for just a second.
1. What is a @property? Its a function that you can call without (). So in our case, normally we would call model.total_votes(), but by making it a property with @property
we can just call it like this - model.total_votes
2. What is it good for? Other than saving a couple of keystrokes, properties are best
used for computed model attributes. You can think of it as a field within the model
(sort of), except that it needs some logic to run on the get, set, or delete.
3. Is it the same thing as a model field? Absoluetly not. Django, for the most part,
just ignores a property - i.e., you cant use it in a querySet as it wont trigger a migration, etc.). You can think of a property like a field that you dont want django to know
about.
4. Why not just use a function? Really @property is just snytactic sugar, so if you
prefer you could use a function all you want. Its a design choice more than anything
else.
Okay. On with the show ..
Take a look at the code again. This uses Djangos aggregate() function to compute the sum
of all the related PollItem() for the poll. aggregate returns a dictionary that in this case
looks like {'votes__sum': <sum_of_votes> }. But we only want the actual number, so
we just call a get on the returned dictionary and default the value to 0 in case its empty.
With this method, we calculate total_votes every time the property is called; that way we
dont have to keep the various database tables in sync.
Because this is a property and not a true field, Django Rest Framework wont pick it up by
default, so we need to modify our serializer as well:
332

1
2
3

class PollSerializer(serializers.ModelSerializer):
items = PollItemSerializer(many=True, required=False)
total_votes = serializers.Field(source='total_votes')

4
5
6
7

class Meta:
model = Poll
fields = ('title', 'publish_date', 'items', 'total_votes')

We instruct the serializer to include a field called total_votes which points to our
total_votes property. With that, our /api/v1/polls will return an additional property total_votes.
Finally, we need to calculate the percentage as well within djangular_polls/models.py.
Lets use a similar technique to calculate the percentage on the fly:
1
2
3
4
5
6

@property
def percentage(self):
total = self.poll.total_votes
if total:
return self.votes / total * 100
return 0

We use the newly created total_votes property of our parent poll to calculate the percentage. We do need to be careful to avoid a divide by zero error. Then we make a similar change
to the PollItemsSerialzer as we did to the PollSerailzer. It now looks like this:
1
2

class PollItemSerializer(serializers.ModelSerializer):
percentage = serializers.Field(source='percentage')

3
4
5
6

class Meta:
model = PollItem
fields = ('id', 'poll', 'name', 'text', 'votes',
'percentage')

Your models.py and the serializers.py should now look like this:

djangular_polls/models.py
1
2

from django.db import models

from django.db.models import Sum

3
4
5

class Poll(models.Model):
333

6
7

title = models.CharField(max_length=255)
publish_date = models.DateTimeField(auto_now=True)

8
9
10
11

@property
def total_votes(self):
return
self.poll_items().aggregate(Sum('votes')).get('votes__sum',
0)

12
13
14

def poll_items(self):
return self.items.all()

15
16
17

class PollItem(models.Model):

18
19
20
21
22

poll = models.ForeignKey(Poll, related_name='items')

name = models.CharField(max_length=30)
text = models.CharField(max_length=300)
votes = models.IntegerField(default=0)

23
24
25
26
27
28
29

@property
def percentage(self):
total = self.poll.total_votes
if total:
return self.votes / total * 100
return 0

30
31
32

class Meta:
ordering = ['-text']

djangular_polls/serializers.py
1
2

from rest_framework import serializers

from djangular_polls.models import Poll, PollItem

3
4
5
6

class PollItemSerializer(serializers.ModelSerializer):
percentage = serializers.Field(source='percentage')

7
8
9

class Meta:
model = PollItem
334

10

fields = ('id', 'poll', 'name', 'text', 'votes',

'percentage')

11
12
13
14
15

class PollSerializer(serializers.ModelSerializer):
items = PollItemSerializer(many=True, required=False)
total_votes = serializers.Field(source='total_votes')

16
17
18
19

class Meta:
model = Poll
fields = ('id', 'title', 'publish_date', 'items',
'total_votes')

With these changes we can now update our Angular controller to push the calculations to
the Django backend so we can share the results amongst multiple users. But first we should
update our database to reflect the changed table structure for the Poll / PollItems models, so
dont forget to run makemigrations and migrate.
Once that is done we can switch back to our angular controller. Update the vote function
within static/js/userPollCtrl.js like so:
1
2
3
4
5
6
7
8
9

10
11
12
13

14
15

$scope.vote = function(item) {
item.votes += 1;
$http.put('/api/v1/poll_items/'+item.id,item).
success(function(data){
$http.get('/api/v1/polls/1').success(function(data){
$scope.poll = data;
}).
error(function(data,status){
console.log("calling /api/v1/polls/1 returned status " +
status);
});
}).
error(function(data,status){
console.log("calling PUT /api/v1/poll_items returned status " +
status);
});
});

Here, were just calling the rest API a few times:

Line 2 - Update vote count for the item just voted on.
335

 Line 3 - Using a PUT request, send the item back to the backend (which will update
the vote count). Note the second parameter of the PUT call is the item itself, which will
be converted to JSON and sent back to the server.
Line 5 - If the PUT request is successful then refetch all the Poll data, which will also
give us updated total_votes and percentages for each item. Its important to only
call this in the success handler so as to ensure the previous PUT request has completed,
otherwise we might not get the most up-to-date data.

336

Using Angular Factories

We now have a multi-user poll working that is updating the backend. Our code works, but
lets clean it up a bit.
One thing that the Angular philosophy stresses is separation of concerns. According to this
philosophy, a controller should just be managing scope; it shouldnt be accessing external
resources directly. This means we should not be callig $http from our controller. (This also
makes testing the controller a whole lot easier.)
This is where factories come into play.
An Angular factory always returns an object, which generally has a number of functions attached that are responsible for working with data, handling business rules, or doing any number of tasks. Factories, unlike controllers, also maintain state throughout the lifetime of your
Angular application.
For example, you might want to cache the poll id the first time a user calls getPoll() for use
in other areas of our application. For our purposes, lets create a PollFactory to handle the
REST API requests for us:
1

pollsApp.factory('pollFactory', function($http) {

2
3
4
5

var baseUrl = '/api/v1/';

var pollUrl = baseUrl + 'polls/';
var pollItemsUrl = baseUrl + 'poll_items/';

6
7

var pollFactory = {};

8
9
10
11

pollFactory.getPoll = function(id) {
return $http.get(pollUrl + id);
};

12
13
14
15
16

pollFactory.vote_for_item = function(poll_item) {
poll_item.votes +=1;
return $http.put(pollItemsUrl + poll_item.id, poll_item);
}

17
18
19

return pollFactory;
});

Lets go through the code line by line:

337

 Line 1 - Here we just declare the factory. Note that we inject the $http service, since
we will need it to make our REST calls.
Lines 3-5 - Define the URLs needed for the REST calls.
Line 7 - This is the factory object that we will eventually return, but first we need to
add some functions to it.
Lines 9-11 - The first function we add is the getPoll() function, which will grab the
poll from our Django REST API. Note that we are returning a promise (we talked about
them previously) and not the actual data. The advantage of returning the promise is
that it allows us to do promise chaining (one call after another), which we will make
use of later in the controller.
Linse 13-16 - Our second function, vote_for_item, takes the poll item as an input,
increments the vote counter, then updates the poll item through the REST API. Again
we are returning a promise which wraps the actual result of the call.
Line 18 - Now that we have finished creating our pollFactory() object and have
given it all the functionality we need. Lets return it, so we can use it in our controller.
Next, lets look at how we use our newly created factory in our controller.
All we have to do is ask Angular to inject the factory into our controller, which we can do with
this initial line:
1
2

pollsApp.controller('UserPollCtrl',
function($scope, $http, pollFactory) {

This is the line used to create our controller. Notice that we added pollFactory to the list
of dependencies, so Angular will inject the factory for us. Also note that we no longer require
$http, as that is all handled by the factory now. With the declaration above, we can use the
factory anywhere in our controller (or application, for that matter) by simply calling it like
this:
1

pollFactory.getPoll(1);

Lets see how the complete controller looks after we add in the factory:
1

pollsApp.controller('UserPollCtrl',function($scope, pollFactory) {

2
3
4
5
6
7

//get the Poll

$scope.poll = ""
function setPoll(promise){
$scope.poll = promise.data;
}

338

9
10
11

function getPoll(){
return pollFactory.getPoll(1);
}

12
13
14
15
16
17
18

$scope.barcolor = function(i) {
var colors = ['progress-bar-success','progress-bar-info',
'progress-bar-warning','progress-bar-danger','']
var idx = i % colors.length;
return colors[idx];
}

19
20

getPoll().then(setPoll);

21
22
23
24
25
26

$scope.vote = function(item) {
pollFactory.vote_for_item(item)
.then(getPoll)
.then(setPoll);
}

27
28

});

Youll notice near the top of the controller we created two functions getPoll which asks the
pollFactory to get the poll, and setPoll which takes a promise and uses it to set the value
of $scope.poll. These functions will come in handy later on.
The next function $scope.barcolor remains unchanged.
Line 22 - getPoll().then(setPoll). This is a simple example of promise chaining. First we call getPoll which in turn calls pollFactory.getPoll and returns a
promise. Next we use a function of the promise called then which will be triggered
once the promise has completed. We pass in our setPoll() function to then, which
will receive the promise returned by getPoll and use the data on that promise to set
our $scope.poll variable.
Line 25 - In our $scope.vote function we have another example of promise chaining. Here we first call pollFactory.vote_for_item, which issues a PUT request to
our poll_items API and returns a promise containing the result of that PUT request.
When the promise returns, we call our getPoll() function with .then(getPoll).
This function then gets the newer version of our Poll object with total_votes and
percentages recalculated, and returns a promise. Finally we call .then(setPoll),
which uses that returned promise to update our $scope.poll variable.
339

As you can see, by structuring our factory methods to return promises, we can chain together
several function calls and have them execute synchronously. As an added advantage, if any
of the calls fail, the subsequent calls in the chain will receive the failed result and can thus
react accordingly.

340

Conclusion
With that, we have some pretty well-factored Angular code that interacts with our RESTful
API to provide a multi-user user poll function. While the Django Rest Framework part of
this chapter should have been mostly a review, we did learn how to ensure that our URL
structure plays nicely with Angular and how to serialize model properties that arent stored
in the database.
On the REST side of things, you should now have a better understanding of controllers and
how they are used to manage scope, factories and how they are used to provide access to data,
the $http service and how it is used to make AJAX calls, and of course promises and how
they can be chained together to achieve some pretty cool functionality.
Also, lets not forget about the ng-repeat directive and how it can be used to achieve the
same functionalty as {% for in Django templates.
All in all, while this is only a small portion of what Angular offers, it should give you some
good ideas about how it can be used to buid well-factored, solid client-side functionality.

341

Exercises
1. For the first exercise lets explore storing state in our factory. Change the pollFactory.getPoll
function to take no parameters and have it return the single most recent poll. Then
cache that polls id, so next time getPoll() is called you have the id of the poll to
retrieve.
2. In this chapter we talk about factories but Angular also has a very similar concept called
services. Have a read through Michael Hermans excellent blog post on services so you
know when to use each.
3. Currently our application is a bit of a mismatch, we are using jQuery on some parts i.e,. showing achievements - and now Angular on the user polls. For practice, convert
the showing acheivement functionality, from application.js, into Angular code.

342

Chapter 15
Angular Forms
The final piece of our application that needs to be converted over to Angular is the registration
from. This chapter will discuss how to handle forms in Angular when dealing with a Django
back-end. We will address submitting and validating forms(client vs. server-side), dealing
with CSRF, and basically making Django and Angular Forms work well together.
When creating a form with Django and Angular, you basically have two choices:
1. You can create the form with Angular along with a REST API for it to call on the Django
backend.
2. You can also create the form from the Django side, using Django Forms, much like we
have already done for the registration form. This allows you to build the form in the
back-end and follow a more traditional Django development style.
We could argue at length as per which method is better. However, since we have talked about
REST APIs and integrating those with Angular before, in this chapter lets cover using Django
Forms (the latter method). If nothing else, at least youll have an idea of how to implement
both approaches and you can see which one works best for you.

343

Field Validation
Regardless of which choice we make for creating the form, we need some sort of form validation. Django Forms offer server side validation out of the box.
NOTE: Server side validation refers to validating the user input of a form
on the server - e.g., via the Python code, in our case. Form submission and validation usually happen at the same time. If server side validation fails, a list of
errors is generally sent back to the client. Meanwhile, Client side validation
refers to validating user inputs directly in the web browser - e.g., via Angular, in
our case.. Validation is often in response to a key press or an input field losing
focus. This allows for fields to be validated one at a time, without a trip to the
server, and generally provides a more real-time feel for the user, meaning that
they are notified as soon as they get an error, instead of after the entire form is
submitted.

Since we already know how to do server side validation with Django Forms, form.is_valid(),
lets look at client side validation with Angular. Before adding any validations, our registration form in templates/payments/register.html looks like this:
1

2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18

<form id="user_form" accept-charset="UTF-8" action="{% url

'register' %}" class="form-stacked" method="post">
{% csrf_token %}
{% if form.is_bound and not form.is_valid %}
<div class="alert alert-{{error.tags}}"> <a class="close"
data-dismiss="alert"></a>
<div class="errors">
{% for field in form.visible_fields %}
{% for error in field.errors %}
<p>{{ field.label }}: {{ error }}</p>
{% endfor %}
{% endfor %}
{% for error in form.non_field_errors %}
<p>{{ error }}</p>
{% endfor %}
</div>
</div>
{% endif %}
{% for field in form.visible_fields %}
{% include "payments/_field.html" %}
344

19
20

21
22
23
24
25
26
27
28

{% endfor %}
<div id="change-card" class="clearfix"{% if not
form.last_4_digits.value %} style="display: none"{% endif %}>
Card
<div class="input">
Using card ending with {{ form.last_4_digits.value }}
(<a href="#">change</a>)
</div>
</div>
{% include "payments/_cardform.html" %}
</form>

Lets parse it into sections:

Lines 4 - 16 - these lines handle displaying errors that are returned from the server
side validation. We will just leave that as is for the time being.
Lines 17 - 19 - these lines handle displaying the fields that are defined in payments.forms.UserForm.
It uses a template called payments/_field.html to render each of the fields. This is
what we will first need to modify to do client side validation with Angular.
Remaining Lines - these lines handle displaying the credit card information portion
of the form, allowing users to input their credit card info. We have this separate form
so we dont have to pass the actual credit card number to our back-end. Instead, once
a user submits the form, we have some jQuery in static/js/application.js that
will call the Stripe API and get a token that is sent to our back-end. This way we dont
have to worry about storing peoples credit cards and the security that comes along
with that.
Coming back to lines 17-19, our user form fields are displayed using the payments/_field.html
template, which looks like this:
1
2
3
4
5
6

<div class="clearfix">
{{ field.label_tag }}
<div class="input">
{{ field }}
</div>
</div>

This produces very simple HTML for each field:

1
2

<div class="clearfix">
<label for="id_name">Name:</label>
345

3
4
5
6

<div class="input">
<input id="id_name" name="name" type="text">
</div>
</div>

In order to do Angular validation, we need to first add an ng-model to each field in our form.
In our template we are using {{field}} to render the input fields. Lets think about what is
happening with that.

field is a property of the form. If you recall, we called {% for field in form.visible_fields
%}. This will loop through all the fields in our UserForm and display them. But how exactly
does it know what to display? Looking at our payments.forms.UserForm class, we see:
1
2
3
4
5
6
7
8
9
10
11
12
13

class UserForm(CardForm):
name = forms.CharField(required=True)
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
label=('Password'),
widget=forms.PasswordInput(render_value=False)
)
ver_password = forms.CharField(
required=True,
label=('Verify Password'),
widget=forms.PasswordInput(render_value=False)
)

This is a list of all the fields. Pay attention to the password and ver_password fields. They
both have an optional argument widget. In Django, a widget controls how a field is rendered
to HTML. Every field gets a default widget based upon the field type. So CharField will have
a default TextInput widget, EmailField will have a default EmailInput widget, and so
on. For our password fields, we have overwritten the default widget to be a PasswordInput
widget so that the password isnt display when the user types. This is handled by simply
adding the HTML attribute type='password' to the input when it is rendered, and the
widget does that.
Since the widget ultimately controls how the field is rendered and we want all of our fields to
be rendered with an ng-model attribute, we can change the widget and tell it to include the
ng-model attribute for us.
The easiest way to do that is to modify the __init__ function of our UserForm. Behold:
1

class UserForm(CardForm):
346

2
3

ng_scope_prefix = "userform"

4
5
6
7
8

def __init__(self, *args, ** kwargs):

super(UserForm, self).__init__(*args, **kwargs)
for name, field in self.fields.items():
attrs = {"ng-model": "%s.%s" % (self.ng_scope_prefix,
name)}

9
10

field.widget.attrs.update(attrs)
Line 3 - In Angular, its simpler to nest all of our models in one object. In other
words, rather than having models named email, name, etc its easier if we have
userform.email, and userform.name. This way, when it comes time to pass that
information around in Angular (i.e., when we are POSTing our form), we can just
pass userform and not have to reference each individual field. This also has the
added advantage of allowing us to change the names and even the number of fields
in the form without having to make corresponding changes in Angular. So grouping
together all of the $scope for a form is a good thing in Angular. And that is what the
ng_scope_prefix field is for.
Line 5 - Our __init__ function.
Line 6 - Always a good idea to call the __init__ of our parent class in case something
is happening there.
Line 7 - Loop through each field in UserForm.
Line 8 - Each widget uses a dictionary called attrs that stores the HTML attributes that will be rendered when the widget is rendered. Here we are creating a dictionary with one entry that has the key of ng-model and the value of
userform.<fieldName>.
Line 10 - Add our dictionary to the list of attrs the widget will render.

The updated payments.forms.UserForm class now looks like:

1

class UserForm(CardForm):

2
3
4
5
6
7

name = forms.CharField(required=True)
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
label=('Password'),
347

8
9
10
11
12
13
14

widget=forms.PasswordInput(render_value=False)
)
ver_password = forms.CharField(
required=True,
label=('Verify Password'),
widget=forms.PasswordInput(render_value=False)
)

15
16

ng_scope_prefix = "userform"

17
18
19
20
21

def __init__(self, *args, ** kwargs):

super(UserForm, self).__init__(*args, **kwargs)
for name, field in self.fields.items():
attrs = {"ng-model": "%s.%s" % (self.ng_scope_prefix,
name)}

22
23

field.widget.attrs.update(attrs)

24
25
26
27
28
29
30
31

def clean(self):
cleaned_data = self.cleaned_data
password = cleaned_data.get('password')
ver_password = cleaned_data.get('ver_password')
if password != ver_password:
raise forms.ValidationError('Passwords do not match')
return cleaned_data

With this change, if we refresh the browser and look at the fields rendered on the registration
page (http://localhost:8000/register, they should look like:
1
2
3
4

5
6

<div class="clearfix">
<label for="id_name">Name:</label>
<div class="input">
<input id="id_name" name="name" ng-model="userform.name"
type="text" class="ng-pristine ng-valid">
</div>
</div>

There you go. We have our nice ng-model="userform.name" added to our input. Pretty
cool, huh? But wait, there is also something else added: class="ng-pristine ng-valid".
We didnt tell it to do that, did we?

348

Well no, not directly. Angular does that for us: Anytime you add an ng-model to an input
element, Angular will add those classes, and thats a good thing because we need those classes
for data/field validation. Lets look at what classes there are that Angular sets for us to help
with the validation. The four we care about are:
Class

Activated

ng-valid
ng-invalid
ng-pristine
ng-dirty

When the field passes validation

When the field fails validation - i.e., an empty field with a required validation
Before any interaction on the field - i.e. before you type in the input field
After the field has been changed - i.e., after the first keystroke in the input field

Notice that by default the field is assigned ng-pristine, meaning it hasnt been touched yet,
and ng-valid, meaning it passes validation.
Try this: Open your JavaScript console and highlight the name input field. Then click on the
name field and type something. Watch the classes change to:

ng-valid ng-dirty

These signifies that text has been inputted into the field and it passes validation. Lets look
at the email field as an example. Type a valid email, then check out the JavaScript console:

<input id="id_email" name="email" ng-model="userform.email"

type="email" class="ng-dirty ng-valid ng-valid-email">

Youll notice it has an extra class, ng-valid-email, because a field can have multiple validators. This class tells us that the email-validator is currently valid. But how does Angular
know this is an email field? Angular uses the HTML5 attributes to determine which validators a field needs and applies them automatically to the field (as long as it has the ng-model
directive attached). In this case, because the field has the attribute type="email", Angular applies the validator. Angular respects the HTML5 type values as well as the required
attribute for form validation. You can also use:

ng-required - just sets the required attribute to true

ng-minlength or min
ng-maxlength or max
ng-pattern - validates based on a regex pattern

With these you have quite a bit of flexibility and can validate almost anything you want. (It
is also possible to use custom directives to do custom validation.)
349

The only issue we have now is that in our UserForm we have also specified some additional
validatiors, like:
1

name = forms.CharField(required = True)

And we would like those validations to also be on the front-end. So we can go back to our
payments.forms.UserForm.__init__ function and add the appropriate attributes. As an
example, if we wanted to add the required HTML attribute for any field that had required
= True we could add the following code:
1
2
3
4

if field.required:
attrs.update({"required": True})
if field.min_length:
attrs.update({"ng-minlength": field.min_length})

Since we dont have any min_length validations, lets go ahead and change our name field
to require a minimum of three letters:
1

name = forms.CharField(required=True, min_length=3)

You could of course add as many parameters as you wanted. You have total control over what
is rendered.
The __init__ method should now look like:
1
2
3
4

def __init__(self, *args, ** kwargs):

super(UserForm, self).__init__(*args, **kwargs)
for name, field in self.fields.items():
attrs = {"ng-model": "%s.%s" % (self.ng_scope_prefix, name)}

5
6
7
8
9
10

if field.required:
attrs.update({"required": True})
if field.min_length:
attrs.update({"ng-minlength": field.min_length})
field.widget.attrs.update(attrs)

350

Display Error Messages

Now that we have a way to perform validation, we need to alert the user when something is
invalid. The simplest way is to add some custom styling to the Angular classes - ng-valid
, ng-invalid , ng-dirty. That way whenever the validation changes it triggers the CSS,
alerting the end user. Add the following two lines to the static/css/mec.css file:
1
2

input.ng-dirty.ng-valid { border:1px solid Green; }

input.ng-dirty.ng-invalid { border:1px solid Red; }

This will color the input green if it is valid and its been changed, and likewise will color
the input red if its invalid and has been changed. Go ahead and try it out. Navigate to
http://localhost:8000/register. If you type py in the email field and then move to a new
field, it should become outlined in red, because thats not a valid email. Try changing it to
[email protected]. It should now be green.
This isnt super helpful to the user though, because they dont know why the field is invalid.
So lets tell them. Angular to the rescue again!
Angular keeps track of validation errors in an $error object. You can access it through the
form, with the only requirement being that the form has a name attribute. Currently our
registration form has no name, so assuming we add the name user_form, you could check
for errors with this code:
1

user_form.name.$error.required

Or, the more general form:

1

<<form_name>>.<<field_name>>.$error.<<validator-name>>

This value will be set to True if the field is failing the particular validator. Now you could
create your own custom error and show it only when the validator is true. For example:
1

<span ng-show="user_form.name.$error.required">This field is

required</span>

Here ng-show only display the span if the name field is failing the required validation. Of
course we dont want to have to write this kind of stuff over and over for each field, so lets
revisit our templates/payments/_field.html template and modify it to include some
custom errors for each field:
1
2
3

<div class="clearfix">
{{ field.label_tag }}
<div class="input">
351

4
5
6
7

8
9

10

11
12

{{ field }}
</div>
<div class="custom-error"
ng-show="{{form.form_name}}.{{field.name}}.$dirty &&
{{form.form_name}}.{{field.name}}.$invalid">
{{field.label}} is invalid:
<span
ng-show="{{form.form_name}}.{{field.name}}.$error.required">value
is required.</span>
<span
ng-show="{{form.form_name}}.{{field.name}}.$error.email">Input
a valid email address.</span>
</div>
</div>

You can see on line 6 that we are adding a div below each field. That div will show only when
the field is in error - e.g., when the field is $dirty (which corresponds to the ng-dirty class)
and $invalid (which corresponds to the ng-invalid class). Then inside of the div we check
each of the error types - required and email - and show an error message for each of those
errors.
Youll notice that we are using the template variable {{form.form_name}}; we need to add
this to our UserForm in payments.forms.UserForm:
1

form_name = 'user_form'

So the class now looks like:

1

class UserForm(CardForm):

2
3
4
5
6
7
8
9
10
11
12
13
14

name = forms.CharField(required=True, min_length=3)

email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
label=('Password'),
widget=forms.PasswordInput(render_value=False)
)
ver_password = forms.CharField(
required=True,
label=('Verify Password'),
widget=forms.PasswordInput(render_value=False)
)
352

15
16
17

form_name = 'user_form'
ng_scope_prefix = "userform"

18
19
20
21
22

def __init__(self, *args, ** kwargs):

super(UserForm, self).__init__(*args, **kwargs)
for name, field in self.fields.items():
attrs = {"ng-model": "%s.%s" % (self.ng_scope_prefix,
name)}

23
24
25
26
27
28

if field.required:
attrs.update({"required": True})
if field.min_length:
attrs.update({"ng-minlength": field.min_length})
field.widget.attrs.update(attrs)

29
30
31
32
33
34
35
36

def clean(self):
cleaned_data = self.cleaned_data
password = cleaned_data.get('password')
ver_password = cleaned_data.get('ver_password')
if password != ver_password:
raise forms.ValidationError('Passwords do not match')
return cleaned_data

And also in the templates/payments/register.html, lets give the form the same name:
1

<form id="{{form.form_name}}" name="{{form.form_name}}"

accept-charset="UTF-8" action="{% url 'register' %}"
class="form-signin" role="form" method="post" novalidate>

In addition to changing the name of the form you may have noticed the last attribute,
novalidate. This turns off the native HTML5 validation, because we are going to have
Angular handle the validation for us. Finally, lets add a CSS class to style the error message
a bit in static/css/mec.css:
1
2
3
4

.custom-error {
color: #FF0000;
font-family: sans-serif;
}

Now try it out. Youll notice that as you type into the field, an error message will be displayed
until you type the correct info to pass the validation, then once you pass the validation, the
353

error message will disappear and the field will become green. Hows that for rapid feedback!
Next we need to apply validation to the credit card fields in templates/payments/_cardform.html,
which well leave as an exercise for you.
As a final touch you may want to disable the register button until all the fields pass validation.
We talked about how Angular assigns classes such as ng-valid and ng-invalid to fields
that fail validation. It also assigns those same classes to the form, so you can quickly tell if
the entire form is valid or invalid. So using that, we can modify our register button (which is
in templates/payments/_cardform.html) to look like this:
1

<input class="btn btn-lg btn-primary" id="user_submit"

name="commit" type="submit" value="Register"
ng-disabled="{{form.form_name}}.$pristine ||
{{form.form_name}}.$invalid">

So, ng-disabled will ensure the button is disabled if the form is $pristine (meaning no
data has been input on the form yet) or if the form is $invalid (meaning one of the field
validators is failing). So once all data is input correctly then the button will become valid and
you can submit the form.
If you do submit the form, the register view function will pick it up and perform server side
validation. If there is an error, such as the email is already in use, it will send back the error
which will be displayed on top of the form.

354

Form submission with Angular

With the form validation out of the way, lets now look at how we handle form submissions
with Angular. First, we need to add a controller, and then we can have the form call a function
within the controller to handle the actual submission in templates/payments/register.html:
1
2

<section class="container" ng-controller="RegisterCtrl">

<form name="{{form.form_name}}" accept-charset="UTF-8" novalidate
class="form-signin" role="form" ng-submit="register()">

Here we first declare a controller called RegisterCtrl (well add a file called static/js/registerCtrl.js
for that). Then notice how we have removed the id attribute (id="{{form.form_name}})
that was being used for the jQuery function that calls Stripe on submit, but we are going to convert that to Angular so we dont need the id anymore. Finally, ng-submit="register()"
is called to handle the form submission, so we will need to create the register() function
in our controller. Also note that we have removed the action attribute. If we leave the
action attribute in place, the form will submit as it normally would, skipping the Angular
ng-submit handling of form submission.
The rest of the template remains unchanged. Now lets look at our register() function to
see what happens when the form is submitted. First off, lets remove the jQuery call to Stripe
in our static/js/application.js file and implement it in Angular. Before we get to
the JavaScript portion, though, lets update our templates/payments/_cardform.html
template with some Angular goodness so it will work better with our Angular call:
1
2
3
4

<!-<input type="hidden" name="last_4_digits" id="last_4_digits">

<input type="hidden" name="stripe_token" id="stripe_token">
-->

5
6
7
8
9
10

<div id="credit-card">
<div id="credit-card-errors" ng-show="stripe_errormsg">
<div class="custom-error" id="stripe-error-message">
[[ stripe_errormsg ]]</div>
</div>

11
12
13
14
15
16

<div class="clearfix">
<label for="credit_card_number">Credit card number</label>
<div class="input">
<input class="field" id="credit_card_number" type="text"
ng-model="card.number" required>
355

17
18
19
20
21
22

23
24
25
26
27
28

29
30

31
32
33
34
35

36
37
38
39
40
41
42
43

44

</div>
</div>
<div class="clearfix">
<label for="cvc">Security code (CVC)</label>
<div class="input">
<input class="small" id="cvc" type="text" ng-model="card.cvc"
required min=3>
</div>
</div>
<div class="clearfix">
<label for="expiry_date">Expiration date</label>
<div class="input">
<select class="small" id="expiry_month"
ng-model="card.expMonth">
{% for month in months %}
<option value="{{ month }}"{% if soon.month == month %}
selected{% endif %}>{{ month }}</option>
{% endfor %}
</select>
<select class="small" id="expiry_year" ng-model="card.expYear">
{% for year in years %}
<option value="{{ year }}"{% if soon.year == year %} selected{%
endif %}>{{ year }}</option>
{% endfor %}
</select>
</div>
</div>
<br/>
</div>
<div class="actions">
<input class="btn btn-lg btn-primary" id="user_submit"
name="commit" type="submit" value="Register"
ng-disabled="{{form.form_name}}.$pristine ||
{{form.form_name}}.$invalid">
</div>

Mainly we have just added the ng-model directive to the various form fields. Also, on lines
7-9 we are using the model stripe_errormsg to determine if there is an error and to display
any error messages returned from Stripe.

356

Also notice the first two hidden fields that are commented out. We do not need those anymore
because we can just store the value in an Angular model without passing them to the template.
Now lets go back to the register function in our controller. A straight-forward attempt
might look like this:

mecApp.controller('RegisterCtrl', function($scope, $http) {

1
2

$scope.register = function() {
$scope.stripe_errormsg = "";
approve_cc();
};

3
4
5
6
7

approve_cc = function() {
Stripe.createToken($scope.card, function(status, response) {
if (response.error) {
$scope.$apply(function(){
$scope.stripe_errormsg = response.error.message;
});
} else {
$scope.userform.last_4_digits = response.card.last4;
$scope.userform.stripe_token = response.id;
}
});
};
});

8
9
10
11
12
13
14
15
16
17
18
19
20

Save this in a new file called static/js/registerCtrl.js. Make sure to update the base
template:

<script src="{% static "js/registerCtrl.js" %}"

type="text/javascript"></script>

Line 3 - Our register function, which is called on form submit.

Line 4 - This clears out any errors messages that may be displayed on the screen.
Line 5 - Call the approve_cc function, which handles the Stripe processing.
Line 8 - The approve_cc function calls Stripe.createToken in the same way we
did with jQuery. And with the result we set the appropriate models, storing the cards
last 4 digits and the Stripe token. If the Stripe call fails, however, we will just store the
error message.

357

 Line 10-12 - when we call Stripe.createToken and then use a callback to handle
the return value (as we are doing in this example), we are outside of Angulars digest
loop, which means we need to tell Angular that we have made changes to the model.
This is what $scope.$apply does. It lets Angular know that we have made some
changes so Angular can apply them. Most of the time this isnt necessary because Angular will automatically wrap our calls in a $scope.$apply for us (as is the
case when we use$http), but because Angular isnt aware of Stripe, we need this
extra step to ensure our changes are applied.
While the above code works, it uses jQuery-style callbacks as opposed to Angular-style
promises. This will make it difficult, or at least a little bit messy, for us to chain together the
functionality that we will need, which will be:
call Stripe -> post data to Django -> log user in -> redirect to members page ->
handle errors if they occur
Lets rework the Stripe function call to use promises. Also, lets factor it out into a factory to
keep our controller nice and tidy. The factory will look like this:
1

mecApp.factory("StripeFactory", function($q, $rootScope) {

2
3
4
5

var factory = {}
factory.createToken = function(card) {
var deferred = $q.defer();

6
7
8
9

10
11
12

Stripe.createToken(card, function(status, response) {

$rootScope.$apply(function() {
if (response.error) return
deferred.reject(response.error);
return deferred.resolve(response);
});
});

13
14
15
16
17

return deferred.promise;
}
return factory;
});

Notice how the service uses the $q API to wrap the Stripe.createToken call in a promise.
In particular:
358

 Line 5 - $q.defer() will create the promise object.

Line 9 - If Stripe.createToken returns errors then resolve the promise as rejected
(think of it like raise an exception in Python)
Line 10 - If Stripe.createToken returns without error then resolve the promise
(e.g., return with the return value from Stripe.createToken)
Also note because the promise is operating outside of Angulars normal digest loop, we
wrapped the promise resolution in $rootScope.$apply. This should ensure that the
models are updated properly when handling promise resolution.
NOTE: We used $rootScope and not $scope because this factory isnt tied to
any particular controller and thus doesnt have access to $scope. In other words,
the consumer of this factory method doesnt need to worry about calling $apply.

Then to call the service:

1

mecApp.controller('RegisterCtrl', function($scope, StripeFactory) {

2
3
4
5
6

var setToken = function(data) {

$scope.userform.last_4_digits = data.card.last4;
$scope.userform.stripe_token = data.id;
}

7
8
9
10

logStripeErrors = function(error) {
$scope.stripe_errormsg = error.message;
}

11
12
13
14
15
16

$scope.register = function() {
$scope.stripe_errormsg = "";
StripeFactory.createToken($scope.card)
.then(setToken, logStripeErrors);
};

17
18

});

This code should now start to look similar to our UserPollCtrl because we are relying on
promises and an external factory to get the work done. With that we have now replaced our
jQuery Stripe call with an Angular one. In doing so we have changed our form to an Angularcontrolled form, so lets now look at submitting forms with Angular.

359

CSRF Protection
The first item that needs to be addressed when submitting forms from Angular to Django is
the CSRF protection. If we were to just submit our form as it is, Djangos CSRF protection
would complain. In order to rectify that, we can add a configuration setting to our Angular
module in static/js/application.js:
1
2
3
4

mecApp.config(function($interpolateProvider, $httpProvider) {
$interpolateProvider.startSymbol('[[')
.endSymbol(']]');
$httpProvider.defaults.headers.common['X-CSRFToken'] =
$('input[name=csrfmiddlewaretoken]').val();
});
Line 4 - is the newly added line here, and it says: add the X-CSRFToken value
to the headers of all requests that we send, and set that value to the input named
csrfmiddlewaretoken. Luckily for us, that is also the name of the input that is
created with Djangos {% csrf_protection %} tag.

CSRF - Sorted.
Next up is making the actual post request from our controller, which would look like:
1

$http.post("/register", $scope.userForm)

There are two issues with this:

1. Angular sends JSON data, while our register view currently expects url encoded
data. If we want to send URL encoded data in Angular, we would have to encode the
data and then add the proper Content-Type header to the post. Alternatively, we could
change our Django form to handle the incoming JSON.
2. $http makes AJAX requests, and our register view is expecting a standard request,
so we will have to change what it returns and how it returns it.
In fact, since we are going to send JSON asynchronously, we should probably be thinking
about extending our JSON API to include a user resource; in which case, form submissions
would create a new user by POSTing to the user resource and then, if successful, log the user
in and redirect to the user page. That is a pretty big step from where we are now, so lets look
at just changing our existing register view function to handle JSON (and we will leave the
more proper REST implementation as an exercise for the user).
360

How I taught an old view some new JSON tricks.

Accepting JSON in our view looks like this (dont make any changes yet, though):
1

import json

2
3
4
5
6

if request.method == 'POST':
# We only talk AJAX posts now
if not request.is_ajax():
return HttpResponseBadRequest("I only speak AJAX nowadays")

7
8
9

data = json.loads(request.body.decode())
form = UserForm(data)

10
11
12

if form.is_valid():
#carry on as usual
Line 1 - We are going to need JSON; import it at the top of the file.
Line 3 - We are going to leave the GET request unchanged, but for POST
Line 5 - This line checks the header to see if the request looks like its an AJAX request.
In other words, if the header has X-Requested-With: XMLHttpRequest, this check
is going to pass.
Line 6 - Insert attitude (err, an error)
Line 8 - JSON data will be sent in the body. Remember, though, we are dealing with
Python 3 now, and so request.body will come back as a byte stream. json.loads
wants a string, so we use .decode() for conversion.
Line 9 - From here we are off to the races; we can load up our form with the incoming
data and use the Django form the same way we always have.

Providing JSON Responses

Despite all the attitude (the error), our register view function is not very smart right now
because it accepts an AJAX request and returns a whole truck load of HTML when it probably
should be just returning status as JSON. If you submit the form as is now, assuming the data
is correct, it will successfully register the user and then just return the user.html page. But
since this is an AJAX request, our front-end will never redirect to the user page. Lets update
the register view function in payments.views.register to just report status, and then
we can do redirection from Angular.

361

1
2
3
4
5
6

def register(request):
user = None
if request.method == 'POST':
# We only talk AJAX posts now
if not request.is_ajax():
return HttpResponseBadRequest("I only speak AJAX
nowadays")

7
8
9

data = json.loads(request.body.decode())
form = UserForm(data)

10
11
12
13
14
15
16
17
18
19
20
21

if form.is_valid():
try:
customer = Customer.create(
"subscription",
email=form.cleaned_data['email'],
description=form.cleaned_data['name'],
card=form.cleaned_data['stripe_token'],
plan="gold",
)
except Exception as exp:
form.addError(exp)

22
23
24
25
26

27

cd = form.cleaned_data
try:
with transaction.atomic():
user = User.create(cd['name'], cd['email'],
cd['password'],
cd['last_4_digits'],
stripe_id="")

28
29
30
31
32
33

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

34
35
36

except IntegrityError:
resp = json.dumps({"status": "fail", "errors":

362

cd['email'] + ' is already a

member'})

37

38
39
40

else:
request.session['user'] = user.pk
resp = json.dumps({"status": "ok", "url": '/'})

41
42

43
44

45

return HttpResponse(resp,
content_type="application/json")
else: # form not valid
resp = json.dumps({"status": "form-invalid", "errors":
form.errors})
return HttpResponse(resp,
content_type="application/json")

46
47
48

else:
form = UserForm()

49
50
51
52
53
54
55
56
57
58
59
60
61

return render_to_response(
'payments/register.html',
{
'form': form,
'months': list(range(1, 12)),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': user,
'years': list(range(2011, 2036)),
},
context_instance=RequestContext(request)
)

The top part of the function is the same as what we covered earlier in the chapter. In fact the
only difference in the function is what is returned for a POST request:
Line 38-39 - If the registration is successful, rather than redirecting the user we now
return some JSON saying the status is OK and providing a URL to redirect to.
Line 35-36 - If there is an error when registering the user (such as email address
already exists) then return some JSON saying status it failed and the associated errors.
Line 44-45 - If form validation fails, return the failed result as JSON. You might think,
why do we still need to validate on the back-end if we have done it on the front-end?
Since we are decoupled from the front-end now, we cant be 100% sure the front-end
363

validation is working correctly or even ran, so its best to check on the back-end as well
just to be safe.
After updating the register view function to return some JSON, lets update the front-end
to take advantage of that. First thing, lets create another factory to interact with our Django
back-end.

364

Fixing Tests
But first, lets keep our unit tests working. Since we switched to returning json responses, we
will need to update several of the tests in tests/payments/testViews.py. Lets look at
the RegisterPageTests class.
Our register view function now handles both GET and POST requests, so the first thing to
do is change our setUp function to create the two request that we need. Doing so will make
RegisterPageTests.setup look like:
1
2
3
4
5
6
7
8
9
10
11

12
13
14

def setUp(self):
self.request_factory = RequestFactory()
data = json.dumps({
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
})
self.post_request = self.request_factory.post(self.url,
data=data,
content_type="application/json",
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
self.post_request.session = {}

15
16

self.request = self.request_factory.get(self.url)

Above we have created two requests 1. The standard GET request, which is self.request. That remains unchanged.
2. And a new POST request called post_request that issues a POST with the proper json
encoded data data.
Also note that post_request sets the extra parameter HTTP_X_REQUESTED_WITH='XMLHttpRequest'.
This will make request.is_ajax() return TRUE.
From there its just a matter of changing most of the tests to use the post_request
and to ensure the data returned is the appropriate JSON response. For example
RegisterPageTests.test_invalid_form_returns_registration_page will now
look like:
365

def test_invalid_form_returns_registration_page(self):

2
3

with mock.patch('payments.forms.UserForm.is_valid') as
user_mock:

4
5
6
7

user_mock.return_value = False
self.post_request._data = b'{}'
resp = register(self.post_request)

8
9
10

#should return a list of errors

self.assertContains(resp, '"status": "form-invalid"')

11
12
13

# make sure that we did indeed call our is_valid function

self.assertEquals(user_mock.call_count, 1)

The important lines are:

Line 6 - set the JSON data to null to ensure the user form is invalid.
Line 10 - Ensure the JSON Response contains a form-invalid status.
Similar changes need to be made through the remainder of the tests in RegisterPageTests.
Go ahead and try to update the tests yourself. If you get stuck you can grab the updated file
from the Github repo:
test/payments/testViews.py
Now back to the front-end javascript code to support the change to our view code.

366

Breaking Chains
Add the following to static/js/registerCtrl.js:
1
2
3
4

5
6
7
8
9
10

mecApp.factory("UserFactory", function($http) {
var factory = {}
factory.register = function(user_data) {
return $http.post("/register",
user_data).then(function(response)
{
return response.data;
});
}
return factory;
});

Here we are taking care of POSTing the data to our back-end and returning the response.
Then we just have our controller call the factory, assuming of course our Stripe call passed.
We do this with a promise chain.
Lets first have a look at the chain:
1
2
3

$scope.register = function() {
$scope.stripe_errormsg = "";
$scope.register_errors = "";

4
5
6
7
8
9
10

StripeFactory.createToken($scope.card)
.then(setToken, logStripeErrors)
.then(UserFactory.register)
.then(redirect_to_user_page)
.then(null,logRegisterErrors);
};

Above is our register function (which if you recall is tied to our form submission with
ng-submit). The first thing it does is clear out any errors that we may have. Then it calls a
promise chain starting on line 5. Lets go through each line in the promise chain.
StripeFactory.createToken($scope.card) - Same as before: Call Stripe, passing in the credit card info.
.then(setToken, logStripeErrors) - This line is called after the createToken
call completes. If createToken completes successfully then setToken is called; if it
fails then logStripeErrors is called. setToken is listed below:
367

1
2
3
4
5

var setToken = function(data) {

$scope.userform.last_4_digits = data.card.last4;
$scope.userform.stripe_token = data.id;
return $scope.userform;
}

Same as before: We appropriate scope data. The final step in this function is to return
$scope.userform. Why? Because we are going to need it in the next step, so we are just
passing on the data to the next promise in the chain.
Moving on to the logStripeErrors function:
1
2
3
4

var logStripeErrors = function(error) {

$scope.stripe_errormsg = error.message;
throw ["There was an error processing the credit card"];
}

error is stored in the stripe_errormsg (same as before). But now we are throwing an
error! This has to do with how promise chaining works. If an error is thrown in a promise
chain, it will be converted to a rejected promise and passed on to the next step in the chain.
If the next step doesnt have an onRejected handler then that step will be skipped and the
promise will be re-thrown, again and again, until there is no more chain or until there is an
onRejected handler. So looking at our promise chain again:
1
2
3
4
5

StripeFactory.createToken($scope.card)
.then(setToken, logStripeErrors)
.then(UserFactory.register)
.then(redirect_to_user_page)
.then(null,logRegisterErrors);

If createToken returns a rejected promise, it will be handled by logStripeErrors. If

logStripeErrors were to store the error message and do nothing else, it would by default
return a fulfilled promise with a value of null. This is like saying deffered.resolve(null),
which means that the next step in the promise chain, submitForm, would be called, and
then our form would be submitted even though there was an error on the Stripe side. Not
good.
So we throw the error, which is in effect the same as saying deferred.reject('error
msg') because it will be wrapped for us automatically. Since the line .then(submitForm)
has no reject handler, the error will be passed to the next line .then(redirect_to_user_page),
which also has no onRejected promise handler - and so it will be passed on again. Finally
the line .then(null,logRegisterErrors) has an onRejected handler, and so it will
receive the value that was thrown from our logStripeErrors function and process it.
368

This makes it possible for us to break out of the chain by throwing an error and never catching
it or trying to recover from an error and continuing execution along the chain.
The next step in the chain is the good ol UserFactory.register, which does the POST and
returns the response, which is handled by redirect_to_user_page:
1
2
3
4
5
6
7

var redirect_to_user_page = function(response) {

if (response.errors) {
throw response.errors;
} else {
window.location = response.url
}
}

Here we receive the JSON response, and if it has an errors key, we throw the errors (so
they will be handled by the next onRegected handler). If no errors then redirect (with
windows.location, old-school style) to the URL that should have been returned by our
POST call.
On a successful registration, this will be the end of our promise chain. But if we have an error,
then there is one more onRejected handler that will get triggered: logRegisterErrors.
1
2
3

var logRegisterErrors = function(errors) {

$scope.register_errors = errors;
}

Just set them to the appropriate scope value. These errors will be displayed at the top of our
form. So if we have a look at /templates/payments/register.html then we can see our
old error displaying functionality.
This old version1
2

3
4
5
6
7
8
9
10
11

{% if form.is_bound and not form.is_valid %}

<div class="alert alert-{{error.tags}}"> <a class="close"
data-dismiss="alert"></a>
<div class="errors">
{% for field in form.visible_fields %}
{% for error in field.errors %}
<p>{{ field.label }}: {{ error }}</p>
{% endfor %}
{% endfor %}
{% for error in form.non_field_errors %}
<p>{{ error }}</p>
{% endfor %}
369

12
13
14

</div>
</div>
{% endif %}

-is replaced by the Angularified version:

1

2
3
4
5

<div class="alert" ng-show="register_errors"> <a class="close"

data-dismiss="alert">x</a>
<div class="errors">
<p ng-repeat="error in register_errors">[[ error ]]</p>
</div>
</div>

And there you have it! We can now submit our form with Angular (and do a whole bunch of
other cool stuff along the way). That was quite a bit of work but we made it. For convenience,
here is the entire static/js/registerCtrl.js:
1

mecApp.factory("StripeFactory", function($q, $rootScope) {

2
3
4
5

var factory = {}
factory.createToken = function(card) {
var deferred = $q.defer();

Stripe.createToken(card, function(status, response) {

$rootScope.$apply(function() {
if (response.error) return
deferred.reject(response.error);
return deferred.resolve(response);
});
});

7
8
9

10
11
12
13

return deferred.promise;

14
15

16

return factory;

17
18

});

19
20
21
22
23

mecApp.factory("UserFactory", function($http) {
var factory = {}
factory.register = function(user_data) {
return $http.post("/register",
user_data).then(function(response)
370

24
25
26
27
28
29

{
return response.data;
});
}
return factory;
});

30
31

mecApp.controller('RegisterCtrl',function($scope, $http,
StripeFactory, UserFactory) {

32
33
34
35
36
37

var setToken = function(data) {

$scope.userform.last_4_digits = data.card.last4;
$scope.userform.stripe_token = data.id;
return $scope.userform;
}

38
39
40
41
42

var logStripeErrors = function(error) {

$scope.stripe_errormsg = error.message;
throw ["There was an error processing the credit card"];
}

43
44
45
46

var logRegisterErrors = function(errors) {

$scope.register_errors = errors;
}

47
48
49
50
51
52
53
54

var redirect_to_user_page = function(response) {

if (response.errors) {
throw response.errors;
} else {
window.location = response.url
}
}

55
56
57
58

$scope.register = function() {
$scope.stripe_errormsg = "";
$scope.register_errors = "";

59
60
61
62

StripeFactory.createToken($scope.card)
.then(setToken, logStripeErrors)
.then(UserFactory.register)
371

.then(redirect_to_user_page)
.then(null,logRegisterErrors);

63
64
65
66

};

67
68

});

372

Conclusion
Okay. So, we got our form all working in Angular. We learned about promises, using them
to wrap third-party APIs, and about breaking out of promise chains. We also talked a good
deal about validation and displaying error messages to the user and how to keep your Django
forms in-sync with your Angular validation. And of course, how to get Angular and Django to
play nicely together. These last three chapters should give you enough Angular background
to tackle the most common issues youll face in developing a Django app with an Angular
front-end.
In other words, you know enough now to be dangerous! That being said, Angular is a large
framework and we have really just scratched the surface. If youre looking for more on Angular there are several resources on the web that you can check out. Below are some good ones
to start you off:
Egghead Videos
Github repo updates with a list of a ton of resources on Angular
Very well done blog with up-to-date angular articles

373

Exercises
1. We are not quite done yet with our conversion to Angular, as that register view
function is begging for a refactor. A good way to organize things would be to have
the current register view function just handle the GET requests and return the
register.html as it does now. As for the POST requests, I would create a new users
resource and add it to our existing REST API. So rather than posting to /register,
our Angular front-end will post to /api/v1/users. This will allow us to separate the
concerns nicely and keep the code a bit cleaner. So, have a go at that.
2. As I said in the earlier part of the chapter, Im leaving the form validation for
_cardform.html to the user. True to my word, here it is as an exercise. Put in some
validation for credit card, and CVC fields.
3. While the code we added to templates/payments/_fields.html is great for our
register page, it also affects our sign in page, which is now constantly displaying errors.
Fix it!

374

Chapter 16
MongoDB Time!
Youve probably heard something about MongoDB or the more general NoSQL database
craze. In this chapter we are going to explore some of the MongoDB features and how you
can use them within the context of Django.
There is a much longer discussion about when to use MongoDB versus when to use a relational database, but Im going to mainly sidestep that for the time being. (Maybe Ill write
more on this later). In the meantime Ill point you to a couple of articles that address that
exact point:
When not to use MongoDB
When to use MongoDB
For our purposes we are going to look at using MongoDB and specifically the Geospatial
capabilities of MongoDB. This will allow us to complete User Story #5 - Galactic Map:
A map displaying the location of each of the registered padwans. Useful for physical meetups, for the purpose of real life re-enactments of course. By Galactic here we mean global.
This map should provide a graphic view of who is where and allow for zooming in on certain
locations.

375

Building the front-end first

Normally I would start out with the backend - build the models and views and what not - but
sometimes its helpful to do things the other way around. If youre having trouble envisioning
how a particular feature should be implemented it can often be helpful to mockup a user
interface to help you think through the user interactions. You could of course just scribble
one out with a pencil, but for giggles we will start to build the entire UI first. Just to get a
different perspective.
Of course we might want to do a tiny bit of back-end work first. Lets create the Django app
that is going to house all of our user map functionality.
1

$ ./manage.py startapp usermap

Dont forget to add the new app to the INSTALLED_APPS tuple in settings.py as well.

Angular and Google Maps for the win

For our front-end we are going to use Google Maps to display the map plus the super handy
and aptly named Google Maps for AngularJS library/Directive to make it easier to integrate
Google Maps and Angular.
Download the minified versions of angular-google-maps and lodash.js (be sure to get the
underscore build) and add them to your static/js directory.
Now add the two libraries plus a link to the external Google Maps API files to your

__base.html template. By now the scripts portion of your __base.html should look like:
1
2

3
4
5
6
7

<!-- scripts -->

<script src="https://js.stripe.com/v2/"
type="text/javascript"></script>
<script type="text/javascript">
//<![CDATA[
Stripe.publishableKey = '{{ publishable }}';
//]]>
</script>

8
9

10
11

<script src="{% static "js/jquery-1.11.1.min.js" %}"

type="text/javascript"></script>
<script src="{% static "js/bootstrap.min.js" %}"></script>
<script src="{% static "js/angular.min.js" %}"
type="text/javascript"></script>
376

12

13

14

15

16

17

18

19

<script src="//maps.googleapis.com/maps/api/js?sensor=false"
type="text/javascript"></script>
<script src="{% static "js/lodash.underscore.min.js" %}"
type="text/javascript"></script>
<script src="{% static "js/angular-google-maps.min.js" %}"
type="text/javascript"></script>
<script src="{% static "js/application.js" %}"
type="text/javascript"></script>
<script src="{% static "js/userPollCtrl.js" %}"
type="text/javascript"></script>
<script src="{% static "js/loggedInCtrl.js" %}"
type="text/javascript"></script>
<script src="{% static "js/registerCtrl.js" %}"
type="text/javascript"></script>
{% block extrajs %}{% endblock %}

You can see on lines 11-13 the three JS files we need to start working with Google Maps.
While were in the __base.html, lets change the menu and replace the about page (since
we are not really using it) with our new user maps page:
1

2
3

4
5
6
7
8
9

10
11
12
13
14
15
16
17

<nav class="navbar navbar-inverse navbar-static-top"

role="navigation">
<header class="navbar-header">
<button type="button" class="navbar-toggle"
data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="{% url 'home' %}">Mos Eisley's
Cantina</a>
</header>
<div class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="active"><a href="{% url 'home' %}">Home</a></li>
<li><a href="{% url 'usermap' %}">User Map</a></li>
<li><a href="{% url 'contact' %}">Contact</a></li>
{% if user %}
<li><a href="{% url 'sign_out' %}">Logout</a></li>
377

18
19
20
21
22
23
24

{% else %}
<li><a href="{% url 'sign_in' %}">Login</a></li>
<li><a href="{% url 'register' %}">Register</a></li>
{% endif %}
</ul>
</div>
</nav>
Line 16 - this is the new navigation item that points to our usermap page. Of course
this line is going to cause the Django templating engine to blow up unless we add the
appropriately named url to django_ecommerce\urls.py. So, just add another item
to the urlpatterns tuple:

url(r'^usermap/', 'usermap.views.usermap', name='usermap'),

Next lets define the view usermap.views.usermap:

1

from django.shortcuts import render_to_response

2
3
4
5

def usermap(request):
return render_to_response('usermap/usermap.html')

And then create the template (templates/usermap/usermap.html):

1

{% extends '__base.html' %}

2
3

{% load staticfiles %}

4
5

{% block content %}

6
7
8

<div ng-controller="UserMapCtrl">
<google-map center="map.center" zoom="map.zoom"
options="map.options"></google-map>
</div>

10
11

{% endblock content %}

12
13
14

15

{% block extrajs %}
<script src='{% static "js/usermapCtrl.js" %}'
type="text/javascript"></script>
{% endblock %}
378

A few lines to take note of here:

Line 7 - As with our earlier chapters on Angular we define a controller just for this
page. And as usual we have come up with a highly creative and original name for our
controller UserMapCtrl.
Line 8 - A nice Directive courtesy of angular-google-maps. As you have probably
already guessed it will insert a google map onto the page. We can also pass in a number
of attributes to control how the map is displayed - center, zoom, and options. Check
out the API documentation for more details.
Line 14 - This line just loads up our controller where we will configure the map options.
Before we add the controller we need to be sure to inject the google-maps service into our
Angular app within static/js/application.js by modifying the first line to look like:
javascript var mecApp = angular.module('mecApp', ['google-maps']);
The second argument there, which if you recall is our list/array of dependencies, just lists
google-maps as a dependency and then Angular will work some dependency injection
magic for us so that we can use the service anywhere in our Angular application.
Finally, we can create our userMapCtrl controller to actually display the map in static/js/userMap.Ctrl.js
1

mecApp.controller('UserMapCtrl', function($scope) {

2
3
4
5
6
7
8
9
10
11
12

$scope.map = {
center: {
latitude: 38.062056,
longitude: -122.643380
},
zoom: 14,
options: {
mapTypeId: google.maps.MapTypeId.HYBRID,
}
};

13
14

});

This is a very simple controller that just initializes our google-map object.
Remember this line?
1

<google-map center="map.center" zoom="map.zoom"

options="map.options"></google-map>
379

We passed a number of attributes to it in order to configure the google-map. Now in our

controller we define the value for those attributes:
map.options stores misc options about how the map is displayed.
map.zoom is the zoom level for the map; the higher the number the more zoomed in
the map will be.
map.center stores the coordinates of where you want the map to be centered on.
(Bonus points if you can figure out the landmark at those particular coordinates.)
Oh and one more thing. If we actually want the map to display we will need to add a height
for the container in our css. So add the following line to the end of static/css/mec.css:
1
2
3

.angular-google-map-container {
height: 400px;
}

With that you will actually be able to see the map on your web page. And thats it for the basic
map.
To complete the usermap feature we are going to place a bunch of markers all over the map
showing the locations of our users based on the user location data in our MongoDB backend
(which we still need to create).

380

MongoDB vs SQL
To provide the functionality for User Story 5 we need a representation of each users location. Lets dive right into some terminology. MongoDB uses different terminology than your
standard SQL database, so to avoid confusion lets compare SQL with MongoDB:
Standard SQL

MongoDB

database
table
row
column
index
table joins
primary key

database
collection
document
field
index
embedded documents or linking
primary key

Why does MongoDB use different terminology?

Data Modeling in MongoDB

MongoDB is a document oriented database, while most SQL databases are table oriented
databases. Table oriented databases are concerned with modeling data in tables and defining the relationships between those tables. MongoDB on the other hand models data in documents.

Documents
Documents define aggregates of data - e.gm data that is combined together to form a total
quantity or thing.
For example:
An advertising Flyer is an aggregate of a main message and several supporting paragraphs, possibly also consisting of other elements such as images, testimonials, footers
and what not. The point here is that a flyer is an aggregate of multiple pieces.
An Article is an aggregate of an author, and abstract and several sections of text. In
the case of a blog post, an article may include comments, likes and/or social mentions.
Again we are modeling multiple things, as a larger aggregate of those things.

381

With a relational database you might model an Article with an Author table, an Article
table, and a Comments table, etc., and then you would create the necessary joins (relationships) to tie all these tables together so they could function as your data-model. Using an
aggregate data-model however you would create an Article Document that would contain
all the data necessary (authors, article, comments, etc.) to display the article.

Trade offs
There are a few tradeoffs to modeling things as a large aggregate versus a relational model.
1. Querying efficiency - using a join to query across several tables as opposed to a single aggregate is generally slower, so often times read speeds will increase by using an
aggregate data model.
2. Data normalization - in relational data models you are encouraged not to duplicate
data; this reduces the storage costs and can also make it easier to maintain the data integrity as each piece of data only lives in one place. However in Aggregate data models
you are encouraged to duplicate data. This reduces the need for joins and makes the
data model simpler, at the cost of increased storage requirements.
3. Cascading Updates - Imagine the Article relational data model. Imagine we have an
author - lets call him Pete - who wrote 20 articles. Now Pete wants to change his
name to Fred. We just update the authors table to Fred and since all the Articles
only hold a relationship to the Authors table all the articles will be updated as well.
Conversely, in the Aggregate data model, where author information is duplicated in
every Article, we would first have to find all articles with author name of Pete and
then update each of them individually.
4. Horizontal scalability - Because aggregate data models store all information together it
is relatively simple to scale the database horizontally by adding more nodes and sharding the database. With relational databases this can be much more difficult as creating
joins across servers can be painful to say the least. So in general relational data models are easier to scale vertically (adding more horse power to the machine), where as
aggregate data models scale horizontally (adding more machines) with relative ease.
These trade offs are by no means set in stone, there are just to get you thinking about the
effects of using one data model vs another. As always the right choice will depend upon the
particular problem that you are trying to solve. Use the trade offs listed here to help get you
thinking about where a particular data model would fit best.
Now that we have a better understanding of Mongo. Lets start using it.

382

Installing Mongo
We arent going to get very far with MongoDB until we install it. Luckily its way easier than
Postgres. Just head on over to the mongoDB downloads page and download it. There are also
a number of package managers supported that can be found here. brew install mongo anyone?
Next up is to get the python packages we need to support MongoDB. There are two that we
are going to use.
pymongo - this is the mongodb database driver for python
mongoengine - a Django ODM (Object-Document Mapper) for mongodb.
Lets install those:
1

$ pip install mongoengine django-rest-framework-mongoengine

pymongo is a dependency of mongoengine so you dont need to explicitly install it. After that
finishes, update your requirements.txt file to include the new libraries:
1

$ pip freeze > requirements.txt

Cool. Now we are ready to configure Django to start working with mongodb.

383

Configuring Django for MongoDB

With the above installations complete we can connect to a MongoDB database - we can do
CRUD operations against documents, we can execute map reduce statements and generally
do anything we want to do with MongoDB.
For example, to connect to a MongoDB database named test on your machine you would
use the following code from the Python Shell:
1
2

>>> import mongoengine

>>> mongoengine.connect('test')

Thats it. That will create the database named test if it doesnt already exist. Of course
there are more complicated forms. Say you want to connect to a MongoDB instance running
on the machine 192.168.1.15 called golaith that requires a username and password. Then
your connection might look like:
1
2

>>> import mongoengine

>>> mongoengine.connect("mongodb://user:[email protected]/golaith")

Django doesnt support NoSQL databases natively. Meaning there is no django.db.backends

for any of the NoSQL databases. So, we need to just configure MongoDB ourselves. To get
things working with Django we need to add the above MongoDB connection in our settings.py
file. Its best to add this right next to the DATABASES settings so that all the database stuff
can be found in one place. Thus, we can update django_ecommerce/settings.py to like so:
1

# DATABASES

2
3

mongoengine.connect("mec-geodata")

4
5
6
7
8
9
10
11
12
13
14

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db',
'USER': 'djangousr',
'PASSWORD': 'djangousr',
'HOST': 'localhost',
'PORT': '5432',
}
}

Make sure to add the import - import mongoengine

384

Some Notes about MongoDB

In truth you dont have to set your MongoDB connection in settings.py. Its just a convenient
place to do so as that is where all your other configuration is done. But since Django isnt
really providing you any support for MongoDB you can make the connection anywhere you
want to.
Also notice that we are now using two databases - our Postgres database,django_db, and our
MongoDB database, mec-geodata. For us, this is exactly what we want as we are only going
to be using MongoDB to store some goespacial stuff and we will use Postgres for everything
else. But what if you wanted to use MongoDB to store everything?
You still need to define a SQL database. I know that seems illogical, but Django, as designed,
depends upon having a relational database. So if you plan on only using Mongodb then your
settings.py would need to look like this:
1

import mongoengine

2
3

...snip...

4
5

mongoengine.connect("my-mongo-database")

6
7
8
9
10
11
12

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME' : 'dummy_db',
}
}

There are many tutorials out there that will tell you otherwise, but in Django 1.8 you are not
going to get very far if you dont have the minimum DATABASES defined. With this setup you
will basically be storing tables for Django admin and user sessions in the sqlite backend and
everything else in MongoDB. If you want to also store the session information in MongoDB
add the following two lines to your settings.py file:
1
2
3

# use MongoDB for sessions as well

SESSION_ENGINE = 'mongoengine.django.sessions'
SESSION_SERIALIZER = 'mongoengine.django.sessions.BSONSerializer'

385

Django Models and MongoDB

With the setup out of the way, lets create our models to be used by Django.
Edit the usermap/models.py file to add the models. With MongoDB, our Models will be inheriting from mongoengine.Document instead of django.models. Mongoengine is the ODM
(Object Document Mapper) developed by 10Gen (the creators of Mongo), which is meant
to be used very much like Djangos standard model classes, but supports MongoDB on the
backend as opposed to a relational database.
The UserLocation Model is the simplest of the two that we will be creating so lets look at
that first (usermap.models.UserLocation):
1
2
3

from mongoengine.document import Document

from mongoengine.fields import EmailField, StringField, \
PointField, SequenceField

4
5
6
7
8
9
10

class UserLocation(Document):
email = EmailField(required=True, unique=True, max_length=200)
name = StringField(required=True, max_length=200)
location = PointField(required=True)
mappoint_id = SequenceField(required=True)

Creating a mongoengine.Document is really similar to creating any other model you would
normally create in Django. There are just a couple of variations to point out:
Line 6 - Our MongoDB documents all need to inherit from mongoengine.document.Document.
Lines 7 - 8 - Declaring fields for you document with MongoDB is the same as with
the standard Django ORM. As such the two fields (email and name) should feel very
familiar to you.
Line 9 - the location field uses a type you may not be familiar with that is the
PointField type. This is one of MongoDBs Geospatial fields which the mongoengine
ODM has full support for.
Line 10 - mappoint_id is used because Google Maps requires a unique numeric id
for all map points in order to improve caching/performance.
Nice! So with that we have our database setup. Now lets talk about storing user locations
- e.g., geodata in MongoDB. In fact lets just give you a crash course in storing GeoSpatial
information in a database. Then well come back to our models and finish coding them up.

386

A Geospatial primer
This primer is not intended to get you up to speed with the enormous GIS (Geographical
Information Systems) industry. Instead, the point here is just to describe the bare minimum
of terms so you have an idea what is going on, and so you can talk GIS at a party without
sounding like, well, an idiot.

Storing Geospatial information

All geospatial features depend upon how you store the geospatial data. In MongoDB you can
store 3 types of geospatial data - Points, Polygons, and LineStrings. Common use cases for
each:
Points - a particular location (GPS coordinate).
Polygons - a large shape that spans multiple GPS coordinates, often used for buildings,
countries, high pressure zone,s forest fire danger zones, etc..
LineStrings - a series of points, like a road, a flight path, a river, etc..
Each of the above items are stored in a special JSON format called GeoJSON (remember
everything in MongoDB is stored as JSON) and thus they are collectively referred to as GeoJSON Objects.

Calculating Distance
In the Geospatial world indexes are used to calculate distance. More specifically they are
used to calculate geometry, but distance is just a function of geometry. The point is that if
you want to calculate distance between various GeoJSON objects, you have to index them.
And there are several different types of indexes that you can use. Lets look at two of them to
give you a feel for how they work.
Generally, when you calculate distance on a graph you use the good old Euclidean geometry:
1

dist((x, y), (a, b)) = (x - a) + (y - b)

This is great for flat surfaces. Thus, if you want to calculate distance against a flat surface in
MongoDB you would use a 2d index.
But the earth is round, so if we want to deal with distances across a globe - aka the spherical distance - we need to be sure we are calculating with the correct distance formula. In
MongoDB we are looking for an index called 2dshpere.
387

Thats all we need to know for now. We are of course glossing over huge amounts of information and ignoring a lot of the finer details, but for now, YAGNI.
Remember:

There are 3 GeoJSON object types (Points, Polygons, and LineStrings) and
There are 2 surfaces/index types used to calculate geometries (2d, and 2dsphere)
For our purposes we will be using 2dsphere indexes.

388

Mongoengine Support for Geospatial Features

All of what we have talked about above is exposed through mongoengine so you dont need
to go down to the Mongodb level to get access to the geospatial stuff. By declaring the appropriate field type in your mongoengine model (aka the Document), mongoengine will handle
creating the appropriate index for you.
GeoJSON Object

2d Index

2dsphere Index

Point
Polygon
LineString

GeoPointField
GeoPolygonField
GeoLineStringField

PointField
PolygonField
LineStringField

Jumping back to our usermap.models.UserLocation Document1

2
3
4
5

class UserLocation(Document):
email = EmailField(required=True, unique=True, max_length=200)
name = StringField(required=True, max_length=200)
location = PointField(required=True)
mappoint_id = SequenceField(required=True)

-you can see from the last line that our location field has a type of PointField - which means
it is setup to use a 2dshpere index so we will be able to use it to calculate distances across
the globe. And thats exactly what we want.

389

Showing a Dot on a Map

After going through some theory and important background information we now have the
following:
1. A front-end displaying a Google Map that will be used to map users.
2. Mongodb installed and configured correctly with Django.
3. A mongoengine Document defined that will function as our main model to store user
location info.
4. Some good background info on MongoDB and Geospatial data storage.
Now we are back on track. The next thing to do is show a dot on a map. Preferably a dot that
represents a user location.
To do that we still need to do the following things:
1. Create the appropriate serializer for our REST API end point as required by Django
Rest Framework.
2. Create a rest endpoint for our UserLocation data so we can query it from our mapping
front-end.
3. Update the front-end to display the dot.
4. We should probably included testing in there somewhere as well.

Building the REST endpoint

Surprisingly enough we will start with the first item - Building the serializer for our Document.
How do we do that?
Well, MongoDB stores everything as JSON right (or more technically BSON which is just a
binary form of JSON). So what do we need to do? Not much actually
MongoDB Serializers

Mongodb speaks JSON. And we want to send back JSON, so we could simply just return the
UserLocation as JSON:
1

UserLocation.objects().to_json()

390

The to_json function exists on mongoengines Queryset and Document classes, and with
only 9 characters of code all of the documents in the UserLocation collection, will be expunged to JSON.
With to_json and the corresponding from_json you can now easily (de)serialize to/from
JSON - which means you dont really need a serializer class at all. Lets leave out the serializer
class and let MongoDB do the heavy lifting for us this time. to_json all the way!
Restful Endpoint for Mongo

Creating a REST endpoint for MongoDB is similar to creating an endpoint for anything else.
With a few minor caveats, of course. Well stick with Django Rest Framework; however, since
we are not using a DRF serializer things will be slightly different than what we did in the
previous DRF chapter.
Diving right in, lets create a usermaps/json_views.py file, and then lets add a function called
user_locations_lists to handle the GET and POST requests for our UserLocations collection.
GET:

Lets start with the GET request:

1
2
3
4

from rest_framework.decorators import api_view

from rest_framework.response import Response
from usermap.models import UserLocation
import json

5
6
7
8
9
10
11

@api_view(['GET', 'POST'])
def user_locations_list(request):
if request.method == 'GET':
locations = json.loads(UserLocation.objects().to_json())
return Response(locations)

Whats happening?

Lines 7 - 8 - Notice here we are using a function and not a class based view. If you
recall from the earlier DRF chapter we used a class based view that inherited from
ListModelMixinand others. However because mongoengine implements

the queryset object slightly differently than Django's internal

queryset (which DRF class based views are based on)the class
based views don't really work out of the box. So we need to
391

just create a function based view and "do it ourselves".Do note

though we are using the@api-view decorator which if you recall provides us
with the graphical test client and the rest of the DRF goodness.
Line 9 - DRF is based upon serializers which return an object or a query set as a Python
dictionary. Thus the DRF Response object expects to get a dictionary which it will then
convert to JSON before returning the JSON representation to the client. However in
our case with the mongoengine function to_json we already have JSON.
So we have a couple of choices here. We could just not use DRF and instead use a standard
Django HTTPResponse which will happily pass on our JSON data; or, we can give DRF the
dictionary it is expecting by using json.loads().
In this example, we have chosen to stick with DRF and use json.loads() to pass DRF a
dictionary. This may seem a bit convoluted as we converting from JSON to a Python dictionary only so DRF can convert back to JSON. But its also the most straight forward way
to integrate MongoDB with DRF. If performance is an issue here you may look at not using
DRF at all for your REST API that have MongoDB backends, or you can explore the mongoengine to_mongo function which can convert a MongoDB Document (but not a queryset) to
a dictionary.
Line 10 - Use DRFs Response object which takes a Python dictionary and returns the
corresponding JSON to the client.
With that we have a way to return a list of UserLocations as JSON. Exactly what we wanted,
and we only used a couple of lines of code.
POST:

The next part of the REST API for UserLocations is to allow the user to crate a new
UserLocation. We do that with a POST request:
1
2
3
4
5

from rest_framework import status

from rest_framework.decorators import api_view
from rest_framework.response import Response
from usermap.models import UserLocation
import json

6
7
8
9
10

@api_view(['GET', 'POST'])
def user_locations_list(request):
if request.method == 'GET':
392

11
12
13
14

15
16
17
18
19

locations = json.loads(UserLocation.objects().to_json())
return Response(locations)
if request.method == 'POST':
locations UserLocation().from_json(json.dumps(request.DATA))
locations.save()
return Response(
json.loads(locations.to_json()),
status=status.HTTP_201_CREATED
)
Line 1 - We need to import rest_framework.status at the top of our module so we
can return the 201 status code.
Line 13 - As said before we are handling POST requests here.
Line 14 - Here we use the mongoengine from_json to create an object from the JSON
passed to us by the client (request.DATA). Again we jump through a few hoops to
get the JSON to the appropriate format, but once we do that we can create our new
UserLocation with no problem. Also note this is really an upsert, meaning if the
UserLocation JSON data passed in from the client includes a MongoDB ObjectID
then from_json will update the existing document, otherwise it will create a new one.
Line 16 - Here we just return the JSON form of the object that was just upserted with
the appropriate status of 201.

There you have it, thats the REST API that works with MongoDB and provides you the standard DRF functionality.
One last thing to do is add the URL routes in usermap/urls.py:
1

from django.conf.urls import patterns, url

2
3
4
5
6
7

urlpatterns = patterns(
'usermap.json_views',
url(r'^user_locations$', 'user_locations_list'),
)

And then of course we need to reference that from our master URL file, django_ecommerce/urls.py:
1
2
3

from django.conf.urls import patterns, include, url

from django.contrib import admin
from payments import views
393

4
5

6
7

from main.urls import urlpatterns as main_json_urls

from djangular_polls.urls import urlpatterns as
djangular_polls_json_urls
from payments.urls import urlpatterns as payments_json_urls
from usermap.urls import urlpatterns as map_json_urls

8
9
10
11
12

admin.autodiscover()
main_json_urls.extend(djangular_polls_json_urls)
main_json_urls.extend(payments_json_urls)
main_json_urls.extend(map_json_urls)

13
14
15
16
17
18
19
20
21
22

urlpatterns = patterns(
'',
url(r'^admin/', include(admin.site.urls)),
url(r'^$', 'main.views.index', name='home'),
url(r'^pages/', include('django.contrib.flatpages.urls')),
url(r'^contact/', 'contact.views.contact', name='contact'),
url(r'^report$', 'main.views.report', name="report"),
url(r'^usermap/', 'usermap.views.usermap', name='usermap'),

23

# user registration/authentication
url(r'^sign_in$', views.sign_in, name='sign_in'),
url(r'^sign_out$', views.sign_out, name='sign_out'),
url(r'^register$', views.register, name='register'),
url(r'^edit$', views.edit, name='edit'),

24
25
26
27
28
29

# api
url(r'^api/v1/', include('main.urls')),

30
31
32

This is the entirely of django_ecommerce/urls.py, but the important lines are:

Line 7 - here we import the URLs from our usermap.
Line 12 - This is were we add the URLs to our list of JSON URLs so we can access this
JSON API for our usermaps application.
Thats it for the REST API. Next up Angular.

394

Connecting the Dots

Earlier we got our basic map displayed and placed a single dot on it. Now we want those
dots to actually represent the user locations. As per usual in Angular, we will create a factory to first grab the data and then use a promise chain in our Controller to bind the data
appropriately.
Update static/js/userMapCtrl.js:
1

mecApp.controller('UserMapCtrl', function($scope, locations) {

2
3
4
5
6
7
8
9
10
11
12

$scope.map = {
center: {
latitude: 38.062056,
longitude: -122.643380
},
zoom: 14,
options: {
mapTypeId: google.maps.MapTypeId.HYBRID,
}
};

13
14
15
16
17
18

//get all the user locations

$scope.locs = [];
cache = function(locs){
$scope.locs = locs;
}

19
20
21

locations.getAll().then(cache);
});

22
23

mecApp.factory('locations', function($http) {

24
25

var locationUrls = '/api/v1/user_locations';

26
27
28
29

30
31

return {
getAll:
function() { return
$http.get(locationUrls).then(function(response) {
console.log(response);
return response.data;
395

});

32

},

33
34

};

35
36

});

From the angular controller we can see that we are basically just adding the functionality to
talk to the JSON API we just created. Now we need to update the Google map Directive in
the template (templates/usermap/usermap.html):
1
2

4
5

<div ng-controller="UserMapCtrl">
<google-map center="map.center" zoom="map.zoom"
options="map.options">
<markers models="locs" coords="'location'"
idKey="'mappoint_id'"></markers>
</google-map>
</div>

Notice that added a markers Directive as a sub-Directive of our google-map Directive. The
markers Directive is an easy way to display a series of markers from a JSON data source. To
setup the markers Directive we passed the following attribute values:
models=locs - this corresponds to the $scope.locs that we populated in our controller.
coords= location - this is telling the directive that each object
in the model has alocation property that stores the coordinates.
idKey= mapppoint_id - recalling from earlier when we created
ourUser_Locationsmodel with aSequenceField, we are now supplying said field to
Google Maps, making the mapping gods happy.
This is the minimal setup we need to display some markers. So were good! Test it out. Fire
up the app and make sure the user maps work as expected.

396

Getting User Locations

Its all good to populate the map with a bunch of random points that we came up with. But
our user story called for real user locations. So lets use some cool HTML5 magic to grab the
users location and store it in MongoDB.
The ideal location to grab the users location is when they first register, so lets look at adding
the functionality there.
First, lets add a bit of code to grab the users location. Well just put it directly into our

registerCtrl()
1
2
3
4
5
6
7
8

$scope.geoloc = "";
if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(function(position){
$scope.$apply(function(){
$scope.geoloc = position;
});
});
}

In the above code segment navigator.geolocation is how Angular exposes the

HTML5 capability of getting a users location. So we call getCurrentPositions() on
the navigator and that gives us the location we need, which we store in $scope.geoloc.
Now lets add/update a factory in order to add the locations to MongoDB. Because this information pertains directly to the user, lets add the function to the UserFactory:
1
2

3
4
5
6

factory.saveUserLoc = function(coords) {
return $http.post("/api/v1/user_locations",
coords).then(function(response)
{
return response.data;
});
}

Next, add the controller to massage the data a bit:

1
2
3
4
5

saveUsrLoc = function() {
var data = {'name' : $scope.userform.name,
'email' :
$scope.userform.email,
'location' : [$scope.geoloc.coords.longitude,
$scope.geoloc.coords.latitude]};
397

UserFactory.saveUserLoc(data);
return $scope.userform;

6
7
8

The main thing to remember here is you need to store the data in longitude, latitude
as opposed to the more obvious latitude / longitude. Also note that we are returning
$scope.userform, this is so the next call in the user story - e. g., to register the user) gets
the userform passed in.
So now with this new functionality, lets update our RegisterCtrl() promise chain:
1
2
3
4
5
6

StripeFactory.createToken($scope.card)
.then(setToken, logStripeErrors)
.then(saveUsrLoc)
.then(UserFactory.register)
.then(redirect_to_user_page)
.then(null,logRegisterErrors);

So, the only difference is that we added the line .then(saveUsrLoc) which is now called
right before calling UserFactory.register.
There you go. You know have a usermap with MongoDB backend.

398

Conclusion
Two things happened in this chapter that are very important. Perhaps you noticed them.
1. We upgraded our usage of Angular in subtle yet important ways. 21 We had to make
design decision where there wasnt a clean right way to do it.
Lets talk about each of these in turn.

Naming Conventions
Remember this promise chain?
1

Locations.getAll().then(cache);

This chains together the process of retrieving location data from our server and it does exactly
what it says it does.
Compare that to the promise chain we wrote previously in the Djangular chapter.
1

pollFactory.vote_for_item(item).then(getPoll).then(setPoll);

While the later is focused on individual parts of the process - e.g., the pollFactory, the poll
- the former is entirely based on the function - getAll().then(cache).
Put another way, the former promise chain uses verbs for names which better describe
their function. While the latter primarily focuses on the nouns which better describe the
thing they are. The seemingly innocuous change of name represent the start of a paradigm
shift from nouns to verbs, from objects to functions. The more we work with Angular and
JavaScript the more we start to identify with the functional nature of the tools and that
identification is communicated back through the code, largely by the way we choose to name
things.
SEE ALSO: Now would be a great time to take a quick dive into the philosophy
of naming things, and who better to guide you than the amazing author Steve
Yegge. Check out the wonderful blog post Execution in the Kingdom of Nouns.

We are starting to move beyond just making stuff work with the language and we are moving
into a stage of deeper appreciation of the design of the language - and how it can be used to express elegant solutions. This is the point where computer science and software development
really start to get interesting. This is where you move beyond being functional in a language
to becoming a student of the language. This is the gateway to mastery but the road becomes
far more individualized from here.
399

Choose Your Own Integrations

The more we use our newly learned skills to tackle problems and come up with solutions the
more, we learn about the framework - and what makes it tick. Having gotten this far in the
course should have given you a good feel for the framework, but well want to practice more.
Well want to open up an empty text editor and create something from nothing, and do that
over and over again. And through the challenge of solving differing and evolving problems
with the same set of tools, you become better with those tools. And you begin to understand
the nuances of the language.
This is part of the reason why we choose to start from an existing application, adding on
bits and bits of functionality to it: We can start to see the repetitive nature of software development, and how each time through the iteration of test -> code -> pub we reworked
things, adjusting parts of the existing framework. Whether its refactoring the URL structure
or adding various pieces of functionality to the sign up process, we turned a dial, flipped a
lever, tweaked some code until things looked right and then moved forward.
This is the process of incrementally refining our solutions and building better and more robust software - this is software craftsmanship.
Its not just going through a tutorial where everything works. Its about getting things to work
together that arent necessarily meant for each other. Like MongoDB and DRF. While at some
level they are a bit incompatible, but by understanding the frameworks overall architecture
and design decisions, we can come up with a seamless integration (in theory, of course).

400

Exercises
1. Remember how we said that Django doesnt officially have a django.db.backends
for Mongo? Well, it also doesnt have any unit testing support for Mongo either. Remember: Django provides a DjangoTestCase that will automatically clear out your
database and migrate the data models. Write a MongoTestCase that clears out a MongoDB database prior to test execution/between test runs.
Need help? Focus on these parts:
django.test.runner.DiscoverRunner
pymongo.Connection
mongoengine.connection.disconnect()
NOTE: This is a difficult exercise, which not everybody will be able to solve. Take
a shot at it though. May the force be with you.

401

Chapter 17
One Admin to Rule Them All
Now that we have finished all of the user stories from Building a Membership Site, its time
to turn our attention to more of the backend, the non-user-facing functionality. For most
membership sites we will need to update the site data from time to time. There are several
use cases as to why we might need to do that:
1.
2.
3.
4.

Adding/removing/ updating marketing items and associated images.

Posting new user polls.
Posting new user announcements.
Dealing with unpaid users

It sure would be nice if there was some sort of way to provide backend access to our MEC
app so we could do all this directly from the browser, via the Django admin site. In this
chapter, we are going to look at the admin site in detail - how to use it and how to add custom
functionality.

402

Basic Admin
I was onced asked to sum up Djangos basic admin site in one sentence It aint pretty, but
it works.
With just a few lines of code you can enable a functioning admin site that will allow you to
modify your models from a web interface. Its not the most beautiful interface, but - hey for two or three lines of code, we cant really complain. Lets start off by taking a look at the
admin site.
If you havent already, you will need to create a user who can access the admin site:
1
2
3
4
5
6

$ ./manage.py createsuperuser
Username: admin
Email address: [email protected]
Password:
Password (again):
Superuser created successfully.

Once done, fire up the server and navigate to the admin site - http://localhost:8000/admin/
NOTE: Starting in Django 1.8 the admin site is enabled by default.

Log in with the user you just created, and you should see the basic admin site, which should
look something like:
Click around a bit. Its pretty intuitive; just click on the Add or Change button to perfrom
the subsequnet action. Did you notice how none of the models we created are actually shown
yet? Lets fix that.

403

Figure 17.1: Basic Admin

404

All your model are belong to us

In order to take control of our created models, all we have to do is create an admin.py file
and specify which models should be editable in the admin site. Lets start with the main
application, so edit main/admin.py to look like:
1
2

from django.contrib import admin

from main.models import MarketingItem, StatusReport, Announcement,
Badge

3
4

admin.site.register((MarketingItem, StatusReport, Announcement,

Badge,))

When we register a model to be displayed in the Admin view, by default Django will do its
best to come up with a standard form allowing the user to add and edit the model. Fire up the
admin view and have a look at what this produces; you should now see all the above models
listed under the main section. Try to edit one of the models; it should show you a list of
existing models, and you can choose one to edit.
NOTE: If youre not seeing any data in the admin view, you can load the system
data with the following command:
1
2
3

```sh
$ ./manage.py loaddata system_data.json
```

Comming back to our auto-generated admin views - for simple cases, what Django creates
is all you need. However, you can create your own ModelAdmin to control how a model is
displayed and/or edited. Lets do that with the Badge model.
First, the obligatory before shot:
Not horribly exciting. Lets add a few columns:
1
2

from django.contrib import admin

from main.models import MarketingItem, StatusReport, Announcement,
Badge

3
4

admin.site.register((MarketingItem, StatusReport, Announcement, ))

5
6
7

@admin.register(Badge)
405

Figure 17.2: listview default

8
9

class BadgeAdmin(admin.ModelAdmin):
list_display = ('img', 'name', 'desc', )

This will add more information to the list view display so that we can tell what each item is at
a glance. Once the above changes are in place, http://localhost:8000/admin/main/badge/
should look like:

Figure 17.3: listview with columns

Images in the Admin

This is slightly better, but wouldnt it be nice to actually see the images as opposed to the
image names?
To do this, and to allow uploading of new images from the admin site, lets update our badge
model to use the models.ImageField. Before we update our model, lets briefly discuss
Djangos ImageField.
406

The Image Field

ImageField is a Django field that allows for uploading files. Note that it doesnt actually store the image in the database; it stores the image in the file system under the
settings.MEDIA_ROOT directory. Only the meta data about the image is stored in the
database.
To get an ImageField to behave properly, we first have to do a bit of setup:
1.
2.
3.
4.

Install the required libraries.

Define where images are saved.
Define the URL from which images are served.
Allow the development server to serve those images.

Install required libraries

Because an ImageField needs to load / save the image to disk, it requires the Pillow library.
We can install that with pip:
1
2

$ pip install pillow

$ pip freeze > requirements.txt

This will install the library and update our requirements.txt file.
Define where images are saved

The ImageField saves images in a directory designated in settings.MEDIA_ROOT. Lets

be super original and call the directory media. In the settings.py file, simply change
MEDIA_ROOT to:
1

MEDIA_ROOT = os.path.join(SITE_ROOT, 'media')

With this setting in place, when you upload an ImageField it will now be stored under
django_ecommerce/media. You can specify a subdirectory with the upload_to parameter
passed to your ImageField constructor. (We will see an example of this in just a minute.)
Define the URL from which images are served

We could do this directly in our urls.py, but lets stick to Djangos recommendation here
and update settings.MEDIA_URL like so:
1

MEDIA_URL = '/media/'

Do note that the trailing forward slash is required.

407

Allow the development server to serve those images

Normally the Django development server wont give two hoots about your MEDIA_URL.
We can add a special route that will make it available on the development server. Update
django_ecommerce.urls.py by adding the following:
1
2
3

from django.conf import settings

from django.conf.urls.static import static

4
5

urlpatterns = patterns('',

6
7

...snipped out existing urls...

8
9
10

#serve media files during deployment

) + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

That last line will allow the Django development server to find the MEDIA_ROOT and serve
it under the MEDIA_URL setting. It is worth noting that this is considered not viable for production, and so if you set DEBUG=False in your settings.py file it will turn off this staticfiles
view.

Image Thumbnails for Admin

Now we are ready to update our main.models.Badge class to show the images in the admin
view. First, update main.models.Badge to look like:
1

class Badge(models.Model):

2
3
4
5

img = models.ImageField(upload_to="images/")
name = models.CharField(max_length=100)
desc = models.TextField()

6
7
8
9

10
11

def thumbnail(self):
if self.img:
return u'<img src="%s" width="100" height="100" />' %
(self.img.url)
else:
return "no image"

12
13

thumbnail.allow_tags = True
408

14

class Meta:
ordering = ('name',)

15
16

What did we change?

1. img = models.ImageField(upload_to="images/"): changed the img field type

to ImageField. We talked about the ImageField already. The upload_to parameter
will place these images in a subdirectory of our MEDIA_ROOT directory. So with this
setup, our badge images will be uploaded to media/images/.
2. We added a function called thumbnail() that returns a thumbnail of the badge. It
simply returns an image tag with specified height and width, or the string no image if
no image has been uploaded.
3. Als, we added the link thumbnail.allow_tags = True, which we will talk about later.
Now we need to change the admin.py slightly so it references our thumbnail function:
1
2

@admin.register(Badge)
class BadgeAdmin(admin.ModelAdmin):

3
4

list_display = ('thumbnail', 'name', 'desc', )

Thats it.
Now look at the admin view and, well, we dont have any images. Huh? Thats because we
previously stored all of our images in the static/img directory. So we need to move them
to /media/images/ and update the fields in the database. But, hey - we just configured the
admin view to allow us to upload images for badges, so just use the admin view to move the
images by clicking on each badge, and then using the choose file button.
Once done, your list view will now look like:
Looking good!

Displaying images from an ImageField

Now if we log into our main site - http://localhost:8000/sign_in - and click the Show
Achievements link, we will see a bunch of missing images. That is because we changed the
location where all of our images are stored.
We need to update our templates/main/_badges.html by changing the <img> tag so that
it looks like:
409

Figure 17.4: listview_with_pics

1
2
3

<img class="img-circle" src="{{ media_url }}{{ bdg.img.url }}"

width="100" height="100" alt=" {{ bdg.name }}"
title="{{ bdg.name }} - {{ bdg.desc }}">

Notice that we are no longer using the {% static %} directive, rather we are just using
{{media_url}}{{bdg.img.url}}. This is the combination of settings.MEDIA_URL and
the relative URL of our uploaded file provided by the ImageField.
And that about wraps things up for Images. Before moving on to the next admin topic, lets
look at one more example of customizing the list_display.

Controlling list_display with Admin class functions

In the above example we defined the thumbnail function in our model. But we could have
just as easily defined it directly in our BadgeAdmin class. For example, if we wanted to also
show a list of usernames that had a particular badge, we could update the BadgeAdmin class
like so:
1
2

@admin.register(Badge)
class BadgeAdmin(admin.ModelAdmin):

410

list_display = ('thumbnail', 'name', 'desc',

'users_with_badge', )

5
6
7

def users_with_badge(self, badge):

return ";".join("User: %s" % (u.name) for u in
badge.user_set.all())

The function users_with_badge could be included in the BadgeAdmin class as shown

above, or we could have put it in our model as we did with the thumbnail function. Its
up to you; in terms of best practices, if youre only ever going to use the function for your
admin view, put it in the Admin class, to keep your model less cluttered. Otherwise, put the
function in your model.

Fine-grained control of the list display

When defining a function like users_with_badge, the list view in the admin view will have
a column with heading Users with badge, as can be seen in this screenshot:

Figure 17.5: user_with_badge

Like most things in the admin view, this is also customizable. To change the column heading
we can change our users_with_badge function to look like this:
1
2

@admin.register(Badge)
class BadgeAdmin(admin.ModelAdmin):

3
4

list_display = ('thumbnail', 'name', 'desc',

'users_with_badge', )

5
6
7

def users_with_badge(self, badge):

return ";".join("User: %s" % (u.name) for u in
badge.user_set.all())

8
9

users_with_badge.short_description = "Those who are worthy"

The last line in the listing is important here.

411

In that line we are setting a property of the users_with_badge function called short_description
to the name we want to be displayed in the column heading. If it seems strange to set a
property of a function, remind yourself that everything in Python is an object. A function is
an object just like a class is an object, and so we can add properties or even other functions
to a function if we want.
From Djangos point of view, it will ask each item in the list_display if it has a
short_description property. If so, it will use that for the column heading.

short_description isnt the only property that we can use to control how things are displayed. Lets say we wanted to format the cells displayed in the users_with_badges column.
If we update our function to use a little HTML:
1
2
3

4
5
6

def users_with_badge(self, badge):

html = "<h3>Users</h3><ul>"
html += "\n".join("<li><strong>%s</strong></li>" % (u.name) for
u in
badge.user_set.all())
html += "</ul>"
return html

Then in the cells for the users_with_badge column we would actually see the raw HTML
code. Not great. So we can add the property allow_tags to take care of that.
1
2
3

4
5
6

def users_with_badge(self, badge):

html = "<h3>Users</h3><ul>"
html += "\n".join("<li><strong>%s</strong></li>" % (u.name) for
u in
badge.user_set.all())
html += "</ul>"
return html

7
8
9

users_with_badge.short_description = "Those who are worthy"

users_with_badge.allow_tags = True

And then we have a list item. It aint pretty but it works.

One last thing: To avoid potential security issues with a known XSS scripting vulnerability,
you should always use format_html when returning HTML code:
1

from django.utils.html import format_html

2
3
4

def users_with_badge(self, badge):

html = "<h3>Users</h3><ul>"
412

6
7
8

html += "\n".join("<li><strong>%s</strong></li>" % (u.name) for

u in
badge.user_set.all())
html += "</ul>"
return format_html(html)

The admin.py file should now look like:

1
2

from django.contrib import admin

from django.utils.html import format_html

3
4

from main.models import MarketingItem, StatusReport, Announcement,

Badge

5
6

admin.site.register((MarketingItem, StatusReport, Announcement, ))

7
8
9
10

@admin.register(Badge)
class BadgeAdmin(admin.ModelAdmin):

11
12

list_display = ('thumbnail', 'name', 'desc',

'users_with_badge', )

13
14
15
16

17
18
19

def users_with_badge(self, badge):

html = "<h3>Users</h3><ul>"
html += "\n".join("<li><strong>%s</strong></li>" % (u.name)
for u in
badge.user_set.all())
html += "</ul>"
return format_html(html)

20
21
22

users_with_badge.short_description = "Those who are worthy"

users_with_badge.allow_tags = True

413

Editing Stuff
Weve seen a bunch of ways to control how existing data is displayed, but what about editing
it? Of course, from the list view of our badges we can just click on any item in the list, and
Django will take us to the change form where we can edit to our hearts content. By default
it looks like this:

Figure 17.6: badges change view

That actually works pretty well for our purposes. So rather than customize this model, lets
look at our djangular_polls app and its two models - Poll and PollItem. Lets start with
a review of what we just covered. See if you can figure out what the following PollAdmin class
does (which should be defined in djangular_polls/admin.py:
1
2

from django.contrib import admin

from django.utils.html import format_html

3
4

from djangular_polls.models import Poll

5
6
7
8

@admin.register(Poll)
class PollAdmin(admin.ModelAdmin):

9
10
11

list_display = ('publish_date', 'title', 'highest_vote',

'list_items', 'total_votes', )

12
13
14
15
16
17

def highest_vote(self, poll):

try:
return poll.poll_items().order_by('-votes')[0].text
except IndexError:
return "No Poll Items for this Poll"

18

414

19
20
21
22

23
24

def list_items(self, poll):

html = "<h3>%s</h3><ul>" % (poll.title)
html += "\n".join("<li><strong>%s</strong> - %d</li>" %
(pi.text, pi.votes) for pi in
poll.poll_items())
html += "</ul>"
return format_html(html)

25
26
27

list_items.short_description = "Poll results"

list_items.allow_tags = True

We are customizing the list view as we did in the previous section. Here is a screenshot of
what the list view will now look like:

Figure 17.7: Poll list view

As can be seen in the image above, we are not only showing information about the Poll
model, but we are showing quite a bit of information about the associated PollItems. They
are tightly coupled after all, so it makes sense to show them together.

Editing with inlines

However, if we click on one of the links to edit the Poll, we will only be able to edit the title
of the poll not cool. It would be much better if we could edit the PollItems inline. Dont
fear - Django has got you covered with InlineModelAdmin. We need to create one for our
PollItems, and then we will tie it to our PollAdmin:
1. We can create either a TabularInline or a StackedInline; the only difference is the
look and feel. The TabularInline is probably better for our purposes here. Create it
with this nice little two-liner:
1
2

class PollItemInline(admin.TabularInline):
model = PollItem
415

2. Next up is to reference the above inline from our PollAdmin. Add this bit of code to
the top of the class:
1

inlines = [PollItemInline,]

inlines is a list of InlineModelAdmin classes that should be inserted in the change

form view. So once we add the inline, our change form for a Poll item becomes much
more interesting. It should look like this:

Figure 17.8: poll_changeview_with_inline

3. Edit away! This should provide all the functionality you need. You can modify the Poll,
the Poll items, add items, delete items, even change the votes if youre feeling a little
bit devious.
Inlines are simple to configure and make it very easy to edit related models in one place. Do
note that in the case above, we were using an inline to edit the reverse relationship (that is,
edit PollItems from Poll) as opposed to editing the Poll from the PollItem. Inlines will
work from either direction, allowing you to edit in any way that feels natural.
Customizing inlines

We should also point out that an InlineModelAdmin can be customized in much the same
way that a regular ModelAdmin can. For example, if we dont want our pesky administrators
to modify the number of votes for a PollItem, we could modify the PollItemInline class
to look like:
1
2

class PollItemInline(admin.TabularInline):
model = PollItem

3
4
5

readonly_fields = ('votes',)
ordering = ('-votes',)
416

Note we also decided to change the ordering to show the items with the most votes at the top.
The complete list of options for InlineModelAdmins is here.

417

Making things Puuurdy

Weve got some functionality out of the admin view, and it was actually really easy to do. Now
lets make it look good.
Since the rest of our site is all using Bootstrap, shouldnt our admin be doing it as well? If
you really want to get into the nitty-gritty of customizing / overhauling the look and feel of
the admin site then head over to the documentation and check out:
Adding a bit of CSS
Overriding Admin Templates
Replacing the entire admin site
We are going to skip all of that, because a team of really talented devs has done it for us. Gotta
love open source.
We can install django-admin-bootstrapped with pip:
1
2

$ pip install django-admin-bootstrapped

$ pip freeze > requirements.txt

Next all we have to do is add it to our INSTALLED_APPS list somewhere above the
django.contrib.admin app. The two apps to add are:
1
2

'django_admin_bootstrapped.bootstrap3',
'django_admin_bootstrapped',

Fire up the server and have a look:

Ahhh, now thats purdy!

418

Figure 17.9: admin view bootstrap

419

Adminstrating Your Users

Since users are the lifeblood of most sites, its worth taking some extra time to consider the
administration of users. First off, our Users (payments.Users) have a pretty horrible admin
view out of the box, so lets at least get the basic list view cleaned in payments.admin.py
1
2

from django.contrib import admin

from .models import User

3
4
5
6

@admin.register(User)
class UserAdmin(admin.ModelAdmin):

7
8

list_display = ('name', 'email', 'rank', 'last_4_digits',

'stripe_id', )
ordering = ('-created_at', )

Weve seen this before. It just controls what values are displayed in the list view of all our
payments / users. Now we can further customize the change form, which is the form used
to edit an individual user. Lets do that by using fieldsets. fieldsets are a way to break up
the form into several different / unique sections. To better understand, have a look at the
following form, which creates a set of three <fieldset>s called User Info, Billing, and
Badges.
As you can see, the form is broken up into three clearly marked sections, each with a title
and some fields underneath. We can achieve this by adding the fieldsets attribute to our
UserAdmin class, like so:
1
2

from django.contrib import admin

from .models import User

3
4
5
6

@admin.register(User)
class UserAdmin(admin.ModelAdmin):

7
8

9
10
11
12

list_display = ('name', 'email', 'rank', 'last_4_digits',

'stripe_id', )
ordering = ('-created_at',)
fieldsets = (
('User Info', {'fields': ('name', 'email', 'rank',)}),
('Billing', {'fields' : ('stripe_id',)}),
420

Figure 17.10: User fieldsets

('Badges', {'fields' : ('badges',)}),

13
14

Notice that fieldsets is a tuple of two-tuples. Each two-tuple contains the title of the section and a dictionary of attributes for that section. In our case the only attribute we used was
the fields attribute, which lists the fields to be displayed in the fieldset. More information
about fieldsets can be found here.
With that, we now have a decent looking interface to administrate our users. However, we
can make the admin interface even more useful. Lets look at exactly how to do that next.

421

Resetting passwords
Someone is bound to call / email and say, Your password reset isnt working, and I cant
get in the site. This usually means they cant remember the password they put in when they
reset their password 30 seconds ago. What we want to do is to get the password reset for
them right. So lets add that capability to the admin view.
Now if we were using Djangos Default User Model, password resets are included out of the
box. You can navigate to http://127.0.0.1:8000/admin/auth/user/, select a user, and you
will see a change password link. Pretty easy.
However, we are using a custom user model - the one in payments.models.User. And the
password reset form doesnt work for custom user models. If you want a quick and dirty hack
to get it to work, check out this Stack Overflow question - but lets do something a bit more
fun and learn more about customizing the admin interface along the way.
So what are we going to do? Simple: We are going to modify the change form for
payments.models.User to use Angular to make a REST call to reset a users password. In
order to accomplish this amazing feat, we need to do the following:
1. Customize the change_form template for payments / users to add a change password link.
2. Create a REST API to reset the password.
3. Integrate angular.js into the change_form so we can call our REST API.
4. Provide some nice status messages.
Lets go through each of these topics in a bit more detail.

Tweaking the built-in admin templates

The admin interface is built using Django views and templates in the same way that we built
our application. There is really no magic there. This means we can extend (or even replace)
the existing templates to add additional functionality. The template we are interested in extending is called change_form.html. This is the form used when editing a model instance.
Since we have already customized the change form with the fieldsets business above, it
would be good to not blow away the work we have done there. Rather, lets see if we can just
add a link somewhere near the top of the page to reset the password.
We have a few options for how to extend the change_form.html template:

422

 Extend the change_form.html for the entire admin site. (Useful if you want to
add something to every single change form for all your models.) To do this add a

templates/admin/change_form.html
Extend the change_form.html for a particular Django app. If we wanted certain editing functionality for every model in our djangular_polls app, we would choose this
option. To do this add a templates/admin/djangular_polls/change_form.html
Extend the change_form.html for a particular model - i.e., for payments.models.user.
This is exactly what well do. To do this we just need to create an HTML template named
change_form.html and put it in templates/admin/payments/user/change_form.html.
This way we will only override the change form for the User model.

Lets have a look at what our template might look like:

1
2
3

{% extends "admin/change_form.html" %}
{% load staticfiles %}
{% load i18n admin_urls %}

4
5

{% block object-tools-items %}

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

<li class="dropdown" id="menuReset" is-open="isopen">

<a class="dropdown-toggle" href id="navLogin">Reset Password</a>
<div class="dropdown-menu" style="padding:17px;">
<form class="form" id="resetpwd" name="resetpwd" method="post"
ng-submit="resetpass('{{original.id}}')" >
{% csrf_token %}
<input name="pass" id="pass" type="password"
placeholder="New Password" ng-model="pass" required>
<input name="pass2" id="pass2" type="password"
placeholder="Repeat Password" ng-model="pass2" required><br>
<button type="submit" id="btnLogin" class="btn"
>Reset Password</button>
</form>
</div>
</li>

22
23
24

{{ block.super }}
{% endblock %}
Line 1 - Notice that we are extending admin/change_form.html which is the
default change form that ships with Djangos Admin. (Actually, since we installed
423

django-admin-bootstrapped, it is the template from there but they are basically

the same.) If youre curious as to what the default change_form.html template that
ships with Django looks like, you can find it here.
Line 5 - By plugging into the object-tools-items block, we can add additional
links across the top of the page (next to the history link).
Line 7 - 21 - This is a bootstrap dropdown that will prompt the user to enter a new
password twice. It will look like this:

Figure 17.11: Reset Password dropdown

Line 7, 11, 14, 16 - If you look closely you can see that we have already Angularfied
this form. It wont work yet since we dont have Angular set up within the page, but it
doesnt hurt to get a headstart on things.
Line 12 - Notice on this line we are using the context variable {{ original.id }}.
In the admin views, the object that we are currently editing is always called original,
so in this case we are grabbing the id from the object.
Second to last line - Dont forget to call block.super to ensure the rest of the stuff
in the block loads.
Okay. So at this point we have a link across the top to reset password and a nice little form,
but it doesnt do anything. Since we are going to integrate Angular, its best to create a REST
endpoint that we can call to update our password.

424

Reset Password REST endpoint

We have created a number of REST endpoints already, and here is yet another example. Lets
run through it really quick.
The URL we want is /api/v1/users/password/<user id>, which translates to the following in payments/urls.py:
1

from django.conf.urls import patterns, url

2
3
4
5
6
7

8
9

urlpatterns = patterns(
'payments.json_views',
url(r'^users$', 'post_user'),
url(r'^users/password/(?P<pk>[0-9]+)$',
json_views.ChangePassword.as_view(),
name='change_password'),
)

Notice on line 5 that we are using the view json_views.ChangePassword, which looks
like:
1
2
3

from rest_framework import generics, status, permissions

from payments.serializers import PasswordSerializer
from django.http import Http404

4
5
6
7
8
9
10
11
12

class ChangePassword(generics.GenericAPIView):
"""
Change password of any user if superadmin.
* pwd
* pwd2
"""
permission_classes = (permissions.IsAdminUser,)
serializer_class = PasswordSerializer

13
14
15
16
17
18

def get_object(self, pk):

try:
return User.objects.get(pk=pk)
except User.DoesNotExist:
raise Http404

19
20

def put(self, request, pk, format=None):

425

21
22
23
24
25
26
27

user = self.get_object(pk)
serializer = PasswordSerializer(user, data=request.DATA)
if serializer.is_valid():
serializer.save()
return Response("Password Changed.")
return Response(serializer.errors,
status=status.HTTP_400_BAD_REQUEST)

A few things to notice about this class.

1. permission_classes = (permissions.IsAdminUser,) - This means the user
making the request must return True for the is_staff property.
2. get_object function - This grabs the appropriate user based on the pk passed in from
the URL parser.
3. put - We use a put function because we are updating an existing user. We
dont actually have the logic to reset the password in this function; we create a
PasswordSerializer and let it handle all the work of updating / setting the users
password, then we return the appropriate status.
And the final piece to our new REST API for resetting a users password is the PasswordSerializer,
which can be found inpayments.serializers.py. It should look like this:
1
2
3

from payments.models import User

from rest_framework import serializers
PASSWORD_MAX_LENGTH = User._meta.get_field('password').max_length

4
5
6
7
8
9
10
11
12
13
14
15

class PasswordSerializer(serializers.Serializer):
"""
Reset password serializer
"""
password = serializers.CharField(
max_length=PASSWORD_MAX_LENGTH
)
password2 = serializers.CharField(
max_length=PASSWORD_MAX_LENGTH,
)

16
17
18

def validate_password2(self, attrs, source):

pwd2 = attrs[source]
426

19
20
21

pwd = attrs['password']
if pwd2 != pwd:
raise serializers.ValidationError("Passwords don't
match")

22
23

return attrs

24
25
26
27
28
29
30

def restore_object(self, attrs, instance=None):

""" change password """
if instance is not None:
print("set password")
instance.set_password(attrs.get('password'))
return instance

31
32
33

# we don't create new instances

return None
For the above serializer we define two fields password and password2, both with a
max_length set to what is defined in the payments.models.User model.
The function validate_password2 is a custom validation function that validates to
ensure the two passwords passed in match. If not, it will return an error telling the user
Passwords dont match. There is no need to create a validation function for each field,
because both fields are checked in the one validator.
If you recall from our earlier chapters on DRF, restore_object is used when you
want to control how an object is created from a Python native object (aka a dictionary).
In our case we want to call instance.set_password(attrs.get('password')),
which will handle encrypting the new password and adding it to the user (note you still
need to call save, but we are doing that in our ChangePassword view). Further note
that we only restore_objects if we are passed in an existing User (for the value of
instance). This is because it doesnt make sense to try to set the password for a new
user (because we dont have any other user data).

And thats it. We now have a fully functioning REST API for changing passwords. You may
want to test it out by going to http://127.0.0.1:8000/api/v1/users/password/1 and using the
DRF form.
Now back to the front end to add Angular support and call our newly minted API.

427

Adding Angular to the Admin Site

Coming back to our template, since we are extending the admin/change_form.html,
adding Angular support is just a matter of finding the appropriate blocks to use to put in the
necessary parts of the template.
First off, we need to reference the appropriate JavaScript files. In our case, angular.min.js
and an admin.js, which we will create to hold the admin-specific code. We will also add
another JavaScript file, ui-bootstrap-tpls-0.<version>.min.js. This is a library
called UI-Bootstrap, which is a set of Bootstrap components written in Angular. We
will use this to control the navLogin dropdown. Add the following to the bottom of
templates/admin/payments/user/change_form.html:
1
2
3

{% block footer %}
{{ block.super }}
<script src="{% static "js/angular.min.js" %}"
type="text/javascript"></script>
<script src="{% static "js/ui-bootstrap-tpls-0.11.0.min.js" %}"
type="text/javascript"></script>
<script src="{% static "js/admin.js" %}"
type="text/javascript"></script>
{% endblock %}

Make sure you also add the UI-Bootsrap library to your static/js folder. You can grab it
here (make sure you get version 0.11.0) and then save it to the correct folder.
Now we need to add our ng-app and our ng-controller directives to the HTML page. To
ensure we can use Angular functionality throughout, we want to wrap the entire page in our
ng-app / ng-controller directives. Due to the way the admin/change_form.html template is set up, its not very straight forward on how to do that.
First, we will extend the navbar block and add our div like so:
1
2
3
4

{% block navbar %}
<div ng-app="adminApp" ng-controller="AdminCtrl">
{{ block.super }}
{% endblock %}

Notice we didnt close the div. We will close it in the footer block. This in effect will create a div that wraps almost the entire page. Its a bit odd to do things this way, but we are
bound by the blocks that are exposed in admin/change_form.html. We could also choose
not to inherit from admin/change_form.html and instead just create our own template
from scratch, but even though that would make for a clean way to declare ng-app and ngcontroller, thats a bit overkill for our purposes.
428

Putting it all together, our templates/admin/payments/user/change_form.html

should now look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26

{% extends "admin/change_form.html" %}
{% load staticfiles %}
{% load i18n admin_urls %}
{% block navbar %}
<div ng-app="adminApp" ng-controller="AdminCtrl">
{{ block.super }}
{% endblock %}
{% block object-tools-items %}
<li class="dropdown" id="menuReset" is-open="isopen">
<a class="dropdown-toggle" href
id="navLogin">Reset Password</a>
<div class="dropdown-menu" style="padding:17px;">
<form class="form" id="resetpwd" name="resetpwd" method="post"
ng-submit="resetpass('{{original.id}}')" >
{% csrf_token %}
<input name="pass" id="pass" type="password"
placeholder="New Password" ng-model="pass" required>
<input name="pass2" id="pass2" type="password"
placeholder="Repeat Password" ng-model="pass2"
required><br>
<button type="submit" id="btnLogin" class="btn">
Reset Password</button>
</form>
</div>
</li>
{{ block.super }}
{% endblock %}

27
28
29
30
31

32

33

{% block footer %}
</div> <!-- closes the ng-app div -->
{{ block.super }}
<script src="{% static "js/angular.min.js" %}"
type="text/javascript"></script>
<script src="{% static 'js/ui-bootstrap-tpls-0.11.0.min.js' %}"
type='text/javascript'></script>
<script src="{% static "js/admin.js" %}"
type="text/javascript"></script>
429

34

{% endblock %}

Fire up the page in the admin view and click on view source or inspect element from your
browser. You should see our ng-app directive just under the content div like so:
1
2
3
4
5
6
7
8

<div id="content" class="colM">

<div ng-app="adminApp" ng-controller="AmdinCtrl" class="ng-scope">
<!-- ...snip basicaly the enire page... -->
<!-- END Content -->
</div>
</div>
<!-- closes the g-app div -->
<footer id="footer"></footer>

Adding the AdminCtrl

Notice that we created an Angular app called adminApp and a controller called AdminCtrl.
We will define both of those in our newly created js/admin.js file. Starting off with just the
basic definitions, our js/admin.js file should look like:
1

var adminApp = angular.module('adminApp',['ui.bootstrap']);

2
3
4
5

6
7

adminApp.config(function($interpolateProvider, $httpProvider) {
$interpolateProvider.startSymbol('[[').endSymbol(']]');
$httpProvider.defaults.headers.common['X-CSRFToken'] =
$('input[name=csrfmiddlewaretoken]').val();
}
);

8
9

adminApp.controller('AdminCtrl', function($scope) {});

In line one we are injecting the ui.bootstrap module into our adminApp (which gives us
access to Bootstrap componenets implemented in Angular). Everything else we have covered
previous in the Angular chapters, so it should be a review.

Calling the reset password REST Endpoint

First thing we would like to do is hook up the reset password REST endpoint to our form.
We can do that with the ng-submit directive for our resetpwd form. As a reminder, here is
what the form looks like in our change_form.html template.
430

1
2
3
4
5
6
7
8
9
10

<form class="form" id="resetpwd" name="resetpwd" method="post"

ng-submit="resetpass('{{original.id}}')" >
{% csrf_token %}
<input name="pass" id="pass" type="password"
placeholder="New Password" ng-model="pass" required>
<input name="pass2" id="pass2" type="password"
placeholder="Repeat Password" ng-model="pass2" required><br>
<button type="submit" id="btnLogin" class="btn">
Reset Password</button>
</form>
Line 2 - Notice that we are calling the resetpass function on ng-submit and passing
it {{original.id}}. As a reminder, original is a context variable set by Django
admin that is the model instance currently being edited.

Lets create the resetpass function in our AdminCtrl (in js/admin.js). But first we are
going to need an Angular factory to call our REST endpoint. Lets call it AdminUserFactory:
1
2
3
4
5
6
7
8
9
10

adminApp.factory("AdminUserFactory", function($http) {
var factory = {};
factory.resetPassword = function(data) {
var pwdData = {password : data.pass, password2 : data.pass2};
return $http.put("/api/v1/users/password/" + data.user, pwdData)
.then(function(response)
{
return response;
});
};

11
12
13

return factory;
});

We have seen this type of factory before. Basically we are just building the url api/v1/users/password/<use
id> and sending a PUT request with the password data.
Now we can hook up the factory into our AdminCtrl controller and create the resetpass
function:
1

adminApp.controller('AdminCtrl', function($scope, AdminUserFactory)

{

431

3
4
5
6
7
8
9
10

$scope.resetpass = function(userId) {
var data = {
'user': userId,
'pass' : $scope.pass,
'pass2': $scope.pass2
};
return AdminUserFactory.resetPassword(data);
};

11
12

});

Above we inject the AdminUserFactory and call it with our resetpass functionality. You
can test the admin form now, and it should successfully reset the password for the user. However, from the UI you wont recieve any success or failure messages, so lets take care of that
as well.
WARNING: In case its not apparent, with this technique we are sending the
passwords in clear-text back to the server. Thus we should really only do this if
we are operating over an HTTPS connection so we can ensure our passwords are
encrypted while sent across the wire.

Adding Success and Failure Messages

Our Admin Site already makes use of Bootstrap-style alerts to display information to the user.
So lets use those alerts to inform the user of the results of the resetpass call. To do that,
we can extend the form_top block from admin/change_form.html, which is a section of
HTML that will appear right across the top of the form (just below our reset password link)
used to edit our model. We can do that by adding the following to our template:
1
2
3
4

{% block form_top %}
<div class="alert" ng-class="alertClass"
ng-show="afterReset">[[ msg ]]</div>
{% endblock %}
Line 2 - Bootstrap alerts use one of four classes for styling - alert-info,alert-warning,
alert-success, or alert-danger. Each class simply changes the background color
of the alert from white to yellow to green to red respectively. Thus if we successfully
reset the password, we want to use the alert-success class on our alert. Well use
the alert-danger class if we fail to update the password. Somewhere in AdminCtrl
432

we will set the value of alertClass to the appropriate class so things will be displayed
with the appropriate background color.
Line 3 - Here we are using a simple ng-show to control if the alert is displayed or not.
And the text in the alert will be tied to $scope.msg.
Now lets update the AdminCtrl:
1

adminApp.controller('AdminCtrl', function($scope, AdminUserFactory)

{

2
3

$scope.afterReset = false;

4
5
6
7
8
9
10
11
12
13
14

$scope.resetpass = function(userId) {
$scope.afterReset = false;
var data = {
'user': userId,
'pass' : $scope.pass,
'pass2': $scope.pass2
};
AdminUserFactory.resetPassword(data)
.then(showAlert,showAlert);
};

15
16
17
18
19

var showAlert = function(data) {

$scope.afterReset = true;
var msg = "";
$scope.alertClass = "alert-danger";

20

if (data.status == 200) {
$scope.alertClass = "alert-success";
$scope.pass = "";
$scope.pass2 = "";
}

21
22
23
24
25
26

$scope.msg = data.data;

27
28
29

};

30
31

});

Okay. Theres a fair bit going on there, so lets break it down bit by bit:
433

 Line 3- Ensure by default that we dont show the alert.

Line 12 - Here we are taking advantage of Angular promises to call our showAlert
function after the REST call completes. Since our REST call will return a 400 error if
the passwords dont match we set showAlert as both the success and failure handler.
(Thats why it appears to be called twice).
Line 15 - Our new showAlert function.
Lines 16-18 - The first bit of the showAlert function sets $scope.afterReset =
true. This will cause our alert to be shown. We also want to be sure to clear out any
previous messages in our alert, so we will initialize the msg to null. Lets default to a
red background.
Lines 20-24 - This is our success handler we want to ensure our alert will have
the proper green colored background, and we can also clear out our form.
Line 26 - Finally, we set the text of our alert to be the response returned from our
REST call.
This will produce alert messages that look like this:
Success message

Figure 17.12: Alert Success

Failure message

Figure 17.13: Alert Failure

The Success message looks fine, but the failure message looks a bit ugly - {"password2":["Passwords

don't match"]}
This is because DRF will return an array of error messages linked to each field that fails validation / has an error. We can add a special case in our showAlert function to handle this.
Replace:
1

$scope.msg = data.data

With:

434

1
2
3
4
5
6
7
8

if (typeof data.data == 'string') {

msg = data.data;
} else {
for (var x in data.data) {
msg += data.data[x].toString() + " ";
}
}
$scope.msg = msg;

This basically says, If we get back a string (which is the success case) then just display it,
otherwise loop through the list of errors and display them all.
It would be nice to hide the password reset form after we are finished, so lets make that
happen. This is where the ui-bootstrap really helps. If you look back at our HTML for the
menuReset list_item, you will see:
1

<li class="dropdown" id="menuReset" is-open="isopen">

The class dropdown is actually an Angular directive defined in ui-bootstrap that allows
us to programmatically control if the dropdown is displayed or hidden. is-open allows you
to specify the scope variable that will control this. Since we have set it to isopen, all we have
to do is set $scope.isopen to false in our controller to hide the dropdown, or true to show
it. Thus we add $scope.isopen = false to the end of our showAlert function and that
will close the dropdown menu. Now if we put it all together, our controller should look like
this:
1

adminApp.controller('AdminCtrl', function($scope, $http,

AdminUserFactory) {

2
3
4

$scope.afterReset = false;
$scope.isopen = false;

5
6
7
8
9
10
11
12
13
14
15

$scope.resetpass = function(userId) {
$scope.afterReset = false;
var data = {
'user': userId,
'pass' : $scope.pass,
'pass2': $scope.pass2
};
AdminUserFactory.resetPassword(data)
.then(showAlert,showAlert);
};
435

16
17
18
19
20

var showAlert = function(data) {

$scope.afterReset = true;
var msg = "";
$scope.alertClass = "alert-danger";

21

if (data.status == 200) {
$scope.alertClass = "alert-success";
$scope.pass = "";
$scope.pass2 = "";
}

22
23
24
25
26
27

if (typeof data.data == 'string') {

msg = data.data;
} else {
for (var x in data.data) {
msg += data.data[x].toString() + " ";
}
}

28
29
30
31
32
33
34
35
36
37
38

$scope.msg = msg;
$scope.isopen = false;
};

39
40

});

And there you have it. A fully functioning Angular-enabled password reset function for a
custom model in the Admin view. That was a lot of work for a password reset, but hopefully
you learned a good deal about the various ways you can customize the admin view and extend
it to do whatever you need.
Oh, and one more thing - Since we are using custom user models, we may want to hide
Djangos Auth Models from the admin view. We can do that by simply adding the following lines to payments/admin.py:
1
2
3

from django.contrib.auth.models import User as DjangoUser

from django.contrib.sites.models import Site
from django.contrib.auth.models import Group

4
5
6

admin.site.unregister(DjangoUser)
admin.site.unregister(Group)
436

admin.site.unregister(Site)

437

Conclusion
In this chapter we have talked about a number of ways to modify the admin interface. Lets
review quickly.

Adding models to the Admin interface

Just create a ModelAdmin and register it with @admin.register like this:
1
2

from django.contrib import admin

from payments.models import User

3
4
5
6

@admin.register(User)
class UserAdmin(admin.ModelAdmin):
pass

Controlling the list view of a model

Control the fields displayed with the list_display attribute of a ModelAdmin, i.e.:
1
2

class UserAdmin(admin.ModelAdmin):
list_display = ('name', 'email', 'rank', 'last_4_digits',
'stripe_id', )

Remember that the list of fields can be a field or a callable. For example, we can create a
thumbnail view with the following callable:
1
2
3

4
5

def thumbnail(self):
if self.img:
return u'<img src="%s" width="100" height="100" />' %
(self.img.url)
else:
return "no image"

6
7

thumbnail.allow_tags = True

Change column headings with the short_description attribute, i.e.:

1

thumbnail.short_description = "badges"

Control ordering of the list with the ordering attribute using - for reverse ordering, i.e.:
1

ordering = ('-created_at', )
438

Controlling how we edit models

To control the look / grouping of the change_form, we can use the fieldsets property,
which will break the fields into sections with headings and fields:
1
2
3
4
5

fieldsets = (
('User Info', {'fields': ('name', 'email', 'rank',)}),
('Billing', {'fields' : ('stripe_id',)}),
('Badges', {'fields' : ('badges',)}),
)

We can edit related models using inlines:

1
2

class PollItemInline(admin.TabularInline):
model = PollItem

3
4
5

@admin.register(Poll)
class PollAdmin(admin.ModelAdmin):

6
7

inlines = (PollItemInline,)

We can control which fields are read-only and thus not editable with the readonly_fields
attribute:
1
2

class PollItemInline(admin.TabularInline):
model = PollItem

3
4

readonly_fields = ('votes',)

Controlling the look and feel of the admin view

The admin view uses Django Views and Templates just like any other part of Django does, so
we can overwrite or extend those views / templates or even create our own. We talked about
how to extend the admin/change_form.html by creating our own template that extends
from that template, and we talked about some of the various blocks that we might want to extend when we customize the template. Just create the template and the appropriate directory
and start overriding the blocks you need.
You can go quite far using this technique, or you could even create your own Views / Templates and URLs and completely replace the entire Django admin. Do whatever makes the
most sense from you.

439

Other Admin functionatliy

I have tried to cover the most commonly used parts of the Django Admin so you can hit the
ground running. But there is a lot more functionality you can control in the admin such as
searching / filtering, customized CSS, the position of the various controls, adding custom
validation and more.
The Admin Docs do a fairly good job of explaining the functionality, so have a read through
to find our more.

440

Exercises
1. We didnt customize the interfaces for several of our models, as the customization
would be very similar to what we have already done in this chapter. For extra practice, customize both the list view and the change form for the following models:
Main.Announcements
Main.Marketing Items
Main.Status reports
Note both Announcements and Marketing Items will benefit from the thumbnail view
and ImageField setup that we did for Badges.
2. For our Payments.Users object in the change form you will notice that badges section
isnt very helpful. See if you can change that section to show a list of the actual badges
so its possible for the user to know what badges they are adding / removing from the
user.

441

Chapter 18
Testing, Testing, and More Testing
Why do we need more testing?
Throughout this book I have been singing the praises of unit testing. It has helped us catch
some errors in our application, made it easier to refactor, and changed our application as our
requirements changed - and hopefully it has given us a sense of security about the working
nature of our application.
However this security may be a bit unfounded, as we have only really been testing half of the
application - namely the Python half. In the more recent chapters we have added a lot of frontend Angular code, which is becoming increasingly important to the overall functionality of
our application. However, at this point we havent done much testing for that functionality.
In this chapter we are going to cover strategies to test the front-end (and the back-end) of
our application as one whole unit. This is commonly referred to as GUI testing or end-to-end
(E2E) testing.

442

What needs to be GUI tested?

If you recall from the chapter on Test Driven Development, we talked about the importance of unit testing and how it can help in producing solid design and quality code. GUI
testing is the next level after your foundation is covered through unit (and some integration)
testing. We can think of GUI testing as the icing on the cake to really help solidify that quality
of your application. For our app in particular, GUI testing can help us in the following areas:
1. Since GUI testing is end-to-end it will help to cover/test our Angular front-end code.
2. Further GUI testing can easily cover integrations with external services, like when we
use Stripe to validate a credit card.
3. Although we know the individual units of our python code work, we dont have tests to
ensure the end-to-end business processes work as expected and that our python backend code is correctly integrating with our front-end Angular code.
While unit testing is very much about ensuring the individual pieces of code function as expected, GUI testing is much more about ensuring that the application as a whole does what it
supposed to do. Put another way, we are now more concerned with meeting the business requirements as opposed to in unit testing, which is about the technical requirements. When we
combine the two styles of testing, we can have a high degree of certainty that the application
functions as expected.
That being said, it is import to put the various types of testing in their appropriate place and
use them accordingly. To that end, its very helpful to look to the Agile Testing Pyramid
as a reference that helps us determine the amount of effort we should invest into each of the
various types of testing:
As can be seen from the Agile Testing Pyramid above, the majority of our focus in testing
should be in creating unit tests, followed closely by system/integration tests, with the smallest
number of tests being the GUI tests. There are many reasons for this, but it all generally boils
down to the following two:
1. Unit tests are cheaper/faster to write and run.
2. Tests that cover more application logic are more prone to failure and difficult maintenance.
With that in mind, we want to roughly shoot for - 50% unit test, 30% integration tests, and
20% GUI tests. While it is important to understand that these numbers are not hard and fast,
they are a good rule of thumb to help us understand where are efforts are best focused.
443

Figure 18.1: Agile Testing Pyramid

444

Ensuring Appropriate Testing Coverage

Looking inward at our application, lets apply these numbers to the testing that we are doing.
To date, we should have roughly the following number of tests:
Test Type

Number of Tests

Percentage

Unit Test
Integration Tests
GUI Tests

32
15
0

68%
32%
0%

Basically we have 47 tests in our ../tests folder. 15 of those are integration tests since they
deal with the database and are not strictly unit tests. But honestly, the split is a bit arbitrary
(as we discussed in the Unit Testing chapter at the beginning of this course).
Looking at the percentages, we can see that we are not too far off from the suggested numbers
in the Agile Testing Pyramid. We just need to add the GUI tests, roughly 12 of them. Do
note though its more important to look at our requirements and make sure we have tests
for those requirements than to just say, we are going to add 12 tests because that will ensure
we have 20% GUI tests. Just keep in mind that if we start having significantly fewer or more
tests than 12 we should be asking ourselves questions about whether we have the appropriate
number of GUI tests.

What about JavaScript testing?

One thing that is missing from our discussion about testing is JavaScript testing - e.g., writing
unit tests specifically for our Angular code base. There is value in unit testing Angular code
as well. However we will cover a lot of what would be covered by Angular unit tests with the
GUI testing. In the interest of brevity we therefore wont be covering Angular unit testing in
this chapter.
For those readers who are interested to dig into angular unit testing. Here are a few resources
which may be of use.
The Angular Docs
Test Angular with Karma
Testing AngularJS With Protractor and Karma

445

GUI Testing with Django

Enough theory, lets jump into GUI Testing with Django.

Install Selenium
First things first, we need to get Selenium (the tool we use for automated GUI Testing) installed:
1
2

$ pip install selenium

$ pip freeze > requirements.txt

Set up the Directory Structure

Next, since GUI Tests can take a much longer time to run than unit tests, lets create a directory structure that allows us to easily run either unit test or GUI tests - or both. To do that,
lets change the test folder to look like:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

__init__.py
gui
__init__.py
testGui.py
unit
__init__.py
contact
__init__.py
testContactModels.py
main
__init__.py
testJSONViews.py
testMainPageView.py
testSerializers.py
payments
__init__.py
testCustomer.py
testForms.py
testUserModel.py
testViews.py
446

Under our tests directory we created two sub-folders, gui and unit(don't forget
that each folder needs an__init__.py. Then we can run the tests we want by typing
the following from thedjango_ecommerce directory:
1. Unit tests: ./manage.py test ../tests/unit
2. GUI tests: ./manage.py test ../tests/gui
3. All tests: ./manage.py test ../tests
With that out of the way, we can write our first test case.

447

Our First GUI Test Case

Create a file called testGui.py in the tests/gui folder:
1

from django.contrib.staticfiles.testing import

StaticLiveServerTestCase
from selenium import webdriver

3
4
5

class LoginTests(StaticLiveServerTestCase):

6
7
8
9
10

@classmethod
def setUpClass(cls):
cls.browser = webdriver.Firefox()
super(LoginTests, cls).setUpClass()

11
12
13
14
15

@classmethod
def tearDownClass(cls):
cls.browser.quit()
super(LoginTests, cls).tearDownClass()

16
17
18

def test_login(self):
self.browser.get('%s%s' % (self.live_server_url,
'/sign_in'))

Whats going on here?

Line 1: Django provides a LiveServerTestCase that will automatically startup a

server, but LiveServerTestCase doesnt correctly serve static files, upon which our
application depends. Thus, we are going to use StaticLiveServerTestCase as that
will ensure our static files are served correctly by the LiveServer.
Line 2 - In Selenium a webdriver represents a browser. There is a webdriver for all
the major browsers. Also the module selenium.webdriver serves as an entry point
into all of Seleniums functionality.
Line 9 - Initialize the webdriver (in this case the Firefox webdriver) and store it
in a class variable. This line will also cause our Firefox web browser to open when the
test is ran.
Line 14 - When our tests finish, we want to close the browser; calling quit() will close
the browser and end the associated system process.

448

 Line 17 - Our first test. Selenium has a rich API, which we will discuss more in this
chapter. The first API call you need to know about is get, which opens a URL in the
browser. self.live_server_url is provided by Djangos LiveServerTestCase
and points to the root URL for the server started by the test case.
Now run the test with the following command from django_ecommerce:
1

$ ./manage.py test ../tests/gui

You should see the Firefox browser open up and show the sign in page briefly before closing.
Be sure that you see the correctly formatted page with the appropriate looking CSS styles. If
you see a page that looks like it doesnt have CSS, you probably didnt configure your static
files correctly.

449

The Selenium API

That test got things working, but it doesnt really do much. To get Selenium to do stuff (i.e.,
execute GUI tests) we need to examine the API first. With Selenium, the main point of entry
into the API is the webdriver (which, again, can be thought of as the browser). So as we have
shown above, the first step is grabbing a reference to the webdriver:
1

browser = webdriver.Firefox()

Once we have a reference called browser in the line above, we can then instruct the webdriver
to perform various actions. We can break these actions up into three main categories (there
are more, but lets start with these three first):
1. Locating elements
2. Acting on those elements
3. Waitings for things to happen
Jumping back to the previous example
1

self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))

This is an example of navigation, as it causes the browser to navigate to the specified URL. (In
our case the sign_in page.) Once we are on that page, we probably want to enter a username
and password. To do that, we have to locate the appropriate elements, which brings us to

Locating Elements
The webdriver in Selenium exposes an API that lets you interact with a webpages DOM (e.g.,
HTML structure) to locate elements. As soon as you locate an element, a WebElement is
returned that exposes element-specific interactions such as send_keys, clear, click, etc.
In Selenium there are several ways to locate elements, but you typically need to concern yourself with only two or three of these methods.
Locating an element by ID

This should be your default, go-to way to locate an element. Why? Because locating an HTML
element by its HTML ID is about the fastest way you get the browser to parse the DOM and
actually locate the element.
Heres how you do it:
450

email_textbox = self.browser.find_element_by_id("id_email")

Likewise, for the password text box we would use this line:

pwd_textbox = self.browser.find_element_by_id("id_password")

The above two examples will tell the browser to search through the DOM for the element with
the ID specified (remember in HTML, IDs should be unique) and return it.
Locating elements that dont have ids

Sometimes we want to interact with elements that dont have IDs. For example maybe we
want to check to see if the sign-in page has a header that says Sign In. The HTML element
for that looks like this:

<h2 class="form-signin-heading">Sign in</h2>

Since this element doesnt have an ID lets look at the other locators provided by Selenium,
namely:

find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

Each one of these will use a different attribute to search through the DOM and find an element.
However as a best practice there are only two of these locators that I would recommend
using:
find_element_by_name
find_element_by_css_selector
All the other selectors have issues; they are either potentially slow, like find_element_by_xpath,
generally not specific enough like find_element_by_tag_name or find_element_by_class_name
or they are vulnerable to breaking when/if you decide to translate your application like
find_element_by_link_text and find_element_by_partial_link_text.
So that leaves us with:
451

 find_element_by_name this is essentially the same as find_element_by_id, but

it searches for the name attribute instead of the id attribute. Since many HTML forms
use name instead of id, this is a pretty safe option to use.
The next option (and my personal favorite locator) is find_element_by_css_selector.
This uses the same syntax you use in your CSS files; you pass it a CSS selector to find and
element.
Given the HTML form below1
2
3
4
5

8
9
10
11
12
13
14

<label for="id_email">Email:</label>
<div class="input">
<input id="id_email" name="email" type="email">
</div>
<div class="custom-error ng-hide"
ng-show="signin_form.email.$dirty &amp;&amp;
signin_form.email.$invalid">
Email is invalid:<span
ng-show="signin_form.email.$error.required"
class="ng-hide">value is required.</span>
<span ng-show="signin_form.email.$error.email"
class="ng-hide">Input a valid email address.</span>
</div>
</div>
<div class="clearfix">
<label for="id_password">Password:</label>
<div class="input">
<input id="id_password" name="password" type="password">
</div>

-here are a few examples of CSS selectors:

1

self.browser.find_element_by_css_selector("[type=email][name=email]")
# returns the element on line 3
self.browser.find_element_by_css_selector("#id_password") # this is
the same as find_element_by_id
self.browser.find_element_by_css_selector(".ng-hide") # find
element with ng-hide class

Basically anything that you can find/style with your CSS you can locate with Selenium by
using find_element_by_css_selector! Pretty cool.

452

WARNING: One big caveat to keep in mind: find_element_by_css_selector

uses CSS 2.0 selectors and not the 3.0 selectors (made popular by jQuery) that
you may be used to. See here for a complete list of valid CSS 2.0 selectors that
will work with Selenium.

Really between the following three golden selectors find_element_by_id

find_element_by_name
find_element_by_css_selector
-you should be able to locate any element in your application.

453

Acting on elements you have located

Once you locate an element, with a command like1

email_textbox = self.browser.find_element_by_id("id_email")

-you will have a WebElement that you can interact with. In Selenium the WebElement has
a rich API which is described here. We will focus on some of the more common methods in
the API.
For our purposes, we want to sign into our application, so we are going to need to input an
email address and password in the appropriate text boxes.
We can do that with the following bit of code. Update testGui.py:
1
2
3
4
5
6

def test_login(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("[email protected]")
pwd_textbox.send_keys("password")

Notice the last two lines where we used the send_keys() function to type a value into the
appropriate text boxes. send_keys simulates a user typing into the text box, so if you run
the test now with the following command:
1

$ ./manage.py test ../tests/gui

You should actually see the values being typed into the appropriate text boxes.
Next step is to click on the Sign In button. We can do that with the following code, which
first locates the element, and then clicks it:
1

self.browser.find_element_by_name("commit").click()

Here we have done two things in one line just to show that you can chain functions in Selenium if you so desire. Also, since we are submitting a form, we can choose to use the
submit() function, which will find the form that encompasses the element and call submit
on it directly:
1

self.browser.find_element_by_name("commit").submit()

Again, update the code. Regardless of which method you choose, if you run the test again,
you should see the following output:

454

1
2

./manage.py test ../tests/gui

Creating test database for alias 'default'...

3
4

5
6
7

<ul class="errorlist"><li>Incorrect email address or

password</li></ul>
.
---------------------------------------------------------------------Ran 1 test in 5.772s

8
9
10

OK
Destroying test database for alias 'default'...

Notice that Selenium outputted to us what gets returned when we submit the form/click the
button. This is helpful for debugging, and it shows us that we are getting an error message
because the user doesnt exist in the database. This is true, because we are starting with a
fresh database where the test user doesnt exist. We could fix that by creating a user in our
setUpClass function. But first, lets convert this to a negative test so we can verify that we
do indeed get and error message if we input invalid data. This will also show us how to verify
information on the screen.
To do that, lets change the test_login() function like so:
1
2
3
4
5
6

def test_failed_login(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("[email protected]")
pwd_textbox.send_keys("password")

7
8
9

# click sign in
self.browser.find_element_by_name("commit").submit()

10
11
12

13
14

# find the error element

invalid_login =
self.browser.find_element_by_css_selector(".errors")
self.assertEquals(invalid_login.text,
"Incorrect email address or password")

What did we change?

1. We renamed the test to the more appropriate test_failed_login
455

2. We added the last three lines, which grab the errors div and check that the text in the
div is equal to the string we expected.
We could do a slight variation to check that our email validator works correctly like so:
1
2
3
4
5
6

def test_failed_login_invalid_email(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("test@")
pwd_textbox.send_keys("password")

7
8
9

# click signin
self.browser.find_element_by_name("commit").submit()

10
11
12

13
14

# find the error element

invalid_login =
self.browser.find_element_by_css_selector(".errors")
self.assertEquals(invalid_login.text,
"Email: Enter a valid email address.")

This is basically the same test, but we change the text we send to email_textbox and change
the associated check for the error message. You could do the same thing to check for other
validations such as password required, email required, etc.
Lets move on to the positive case where login is successful. First, lets create a user in the
setUp() function.
1
2
3

def setUp(self):
self.valid_test_user = User.create(
"tester", "[email protected]", "test", 1234)

4
5
6

def tearDown(self):
self.valid_test_user.delete()

Above we create the user, and then clean up the created user. Now to write our login test:
1
2
3
4
5

def test_login(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("[email protected]")
456

pwd_textbox.send_keys("test")

7
8
9

# click sign in
self.browser.find_element_by_name("commit").submit()

10
11
12
13

# ensure the user box is shown, as this means we have logged in

self.assertTrue(
self.browser.find_element_by_id("user_info").is_displayed())

Again, very similar to the previous tests, but here on the last line we are trying to find an
element with the ID of user_info. Then we call is_displayed() on that element, which
will return true if it is visible to the user. Since we know the User Info box only shows up
after a successful login, this is a good visual cue we can use to verify that our test is running
correctly.
Selenium Best Practice #1 You should add a check for a visual cue after each
page transition or AJAX action (that should change the display). This way our
test will start to function in the same way a manual tester would test your application, e.g., do a bit of work, check that the application is responding correctly,
do a bit more work, check again, etc.

457

Waiting for things to happen

The way our script is written right now, it will basically run as fast as possible, expecting
the browser/our application to keep up. (Actually thats not entirely true; Selenium is smart
enough to wait for a page to load, but page loads dont include AJAX actions.) Thus, in our
test cases so far, Selenium will wait for the page to load after we navigate to the /sign_in
page, but for many of the other steps, such as displaying the error message, we are using AJAX
calls to update the page. Selenium doesnt know anything about AJAX. Since everything is
running on our local machine, we should be okay, but if we start to test in parallel on a remote
server or our computer randomly slows down for other reasons, our tests may fail because
Selenium has gotten ahead of our application.
Selenium Best Practice #2 Always wait for things to happen, to make your
tests more robust. Things like AJAX calls, javascript animations, or slow test
clients can take time and Selenium isnt very patient. Best to wait

Selenium gives us two main ways of waiting for things to happen:

1. Implicitly
2. Explicitly

Implicit Waits
Webdriver provides the function implicitly_wait() that will set up an implicit wait for
each locator call. We can enable this in our setUpClass method by changing it to read as
follows:
1
2
3
4
5

@classmethod
def setUpClass(cls):
cls.browser = webdriver.Firefox()
cls.browser.implicitly_wait(10)
super(LoginTests, cls).setUpClass()

Line 4 is the important one here. We are telling the webdriver that if at first it doesnt find
an element (when we call any of our find_element functions), then keep retrying to find
that element for up to ten seconds. If the webdriver finds that element in the time span, then
everything is good and the script will continue; if not, then the script will fail.
In the interest of making our test less brittle and more reliable, it is a good idea to always
include a call to implicitly_wait. However, there is one drawback: if you have an
458

implicitly_wait call and you want to check that an element is not displayed, you would
write code like this:
1

self.assertFalse(self.browser.find_element_by_id("user_info").is_displayed())

Keep in mind that in this case, it would take the full 10 seconds for this line to complete
because of the implicitly_wait call we made in our setUpClass.

Explicit Waits
For this reason - waiting - some people dont like to use implicit waits. There are also times
when you know a particular operation may take longer than other things on the page (perhaps
because youre accessing an external API). In those cases (or if you just want to be explicit as
in The Zen of Python, you can use an explicit wait.
Going back to our test_failed_login(), lets put an explicit wait in when checking for
errors. The entire updated test case would look like this:
1
2
3

from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

4
5
6
7
8
9
10

def test_falied_login(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("[email protected]")
pwd_textbox.send_keys("password")

11
12
13

# click sign in
self.browser.find_element_by_name("commit").submit()

14
15
16
17

# find the error element

invalid_login = WebDriverWait(self.browser, 10).until(
EC.visibility_of_element_located((By.CSS_SELECTOR,
".errors")))

18
19
20

self.assertEquals(invalid_login.text,
"Incorrect email address or password")
Lines 1 - 3 - We need these additional imports; put them at the top of your module.
459

 Lines 5 - 14 - Nothing new here.

Lines 16 - 17 - Lets break this apart as there are a number of thing going on here.
The first is:
1

WebDriverWait(self.browser, 10)

Alone this command will just cause the script to pause for ten seconds - not a great idea (as
we already discussed). However, we can chain the until function to our wait, which will
cause the wait to stop waiting as soon as the until function returns a truthy result. until
takes a callable that accepts a single argument.
Lets look at a quick example:
1

WebDriverWait(self.browser,10).until(lambda x: True)

This code will immediately exit out of the WebDriverWait since the result is truthy. Then,
when WebDriverWait exits, it will return the value returned by the callable passed to until,
which in this case is True.
Usually, we want to wait until something happens on the screen, like and element becomes present, or visible, or not visible, or something like that. This is where the
expected_conditions comes in to play. expected_conditions is a Selenium-provided
module that allows us to do a number of checks on the web page. In our specific example,
we are using this bit of code (where EC is the expected_conditions module):
1

EC.visibility_of_element_located((By.CSS_SELECTOR, ".errors"))

Here, we are calling the visiblity_of_element_located() class in the expected_conditions

module. This class will not only call webdriver.find_element but will also call
is_displayed and wait until that returns True before it returns the WebElement.
In effect these two lines will just keep trying to find the .errors element (by default every
0.5 seconds) until it is either found (and visible) or until ten seconds pass. If not found then
the call will raise a selenium.common.exceptions.TimeoutException.
There are other expected_conditions you can use; have a look here. Or you can call your
own function. Besides until there is also an until_not if you want to wait until something
is Falsy.

460

Page Objects
Now that we know how to find elements, wait for them and act on them, we can do most
anything we need to do with Selenium. At this point most people just start writing a lot
of test cases, not giving much thought to maintaining and/or updating the test cases in the
future. Since we have already learned the fact that GUI tests are expensive to maintain, its
worth looking at how we might reduce the cost of maintaining these test cases.
One way to do that is to use the Page Objects pattern.
As the name implies, using Page Objects basically means creating an object for each page
of your application, and encapsulating the GUI testing code for the corresponding page in
that object. For example we could create a login page object and put all the functionality
associated with login in that page object, like filling our username and password, checking
error messages and so on.
This analogy works pretty well with a simple login page but with AJAX-rich applications, the
name Page Object might be a bit misleading, because often a page is too big to encapsulate
in a single object. While our sign_in page maps well to the term page object, our user
page probably doesnt. This is because our user page consists of multiple separate pieces
of functionality - i.e., announcements, status updates, badges, user info, polls, etc.. Thus,
for the user page, we should look to create a page object for each of the separate pieces of
functionality (such as user polls, status updates, etc..)
Again, the point is to make things more maintainable and easy to understand. Motivation for
the need to use page objects can be obtained by looking at the GUI tests we have written thus
far in the chapter, like:
1
2
3
4
5
6

def test_failed_login(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("[email protected]")
pwd_textbox.send_keys("password")

7
8
9

# click sign in
self.browser.find_element_by_name("commit").submit()

10
11
12
13

# find the error element

invalid_login = WebDriverWait(self.browser, 10).until(
EC.visibility_of_element_located((By.CSS_SELECTOR,
".errors")))
461

14
15
16

self.assertEquals(invalid_login.text,
"Incorrect email address or password")

17
18
19
20
21
22
23

def test_failed_login_invalid_email(self):
self.browser.get('%s%s' % (self.live_server_url, '/sign_in'))
email_textbox = self.browser.find_element_by_id("id_email")
pwd_textbox = self.browser.find_element_by_id("id_password")
email_textbox.send_keys("test@")
pwd_textbox.send_keys("password")

24
25
26

# click signin
self.browser.find_element_by_name("commit").submit()

27
28
29

30
31

# find the error element

invalid_login =
self.browser.find_element_by_css_selector(".errors")
self.assertEquals(invalid_login.text,
"Email: Enter a valid email address.")

We can see there is a lot of duplication of code. And just like with our production code, if we
remove this duplication and share code it will make it cheaper to maintain our GUI tests. To
do that, we use the Page Object pattern, of course.
Lets get a clearer picture of how this works by creating a sign_in page object.
First, lets create a new folder - tests/gui/pages/. Be sure to put the __init__.py file in there,
and then create our first page object class in a new file called testPage.py. Starting off with a
base class, lets create the SeleniumPage class:

Page Object Base Classes

1
2
3
4

class SeleniumPage(object):
'''Place to allow for any site-wide configuration you may want
for you GUI testing.
'''

5
6
7
8
9

def __init__(self, driver, base_url="", wait_time=10):

self.driver = driver
self.base_url= base_url
self.wait_time= wait_time
462

Nothing fancy here - The class just allows us to set a few necessary variables. Next, create a
base class for the elements on the page:
Python from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support
import expected_conditions as EC
class SeleniumElement(object):
1
2

def __init__(self, locator):

self.locator = locator

3
4
5
6
7
8

def __get__(self, obj, owner):

driver = obj.driver
wait_time = obj.wait_time
return WebDriverWait(driver, wait_time).until(
EC.visibility_of_element_located(self.locator))

A couple of things of interest here:

1. The __init__ function initializes the element with a locator so we can find it. A locator, here, takes the form of something like (BY.ID, "id_email")
2. The __get__ function is one of those special Python methods. Its used for property
access, so when we call the get property for this object, this bit of code will run. Pay
attention to the second parameter, obj. This is the object that the property is called
on. In other words, if you were do to this1
2
3
4

class Page(SeleniumPage):
email = SeleniumElement((By.ID, "id_email"))
def dummy_method(self):
self.email

-the last line of the code above, where we call self.email, would call the __get__ method
of email and pass in Page for the obj parameter.
Why does this matter? For our purposes, its quite handy as it allows us an easy way to
pass values (i.e. driver and wait_time) from the containing class (Page) to our element
(email). This lets us set explicit wait times once per page object and not have to worry about
setting it each time we try to access an element.

Without further ado the Page Object

With the base classes written, lets look at what our SignInPage page object looks like:
463

class SignInPage(SeleniumPage):

2
3
4
5
6
7

email_textbox = SeleniumElement((By.ID, "id_email"))

pwd_textbox = SeleniumElement((By.ID, "id_password"))
sign_in_button = SeleniumElement((By.NAME, "commit"))
error_msg_elem = SeleniumElement((By.CSS_SELECTOR, ".errors"))
sign_in_title = SeleniumElement((By.CSS_SELECTOR,
".form-signin-heading"))

8
9
10
11
12

def do_login(self, email, pwd):

self.email_textbox.send_keys(email)
self.pwd_textbox.send_keys(pwd)
self.sign_in_button.submit()

13
14
15
16

@property
def error_msg(self):
return self.error_msg_elem.text

17
18
19
20

@property
def rel_url(self):
return '/sign_in'

21
22
23
24

def go_to(self):
self.driver.get('%s%s' % (self.base_url, self.rel_url))
assert self.sign_in_title.text == "Sign in"

Lets look at this class piece-by-piece:

Lines 3 - 7 - This is where we define the element on the page. Defining elements this way ensures that all the elements are using our custom locating code in
SeleniumElement.__get__, plus it exposes the elements to the unit test in case we
want to use them directly.
Lines 9 - 12 - For common functions that we need to perform on a page to test
it, its helpful to create a method in our page object. As such, we have created the
do_login() method on these lines, which fills in the email and password fields and
submits the form.
Lines 14-16 - Since we will be testing the error messages displayed frequently, we
have created a nice helper property to make that easy to do.
Lines 18-20 - In the spirit of encapsulation, we can put the relative URL for our page
here so we dont need to litter our tests with the URL.
464

 Lines 22-24 - For page objects that represent an actual page, its helpful to have a
go_to() function that will navigate to the page and verify that we are indeed on the
correct page.
Thats our page object. For clarification purposes, here is the entire code for the sign in test:
1
2
3
4

from
from
from
from

selenium import webdriver

selenium.webdriver.common.by import By
selenium.webdriver.support.ui import WebDriverWait
selenium.webdriver.support import expected_conditions as EC

5
6
7

from payments.models import User

from django.contrib.staticfiles.testing import
StaticLiveServerTestCase

8
9
10
11
12
13

class SeleniumPage(object):
'''Place to allow for any site-wide configuration you may want
for you GUI testing.
'''

14
15
16
17
18

def __init__(self, driver, base_url=None, wait_time=10):

self.driver = driver
self.base_url = base_url
self.wait_time = wait_time

19
20
21

class SeleniumElement(object):

22
23
24

def __init__(self, locator):

self.locator = locator

25
26
27
28
29
30

def __get__(self, obj, owner):

driver = obj.driver
wait_time = obj.wait_time
return WebDriverWait(driver, wait_time).until(
EC.visibility_of_element_located(self.locator))

31
32
33

class SignInPage(SeleniumPage):
465

34
35
36
37
38
39

email_textbox = SeleniumElement((By.ID, "id_email"))

pwd_textbox = SeleniumElement((By.ID, "id_password"))
sign_in_button = SeleniumElement((By.NAME, "commit"))
error_msg_elem = SeleniumElement((By.CSS_SELECTOR, ".errors"))
sign_in_title = SeleniumElement((By.CSS_SELECTOR,
".form-signin-heading"))

40
41
42
43
44

def do_login(self, email, pwd):

self.email_textbox.send_keys(email)
self.pwd_textbox.send_keys(pwd)
self.sign_in_button.submit()

45
46
47
48

@property
def error_msg(self):
return self.error_msg_elem.text

49
50
51
52

@property
def rel_url(self):
return '/sign_in'

53
54
55
56

def go_to(self):
self.driver.get('%s%s' % (self.base_url, self.rel_url))
assert self.sign_in_title.text == "Sign in"

Thats it! Now lets refactor our tests.

Page Object in Action

We could rewrite our entire page test set:
1
2
3
4

from
from
from
from

selenium import webdriver

selenium.webdriver.common.by import By
selenium.webdriver.support.ui import WebDriverWait
selenium.webdriver.support import expected_conditions as EC

5
6
7

from payments.models import User

from django.contrib.staticfiles.testing import
StaticLiveServerTestCase

466

9
10

class LoginTests(StaticLiveServerTestCase):

11
12
13
14
15
16

@classmethod
def setUpClass(cls):
cls.browser = webdriver.Firefox()
cls.browser.implicitly_wait(10)
super(LoginTests, cls).setUpClass()

17
18
19
20
21

@classmethod
def tearDownClass(cls):
cls.browser.quit()
super(LoginTests, cls).tearDownClass()

22
23
24
25
26

def setUp(self):
self.valid_test_user = User.create(
"tester", "[email protected]", "test", 1234)
self.sign_in_page = SignInPage(self.browser,
self.live_server_url)

27
28
29

def tearDown(self):
self.valid_test_user.delete()

30
31
32
33
34
35

def test_login(self):
self.sign_in_page.go_to()
self.sign_in_page.do_login("[email protected]", "test")
self.assertTrue(
self.browser.find_element_by_id("user_info").is_displayed())

36
37
38
39
40
41

def test_falied_login(self):
self.sign_in_page.go_to()
self.sign_in_page.do_login("[email protected]", "password")
self.assertEquals(self.sign_in_page.error_msg,
"Incorrect email address or password")

42
43
44
45
46
47

def test_failed_login_invalid_email(self):
self.sign_in_page.go_to()
self.sign_in_page.do_login("test@", "password")
self.assertEquals(self.sign_in_page.error_msg,
"Email: Enter a valid email address.")
467

48
49
50
51
52
53

class SeleniumPage(object):
'''Place to allow for any site-wide configuration you may want
for you GUI testing.
'''

54
55
56
57
58

def __init__(self, driver, base_url=None, wait_time=10):

self.driver = driver
self.base_url = base_url
self.wait_time = wait_time

59
60
61

class SeleniumElement(object):

62
63
64

def __init__(self, locator):

self.locator = locator

65
66
67
68
69
70

def __get__(self, obj, owner):

driver = obj.driver
wait_time = obj.wait_time
return WebDriverWait(driver, wait_time).until(
EC.visibility_of_element_located(self.locator))

71
72
73

class SignInPage(SeleniumPage):

74
75
76
77
78
79

email_textbox = SeleniumElement((By.ID, "id_email"))

pwd_textbox = SeleniumElement((By.ID, "id_password"))
sign_in_button = SeleniumElement((By.NAME, "commit"))
error_msg_elem = SeleniumElement((By.CSS_SELECTOR, ".errors"))
sign_in_title = SeleniumElement((By.CSS_SELECTOR,
".form-signin-heading"))

80
81
82
83
84

def do_login(self, email, pwd):

self.email_textbox.send_keys(email)
self.pwd_textbox.send_keys(pwd)
self.sign_in_button.submit()

85
86

@property
468

87
88

def error_msg(self):
return self.error_msg_elem.text

89
90
91
92

@property
def rel_url(self):
return '/sign_in'

93
94
95
96

def go_to(self):
self.driver.get('%s%s' % (self.base_url, self.rel_url))
assert self.sign_in_title.text == "Sign in"

As you can see, not only have we saved a lot of code, but our tests are pretty clear and readable
now as well. Double win!
Believe it or not, thats 90% of what you need to know to successfully test your web page with
Selenium. Of course, that remaining 10% can be pretty tricky. In order to help with that, the
final section in this chapter will list a few gotchas you may run into when writing tests for our
application.

469

When Selenium Doesnt Work

Sometimes in GUI automation, things dont really work the way you think they might. This
section is meant to be a rapid-fire list of common problems that you may have when testing
our application and how to solve them.

Selecting elements from a dropdown

This is not supported out of the box.

How do you select from drop-downs such as expiration month/year on the registration page?
Keep in mind that Selenium works on the DOM. So when you call driver.find_element_by(),
Selenium will search through the entire page to find the element. But did you know that
each WebElement also supports the find_element_by function? Thus to set a dropdown,
find the dropdown element, then use that element to find the option you want. Then call
click. So if we wanted to select the year 2017 from our expiration year dropdown, the code
would look like this:
1
2
3

dd = self.driver.find_element_by_id('expiry_year')
option = dd.find_element_by_css_selector("option[value='2017']")
option.click()

In the second line we are only searching through the HTML elements that are children of the
expiry_year drop down. This ensures we dont click on the wrong dropdown option.

Controlling Browser Options

If you remember back to the MongoDB Time chapter, we created a function to get the users
current location from the browser. So if we try to register while running a Selenium test, the
browser will give us a popup asking if we want to allow the website to get our current location.
We can use a browser profile to let the browser know that we always want to answer yes to
that question. For Firefox, we have the Selenium object FirefoxProfile that allows us to
control those sorts of things. Here is how we would use that to ensure we always get back a
geolocation:
1

from selenium.webdriver.firefox.firefox_profile import

FirefoxProfile

2
3
4

@classmethod
def setUpClass(cls):
470

5
6
7
8

profile = FirefoxProfile()
profile.set_preference('geo.prompt.testing', True)
profile.set_preference('geo.prompt.testing.allow', True)
cls.browser = webdriver.Firefox(profile)

9
10
11

cls.browser.implicitly_wait(10)
super(RegistrationTests, cls).setUpClass()
Line 1 - The FirefoxProfile() is the class that encapsulates all the browser specific
settings for Firefox.
Lines 5 - 7 - Create the FirefoxProfile() and set the preferences that we need for
our test.
Line 8 - Create a new Firefox webdriver, passing in the profile we just created.

Now you can control any of the hundreds of preferences that are available in Firefox.

You need to run some JavaScript code

The above code only works in certain versions of Firefox, and not at all for other browsers.
But you can also mock out the geolocation call with a bit of JavaScript. To run JavaScript
within Selenium you use the webdriver.execute_script() function. Heres an example
that will set our Angular scope as if the HTML5 geolocation was called:
1

email_textbox = self.driver.get_element_by_id("id_email")

2
3
4
5
6
7

self.driver.execute_script('''
angular.element($("#id_email")).scope().geoloc = {
'coords': {'latitude':'1', 'longitude':'2'}
};
angular.element(document.body).injector().get('$rootScope').$apply();''')

How does that work?

A few things are going on here:
1. driver.execute_script is Seleniums way to call JavaScript from your GUI tests.
2. We call angular.element($("#id_email")).scope() to get the scope of our controller. Keep in mind that Angular can have multiple scopes on a page so we are explicitly saying get the scope for the element #id_email. Note we can use any element
that is in the Angular scope we are looking for. On the registration page, this could be
anything in the registration form.
471

3. As for the second command in our execute_script call, we are getting Angulars
rootScope and calling apply (which you should always do when updating scope variables from outside of Angular).
After this bit of JavaScript is executed, we will have $scope.geoloc in our registrationCtrl
set to the value we specified, thus we can do the geolocation for the user with whatever lat /
long we decide to provide.
This is helpful as sometimes you may need to set certain Angular values to get this to work
correctly in your Selenium tests.

Running tests on multiple browsers

Selenium supports many different browsers, including Firefox, Chrome, IE, Opera, Safari,
PhantomJS, and Android / iOS mobile browsers out of the box. Using a different browser
is as simple as creating a different webdriver. For example, to run against Chrome, simply
change your setUpClass as such:

@classmethod
def setUpClass(cls):
cls.browser = webdriver.Chrome()
cls.browser.implicitly_wait(10)
super(RegistrationTests, cls).setUpClass()

1
2
3
4
5

Likewise just change the name to the browser that you want. Do note however that other
browsers may have certain setup requirements. Details about what needs to be configured
before using a particular browser can be found here:

IE
Chrome
Opera
Firefox
Safari
PhantomJS
mobile browsers

472

Limitations of Djangos LiveServerTestCase

While Djangos LiveServerTestCase and related StaticLiveServerTestCase are indeed helpful when it comes to testing with Selenium, they do have some limitations. The
main limitation is due to the fact that LiveServerTestCase is single-threaded.
For many scenarios, like the login scenario we covered in this chapter, a single thread is all
you need. However, this will lead to problems when you start to make calls to external servers.
In particular, on our registration page when we try to make a call to Stripe both on the front
and back end, this will cause strange errors that are difficult to debug. The solution would
be to have LiveServerTestCase be multi-threaded or run in a completely separate process.
This is a lot more difficult than one might imagine, so this hasnt been implemented yet in
Django.
However, we can do a simplistic version of this on our own by creating a custom test runner
(much like we did in the Mongo exercises). You will need this for the exercises, so here is the
code for the custom test runner:
1
2
3
4
5

import os
import sys
import subprocess
from django.test.runner import DiscoverRunner
from django_ecommerce.guitest_settings import SERVER_ADDR

6
7
8

class LiveServerTestRunner(DiscoverRunner):

9
10
11

12
13

def setup_databases(self, **kwargs):

retval = super(LiveServerTestRunner,
self).setup_databases(**kwargs)
self.spawn_server()
return retval

14
15
16
17
18
19
20
21
22

def spawn_server(self):
gui_settings = 'django_ecommerce.guitest_settings'
server_command = ["./manage.py", "runserver",
SERVER_ADDR, "--settings="+gui_settings]
self.server_p = subprocess.Popen(
server_command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
473

close_fds=True,
preexec_fn=os.setsid

23
24

)
print("server process started up... continuing with test
execution")

25
26

27
28
29
30
31
32
33
34

def kill_server(self):
try:
print("killing server process...")
os.killpg(os.getpgid(self.server_p.pid), 15)
self.server_p.wait()
except:
print("exception", sys.exc_info()[0])

35
36
37
38
39

def teardown_databases(self, old_config, **kwargs):

self.kill_server()
return super(LiveServerTestRunner, self).teardown_databases(
old_config, **kwargs)

Save it as runner.py in the django_ecommerce directory.

How does it work?

In a nutshell, we create a custom test runner, and after we have created the test database in
setup_databases, we use subprocess.Popen to launch a Django development server in
a different process. Then in teardown_databases before we delete the database we kill off
the development server.
Timing is import here, because we have used a custom settings file (shown later) that will tell
our development server to connect to our test database. So we must be sure the development
server starts after the database is created, and before we try to drop the database.
The actual work of spawning the development server in a child process is housed in the
spawn_server function. Here are the important pieces of that function:
Line 1 - We are using a settings file called django_ecommerce.guitest_settings,
which houses the database connection definition and the address we want to use for
our development server:
1

# Django settings for running gui tests with django_ecommerce

project.
from .settings import *

474

4
5
6
7
8
9
10
11
12
13

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'test_django_db',
'USER': 'djangousr',
'PASSWORD': 'djangousr',
'HOST': 'localhost',
'PORT': '5432',
}
}

14
15

SERVER_ADDR = "127.0.0.1:9001"

Notice that at the start we import the existing settings to make sure we have all
the settings we might need. Also notice how we have set the database name to
test_django_db. This is the name of the database that our test runner will generate.
Since both our test and our development server are using the same database, it makes
it easier to set up test data in our tests. Bonus!
Make sure to create the new settings file as guitest_settings.py.
Lines 2 - 3 - Coming back to our spawn_server function, these two lines are the
command line that we want to execute. Notice how we grab the SERVER_ADDR from
our guitest_settings and also pass in guitest_settings as our custom settings
file for our development server.
Lines 4 - 8 - This is how we spawn the child process. Basically we are setting it to run
in a child process and capturing the stdout and sterr so they arent displayed to the
terminal. The last two options are necessary to properly clean up the process when we
want to kill it later. For more information on the highly useful subprocess library,
check out the official docs
Finally, after we spawn the server, our tests will run. When we have completed all of our tests,
we want to kill the development server, which is done in the kill_server function. Here
we are using the special function os.killpg because when we run the initial manage.py
runserver ... command, it actually spawns a couple of processes, so os.killpg will kill
the whole process tree.
With this test runner, we no longer need to use the LiveServerTestCase because our test
runner is firing up the LiveServer for us. All we need to do is use the appropriate URL. We
can get this in our tests by importing it directly from our custom settings files. A test using
our LiveServerTestRunner could look like this:
475

from django_ecommerce.guitest_settings import SERVER_ADDR

2
3

class LoginTests(TestCase):

4
5
6
7
8

def setUp(self):
self.valid_test_user = User.create("tester",
"[email protected]", "test", 1234)
self.sign_in_page = SignInPage(self.browser,
"http://"+SERVER_ADDR)

There are only two differences from our previous test case:
We are inheriting from a regular TestCase and not StaticLiveServerTestCase,
and
We import the URL for the server from our guitest_settings because our regular
TestCase doesnt have an associated live_server_url.
Make sure to update all of the GUI tests using the above format.
With that, you can run your Selenium tests without having to worry about the limitations
of the LiveServerTestCase. To run your tests using this test runner, use the following
command:
1

$ ./manage.py test ../tests --testrunner=runner.LiveServerTestRunner

476

Conclusion
As you have seen, Selenium can be a very powerful way to execute GUI tests with Python.
It integrates well into Django and makes controlling the browser relatively straight-forward.
We can easily find elements and act on them, and it has a simple API for waiting for things
on the page to happen. While Selenium focus mainly on basic web functionality, it can also
test web 2.0 AJAX functionality without too much trouble.
The difficulties you run into will likely be around Selenium + Angular; while its not perfect,
it is getting better every day. Dont be afraid of the execute_script function so you can
modify your Angular variables / values as necessary. Take some time to learn the features
of Selenium and see if its the right GUI testing tool for you. Otherwise you can always look
at the Angular specific testing tools, such as Protractor which are mentioned in the What
about javascript testing section of this chapter.

477

Exercises
1. Write a Page Object for the Registration Page, and use it to come up with test for successful and unsuccessful registrations. Youll need to use the LiveServerTestRunner
for the tests to execute successfully.

478

Chapter 19
Deploy
You, dear reader, have come a long way. We started with a simple login and registration
page, and built the greatest Star Wars Fan Site the world has ever know. (Greatest being a
somewhat subjective term.) Nevertheless, we built something cool. Pat your self on the back.
You did it!
But we are not quite done, though. Its time to share the site with the world. That means
sticking it up on a server somewhere, maybe getting a domain name and, well deploying
the site. This chapter will walk you through what you need to know to get a Django site
deployed on a server in the cloud.

479

Where to Deploy
There are literally thousands of different web hosts that you could use to deploy your Django
app. But basically all of them fall into three different types:

Server/VPS (Virtual Private Server)

The standard and most common type of hosting environment is one where you get access
to a (usually) virtualized machine that has a basic OS installed on it and not much else. This
give you the greatest control and flexibility, but it can be a bit more work to manage. This
option also has the advantage of generally being the cheapest option. Because of its ubiquity
and affordability, this is the type of deployment we will cover in this chapter.

Managed (Server/VPS)
Basically the same as a Server/VPS (which by contrast is often called Unmanaged Hosting),
but the hosting company will provide support for OS upgrades, backups, etc Because the
company needs to support the server, there are generally some limitation as to what software
and configurations you can install. Rackspace made a name for itself by providing Managed
Servers/VPS.

PaaS
PaaS, or Platform as a Service, is a setup that attempts to remove the complexity of deployment from the user. Using a PaaS, such as Heroku. can make deployment as simple as pushing a commit to GitHub (after the prerequisite setup has been complete). Oftentimes a PaaS
will offer additional features such as automatic scaling, backups, deployment, etc They are a
good option if you dont want to worry about deployment, but you generally have very limited
flexibility when using the PaaS. Example PaaS providers include Heroku, Amazons Elastic
Beanstalk, and Google App Engine.
Its worth clicking some of the links above and exploring some of the options, but for now,
we are going to stick with deployment on a Server/VPS. You will learn the most by going this
route, and its always good to understand how deployment works.

480

What to Deploy
Since we are going to use a VPS that means we will start with a blank OS. To host a Django web
app in production we will need three additional pieces of software (aka servers) to ensure the
application will run without any issues in the more demanding environment (e.g., multiple
users, more requests). The three (actually four) servers that we will need are:
1. Database Server - PostgreSQL and MongoDB, by now youre familiar with the database
server, as youve been using it in development also.
2. Web Server / Proxy - In development we rely on the Django Test Server (i.e.
manage.py runserver) to handle web requests and return results. The Django
Test Server is great for development, but its not meant to be deployed into production.
So since we are going into Production we are going to use a Server designed to be
able to handle multiple simultaneous requests, serve static files (i.e., javascript, css,
images, etc..), and generally not fall over when multiple users start making requests.
The particular server we are going to use for this is called Nginx. Nginx is a fast,
powerful, and most importantly easy to configure HTTP server. In our setup it will be
the first thing that responds when a user requests a web page, and it will either serve
the static files directly to the user, or pass the request to our application server (see
below) which will run the Python code and return the results to the user.
3. Application Server - Again in development manage.py runserver supplies us with
both a web server (to handle web requests) and an application server that runs the
Django code and returns the results for a particular request. In production we will use
a separate application server called Gunicorn. Gunicorn is a WSGI server written in
Python that supports Django out of the box. This means we can use Gunicorn to host
our Django application (much like the Django test server would). The difference is
that Gunicorn is written with a production scenario in mind - its more secure and will
perform better than Djangos Test Server.
When we have all of these servers setup correctly. The flow of a users web request will look
like this:
Thats how it all works. Now that we have a better idea of how the pieces fit together, lets
jump right into the nitty-gritty of deployment.

481

Figure 19.1: Deployment Description

482

Firing up the VPS

For the purpose of this chapter we are going to utilize an excellent VPS provider called DigitalOcean (but these instructions should work with most any VPS). If you want to use another
provider, we are going to set up a VPS running Ubuntu 14.4 x64. If you want to use DigitalOcean, you can set up an account using this link, which will give you $10 in credit for free,
which is enough to go through this tutorial and then some. Once youre signed up, follow
these steps to get your VPS (what DigitalOcean calls a Droplet) up and running:

DigitalOcean Setup

Log into your DigitalOcean Portal

Click the top button on the left-hand navigation panel that says Create Droplet
Type in a host name; lets call it MEC.
Under select size you can choose the first option, $5/month.
Select whatever region you want.
Dont check anything in Available Settings.
For Select Image, select Ubuntu 14.04 x64.
Click Create Droplet.

This will create a Droplet (a VPS) based on the settings you just specified. Setup should take
just a couple of minutes, and then you can access your droplet through SSH; DigitalOcean
will email you the root password and IP address. Once the creation is done, you have a freshly
minted VPS and root access.
You can now do whatever you want.

483

Configuring the OS
Now how do we install the software we need on the VPS?
First, SSH into your Ubuntu machine as root and then run an upgrade and install the necessary packages
1
2
3

$ ssh root@<your-machines-ip>
$ apt-get update && apt-get upgrade
$ sudo apt-get install python-virtualenv libpq-dev python-dev
postgresql postgresql-contrib nginx git libpython3.4-dev

Here we installed nginx, postgres, git and virtualenv, plus the necessary support packages.
This will ensure we have the necessary operating system packages available. Now we need to
configure things.

Postgres setup
Lets work back to front, and configure the database first. These are the same instructions
covered in the Upgrade chapter when we installed postgres on our dev machine, but we will
repeat them here to make things easier to find:
1. Verify installation is correct:
1

$ psql --version

And you should get something like the following:

1

psql (PostgreSQL) 9.3.7

2. Now that Postgres is installed, you need to set up a database user and create an account
for Django to use. When Postgres is installed, the system will create a user named
postgres. Lets switch to that user so we can create an account for Django to use:
1

$ sudo su postgres

3. Create a Django user in the database:

1

$ createuser -P djangousr

Enter the password twice and remember it; for security reasons its best not to use
the same password you used in development. We will show you how to update your
settings.py file accordingly later in the chapter.
484

4. Now using the postgres shell, create a new database to use with Django. Note: Dont
type the lines starting with a #. These are comments for your benefit.
1

$ psql

2
3
4

# Once in the shell, type the following to create the database:

CREATE DATABASE django_db OWNER djangousr ENCODING 'UTF8';

5
6
7

# Then to quit the shell, type:

$ \q

5. Then we can set up permissions for Postgres to use by editing the file /etc/postgresql/9.3/main/pg_hba.conf with vim. Just add the following line to the end of the
file:
1

local

django_db

djangousr

md5

Then save the file and exit the text editor. The above line says,
"`djangousr` user can access the `django_db` database if they
are initiating a local connection and using an md5-encrypted
password".

6. Switch back to the root user:

1

$ exit

7. Finally restart the postgres service:

1

$ /etc/init.d/postgresql restart

This should restart postgres, and you should now be able to access the database. Check
that your newly created user can access the database with the following command:
1

$ psql django_db djangousr

This will prompt you for the password; type it in, and you should get to the database
prompt. You can execute any SQL statements that you want from here, but at this point
we just want to make sure we can access the database, so just do a \q and exit out of
the database shell. Youre all set; postgres is working! You probably want to do a final
exit from the command line to get back to the shell of your normal user.
If you do encounter any problems installing PostgreSQL, check the wiki. It has a lot of good
troubleshooting tips.
485

Set up Mongo
We have two databases now, so lets not forget to install mongo. Start off by installing the
packages:
1

$ apt-get install mongo-db

Thats all the configuration necessary. You can make sure its working by typing mongo from
the command prompt and making sure it drops you into the mongo shell.

Python Setup
We can start by creating a virtual environment. The following line will create one using
Python 3:
1

$ virtualenv /opt/mec_env -p `which python3`

Switch to the mec_env directory, then clone our mec app from GitHub:
1
2

$ cd /opt/mec_env
$ git clone https://github.com/realpython/book3-exercises.git
mec_app
NOTE Make sure you clone the app from your Github repo.

That will fetch the code from GitHub and put it into a directory called mec_app.
Lets activate our virtualenv:
1

$ source bin/activate

And then install all the necessary dependencies for our app:
1
2

$ cd mec_app
$ pip install -r requirements.txt

This will get all of our dependencies set up. However, we are going to need to make some
changes in order for Django to run in our new production environment. We will do that in
a later section called Django Setup.

486

Configuring Nginx
Nginx configuration just involves setting up a config file to get Nginx to listen on the incoming
port and to have it pass requests onto Gunicorn, which is what will be hosting our Django
app. By default, after you install Nginx (which we did in the earlier step where we executed
apt-get install nginx ) it will listen on port 80. So if you open up your browser and
navigate to the URL of your DigitalOcean Droplet, you should see the following:

Figure 19.2: nginx main screen

This means that Nginx is configured correctly. Now we need to configure it to serve our static
files and to point to Gunicorn for requests that need to be routed to Django. To do that, we
need to add a config file.
Nginx lives in the directory /etc/nginx. The main configuration file is called /etc/nginx/nginx.conf;
this is where you make changes that will affect the entire system. For our purposes, you can
safely leave this file alone, but its worth having a look through it if you want to get a better
idea of whats going on with Nginx.
We are going to add a new file called /etc/nginx/sites-available/mec, the contents of
which should look like:
1
2
3

upstream app_server_djangoapp {
server localhost:8001 fail_timeout=0;
}

4
5
6
7

server {
listen 80;
server_name <<put in ipaddress or hostname of your server
here>>;
487

access_log /var/log/nginx/mec-access.log;
error_log /var/log/nginx/mec-error.log info;

9
10
11

keepalive_timeout 5;

12
13

# path for static files

location /static {
autoindex on;
alias /opt/mec_env/static;
}

14
15
16
17
18
19

location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;

20
21
22
23
24

if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}

25
26
27
28

29
30

This will tell Nginx to serve static files from the directory /opt/mec_env/static/, which
will be set up shortly. Other requests will be passed on to whatever is running on port 8001
on the server, which should be Gunicorn and our Django application.
To activate this configuration, do the following:
1
2

$ cd /etc/nginx/sites-enabled
$ ln -s ../sites-avaliable/mec

The above will establish a symbolic link from the file we just created in the /etc/nginx/sites-enabled
folder. This folder is where nginx will look for configuration so we need to do this so Nginx
will see our configuration file and then things will work properly.
Next we need to deactivate the default Nginx configuration:
1

$ rm default

Then your Nginx configuration should be solid.

488

Configuring Gunicorn
Gunicorn is the server that will run your Django app. We can install it with pip:
1
2
3

$ cd /opt/mec_evn
$ source bin/activate
$ pip install gunicorn

This will install Gunicorn. It takes a number of configuration options to work correctly, so
lets create a new bash script that will run Gunicorn for us when we want to. Edit the file
/opt/mec_app/mec_env/deploy/gunicorn_start:
1

#!/bin/bash

2
3
4

10

#Section 1 - Define Script Variables

NAME="Mec"
application
DJANGODIR=/opt/mec_env/mec_app/django_ecommerce
project directory
USER=www-data
run as
GROUP=www-data
run as
NUM_WORKERS=3
worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=django_ecommerce.settings
settings file should Django use
DJANGO_WSGI_MODULE=django_ecommerce.wsgi
name

# Name of the
# Django
# the user to
# the group to
# how many
# which
# WSGI module

11
12
13

#Section 2 - activate virtualenv and create environment variables

echo "Starting $NAME as `whoami`"

14
15
16
17
18
19

# Activate the virtual environment

cd $DJANGODIR
source ../../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

20
21

# Section 3 - Start Gunicorn

22

489

23
24
25
26
27
28

exec gunicorn ${DJANGO_WSGI_MODULE}:application \

--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP\
--log-level=debug \
--bind=127.0.0.1:8001

Lets go through the script. From the comments we can see that it is broken up into three
section. The first section defines the variables that we will use to execute Gunicorn in the
third section.
USER and GROUP are used to set Gunicorn to run as www-data, which is a user created
during our installation that is most often used to run web/app server. This is for security
concerns to limit access to the system.
In section 2 we activate our virtualenv (since that is where Gunicorn is installed) and export
the environment variables needed. Its not uncommon to use this section to export environment variables that influence how Django runs, such as a new database connection for
example.
The final section starts Gunicorn using the variables defined in section 1.
We can now test this out by running our gunicorn_start script from the command line.
That can be done with the following commands:
1
2
3

$ cd /opt/mec_env/mec_app/deploy
$ chmod +x gunicorn_start
# make sure the script is executable
$ ./gunicorn_start

Note that the second line of the above listing only has to be executed once to ensure we can
execute the file. When you run the script, you should see a bunch of output about your configuration, ending with:
1
2
3

4
5

[2015-02-15 23:02:35 -0500] [23401]

[2015-02-15 23:02:35 -0500] [23401]
[2015-02-15 23:02:35 -0500] [23401]
http://127.0.0.1:8001 (23401)
[2015-02-15 23:02:35 -0500] [23401]
[2015-02-15 23:02:35 -0500] [23410]
23410
[2015-02-15 23:02:35 -0500] [23411]
23411
[2015-02-15 23:02:35 -0500] [23412]
23412
490

[INFO] Starting gunicorn 19.2.1

[DEBUG] Arbiter booted
[INFO] Listening at:
[INFO] Using worker: sync
[INFO] Booting worker with pid:
[INFO] Booting worker with pid:
[INFO] Booting worker with pid:

8
9

[2015-02-15 23:02:35 -0500] [23401] [DEBUG] 3 workers

[2015-02-15 23:02:36 -0500] [23401] [DEBUG] 3 workers

Of course the dates and the actual pid numbers will change, but it should have that basic
output, and then it will continue to the line1

[2015-02-15 23:02:36 -0500] [23401] [DEBUG] 3 workers

-repeated over and over again. This is Gunicorn running. You can now go to your browser
and navigate to the server IP, and you should no longer see the welcome to nginx screen.
Instead youll probably see some ugly Django error. But thats a good thing; it means you have
Nginx talking to your Django application. We just havent finished configuring the Django
application.
Before we switch to the Django application though, we need to finish up with Gunicorn. Right
now if we execute gunicorn_start, that command will stop running as soon as we log out
of our SSH session. What we want is for Gunicorn to run like a service, meaning it starts on
system boot and keeps running forever. We can do that with Supervisor.

Configuring Supervisor
Supervisor is a Python program that is intended to keep other programs running. From the
projects own description: Supervisor is a client/server system that allows its users to monitor
and control a number of processes on UNIX-like operating systems.
In other words, we use Supervisor to make sure other processes (like Gunicorn) keep running.
Setup is straightforward. Start by installing it via:
1

$ apt-get install supervisor

Now create a configuration that will keep Gunicorn running. Edit the file /etc/supervisor/conf.d/mec.co
to include the following:
1
2
3
4
5
6

[program: mec]
directory = /opt/mec_env/mec_app
user = www-data
command = /opt/mec_env/mec_app/deploy/gunicorn_start
stdout_logfile = /opt/mec_env/log/supervisor-logfile.log
stderr_logfile = /opt/mec_env/log/supervisor-error-logfile.log

We will need to create the log directory for the log files:
1

$ mkdir /opt/mec_env/log

491

Finally, restart supervisor:

1

$ /etc/init.d/supervisor restart

It will probably take a few seconds. To see if it started, check the web page and see if you still
have the ugly Django error. If so, youre good.
If not, check the main supervisor log file:
1

$ less /var/log/supervisor/supervisord.log

Or the application specific log files we just set up: sh

1

$ tail /opt/mec_env/log/*

Somewhere in there, it should have your problem.

Supervisor will now manage Gunicorn and ensure it keeps running all the time.
Now lets get the Django app itself configured correctly to work on the new environment.

492

Django Setup
There are a few things we need to modify in our django_ecommerce/settings.py file to
get things to run on our production server. However, if we make the necessary changes for
production to django_ecommerce/settings.py then things wont work in our development environment.
We need two settings.py files - one for development and one for production. There are
two common ways to handle this. One is to actually create two separate settings.py files,
and the other is to use environment variables. We will discuss using separate files here as its
probably the easiest when getting started.

Step 1. Create a settings_prod.py file

In our deployment directory, lets create a file called, you guessed it, settings_prod.py it should look like this:
1
2

# turn off debugging in production

DEBUG = False

3
4
5
6
7
8
9
10
11
12
13
14

# settings for the production database

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db',
'USER': 'djangousr',
'PASSWORD': '<< password you set for you new database>>',
'HOST': 'localhost',
'PORT': '5432',
}
}

15
16
17

# allowed hosts for our production site

ALLOWED_HOSTS = ['<<ip address of your server>>']

18
19
20

STATIC_ROOT = '/opt/mec_env/static/'
MEDIA_ROOT = '/opt/mec_env/media/'

This file will be used to overwrite the settings in your development settings.py file. The settings should be self-explanatory; we are configuring things to work in our production server.
493

Also, be sure that both /opt/mec_env/static and /opt/mec_env/media exist on the

server with the following command:
1
2

$ mkdir /opt/mec_env/static
$ mkdir /opt/mec_env/media

Step 2. Link to the new settings file.

Now to make Django aware of the new settings_prod.py file, add the following lines to
the end of your existing django_ecommerce/settings.py file:
1
2
3
4

try:
from .settings_prod import *
except ImportError:
pass

What does this do? It tries to import settings_prod from the same directory as our
normal settings.py file. Since settings_prod is imported as the last statement in
settings.py, whatever values are set in settings_prod.py will overwrite the values
already set in settings.py.

Step 3. Move settings_prod.py into place

To make things work (on our server), copy settings_prod.py into the correct place:
1
2

$ cd /opt/mec_env/mec_app
$ cp deploy/settings_prod.py django_ecommerce/django_ecommerce/

This will move the file into place, and then when Django starts up it will read our production
settings. To get Django to restart, just restart Gunicorn, which is done using Supervisor:
1

$ /etc/init.d/supervisor restart

How do we know it worked? Reload your web page, and youll see a 500 error, no more debug
error messages, because we have set DEBUG=False. Now to get rid of those pesky 500 errors.

Step 4. Run your migrations

We are getting the 500 error because we dont have our database set up correctly. Lets fix
that:

494

1
2
3

$ cd /opt/mec_env/mec_app/django_ecommerce
$ source ../../bin/activate
$ ./manage.py migrate

Now reload the web page, and no more 500 error, yay! But, the page looks very ugly, boo!

Step 5. Put the static files in the correct place

The page looks bad because Django cant find our static files. Remember in our settings_prod.py
we set the static root to be /opt/mec_env/static. But there is nothing there. Django has a
handy built-in command that will move all of our static files to that directory for us, though:
1

$ ./manage.py collectstatic

The command will warn you that its going to overwrite anything in that directory. Say yes,
and then it should copy all the files, finishing up with a message like:
1

414 static files copied to '/opt/mec_env/static'

Now reload the web page and voil (spoken with a strong French accent)! And there you
have it. You are deployed into production! Congratulations.
So thats it, right? Wrong we are just getting started, actually. Now that we have the deployment working, we need to1. Keep track of all the config files
2. Create a deployment script so that we can update production with a single script when
the time comes to make updates.

495

Configuration as Code
A term often used in DevOps circles, configuration as code, means keeping track of your configuration files so that deployment can be automated, thus reducing the dreaded it worked
on my machine issue.
For our setup, this means gathering the configuration files for Nginx, Gunicorn, and Supervisor, plus our settings_prod.py, and storing them in GitHub. We started to do this already,
but lets make sure we have everything in the correct place.
We want a directory structure like this:
1
2
3
4
5
6
7
8

deploy
gunicorn_start
nginx
sites-avaliable

mec
settings_prod.py
supervisor
mec.conf

As a review of the deployment process, here are the contents of each of those files:
gunicorn_start
1

#!/bin/bash

2
3

NAME="Mec"
application
DJANGODIR=/opt/mec_env/mec_app/django_ecommerce
project directory
USER=www-data
run as
GROUP=www-data
run as
NUM_WORKERS=3
worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=django_ecommerce.settings
settings file should Django use
DJANGO_WSGI_MODULE=django_ecommerce.wsgi
name

10
11

echo "Starting $NAME as `whoami`"

496

# Name of the
# Django
# the user to
# the group to
# how many
# which
# WSGI module

12
13
14
15
16
17

# Activate the virtual environment

cd $DJANGODIR
source ../../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

18
19
20
21
22
23
24
25

# Start your Django Unicorn

exec gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP\
--log-level=debug \
--bind=127.0.0.1:8001

nginx/sites-avaliable/mec
1
2
3

upstream app_server_djangoapp {
server localhost:8001 fail_timeout=0;
}

4
5
6
7

server {
listen 80;
server_name 128.199.202.178;

8
9
10

access_log /var/log/nginx/mec-access.log;
error_log /var/log/nginx/mec-error.log info;

11
12

keepalive_timeout 5;

13
14
15
16
17
18

# path for static files

location /static {
autoindex on;
alias / opt/mec_env/static;
}

19
20
21
22
23

location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;

24

497

if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}

25
26
27
28

29
30

supervisor/mec.conf
1
2
3
4
5
6

[program: mec]
directory = /opt/mec_env/mec_app
user = www-data
command = /opt/mec_env/mec_app/deploy/gunicorn_start
stdout_logfile = /opt/mec_env/log/supervisor-logfile.log
stderr_logfile = /opt/mec_env/log/supervisor-error-logfile.log

settings_prod.py
1
2

# turn off debugging in production

DEBUG=False

3
4
5
6
7

8
9
10
11
12
13
14

# settings for the production database

DATABASES = {
'default': {
'ENGINE':
'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db',
'USER': 'djangousr',
'PASSWORD': 'djangousr',
'HOST' : 'localhost',
'PORT' : '5432',
}
}

15
16
17

# allowed hosts for our production site

ALLOWED_HOSTS = ['128.199.202.178']

18
19
20

STATIC_ROOT = '/opt/mec_env/static/'
MEDIA_ROOT = '/opt/mec_env/media/'

Thats it for our configuration. Once you have all those files in the correct place, commit it all
and push it up to your GitHub account.
498

Lets script it
At the point, we have a production environment working, and we are keeping track of all the
configuration for our production environment. But next time we need to update the configuration, move to a different server, or simply deploy some new code, there is a good chance
that we will probably have forgotten the steps necessary for installation. Plus, its annoying
to have to manually type in all that stuff if we just want to update production.
Lets fix that with an auto-deployment script.

An auto-what script?
Its a simple Python file that will SSH into our server, grab the latest code, make sure everything is configured right, and redeploy the application. Sounds like a lot? Dont worry; its
actually not too much work thanks to a nice Python library called Fabric.
Fabric allows us to script SSH and shell commands using Python. What could be better?

Writing your first Fabric script

First thing to do is of course install Fabric. Unfortunately as of this writing, Fabric is not
quite ready for Python 3, although it is close. So we will use a fork that does work on Python
3. To use the fork, update your requirements.txt file and add this line to the end:
1
2

-e git+https://github.com/pashinin/fabric.git@p33#egg=Fabric
six==1.9.0

Then you can install it with:

1

$ pip install -r requirements.txt

Once you have Fabric installed, we can use it to create a script that will automate our deployment. Lets look at a simple version of the script first (we will expand upon this as we go
on).
Start by creating deploy/fabfile.py:
1

from fabric.api import env, cd, run, prefix, lcd, settings, local

2
3
4

env.hosts = ['<<domain or ip of your server>>']

env.user = 'root'

499

6
7
8
9
10
11
12
13
14

def update_app():
with cd("/opt/mec_env/mec_app"):
run("git pull")
with cd("/opt/mec_env/mec_app/django_ecommerce"):
with prefix("source /opt/mec_env/bin/activate"):
run("pip install -r ../requirements.txt")
run("./manage.py migrate")
run("./manage.py collectstatic")

Lets look at whats going on, line-by-line:

Line 1 - import the fabric API
Line 3 - specify the host we are going to run on by IP or domain; if there are multiple
items in this list then Fabric will apply the same function to all servers in the list
Line 4 - env.user specifies the username you want to SSH with. If you dont provide
this, Fabric will try to use the username you are logged in as on your local machine
Line 7 - Fabric works by executing a number of commands over SSH; this is the function that has a series of commands necessary to update the application on our remote
machine
Line 8 - the cd function does just what you think; it changes to a directory; here we use
it in a context manager, so each subsequent command will execute inside that directory
Line 9 - execute git pull to get the latest code from GitHub; because this
line is inside the cd context manager, Fabric will actually be executing a command like: cd /opt/mec_env/mec_app && git pull Line 10 - another cd
context manager, this time to the directory where manage.py lives Line 11 prefix is another context manager that says, execute this command just before you execute any other command; here we are using this to ensure we have
our virtualenv activated before we execute any of our commands Line 12 - install any new requirements listed in requirements.txt; because this command
is nested inside of two context managers, the ultimate command that will be
executed is:
cd /opt/mec_env_mec_app/django_ecommerce && source

/opt/mec_env/bin/activate && pip install -r ../requirements.txt

Line 12 - run migrations
Line 13 - run collectstatic; on this line because collectstatic prompts the
user to enter yes or no, Fabric will detect this and prompt you on your local machine,
which makes it easier as sometimes you need to interact with the remote machine
Keep in mind that we can tweak the last couple of commands so they never ask us any questions:
500

1
2

run("./manage.py migrate --noinput")

run("./manage.py collectstatic --noinput")

The --noinput argument is provided by Django for just this type of situation where you
always want to answer yes and not be prompted.
In addition, if you dont want to be prompted with a password to log into your server, you
can set up an SSH keypair so that no password is required. For DigitialOcean this how-to
will describe just how thats done.
Once the script is done, use the fab command to run it. Make sure youre in the deploy
directory, then run:
1

$ fab update_app

And off to the races you go!

As you can see, scripting a basic deployment with Fabric is pretty straightforward. We probably want to add one more function to update the config files (for Nginx, Supervisor, and
Django) automatically just in case they ever change, but Ill leave that as an exercise for the
reader.

501

Continuous Integration
By automating the deployment of your application with Fabric, you not only save time, but
you also help to prevent errors during deployment by ensuring steps arent missed or forgotten. Taken to the next level, if there were a way to ensure that all of our tests were run and
passed before any deployment happened, we would be adding even more quality assurance.
If we then performed the step of running all of our tests (plus potentially deploying) each
time we made a commit to our codebase, we would be practicing continuous integration.
The term, coined by Grady Booch back in the Extreme Programming days, and often referred
to as simply CI, literally refers to integrating the various branches of code (e.g., each developers local branch) with the mainline code daily. In git this translates to:
git commit any local changes
git pull fix any merge conflicts
git push
But with automated unit and integration tests, we can do better by adding test runs into the
mix.
For a single developer, we can actually build a continuous integration system like this
with Fabric and a few more lines of code. Lets add a function called integrate to our
deploy/fabfile.py:
1
2
3

def integrate():
with lcd("../django_ecommerce/"):
local("./manage.py test ../tests/unit")

4
5
6

with settings(warn_only=True):
local("git add -p && git commit")

7
8
9
10

local("git pull")
local("./manage.py test ../tests")
local("git push")

And what does all that do?

Line 2 - lcd changes the directory on the local machine; here we make sure we are in
the correct directory for running our tests.

502

 Line 4 - normally if a line fails, Fabric will abort execution; for the git add / git commit line, it will fail if we havent made any local changes, but maybe we still want to
integrate, so this line tells Fabric to spew out a warning and continue execution
Line 8 - after we commit, lets get the latest from origin (aka GitHub); this is the integration step.
Line 9 - after we integrate, run the full suite of tests to make sure everything is still
working
Line 10 - if all of our tests pass, push all changes back to GitHub
Then to practice continuous integration, instead of running git push / git pull manually, every time youre ready to push a change, just execute:
1

$ fab integrate update_app

That will run the two functions and your fabfile, and youre cooking up some CI goodness.

Continuous Integration Tools

In addition to the simple Fabric-based CI we just implemented, which Im dubbing the simplest CI system on earth, there are several third-party CI servers, the entire purpose of which
is to listen to your code repository and automatically run CI builds every time some new code
appears in the repository. There are many tools out there that do this, so I dont intend to
cover them all. Rather Ill point you to a couple of great resources to get more information
about CI system and how to work with them.
1. The second book in the Real Python series Web Development for Python has a full
chapter on Travis CI, one of the most popular CI servers out there.
2. The article Docker in Action: Fitter, Happier, More Productive discusses CI setup with
Docker.

503

Conclusion
We covered a lot in this chapter. We went through the complete set up of a virgin VPS,
getting it set up with Nginx, Gunicorn, postgresql, MongoDB and Django. Then, as if that
wasnt enough, we showed you how to automate the entire process. And we even looked at
some suggestions on how to implement continuous integration to ensure your app is always
well tested, integrated and behaving nicely. We came a long way in this chapter; you should
be proud of yourself for getting through it.
Furthermore, you should now have a better understanding of what goes into deployment,
how to configure the various servers and hopefully a bit about how to troubleshoot problems
when they arise. Now you know enough to be dangerous.

504

Exercises
1. While a VPS represents the most common case for deploying a Django app, there are
several others. Have a read through the following articles to get an understanding of
how to deploy using other architectures:
Amazon Elastic Beanstalk - This article details a step-by-step guide on how
to deploy a Django app to AWS with Elastic Beanstalk.
Heroku - How to Migrate your Django Project to Heroku.
Dokku - This is basically Heroku on the cheap. Find out how to get Django set
up with Dokku here.
Docker - While mentioned previous in the chapter, this article is an excellent
write-up on how to use Docker for deployment + development + CI.
2. Didnt think you were going to get out of this chapter without writing any code, did you?
Remember back in the Fabric section when we talked about creating another function
to update all of your configuration files? Well, now is the time to do that. Create a
function called update_config() that automatically updates your Nginx, Supervisor
and Django config / setting files for you.
When the function is complete you should be able to execute:
1

$ fab integrate update_app update_config

Bonus points if you can get it all to work so the user only has to type:
1

$ fab ci

505

Chapter 20
Conclusion

Thats about it. You should now have a fully functioning website, deployed to production,
with tests and auto-deployment and all kinds of goodness! Hurray, you made it. It was a lot
of hard work and dedication, but doesnt if feel great knowing that you have come this far? By
now you should have a good understanding of what it takes to make a web site. We covered a
ton of stuff - from mixins to transactions, from inclusion tags to form handling, from Mongo
to Python 3 to Angular to promises and a whole lot more in between.
While we have tried to cover a great deal, there is so much out there in the world of computer
programming that we have barely scratched the surface. But thats okay. You dont need to
know everything - nobody does. In fact my advise if you intend to continue growing as a software developer is to go deep not broad. There are far to many generalists / script kiddies /
developers who only know enough to be dangerous. Hopefully we have not only geared you
up with a well-rounded knowledge about how to develop web pages but also an understanding of how things work architecturally, what informs the choices that developers make when
structuring code or web apps, and the tradeoffs associated with any sizable development effort.
At this point if you just start thinking more about the structure and layout of your data and
code, then I think Ive done my job. Keep coding. Keep learning. And always strive for a
deeper understanding of why code works the way it does. That curiosity and subsequent
exploration is what leads to true software craftsmanship.
Thank you very much for allowing me to share what knowledge I have with you. I do hope it
was a fulfilling and pleasurable experience. Im always eager, to hear how things went. Let
me know what you build, what you learn, or what problems you encounter along your journey.
Until then, all the best and do enjoy your coding adventures (even when the code doesnt do
what you tell it to.)
506

Jeremy Johnson, Michael Herman, and Fletcher Heisler

@j1z0 / [email protected]

507

Chapter 21
Appendix A - Solutions to Exercises
Software Craftsmanship
Exercise 1
Question: Our URL routing testing example only tested one route. Write tests to
test the other routes. Where would you put the test to verify the pages/ route?
Do you need to do anything special to test the admin routes?
There are, of course, many solutions to this. Lets add a ViewTesterMixin() class to
payments/test in order to help with the view tests so were not repeating ourselves:
1
2

from .views import sign_in, sign_out

from django.core.urlresolvers import resolve

3
4
5

class ViewTesterMixin(object):

6
7
8
9
10
11
12
13
14
15

@classmethod
def setupViewTester(cls, url, view_func, expected_html,
status_code=200,
session={}):
request_factory = RequestFactory()
cls.request = request_factory.get(url)
cls.request.session = session
cls.status_code = status_code
cls.url = url
508

16
17

cls.view_func = staticmethod(view_func)
cls.expected_html = expected_html

18
19
20
21

def test_resolves_to_correct_view(self):
test_view = resolve(self.url)
self.assertEquals(test_view.func, self.view_func)

22
23
24
25

def test_returns_appropriate_respose_code(self):
resp = self.view_func(self.request)
self.assertEquals(resp.status_code, self.status_code)

26
27
28
29

def test_returns_correct_html(self):
resp = self.view_func(self.request)
self.assertEquals(resp.content, self.expected_html)

And its meant to be used like this:

1

class SignInPageTests(TestCase, ViewTesterMixin):

2
3
4
5
6
7
8
9
10
11

@classmethod
def setUpClass(cls):
html = render_to_response(
'sign_in.html',
{
'form': SigninForm(),
'user': None
}
)

12
13
14
15
16
17

ViewTesterMixin.setupViewTester(
'/sign_in',
sign_in,
html.content
)

The ViewTesterMixin() creates some default tests that can be run against most standard
tests.
In our case, we use it for all the view functions in the payments app. These are the same functions that we previously implemented to test our view function, except these are all placed in
a base class.

509

So, all you have to do is call the setupViewTest() class method, which is meant to be
called from the derived setUpClass class method. Once you the setup is taken care of, the
ViewTesterMixin() will simply perform the basic tests for routing, checking return codes,
and making sure youre using the appropriate template.
Once setup, you can implement any other tests you like.
he great thing about using Mixins for testing is that they can reduce a lot of the boilerplate
(e.g., common test routines) that you often run against Djangos system.
As for the second part of the question. Dont bother testing the admin views as they are
generated by Django, so we can assume they are correct because they are already tested by
the Django Unit Tests.
For the pages route, that is actually not that easy to do in Django 1.5. If you are sticking
to the out-of-the-box test discovery, there isnt a base test file. So well leave this out until
we get to the chapter on upgrading to Django 1.8, and then you can see how the improved
test discovery in Django 1.8 (which was introduced in Django 1.6) will allow you to better
structure your tests to model the application under test.

Exercise 2
Question: Write a simple test to verify the functionality of the sign_out view. Do
you recall how to handle the session?
1

class SignOutPageTests(TestCase, ViewTesterMixin):

2
3
4
5
6
7
8
9
10
11

@classmethod
def setUpClass(cls):
ViewTesterMixin.setupViewTester(
'/sign_out',
sign_out,
"", # a redirect will return no html
status_code=302,
session={"user": "dummy"},
)

12
13
14
15

def setUp(self):
#sign_out clears the session, so let's reset it overtime
self.request.session = {"user": "dummy"}

There are two things to note here.

510

1. We are passing in session = {"user": "dummy"} to our setupViewTest() function.

2. And if we look at the third and fourth lines of setupViewTester():
1
2

cls.request = request_factory.get(url)
cls.request.session = session

We can see that it shoves the session into the request returned by our request factory. If
you dont recall what the RequestFactory() is all about, have a quick re-read of the Mocks,
fakes, test doubles, dummy object, stubs section of the Testing Routes and Views chapter.

Exercise 3
Question: Write a test for the contact/models. What do you really need to test?
Do you need to use the database backend?
If you recall from the Testing Models chapter, we said to only test the functionality you write
for a model. Thus, for the ContactForm model we only need to test two things:
1. The fact that the model will return the email value as the string representation.
2. The fact that the queries to ContactForms are always ordered by the timestamp.
Note that there is a bug in the ContactForm() class; the class Meta needs
to be indented so it is a member class of ContactForm. Make sure to updated
before running the tests!
Here is the code for the test:
1
2
3

from django.test import TestCase

from .models import ContactForm
from datetime import datetime, timedelta

4
5
6

class UserModelTest(TestCase):

7
8
9
10
11
12

@classmethod
def setUpClass(cls):
ContactForm(email="[email protected]", name="test").save()
ContactForm(email="[email protected]", name="jj").save()
cls.firstUser = ContactForm(
511

email="[email protected]",
name="first",
timestamp=datetime.today() + timedelta(days=2)

13
14
15

)
cls.firstUser.save()
#cls.test_user=User(email="[email protected]", name ='test user')
#cls.test_user.save()

16
17
18
19
20
21
22

def test_contactform_str_returns_email(self):
self.assertEquals("[email protected]", str(self.firstUser))

23
24
25
26

def test_ordering(self):
contacts = ContactForm.objects.all()
self.assertEquals(self.firstUser, contacts[0])

Exercise 4
Question: QA teams are particularly keen on boundary checking. Research what
it is, if you are not familiar with it, then write some unit tests for the CardForm
from the payments app to ensure that boundary checking is working correctly.
First, whats boundary checking? Check out the Wikipedia article.
To accomplish boundary checking we can make use of our FormTesterMixin() and have it
check for validation errors when we pass in values that are too long or too short.
Here is what the test might look like:
1
2
3
4
5
6
7

8
9
10
11
12

def test_card_form_data_validation_for_invalid_data(self):
invalid_data_list = [
{
'data': {'last_4_digits': '123'},
'error': (
'last_4_digits',
[u'Ensure this value has at least 4 characters
(it has 3).']
)
},
{
'data': {'last_4_digits': '12345'},
'error': (
512

'last_4_digits',
[u'Ensure this value has at most 4 characters
(it has 5).']

13
14

15

16
17

18
19
20
21
22
23
24
25

for invalid_data in invalid_data_list:

self.assertFormError(
CardForm,
invalid_data['error'][0],
invalid_data['error'][1],
invalid_data["data"]
)

Before moving on, be sure that all your new tests pass: ./manage.py test.

513

Test Driven Development

Exercise 1
Question: If you really want to get your head around mocks, try mocking out the
test_registering_user_twice_cause_error_msg() test. Start by mocking out
the User.create function so that it throws the appropriate errors.
HINT: this documentation is a great resource and the best place to start. In particular, search for side_effect.

The first part of this exercise is relatively straightforward. Use the patch decorator with the
side_effect parameter like this:
1
2
3
4
5

@mock.patch('payments.models.User.save', side_effect=IntegrityError)
def test_registering_user_twice_cause_error_msg(self, save_mock):
# create the request used to test the view
self.request.session = {}
#...snipped the rest for brevity...#

The side_effect parameter says, When this function is called, raise the IntegrityError
exception. Also note that since you are manually throwing the error, you dont need to create
the data in the database, so you can remove the first couple of lines from the test function as
well:
1
2
3

#create a user with same email so we get an integrity error

user = User(name='pyRock', email='[email protected]')
user.save()

Run the test. Did you get an error?

1
2
3
4
5
6

7
8
9

Creating test database for alias 'default'...

......F....
.
........
======================================================================
FAIL: test_registering_user_twice_cause_error_msg
(payments.tests.RegisterPageTests)
---------------------------------------------------------------------Traceback (most recent call last):
File
"/Users/michaelherman/.virtualenvs/chp9/lib/python2.7/site-packages/mock.py",
line 1201, in patched
514

10
11

12
13

return func(*args, **keywargs)

File
"/Users/michaelherman/Documents/repos/realpython/book3-exercises/_chapters/ch
line 362, in test_registering_user_twice_cause_error_msg
self.assertEquals(len(users), 1)
AssertionError: 0 != 1

14
15
16

---------------------------------------------------------------------Ran 20 tests in 0.295s

17
18
19

FAILED (failures=1)
Destroying test database for alias 'default'...

What else do you need to change - and why?

Question: Want more? Mock out the UserForm as well. Good luck.
This part is a bit more involved. There are many ways to achieve this with the mock library,
but the simplest way is to create a class and use the patch decorator to replace the UserForm
with the class you created specifically for testing.
Here is what the class looks like:
1
2

def get_MockUserForm(self):
from django import forms

3
4

class MockUserForm(forms.Form):

5
6
7

def is_valid(self):
return True

8
9
10
11
12
13
14
15
16
17
18

@property
def cleaned_data(self):
return {
'email': '[email protected]',
'name': 'pyRock',
'stripe_token': '...',
'last_4_digits': '4242',
'password': 'bad_password',
'ver_password': 'bad_password',
}

19

515

20
21

def addError(self, error):

pass

22
23

return MockUserForm()

A few quick points:

1. The class is wrapped in a function so its easy to reach.
2. Only the methods/properties that will be called by this test are created. This, to get the
tests to behave right, we only used the methods need to do the as minimal amount of
work as possible.
Once the Mock class is created, it can override the real UserForm class with the mock class
by using the patch decorator.
The full test now looks like this:
1
2

@mock.patch('payments.views.UserForm', get_MockUserForm)
@mock.patch('payments.models.User.save',
side_effect=IntegrityError)
def test_registering_user_twice_cause_error_msg(self,
save_mock):

4
5
6
7
8

#create the request used to test the view

self.request.session = {}
self.request.method = 'POST'
self.request.POST = {}

9
10
11
12
13
14
15
16
17
18
19
20
21

#create the expected html

html = render_to_response(
'register.html',
{
'form': self.get_MockUserForm(),
'months': list(range(1, 12)),
'publishable': settings.STRIPE_PUBLISHABLE,
'soon': soon(),
'user': None,
'years': list(range(2011, 2036)),
}
)

22

516

#mock out stripe so we don't hit their server

with mock.patch('stripe.Customer') as stripe_mock:

23
24
25

config = {'create.return_value': mock.Mock()}

stripe_mock.configure_mock(**config)

26
27
28

#run the test

resp = register(self.request)

29
30
31

#verify that we did things correctly

self.assertEqual(resp.content, html.content)
self.assertEqual(resp.status_code, 200)
self.assertEqual(self.request.session, {})

32
33
34
35
36

#assert there is no records in the database.

users = User.objects.filter(email="[email protected]")
self.assertEqual(len(users), 0)

37
38
39

Notice that the patched object is payments.views.UserForm NOT payments.forms.UserForm.

The trickiest thing about the mock library is knowing where to patch. Keep in mind that you
want to patch the object used in your SUT (System Under Test).
In this case since were testing payments.views.registration, we want the UserForm
imported in the view module. If youre still unclear about this, re-read this section of the
docs.

Exercise 2
Question: As alluded to in the conclusion, remove the customer creation logic
from register() and place it into a separate CustomerManager() class. Re-read
the first paragraph of the conclusion before you start, and dont forget to update
the tests accordingly.
The solution for this is pretty straightforward: Grab all the Stripe logic and wrap it in a class.
This example is a bit contrived because at this point in our application it may not make a lot
of sense to do this, but the point here is about TDD and how it helps you with refactoring. To
make this change, the first thing you would do is create a simple test to help design the new
Customer() class:
1

class CustomerTests(TestCase):

517

3
4
5

6
7

def test_create_subscription(self):
with mock.patch('stripe.Customer.create') as create_mock:
cust_data = {'description': 'test user', 'email':
'[email protected]',
'card': '4242', 'plan': 'gold'}
Customer.create(**cust_data)

8
9

create_mock.assert_called_with(**cust_data)

This test says, The Customer.create function is used to call Stripe with the arguments
passed in.
You could implement a simple solution to that to place in payments.views like this:
1

class Customer(object):

2
3
4
5

@classmethod
def create(cls, **kwargs):
return stripe.Customer.create(**kwargs)

Then make sure to update the following import in payments.tests:

1

from payments.views import soon, register, Customer

This is probably the simplest way to achieve this. (Note: Instead of using **kwargs you
could enumerate the names.) This will pass the test, but the next requirement is to support
both subscription and one_time payments.
To do that, lets update the tests:
1

class CustomerTests(TestCase):

2
3
4
5

6
7

def test_create_subscription(self):
with mock.patch('stripe.Customer.create') as create_mock:
cust_data = {'description': 'test user', 'email':
'[email protected]',
'card': '4242', 'plan': 'gold'}
Customer.create("subscription", **cust_data)

8
9

create_mock.assert_called_with(**cust_data)

10
11
12

def test_create_one_time_bill(self):
with mock.patch('stripe.Charge.create') as charge_mock:
518

13
14
15
16
17

cust_data = {'description': 'email',

'card': '1234',
'amount': '5000',
'currency': 'usd'}
Customer.create("one_time", **cust_data)

18
19

charge_mock.assert_called_with(**cust_data)

Here, you could have created two separate functions, but we designed it with one function
and passed in the type of billing as the first argument. You could make both of these test pass
with a slight modification to the original function:
1

class Customer(object):

2
3
4
5
6
7
8

@classmethod
def create(cls, billing_method="subscription", **kwargs):
if billing_method == "subscription":
return stripe.Customer.create(**kwargs)
elif billing_method == "one_time":
return stripe.Charge.create(**kwargs)

And there you have it.

Next, lets actually substitute the new Customer() class in place of stripe.Customer in
our register function:
1
2
3
4
5
6

customer = Customer.create(
email=form.cleaned_data['email'],
description=form.cleaned_data['name'],
card=form.cleaned_data['stripe_token'],
plan="gold",
)

Finally, update our test cases so that we do not reference Stripe at all. Heres an example of
an updated test case:
1
2
3

@mock.patch('payments.views.Customer.create')
@mock.patch.object(User, 'create')
def test_registering_new_user_returns_succesfully(self,
create_mock, stripe_mock):

Not too much different. We changed the first patch to payments.views.Customer.create

instead of stripe. In fact, we would have left it as is, and the test would have still passed - but
519

this way we keep our test ignorant of the fact that we are using Stripe in case we later want to
use something other than Stripe to process payments or change how we interact with Stripe.
And of course you could continue factoring out the Stripe stuff from the edit() function as
well if you wanted, but well stop here.
Finally, run all of your tests again to ensure we didnt break any functionality in another app:
1

./manage.py test

They should all pass:

1
2
3
4
5
6

reating test database for alias 'default'...

..................................................................................
.
........
---------------------------------------------------------------------Ran 555 tests in 11.508s

7
8

OK (skipped=1, expected failures=1)

Cheers!

520

Bootstrap 3 and Best Effort Design

Exercise 1
Questions: **Bootstrap is a front-end framework, and although we didnt touch much on it
in this chapter, it uses a number of CSS classes to insert things on the page, making it look
nice and provide the responsive nature of the page. It does this by providing a large number
of classes that can be attached to any HTML element to help with placement. All of these
capabilities are described on the Bootstraps CSS page. Have a look through it, and then lets
put some of those classes to use.
1

- In the main carousel, the text, "Join the Dark Side" on the Darth
Vader image, blocks the image of Darth himself. Using the
Bootstrap / carousel CSS, can you move the text and sign up
button to the left of the image so as to not cover Lord Vader?
- If we do the above change, everything looks fine until we view
things on a phone (or make our browser really small). Once we
do that, the text covers up Darth Vader completely. Can you
make it so on small screens the text is in the "normal
position" (centered / lower portion of the image) and for
larger images it's on the left.**

Part 1

For the first part, you could just add the style inline.
1
2
3

4
5

7
8

<figure class="item active row">

<img src="{% static 'img/darth.jpg' %}" alt="Join the Dark Side">
<figcaption class="carousel-caption"
style="top:0%;left:5%;right:60%;text-align:left">
<h1 class>Join the Dark Side</h1>
<p>Or the light side. Doesn't matter. If you're into Star Wars
then this is the place for you.</p>
<p><a class="btn btn-lg btn-primary" href="{% url 'register'
%}" role="button">Sign up today</a></p>
</figcaption>
</figure>

Here, we simply changed the style of the figcaption to push the caption to the top and left
top:0%;left:5% and have the text wrap at 60% of the right edge, right:60%. We then
aligned the text left: text-align:left.
521

Doing this overrides the CSS for the class carousel-caption that is defined both in
bootstrap.css and carousel.css. As a best practice, though, we should keep our
styling in CSS files and out of our HTML/templates - but we dont want this style to
apply to all .carousel-caption items. So lets give this particular caption an id (say
id="darth_caption") and then update our mec.css (where we put all of our custom CSS
rules) files as follows:
1
2
3
4
5
6

#darth_caption {
top: 0%;
left: 5%;
right: 60%;
text-align: left;
}

This has the same effect but keeps the styling from cluttering up our HTML.
Part 2

Bootstraps Responsive Utilities give you the power to create different styling depending upon
the size of the screen that is being used. Check out the chart on the Bootstrap page. By using
those special classes, you can show or hide items based on the screen size. For example,
if your used class=hidden-lg, then the HTML element associated to that class would be
hidden on large screens.
In our example, we essentially use two figcaptions. One for large screens, and the other
for all other screens:
1

2
3

5
6
7
8

10

<figcaption class="carousel-caption visible-lg-block"

id="darth_caption">
<h1>Join the Dark Side</h1>
<p>Or the light side. Doesn't matter. If your into Star Wars
this is the place for you.</p>
<p><a class="btn btn-lg btn-primary" href="{% url 'register'
role="button">Sign up today</a></p>
</figcaption>
<figcaption class="carousel-caption hidden-lg">
<h1>Join the Dark Side</h1>
<p>Or the light side. Doesn't matter. If your into Star Wars
this is the place for you.</p>
<p><a class="btn btn-lg btn-primary" href="{% url 'register'
role="button">Sign up today</a></p>
</figcaption>
522

then
%}"

then
%}"

That should do it! If you make your browser screen smaller and smaller, you will see that
eventually the caption will jump back to the middle lower third. If you make your browser
larger, then the caption will jump to the left of Darth Vader in the image. Congratulations,
you now understand the basics of how responsive websites are built.

Exercise 2
Question: In this chapter, we updated the Home Page but we havent done anything about the Contact Page, the Login Page, or the Register Page. Bootstrapify
them. Try to make them look awesome. The Bootstrap examples page is a good
place to go to get some simple ideas to implement. Remember: try to make the
pages semantic, reuse the Django templates that you already wrote where possible, and most of all have fun.
There is no right or wrong answer for this section. The point is just to explore Bootstrap and
see what you come up with. Below youll see some examples. Lets start with the login page.
Short and sweet.
Sign-in page

Figure 21.1: Sign-in page

523

1
2
3
4
5
6
7
8

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26

27
28
29

{% extends "__base.html" %}
{% load staticfiles %}
{% block extra_css %}
<link href="{% static 'css/signin.css' %}" rel="stylesheet">
{% endblock %}
{% block content %}
<div class="container">
<form accept-charset="UTF-8" action="{% url 'sign_in' %}"
class="form-signin" role="form" method="post">
{% csrf_token %}
<h1 class="form-signin-heading">Sign in</h1>
{% if form.is_bound and not form.is_valid %}
<div class="alert-message block-message error">
<div class="errors">
{% for field in form.visible_fields %}
{% for error in field.errors %}
<p>{{ field.label }}: {{ error }}</p>
{% endfor %}
{% endfor %}
{% for error in form.non_field_errors %}
<p>{{ error }}</p>
{% endfor %}
</div>
</div>
{% endif %}
{% for field in form %}{% include "payments/_field.html" %}{%
endfor %}
<input class="btn btn-lg btn-primary btn-lg" name="commit"
type="submit" value="Sign in">
</form>
</div>
{% endblock %}

This is not much change to our previous sign-in template. The main difference is the new
stylesheet. Notice on lines 3-5 we have the following code.
1
2
3

{% block extra_css %}
<link href="{% static 'css/signin.css' %}" rel="stylesheet">
{% endblock %}

Here we created another block in the _base.html template so that we can easily add new
524

stylesheets from inherited pages:

1
2

{% block extra_css %}
{% endblock %}

The modified __base.html now looks like:

1

{% load static %}

2
3
4
5
6
7
8

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width,
initial-scale=1">
<title>Mos Eisley's Cantina</title>

10
11
12

13
14
15
16

<!-- Bootstrap -->

<link href= "{% static "css/bootstrap.min.css" %}"
rel="stylesheet">
<!-- custom styles -->
<link href= "{% static "css/mec.css" %}" rel="stylesheet">
{% block extra_css %}
{% endblock %}

17
18

19

20
21

22

23
24
25

<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements

and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via
file:// -->
<!--[if lt IE 9]>
<script
src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script
src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>

26
27

... snip ...

525

As you can see, just after the stylesheets on line 23-24 we added the {% block extra_css
%} block. This allows us to add additional CSS or tags in the header. This is a helpful technique to make your reusable templates more flexible. Its always a balance between making
your templates more flexible and making things easy to maintain; dont get carried away with
adding blocks everywhere. But a few blocks in the right places in your __base.html can be
very useful.
The final piece of the puzzle of the sign-in page is the signin.css that is responsible for the
styling. It looks like this:
1
2
3
4

body {
padding-bottom: 40px;
background-color: #eee;
}

5
6
7
8
9
10
11

.form-signin {
max-width: 330px;
padding: 15px;
margin: 0 auto;
font-family: 'starjedi', sans-serif;
}

12
13
14
15
16

.form-signin .form-signin-heading,
.form-signin .checkbox {
margin-bottom: 10px;
}

17
18
19
20

.form-signin .checkbox {
font-weight: normal;
}

21
22
23
24
25
26
27
28
29
30

.form-signin .form-control {
position: relative;
height: auto;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;
padding: 10px;
font-size: 16px;
}

31

526

32
33
34

.form-signin .form-control:focus {
z-index: 2;
}

35
36
37
38
39
40

.form-signin input[type="email"] {
margin-bottom: -1px;
border-bottom-right-radius: 0;
border-bottom-left-radius: 0;
}

41
42
43
44
45
46

.form-signin input[type="password"] {
margin-bottom: 10px;
border-top-left-radius: 0;
border-top-right-radius: 0;
}
NOTE: This particular stylesheet is relatively short as far as stylesheets go, so
we could have included it into the mec.css. This would reduce the number of
extra pages that need to be downloaded and thus improve the response time of
the website slightly. However, weve left it separate as a way to demonstrate a
good use-case for the Django templates block directive.

We can apply similar formatting to the contact page so that we have a nice consistent theme
to our site. Grab the file from the chp08 folder in the repo.

Exercise 3
Question: Previously in the chapter we introduced the marketing__circle_item
template tag. The one issue we had with it was that it required a whole lot of data
to be passed into it. Lets see if we can fix that. Inclusion tags dont have to have
data passed in. Instead, they can inherit context from the parent template. This
is done by passing in takes_context=True to the inclusion tag decorator like so:
1

@register.inclusion_tag('main/templatetags/circle_item.html',
takes_context=True)

If we did this for our marketing__circle_item tag, we wouldnt have to pass in

all that data; we could just read it from the context.
Go ahead and make that change, then you will need to update the main.views.index
function to add the appropriate data to the context when you call render_to_response.

527

Once that is all done, you can stop hard-coding all the data in the HTML template
and instead pass it to the template from the view function.
For bonus points, create a marketing_info model. Read all the necessary data
from the model in the index view function and pass it into the template.

After ensuring that we have indeed set our marketing_circle_item to takes_context=True-

1
2

from django import template

register = template.Library()

3
4
5
6
7
8
9
10

@register.inclusion_tag(
'main/templatetags/circle_item.html',
takes_context=True
)
def marketing__circle_item(context):
return context

-the next thing to do is update our associated view function to pass in the context that we need
for our marketing items. Updating main.views to do that will cause it to look something like:
1
2
3

from django.shortcuts import render_to_response

from payments.models import User
#from main.templatetags.main_marketing import marketing__circle_item

4
5
6

class market_item(object):

7
8

9
10
11
12
13
14

def __init__(self, img_name, heading, caption,

button_link="register",
button_title="View details"):
self.img_name = img_name
self.heading = heading
self.caption = caption
self.button_link = button_link
self.button_title = button_title

15
16
17
18

market_items = [
market_item(
img_name="yoda.jpg",
528

heading="Hone your Jedi Skills",

caption="All members have access to our unique"
" training and achievements latters. Progress through the "
"levels and show everyone who the top Jedi Master is!",
button_title="Sign Up Now"

19
20
21
22
23

),
market_item(
img_name="clone_army.jpg",
heading="Build your Clan",
caption="Engage in meaningful conversation, or "
"bloodthirsty battle! If it's related to "
"Star Wars, in any way, you better believe we do it.",
button_title="Sign Up Now"
),
market_item(
img_name="leia.jpg",
heading="Find Love",
caption="Everybody knows Star Wars fans are the "
"best mates for Star Wars fans. Find your "
"Princess Leia or Han Solo and explore the "
"stars together.",
button_title="Sign Up Now"
),

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

def index(request):
uid = request.session.get('user')
# for now just hard-code all the marketing info stuff
# to see how this works
if uid is None:
return render_to_response(
'main/index.html',
{'marketing_items': market_items}
)
else:
return render_to_response(
'main/user.html',
{'marketing_items': market_items,
'user': User.get_by_id(uid)}
529

59

1. We created a dummy class called market_item. The class is used to make the variables
easier to access in the template, and in fact later when we make the model class it is
going to end up looking pretty similar to the dummy class.
2. Next we added dummy data. Normally you would read this from the database, but lets
start quick and dirty and stick all the data in a list called market_item. Notice how
we put the list at the module level namespace; this will make it easier to access from
the unit tests (which are going to break as soon as we implement this). 1/ The final
thing we did was pass the newly created list of marketing items to the template as the
context:
1
2
3
4
5
6
7
8
9
10
11

if uid is None:
return render_to_response(
'main/index.html',
{'marketing_items': market_items}
)
else:
return render_to_response(
'main/user.html',
{'marketing_items': market_items,
'user': User.get_by_id(uid)}
)

Simply passing in the dictionary with key marketing_items set to the market_items list
to the render_to_response() function will get the context set up so its accessible to our
templates. Then our inclusion tag, which now has access to the context, can pick it up and
pass it to the template main\templatetags\circle_item.html'. First a look at
themarketing__circle_item template. It now does more or less nothing:
1
2

from django import template

register = template.Library()

3
4
5
6
7
8
9
10

@register.inclusion_tag(
'main/templatetags/circle_item.html',
takes_context=True
)
def marketing__circle_item(context):
return context
530

It does have to take the context as the first argument, and whatever it returns will be the
context that circle_item.html template has access to. We pass the entire context. Finally
our template can now loop through the list of market_items and display the nicely formatted
marketing blurb:
1

{% load staticfiles %}

2
3
4
5

6
7
8

9
10

{% for m in marketing_items %}
<div class="col-lg-4">
<img class="img-circle" src="{% static 'img/'|add:m.img %}"
width="140" height="140" alt="{{ m.img }}">
<h2>{{ m.heading }}</h2>
<p>{{ m.caption }}</p>
<p><a class="btn btn-default" href="{% url m.button_link %}"
role="button">{{ m.button_title }} &raquo;</a></p>
</div>
{% endfor %}

What we are doing here is looping through the marketing_items list (that we passed in from
the main.views.index function) and created a new circle marketing HTML for each. This
has the added advantage that it will allow us to add a variable number of marketing messages
to our page!
Finally, make sure to update the main/index.html file:
1
2
3
4
5
6

<section class="container marketing">

<!-- Three columns of text below the carousel -->
<div class="row center-text">
{% marketing__circle_item %}
</div>
</section>

With all that done, fire up the browser and have a look at your site; it should all look as it did
before.

Dont forget to check your unit tests. You should see a big ugly failure in tests.main.testMainPageView.Ma
This is because your index page now requires the marketing_info context variable, and
we are not passing it into our test. Remember earlier when we said we were putting
the marketing_items list at the module level to aid with our testing? Well, lets fix
tests.main.testMainPageView.MainPageTests:
1. First import the marketing_items into the test, so we can reuse the same data:
531

from main.views import index, market_items

2. Next update the test test_returns_exact_html to use market_items:

1
2
3
4
5
6
7
8
9

def test_returns_exact_html(self):
resp = index(self.request)
self.assertEqual(
resp.content,
render_to_response(
"main/index.html",
{"marketing_items": market_items}
).content
)

Now rerun and all your tests should pass! Great work.
Bonus

Going for the bonus?

Question: ** For bonus points, create a marketing_info model. Read all the necessary data
from the model in the index view function and pass it into the template.**
Essentially, youll data drive the template from the database (as opposed to hardcoding a
bunch of stuff in the view function). Youre actually almost there already. All you need to
do is create a model, then have your view read from the model as opposed to the hardcoded
values. First lets add a new class to main/models.py:
1

from django.db import models

2
3
4
5
6
7
8
9
10

# Create your models here.

class MarketingItem(models.Model):
img = models.CharField(max_length=255)
heading = models.CharField(max_length=300)
caption = models.TextField()
button_link = models.URLField(null=True, default="register")
button_title = models.CharField(max_length=20, default="View
details")

Its a simple model with a couple of default values. Next we can change our main.views to
read from the model instead of from the hardcoded values like so:
532

from main.models import MarketingItem

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

def index(request):
uid = request.session.get('user')
market_items = MarketingItem.objects.all()
if uid is None:
return render_to_response(
'main/index.html',
{'marketing_items': market_items}
)
else:
return render_to_response(
'main/user.html',
{'marketing_items': market_items,
'user': User.get_by_id(uid)}
)

Here, we call market_items = MarketingItem.objects.all() and then whatever we

have in our database will appear on the screen. Groovy!
Fire up the application and check to see if this works. Youll probably get some errors because
you didnt run syncdb to create the new table. (Also note that we are going to stick with
syncdb until we get to the migrations chapter, which will show us a better way of updating
our models). But even if you did, you wont see any of the marketing items on your index
page since theres nothing in your database. We need to load some data.
1. Create a directory main/fixtures
2. In that directory create a file called initial_data.json (Note: you could also use YAML
or XML if you prefer)
3. Now chuck the data you want to load in the initial_data.json:
1
2
3
4
5
6
7
8

[
{
"model": "main.MarketingItem",
"pk": 1,
"fields":
{
"img":"yoda.jpg",
"heading":"Hone your Jedi Skills",
533

"caption":"All members have access to our unique training

and achievements latters. Progress through the levels
and show everyone who the top Jedi Master is!",
"button_title":"Sign Up Now"

10

}
},
{
"model": "main.MarketingItem",
"pk": 2,
"fields":
{
"img":"clone_army.jpg",
"heading":"Build your Clan",
"caption":"Engage in meaningful conversation, or
bloodthirsty battle! If it's related to Star Wars, in
any way, you better believe we do it.",
"button_title":"Sign Up Now"
}
},
{
"model": "main.MarketingItem",
"pk": 3,
"fields":
{
"img":"leia.jpg",
"heading":"Find Love",
"caption":"Everybody knows Star Wars fans are the best
mates for Star Wars fans. Find your Princess Leia or
Han Solo and explore the stars together.",
"button_title":"Sign Up Now"
}
}

11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31

32
33
34
35

Once this is all in place, every time you run syncdb, the data from initial_data.json will automatically be put into the database. Re-run syncdb, fire up the Django development server,
and you should now see your marketing info. Awesome!
Before you pack everything up, make sure to run the tests. They should all pass.

534

Another Migrations Note:

Please note that while syncdb will load the data in your fixtures, if you have created any migrations for the application this feature wont work. Again we will
cover migrations in an upcoming chapter. So try not to jump ahead or this stuff
might not work.

535

Building the Members Page

Exercise 1
Questions: Our User Story US3 Main Page** says that the members page is a place for announcements and to list current happenings. We have implemented user announcements in
the form of status reports, but we should also have a section for system announcements/current events. Using the architecture described in this chapter, create an Announcements
info_box to display system-wide announcements?**
The simplest way to implement this requirement is with a simple template. Lets call it
main/_announcements.html:
1
2
3
4
5
6
7
8
9
10

11
12

<!-- system announcements -->

{% load staticfiles %}
<section class="info-box" id="announcements">
<h1>orders from the Council</h1>
<div class="full-image">
<img src="{% static 'img/beat_jarjar.jpg' %}"/>
</div>
<div >
<h3>April 1: Join us for our annual Smash Jar Jar bash</h2>
<p>Bring the whole family to MEC for a fun-filled day of
smashing Jar Jar Binks!</p>
</div>
</section>

This uses the same .info_box class as all our other boxes on the user.html page. It includes a heading, image and some details about the announcement. We need a few CSS rules
to control the size and position of the image.
Update mec.css:
1
2
3

.full-image {
overflow: hidden;
}

4
5
6
7
8
9

.full-image img {
position: relative;
display: block;
margin: auto;
min-width: 100%;
536

min-height: 100px;

10
11

This causes the image to center and autoscale to be the size of the info-box, minus the
border. Also if the image gets too large, rather than trying to shrink the image to a tiny size
(which will cause the image to look pretty bad) it will just be cropped.
Finally we can hook the image into our user.html page like so:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

{% extends "__base.html" %}
{% load staticfiles %}
{% block content %}
<div class="row member-page">
<div class="col-sm-8">
<div class="row">
{% include "main/_announcements.html" %}
{% include "main/_lateststatus.html" %}
</div>
</div>
<div class="col-sm-4">
<div class="row">
{% include "main/_jedibadge.html" %}
{% include "main/_statusupdate.html" %}
</div>
</div>
</div>
{% endblock %}

Its just another include. We also moved the info-boxes around a bit so the announcements
are the first thing the user will see. With this, you should have a page that looks like so:
[User.html with announcements](images/announcements.png]
The advantage to doing something simple like this is:
1. It doesnt take much time.
2. You are free to use whatever HTML you like for each announcement.
The disadvantages:
1. Its static; you need to update the template for each new announcement.

537

2. You are limited to one announcement, unless of course you update the template for
more announcements.
Lets address the disadvantages by data driving the announcements info box from the
database in the same way we did for main_lateststatus.html.
Step 1: Create a model.

Lets call it main.models.Announcement:

1
2
3
4
5

class Announcement(models.Model):
when = models.DateTimeField(auto_now=True)
img = models.CharField(max_length=255, null=True)
vid = models.URLField(null=True)
info = models.TextField()

Were allowing either an image or a video as the main part of the content, and then info will
allow us to store arbitrary HTML in the database, so we can put whatever we want in there.
We also time stamp the Announcements as we dont want to keep them on the site forever
because it doesnt look good to have announcements that are several years old on the site.
Sync the database.
In order to support embedding videos, lets turn to a pretty solid third party app. Its called
django-embed-video. Its not very customizable, but its pretty easy to use and does all the
heavy lifting for us.
As always, install it with pip:
1

$ pip install django-embed-video

Dont forget to add it to the requirements.txt file as well as embed_video to the INSTALLED_APPS
tuple in django_ecommerce\settings.py. Once done, update that main/_announcements.html
template.
1
2
3
4
5
6
7
8

<!-- system announcements -->

{% load staticfiles %}
{% load embed_video_tags %}
<section class="info-box" id="announcements">
<h1>orders from the Council</h1>
{% for a in announce %}
<div class="full-image">
{% if a.vid %}
538

{% video a.vid "medium" %}

{% else %}
<img src="{% static 'img/'|add:a.img %}"/>
{% endif %}
</div>
<div > {{ a.info | safe }}</div>
<br>
{% endfor %}
</section>

9
10
11
12
13
14
15
16
17

Line 3: loads a new set of tags from django-embed-video

Line 6: loops through all the announcements
Line 8: gives videos priority; if we have a video entry in the database, show that
Line 9: load the video in an iframe using django-embed-vide
Line 11: if its an image, show that instead
Line 14: inserts the HTML from the database; the safe filter tells Django to not escape
the data so it will be rendered as HTML instead of plain text

In main/views.index add another context variable. The relevant part is below:

1

def index(request):

2
3

...snip...

4
5
6
7
8
9
10
11

else:
#membership page
status = StatusReport.objects.all().order_by('-when')[:20]
announce_date = date.today() - timedelta(days=30)
announce = (Announcement.objects.filter(
when__gt=announce_date).order_by('-when')
)

12
13
14
15
16
17
18
19

return render_to_response(
'main/user.html',
{
'user': User.get_by_id(uid),
'reports': status,
'announce': announce
},
539

context_instance=RequestContext(request),

20

21

Update the imports as well:

1
2
3
4

from
from
from
from

django.shortcuts import render_to_response, RequestContext

payments.models import User
main.models import MarketingItem, StatusReport, Announcement
datetime import date, timedelta

Basically we are grabbing all the announcements in the last thirty days and ordering them,
so the most recent will appear at the top of the page.
Run the tests. Make sure to manually test as well.

Exercise 2
Question: You may have noticed that in the Jedi Badge box there is a list achievements link. What if the user could get achievements for posting status reports,
attending events, and any other arbitrary action that we create in the future?
This may be a nice way to increase participation, because everybody likes badges,
right? Go ahead and implement this achievements feature. Youll need a model
to represent the badges and a link between each user and the badges they own
(maybe a user_badges table). Then youll want your template to loop through
and display all the badges that the given user has.
There are several ways to do this. Well look at the most straightforward.
First create the model main.models.Badge:
1
2
3
4

class Badge(models.Model):
img = models.CharField(max_length=255)
name = models.CharField(max_length=100)
desc = models.TextField()

5
6
7

class Meta:
ordering = ('name',)

Each user will have a reference to the badges and many users can get the same badge, so this
creates a many-to-many relationship. Thus we will update payments.models.User adding
the new relationship field:
1

badges = models.ManyToManyField(Badge)
540

The default for a ManyToManyField is to create a lookup table. After adding this code,
drop the database, re-create it, and then run syncdb and in your database you will have a
table called payments_user_badges. The badges field will manage all the relationship
stuff for you. Of course we have to add from main.models import Badge for this to work.
That causes a problem though because we already have from payment.models import
User in main.models (because we are using it in the StatusReport model). This creates a
circular reference and will cause the import to break.
NOTE Circular references dont always cause imports to break, and there are
ways to make them work, but it is generally considered bad practice to have circular references. You can find a good discussion on circular references here.

We can remove this circular reference by changing our StatusReport model class so it
doesnt have to import payments.Users. We do that like so:
1
2
3

class StatusReport(models.Model):
user = models.ForeignKey('payments.User')
...snip...

In the case of the ForeignKey field, Django allows us to reference a model by using its
name as a string. This means that Django will wait until the payments.User model is
created and then link it up with StatusReport. It also means we can remove our from
payments.main import User statement, and then we dont have a circular reference.
Next up is to return the list of badges as part of the request for the user.html page. Updating
our main.views.index() function we now have:
1

def index(request):

2
3

...snip...

4
5
6
7

else:
#membership page
status = StatusReport.objects.all().order_by('-when')[:20]

8
9
10
11
12

announce_date = date.today() - timedelta(days=30)

announce = (Announcement.objects.filter(
when__gt=announce_date).order_by('-when')
)

13
14

usr = User.get_by_id(uid)
541

15

badges = usr.badges.all()

16
17
18
19
20
21
22
23
24
25
26

return render_to_response(
'main/user.html',
{
'user': usr,
'badges': badges,
'reports': status,
'announce': announce
},
context_instance=RequestContext(request),
)

On Line 13 we get all the badges linked to the current user by calling user.badges.all().
badges is the ManyToManyField we just created. The all() function will return a list of all
the related badges. We just set that list to the context variable badges and pass it into the
template.
Speaking of which, update the user.html template:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

{% extends "__base.html" %}
{% load staticfiles %}
{% block content %}
<div id="achievements" class="row member-page hide">
{% include "main/_badges.html" %}
</div>
<div class="row member-page">
<div class="col-sm-8">
<div class="row">
{% include "main/_announcements.html" %}
{% include "main/_lateststatus.html" %}
</div>
</div>
<div class="col-sm-4">
<div class="row">
{% include "main/_jedibadge.html" %}
{% include "main/_statusupdate.html" %}
</div>
</div>
</div>
{% endblock %}
542

Lines 4-6 are the important ones here. Basically we are creating another info box that will
stretch across the top of the screen to show all the badges. But we are adding the Bootstrap
CSS class hide so the achievements row wont be shown.
The main/_badges.html template:
1
2
3
4
5
6
7
8
9
10
11
12
13

{% load staticfiles %}
<section class=" row info-box text-center" id="badges">
<h1 id="achieve">Achievements</h1>
{% for bdg in badges %}
<div class="col-lg-4">
<h2>{{ bdg.name }}</h2>
<img class="img-circle" src="{% static 'img/'|add:bdg.img %}"
width="100" height="100" alt=" {{ bdg.name }}"
title="{{ bdg.name }} - {{ bdg.desc }}">
<p>{{ bdg.desc }}</p>
</div>
{% endfor %}
</section>

Here, we loop through the badges and show them with a heading and description. Using
col-lg-4 means there will be three columns per row; if there are more than three badges, it
will just wrap and add another row.
The final thing to do is to make the Show Achievements link work, since by default the
Achievements info-box is hidden. Clicking the Show Achievements link should show them.
And once they are visible, clicking the link again should hide them. The easiest way to do
this is to use some JavaScript. We havent really talked much about JavaScript in this course
yet, but we will in an upcoming chapter. For now, just have a look at the code, which goes in
application.js:
1
2
3
4
5
6
7
8
9
10
11

//show status
$("#show-achieve").click(function() {
a = $("#achievements");
l = $("#show-achieve");
if (a.hasClass("hide")) {
a.hide().removeClass('hide').slideDown('slow');
l.html("Hide Achievements");
} else {
a.addClass("hide");
l.html("Show Achievements");
}
543

return false;
});

12
13

This handles the click event that is called when you click on the Show Achievements link.
Lets walk through the code.

Line 2: sets up the click even handler.

Line 3: grabs a reference to the achievements info box.
Line 4: grabs a reference to the Show Achievements link itself.
Line 5: if a has the hide class (i.e. Bootstrap is currently hiding the info-box), then
Line 6: remove the class hide, which will make the info box visible, and then use a
jQuery animation slideDown to make it scroll in slowly.
Line 7: changes the text of the Show Achievements link to Hide Achievements.
Line 9-11: do the opposite of lines 5-7; hide the info box and change the link text to
Show Achievements.
Line 12: return false, which prevents the screen from redrawing.

Finally, update the test_index_handles_logged_in_user() test in tests.main.testMainPageView.MainPa

so that the foreign key relationship between users and badges is established:
1
2
3

def test_index_handles_logged_in_user(self):
#create a session that appears to have a logged in user
self.request.session = {"user": "1"}

4
5
6
7
8

#setup dummy user

#we need to save user so user -> badges relationship is created
u = User(email="[email protected]")
u.save()

9
10

with mock.patch('main.views.User') as user_mock:

11
12
13
14

#tell the mock what to do when called

config = {'get_by_id.return_value': u}
user_mock.configure_mock(**config)

15
16
17

#run the test

resp = index(self.request)

18
19
20

#ensure we return the state of the session back to normal

self.request.session = {}
544

u.delete()

21
22

#we are now sending a lot of state for logged in users,

rather than
#recreating that all here, let's just check for some text
#that should only be present when we are logged in.
self.assertContains(resp, "Report back to base")

23

24
25
26

Be sure to add the import as well:

1

from payments.models import User

Finally, we need to update the database:

1. Enter the Postgres Shell:
1
2

DROP DATABASE django_db;

CREATE DATABASE django_db;

2. Exit the shell and run syncdb:

1

$ ./manage.py syncdb

Thats it. Make sure to test, then commit your code.

545

REST
Exercise 1
Question: Flesh out the unit tests. In the JsonViewTests, check the case where
there is no data to return at all, and test a POST request with and without valid
data.
Expanding the tests, we can write a test_get_member():
1
2
3

def test_get_member(self):
stat = StatusReport(user=self.test_user, status="testing")
stat.save()

4
5
6

status = StatusReport.objects.get(pk=stat.id)
expected_json = StatusReportSerializer(status).data

7
8

response = StatusMember.as_view()(self.get_request(),
pk=stat.id)

9
10

self.assertEqual(expected_json, response.data)

11
12

stat.delete()

This is very similar to the test_get_collection(). The main difference here is that we
are saving a status report for our test test user and then passing in the pk to our view. This is
the same as calling the url /api/v1/status_reports/1.
Likewise, we can test other methods such as DELETE:
1
2
3

def test_delete_member(self):
stat = StatusReport(user=self.test_user, status="testing")
stat.save()

4
5
6

response = StatusMember.as_view()(
self.get_request(method='DELETE'), pk=stat.pk)

7
8

self.assertEqual(response.status_code,
status.HTTP_204_NO_CONTENT)

9
10

stat.delete()

546

Now, in order to get this to pass, we need to update the setUpClass() method and add a
tearDownClass() method.
1
2
3
4
5

@classmethod
def setUpClass(cls):
cls.factory = APIRequestFactory()
cls.test_user = User(id=2222, email="[email protected]")
cls.test_user.save()

6
7
8
9

@classmethod
def tearDownClass(cls):
cls.test_user.delete()

Why do you think we need to add these?

Run the tests. All 38 should pass.

Exercise 2
Questions: Extend the REST API to cover the user.models.Badge.
To create a new endpoint you need to update three files:
1. main/serializers.py - create the JSON serializer
2. main/json_views.py - create the DRF views
3. main/urls.py - add the REST URIs (endpoints)
First, create the serializer:
1

class BadgeSerializer(serializers.ModelSerializer):

2
3
4
5

class Meta:
model = Badge
fields = ('id', 'img', 'name', 'desc')

Nothing to it. Just use the handy serializer.ModelSerializer. Make sure to update
the imports as well:
1

from payments.models import User, Badge

Now lets create the Collection and Member views:

547

1
2

class BadgeCollection(
mixins.ListModelMixin, mixins.CreateModelMixin,
generics.GenericAPIView
):

4
5
6
7

queryset = Badge.objects.all()
serializer_class = BadgeSerializer
permission_classes = (permissions.IsAuthenticated,)

8
9
10

def get(self, request):

return self.list(request)

11
12
13

def post(self, request):

return self.create(request)

14
15
16
17
18
19

class BadgeMember(
mixins.RetrieveModelMixin, mixins.UpdateModelMixin,
mixins.DestroyModelMixin, generics.GenericAPIView
):

20
21
22
23

queryset = Badge.objects.all()
serializer_class = BadgeSerializer
permission_classes = (permissions.IsAuthenticated,)

24
25
26

def get(self, request, *args, **kwargs):

return self.retrieve(request, *args, **kwargs)

27
28
29

def put(self, request, *args, **kwargs):

return self.update(request, *args, **kwargs)

30
31
32

def delete(self, request, *args, **kwargs):

return self.destroy(request, *args, **kwargs)

Its a bit of copy and paste from the StatusMember and StatusCollection, but its pretty
clear exactly what is going on here.
Again, update the imports:
1
2

from main.serializers import StatusReportSerializer, BadgeSerializer

from main.models import StatusReport, Badge
548

Finally we need to wire up our URIs in main/urls.py:

1
2
3
4
5
6

7
8
9

10

urlpatterns = patterns(
'main.json_views',
url(r'^$', 'api_root'),
url(r'^status_reports/$', json_views.StatusCollection.as_view(),
name='status_reports_collection'),
url(r'^status_reports/(?P<pk>[0-9]+)/$',
json_views.StatusMember.as_view()),
url(r'^badges/$', json_views.BadgeCollection.as_view(),
name='badges_collection'),
url(r'^badges/(?P<pk>[0-9]+)/$',
json_views.BadgeMember.as_view()),
)

Dont forget to add the unit tests as well. This is all you since you should be an expert now
after doing all the testing from exercise 1. If youll notice, though, once you do all your tests for
Badges, they are probably pretty similar to your tests for StatusReport. Can you factor
out arest_api_test_case? Have a look back at the testing mixins in Chapter 2 and see if you
can do something similar here.

Exercise 3
Question: Did you know that the browsable API uses Bootstrap for the look and
feel? Since we just learned Bootstrap, update the browsable API Template to fit
with our overall site template.
This is actually pretty easy to do once you know how. In case your Googling didnt find it,
the DRF documentation tells you how on this page.
Basically, all you have to do is create a template rest_framework/api.html that extends
the DRF template rest_framework/base.html. The simplest thing we can do is to
change the CSS to use the CSS we are using for the rest of our site. To do that, make a
rest_framework/api.html template look like this:
1
2
3
4

{% extends "rest_framework/base.html" %}
{% load staticfiles %}
{% block bootstrap_theme %}
<link href="{% static "css/bootstrap.min.css" %}"
rel="stylesheet">
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements
and media queries -->
549

7
8

10
11
12

<!-- WARNING: Respond.js doesn't work if you view the page via
file:// -->
<!--[if lt IE 9]>
<script
src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script
src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></scrip
<![endif]-->
<link href="{% static "css/mec.css" %}" rel="stylesheet">
{% endblock %}

Here we added the bootstrap_theme block and set it to use the Bootstrap CSS that we are
using and then also use our custom mec.css stylesheet, mainly to get our cool star_jedi font.
Super simple.
Go ahead. Test it out. Fire up the server, and then navigate to http://127.0.0.1:8000/api/v1/
status_reports/.
Again the docs have a lot more information about how to customize the look and feel of the
browsable API, so have a look if youre interested.

Exercise 4
Question: We dont have permissions on the browsable API. Add them in.
Right now, since our REST API requires authenticated users, the browsable API doesnt work
very well unless you can log in. So we can use the built-in DRF login and logout forms so
that a user can login and use the browsable API. To do that, all we need to do is update our
main/urls.py file by adding this line at the end of the urlpatterns member:
1
2

url(r'^api-auth/', include(
'rest_framework.urls', namespace='rest_framework')),

With this, you will now be able to browse to http://localhost:8000/api/v1/api-auth/login/

where you can log in as the superuser, for example. Then you can browse the API.
Another part of the browsable API that is missing is a main page where we can see all of the
URIs that the API contains. We can create one by updating our main/json_views.py file
as such:
1
2
3

from rest_framework.decorators import api_view

from rest_framework.response import Response
from rest_framework.reverse import reverse
550

4
5
6
7
8
9
10
11
12

@api_view(('GET',))
def api_root(request):
return Response({
'status_reports': reverse(
'status_reports_collection', request=request),
'badges': reverse('badges_collection', request=request),
})

This view function returns a list of the two URIs that we currently support. This way the user
can start here and click through to view the entire API. All we need to do now is add the URL
to our main/urls.py:
1

url(r'^$', 'api_root'),

Now as an API consumer we dont have to guess what the available resources are in our REST
API - just browse to http://localhost:8000/api/v1/ to view them all.

551

Django Migrations
Exercise 1
Question: At this point if you drop your database, run migrations and then run the
tests you will have a failing test because there are no MarketItems in the database.
For testing you have two options:
Load the data in the test (or use a fixture).
Load the data by using a datamigration.
The preferred option for this case is to create a data migration to load the
MarketingItems. Can you explain why?. Create the migration.

First, create a new migrations file. Use the following command to help out:
1

$ ./manage.py makemigrations --empty main

This will create a new migration file in main\migrations, which should be called
0003_auto_<some ugly date string>.py. I dont particularly care for that name
so lets rename it to data_load_marketing_itmes_0003.py. (Later it will become clear
why we put the number at the end). Then we can update the contents of the file to create the
marketing items that we need. In total the file should now look like:
1
2

# -*- coding: utf-8 -*from __future__ import unicode_literals

3
4

from django.db import models, migrations

5
6
7
8
9
10
11
12
13
14
15
16
17

#start with a list of all the data we want to add.

init_marketing_data = [
{
"img":"yoda.jpg",
"heading":"Hone your Jedi Skills",
"caption":"All members have access to our unique"
" training and achievements latters. Progress through the "
"levels and show everyone who the top Jedi Master is!",
"button_title":"Sign Up Now"
},
{
552

"img":"clone_army.jpg",
"heading":"Build your Clan",
"caption":"Engage in meaningful conversation, or "
"bloodthirsty battle! If it's related to "
"Star Wars, in any way, you better believe we do it.",
"button_title":"Sign Up Now"

18
19
20
21
22
23

},
{

24
25

"img":"leia.jpg",
"heading":"Find Love",
"caption":"Everybody knows Star Wars fans are the "
"best mates for Star Wars fans. Find your "
"Princess Leia or Han Solo and explore the "
"stars together.",
"button_title":"Sign Up Now"

26
27
28
29
30
31
32

},

33
34

35
36

def create_marketing_items(apps, schema_editor):

37
38

MarketingItem = apps.get_model("main", "MarketingItem")

39
40
41

#stare data in database

[MarketingItem(**d).save() for d in init_marketing_data]

42
43
44

class Migration(migrations.Migration):

45
46
47
48

dependencies = [
('main', '0002_statusreport'),
]

49
50
51
52

operations = [
migrations.RunPython(create_marketing_items)
]

This follows the same pattern as the user creation data migration we did in the Data Migrations section of this chapter. The main differences are as follows:
1. The top part of the file just lists all the data that we are going to load. This makes it
553

easy to see what system data we will be loading.

2. In our data creation function create_marketing_items() notice the following line:
1

MarketingItem = apps.get_model("main", "MarketingItem")

This use the app registry that is provided by the migration framework so we can get the version of the MarketingItem that corresponds to our particular migration. This is important
in case the MarketingItem model later changes. By using the app registry we can be sure
we are getting the correct version of the model.
Once we have that correct version we just create the data using a list comprehension.
But why use a funny name for the file?

Remember that we are trying to load the data so that our test main.testMainPageView.test_returns_exac
will run correctly. If you recall the test itself looks like:
1
2
3
4
5
6
7
8
9
10

```python
def test_returns_exact_html (self):
market_items = MarketingItem.objects.all()
resp = index(self.request)
self.assertEqual(
resp.content,
render_to_response("main/index.html",
{"marketing_items":market_items}
).content)
```

So we are comparing the html rendered in index.html to the html rendered when we pass in
market_items. If you recall from earlier chapters, market_items is a list that contains all
the items that we want to pass into our template to render the template. This was left over
from before we switch the view function to read from the database. So this leaves us in a state
where we have duplication. We have two lists of system data:
1. data_load_marketing_items_0003.py - the data we used to load the database
2. main.views.market_items - a left over from when we refactored the code to load
marketing items from the database.
So to remove that duplication and only have one place to store / load the initial system data
we can change our test as follows:

554

from main.migrations.data_load_marketing_items_0003 import

init_marketing_data

2
3
4
5
6
7
8
9
10
11
12
13

def test_returns_exact_html(self):
data = [MarketingItem(**d) for d in init_marketing_data]
resp = index(self.request)
self.assertEqual(
resp.content,
render_to_response(
"main/index.html",
{"marketing_items": data}
).content
)
NOTE: Dont forget to remove the market_items import at the top of this file.

What we have done here is load the init_marketing_data list from our migration directly
into our test, so we dont need to keep the list in main.views.market_items around anymore and we remove the code duplication!
Again, so why the funny name for the migration file?

Because python doesnt allow you to import a file name that starts with a number. In other
words1

from main.migrations.0003_data_load_marketing_items import

init_marketing_data

-will fail. Of course as with all things in python you can get around it by doing something like:
1

init_marketing_data =
__import__('main.migrations.0003_data_load_marketing_items.init_marketing_data'

But you really should avoid calling anything with __ if you can. So we renamed the file instead.

Exercise 2
Question We have a new requirement for two-factor authentication. Add a new
field to the user model called second_factor. Run ./manage.py makemigration
payments. What did it create? Can you explain what is going on in each line of
555

the migration? Now run ./manage.py migrate and check the database to see
the change that has been made. What do you see in the database? Now assume
management comes back and says two-factor is too complex for users; we dont
want to add it after all. List two different ways you can remove the newly added
field using migrations.

We are not going to show the code in the first part; just try it out and see what happens. For
the question at the end - list two different ways you can remove the newly added field using
migrations - the answers are:
1. You can migrate backwards to the previous migration. So assuming this was migration
0004 that came right after migration 0003 you could simply run:
1

$ ./manage.py migrate payments 0003

That will return you to before the field was created. Then just drop the field from your
model and continue.
2. The second way is to just drop the field from the model and then run makemigrations
again, which will then drop the field. At that point, since your migrations are going
around in a circle you may want to look at Squashing Migrations. Do keep in mind
though that squashing migrations is completely optional and not at all required.

Exercise 3
Question: Lets pretend that MEC has been bought by a big corporation - well call
it BIGCO. BIGCO loves making things complicated. They say that all users must
have a bigCoID, and that ID has to follow a certain formula. The ID should look
like this: <first_two_digits_in_name><1-digit-Rank_code><sign-up-date>
1-digit-Rank_code = Y for youngling, P for padwan, J for Jedi
sign-up-date = Use whatever date format you like
Since this is an ID field we need to ensure that its unique.
Now create the new field and a migration for the field, then manually write a data
migration to populate the new field with the data from the pre-existing users.

First add the field to payments.models.User:

1

bigCoID = models.CharField(max_length=50)

Then create the migration with:

556

$ ./manage.py makemigrations payments

Which will give you a warning about non-nullable fields:

1

3
4
5
6

You are trying to add a non-nullable field 'bigCoID' to user

without a default;
we can't do that (the database needs something to populate existing
rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows)
2) Quit, and let me add a default in models.py
Select an option:

Just select 1 and put in 'foo'. We are going to change it in a second.

This will create a migration with a name like 0004_<some ugly string of numbers
and dashes>, which will house your migration.
It should include this operation (notice we have changed the default value):
1
2
3
4
5
6

migrations.AddField(
model_name='user',
name='bigCoID',
field=models.CharField(max_length=50, default='foo'),
preserve_default=False,
),

There maybe another AlterField statement for the last_notification field, which you
can ignore or delete. Now just apply the migration with:
1

$ ./manage.py migrate

This should create the field for you.

Next we need to create the data migration, which we have done a couple of times now already
so you should be getting good at this now. Create a file called 0005_bigcoId_migration.py:
1
2

# -*- coding: utf-8 -*from __future__ import unicode_literals

3
4

from django.db import migrations

5
6
7

def migrate_bigcoid(apps, schema_editor):

557

User = apps.get_model('payments', 'User')

10
11
12
13
14
15
16
17

for u in User.objects.all():
bid = ("%s%s%s%s" % (u.name[:2],
u.rank[:1],
u.created_at.strftime("%Y%m%d%H%M%S%f"),
))
u.bigCoID = bid
u.save()

18
19
20

class Migration(migrations.Migration):

21
22
23
24

dependencies = [
('payments', '0004_auto_20141001_0546'),
]

25
26
27
28

operations = [
migrations.RunPython(migrate_bigcoid)
]

That should do it. Apply the migration.

What about the unique field?

You may be thinking that bigCoID should be a unique field. And youre right. The problem
is if we try to add a unique field, to a table, we need to ensure the data is unique so we have
to do that as a three step process:
1. Create a new field without unique constraint (Schema Migration)
2. Update the data in the field to make it unique (Data Migration)
3. Add a unique constraint to the field (Schema Migration)
The only other way to do it would be to create a custom migration using the RunSQL operation. But that means you have to write the SQL by hand.
Coming back to our specific example. We have already completed step 1 and 2. So all we
have to do now is edit payments/models.py and add unique=True to bigCoID. Then run
create and apply the migration as usual. And that will apply the unique constraint in the
database and you are good to go.
558

And creating new Users

One last thing we need to handle: Creating a new user. If we stop here, it will become difficult
to create new users without getting a unique key constraint error. So lets update the create
function in payments.models.User so that it will generate the bigCoID for us. The create
function will now look like this:
1
2
3
4
5

@classmethod
def create(cls, name, email, password, last_4_digits, stripe_id=''):
new_user = cls(name=name, email=email,
last_4_digits=last_4_digits, stripe_id=stripe_id)
new_user.set_password(password)

6
7
8
9
10
11
12
13

#set bigCoID
new_user.bigCoID = ("%s%s%s" % (new_user.name[:2],
new_user.rank[:1],
datetime.now().strftime("%Y%m%d%H%M%S%f"),
))
new_user.save()
return new_user

Notice that we are using datetime.now().strftime("%Y%m%d%H%M%S%f"). This is a datetime string accurate to the micro second. Otherwise known as a poor mans unique id. Since
we are accurate down to the micro second there is an extremely low probability that there will
ever be two users created in the same micro second and thus this should basically always produce a unique value. You could also use a database sequence to ensure uniqueness. Take a
look at this django snippet, for example.
Also, we need to update a few test in tests.payments.testUserModel to use the create
method (as we didnt strictly require it until now):
1
2
3
4

@classmethod
def setUpClass(cls):
cls.test_user = User.create(email="[email protected]", name='test user',
password="pass",
last_4_digits="1234")

5
6
7

def test_create_user_function_stores_in_database(self):
self.assertEquals(User.objects.get(email="[email protected]"),
self.test_user)

Here we simply update the setUpClass() and the test_user_function_stores_in_database()

function to use User.create(). Also to be extra safe you may want to create another test
559

called something like test_create_two_users_each_user_has_unique_bigCoID()

to ensure that our create function is indeed generating unique values for bigCoID.

560

AngularJS Primer
Exercise 1
Question: Our User Poll example uses progress bars, which are showing a percentage of votes. But our vote function just shows the raw number of votes, so
the two dont actually match up. Can you update the vote() function to return
the current percentage of votes. (HINT: You will also need to keep track of the
total number of votes.)
The answer to this question relies on storing a bit more information in the controller. Since
we are updating percentages, we will have to update every item in the list each time a vote is
placed. Heres a look at the controller in its entirety, and then we will break it down line by
line:
1
2
3

mecApp.controller('UserPollCtrl', function($scope) {
$scope.total_votes = 0;
$scope.vote_data = {}

4
5
6
7
8
9
10

11
12
13
14
15
16

$scope.vote = function(voteModel) {
if (!$scope.vote_data.hasOwnProperty(voteModel)) {
$scope.vote_data[voteModel] = {"votes": 0, "percent": 0};
$scope[voteModel]=$scope.vote_data[voteModel];
}
$scope.vote_data[voteModel]["votes"] =
$scope.vote_data[voteModel]["votes"] + 1;
$scope.total_votes = $scope.total_votes + 1;
for (var key in $scope.vote_data) {
item = $scope.vote_data[key];
item["percent"] = item["votes"] / $scope.total_votes * 100;
}
};

17
18

});

Okay. Lets break it down line by line:

Line 1 - No change here; just the definition of our controller.
Line 2 - $scope.total_votes = 0;. We are keeping track of the total number of
votes so we can calculate percentages.
561

 Line 3 - $scope.vote_data = {}. This creates an empty JavaScript object (which

in practice functions much like a Python dictionary). We will use this as the data structure to store information about the votes. We are not defining the data structure yet,
but when we do, it will look like this:

1
2
3
4
5
6
7
8
9

{ 'votes_for_yoda':
{ 'votes': 0,
'percent': 0'
},
'votes_for_qui':
{ 'votes': 0,
'percent': 0'
},
}

Of course the structure will continue for each Jedi we add to the list. But the basic idea is that
we have a key with votes_for_ + the jedi name, and that key returns the number of votes
and the percentage of all votes.
Line 6 and 7 - Since we have not defined any data in our votes_data data
structure, we need to dynamically add it each time the vote() function is called.
hasOwnProperty checks to see if we have a key equal to the voteModel string passed
in, which should be something like votes_for_yoda. If we dont, then this is the first
time that particular Jedi has been voted for, so we will create a new data structure for
the Jedi and initialize everything to 0.
Line 8 - This line is completely optional, and were adding it to save typing in the view.
Instead of typing something like {{ vote_data.votes_for_yoda.percent }}, we
can simply say - {{ votes_for_yoda.percent }}.
Line 10 - Increase the number of votes for the current Jedi by one.
Line 11 - Keep track of the total number of votes.
Lines 12-14 - Loop through the data structure of all Jedis and recalculate their associated percentage of votes.
That gets us through the controller, which will automatically update all associated vote percentages.
Now we do need to make some updates to the view as well. To save space, lets just look at
Yoda:

562

2
3

4
5

7
8

<span ng-click='vote("votes_for_yoda")' class="glyphicon

glyphicon-plus"></span>
<strong>Yoda</strong>
<span class="pull-right">{{ votes_for_yoda.percent | number:0
}}%</span>
<div class="progress">
<div class="progress-bar progress-bar-danger" role="progressbar"
aria-value="{{ votes_for_yoda.percent }}"
aria-valuemin="0" aria-valuemax="100" style="width: {{
votes_for_yoda.percent }}%;">
</div>
</div>
Line 1 - The call to the controller - e.g., ng-click='vote("votes_for_yoda")' is unchanged.
Line 3 - {{ votes_for_yoda.percent | number:0 }}. Here the expression being evaluated asks for the percentage attached to votes_for_yoda and uses the Angular number filter to format the number with 0 decimal places. Remember: We can
use the shorthand to get the model because of what we did in Line 8 of the controller.
Lines 3, 5, and 6 - Again, we just access the percent for the appropriate votes item
from the controller.

Likewise, we repeat the same structure for each vote item, and now you have a set of progress
bars that always total up to 100 and all dynamically update each time you cast a vote!
Curious to see the vote count for each individual Jedi? Update the HTML like so:
1

<strong>Yoda (votes: {{ votes_for_yoda.votes }})</strong>

Be sure to do this for all Jedis.

563

Djangular: Integrating Django and Angular

Exercise 1
Question: For the first exercise lets explore storing state in our factory. Change
the pollFactory.getPoll function to take no parameters and have it return the
single most recent poll. Then cache that polls id, so next time getPoll() is called
you have the id of the poll to retrieve.
Before we even begin to element caching, we need to make sure out API returns a poll.id.
Lets test it out. Fire up the server, then navigate to http://localhost:8000/api/v1/polls/
?format=json. You should see something like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

[
[
{
"id": 1,
"title": "Who is the best Jedi?",
"publish_date": "2014-10-21T04:05:24.107Z",
"items": [
{
"id": 1,
"poll": 1,
"name": "Yoda",
"text": "Yoda",
"votes": 1,
"percentage": 50
},
{
"id": 2,
"poll": 1,
"name": "Vader",
"text": "Vader",
"votes": 1,
"percentage": 50
},
{
"id": 3,
"poll": 1,
"name": "Luke",
564

"text": "Luke",
"votes": 0,
"percentage": 0

28
29
30

31

],
"total_votes": 2

32
33

34
35

So, yes - there is a poll.id. Where is the code for this? Open djangular_polls/serializers.py.
Its the line:
1

fields = ('id', 'title', 'publish_date', 'items', 'total_votes')

Okay. Now to update the controller. We could change the JSON API to provide a way to
return only the latest poll, but lets learn a bit more about Angular instead. Update the
pollFactory:
1

pollsApp.factory('pollFactory', function($http, $filter) {

2
3
4
5
6
7

var
var
var
var
var

baseUrl = '/api/v1/';
pollUrl = baseUrl + 'polls/';
pollItemsUrl = baseUrl + 'poll_items/';
pollId = 0;
pollFactory = {};

8
9
10
11

pollFactory.getPoll = function() {
var tempUrl = pollUrl;
if (pollId != 0) { tempUrl = pollUrl + pollId; }

12
13
14
15

16
17
18
19

return $http.get(pollUrl).then(function(response)
{
var latestPoll = $filter('orderBy')(response.data,
'-publish_date')[0];
pollId = latestPoll.id;
return latestPoll;
});
};

20
21
22
23

pollFactory.vote_for_item = function(poll_item) {
poll_item.votes +=1;
return $http.put(pollItemsUrl + poll_item.id, poll_item);
565

24

25
26
27

return pollFactory;
});
Line 5 - var pollId = 0;: This is our state variable to store the poll id.
Line 8 - Notice we arent taking in any parameters (before the function took an id).
Lines 9-10 - If we dont have a pollId cached then just call /api/v1/polls/,
which returns the complete list of polls. Otherwise pass the id, so we would be calling
/api/v1/polls/<pollId>.
Line 11 Notice we are calling return on our $http.get().then() function. then()
returns a promise. So, we will be returning a promise, but only after we first call
$http.get, then we call our then function (i.e., lines 12-16), then the caller or controller gets the return value from line 13.
Line 13 - This may be something new to you. Its the $filter function from Angular,
which is the same function that we use in the Angular templates - i.e., [[ model |
filter ]]. Here we are just calling it from the controller.

Lets digress for a second

To access the $filter function we need to inject it into the top of our factory like so:
1

pollsApp.factory('pollFactory', function($http, $filter) {

Then a general form of the filter function looks like this:

1

$filter(filterName)(array, expression, reverse)

filterName is the name of the filter to use. In our case we are using the orderBy
filter.
array is the array to filter. For us, its response.data from our $http.get - i.e., our
list of Polls.
expression generally takes the name of the attribute on the object you want to order
by. You can also prepend + or - to the front of the attribute to control the order to be
ascending or descending. + is default.
reverse - instead of prepending + or -, you can pass reverse, which is a boolean value.

So this line orders the list for us, and we take the first element in the list after it is ordered.
Back to the pollFactory code:
566

 Line 14 - grab the id from the poll item we are working with and
Line 15 - return the poll item.
Finally, there is a slight change we need to make in our controller. Because our getPoll
function is now returning a Poll as the promise instead of the response as the promise, we
need to change the setPoll function. It should now look like this:
1
2
3

function setPoll(promise){
$scope.poll = promise;
}

And thats it. You will now get the whole list of polls for the first call, and after that only get
the poll you are interested in.

Exercise 3
Question: Currently our application is a bit of a mismatch, we are using jQuery on
some parts - i.e,. showing achievements - and now Angular on the user polls. For
practice, convert the showing achievement functionality, from application.js,
into Angular code.
Lets handle this in four parts
Part 1

The first thing we need to do is initialize our ng-app somewhere more general. Im a big fan
of DRY, so lets just declare the ng-app on the <html> tag in __base.html.
1

<html lang="en" ng-app="mecApp">

Well call it mecApp since thats more fitting for our application. This also means we need to
remove it from __polls.html. Also, lets move the declaration of the angular.module to
application.js (so it will be the first thing created). The top of application.js should
now look like:
1

var mecApp = angular.module('mecApp', []);

2
3
4
5
6

mecApp.config(function($interpolateProvider) {
$interpolateProvider.startSymbol('[[')
.endSymbol(']]');
});
567

And, of course, in userPollCtrl.js we no longer need the angular.module declaration:

1

var pollsApp = angular.module('pollsApp',[]);

2
3
4
5
6

pollsApp.config(function($interpolateProvider){
$interpolateProvider.startSymbol('[[')
.endSymbol(']]');
});

Then everywhere that we referenced pollsApp we need to replace with mecApp. With that
change, everything should continue to work fine. It will allow us to use Angular throughout
our entire application as opposed to just for the poll app.
Part 2

Now to make the BIG changes.

Lets start with declaring a LoggedInCtrl in a new file called static/js/loggedInCtrl.js:
1
2
3

mecApp.controller('LoggedInCtrl', function($scope) {
$scope.show_badges = false ;
$scope.show_hide_label = "Show";

4
5
6
7
8
9

$scope.show = function() {
$scope.show_badges = ! $scope.show_badges;
$scope.show_hide_label = ($scope.show_badges) ? 'Hide': 'Show';
}
});

The above is a very simple controller where we define two models, $scope.show_badges
and $scope.show_hide_label. We also define a function called show(), which will be
called when the user clicks on the Show Achievements link. So lets look at that link in
templates/main/_jedibadge.html
Change this:
1

<li><a id="show-achieve" href="#">Show Achievements</a></li>

To:
1

<li ng-click="show()"><a href="#">[[ show_hide_label ]]

Achievements</a></li>

Weve got two bits of Angular functionality tied into the list item:
568

1. ng-click="show()" calls the $scope.show() function from our LoggedInCtrl,

which toggles the value of $scope.show_badges between true and false. The function also sets the model $scope.show_hide_label to either Hide (if show_badges
is true) or Show.
2. [[ show_hide_label ]] substitutes the value of $scope.show_hide_label so
that the link text will read Show Achievements or Hide Achievements appropriately.
Part 3

The next piece of the puzzle is the updated template templates/main/user.html:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

{% extends "__base.html" %}
{% load staticfiles %}
{% block content %}
<div ng-controller="LoggedInCtrl">
<div id="achievements" class="row member-page"
ng-class="{hide: show_badges==false}">
{% include "main/_badges.html" %}
</div>
<div class="row member-page">
<div class="col-sm-8">
<div class="row">
{% include "main/_announcements.html" %}
{% include "main/_lateststatus.html" %}
</div>
</div>
<div class="col-sm-4">
<div class="row">
{% include "main/_jedibadge.html" %}
{% include "main/_statusupdate.html" %}
{% include "djangular_polls/_polls.html" %}
</div>
</div>
</div>
</div>
{% endblock %}
{% block extrajs %}
<script src="{% static "js/userPollCtrl.js" %}"
type="text/javascript"></script>
569

28

29

<script src="{% static "js/loggedInCtrl.js" %}"

type="text/javascript"></script>
{% endblock %}

The interesting part is lines 4-8 of the template:

1
2
3
4
5

<div ng-controller="LoggedInCtrl">
<div id="achievements" class="row member-page"
ng-class="{hide: show_badges==false}">
{% include "main/_badges.html" %}
</div>
** Line 1** - Here we declare our controller.
** Line 3** - ng-class is a core Angular directive used to add/remove classes dynamically. So, if the value of $scope.show_badges is false, then we add the class hide to
our div; otherwise, remove the class from our div. This is all we need to do to show/hide
the div.

Part 4

Finally, we need to update our scripts in __base.html:

1
2

4
5

<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->

<script src="{% static "js/jquery-1.11.1.min.js" %}"
type="text/javascript"></script>
<!-- Include all compiled plugins (below), or include individual
files as needed -->
<script src="{% static "js/bootstrap.min.js" %}"></script>
<script src="{% static "js/angular.min.js" %}"
type="text/javascript"></script>
<script src="{% static "js/application.js" %}"
type="text/javascript"></script>
<script src="{% static "js/userPollCtrl.js" %}"
type="text/javascript"></script>
<script src="{% static "js/loggedInCtrl.js" %}"
type="text/javascript"></script>

And then we can remove the following from templates/main/user.html:

1

<script src="{% static "js/userPollCtrl.js" %}"

type="text/javascript"></script>
570

<script src="{% static "js/loggedInCtrl.js" %}"

type="text/javascript"></script>

Thats everything to convert our Show Achievements functionality to Angular, so we can

remove the corresponding jQuery code from application.js.

571

Angular Forms
Exercise 1
Question: We are not quite done yet with our conversion to Angular, as that
register view function is begging for a refactor. A good way to organize things
would be to have the current register view function just handle the GET requests and return the register.html as it does now. As for the POST requests, I
would create a new users resource and add it to our existing REST API. So rather
than posting to /register, our Angular front-end will post to /api/v1/users.
This will allow us to separate the concerns nicely and keep the code a bit cleaner.
So, have a go at that.
The most straight forward way to do this is to leverage Django Rest Framework. The first
thing to do is to create a new file payments/json_views.py. Then add a post_user()
function to that file:
1
2
3
4
5
6
7

from
from
from
from
from
from
from

payments.models import User, UnpaidUsers

payments.forms import UserForm
payments.views import Customer
rest_framework.response import Response
django.db import transaction
django.db import IntegrityError
rest_framework.decorators import api_view

8
9
10
11
12

@api_view(['POST'])
def post_user(request):
form = UserForm(request.DATA)

13
14
15
16

17
18
19
20
21
22

if form.is_valid():
try:
#update based on your billing method (subscription vs
one time)
customer = Customer.create(
"subscription",
email=form.cleaned_data['email'],
description=form.cleaned_data['name'],
card=form.cleaned_data['stripe_token'],
plan="gold",
572

23
24
25

)
except Exception as exp:
form.addError(exp)

26
27
28
29
30
31
32

cd = form.cleaned_data
try:
with transaction.atomic():
user = User.create(
cd['name'], cd['email'],
cd['password'], cd['last_4_digits'])

33
34
35
36
37
38

if customer:
user.stripe_id = customer.id
user.save()
else:
UnpaidUsers(email=cd['email']).save()

39
40
41
42
43
44
45

except IntegrityError:
form.addError(cd['email'] + ' is already a member')
else:
request.session['user'] = user.pk
resp = {"status": "ok", "url": '/'}
return Response(resp, content_type="application/json")

46
47
48
49
50
51

resp = {"status": "fail", "errors": form.non_field_errors()}

return Response(resp)
else: # for not valid
resp = {"status": "form-invalid", "errors": form.errors}
return Response(resp)

After all the imports, we declare our view function (on Line 10 ) using the api_view decorator and passing in the list of http methods we support - just POST, in our case. The
post_user() function is basically the same as it was in the register view but we can take
a few shortcuts because we are using DRF:
Line 10 - No need to convert the request to JSON; DRF does that for us; we just load
up the UserForm.
Lines 40-41, 43-44, 46-47 - No need to convert the response back to JSON; just use
rest_framework.response.Response and it converts for us.

573

Now we just need to update our url config, so lets add a file payments/urls.py
1

from django.conf.urls import patterns, url

2
3
4
5
6
7

urlpatterns = patterns(
'payments.json_views',
url(r'^users$', 'post_user'),
)

Lets update our main url file, django_ecommerce/urls.py, the same way we did when we
added djangular_polls
Add the import:
1

from payments.urls import urlpatterns as payments_json_urls

And code:
1

main_json_urls.extend(payments_json_urls)

The file should now look like:

1
2
3
4
5

from django.conf.urls import patterns, include, url

from django.contrib import admin
from payments import views
from main.urls import urlpatterns as main_json_urls
from djangular_polls.urls import urlpatterns as
djangular_polls_json_urls
from payments.urls import urlpatterns as payments_json_urls

7
8
9
10

admin.autodiscover()
main_json_urls.extend(djangular_polls_json_urls)
main_json_urls.extend(payments_json_urls)

11
12
13
14
15
16
17
18

urlpatterns = patterns(
'',
url(r'^admin/', include(admin.site.urls)),
url(r'^$', 'main.views.index', name='home'),
url(r'^pages/', include('django.contrib.flatpages.urls')),
url(r'^contact/', 'contact.views.contact', name='contact'),
url(r'^report$', 'main.views.report', name="report"),

19

574

# user registration/authentication
url(r'^sign_in$', views.sign_in, name='sign_in'),
url(r'^sign_out$', views.sign_out, name='sign_out'),
url(r'^register$', views.register, name='register'),
url(r'^edit$', views.edit, name='edit'),

20
21
22
23
24
25

# api
url(r'^api/v1/', include('main.urls')),

26
27
28

Finally, we have one last tiny change in our static/js/RegisterCtrl.js in order to point
it to the new url:
1
2
3
4

5
6
7
8
9
10

mecApp.factory("UserFactory", function($http) {
var factory = {}
factory.register = function(user_data) {
return $http.post("/api/v1/users",
user_data).then(function(response)
{
return response.data;
});
}
return factory;
});

Easy, right?

Exercise 2
Question: As I said in the earlier part of the chapter, Im leaving the form validation for _cardform.html to the user. True to my word, here it is as an exercise.
Put in some validation for credit card, and CVC fields.
Lets look at a simple way to add field validation for our credit card number and CVC fields.
Update django_ecommerce/templates/payments/_cardform.html:
1
2
3
4

<div class="clearfix">
<label for="credit_card_number">Credit card number</label>
<div class="input">
<input class="field" id="credit_card_number" name="cc_num"
type="text"
ng-model="card.number" ng-minlength="16" required>
575

6
7
8

10

11
12

</div>
<div class="custom-error"
ng-show="user_form.cc_num.$dirty && user_form.cc_num.$invalid"
style="display: none;">
Credit card number is invalid:<span
ng-show="user_form.cc_num.$error.required">value is
required</span>
<span ng-show="user_form.cc_num.$error.minlength">min length of
sixteen</span>
</div>
</div>

13
14
15
16
17

18
19
20

21

22

23
24

<div class="clearfix">
<label for="cvv">Security code (CVC)</label>
<div class="input">
<input class="small" id="cvc" type="text" name="cvc"
ng-model="card.cvc" required
ng-minlength="3">
</div>
<div class="custom-error" ng-show="user_form.cvc.$dirty &&
user_form.cvc.$invalid" style="display: none;">
CVC is invalid:<span
ng-show="user_form.cvc.$error.required">value is
required</span>
<span ng-show="user_form.cvc.$error.minlength">min length of
three</span>
</div>
</div>

This is similar to how we added field validation to our user_form. We add the ng-minlength
attribute and the required attribute to each of our input field and then have a custom error
div ( Line 7-11, 20-23 ) that displays an error message based upon the error that was raised
by angular.
SEE ALSO: If you wanted to validate to ensure an input is a number you would
think you could use the attribute type=number but there apparently is a bug
in Angular for those types of validations, so you would have to create your own
custom directive to get around that. There is a good explanation of how to do
that on blakeembrey.com.

576

This validation is a bit week, as you might still get an invalid credit card that is 16 digits long.
The Stripe API actually provides validation methods that you can call. Like the link above
you would need to create a custom directive or filter to do the validation for you. But there is
already an angular-stripe library that does most of that heavy lifting for you. Have a look at
the library on Github

Exercise 3
Question: While the code we added to templates/payments/_field.html is great
for our register page, it also affects our sign in page, which is now constantly
displaying errors. Fix it!
Remember that _field.html uses the template expression1

ng-show="{{form.form_name}}.{{field.name}}.$dirty &&
{{form.form_name}}.{{field.name}}.$invalid"

-which relies on the form having a form_name attribute which our SigninForm doesnt have.
So edit payments/forms.SigninForm and add the attribute just like you did previously for
payments.forms.UserForm. So now your SigninForm should look like:
1
2
3
4

class SigninForm(PaymentForm):
email = forms.EmailField(required=True)
password = forms.CharField(
required=True,
widget=forms.PasswordInput(render_value=False)
)

6
7
8

form_name = 'signin_form'
ng_scope_prefix = 'signinform'

Then update templates/payments/sign_in.html to use that form name, by modifying

the from tag so it looks like this:
1
2

<form accept-charset="UTF-8" action="{% url 'sign_in' %}"

name="{{form_name}}" class="form-signin" role="form"
method="post">{% csrf_token %}

Now your error messages will be gone!

Bonus points if you can find one more form that also needs the same treatment. Give
up? Take a look at contact.forms.ContactView. It needs the same treatment as our
SigninForm got.
577

MongoDB Time!
Exercise 1
Question: Write a MongoTestCase that clears out a MongoDB database prior to
test execution/between test runs.
To accomplish this we just need to extend Djangos built-in testing capabilities. Lets use
django.test.runner.DiscoverRunner for that:
1
2
3
4
5

from
from
from
from
from

django.test.runner import DiscoverRunner

mongoengine import connect
mongoengine.connection import disconnect
pymongo import Connection
django.conf import settings

6
7
8

class MongoTestRunner(DiscoverRunner):

9
10
11
12

def setup_databases(self, **kwargs):

# since we connect to the database in settings.py
disconnect()

13
14
15
16

db_name = "test_%s" % settings._MONGO_NAME

connect(db_name,alias='default')
print('Creating test-database:', db_name)

17
18

return db_name

19
20
21
22
23

def teardown_databases(self, db_name, *args):

conn = Connection()
conn.drop_database(db_name)
print('Dropping test-database:', db_name)

In the above code example we are overwriting the setup_databases() and teardown_databases()
methods from the parent DiscoverRunner.
In the setup_databases() method we use mongoengine to create a new MongoDB
database. The test database has its named derived from the word test_ and the name
defined in our settings.py file. Which in our case is mec-geodata.

578

Then in the teardown_databases() method we use pymongo to drop all collections in the
database.
We can also make a MongoTestCase if we need to do any setup/configuration per test. It
would look like this:
1

from django.test import TestCase

2
3
4

class MongoTestCase(TestCase):

5
6
7

def _fixture_setup(self):
pass

8
9
10

def _fixture_teardown(self):
pass

There is a skeleton to use. Feel free to insert your own functionality.

The final step is to get django to use our new MongoTestRunner. Which can be done through
settings.py, by setting TEST_RUNNER to be what you want. For us, it looks like this:
1

TEST_RUNNER = 'django_ecommerce.tests.MongoTestRunner'

This will tell Django to use your runner always. Of course you can use a temporary settings
file for this as well, if you want to switch back and forth between database types:
1

from django_ecommerce.settings import *

2
3

TEST_RUNNER = 'django_ecommerce.tests.MongoTestRunner'

And that can be called from the command line by using something like:
1

$ ./manage.py test ../tests --settings=mongo_settings

Thats all there is to it. Now you have a way to test MongoDB as well. So get cracking - write
some tests!

579

One Admin to Rule Them All

Exercise 1
Question We didnt customize the interfaces for several of our models, as the customization would be very similar to what we have already done in this chapter.
For extra practice, customize both the list view and the change form for the following models:
Main.Announcements
Main.Marketing Items
Main.Status reports
Note both Announcements and Marketing Items will benefit from the thumbnail
view and ImageField setup that we did for Badges.

Lets go through the models one at a time. All the below code will go into main/admin.py
First up Main.Announcements

Lets create the ModelAdmin and get the list view working correctly.
1
2

@admin.register(Announcement)
class AnnouncementAdmin(admin.ModelAdmin):

3
4

list_display = ('when', 'img', 'vid', 'info_html')

5
6
7

def info_html(self, announcement):

return format_html(announcement.info)

8
9
10

info_html.short_description = "Info"
info_html.allow_tags = True

Now we are going to fix the img field in the same way we did for badges by making it an
ImageField and adding a thumbnail view. After doing that our model should now look like
this:
1

class Announcement(models.Model):

2
3

when = models.DateTimeField(auto_now=True)
580

5
6

img = models.ImageField(upload_to="announce/", null=True,

blank=True)
vid = models.URLField(null=True, blank=True)
info = models.TextField()

7
8
9
10

11
12

def thumbnail(self):
if self.img:
return u'<img src="%s" width="100" height="100" />' %
(self.img.url)
else:
return "no image"

13
14

thumbnail.allow_tags = True

And we need to update our AnnouncementAdmin.list_display to use thumbnail instead

of img. Then you can add / view images from the admin view. Also dont forget to run
./manage.py makemigrations and ./manage.py migrate to update the database accordingly.
Then of course just like for Badges we will have to update our main site to point to the new
images. So we update templates/main/_announcements.html to load the image from
the media URL by changing this line:
1

<img src="{% static 'img/'|add:a.img %}"/>

<img src="{{MEDIA_URL}}{{a.img}}"/>

to

With that images should display nicely on the front end.

How about the videos? Well if you remember from way back early in the book, we used a
library called django-embed-video to add the links to youtube videos to our user page. Well
as it turns out that same library offers Admin support. Bonus! We need to do a couple of
things.
First, update our model again to use the EmbedVideoField which is just a lightweight wrapper around a URLField:
1

from embed_video.fields import EmbedVideoField

2
3

class Announcement(models.Model):

4
5

when = models.DateTimeField(auto_now=True)
581

7
8

img = models.ImageField(upload_to="announce/", null=True,

blank=True)
vid = EmbedVideoField(null=True, blank=True)
#----- snip --- #

This is necessary so we can get the correct widget displayed in our Admin View. Also, this
will require you to make and run migrations again so go ahead and do that.
Now lets modify our AnnouncementAdmin to use embed_video.admin.AdminVidoMixin.
All we have to change is the class definition:
1

from embed_video.admin import AdminVideoMixin

2
3
4

@admin.register(Announcement)
class AnnouncementAdmin(AdminVideoMixin, admin.ModelAdmin):

Once that Mixin is in place, it will take care of finding any fields of type EmbedVideoField
and will create a custom widget that displayes the video and the url for the video in the change
form. So if you now look go to http://127.0.0.1:8000/admin/main/announcement/2/ you
will see a playable thumbnail of the video plus a text box to input the url for the video.
There you go. That should do it for Announcements. Feel free to tweak anything else you like,
but that should get you most of the functionality you need.
Main.Marketing Items

Marketing Items will end up being very similar to Announcements. Lets start off with the
model updates, since we now have three models that will likely use the thumbnail view, it
time to refactor.
First lets create a ThumbnailMixin:
1
2

3
4

class ThumbnailMixin(object):
'''use this mixin if you want to easily show thumbnails for an
image field
in your admin view
'''

5
6
7
8

def thumbnail(self):
if self.img:
return u'<img src="%s" width="100" height="100" />' %
(self.img.url)
else:
582

10

return "no image"

11
12

thumbnail.allow_tags = True

Then we can add the ThumbnailMixin to the models MarketingItem, Announcement, and
Badge. With that here is what MarketingItem will look like:
1
2
3
4
5

class MarketingItem(models.Model, ThumbnailMixin):

img = models.ImageField(upload_to="marketing/")
heading = models.CharField(max_length=300)
caption = models.TextField()
button_link = models.CharField(null=True, blank=True,
max_length=200, default="register")
button_title = models.CharField(max_length=20,default="View
details")

As per usual, dont forget to run migrations:

1
2

$ ./manage.py makemigrations
$ ./manage.py migrate

For the Admin view, lets do something a little different. If you remember from the Bootstrap
chapter we created a template tag marketing__circle_item to display marketing items.
So why not use that tag in the admin view so Administrators can immediately see what the
marketing item will look like on the live site. Below is the admin class that will make that
happen:
1

from django.template.loader import render_to_string

2
3
4

@admin.register(MarketingItem)
class MarketingItemAdmin(admin.ModelAdmin):

5
6

list_display = ('heading', 'live_view')

7
8
9

10

def live_view(self, mi):

return
render_to_string("main/templatetags/circle_item.html",
{'marketing_items': (mi,)})

11
12
13

live_view.short_description = "Rendered Template"

live_view.allow_tags = True

583

The magic is in the live_view function which simply renders the template associated with
the marketing__circle_item template tag and passes in the MarketItem mi as the context
for the template.
Try it out, youll see the fully rendered marketing item in the admin list view!
One more to go
Main.StatusReports

This one is even simpler. No need to change the model. Lets just create an Admin class:
1
2

from django.forms import Textarea

from django.db import models

3
4
5

@admin.register(StatusReport)
class StatusReportAdmin(admin.ModelAdmin):

6
7

list_display = ('status', 'user', 'when')

8
9
10

11

formfield_overrides = {
models.CharField: {'widget': Textarea(attrs={'rows': 4,
'cols': 70})},
}

The formfield_overrides is the only thing new here. What it says is to change the widget used to edit any field that is a models.CharField to a Textarea and pass attrs to
Textarea.__init__(). This will give us a large Textarea to edit the status message. We
want to do this since the max_length for the field is 200 and its just easier to type all that
in a Textarea.
Okay. Thats it for the models. Things should be looking better now.

Exercise 2
Question: For our Payments.Users object in the change form you will notice that
badges section isnt very helpful. See if you can change that section to show a list
of the actual badges so its possible for the user to know what badges they are
adding / removing from the user.
There are a few ways to do this. Probably the simplest is to use filter_horizontal or
filter_vertical which will give you a nice jQuery select that shows what you have chosen.
To do that just add a single line to the bottom of your UserAdmin. Here is the full UserAdmin:
584

1
2

@admin.register(User)
class UserAdmin(admin.ModelAdmin):

3
4

5
6
7
8
9
10
11

list_display = ('name', 'email', 'rank', 'last_4_digits',

'stripe_id')
ordering = ('-created_at',)
fieldsets = (
('User Info', {'fields': ('name', 'email', 'rank',)}),
('Billing', {'fields': ('stripe_id',)}),
('Badges', {'fields': ('badges',)}),
)
filter_horizontal = ('badges',)

That will give us a nice select, but all options will just say badge object. We can change
this like we changed the SelectWidget by adding a __str__ function to our Badge model.
Such as:
1
2

def __str__(self):
return self.name

585

Testing, Testing, and More Testing

Exercise 1
Question: Write a Page Object for the Registration Page, and use it to come up
with test for successful and unsuccessful registrations. Youll need to use the
LiveServerTestRunner for the tests to execute successfully.
Well start by creating our page object which is very similar to the page object we created for
the login page:
1

class RegisterPage(SeleniumPage):

2
3
4
5
6
7
8
9
10
11

12
13

name_textbox = SeleniumElement((By.ID, 'id_name'))

email_textbox = SeleniumElement((By.ID, 'id_email'))
pwd_textbox = SeleniumElement((By.ID, 'id_password'))
ver_pwd_textbox = SeleniumElement((By.ID, 'id_ver_password'))
cc_textbox = SeleniumElement((By.ID, 'credit_card_number'))
cvc_textbox = SeleniumElement((By.ID, 'cvc'))
expiry_month_dd = SeleniumElement((By.ID, 'expiry_month'))
expiry_year_dd = SeleniumElement((By.ID, 'expiry_year'))
register_title = SeleniumElement((By.CSS_SELECTOR,
'.form-signin-heading'))
register_button = SeleniumElement((By.ID, 'user_submit'))
errors_div = SeleniumElement((By.CSS_SELECTOR, ".alert"))

Nothing special here; we are just defining our elements like we did with SignInPage.
Next are a few properties:
1
2
3
4
5
6

@property
def error_msg(self):
'''the errors div has a 'x' to close it
let's not return that
'''
return self.errors_div.text[2:]

7
8
9
10

@property
def rel_url(self):
return '/register'

Again, similar to SignInPage do note for the error_msg property we trim off the first two
characters since they are used to show the X a user can click on to hide the errors.
586

Continuing with the rest of the class

1
2
3

def go_to(self):
self.driver.get('%s%s' % (self.base_url, self.rel_url))
assert self.register_title.text == "Register Today!"

4
5
6
7
8
9
10

def mock_geoloc(self, lat, lon):

self.driver.execute_script('''angular.element($('#id_email'))
.scope().geoloc = {
'coords': {'latitude': '%s','longitude':'%s'}};
angular.element(document.body).injector().get('$rootScope').$apply();
''' % (lat, lon))

Okay. Lets look at that mock_geoloc(). We talked about this in the You need to run
some JavaScript code section of the chapter. Basically, we are setting the value of the
$scope.geoloc to whatever the user passes in. This way we can test that geolocation is
working, because with the default browser you get when you run selenium, the geolocation
functionality probably wont work correctly.
1
2
3

def set_expiry_month(self, month_as_int):

selector = "option[value='%s']" % (month_as_int)
self.expiry_month_dd.find_element_by_css_selector(selector).click()

4
5
6
7

def set_expiry_year(self, year_as_int):

selector = "option[value='%s']" % (year_as_int)
self.expiry_year_dd.find_element_by_css_selector(selector).click()

What are we doing here?

1. Setting our drop downs, by clicking on the option tag in the drop down. Note here
that normally we say webdriver.find_element..., but now we are in effect saying
webelement.find_element.... When we do this, our find will be restricted to only
search through the child DOM objects of the web element. This is how we can ensure
that we are finding only the option tags that belong to our drop down. These functions
could be refactored up in the SeleniumElement class consider that extra credit.
1
2

def do_reg(self, name, email, pwd, pwd2, cc, cvc,

expiry_month, expiry_year, lat=1, lon=2):

3
4
5

self.name_textbox.send_keys(name)
self.email_textbox.send_keys(email)
587

6
7
8
9
10
11
12
13

self.pwd_textbox.send_keys(pwd)
self.ver_pwd_textbox.send_keys(pwd2)
self.cc_textbox.send_keys(cc)
self.cvc_textbox.send_keys(cvc)
self.set_expiry_month(expiry_month)
self.set_expiry_year(expiry_year)
self.mock_geoloc(lat, lon)
self.register_button.click()

This mimics the doLogin() function we have on the SignInPage. It just fills out all the
necessary fields, and, as a bonus, mocks out the geolocation value as well.
Now onto the actual test.
1

class RegistrationTests(TestCase):

2
3
4
5

6
7
8
9

@classmethod
def setUpClass(cls):
from selenium.webdriver.firefox.firefox_profile import
FirefoxProfile
profile = FirefoxProfile()
profile.set_preference('geo.prompt.testing', True)
profile.set_preference('geo.prompt.testing.allow', True)
cls.browser = webdriver.Firefox(profile)

10
11
12

cls.browser.implicitly_wait(10)
super(RegistrationTests, cls).setUpClass()

This is the hardest part of the test, and probably sent you googling for a bit. Thats all part
of the programming game. If you ran the test as you may have noticed the browser kept
asking you if it was okay to give your location. (Wether or not the browser asks you this is
controlled by a setting in your firefox profile). Luckily with selenium we have access to the
Firefox profile and we can set whatever values we want. These settings above will ensure that
you are not prompted with that annoying popup anymore.
1
2
3
4

@classmethod
def tearDownClass(cls):
cls.browser.quit()
super(RegistrationTests, cls).tearDownClass()

5
6

def setUp(self):

588

self.reg = RegisterPage(self.browser, "http://" +

127.0.0.1:9001)

And now finally the actual tests:

1
2
3
4
5
6
7

def test_registration(self):
self.reg.go_to()
self.reg.do_reg(name="somebodynew", email="[email protected]",
pwd="test", pwd2="test", cc="4242424242424242",
cvc="123", expiry_month="4", expiry_year="2020")
self.assertTrue(
self.browser.find_element_by_id("user_info").is_displayed())

Nothing special here we are just filling out the form, using our PageObject and checking to
make sure the user_info element is displayed (which should only be on the logged in members page).
1
2
3
4
5
6

def test_failed_registration(self):
self.reg.go_to()
self.reg.do_reg(name="somebodynew", email="[email protected]",
pwd="test", pwd2="test2", cc="4242424242424242",
cvc="123", expiry_month="4", expiry_year="2020")
self.assertIn("Passwords do not match", self.reg.error_msg)

And for our final test we fill out all the information (with passwords that dont match) and
check to see that we get the right error message. We could also come up with other combinations of this test to check the various error messages one might encounter.
One final note, even though test_failed_registration() comes later in the file then
test_registration(), it runs first. Why? by default the test runner will run the tests in
alphabetical order. This useful tidbit is helpful if you want your test to run in a certain order.
For example, if you run all your failed registrations before your successful registration, this
will improve the execution speed of the tests.
While this is the only exercise in this chapter, feel free to continue the testing and see if you
can come up with test cases for the major functionality pieces. It is good practice. Cheers!

589

Deploy
Exercise 2
Question: Didnt think you were going to get out of this chapter without writing
any code, did you? Remember back in the Fabric section when we talked about
creating another function to update all of your configuration files? Well, now is
the time to do that. Create a function called update_config() that automatically
updates your Nginx, Supervisor and Django config / setting files for you.
When the function is complete you should be able to execute:
1

$ fab integrate update_app update_config

Bonus points if you can get it all to work so the user only has to type:
1

$ fab ci

For the first part - e.g, the update_config() function - our function should look something
like this:
1
2
3

4
5

6
7

def update_config():
with cd("/opt/mec_env/mec_app/deploy"):
run("cp settings_prod.py
../django_ecommerce/django_ecommerce/")
run("cp supervisor/mec.conf /etc/supervisor/conf.d/")
run("cp nginx/sites-avaliable/mec
/etc/nginx/sites-available/")
run("/etc/init.d/supervisor restart")
run("/etc/init.d/nginx restart")

Not a lot new above. Basically all I did was start off in the deploy directory, copy the three
files to their necessary places (unix will automatically overwrite on copy), and then I restarted
both supervisor and nginx to make sure any configuration changes get updated. With that
you can now run
1

$ fab integrate update_app update_config

For the bonus section well there is really nothing to it. Keep in mind that fabric is just
python so you can structure it anyway you like. Which means you just need to create a function called ci that in turn calls the other three functions.
Below is a listing of the entire fabfile.py including the new ci function.

590

from fabric.api import env, cd, run, prefix, lcd, settings, local

2
3
4

env.hosts = ['<< server ip>>']

env.user = 'root'

5
6
7
8
9
10

def ci():
integrate()
update_app()
update_config()

11
12
13
14
15
16
17
18
19
20

def update_app():
with cd("/opt/mec_env/mec_app"):
run("git pull")
with cd("/opt/mec_env/mec_app/django_ecommerce"):
with prefix("source /opt/mec_env/bin/activate"):
run("pip install -r ../requirements.txt")
run("./manage.py migrate --noinput")
run("./manage.py collectstatic --noinput")

21
22
23
24
25

26
27

28
29

def update_config():
with cd("/opt/mec_env/mec_app/deploy"):
run("cp settings_prod.py
../django_ecommerce/django_ecommerce/")
run("cp supervisor/mec.conf /etc/supervisor/conf.d/")
run("cp nginx/sites-avaliable/mec
/etc/nginx/sites-available/")
run("/etc/init.d/supervisor restart")
run("/etc/init.d/nginx restart")

30
31
32
33
34
35

def integrate():
with lcd("../django_ecommerce/"):
local("pwd")
local("./manage.py test ../tests/unit")

36
37
38

with settings(warn_only=True):
local("git add -p && git commit")
591

39
40
41
42

local("git pull")
local("./manage.py test ../tests")
local("git push")

592