Gunicorn, Django

& WSGI
Benoît Chesneau
23/05/2010 - djangocon berlin
Tuesday, May 25, 2010
 benoît chesneau
Gunicorn author
[email protected]
Minimal web & Opensource
Web craftman
http://www.e-engura.com &
http://benoitc.im

Tuesday, May 25, 2010

 • WSGI ?
• Django & WSGI
• Gunicorn

Tuesday, May 25, 2010

 WSGI ?

• Web Server Gateway Interface

• PEP 333
• Web interface between the HTTP server
and Python applications.

Tuesday, May 25, 2010

 WEB Server

WSGI

Python app

Tuesday, May 25, 2010

 Serveur web

WSGI

Application Python

App App App

Tuesday, May 25, 2010

 simple application
def app(environ, start_response):
"""Simplest possible application object"""
data = 'Hello, World!\n'
status = '200 OK'
response_headers = [
('Content-type','text/plain'),
('Content-Length', str(len(data)))
]
start_response(status, response_headers)
return [data]

Tuesday, May 25, 2010

 WSGI Middleware

class CustomHeader(object):

def __init__(self, app):

self.app = app

def __call__(self, environ, start_response):

environ["HTTP_X_MY_HEADER"] = "1"
return self.app(environ, start_response)

Tuesday, May 25, 2010

 Chain WSGI aps

application = CustomHeader(app)

Tuesday, May 25, 2010

 Django & WSGI

Tuesday, May 25, 2010

 a way to deploy

Tuesday, May 25, 2010

 but we can discuss
with WSGI

• Why reinvent the wheel

• Common principles: middlewares,
request, ...
• intégrate != compatible wsgi

Tuesday, May 25, 2010

 • django-wsgi
• twod.wsgi

Tuesday, May 25, 2010

 twod.wsgi - embed

import os
from twod.wsgi import DjangoApplication

os.environ['DJANGO_SETTINGS_MODULE'] = "yourpackage.settings"
django_app = DjangoApplication()

Tuesday, May 25, 2010

 embed WSGI apps

from twod.wsgi import call_wsgi_app

from somewhere import wsgi_app

def run_app(request, path_info):

response = call_wsgi_app(wsgi_app, request, path_info)
response['Server'] = "twod.wsgi 1.0"
return response

Tuesday, May 25, 2010

 Deploy

• The real thing

• 2 kindsof deploiements :
• Modules to HTTP servers
• WSGI servers

Tuesday, May 25, 2010

 Modules

• uWSGI (NGINX)
• mod_wsgi (Apache)

Tuesday, May 25, 2010

 WSGI server

• spawning, paster, ...

• cherrypy, ..
• gunicorn

Tuesday, May 25, 2010

Tuesday, May 25, 2010
Tuesday, May 25, 2010
 Why

• Unicorn is awesome
• need something stable
• need something simple

Tuesday, May 25, 2010

 Philosophy

• Simple
• Minimal
• Performant
• Unix

Tuesday, May 25, 2010

 gunicorn
• Green unicorn
• WSGI 1.0
• Load balancing via pre-fork and a shared
socket
• async, sync worker (slow and fast clients)
• Easy integration with Paster compatible app
& ... Django

Tuesday, May 25, 2010

 More...

• HTTP Stream. Decode on the fly HTTP

chunks
• Upgrade “à la nginx”
• Eventlet, Gevent, Tornado
• DSL Config

Tuesday, May 25, 2010

 http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi

gunicorn is the winner in my eyes. It's easy to configure

and get up and running and certainly fast enough [..] .
Tuesday, May 25, 2010
 blogg.se - swedens largest blog provider
(stats for week 18 '10)
* 2.5M unique visitors (source: kiaindex.net)
* 800 000 new entries
* 850 000 new comments
* 16 000 new blogs
Tuesday, May 25, 2010
 DSL Config
bind = "127.0.0.1:8000"
daemon = True
debug = False
workers = 3

def when_ready(server):
....

Tuesday, May 25, 2010

 Simple command line

• gunicorn_django -w 3 -d -p /path/to/
pidfile /myproject/settings.py
• ./manage.py run_gunicorn -w 3
• gunicorn_django -w 3 -k
“egg:gunicorn#eventlet” /myproject/
settings.py

Tuesday, May 25, 2010

 Graceful reload

KILL -HUP `head -1 /path/to/pidfile`

Tuesday, May 25, 2010

 QUIT

kill -QUIT `cat /path/to/pidfile`

Tuesday, May 25, 2010

 Increase/ Decrease
number of workers

• KILL -TTIN PID

• KILL -TTOU PID

Tuesday, May 25, 2010

 use it behind nginx

• proxy
• buffering
• secure your app against DOS

Tuesday, May 25, 2010

 nginx.conf
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;

proxy_redirect off;

if (!-f $request_filename) {
proxy_pass http://app_server;
break;
}

Tuesday, May 25, 2010

 upstream app_server {
server 192.168.0.7:8080 fail_timeout=0;
server 192.168.0.8:8080 fail_timeou=0;
}

server {
listen 80 default;
client_max_body_size 4G;
server_name _;
keepalive_timeout 5;
root /path/to/app/current/public;

location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;

proxy_redirect off;

if (!-f $request_filename) {
proxy_pass http://app_server;
break;
}
}
}

Tuesday, May 25, 2010

 async - the trick

proxy_buffering off;

Tuesday, May 25, 2010

 Some tricks

• number of workers = 2xCPUs +1

• preload your project
• pre/before hooks
• when_ready hook

Tuesday, May 25, 2010

 Tune your os

• Increment fd limits : ulimit -n 2048

• Increment the connections queue
(somaxcon)
• Play with tcp windows
• More : http://gunicorn.org/tuning.html

Tuesday, May 25, 2010

 DEMO

Tuesday, May 25, 2010

 0.10

• HTTP parser (in C?)

• Increase unitests
• Reload hook
• status

Tuesday, May 25, 2010

 Liens

• http://gunicorn.org
• http://e-engura.org
• http://www.python.org/dev/peps/pep-0333/
• http://bitbucket.org/2degrees/twod.wsgi/
• http://github.com/alex/django-wsgi

Tuesday, May 25, 2010

 Questions

Tuesday, May 25, 2010

 @benoitc

Tuesday, May 25, 2010

 Cette création est mise à disposition selon le Contrat
Paternité 2.0 France disponible en ligne http://
creativecommons.org/licenses/by/2.0/fr/ ou par courrier
postal à Creative Commons, 171 Second Street, Suite
300, San Francisco, California 94105, USA.

Tuesday, May 25, 2010