Python

and
Google App Engine
Dan Sanderson
June 14, 2012

Thursday, June 14, 12

Thursday, June 14, 12

Thursday, June 14, 12

Google App Engine

Platform for building scalable web applications

Built on Google infrastructure
Pay for what you use

Thursday, June 14, 12

Apps, instance hours, storage, bandwidth, service calls

Free to start!

Launched with Python 2.5 exclusively in 2008; then

Java, Go, Python 2.7

Google App Engine

Easy development
Easy deployment
No servers to manage, no OS to update;
App Engine does this for you

Based on standard technologies:

Python 2.7, WSGI

Thursday, June 14, 12

Agenda
Development environment
Tools and IDEs
Unit testing
Remote access
(more more more!)
Thursday, June 14, 12

Demo

Thursday, June 14, 12

Development
Environment
Make your computer look like AE.

Thursday, June 14, 12

Development
Environment
Make your computer look like AE.

Thursday, June 14, 12

Development
Environment
Python 2.7
App Engine SDK (development server)
Optional packages
virtualenv and pip
Thursday, June 14, 12

Python 2.7
python.org
2.7.x, not 3.x
Interpreter, standard library
(App Engine sandbox)
Thursday, June 14, 12

Python 2.7
Windows
Python 2.7.3 Windows (X86-64) Installer
C:\Python27\
python -V
Thursday, June 14, 12

Python 2.7
Mac OS X
10.7 Lion: you already have it
python -V
Python 2.7.3 Mac OS X installers
Homebrew (mxcl.github.com/homebrew)
Thursday, June 14, 12

Python 2.7
Linux
You probably have it.
python -V
Ubuntu Precise: python 2.7.3
sudo apt-get install python
Thursday, June 14, 12

App Engine SDK

developers.google.com/appengine

Windows:

GoogleAppEngine-1.6.6.msi

Mac OS X:

GoogleAppEngineLauncher-1.6.6.dmg

Linux:

google_appengine_1.6.6.zip

Thursday, June 14, 12

App Engine SDK

Thursday, June 14, 12

App Engine SDK

dev_appserver.py appdir
appcfg.py command appdir
API libraries
SDK includes all dependencies
More tools! (webapp)
Not installed in site-packages
Thursday, June 14, 12

Optional Packages
app.yaml:
application: myapp
version: 1
runtime: python27
api_version: 1
threadsafe: true
# ...
libraries:
- name: django
version: 1.3
- name: jinja2
version: 2.6
- name: markupsafe
version: 0.15

Thursday, June 14, 12

Optional Packages

Thursday, June 14, 12

Django
Jinja2, markupsafe
lxml
numpy
PIL
pycrypto
webapp2*
webob
yaml

Special access to fast

implementations on
App Engine
Versioned access
Not in the SDK;
install them locally,
as needed

Optional Packages

Thursday, June 14, 12

AE live environment:
AE modules + sandbox + AEs Python +
requested libraries

Development server environment:

AE modules + simulated sandbox +
your Python environment

setup.py, easy_install, pip add to

site-packages; one big pile

Do I have libs installed that AE doesnt have?

What if different apps need different versions?

virtualenv
Create virtual Python environments, each
with its own set of packages

Imports use the active environment

Versions isolated, dependencies controlled

Thursday, June 14, 12

virtualenv
virtualenv (once):
Install
sudo easy_install virtualenv

Create an environment for your App

Engine project:

virtualenv myapp_env

the environment:
Activate
source ./myapp_env/bin/activate

Thursday, June 14, 12

pip
Python package installation manager
Like easy_install but better
Python Package Index (PyPI):
pypi.python.org

Works well with virtualenv (no sudo!)

Thursday, June 14, 12

pip
pip (once):
Install
sudo easy_install pip
install a package from PyPI:
To
pip install jinja2
install a specific version:
To
pip install jinja2==2.6

Thursday, June 14, 12

pip
file:
Requirements
pip install -r requirements.txt

File is just a list of package names, with

optional versions

Thursday, June 14, 12

app.yaml:

requirements.txt:

libraries:
- name: django
version: 1.3
- name: jinja2
version: 2.6
- name: markupsafe
version: 0.15

django==1.3
jinja2==2.6
markupsafe==0.15

virtualenv myapp_env
source ./myapp_env/bin/activate
pip install -r requirements.txt

Thursday, June 14, 12

django==1.3
jinja2==2.6
lxml==2.3
markupsafe==0.15
numpy==1.6.1
pycrypto==2.3
PyYAML==3.10
webapp2==2.5.1
webob==1.1.1
PIL==1.1.7

Thursday, June 14, 12

Thursday, June 14, 12

Tools and IDEs

Thursday, June 14, 12

Tools and IDEs

dev_appserver.py, dev server console
appcfg.py
Google App Engine Launcher
JetBrains PyCharm
Aptana PyDev
Thursday, June 14, 12

dev_appserver.py

Thursday, June 14, 12

dev_appserver.py appdir
Runs a local web server
Watches your files for changes, re-imports modules
Simulates runtime environment and services locally

Datastore, Blobstore, Memcache; MySQL

Mail, XMPP
Task queues

Automatic datastore index generation

Development Server
Console
localhost:8080/_ah/admin
Web UI to inspect and manipulate the
development environment

datastore, memcache; task queues;

incoming mail, XMPP

Python console: run Python code from the

browser

Thursday, June 14, 12

appcfg.py

Thursday, June 14, 12

appcfg.py command appdir

Download app code (download_app)

Download logs (request_logs)

Deploy your app (update)

Manage datastore indexes (update_indexes,
vacuum_indexes)
Import/export datastore data (upload_data,
download_data, create_bulkloader_config)

appcfg.py
Use --oauth2
Use two-factor authentication
For tools that dont support OAuth

deployment, use app-specific passwords:

google.com/settings

Thursday, June 14, 12

Google App Engine

Launcher

Thursday, June 14, 12

JetBrains PyCharm

Thursday, June 14, 12

jetbrains.com/pycharm

code completion; interactive debugger; unit testing with

coverage reports; doc strings and doc access; auto
imports; refactoring

AE project template; AE lib-aware; run dev server in

debugger; deploy to App Engine, fetch logs

Special Django, HTML, CSS, JavaScript support

$99 personal, $199 commercial;

free Open Source Project license;
30-day free trial

JetBrains PyCharm

Thursday, June 14, 12

JetBrains PyCharm

Thursday, June 14, 12

JetBrains PyCharm

Thursday, June 14, 12

Aptana PyDev

Thursday, June 14, 12

Aptana Studio 3 with PyDev: aptana.com

AE project set-up wizard (no starter files though); AE

lib-aware; run dev server in debugger

Special JavaScript, HTML, CSS support

Eclipse + PyDev plugin: pydev.org

Free!
code completion; interactive debugger; doc strings
and doc access; unit testing; auto imports; refactoring

Aptana PyDev

Thursday, June 14, 12

Thursday, June 14, 12

Thursday, June 14, 12

Thursday, June 14, 12

Thursday, June 14, 12

Unit Testing

Thursday, June 14, 12

Unit Testing
Its just Python, right?
Activate your virtualenv
Make sure google_appengine and your apps
root dir are in PYTHONPATH

(Use a test runner script, or a test tool:

nose-gae, gaeunit)

Thursday, June 14, 12

Unit Testing
% source ./myapp_env/bin/activate
(myapp_env)% export PYTHONPATH=$PYTHONPATH:/
Users/dan/google_appengine
(myapp_env)% python
>>> from google.appengine.ext import db
>>> import models
>>> m = models.Entry()
>>> m.title = Test
>>> m.put()
# (in theory)
>>>

Thursday, June 14, 12

Unit Testing
>>> from google.appengine.api import mail
>>> mail.send_mail([email protected]', [email protected]',
subj, body)
...
AssertionError: No api proxy found for service
mail
>>>

Thursday, June 14, 12

testbed

Thursday, June 14, 12

Service API stubs for a clean test environment

Provided by the App Engine SDK

google.appengine.ext.testbed

Mail and XMPP: records outgoing messages

Datastore: can fuzz test eventual consistency
Users: can set signed-in status, address,
is-admin status

testbed
>>>
>>>
>>>
>>>

from google.appengine.ext import testbed

tb = testbed.Testbed()
tb.activate()
tb.init_datastore_v3_stub()

>>> from google.appengine.ext import db

>>> import models
>>> m = models.Entry()
>>> m.title = Test
>>> m.put()
# (success!)
datastore_types.Key.from_path(uEntry, 1L,
_app=utestbed-test)

Thursday, June 14, 12

testbed
>>> tb.activate()
>>> tb.init_mail_stub()
>>> from google.appengine.api import mail
>>> mail.send_mail([email protected]', [email protected]',
subj, body)
# (success!)
>>> tb.get_stub(testbed.MAIL_SERVICE_NAME)
.get_sent_messages([email protected]')
[<google.appengine.api.mail.EmailMessage
object at 0x10b5115d0>]
>>>

Thursday, June 14, 12

testbed
import unittest
from google.appengine.ext import testbed
import models
class EntryTestBase(unittest.TestCase):
def setUp(self):
self.tb = testbed.Testbed()
self.tb.activate()
self.tb.init_datastore_v3_stub()
def tearDown(self):
self.tb.deactivate()

Thursday, June 14, 12

testbed
class EntryTestBase(unittest.TestCase):
# ...
def testEntry(self):
m = models.Entry()
m.title = Test
m.put()
self.assertIsNotNone(m.create_date)
if __name__ == __main__:
unittest.main()

Thursday, June 14, 12

testbed
tb.init_all_stubs()
tb.init_blobstore_stub()
tb.init_datastore_v3_stub()
tb.init_memcache_stub()
tb.init_images_stub()
tb.init_mail_stub()
tb.init_taskqueue_stub()
tb.init_urlfetch_stub()
tb.init_user_stub()
tb.init_xmpp_stub()

developers.google.com/appengine/docs/python/tools/
localunittesting
Thursday, June 14, 12

Unit Testing
>>> import webob
>>> import blog
>>> h = blog.FrontPageHandler()
>>> h.request = webob.Request.blank(/)
>>> h.get()
# (assuming testbed is set up as needed)
>>> h.response.status
200 OK
>>>

Thursday, June 14, 12

WebTest
Test helper for WSGI applications, wraps

any WSGI app instance in a webob-based

interface

Third-party tool, not included

install WebTest (in your virtualenv):
To
pip install WebTest
Thursday, June 14, 12

WebTest
import unittest
import webtest
import blog
class BlogTest(unittest.TestCase):
def setUp(self):
# (testbed init goes here...)
self.testapp = webtest.TestApp(
blob.application)

Thursday, June 14, 12

WebTest
class BlogTest(unittest.TestCase):
# ...
def testFrontPage(self):
response = self.testapp.get(/)
self.assertEqual(200, response.status_int)
# ...

webtest.pythonpaste.org
www.webob.org

Thursday, June 14, 12

Thursday, June 14, 12

Remote Access

Thursday, June 14, 12

Remote Access
Service API stubs: live site, development
server, testbed

Remote access: configure API stubs locally

to access the live services for the app

Uses a proxy in the app itself

Thursday, June 14, 12

Remote Access
Activate the remote_api builtin
(/_ah/remote_api) in app.yaml:
builtins:
- remote_api: on

remote_api_shell.py appid
(Does not support OAuth, must use appspecific password: google.com/settings)

Remote API calls consume app resources!

Thursday, June 14, 12

Remote Access
% cd ae-book
% remote_api_shell.py ae-book
Email: ...
Password: ...
App Engine remote_api shell
Python 2.7.1 ...
s~ae-book> import models
s~ae-book> m = models.Entry.all().fetch(3)
s~ae-book> m[2].title
uCode Samples Are Coming

Thursday, June 14, 12

Remote Access
#!/usr/bin/python
from google.appengine.ext.remote_api import \
remote_api_stub
def auth_func():
# prompt for email address and password
return email_address, password
remote_api_stub.ConfigureRemoteApi(
appid, /_ah/remote_api, auth_func)
remote_api_stub.MaybeInvokeAuthentication()

Thursday, June 14, 12

Review

Development environment

Python, AE SDK, dev server

Optional libs, virtualenv, pip;
reproducable environments with requirements files

Tools and IDEs

Thursday, June 14, 12

Launcher, PyCharm, Aptana PyDev

Using IDEs with virtualenv
Deployment with appcfg.py

Review

Unit testing

WebTest
PyCharm code coverage

Remote access

Thursday, June 14, 12

testbed

Remote shell
Remote access from a script

developers.google.com/
appengine
appengine.google.com
ae-book.appspot.com
Programming Google App
Engine, 2nd ed.
Summer 2012
Dan Sanderson
profiles.google.com/
dan.sanderson

Thursday, June 14, 12