Python For Programmers - A Project-Based Tutorial
Python For Programmers - A Project-Based Tutorial
Introduction to Workshop
Alexandra (Sandy) Strong (@sandymahalo): Systems Engineer at Dreamhost, Python Education, PyLadies
http://bit.ly/YXKWic
I recommend downloading and saving the slides as a PDF now!
Why Python?
Simple to get started, easy for beginners, powerful enough for professionals Code is clean, modular, and elegant, making it easy to read and edit Whitespace enforcement Extensive standard library, many modules to import from Cross platform - Windows, Mac, Linux Supportive, large, and helpful community
Why a project?
Getting your hands dirty is the best way to develop a new skill! Your efforts in this tutorial will produce a meaningful project, and give you something to show for your work.
Questions so far?
Jinja2
Commonly used python templating language Easy to grok syntax Integrates with many python libraries
Raise your hand if you were not able to install memcached and/or pylibmc successfully!
IPython
? operator %who and %whos %hist %pdb More details: http://pages.physics.cornell.
edu/~myers/teaching/ComputationalMethods/python/ipy thon.html
Strings
In [1]: my_name = "Sandy Strong" In [2]: print my_name Sandy Strong Now type, "my_name." and hit the tab key: In [3]: my_name. my_name.capitalize my_name.center my_name.count my_name.decode my_name.encode <snip>
upper
makes all characters upper case
lower
makes all characters lower case
split
splits string into a list, whitespace delimited
find
search for a string within your string
startswith
test for what your string starts with
String formatting
You can pass variables into strings to format them in a specific way, for example:
In [14]: age = 28 In [15]: name = 'Sandy' In [16]: print "Hello, my name is %s." % name Hello, my name is Sandy. In [17]: print "Hello, my name is %s, and I'm %s years old." % (name, age) Hello, my name is Sandy, and I'm 28 years old.
Lists
[2]: items = ['bacon', 3.14, ['bread', 'milk'] ] In [3]: print items ['bacon', 3.14, ['bread', 'milk']]
You can put lists (and other Python data types) inside of lists.
append
append an item to the end of your list
pop
provide index position to "pop" an item out of your list
Tuples
In [12]: colors = ('red', 'blue', 'green') In [13]: print colors ('red', 'blue', 'green')
Tuples are immutable. Once they're created, they cannot be changed. Because they are immutable, they are hashable.
Dictionaries
In [1]: favorite_sports = {'John': 'Football', 'Sally': 'Soccer'} In [2]: print favorite_sports {'John': 'Football', 'Sally': 'Soccer'}
The first parameter is the "key" and the second parameter is the "value". A dictionary can have as many key/value pairs as you wish, and keys are immutable.
values
all values in your dictionary
keys
all keys in your dictionary
items
list of 2-element tuples that correspond to the key/value pairs in your dictionary
update
add a new key/value pair to your dictionary
Comparison operators
Equal to: == Less than: < Greater than: > Less than or equal to: <= Greater than or equal to: >=
if and else
In [70]: age = 20 In [71]: if age <= 20: ....: print "You're old enough!" ....: You're old enough!
In [72]: if age > 20: ....: print "You're too old!" ....: else: ....: print "You're the right age." ....: You're the right age.
Using elif
The elif keyword allows you to expand your control flow block, and perform additional comparisons within an if-else statement.
In [74]: if age < 20: ....: print "You're too young." ....: elif age >= 20: ....: print "You're the right age!" ....: You're the right age!
We will only add the pet to pet_list if the user is not Bob.
CherryPy Setup
You should already have all libraries installed and working on your computer. We will be creating a Poll web application. You will be able to create and edit polls, add and edit choices, and vote on the poll. The first step is to create a directory on your computer named my-cherrypy-poll: mkdir my-cherrypy-poll
If you do not have git installed, please review the instructions from our tutorial wiki:
https://us.pycon.org/2013/community/tutorials/5/
db_tools.py (continued)
def set_value(key,value): ''' Set a given key/value pair in mc ''' mc = get_mc() mc.set(key,value) del mc return True def del_value(key): ''' Delete a key/value pair from mc ''' mc = get_mc() mc.delete(key) del mc return True
We leveraged methods from pylibmc to build our ORM. This library allows Python to easily talk to memcache.
Questions so far?
_punct_re = re.compile(r'[\t !"#$%&\'()*\-/<=>?@\[\\\]^_`{|},.]+') def slugify(text, delim=u'-'): """Generates an ASCII-only slug.""" result = [] for word in _punct_re.split(text.lower()): word = normalize('NFKD', word).encode('ascii', 'ignore') if word: result.append(word) return str(delim.join(result))
What is "slugify"?
Suppose a user created a poll named:
"My favorite food is oranges, what is yours? "
Why? Prettier URLs, more SEO-friendly, handy unique identifier for the record in the datastore.
Normalizing unicode
Unicode is great-- it represents a much larger character set than ASCII. However, it can be challenging to work with. For the purpose of storing slugs in our datastore, it's simpler if we normalize our slug string to only ASCII characters. This code normalizes the string, replacing unicode characters with their ASCII equivalent, and then encodes the string in ASCII format, ignoring any errors raised during the encoding process:
normalize('NFKD', word).encode('ascii', 'ignore')
For a production web application, you would want to use a more robust method for slugifying your Poll names.
models.py
The models.py file in a Python application is often used to define class types and create and inherit model instances. In our application, we are essentially using the theory of this to help manipulate data within our memcache datastore. We will use our models to edit and create "poll" instances which we update in our memcache storage.
def add_poll(**kwargs): choices_arr = [] count = 1 poll_dict = {} poll_dict['question'] = kwargs.get('question') for k,v in kwargs.items(): if 'choice' not in k: continue choice_dict = { 'id': count, 'text': v, 'value': 0 } choices_arr.append(choice_dict) count += 1 slug = slugify(kwargs.get('question')) poll_dict['slug'] = slug poll_dict['choices'] = choices_arr set_value(slug,poll_dict) if kwargs.get('publish'): publish_poll(slug)
We grab the poll from kwargs, and then if choices are present, we build out a choice_dict of the choices associated with that poll. The poll question gets slugified, and added to the poll_dict, and then a list containing the choices dictionary is added. to poll_dict. We call set_value and pass in the slug (unique key for the poll), and the contents of poll_dict. Finally, if kwargs contains a key named "publish" we publish the poll.
def edit_poll(**kwargs): choices_arr = [] poll = get_poll(str(kwargs.get('slug'))) poll_dict = {} poll_dict['question'] = kwargs.get('question') for k,v in kwargs.items(): if 'choice' not in k: continue this_choice = [ c for c in poll.get('choices') if int(k.strip('choice')) == c.get('id') ] if not len(this_choice): return False else: this_choice = this_choice[0] choice_dict = { 'id': this_choice.get('id'), 'text': v, 'value': this_choice.get('value'), } choices_arr.append(choice_dict)
edit_poll (continued)
slug = str(kwargs.get('slug')) poll_dict['slug'] = slug poll_dict['choices'] = choices_arr set_value(slug,poll_dict) if kwargs.get('publish'): publish_poll(slug) else: unpublish_poll(slug) return poll_dict
The edit_poll function and add_poll functions have some similarities. They start to differ with the list comprehension used to assign this_choice. We build out this_choice using a conditional to validate that the id for the choice passed in is equivalent to the choice id on the poll object from memcache. From there, we build out the choice_dict , get slug from **kwargs , add slug and choices to the poll_dict , and set the value in memcache.
This is the view that allows visitors to vote on a poll. When executed, it accepts the poll_key (its unique identifier), and the choice that the user selected. The poll is retrieved from memcached, validated, and it's vote count is incremented in memcache.
views.py (continued)
def get_global_links(): return [{ 'name': 'Add a poll', 'url': '/polls/add', }, { 'name': 'Home', 'url': '/' }, { 'name': 'Poll list', 'url': '/polls', }]
This is essentially our site navigation. We return a list of dictionaries, each one containing a pretty name for the view, along with the url path where the view is located within our application. The view to "Add a poll" is mapped to the "/polls/add" url.
views.py (continued)
def get_poll(key): poll = get_value(key) return poll
Retrieve a specific poll from memcache, based on the key-- the slug-- for the poll.
views.py (continued)
def get_polls(): poll_list = [] published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) for p in published_polls: poll = get_value(p) poll_list.append(poll) return poll_list
Return the value from memcache with the key 'published_polls', loop through each poll id (slug), and retrieve the poll object for that id. Build a list of poll objects, and return them.
views.py (continued)
def publish_poll(key): published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) if key not in published_polls: published_polls.append(key) set_value('published_polls',published_polls)
For a given poll id (key), add it to the list of published_polls in memcache if it is not already there. Notice how we're using if-statements to perform validation on published_polls and the individual key we're attempting to add to the published_polls list.
views.py (continued)
def unpublish_polls(key): published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) if key in published_polls: published_polls.remove(key) set_value('published_polls',published_polls)
For a given poll id (key), remove it from list of published_polls in memcache, if it is currently published. Note again how we perform validation.
So, who has questions? ...I know someone must have a question! ... c'mon...
env tells our app where to find templates at, and conf tells our app where to find the app.conf file.
You will observe these same patterns in other methods within the PollViews class.
This lets us add a poll to our datastore. If method is POST, we pass in **kwargs to add_poll, then we render the contents of our data_dict out to the add_poll.html template.
Congratulations! You've completed writing your Python code for this app!
App Homework!
Raise an error if a user enters bad data into the form Use the unpublish_polls function in the application to properly unpublish polls Use CherryPy Sessions to validate that each user can only vote once http://docs.cherrypy.
org/dev/refman/lib/sessions.html
Allow users to add more than 4 choices to a poll using a more dynamically generated form
add_poll.html
<html> <head> </head> <body> <h1> {{title}}</h1> <h2>Add a poll</h2> <form action= "/polls/add" method= 'POST'> Question: <input type="text" name="question" value=""><br> Choice 1: <input type="text" name="choice1" value=""><br> Choice 2: <input type="text" name="choice2" value=""><br> Choice 3: <input type="text" name="choice3" value=""><br> Choice 4: <input type="text" name="choice4" value=""><br> Publish? <input type="checkbox" name="publish" value="checked"><br> <input type="submit" value="Add poll"> </form> {% for l in links %} <span><a href=" {{l.url}}">{{l.name}}</a></span> {% endfor %}
edit_poll.html
<html> <head></head> <body> <h1> {{title}}</h1> <h2>Edit your poll</h2> <form action=" /polls/edit/{{poll.slug}}" method= 'POST'> Question: <input type="text" name="question" value=" {{poll.question}}" ><br> {% for c in poll.choices %} Choice {{c.id}}: <input type="text" name="choice {{c.id}}" value=" {{c. text}}"><br> {% endfor %} Publish? <input type="checkbox" name="publish" checked=" {% if poll. published %}checked {% endif %}"><br> <input type="hidden" name="slug" value=" {{poll.slug}}"> <input type="submit" value="Edit poll"> </form> {% for l in links %} <span><a href=" {{l.url}}">{{l.name}}</a></span> {% endfor %} </body> </html>
index.html
<html> <head> </head> <body> <h1> {{title}}</h1> <p>Welcome to the polls app. Feel free to add a poll or answer some questions! :)</p> {% for l in links %} <span><a href=" {{l.url}}">{{l.name}}</a></span> {% endfor %} </body> </html>
poll.html
<html> <head></head> <body> <h1> {{title}}</h1> <h2>Poll: {{poll.question}}</h2> {% if success %} <p>Thanks for voting!</p> <h3>Results</h3> {% for c in poll.choices %} <p> {{c.text}}: {{c.value}}</p> {% endfor %} {% else %} <form action=" /polls/poll/{{poll.slug}}" method= 'POST'> {% for c in poll.choices %} <input type="radio" name="choice" value=" {{c.id}}">{{c.text}}<br> {% endfor %} <input type="submit" value="Vote"> </form> {% endif %} {% for l in links %} <span><a href=" {{l.url}}">{{l.name}}</a></span> | {% endfor %} </body></html>
polls.html
<html> <head></head> <body> <h1>{{title}}</h1> <h2>Polls</h2> <ul> {% for p in polls %} <li><a href="/polls/poll/{{p.slug}}">{{p.question}} </a></li> {% endfor %} </ul> {% for l in links %} <span><a href="{{l.url}}">{{l.name}}</a></span> {% endfor %} </body> </html>
Testing
(/me facepalms)
Writing Tests
Unit Tests Mocking Request/Response Mocking Objects
Unfortunately...
... we don't have time to go into testing in-depth during this tutorial. We want you guys to come away with: Understanding of why writing tests is so important A basic understanding of how tests might be written Resources that inspire you to write your own tests!
class BlogLogin(object): def __init__(self, login, password): self._blogin = Blog(login, password) def my_subscriptions(self): return self._subs(self._blogin.subs(self._blogin.login) def _subs(self, call): crs = -1 results = [] while crs != 0: page = call(crs) subscriptions = page['subscriptions'] crs = page['next_cursor'] results.extend(subscriptions) return results
Testing homework:
Your app should be well-tested! After PyCon, go home and write a test suite using unittest and mock, you can find great documentation here:
Python unittest Documentation Mock Library Documentation
Errors
(FML)
Exceptions:
Python has built-in exceptions that can be caught or raised Alternatively, you can write your own exception classes specific to your application
Catching exceptions
try: # Try to get "my-first-poll" from memcache return get_value('my-first-poll') except KeyError: # If the key is not found in memcache, # return an empty dictionary instead return {} Catching exceptions is useful when you can anticipate what exception might occur, and have the application do something else instead of raising the exception. In this case, if our key is not found, we want our app to return an empty dictionary.
Raising exceptions
If you do not attempt to catch an exception, then Python will raise it automatically. There are many cases where you can catch an exception and mitigate the negative impact for users. However, there are some exceptions that you can never programmatically predict-- and those bubble up into your application error log.
Exception homework:
Go through your completed my-cherrypy-poll app and find places that are missing exceptions. Put in your own try/catch blocks, and then try writing your own custom exception classes:
Python Built-In Exceptions Creating User-Defined Exception Classes
You can find the full pdb documentation here. Have fun, and happy bug hunting!
Conclusions
(what did we learn?)
Whew!
Today we gave you a whirlwind tour of basic Python, and we're hoping it's enough to get you guys hooked! We covered data types, loops, control flow statements, conditionals, functions, classes, testing, error handling, debugging, and more!
IRC:
Freenode #python channel
Cool stuff:
Github
Questions?
Thank you so much for attending our tutorial, we hope you enjoyed it!