APythonBook

APythonBook:BeginningPython,Advanced
Python,andPythonExercises
Author:
DaveKuhlman
Contact:
[email protected]
Address:
http://www.davekuhlman.org

Page1

APythonBook

Revision
1.3a
Date
December15,2013
Copyright
Copyright(c)2009DaveKuhlman.AllRightsReserved.Thisdocumentissubject
totheprovisionsoftheOpenSourceMITLicense
http://www.opensource.org/licenses/mitlicense.php.
Abstract
ThisdocumentisaselflearningdocumentforacourseinPythonprogramming.
Thiscoursecontains(1)apartforbeginners,(2)adiscussionofseveraladvanced
topicsthatareofinteresttoPythonprogrammers,and(3)aPythonworkbookwith
lotsofexercises.

Page2

APythonBook

Contents
1Part1BeginningPython...........................................................................................10
1.1IntroductionsEtc...................................................................................................10
1.1.1Resources.......................................................................................................11
1.1.2AgeneraldescriptionofPython....................................................................12
1.1.3InteractivePython..........................................................................................15
1.2Lexicalmatters......................................................................................................15
1.2.1Lines..............................................................................................................15
1.2.2Comments......................................................................................................16
1.2.3Namesandtokens..........................................................................................16
1.2.4Blocksandindentation..................................................................................16
1.2.5Docstrings.....................................................................................................17
1.2.6Programstructure..........................................................................................17
1.2.7Operators.......................................................................................................18
1.2.8Alsosee.........................................................................................................19
1.2.9Codeevaluation.............................................................................................19
1.3Statementsandinspectionpreliminaries...........................................................20
1.4Builtindatatypes.................................................................................................21
1.4.1Numerictypes................................................................................................21
1.4.2Tuplesandlists..............................................................................................21
1.4.3Strings............................................................................................................24
1.4.3.1Thenewstring.formatmethod...............................................................26
1.4.3.2Unicodestrings......................................................................................27
1.4.4Dictionaries....................................................................................................29
1.4.5Files...............................................................................................................32
1.4.6Otherbuiltintypes........................................................................................35
1.4.6.1TheNonevalue/type..............................................................................35
1.4.6.2Booleanvalues.......................................................................................36
1.4.6.3Setsandfrozensets.................................................................................36
1.5FunctionsandClassesAPreview......................................................................36
1.6Statements.............................................................................................................37
1.6.1Assignmentstatement....................................................................................37
1.6.2importstatement............................................................................................39
1.6.3printstatement...............................................................................................41
1.6.4if:elif:else:statement...................................................................................43
1.6.5for:statement.................................................................................................44
1.6.6while:statement.............................................................................................48
Page3

APythonBook
1.6.7continueandbreakstatements.......................................................................48
1.6.8try:except:statement.....................................................................................49
1.6.9raisestatement...............................................................................................51
1.6.10with:statement.............................................................................................52
1.6.10.1Writingacontextmanager...................................................................52
1.6.10.2Usingthewith:statement....................................................................53
1.6.11del................................................................................................................54
1.6.12casestatement..............................................................................................55
1.7Functions,Modules,Packages,andDebugging....................................................55
1.7.1Functions.......................................................................................................55
1.7.1.1Thedefstatement...................................................................................55
1.7.1.2Returningvalues....................................................................................55
1.7.1.3Parameters..............................................................................................56
1.7.1.4Arguments..............................................................................................56
1.7.1.5Localvariables.......................................................................................57
1.7.1.6Otherthingstoknowaboutfunctions....................................................57
1.7.1.7Globalvariablesandtheglobalstatement.............................................58
1.7.1.8Docstringsforfunctions.......................................................................60
1.7.1.9Decoratorsforfunctions........................................................................60
1.7.2lambda...........................................................................................................61
1.7.3Iteratorsandgenerators.................................................................................62
1.7.4Modules.........................................................................................................67
1.7.4.1Docstringsformodules.........................................................................68
1.7.5Packages........................................................................................................68
1.8Classes...................................................................................................................69
1.8.1Asimpleclass................................................................................................69
1.8.2Definingmethods..........................................................................................70
1.8.3Theconstructor..............................................................................................70
1.8.4Membervariables..........................................................................................70
1.8.5Callingmethods.............................................................................................71
1.8.6Addinginheritance........................................................................................71
1.8.7Classvariables...............................................................................................72
1.8.8Classmethodsandstaticmethods.................................................................72
1.8.9Properties.......................................................................................................74
1.8.10Interfaces.....................................................................................................75
1.8.11Newstyleclasses.........................................................................................75
1.8.12Docstringsforclasses.................................................................................77
1.8.13Privatemembers..........................................................................................77
1.9SpecialTasks.........................................................................................................77
1.9.1Debuggingtools.............................................................................................77
Page4

APythonBook
1.9.2Fileinputandoutput......................................................................................78
1.9.3Unittests........................................................................................................80
1.9.3.1Asimpleexample..................................................................................80
1.9.3.2Unittestsuites........................................................................................81
1.9.3.3Additionalunittestfeatures....................................................................83
1.9.3.4GuidanceonUnitTesting......................................................................85
1.9.4doctest............................................................................................................85
1.9.5ThePythondatabaseAPI..............................................................................87
1.9.6InstallingPythonpackages............................................................................88
1.10MorePythonFeaturesandExercises..................................................................89
2Part2AdvancedPython............................................................................................90
2.1IntroductionPython201(Slightly)AdvancedPythonTopics.......................90
2.2RegularExpressions..............................................................................................90
2.2.1Definingregularexpressions.........................................................................90
2.2.2Compilingregularexpressions......................................................................91
2.2.3Usingregularexpressions..............................................................................91
2.2.4Usingmatchobjectstoextractavalue..........................................................92
2.2.5Extractingmultipleitems..............................................................................93
2.2.6Replacingmultipleitems...............................................................................94
2.3IteratorObjects......................................................................................................96
2.3.1ExampleAgeneratorfunction....................................................................98
2.3.2ExampleAclasscontainingageneratormethod......................................100
2.3.3ExampleAniteratorclass.........................................................................102
2.3.4ExampleAniteratorclassthatusesyield.................................................104
2.3.5ExampleAlistcomprehension.................................................................105
2.3.6ExampleAgeneratorexpression..............................................................105
2.4UnitTests............................................................................................................106
2.4.1Definingunittests........................................................................................106
2.4.1.1Createatestclass.................................................................................106
2.5ExtendingandembeddingPython......................................................................109
2.5.1Introductionandconcepts............................................................................109
2.5.2Extensionmodules.......................................................................................110
2.5.3SWIG...........................................................................................................112
2.5.4Pyrex............................................................................................................115
2.5.5SWIGvs.Pyrex...........................................................................................120
2.5.6Cython.........................................................................................................120
2.5.7Extensiontypes............................................................................................122
2.5.8Extensionclasses.........................................................................................122
2.6Parsing.................................................................................................................122
2.6.1Specialpurposeparsers...............................................................................123
Page5

APythonBook
2.6.2Writingarecursivedescentparserbyhand.................................................124
2.6.3Creatingalexer/tokenizerwithPlex...........................................................131
2.6.4Asurveyofexistingtools............................................................................141
2.6.5CreatingaparserwithPLY.........................................................................141
2.6.6Creatingaparserwithpyparsing.................................................................148
2.6.6.1Parsingcommadelimitedlines............................................................148
2.6.6.2Parsingfunctors...................................................................................149
2.6.6.3Parsingnames,phonenumbers,etc.....................................................150
2.6.6.4Amorecomplexexample....................................................................151
2.7GUIApplications................................................................................................153
2.7.1Introduction.................................................................................................153
2.7.2PyGtk...........................................................................................................153
2.7.2.1Asimplemessagedialogbox..............................................................153
2.7.2.2Asimpletextinputdialogbox.............................................................156
2.7.2.3Afileselectiondialogbox...................................................................158
2.7.3EasyGUI......................................................................................................160
2.7.3.1AsimpleEasyGUIexample................................................................161
2.7.3.2AnEasyGUIfileopendialogexample................................................161
2.8GuidanceonPackagesandModules...................................................................161
2.8.1Introduction.................................................................................................161
2.8.2ImplementingPackages...............................................................................162
2.8.3UsingPackages............................................................................................162
2.8.4DistributingandInstallingPackages...........................................................162
2.9EndMatter...........................................................................................................164
2.9.1AcknowledgementsandThanks..................................................................164
2.9.2SeeAlso.......................................................................................................164
3Part3PythonWorkbook.........................................................................................165
3.1Introduction.........................................................................................................165
3.2LexicalStructures................................................................................................165
3.2.1Variablesandnames....................................................................................165
3.2.2Linestructure...............................................................................................167
3.2.3Indentationandprogramstructure...............................................................168
3.3ExecutionModel.................................................................................................169
3.4BuiltinDataTypes.............................................................................................170
3.4.1Numbers......................................................................................................170
3.4.1.1Literalrepresentationsofnumbers......................................................171
3.4.1.2Operatorsfornumbers.........................................................................173
3.4.1.3Methodsonnumbers............................................................................175
3.4.2Lists.............................................................................................................175
3.4.2.1Literalrepresentationoflists...............................................................176
Page6

APythonBook
3.4.2.2Operatorsonlists.................................................................................178
3.4.2.3Methodsonlists...................................................................................178
3.4.2.4Listcomprehensions............................................................................180
3.4.3Strings..........................................................................................................182
3.4.3.1Characters............................................................................................183
3.4.3.2Operatorsonstrings.............................................................................184
3.4.3.3Methodsonstrings...............................................................................185
3.4.3.4Rawstrings..........................................................................................187
3.4.3.5Unicodestrings....................................................................................188
3.4.4Dictionaries..................................................................................................190
3.4.4.1Literalrepresentationofdictionaries...................................................190
3.4.4.2Operatorsondictionaries.....................................................................191
3.4.4.3Methodsondictionaries.......................................................................192
3.4.5Files.............................................................................................................195
3.4.6Afewmiscellaneousdatatypes..................................................................197
3.4.6.1None.....................................................................................................197
3.4.6.2ThebooleansTrueandFalse...............................................................197
3.5Statements...........................................................................................................198
3.5.1Assignmentstatement..................................................................................198
3.5.2printstatement.............................................................................................200
3.5.3if:statementexercises..................................................................................201
3.5.4for:statementexercises...............................................................................202
3.5.5while:statementexercises...........................................................................205
3.5.6breakandcontinuestatements.....................................................................206
3.5.7Exceptionsandthetry:except:andraisestatements...................................207
3.6Functions.............................................................................................................210
3.6.1Optionalargumentsanddefaultvalues.......................................................211
3.6.2Passingfunctionsasarguments...................................................................213
3.6.3Extraargsandkeywordargs.......................................................................214
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)..................216
3.6.4Functionsandducktypingandpolymorphism...........................................216
3.6.5Recursivefunctions.....................................................................................217
3.6.6Generatorsanditerators...............................................................................219
3.7Objectorientedprogrammingandclasses..........................................................223
3.7.1Theconstructor............................................................................................224
3.7.2InheritanceImplementingasubclass.......................................................225
3.7.3Classesandpolymorphism..........................................................................227
3.7.4Recursivecallstomethods..........................................................................228
3.7.5Classvariables,classmethods,andstaticmethods.....................................230
3.7.5.1Decoratorsforclassmethodandstaticmethod.....................................233
Page7

APythonBook
3.8AdditionalandAdvancedTopics........................................................................234
3.8.1Decoratorsandhowtoimplementthem......................................................234
3.8.1.1Decoratorswitharguments..................................................................235
3.8.1.2Stackeddecorators...............................................................................236
3.8.1.3Morehelpwithdecorators...................................................................238
3.8.2Iterables.......................................................................................................239
3.8.2.1AfewpreliminariesonIterables..........................................................239
3.8.2.2Morehelpwithiterables......................................................................240
3.9ApplicationsandRecipes....................................................................................240
3.9.1XMLSAX,minidom,ElementTree,Lxml..............................................241
3.9.2Relationaldatabaseaccess...........................................................................249
3.9.3CSVcommaseparatedvaluefiles...........................................................255
3.9.4YAMLandPyYAML..................................................................................256
3.9.5Json..............................................................................................................258
4Part4GeneratingPythonBindingsforXML.........................................................260
4.1Introduction.........................................................................................................260
4.2Generatingthecode.............................................................................................261
4.3UsingthegeneratedcodetoparseandexportanXMLdocument.....................263
4.4Somecommandlineoptionsyoumightwanttoknow.......................................263
4.5Thegraphicalfrontend.......................................................................................264
4.6Addingapplicationspecificbehavior.................................................................265
4.6.1Implementingcustomsubclasses................................................................265
4.6.2Usingthegenerated"API"fromyourapplication......................................266
4.6.3Acombinedapproach..................................................................................267
4.7Specialsituationsanduses..................................................................................269
4.7.1Generic,typeindependentprocessing.........................................................269
4.7.1.1Step1generatethebindings............................................................270
4.7.1.2Step2addapplicationspecificcode................................................270
4.7.1.3Step3writeatest/driverharness.....................................................274
4.7.1.4Step4runthetestapplication..........................................................276
4.8Somehints...........................................................................................................276
4.8.1ChildrendefinedwithmaxOccursgreaterthan1........................................276
4.8.2Childrendefinedwithsimplenumerictypes...............................................277
4.8.3Thetypeofanelement'scharactercontent..................................................277
4.8.4Constructorsandtheirdefaultvalues..........................................................277

Page8

APythonBook

Preface
ThisbookisacollectionofmaterialsthatI'veusedwhenconductingPythontrainingand
alsomaterialsfrommyWebsitethatareintendedforselfinstruction.
Youmaypreferamachinereadablecopyofthisbook.Youcanfinditinvariousformats
here:
HTMLhttp://www.davekuhlman.org/python_book_01.html
PDFhttp://www.davekuhlman.org/python_book_01.pdf
ODF/OpenOfficehttp://www.davekuhlman.org/python_book_01.odt
And,letmethankthestudentsinmyPythonclasses.Theirquestionsandsuggestions
wereagreathelpinthepreparationofthesematerials.

Page9

APythonBook

1Part1BeginningPython
1.1IntroductionsEtc
Introductions
Practicalmatters:restrooms,breakroom,lunchandbreaktimes,etc.
StartingthePythoninteractiveinterpreter.Also,IPythonandIdle.
Runningscripts
EditorsChooseaneditorwhichyoucanconfiguresothatitindentswith4spaces,not
tabcharacters.ForalistofeditorsforPython,see:
http://wiki.python.org/moin/PythonEditors.Afewpossibleeditors:
SciTEhttp://www.scintilla.org/SciTE.html.
MSWindowsonly(1)TextPadhttp://www.textpad.com;(2)UltraEdit
http://www.ultraedit.com/.
JedSeehttp://www.jedsoft.org/jed/.
EmacsSeehttp://www.gnu.org/software/emacs/and
http://www.xemacs.org/faq/xemacsfaq.html.
jEditRequiresabitofcustomizationforPythonSeehttp://jedit.org.
Vimhttp://www.vim.org/
Geanyhttp://www.geany.org/
Andmanymore.
Interactiveinterpreters:

python
ipython
Idle
IDEsAlsosee
http://en.wikipedia.org/wiki/List_of_integrated_development_environments_for_Python:

PyWinMSWindowsonly.Availableat:
http://sourceforge.net/projects/pywin32/.
WingIDESeehttp://wingware.com/wingide/.
Eclipsehttp://eclipse.org/.ThereisapluginthatsupportsPython.
KdevelopLinux/KDESeehttp://www.kdevelop.org/.
EricLinuxKDE?Seehttp://ericide.pythonprojects.org/index.html
EmacsandSciTEwillevaluateaPythonbufferwithintheeditor.
Page10

APythonBook

1.1.1Resources
Whereelsetogethelp:
Pythonhomepagehttp://www.python.org
Pythonstandarddocumentationhttp://www.python.org/doc/.
Youwillalsofindlinkstotutorialsthere.
FAQshttp://www.python.org/doc/faq/.
ThePythonWikihttp://wiki.python.org/
ThePythonPackageIndexLotsofPythonpackages
https://pypi.python.org/pypi
Specialinterestgroups(SIGs)http://www.python.org/sigs/
Otherpythonrelatedmailinglistsandlistsforspecificapplications(forexample,
Zope,Twisted,etc).Try:http://dir.gmane.org/search.php?match=python.
http://sourceforge.netLotsofprojects.Searchfor"python".
USENETcomp.lang.python.CanalsobeaccessedthroughGmane:
http://dir.gmane.org/gmane.comp.python.general.
ThePythontutoremaillisthttp://mail.python.org/mailman/listinfo/tutor
Localdocumentation:

OnMSWindows,thePythondocumentationisinstalledwiththestandard
installation.
InstallthestandardPythondocumentationonyourmachinefrom
http://www.python.org/doc/.
pydoc.Example,onthecommandline,type:pydocre.
Importamodule,thenviewits.__doc__attribute.
Attheinteractiveprompt,usehelp(obj).Youmightneedtoimportitfirst.
Example:
>>>importurllib
>>>help(urllib)

InIPython,thequestionmarkoperatorgiveshelp.Example:
In[13]:open?
Type:builtin_function_or_method
BaseClass:<type'builtin_function_or_method'>
StringForm:<builtinfunctionopen>
Namespace:Pythonbuiltin
Docstring:
open(name[,mode[,buffering]])>fileobject
Openafileusingthefile()type,returnsafile
object.
ConstructorDocstring:
x.__init__(...)initializesx;see
x.__class__.__doc__forsignature

Page11

APythonBook
Callable:Yes
Calldef:Callingdefinitionnotavailable.Call
docstring:
x.__call__(...)<==>x(...)

1.1.2AgeneraldescriptionofPython
Pythonisahighlevelgeneralpurposeprogramminglanguage:
Becausecodeisautomaticallycompiledtobytecodeandexecuted,Pythonis
suitableforuseasascriptinglanguage,Webapplicationimplementation
language,etc.
BecausePythoncanbeextendedinCandC++,Pythoncanprovidethespeed
neededforevencomputeintensivetasks.
Becauseofitsstrongstructuringconstructs(nestedcodeblocks,functions,
classes,modules,andpackages)anditsconsistentuseofobjectsand
objectorientedprogramming,Pythonenablesustowriteclear,logical
applicationsforsmallandlargetasks.
ImportantfeaturesofPython:

Builtinhighleveldatatypes:strings,lists,dictionaries,etc.
Theusualcontrolstructures:if,ifelse,ifelifelse,while,plusapowerful
collectioniterator(for).
Multiplelevelsoforganizationalstructure:functions,classes,modules,and
packages.Theseassistinorganizingcode.Anexcellentandlargeexampleisthe
Pythonstandardlibrary.
CompileontheflytobytecodeSourcecodeiscompiledtobytecodewithouta
separatecompilestep.Sourcecodemodulescanalsobe"precompiled"tobyte
codefiles.
ObjectorientedPythonprovidesaconsistentwaytouseobjects:everythingis
anobject.And,inPythonitiseasytoimplementnewobjecttypes(calledclasses
inobjectorientedprogramming).
ExtensionsinCandC++Extensionmodulesandextensiontypescanbewritten
byhand.Therearealsotoolsthathelpwiththis,forexample,SWIG,sip,Pyrex.
JythonisaversionofPythonthat"playswellwith"Java.See:TheJythonProject
http://www.jython.org/Project/.
Somethingsyouwillneedtoknow:

Pythonusesindentationtoshowblockstructure.Indentoneleveltoshowthe
beginningofablock.Outdentoneleveltoshowtheendofablock.Asan
example,thefollowingCstylecode:
if(x)
{

Page12

APythonBook
if(y)
{
f1()
}
f2()
}

inPythonwouldbe:
ifx:
ify:
f1()
f2()

And,theconventionistousefourspaces(andnohardtabs)foreachlevelofindentation.
Actually,it'smorethanaconvention;it'spracticallyarequirement.Followingthat
"convention"willmakeitsomucheasiertomergeyourPythoncodewithcodefrom
othersources.
AnoverviewofPython:

AscriptinglanguagePythonissuitable(1)forembedding,(2)forwritingsmall
unstructuredscripts,(3)for"quickanddirty"programs.
Notascriptinglanguage(1)Pythonscales.(2)Pythonencouragesustowrite
codethatisclearandwellstructured.
Interpreted,butalsocompiledtobytecode.Modulesareautomaticallycompiled
(to.pyc)whenimported,butmayalsobeexplicitlycompiled.
Providesaninteractivecommandlineandinterpretershell.Infact,thereare
several.
DynamicForexample:
Typesareboundtovalues,nottovariables.
Functionandmethodlookupisdoneatruntime.
Valuesareinspectable.
Thereisaninteractiveinterpreter,morethanone,infact.
Youcanlistthemethodssupportedbyanygivenobject.
Stronglytypedatruntime,notcompiletime.Objects(values)haveatype,but
variablesdonot.
ReasonablyhighlevelHighlevelbuiltindatatypes;highlevelcontrol
structures(forwalkinglistsanditerators,forexample).
ObjectorientedAlmosteverythingisanobject.Simpleobjectdefinition.Data
hidingbyagreement.Multipleinheritance.Interfacesbyconvention.
Polymorphism.
HighlystructuredStatements,functions,classes,modules,andpackagesenable
ustowritelarge,wellstructuredapplications.Whystructure?Readability,
locateability,modifiability.
Explicitness
Page13

APythonBook

Firstclassobjects:
Definition:Can(1)passtofunction;(2)returnfromfunction;(3)stuffintoa
datastructure.
Operatorscanbeappliedtovalues(notvariables).Example:f(x)[3]
Indentedblockstructure"Pythonispseudocodethatruns."
EmbeddingandextendingPythonPythonprovidesawelldocumentedand
supportedway(1)toembedthePythoninterpreterinC/C++applicationsand(2)
toextendPythonwithmodulesandobjectsimplementedinC/C++.
Insomecases,SWIGcangeneratewrappersforexistingC/C++code
automatically.Seehttp://www.swig.org/
CythonenablesustogenerateCcodefromPythonandto"easily"create
wrappersforC/C++functions.See
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ToembedandextendPythonwithJava,thereisJython.See
http://www.jython.org/
Automaticgarbagecollection.(But,thereisagcmoduletoallowexplicitcontrol
ofgarbagecollection.)
Comparisonwithotherlanguages:compiledlanguages(e.g.C/C++);Java;Perl,
Tcl,andRuby.Pythonexcellsat:developmentspeed,executionspeed,clarityand
maintainability.
VarietiesofPython:
CPythonStandardPython2.ximplementedinC.
JythonPythonfortheJavaenvironmenthttp://www.jython.org/
PyPyPythonwithaJITcompilerandstacklessmodehttp://pypy.org/
StacklessPythonwithenhancedthreadsupportandmicrothreadsetc.
http://www.stackless.com/
IronPythonPythonfor.NETandtheCLRhttp://ironpython.net/
Python3Thenew,newPython.Thisisintendedasareplacementfor
Python2.x.http://www.python.org/doc/.Afewdifferences(fromPython
2.x):
Theprintstatementchangedtotheprintfunction.
Stringsareunicodebydefault.
Classesareall"newstyle"classes.
Changestosyntaxforcatchingexceptions.
Changestointegersnolonginteger;integerdivisionwithautomatic
converttofloat.
Morepervasiveuseofiterables(ratherthancollections).
Etc.
ForamoreinformationaboutdifferencesbetweenPython2.xandPython3.x,
seethedescriptionofthevariousfixesthatcanbeappliedwiththe2to3tool:
Page14

APythonBook

http://docs.python.org/3/library/2to3.html#fixers
Themigrationtool,2to3,easestheconversionof2.xcodeto3.x.
AlsoseeTheZenofPythonhttp://www.python.org/peps/pep0020.html.Or,at
thePythoninteractiveprompt,type:
>>>importthis

1.1.3InteractivePython
IfyouexecutePythonfromthecommandlinewithnoscript(noarguments),Python
givesyouaninteractiveprompt.ThisisanexcellentfacilityforlearningPythonandfor
tryingsmallsnippetsofcode.Manyoftheexamplesthatfollowweredevelopedusing
thePythoninteractiveprompt.
StartthePythoninteractiveinterpreterbytypingpythonwithnoargumentsatthe
commandline.Forexample:
$python
Python2.6.1(r261:67515,Jan112009,15:19:23)
[GCC4.3.2]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>print'hello'
hello
>>>

YoumayalsowanttoconsiderusingIDLE.IDLEisagraphicalintegrateddevelopment
environmentforPython;itcontainsaPythonshell.ItislikelythatIdlewasinstalledfor
youwhenyouinstalledPython.YouwillfindascripttostartupIDLEinthe
Tools/scriptsdirectoryofyourPythondistribution.IDLErequiresTkinter.
Inaddition,therearetoolsthatwillgiveyouamorepowerfulandfancyPython
interactiveinterpreter.OneexampleisIPython,whichisavailableat
http://ipython.scipy.org/.

1.2Lexicalmatters
1.2.1Lines

Pythondoeswhatyouwantittodomostofthetimesothatyouonlyhavetoadd
extracharacterssomeofthetime.
Statementseparatorisasemicolon,butisonlyneededwhenthereismorethan
onestatementonaline.And,writingmorethanonestatementonthesamelineis
consideredbadform.
ContinuationlinesAbackslashaslastcharacterofthelinemakesthe
Page15

APythonBook
followinglineacontinuationofthecurrentline.But,notethatanopening
"context"(parenthesis,squarebracket,orcurlybracket)makesthebackslash
unnecessary.

1.2.2Comments
Everythingafter"#"onalineisignored.Noblockcomments,butdocstringsarea
commentinquotesatthebeginningofamodule,class,methodorfunction.Also,editors
withsupportforPythonoftenprovidetheabilitytocommentoutselectedblocksofcode,
usuallywith"##".

1.2.3Namesandtokens

Allowedcharacters:azAZ09underscore,andmustbeginwithaletteror
underscore.
Namesandidentifiersarecasesensitive.
Identifierscanbeofunlimitedlength.
Specialnames,customizing,etc.Usuallybeginandendindoubleunderscores.
SpecialnameclassesSingleanddoubleunderscores.
SingleleadingsingleunderscoreSuggestsa"private"methodorvariable
name.Notimportedby"frommoduleimport*".
SingletrailingunderscoreUseittoavoidconflictswithPythonkeywords.
DoubleleadingunderscoresUsedinaclassdefinitiontocausename
mangling(weakhiding).But,notoftenused.
NamingconventionsNotrigid,but:
Modulesandpackagesalllowercase.
GlobalsandconstantsUppercase.
ClassesBumpycapswithinitialupper.
MethodsandfunctionsAlllowercasewithwordsseparatedbyunderscores.
LocalvariablesLowercase(withunderscorebetweenwords)orbumpy
capswithinitialloweroryourchoice.
GoodadviceFollowtheconventionsusedinthecodeonwhichyouare
working.
Names/variablesinPythondonothaveatype.Valueshavetypes.

1.2.4Blocksandindentation
Pythonrepresentsblockstructureandnestedblockstructurewithindentation,notwith
beginandendbrackets.
TheemptyblockUsethepassnoopstatement.
Benefitsoftheuseofindentationtoindicatestructure:
Page16

APythonBook
Reducestheneedforacodingstandard.Onlyneedtospecifythatindentationis4
spacesandnohardtabs.
Reducesinconsistency.Codefromdifferentsourcesfollowthesameindentation
style.Ithasto.
Reduceswork.Onlyneedtogettheindentationcorrect,notbothindentationand
brackets.
Reducesclutter.Eliminatesallthecurlybrackets.
Ifitlookscorrect,itiscorrect.Indentationcannotfoolthereader.
EditorconsiderationsThestandardis4spaces(nohardtabs)foreachindentationlevel.
Youwillneedatexteditorthathelpsyourespectthat.

1.2.5Docstrings
Docstringsarelikecomments,buttheyarecarriedwithexecutingcode.Docstringscan
beviewedwithseveraltools,e.g.help(),obj.__doc__,and,inIPython,aquestion
mark(?)afteranamewillproducehelp.
Adocstringiswrittenasaquotedstringthatisatthetopofamoduleorthefirstlines
aftertheheaderlineofafunctionorclass.
Wecanusetriplequotingtocreatedocstringsthatspanmultiplelines.
Therearealsotoolsthatextractandformatdocstrings,forexample:
pydocDocumentationgeneratorandonlinehelpsystem
http://docs.python.org/lib/modulepydoc.html.
epydocEpydoc:AutomaticAPIDocumentationGenerationforPython
http://epydoc.sourceforge.net/index.html
SphinxCanalsoextractdocumentationfromPythondocstrings.See
http://sphinxdoc.org/index.html.
Seethefollowingforsuggestionsandmoreinformationondocstrings:Docstring
conventionshttp://www.python.org/dev/peps/pep0257/.

1.2.6Programstructure

Executiondef,class,etcareexecutablestatementsthataddsomethingtothe
currentnamespace.Modulescanbebothexecutableandimportable.
Statements,datastructures,functions,classes,modules,packages.
Functions
Classes
Modulescorrespondtofileswitha"*.py"extension.Packagescorrespondtoa
directory(orfolder)inthefilesystem;apackagecontainsafilenamed
"__init__.py".Bothmodulesandpackagescanbeimported(seesectionimport
Page17

APythonBook

statement).
PackagesAdirectorycontainingafilenamed"__init__.py".Canprovide
additionalinitializationwhenthepackageoramoduleinitisloaded(imported).

1.2.7Operators

See:http://docs.python.org/ref/operators.html.Pythondefinesthefollowing
operators:
+***///%
<<>>&|^~
<><=>===!=<>

Thecomparisonoperators<>and!=arealternatespellingsofthesameoperator.
!=isthepreferredspelling;<>isobsolescent.
Logicaloperators:
andorisnotin

Therearealso(1)thedotoperator,(2)thesubscriptoperator[],andthe
function/methodcalloperator().
Forinformationontheprecedencesofoperators,seethetableat
http://docs.python.org/2/reference/expressions.html#operatorprecedence,which
isreproducedbelow.
Forinformationonwhatthedifferentoperatorsdo,thesectioninthe"Python
LanguageReference"titled"Specialmethodnames"maybeofhelp:
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
ThefollowingtablesummarizestheoperatorprecedencesinPython,fromlowest
precedence(leastbinding)tohighestprecedence(mostbinding).Operatorsonthe
samelinehavethesameprecedence.Unlessthesyntaxisexplicitlygiven,
operatorsarebinary.Operatorsonthesamelinegrouplefttoright(exceptfor
comparisons,includingtests,whichallhavethesameprecedenceandchainfrom
lefttorightseesection5.9andexponentiation,whichgroupsfromrightto
left):
OperatorDescription
==========================================
lambdaLambdaexpression
orBooleanOR
andBooleanAND
notxBooleanNOT
in,notinMembershiptests
is,isnotIdentitytests
<,<=,>,>=,<>,!=,==Comparisons
|BitwiseOR
^BitwiseXOR
&BitwiseAND
<<,>>Shifts

Page18

APythonBook
+,Additionandsubtraction
*,/,%Multiplication,division,
remainder
+x,xPositive,negative
~xBitwisenot
**Exponentiation
x.attributeAttributereference
x[index]Subscription
x[index:index]Slicing
f(arguments...)Functioncall
(expressions...)Bindingortupledisplay
[expressions...]Listdisplay
{key:datum...}Dictionarydisplay
`expressions...`Stringconversion

Notethatmostoperatorsresultincallstomethodswithspecialnames,for
example__add__,__sub__,__mul__,etc.SeeSpecialmethodnames
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
Later,wewillseehowtheseoperatorscanbeemulatedinclassesthatyoudefine
yourself,throughtheuseofthesespecialnames.

1.2.8Alsosee
FormoreonlexicalmattersandPythonstyles,see:

CodeLikeaPythonista:IdiomaticPython
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html.
StyleGuideforPythonCodehttp://www.python.org/dev/peps/pep0008/
Theflake8stylecheckingprogram.Seehttps://pypi.python.org/pypi/flake8.Also
seethepylintcodechecker:https://pypi.python.org/pypi/pylint.

1.2.9Codeevaluation
UnderstandingthePythonexecutionmodelHowPythonevaluatesandexecutesyour
code.
Evaluatingexpressions.
Creatingnames/variablesBindingThefollowingallcreatenames(variables)and
bindvalues(objects)tothem:(1)assignment,(2)functiondefinition,(3)classdefinition,
(4)functionandmethodcall,(5)importingamodule,...
FirstclassobjectsAlmostallobjectsinPythonarefirstclass.Definition:Anobjectis
firstclassif:(1)wecanputitinastructuredobject;(2)wecanpassittoafunction;(3)
wecanreturnitfromafunction.
ReferencesObjects(orreferencestothem)canbeshared.Whatdoesthismean?

Theobject(s)satisfytheidentitytestoperatoris.
Page19

APythonBook

Thebuiltinfunctionid()returnsthesamevalue.
Theconsequencesformutableobjectsaredifferentfromthoseforimmutable
objects.
Changing(updating)amutableobjectreferencedthroughonevariableor
containeralsochangesthatobjectreferencedthroughothervariablesor
containers,becauseitisthesameobject.
del()Thebuiltinfunctiondel()removesareference,not(necessarily)the
objectitself.

1.3Statementsandinspectionpreliminaries
printExample:
printobj
print"one","two",'three'

for:Example:
stuff=['aa','bb','cc']
foriteminstuff:
printitem

LearnwhatthetypeofanobjectisExample:
type(obj)

Learnwhatattributesanobjecthasandwhatit'scapabilitiesareExample:
dir(obj)
value="amessage"
dir(value)

GethelponaclassoranobjectExample:
help(str)
help("")
value="abc"
help(value)
help(value.upper)

InIPython(butnotstandardPython),youcanalsogethelpattheinteractivepromptby
typing"?"and"??"afteranobject.Example:
In[48]:a=''
In[49]:a.upper?
Type:builtin_function_or_method
StringForm:<builtinmethodupperofstrobjectat0x7f1c426e0508>
Docstring:
S.upper()>string

Page20

APythonBook
ReturnacopyofthestringSconvertedtouppercase.

1.4Builtindatatypes
Forinformationonbuiltindatatypes,seesectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.

1.4.1Numerictypes
Thenumerictypesare:
PlainintegersSameprecisionasaClong,usuallya32bitbinarynumber.
LongintegersDefinewith100L.But,plainintegersareautomatically
promotedwhenneeded.
FloatsImplementedasaCdouble.Precisiondependsonyourmachine.See
sys.float_info.
ComplexnumbersDefinewith,forexample,3jorcomplex(3.0,2.0).
See2.3.4NumericTypesint,float,long,complex
http://docs.python.org/lib/typesnumeric.html.

Pythondoesmixedarithmetic.
Integerdivisiontruncates.ThisischangedinPython3.Usefloat(n)toforcecoercion
toafloat.Example:
In[8]:a=4
In[9]:b=5
In[10]:a/b
Out[10]:0#possiblywrong?
In[11]:float(a)/b
Out[11]:0.8

Applyingthefunctioncalloperator(parentheses)toatypeorclasscreatesaninstanceof
thattypeorclass.
ScientificandheavilynumericprogrammingHighlevelPythonisnotveryefficientfor
numericalprogramming.But,therearelibrariesthathelpNumpyandSciPySee:
SciPy:ScientificToolsforPythonhttp://scipy.org/

1.4.2Tuplesandlists
ListAlistisadynamicarray/sequence.Itisorderedandindexable.Alistismutable.
Listconstructors:[],list().
range()andxrange():
Page21

APythonBook
range(n)createsalistofnintegers.Optionalargumentsarethestartinginteger
andastride.
xrangeislikerange,exceptthatitcreatesaniteratorthatproducestheitems
inthelistofintegersinsteadofthelistitself.
TuplesAtupleisasequence.Atupleisimmutable.

Tupleconstructors:(),butreallyacomma;alsotuple().
Tuplesarelikelists,butarenotmutable.
Pythonlistsare(1)heterogeneous(2)indexable,and(3)dynamic.Forexample,wecan
addtoalistandmakeitlonger.
Notesonsequenceconstructors:
Toconstructatuplewithasingleelement,use(x,);atuplewithasingle
elementrequiresacomma.
Youcanspreadelementsacrossmultiplelines(andnoneedforbackslash
continuationcharacter"\").
Acommacanfollowthelastelement.
Thelengthofatupleorlist(orothercontainer):len(mylist).

Operatorsforlists:

Try:list1+list2,list1*n,list1+=list2,etc.
Comparisonoperators:<,==,>=,etc.
Testformembershipwiththeinoperator.Example:
In[77]:a=[11,22,33]
In[78]:a
Out[78]:[11,22,33]
In[79]:22ina
Out[79]:True
In[80]:44ina
Out[80]:False

Subscription:
Indexingintoasequence
NegativeindexesEffectively,lengthofsequenceplus(minus)index.
SlicingExample:data[2:5].Defaultvalues:beginningandendoflist.
SlicingwithstridesExample:data[::2].
OperationsontuplesNooperationsthatchangethetuple,sincetuplesareimmutable.
Wecandoiterationandsubscription.Wecando"contains"(theinoperator)andgetthe
length(thelen()operator).Wecanusecertainbooleanoperators.

OperationsonlistsOperationssimilartotuplesplus:

Appendmylist.append(newitem).
Page22

APythonBook
Insertmylist.insert(index,newitem).Noteonefficiency:The
insertmethodisnotasfastastheappendmethod.Ifyoufindthatyouneed
todoalargenumberofmylist.insert(0,obj)(thatis,insertingatthe
beginningofthelist)considerusingadequeinstead.See:
http://docs.python.org/2/library/collections.html#collections.deque.Or,use
appendandreverse.
Extendmylist.extend(anotherlist).Alsocanuse+and+=.
Removemylist.remove(item)andmylist.pop().Notethat
append()togetherwithpop()implementsastack.
Deletedelmylist[index].
PopGetlast(rightmost)itemandremovefromlistmylist.pop().
Listoperators+,*,etc.

Formoreoperationsandoperatorsonsequences,see:
http://docs.python.org/2/library/stdtypes.html#sequencetypesstrunicodelisttuplebyte
arraybufferxrange.
Exercises:

Createanemptylist.Append4stringstothelist.Thenpoponeitemofftheend
ofthelist.Solution:
In[25]:a=[]
In[26]:a.append('aaa')
In[27]:a.append('bbb')
In[28]:a.append('ccc')
In[29]:a.append('ddd')
In[30]:printa
['aaa','bbb','ccc','ddd']
In[31]:a.pop()
Out[31]:'ddd'

Usetheforstatementtoprinttheitemsinthelist.Solution:
In[32]:foritemina:
....:printitem
....:
aaa
bbb
ccc

Usethestringjoinoperationtoconcatenatetheitemsinthelist.Solution:
In[33]:'||'.join(a)
Out[33]:'aaa||bbb||ccc'

Uselistscontainingthree(3)elementstocreateandshowatree:
In[37]:b=['bb',None,None]
In[38]:c=['cc',None,None]
In[39]:root=['aa',b,c]

Page23

APythonBook
In[40]:
In[40]:
In[40]:defshow_tree(t):
....:ifnott:
....:return
....:printt[0]
....:show_tree(t[1])
....:show_tree(t[2])
....:
....:
In[41]:show_tree(root)
aa
bb
cc

Notethatwewilllearnabetterwaytorepresenttreestructureswhenwecover
implementingclassesinPython.

1.4.3Strings
Stringsaresequences.Theyareimmutable.Theyareindexable.Theyareiterable.
Foroperationsonstrings,seehttp://docs.python.org/lib/stringmethods.htmloruse:
>>>help(str)

Or:
>>>dir("abc")

Stringoperations(methods).
Stringoperators,e.g.+,<,<=,==,etc..
Constructors/literals:
Quotes:singleanddouble.Escapingquotesandotherspecialcharacterswitha
backslash.
TriplequotingUsetriplesinglequotesordoublequotestodefinemultiline
strings.
str()Theconstructorandthenameofthetype/class.
'aSeparator'.join(aList)
Manymore.
Escapecharactersinstrings\t,\n,\\,etc.

StringformattingSee:
http://docs.python.org/2/library/stdtypes.html#stringformattingoperations
Examples:
In[18]:name='dave'

Page24

APythonBook
In[19]:size=25
In[20]:factor=3.45
In[21]:print'Name:%sSize:%dFactor:%3.4f'%(name,size,
factor,)
Name:daveSize:25Factor:3.4500
In[25]:print'Name:%sSize:%dFactor:%08.4f'%(name,size,
factor,)
Name:daveSize:25Factor:003.4500

Iftherighthandargumenttotheformattingoperatorisadictionary,thenyoucan
(actually,must)usethenamesofkeysinthedictionaryinyourformatstrings.Examples:
In[115]:values={'vegetable':'chard','fruit':'nectarine'}
In[116]:'Ilove%(vegetable)sandIlove%(fruit)s.'%values
Out[116]:'IlovechardandIlovenectarine.'

Alsoconsiderusingtherightjustifyandleftjustifyoperations.Examples:
mystring.rjust(20),mystring.ljust(20,':').
InPython3,thestr.formatmethodispreferredtothestringformattingoperator.
ThismethodisalsoavailableinPython2.7.Ithasbenefitsandadvantagesoverthestring
formattingoperator.Youcanstartlearningaboutithere:
http://docs.python.org/2/library/stdtypes.html#stringmethods
Exercises:

Usealiteraltocreateastringcontaining(1)asinglequote,(2)adoublequote,(3)
bothasingleanddoublequote.Solutions:
"Some'quoted'text."
'Some"quoted"text.'
'Some"quoted"\'extra\'text.'

Writeastringliteralthatspansmultiplelines.Solution:
"""Thisstring
spansseverallines
becauseitisalittlelong.
"""

Usethestringjoinoperationtocreateastringthatcontainsacolonasa
separator.Solution:
>>>content=[]
>>>content.append('finch')
>>>content.append('sparrow')
>>>content.append('thrush')
>>>content.append('jay')
>>>contentstr=':'.join(content)
>>>printcontentstr
finch:sparrow:thrush:jay

Usestringformattingtoproduceastringcontainingyourlastandfirstnames,
Page25

APythonBook
separatedbyacomma.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>full='%s,%s'%(last,first,)
>>>printfull
Kuhlman,Dave

IncrementallybuildinguplargestringsfromlotsofsmallstringstheoldwaySince
stringsinPythonareimmutable,appendingtoastringrequiresareallocation.So,itis
fastertoappendtoalist,thenusejoin.Example:
In[25]:strlist=[]
In[26]:strlist.append('Line#1')
In[27]:strlist.append('Line#2')
In[28]:strlist.append('Line#3')
In[29]:str='\n'.join(strlist)
In[30]:printstr
Line#1
Line#2
Line#3

IncrementallybuildinguplargestringsfromlotsofsmallstringsthenewwayThe
+=operationonstringshasbeenoptimized.So,whenyoudothisstr1+=str2,
evenmanytimes,itisefficient.
Thetranslatemethodenablesustomapthecharactersinastring,replacingthosein
onetablebythoseinanother.And,themaketransfunctioninthestringmodule,
makesiteasytocreatethemappingtable:
importstring
deftest():
a='axbycz'
t=string.maketrans('abc','123')
printa
printa.translate(t)
test()

1.4.3.1Thenewstring.formatmethod

Thenewwaytodostringformatting(whichisstandardinPython3andperhaps
preferredfornewcodeinPython2)istousethestring.formatmethod.Seehere:
http://docs.python.org/2/library/stdtypes.html#str.format
http://docs.python.org/2/library/string.html#formatstringsyntax
http://docs.python.org/2/library/string.html#formatspecificationminilanguage
Someexamples:

Page26

APythonBook
In[1]:'aaa{1}bbb{0}ccc{1}ddd'.format('xx','yy',)
Out[1]:'aaayybbbxxcccyyddd'
In[2]:'number:{0:05d}ok'.format(25)
Out[2]:'number:00025ok'
In[4]:'n1:{num1}n2:{num2}'.format(num2=25,num1=100)
Out[4]:'n1:100n2:25'
In[5]:'n1:{num1}n2:{num2}again:{num1}'.format(num2=25,
num1=100)
Out[5]:'n1:100n2:25again:100'
In[6]:'number:{:05d}ok'.format(25)
Out[6]:'number:00025ok'
In[7]:values={'name':'dave','hobby':'birding'}
In[8]:'user:{name}activity:{hobby}'.format(**values)
Out[8]:'user:daveactivity:birding'

1.4.3.2Unicodestrings

Representingunicode:
In[96]:a=u'abcd'
In[97]:a
Out[97]:u'abcd'
In[98]:b=unicode('efgh')
In[99]:b
Out[99]:u'efgh'

Converttounicode:a_string.decode(encoding).Examples:
In[102]:'abcd'.decode('utf8')
Out[102]:u'abcd'
In[103]:
In[104]:'abcd'.decode(sys.getdefaultencoding())
Out[104]:u'abcd'

Convertoutofunicode:a_unicode_string.encode(encoding).Examples:
In[107]:a=u'abcd'
In[108]:a.encode('utf8')
Out[108]:'abcd'
In[109]:a.encode(sys.getdefaultencoding())
Out[109]:'abcd'
In[110]:b=u'Sel\xe7uk'
In[111]:printb.encode('utf8')
Seluk

TestforunicodetypeExample:
In[122]:importtypes
In[123]:a=u'abcd'
In[124]:type(a)istypes.UnicodeType
Out[124]:True
In[125]:

Page27

APythonBook
In[126]:type(a)istype(u'')
Out[126]:True

Orbetter:
In[127]:isinstance(a,unicode)
Out[127]:True

Anexamplewithacharacter"c"withahachek:
In[135]:name='IvanKrsti\xc4\x87'
In[136]:name.decode('utf8')
Out[136]:u'IvanKrsti\u0107'
In[137]:
In[138]:len(name)
Out[138]:12
In[139]:len(name.decode('utf8'))
Out[139]:11

Youcanalsocreateaunicodecharacterbyusingtheunichr()builtinfunction:
In[2]:a='aa'+unichr(170)+'bb'
In[3]:a
Out[3]:u'aa\xaabb'
In[6]:b=a.encode('utf8')
In[7]:b
Out[7]:'aa\xc2\xaabb'
In[8]:printb
aabb

GuidanceforuseofencodingsandunicodeIfyouareworkingwithamultibyte
characterset:
1. Convert/decodefromanexternalencodingtounicodeearly
(my_string.decode(encoding)).
2. Doyourworkinunicode.
3. Convert/encodetoanexternalencodinglate
(my_string.encode(encoding)).
Formoreinformation,see:
UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
PEP100:PythonUnicodeIntegration
http://www.python.org/dev/peps/pep0100/
InthePythonstandardlibrary:
codecsCodecregistryandbaseclasses
http://docs.python.org/2/library/codecs.html#modulecodecs
StandardEncodings
http://docs.python.org/2/library/codecs.html#standardencodings
Ifyouarereadingandwritingmultibytecharacterdatafromortoafile,thenlookatthe

Page28

APythonBook
codecs.open()inthecodecsmodule
http://docs.python.org/2/library/codecs.html#codecs.open.
HandlingmultibytecharactersetsinPython3iseasier,Ithink,butdifferent.Onehintis
tousetheencodingkeywordparametertotheopenbuiltinfunction.Hereisan
example:
deftest():
infile=open('infile1.txt','r',encoding='utf8')
outfile=open('outfile1.txt','w',encoding='utf8')
forlineininfile:
line=line.upper()
outfile.write(line)
infile.close()
outfile.close()
test()

1.4.4Dictionaries
Adictionaryisacollection,whosevaluesareaccessiblebykey.Itisacollectionof
namevaluepairs.
Theorderofelementsinadictionaryisundefined.But,wecaniterateover(1)thekeys,
(2)thevalues,and(3)theitems(keyvaluepairs)inadictionary.Wecansetthevalueof
akeyandwecangetthevalueassociatedwithakey.
Keysmustbeimmutableobjects:ints,strings,tuples,...
Literalsforconstructingdictionaries:
d1={}
d2={key1:value1,key2:value2,}

Constructorfordictionariesdict()canbeusedtocreateinstancesoftheclassdict.
Someexamples:
dict({'one':2,'two':3})
dict({'one':2,'two':3}.items())
dict({'one':2,'two':3}.iteritems())
dict(zip(('one','two'),(2,3)))
dict([['two',3],['one',2]])
dict(one=2,two=3)
dict([(['one','two'][i2],i)foriin(2,3)])

Foroperationsondictionaries,seehttp://docs.python.org/lib/typesmapping.htmloruse:
>>>help({})

Or:
Page29

APythonBook
>>>dir({})

IndexingAccessoradditemstoadictionarywiththeindexingoperator[].Example:
In[102]:dict1={}
In[103]:dict1['name']='dave'
In[104]:dict1['category']=38
In[105]:dict1
Out[105]:{'category':38,'name':'dave'}

Someoftheoperationsproducethekeys,thevalues,andtheitems(pairs)inadictionary.
Examples:
In[43]:d={'aa':111,'bb':222}
In[44]:d.keys()
Out[44]:['aa','bb']
In[45]:d.values()
Out[45]:[111,222]
In[46]:d.items()
Out[46]:[('aa',111),('bb',222)]

Wheniteratingoverlargedictionaries,usemethodsiterkeys(),itervalues(),
anditeritems().Example:
In[47]:
In[47]:d={'aa':111,'bb':222}
In[48]:forkeyind.iterkeys():
....:printkey
....:
....:
aa
bb

Totestfortheexistenceofakeyinadictionary,usetheinoperatororthe
mydict.has_key(k)method.Theinoperatorispreferred.Example:
>>>d={'tomato':101,'cucumber':102}
>>>k='tomato'
>>>kind
True
>>>d.has_key(k)
True

Youcanoftenavoidtheneedforatestbyusingmethodget.Example:
>>>d={'tomato':101,'cucumber':102}
>>>d.get('tomato',1)
101
>>>d.get('chard',1)
1
>>>ifd.get('eggplant')isNone:
...print'missing'

Page30

APythonBook
...
missing

Dictionary"view"objectsprovidedynamic(automaticallyupdated)viewsofthekeysor
thevaluesortheitemsinadictionary.Viewobjectsalsosupportsetoperations.Create
viewswithmydict.viewkeys(),mydict.viewvalues(),and
mydict.viewitems().See:
http://docs.python.org/2/library/stdtypes.html#dictionaryviewobjects.
Thedictionarysetdefaultmethodprovidesawaytogetthevalueassociatedwitha
keyfromadictionaryandtosetthatvalueifthekeyismissing.Example:
In[106]:a
Out[106]:{}
In[108]:a.setdefault('cc',33)
Out[108]:33
In[109]:a
Out[109]:{'cc':33}
In[110]:a.setdefault('cc',44)
Out[110]:33
In[111]:a
Out[111]:{'cc':33}

Exercises:

Writealiteralthatdefinesadictionaryusingbothstringliteralsandvariables
containingstrings.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>name_dict={first:last,'Elvis':'Presley'}
>>>printname_dict
{'Dave':'Kuhlman','Elvis':'Presley'}

Writestatementsthatiterateover(1)thekeys,(2)thevalues,and(3)theitemsin
adictionary.(Note:Requiresintroductionoftheforstatement.)Solutions:
>>>d={'aa':111,'bb':222,'cc':333}
>>>forkeyind.keys():
...printkey
...
aa
cc
bb
>>>forvalueind.values():
...printvalue
...
111
333
222
>>>foritemind.items():
...printitem

Page31

APythonBook
...
('aa',111)
('cc',333)
('bb',222)
>>>forkey,valueind.items():
...printkey,'::',value
...
aa::111
cc::333
bb::222

Additionalnotesondictionaries:

Youcanuseiterkeys(),itervalues(),iteritems()toobtain
iteratorsoverkeys,values,anditems.
Adictionaryitselfisiterable:ititeratesoveritskeys.So,thefollowingtwolines
areequivalent:
forkinmyDict:printk
forkinmyDict.iterkeys():printk

Theinoperatortestsforakeyinadictionary.Example:
In[52]:mydict={'peach':'sweet','lemon':'tangy'}
In[53]:key='peach'
In[54]:ifkeyinmydict:
....:printmydict[key]
....:
sweet

1.4.5Files
Openafilewiththeopenfactorymethod.Example:
In[28]:f=open('mylog.txt','w')
In[29]:f.write('message#1\n')
In[30]:f.write('message#2\n')
In[31]:f.write('message#3\n')
In[32]:f.close()
In[33]:f=file('mylog.txt','r')
In[34]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
In[35]:f.close()

Notes:

Usethe(builtin)open(path,mode)functiontoopenafileandcreateafile
object.Youcouldalsousefile(),butopen()isrecommended.
Page32

APythonBook

Afileobjectthatisopenforreadingatextfilesupportstheiteratorprotocoland,
therefore,canbeusedinaforstatement.Ititeratesoverthelinesinthefile.This
ismostlikelyonlyusefulfortextfiles.
openisafactorymethodthatcreatesfileobjects.Useittoopenfilesforreading,
writing,andappending.Examples:
infile=open('myfile.txt','r')#openforreading
outfile=open('myfile.txt','w')#openfor(over)
writing
log=open('myfile.txt','a')#openfor
appendingtoexistingcontent

Whenyouhavefinishedwithafile,closeit.Examples:
infile.close()
outfile.close()

Youcanalsousethewith:statementtoautomaticallyclosethefile.Example:
withopen('tmp01.txt','r')asinfile:
forxininfile:
printx,

Theaboveworksbecauseafileisacontextmanager:itobeysthecontext
managerprotocol.Afilehasmethods__enter__and__exit__,andthe
__exit__methodautomaticallyclosesthefileforus.Seethesectiononthe
with:statement.
Inordertoopenmultiplefiles,youcannestwith:statements,oruseasingle
with:statementwithmultiple"expressionastarget"clauses.Example:
deftest():
#
#usemultiplenestedwith:statements.
withopen('small_file.txt','r')asinfile:
withopen('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%
line.upper())
printinfile
printoutfile
#
#useasinglewith:statement.
withopen('small_file.txt','r')asinfile,\
open('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%line.upper())
printinfile
printoutfile
test()

fileisthefiletypeandcanbeusedasaconstructortocreatefileobjects.But,
Page33

APythonBook
openispreferred.
Linesreadfromatextfilehaveanewline.Stripitoffwithsomethinglike:
line.rstrip('\n').
Forbinaryfilesyoushouldaddthebinarymode,forexample:rb,wb.Formore
aboutmodes,seethedescriptionoftheopen()functionatBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.html.
Learnmoreaboutfileobjectsandthemethodstheyprovideat:2.3.9FileObjects
http://docs.python.org/2/library/stdtypes.html#fileobjects.
Youcanalsoappendtoanexistingfile.Notethe"a"modeinthefollowingexample:
In[39]:f=open('mylog.txt','a')
In[40]:f.write('message#4\n')
In[41]:f.close()
In[42]:f=file('mylog.txt','r')
In[43]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
message#4
In[44]:f.close()

Forbinaryfiles,add"b"tothemode.NotstrictlynecessaryonUNIX,butneededonMS
Windows.And,youwillwanttomakeyourcodeportableacrossplatforms.Example:
In[62]:importzipfile
In[63]:outfile=open('tmp1.zip','wb')
In[64]:zfile=zipfile.ZipFile(outfile,'w',zipfile.ZIP_DEFLATED)
In[65]:zfile.writestr('entry1','myheroeshavealwaysbeen
cowboys')
In[66]:zfile.writestr('entry2','andtheystillareitseems')
In[67]:zfile.writestr('entry3','sadlyinsearchofand')
In[68]:zfile.writestr('entry4','onstepinbackof')
In[69]:
In[70]:zfile.writestr('entry4','onestepinbackof')
In[71]:zfile.writestr('entry5','themselvesandtheirslowmoving
ways')
In[72]:zfile.close()
In[73]:outfile.close()
In[75]:
$
$unziplvtmp1.zip
Archive:tmp1.zip
LengthMethodSizeRatioDateTimeCRC32Name

34Defl:N366%05290817:04f6b7d921entry1
27Defl:N297%05290817:0710da8f3dentry2
22Defl:N249%05290817:073fd17fdaentry3
18Defl:N2011%05290817:08d55182e6entry4

Page34

APythonBook
19Defl:N2111%05290817:081a892acdentry4
37Defl:N395%05290817:09e213708centry5

1571698%6files

Exercises:

Readallofthelinesofafileintoalist.Printthe3rdand5thlinesinthefile/list.
Solution:
In[55]:f=open('tmp1.txt','r')
In[56]:lines=f.readlines()
In[57]:f.close()
In[58]:lines
Out[58]:['the\n','big\n','brown\n','dog\n',
'had\n','long\n','hair\n']
In[59]:printlines[2]
brown
In[61]:printlines[4]
had

Morenotes:

Stripnewlines(andotherwhitespace)fromastringwithmethodsstrip(),
lstrip(),andrstrip().
Getthecurrentpositionwithinafilebyusingmyfile.tell().
Setthecurrentpositionwithinafilebyusingmyfile.seek().Itmaybe
helpfultouseos.SEEK_CURandos.SEEK_END.Forexample:
f.seek(2,os.SEEK_CUR)advancesthepositionbytwo
f.seek(3,os.SEEK_END)setsthepositiontothethirdtolast.
f.seek(25)setsthepositionrelativetothebeginningofthefile.

1.4.6Otherbuiltintypes
OtherbuiltindatatypesaredescribedinsectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.
1.4.6.1TheNonevalue/type

TheuniquevalueNoneisusedtoindicate"novalue","nothing","nonexistence",etc.
ThereisonlyoneNonevalue;inotherwords,it'sasingleton.
UseistotestforNone.Example:
>>>flag=None
>>>
>>>ifflagisNone:
...print'clear'

Page35

APythonBook
...
clear
>>>ifflagisnotNone:
...print'hello'
...
>>>

1.4.6.2Booleanvalues

TrueandFalsearethebooleanvalues.
Thefollowingvaluesalsocountasfalse,forexample,inanif:statement:False,
numericzero,None,theemptystring,anemptylist,anemptydictionary,anyempty
container,etc.Allothervalues,includingTrue,actastruevalues.
1.4.6.3Setsandfrozensets

Asetisanunorderedcollectionofimmutableobjects.Asetdoesnotcontainduplicates.
Setssupportseveralsetoperations,forexample:union,intersection,difference,...
Afrozensetislikeaset,exceptthatafrozensetisimmutable.Therefore,afrozensetis
hashableandcanbeusedasakeyinadictionary,anditcanbeaddedtoaset.
Createasetwiththesetconstructor.Examples:
>>>a=set()
>>>a
set([])
>>>a.add('aa')
>>>a.add('bb')
>>>a
set(['aa','bb'])
>>>b=set([11,22])
>>>b
set([11,22])
>>>c=set([22,33])
>>>b.union(c)
set([33,11,22])
>>>b.intersection(c)
set([22])

Formoreinformationonsets,see:SetTypesset,frozenset
http://docs.python.org/lib/typesset.html

1.5FunctionsandClassesAPreview
StructuredcodePythonprogramsaremadeupofexpressions,statements,functions,
classes,modules,andpackages.
Page36

APythonBook
Pythonobjectsarefirstclassobjects.
Expressionsareevaluated.
Statementsareexecuted.
Functions(1)areobjectsand(2)arecallable.
ObjectorientedprogramminginPython.Modeling"realworld"objects.(1)
Encapsulation;(2)datahiding;(3)inheritance.Polymorphism.
Classes(1)encapsulation;(2)datahiding;(3)inheritance.
Anoverviewofthestructureofatypicalclass:(1)methods;(2)theconstructor;(3)class
(static)variables;(4)super/subclasses.

1.6Statements
1.6.1Assignmentstatement
Formtarget=expression.
Possibletargets:

Identifier
TupleorlistCanbenested.Leftandrightsidesmusthaveequivalentstructure.
Example:
>>>x,y,z=11,22,33
>>>[x,y,z]=111,222,333
>>>a,(b,c)=11,(22,33)
>>>a,B=11,(22,33)

Thisfeaturecanbeusedtosimulateanenum:
In[22]:LITTLE,MEDIUM,LARGE=range(1,4)
In[23]:LITTLE
Out[23]:1
In[24]:MEDIUM
Out[24]:2

Subscriptionofasequence,dictionary,etc.Example:
In[10]:a=range(10)
In[11]:a
Out[11]:[0,1,2,3,4,5,6,7,8,9]
In[12]:a[3]='abc'
In[13]:a
Out[13]:[0,1,2,'abc',4,5,6,7,8,9]
In[14]:
In[14]:b={'aa':11,'bb':22}
In[15]:b

Page37

APythonBook
Out[15]:{'aa':11,'bb':22}
In[16]:b['bb']=1000
In[17]:b['cc']=2000
In[18]:b
Out[18]:{'aa':11,'bb':1000,'cc':2000}

AsliceofasequenceNotethatthesequencemustbemutable.Example:
In[1]:a=range(10)
In[2]:a
Out[2]:[0,1,2,3,4,5,6,7,8,9]
In[3]:a[2:5]=[11,22,33,44,55,66]
In[4]:a
Out[4]:[0,1,11,22,33,44,55,66,5,6,7,8,9]

AttributereferenceExample:
>>>classMyClass:
...pass
...
>>>anObj=MyClass()
>>>anObj.desc='pretty'
>>>printanObj.desc
pretty

Thereisalsoaugmentedassignment.Examples:
>>>index=0
>>>index+=1
>>>index+=5
>>>index+=f(x)
>>>index=1
>>>index*=3

Thingstonote:

Assignmenttoanamecreatesanewvariable(ifitdoesnotexistinthe
namespace)andabinding.Specifically,itbindsavaluetothenewname.Calling
afunctionalsodoesthistothe(formal)parameterswithinthelocalnamespace.
InPython,alanguagewithdynamictyping,thedatatypeisassociatedwiththe
value,notthevariable,asisthecaseinstaticallytypedlanguages.
Assignmentcanalsocausesharingofanobject.Example:
obj1=A()
obj2=obj1

Checktodeterminethatthesameobjectissharedwithid(obj)ortheis
operator.Example:
In[23]:a=range(10)
In[24]:a
Out[24]:[0,1,2,3,4,5,6,7,8,9]
In[25]:b=a
In[26]:b

Page38

APythonBook
Out[26]:[0,1,2,3,4,5,6,7,8,9]
In[27]:b[3]=333
In[28]:b
Out[28]:[0,1,2,333,4,5,6,7,8,9]
In[29]:a
Out[29]:[0,1,2,333,4,5,6,7,8,9]
In[30]:aisb
Out[30]:True
In[31]:printid(a),id(b)
3103792031037920

Youcanalsodomultipleassignmentinasinglestatement.Example:
In[32]:a=b=123
In[33]:a
Out[33]:123
In[34]:b
Out[34]:123
In[35]:
In[35]:
In[35]:a=b=[11,22]
In[36]:aisb
Out[36]:True

Youcaninterchange(swap)thevalueoftwovariablesusingassignmentand
packing/unpacking:
>>>a=111
>>>b=222
>>>a,b=b,a
>>>a
222
>>>b
111

1.6.2importstatement
Makemodule(orobjectsinthemodule)available.
Whatimportdoes:

Evaluatethecontentofamodule.
Likelytocreatevariablesinthelocal(module)namespace.
Evaluationofaspecificmoduleonlyhappensonceduringagivenrunofthe
program.Therefore,amoduleissharedacrossanapplication.
Amoduleisevaluatedfromtoptobottom.Laterstatementscanreplacevalues
createdearlier.Thisistrueoffunctionsandclasses,aswellas(other)variables.
Whichstatementsareevaluated?Assignment,class,def,...
Usethefollowingidiomtomakeamodulebothrunableandimportable:
if__name__=='__main__':

Page39

APythonBook
#importpdb;pdb.set_trace()
main()#or"test()"orsomeotherfunction
definedinmodule

Notes:
Theaboveconditionwillbetrueonlywhenthemoduleisrunasascriptand
willnotbetruewhenthemoduleisimported.
Thelinecontainingpdbcanbecopiedanyplaceinyourprogramand
uncommented,andthentheprogramwilldropintothePythondebugger
whenthatlocationisreached.
Whereimportlooksformodules:

sys.pathshowswhereitlooks.
Therearesomestandardplaces.
AddadditionaldirectoriesbysettingtheenvironmentvariablePYTHONPATH.
Youcanalsoaddpathsbymodifyingsys.path,forexample:
importsys
sys.path.insert(0,'/path/to/my/module')

Packagesneedafilenamed__init__.py.
ExtensionsTodeterminewhatextensionsimportlooksfor,do:
>>>importimp
>>>imp.get_suffixes()
[('.so','rb',3),('module.so','rb',3),('.py','U',
1),('.pyc','rb',2)]

Formsoftheimportstatement:
importANamesinthelocal(module)namespaceareaccessiblewiththedot
operator.
importAasBImportthemoduleA,butbindthemoduleobjecttothe
variableB.
importA1,A2Notrecommended
fromAimportB
fromAimportB1,B2
fromAimportBasC
fromAimport*Notrecommended:cluttersandmixesnamespaces.
fromA.BimportC(1)PossiblyimportobjectCfrommoduleBin
packageAor(2)possiblyimportmoduleCfromsubpackageBinpackageA.
importA.B.CToreferenceattributesinC,mustusefullyqualifiedname,
forexampleuseA.B.C.DtoreferenceDinsideofC.
Morenotesontheimportstatement:

TheimportstatementandpackagesAfilenamed__init__.pyisrequiredin
apackage.Thisfileisevaluatedthefirsttimeeitherthepackageisimportedora
Page40

APythonBook

fileinthepackageisimported.Question:Whatismadeavailablewhenyoudo
importaPackage?Answer:Allvariables(names)thatareglobalinsidethe
__init__.pymoduleinthatpackage.But,seenotesontheuseof__all__:
Theimportstatementhttp://docs.python.org/ref/import.html
Theuseofif__name__=="__main__":Makesamoduleboth
importableandexecutable.
UsingdotsintheimportstatementFromthePythonlanguagereferencemanual:
"Hierarchicalmodulenames:whenthemodulenamescontains
oneormoredots,themodulesearchpathiscarriedout
differently.Thesequenceofidentifiersuptothelastdotisused
tofindapackage;thefinalidentifieristhensearchedinside
thepackage.Apackageisgenerallyasubdirectoryofa
directoryonsys.paththathasafile__init__.py."

See:Theimportstatementhttp://docs.python.org/ref/import.html
Exercises:

Importamodulefromthestandardlibrary,forexamplere.
Importanelementfromamodulefromthestandardlibrary,forexampleimport
compilefromtheremodule.
CreateasimplePythonpackagewithasinglemoduleinit.Solution:
1. Createadirectorynamedsimplepackageinthecurrentdirectory.
2. Createan(empty)__init__.pyinthenewdirectory.
3. Createansimple.pyinthenewdirectory.
4. Addasimplefunctionnametest1insimple.py.
5. Importusinganyofthefollowing:
>>>importsimplepackage.simple
>>>fromsimplepackageimportsimple
>>>fromsimplepackage.simpleimporttest1
>>>fromsimplepackage.simpleimporttest1asmytest

1.6.3printstatement
printsendsoutputtosys.stdout.Itaddsanewline,unlessanextracommais
added.
Argumentstoprint:

MultipleitemsSeparatedbycommas.
Endwithcommatosuppresscarriagereturn.
Usestringformattingformorecontroloveroutput.
Alsoseevarious"prettyprinting"functionsandmethods,inparticular,pprint.
See3.27pprintDataprettyprinter
Page41

APythonBook
http://docs.python.org/lib/modulepprint.html.
StringformattingArgumentsareatuple.Reference:2.3.6.2StringFormatting
Operationshttp://docs.python.org/lib/typesseqstrings.html.
Canalsousesys.stdout.Notethatacarriagereturnisnotautomaticallyadded.
Example:
>>>importsys
>>>sys.stdout.write('hello\n')

ControllingthedestinationandformatofprintReplacesys.stdoutwithaninstance
ofanyclassthatimplementsthemethodwritetakingoneparameter.Example:
importsys
classWriter:
def__init__(self,file_name):
self.out_file=file(file_name,'a')
defwrite(self,msg):
self.out_file.write('[[%s]]'%msg)
defclose(self):
self.out_file.close()
deftest():
writer=Writer('outputfile.txt')
save_stdout=sys.stdout
sys.stdout=writer
print'hello'
print'goodbye'
writer.close()
#Showtheoutput.
tmp_file=file('outputfile.txt')
sys.stdout=save_stdout
content=tmp_file.read()
tmp_file.close()
printcontent
test()

Thereisanalternativeformoftheprintstatementthattakesafilelikeobject,in
particularanobjectthathasawritemethod.Forexample:
In[1]:outfile=open('tmp.log','w')
In[2]:print>>outfile,'Message#1'
In[3]:print>>outfile,'Message#2'
In[4]:print>>outfile,'Message#3'
In[5]:outfile.close()
In[6]:
In[6]:infile=open('tmp.log','r')
In[7]:forlineininfile:
...:print'Line:',line.rstrip('\n')
...:

Page42

APythonBook
Line:Message#1
Line:Message#2
Line:Message#3
In[8]:infile.close()

FuturedeprecationwarningThereisnoprintstatementinPython3.Thereisaprint
builtinfunction.

1.6.4if:elif:else:statement
Atemplatefortheif:statement:
ifcondition1:
statements
elifcondition2:
statements
elifcondition3:
statements
else:
statements

Theelifandelseclausesareoptional.
ConditionsExpressionsAnythingthatreturnsavalue.Comparewitheval()and
exec.
Truthvalues:
FalseFalse,None,numericzero,theemptystring,anemptycollection(list
ortupleordictionaryor...).
TrueTrueandeverythingelse.
Operators:

andandorNotethatbothandandordoshortcircuitevaluation.
not
isandisnotTheidenticalobject.Cf.aisbandid(a)==id(b).
UsefultotestforNone,forexample:
ifxisNone:
...
ifxisnotNone:
...

inandnotinCanbeusedtotestforexistenceofakeyinadictionaryorfor
thepresenceofavalueinacollection.
Theinoperatortestsforequality,notidentity.
Example:
>>>d={'aa':111,'bb':222}
>>>'aa'ind

Page43

APythonBook
True
>>>'aa'notind
False
>>>'xx'ind
False

Comparisonoperators,forexample==,!=,<,<=,...
Thereisanifexpression.Example:

>>>a='aa'
>>>b='bb'
>>>x='yes'ifa==belse'no'
>>>x
'no'

Notes:
Theelif:clausesandtheelse:clauseareoptional.
Theif:,elif:,andelse:clausesareallheaderlinesinthesensethatthey
areeachfollowedbyanindentedblockofcodeandeachoftheseheaderlines
endswithacolon.(Toputanemptyblockafteroneofthese,oranyother,
statementheaderline,usethepassstatement.It'seffectivelyanoop.)
Parenthesesaroundtheconditioninanif:orelif:arenotrequiredandare
consideredbadform,unlesstheconditionextendsovermultiplelines,inwhich
caseparenthesesarepreferredoveruseofalinecontinuationcharacter(backslash
attheendoftheline).
Exercises:

Writeanifstatementwithanandoperator.
Writeanifstatementwithanoroperator.
Writeanifstatementcontainingbothandandoroperators.

1.6.5for:statement
Iterateoverasequenceoran"iterable"object.
Form:
forxiny:
block

IteratorSomenotesonwhatitmeanstobeiterable:

Aniterableissomethingthatcanbeusedinaniteratorcontext,forexample,ina
for:statement,inalistcomprehension,andinageneratorexpression.
Sequencesandcontainersareiterable.Examples:tuples,lists,strings,
dictionaries.
Instancesofclassesthatobeytheiteratorprotocolareiterable.See
Page44

APythonBook
http://docs.python.org/lib/typeiter.html.
Wecancreateaniteratorobjectwithbuiltinfunctionssuchasiter()and
enumerate().SeeBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlinthePythonstandardlibrary
reference.
Functionsthatusetheyieldstatement,produceaniterator,althoughit'sactually
calledagenerator.
Aniterableimplementstheiteratorinterfaceandsatisfiestheiteratorprotocol.
Theiteratorprotocol:__iter__()andnext()methods.See2.3.5Iterator
Types(http://docs.python.org/lib/typeiter.html).
Testingfor"iterability":
Ifyoucanuseanobjectinafor:statement,it'siterable.
Iftheexpresioniter(obj)doesnotproduceaTypeErrorexception,it's
iterable.
Somewaystoproduceiterators:

iter()andenumerate()See:
http://docs.python.org/lib/builtinfuncs.html.
some_dict.iterkeys(),some_dict.itervalues(),
some_dict.iteritems().
Useasequenceinaniteratorcontext,forexampleinaforstatement.Lists,
tuples,dictionaries,andstringscanbeusedinaniteratorcontexttoproducean
iterator.
GeneratorexpressionsLatestPythononly.Syntacticallylikelist
comprehensions,but(1)surroundedbyparenthesesinsteadofsquarebracketsand
(2)uselazyevaluation.
AclassthatimplementstheiteratorprotocolExample:
classA(object):
def__init__(self):
self.data=[11,22,33]
self.idx=0
def__iter__(self):
returnself
defnext(self):
ifself.idx<len(self.data):
x=self.data[self.idx]
self.idx+=1
returnx
else:
raiseStopIteration
deftest():
a=A()
forxina:

Page45

APythonBook
printx
test()

NotethattheiteratorprotocolchangesinPython3.
Afunctioncontainingayieldstatement.See:
Yieldexpressions
http://docs.python.org/2/reference/expressions.html#yieldexpressions
Theyieldstatement
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
AlsoseeitertoolsmoduleinthePythonstandardlibraryformuchmorehelp
withiterators:itertoolsFunctionscreatingiteratorsforefficientlooping
http://docs.python.org/2/library/itertools.html#moduleitertools
Thefor:statementcanalsodounpacking.Example:

In[25]:items=['apple','banana','cherry','date']
In[26]:foridx,iteminenumerate(items):
....:print'%d.%s'%(idx,item,)
....:
0.apple
1.banana
2.cherry
3.date

Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.
Helpfulfunctionswithfor:

enumerate(iterable)Returnsaniterablethatproducespairs(tuples)
containingcount(index)andvalue.Example:
>>>foridx,valueinenumerate([11,22,33]):
...printidx,value
...
011
122
233

range([start,]stop[,step])andxrange([start,]stop[,
step]).
ListcomprehensionsSincelistcomprehensionscreatelists,theyareusefulinfor
statements,although,whenthenumberofelementsislarge,youshouldconsiderusinga
generatorexpressioninstead.Alistcomprehensionlooksabitlikeafor:statement,but
isinsidesquarebrackets,anditisanexpression,notastatement.Twoforms(among
others):

[f(x)forxiniterable]
Page46

APythonBook
[f(x)forxiniterableift(x)]
GeneratorexpressionsAgeneratorexpressionlookssimilartoalistcomprehension,
exceptthatitissurroundedbyparenthesesratherthansquarebrackets.Example:

In[28]:items=['apple','banana','cherry','date']
In[29]:gen1=(item.upper()foriteminitems)
In[30]:forxingen1:
....:print'x:',x
....:
x:APPLE
x:BANANA
x:CHERRY
x:DATE

Exercises:

Writealistcomprehensionthatreturnsallthekeysinadictionarywhose
associatedvaluesaregreaterthanzero.
Thedictionary:{'aa':11,'cc':33,'dd':55,'bb':22}
Solution:[x[0]forxinmy_dict.iteritems()ifx[1]>
0]
Writealistcomprehensionthatproducesevenintegersfrom0to10.Useafor
statementtoiterateoverthosevalues.Solution:
forxin[yforyinrange(10)ify%2==0]:
print'x:%s'%x

Writealistcomprehensionthatiteratesovertwolistsandproducesallthe
combinationsofitemsfromthelists.Solution:
In[19]:a=range(4)
In[20]:b=[11,22,33]
In[21]:a
Out[21]:[0,1,2,3]
In[22]:b
Out[22]:[11,22,33]
In[23]:c=[(x,y)forxinaforyinb]
In[24]:printc
[(0,11),(0,22),(0,33),(1,11),(1,22),(1,33),
(2,11),(2,22),(2,33),(3,11),(3,22),(3,33)]

But,notethatinthepreviousexercise,ageneratorexpressionwouldoftenbebetter.A
generatorexpressionislikealistcomprehension,exceptthat,insteadofcreatingthe
entirelist,itproducesageneratorthatcanbeusedtoproduceeachoftheelements.
Thebreakandcontinuestatementsareoftenusefulinaforstatement.Seecontinue
andbreakstatements
Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.Example:
Page47

APythonBook
foritemindata1:
ifitem>100:
value1=item
break
else:
value1='notfound'
print'value1:',value1

Whenrun,itprints:
value1:notfound

1.6.6while:statement
Form:
whilecondition:
block

Thewhile:statementisnotoftenusedinPythonbecausethefor:statementis
usuallymoreconvenient,moreidiomatic,andmorePythonic.
Exercises:

Writeawhilestatementthatprintsintegersfromzeroto5.Solution:
count=0
whilecount<5:
count+=1
printcount

Thebreakandcontinuestatementsareoftenusefulinawhilestatement.See
continueandbreakstatements
Thewhilestatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executedifthewhilestatementcompletesnormally,thatisifabreakstatementisnot
executed.

1.6.7continueandbreakstatements
Thebreakstatementexitsfromaloop.
Thecontinuestatementcausesexecutiontoimmediatelycontinueatthestartofthe
loop.
Canbeusedinfor:andwhile:.
Whenthefor:statementorthewhile:statementhasanelse:clause,theblockin
theelse:clauseisexecutedonlyifabreakstatementisnotexecuted.
Exercises:
Page48

APythonBook

Usingbreak,writeawhilestatementthatprintsintegersfromzeroto5.
Solution:
count=0
whileTrue:
count+=1
ifcount>5:
break
printcount

Notes:
Aforstatementthatusesrange()orxrange()wouldbebetterthana
whilestatementforthisuse.
Usingcontinue,writeawhilestatementthatprocessesonlyevenintegers
from0to10.Note:%isthemodulooperator.Solution:
count=0
whilecount<10:
count+=1
ifcount%2==0:
continue
printcount

1.6.8try:except:statement
Exceptionsareasystematicandconsistentwayofprocessingerrorsand"unusual"events
inPython.
CaughtanduncaughtexceptionsUncaughtexceptionsterminateaprogram.
Thetry:statementcatchesanexception.
AlmostallerrorsinPythonareexceptions.
Evaluation(executionmodel)ofthetrystatementWhenanexceptionoccursinthe
tryblock,evenifinsideanestedfunctioncall,executionofthetryblockendsandthe
exceptclausesaresearchedforamatchingexception.
TracebacksAlsoseethetracebackmodule:
http://docs.python.org/lib/moduletraceback.html
Exceptionsareclasses.
Exceptionclassessubclassing,args.
Anexceptionclassinanexcept:clausecatchesinstancesofthatexceptionclassand
allsubclasses,butnotsuperclasses.
BuiltinexceptionclassesSee:

Moduleexceptions.
Page49

APythonBook
Builtinexceptionshttp://docs.python.org/lib/moduleexceptions.html.
UserdefinedexceptionclassessubclassesofException.

Example:
try:
raiseRuntimeError('thissillyerror')
exceptRuntimeError,exp:
print"[[[%s]]]"%exp

Reference:http://docs.python.org/lib/moduleexceptions.html
Youcanalsogettheargumentspassedtotheconstructorofanexceptionobject.Inthe
aboveexample,thesewouldbe:
exp.args

Whywouldyoudefineyourownexceptionclass?Oneanswer:Youwantauserofyour
codetocatchyourexceptionandnoothers.
Catchinganexceptionbyexceptionclasscatchesexceptionsofthatclassandallits
subclasses.So:
exceptSomeExceptionClass,exp:

matchesandcatchesanexceptionifSomeExceptionClassistheexceptionclassorabase
class(superclass)oftheexceptionclass.Theexceptionobject(usuallyaninstanceof
someexceptionclass)isboundtoexp.
Amore"modern"syntaxis:
exceptSomeExceptionClassasexp:

So:
classMyE(ValueError):
pass
try:
raiseMyE()
exceptValueError:
print'caughtexception'

willprint"caughtexception",becauseValueErrorisabaseclassofMyE.
Alsoseetheentriesfor"EAFP"and"LBYL"inthePythonglossary:
http://docs.python.org/3/glossary.html.
Exercises:

Writeaverysimple,emptyexceptionsubclass.Solution:
classMyE(Exception):

Page50

APythonBook
pass

Writeatry:except:statementthatraisesyourexceptionandalsocatchesit.
Solution:
try:
raiseMyE('hellotheredave')
exceptMyE,e:
printe

1.6.9raisestatement
Throworraiseanexception.
Forms:
raiseinstance
raiseMyExceptionClass(value)preferred.
raiseMyExceptionClass,value
Theraisestatementtakes:

An(instanceof)abuiltinexceptionclass.
AninstanceofclassExceptionor
AninstanceofabuiltinsubclassofclassExceptionor
AninstanceofauserdefinedsubclassofclassExceptionor
Oneoftheaboveclassesand(optionally)avalue(forexample,astringora
tuple).
Seehttp://docs.python.org/ref/raise.html.

Foralistofbuiltinexceptions,seehttp://docs.python.org/lib/moduleexceptions.html.
Thefollowingexampledefinesanexceptionsubclassandthrowsaninstanceofthat
subclass.Italsoshowshowtopassandcatchmultipleargumentstotheexception:
classNotsobadError(Exception):
pass
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:%s'%(e.args,)
test(0)

Whichprintsoutthefollowing:
Errorargs:('amoderatelybaderror','nottoobad')

Page51

APythonBook
Notes:
Inordertopassinmultipleargumentswiththeexception,weuseatuple,orwe
passmultipleargumentstotheconstructor.
Thefollowingexampledoesasmallamountofprocessingofthearguments:

classNotsobadError(Exception):
"""Anexceptionclass.
"""
defget_args(self):
return'::::'.join(self.args)
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:{{{%s}}}'%(e.get_args(),)
test(0)

1.6.10with:statement
Thewithstatementenablesustouseacontextmanager(anyobjectthatsatisfiesthe
contextmanagerprotocol)toaddcodebefore(onentryto)andafter(onexitfrom)a
blockofcode.
1.6.10.1Writingacontextmanager

Acontextmanagerisaninstanceofaclassthatsatisfiesthisinterface:
classContext01(object):
def__enter__(self):
pass
def__exit__(self,exc_type,exc_value,traceback):
pass

Hereisanexamplethatusestheabovecontextmanager:
classContext01(object):
def__enter__(self):
print'in__enter__'
return'somevalueorother'#usuallywewanttoreturn
self
def__exit__(self,exc_type,exc_value,traceback):
print'in__exit__'

Notes:

Page52

APythonBook

The__enter__methodiscalledbeforeourblockofcodeisentered.
Usually,butnotalways,wewillwantthe__enter__methodtoreturnself,
thatis,theinstanceofourcontextmanagerclass.Wedothissothatwecanwrite:
withMyContextManager()asobj:
pass

andthenusetheinstance(objinthiscase)inthenestedblock.
The__exit__methodiscalledwhenourblockofcodeisexitedeither
normallyorbecauseofanexception.
Ifanexceptionissupplied,andthemethodwishestosuppresstheexception(i.e.,
preventitfrombeingpropagated),itshouldreturnatruevalue.Otherwise,the
exceptionwillbeprocessednormallyuponexitfromthismethod.
Iftheblockexitsnormally,thevalueofexc_type,exc_value,and
tracebackwillbeNone.
Formoreinformationonthewith:statement,seeContextManagerTypes
http://docs.python.org/2/library/stdtypes.html#contextmanagertypes.

Seemodulecontextlibforstrangewaysofwritingcontextmanagers:
http://docs.python.org/2/library/contextlib.html#modulecontextlib
1.6.10.2Usingthewith:statement

Hereareexamples:
#example1
withContext01():
print'inbody'
#example2
withContext02()asa_value:
print'inbody'
print'a_value:"%s"'%(a_value,)
a_value.some_method_in_Context02()
#example3
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.upper())

Notes:

Intheformwith...asval,thevaluereturnedbythe__enter__
methodisassignedtothevariable(valinthiscase).
Inordertousemorethanonecontextmanager,youcannestwith:statements,
orseparateusesofofthecontextmanagerswithcommas,whichisusually
Page53

APythonBook
preferred.Seeexample3above.

1.6.11del
Thedelstatementcanbeusedto:
Removenamesfromnamespace.
Removeitemsfromacollection.
Ifnameislistedinaglobalstatement,thendelremovesnamefromtheglobal
namespace.

Namescanbea(nested)list.Examples:
>>>dela
>>>dela,b,c

Wecanalsodeleteitemsfromalistordictionary(andperhapsfromotherobjectsthatwe
cansubscript).Examples:
In[9]:d={'aa':111,'bb':222,'cc':333}
In[10]:printd
{'aa':111,'cc':333,'bb':222}
In[11]:deld['bb']
In[12]:printd
{'aa':111,'cc':333}
In[13]:
In[13]:a=[111,222,333,444]
In[14]:printa
[111,222,333,444]
In[15]:dela[1]
In[16]:printa
[111,333,444]

And,wecandeleteanattributefromaninstance.Example:
In[17]:classA:
....:pass
....:
In[18]:a=A()
In[19]:a.x=123
In[20]:dir(a)
Out[20]:['__doc__','__module__','x']
In[21]:printa.x
123
In[22]:dela.x
In[23]:dir(a)
Out[23]:['__doc__','__module__']
In[24]:printa.x

exceptions.AttributeErrorTraceback(mostrecentcalllast)

Page54

APythonBook
/home/dkuhlman/a1/Python/Test/<console>
AttributeError:Ainstancehasnoattribute'x'

1.6.12casestatement
ThereisnocasestatementinPython.Usetheif:statementwithasequenceofelif:
clauses.Or,useadictionaryoffunctions.

1.7Functions,Modules,Packages,andDebugging
1.7.1Functions
1.7.1.1Thedefstatement

Thedefstatementisusedtodefinefunctionsandmethods.
Thedefstatementisevaluated.Itproducesafunction/method(object)andbindsittoa
variableinthecurrentnamespace.
Althoughthedefstatementisevaluated,thecodeinitsnestedblockisnotexecuted.
Therefore,manyerrorsmaynotbedetecteduntileachandeverypaththroughthatcodeis
tested.Recommendations:(1)UseaPythoncodechecker,forexampleflake8or
pylint;(2)DothoroughtestingandusethePythonunittestframework.Pythonic
wisdom:Ifit'snottested,it'sbroken.
1.7.1.2Returningvalues

Thereturnstatementisusedtoreturnvaluesfromafunction.
Thereturnstatementtakeszeroormorevalues,separatedbycommas.Usingcommas
actuallyreturnsasingletuple.
ThedefaultvalueisNone.
Toreturnmultiplevalues,useatupleorlist.Don'tforgetthat(assignment)unpacking
canbeusedtocapturemultiplevalues.Returningmultipleitemsseparatedbycommasis
equivalenttoreturningatuple.Example:
In[8]:deftest(x,y):
...:returnx*3,y*4
...:
In[9]:a,b=test(3,4)
In[10]:printa
9

Page55

APythonBook
In[11]:printb
16

1.7.1.3Parameters

DefaultvaluesExample:
In[53]:deft(max=5):
....:forvalinrange(max):
....:printval
....:
....:
In[54]:t(3)
0
1
2
In[55]:t()
0
1
2
3
4

Givingaparameteradefaultvaluemakesthatparameteroptional.
Note:Ifafunctionhasaparameterwithadefaultvalue,thenall"normal"arguments
mustproceedtheparameterswithdefaultvalues.Morecompletely,parametersmustbe
givenfromlefttorightinthefollowingorder:
1. Normalarguments.
2. Argumentswithdefaultvalues.
3. Argumentlist(*args).
4. Keywordarguments(**kwargs).
Listparameters*args.It'satuple.
Keywordparameters**kwargs.It'sadictionary.
1.7.1.4Arguments

Whencallingafunction,valuesmaybepassedtoafunctionwithpositionalargumentsor
keywordarguments.
Positionalargumentsmustplacedbefore(totheleftof)keywordarguments.
Passingliststoafunctionasmultipleargumentssome_func(*aList).Effectively,
thissyntaxcausesPythontounrollthearguments.Example:
deffn1(*args,**kwargs):
fn2(*args,**kwargs)

Page56

APythonBook
1.7.1.5Localvariables

CreatinglocalvariablesAnybindingoperationcreatesalocalvariable.Examplesare
(1)parametersofafunction;(2)assignmenttoavariableinafunction;(3)theimport
statement;(4)etc.Contrastwithaccessingavariable.
VariablelookupTheLGB/LEGBruleThelocal,enclosing,global,builtinscopes
aresearchedinthatorder.See:http://www.python.org/dev/peps/pep0227/
TheglobalstatementInsideafunction,wemustuseglobalwhenwewanttoset
thevalueofaglobalvariable.Example:
deffn():
globalSome_global_variable,Another_global_variable
Some_global_variable='hello'
...

1.7.1.6Otherthingstoknowaboutfunctions

FunctionsarefirstclassYoucanstoretheminastructure,passthemtoa
function,andreturnthemfromafunction.
Functioncallscantakekeywordarguments.Example:
>>>test(size=25)

Formalparameterstoafunctioncanhavedefaultvalues.Example:
>>>deftest(size=0):
...

Donotusemutableobjectsasdefaultvalues.
Youcan"capture"remainingargumentswith*args,and**kwargs.(Spelling
isnotsignificant.)Example:
In[13]:deftest(size,*args,**kwargs):
....:printsize
....:printargs
....:printkwargs
....:
....:
In[14]:test(32,'aa','bb',otherparam='xyz')
32
('aa','bb')
{'otherparam':'xyz'}

Normalargumentsmustcomebeforedefaultargumentswhichmustcomebefore
keywordarguments.
Afunctionthatdoesnotexplicitlyreturnavalue,returnsNone.
Inordertosetthevalueofaglobalvariable,declarethevariablewithglobal.
Exercises:

Page57

APythonBook

Writeafunctionthattakesasingleargument,printsthevalueoftheargument,
andreturnstheargumentasastring.Solution:
>>>deft(x):
...print'x:%s'%x
...return'[[%s]]'%x
...
>>>t(3)
x:3
'[[3]]'

Writeafunctionthattakesavariablenumberofargumentsandprintsthemall.
Solution:
>>>deft(*args):
...forarginargs:
...print'arg:%s'%arg
...
>>>t('aa','bb','cc')
arg:aa
arg:bb
arg:cc

Writeafunctionthatprintsthenamesandvaluesofkeywordargumentspassedto
it.Solution:
>>>deft(**kwargs):
...forkeyinkwargs.keys():
...print'key:%svalue:%s'%(key,
kwargs[key],)
...
>>>t(arg1=11,arg2=22)
key:arg1value:11
key:arg2value:22

1.7.1.7Globalvariablesandtheglobalstatement

Bydefault,assignmentinafunctionormethodcreateslocalvariables.
Reference(notassignment)toavariable,accessesalocalvariableifithasalreadybeen
created,elseaccessesaglobalvariable.
Inordertoassignavaluetoaglobalvariable,declarethevariableasglobalatthe
beginningofthefunctionormethod.
Ifinafunctionormethod,youbothreferenceandassigntoavariable,thenyoumust
either:
1. Assigntothevariablefirst,or
2. Declarethevariableasglobal.
Theglobalstatementdeclaresoneormorevariables,separatedbycommas,tobe
global.
Page58

APythonBook
Someexamples:
In[1]:
In[1]:X=3
In[2]:deft():
...:printX
...:
In[3]:
In[3]:t()
3
In[4]:defs():
...:X=4
...:
In[5]:
In[5]:
In[5]:s()
In[6]:t()
3
In[7]:X=1
In[8]:defu():
...:globalX
...:X=5
...:
In[9]:
In[9]:u()
In[10]:t()
5
In[16]:defv():
....:x=X
....:X=6
....:returnx
....:
In[17]:
In[17]:v()

Traceback(mostrecentcalllast):
File"<ipythonconsole>",line1,in<module>
File"<ipythonconsole>",line2,inv
UnboundLocalError:localvariable'X'referencedbeforeassignment
In[18]:defw():
....:globalX
....:x=X
....:X=7
....:returnx
....:
In[19]:
In[19]:w()
Out[19]:5
In[20]:X
Out[20]:7

Page59

APythonBook
1.7.1.8Docstringsforfunctions

Adddocstringsasatriplequotedstringbeginningwiththefirstlineofafunctionor
method.Seeepydocforasuggestedformat.
1.7.1.9Decoratorsforfunctions

Adecoratorperformsatransformationonafunction.Examplesofdecoratorsthatare
builtinfunctionsare:@classmethod,@staticmethod,[email protected]:
http://docs.python.org/2/library/functions.html#builtinfunctions
Adecoratorisappliedusingthe"@"characteronalineimmediatelypreceedingthe
functiondefinitionheader.Examples:
classSomeClass(object):
@classmethod
defHelloClass(cls,arg):
pass
##HelloClass=classmethod(HelloClass)
@staticmethod
defHelloStatic(arg):
pass
##HelloStatic=staticmethod(HelloStatic)
#
#Define/implementadecorator.
defwrapper(fn):
definner_fn(*args,**kwargs):
print'>>'
result=fn(*args,**kwargs)
print'<<'
returnresult
returninner_fn
#
#Applyadecorator.
@wrapper
deffn1(msg):
pass
##fn1=wrapper(fn1)

Notes:

Thedecoratorform(withthe"@"character)isequivalenttotheform
(commentedout)thatcallsthedecoratorfunctionexplicitly.
Theuseofclassmethodsandstaticmethodwillbeexplainedlaterinthe
sectiononobjectorientedprogramming.
Adecoratorisimplementedasafunction.Therefore,tolearnaboutsomespecific
Page60

APythonBook

decorator,youshouldsearchforthedocumentationonortheimplementationof
thatfunction.Rememberthatinordertouseafunction,itmustbedefinedinthe
currentmoduleorimportedbythecurrentmoduleorbeabuiltin.
Theformthatexplicitlycallsthedecoratorfunction(commentedoutinthe
exampleabove)isequivalenttotheformusingthe"@"character.

1.7.2lambda
Usealambda,asaconvenience,whenyouneedafunctionthatboth:
isanonymous(doesnotneedaname)and
containsonlyanexpressionandnostatements.
Example:

In[1]:fn=lambdax,y,z:(x**2)+(y*2)+z
In[2]:fn(4,5,6)
Out[2]:32
In[3]:
In[3]:format=lambdaobj,category:'The"%s"isa"%s".'%(obj,
category,)
In[4]:format('pinetree','conifer')
Out[4]:'The"pinetree"isa"conifer".'

Alambdacantakemultipleargumentsandcanreturn(likeafunction)multiplevalues.
Example:
In[79]:a=lambdax,y:(x*3,y*4,(x,y))
In[80]:
In[81]:a(3,4)
Out[81]:(9,16,(3,4))

Suggestion:Insomecases,alambdamaybeusefulasaneventhandler.
Example:
classTest:
def__init__(self,first='',last=''):
self.first=first
self.last=last
deftest(self,formatter):
"""
Testforlambdas.
formatterisafunctiontaking2arguments,firstandlast
names.Itshouldreturntheformattedname.
"""
msg='Mynameis%s'%(formatter(self.first,self.last),)
printmsg
deftest():
t=Test('Dave','Kuhlman')

Page61

APythonBook
t.test(lambdafirst,last:'%s%s'%(first,last,))
t.test(lambdafirst,last:'%s,%s'%(last,first,))
test()

Alambdaenablesustodefine"functions"wherewedonotneednamesforthose
functions.Example:
In[45]:a=[
....:lambdax:x,
....:lambdax:x*2,
....:]
In[46]:
In[46]:a
Out[46]:[<function__main__.<lambda>>,<function__main__.<lambda>>]
In[47]:a[0](3)
Out[47]:3
In[48]:a[1](3)
Out[48]:6

Reference:http://docs.python.org/2/reference/expressions.html#lambda

1.7.3Iteratorsandgenerators
Concepts:
iterator
Anditeratorissomethingthatsatisfiestheiteratorprotocol.Clue:Ifit'saniterator,
youcanuseitinafor:statement.
generator
Ageneratorisaclassorfunctionthatimplementsaniterator,i.e.thatimplementsthe
iteratorprotocol.
theiteratorprotocol
Anobjectsatisfiestheiteratorprotocolifitdoesthefollowing:

Itimplementsa__iter__method,whichreturnsaniteratorobject.
Itimplementsanextfunction,whichreturnsthenextitemfromthe
collection,sequence,stream,etcofitemstobeiteratedover
ItraisestheStopIterationexceptionwhentheitemsareexhaustedand
thenext()methodiscalled.

yield
Theyieldstatementenablesustowritefunctionsthataregenerators.Such
functionsmaybesimilartocoroutines,sincetheymay"yield"multipletimesandare
resumed.
Page62

APythonBook
Formoreinformationoniterators,seethesectiononiteratortypesinthePythonLibrary
Referencehttp://docs.python.org/2/library/stdtypes.html#iteratortypes.
Formoreontheyieldstatement,see:
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
Actually,yieldisanexpression.Formoreonyieldexpressionsandonthenext()
andsend()generatormethods,aswellasothers,see:Yieldexpression
http://docs.python.org/2/reference/expressions.html#yieldexpressionsinthePython
languagereferencemanual.
Afunctionormethodcontainingayieldstatementimplementsagenerator.Addingthe
yieldstatementtoafunctionormethodturnsthatfunctionormethodintoonewhich,
whencalled,returnsagenerator,i.e.anobjectthatimplementstheiteratorprotocol.
Agenerator(afunctioncontainingyield)providesaconvenientwaytoimplementa
filter.But,alsoconsider:
Thefilter()builtinfunction
Listcomprehensionswithanifclause
Hereareafewexamples:

defsimplegenerator():
yield'aaa'#Note1
yield'bbb'
yield'ccc'
deflist_tripler(somelist):
foriteminsomelist:
item*=3
yielditem
deflimit_iterator(somelist,max):
foriteminsomelist:
ifitem>max:
return#Note2
yielditem
deftest():
print'1.',''*30
it=simplegenerator()
foriteminit:
printitem
print'2.',''*30
alist=range(5)
it=list_tripler(alist)
foriteminit:
printitem
print'3.',''*30
alist=range(8)

Page63

APythonBook
it=limit_iterator(alist,4)
foriteminit:
printitem
print'4.',''*30
it=simplegenerator()
try:
printit.next()#Note3
printit.next()
printit.next()
printit.next()
exceptStopIteration,exp:#Note4
print'reachedendofsequence'
if__name__=='__main__':
test()

Notes:
1. Theyieldstatementreturnsavalue.Whenthenextitemisrequestedandthe
iteratoris"resumed",executioncontinuesimmediatelyaftertheyield
statement.
2. Wecanterminatethesequencegeneratedbyaniteratorbyusingareturn
statementwithnovalue.
3. Toresumeagenerator,usethegenerator'snext()orsend()methods.
send()islikenext(),butprovidesavaluetotheyieldexpression.
4. Wecanalternativelyobtaintheitemsinasequencebycallingtheiterator's
next()method.Sinceaniteratorisafirstclassobject,wecansaveitinadata
structureandcanpassitaroundforuseatdifferentlocationsandtimesinour
program.
1. Whenaniteratorisexhaustedorempty,itthrowstheStopIteration
exception,whichwecancatch.
Andhereistheoutputfromrunningtheaboveexample:
$pythontest_iterator.py
1.
aaa
bbb
ccc
2.
0
3
6
9
12
3.
0
1
2
3

Page64

APythonBook
4
4.
aaa
bbb
ccc
reachedendofsequence

Aninstanceofaclasswhichimplementsthe__iter__method,returninganiterator,is
iterable.Forexample,itcanbeusedinaforstatementorinalistcomprehension,orin
ageneratorexpression,orasanargumenttotheiter()builtinmethod.But,notice
thattheclassmostlikelyimplementsageneratormethodwhichcanbecalleddirectly.
ExamplesThefollowingcodeimplementsaniteratorthatproducesalltheobjectsina
treeofobjects:
classNode:
def__init__(self,data,children=None):
self.initlevel=0
self.data=data
ifchildrenisNone:
self.children=[]
else:
self.children=children
defset_initlevel(self,initlevel):self.initlevel=initlevel
defget_initlevel(self):returnself.initlevel
defaddchild(self,child):
self.children.append(child)
defget_data(self):
returnself.data
defget_children(self):
returnself.children
defshow_tree(self,level):
self.show_level(level)
print'data:%s'%(self.data,)
forchildinself.children:
child.show_tree(level+1)
defshow_level(self,level):
print''*level,
#
#Generatormethod#1
#Thisgeneratorturnsinstancesofthisclassintoiterable
objects.
#
defwalk_tree(self,level):
yield(level,self,)
forchildinself.get_children():
forlevel1,tree1inchild.walk_tree(level+1):
yieldlevel1,tree1
def__iter__(self):
returnself.walk_tree(self.initlevel)

Page65

APythonBook
#
#Generatormethod#2
#Thisgeneratorusesasupportfunction(walk_list)whichcalls
#thisfunctiontorecursivelywalkthetree.
#Ifeffect,thisiteratesoverthesupportfunction,which
#iteratesoverthisfunction.
#
defwalk_tree(tree,level):
yield(level,tree)
forchildinwalk_list(tree.get_children(),level+1):
yieldchild
defwalk_list(trees,level):
fortreeintrees:
fortreeinwalk_tree(tree,level):
yieldtree
#
#Generatormethod#3
#Thisgeneratorislikemethod#2,butcallsitself(asan
iterator),
#ratherthancallingasupportfunction.
#
defwalk_tree_recur(tree,level):
yield(level,tree,)
forchildintree.get_children():
forlevel1,tree1inwalk_tree_recur(child,level+1):
yield(level1,tree1,)
defshow_level(level):
print''*level,
deftest():
a7=Node('777')
a6=Node('666')
a5=Node('555')
a4=Node('444')
a3=Node('333',[a4,a5])
a2=Node('222',[a6,a7])
a1=Node('111',[a2,a3])
initLevel=2
a1.show_tree(initLevel)
print'='*40
forlevel,iteminwalk_tree(a1,initLevel):
show_level(level)
print'item:',item.get_data()
print'='*40
forlevel,iteminwalk_tree_recur(a1,initLevel):
show_level(level)
print'item:',item.get_data()

Page66

APythonBook
print'='*40
a1.set_initlevel(initLevel)
forlevel,itemina1:
show_level(level)
print'item:',item.get_data()
iter1=iter(a1)
printiter1
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
##printiter1.next()
returna1
if__name__=='__main__':
test()

Notes:

AninstanceofclassNodeis"iterable".Itcanbeuseddirectlyinafor
statement,alistcomprehension,etc.So,forexample,whenaninstanceofNode
isusedinaforstatement,itproducesaniterator.
WecouldalsocalltheNode.walk_methoddirectlytoobtainaniterator.
MethodNode.walk_treeandfunctionswalk_treeand
walk_tree_recuraregenerators.Whencalled,theyreturnaniterator.They
dothisbecausetheyeachcontainayieldstatement.
Thesemethods/functionsarerecursive.Theycallthemselves.Sincetheyare
generators,theymustcallthemselvesinacontextthatusesaniterator,for
exampleinaforstatement.

1.7.4Modules
AmoduleisaPythonsourcecodefile.
Amodulecanbeimported.Whenimported,themoduleisevaluated,andamoduleobject
iscreated.Themoduleobjecthasattributes.Thefollowingattributesareofspecial
interest:
__doc__Thedocstringofthemodule.
__name__Thenameofthemodulewhenthemoduleisimported,butthe
string"__main__"whenthemoduleisexecuted.
Othernamesthatarecreated(bound)inthemodule.
Amodulecanberun.

Tomakeamodulebothimportableandrunable,usethefollowingidiom(attheendof
Page67

APythonBook
themodule):
defmain():
o
o
o
if__name__=='__main__':
main()

WherePythonlooksformodules:
Seesys.path.
Standardplaces.
EnvironmentvariablePYTHONPATH.
Notesaboutmodulesandobjects:

Amoduleisanobject.
Amodule(object)canbeshared.
Aspecificmoduleisimportedonlyonceinasinglerun.Thismeansthatasingle
moduleobjectissharedbyallthemodulesthatimportit.

1.7.4.1Docstringsformodules

Adddocstringsasatriplequotedstringatornearthetopofthefile.Seeepydocfora
suggestedformat.

1.7.5Packages
Apackageisadirectoryonthefilesystemwhichcontainsafilenamed__init__.py.
The__init__.pyfile:

Whyisitthere?Itmakesmodulesinthedirectory"importable".
Can__init__.pybeempty?Yes.Or,justincludeacomment.
Whenisitevaluated?Itisevaluatedthefirsttimethatanapplicationimports
anythingfromthatdirectory/package.
Whatcanyoudowithit?Anycodethatshouldbeexecutedexactlyonceand
duringimport.Forexample:
Performinitializationneededbythepackage.
Makevariables,functions,classes,etcavailable.Forexample,whenthe
packageisimportedratherthanmodulesinthepackage.Youcanalsoexpose
objectsdefinedinmodulescontainedinthepackage.
Defineavariablenamed__all__tospecifythelistofnamesthatwillbe
importedbyfrommy_packageimport*.Forexample,ifthefollowingis
presentinmy_package/__init__.py:
Page68

APythonBook
__all__=['func1','func2',]

Then,frommy_packageimport*willimportfunc1andfunc2,but
notothernamesdefinedinmy_package.
Notethat__all__canbeusedatthemodulelevel,aswellasatthepackage
level.
Formoreinformation,seethesectiononpackagesinthePythontutorial:
http://docs.python.org/2/tutorial/modules.html#packages.
Guidanceandsuggestions:

"Flatisbetter"Usethe__init__.pyfiletopresenta"flat"viewoftheAPI
foryourcode.Enableyouruserstodoimportmypackageandthen
reference:
mypackage.item1
mypackage.item2
mypackage.item3
Etc.
Whereitem1,item2,etccomposetheAPIyouwantyouruserstouse,even
thoughtheimplementationoftheseitemsmaybeburieddeepinyourcode.
Usethe__init__.pymoduletopresenta"clean"API.Presentonlytheitems
thatyouintendyouruserstouse,andbydoingso,"hide"itemsyoudonotintend
themtouse.

1.8Classes
Classesmodelthebehaviorofobjectsinthe"real"world.Methodsimplementthe
behaviorsofthesetypesofobjects.Membervariableshold(current)state.Classesenable
ustoimplementnewdatatypesinPython.
Theclass:statementisusedtodefineaclass.Theclass:statementcreatesaclass
objectandbindsittoaname.

1.8.1Asimpleclass
In[104]:classA:
.....:pass
.....:
In[105]:a=A()

Todefineanewstyleclass(recommended),inheritfromobjectorfromanotherclass
thatdoes.Example:
In[21]:classA(object):
....:pass
....:

Page69

APythonBook
In[22]:
In[22]:a=A()
In[23]:a
Out[23]:<__main__.Aobjectat0x82fbfcc>

1.8.2Definingmethods
Amethodisafunctiondefinedinclassscopeandwithfirstparameterself:
In[106]:classB(object):
.....:defshow(self):
.....:print'hellofromB'
.....:
In[107]:b=B()
In[108]:b.show()
hellofromB

Amethodaswedescribeithereismoreproperlycalledaninstancemethod,inorderto
distinguishitfromclassmethodsandstaticmethods.

1.8.3Theconstructor
Theconstructorisamethodnamed__init__.
Exercise:Defineaclasswithamembervariablenameandashowmethod.Useprint
toshowthename.Solution:
In[109]:classA(object):
.....:def__init__(self,name):
.....:self.name=name
.....:defshow(self):
.....:print'name:"%s"'%self.name
.....:
In[111]:a=A('dave')
In[112]:a.show()
name:"dave"

Notes:

Theselfvariableisexplicit.Itreferencesthecurrentobject,thatistheobject
whosemethodiscurrentlyexecuting.
Thespelling("self")isoptional,buteveryonespellsitthatway.

1.8.4Membervariables
DefiningmembervariablesMembervariablesarecreatedwithassignment.Example:
classA(object):
def__init__(self,name):

Page70

APythonBook
self.name=name

AsmallgotchaDothis:
In[28]:classA(object):
....:def__init__(self,items=None):
....:ifitemsisNone:
....:self.items=[]
....:else:
....:self.items=items

Donotdothis:
In[29]:classB:
....:def__init__(self,items=[]):#wrong.listctor
evaluatedonlyonce.
....:self.items=items

Inthesecondexample,thedefstatementandthelistconstructorareevaluatedonly
once.Therefore,theitemmembervariableofallinstancesofclassB,willsharethesame
value,whichismostlikelynotwhatyouwant.

1.8.5Callingmethods

Usetheinstanceandthedotoperator.
Callingamethoddefinedinthesameclassorasuperclass:
FromoutsidetheclassUsetheinstance:
some_object.some_method()
an_array_of_of_objects[1].another_method()

FromwithinthesameclassUseself:
self.a_method()

Fromwithasubclasswhenthemethodisinthesuperclassandthereisa
methodwiththesamenameinthecurrentclassUsetheclass(name)oruse
super:
SomeSuperClass.__init__(self,arg1,arg2)
super(CurrentClass,
self).__init__(arg1,arg2)

CallingamethoddefinedinaspecificsuperclassUsetheclass(name).

1.8.6Addinginheritance
ReferencingsuperclassesUsethebuiltinsuperortheexplicitnameofthe
superclass.Useofsuperispreferred.Forexample:
In[39]:classB(A):

Page71

APythonBook
....:def__init__(self,name,size):
....:super(B,self).__init__(name)
....:#A.__init__(self,name)#anolderalternative
form
....:self.size=size

Theuseofsuper()maysolveproblemssearchingforthebaseclasswhenusing
multipleinheritance.Abettersolutionistonotusemultipleinheritance.
Youcanalsousemultipleinheritance.But,itcancauseconfusion.Forexample,inthe
following,classCinheritsfrombothAandB:
classC(A,B):
...

PythonsearchessuperclassesMRO(methodresolutionorder).Ifonlysingleinheritance
isinvolved,thereislittleconfusion.Ifmultipleinheritanceisbeingused,thesearchorder
ofsuperclassescangetcomplexseehere:
http://www.python.org/download/releases/2.3/mro
Formoreinformationoninheritance,seethetutorialinthestandardPython
documentationset:9.5Inheritanceand9.5.1MultipleInheritance.
Watchoutforproblemswithinheritingfrommultipleclassesthathaveacommonbase
class.

1.8.7Classvariables

Alsocalledstaticdata.
Aclassvariableissharedbyinstancesoftheclass.
Defineatclasslevelwithassignment.Example:
classA(object):
size=5
defget_size(self):
returnA.size

Referencewithclassname.variable.
Caution:self.variable=xcreatesanewmembervariable.

1.8.8Classmethodsandstaticmethods
Instance(plain)methods:
Aninstancemethodreceivestheinstanceasitsfirstargument.
Classmethods:

Aclassmethodreceivestheclassasitsfirstargument.
Defineclassmethodswithbuiltinfunctionclassmethod()orwithdecorator
Page72

APythonBook
@classmethod.
Seethedescriptionofclassmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#classmethod
Staticmethods:
Astaticmethodreceivesneithertheinstancenortheclassasitsfirstargument.
Definestaticmethodswithbuiltinfunctionstaticmethod()orwith
decorator@staticmethod.
Seethedescriptionofstaticmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#staticmethod
Notesondecorators:

Adecoratoroftheform@afuncisthesameasm=afunc(m).So,this:
@afunc
defm(self):pass

isthesameas:
defm(self):pass
m=afunc(m)

Youcanusedecorators@classmethodand@staticmethod(insteadofthe
classmethod()andstaticmethod()builtinfunctions)todeclareclass
methodsandstaticmethods.
Example:

classB(object):
Count=0
defdup_string(x):
s1='%s%s'%(x,x,)
returns1
dup_string=staticmethod(dup_string)
@classmethod
defshow_count(cls,msg):
print'%s%d'%(msg,cls.Count,)
deftest():
printB.dup_string('abcd')
B.show_count('hereisthecount:')

Analternativewaytoimplement"staticmethods"Usea"plain",modulelevel
function.Forexample:
In[1]:definc_count():
...:A.count+=1
...:
In[2]:

Page73

APythonBook
In[2]:defdec_count():
...:A.count=1
...:
In[3]:
In[3]:classA:
...:count=0
...:defget_count(self):
...:returnA.count
...:
In[4]:
In[4]:a=A()
In[5]:a.get_count()
Out[5]:0
In[6]:
In[6]:
In[6]:inc_count()
In[7]:inc_count()
In[8]:a.get_count()
Out[8]:2
In[9]:
In[9]:b=A()
In[10]:b.get_count()
Out[10]:2

1.8.9Properties
Thepropertybuiltinfunctionenablesustowriteclassesinawaythatdoesnotrequirea
useroftheclasstousegettersandsetters.Example:
classTestProperty(object):
def__init__(self,description):
self._description=description
def_set_description(self,description):
print'settingdescription'
self._description=description
def_get_description(self):
print'gettingdescription'
returnself._description
description=property(_get_description,_set_description)

Thepropertybuiltinfunctionisalsoadecorator.So,thefollowingisequivalentto
theaboveexample:
classTestProperty(object):
def__init__(self,description):
self._description=description
@property
defdescription(self):
print'gettingdescription'
returnself._description

Page74

APythonBook
@description.setter
defdescription(self,description):
print'settingdescription'
self._description=description

Notes:
Wemarktheinstancevariableasprivatebyprefixingitwithandunderscore.
Thenameoftheinstancevariableandthenameofthepropertymustbedifferent.
Iftheyarenot,wegetrecursionandanerror.
Formoreinformationonproperties,seeBuiltinFunctionsproperties
http://docs.python.org/2/library/functions.html#property

1.8.10Interfaces
InPython,toimplementaninterfaceistoimplementamethodwithaspecificnameanda
specificarguments.
"Ducktyping"Ifitwalkslikeaduckandquackslikeaduck...
Onewaytodefinean"interface"istodefineaclasscontainingmethodsthathavea
headerandadocstringbutnoimplementation.
Additionalnotesoninterfaces:

Interfacesarenotenforced.
Aclassdoesnothavetoimplementallofaninterface.

1.8.11Newstyleclasses
Anewstyleclassisonethatsubclassesobjectoraclassthatsubclassesobject(that
is,anothernewstyleclass).
YoucansubclassPython'sbuiltindatatypes.

Asimpleexamplethefollowingclassextendsthelistdatatype:
classC(list):
defget_len(self):
returnlen(self)
c=C((11,22,33))
c.get_len()
c=C((11,22,33,44,55,66,77,88))
printc.get_len()
#Prints"8".

Aslightlymorecomplexexamplethefollowingclassextendsthedictionary
Page75

APythonBook
datatype:
classD(dict):
def__init__(self,data=None,name='no_name'):
ifdataisNone:
data={}
dict.__init__(self,data)
self.name=name
defget_len(self):
returnlen(self)
defget_keys(self):
content=[]
forkeyinself:
content.append(key)
contentstr=','.join(content)
returncontentstr
defget_name(self):
returnself.name
deftest():
d=D({'aa':111,'bb':222,'cc':333})
#Prints"3"
printd.get_len()
#Prints"'aa,cc,bb'"
printd.get_keys()
#Prints"no_name"
printd.get_name()

Somethingstorememberaboutnewstyleclasses:
Inordertobenewstyle,aclassmustinherit(directlyorindirectly)from
object.Notethatifyouinheritfromabuiltintype,yougetthisautomatically.
Newstyleclassesunifytypesandclasses.
Youcansubclass(builtin)typessuchasdict,str,list,file,etc.
Thebuiltintypesnowprovidefactoryfunctions:dict(),str(),int(),
file(),etc.
ThebuiltintypesareintrospectableUsex.__class__,
dir(x.__class__),isinstance(x,list),etc.
Newstyleclassesgiveyoupropertiesanddescriptors.
Newstyleclassesenableyoutodefinestaticmethods.Actually,allclassesenable
youtodothis.
Anewstyleclassisauserdefinedtype.Foraninstanceofanewstyleclassx,
type(x)isthesameasx.__class__.
Formoreonnewstyleclasses,see:http://www.python.org/doc/newstyle/

Exercises:

Writeaclassandasubclassofthisclass.
Givethesuperclassonemembervariable,aname,whichcanbeenteredwhen
Page76

APythonBook
aninstanceisconstructed.
Givethesubclassonemembervariable,adescription;thesubclassconstructor
shouldallowentryofbothnameanddescription.
Putashow()methodinthesuperclassandoverridetheshow()methodin
thesubclass.
Solution:
classA(object):
def__init__(self,name):
self.name=name
defshow(self):
print'name:%s'%(self.name,)
classB(A):
def__init__(self,name,desc):
A.__init__(self,name)
self.desc=desc
defshow(self):
A.show(self)
print'desc:%s'%(self.desc,)

1.8.12Docstringsforclasses
Adddocstringsasa(triplequoted)stringbeginningwiththefirstlineofaclass.See
epydocforasuggestedformat.

1.8.13Privatemembers
Addanleadingunderscoretoamembername(methodordatavariable)to"suggest"that
thememberisprivate.

1.9SpecialTasks
1.9.1Debuggingtools
pdbThePythondebugger:

Startthedebuggerbyrunninganexpression:
pdb.run('expression')

Example:
if__name__=='__main__':
importpdb
pdb.run('main()')

Startupthedebuggerataspecificlocationwiththefollowing:
Page77

APythonBook
importpdb;pdb.set_trace()

Example:
if__name__=='__main__':
importpdb
pdb.set_trace()
main()

Gethelpfromwithinthedebugger.Forexample:
(Pdb)help
(Pdb)helpnext

CanalsoembedIPythonintoyourcode.See
http://ipython.scipy.org/doc/manual/manual.html.
ipdbAlsoconsiderusingipdb(andIPython).Theipdbdebuggerinteractive
prompthassomeadditionalfeatures,forexample,itdoestabnamecompletion.
Inspecting:
importinspect
Seehttp://docs.python.org/lib/moduleinspect.html.
Don'tforgettotrydir(obj)andtype(obj)andhelp(obj),first.
Miscellaneoustools:

id(obj)
globals()andlocals().
dir(obj)Returnsinterestingnames,butlistisnotnecessarilycomplete.
obj.__class__
cls.__bases__
obj.__class__.__bases__
obj.__doc__.Butusually,help(obj)isbetter.Itproducesthedocstring.
Customizetherepresentationofyourclass.Definethefollowingmethodsinyour
class:
__repr__()Calledby(1)repr(),(2)interactiveinterpreterwhen
representationisneeded.
__str__()Calledby(1)str(),(2)stringformatting.
pdbisimplementedwiththecmdmoduleinthePythonstandardlibrary.Youcan
implementsimilarcommandlineinterfacesbyusingcmd.See:cmdSupportfor
lineorientedcommandinterpretershttp://docs.python.org/lib/modulecmd.html.

1.9.2Fileinputandoutput
Createafileobject.Useopen().
Thisexamplereadsandprintseachlineofafile:
Page78

APythonBook
deftest():
f=file('tmp.py','r')
forlineinf:
print'line:',line.rstrip()
f.close()
test()

Notes:

Atextfileisaniterable.Ititeratesoverthelinesinafile.Thefollowingisa
commonidiom:
infile=file(filename,'r')
forlineininfile:
process_a_line(line)
infile.close()

string.rstrip()stripsnewlineandotherwhitespacefromtherightsideof
eachline.Tostripnewlinesonly,butnototherwhitespace,tryrstrip('\n').
Otherwaysofreadingfromafile/streamobject:my_file.read(),
my_file.readline(),my_file.readlines(),
Thisexamplewriteslinesoftexttoafile:

deftest():
f=file('tmp.txt','w')
forchin'abcdefg':
f.write(ch*10)
f.write('\n')
f.close()
test()

Notes:
Thewritemethod,unliketheprintstatement,doesnotautomaticallyadd
newlinecharacters.
Mustclosefileinordertoflushoutput.Or,usemy_file.flush().
And,don'tforgetthewith:statement.Itmakesclosingfilesautomatic.Thefollowing
exampleconvertsallthevowelsinaninputfiletouppercaseandwritestheconverted
linestoanoutputfile:

importstring
defshow_file(infilename,outfilename):
tran_table=string.maketrans('aeiou','AEIOU')
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.translate(tran_table))

Page79

APythonBook

1.9.3Unittests
Formoredocumentationontheunittestframework,seeunittestUnittesting
frameworkhttp://docs.python.org/2/library/unittest.html#moduleunittest
ForhelpandmoreinformationdothefollowingatthePythoninteractiveprompt:
>>>importunittest
>>>help(unittest)

And,youcanreadthesource:Lib/unittest.pyinthePythonstandardlibrary.
1.9.3.1Asimpleexample

Hereisaverysimpleexample.Youcanfindmoreinformationaboutthisprimitiveway
ofstructuringunittestsinthelibrarydocumentationfortheunittestmoduleBasic
examplehttp://docs.python.org/lib/minimalexample.html
importunittest
classUnitTests02(unittest.TestCase):
deftestFoo(self):
self.failUnless(False)
classUnitTests01(unittest.TestCase):
deftestBar01(self):
self.failUnless(False)
deftestBar02(self):
self.failUnless(False)
defmain():
unittest.main()
if__name__=='__main__':
main()

Notes:

Thecalltounittest.main()runsalltestsinalltestfixturesinthemodule.It
actuallycreatesaninstanceofclassTestPrograminmodule
Lib/unittest.py,whichautomaticallyrunstests.
Testfixturesareclassesthatinheritfromunittest.TestCase.
Withinatestfixture(aclass),thetestsareanymethodswhosenamesbeginwith
theprefix"test".
Inanytest,wecheckforsuccessorfailurewithinheritedmethodssuchas
failIf(),failUnless(),assertNotEqual(),etc.Formoreonthese
Page80

APythonBook

methods,seethelibrarydocumentationfortheunittestmoduleTestCase
Objectshttp://docs.python.org/lib/testcaseobjects.html.
Ifyouwanttochange(1)thetestmethodprefixor(2)thefunctionusedtosort
(theorderof)executionoftestswithinatestfixture,thenyoucancreateyourown
instanceofclassunittest.TestLoaderandcustomizeit.Forexample:
defmain():
my_test_loader=unittest.TestLoader()
my_test_loader.testMethodPrefix='check'
my_test_loader.sortTestMethodsUsing=my_cmp_func
unittest.main(testLoader=my_test_loader)
if__name__=='__main__':
main()

But,seethenotesinsectionAdditionalunittestfeaturesforinstructionsona
(possibly)betterwaytodothis.
1.9.3.2Unittestsuites

Hereisanother,notquitesosimple,example:
#!/usr/bin/envpython
importsys,popen2
importgetopt
importunittest
classGenTest(unittest.TestCase):
deftest_1_generate(self):
cmd='python../generateDS.pyfoout2sup.pysout2sub.py
people.xsd'
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
self.failUnless(len(result)==0)
deftest_2_compare_superclasses(self):
cmd='diffout1sup.pyout2sup.py'
outfile,infile=popen2.popen2(cmd)
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
#print'len(result):',len(result)
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)

Page81

APythonBook
self.failUnless(check_result(result))
deftest_3_compare_subclasses(self):
cmd='diffout1sub.pyout2sub.py'
outfile,infile=popen2.popen2(cmd)
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)
self.failUnless(check_result(result))
defcheck_result(result):
flag1=0
flag2=0
lines=result.split('\n')
len1=len(lines)
iflen1<=5:
flag1=1
s1='\n'.join(lines[:4])
ifs1.find('Generated')>1:
flag2=1
returnflag1andflag2
#Makethetestsuite.
defsuite():
#Thefollowingisobsolete.SeeLib/unittest.py.
#returnunittest.makeSuite(GenTest)
loader=unittest.TestLoader()
#oralternatively
#loader=unittest.defaultTestLoader
testsuite=loader.loadTestsFromTestCase(GenTest)
returntestsuite
#Makethetestsuiteandrunthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
runner.run(testsuite)
USAGE_TEXT="""
Usage:
pythontest.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythontest.py

Page82

APythonBook
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
main()
#importpdb
#pdb.run('main()')

Notes:

GenTestisourtestsuiteclass.Itinheritsfromunittest.TestCase.
EachmethodinGenTestwhosenamebeginswith"test"willberunasatest.
Thetestsareruninalphabeticorderbymethodname.
DefaultsinclassTestLoaderforthetestnameprefixandsortcomparison
functioncanbeoverridden.See5.3.8TestLoaderObjects
http://docs.python.org/lib/testloaderobjects.html.
AtestcaseclassmayalsoimplementmethodsnamedsetUp()and
tearDown()toberunbeforeandaftertests.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.Actually,thefirsttestmethodin
ourexampleshould,perhaps,beasetUp()method.
Thetestsusecallssuchasself.failUnless()toreporterrors.Theseare
inheritedfromclassTestCase.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.
Functionsuite()createsaninstanceofthetestsuite.
Functiontest()runsthetests.

1.9.3.3Additionalunittestfeatures

And,thefollowingexampleshowsseveraladditionalfeatures.Seethenotesthatfollow
Page83

APythonBook
thecode:
importunittest
classUnitTests02(unittest.TestCase):
deftestFoo(self):
self.failUnless(False)
defcheckBar01(self):
self.failUnless(False)
classUnitTests01(unittest.TestCase):
#Note1
defsetUp(self):
print'settingupUnitTests01'
deftearDown(self):
print'tearingdownUnitTests01'
deftestBar01(self):
print'testingtestBar01'
self.failUnless(False)
deftestBar02(self):
print'testingtestBar02'
self.failUnless(False)
deffunction_test_1():
name='mona'
assertnotname.startswith('mo')
defcompare_names(name1,name2):
ifname1<name2:
return1
elifname1>name2:
return1
else:
return0
defmake_suite():
suite=unittest.TestSuite()
#Note2
suite.addTest(unittest.makeSuite(UnitTests01,
sortUsing=compare_names))
#Note3
suite.addTest(unittest.makeSuite(UnitTests02,prefix='check'))
#Note4
suite.addTest(unittest.FunctionTestCase(function_test_1))
returnsuite
defmain():
suite=make_suite()
runner=unittest.TextTestRunner()
runner.run(suite)
if__name__=='__main__':

Page84

APythonBook
main()

Notes:
1. Ifyourunthiscode,youwillnoticethatthesetUpandtearDownmethodsin
classUnitTests01arerunbeforeandaftereachtestinthatclass.
2. Wecancontroltheorderinwhichtestsarerunbypassingacomparefunctionto
themakeSuitefunction.Thedefaultisthecmpbuiltinfunction.
3. Wecancontrolwhichmethodsinatestfixtureareselectedtoberunbypassing
theoptionalargumentprefixtothemakeSuitefunction.
4. Ifwehaveanexistingfunctionthatwewantto"wrap"andrunasaunittest,we
cancreateatestcasefromafunctionwiththeFunctionTestCasefunction.If
wedothat,noticethatweusetheassertstatementtotestandpossiblycause
failure.
1.9.3.4GuidanceonUnitTesting

Whyshouldweuseunittests?Manyreasons,including:
Withoutunittests,cornercasesmaynotbechecked.Thisisespeciallyimportant,
sincePythondoesrelativelylittlecompiletimeerrorchecking.
Unittestsfacilitateafrequentandshortdesignandimplementandrelease
developmentcycle.SeeONLamp.comExtremePython
http://www.onlamp.com/pub/a/python/2001/03/28/pythonnews.htmlandWhatis
XPhttp://www.xprogramming.com/what_is_xp.htm.
Designingthetestsbeforewritingthecodeis"agoodidea".
Additionalnotes:

Inatestclass,instancemethodssetUpandtearDownarerunautomatically
beforeeachandaftereachindividualtest.
Inatestclass,classmethodssetUpClassandtearDownClassarerun
automaticallyoncebeforeandafterallthetestsinaclass.
ModulelevelfunctionssetUpModuleandtearDownModulearerunbefore
andafteranytestsinamodule.
Insomecasesyoucanalsoruntestsdirectlyfromthecommandline.Dothe
followingforhelp:
$pythonmunittesthelp

1.9.4doctest
Forsimpletestharnesses,considerusingdoctest.Withdoctestyoucan(1)runa
testatthePythoninteractiveprompt,then(2)copyandpastethattestintoadocstringin
yourmodule,andthen(3)runthetestsautomaticallyfromwithinyourmoduleunder
Page85

APythonBook
doctest.
ThereareexamplesandexplanationinthestandardPythondocumentation:5.2doctest
TestinteractivePythonexampleshttp://docs.python.org/lib/moduledoctest.html.
Asimplewaytousedoctestinyourmodule:
1. RunseveraltestsinthePythoninteractiveinterpreter.Notethatbecause
doctestlooksfortheinterpreter's">>>"prompt,youmustusethestandard
interpreter,andnot,forexample,IPython.Also,makesurethatyouincludealine
withthe">>>"promptaftereachsetofresults;thisenablesdoctestto
determinetheextentofthetestresults.
2. Usecopyandpaste,toinsertthetestsandtheirresultsfromyourinteractive
sessionintothedocstrings.
3. Addthefollowingcodeatthebottomofyourmodule:
def_test():
importdoctest
doctest.testmod()
if__name__=="__main__":
_test()

Hereisanexample:
deff(n):
"""
Printsomethingfunny.
>>>f(1)
10
>>>f(2)
10
>>>f(3)
0
"""
ifn==1:
return10
elifn==2:
return10
else:
return0
deftest():
importdoctest,test_doctest
doctest.testmod(test_doctest)
if__name__=='__main__':
test()

And,hereistheoutputfromrunningtheabovetestwiththevflag:
Page86

APythonBook
$pythontest_doctest.pyv
Runningtest_doctest.__doc__
0of0examplesfailedintest_doctest.__doc__
Runningtest_doctest.f.__doc__
Trying:f(1)
Expecting:10
ok
Trying:f(2)
Expecting:10
ok
Trying:f(3)
Expecting:0
ok
0of3examplesfailedintest_doctest.f.__doc__
Runningtest_doctest.test.__doc__
0of0examplesfailedintest_doctest.test.__doc__
2itemshadnotests:
test_doctest
test_doctest.test
1itemspassedalltests:
3testsintest_doctest.f
3testsin3items.
3passedand0failed.
Testpassed.

1.9.5ThePythondatabaseAPI
PythondatabaseAPIdefinesastandardinterfaceforaccesstoarelationaldatabase.
InordertousethisAPIyoumustinstallthedatabaseadapter(interfacemodule)foryour
particulardatabase,e.g.PostgreSQL,MySQL,Oracle,etc.
YoucanlearnmoreaboutthePythonDBAPIhere:
http://www.python.org/dev/peps/pep0249/
Thefollowingsimpleexampleusessqlite3http://docs.python.org/2/library/sqlite3.html
#!/usr/bin/envpython
"""
Createarelationaldatabaseandatableinit.
Addsomerecords.
Readanddisplaytherecords.
"""
importsys
importsqlite3
defcreate_table(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''CREATETABLEplants

Page87

APythonBook
(nametext,desctext,catint)''')
cursor.execute(
'''INSERTINTOplantsVALUES('tomato','redandjuicy',
1)''')
cursor.execute(
'''INSERTINTOplantsVALUES('pepper','greenandcrunchy',
2)''')
cursor.execute('''INSERTINTOplantsVALUES('pepper','purple',
2)''')
con.commit()
con.close()
defretrieve(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''select*fromplants''')
rows=cursor.fetchall()
printrows
print''*40
cursor.execute('''select*fromplants''')
forrowincursor:
printrow
con.close()
deftest():
args=sys.argv[1:]
iflen(args)!=1:
sys.stderr.write('\nusage:test_db.py<db_name>\n\n')
sys.exit(1)
db_name=args[0]
create_table(db_name)
retrieve(db_name)
test()

1.9.6InstallingPythonpackages
Simple:
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot

Morecomplex:

LookforaREADMEorINSTALLfileattherootofthepackage.
Typethefollowingforhelp:
$pythonsetup.pycmdhelp
$pythonsetup.pyhelpcommands
$pythonsetup.pyhelp[cmd1cmd2...]

And,forevenmoredetails,seeInstallingPythonModules
Page88

APythonBook
http://docs.python.org/inst/inst.html
pipisbecomingpopularforinstallingandmanagingPythonpackages.See:
https://pypi.python.org/pypi/pip
Also,considerusingvirtualenv,especiallyifyoususpectorworrythatinstalling
somenewpackagewillalterthebehaviorofapackagecurrentlyinstalledonyour
machine.See:https://pypi.python.org/pypi/virtualenv.virtualenvcreatesadirectory
andsetsupaPythonenvironmentintowhichyoucaninstallandusePythonpackages
withoutchangingyourusualPythoninstallation.

1.10MorePythonFeaturesandExercises
[Astimepermits,explainmorefeaturesanddomoreexercisesasrequestedbyclass
members.]
ThankstoDavidGoodgerforthefollowinglistorreferences.His"CodeLikea
Pythonista:IdiomaticPython"
(http://python.net/~goodger/projects/pycon/2007/idiomatic/)isworthacarefulreading:

"PythonObjects",FredrikLundh,http://www.effbot.org/zone/pythonobjects.htm
"HowtothinklikeaPythonista",MarkHammond,
http://python.net/crew/mwh/hacks/objectthink.html
"Pythonmain()functions",GuidovanRossum,
http://www.artima.com/weblogs/viewpost.jsp?thread=4829
"PythonIdiomsandEfficiency",http://jaynes.colorado.edu/PythonIdioms.html
"Pythontrack:pythonidioms",
http://www.cs.caltech.edu/courses/cs11/material/python/misc/python_idioms.html
"BePythonic",ShalabhChaturvedi,http://shalabh.infogami.com/Be_Pythonic2
"PythonIsNotJava",PhillipJ.Eby,
http://dirtsimple.org/2004/12/pythonisnotjava.html
"WhatisPythonic?",MartijnFaassen,
http://faassen.ntree.net/blog/view/weblog/2005/08/06/0
"SortingMiniHOWTO",AndrewDalke,
http://wiki.python.org/moin/HowTo/Sorting
"PythonIdioms",http://www.gungfu.de/facts/wiki/Main/PythonIdioms
"PythonFAQs",http://www.python.org/doc/faq/

Page89

APythonBook

2Part2AdvancedPython
2.1IntroductionPython201(Slightly)AdvancedPythonTopics
Thisdocumentisintendedasnotesforacourseon(slightly)advancedPythontopics.

2.2RegularExpressions
Formorehelponregularexpressions,see:

reRegularexpressionoperationshttp://docs.python.org/library/re.html
RegularExpressionHOWTOhttp://docs.python.org/howto/regex.html

2.2.1Definingregularexpressions
Aregularexpressionpatternisasequenceofcharactersthatwillmatchsequencesof
charactersinatarget.
Thepatternsorregularexpressionscanbedefinedasfollows:

Literalcharactersmustmatchexactly.Forexample,"a"matches"a".
Concatenatedpatternsmatchconcatenatedtargets.Forexample,"ab"("a"
followedby"b")matches"ab".
Alternatepatterns(separatedbyaverticalbar)matcheitherofthealternative
patterns.Forexample,"(aaa)|(bbb)"willmatcheither"aaa"or"bbb".
Repeatingandoptionalitems:
"abc*"matches"ab"followedbyzeroormoreoccurancesof"c",forexample,
"ab","abc","abcc",etc.
"abc+"matches"ab"followedbyoneormoreoccurancesof"c",forexample,
"abc","abcc",etc,butnot"ab".
"abc?"matches"ab"followedbyzerooroneoccurancesof"c",forexample,
"ab"or"abc".
SetsofcharactersCharactersandsequencesofcharactersinsquarebrackets
formaset;asetmatchesanycharacterinthesetorrange.Forexample,"[abc]"
matches"a"or"b"or"c".And,forexample,"[_az09]"matchesanunderscore
oranylowercaseletteroranydigit.
GroupsParenthesesindicateagroupwithapattern.Forexample,"ab(cd)*ef"is
apatternthatmatches"ab"followedbyanynumberofoccurancesof"cd"
followedby"ef",forexample,"abef","abcdef","abcdcdef",etc.
Therearespecialnamesforsomesetsofcharacters,forexample"\d"(anydigit),
Page90

APythonBook
"\w"(anyalphanumericcharacter),"\W"(anynonalphanumericcharacter),etc.
Moremoreinformation,seePythonLibraryReference:RegularExpression
Syntaxhttp://docs.python.org/library/re.html#regularexpressionsyntax
Becauseoftheuseofbackslashesinpatterns,youareusuallybetteroffdefiningregular
expressionswithrawstrings,e.g.r"abc".

2.2.2Compilingregularexpressions
Whenaregularexpressionistobeusedmorethanonce,youshouldconsidercompiling
it.Forexample:
importsys,re
pat=re.compile('aa[bc]*dd')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
ifpat.search(line):
print'matched:',line
else:
print'nomatch:',line

Comments:

Weimportmodulereinordertouseregularexpresions.
re.compile()compilesaregularexpressionsothatwecanreusethe
compiledregularexpressionwithoutcompilingitrepeatedly.

2.2.3Usingregularexpressions
Usematch()tomatchatthebeginningofastring(ornotatall).
Usesearch()tosearchastringandmatchthefirststringfromtheleft.
Herearesomeexamples:
>>>importre
>>>pat=re.compile('aa[09]*bb')
>>>x=pat.match('aa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9608>
>>>x=pat.match('xxxxaa1234bbccddee')
>>>x
>>>type(x)
<type'NoneType'>
>>>x=pat.search('xxxxaa1234bbccddee')
>>>x

Page91

APythonBook
<_sre.SRE_Matchobjectat0x401e9608>

Notes:

Whenamatchorsearchissuccessful,itreturnsamatchobject.Whenitfails,it
returnsNone.
Youcanalsocallthecorrespondingfunctionsmatchandsearchintheremodule,
e.g.:
>>>x=re.search(pat,'xxxxaa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9560>

Foralistoffunctionsintheremodule,seeModuleContents
http://docs.python.org/library/re.html#modulecontents.

2.2.4Usingmatchobjectstoextractavalue
Matchobjectsenableyoutoextractmatchedsubstringsafterperformingamatch.A
matchobjectisreturnedbysuccessfulmatch.Thepartofthetargetavailableinthematch
objectistheportionmatchedbygroupsinthepattern,thatistheportionofthepattern
insideparentheses.Forexample:
In[69]:mo=re.search(r'height:(\d*)width:(\d*)','height:123
width:456')
In[70]:mo.groups()
Out[70]:('123','456')

Hereisanotherexample:
importsys,re
Targets=[
'Thereare<<25>>sparrows.',
'Isee<<15>>finches.',
'Thereisnothinghere.',
]
deftest():
pat=re.compile('<<([09]*)>>')
forlineinTargets:
mo=pat.search(line)
ifmo:
value=mo.group(1)
print'value:%s'%value
else:
print'nomatch'
test()

Whenweruntheabove,itprintsoutthefollowing:
Page92

APythonBook
value:25
value:15
nomatch

Explanation:
Intheregularexpression,putparenthesesaroundtheportionoftheregular
expressionthatwillmatchwhatyouwanttoextract.Eachpairofparentheses
marksoffagroup.
Afterthesearch,checktodetermineiftherewasasuccessfulmatchbychecking
foramatchingobject."pat.search(line)"returnsNoneifthesearchfails.
Ifyouspecifymorethanonegroupinyourregularexpression(morethatonepair
ofparentheses),thenyoucanuse"value=mo.group(N)"toextractthevalue
matchedbytheNthgroupfromthematchingobject."value=mo.group(1)"
returnsthefirstextractedvalue;"value=mo.group(2)"returnsthesecond;etc.An
argumentof0returnsthestringmatchedbytheentireregularexpression.
Inaddition,youcan:

Use"values=mo.groups()"togetatuplecontainingthestringsmatchedbyall
groups.
Use"mo.expand()"tointerpolatethegroupvaluesintoastring.Forexample,
"mo.expand(r'value1:\1value2:\2')"insertsthevaluesofthefirstandsecond
groupintoastring.Ifthefirstgroupmatched"aaa"andthesecondmatched
"bbb",thenthisexamplewouldproduce"value1:aaavalue2:bbb".Forexample:
In[76]:mo=re.search(r'h:(\d*)w:(\d*)','h:123
w:456')
In[77]:mo.expand(r'Height:\1Width:\2')
Out[77]:'Height:123Width:456'

2.2.5Extractingmultipleitems
Youcanextractmultipleitemswithasinglesearch.Hereisanexample:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
print'value1:%svalue2:%s'%(value1,value2)
else:
print'nomatch'

Page93

APythonBook
Comments:

Usemultipleparenthesizedsubstringsintheregularexpressiontoindicatethe
portions(groups)tobeextracted.
"mo.group(1,2)"returnsthevaluesofthefirstandsecondgroupinthestring
matched.
Wecouldalsohaveused"mo.groups()"toobtainatuplethatcontainsboth
values.
Yetanotheralternativewouldhavebeentousethefollowing:print
mo.expand(r'value1:\1value2:\2').

2.2.6Replacingmultipleitems
Asimplewaytoperformmultiplereplacementsusingaregularexpressionistousethe
re.subn()function.Hereisanexample:
In[81]:re.subn(r'\d+','***','thereare203birdssittingin2
trees')
Out[81]:('thereare***birdssittingin***trees',2)

Formorecomplexreplacements,useafunctioninsteadofaconstantreplacementstring:
importre
defrepl_func(mo):
s1=mo.group(1)
s2='*'*len(s1)
returns2
deftest():
pat=r'(\d+)'
in_str='thereare2034birdsin21trees'
out_str,count=re.subn(pat,repl_func,in_str)
print'in:"%s"'%in_str
print'out:"%s"'%out_str
print'count:%d'%count
test()

Andwhenweruntheabove,itproduces:
in:"thereare2034birdsin21trees"
out:"thereare****birdsin**trees"
count:2

Notes:

Thereplacementfunctionreceivesoneargument,amatchobject.
There.subn()functionreturnsatuplecontainingtwovalues:(1)thestring
afterreplacementsand(2)thenumberofreplacementsperformed.
Page94

APythonBook
HereisanevenmorecomplexexampleYoucanlocatesubstrings(slices)ofamatch
andreplacethem:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
start1=mo.start(1)
end1=mo.end(1)
start2=mo.start(2)
end2=mo.end(2)
print'value1:%sstart1:%dend1:%d'%(value1,start1,
end1)
print'value2:%sstart2:%dend2:%d'%(value2,start2,
end2)
repl1=raw_input('Enterreplacement#1:')
repl2=raw_input('Enterreplacement#2:')
newline=(line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:])
print'newline:%s'%newline
else:
print'nomatch'

Explanation:

Alternatively,use"mo.span(1)"insteadof"mo.start(1)"and"mo.end(1)"inorder
togetthestartandendofasubmatchinasingleoperation."mo.span(1)"returnsa
tuple:(start,end).
Puttogetheranewstringwithstringconcatenationfrompiecesoftheoriginal
stringandreplacementvalues.Youcanusestringslicestogetthesubstringsof
theoriginalstring.Inourcase,thefollowinggetsthestartofthestring,addsthe
firstreplacement,addsthemiddleoftheoriginalstring,addsthesecond
replacement,andfinally,addsthelastpartoftheoriginalstring:
newline=line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:]

Youcanalsousethesubfunctionormethodtodosubstitutions.Hereisanexample:
importsys,re
pat=re.compile('[09]+')
print'Replacingdecimaldigits.'

Page95

APythonBook
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
repl=raw_input('Enterareplacement:')
result=pat.sub(repl,target)
print'result:%s'%result

Hereisanotherexampleoftheuseofafunctiontoinsertcalculatedreplacements.
importsys,re,string
pat=re.compile('[am]+')
defreplacer(mo):
returnstring.upper(mo.group(0))
print'Uppercasingam.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
result=pat.sub(replacer,target)
print'result:%s'%result

Notes:
Ifthereplacementargumenttosubisafunction,thatfunctionmusttakeone
argument,amatchobject,andmustreturnthemodified(orreplacement)value.
Thematchedsubstringwillbereplacedbythevaluereturnedbythisfunction.
Inourcase,thefunctionreplacerconvertsthematchedvaluetouppercase.
Thisisalsoaconvenientuseforalambdainsteadofanamedfunction,forexample:

importsys,re,string
pat=re.compile('[am]+')
print'Uppercasingam.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
result=pat.sub(
lambdamo:string.upper(mo.group(0)),
target)
print'result:%s'%result

2.3IteratorObjects
Note1:YouwillneedasufficientlyrecentversionofPythoninordertouseiteratorsand
generators.IbelievethattheywereintroducedinPython2.2.
Page96

APythonBook
Note2:TheiteratorprotocolhaschangedslightlyinPythonversion3.0.
Goalsforthissection:
Learnhowtoimplementageneratorfunction,thatis,afunctionwhich,when
called,returnsaniterator.
Learnhowtoimplementaclasscontainingageneratormethod,thatis,amethod
which,whencalled,returnsaniterator.
Learntheiteratorprotocol,specificallywhatmethodsaniteratormustsupportand
whatthosemethodsmustdo.
Learnhowtoimplementaniteratorclass,thatis,aclasswhoseinstancesare
iteratorobjects.
Learnhowtoimplementrecursiveiteratorgenerators,thatis,aniteratorgenerator
whichrecursivelyproducesiteratorgenerators.
Learnthatyourimplementationofaniteratorobject(aniteratorclass)can
"refresh"itselfandlearnatleastonewaytodothis.
Definitions:

IteratorAnditeratorisanobjectthatsatisfies(implements)theiteratorprotocol.
IteratorprotocolAnobjectimplementstheiteratorprotocolifitimplementsboth
anext()andan__iter__()methodwhichsatisfytheserules:(1)the
__iter__()methodmustreturntheiterator;(2)thenext()methodshould
returnthenextitemtobeiteratedoverandwhenfinished(therearenomore
items)shouldraisetheStopIterationexception.Theiteratorprotocolis
describedatIteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
IteratorclassAclassthatimplements(satisfies)theiteratorprotocol.In
particular,theclassimplementsnext()and__iter__()methodsas
describedaboveandinIteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
(Iterator)generatorfunctionAfunction(ormethod)which,whencalled,returns
aniteratorobject,thatis,anobjectthatsatisfiestheiteratorprotocol.Afunction
containingayieldstatementautomaticallybecomesagenerator.
GeneratorexpressionAnexpressionwhichproducesaniteratorobject.
Generatorexpressionshaveaformsimilartoalistcomprehension,butare
enclosedinparenthesesratherthansquarebrackets.Seeexamplebelow.
Afewadditionalbasicpoints:

Afunctionthatcontainsayieldstatementisageneratorfunction.Whencalled,it
returnsaniterator,thatis,anobjectthatprovidesnext()and__iter__()
methods.
Theiteratorprotocolisdescribedhere:PythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
Page97

APythonBook
Aclassthatdefinesbothanext()methodanda__iter__()methodsatisfies
theiteratorprotocol.So,instancesofsuchaclasswillbeiterators.
Pythonprovidesavarietyofwaystoproduce(implement)iterators.Thissection
describesafewofthoseways.Youshouldalsolookattheiter()builtin
function,whichisdescribedinThePythonStandardLibrary:BuiltinFunctions:
iter()http://docs.python.org/library/functions.html#iter.
Aniteratorcanbeusedinaniteratorcontext,forexampleinaforstatement,ina
listcomprehension,andinageneratorexpression.Whenaniteratorisusedinan
iteratorcontext,theiteratorproducesitsvalues.
Thissectionattemptstoprovideexamplesthatillustratethegenerator/iteratorpattern.

Whyisthisimportant?
Oncemastered,itisasimple,convenient,andpowerfulprogrammingpattern.
Ithasmanyandpervasiveuses.
Ithelpstolexicallyseparatetheproducercodefromtheconsumercode.Doingso
makesiteasiertolocateproblemsandtomodifyorfixcodeinawaythatis
localizedanddoesnothaveunwantedsideeffects.
Implementingyourowniterators(andgenerators)enablesyoutodefineyourown
abstractsequences,thatis,sequenceswhosecompositionaredefinedbyyour
computationsratherthanbytheirpresenceinacontainer.Infact,youriteratorcan
calculateorretrievevaluesaseachoneisrequested.
ExamplesTheremainderofthissectionprovidesasetofexampleswhichimplement
anduseiterators.

2.3.1ExampleAgeneratorfunction
Thisfunctioncontainsayieldstatement.Therefore,whenwecallit,itproducesan
iterator:
defgenerateItems(seq):
foriteminseq:
yield'item:%s'%item
anIter=generateItems([])
print'dir(anIter):',dir(anIter)
anIter=generateItems([111,222,333])
forxinanIter:
printx
anIter=generateItems(['aaa','bbb','ccc'])
printanIter.next()
printanIter.next()
printanIter.next()
printanIter.next()

Runningthisexampleproducesthefollowingoutput:
Page98

APythonBook
dir(anIter):['__class__','__delattr__','__doc__',
'__getattribute__',
'__hash__','__init__','__iter__','__new__','__reduce__',
'__reduce_ex__','__repr__','__setattr__','__str__','gi_frame',
'gi_running','next']
item:111
item:222
item:333
item:aaa
item:bbb
item:ccc
Traceback(mostrecentcalllast):
File"iterator_generator.py",line14,in?
printanIter.next()
StopIteration

Notesandexplanation:
Thevaluereturnedbythecalltothegenerator(function)isaniterator.Itobeys
theiteratorprotocol.Thatis,dir(anIter)showsthatithasboth
__iter__()andnext()methods.
Becausethisobjectisaniterator,wecanuseaforstatementtoiterateoverthe
valuesreturnedbythegenerator.
Wecanalsogetitsvaluesbyrepeatedlycallingthenext()method,untilit
raisestheStopIterationexception.Thisabilitytocallthenextmethodenablesus
topasstheiteratorobjectaroundandgetvaluesatdifferentlocationsinourcode.
Oncewehaveobtainedallthevaluesfromaniterator,itis,ineffect,"empty"or
"exhausted".Theiteratorprotocol,infact,specifiesthatonceaniteratorraisesthe
StopIterationexception,itshouldcontinuetodoso.Anotherwaytosaythisis
thatthereisno"rewind"operation.But,youcancallthethegeneratorfunction
againtogeta"fresh"iterator.
Analternativeandperhapssimplerwaytocreateaninteratoristouseagenerator
expression.Thiscanbeusefulwhenyoualreadyhaveacollectionoriteratortowork
with.

Thenfollowingexampleimplementsafunctionthatreturnsageneratorobject.Theeffect
istogeneratetheobjectsinacollectionwhichexcludingitemsinasepartecollection:
DATA=[
'lemon',
'lime',
'grape',
'apple',
'pear',
'watermelon',
'canteloupe',
'honeydew',
'orange',

Page99

APythonBook
'grapefruit',
]
defmake_producer(collection,excludes):
gen=(itemforitemincollectionifitemnotinexcludes)
returngen
deftest():
iter1=make_producer(DATA,('apple','orange','honeydew',))
print'%s'%iter1
forfruitiniter1:
printfruit
test()

Whenrun,thisexampleproducesthefollowing:
$pythonworkbook063.py
<generatorobject<genexpr>at0x7fb3d0f1bc80>
lemon
lime
grape
pear
watermelon
canteloupe
grapefruit

Notes:

Ageneratorexpressionlooksalmostlikealistcomprehension,butissurrounded
byparenthesesratherthansquarebrackets.Formoreonlistcomprehensionssee
sectionExampleAlistcomprehension.
Themake_producerfunctionreturnstheobjectproducedbythegenerator
expression.

2.3.2ExampleAclasscontainingageneratormethod
Eachtimethismethodiscalled,itproducesa(new)iteratorobject.Thismethodis
analogoustotheiterkeysanditervaluesmethodsinthedictionarybuiltinobject:
#
#Aclassthatprovidesaniteratorgeneratormethod.
#
classNode:
def__init__(self,name='<noname>',value='<novalue>',
children=None):
self.name=name
self.value=value
self.children=children
ifchildrenisNone:
self.children=[]

Page100

APythonBook
else:
self.children=children
defset_name(self,name):self.name=name
defget_name(self):returnself.name
defset_value(self,value):self.value=value
defget_value(self):returnself.value
defiterchildren(self):
forchildinself.children:
yieldchild
#
#Printinformationonthisnodeandwalkoverallchildrenand
#grandchildren...
defwalk(self,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),self.get_name(),self.get_value(),)
forchildinself.iterchildren():
child.walk(level+1)
#
#Anfunctionthatistheequivalentofthewalk()methodin
#classNode.
#
defwalk(node,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),node.get_name(),node.get_value(),)
forchildinnode.iterchildren():
walk(child,level+1)
defget_filler(level):
return''*level
deftest():
a7=Node('gilbert','777')
a6=Node('fred','666')
a5=Node('ellie','555')
a4=Node('daniel','444')
a3=Node('carl','333',[a4,a5])
a2=Node('bill','222',[a6,a7])
a1=Node('alice','111',[a2,a3])
#Usethewalkmethodtowalktheentiretree.
print'Usingthemethod:'
a1.walk()
print'='*30
#Usethewalkfunctiontowalktheentiretree.
print'Usingthefunction:'
walk(a1)
test()

Runningthisexampleproducesthefollowingoutput:
Usingthemethod:
name:alicevalue:111

Page101

APythonBook
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
==============================
Usingthefunction:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555

Notesandexplanation:

Thisclasscontainsamethoditerchildrenwhich,whencalled,returnsaniterator.
Theyieldstatementinthemethoditerchildrenmakesitintoagenerator.
Theyieldstatementreturnsoneitemeachtimeitisreached.Thenexttimethe
iteratorobjectis"called"itresumesimmediatelyaftertheyieldstatement.
Afunctionmayhaveanynumberofyieldstatements.
Aforstatementwilliterateoveralltheitemsproducedbyaniteratorobject.
Thisexampleshowstwowaystousethegenerator,specifically:(1)thewalk
methodintheclassNodeand(2)thewalkfunction.Bothcallthegenerator
iterchildrenandbothdoprettymuchthesamething.

2.3.3ExampleAniteratorclass
Thisclassimplementstheiteratorprotocol.Therefore,instancesofthisclassareiterators.
Thepresenceofthenext()and__iter__()methodsmeansthatthisclass
implementstheiteratorprotocolandmakesinstancesofthisclassiterators.
Notethatwhenaniteratoris"exhausted"it,normally,cannotbereusedtoiterateoverthe
sequence.However,inthisexample,weprovidearefreshmethodwhichenablesusto
"rewind"andreusetheiteratorinstance:
#
#Aniteratorclassthatdoes*not*use``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classIteratorExample:
def__init__(self,seq):
self.seq=seq
self.idx=0
defnext(self):
self.idx+=1
ifself.idx>=len(self.seq):

Page102

APythonBook
raiseStopIteration
value=self.seq[self.idx]
self.idx+=1
returnvalue
def__iter__(self):
returnself
defrefresh(self):
self.idx=0
deftest_iteratorexample():
a=IteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=IteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_iteratorexample()

Runningthisexampleproducesthefollowingoutput:
d
b

d
b
==============================
b
d
stopping

Notesandexplanation:

Thenextmethodmustkeeptrackofwhereitisandwhatitemitshouldproduce
next.
Alert:TheiteratorprotocolhaschangedslightlyinPython3.0.Inparticular,the
next()methodhasbeenrenamedto__next__().See:PythonStandard
Library:IteratorTypes
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
Page103

APythonBook

2.3.4ExampleAniteratorclassthatusesyield
Theremaybetimeswhenthenextmethodiseasierandmorestraightforwardto
implementusingyield.Ifso,thenthisclassmightserveasanmodel.Ifyoudonotfeel
theneedtodothis,thenyoushouldignorethisexample:
#
#Aniteratorclassthatuses``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classYieldIteratorExample:
def__init__(self,seq):
self.seq=seq
self.iterator=self._next()
self.next=self.iterator.next
def_next(self):
flag=0
forxinself.seq:
ifflag:
flag=0
yieldx
else:
flag=1
def__iter__(self):
returnself.iterator
defrefresh(self):
self.iterator=self._next()
self.next=self.iterator.next
deftest_yielditeratorexample():
a=YieldIteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=YieldIteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_yielditeratorexample()

Runningthisexampleproducesthefollowingoutput:
Page104

APythonBook
d
b

d
b
==============================
b
d
stopping

Notesandexplanation:

Becausethe_nextmethodusesyield,callingit(actually,callingtheiterator
objectitproduces)inaniteratorcontextcausesittobe"resumed"immediately
aftertheyieldstatement.Thisreducesbookkeepingabit.
However,withthisstyle,wemustexplicitlyproduceaniterator.Wedothisby
callingthe_nextmethod,whichcontainsayieldstatement,andisthereforea
generator.Thefollowingcodeinourconstructor(__init__)completesthe
setupofourclassasaniteratorclass:
self.iterator=self._next()
self.next=self.iterator.next

Rememberthatweneedboth__iter__()andnext()methodsin
YieldIteratorExampletosatisfytheiteratorprotocol.The__iter__()
methodisalreadythereandtheabovecodeintheconstructorcreatesthenext()
method.

2.3.5ExampleAlistcomprehension
Alistcomprehensionlooksabitlikeaniterator,butitproducesalist.See:ThePython
LanguageReference:Listdisplays
http://docs.python.org/reference/expressions.html#listdisplaysformoreonlist
comprehensions.
Hereisanexample:
In[4]:deff(x):
...:returnx*3
...:
In[5]:list1=[11,22,33]
In[6]:list2=[f(x)forxinlist1]
In[7]:printlist2
[33,66,99]

2.3.6ExampleAgeneratorexpression
Ageneratorexpressionlooksquitesimilartoalistcomprehension,butisenclosedin
Page105

APythonBook
parenthesesratherthansquarebrackets.Unlikealistcomprehension,agenerator
expressiondoesnotproducealist;itproducesangeneratorobject.Ageneratorobjectis
aniterator.
Formoreongeneratorexpressions,seeThePythonLanguageReference:Generator
expressionshttp://docs.python.org/reference/expressions.html#generatorexpressions.
Thefollowingexampleusesageneratorexpressiontoproduceaniterator:
mylist=range(10)
deff(x):
returnx*3
genexpr=(f(x)forxinmylist)
forxingenexpr:
printx

Notesandexplanation:

Thegeneratorexpression(f(x)forxinmylist)producesaniteratorobject.
Noticethatwecanusetheiteratorobjectlaterinourcode,cansaveitinadata
structure,andcanpassittoafunction.

2.4UnitTests
UnittestandthePythonunittestframeworkprovideaconvenientwaytodefineandrun
teststhatensurethataPythonapplicationproducesspecifiedresults.
Thissection,whileitwillnotattempttoexplaineverythingabouttheunittestframework,
willprovideexamplesofseveralstraightforwardwaystoconstructandruntests.
Someassumptions:

Wearegoingtodevelopasoftwareprojectincrementally.Wewillnotimplement
andreleaseallatonce.Therefore,eachtimeweaddtoourexistingcodebase,we
needawaytoverifythatouradditions(andfixes)havenotcausednewproblems
inoldcode.
Addingnewcodetoexistingcodewillcauseproblems.Weneedtobeableto
check/testforthoseproblemsateachstep.
Asweaddcode,weneedtobeabletoaddtestsforthatnewcode,too.

2.4.1Definingunittests
2.4.1.1Createatestclass.

Inthetestclass,implementanumberofmethodstoperformyourtests.Nameyourtest
Page106

APythonBook
methodswiththeprefix"test".Hereisanexample:
importunittest
classMyTest(unittest.TestCase):
deftest_one(self):
#sometestcode
pass
deftest_two(self):
#sometestcode
pass

Createatestharness.Hereisanexample:
importunittest
#makethetestsuite.
defsuite():
loader=unittest.TestLoader()
testsuite=loader.loadTestsFromTestCase(MyTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)

Hereisamorecompleteexample:
importsys,StringIO,string
importunittest
importwebserv_example_heavy_sub
#Acomparisonfunctionforcaseinsenstivesorting.
defmycmpfunc(arg1,arg2):
returncmp(string.lower(arg1),string.lower(arg2))
classXmlTest(unittest.TestCase):
deftest_import_export1(self):
inFile=file('test1_in.xml','r')
inContent=inFile.read()
inFile.close()
doc=webserv_example_heavy_sub.parseString(inContent)
outFile=StringIO.StringIO()
outFile.write('<?xmlversion="1.0"?>\n')
doc.export(outFile,0)
outContent=outFile.getvalue()
outFile.close()
self.failUnless(inContent==outContent)
#makethetestsuite.
defsuite():

Page107

APythonBook
loader=unittest.TestLoader()
#Changethetestmethodprefix:test>trial.
#loader.testMethodPrefix='trial'
#Changethecomparisonfunctionthatdeterminestheorderof
tests.
#loader.sortTestMethodsUsing=mycmpfunc
testsuite=loader.loadTestsFromTestCase(XmlTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest_main():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
if__name__=="__main__":
test_main()

Runningtheabovescriptproducesthefollowingoutput:
test_import_export(__main__.XmlTest)...ok

Ran1testin0.035s
OK

Afewnotesonthisexample:

Thisexampleteststheabilitytoparseanxmldocumenttest1_in.xmlandexport
thatdocumentbacktoXML.ThetestsucceedsiftheinputXMLdocumentand
theexportedXMLdocumentarethesame.
ThecodewhichisbeingtestedparsesanXMLdocumentreturnedbyarequestto
AmazonWebservices.YoucanlearnmoreaboutAmazonWebservicesat:
http://www.amazon.com/webservices.ThiscodewasgeneratedfromanXML
SchemadocumentbygenerateDS.py.Soweareineffect,testinggenerateDS.py.
YoucanfindgenerateDS.pyat:
http://http://www.davekuhlman.org/#generatedspy.
Testingforsuccess/failureandreportingfailuresUsethemethodslistedat
http://www.python.org/doc/current/lib/testcaseobjects.htmltotestforandreport
successandfailure.Inourexample,weused"self.failUnless(inContent==
outContent)"toensurethatthecontentweparsedandthecontentthatwe
exportedwerethesame.
Addadditionaltestsbyaddingmethodswhosenameshavetheprefix"test".If
youpreferadifferentprefixfortestsnames,addsomethinglikethefollowingto
theabovescript:
loader.testMethodPrefix='trial'

Page108

APythonBook

Bydefault,thetestsarerunintheorderoftheirnamessortedbythecmp
function.So,ifneeded,youcancontroltheorderofexecutionoftestsby
selectingtheirnames,forexample,usingnamesliketest_1_checkderef,
test_2_checkcalc,etc.Or,youcanchangethecomparisonfunctionbyadding
somethinglikethefollowingtotheabovescript:
loader.sortTestMethodsUsing=mycmpfunc

Asabitofmotivationforcreatingandusingunittests,whiledevelopingthisexample,I
discoveredseveralerrors(ormaybe"specialfeatures")ingenerateDS.py.

2.5ExtendingandembeddingPython
2.5.1Introductionandconcepts
Extendingvs.embeddingTheyaredifferentbutrelated:
ExtendingPythonmeanstoimplementanextensionmoduleoranextensiontype.
AnextensionmodulecreatesanewPythonmodulewhichisimplementedin
C/C++.FromPythoncode,anextensionmoduleappearstobejustlikeamodule
implementedinPythoncode.AnextensiontypecreatesanewPython(builtin)
typewhichisimplementedinC/C++.FromPythoncode,anextensiontype
appearstobejustlikeabuiltintype.
EmbeddingPython,bycontrast,istoputthePythoninterpreterwithinan
application(i.e.linkitin)sothattheapplicationcanrunPythonscripts.The
scriptscanbeexecutedortriggeredinavarietyofways,e.g.theycanbeboundto
keysonthekeyboardortomenuitems,theycanbetriggeredbyexternalevents,
etc.Usually,inordertomaketheembeddedPythoninterpreteruseful,Pythonis
alsoextendedwithfunctionsfromtheembeddingapplication,sothatthescripts
cancallfunctionsthatareimplementedbytheembeddingC/C++application.
DocumentationThetwoimportantsourcesforinformationaboutextendingand
embeddingarethefollowing:

ExtendingandEmbeddingthePythonInterpreter
http://www.python.org/doc/current/ext/ext.html
Python/CAPIReferenceManual
http://www.python.org/doc/current/api/api.html
Typesofextensions:

ExtensionmodulesFromthePythonside,itappearstobeaPythonmodule.
Usuallyitexportsfunctions.
ExtensiontypesUsedtoimplementanewPythondatatype.
ExtensionclassesFromthePythonside,itappearstobeaclass.
Page109

APythonBook
ToolsThereareseveraltoolsthatsupportthedevelopmentofPythonextensions:

SWIGLearnaboutSWIGat:http://www.swig.org
PyrexLearnaboutPyrexat:
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ThereisalsoCython,whichseemstobeanadvancedversionof,oratleastan
alternativetoPyrex.See:CythonCExtensionsforPython
http://www.cython.org/

2.5.2Extensionmodules
WritinganextensionmodulebyhandWhattodo:
Createthe"init"functionThenameofthisfunctionmustbe"init"followedby
thenameofthemodule.Everyextensionmodulemusthavesuchafunction.
CreatethefunctiontableThistablemapsfunctionnames(referencedfrom
Pythoncode)tofunctionpointers(implementedinC/C++).
Implementeachwrapperfunction.
ImplementingawrapperfunctionWhattodo:

1. CapturetheargumentswithPyArg_ParseTuple.Theformatstringspecifieshow
argumentsaretobeconvertedandcaptured.See1.7ExtractingParametersin
ExtensionFunctions.Herearesomeofthemostcommonlyusedtypes:
Use"i","s","f",etctoconvertandcapturesimpletypessuchasintegers,
strings,floats,etc.
Use"O"togetapointertoPython"complex"typessuchaslists,tuples,
dictionaries,etc.
Useitemsinparenthesestocaptureandunpacksequences(e.g.listsand
tuples)offixedlength.Example:
if(!PyArg_ParseTuple(args,"(ii)(ii)",&x,&y,
&width,&height))
{
returnNULL;
}/*if*/

Asamplecallmightbe:
lowerLeft=(x1,y1)
extent=(width1,height1)
scan(lowerLeft,extent)

Use":aName"(colon)attheendoftheformatstringtoprovideafunction
nameforerrormessages.Example:
if(!PyArg_ParseTuple(args,"O:setContentHandler",
&pythonInstance))
{

Page110

APythonBook
returnNULL;
}/*if*/

Use";anerrormessage"(semicolon)attheendoftheformatstringtoprovide
astringthatreplacesthedefaulterrormessage.
Docsareavailableat:http://www.python.org/doc/current/ext/parseTuple.html.
2. Writethelogic.
3. HandleerrorsandexceptionsYouwillneedtounderstandhowto(1)clearing
errorsandexceptionsand(2)Raiseerrors(exceptions).
ManyfunctionsinthePythonCAPIraiseexceptions.Youwillneedtocheck
forandcleartheseexceptions.Hereisanexample:

char*message;
intmessageNo;
message=NULL;
messageNo=1;
/*Istheargumentastring?
*/
if(!PyArg_ParseTuple(args,"s",&message))
{
/*It'snotastring.Cleartheerror.
*Thentrytogetamessagenumber(an
integer).
*/
PyErr_Clear();
if(!PyArg_ParseTuple(args,"i",&messageNo))
{
o
o
o

YoucanalsoraiseexceptionsinyourCcodethatcanbecaught(ina
"try:except:"block)backinthecallingPythoncode.Hereisanexample:
if(n==0)
{
PyErr_SetString(PyExc_ValueError,"Valuemust
notbezero");
returnNULL;
}

SeeInclude/pyerrors.hinthePythonsourcedistributionformore
exception/errortypes.
And,youcantestwhetherafunctioninthePythonCAPIthatyouhavecalled
hasraisedanexception.Forexample:
if(PyErr_Occurred())
{
/*Anexceptionwasraised.
*Dosomethingaboutit.

Page111

APythonBook
*/
o
o
o

Formoredocumentationonerrorsandexceptions,see:
http://www.python.org/doc/current/api/exceptionHandling.html.
4. Createandreturnavalue:
ForeachbuiltinPythontypethereisasetofAPIfunctionstocreateand
manipulateit.Seethe"Python/CAPIReferenceManual"foradescriptionof
thesefunctions.Forexample,see:
http://www.python.org/doc/current/api/intObjects.html
http://www.python.org/doc/current/api/stringObjects.html
http://www.python.org/doc/current/api/tupleObjects.html
http://www.python.org/doc/current/api/listObjects.html
http://www.python.org/doc/current/api/dictObjects.html
Etc.
ThereferencecountYouwillneedtofollowPython'srulesforreference
countingthatPythonusestogarbagecollectobjects.Youcanlearnabout
theserulesathttp://www.python.org/doc/current/ext/refcounts.html.Youwill
notwantPythontogarbagecollectobjectsthatyoucreatetooearlyortoolate.
WithrespecttoPythonobjectscreatedwiththeabovefunctions,thesenew
objectsareownedandmaybepassedbacktoPythoncode.However,there
aresituationswhereyourC/C++codewillnotautomaticallyownareference,
forexamplewhenyouextractanobjectfromacontainer(alist,tuple,
dictionary,etc).Inthesecasesyoushouldincrementthereferencecountwith
Py_INCREF.

2.5.3SWIG
Note:OurdiscussionandexamplesareforSWIGversion1.3
SWIGwilloftenenableyoutogeneratewrappersforfunctionsinanexistingCfunction
library.SWIGdoesnotunderstandeverythinginCheaderfiles.Butitdoesafairly
impressivejob.Youshouldtryitfirstbeforeresortingtothehardworkofwriting
wrappersbyhand.
MoreinformationonSWIGisathttp://www.swig.org.
Herearesomestepsthatyoucanfollow:
1. CreateaninterfacefileEvenwhenyouarewrappingfunctionsdefinedinan
existingheaderfile,creatinganinterfacefileisagoodidea.Includeyourexisting
headerfileintoit,thenaddwhateverelseyouneed.Hereisanextremelysimple
exampleofaSWIGinterfacefile:
Page112

APythonBook
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
%include"MyLibrary.h"

Comments:
The"%{"and"%}"bracketsaredirectivestoSWIG.Theysay:"Addthecode
betweenthesebracketstothegeneratedwrapperfilewithoutprocessingit.
The"%include"statementsays:"Copythefileintotheinterfacefilehere.In
effect,youareaskingSWIGtogeneratewrappersforallthefunctionsinthis
headerfile.Ifyouwantwrappersforonlysomeofthefunctionsinaheader
file,thencopyorreproducefunctiondeclarationsforthedesiredfunctions
here.Anexample:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
intcalcArea(intwidth,intheight);
intcalcVolume(intradius);

Thisexamplewillgeneratewrappersforonlytwofunctions.
YoucanfindmoreinformationaboutthedirectivesthatareusedinSWIG
interfacefilesintheSWIGUserManual,inparticularat:
http://www.swig.org/Doc1.3/Preprocessor.html
http://www.swig.org/Doc1.3/Python.html
2. Generatethewrappers:

swigpythonMyLibrary.i

3. Compileandlinkthelibrary.OnLinux,youcanusesomethinglikethefollowing:
gcccMyLibrary.c
gcccI/usr/local/include/python2.3MyLibrary_wrap.c
gccsharedMyLibrary.oMyLibrary_wrap.oo
_MyLibrary.so

Notethatweproduceasharedlibrarywhosenameisthemodulenameprefixed
withanunderscore.SWIGalsogeneratesa.pyfile,withouttheleading
underscore,whichwewillimportfromourPythoncodeandwhich,inturn,
importsthesharedlibrary.
4. Usetheextensionmoduleinyourpythoncode:
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2

Page113

APythonBook
Type"help","copyright","credits"or"license"for
moreinformation.
>>>importMyLibrary
>>>MyLibrary.calcArea(4.0,5.0)
20.0

Hereisamakefilethatwillexecuteswigtogeneratewrappers,thencompileandlinkthe
extension.
CFLAGS=I/usr/local/include/python2.3
all:_MyLibrary.so
_MyLibrary.so:MyLibrary.oMyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
MyLibrary.o:MyLibrary.c
gcccMyLibrary.coMyLibrary.o
MyLibrary_wrap.o:MyLibrary_wrap.c
gccc${CFLAGS}MyLibrary_wrap.coMyLibrary_wrap.o
MyLibrary_wrap.c:MyLibrary.i
swigpythonMyLibrary.i
clean:
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c
MyLibrary_wrap.o_MyLibrary.so
Hereisanexampleofrunningthismakefile:
$makefMyLibrary_makefileclean
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c\
MyLibrary_wrap.o_MyLibrary.so
$makefMyLibrary_makefile
gcccMyLibrary.coMyLibrary.o
swigpythonMyLibrary.i
gcccI/usr/local/include/python2.3MyLibrary_wrap.co
MyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so

And,hereareCsourcefilesthatcanbeusedinourexample.
MyLibrary.h:
/*MyLibrary.h
*/

Page114

APythonBook
floatcalcArea(floatwidth,floatheight);
floatcalcVolume(floatradius);
intgetVersion();
intgetMode();

MyLibrary.c:
/*MyLibrary.c
*/
floatcalcArea(floatwidth,floatheight)
{
return(width*height);
}
floatcalcVolume(floatradius)
{
return(3.14*radius*radius);
}
intgetVersion()
{
return123;
}
intgetMode()
{
return1;
}

2.5.4Pyrex
PyrexisausefultoolforwritingPythonextensions.BecausethePyrexlanguageis
similartoPython,writingextensionsinPyrexiseasierthandoingsoinC.Cython
appearstobetheanewerversionofPyrex.
MoreinformationisonPyrexandCythonisat:
Pyrexhttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
CythonCExtensionsforPythonhttp://www.cython.org/
HereisasimplefunctiondefinitioninPyrex:

#python_201_pyrex_string.pyx
importstring
defformatString(objects1,objects2):
s1=string.strip(s1)
s2=string.strip(s2)

Page115

APythonBook
s3='<<%s||%s>>'%(s1,s2)
s4=s3*4
returns4

And,hereisamakefile:
CFLAGS=DNDEBUGO3WallWstrictprototypesfPIC\
I/usr/local/include/python2.3
all:python_201_pyrex_string.so
python_201_pyrex_string.so:python_201_pyrex_string.o
gccsharedpython_201_pyrex_string.oo
python_201_pyrex_string.so
python_201_pyrex_string.o:python_201_pyrex_string.c
gccc${CFLAGS}python_201_pyrex_string.co
python_201_pyrex_string.o
python_201_pyrex_string.c:python_201_pyrex_string.pyx
pyrexcpython_201_pyrex_string.pyx
clean:
rmfpython_201_pyrex_string.sopython_201_pyrex_string.o\
python_201_pyrex_string.c

Hereisanotherexample.Inthisone,onefunctioninthe.pyxfilecallsanother.Hereis
theimplementationfile:
#python_201_pyrex_primes.pyx
defshowPrimes(intkmax):
plist=primes(kmax)
forpinplist:
print'prime:%d'%p
cdefprimes(intkmax):
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1

Page116

APythonBook
returnresult

And,hereisamakefile:
#CFLAGS=DNDEBUGgO3WallWstrictprototypesfPIC#
I/usr/local/include/python2.3CFLAGS=DNDEBUG
I/usr/local/include/python2.3
all:python_201_pyrex_primes.so
python_201_pyrex_primes.so:python_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
python_201_pyrex_primes.o:python_201_pyrex_primes.c
gccc${CFLAGS}python_201_pyrex_primes.copython_201_pyrex_primes.o
python_201_pyrex_primes.c:python_201_pyrex_primes.pyx
pyrexcpython_201_pyrex_primes.pyx
clean:
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o
python_201_pyrex_primes.c
Hereistheoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeprimesclean
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o\
python_201_pyrex_primes.c
$makefpython_201_pyrex_makeprimes
pyrexcpython_201_pyrex_primes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_primes.copython_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so

Hereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_primes
>>>dir(python_201_pyrex_primes)
['__builtins__','__doc__','__file__','__name__','showPrimes']
>>>python_201_pyrex_primes.showPrimes(5)
prime:2
prime:3
prime:5
prime:7

Page117

APythonBook
prime:11

ThisnextexampleshowshowtousePyrextoimplementanewextensiontype,thatisa
newPythonbuiltintype.Noticethattheclassisdeclaredwiththecdefkeyword,which
tellsPyrextogeneratetheCimplementationofatypeinsteadofaclass.
Hereistheimplementationfile:
#python_201_pyrex_clsprimes.pyx
"""Animplementationofprimeshandlingclass
forademonstrationofPyrex.
"""
cdefclassPrimes:
"""Aclasscontainingfunctionsfor
handlingprimes.
"""
defshowPrimes(self,intkmax):
"""Showarangeofprimes.
Usethemethodprimes()togeneratetheprimes.
"""
plist=self.primes(kmax)
forpinplist:
print'prime:%d'%p
defprimes(self,intkmax):
"""Generatetheprimesintherange0kmax.
"""
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult

And,hereisamakefile:
CFLAGS=DNDEBUGI/usr/local/include/python2.3
all:python_201_pyrex_clsprimes.so

Page118

APythonBook
python_201_pyrex_clsprimes.so:python_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o:python_201_pyrex_clsprimes.c
gccc${CFLAGS}python_201_pyrex_clsprimes.co
python_201_pyrex_clsprimes.o
python_201_pyrex_clsprimes.c:python_201_pyrex_clsprimes.pyx
pyrexcpython_201_pyrex_clsprimes.pyx
clean:
rmfpython_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c

Hereisoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeclsprimesclean
rmfpython_201_pyrex_clsprimes.sopython_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
$makefpython_201_pyrex_makeclsprimes
pyrexcpython_201_pyrex_clsprimes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_clsprimes.copython_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so

Andhereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_clsprimes
>>>dir(python_201_pyrex_clsprimes)
['Primes','__builtins__','__doc__','__file__','__name__']
>>>primes=python_201_pyrex_clsprimes.Primes()
>>>dir(primes)
['__class__','__delattr__','__doc__','__getattribute__',
'__hash__',
'__init__','__new__','__reduce__','__reduce_ex__','__repr__',
'__setattr__','__str__','primes','showPrimes']
>>>primes.showPrimes(4)
prime:2
prime:3
prime:5
prime:7

DocumentationAlsonoticethatPyrexpreservesthedocumentationforthemodule,the
class,andthemethodsintheclass.Youcanshowthisdocumentationwithpydoc,as
Page119

APythonBook
follows:
$pydocpython_201_pyrex_clsprimes

Or,inPythoninteractivemode,use:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_clsprimes
>>>help(python_201_pyrex_clsprimes)

2.5.5SWIGvs.Pyrex
ChooseSWIGwhen:
YoualreadyhaveanexistingCorC++implementationofthecodeyouwantto
callfromPython.InthiscaseyouwantSWIGtogeneratethewrappers.Butnote
thatCythonpromisestoenableyoutoquicklywrapandcallfunctions
implementedinC.
YouwanttowritetheimplementationinCorC++byhand.Perhaps,becauseyou
thinkyoucandosoquickly,forexample,orbecauseyoubelievethatyoucan
makeithighlyoptimized.Then,youwanttobeabletogeneratethePython
(extension)wrappersforitquickly.
ChoosePyrexwhen:

YoudonothaveaC/C++implementationandyouwantaneasierwaytowrite
thatCimplementation.WritingPyrexcode,whichisalotlikePython,iseasier
thanwritingCorC++codebyhand).
YoustarttowritetheimplementationinC,thenfindthatitrequireslotsofcallsto
thePythonCAPI,andyouwanttoavoidhavingtolearnhowtodothat.

2.5.6Cython
HereisasimpleexamplethatusesCythontowrapafunctionimplementedinC.
FirsttheCheaderfile:
/*test_c_lib.h*/
intcalculate(intwidth,intheight);

And,theCimplementationfile:
/*test_c_lib.c*/

Page120

APythonBook
#include"test_c_lib.h"
intcalculate(intwidth,intheight)
{
intresult;
result=width*height*3;
returnresult;
}

HereisaCythonfilethatcallsourCfunction:
#test_c.pyx
#DeclaretheexternalCfunction.
cdefexternfrom"test_c_lib.h":
intcalculate(intwidth,intheight)
deftest(w,h):
#CalltheexternalCfunction.
result=calculate(w,h)
print'resultfromcalculate:%d'%result

Wecancompileourcodeusingthisscript(onLinux):
#!/bin/bashx
cythontest_c.pyx
gcccfPICI/usr/local/include/python2.6otest_c.otest_c.c
gcccfPICI/usr/local/include/python2.6otest_c_lib.o
test_c_lib.c
gccsharedfPICI/usr/local/include/python2.6otest_c.so
test_c.otest_c_lib.o

HereisasmallPythonfilethatusesthewrapperthatwewroteinCython:
#run_test_c.py
importtest_c
deftest():
test_c.test(4,5)
test_c.test(12,15)
if__name__=='__main__':
test()

And,whenwerunit,weseethefollowing:
$pythonrun_test_c.py
resultfromcalculate:60
resultfromcalculate:540

Page121

APythonBook

2.5.7Extensiontypes
ThegoalAnewbuiltindatatypeforPython.
ExistingexamplesObjects/listobject.c,Objects/stringobject.c,Objects/dictobject.c,etc
inthePythonsourcecodedistribution.
InolderversionsofthePythonsourcecodedistribution,atemplatefortheCcodewas
providedinObjects/xxobject.c.Objects/xxobject.cisnolongerincludedinthePython
sourcecodedistribution.However:
Thediscussionandexamplesforcreatingextensiontypeshavebeenexpanded.
See:ExtendingandEmbeddingthePythonInterpreter,2.DefiningNewTypes
http://docs.python.org/extending/newtypes.html.
IntheTools/framerdirectoryofthePythonsourcecodedistributionthereisan
applicationthatwillgenerateaskeletonforanextensiontypefromaspecification
objectwritteninPython.RunTools/framer/example.pytoseeitinaction.
And,youcanusePyrextogenerateanewbuiltintype.Todoso,implementa
Python/PyrexclassanddeclaretheclasswiththePyrexkeywordcdef.Infact,youmay
wanttousePyrextogenerateaminimalextensiontype,andtheneditthatgeneratedcode
toinsertandaddfunctionalitybyhand.SeethePyrexsectionforanexample.

Pyrexalsogoessomewaytowardgivingyouaccessto(existing)Cstructsandfunctions
fromPython.

2.5.8Extensionclasses
ExtensionclassestheeasywaySWIGshadowclasses.
StartwithanimplementationofaC++classanditsheaderfile.
UsethefollowingSWIGflags:
swigc++pythonmymodule.i

MoreinformationisavailablewiththeSWIGdocumentationat:
http://www.swig.org/Doc1.3/Python.html.
ExtensionclassesthePyrexwayAnalternatieistousePyrextocompileaclass
definitionthatdoesnothavethecdefkeyword.UsingcdefontheclasstellsPyrexto
generateanextensiontypeinsteadofaclass.Youwillhavetodeterminewhetheryou
wantanextensionclassoranextensiontype.

2.6Parsing
Pythonisanexcellentlanguagefortextanalysis.
Page122

APythonBook
Insomecases,simplysplittinglinesoftextintowordswillbeenough.Inthesecasesuse
string.split().
Inothercases,regularexpressionsmaybeabletodotheparsingyouneed.Ifso,seethe
sectiononregularexpressionsinthisdocument.
However,insomecases,morecomplexanalysisofinputtextisrequired.Thissection
describessomeofthewaysthatPythoncanhelpyouwiththiscomplexparsingand
analysis.

2.6.1Specialpurposeparsers
ThereareanumberofspecialpurposeparserswhichyouwillfindinthePythonstandard
library:
ConfigParserparserConfigurationfileparser
http://docs.python.org/library/configparser.html
getoptParserforcommandlineoptions
http://docs.python.org/library/getopt.html
optparseMorepowerfulcommandlineoptionparser
http://docs.python.org/library/optparse.html
urlparseParseURLsintocomponents
http://docs.python.org/library/urlparse.html
csvCSV(commaseparatedvalues)FileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv
os.pathCommonpathnamemanipulations
http://docs.python.org/library/os.path.html
XMLparsersandXMLtoolsThereislotsofsupportforparsingandprocessingXML
inPython.Hereareafewplacestolookforsupport:

ThePythonstandardlibraryStructuredMarkupProcessingTools
http://docs.python.org/library/markup.html.
Inparticular,youmaybeinterestedinxml.dom.minidomLightweightDOM
implementationhttp://docs.python.org/library/xml.dom.minidom.html.
ElementTreeYoucanthinkofElementTreeasanenhancedDOM(document
objectmodel).Manyfinditeasiertousethanminidom.ElementTreeisinthe
Pythonstandardlibrary,anddocumentationishere:ElementTreeOverview
http://effbot.org/zone/elementindex.htm.
LxmlmimicstheElementTreeAPI,buthasadditionalcapabilities.Findoutabout
Lxmlatlxmlhttp://codespeak.net/lxml/index.htmlNotethatlxmlalsohas
supportforXPathandXSLT.
Dave'ssupportforPythonandXMLhttp://www.rexx.com/~dkuhlman.

Page123

APythonBook

2.6.2Writingarecursivedescentparserbyhand
Forsimplegrammars,thisisnotsohard.
Youwillneedtoimplement:
Arecognizermethodorfunctionforeachproductionruleinyourgrammar.Each
recognizermethodbeginslookingatthecurrenttoken,thenconsumesasmany
tokensasneededtorecognizeit'sownproductionrule.Itcallstherecognizer
functionsforanynonterminalsonitsrighthandside.
AtokenizerSomethingthatwillenableeachrecognizerfunctiontogettokens,
onebyone.Thereareavarietyofwaystodothis,e.g.(1)afunctionthat
producesalistoftokensfromwhichrecognizerscanpoptokens;(2)agenerator
whosenextmethodreturnsthenexttoken;etc.
Asanexample,we'llimplementarecursivedescentparserwritteninPythonforthe
followinggrammer:

Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>

Hereisanimplementationofarecursivedescentparserfortheabovegrammar:
#!/usr/bin/envpython
"""
Arecursivedescentparserexample.
Usage:
pythonrparser.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonrparser.pymyfile.txt
Thegrammar:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
"""
importsys
importstring
importtypes
importgetopt

Page124

APythonBook
#
#TousetheIPythoninteractiveshelltoinspectyourrunning
#application,uncommentthefollowinglines:
#
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Thenaddthefollowinglineatthepointinyourcodewhere
#youwanttoinspectruntimevalues:
#
#ipshell('somemessagetoidentifywhereweare')
#
#Formoreinformationsee:http://ipython.scipy.org/moin/
#
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType

Page125

APythonBook
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
pass
defparseFile(self,infileName):
self.infileName=infileName
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(self.infileName,'r')
self.tokens=genTokens(self.infile)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=genTokens(instream,'<instream>')
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:

Page126

APythonBook
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokenType,self.token,self.lineNo=
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
result=self.func_call_reco()
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:

Page127

APythonBook
break
self.tokenType,self.token,self.lineNo=
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
defis_word(token):
forletterintoken:
ifletternotinstring.ascii_letters:
returnNone
return1
#
#Generatethetokens.
#Usage:
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile):
lineNo=0
while1:
lineNo+=1
try:
line=infile.next()
except:
yield(EOFTokType,None,lineNo)
toks=line.split()
fortokintoks:
ifis_word(tok):
tokType=WordTokType
eliftok=='(':
tokType=LParTokType
eliftok==')':
tokType=RParTokType
eliftok==',':
tokType=CommaTokType
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')

Page128

APythonBook
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
inputfile=args[0]
test(inputfile)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Commentsandexplanation:

ThetokenizerisaPythongenerator.ItreturnsaPythongeneratorthatcan
produce"(tokType,tok,lineNo)"tuples.Ourtokenizerissosimplemindedthat
wehavetoseparateallofourtokenswithwhitespace.(Alittlelater,we'llseehow
tousePlextoovercomethislimitation.)
Theparserclass(ProgParser)containstherecognizermethodsthatimplementthe
productionrules.Eachofthesemethodsrecognizesasyntacticconstructdefined
byarule.Inourexample,thesemethodshavenamesthatendwith"_reco".
Wecouldhave,alternatively,implementedourrecognizersasglobalfunctions,
insteadofasmethodsinaclass.However,usingaclassgivesusaplaceto"hang"
thevariablesthatareneededacrossmethodsandsavesusfromhavingtouse
("evil")globalvariables.
Arecognizermethodrecognizesterminals(syntacticelementsontherighthand
sideofthegrammarruleforwhichthereisnogrammarrule)by(1)checkingthe
tokentypeandthetokenvalue,andthen(2)callingthetokenizertogetthenext
token(becauseithasconsumedatoken).
Page129

APythonBook
Arecognizermethodchecksforandprocessesanonterminal(syntacticelements
ontherighthandsideforwhichthereisagrammarrule)bycallingtherecognizer
methodthatimplementsthatnonterminal.
Ifarecognizermethodfindsasyntaxerror,itraisesanexceptionofclass
ParserError.
SinceourexamplerecursivedescentparsercreatesanAST(anabstractsyntax
tree),wheneverarecognizermethodsuccessfullyrecognizesasyntacticconstruct,
itcreatesaninstanceofclassASTNodetorepresentitandreturnsthatinstanceto
itscaller.TheinstanceofASTNodehasanodetypeandcontainschildnodes
whichwereconstructedbyrecognizermethodscalledbythisone(i.e.that
representnonterminalsontherighthandsideofagrammarrule).
Eachtimearecognizermethod"consumesatoken",itcallsthetokenizertoget
thenexttoken(andtokentypeandlinenumber).
Thetokenizerreturnsatokentypeinadditiontothetokenvalue.Italsoreturnsa
linenumberforerrorreporting.
ThesyntaxtreeisconstructedfrominstancesofclassASTNode.
TheASTNodeclasshasashowmethod,whichwalkstheASTandproduces
output.Youcanimaginethatasimilarmethodcoulddocodegeneration.And,
youshouldconsiderthepossibilityofwritinganalogoustreewalkmethodsthat
performtaskssuchasoptimization,annotationoftheAST,etc.
And,hereisasampleofthedatawecanapplythisparserto:

aaa()
bbb(ccc())
ddd(eee(),fff(ggg(),hhh(),iii()))

And,ifweruntheparseronthethisinputdata,wesee:
$pythonworkbook045.pyworkbook045.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:bbb
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ccc
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType

Page130

APythonBook
Child:ddd
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:eee
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:fff
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ggg
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:hhh
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:iii
NodeTypeFuncCallListNodeType

2.6.3Creatingalexer/tokenizerwithPlex
LexicalanalysisThetokenizerinourrecursivedescentparserexamplewas(for
demonstrationpurposes)overlysimple.Youcanalwayswritemorecomplextokenizers
byhand.However,formorecomplex(andreal)tokenizers,youmaywanttouseatoolto
buildyourtokenizer.
Inthissectionwe'lldescribePlexanduseittoproduceatokenizerforourrecursive
descentparser.
YoucanobtainPlexathttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inordertouseit,youmaywanttoaddPlex1.1.4/PlextoyourPYTHONPATH.
HereisasimpleexamplefromthePlextutorial:
#!/usr/bin/envpython
"""
SamplePlexlexer
Usage:
pythonplex_example.pyinputfile
"""
importsys
importPlex

Page131

APythonBook
defcount_lines(scanner,text):
scanner.line_count+=1
print''*60
deftest(infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
number=Plex.Rep1(digit)
space=Plex.Any("\t")
endline=Plex.Str('\n')
#comment=Plex.Str('"')+Plex.Rep(Plex.AnyBut('"'))+
Plex.Str('"')
resword=Plex.Str("if","then","else","end")
lexicon=Plex.Lexicon([
(endline,count_lines),
(resword,'keyword'),
(name,'ident'),
(number,'int'),
(Plex.Any("+*/=<>"),'operator'),
(space,Plex.IGNORE),
#(comment,'comment'),
(Plex.Str('('),'lpar'),
(Plex.Str(')'),'rpar'),
#commentssurroundedby(*and*)
(Plex.Str("(*"),Plex.Begin('comment')),
Plex.State('comment',[
(Plex.Str("*)"),Plex.Begin('')),
(Plex.AnyChar,Plex.IGNORE),
]),
])
infile=open(infileName,"r")
scanner=Plex.Scanner(lexicon,infile,infileName)
scanner.line_count=0
whileTrue:
token=scanner.read()
iftoken[0]isNone:
break
position=scanner.position()
posstr=('(%d,%d)'%(position[1],
position[2],)).ljust(10)
tokstr='"%s"'%token[1]
tokstr=tokstr.ljust(20)
print'%stok:%stokType:%s'%(posstr,tokstr,token[0],)
print'line_count:%d'%scanner.line_count
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]

Page132

APythonBook
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Hereisabitofdataonwhichwecanusetheabovelexer:
mass=(height*(*somecomment*)width*depth)/density
totalmass=totalmass+mass

And,whenweapplytheabovetestprogramtothisdata,hereiswhatwesee:
$pythonplex_example.pyplex_example.data
(1,0)tok:"mass"tokType:ident
(1,5)tok:"="tokType:operator
(1,7)tok:"("tokType:lpar
(1,8)tok:"height"tokType:ident
(1,15)tok:"*"tokType:operator
(1,36)tok:"width"tokType:ident
(1,42)tok:"*"tokType:operator
(1,44)tok:"depth"tokType:ident
(1,49)tok:")"tokType:rpar
(1,51)tok:"/"tokType:operator
(1,53)tok:"density"tokType:ident

(2,0)tok:"totalmass"tokType:ident
(2,10)tok:"="tokType:operator
(2,12)tok:"totalmass"tokType:ident
(2,22)tok:"+"tokType:operator
(2,24)tok:"mass"tokType:ident

line_count:2

Commentsandexplanation:

Createalexiconfromscanningpatterns.
SeethePlextutorialandreference(andbelow)formoreinformationonhowto
constructthepatternsthatmatchvarioustokens.
Createascannerwithalexicon,aninputfile,andaninputfilename.
Thecall"scanner.read()"getsthenexttoken.Itreturnsatuplecontaining(1)the
tokenvalueand(2)thetokentype.
Thecall"scanner.position()"getsthepositionofthecurrenttoken.Itreturnsa
tuplecontaining(1)theinputfilename,(2)thelinenumber,and(3)thecolumn
number.
Wecanexecuteamethodwhenagiventokenisfoundbyspecifyingthefunction
asthetokenaction.Inourexample,thefunctioniscount_lines.Maintainingaline
Page133

APythonBook
countisactuallyunneeded,sincethepositiongivesusthisinformation.However,
noticehowweareabletomaintainavalue(inourcaseline_count)asan
attributeofthescanner.
And,herearesomecommentsonconstructingthepatternsusedinalexicon:
Plex.Rangeconstructsapatternthatmatchesanycharacterintherange.
Plex.Repconstructsapatternthatmatchesasequenceofzeroormoreitems.
Plex.Rep1constructsapatternthatmatchesasequenceofoneormoreitems.
pat1+pat2constructsapatternthatmatchesasequencecontainingpat1
followedbypat2.
pat1|pat2constructsapatternthatmatcheseitherpat1orpat2.
Plex.Anyconstructsapatternthatmatchesanyonecharacterinitsargument.
Nowlet'srevisitourrecursivedescentparser,thistimewithatokenizerbuiltwithPlex.
Thetokenizeristrivial,butwillserveasanexampleofhowtohookitintoaparser:

#!/usr/bin/envpython
"""
ArecursivedescentparserexampleusingPlex.
ThisexampleusesPlextoimplementatokenizer.
Usage:
pythonpython_201_rparser_plex.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonpython_201_rparser_plex.pymyfile.txt
Thegrammar:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
"""
importsys,string,types
importgetopt
importPlex
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Constants
#

Page134

APythonBook
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',

Page135

APythonBook
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=None
self.tokens=None
defparseFile(self,infileName):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(infileName,'r')
self.tokens=genTokens(self.infile,infileName)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.tokens=genTokens(self.instream,'<stream>')
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptystream'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break

Page136

APythonBook
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokenType,self.token,self.lineNo=
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
result=self.func_call_reco()
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokenType,self.token,self.lineNo=
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):

Page137

APythonBook
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
#
#Generatethetokens.
#Usageexample
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile,infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
lpar=Plex.Str('(')
rpar=Plex.Str(')')
comma=Plex.Str(',')
comment=Plex.Str("#")+Plex.Rep(Plex.AnyBut("\n"))
space=Plex.Any("\t\n")
lexicon=Plex.Lexicon([
(name,'word'),
(lpar,'lpar'),
(rpar,'rpar'),
(comma,'comma'),
(comment,Plex.IGNORE),
(space,Plex.IGNORE),
])
scanner=Plex.Scanner(lexicon,infile,infileName)
while1:
tokenType,token=scanner.read()
name,lineNo,columnNo=scanner.position()
iftokenType==None:
tokType=EOFTokType
token=None
eliftokenType=='word':
tokType=WordTokType
eliftokenType=='lpar':
tokType=LParTokType
eliftokenType=='rpar':
tokType=RParTokType
eliftokenType=='comma':
tokType=CommaTokType
else:
tokType=NoneTokType
tok=token
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()

Page138

APythonBook
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

And,hereisasampleofthedatawecanapplythisparserto:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest

And,whenwerunourparser,itproducesthefollowing:
$pythonplex_recusive.pyplex_recusive.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType

Page139

APythonBook
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:bbb
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ccc
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ddd
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:eee
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:fff
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ggg
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:hhh
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:iii
NodeTypeFuncCallListNodeType

Comments:

Wecannowputcommentsinourinput,andtheywillbeignored.Comments
beginwitha"#"andcontinuetotheendofline.Seethedefinitionofcommentin
functiongenTokens.
Thistokenizerdoesnotrequireustoseparatetokenswithwhitespaceasdidthe
simpletokenizerintheearlierversionofourrecursivedescentparser.
Thechangeswemadeovertheearlierversionwereto:
1. ImportPlex.
2. ReplacethedefinitionofthetokenizerfunctiongenTokens.
3. ChangethecalltogenTokenssothatthecallpassesinthefilename,whichis
neededtocreatethescanner.
OurnewversionofgenTokensdoesthefollowing:
1. Createpatternsforscanning.
2. Createalexicon(aninstanceofPlex.Lexicon),whichusesthepatterns.
Page140

APythonBook
3. Createascanner(aninstanceofPlex.Scanner),whichusesthelexicon.
4. Executealoopthatreadstokens(fromthescanner)and"yields"eachone.

2.6.4Asurveyofexistingtools
Forcomplexparsingtasks,youmaywanttoconsiderthefollowingtools:
kwParsingAparsergeneratorinPython
http://gadfly.sourceforge.net/kwParsing.html
PLYPythonLexYacchttp://systems.cs.uchicago.edu/ply/
PyLRFastLRparsinginpython
http://starship.python.net/crew/scott/PyLR.html
YappsTheYappsParserGeneratorSystem
http://theory.stanford.edu/~amitp/Yapps/
And,forlexicalanalysis,youmayalsowanttolookhere:

UsingRegularExpressionsforLexicalAnalysis
http://effbot.org/zone/xmlscanner.htm
Plexhttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inthesectionsbelow,wegiveexamplesandnotesabouttheuseofPLYandpyparsing.

2.6.5CreatingaparserwithPLY
InthissectionwewillshowhowtoimplementourparserexamplewithPLY.
FirstdownloadPLY.Itisavailablehere:PLY(PythonLexYacc)
http://www.dabeaz.com/ply/
ThenaddthePLYdirectorytoyourPYTHONPATH.
LearnhowtoconstructlexersandparserswithPLYbyreadingdoc/ply.htmlinthe
distributionofPLYandbylookingattheexamplesinthedistribution.
Forthoseofyouwhowantamorecomplexexample,seeAPythonParserforthe
RELAXNGCompactSyntax,whichisimplementedwithPLY.
Now,hereisourexampleparser.Commentsandexplanationsarebelow:
#!/usr/bin/envpython
"""
Aparserexample.
ThisexampleusesPLYtoimplementalexerandparser.
Thegrammar:
Prog::=Command*
Command::=Func_call

Page141

APythonBook
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call*
Term=<word>
Hereisasample"program"touseasinput:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest
"""
importsys
importtypes
importgetopt
importply.lexaslex
importply.yaccasyacc
#
#Globals
#
startlinepos=0
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
CommandListNodeType=3
FuncCallNodeType=4
FuncCallListNodeType=5
TermNodeType=6
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
CommandListNodeType:'CommandListNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#

Page142

APythonBook
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defappend(self,item):
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType:%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Value:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Exceptionclasses
#
classLexerError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Lexererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
classParserError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Parsererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
#
#Lexerspecification
#
tokens=(
'NAME',

Page143

APythonBook
'LPAR','RPAR',
'COMMA',
)
#Tokens
t_LPAR=r'\('
t_RPAR=r'\)'
t_COMMA=r'\,'
t_NAME=r'[azAZ_][azAZ09_]*'
#Ignorewhitespace
t_ignore='\t'
#Ignorecomments('#'toendofline)
deft_COMMENT(t):
r'\#[^\n]*'
pass
deft_newline(t):
r'\n+'
globalstartlinepos
startlinepos=t.lexer.lexpos1
t.lineno+=t.value.count("\n")
deft_error(t):
globalstartlinepos
msg="Illegalcharacter'%s'"%(t.value[0])
columnno=t.lexer.lexposstartlinepos
raiseLexerError(msg,t.lineno,columnno)
#
#Parserspecification
#
defp_prog(t):
'prog:command_list'
t[0]=ASTNode(ProgNodeType,t[1])
defp_command_list_1(t):
'command_list:command'
t[0]=ASTNode(CommandListNodeType,t[1])
defp_command_list_2(t):
'command_list:command_listcommand'
t[1].append(t[2])
t[0]=t[1]
defp_command(t):
'command:func_call'
t[0]=ASTNode(CommandNodeType,t[1])
defp_func_call_1(t):
'func_call:termLPARRPAR'

Page144

APythonBook
t[0]=ASTNode(FuncCallNodeType,t[1])
defp_func_call_2(t):
'func_call:termLPARfunc_call_listRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1],t[3])
defp_func_call_list_1(t):
'func_call_list:func_call'
t[0]=ASTNode(FuncCallListNodeType,t[1])
defp_func_call_list_2(t):
'func_call_list:func_call_listCOMMAfunc_call'
t[1].append(t[3])
t[0]=t[1]
defp_term(t):
'term:NAME'
t[0]=ASTNode(TermNodeType,t[1])
defp_error(t):
globalstartlinepos
msg="Syntaxerrorat'%s'"%t.value
columnno=t.lexer.lexposstartlinepos
raiseParserError(msg,t.lineno,columnno)
#
#ParsetheinputanddisplaytheAST(abstractsyntaxtree)
#
defparse(infileName):
startlinepos=0
#Buildthelexer
lex.lex(debug=1)
#Buildtheparser
yacc.yacc()
#Readtheinput
infile=file(infileName,'r')
content=infile.read()
infile.close()
try:
#Dotheparse
result=yacc.parse(content)
#DisplaytheAST
result.show(0)
exceptLexerError,exp:
exp.show()
exceptParserError,exp:
exp.show()
USAGE_TEXT=__doc__
defusage():
printUSAGE_TEXT
sys.exit(1)

Page145

APythonBook
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
infileName=args[0]
parse(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Applyingthisparsertothefollowinginput:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest

producesthefollowingoutput:
NodeType:ProgNodeType
NodeType:CommandListNodeType
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:aaa
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:bbb
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ccc
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ddd
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType

Page146

APythonBook
NodeType:TermNodeType
Value:eee
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:fff
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ggg
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:hhh
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:iii

Commentsandexplanation:

CreatingthesyntaxtreeBasically,eachrule(1)recognizesanonterminal,(2)
createsanode(possiblyusingthevaluesfromtherighthandsideoftherule),and
(3)returnsthenodebysettingthevalueoft[0].Adeviationfromthisisthe
processingofsequences,discussedbelow.
Sequencesp_command_list_1andp_command_list_1showhowtohandle
sequencesofitems.Inthiscase:
p_command_list_1recognizesacommandandcreatesaninstanceof
ASTNodewithtypeCommandListNodeTypeandaddsthecommandtoitasa
child,and
p_command_list_2recognizesanadditionalcommandandaddsit(asachild)
totheinstanceofASTNodethatrepresentsthelist.
DistinguishingbetweendifferentformsofthesameruleInordertoprocess
alternativestothesameproductionruledifferently,weusedifferentfunctions
withdifferentimplementations.Forexample,weuse:
p_func_call_1torecognizeandprocess"func_call:termLPARRPAR"(a
functioncallwithoutarguments),and
p_func_call_2torecognizeandprocess"func_call:termLPARfunc_call_list
RPAR"(afunctioncallwitharguments).
ReportingerrorsOurparserreportsthefirsterrorandquits.We'vedonethisby
raisinganexceptionwhenwefindanerror.Weimplementtwoexceptionclasses:
LexerErrorandParserError.Implementingmorethanoneexceptionclassenables
ustodistinguishbetweendifferentclassesoferrors(notethemultipleexcept:
clausesonthetry:statementinfunctionparse).And,weuseaninstanceofthe
exceptionclassasacontainerinorderto"bubbleup"informationabouttheerror
(e.g.amessage,alinenumber,andacolumnnumber).

Page147

APythonBook

2.6.6Creatingaparserwithpyparsing
pyparsingisarelativelynewparsingpackageforPython.Itwasimplementedandis
supportedbyPaulMcGuireanditshowspromise.Itappearsespeciallyeasytouseand
seemsespeciallyappropriateinparticularforquickparsingtasks,althoughithasfeatures
thatmakesomecomplexparsingtaskseasy.ItfollowsaverynaturalPythonstylefor
constructingparsers.
Gooddocumentationcomeswiththepyparsingdistribution.Seefile
HowToUseParsing.html.So,Iwon'ttrytorepeatthathere.Whatfollowsisanattemptto
provideseveralquickexamplestohelpyousolvesimpleparsingtasksasquicklyas
possible.
Youwillalsowanttolookatthesamplesintheexamplesdirectory,whicharevery
helpful.Myexamplesbelowarefairlysimple.Youcanseemoreoftheabilityof
pyparsingtohandlecomplextasksintheexamples.
WheretogetitYoucanfindpyparsingat:PyparsingWikiHome
http://pyparsing.wikispaces.com/
HowtoinstallitPutthepyparsingmodulesomewhereonyourPYTHONPATH.
Andnow,hereareafewexamples.
2.6.6.1Parsingcommadelimitedlines

Note:Thisexampleisfordemonstrationpurposesonly.Ifyoureallytoneedtoparse
commadelimitedfields,youcanprobablydosomuchmoreeasilywiththeCSV(comma
separatedvalues)moduleinthePythonstandardlibrary.
Hereisasimplegrammarforlinescontainingfieldsseparatedbycommas:
importsys
frompyparsingimportalphanums,ZeroOrMore,Word
fieldDef=Word(alphanums)
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test1.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
fields=lineDef.parseString(line)
printfields

Page148

APythonBook
test()

Hereissomesampledata:
abcd,defg
11111,22222,33333

And,whenwerunourparseronthisdatafile,hereiswhatwesee:
$pythoncomma_parser.pysample1.data
['abcd',',','defg']
['11111',',','22222',',','33333']

Notesandexplanation:

NotehowthegrammarisconstructedfromnormalPythoncallstofunctionand
object/classconstructors.I'veconstructedtheparserinlinebecausemyexample
issimple,butconstructingtheparserinafunctionorevenamodulemightmake
senseformorecomplexgrammars.pyparsingmakesiteasytousethesethese
differentstyles.
Use"+"tospecifyasequence.Inourexample,alineDefisafieldDef
followedby....
UseZeroOrMoretospecifyrepetition.Inourexample,alineDefisa
fieldDeffollowedbyzeroormoreoccurancesofcommaandfieldDef.
ThereisalsoOneOrMorewhenyouwanttorequireatleastoneoccurance.
Parsingcommadelimitedtexthappenssofrequentlythatpyparsingprovidesa
shortcut.Replace:
lineDef=fieldDef+ZeroOrMore(","+fieldDef)

with:
lineDef=delimitedList(fieldDef)

AndnotethatdelimitedListtakesanoptionalargumentdelimusedtospecify
thedelimiter.Thedefaultisacomma.
2.6.6.2Parsingfunctors

Thisexampleparsesexpressionsoftheformfunc(arg1,arg2,arg3):
frompyparsingimportWord,alphas,alphanums,nums,ZeroOrMore,
Literal
lparen=Literal("(")
rparen=Literal(")")
identifier=Word(alphas,alphanums+"_")
integer=Word(nums)
functor=identifier
arg=identifier|integer

Page149

APythonBook
args=arg+ZeroOrMore(","+arg)
expression=functor+lparen+args+rparen
deftest():
content=raw_input("Enteranexpression:")
parsedContent=expression.parseString(content)
printparsedContent
test()

Explanation:

UseLiteraltospecifyafixedstringthatistobematchedexactly.Inourexample,
alparenisa(.
Wordtakesanoptionalsecondargument.Withasingle(string)argument,it
matchesanycontiguouswordmadeupofcharactersinthestring.Withtwo
(string)argumentsitmatchesawordwhosefirstcharacterisinthefirststringand
whoseremainingcharactersareinthesecondstring.So,ourdefinitionof
identifiermatchesawordwhosefirstcharacterisanalphaandwhoseremaining
charactersarealphanumericsorunderscore.Asanotherexample,youcanthink
ofWord("0123456789")asanalogoustoaregexpcontainingthepattern"[09]+".
Useaverticalbarforalternation.Inourexample,anargcanbeeitheranidentifier
oraninteger.

2.6.6.3Parsingnames,phonenumbers,etc.

Thisexampleparsesexpressionshavingthefollowingform:
Inputformat:
[name][phone][city,statezip]
Last,first1112223333city,ca99999

Hereistheparser:
importsys
frompyparsingimportalphas,nums,ZeroOrMore,Word,Group,
Suppress,Combine
lastname=Word(alphas)
firstname=Word(alphas)
city=Group(Word(alphas)+ZeroOrMore(Word(alphas)))
state=Word(alphas,exact=2)
zip=Word(nums,exact=5)
name=Group(lastname+Suppress(",")+firstname)
phone=Combine(Word(nums,exact=3)+""+Word(nums,exact=3)+""
+Word(nums,exact=4))
location=Group(city+Suppress(",")+state+zip)
record=name+phone+location

Page150

APythonBook
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test3.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
line=line.strip()
iflineandline[0]!="#":
fields=record.parseString(line)
printfields
test()

And,hereissomesampleinput:
Jabberer,Jerry1112223333Bakersfield,CA95111
Kackler,Kerry1112223334Fresno,CA95112
Louderdale,Larry1112223335LosAngeles,CA94001

Hereisoutputfromparsingtheaboveinput:
[['Jabberer','Jerry'],'1112223333',[['Bakersfield'],'CA',
'95111']]
[['Kackler','Kerry'],'1112223334',[['Fresno'],'CA','95112']]
[['Louderdale','Larry'],'1112223335',[['Los','Angeles'],'CA',
'94001']]

Comments:

Weusethelen=nargumenttotheWordconstructortoresticttheparserto
acceptingaspecificnumberofcharacters,forexampleinthezipcodeandphone
number.Wordalsoacceptsmin=n''and``max=ntoenableyoutorestrict
thelengthofawordtowithinarange.
WeuseGrouptogrouptheparsedresultsintosublists,forexampleinthe
definitionofcityandname.Groupenablesustoorganizetheparseresultsinto
simpleparsetrees.
WeuseCombinetojoinparsedresultsbackintoasinglestring.Forexample,in
thephonenumber,wecanrequiredashesandyetjointheresultsbackintoa
singlestring.
WeuseSuppresstoremoveunneededsubelementsfromparsedresults.For
example,wedonotneedthecommabetweenlastandfirstname.

2.6.6.4Amorecomplexexample

Thisexample(thankstoPaulMcGuire)parsesamorecomplexstructureandproducesa
dictionary.
Page151

APythonBook
Hereisthecode:
frompyparsingimportLiteral,Word,Group,Dict,ZeroOrMore,alphas,
nums,\
delimitedList
importpprint
testData="""
++++++++++
||A1|B1|C1|D1|A2|B2|C2|D2|
+=======+======+======+======+======+======+======+======+======+
|min|7|43|7|15|82|98|1|37|
|max|11|52|10|17|85|112|4|39|
|ave|9|47|8|16|84|106|3|38|
|sdev|1|3|1|1|1|3|1|1|
++++++++++
"""
#Definegrammarfordatatable
heading=(Literal(
"++++++++++")
+
"||A1|B1|C1|D1|A2|B2|C2|D2|"+
"+=======+======+======+======+======+======+======+======+======+").
suppress()
vert=Literal("|").suppress()
number=Word(nums)
rowData=Group(vert+Word(alphas)+vert+
delimitedList(number,"|")+
vert)
trailing=Literal(
"++++++++++").
suppress()
datatable=heading+Dict(ZeroOrMore(rowData))+trailing
defmain():
#Nowparsedataandprintresults
data=datatable.parseString(testData)
print"data:",data
print"data.asList():",
pprint.pprint(data.asList())
print"datakeys:",data.keys()
print"data['min']:",data['min']
print"data.max:",data.max
if__name__=='__main__':
main()

Whenwerunthis,itproducesthefollowing:
data:[['min','7','43','7','15','82','98','1','37'],

Page152

APythonBook
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
data.asList():[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
datakeys:['ave','min','sdev','max']
data['min']:['7','43','7','15','82','98','1','37']
data.max:['11','52','10','17','85','112','4','39']

Notes:

NotetheuseofDicttocreateadictionary.Theprintstatementsshowhowtoget
attheitemsinthedictionary.
NotehowwecanalsogettheparseresultsasalistbyusingmethodasList.
Again,weusesuppresstoremoveunneededitemsfromtheparseresults.

2.7GUIApplications
2.7.1Introduction
ThissectionwillhelpyoutoputaGUI(graphicaluserinterface)inyourPython
program.
WewilluseaparticularGUIlibrary:PyGTK.We'vechosenthisbecauseitisreasonably
lightweightandourgoalistoembedlightweightGUIinterfacesinan(possibly)
existingapplication.
ForsimplerGUIneeds,considerEasyGUI,whichisalsodescribedbelow.
FormoreheavyweightGUIneeds(forexample,completeGUIapplications),youmay
wanttoexploreWxPython.SeetheWxPythonhomepageat:http://www.wxpython.org/

2.7.2PyGtk
InformationaboutPyGTKishere:ThePyGTKhomepagehttp://www.pygtk.org//.
2.7.2.1Asimplemessagedialogbox

InthissectionweexplainhowtopopupasimpledialogboxfromyourPython
application.
Todothis,dothefollowing:
1. ImportgtkintoyourPythonmodule.
2. Definethedialoganditsbehavior.
Page153

APythonBook
3. Createaninstanceofthedialog.
4. Runtheeventloop.
Hereisasamplethatdisplaysamessagebox:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classMessageBox(gtk.Dialog):
def__init__(self,message="",buttons=(),pixmap=None,
modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
hbox=gtk.HBox(spacing=5)
hbox.set_border_width(5)
self.vbox.pack_start(hbox)
hbox.show()
ifpixmap:
self.realize()
pixmap=Pixmap(self,pixmap)
hbox.pack_start(pixmap,expand=False)
pixmap.show()
label=gtk.Label(message)
hbox.pack_start(label)
label.show()
fortextinbuttons:
b=gtk.Button(text)
b.set_flags(gtk.CAN_DEFAULT)
b.set_data("user_data",text)
b.connect("clicked",self.click)
self.action_area.pack_start(b)
b.show()
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=button.get_data("user_data")
self.quit()
#createamessagebox,andreturnwhichbuttonwaspressed
defmessage_box(title="MessageBox",message="",buttons=(),
pixmap=None,
modal=True):
win=MessageBox(message,buttons,pixmap=pixmap,modal=modal)
win.set_title(title)

Page154

APythonBook
win.show()
gtk.main()
returnwin.ret
deftest():
result=message_box(title='Test#1',
message='Hereisyourmessage',
buttons=('Ok','Cancel'))
print'result:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Someexplanation:

First,weimportgtk
NextwedefineaclassMessageBoxthatimplementsamessagebox.Herearea
fewimportantthingstoknowaboutthatclass:
Itisasubclassofgtk.Dialog.
Itcreatesalabelandpacksitintothedialog'sclientarea.NotethataDialogis
aWindowthatcontainsavboxatthetopofandanaction_areaatthebottom
ofitsclientarea.Theintensionisforustopackmiscellaneouswidgetsinto
thevboxandtoputbuttonssuchas"Ok","Cancel",etcintotheaction_area.
Page155

APythonBook
Itcreatesonebuttonforeachbuttonlabelpassedtoitsconstructor.The
buttonsareallconnectedtotheclickmethod.
Theclickmethodsavesthevalueoftheuser_dataforthebuttonthatwas
clicked.Inourexample,thisvaluewillbeeither"Ok"or"Cancel".
And,wedefineafunction(message_box)that(1)createsaninstanceofthe
MessageBoxclass,(2)setsitstitle,(3)showsit,(4)startsitseventloopsothatit
cangetandprocesseventsfromtheuser,and(5)returnstheresulttothecaller(in
thiscase"Ok"or"Cancel").
Ourtestingfunction(test)callsfunctionmessage_boxandprintstheresult.
Thislookslikequiteabitofcode,untilyounoticethattheclassMessageBoxand
thefunctionmessage_boxcouldbeputitautilitymoduleandreused.

2.7.2.2Asimpletextinputdialogbox

And,hereisanexamplethatdisplaysantextinputdialog:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classEntryDialog(gtk.Dialog):
def__init__(self,message="",default_text='',modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
box=gtk.VBox(spacing=10)
box.set_border_width(10)
self.vbox.pack_start(box)
box.show()
ifmessage:
label=gtk.Label(message)
box.pack_start(label)
label.show()
self.entry=gtk.Entry()
self.entry.set_text(default_text)
box.pack_start(self.entry)
self.entry.show()
self.entry.grab_focus()
button=gtk.Button("OK")
button.connect("clicked",self.click)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
button.grab_default()
button=gtk.Button("Cancel")

Page156

APythonBook
button.connect("clicked",self.quit)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
self.ret=None
defquit(self,w=None,event=None):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=self.entry.get_text()
self.quit()
definput_box(title="InputBox",message="",default_text='',
modal=True):
win=EntryDialog(message,default_text,modal=modal)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deftest():
result=input_box(title='Test#2',
message='Enteravaluexxx:',
default_text='adefaultvalue')
ifresultisNone:
print'Canceled'
else:
print'result:"%s"'%result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()

Page157

APythonBook
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Mostoftheexplanationforthemessageboxexampleisrelevanttothisexample,too.
Herearesomedifferences:

OurEntryDialogclassconstructorcreatesinstanceofgtk.Entry,setsitsdefault
value,andpacksitintotheclientarea.
Theconstructoralsoautomaticallycreatestwobuttons:"OK"and"Cancel".The
"OK"buttonisconnecttotheclickmethod,whichsavesthevalueoftheentry
field.The"Cancel"buttonisconnecttothequitmethod,whichdoesnotsavethe
value.
And,ifclassEntryDialogandfunctioninput_boxlookusableanduseful,add
themtoyourutilityguimodule.

2.7.2.3Afileselectiondialogbox

Thisexampleshowsafileselectiondialogbox:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classFileChooser(gtk.FileSelection):
def__init__(self,modal=True,multiple=True):
gtk.FileSelection.__init__(self)
self.multiple=multiple
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
self.cancel_button.connect('clicked',self.quit)
self.ok_button.connect('clicked',self.ok_cb)
ifmultiple:
self.set_select_multiple(True)
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defok_cb(self,b):
ifself.multiple:
self.ret=self.get_selections()

Page158

APythonBook
else:
self.ret=self.get_filename()
self.quit()
deffile_sel_box(title="Browse",modal=False,multiple=True):
win=FileChooser(modal=modal,multiple=multiple)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deffile_open_box(modal=True):
returnfile_sel_box("Open",modal=modal,multiple=True)
deffile_save_box(modal=True):
returnfile_sel_box("SaveAs",modal=modal,multiple=False)
deftest():
result=file_open_box()
print'openresult:',result
result=file_save_box()
print'saveresult:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
main()
#importpdb
#pdb.run('main()')

Page159

APythonBook
Alittleguidance:
Thereisapredefinedfileselectiondialog.Wesubclassit.
Thisexampledisplaysthefileselectiondialogtwice:oncewithatitle"Open"and
oncewithatitle"SaveAs".
Notehowwecancontrolwhetherthedialogallowsmultiplefileselections.And,
ifweselectthemultipleselectionmode,thenweuseget_selectionsinsteadof
get_filenameinordertogettheselectedfilenames.
Thedialogcontainsbuttonsthatenabletheuserto(1)createanewfolder,(2)
deleteafile,and(3)renameafile.Ifyoudonotwanttheusertoperformthese
operations,thencallhide_fileop_buttons.Thiscalliscommentedoutinour
samplecode.
Notethattherearealsopredefineddialogsforfontselection(FontSelectionDialog)and
colorselection(ColorSelectionDialog)

2.7.3EasyGUI
IfyourGUIneedsareminimalist(maybeapopupdialogortwo)andyourapplicationis
imperativeratherthaneventdriven,thenyoumaywanttoconsiderEasyGUI.Asthe
namesuggests,itisextremelyeasytouse.
HowtoknowwhenyoumightbeabletouseEasyGUI:
Yourapplicationdoesnotneedtoruninawindowcontainingmenusandamenu
bar.
YourGUIneedsamounttolittlemorethandisplayingadialognowandthento
getresponsesfromtheuser.
Youdonotwanttowriteaneventdrivenapplication,thatis,oneinwhichyour
codesitsandwaitsforthetheusertoinitiateoperation,forexample,withmenu
items.
EasyGUIplusdocumentationandexamplesareavailableatEasyGUIhomepageat
SourceForgehttp://easygui.sourceforge.net/

EasyGUIprovidesfunctionsforavarietyofcommonlyneededdialogboxes,including:

Amessageboxdisplaysamessage.
Ayes/nomessageboxdisplays"Yes"and"No"buttons.
Acontinue/cancelmessageboxdisplays"Continue"and"Cancel"buttons.
Achoiceboxdisplaysaselectionlist.
Anenterboxallowsentryofalineoftext.
Anintegerboxallowsentryofaninterger.
Amultipleentryboxallowsentryintomultiplefields.
Codeandtextboxessupportthedisplayoftextinmonospacedorporportional
Page160

APythonBook
fonts.
Fileanddirectoryboxesenabletheusertoselectafileoradirectory.
SeethedocumentationattheEasyGUIWebsiteformorefeatures.
ForademonstrationofEasyGUI'scapabilities,runtheeasygui.pyasaPythonscript:
$pythoneasygui.py

2.7.3.1AsimpleEasyGUIexample

Hereisasimpleexamplethatpromptstheuserforanentry,thenshowstheresponseina
messagebox:
importeasygui
deftesteasygui():
response=easygui.enterbox(msg='Enteryourname:',title='Name
Entry')
easygui.msgbox(msg=response,title='YourResponse')
testeasygui()

2.7.3.2AnEasyGUIfileopendialogexample

Thisexamplepresentsadialogtoallowtheusertoselectafile:
importeasygui
deftest():
response=easygui.fileopenbox(msg='Selectafile')
print'filename:%s'%response
test()

2.8GuidanceonPackagesandModules
2.8.1Introduction
Pythonhasanexcellentrangeofimplementationorganizationstructures.Theserange
fromstatementsandcontrolstructures(atalowlevel)throughfunctions,methods,and
classes(atanintermediatelevel)andmodulesandpackagesatanupperlevel.
Thissectionprovidessomeguidancewiththeuseofpackages.Inparticular:

Howtoconstructandimplementthem.
Howtousethem.
Howtodistributeandinstallthem.
Page161

APythonBook

2.8.2ImplementingPackages
APythonpackageisacollectionofPythonmodulesinadiskdirectory.
Inordertobeabletoimportindividualmodulesfromadirectory,thedirectorymust
containafilenamed__init__.py.(Notethatrequirementdoesnotapplytodirectoriesthat
arelistedinPYTHONPATH.)The__init__.pyservesseveralpurposes:

Thepresenceofthefile__init__.pyinadirectorymarksthedirectoryasaPython
package,whichenablesimportingmodulesfromthedirectory.
Thefirsttimeanapplicationimportsanymodulefromthedirectory/package,the
codeinthemodule__init__isevaluated.
Ifthepackageitselfisimported(asopposedtoanindividualmodulewithinthe
directory/package),thenitisthe__init__thatisimported(andevaluated).

2.8.3UsingPackages
Onesimplewaytoenabletheusertoimportanduseapackageistoinstructtheuseto
importindividualmodulesfromthepackage.
Asecond,slightlymoreadvancedwaytoenabletheusertoimportthepackageisto
exposethosefeaturesofthepackageinthe__init__module.Supposethatmodulemod1
containsfunctionsfun1aandfun1bandsupposethatmodulemod2containsfunctions
fun2aandfun2b.Thenfile__init__.pymightcontainthefollowing:
frommod1importfun1a,fun1b
frommod2importfun2a,fun2b

Then,ifthefollowingisevaluatedintheuser'scode:
importtestpackages

Thentestpackageswillcontainfun1a,fun1b,fun2a,andfun2b.
Forexample,hereisaninteractivesessionthatdemostratesimportingthepackage:
>>>importtestpackages
>>>printdir(testpackages)
[`__builtins__',`__doc__',`__file__',`__name__',
`__path__',
`fun1a',`fun1b',`fun2a',`fun2b',`mod1',`mod2']

2.8.4DistributingandInstallingPackages
Distutils(PythonDistributionUtilities)hasspecialsupportfordistrubutingandinstalling
packages.Learnmorehere:DistributingPythonModules
http://docs.python.org/distutils/index.html.
Page162

APythonBook
Asourexample,imaginethatwehaveadirectorycontainingthefollowing:
Testpackages
Testpackages/README
Testpackages/MANIFEST.in
Testpackages/setup.py
Testpackages/testpackages/__init__.py
Testpackages/testpackages/mod1.py
Testpackages/testpackages/mod2.py

NoticethesubdirectoryTestpackages/testpackagescontainingthefile__init__.py.
ThisisthePythonpackagethatwewillinstall.
We'lldescribehowtoconfiguretheabovefilessothattheycanbepackagedasasingle
distributionfileandsothatthePythonpackagetheycontaincanbeinstalledasapackage
byDistutils.
TheMANIFEST.infileliststhefilesthatwewantincludedinourdistribution.Hereis
thecontentsofourMANIFEST.infile:
includeREADMEMANIFESTMANIFEST.in
includesetup.py
includetestpackages/*.py

Thesetup.pyfiledescribestoDistutils(1)howtopackagethedistributionfileand(2)
howtoinstallthedistribution.Hereisthecontentsofoursamplesetup.py:
#!/usr/bin/envpython
fromdistutils.coreimportsetup#[1]
long_description='TestsforinstallinganddistributingPython
packages'
setup(name='testpackages',#[2]
version='1.0a',
description='TestsforPythonpackages',
maintainer='DaveKuhlman',
maintainer_email='[email protected]',
url='http://www.rexx.com/~dkuhlman',
long_description=long_description,
packages=['testpackages']#[3]
)

Explanation:
1. WeimportthenecessarycomponentfromDistutils.
2. Wedescribethepackageanditsdeveloper/maintainer.
3. Wespecifythedirectorythatistobeinstalledasapackage.Whentheuser
installsourdistribution,thisdirectoryandallthemodulesinitwillbeinstalledas
apackage.
Page163

APythonBook
Now,tocreateadistributionfile,werunthefollowing:
pythonsetup.pysdistformats=gztar

whichwillcreateafiletestpackages1.0a.tar.gzunderthedirectorydist.
Then,youcangivethisdistributionfiletoapotentialuser,whocaninstallitbydoingthe
following:
$tarxvzftestpackages1.0a.tar.gz
$cdtestpackages1.0a
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot

2.9EndMatter
2.9.1AcknowledgementsandThanks

ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
ThankstoDaveBeazleyandothersforSWIGandPLY.
ThankstoGregEwingforPyrexandPlex.
ThankstoJamesHenstridgeforPyGTK.

2.9.2SeeAlso

ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
PythonDocumentationhttp://www.python.org/doc/forlotsofdocumentation
onPython
Dave'sWebSitehttp://http://www.davekuhlman.orgformoresoftwareand
informationonusingPythonforXMLandtheWeb.
TheSWIGhomepagehttp://www.swig.orgformoreinformationonSWIG
(SimplifiedWrapperandInterfaceGenerator).
ThePyrexhomepagehttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
formoreinformationonPyrex.
PLY(PythonLexYacc)homepagehttp://www.dabeaz.com/ply/formore
informationonPLY.
ThePlexhomepagehttp://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/
formoreinformationonPlex.
DistributingPythonModuleshttp://docs.python.org/distutils/index.htmlfor
informationonthePythonDistributionUtilities(Distutils).

Page164

APythonBook

3Part3PythonWorkbook
3.1Introduction
ThisdocumenttakesaworkbookandexercisewithsolutionsapproachtoPython
training.Itishopedthatthosewhofeelaneedforlessexplanationandmorepractical
exerciseswillfindthisuseful.
Afewnotesabouttheexercises:
I'vetriedtoincludesolutionsformostoftheexercises.Hopefully,youwillbe
abletocopyandpastethesesolutionsintoyourtexteditor,thenextendand
experimentwiththem.
IusetwointeractivePythoninterpreters(althoughtheyarethesamePython
underneath).Whenyouseethisprompt>>>,it'sthestandardPythoninterpreter.
And,whenyouseethispromptIn[1]:,it'sIPython
http://ipython.scipy.org/moin/.
ThelatestversionofthisdocumentisatmyWebsite(URLabove).

Ifyouhavecommentsorsuggestions,pleasesendthemmyway.

3.2LexicalStructures
3.2.1Variablesandnames
Anameisanycombinationofletters,digits,andtheunderscore,butthefirstcharacter
mustbealetteroranunderscore.Namesmaybeofanylength.
Caseissignificant.
Exercises:
1. Whichofthefollowingarevalidnames?
1. total
2. total_of_all_vegetables
3. bigtitle1
4. _inner_func
5. 1bigtitle
6. bigtitle1
2. Whichorthefollowingpairsarethesamename:
1. the_last_itemandthe_last_item
Page165

APythonBook
2. the_last_itemandThe_Last_Item
3. itemianditemj
4. item1anditeml
Solutions:
1. Items1,2,4,and6arevalid.Item3isnotasinglename,butisthreeitems
separatedbytheminusoperator.Item5isnotvalidbecauseitbeginswithadigit.
2. Pythonnamesarecasesensitive,whichmeans:
1. the_last_itemandthe_last_itemarethesame.
2. the_last_itemandThe_Last_ItemaredifferentThesecondname
hasanuppercasecharacters.
3. itemianditemjaredifferent.
4. item1anditemlaredifferentThisonemaybedifficulttosee,
dependingonthefontyouareviewing.Onenameendswiththedigitone;the
otherendswiththealphacharacter"el".Andthisexampleprovidesagood
reasontouse"1"and"l"judiciouslyinnames.
ThefollowingarekeywordsinPythonandshouldnotbeusedasvariablenames:
anddelfromnotwhile
aselifglobalorwith
assertelseifpassyield
breakexceptimportprint
classexecinraise
continuefinallyisreturn
defforlambdatry

Exercises:
1. WhichofthefollowingarevalidnamesinPython?
1. _global
2. global
3. file
Solutions:
1. Donotusekeywordsforvariablenames:
1. Valid
2. Notavalidname."global"isakeyword.
3. Valid,however,"file"isthenameofabuiltintype,asyouwilllearnlater,so
youareadvisednottoredefineit.Hereareafewofthenamesofbuiltin
types:"file","int","str","float","list","dict",etc.SeeBuiltinTypes
http://docs.python.org/lib/types.htmlformorebuiltintypes..
ThefollowingareoperatorsinPythonandwillseparatenames:
+***///%
<<>>&|^~

Page166

APythonBook
<><=>===!=<>
andorisnotin
Also:()[].(dot)

But,notethatthePythonstyleguidesuggeststhatyouplaceblanksaroundbinary
operators.Oneexceptiontothisruleisfunctionargumentsandparametersforfunctions:
itissuggestedthatyounotputblanksaroundtheequalsign(=)usedtospecifykeyword
argumentsanddefaultparameters.
Exercises:
1. Whichofthefollowingaresinglenamesandwhicharenamesseparatedby
operators?
1. fruit_collection
2. fruitcollection
Solutions:
1. Donotuseadash,orotheroperator,inthemiddleofaname:
1. fruit_collectionisasinglename
2. fruitcollectionistwonamesseparatedbyadash.

3.2.2Linestructure
InPython,normallywewriteonestatementperline.Infact,Pythonassumesthis.
Therefore:
Statementseparatorsarenotnormallyneeded.
But,ifwewantmorethanonestatementonaline,weuseastatementseparator,
specificallyasemicolon.
And,ifwewanttoextendastatementtoasecondorthirdlineandsoon,we
sometimesneedtodoabitextra.
ExtendingaPythonstatementtoasubsequentlineFollowthesetworules:

1. Ifthereisanopencontext,nothingspecialneedbedonetoextendastatement
acrossmultiplelines.Anopencontextisanopenparenthesis,anopensquare
bracket,oranopencurlybracket.
2. Wecanalwaysextendastatementonafollowinglinebyplacingabackslashas
thelastcharacteroftheline.
Exercises:
1. Extendthefollowingstatementtoasecondlineusingparentheses:
total_count=tree_count+vegetable_count+
fruit_count

2. Extendthefollowingstatementtoasecondlineusingthebackslashline
Page167

APythonBook
continuationcharacter:
total_count=tree_count+vegetable_count+
fruit_count

Solutions:
1. ParenthesescreateanopencontextthattellsPythonthatastatementextendsto
thenextline:
total_count=(tree_count+
vegetable_count+fruit_count)

2. AbackslashasthelastcharacteronlinetellsPythonthatthecurrentstatement
extendstothenextline:
total_count=tree_count+\
vegetable_count+fruit_count

Forextendingalineonasubsequentline,whichisbetter,parenthesesorabackslash?
Hereisaquote:
"ThepreferredwayofwrappinglonglinesisbyusingPython'simplied
linecontinuationinsideparentheses,bracketsandbraces.Ifnecessary,
youcanaddanextrapairofparenthesesaroundanexpression,but
sometimesusingabackslashlooksbetter."
PEP8:StyleGuideforPythonCode
http://www.python.org/dev/peps/pep0008/

3.2.3Indentationandprogramstructure
Pythonusesindentationtoindicateprogramstructure.Thatistosay,inordertonesta
blockofcodeinsideacompoundstatement,youindentthatnestedcode.Thisisdifferent
frommanyprogramminglanguageswhichusesomesortofbeginandendmarkers,for
examplecurlybrackets.
ThestandardcodingpracticeforPythonistousefourspacesperindentationlevelandto
notusehardtabs.(SeetheStyleGuideforPythonCode.)Becauseofthis,youwillwant
touseatexteditorthatyoucanconfiguresothatitwillusefourspacesforindentation.
SeehereforalistofPythonfriendlytexteditors:PythonEditors.
Exercises:
1. Giventhefollowing,nesttheprintstatementinsidetheifstatement:
ifx>0:
printx

2. Nestthesetwolines:
Page168

APythonBook
z=x+y
printz

insidethefollowingfunctiondefinitionstatement:
defshow_sum(x,y):

Solutions:
1. Indentationindicatesthatonestatementisnestedinsideanotherstatement:
ifx>0:
printx

2. Indentationindicatesthatablockofstatementsisnestedinsideanotherstatement:
defshow_sum(x,y):
z=x+y
printz

3.3ExecutionModel
Hereareafewrules:
1. PythonevaluatesPythoncodefromthetopofamoduledowntothebottomofa
module.
2. Bindingstatementsattoplevelcreatenames(andbindvaluestothosenames)as
Pythonevaluatescode.Furthermore,anameisnotcreateduntilitisboundtoa
value/object.
3. Anestedreferencetoaname(forexample,insideafunctiondefinitionorinthe
nestedblockofanifstatement)isnotuseduntilthatnestedcodeisevaluated.
Exercises:
1. Willthefollowingcodeproduceanerror?
show_version()
defshow_version():
print'Version1.0a'

2. Willthefollowingcodeproduceanerror?
deftest():
show_version()
defshow_version():
print'Version1.0a'
test()

3. Willthefollowingcodeproduceanerror?Assumethatshow_configisnot
defined:
x=3

Page169

APythonBook
ifx>5:
show_config()

Solutions:
1. Answer:Yes,itgeneratesanerror.Thenameshow_versionwouldnotbe
createdandboundtoavalueuntilthedeffunctiondefinitionstatementbindsa
functionobjecttoit.Thatisdoneaftertheattempttouse(call)thatobject.
2. Answer:No.Thefunctiontest()doescallthefunctionshow_version(),
butsincetest()isnotcalleduntilaftershow_version()isdefined,thatis
OK.
3. Answer:No.It'sbadcode,butinthiscasewillnotgenerateanerror.Sincexis
lessthan5,thebodyoftheifstatementisnotevaluated.
N.B.Thisexampleshowswhyitisimportantduringtestingthateverylineof
codeinyourPythonprogrambeevaluated.HereisgoodPythonicadvice:"Ifit's
nottested,it'sbroken."

3.4BuiltinDataTypes
Eachofthesubsectionsinthissectiononbuiltindatatypeswillhaveasimilarstructure:
1. Abriefdescriptionofthedatatypeanditsuses.
2. RepresentationandconstructionHowtorepresentaninstanceofthedatatype.
Howtocodealiteralrepresentationthatcreatesanddefinesaninstance.Howto
createaninstanceofthebuiltintype.
3. Operatorsthatareapplicabletothedatatype.
4. Methodsimplementedandsupportedbythedatatype.

3.4.1Numbers
Thenumbersyouwillusemostcommonlyarelikelytobeintegersandfloats.Python
alsohaslongintegersandcomplexnumbers.
Afewfactsaboutnumbers(inPython):

Pythonwillconverttousingalongintegerautomaticallywhenneeded.Youdo
notneedtoworryaboutexceedingthesizeofa(standard)integer.
ThesizeofthelargestintegerinyourversionofPythonisinsys.maxint.To
learnwhatitis,do:
>>>importsys
>>>printsys.maxint
9223372036854775807

Theaboveshowthemaximumsizeofanintegerona64bitversionofPython.
Youcanconvertfromintegertofloatbyusingthefloatconstructor.Example:
Page170

APythonBook
>>>x=25
>>>y=float(x)
>>>printy
25.0

Pythondoes"mixedarithmetic".Youcanadd,multiply,anddivideintegersand
floats.Whenyoudo,Python"promotes"theresulttoafloat.

3.4.1.1Literalrepresentationsofnumbers

Anintegerisconstructedwithaseriesofdigitsortheintegerconstructor(int(x)).Be
awarethatasequenceofdigitsbeginningwithzerorepresentsanoctalvalue.Examples:
>>>x1=1234
>>>x2=int('1234')
>>>x3=25
>>>x1
1234
>>>x2
1234
>>>x3
25

Afloatisconstructedeitherwithdigitsandadot(example,12.345)orwith
engineering/scientificnotationorwiththefloatconstructor(float(x)).Examples:
>>>x1=2.0e3
>>>x1=1.234
>>>x2=1.234
>>>x3=float('1.234')
>>>x4=2.0e3
>>>x5=2.0e3
>>>printx1,x2,x3,x4,x5
1.2341.2341.2342000.00.002

Exercises:
Constructthesenumericvalues:
1.
2.
3.
4.
5.
6.
7.

Integerzero
Floatingpointzero
Integeronehundredandone
Floatingpointonethousand
Floatingpointonethousandusingscientificnotation
Createapositiveinteger,anegativeinteger,andzero.Assignthemtovariables
Writeseveralarithmeticexpressions.Bindthevaluestovariables.Useavariety
ofoperators,e.g.+,,/,*,etc.Useparenthesestocontroloperatorscope.
8. Createseveralfloatsandassignthemtovariables.
9. Writeseveralarithmeticexpressionscontainingyourfloatvariables.
Page171

APythonBook
10. Writeseveralexpressionsusingmixedarithmetic(integersandfloats).Obtaina
floatasaresultofdivisionofoneintegerbyanother;dosobyexplicitly
convertingoneintegertoafloat.
Solutions:
1.
2.
3.
4.
5.
6.

0
0.0,0.,or.0
101
1000.0
1e3or1.0e3
Asigningintegervaluestovariables:
In[7]:value1=23
In[8]:value2=14
In[9]:value3=0
In[10]:value1
Out[10]:23
In[11]:value2
Out[11]:14
In[12]:value3
Out[12]:0

7. Assigningexpressionvaluestovariables:
value1=4*(3+5)
value2=(value1/3.0)2

8. Assigningfloatstovariables:
value1=0.01
value2=3.0
value3=3e4

9. Assigningexpressionscontainingvarialbes:
value4=value1*(value2value3)
value4=value1+value2+value3value4

10. Mixedarithmetic:
x=5
y=8
z=float(x)/y

Youcanalsoconstructintegersandfloatsusingtheclass.Callingaclass(using
parenthesesafteraclassname,forexample)producesaninstanceoftheclass.
Exercises:
1. Constructanintegerfromthestring"123".
2. Constructafloatfromtheinteger123.
3. Constructanintegerfromthefloat12.345.
Solutions:
Page172

APythonBook
1. Usetheintdatatypetoconstructanintegerinstancefromastring:
int("123")

2. Usethefloatdatatypetoconstructafloatinstancefromaninteger:
float(123)

3. Usetheintdatatypetoconstructanintegerinstancefromafloat:
int(12.345)#>12

Noticethattheresultistruncatedtotheintegerpart.
3.4.1.2Operatorsfornumbers

Youcanusemostofthefamiliaroperatorswithnumbers,forexample:
+***///%
<<>>&|^~
<><=>===!=<>

Lookhereforanexplanationoftheseoperatorswhenappliedtonumbers:Numeric
Typesint,float,long,complexhttp://docs.python.org/lib/typesnumeric.html.
Someoperatorstakeprecedenceoverothers.ThetableintheWebpagejustreferenced
abovealsoshowsthatorderofpriority.
Hereisabitofthattable:
Allnumerictypes(exceptcomplex)supportthefollowingoperations,
sortedbyascendingpriority(operationsinthesameboxhavethe
same
priority;allnumericoperationshaveahigherprioritythan
comparison
operations):
OperationResult

x+ysumofxandy
xydifferenceofxandy
x*yproductofxandy
x/yquotientofxandy
x//y(floored)quotientofxandy
x%yremainderofx/y
xxnegated
+xxunchanged
abs(x)absolutevalueormagnitudeofx
int(x)xconvertedtointeger
long(x)xconvertedtolonginteger
float(x)xconvertedtofloatingpoint
complex(re,im)acomplexnumberwithrealpartre,imaginarypart
im.imdefaultstozero.
c.conjugate()conjugateofthecomplexnumberc

Page173

APythonBook
divmod(x,y)thepair(x//y,x%y)
pow(x,y)xtothepowery
x**yxtothepowery

Noticealsothatthesameoperatormayperformadifferentfunctiondependingonthe
datatypeofthevaluetowhichitisapplied.
Exercises:
1. Addthenumbers3,4,and5.
2. Add2totheresultofmultiplying3by4.
3. Add2plus3andmultiplytheresultby4.
Solutions:
1. Arithmeticexpressionsarefollowstandardinfixalgebraicsyntax:
3+4+5

2. Useanotherinfixexpression:
2+3*4

Or:
2+(3*4)

But,inthiscasetheparenthesesarenotnecessarybecausethe*operatorbinds
moretightlythanthe+operator.
3. Useparenthesestocontrolorderofevaluation:
(2+3)*4

Notethatthe*operatorhasprecedenceover(bindstighterthan)the+operator,
sotheparenthesesareneeded.
Pythondoesmixedarithemetic.Whenyouapplyanoperationtoanintegerandafloat,it
promotestheresulttothe"higher"datatype,afloat.
Ifyouneedtoperformanoperationonseveralintegers,butwantuseafloatingpoint
operation,firstconvertoneoftheintegerstoafloatusingfloat(x),whicheffectively
createsaninstanceofclassfloat.
TrythefollowingatyourPythoninteractiveprompt:
1. 1.0+2
2. 2/3Noticethattheresultistruncated.
3. float(2)/3Noticethattheresultisnottruncated.
Exercises:
1. Giventhefollowingassignments:
x=20
y=50

Page174

APythonBook
Dividexbyygivingafloatresult.
Solutions:
1. Promoteoneoftheintegerstofloatbeforeperformingthedivision:
z=float(x)/y

3.4.1.3Methodsonnumbers

Mostofthemethodsimplementedbythedatatypes(classes)intandfloatarespecial
methodsthatarecalledthroughtheuseofoperators.Specialmethodsoftenhavenames
thatbeginandendwithadoubleunderscore.Toseealistofthespecialnamesandabit
ofanindicationofwheneachiscalled,doanyofthefollowingatthePythoninteractive
prompt:
>>>help(int)
>>>help(32)
>>>help(float)
>>>help(1.23)
>>>dir(1)
>>>dir(1.2)

3.4.2Lists
Listsareacontainerdatatypethatactsasadynamicarray.Thatistosay,alistisa
sequencethatcanbeindexedintoandthatcangrowandshrink.
Atupleisanindexablecontainer,likealist,exceptthatatupleisimmutable.
Afewcharacteristicsoflistsandtuples:

Alisthasa(current)lengthGetthelengthofalistwithlen(mylist).
AlisthasanorderTheitemsinalistareordered,andyoucanthinkofthat
orderasgoingfromlefttoright.
AlistisheterogeneousYoucaninsertdifferenttypesofobjectsintothesame
list.
Listsaremutable,buttuplesarenot.Thus,thefollowingaretrueoflists,butnot
oftuples:
Youcanextendedoraddtoalist.
Youcanshrinkalistbydeletingitemsfromit.
Youcaninsertitemsintothemiddleofalistoratthebeginningofalist.You
canadditemstotheendofalist.
Youcanchangewhichitemisatagivenpositioninalist.

Page175

APythonBook
3.4.2.1Literalrepresentationoflists

Theliteralrepresentationofalistissquarebracketscontainingzeroormoreitems
separatedbycommas.
Examples:
1. TrytheseatthePythoninteractiveprompt:
>>>[11,22,33]
>>>['aa','bb','cc',]
>>>[100,'apple',200,'banana',]#Thelastcomma
is
>>>optional.

2. Alistcancontainlists.Infactalistcancontainanykindofobject:
>>>[1,[2,3],4,[5,6,7,],8]

3. Listsareheterogenous,thatis,differentkindsofobjectscanbeinthesamelist.
Hereisalistthatcontainsanumber,astring,andanotherlist:
>>>[123,'abc',[456,789]]

Exercises:
1. Create(define)thefollowingtuplesandlistsusingaliteral:
1. Atupleofintegers
2. Atupleofstrings
3. Alistofintegers
4. Alistofstrings
5. Alistoftuplesortupleoflists
6. Alistofintegersandstringsandtuples
7. Atuplecontainingexactlyoneitem
8. Anemptytuple
2. Doeachofthefollowing:
1. Printthelengthofalist.
2. PrinteachiteminthelistIterateovertheitemsinoneofyourlists.Print
eachitem.
3. Appendanitemtoalist.
4. Insertanitematthebeginningofalist.Insertaniteminthemiddleofalist.
5. Addtwoliststogether.Dosobyusingboththeextendmethodandtheplus
(+)operator.Whatisthedifferencebetweenextendingalistandaddingtwo
lists?
6. Retrievethe2nditemfromoneofyourtuplesorlists.
7. Retrievethe2nd,3rd,and4thitems(aslice)fromoneofyourtuplesorlists.
8. Retrievethelast(rightmost)iteminoneofyourlists.
9. Replaceaniteminalistwithanewitem.
Page176

APythonBook
10. Poponeitemofftheendofyourlist.
11. Deleteanitemfromalist.
12. Dothefollowinglistmanipulations:
1. Writeafunctionthattakestwoarguments,alistandanitem,andthat
appendstheitemtothelist.
2. Createanemptylist,
3. Callyourfunctionseveraltimestoappenditemstothelist.
4. Then,printouteachiteminthelist.
Solutions:
1. WecandefinelistliteralsatthePythonorIPythoninteractiveprompt:
1. Createatupleusingcommas,optionallywithparentheses:
In[1]:a1=(11,22,33,)
In[2]:a1
Out[2]:(11,22,33)

2. Quotedcharactersseparatedbycommascreateatupleofstrings:
In[3]:a2=('aaa','bbb','ccc')
In[4]:a2
Out[4]:('aaa','bbb','ccc')

3. Itemsseparatedbycommasinsidesquarebracketscreatealist:
In[26]:a3=[100,200,300,]
In[27]:a3
Out[27]:[100,200,300]

4. Stringsseparatedbycommasinsidesquarebracketscreatealistofstrings:
In[5]:a3=['basil','parsley','coriander']
In[6]:a3
Out[6]:['basil','parsley','coriander']
In[7]:

5. Atupleoralistcancontaintuplesandlists:
In[8]:a5=[(11,22),(33,44),(55,)]
In[9]:a5
Out[9]:[(11,22),(33,44),(55,)]

6. Alistortuplecancontainitemsofdifferenttypes:
In[10]:a6=[101,102,'abc',"def",(201,202),
('ghi','jkl')]
In[11]:a6
Out[11]:[101,102,'abc','def',(201,202),
('ghi','jkl')]

7. Inordertocreateatuplecontainingexactlyoneitem,wemustuseacomma:
In[13]:a7=(6,)
In[14]:a7

Page177

APythonBook
Out[14]:(6,)

8. Inordertocreateanemptytuple,usethetupleclass/typetocreateaninstance
ofaemptytuple:
In[21]:a=tuple()
In[22]:a
Out[22]:()
In[23]:type(a)
Out[23]:<type'tuple'>

3.4.2.2Operatorsonlists

Thereareseveraloperatorsthatareapplicabletolists.Hereishowtofindoutabout
them:
Dodir([])ordir(any_list_instance).Someoftheitemswith
specialnames(leadingandtrainingdoubleunderscores)willgiveyoucluesabout
operatorsimplementedbythelisttype.
Dohelp([])orhelp(list)atthePythoninteractiveprompt.
Dohelp(any_list_instance.some_method),wheresome_method
isoneoftheitemslistedusingdir(any_list_instance).
SeeSequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/lib/typesseq.html
Exercises:

1. Concatenate(add)twoliststogether.
2. Createasinglelistthatcontainstheitemsinaninitiallistrepeated3times.
3. Comparetwolists.
Solutions:
1. Theplusoperator,appliedtotwolistsproducesanewlistthatisaconcatenation
oftwolists:
>>>[11,22]+['aa','bb']

2. Multiplyingalistbyanintegerncreatesanewlistthatrepeatstheoriginallistn
times:
>>>[11,'abc',4.5]*3

3. Thecomparisonoperatorscanbeusedtocomparelists:
>>>[11,22]==[11,22]
>>>[11,22]<[11,33]

3.4.2.3Methodsonlists

Again,usedir()andhelp()tolearnaboutthemethodssupportedbylists.
Page178

APythonBook
Examples:
1. Createtwo(small)lists.Extendthefirstlistwiththeitemsinthesecond.
2. Appendseveralindividualitemstotheendofalist.
3. (a)Insertaitematthebeginningofalist.(b)Insertanitemsomewhereinthe
middleofalist.
4. Popanitemofftheendofalist.
Solutions:
1. Theextendmethodaddselementsfromanotherlist,orotheriterable:
>>>a=[11,22,33,44,]
>>>b=[55,66]
>>>a.extend(b)
>>>a
[11,22,33,44,55,66]

2. Usetheappendmethodonalisttoadd/appendanitemtotheendofalist:
>>>a=['aa',11]
>>>a.append('bb')
>>>a.append(22)
>>>a
['aa',11,'bb',22]

3. Theinsertmethodonalistenablesustoinsertitemsatagivenpositionina
list:
>>>a=[11,22,33,44,]
>>>a.insert(0,'aa')
>>>a
['aa',11,22,33,44]
>>>a.insert(2,'bb')
>>>a
['aa',11,'bb',22,33,44]

But,notethatweuseappendtoadditemsattheendofalist.
4. Thepopmethodonalistreturnsthe"rightmost"itemfromalistandremoves
thatitemfromthelist:
>>>a=[11,22,33,44,]
>>>
>>>b=a.pop()
>>>a
[11,22,33]
>>>b
44
>>>b=a.pop()
>>>a
[11,22]
>>>b
33

Page179

APythonBook
Notethattheappendandpopmethodstakentogethercanbeusedtoimplement
astack,thatisaLIFO(lastinfirstout)datastructure.
3.4.2.4Listcomprehensions

Alistcomprehensionisaconvenientwaytoproducealistfromaniterable(asequence
orotherobjectthatcanbeiteratedover).
Initssimplestform,alistcomprehensionresemblestheheaderlineofaforstatement
insidesquarebrackets.However,inalistcomprehension,theforstatementheaderis
prefixedwithanexpressionandsurroundedbysquarebrackets.Hereisatemplate:
[expr(x)forxiniterable]

where:
expr(x)isanexpression,usually,butnotalways,containingx.
iterableissomeiterable.Aniterablemaybeasequence(forexample,alist,a
string,atuple)oranunorderedcollectionoraniterator(somethingoverwhichwe
caniterateorapplyaforstatementto).
Hereisanexample:

>>>a=[11,22,33,44]
>>>b=[x*2forxina]
>>>b
[22,44,66,88]

Exercises:
1. Giventhefollowinglistofstrings:
names=['alice','bertrand','charlene']

producethefollowinglists:(1)alistofalluppercasenames;(2)alistof
capitalized(firstletteruppercase);
2. Giventhefollowingfunctionwhichcalculatesthefactorialofanumber:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)

andthefollowinglistofnumbers:
numbers=[2,3,4,5]

createalistofthefactorialsofeachofthenumbersinthelist.
Solutions:
1. Forourexpressioninalistcomprehension,usetheupperandcapitalize
Page180

APythonBook
methods:
>>>names=['alice','bertrand','charlene']
>>>[name.upper()fornameinnames]
['ALICE','BERTRAND','CHARLENE']
>>>[name.capitalize()fornameinnames]
['Alice','Bertrand','Charlene']

2. Theexpressioninourlistcomprehensioncallsthefactorialfunction:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
deftest():
numbers=[2,3,4,5]
factorials=[t(n)forninnumbers]
print'factorials:',factorials
if__name__=='__main__':
test()

Alistcomprehensioncanalsocontainanifclause.Hereisatemplate:
[expr(x)forxiniterableifpred(x)]

where:
pred(x)isanexpressionthatevaluatestoatrue/falsevalue.Valuesthatcount
asfalsearenumericzero,False,None,andanyemptycollection.Allother
valuescountastrue.
Onlyvaluesforwhichtheifclauseevaluatestotrueareincludedincreatingtheresulting
list.

Examples:
>>>a=[11,22,33,44]
>>>b=[x*3forxinaifx%2==0]
>>>b
[66,132]

Exercises:
1. Giventwolists,generatealistofallthestringsinthefirstlistthatarenotinthe
secondlist.Herearetwosamplelists:
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']

Solutions:
1. Theifclauseofourlistcomprehensionchecksforcontainmentinthelistnames2:
Page181

APythonBook
deftest():
names1=['alice','bertrand','charlene',
'daniel']
names2=['bertrand','charlene']
names3=[namefornameinnames1ifnamenotin
names2]
print'names3:',names3
if__name__=='__main__':
test()

Whenrun,thisscriptprintsoutthefollowing:
names3:['alice','daniel']

3.4.3Strings
Astringisanorderedsequenceofcharacters.Hereareafewcharacteristicsofstrings:
Astringhasalength.Getthelengthwiththelen()builtinfunction.
Astringisindexable.Getasinglecharacteratapositioninastringwiththe
squarebracketoperator,forexamplemystring[5].
Youcanretrieveaslice(substring)ofastringwithasliceoperation,forexample
mystring[5:8].
Createstringswithsinglequotesordoublequotes.Youcanputsinglequotesinside
doublequotesandyoucanputdoublequotesinsidesinglequotes.Youcanalsoescape
characterswithabackslash.

Exercises:
1. Createastringcontainingasinglequote.
2. Createastringcontainingadoublequote.
3. Createastringcontainingbothasinglequoteadoublequote.
Solutions:
1. Createastringwithdoublequotestoincludesinglequotesinsidethestring:
>>>str1="thatisjerry'sball"

2. Createastringenclosedwithsinglequotesinordertoincludedoublequotes
insidethestring:
>>>str1='say"goodbye",bullwinkle'

3. Takeyourchoice.Escapeeitherthesinglequotesorthedoublequoteswitha
backslash:
>>>str1='say"hello"tojerry\'smom'
>>>str2="say\"hello\"tojerry'smom"
>>>str1
'say"hello"tojerry\'smom'

Page182

APythonBook
>>>str2
'say"hello"tojerry\'smom'

Triplequotesenableyoutocreateastringthatspansmultiplelines.Usethreesingle
quotesorthreedoublequotestocreateasinglequotedstring.
Examples:
1. Createatriplequotedstringthatcontainssingleanddoublequotes.
Solutions:
1. Usetriplesinglequotesortripledoublequotestocreatemultilinestrings:
String1='''Thisstringextends
acrossseverallines.And,soithas
endoflinecharactersinit.
'''
String2="""
Thisstringbeginsandendswithanendofline
character.Itcanhaveboth'single'
quotesand"double"quotesinit.
"""
deftest():
printString1
printString2
if__name__=='__main__':
test()

3.4.3.1Characters

Pythondoesnothaveadistinctcharactertype.InPython,acharacterisastringoflength
1.Youcanusetheord()andchr()builtinfunctionstoconvertfromcharacterto
integerandback.
Exercises:
1. Createacharacter"a".
2. Createacharacter,thenobtainitsintegerrepresentation.
Solutions:
1. Thecharacter"a"isaplainstringoflength1:
>>>x='a'

2. Theintegerequivalentoftheletter"A":
>>>x="A"
>>>ord(x)
65

Page183

APythonBook
3.4.3.2Operatorsonstrings

Youcanconcatenatestringswiththe"+"operator.
Youcancreatemultipleconcatenatedcopiesofastringwiththe"*"operator.
And,augmentedassignment(+=and*=)alsowork.
Examples:
>>>'cat'+'and'+'dog'
'catanddog'
>>>'#'*40
'########################################'
>>>
>>>s1='flower'
>>>s1+='s'
>>>s1
'flowers'