matplotlib Code

2864 lines (2323 with data), 62.5 kB

#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass amsbook
\language english
\inputencoding auto
\fontscheme default
\graphics default
\paperfontsize default
\papersize Default
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default

\layout Chapter

A tour of IPython
\layout Standard

One of Python's most useful features is its interactive interpreter.
 This system allows very fast testing of ideas without the overhead of creating
 test files as is typical in most programming languages.
 In scientific computing, one of the reasons behind the popularity of systems
 like Matlab\SpecialChar ~

\begin_inset ERT
status Collapsed

\layout Standard

\backslash 
texttrademark
\end_inset 

, IDL\SpecialChar ~

\begin_inset ERT
status Collapsed

\layout Standard

\backslash 
texttrademark
\end_inset 

 or Mathematica\SpecialChar ~

\begin_inset ERT
status Collapsed

\layout Standard

\backslash 
texttrademark
\end_inset 

, is precisely their interactive nature.
 Scientific computing is an inherently exploratory problem domain, where
 one is rarely faced with writing a program against a set of well-defined
 explicit constraints.
 Being able to load data, process it with different algorithms or test parameter
s, visualize it, save results, and do all of this in a fluid and efficient
 way, can make a big productivity difference in day to day scientific work.
 Even for the development of large codes, a good interactive interpreter
 can be a major asset, though this is a less commonly held view; later in
 this document we will discuss this aspect of the problem.
\layout Standard

However, the interpreter supplied with the standard Python distribution
 is somewhat limited for extended interactive use.
 The IPython project 
\begin_inset LatexCommand \cite{IPython}

\end_inset 

 was born out of a desire to have a better Python interactive environment,
 which could combine the advantages of the Python language with some of
 the best ideas found in systems like IDL or Mathematica, along with many
 more enhancements.
 IPython is a free software project (released under the BSD license) which
 tries to:
\layout Enumerate

Provide an interactive shell superior to Python's default.
 IPython has many features for object introspection, system shell access,
 and its own special command system for adding functionality when working
 interactively.
 It tries to be a very efficient environment both for Python code development
 and for exploration of problems using Python objects (in situations like
 data analysis).
\layout Enumerate

Serve as an embeddable, ready to use interpreter for your own programs.
 IPython can be started with a single call from inside another program,
 providing access to the current namespace.
 This can be very useful both for debugging purposes and for situations
 where a blend of batch-processing and interactive exploration are needed.
\layout Enumerate

Offer a flexible framework which can be used as the base environment for
 other systems with Python as the underlying language.
 Specifically scientific environments like Mathematica, IDL and Matlab inspired
 its design, but similar ideas can be useful in many fields.
\layout Standard

This document is not meant to replace the comprehensive IPython manual,
 which ships with the IPython distribution and is also available online
 at 
\begin_inset LatexCommand \htmlurl{http://ipython.scipy.org/doc/manual}

\end_inset 

.
 Instead, we will present here some relevant parts of it for everyday use,
 and refer readers to the full manual for in-depth details.
 
\layout Standard

Additionally, this article by Jeremy Jones provides an introductory tutorial
 about IPython:
\newline 

\begin_inset LatexCommand \htmlurl{http://www.onlamp.com/pub/a/python/2005/01/27/ipython.html}

\end_inset 

.
\layout Section

Main IPython features
\begin_inset OptArg
collapsed true

\layout Standard

Main features
\end_inset 


\layout Standard

This section summarizes the most important user-visible features of IPython,
 which are not a part of the default Python shell or other interactive Python
 systems.
 While you can use IPython as a straight replacement for the normal Python
 shell, a quick read of these will allow you to take advantage of many enhanceme
nts which can be very useful in everyday work.
\layout Standard

A bird's eye view of IPython's feature set:
\layout Itemize

Dynamic object introspection.
 You can access docstrings, function definition prototypes, source code,
 source files and other details of any object accessible to the interpreter
 with a single keystroke (`
\family typewriter 
?
\family default 
').
 Adding a second 
\family typewriter 
?
\family default 
 produces more details when possible.
\layout Itemize

Completion in the local namespace, via the TAB key.
 This works for keywords, methods, variables and files in the current directory.
 TAB-completion, especially for attributes, is a convenient way to explore
 the structure of any object you're dealing with.
 Simply type object_name.<TAB> and a list of the object's attributes will
 be printed.
\layout Itemize

Numbered input/output prompts with command history (persistent across sessions
 and tied to each profile), full searching in this history and caching of
 all input and output.
\layout Itemize

User-extensible `magic' commands.
 A set of commands prefixed with 
\family typewriter 
%
\family default 
 is available for controlling IPython itself and provides directory control,
 namespace information and many aliases to common system shell commands.
\layout Itemize

Alias facility for defining your own system aliases.
\layout Itemize

Complete system shell access.
 Lines starting with ! are passed directly to the system shell, and using
 !! captures shell output into python variables for further use.
\layout Itemize

The ability to expand python variables when calling the system shell.
 In a shell command, any python variable prefixed with 
\family typewriter 
$
\family default 
 is expanded.
 A double 
\family typewriter 
$$
\family default 
 allows passing a literal 
\family typewriter 
$
\family default 
 to the shell (for access to shell and environment variables like 
\family typewriter 
$PATH
\family default 
).
\layout Itemize

Filesystem navigation, via a magic 
\family typewriter 
%cd
\family default 
 command, along with a persistent bookmark system (using 
\family typewriter 
%bookmark
\family default 
) for fast access to frequently visited directories.
\layout Itemize

A macro system for quickly re-executing multiple lines of previous input
 with a single name, implemented via the 
\family typewriter 
%macro
\family default 
 magic command.
\layout Itemize

Session logging and restoring via the 
\family typewriter 
%logstart
\family default 
, 
\family typewriter 
%logon/off
\family default 
 and 
\family typewriter 
%logstate
\family default 
 magics.
 You can then later use these log files as code in your programs.
\layout Itemize

Verbose and colored exception traceback printouts.
 Easier to parse visually, and in verbose mode they produce a lot of useful
 debugging information.
\layout Itemize

Auto-parentheses: callable objects can be executed without parentheses:
 
\family typewriter 
`sin 3'
\family default 
 is automatically converted to 
\family typewriter 
`sin(3)
\family default 
'.
\layout Itemize

Auto-quoting: using `
\family typewriter 
,
\family default 
' as the first character forces auto-quoting of the rest of the line: 
\family typewriter 
`,my_function a b'
\family default 
 becomes automatically 
\family typewriter 
`my_function("a","b")'.
\layout Itemize

Flexible configuration system.
 It uses a configuration file which allows permanent setting of all command-line
 options, module loading, code and file execution.
 The system allows recursive file inclusion, so you can have a base file
 with defaults and layers which load other customizations for particular
 projects.
\layout Itemize

Embeddable.
 You can call IPython as a python shell inside your own python programs.
 This can be used both for debugging code or for providing interactive abilities
 to your programs with knowledge about the local namespaces (very useful
 in debugging and data analysis situations).
\layout Itemize

Easy debugger access.
 You can set IPython to call up the Python debugger (pdb) every time there
 is an uncaught exception.
 This drops you inside the code which triggered the exception with all the
 data live and it is possible to navigate the stack to rapidly isolate the
 source of a bug.
 The 
\family typewriter 
%run
\family default 
 magic command --with the 
\family typewriter 
-d
\family default 
 option-- can run any script under 
\family typewriter 
pdb
\family default 
's control, automatically setting initial breakpoints for you.
\layout Itemize

Profiler support.
 You can run single statements (similar to 
\family typewriter 
profile.run()
\family default 
) or complete programs under the profiler's control.
 While this is possible with the standard 
\family typewriter 
profile
\family default 
 module, IPython wraps this functionality with magic commands (see 
\family typewriter 
`%prun'
\family default 
 and 
\family typewriter 
`%run -p
\family default 
') convenient for rapid interactive work.
\layout Section

Effective interactive work 
\begin_inset OptArg
collapsed true

\layout Standard

Interactive use
\end_inset 


\layout Standard

IPython has been designed to try to make interactive work as fluid and efficient
 as possible.
 All of its features try to maximize the output-per-keystroke, so that as
 you work at an interactive console, minimal typing produces results.
 It makes extensive use of the readline library, has its own control system
 (magics), caches previous inputs and outputs, has a macro system, etc.
 Becoming familiar with these features, while not necessary for basic use,
 will make long-term use of the system much more pleasant and productive.
\layout Subsection

Magic functions
\layout Standard

The default Python interactive shell only allows valid Python code to be
 typed at its input prompt.
 While this appears like a reasonable approach in principle, in practical
 use it turns out to be rather limiting.
 A good interactive environment should allow you to control the environment
 itself, in hopefully the most typing-efficient way.
 
\layout Standard

Verbosity in code is a good thing, since code is a long-lived entity, and
 deciphering three-letter acronyms for variable names, 6 months after a
 program was written, is typically an exercise in frustration.
 However at an interactive prompt, where every keystroke counts and things
 are not meant to be permanent, compact and efficient control of your environmen
t is an important feature.
 The default Python shell does not offer this, and the Python language's
 verbosity, which is an asset for the long-term readability of code, becomes
 a bit of a liability in this context.
\layout Standard

For this reason, IPython offers a system of `magic' commands, which serve
 to control IPython itself and perform a number of common tasks.
 Users of IDL will be familiar with the `dot' commands, like 
\family typewriter 
.stop
\family default 
, which perform similar functions in that system.
 In IPython, the magic system covers much more functionality and is fully
 user-extensible.
 This allows users to add all the control they may desire to their everyday
 working environment.
 
\layout Standard

The magics system is patterned after the time-honored Unix shells, with
 whitespace separating arguments, no parentheses required, and dashes for
 specifying options to commands.
 Many builtin magics also are named like the Unix commands they mimic, so
 that an IPython environment can be used `out of the box' by any Unix user
 with ease.
\layout Standard

IPython will treat any line whose first character is a 
\family typewriter 
%
\family default 
 as a special call to a magic function.
 For example: typing 
\family typewriter 
`%cd mydir'
\family default 
 (without the quotes) changes you working directory to 
\family typewriter 
`mydir'
\family default 
, if it exists.
 For any magic function, typing its name followed by 
\family typewriter 
?
\family default 
 will show you the magic's information and docstring, just like for other
 regular Python objects.
 Simply typing 
\family typewriter 
magic
\family default 
 at the prompt will print an overview of the system, and a list of all the
 existing magics with their docstrings.
\layout Standard

If you have 'automagic' enabled, you don't need to type in the 
\family typewriter 
%
\family default 
 explicitly.
 Automagic is enabled by default, and you can configure this in your 
\family typewriter 
ipythonrc
\family default 
 file, via the command line option 
\family typewriter 
-automagic 
\family default 
or even toggle it at runtime with the 
\family typewriter 
%automagic
\family default 
 function.
 IPython will scan its internal list of magic functions and call one if
 it exists.
 With automagic on you can then just type `
\family typewriter 
cd mydir
\family default 
' to go to directory `
\family typewriter 
mydir
\family default 
'.
 The automagic system has the lowest possible precedence in name searches,
 so defining an identifier with the same name as an existing magic function
 will shadow it for automagic use.
 You can still access the shadowed magic function by explicitly using the
 
\family typewriter 
%
\family default 
 character at the beginning of the line.
\layout Standard

An example (with automagic on) should clarify all this:
\layout LyX-Code

In [1]: cd ipython 
\color blue
# %cd is called by automagic
\layout LyX-Code

/home/fperez/ipython
\layout LyX-Code

In [2]: cd = 1 
\color blue
# now cd is just a variable
\layout LyX-Code

In [3]: cd ..
 
\color blue
# and doesn't work as a function anymore
\layout LyX-Code

------------------------------------------------------------
\layout LyX-Code

    File "<console>", line 1
\layout LyX-Code

      cd ..
\layout LyX-Code

          ^
\layout LyX-Code

SyntaxError: invalid syntax
\layout LyX-Code

\layout LyX-Code

In [4]: %cd ..
 
\color blue
# but %cd always works
\layout LyX-Code

/home/fperez
\layout LyX-Code

In [5]: del cd 
\color blue
# if you remove the cd variable
\layout LyX-Code

In [6]: cd ipython 
\color blue
# automagic can work again
\layout LyX-Code

/home/fperez/ipython
\layout Subsection

Object exploration
\layout Standard

Python is a language with exceptional introspection capabilities.
 This means that, within the language itself, it is possible to extract
 a remarkable amount of information about all objects currently in memory.
 However the default Python shell exposes very little of this power in an
 easy to use manner; IPython provides a lot of functionality to remedy this.
\layout Standard

The bulk of IPython's introspection system is accessible via only two keys:
 the question mark 
\family typewriter 
?
\family default 
 and the 
\family typewriter 
<TAB>
\family default 
 key.
 Under the hood, these two keys control a fairly complex set of libraries
 which ultimately rely on the 
\family typewriter 
readline 
\family default 
and 
\family typewriter 
inspect
\family default 
 modules from the Python standard library.
 But for regular use, you should never need to remember anything beyond
 these two.
 As an example, consider defining a variable named 
\family typewriter 
mylist
\family default 
, which starts as an empty list:
\layout LyX-Code

In [1]: mylist=[]
\layout Standard

now you can find out some things about it by using the question mark:
\layout LyX-Code

In [2]: mylist?
\layout LyX-Code

Type:           list
\layout LyX-Code

Base Class:     <type 'list'>
\layout LyX-Code

String Form:    []
\layout LyX-Code

Namespace:      Interactive
\layout LyX-Code

Length:         0
\layout LyX-Code

Docstring:
\layout LyX-Code

    list() -> new list
\layout LyX-Code

    list(sequence) -> new list initialized from sequence's items
\layout Standard

next, by adding a period (the standard Python attribute separator) and hitting
 
\family typewriter 
TAB
\family default 
, IPython will show you all the attributes which this object has:
\layout LyX-Code

In [3]: mylist.
\emph on 
\color blue
<The TAB key was pressed here>
\layout LyX-Code

mylist.append   mylist.extend   mylist.insert   mylist.remove mylist.sort
\layout LyX-Code

mylist.count    mylist.index    mylist.pop      mylist.reverse
\layout Standard

you can then request further details about any of them:
\layout LyX-Code

In [3]: mylist.append?
\layout LyX-Code

Type:           builtin_function_or_method
\layout LyX-Code

Base Class:     <type 'builtin_function_or_method'>
\layout LyX-Code

String Form:    <built-in method append of list object at 0x403b2b6c>
\layout LyX-Code

Namespace:      Interactive
\layout LyX-Code

Docstring:
\layout LyX-Code

    L.append(object) -- append object to end
\layout Standard

The 
\family typewriter 
? 
\family default 
system can be doubled.
 The first screenshot in Fig.\SpecialChar ~

\begin_inset LatexCommand \ref{fig:ipscr_code}

\end_inset 

 was generated by typing at the IPython prompt: 
\layout LyX-Code

In [1]: import code
\layout LyX-Code

In [2]: code??
\layout Standard

Using 
\family typewriter 
??
\family default 
 shows the syntax-highlighted source for the 
\family typewriter 
code
\family default 
 module from the Python standard library.
 This is an excellent way to explore modules or objects which you are not
 familiar with.
 As long as Python's 
\family typewriter 
inspect
\family default 
 system is capable of finding the source code for an object, IPython will
 show it to you, with nice syntax highlights.
 
\layout Standard

This can be done for entire modules, as in the prvious example, for individual
 functions, or even methods of object instances.
 The second screenshot in the same figure shows source for the 
\family typewriter 
timeit
\family default 
 method of a 
\family typewriter 
timeit.Timer
\family default 
 object.
\layout Standard

The magic commands 
\family typewriter 
%pdoc
\family default 
, 
\family typewriter 
%pdef
\family default 
, 
\family typewriter 
%psource
\family default 
 and 
\family typewriter 
%pfile
\family default 
 will respectively print the docstring, function definition line, full source
 code and the complete file for any object (when they can be found).
\layout Standard


\begin_inset Float figure
wide false
collapsed false

\layout Standard
\align center 

\begin_inset Graphics
	filename fig/ipscr_code.png
	lyxscale 40
	width 48line%

\end_inset 

\SpecialChar ~

\begin_inset Graphics
	filename fig/ipscr_meth_src.png
	lyxscale 40
	width 48line%

\end_inset 


\layout Caption


\begin_inset LatexCommand \label{fig:ipscr_code}

\end_inset 

IPython can show syntax-highlighted source code for objects whose source
 is available.
\end_inset 


\layout Subsection

Input and Ouptut cached prompts
\layout Standard

In IPython, all output results are automatically stored in a global dictionary
 named 
\family typewriter 
Out
\family default 
 and variables named 
\family typewriter 
_1
\family default 
, 
\family typewriter 
_2
\family default 
, etc.
 alias them.
 For example, the result of input line 4 is available either as 
\family typewriter 
Out[4]
\family default 
 or as 
\family typewriter 
_4
\family default 
.
 Additionally, three variables named 
\family typewriter 
_
\family default 
, 
\family typewriter 
__
\family default 
 and 
\family typewriter 
___
\family default 
 are always kept updated with the for the last three results.
 This allows you to recall any previous result and further use it for new
 calculations.
 For example:
\layout LyX-Code

In [1]: 2+4
\layout LyX-Code

Out[1]: 6
\layout LyX-Code

\layout LyX-Code

In [2]: _+9
\layout LyX-Code

Out[2]: 15
\layout LyX-Code

\layout LyX-Code

In [3]: _+__
\layout LyX-Code

Out[3]: 21
\layout LyX-Code

\layout LyX-Code

In [4]: print _1
\layout LyX-Code

6
\layout LyX-Code

\layout LyX-Code

In [5]: print Out[1]
\layout LyX-Code

6
\layout LyX-Code

\layout LyX-Code

In [6]: _2**3
\layout LyX-Code

Out[6]: 3375
\layout Standard

You can put a 
\family typewriter 
`;
\family default 
' at the end of a line to supress the printing of output.
 This is useful when doing calculations which generate long output you are
 not interested in seeing.
 The 
\family typewriter 
_*
\family default 
 variables and the 
\family typewriter 
Out[]
\family default 
 list do get updated with the contents of the output, even if it is not
 printed.
 You can thus still access the generated results this way for further processing.
\layout Standard

A similar system exists for caching input.
 All input is stored in a global list called 
\family typewriter 
In
\family default 
 , so you can re-execute lines 22 through 28 plus line 34 by typing 
\family typewriter 
'exec In[22:29]+In[34]'
\family default 
 (using Python slicing notation).
 
\layout Standard

At any time, your input history remains available.
 The 
\family typewriter 
%hist
\family default 
 command can show you all previous input, without line numbers if desired
 (option 
\family typewriter 
-n
\family default 
) so you can directly copy and paste code either back in IPython or in a
 text editor.
 You can also save all your history by turning on logging via 
\family typewriter 
%logstart
\family default 
; these logs can later be either reloaded as IPython sessions or used as
 code for your programs.
\layout Standard

If you need to execute the same set of lines often, you can assign them
 to a macro with the 
\family typewriter 
%macro 
\family default 
magic function.
 Macros are simply short names for groups of input lines, which can be re-execut
ed by only typing that name.
 Typing 
\family typewriter 
macro?
\family default 
 at the prompt will show you the function's full documentation.
 For example, if your history contains:
\layout LyX-Code

44: x=1
\layout LyX-Code

45: y=3
\layout LyX-Code

46: z=x+y
\layout LyX-Code

47: print x
\layout LyX-Code

48: a=5
\layout LyX-Code

49: print 'x',x,'y',y
\layout Standard

You can create a macro with lines 44 through 47 (included) and line 49 called
 
\family typewriter 
my_macro
\family default 
 with:
\layout LyX-Code

In [51]: %macro my_macro 44:48 49
\layout Standard

Now, simply typing 
\family typewriter 
my_macro
\family default 
 will re-execute all this code in one pass.
 The number range follows standard Python list slicing notation, where
\family typewriter 
 n:m
\family default 
 means the numbers 
\begin_inset Formula $(n,n+1,\ldots,m-1).$
\end_inset 


\layout Standard

You should note that macros execute in the current context, so if any variable
 changes, the macro will pick up the new value every time it is executed:
\layout LyX-Code

In [1]: x=1
\layout LyX-Code

In [2]: y=x*5
\layout LyX-Code

In [3]: z=x+3
\layout LyX-Code

In [4]: print 'y is:',y,'and z is:',z
\layout LyX-Code

y is: 5 and z is: 4
\layout LyX-Code


\color blue
# make a macro with lines 2,3,4 (note Python list slice syntax):
\layout LyX-Code

In [5]: macro yz 2:5
\layout LyX-Code

Macro `yz` created.
 To execute, type its name (without quotes).
\layout LyX-Code

Macro contents:
\layout LyX-Code

y=x*5
\layout LyX-Code

z=x+3
\layout LyX-Code

print 'y is:',y,'and z is:',z
\layout LyX-Code


\color blue
# now, run the macro directly:
\layout LyX-Code

In [6]: yz
\layout LyX-Code

Out[6]: Executing Macro...
\layout LyX-Code

y is: 5 and z is: 4
\layout LyX-Code


\color blue
# we change the value of x
\layout LyX-Code

In [7]: x=9
\layout LyX-Code


\color blue
# and now if we rerun the macro, we get the new values:
\layout LyX-Code

In [8]: yz
\layout LyX-Code

Out[8]: Executing Macro...
\layout LyX-Code

y is: 45 and z is: 12
\layout Subsection

Running code
\layout Standard

The 
\family typewriter 
%run
\family default 
 magic command allows you to run any python script and load all of its data
 directly into the interactive namespace.
 
\family typewriter 
%run 
\family default 
is a sophisticated wrapper around the Python 
\family typewriter 
execfile()
\family default 
 builtin function; since the file is re-read from disk each time, changes
 you make to it are reflected immediately (in contrast to the behavior of
 
\family typewriter 
import
\family default 
).
 I rarely use 
\family typewriter 
import
\family default 
 for code I am testing, relying on 
\family typewriter 
%run
\family default 
 instead.
 
\layout Standard

By default, 
\layout LyX-Code

%run myfile arg1 arg2 ...
\layout Standard

executes 
\family typewriter 
myfile 
\family default 
in a namespace initially consisting only of 
\family typewriter 
__name__=='__main__'
\family default 
 and 
\family typewriter 
sys.argv
\family default 
 being filled with arg1, arg2, etc.
 This means that using 
\family typewriter 
%run
\family default 
 is functionally very simlar to executing a script at the system command
 line, but you get all the functionality of IPython (better tracebacks,
 debugger and profiler access, etc.).
 The 
\family typewriter 
-n
\family default 
 option prevents 
\family typewriter 
__name__ 
\family default 
from being set equal to 
\family typewriter 
'__main__'
\family default 
, in case you want to test the part of a script which only runs when 
\family typewriter 
import
\family default 
ed.
 
\layout Standard

Additionally, the fact that IPython then updates your interactive namespace
 with the variables defined in the script is very useful, because you can
 run your code to do a lot of processing, and then continue using and exploring
 interactively the objects created by the program.
 
\layout Standard

For example, if the file 
\family typewriter 
ip_simple.py
\family default 
 contains:
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lstinputlisting{examples/ip_simple.py}
\end_inset 

you can run it in IPython as follows:
\layout LyX-Code


\color blue
# First, let's check that x is undefined
\layout LyX-Code

In [1]: x
\layout LyX-Code

---------------------------------------------------------------------------
\layout LyX-Code

exceptions.NameError                                 Traceback (most recent
 call last)
\layout LyX-Code

/usr/local/home/fperez/teach/course/examples/<console>
\layout LyX-Code

NameError: name 'x' is not defined
\layout LyX-Code

\layout LyX-Code


\color blue
# Now we run the script (the .py extension is optional):
\layout LyX-Code

In [2]: run ip_simple
\layout LyX-Code

sys.argv is: ['ip_simple.py']
\layout LyX-Code

__name__ is: __main__
\layout LyX-Code

\layout LyX-Code


\color blue
# If we print x, now it has the value from the script
\layout LyX-Code

In [3]: x
\layout LyX-Code

Out[3]: 1
\layout LyX-Code

\layout LyX-Code


\color blue
# Again, but now running with some arguments:
\layout LyX-Code

In [4]: run ip_simple -x arg1 "hello world"
\layout LyX-Code

sys.argv is: ['ip_simple.py', '-x', 'arg1', 'hello world']
\layout LyX-Code

__name__ is: __main__
\layout Standard

With the 
\family typewriter 
-i
\family default 
 option, the namespace where your script runs is actually your interactive
 one.
 This can be used for two sligthly different purposes.
 The simpler case, is just to quickly type up a set of commands in an editor
 which you want to execute on your current environment (although the 
\family typewriter 
%edit
\family default 
 command can also be used for this).
 Consider running the file 
\family typewriter 
ip_simple2.py
\family default 
:
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lstinputlisting{examples/ip_simple2.py}
\end_inset 

in IPython:
\layout LyX-Code


\color blue
# A regular %run will produce an error:
\layout LyX-Code

In [1]: run ip_simple2
\layout LyX-Code

---------------------------------------------------------
\layout LyX-Code

exceptions.NameError              Traceback (most recent call last)
\layout LyX-Code

/usr/local/home/fperez/teach/course/examples/ip_simple2.py
\layout LyX-Code

      2
\layout LyX-Code

      3 It should be run via IPython's %run with the -i option."""
\layout LyX-Code

      4
\layout LyX-Code

----> 5 print 'x is:',x
\layout LyX-Code

      6
\layout LyX-Code

NameError: name 'x' is not defined
\layout LyX-Code

WARNING: Failure executing file: <ip_simple2.py>
\layout LyX-Code

x is:
\layout LyX-Code


\color blue
# However, if you do have a variable x defined:
\layout LyX-Code

In [2]: x='hello'
\layout LyX-Code


\color blue
# you can use the -i option and the code will see x:
\layout LyX-Code

In [3]: run -i ip_simple2
\layout LyX-Code

x is: hello
\layout Standard

A different use of 
\family typewriter 
%run -i
\family default 
, is to repeatedly run scripts which may have a potentially expensive initializa
tion phase.
 If this initialization does not need to be repeated on each run (for example,
 you are debugging some other submodule and can reuse the same expensive
 object several times), you can avoid it by protecting the expensive object
 with a 
\family typewriter 
try/except
\family default 
 block.
 This simple script illustrates the technique:
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lstinputlisting{examples/ip_expensive_init.py}
\end_inset 

In IPython, here is how you can use it:
\layout LyX-Code


\color blue
# The first time it runs, it will have to initialize
\layout LyX-Code

In [1]: run -i ip_expensive_init.py
\layout LyX-Code

bigobject not found, performing expensive initialization...
\layout LyX-Code

total is: 499500
\layout LyX-Code


\color blue
# but successive runs don't require initialization
\layout LyX-Code

In [2]: run -i ip_expensive_init.py
\layout LyX-Code

We found bigobject! No need to initialize it.
\layout LyX-Code

total is: 499500
\layout LyX-Code


\color blue
# you can still run without -i, to achieve a full reload 
\layout LyX-Code


\color blue
# if you need it for any reason
\layout LyX-Code

In [3]: run ip_expensive_init.py
\layout LyX-Code

bigobject not found, performing expensive initialization...
\layout LyX-Code

total is: 499500
\layout Standard

In the third run, by not using 
\family typewriter 
-i
\family default 
, your script runs in an empty namespace and this forces a full initialization
 (the 
\family typewriter 
NameError
\family default 
 exception is triggered).
\layout Standard


\family typewriter 
%run
\family default 
 also has special flags for timing the execution of your scripts (
\family typewriter 
-t
\family default 
) and for executing them under the control of either Python's 
\family typewriter 
pdb 
\family default 
debugger (
\family typewriter 
-d
\family default 
) or profiler (
\family typewriter 
-p
\family default 
).
 You can get all of its docstring with the usual 
\family typewriter 
run?
\family default 
 mechanism.
\layout Standard

Thanks to all of its various control options, 
\family typewriter 
%run
\family default 
 can be used as the main tool for efficient interactive development of code
 which you write in your editor of choice.
 My personal operation mode, which has served me well for several years
 of scientific work in Python, is to have a good editor (XEmacs in my case)
 open with all my Python code, and IPython open in a terminal where I run,
 debug, explore, plot, etc.
\layout Section

Access to the underlying Operating System
\begin_inset OptArg
collapsed true

\layout Standard

OS access
\end_inset 


\layout Subsection

Basic usage
\layout Standard

IPython allows you to always access the underlying OS very easily.
 Any lines starting with 
\family typewriter 
!
\family default 
 are passed directly to the system shell:
\layout LyX-Code

In [6]: !ls ip*.py
\layout LyX-Code

ip_expensive_init.py  ip_simple2.py  ip_simple.py
\layout Standard

and using 
\family typewriter 
!!
\family default 
 captures shell output into python variables for further use:
\layout LyX-Code

In [7]: !!ls ip*.py
\layout LyX-Code

Out[7]: ['ip_expensive_init.py', 'ip_simple2.py', 'ip_simple.py']
\layout Standard

There is a difference between the two cases: in the first, the 
\family typewriter 
ls
\family default 
 command simply prints its results to the terminal as text, but no value
 is returned.
 In the second, IPython actually captures the output of the command, splits
 it as a list (one line per entry), and returns its value.
 This allows you to then operate on the results with Python routines.
\layout Standard

Additionally, IPython plays a few interesting syntactic tricks for your
 convenience.
 Whenever you make a system call, IPython will expand any call of the type
 
\family typewriter 
$var
\family default 
 into the actual value of the python variable 
\family typewriter 
var
\family default 
, so that you can call shell commands on Python values.
 Continuing the session above, and remembering that 
\family typewriter 
_
\family default 
 holds the previously returned value, we can call the `
\family typewriter 
wc -l
\family default 
' Unix command (which does a line count on a file) on the files we just
 obtained:
\layout LyX-Code

In [8]: for f in _:
\layout LyX-Code

   ...:      if 'simple' in f:
\layout LyX-Code

   ...:          !wc -l $f
\layout LyX-Code

   ...:
\layout LyX-Code

3 ip_simple2.py
\layout LyX-Code

4 ip_simple.py
\layout Standard

While this is completely unorthodox (actually, invalid) Python, it is the
 kind of functionality which can make for extremely efficient uses when
 working at an interactive command line.
 Obviously all of this can be done (and it 
\emph on 
is
\emph default 
 done that way by IPython internally) with regular Python code, but that
 approach requires a fair amount more typing, the use of 
\family typewriter 
%
\family default 
-based string interpolation, and making system calls via the 
\family typewriter 
os.system()
\family default 
 function.
\layout Standard

If you actually need to pass a 
\family typewriter 
$
\family default 
 character to a shell command, you simply use 
\family typewriter 
$$
\family default 
 in the IPython command line:
\layout LyX-Code

In [11]: !echo $$SHELL
\layout LyX-Code

/bin/tcsh
\layout Standard

If you want to capture the output of a system command directly to a named
 Python variable, you can use the 
\family typewriter 
%sc
\family default 
 magic function:
\layout LyX-Code


\color blue
# by default, %sc captures to a plain string:
\layout LyX-Code

In [16]: %sc astr=ls ip*.py
\layout LyX-Code

In [17]: astr
\layout LyX-Code

Out[17]: 'ip_expensive_init.py
\backslash 
nip_simple2.py
\backslash 
nip_simple.py'
\layout LyX-Code


\color blue
# but with the -l option, it splits to a list (like !! does)
\layout LyX-Code

In [18]: %sc -l alist=ls ip*.py
\layout LyX-Code

In [19]: alist
\layout LyX-Code

Out[19]: ['ip_expensive_init.py', 'ip_simple2.py', 'ip_simple.py']
\layout Subsection

System aliases
\layout Standard

In IPython, you can also define your own system aliases.
 Even though IPython gives you access to your system shell via the 
\family typewriter 
!
\family default 
 prefix, it is convenient to have aliases to the system commands you use
 most often.
 This allows you to work seamlessly from inside IPython with the same commands
 you are used to in your system shell:
\layout Standard


\family typewriter 
`%alias alias_name cmd'
\family default 
 defines 
\family typewriter 
`alias_name'
\family default 
 as an alias for 
\family typewriter 
`cmd'
\layout Standard

Then, typing 
\family typewriter 
`alias_name params'
\family default 
 will execute the system command 
\family typewriter 
`cmd params'
\family default 
 (from your underlying operating system).
 Aliases have lower precedence than magic functions and Python normal variables,
 so if 
\family typewriter 
`foo'
\family default 
 is both a Python variable and an alias, the alias can not be executed until
 
\family typewriter 
`del foo'
\family default 
 removes the Python variable.
 If you need to access an alias directly, you can use the builtin function
 
\family typewriter 
ipalias 
\family default 
as
\family typewriter 
 ipalias('foo')
\family default 
.
\layout Standard

You can use the 
\family typewriter 
%l
\family default 
 specifier in an alias definition to represent the whole line when the alias
 is called.
 For example:
\layout LyX-Code

In [2]: alias all echo "Input in brackets: <%l>"
\layout LyX-Code

In [3]: all hello world
\layout LyX-Code

Input in brackets: <hello world>
\layout Standard

You can also define aliases with positional parameters using 
\family typewriter 
%s
\family default 
 specifiers (one per parameter):
\layout LyX-Code

In [1]: alias parts echo first %s second %s
\layout LyX-Code

In [2]: %parts A B
\layout LyX-Code

first A second B
\layout LyX-Code

In [3]: %parts A
\layout LyX-Code

Incorrect number of arguments: 2 expected.
\layout LyX-Code

parts is an alias to: 'echo first %s second %s'
\layout Standard

Aliases expand Python variables just like system calls using 
\family typewriter 
!
\family default 
 or 
\family typewriter 
!!
\family default 
 do: all expressions prefixed with '
\family typewriter 
$
\family default 
' get expanded.
 For details of the semantic rules, see PEP-215: 
\begin_inset LatexCommand \htmlurl{http://www.python.org/peps/pep-0215.html}

\end_inset 

.
 This is the library used by IPython for variable expansion.
\layout Standard

Simply typing 
\family typewriter 
alias
\family default 
 will print a list of the current aliases, and 
\family typewriter 
unalias
\family default 
 can be used to remove an alias.
 For further details, use 
\family typewriter 
alias?
\family default 
.
\layout Subsection

Directory management
\layout Standard

IPython comes with some pre-defined aliases and a complete system for changing
 directories, both via a stack (see 
\family typewriter 
%pushd
\family default 
, 
\family typewriter 
%popd
\family default 
 and 
\family typewriter 
%ds
\family default 
) and via direct 
\family typewriter 
%cd
\family default 
.
 The latter keeps a history of visited directories and allows you to go
 to any previously visited one.
 You can see this history with the 
\family typewriter 
%dhist
\family default 
 magic:
\layout LyX-Code

In [1]: cd ~/code/python
\layout LyX-Code

/home/fperez/code/python
\layout LyX-Code

In [2]: cd ~/teach/
\layout LyX-Code

/home/fperez/teach
\layout LyX-Code

In [3]: cd ~/research
\layout LyX-Code

/home/fperez/research
\layout LyX-Code

In [4]: dhist
\layout LyX-Code

Directory history (kept in _dh)
\layout LyX-Code

0: /home/fperez/teach/course/examples
\layout LyX-Code

1: /home/fperez/code/python
\layout LyX-Code

2: /home/fperez/teach
\layout LyX-Code

3: /home/fperez/research
\layout LyX-Code

In [5]: cd -1
\layout LyX-Code

/home/fperez/code/python
\layout Standard

The 
\family typewriter 
%bookmark
\family default 
 magic allows you to create named bookmarks in your filesystem, which 
\family typewriter 
cd
\family default 
 can be directed to go to (with the 
\family typewriter 
-b
\family default 
 flag), and to which it will try to default automatically if no such named
 directory exists.
 The system is very easy to use and quite natural in practice:
\layout LyX-Code

In [8]: bookmark course
\layout LyX-Code

In [9]: cd
\layout LyX-Code

/home/fperez
\layout LyX-Code

In [10]: ls course
\layout LyX-Code

ls: course: No such file or directory
\layout LyX-Code

In [11]: cd course
\layout LyX-Code

(bookmark:course) -> /home/fperez/teach/course
\layout LyX-Code

/home/fperez/teach/course
\layout Subsection

IPython as a system shell
\layout Standard

While IPython is 
\emph on 
not
\emph default 
 a system shell, it ships with a special profile called 
\family typewriter 
pysh
\family default 
, which you can activate at the command line as 
\family typewriter 
`ipython -p pysh'
\family default 
.
 This modifies IPython's behavior and adds some additional facilities and
 a prompt customized for filesystem navigation.
\layout Standard

Note that this does 
\emph on 
not
\emph default 
 make IPython a full-fledged system shell.
 In particular, it has no job control, so if you type Ctrl-Z (under Unix),
 you'll suspend pysh itself, not the process you just started.
 
\layout Standard

What the shell profile allows you to do is to use the convenient and powerful
 syntax of Python to do quick scripting at the command line.
 Below we describe some of its features.
\layout Subsubsection

Aliases
\layout Standard

All of your 
\family typewriter 
$PATH
\family default 
 has been loaded as IPython aliases, so you should be able to type any normal
 system command and have it executed.
 See 
\family typewriter 
%alias?
\family default 
 and 
\family typewriter 
%unalias?
\family default 
 for details on the alias facilities.
 See also 
\family typewriter 
%rehash?
\family default 
 and 
\family typewriter 
%rehashx? 
\family default 
for details on the mechanism used to load 
\family typewriter 
$PATH
\family default 
.
\layout Subsubsection

Special syntax
\layout Standard

Any lines which begin with 
\family typewriter 
`~'
\family default 
, 
\family typewriter 
`/'
\family default 
 and 
\family typewriter 
`.'
\family default 
 will be executed as shell commands instead of as Python code.
 The special escapes below are also recognized.
 
\family typewriter 
!cmd
\family default 
 is valid in single or multi-line input, all others are only valid in single-lin
e input:
\layout Description


\family typewriter 
!cmd
\family default 
 pass `cmd' directly to the shell 
\layout Description


\family typewriter 
!!cmd
\family default 
 execute `cmd' and return output as a list (split on `
\backslash 
n') 
\layout Description


\family typewriter 
$var=cmd
\family default 
 capture output of cmd into var, as a string (shorthand for 
\family typewriter 
%sc var=cmd
\family default 
)
\layout Description


\family typewriter 
$$var=cmd
\family default 
 capture output of cmd into var, as a list (split on `
\backslash 
n', shorthand for 
\family typewriter 
%sc -l var=cmd
\family default 
)
\layout Subsubsection

Useful functions and modules
\layout Standard

The os, sys and shutil modules from the Python standard library are automaticall
y loaded.
 Some additional functions, useful for shell usage, are listed below.
 You can request more help about them with `
\family typewriter 
?
\family default 
'.
\layout Description


\family typewriter 
shell
\family default 
 - execute a command in the underlying system shell 
\layout Description


\family typewriter 
system
\family default 
 - like 
\family typewriter 
shell()
\family default 
, but return the exit status of the command
\layout Description


\family typewriter 
sout
\family default 
 - capture the output of a command as a string
\layout Description


\family typewriter 
lout
\family default 
 - capture the output of a command as a list (split on `
\backslash 
n')
\layout Description


\family typewriter 
getoutputerror
\family default 
 - capture (output,error) of a shell commandss
\layout Standard


\family typewriter 
sout
\family default 
/
\family typewriter 
lout
\family default 
 are the functional equivalents of 
\family typewriter 
$
\family default 
/
\family typewriter 
$$
\family default 
.
 They are provided to allow you to capture system output in the middle of
 true python code, function definitions, etc (where 
\family typewriter 
$
\family default 
 and 
\family typewriter 
$$
\family default 
 are invalid)
\layout Section

Access to an editor
\layout Standard

You can use 
\family typewriter 
%edit
\family default 
 to have almost multiline editing.
 While IPython doesn't support true multiline editing, this command allows
 you to call an editor on the spot, and IPython will execute the code you
 type in there as if it were typed interactively.
 
\layout Standard


\family typewriter 
%edit
\family default 
 runs your IPython configured editor.
 By default this is read from your environment variable 
\family typewriter 
$EDITOR
\family default 
.
 If this isn't found, it will default to 
\family typewriter 
vi
\family default 
 under Linux/Unix and to 
\family typewriter 
notepad
\family default 
 under Windows.
 
\layout Standard

You can also set the value of this editor via the command-line option 
\family typewriter 
`-editor'
\family default 
 or in your 
\family typewriter 
ipythonrc
\family default 
 file.
 This is useful if you wish to use specifically for IPython an editor different
 from your typical default (and for Windows users who typically don't set
 environment variables).
\layout Standard

This command allows you to conveniently edit multi-line code right in your
 IPython session.
\layout Standard

If called without arguments, 
\family typewriter 
%edit
\family default 
 opens up an empty editor with a temporary file and will execute the contents
 of this file when you close it (don't forget to save it!).
 
\layout Section

Customizing IPython
\layout Subsection

Basics
\layout Standard

IPython has a very flexible configuration system.
 It uses a configuration file which allows permanent setting of all command-line
 options, module loading, code and file execution.
 The system allows recursive file inclusion, so you can have a base file
 with defaults and layers which load other customizations for particular
 projects.
\layout Standard

IPython reads a configuration file which can be specified at the command
 line (
\family typewriter 
-rcfile
\family default 
) or which by default is assumed to be called 
\family typewriter 
ipythonrc
\family default 
.
 Such a file is looked for in the current directory where IPython is started
 and then in your 
\family typewriter 
IPYTHONDIR
\family default 
, which allows you to have local configuration files for specific projects.
 The default value for this directory is 
\family typewriter 
$HOME/.ipython
\family default 
 (
\family typewriter 
_ipython
\family default 
 under Windows).
 Under Unix operating systems 
\family typewriter 
$HOME
\family default 
 always exists; for Windows, IPython will try to find such an environment
 variable; if it doesn't exist, it uses 
\family typewriter 
HOMEDRIVE
\backslash 
HOMEPATH 
\family default 
(these are always defined by Windows).
 This typically gives something like 
\family typewriter 
C:
\backslash 
Documents and Settings
\backslash 
YourUserName
\family default 
, but your local details may vary.
 Finally, you can make this directory live anywhere you want by creating
 an environment variable called 
\family typewriter 
$IPYTHONDIR
\family default 
.
\layout Standard

In this directory you will find all the files that configure IPython's defaults,
 and you can put there your profiles and extensions.
 This directory is automatically added by IPython to 
\family typewriter 
sys.path
\family default 
, so anything you place there can be found by 
\family typewriter 
import
\family default 
 statements.
\layout Standard

The syntax of an rcfile is one of key-value pairs separated by whitespace,
 one per line.
 Lines beginning with a 
\family typewriter 
#
\family default 
 are ignored as comments, but comments can 
\series bold 
not
\series default 
 be put on lines with data (the parser is fairly primitive).
 You can study the default rcfile created by IPython at startup for customizatio
n details, it is extremely commented.
\layout Subsection

Profiles
\layout Standard

IPython can load any configuration file you want if you give its name at
 startup with the 
\family typewriter 
-rcfile
\family default 
 flag.
 However, for convenience it provides a shorthand based on a naming convention
 for loading such profiles.
 This system allows you to easily maintain customized versions of IPython
 for specific purposes.
\layout Standard

With the 
\family typewriter 
-profile <name>
\family default 
 flag (you can abbreviate it to 
\family typewriter 
-p
\family default 
), IPython will assume that your config file is called 
\family typewriter 
ipythonrc-<name>
\family default 
 (it looks in current dir first, then in 
\family typewriter 
IPYTHONDIR
\family default 
).
 This is a quick way to keep and load multiple config files for different
 tasks, especially if you use the include option of config files.
 You can keep a basic 
\family typewriter 
IPYTHONDIR/ipythonrc
\family default 
 file and then have other profiles which include this one and load extra
 things for particular tasks.
 For example:
\layout Enumerate


\family typewriter 
$HOME/.ipython/ipythonrc
\family default 
: load basic things you always want.
 
\layout Enumerate


\family typewriter 
$HOME/.ipython/ipythonrc-math
\family default 
: load (1) and basic math-related modules.
 
\layout Enumerate


\family typewriter 
$HOME/.ipython/ipythonrc-numeric
\family default 
: load (1) and Numeric and plotting modules.
\layout Standard

Since it is possible to create an endless loop by having circular file inclusion
s, IPython will stop if it reaches 15 recursive inclusions.
\layout Section

Debugging and profiling with IPython 
\begin_inset OptArg
collapsed true

\layout Standard

Debugging and profiling
\end_inset 


\layout Standard

The Python standard library includes powerful facilities for debugging and
 profiling code, but it is common to find even experienced Python programmers
 who still do not take advantage of them.
 In part, this is due to the fact that loading and configuring them requires
 reading an extra documentation section, and keeping a bit of additional
 information about their use in your head.
 IPython tries to automate their use to the point where, with a single command,
 you can use either of these subsystems in a transparent manner.
 Hopefully they will become part of your daily workflow.
\layout Standard

At its most basic, for debugging your programs, you can rely on using 
\family typewriter 
%run
\family default 
 to execute them, see the results, play with all variables loaded into the
 interactive namespace, etc.
 A typical working session involves keeping your favorite editor open with
 the file you are working on, and repeatedly calling 
\family typewriter 
%run
\family default 
 on it as you make changes and save them.
\layout Standard


\begin_inset Float figure
wide false
collapsed false

\layout Standard
\align center 

\begin_inset Graphics
	filename fig/ipscr_traceback.png
	lyxscale 40
	width 70line%

\end_inset 


\layout Caption


\begin_inset LatexCommand \label{fig:ipscr_traceback}

\end_inset 

IPython can provide extremely detailed tracebacks.
\end_inset 


\layout Standard

If your program raises an exception, IPython will provide you with a more
 detailed traceback than the default Python ones.
 You can even increase the level of detail further by using 
\family typewriter 
%xmode Verbose
\family default 
, which forces the printing of variable values at all stack frames.
 This option should be used with care though (and that's why it is not the
 default), as printing a ten-million-entry array can lock up your computer
 for a very long time.
 An example of this kind of very informative traceback is shown in Fig.\SpecialChar ~

\begin_inset LatexCommand \ref{fig:ipscr_traceback}

\end_inset 

.
\layout Subsection

Automatic invocation of 
\family typewriter 
pdb
\family default 
 on exceptions
\layout Standard

IPython, if started with the 
\family typewriter 
-pdb
\family default 
 option (or if the option is set in your rc file) can call the Python 
\family typewriter 
pdb
\family default 
 debugger every time your code triggers an uncaught exception.
 This feature can also be toggled at any time with the 
\family typewriter 
%pdb
\family default 
 magic command.
 This can be extremely useful in order to find the origin of subtle bugs,
 because 
\family typewriter 
pdb
\family default 
 opens up at the point in your code which triggered the exception, and while
 your program is at this point `dead', all the data is still available and
 you can walk up and down the stack frame and understand the origin of the
 problem.
\layout Standard

Furthermore, you can use these debugging facilities both with the embedded
 IPython mode and without IPython at all.
 For an embedded shell (see sec.
 
\begin_inset LatexCommand \ref{sec:ipython_embed}

\end_inset 

), simply call the constructor with 
\family typewriter 
`-pdb'
\family default 
 in the argument string and automatically 
\family typewriter 
pdb
\family default 
 will be called if an uncaught exception is triggered by your code.
 
\layout Standard

For stand-alone use of the feature in your programs which do not use IPython
 at all, put the following lines toward the top of your `main' routine:
\layout LyX-Code

import sys,IPython.ultraTB
\layout LyX-Code

sys.excepthook = IPython.ultraTB.FormattedTB(mode=`Verbose', 
\newline 
color_scheme=`Linux', call_pdb=1)
\layout Standard

The 
\family typewriter 
mode
\family default 
 keyword can be either 
\family typewriter 
`Verbose'
\family default 
 or 
\family typewriter 
`Plain'
\family default 
, giving either very detailed or normal tracebacks respectively.
 The 
\family typewriter 
color_scheme
\family default 
 keyword can be one of 
\family typewriter 
`NoColor'
\family default 
, 
\family typewriter 
`Linux'
\family default 
 (default) or 
\family typewriter 
`LightBG'
\family default 
.
 These are the same options which can be set in IPython with 
\family typewriter 
-colors
\family default 
 and 
\family typewriter 
-xmode
\family default 
.
\layout Standard

This will give any of your programs detailed, colored tracebacks with automatic
 invocation of 
\family typewriter 
pdb
\family default 
.
\layout Subsection

Running entire programs via 
\family typewriter 
pdb
\layout Standard


\family typewriter 
pdb
\family default 
, the Python debugger, is a powerful interactive debugger which allows you
 to step through code, set breakpoints, watch variables, etc.
 IPython makes it very easy to start any script under the control of 
\family typewriter 
pdb
\family default 
, regardless of whether you have wrapped it into a 
\family typewriter 
`main()'
\family default 
 function or not.
 For this, simply type 
\family typewriter 
`%run -d myscript'
\family default 
 at an IPython prompt.
 See the 
\family typewriter 
%run
\family default 
 command's documentation (
\family typewriter 
run?
\family default 
) for more details, including how to control where 
\family typewriter 
pdb
\family default 
 will stop execution first.
\layout Standard

For more information on the use of the 
\family typewriter 
pdb
\family default 
 debugger, read the included 
\family typewriter 
pdb.doc
\family default 
 file (part of the standard Python distribution).
 On a stock Linux system it is located at 
\family typewriter 
/usr/lib/python2.3/pdb.doc
\family default 
, but the easiest way to read it is by using the 
\family typewriter 
help()
\family default 
 function of the 
\family typewriter 
pdb
\family default 
 module as follows (in an IPython prompt):
\layout LyX-Code

In [1]: import pdb
\layout LyX-Code

In [2]: pdb.help()
\layout Standard

This will load the 
\family typewriter 
pdb.doc
\family default 
 document in a file viewer for you automatically.
\layout Subsection

Profiling
\layout Standard

When dealing with performance issues, the 
\family typewriter 
%run
\family default 
 command with a 
\family typewriter 
-p
\family default 
 option allows you to run complete programs under the control of the Python
 profiler.
 The 
\family typewriter 
%prun
\family default 
 command does a similar job for single Python expressions (like function
 calls, similar to 
\family typewriter 
profile.run()
\family default 
).
 While this is possible with the standard 
\family typewriter 
profile
\family default 
 module, IPython wraps this functionality with magic commands convenient
 for rapid interactive work.
\layout Section


\begin_inset LatexCommand \label{sec:ipython_embed}

\end_inset 

Embedding IPython into your programs 
\begin_inset OptArg
collapsed true

\layout Standard

Embedding
\end_inset 


\layout Standard

A few lines of code are enough to load a complete IPython inside your own
 programs, giving you the ability to work with your data interactively after
 automatic processing has been completed.
 
\layout Standard

You can call IPython as a python shell inside your own python programs.
 This can be used both for debugging code or for providing interactive abilities
 to your programs with knowledge about the local namespaces (very useful
 in debugging and data analysis situations).
\layout Standard

It is possible to start an IPython instance 
\emph on 
inside
\emph default 
 your own Python programs.
 This allows you to evaluate dynamically the state of your code, operate
 with your variables, analyze them, etc.
 Note however that any changes you make to values while in the shell do
 
\emph on 
not
\emph default 
 propagate back to the running code, so it is safe to modify your values
 because you won't break your code in bizarre ways by doing so.
\layout Standard

This feature allows you to easily have a fully functional python environment
 for doing object introspection anywhere in your code with a simple function
 call.
 In some cases a simple print statement is enough, but if you need to do
 more detailed analysis of a code fragment this feature can be very valuable.
\layout Standard

It can also be useful in scientific computing situations where it is common
 to need to do some automatic, computationally intensive part and then stop
 to look at data, plots, etc
\begin_inset Foot
collapsed true

\layout Standard

This functionality was inspired by IDL's combination of the 
\family typewriter 
stop
\family default 
 keyword and the 
\family typewriter 
.continue
\family default 
 executive command, which I have found very useful in the past, and by a
 posting on comp.lang.python by cmkl <cmkleffner@gmx.de> on Dec.
 06/01 concerning similar uses of pyrepl.
\end_inset 

.
 Opening an IPython instance will give you full access to your data and
 functions, and you can resume program execution once you are done with
 the interactive part (perhaps to stop again later, as many times as needed).
\layout Standard

The following code snippet is the bare minimum you need to include in your
 Python programs for this to work (detailed examples follow later):
\layout LyX-Code

from IPython.Shell import IPShellEmbed
\layout LyX-Code

ipshell = IPShellEmbed()
\layout LyX-Code

ipshell() # this call anywhere in your program will start IPython
\layout Standard

You can run embedded instances even in code which is itself being run at
 the IPython interactive prompt with '
\family typewriter 
%run\SpecialChar ~
<filename>
\family default 
'.
 Since it's easy to get lost as to where you are (in your top-level IPython
 or in your embedded one), it's a good idea in such cases to set the in/out
 prompts to something different for the embedded instances.
 The code examples below illustrate this.
\layout Standard

You can also have multiple IPython instances in your program and open them
 separately, for example with different options for data presentation.
 If you close and open the same instance multiple times, its prompt counters
 simply continue from each execution to the next.
\layout Standard

Please look at the docstrings in the 
\family typewriter 
Shell.py
\family default 
 module for more details on the use of this system.
\layout Standard

The following sample file illustrating how to use the embedding functionality
 is provided in the examples directory as 
\family typewriter 
example-embed.py
\family default 
.
 It should be fairly self-explanatory:
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lstinputlisting{examples/ip_embed.py}
\end_inset 


\layout Standard

Once you understand how the system functions, you can use the following
 code fragments in your programs which are ready for cut and paste:
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lstinputlisting{examples/ip_embed-short.py}
\end_inset 


\layout Section


\begin_inset LatexCommand \label{sec:ipython_pylab}

\end_inset 

Integration with Matplotlib
\begin_inset OptArg
collapsed true

\layout Standard

Matplotlib
\end_inset 


\layout Standard

The matplotlib library (
\begin_inset LatexCommand \htmlurl{http://matplotlib.sourceforge.net}

\end_inset 

) provides high quality 2D plotting for Python.
 Matplotlib can produce plots on screen using a variety of GUI toolkits,
 including Tk, GTK and WXPython.
 It also provides a number of commands useful for scientific computing,
 all with a syntax compatible with that of the popular Matlab program.
\layout Standard

IPython accepts the special option 
\family typewriter 
-pylab
\family default 
.
 This configures it to support matplotlib, honoring the settings in the
 
\family typewriter 
.matplotlibrc
\family default 
 file.
 IPython will detect the user's choice of matplotlib GUI backend, and automatica
lly select the proper threading model to prevent blocking.
 It also sets matplotlib in interactive mode and modifies 
\family typewriter 
%run
\family default 
 slightly, so that any matplotlib-based script can be executed using 
\family typewriter 
%run
\family default 
 and the final 
\family typewriter 
show()
\family default 
 command does not block the interactive shell.
\layout Standard

The 
\family typewriter 
-pylab
\family default 
 option must be given first in order for IPython to configure its threading
 mode.
 However, you can still issue other options afterwards.
 This allows you to have a matplotlib-based environment customized with
 additional modules using the standard IPython profile mechanism: ``
\family typewriter 
ipython -pylab -p myprofile
\family default 
'' will load the profile defined in 
\family typewriter 
ipythonrc-myprofile
\family default 
 after configuring matplotlib.
\the_end