\texttt{pyrex} is a pure python packages that utilizes a custom
language which is a hybid of C and python to write code that looks
like python, but is converted by \texttt{pyrex} into python C
extension code. It can be used to write custom C extension modules in
a python like module to remove performance bottlenecks in code, as
well as to wrap and existing C API with a python binding. \texttt{pyrex}
generates C code, so you can use it to automatically generate C
extensions that you can ship with your code and users can build your
code without \texttt{pyrex} installed.
\section{Writing C extensions \texttt{pyrex}}
The canonical \texttt{pyrex} example generates a list of \texttt{N}
prime numbers, and illustrates the hybrid nature of \texttt{pyrex}
syntax
\begin{lstlisting}
# name this file with the pyx extension for pyrex, rather than the py
# extension for python, eg primes.pyx
def primes(int kmax):
# pyrex uses cdef to declare a c type
cdef int n, k, i
cdef int p[1000]
# you can use normal python too, eg a python list
result = []
if kmax > 1000:
kmax = 1000
k = 0
n = 2
while k < kmax:
i = 0
while i < k and n % p[i] <> 0:
i = i + 1
if i == k:
p[k] = n
k = k + 1
result.append(n)
n = n + 1
return result
\end{lstlisting}
To build our python extension, we will use the \texttt{pyrex.distutils}
extensions. Here is a typical setup.py
\begin{lstlisting}
from distutils.core import setup
# we use the Pyrex distutils Extension class rather than the standard
# python one
#from distutils.extension import Extension
from Pyrex.Distutils.extension import Extension
from Pyrex.Distutils import build_ext
setup(
name = 'Demos',
ext_modules=[
Extension("primes", ["primes.pyx"]),
],
cmdclass = {'build_ext': build_ext}
)
\end{lstlisting}
and we can build it in place using
\begin{lstlisting}
python setup.py build_ext --inplace
\end{lstlisting}
This creates a primes.c module which is the generated C code that we
can ship with our python code to users who may not have \texttt{pyrex}
installed, and a primes.so file which is the python shared library
extension. We can now fire up ipython, import primes, and call our
function with C performance. Here is an example shell session in
which we build and test our new extension code
\begin{lstlisting}
# our single pyx file from above
pyrex_demos> ls primes*
primes.pyx
# build the module in place
pyrex_demos> python setup.py build_ext --inplace
running build_ext
pyrexc primes.pyx --> primes.c
building 'primes' extension
creating build
creating build/temp.macosx-10.3-fat-2.5
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c primes.c -o build/temp.macosx-10.3-fat-2.5/primes.o
gcc -arch i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.5/primes.o -o primes.so
# now we have the original pyx and also the autogenerated C file and
# the extension module
pyrex_demos> ls primes*
primes.cprimes.pyxprimes.so
# let's test drive this in ipython
pyrex_demos> ipython
IPython 0.8.3.svn.r2876 -- An enhanced Interactive Python.
In [1]: import primes
In [2]: dir(primes)
Out[2]: ['__builtins__', '__doc__', '__file__', '__name__', 'primes']
In [3]: print primes.primes(20)
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71]
\end{lstlisting}
\section{Working with \texttt{numpy} arrays}
\texttt{numpy} arrays are the core of high performance computing in
python, and one of the most common data formats for passing large data
sets around between pyhton code and other wrappers. There are many
things that arrays do very well and are practically as fast as a
native C or FORTRAN implementations, eg convolutions and FFTs. But
there are somethings that can be painfully slow in python when working
with arrays, for example iterative algorithms over an array of values.
For these cases, it is nice to be able to quickly generate some python
extension code for working with \texttt{numpy} array data.
\texttt{numpy} provides a file which exposes its C API for use in
\texttt{pyrex} extension code, you can find it, and another file which
\texttt{numpy} uses to expose the requisite bits of the Python C API
which it needs, in the \texttt{numpy} source code directory
\texttt{numpy/doc/pyrex}. These files are \texttt{c\_numpy.pxd} and
\texttt{c\_python.pxd}. In addition, \texttt{numpy} provides and
example file \texttt{numpyx.pyx}that shows you how to build a pyx
extension file for multi-dimensional array sof different data types
(eg int, float, python object). Here we will be a little less
ambitious for starters, and write a simple toy function that sums a 1D
array of floats.
\begin{lstlisting}
# import the numpy c API (you need to have c_python.pxd and
# c_numpy.pxd from the numpy source directory in your build directory
cimport c_numpy
# since this is pyrex, we can import normal python modules too
import numpy
# numpy must be initialized -- don't forget to do this when writing
# numpy extension code. It's a common gotcha
c_numpy.import_array()
def sum_elements(c_numpy.ndarray arr):
cdef int i
cdef double x, val
x = 0.
val = 0.
for i from 0<=i<arr.dimensions[0]:
val = (<double*>(arr.data + i*arr.strides[0]))[0]
x = x + val
return x
\end{lstlisting}
