Menu
▾
▴
[r4503]: / trunk / course / scipy_tour.lyx Maximize Restore History
5717 lines (4358 with data), 132.0 kB
#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass amsbook
\language english
\inputencoding auto
\fontscheme default
\graphics default
\paperfontsize default
\papersize Default
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default
\layout Chapter
A tour of SciPy
\layout Standard
\align right
Chapter contributed by Travis E.
Oliphant
\layout Section
Introduction
\layout Standard
SciPy is a collection of mathematical algorithms and convenience functions
built on the Numeric extension for Python.
It adds significant power to the interactive Python session by exposing
the user to high-level commands and classes for the manipulation and visualizat
ion of data.
With SciPy, an interactive Python session becomes a data-processing and
system-prototyping environment rivaling sytems such as Matlab, IDL, Octave,
R-Lab, and SciLab.
\layout Standard
The additional power of using SciPy within Python, however, is that a powerful
programming language is also available for use in developing sophisticated
programs and specialized applications.
Scientific applications written in SciPy benefit from the development of
additional modules in numerous niche's of the software landscape by developers
across the world.
Everything from parallel programming to web and data-base subroutines and
classes have been made available to the Python programmer.
All of this power is available in addition to the mathematical libraries
in SciPy.
\layout Standard
This document provides a tutorial for the first-time user of SciPy to help
get started with some of the features available in this powerful package.
It is assumed that the user has already installed the package.
Some general Python facility is also assumed such as could be acquired
by working through the Tutorial in the Python distribution.
For further introductory help the user is directed to the Numeric documentation.
Throughout this tutorial it is assumed that the user has imported all of
the names defined in the SciPy namespace using the command
\begin_inset ERT
status Collapsed
\layout Standard
\backslash
begin{verbatim}
\newline
>>> from scipy import *
\backslash
end{verbatim}
\end_inset
\layout Subsection
General Help
\layout Standard
Python provides the facility of documentation strings.
The functions and classes available in SciPy use this method for on-line
documentation.
There are two methods for reading these messages and getting help.
Python provides the command
\series bold
help
\series default
in the pydoc module.
Entering this command with no arguments (i.e.
>>> help ) launches an interactive help session that allows searching through
the keywords and modules available to all of Python.
Running the command help with an object as the argument displays the calling
signature, and the documentation string of the object.
\layout Standard
The pydoc method of help is sophisticated but uses a pager to display the
text.
Sometimes this can interfere with the terminal you are running the interactive
session within.
A scipy-specific help system is also available under the command scipy.info.
The signature and documentation string for the object passed to the help
command are printed to standard output (or to a writeable object passed
as the third argument).
The second keyword argument of
\begin_inset Quotes eld
\end_inset
scipy.info
\begin_inset Quotes erd
\end_inset
defines the maximum width of the line for printing.
If a module is passed as the argument to help than a list of the functions
and classes defined in that module is printed.
For example:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example1.1}
preview false
\end_inset
\layout Standard
Another useful command is
\series bold
source.
\family typewriter
\series default
\family default
When given a function written in Python as an argument, it prints out a
listing of the source code for that function.
This can be helpful in learning about an algorithm or understanding exactly
what a function is doing with its arguments.
Also don't forget about the Python command
\family typewriter
dir
\family default
which can be used to look at the namespace of a module or package.
\layout Subsection
SciPy Organization
\layout Standard
SciPy is organized into subpackages covering different scientific computing
domains.
Some common functions which several subpackages rely on live under the
\family typewriter
scipy_base
\family default
package which is installed at the same directory level as the scipy package
itself and could be installed separately.
This allows for the possibility of separately distributing the subpackages
of scipy as long as scipy_base package is provided as well.
\layout Standard
Two other packages are installed at the higher-level: scipy_distutils and
weave.
These two packages while distributed with main scipy package could see
use independently of scipy and so are treated as separate packages and
described elsewhere.
\layout Standard
The remaining subpackages are summarized in the following table (a * denotes
an optional sub-package that requires additional libraries to function
or is not available on all platforms).
\layout Standard
\align center
\begin_inset Tabular
<lyxtabular version="3" rows="17" columns="2">
<features>
<column alignment="center" valignment="top" leftline="true" width="0pt">
<column alignment="center" valignment="top" leftline="true" rightline="true" width="0pt">
<row topline="true" bottomline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
Subpackage
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Description
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
cluster
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Clustering algorithms
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
cow
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Cluster of Workstations code for parallel programming
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
fftpack
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
FFT based on fftpack -- default
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
fftw*
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
FFT based on fftw --- requires FFTW libraries (is this still needed?)
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
ga
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Genetic algorithms
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
gplt*
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Plotting --- requires gnuplot
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
integrate
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Integration
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
interpolate
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Interpolation
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
io
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Input and Output
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
linalg
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Linear algebra
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
optimize
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Optimization and root-finding routines
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
plt*
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Plotting --- requires wxPython
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
signal
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Signal processing
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
special
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Special functions
\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
stats
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Statistical distributions and functions
\end_inset
</cell>
</row>
<row topline="true" bottomline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text
\layout Standard
xplt
\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
\layout Standard
Plotting with gist
\end_inset
</cell>
</row>
</lyxtabular>
\end_inset
\layout Standard
Because of their ubiquitousness, some of the functions in these subpackages
are also made available in the scipy namespace to ease their use in interactive
sessions and programs.
In addition, many convenience functions are located in the scipy_base package
and the in the top-level of the scipy package.
Before looking at the sub-packages individually, we will first look at
some of these common functions.
\layout Section
Basic functions in scipy_base and top-level scipy
\begin_inset OptArg
collapsed true
\layout Standard
Basic functions
\end_inset
\layout Subsection
Interaction with Numeric
\layout Standard
To begin with, all of the Numeric functions have been subsumed into the
scipy namespace so that all of those functions are available without additional
ly importing Numeric.
In addition, the universal functions (addition, subtraction, division)
have been altered to not raise exceptions if floating-point errors are
encountered
\begin_inset Foot
collapsed true
\layout Standard
These changes are all made in a new module (fastumath) that is part of the
scipy_base package.
The old functionality is still available in umath (part of Numeric) if
you need it (note: importing umath or fastumath resets the behavior of
the infix operators to use the umath or fastumath ufuncs respectively).
\end_inset
, instead NaN's and Inf's are returned in the arrays.
To assist in detection of these events new universal functions (isnan,
isfinite, isinf) have been added.
\layout Standard
In addition, the comparision operators have been changed to allow comparisons
and logical operations of complex numbers (only the real part is compared).
Also, with the new universal functions in SciPy, the logical operations
(except logical_XXX functions) all return arrays of unsigned bytes (8-bits
per element instead of the old 32-bits, or even 64-bits) per element
\begin_inset Foot
collapsed true
\layout Standard
Be careful when treating logical expressions as integers as the 8-bit integers
may silently overflow at 256.
\end_inset
.
\layout Standard
In an effort to get a consistency for default arguments, some of the default
arguments have changed from Numeric.
The idea is for you to use scipy as a base package instead of Numeric anyway.
\layout Standard
Finally, some of the basic functions like log, sqrt, and inverse trig functions
have been modified to return complex numbers instead of NaN's where appropriate
(
\emph on
i.e.
\family typewriter
\emph default
scipy.sqrt(-1)
\family default
returns
\family typewriter
1j
\family default
).
\layout Subsection
Alter numeric
\layout Standard
With the command
\series bold
scipy.alter_numeric()
\series default
you can now use index and mask arrays inside brackets and the coercion rules
of Numeric will be changed so that Python scalars will not be used to determine
the type of the output of an expression.
\layout Subsection
Scipy_base routines
\layout Standard
The purpose of scipy_base is to collect general-purpose routines that the
other sub-packages can use and to provide a simple replacement for Numeric.
Anytime you might think to import Numeric, you can import scipy_base instead
and remove yourself from direct dependence on Numeric.
These routines are divided into several files for organizational purposes,
but they are all available under the scipy_base namespace (and the scipy
namespace).
There are routines for type handling and type checking, shape and matrix
manipulation, polynomial processing, and other useful functions.
Rather than giving a detailed description of each of these functions (which
is available using the
\series bold
help
\series default
,
\series bold
info
\series default
and
\series bold
source
\series default
commands), this tutorial will discuss some of the more useful commands
which require a little introduction to use to their full potential.
\layout Subsubsection
Type handling
\layout Standard
Note the difference between
\series bold
iscomplex
\series default
(
\series bold
isreal
\series default
) and
\series bold
iscomplexobj
\series default
(
\series bold
isrealobj
\series default
).
The former command is array based and returns byte arrays of ones and zeros
providing the result of the element-wise test.
The latter command is object based and returns a scalar describing the
result of the test on the entire object.
\layout Standard
Often it is required to get just the real and/or imaginary part of a complex
number.
While complex numbers and arrays have attributes that return those values,
if one is not sure whether or not the object will be complex-valued, it
is better to use the functional forms
\series bold
real
\series default
and
\series bold
imag
\series default
.
These functions succeed for anything that can be turned into a Numeric
array.
Consider also the function
\series bold
real_if_close
\series default
which transforms a complex-valued number with tiny imaginary part into a
real number.
\layout Standard
Occasionally the need to check whether or not a number is a scalar (Python
(long)int, Python float, Python complex, or rank-0 array) occurs in coding.
This functionality is provided in the convenient function
\series bold
isscalar
\series default
which returns a 1 or a 0.
\layout Standard
Finally, ensuring that objects are a certain Numeric type occurs often enough
that it has been given a convenient interface in SciPy through the use
of the
\series bold
cast
\series default
dictionary.
The dictionary is keyed by the type it is desired to cast to and the dictionary
stores functions to perform the casting.
Thus,
\family typewriter
>>> a = cast['f'](d)
\family default
returns an array of float32 from d.
This function is also useful as an easy way to get a scalar of a certain
type:
\family typewriter
>>> fpi = cast['f'](pi)
\family default
although this should not be needed if you use alter_numeric().
\layout Subsubsection
Index Tricks
\layout Standard
Thre are some class instances that make special use of the slicing functionality
to provide efficient means for array construction.
This part will discuss the operation of
\series bold
mgrid
\series default
,
\series bold
ogrid
\series default
,
\series bold
r_
\series default
,
\series bold
\series default
and
\series bold
c_
\series default
for quickly constructing arrays.
\layout Standard
One familiar with Matlab may complain that it is difficult to construct
arrays from the interactive session with Python.
Suppose, for example that one wants to construct an array that begins with
3 followed by 5 zeros and then contains 10 numbers spanning the range -1
to 1 (inclusive on both ends).
Before SciPy, you would need to enter something like the following
\family typewriter
\newline
>>> concatenate(([3],[0]*5,arange(-1,1.002,2/9.0))
\family default
.
\newline
With the
\series bold
r_
\series default
command one can enter this as
\family typewriter
\newline
>>> r_[3,[0]*5,-1:1:10j]
\family default
\newline
which can ease typing and make for more readable code.
Notice how objects are concatenated, and the slicing syntax is (ab)used
to construct ranges.
The other term that deserves a little explanation is the use of the complex
number 10j as the step size in the slicing syntax.
This non-standard use allows the number to be interpreted as the number
of points to produce in the range rather than as a step size (note we would
have used the long integer notation, 10L, but this notation may go away
in Python as the integers become unified).
This non-standard usage may be unsightly to some, but it gives the user
the ability to quickly construct complicated vectors in a very readable
fashion.
When the number of points is specified in this way, the end-point is inclusive.
\layout Standard
The
\begin_inset Quotes eld
\end_inset
r
\begin_inset Quotes erd
\end_inset
stands for row concatenation because if the objects between commas are
2 dimensional arrays, they are stacked by rows (and thus must have commensurate
columns).
There is an equivalent command
\series bold
c_
\series default
that stacks 2d arrays by columns but works identically to
\series bold
r_
\series default
for 1d arrays.
\layout Standard
Another very useful class instance which makes use of extended slicing notation
is the function
\series bold
mgrid
\series default
.
\series bold
\series default
In the simplest case, this function can be used to construct 1d ranges as
a convenient substitute for arange.
It also allows the use of complex-numbers in the step-size to indicate
the number of points to place between the (inclusive) end-points.
The real purpose of this function however is to produce N, N-d arrays which
provide coordinate arrays for an N-dimensional volume.
The easiest way to understand this is with an example of its usage:
\begin_inset Include \verbatiminput{examples/scipy/example2.1}
preview false
\end_inset
\layout Standard
Having meshed arrays like this is sometimes very useful.
However, it is not always needed just to evaluate some N-dimensional function
over a grid due to the array-broadcasting rules of Numeric and SciPy.
If this is the only purpose for generating a meshgrid, you should instead
use the function
\series bold
ogrid
\series default
which generates an
\begin_inset Quotes eld
\end_inset
open
\begin_inset Quotes erd
\end_inset
grid using NewAxis judiciously to create N, N-d arrays where only one-dimension
in each array has length greater than 1.
This will save memory and create the same result if the only purpose for
the meshgrid is to generate sample points for evaluation of an N-d function.
\layout Subsubsection
Shape manipulation
\layout Standard
In this category of functions are routines for squeezing out length-one
dimensions from N-dimensional arrays, ensuring that an array is at least
1-, 2-, or 3-dimensional, and stacking (concatenating) arrays by rows,
columns, and
\begin_inset Quotes eld
\end_inset
pages
\begin_inset Quotes erd
\end_inset
(in the third dimension).
Routines for splitting arrays (roughly the opposite of stacking arrays)
are also available.
\layout Subsubsection
Matrix manipulations
\layout Standard
These are functions specifically suited for 2-dimensional arrays that were
part of MLab in the Numeric distribution, but have been placed in scipy_base
for completeness so that users are not importing Numeric.
\layout Subsubsection
Polynomials
\layout Standard
There are two (interchangeable) ways to deal with 1-d polynomials in SciPy.
The first is to use the
\series bold
poly1d
\series default
class in
\series bold
scipy_base
\series default
.
This class accepts coefficients or polynomial roots to initialize a polynomial.
The polynomial object can then be manipulated in algebraic expressions,
integrated, differentiated, and evaluated.
It even prints like a polynomial:
\begin_inset Include \verbatiminput{examples/scipy/example2.2}
preview false
\end_inset
\layout Standard
The other way to handle polynomials is as an array of coefficients with
the first element of the array giving the coefficient of the highest power.
There are explicit functions to add, subtract, multiply, divide, integrate,
differentiate, and evaluate polynomials represented as sequences of coefficient
s.
\layout Subsubsection
Vectorizing functions (vectorize)
\layout Standard
One of the features that SciPy provides is a class
\series bold
vectorize
\series default
to convert an ordinary Python function which accepts scalars and returns
scalars into a
\begin_inset Quotes eld
\end_inset
vectorized-function
\begin_inset Quotes erd
\end_inset
with the same broadcasting rules as other Numeric functions (
\emph on
i.e.
\emph default
the Universal functions, or ufuncs).
For example, suppose you have a Python function named
\series bold
addsubtract
\series default
defined as:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example3.1}
preview false
\end_inset
which defines a function of two scalar variables and returns a scalar result.
The class vectorize can be used to
\begin_inset Quotes eld
\end_inset
vectorize
\begin_inset Quotes erd
\end_inset
this function so that
\begin_inset ERT
status Open
\layout Standard
\backslash
begin{verbatim}
\newline
>>> vec_addsubtract = vectorize(addsubtract)
\backslash
end{verbatim}
\newline
\backslash
noindent
\end_inset
returns a function which takes array arguments and returns an array result:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example3.2}
preview false
\end_inset
\layout Standard
This particular function could have been written in vector form without
the use of
\series bold
vectorize
\series default
.
But, what if the function you have written is the result of some optimization
or integration routine.
Such functions can likely only be vectorized using
\family typewriter
vectorize.
\layout Subsubsection
Other useful functions
\layout Standard
There are several other functions in the scipy_base package including most
of the other functions that are also in MLab that comes with the Numeric
package.
The reason for duplicating these functions is to allow SciPy to potentially
alter their original interface and make it easier for users to know how
to get access to functions
\family typewriter
>>> from scipy import *.
\layout Standard
New functions which should be mentioned are
\series bold
mod(x,y)
\series default
which can replace
\series bold
x%y
\series default
when it is desired that the result take the sign of
\series bold
y
\series default
instead of
\series bold
x
\series default
.
Also included is
\series bold
fix
\series default
which always rounds to the nearest integer towards zero.
For doing phase processing, the functions
\series bold
angle,
\series default
and
\series bold
unwrap
\series default
are also useful.
Also, the
\series bold
linspace
\series default
and
\series bold
logspace
\series default
functions return equally spaced samples in a linear or log scale.
Finally, mention should be made of the new function
\series bold
select
\series default
which extends the functionality of
\series bold
where
\series default
to include multiple conditions and multiple choices.
The calling convention is
\family typewriter
select(condlist,choicelist,default=0).
\family default
\series bold
Select
\series default
is
\series bold
\series default
a vectorized form of the multiple if-statement.
It allows rapid construction of a function which returns an array of results
based on a list of conditions.
Each element of the return array is taken from the array in a
\family typewriter
choicelist
\family default
corresponding to the first condition in
\family typewriter
condlist
\family default
that is true.
For example
\begin_inset Include \verbatiminput{examples/scipy/example2.3}
preview false
\end_inset
\layout Subsection
Common functions
\layout Standard
Some functions depend on sub-packages of SciPy but should be available from
the top-level of SciPy due to their common use.
These are functions that might have been placed in scipy_base except for
their dependence on other sub-packages of SciPy.
For example the
\series bold
factorial
\series default
and
\series bold
comb
\series default
functions compute
\begin_inset Formula $n!$
\end_inset
and
\begin_inset Formula $n!/k!(n-k)!$
\end_inset
using either exact integer arithmetic (thanks to Python's Long integer
object), or by using floating-point precision and the gamma function.
The functions
\series bold
rand
\series default
and
\series bold
randn
\series default
are used so often that they warranted a place at the top level.
There are convenience functions for the interactive use:
\series bold
disp
\series default
(similar to print),
\series bold
\series default
and
\series bold
who
\series default
(returns a list of defined variables and memory consumption--upper bounded).
Another function returns a common image used in image processing:
\series bold
lena.
\series default
\layout Standard
Finally, two functions are provided that are useful for approximating derivative
s of functions using discrete-differences.
The function
\series bold
central_diff_weights
\series default
returns weighting coefficients for an equally-spaced
\begin_inset Formula $N$
\end_inset
-point approximation to the derivative of order
\begin_inset Formula $o.$
\end_inset
These weights must be multiplied by the function corresponding to these
points and the results added to obtain the derivative approximation.
This function is intended for use when only samples of the function are
avaiable.
When the function is an object that can be handed to a routine and evaluated,
the function
\series bold
derivative
\series default
can be used to automatically evaluate the object at the correct points
to obtain an N-point approximation to the
\begin_inset Formula $o^{\textrm{th}}$
\end_inset
-derivative at a given point.
\layout Section
Special functions (special)
\begin_inset OptArg
collapsed true
\layout Standard
Special functions
\end_inset
\layout Standard
The main feature of the
\series bold
special
\series default
package is the definition of numerous special functions of mathematical
physics.
Available functions include airy, elliptic, bessel, gamma, beta, hypergeometric
, parabolic cylinder, mathieu, spheroidal wave, struve, and kelvin.
There are also some low-level stats functions that are not intended for
general use as an easier interface to these functions is provided by the
\family typewriter
stats
\family default
module.
Most of these functions can take array arguments and return array results
following the same broadcasting rules as other math functions in Numerical
Python.
Many of these functions also accept complex-numbers as input.
For a complete list of the available functions with a one-line description
type
\family typewriter
>>>info(special).
\family default
Each function also has it's own documentation accessible using help.
If you don't see a function you need, consider writing it and contributing
it to the library.
You can write the function in either C, Fortran, or Python.
Look in the source code of the library for examples of each of these kind
of functions.
\layout Section
Integration (integrate)
\begin_inset OptArg
collapsed true
\layout Standard
Integration
\end_inset
\layout Standard
The
\series bold
integrate
\series default
sub-package provides several integration techniques including an ordinary
differential equation integrator.
An overview of the module is provided by the help command:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.1}
preview false
\end_inset
\layout Subsection
General integration (integrate.quad)
\layout Standard
The function
\series bold
quad
\series default
is provided to integrate a function of one variable between two points.
The points can be
\begin_inset Formula $\pm\infty$
\end_inset
(
\begin_inset Formula $\pm$
\end_inset
\series bold
integrate.inf
\series default
) to indicate infinite limits.
For example, suppose you wish to integrate a bessel function
\family typewriter
jv(2.5,x)
\family default
along the interval
\begin_inset Formula $[0,4.5].$
\end_inset
\begin_inset Formula \[
I=\int_{0}^{4.5}J_{2.5}\left(x\right)\, dx.\]
\end_inset
This could be computed using
\series bold
quad:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.2}
preview false
\end_inset
\layout Standard
The first argument to quad is a
\begin_inset Quotes eld
\end_inset
callable
\begin_inset Quotes erd
\end_inset
Python object (
\emph on
i.e
\emph default
a function, method, or class instance).
Notice the use of a lambda-function in this case as the argument.
The next two arguments are the limits of integration.
The return value is a tuple, with the first element holding the estimated
value of the integral and the second element holding an upper bound on
the error.
Notice, that in this case, the true value of this integral is
\begin_inset Formula \[
I=\sqrt{\frac{2}{\pi}}\left(\frac{18}{27}\sqrt{2}\cos\left(4.5\right)-\frac{4}{27}\sqrt{2}\sin\left(4.5\right)+\sqrt{2\pi}\textrm{Si}\left(\frac{3}{\sqrt{\pi}}\right)\right),\]
\end_inset
where
\begin_inset Formula \[
\textrm{Si}\left(x\right)=\int_{0}^{x}\sin\left(\frac{\pi}{2}t^{2}\right)\, dt.\]
\end_inset
is the Fresnel sine integral.
Note that the numerically-computed integral is within
\begin_inset Formula $1.04\times10^{-11}$
\end_inset
of the exact result --- well below the reported error bound.
\layout Standard
Infinite inputs are also allowed in
\series bold
quad
\series default
by using
\begin_inset Formula $\pm$
\end_inset
\series bold
integrate.inf
\series default
(or
\series bold
inf)
\series default
as one of the arguments.
For example, suppose that a numerical value for the exponential integral:
\begin_inset Formula \[
E_{n}\left(x\right)=\int_{1}^{\infty}\frac{e^{-xt}}{t^{n}}\, dt.\]
\end_inset
is desired (and the fact that this integral can be computed as
\family typewriter
special.expn(n,x)
\family default
is forgotten).
The functionality of the function
\series bold
special.expn
\series default
can be replicated by defining a new function
\series bold
vec_expint
\series default
based on the routine
\series bold
quad:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.3}
preview false
\end_inset
\layout Standard
The function which is integrated can even use the quad argument (though
the error bound may underestimate the error due to possible numerical error
in the integrand from the use of
\series bold
quad
\series default
).
The integral in this case is
\begin_inset Formula \[
I_{n}=\int_{0}^{\infty}\int_{1}^{\infty}\frac{e^{-xt}}{t^{n}}\, dt\, dx=\frac{1}{n}.\]
\end_inset
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.4}
preview false
\end_inset
\layout Standard
This last example shows that multiple integration can be handled using repeated
calls to
\series bold
quad.
\series default
The mechanics of this for double and triple integration have been wrapped
up into the functions
\series bold
dblquad
\series default
and
\series bold
tplquad.
\series default
The function,
\series bold
dblquad
\series default
performs double integration.
Use the help function to be sure that the arguments are defined in the
correct order.
In addition, the limits on all inner integrals are actually functions which
can be constant functions.
An example of using double integration to compute several values of
\begin_inset Formula $I_{n}$
\end_inset
is shown below:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.5}
preview false
\end_inset
\layout Subsection
Gaussian quadrature (integrate.gauss_quadtol)
\layout Standard
A few functions are also provided in order to perform simple Gaussian quadrature
over a fixed interval.
The first is
\series bold
fixed_quad
\series default
which performs fixed-order Gaussian quadrature.
The second function is
\series bold
quadrature
\series default
which performs Gaussian quadrature of multiple orders until the difference
in the integral estimate is beneath some tolerance supplied by the user.
These functions both use the module
\series bold
special.orthogonal
\series default
which can calculate the roots and quadrature weights of a large variety
of orthogonal polynomials (the polynomials themselves are available as
special functions returning instances of the polynomial class --- e.g.
\family typewriter
special.legendre
\family default
).
\layout Subsection
Integrating using samples
\layout Standard
There are three functions for computing integrals given only samples:
\series bold
trapz
\series default
,
\series bold
simps
\series default
, and
\series bold
romb
\series default
.
The first two functions use Newton-Coates formulas of order 1 and 2 respectivel
y to perform integration.
These two functions can handle, non-equally-spaced samples.
The trapezoidal rule approximates the function as a straight line between
adjacent points, while Simpson's rule approximates the function between
three adjacent points as a parabola.
\layout Standard
If the samples are equally-spaced and the number of samples available is
\begin_inset Formula $2^{k}+1$
\end_inset
for some integer
\begin_inset Formula $k$
\end_inset
, then Romberg integration can be used to obtain high-precision estimates
of the integral using the available samples.
Romberg integration uses the trapezoid rule at step-sizes related by a
power of two and then performs Richardson extrapolation on these estimates
to approximate the integral with a higher-degree of accuracy.
(A different interface to Romberg integration useful when the function
can be provided is also available as
\series bold
integrate.romberg)
\series default
.
\series bold
\layout Subsection
Ordinary differential equations (integrate.odeint)
\layout Standard
Integrating a set of ordinary differential equations (ODEs) given initial
conditions is another useful example.
The function
\series bold
odeint
\series default
is available in SciPy for integrating a first-order vector differential
equation:
\begin_inset Formula \[
\frac{d\mathbf{y}}{dt}=\mathbf{f}\left(\mathbf{y},t\right),\]
\end_inset
given initial conditions
\begin_inset Formula $\mathbf{y}\left(0\right)=y_{0},$
\end_inset
where
\begin_inset Formula $\mathbf{y}$
\end_inset
is a length
\begin_inset Formula $N$
\end_inset
vector and
\begin_inset Formula $\mathbf{f}$
\end_inset
is a mapping from
\begin_inset Formula $\mathcal{R}^{N}$
\end_inset
to
\begin_inset Formula $\mathcal{R}^{N}.$
\end_inset
A higher-order ordinary differential equation can always be reduced to
a differential equation of this type by introducing intermediate derivatives
into the
\begin_inset Formula $\mathbf{y}$
\end_inset
vector.
\layout Standard
For example suppose it is desired to find the solution to the following
second-order differential equation:
\begin_inset Formula \[
\frac{d^{2}w}{dz^{2}}-zw(z)=0\]
\end_inset
with initial conditions
\begin_inset Formula $w\left(0\right)=\frac{1}{\sqrt[3]{3^{2}}\Gamma\left(\frac{2}{3}\right)}$
\end_inset
and
\begin_inset Formula $\left.\frac{dw}{dz}\right|_{z=0}=-\frac{1}{\sqrt[3]{3}\Gamma\left(\frac{1}{3}\right)}.$
\end_inset
It is known that the solution to this differential equation with these
boundary conditions is the Airy function
\begin_inset Formula \[
w=\textrm{Ai}\left(z\right),\]
\end_inset
which gives a means to check the integrator using
\series bold
special.airy.
\layout Standard
First, convert this ODE into standard form by setting
\begin_inset Formula $\mathbf{y}=\left[\frac{dw}{dz},w\right]$
\end_inset
and
\begin_inset Formula $t=z.$
\end_inset
Thus, the differential equation becomes
\begin_inset Formula \[
\frac{d\mathbf{y}}{dt}=\left[\begin{array}{c}
ty_{1}\\
y_{0}\end{array}\right]=\left[\begin{array}{cc}
0 & t\\
1 & 0\end{array}\right]\left[\begin{array}{c}
y_{0}\\
y_{1}\end{array}\right]=\left[\begin{array}{cc}
0 & t\\
1 & 0\end{array}\right]\mathbf{y}.\]
\end_inset
In other words,
\begin_inset Formula \[
\mathbf{f}\left(\mathbf{y},t\right)=\mathbf{A}\left(t\right)\mathbf{y}.\]
\end_inset
\layout Standard
As an interesting reminder, if
\begin_inset Formula $\mathbf{A}\left(t\right)$
\end_inset
commutes with
\begin_inset Formula $\int_{0}^{t}\mathbf{A}\left(\tau\right)\, d\tau$
\end_inset
under matrix multiplication, then this linear differential equation has
an exact solution using the matrix exponential:
\begin_inset Formula \[
\mathbf{y}\left(t\right)=\exp\left(\int_{0}^{t}\mathbf{A}\left(\tau\right)d\tau\right)\mathbf{y}\left(0\right),\]
\end_inset
However, in this case,
\begin_inset Formula $\mathbf{A}\left(t\right)$
\end_inset
and its integral do not commute.
\layout Standard
There are many optional inputs and outputs available when using odeint which
can help tune the solver.
These additional inputs and outputs are not needed much of the time, however,
and the three required input arguments and the output solution suffice.
The required inputs are the function defining the derivative,
\emph on
fprime,
\emph default
the initial conditions vector,
\emph on
y0
\emph default
, and the time points to obtain a solution,
\emph on
t,
\emph default
(with the initial value point as the first element of this sequence).
The output to
\series bold
odeint
\series default
is a matrix where each row contains the solution vector at each requested
time point (thus, the initial conditions are given in the first output
row).
\layout Standard
The following example illustrates the use of odeint including the usage
of the
\series bold
Dfun
\series default
option which allows the user to specify a gradient (with respect to
\begin_inset Formula $\mathbf{y}$
\end_inset
) of the function,
\series bold
\begin_inset Formula $\mathbf{f}\left(\mathbf{y},t\right)$
\end_inset
.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example4.6}
preview false
\end_inset
\layout Section
Optimization (optimize)
\layout Standard
There are several classical optimization algorithms provided by SciPy in
the
\series bold
optimize
\series default
package.
An overview of the module is available using
\series bold
help
\series default
(or pydoc.help):
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.1}
preview false
\end_inset
The first four algorithms are unconstrained minimization algorithms (fmin:
Nelder-Mead simplex, fmin_bfgs: BFGS, fmin_ncg: Newton Conjugate Gradient,
and leastsq: Levenburg-Marquardt).
The fourth algorithm only works for functions of a single variable but
allows minimization over a specified interval.
The last algorithm actually finds the roots of a general function of possibly
many variables.
It is included in the optimization package because at the (non-boundary)
extreme points of a function, the gradient is equal to zero.
\layout Subsection
Nelder-Mead Simplex algorithm (optimize.fmin)
\layout Standard
The simplex algorithm is probably the simplest way to minimize a fairly
well-behaved function.
The simplex algorithm requires only function evaluations and is a good
choice for simple minimization problems.
However, because it does not use any gradient evaluations, it may take
longer to find the minimum.
To demonstrate the minimization function consider the problem of minimizing
the Rosenbrock function of
\begin_inset Formula $N$
\end_inset
variables:
\begin_inset Formula \[
f\left(\mathbf{x}\right)=\sum_{i=1}^{N-1}100\left(x_{i}-x_{i-1}^{2}\right)^{2}+\left(1-x_{i-1}\right)^{2}.\]
\end_inset
The minimum value of this function is 0 which is achieved when
\begin_inset Formula $x_{i}=1.$
\end_inset
This minimum can be found using the
\series bold
fmin
\series default
routine as shown in the example below:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.2}
preview false
\end_inset
\layout Standard
Another optimization algorithm that needs only function calls to find the
minimum is Powell's method available as
\series bold
optimize.fmin_powell
\series default
.
\layout Subsection
Broyden-Fletcher-Goldfarb-Shanno algorithm (optimize.fmin_bfgs)
\layout Standard
In order to converge more quickly to the solution, this routine uses the
gradient of the objective function.
If the gradient is not given by the user, then it is estimated using first-diff
erences.
The Broyden-Fletcher-Goldfarb-Shanno (BFGS) method typically requires fewer
function calls than the simplex algorithm even when the gradient must be
estimated.
\layout Standard
To demonstrate this algorithm, the Rosenbrock function is again used.
The gradient of the Rosenbrock function is the vector:
\begin_inset Formula \begin{eqnarray*}
\frac{\partial f}{\partial x_{j}} & = & \sum_{i=1}^{N}200\left(x_{i}-x_{i-1}^{2}\right)\left(\delta_{i,j}-2x_{i-1}\delta_{i-1,j}\right)-2\left(1-x_{i-1}\right)\delta_{i-1,j}.\\
& = & 200\left(x_{j}-x_{j-1}^{2}\right)-400x_{j}\left(x_{j+1}-x_{j}^{2}\right)-2\left(1-x_{j}\right).\end{eqnarray*}
\end_inset
This expression is valid for the interior derivatives.
Special cases are
\begin_inset Formula \begin{eqnarray*}
\frac{\partial f}{\partial x_{0}} & = & -400x_{0}\left(x_{1}-x_{0}^{2}\right)-2\left(1-x_{0}\right),\\
\frac{\partial f}{\partial x_{N-1}} & = & 200\left(x_{N-1}-x_{N-2}^{2}\right).\end{eqnarray*}
\end_inset
A Python function which computes this gradient is constructed by the code-segme
nt:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.3}
preview false
\end_inset
\layout Standard
The calling signature for the BFGS minimization algorithm is similar to
\series bold
fmin
\series default
with the addition of the
\emph on
fprime
\emph default
argument.
An example usage of
\series bold
fmin_bfgs
\series default
is shown in the following example which minimizes the Rosenbrock function.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.4}
preview false
\end_inset
\layout Subsection
Newton-Conjugate-Gradient (optimize.fmin_ncg)
\layout Standard
The method which requires the fewest function calls and is therefore often
the fastest method to minimize functions of many variables is
\series bold
fmin_ncg.
\series default
This method is a modified Newton's method and uses a conjugate gradient
algorithm to (approximately) invert the local Hessian.
Newton's method is based on fitting the function locally to a quadratic
form:
\begin_inset Formula \[
f\left(\mathbf{x}\right)\approx f\left(\mathbf{x}_{0}\right)+\nabla f\left(\mathbf{x}_{0}\right)\cdot\left(\mathbf{x}-\mathbf{x}_{0}\right)+\frac{1}{2}\left(\mathbf{x}-\mathbf{x}_{0}\right)^{T}\mathbf{H}\left(\mathbf{x}_{0}\right)\left(\mathbf{x}-\mathbf{x}_{0}\right).\]
\end_inset
where
\begin_inset Formula $\mathbf{H}\left(\mathbf{x}_{0}\right)$
\end_inset
is a matrix of second-derivatives (the Hessian).
If the Hessian is positive definite then the local minimum of this function
can be found by setting the gradient of the quadratic form to zero, resulting
in
\begin_inset Formula \[
\mathbf{x}_{\textrm{opt}}=\mathbf{x}_{0}-\mathbf{H}^{-1}\nabla f.\]
\end_inset
The inverse of the Hessian is evaluted using the conjugate-gradient method.
An example of employing this method to minimizing the Rosenbrock function
is given below.
To take full advantage of the NewtonCG method, a function which computes
the Hessian must be provided.
The Hessian matrix itself does not need to be constructed, only a vector
which is the product of the Hessian with an arbitrary vector needs to be
available to the minimization routine.
As a result, the user can provide either a function to compute the Hessian
matrix, or a function to compute the product of the Hessian with an arbitrary
vector.
\layout Subsubsection
Full Hessian example:
\layout Standard
The Hessian of the Rosenbrock function is
\begin_inset Formula \begin{eqnarray*}
H_{ij}=\frac{\partial^{2}f}{\partial x_{i}\partial x_{j}} & = & 200\left(\delta_{i,j}-2x_{i-1}\delta_{i-1,j}\right)-400x_{i}\left(\delta_{i+1,j}-2x_{i}\delta_{i,j}\right)-400\delta_{i,j}\left(x_{i+1}-x_{i}^{2}\right)+2\delta_{i,j},\\
& = & \left(202+1200x_{i}^{2}-400x_{i+1}\right)\delta_{i,j}-400x_{i}\delta_{i+1,j}-400x_{i-1}\delta_{i-1,j},\end{eqnarray*}
\end_inset
if
\begin_inset Formula $i,j\in\left[1,N-2\right]$
\end_inset
with
\begin_inset Formula $i,j\in\left[0,N-1\right]$
\end_inset
defining the
\begin_inset Formula $N\times N$
\end_inset
matrix.
Other non-zero entries of the matrix are
\begin_inset Formula \begin{eqnarray*}
\frac{\partial^{2}f}{\partial x_{0}^{2}} & = & 1200x_{0}^{2}-400x_{1}+2,\\
\frac{\partial^{2}f}{\partial x_{0}\partial x_{1}}=\frac{\partial^{2}f}{\partial x_{1}\partial x_{0}} & = & -400x_{0},\\
\frac{\partial^{2}f}{\partial x_{N-1}\partial x_{N-2}}=\frac{\partial^{2}f}{\partial x_{N-2}\partial x_{N-1}} & = & -400x_{N-2},\\
\frac{\partial^{2}f}{\partial x_{N-1}^{2}} & = & 200.\end{eqnarray*}
\end_inset
For example, the Hessian when
\begin_inset Formula $N=5$
\end_inset
is
\begin_inset Formula \[
\mathbf{H}=\left[\begin{array}{ccccc}
1200x_{0}^{2}-400x_{1}+2 & -400x_{0} & 0 & 0 & 0\\
-400x_{0} & 202+1200x_{1}^{2}-400x_{2} & -400x_{1} & 0 & 0\\
0 & -400x_{1} & 202+1200x_{2}^{2}-400x_{3} & -400x_{2} & 0\\
0 & & -400x_{2} & 202+1200x_{3}^{2}-400x_{4} & -400x_{3}\\
0 & 0 & 0 & -400x_{3} & 200\end{array}\right].\]
\end_inset
The code which computes this Hessian along with the code to minimize the
function using
\series bold
fmin_ncg
\series default
is shown in the following example:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.5}
preview false
\end_inset
\layout Subsubsection
Hessian product example:
\layout Standard
For larger minimization problems, storing the entire Hessian matrix can
consume considerable time and memory.
The Newton-CG algorithm only needs the product of the Hessian times an
arbitrary vector.
As a result, the user can supply code to compute this product rather than
the full Hessian by setting the
\emph on
fhess_p
\emph default
keyword to the desired function.
The fhess_p function should take
\series bold
\series default
the minimization vector as the first argument and the arbitrary vector as
the second argument.
Any extra arguments passed to the function to be minimized will also be
passed to this function.
If possible, using Newton-CG with the hessian product option is probably
the fastest way to minimize the function.
\layout Standard
In this case, the product of the Rosenbrock Hessian with an arbitrary vector
is not difficult to compute.
If
\begin_inset Formula $\mathbf{p}$
\end_inset
is the arbitrary vector, then
\begin_inset Formula $\mathbf{H}\left(\mathbf{x}\right)\mathbf{p}$
\end_inset
has elements:
\begin_inset Formula \[
\mathbf{H}\left(\mathbf{x}\right)\mathbf{p}=\left[\begin{array}{c}
\left(1200x_{0}^{2}-400x_{1}+2\right)p_{0}-400x_{0}p_{1}\\
\vdots\\
-400x_{i-1}p_{i-1}+\left(202+1200x_{i}^{2}-400x_{i+1}\right)p_{i}-400x_{i}p_{i+1}\\
\vdots\\
-400x_{N-2}p_{N-2}+200p_{N-1}\end{array}\right].\]
\end_inset
Code which makes use of the
\emph on
fhess_p
\emph default
keyword to minimize the Rosenbrock function using
\series bold
fmin_ncg
\series default
follows:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.6}
preview false
\end_inset
\layout Subsection
Least-square fitting (minimize.leastsq)
\layout Standard
All of the previously-explained minimization procedures can be used to solve
a least-squares problem provided the appropriate objective function is
constructed.
For example, suppose it is desired to fit a set of data
\begin_inset Formula $\left\{ \mathbf{x}_{i},\mathbf{y}_{i}\right\} $
\end_inset
to a known model,
\begin_inset Formula $\mathbf{y}=\mathbf{f}\left(\mathbf{x},\mathbf{p}\right)$
\end_inset
where
\begin_inset Formula $\mathbf{p}$
\end_inset
is a vector of parameters for the model that need to be found.
A common method for determining which parameter vector gives the best fit
to the data is to minimize the sum of squares of the residuals.
The residual is usually defined for each observed data-point as
\begin_inset Formula \[
e_{i}\left(\mathbf{p},\mathbf{y}_{i},\mathbf{x}_{i}\right)=\left\Vert \mathbf{y}_{i}-\mathbf{f}\left(\mathbf{x}_{i},\mathbf{p}\right)\right\Vert .\]
\end_inset
An objective function to pass to any of the previous minization algorithms
to obtain a least-squares fit is.
\begin_inset Formula \[
J\left(\mathbf{p}\right)=\sum_{i=0}^{N-1}e_{i}^{2}\left(\mathbf{p}\right).\]
\end_inset
\layout Standard
The
\series bold
leastsq
\series default
algorithm performs this squaring and summing of the residuals automatically.
It takes as an input argument the vector function
\begin_inset Formula $\mathbf{e}\left(\mathbf{p}\right)$
\end_inset
and returns the value of
\begin_inset Formula $\mathbf{p}$
\end_inset
which minimizes
\begin_inset Formula $J\left(\mathbf{p}\right)=\mathbf{e}^{T}\mathbf{e}$
\end_inset
directly.
The user is also encouraged to provide the Jacobian matrix of the function
(with derivatives down the columns or across the rows).
If the Jacobian is not provided, it is estimated.
\layout Standard
An example should clarify the usage.
Suppose it is believed some measured data follow a sinusoidal pattern
\begin_inset Formula \[
y_{i}=A\sin\left(2\pi kx_{i}+\theta\right)\]
\end_inset
where the parameters
\begin_inset Formula $A,$
\end_inset
\begin_inset Formula $k$
\end_inset
, and
\begin_inset Formula $\theta$
\end_inset
are unknown.
The residual vector is
\begin_inset Formula \[
e_{i}=\left|y_{i}-A\sin\left(2\pi kx_{i}+\theta\right)\right|.\]
\end_inset
By defining a function to compute the residuals and (selecting an appropriate
starting position), the least-squares fit routine can be used to find the
best-fit parameters
\begin_inset Formula $\hat{A},\,\hat{k},\,\hat{\theta}$
\end_inset
.
This is shown in the following example and a plot of the results is shown
in Figure
\begin_inset LatexCommand \ref{fig:least_squares_fit}
\end_inset
.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.7}
preview false
\end_inset
\layout Standard
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/leastsqfit.pdf
display color
width 50page%
rotateOrigin center
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:least_squares_fit}
\end_inset
Least-square fitting to noisy data using
\series bold
scipy.optimize.leastsq
\end_inset
\layout Subsection
Scalar function minimizers
\layout Standard
Often only the minimum of a scalar function is needed (a scalar function
is one that takes a scalar as input and returns a scalar output).
In these circumstances, other optimization techniques have been developed
that can work faster.
\layout Subsubsection
Unconstrained minimization (optimize.brent)
\layout Standard
There are actually two methods that can be used to minimize a scalar function
(
\series bold
brent
\series default
and
\series bold
golden)
\series default
, but
\series bold
golden
\series default
is included only for academic purposes and should rarely be used.
The brent method uses Brent's algorithm for locating a minimum.
Optimally a bracket should be given which contains the minimum desired.
A bracket is a triple
\begin_inset Formula $\left(a,b,c\right)$
\end_inset
such that
\begin_inset Formula $f\left(a\right)>f\left(b\right)<f\left(c\right)$
\end_inset
and
\begin_inset Formula $a<b<c$
\end_inset
.
If this is not given, then alternatively two starting points can be chosen
and a bracket will be found from these points using a simple marching algorithm.
If these two starting points are not provided 0 and 1 will be used (this
may not be the right choice for your function and result in an unexpected
minimum being returned).
\layout Subsubsection
Bounded minimization (optimize.fminbound)
\layout Standard
Thus far all of the minimization routines described have been unconstrained
minimization routines.
Very often, however, there are constraints that can be placed on the solution
space before minimization occurs.
The
\series bold
fminbound
\series default
function is an example of a constrained minimization procedure that provides
a rudimentary interval constraint for scalar functions.
The interval constraint allows the minimization to occur only between two
fixed endpoints.
\layout Standard
For example, to find the minimum of
\begin_inset Formula $J_{1}\left(x\right)$
\end_inset
near
\begin_inset Formula $x=5$
\end_inset
,
\series bold
fminbound
\series default
can be called using the interval
\begin_inset Formula $\left[4,7\right]$
\end_inset
as a constraint.
The result is
\begin_inset Formula $x_{\textrm{min}}=5.3314$
\end_inset
:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.8}
preview false
\end_inset
\layout Subsection
Root finding
\layout Subsubsection
Sets of equations
\layout Standard
To find the roots of a polynomial, the command
\series bold
roots
\series default
is useful.
To find a root of a set of non-linear equations, the command
\series bold
optimize.fsolve
\series default
is needed.
For example, the following example finds the roots of the single-variable
transcendental equation
\begin_inset Formula \[
x+2\cos\left(x\right)=0,\]
\end_inset
and the set of non-linear equations
\begin_inset Formula \begin{eqnarray*}
x_{0}\cos\left(x_{1}\right) & = & 4,\\
x_{0}x_{1}-x_{1} & = & 5.\end{eqnarray*}
\end_inset
The results are
\begin_inset Formula $x=-1.0299$
\end_inset
and
\begin_inset Formula $x_{0}=6.5041,\, x_{1}=0.9084$
\end_inset
.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example5.9}
preview false
\end_inset
\layout Subsubsection
Scalar function root finding
\layout Standard
If one has a single-variable equation, there are four different root finder
algorithms that can be tried.
Each of these root finding algorithms requires the endpoints of an interval
where a root is suspected (because the function changes signs).
In general
\series bold
brentq
\series default
is the best choice, but the other methods may be useful in certain circumstance
s or for academic purposes.
\layout Subsubsection
Fixed-point solving
\layout Standard
A problem closely related to finding the zeros of a function is the problem
of finding a fixed-point of a function.
A fixed point of a function is the point at which evaluation of the function
returns the point:
\begin_inset Formula $g\left(x\right)=x.$
\end_inset
Clearly the fixed point of
\begin_inset Formula $g$
\end_inset
is the root of
\begin_inset Formula $f\left(x\right)=g\left(x\right)-x.$
\end_inset
Equivalently, the root of
\begin_inset Formula $f$
\end_inset
is the fixed_point of
\begin_inset Formula $g\left(x\right)=f\left(x\right)+x.$
\end_inset
The routine
\series bold
fixed_point
\series default
provides a simple iterative method using Aitkens sequence acceleration
to estimate the fixed point of
\begin_inset Formula $g$
\end_inset
given a starting point.
\layout Section
Interpolation (interpolate)
\begin_inset OptArg
collapsed true
\layout Standard
Interpolation
\end_inset
\layout Standard
There are two general interpolation facilities available in SciPy.
The first facility is an interpolation class which performs linear 1-dimensiona
l interpolation.
The second facility is based on the FORTRAN library FITPACK and provides
functions for 1- and 2-dimensional (smoothed) cubic-spline interpolation.
\layout Subsection
Linear 1-d interpolation (interpolate.linear_1d)
\layout Standard
The linear_1d class in scipy.interpolate is a convenient method to create
a function based on fixed data points which can be evaluated anywhere within
the domain defined by the given data using linear interpolation.
An instance of this class is created by passing the 1-d vectors comprising
the data.
The instance of this class defines a
\emph on
__call__
\emph default
method and can therefore by treated like a function which interpolates
between known data values to obtain unknown values (it even has a docstring
for help).
Behavior at the boundary can be specified at instantiation time.
The following example demonstrates it's use.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example6.1}
preview false
\end_inset
Figure shows the result:
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/inter_1d.pdf
display color
width 50page%
rotateOrigin center
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:inter_1d}
\end_inset
One-dimensional interpolation using the class
\series bold
interpolate.linear_1d.
\end_inset
\layout Subsection
Spline interpolation in 1-d (interpolate.splXXX)
\layout Standard
Spline interpolation requires two essential steps: (1) a spline representation
of the curve is computed, and (2) the spline is evaluated at the desired
points.
In order to find the spline representation, there are two different was
to represent a curve and obtain (smoothing) spline coefficients: directly
and parametrically.
The direct method finds the spline representation of a curve in a two-dimension
al plane using the function
\series bold
interpolate.splrep.
\series default
The first two arguments are the only ones required, and these provide the
\begin_inset Formula $x$
\end_inset
and
\begin_inset Formula $y$
\end_inset
components of the curve.
The normal output is a 3-tuple,
\begin_inset Formula $\left(t,c,k\right)$
\end_inset
, containing the knot-points,
\begin_inset Formula $t$
\end_inset
, the coefficients
\begin_inset Formula $c$
\end_inset
and the order
\begin_inset Formula $k$
\end_inset
of the spline.
The default spline order is cubic, but this can be changed with the input
keyword,
\emph on
k.
\layout Standard
For curves in
\begin_inset Formula $N$
\end_inset
-dimensional space the function
\series bold
interpolate.splprep
\series default
allows defining the curve parametrically.
For this function only 1 input argument is required.
This input is a list of
\begin_inset Formula $N$
\end_inset
-arrays representing the curve in
\begin_inset Formula $N$
\end_inset
-dimensional space.
The length of each array is the number of curve points, and each array
provides one component of the
\begin_inset Formula $N$
\end_inset
-dimensional data point.
The parameter variable is given with the keword argument,
\emph on
u,
\emph default
which defaults to an equally-spaced monotonic sequence between
\begin_inset Formula $0$
\end_inset
and
\begin_inset Formula $1$
\end_inset
.
The default output consists of two objects: a 3-tuple,
\begin_inset Formula $\left(t,c,k\right)$
\end_inset
, containing the spline representation and the parameter variable
\begin_inset Formula $u.$
\end_inset
\layout Standard
The keyword argument,
\emph on
s
\emph default
, is used to specify the amount of smoothing to perform during the spline
fit.
The default value of
\begin_inset Formula $s$
\end_inset
is
\begin_inset Formula $s=m-\sqrt{2m}$
\end_inset
where
\begin_inset Formula $m$
\end_inset
is the number of data-points being fit.
Therefore,
\series bold
if no smoothing is desired a value of
\begin_inset Formula $\mathbf{s}=0$
\end_inset
should be passed to the routines.
\layout Standard
Once the spline representation of the data has been determined, functions
are available for evaluating the spline (
\series bold
interpolate.splev)
\series default
and its derivatives (
\series bold
interpolate.splev, interpolate.splade
\series default
) at any point and the integral of the spline between any two points (
\series bold
interpolate.splint)
\series default
.
In addition, for cubic splines (
\begin_inset Formula $k=3$
\end_inset
) with 8 or more knots, the roots of the spline can be estimated (
\series bold
interpolate.sproot)
\series default
.
These functions are demonstrated in the example that follows (see also
Figure
\begin_inset LatexCommand \ref{fig:spline-1d}
\end_inset
).
\layout Standard
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/interp_cubic.pdf
display color
width 45line%
rotateOrigin center
subcaption
subcaptionText "Cubic-spline (splrep)"
\end_inset
\SpecialChar ~
\begin_inset Graphics
filename fig/scipy/interp_cubic_der.pdf
display color
width 45line%
rotateOrigin center
subcaption
subcaptionText "Derivative of spline (splev)"
\end_inset
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/interp_cubic_int.pdf
display color
width 45line%
rotateOrigin center
subcaption
subcaptionText "Integral of spline (splint)"
\end_inset
\SpecialChar ~
\begin_inset Graphics
filename fig/scipy/interp_cubic_param.pdf
display color
width 45line%
rotateOrigin center
subcaption
subcaptionText "Spline of parametric curve (splprep)"
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:spline-1d}
\end_inset
Examples of using cubic-spline interpolation.
\end_inset
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example6.2}
preview false
\end_inset
\layout Subsection
Two-dimensionsal spline representation (interpolate.bisplrep)
\layout Standard
For (smooth) spline-fitting to a two dimensional surface, the function
\series bold
interpolate.bisplrep
\series default
is available.
This function takes as required inputs the
\series bold
1-D
\series default
arrays
\emph on
x, y,
\emph default
and
\emph on
z
\emph default
which represent points on the surface
\begin_inset Formula $z=f\left(x,y\right).$
\end_inset
The default output is a list
\begin_inset Formula $\left[tx,ty,c,kx,ky\right]$
\end_inset
whose entries represent respectively, the components of the knot positions,
the coefficients of the spline, and the order of the spline in each coordinate.
It is convenient to hold this list in a single object,
\emph on
tck,
\emph default
so that it can be passed easily to the function
\series bold
interpolate.bisplev.
\series default
The keyword,
\emph on
s
\emph default
,
\emph on
\emph default
can be used to change the amount of smoothing performed on the data while
determining the appropriate spline.
The default value is
\begin_inset Formula $s=m-\sqrt{2m}$
\end_inset
where
\begin_inset Formula $m$
\end_inset
is the number of data points in the
\emph on
x, y,
\emph default
and
\emph on
z
\emph default
vectors.
As a result, if no smoothing is desired, then
\begin_inset Formula $s=0$
\end_inset
should be passed to
\series bold
interpolate.bisplrep.
\layout Standard
To evaluate the two-dimensional spline and it's partial derivatives (up
to the order of the spline), the function
\series bold
interpolate.bisplev
\series default
is required.
This function takes as the first two arguments
\series bold
two 1-D arrays
\series default
whose cross-product specifies the domain over which to evaluate the spline.
The third argument is the
\emph on
tck
\emph default
list returned from
\series bold
interpolate.bisplrep.
\series default
If desired, the fourth and fifth arguments provide the orders of the partial
derivative in the
\begin_inset Formula $x$
\end_inset
and
\begin_inset Formula $y$
\end_inset
direction respectively.
\layout Standard
It is important to note that two dimensional interpolation should not be
used to find the spline representation of images.
The algorithm used is not amenable to large numbers of input points.
The signal processing toolbox contains more appropriate algorithms for
finding the spline representation of an image.
The two dimensional interpolation commands are intended for use when interpolat
ing a two dimensional function as shown in the example that follows (See
also Figure
\begin_inset LatexCommand \ref{fig:2d_interp}
\end_inset
).
This example uses the
\series bold
mgrid
\series default
command in SciPy which is useful for defining a
\begin_inset Quotes eld
\end_inset
mesh-grid
\begin_inset Quotes erd
\end_inset
in many dimensions.
(See also the
\series bold
ogrid
\series default
command if the full-mesh is not needed).
The number of output arguments and the number of dimensions of each argument
is determined by the number of indexing objects passed in
\series bold
mgrid[].
\series default
\begin_inset Include \verbatiminput{examples/scipy/example6.3}
preview false
\end_inset
\layout Standard
\added_space_top 0.3cm \added_space_bottom 0.3cm \align center
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/2d_func.pdf
display color
width 45line%
rotateOrigin center
\end_inset
\SpecialChar ~
\SpecialChar ~
\begin_inset Graphics
filename fig/scipy/2d_interp.pdf
display color
width 45line%
rotateOrigin center
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:2d_interp}
\end_inset
Example of two-dimensional spline interpolation.
\end_inset
\layout Section
Signal Processing (signal)
\begin_inset OptArg
collapsed true
\layout Standard
Signal Processing
\end_inset
\layout Standard
The signal processing toolbox currently contains some filtering functions,
a limited set of filter design tools, and a few B-spline interpolation
algorithms for one- and two-dimensional data.
While the B-spline algorithms could technically be placed under the interpolati
on category, they are included here because they only work with equally-spaced
data and make heavy use of filter-theory and transfer-function formalism
to provide a fast B-spline transform.
To understand this section you will need to understand that a signal in
SciPy is an array of real or complex numbers.
\layout Subsection
B-splines
\layout Standard
A B-spline is an approximation of a continuous function over a finite-domain
in terms of B-spline coefficients and knot points.
If the knot-points are equally spaced with spacing
\begin_inset Formula $\Delta x$
\end_inset
, then the B-spline approximation to a 1-dimensional function is the finite-basi
s expansion.
\begin_inset Formula \[
y\left(x\right)\approx\sum_{j}c_{j}\beta^{o}\left(\frac{x}{\Delta x}-j\right).\]
\end_inset
In two dimensions with knot-spacing
\begin_inset Formula $\Delta x$
\end_inset
and
\begin_inset Formula $\Delta y$
\end_inset
, the function representation is
\begin_inset Formula \[
z\left(x,y\right)\approx\sum_{j}\sum_{k}c_{jk}\beta^{o}\left(\frac{x}{\Delta x}-j\right)\beta^{o}\left(\frac{y}{\Delta y}-k\right).\]
\end_inset
In these expressions,
\begin_inset Formula $\beta^{o}\left(\cdot\right)$
\end_inset
is the space-limited B-spline basis function of order,
\begin_inset Formula $o$
\end_inset
.
The requirement of equally-spaced knot-points and equally-spaced data points,
allows the development of fast (inverse-filtering) algorithms for determining
the coefficients,
\begin_inset Formula $c_{j}$
\end_inset
, from sample-values,
\begin_inset Formula $y_{n}$
\end_inset
.
Unlike the general spline interpolation algorithms, these algorithms can
quickly find the spline coefficients for large images.
\layout Standard
The advantage of representing a set of samples via B-spline basis functions
is that continuous-domain operators (derivatives, re-sampling, integral,
etc.) which assume that the data samples are drawn from an underlying continuous
function can be computed with relative ease from the spline coefficients.
For example, the second-derivative of a spline is
\begin_inset Formula \[
y{}^{\prime\prime}\left(x\right)=\frac{1}{\Delta x^{2}}\sum_{j}c_{j}\beta^{o\prime\prime}\left(\frac{x}{\Delta x}-j\right).\]
\end_inset
Using the property of B-splines that
\begin_inset Formula \[
\frac{d^{2}\beta^{o}\left(w\right)}{dw^{2}}=\beta^{o-2}\left(w+1\right)-2\beta^{o-2}\left(w\right)+\beta^{o-2}\left(w-1\right)\]
\end_inset
it can be seen that
\begin_inset Formula \[
y^{\prime\prime}\left(x\right)=\frac{1}{\Delta x^{2}}\sum_{j}c_{j}\left[\beta^{o-2}\left(\frac{x}{\Delta x}-j+1\right)-2\beta^{o-2}\left(\frac{x}{\Delta x}-j\right)+\beta^{o-2}\left(\frac{x}{\Delta x}-j-1\right)\right].\]
\end_inset
If
\begin_inset Formula $o=3$
\end_inset
, then at the sample points,
\begin_inset Formula \begin{eqnarray*}
\Delta x^{2}\left.y^{\prime}\left(x\right)\right|_{x=n\Delta x} & = & \sum_{j}c_{j}\delta_{n-j+1}-2c_{j}\delta_{n-j}+c_{j}\delta_{n-j-1},\\
& = & c_{n+1}-2c_{n}+c_{n-1}.\end{eqnarray*}
\end_inset
Thus, the second-derivative signal can be easily calculated from the spline
fit.
if desired, smoothing splines can be found to make the second-derivative
less sensitive to random-errors.
\layout Standard
The savvy reader will have already noticed that the data samples are related
to the knot coefficients via a convolution operator, so that simple convolution
with the sampled B-spline function recovers the original data from the
spline coefficients.
The output of convolutions can change depending on how boundaries are handled
(this becomes increasingly more important as the number of dimensions in
the data-set increases).
The algorithms relating to B-splines in the signal-processing sub package
assume mirror-symmetric boundary conditions.
Thus, spline coefficients are computed based on that assumption, and data-sampl
es can be recovered exactly from the spline coefficients by assuming them
to be mirror-symmetric also.
\layout Standard
Currently the package provides functions for determining second- and third-order
cubic spline coefficients from equally spaced samples in one- and two-dimension
s (
\series bold
signal.qspline1d, signal.qspline2d, signal.cspline1d, signal.cspline2d
\series default
).
The package also supplies a function (
\series bold
signal.bspline
\series default
) for evaluating the bspline basis function,
\begin_inset Formula $\beta^{o}\left(x\right)$
\end_inset
for arbitrary order and
\begin_inset Formula $x.$
\end_inset
For large
\begin_inset Formula $o$
\end_inset
, the B-spline basis function can be approximated well by a zero-mean Gaussian
function with standard-deviation equal to
\begin_inset Formula $\sigma_{o}=\left(o+1\right)/12$
\end_inset
:
\begin_inset Formula \[
\beta^{o}\left(x\right)\approx\frac{1}{\sqrt{2\pi\sigma_{o}^{2}}}\exp\left(-\frac{x^{2}}{2\sigma_{o}}\right).\]
\end_inset
A function to compute this Gaussian for arbitrary
\begin_inset Formula $x$
\end_inset
and
\begin_inset Formula $o$
\end_inset
is also available (
\series bold
signal.gauss_spline
\series default
).
The following code and Figure uses spline-filtering to compute an edge-image
(the second-derivative of a smoothed spline) of Lena's face which is an
array returned by the command
\series bold
lena().
\series default
The command
\series bold
signal.sepfir2d
\series default
was used to apply a separable two-dimensional FIR filter with mirror-symmetric
boundary conditions to the spline coefficients.
This function is ideally suited for reconstructing samples from spline
coefficients and is faster than
\series bold
signal.convolve2d
\series default
which convolves arbitrary two-dimensional filters and allows for choosing
mirror-symmetric boundary conditions.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example6.4}
preview false
\end_inset
\layout Standard
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/lena_image.pdf
display color
width 45line%
BoundingBox 3.5cm 10cm 18cm 25cm
clip
rotateOrigin center
\end_inset
\begin_inset Graphics
filename fig/scipy/lena_edge.pdf
display color
width 45line%
BoundingBox 3.5cm 10cm 18cm 25cm
clip
rotateOrigin center
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:lena_edge_spline}
\end_inset
Example of using smoothing splines to filter images.
\end_inset
\layout Subsection
Filtering
\layout Standard
Filtering is a generic name for any system that modifies an input signal
in some way.
In SciPy a signal can be thought of as a Numeric array.
There are different kinds of filters for different kinds of operations.
There are two broad kinds of filtering operations: linear and non-linear.
Linear filters can always be reduced to multiplication of the flattened
Numeric array by an appropriate matrix resulting in another flattened Numeric
array.
Of course, this is not usually the best way to compute the filter as the
matrices and vectors involved may be huge.
For example filtering a
\begin_inset Formula $512\times512$
\end_inset
image with this method would require multiplication of a
\begin_inset Formula $512^{2}x512^{2}$
\end_inset
matrix with a
\begin_inset Formula $512^{2}$
\end_inset
vector.
Just trying to store the
\begin_inset Formula $512^{2}\times512^{2}$
\end_inset
matrix using a standard Numeric array would require
\begin_inset Formula $68,719,476,736$
\end_inset
elements.
At 4 bytes per element this would require
\begin_inset Formula $256\textrm{GB}$
\end_inset
of memory.
In most applications most of the elements of this matrix are zero and a
different method for computing the output of the filter is employed.
\layout Subsubsection
Convolution/Correlation
\layout Standard
Many linear filters also have the property of shift-invariance.
This means that the filtering operation is the same at different locations
in the signal and it implies that the filtering matrix can be constructed
from knowledge of one row (or column) of the matrix alone.
In this case, the matrix multiplication can be accomplished using Fourier
transforms.
\layout Standard
Let
\begin_inset Formula $x\left[n\right]$
\end_inset
define a one-dimensional signal indexed by the integer
\begin_inset Formula $n.$
\end_inset
Full convolution of two one-dimensional signals can be expressed as
\begin_inset Formula \[
y\left[n\right]=\sum_{k=-\infty}^{\infty}x\left[k\right]h\left[n-k\right].\]
\end_inset
This equation can only be implemented directly if we limit the sequences
to finite support sequences that can be stored in a computer, choose
\begin_inset Formula $n=0$
\end_inset
to be the starting point of both sequences, let
\begin_inset Formula $K+1$
\end_inset
be that value for which
\begin_inset Formula $y\left[n\right]=0$
\end_inset
for all
\begin_inset Formula $n>K+1$
\end_inset
and
\begin_inset Formula $M+1$
\end_inset
be that value for which
\begin_inset Formula $x\left[n\right]=0$
\end_inset
for all
\begin_inset Formula $n>M+1$
\end_inset
, then the discrete convolution expression is
\begin_inset Formula \[
y\left[n\right]=\sum_{k=\max\left(n-M,0\right)}^{\min\left(n,K\right)}x\left[k\right]h\left[n-k\right].\]
\end_inset
For convenience assume
\begin_inset Formula $K\geq M.$
\end_inset
Then, more explicitly the output of this operation is
\begin_inset Formula \begin{eqnarray*}
y\left[0\right] & = & x\left[0\right]h\left[0\right]\\
y\left[1\right] & = & x\left[0\right]h\left[1\right]+x\left[1\right]h\left[0\right]\\
y\left[2\right] & = & x\left[0\right]h\left[2\right]+x\left[1\right]h\left[1\right]+x\left[2\right]h\left[0\right]\\
\vdots & \vdots & \vdots\\
y\left[M\right] & = & x\left[0\right]h\left[M\right]+x\left[1\right]h\left[M-1\right]+\cdots+x\left[M\right]h\left[0\right]\\
y\left[M+1\right] & = & x\left[1\right]h\left[M\right]+x\left[2\right]h\left[M-1\right]+\cdots+x\left[M+1\right]h\left[0\right]\\
\vdots & \vdots & \vdots\\
y\left[K\right] & = & x\left[K-M\right]h\left[M\right]+\cdots+x\left[K\right]h\left[0\right]\\
y\left[K+1\right] & = & x\left[K+1-M\right]h\left[M\right]+\cdots+x\left[K\right]h\left[1\right]\\
\vdots & \vdots & \vdots\\
y\left[K+M-1\right] & = & x\left[K-1\right]h\left[M\right]+x\left[K\right]h\left[M-1\right]\\
y\left[K+M\right] & = & x\left[K\right]h\left[M\right].\end{eqnarray*}
\end_inset
Thus, the full discrete convolution of two finite sequences of lengths
\begin_inset Formula $K+1$
\end_inset
and
\begin_inset Formula $M+1$
\end_inset
respectively results in a finite sequence of length
\begin_inset Formula $K+M+1=\left(K+1\right)+\left(M+1\right)-1.$
\end_inset
\layout Standard
One dimensional convolution is implemented in SciPy with the function
\family typewriter
signal.convolve
\family default
.
This function takes as inputs the signals
\begin_inset Formula $x,$
\end_inset
\begin_inset Formula $h$
\end_inset
, and an optional flag and returns the signal
\begin_inset Formula $y.$
\end_inset
The optional flag allows for specification of which part of the output
signal to return.
The default value of 'full' returns the entire signal.
If the flag has a value of 'same' then only the middle
\begin_inset Formula $K$
\end_inset
values are returned starting at
\begin_inset Formula $y\left[\left\lfloor \frac{M-1}{2}\right\rfloor \right]$
\end_inset
so that the output has the same length as the largest input.
If the flag has a value of 'valid' then only the middle
\begin_inset Formula $K-M+1=\left(K+1\right)-\left(M+1\right)+1$
\end_inset
output values are returned where
\begin_inset Formula $z$
\end_inset
depends on all of the values of the smallest input from
\begin_inset Formula $h\left[0\right]$
\end_inset
to
\begin_inset Formula $h\left[M\right].$
\end_inset
In other words only the values
\begin_inset Formula $y\left[M\right]$
\end_inset
to
\begin_inset Formula $y\left[K\right]$
\end_inset
inclusive are returned.
\layout Standard
This same function
\family typewriter
signal.convolve
\family default
can actually take
\begin_inset Formula $N$
\end_inset
-dimensional arrays as inputs and will return the
\begin_inset Formula $N$
\end_inset
-dimensional convolution of the two arrays.
The same input flags are available for that case as well.
\layout Standard
Correlation is very similar to convolution except for the minus sign becomes
a plus sign.
Thus
\begin_inset Formula \[
w\left[n\right]=\sum_{k=-\infty}^{\infty}y\left[k\right]x\left[n+k\right]\]
\end_inset
is the (cross) correlation of the signals
\begin_inset Formula $y$
\end_inset
and
\begin_inset Formula $x.$
\end_inset
For finite-length signals with
\begin_inset Formula $y\left[n\right]=0$
\end_inset
outside of the range
\begin_inset Formula $\left[0,K\right]$
\end_inset
and
\begin_inset Formula $x\left[n\right]=0$
\end_inset
outside of the range
\begin_inset Formula $\left[0,M\right],$
\end_inset
the summation can simplify to
\begin_inset Formula \[
w\left[n\right]=\sum_{k=\max\left(0,-n\right)}^{\min\left(K,M-n\right)}y\left[k\right]x\left[n+k\right].\]
\end_inset
Assuming again that
\begin_inset Formula $K\geq M$
\end_inset
this is
\begin_inset Formula \begin{eqnarray*}
w\left[-K\right] & = & y\left[K\right]x\left[0\right]\\
w\left[-K+1\right] & = & y\left[K-1\right]x\left[0\right]+y\left[K\right]x\left[1\right]\\
\vdots & \vdots & \vdots\\
w\left[M-K\right] & = & y\left[K-M\right]x\left[0\right]+y\left[K-M+1\right]x\left[1\right]+\cdots+y\left[K\right]x\left[M\right]\\
w\left[M-K+1\right] & = & y\left[K-M-1\right]x\left[0\right]+\cdots+y\left[K-1\right]x\left[M\right]\\
\vdots & \vdots & \vdots\\
w\left[-1\right] & = & y\left[1\right]x\left[0\right]+y\left[2\right]x\left[1\right]+\cdots+y\left[M+1\right]x\left[M\right]\\
w\left[0\right] & = & y\left[0\right]x\left[0\right]+y\left[1\right]x\left[1\right]+\cdots+y\left[M\right]x\left[M\right]\\
w\left[1\right] & = & y\left[0\right]x\left[1\right]+y\left[1\right]x\left[2\right]+\cdots+y\left[M-1\right]x\left[M\right]\\
w\left[2\right] & = & y\left[0\right]x\left[2\right]+y\left[1\right]x\left[3\right]+\cdots+y\left[M-2\right]x\left[M\right]\\
\vdots & \vdots & \vdots\\
w\left[M-1\right] & = & y\left[0\right]x\left[M-1\right]+y\left[1\right]x\left[M\right]\\
w\left[M\right] & = & y\left[0\right]x\left[M\right].\end{eqnarray*}
\end_inset
\layout Standard
The SciPy function
\family typewriter
signal.correlate
\family default
implements this operation.
Equivalent flags are available for this operation to return the full
\begin_inset Formula $K+M+1$
\end_inset
length sequence ('full') or a sequence with the same size as the largest
sequence starting at
\begin_inset Formula $w\left[-K+\left\lfloor \frac{M-1}{2}\right\rfloor \right]$
\end_inset
('same') or a sequence where the values depend on all the values of the
smallest sequence ('valid').
This final option returns the
\begin_inset Formula $K-M+1$
\end_inset
values
\begin_inset Formula $w\left[M-K\right]$
\end_inset
to
\begin_inset Formula $w\left[0\right]$
\end_inset
inclusive.
\layout Standard
The function
\series bold
signal.correlate
\series default
can also take arbitrary
\begin_inset Formula $N$
\end_inset
-dimensional arrays as input and return the
\begin_inset Formula $N$
\end_inset
-dimensional convolution of the two arrays on output.
\layout Standard
When
\begin_inset Formula $N=2,$
\end_inset
\series bold
signal.correlate
\series default
and/or
\series bold
signal.convolve
\series default
can be used to construct arbitrary image filters to perform actions such
as blurring, enhancing, and edge-detection for an image.
\layout Standard
Convolution is mainly used for filtering when one of the signals is much
smaller than the other (
\begin_inset Formula $K\gg M$
\end_inset
), otherwise linear filtering is more easily accomplished in the frequency
domain (see Fourier Transforms).
\layout Subsubsection
Difference-equation filtering
\layout Standard
A general class of linear one-dimensional filters (that includes convolution
filters) are filters described by the difference equation
\begin_inset Formula \[
\sum_{k=0}^{N}a_{k}y\left[n-k\right]=\sum_{k=0}^{M}b_{k}x\left[n-k\right]\]
\end_inset
where
\begin_inset Formula $x\left[n\right]$
\end_inset
is the input sequence and
\begin_inset Formula $y\left[n\right]$
\end_inset
is the output sequence.
If we assume initial rest so that
\begin_inset Formula $y\left[n\right]=0$
\end_inset
for
\begin_inset Formula $n<0$
\end_inset
, then this kind of filter can be implemented using convolution.
However, the convolution filter sequence
\begin_inset Formula $h\left[n\right]$
\end_inset
could be infinite if
\begin_inset Formula $a_{k}\neq0$
\end_inset
for
\begin_inset Formula $k\geq1.$
\end_inset
In addition, this general class of linear filter allows initial conditions
to be placed on
\begin_inset Formula $y\left[n\right]$
\end_inset
for
\begin_inset Formula $n<0$
\end_inset
resulting in a filter that cannot be expressed using convolution.
\layout Standard
The difference equation filter can be thought of as finding
\begin_inset Formula $y\left[n\right]$
\end_inset
recursively in terms of it's previous values
\begin_inset Formula \[
a_{0}y\left[n\right]=-a_{1}y\left[n-1\right]-\cdots-a_{N}y\left[n-N\right]+\cdots+b_{0}x\left[n\right]+\cdots+b_{M}x\left[n-M\right].\]
\end_inset
Often
\begin_inset Formula $a_{0}=1$
\end_inset
is chosen for normalization.
The implementation in SciPy of this general difference equation filter
is a little more complicated then would be implied by the previous equation.
It is implemented so that only one signal needs to be delayed.
The actual implementation equations are (assuming
\begin_inset Formula $a_{0}=1$
\end_inset
).
\begin_inset Formula \begin{eqnarray*}
y\left[n\right] & = & b_{0}x\left[n\right]+z_{0}\left[n-1\right]\\
z_{0}\left[n\right] & = & b_{1}x\left[n\right]+z_{1}\left[n-1\right]-a_{1}y\left[n\right]\\
z_{1}\left[n\right] & = & b_{2}x\left[n\right]+z_{2}\left[n-1\right]-a_{2}y\left[n\right]\\
\vdots & \vdots & \vdots\\
z_{K-2}\left[n\right] & = & b_{K-1}x\left[n\right]+z_{K-1}\left[n-1\right]-a_{K-1}y\left[n\right]\\
z_{K-1}\left[n\right] & = & b_{K}x\left[n\right]-a_{K}y\left[n\right],\end{eqnarray*}
\end_inset
where
\begin_inset Formula $K=\max\left(N,M\right).$
\end_inset
Note that
\begin_inset Formula $b_{K}=0$
\end_inset
if
\begin_inset Formula $K>M$
\end_inset
and
\begin_inset Formula $a_{K}=0$
\end_inset
if
\begin_inset Formula $K>N.$
\end_inset
In this way, the output at time
\begin_inset Formula $n$
\end_inset
depends only on the input at time
\begin_inset Formula $n$
\end_inset
and the value of
\begin_inset Formula $z_{0}$
\end_inset
at the previous time.
This can always be calculated as long as the
\begin_inset Formula $K$
\end_inset
values
\begin_inset Formula $z_{0}\left[n-1\right]\ldots z_{K-1}\left[n-1\right]$
\end_inset
are computed and stored at each time step.
\layout Standard
The difference-equation filter is called using the command
\series bold
signal.lfilter
\series default
in SciPy.
This command takes as inputs the vector
\begin_inset Formula $b,$
\end_inset
the vector,
\begin_inset Formula $a,$
\end_inset
a signal
\begin_inset Formula $x$
\end_inset
and returns the vector
\begin_inset Formula $y$
\end_inset
(the same length as
\begin_inset Formula $x$
\end_inset
) computed using the equation given above.
If
\begin_inset Formula $x$
\end_inset
is
\begin_inset Formula $N$
\end_inset
-dimensional, then the filter is computed along the axis provided.
If, desired, initial conditions providing the values of
\begin_inset Formula $z_{0}\left[-1\right]$
\end_inset
to
\begin_inset Formula $z_{K-1}\left[-1\right]$
\end_inset
can be provided or else it will be assumed that they are all zero.
If initial conditions are provided, then the final conditions on the intermedia
te variables are also returned.
These could be used, for example, to restart the calculation in the same
state.
\layout Standard
Sometimes it is more convenient to express the initial conditions in terms
of the signals
\begin_inset Formula $x\left[n\right]$
\end_inset
and
\begin_inset Formula $y\left[n\right].$
\end_inset
In other words, perhaps you have the values of
\begin_inset Formula $x\left[-M\right]$
\end_inset
to
\begin_inset Formula $x\left[-1\right]$
\end_inset
and the values of
\begin_inset Formula $y\left[-N\right]$
\end_inset
to
\begin_inset Formula $y\left[-1\right]$
\end_inset
and would like to determine what values of
\begin_inset Formula $z_{m}\left[-1\right]$
\end_inset
should be delivered as initial conditions to the difference-equation filter.
It is not difficult to show that for
\begin_inset Formula $0\leq m<K,$
\end_inset
\begin_inset Formula \[
z_{m}\left[n\right]=\sum_{p=0}^{K-m-1}\left(b_{m+p+1}x\left[n-p\right]-a_{m+p+1}y\left[n-p\right]\right).\]
\end_inset
Using this formula we can find the intial condition vector
\begin_inset Formula $z_{0}\left[-1\right]$
\end_inset
to
\begin_inset Formula $z_{K-1}\left[-1\right]$
\end_inset
given initial conditions on
\begin_inset Formula $y$
\end_inset
(and
\begin_inset Formula $x$
\end_inset
).
The command
\series bold
signal.lfiltic
\series default
performs this function.
\layout Subsubsection
Other filters
\layout Standard
The signal processing package provides many more filters as well.
\layout Paragraph
Median Filter
\layout Standard
A median filter is commonly applied when noise is markedly non-Gaussian
or when it is desired to preserve edges.
The median filter works by sorting all of the array pixel values in a rectangul
ar region surrounding the point of interest.
The sample median of this list of neighborhood pixel values is used as
the value for the output array.
The sample median is the middle array value in a sorted list of neighborhood
values.
If there are an even number of elements in the neighborhood, then the average
of the middle two values is used as the median.
A general purpose median filter that works on N-dimensional arrays is
\series bold
signal.medfilt
\series default
.
A specialized version that works only for two-dimensional arrays is available
as
\series bold
signal.medfilt2d
\series default
.
\family typewriter
\layout Paragraph
Order Filter
\layout Standard
A median filter is a specific example of a more general class of filters
called order filters.
To compute the output at a particular pixel, all order filters use the
array values in a region surrounding that pixel.
These array values are sorted and then one of them is selected as the output
value.
For the median filter, the sample median of the list of array values is
used as the output.
A general order filter allows the user to select which of the sorted values
will be used as the output.
So, for example one could choose to pick the maximum in the list or the
minimum.
The order filter takes an additional argument besides the input array and
the region mask that specifies which of the elements in the sorted list
of neighbor array values should be used as the output.
The command to perform an order filter is
\series bold
signal.order_filter
\series default
.
\layout Paragraph
Wiener filter
\layout Standard
The Wiener filter is a simple deblurring filter for denoising images.
This is not the Wiener filter commonly described in image reconstruction
problems but instead it is a simple, local-mean filter.
Let
\begin_inset Formula $x$
\end_inset
be the input signal, then the output is
\layout Standard
\begin_inset Formula \[
y=\left\{ \begin{array}{cc}
\frac{\sigma^{2}}{\sigma_{x}^{2}}m_{x}+\left(1-\frac{\sigma^{2}}{\sigma_{x}^{2}}\right)x & \sigma_{x}^{2}\geq\sigma^{2},\\
m_{x} & \sigma_{x}^{2}<\sigma^{2}.\end{array}\right.\]
\end_inset
Where
\begin_inset Formula $m_{x}$
\end_inset
is the local estimate of the mean and
\begin_inset Formula $\sigma_{x}^{2}$
\end_inset
is the local estimate of the variance.
The window for these estimates is an optional input parameter (default
is
\begin_inset Formula $3\times3$
\end_inset
).
The parameter
\begin_inset Formula $\sigma^{2}$
\end_inset
is a threshold noise parameter.
If
\begin_inset Formula $\sigma$
\end_inset
is not given then it is estimated as the average of the local variances.
\layout Paragraph
Hilbert filter
\layout Standard
The Hilbert transform constructs the complex-valued analytic signal from
a real signal.
For example if
\begin_inset Formula $x=\cos\omega n$
\end_inset
then
\begin_inset Formula $y=\textrm{hilbert}\left(x\right)$
\end_inset
would return (except near the edges)
\begin_inset Formula $y=\exp\left(j\omega n\right).$
\end_inset
In the frequency domain, the hilbert transform performs
\begin_inset Formula \[
Y=X\cdot H\]
\end_inset
where
\begin_inset Formula $H$
\end_inset
is 2 for positive frequencies,
\begin_inset Formula $0$
\end_inset
for negative frequencies and
\begin_inset Formula $1$
\end_inset
for zero-frequencies.
\layout Paragraph
Detrend
\layout Subsection
Filter design
\layout Subsubsection
Finite-impulse response design
\layout Subsubsection
Inifinite-impulse response design
\layout Subsubsection
Analog filter frequency response
\layout Subsubsection
Digital filter frequency response
\layout Subsection
Linear Time-Invariant Systems
\layout Subsubsection
LTI Object
\layout Subsubsection
Continuous-Time Simulation
\layout Subsubsection
Step response
\layout Subsubsection
Impulse response
\layout Section
Input/Output
\layout Subsection
Binary
\layout Subsubsection
Arbitrary binary input and output (fopen)
\layout Subsubsection
Read and write Matlab .mat files
\layout Subsubsection
Saving workspace
\layout Subsection
Text-file
\layout Subsubsection
Read text-files (read_array)
\layout Subsubsection
Write a text-file (write_array)
\layout Section
Fourier Transforms
\layout Subsection
One-dimensional
\layout Subsection
Two-dimensional
\layout Subsection
N-dimensional
\layout Subsection
Shifting
\layout Subsection
Sample frequencies
\layout Subsection
Hilbert transform
\layout Subsection
Tilbert transform
\layout Section
Linear Algebra
\layout Standard
When SciPy is built using the optimized ATLAS LAPACK and BLAS libraries,
it has very fast linear algebra capabilities.
If you dig deep enough, all of the raw lapack and blas libraries are available
for your use for even more speed.
In this section, some easier-to-use interfaces to these routines are described.
\layout Standard
All of these linear algebra routines expect an object that can be converted
into a 2-dimensional array.
The output of these routines is also a two-dimensional array.
There is a matrix class defined in Numeric that scipy inherits and extends.
You can initialize this class with an appropriate Numeric array in order
to get objects for which multiplication is matrix-multiplication instead
of the default, element-by-element multiplication.
\layout Subsection
Matrix Class
\layout Standard
The matrix class is initialized with the SciPy command
\family typewriter
\series bold
mat
\family default
\series default
which is just convenient short-hand for
\family typewriter
\series bold
Matrix.Matrix
\family default
\series default
.
If you are going to be doing a lot of matrix-math, it is convenient to
convert arrays into matrices using this command.
One convencience of using the mat command is that you can enter two-dimensional
matrices using MATLAB-like syntax with commas or spaces separating columns
and semicolons separting rows as long as the matrix is placed in a string
passed to
\series bold
mat
\series default
.
\layout Subsection
Basic routines
\layout Subsubsection
Finding Inverse
\layout Standard
The inverse of a matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
is the matrix
\begin_inset Formula $\mathbf{B}$
\end_inset
such that
\begin_inset Formula $\mathbf{AB}=\mathbf{I}$
\end_inset
where
\begin_inset Formula $\mathbf{I}$
\end_inset
is the identity matrix consisting of ones down the main diagonal.
Usually
\begin_inset Formula $\mathbf{B}$
\end_inset
is denoted
\begin_inset Formula $\mathbf{B}=\mathbf{A}^{-1}$
\end_inset
.
In SciPy, the matrix inverse of the Numeric array, A, is obtained using
\family typewriter
\series bold
linalg.inv
\series default
(A)
\family default
, or using
\family typewriter
A.I
\family default
if
\family typewriter
A
\family default
is a Matrix.
For example, let
\begin_inset Formula \[
\mathbf{A=}\left[\begin{array}{ccc}
1 & 3 & 5\\
2 & 5 & 1\\
2 & 3 & 8\end{array}\right]\]
\end_inset
then
\begin_inset Formula \[
\mathbf{A^{-1}=\frac{1}{25}\left[\begin{array}{ccc}
-37 & 9 & 22\\
14 & 2 & -9\\
4 & -3 & 1\end{array}\right]=\left[\begin{array}{ccc}
-1.48 & 0.36 & 0.88\\
0.56 & 0.08 & -0.36\\
0.16 & -0.12 & 0.04\end{array}\right].}\]
\end_inset
The following example demonstrates this computation in SciPy
\begin_inset Include \verbatiminput{examples/scipy/example10.2.1}
preview false
\end_inset
\layout Subsubsection
Solving linear system
\layout Standard
Solving linear systems of equations is straightforward using the scipy command
\series bold
linalg.solve.
\series default
This command expects an input matrix and a right-hand-side vector.
The solution vector is then computed.
An option for entering a symmetrix matrix is offered which can speed up
the processing when applicable.
As an example, suppose it is desired to solve the following simultaneous
equations:
\begin_inset Formula \begin{eqnarray*}
x+3y+5z & = & 10\\
2x+5y+z & = & 8\\
2x+3y+8z & = & 3\end{eqnarray*}
\end_inset
We could find the solution vector using a matrix inverse:
\begin_inset Formula \[
\left[\begin{array}{c}
x\\
y\\
z\end{array}\right]=\left[\begin{array}{ccc}
1 & 3 & 5\\
2 & 5 & 1\\
2 & 3 & 8\end{array}\right]^{-1}\left[\begin{array}{c}
10\\
8\\
3\end{array}\right]=\frac{1}{25}\left[\begin{array}{c}
-232\\
129\\
19\end{array}\right]=\left[\begin{array}{c}
-9.28\\
5.16\\
0.76\end{array}\right].\]
\end_inset
However, it is better to use the linalg.solve command which can be faster
and more numerically stable.
In this case it gives the same answer as shown in the following example:
\begin_inset Include \verbatiminput{examples/scipy/example10.2.2}
preview false
\end_inset
\layout Subsubsection
Finding Determinant
\layout Standard
The determinant of a square matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
is often denoted
\begin_inset Formula $\left|\mathbf{A}\right|$
\end_inset
and is a quantity often used in linear algebra.
Suppose
\begin_inset Formula $a_{ij}$
\end_inset
are the elements of the matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
and let
\begin_inset Formula $M_{ij}=\left|\mathbf{A}_{ij}\right|$
\end_inset
be the determinant of the matrix left by removing the
\begin_inset Formula $i^{\textrm{th}}$
\end_inset
row and
\begin_inset Formula $j^{\textrm{th}}$
\end_inset
column from
\begin_inset Formula $\mathbf{A}$
\end_inset
.
Then for any row
\begin_inset Formula $i,$
\end_inset
\begin_inset Formula \[
\left|\mathbf{A}\right|=\sum_{j}\left(-1\right)^{i+j}a_{ij}M_{ij}.\]
\end_inset
This is a recursive way to define the determinant where the base case is
defined by accepting that the determinant of a
\begin_inset Formula $1\times1$
\end_inset
matrix is the only matrix element.
In SciPy the determinant can be calculated with
\series bold
linalg.det
\series default
.
For example, the determinant of
\begin_inset Formula \[
\mathbf{A=}\left[\begin{array}{ccc}
1 & 3 & 5\\
2 & 5 & 1\\
2 & 3 & 8\end{array}\right]\]
\end_inset
is
\begin_inset Formula \begin{eqnarray*}
\left|\mathbf{A}\right| & = & 1\left|\begin{array}{cc}
5 & 1\\
3 & 8\end{array}\right|-3\left|\begin{array}{cc}
2 & 1\\
2 & 8\end{array}\right|+5\left|\begin{array}{cc}
2 & 5\\
2 & 3\end{array}\right|\\
& = & 1\left(5\cdot8-3\cdot1\right)-3\left(2\cdot8-2\cdot1\right)+5\left(2\cdot3-2\cdot5\right)=-25.\end{eqnarray*}
\end_inset
In SciPy this is computed as shown in this example:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.2.3}
preview false
\end_inset
\layout Subsubsection
Computing norms
\layout Standard
Matrix and vector norms can also be computed with SciPy.
A wide range of norm definitions are available using different parameters
to the order argument of
\series bold
linalg.norm
\series default
.
This function takes a rank-1 (vectors) or a rank-2 (matrices) array and
an optional order argument (default is 2).
Based on these inputs a vector or matrix norm of the requested order is
computed.
\layout Standard
For vector
\series bold
x
\series default
, the order parameter can be any real number including
\series bold
inf
\series default
or -
\series bold
inf
\series default
.
The computed norm is
\begin_inset Formula \[
\left\Vert \mathbf{x}\right\Vert =\left\{ \begin{array}{cc}
\max\left|x_{i}\right| & \textrm{ord}=\textrm{inf}\\
\min\left|x_{i}\right| & \textrm{ord}=-\textrm{inf}\\
\left(\sum_{i}\left|x_{i}\right|^{\textrm{ord}}\right)^{1/\textrm{ord}} & \left|\textrm{ord}\right|<\infty.\end{array}\right.\]
\end_inset
\layout Standard
For matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
the only valid values for norm are
\begin_inset Formula $\pm2,\pm1,$
\end_inset
\begin_inset Formula $\pm$
\end_inset
inf, and 'fro' (or 'f') Thus,
\begin_inset Formula \[
\left\Vert \mathbf{A}\right\Vert =\left\{ \begin{array}{cc}
\max_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=\textrm{inf}\\
\min_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=-\textrm{inf}\\
\max_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=1\\
\min_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=-1\\
\max\sigma_{i} & \textrm{ord}=2\\
\min\sigma_{i} & \textrm{ord}=-2\\
\sqrt{\textrm{trace}\left(\mathbf{A}^{H}\mathbf{A}\right)} & \textrm{ord}=\textrm{'fro'}\end{array}\right.\]
\end_inset
where
\begin_inset Formula $\sigma_{i}$
\end_inset
are the singular values of
\begin_inset Formula $\mathbf{A}$
\end_inset
.
\layout Subsubsection
Solving linear least-squares problems and pseudo-inverses
\layout Standard
Linear least-squares problems occur in many branches of applied mathematics.
In this problem a set of linear scaling coefficients is sought that allow
a model to fit data.
In particular it is assumed that data
\begin_inset Formula $y_{i}$
\end_inset
is related to data
\begin_inset Formula $\mathbf{x}_{i}$
\end_inset
through a set of coefficients
\begin_inset Formula $c_{j}$
\end_inset
and model functions
\begin_inset Formula $f_{j}\left(\mathbf{x}_{i}\right)$
\end_inset
via the model
\begin_inset Formula \[
y_{i}=\sum_{j}c_{j}f_{j}\left(\mathbf{x}_{i}\right)+\epsilon_{i}\]
\end_inset
where
\begin_inset Formula $\epsilon_{i}$
\end_inset
represents uncertainty in the data.
The strategy of least squares is to pick the coefficients
\begin_inset Formula $c_{j}$
\end_inset
to minimize
\begin_inset Formula \[
J\left(\mathbf{c}\right)=\sum_{i}\left|y_{i}-\sum_{j}c_{j}f_{j}\left(x_{i}\right)\right|^{2}.\]
\end_inset
\layout Standard
Theoretically, a global minimum will occur when
\begin_inset Formula \[
\frac{\partial J}{\partial c_{n}^{*}}=0=\sum_{i}\left(y_{i}-\sum_{j}c_{j}f_{j}\left(x_{i}\right)\right)\left(-f_{n}^{*}\left(x_{i}\right)\right)\]
\end_inset
or
\begin_inset Formula \begin{eqnarray*}
\sum_{j}c_{j}\sum_{i}f_{j}\left(x_{i}\right)f_{n}^{*}\left(x_{i}\right) & = & \sum_{i}y_{i}f_{n}^{*}\left(x_{i}\right)\\
\mathbf{A}^{H}\mathbf{Ac} & = & \mathbf{A}^{H}\mathbf{y}\end{eqnarray*}
\end_inset
where
\begin_inset Formula \[
\left\{ \mathbf{A}\right\} _{ij}=f_{j}\left(x_{i}\right).\]
\end_inset
When
\begin_inset Formula $\mathbf{A^{H}A}$
\end_inset
is invertible, then
\begin_inset Formula \[
\mathbf{c}=\left(\mathbf{A}^{H}\mathbf{A}\right)^{-1}\mathbf{A}^{H}\mathbf{y}=\mathbf{A}^{\dagger}\mathbf{y}\]
\end_inset
where
\begin_inset Formula $\mathbf{A}^{\dagger}$
\end_inset
is called the pseudo-inverse of
\begin_inset Formula $\mathbf{A}.$
\end_inset
Notice that using this definition of
\series bold
\begin_inset Formula $\mathbf{A}$
\end_inset
\series default
the model can be written
\begin_inset Formula \[
\mathbf{y}=\mathbf{Ac}+\boldsymbol{\epsilon}.\]
\end_inset
The command
\series bold
linalg.lstsq
\series default
will solve the linear least squares problem for
\begin_inset Formula $\mathbf{c}$
\end_inset
given
\begin_inset Formula $\mathbf{A}$
\end_inset
and
\begin_inset Formula $\mathbf{y}$
\end_inset
.
In addition
\series bold
linalg.pinv
\series default
or
\series bold
linalg.pinv2
\series default
(uses a different method based on singular value decomposition) will find
\begin_inset Formula $\mathbf{A}^{\dagger}$
\end_inset
given
\begin_inset Formula $\mathbf{A}.$
\end_inset
\layout Standard
The following example Fig.\SpecialChar ~
\begin_inset LatexCommand \ref{fig:lstsq_fit}
\end_inset
demonstrate the use of
\series bold
linalg.lstsq
\series default
and
\series bold
linalg.pinv
\series default
for solving a data-fitting problem.
The data shown below were generated using the model:
\begin_inset Formula \[
y_{i}=c_{1}e^{-x_{i}}+c_{2}x_{i}\]
\end_inset
where
\begin_inset Formula $x_{i}=0.1i$
\end_inset
for
\begin_inset Formula $i=1\ldots10$
\end_inset
,
\begin_inset Formula $c_{1}=5$
\end_inset
, and
\begin_inset Formula $c_{2}=4.$
\end_inset
Noise is added to
\begin_inset Formula $y_{i}$
\end_inset
and the coefficients
\begin_inset Formula $c_{1}$
\end_inset
and
\begin_inset Formula $c_{2}$
\end_inset
are estimated using linear least squares.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.2.5}
preview false
\end_inset
\layout Standard
\begin_inset Float figure
placement htbp
wide false
collapsed false
\layout Standard
\align center
\begin_inset Graphics
filename fig/scipy/lstsq_fit.pdf
display color
width 70line%
BoundingBox 3cm 10cm 18cm 25cm
clip
rotateOrigin center
\end_inset
\layout Caption
\begin_inset LatexCommand \label{fig:lstsq_fit}
\end_inset
Least squares fitting example
\end_inset
\layout Subsubsection
Generalized inverse
\layout Standard
The generalized inverse is calculated using the command
\series bold
linalg.pinv
\series default
or
\series bold
linalg.pinv2.
\series default
These two commands differ in how they compute the generalized inverse.
The first uses the linalg.lstsq algorithm while the second uses singular
value decomposition.
Let
\begin_inset Formula $\mathbf{A}$
\end_inset
be an
\begin_inset Formula $M\times N$
\end_inset
matrix, then if
\begin_inset Formula $M>N$
\end_inset
the generalized inverse is
\begin_inset Formula \[
\mathbf{A}^{\dagger}=\left(\mathbf{A}^{H}\mathbf{A}\right)^{-1}\mathbf{A}^{H}\]
\end_inset
while if
\begin_inset Formula $M<N$
\end_inset
matrix the generalized inverse is
\begin_inset Formula \[
\mathbf{A}^{\#}=\mathbf{A}^{H}\left(\mathbf{A}\mathbf{A}^{H}\right)^{-1}.\]
\end_inset
In both cases for
\begin_inset Formula $M=N$
\end_inset
, then
\begin_inset Formula \[
\mathbf{A}^{\dagger}=\mathbf{A}^{\#}=\mathbf{A}^{-1}\]
\end_inset
as long as
\begin_inset Formula $\mathbf{A}$
\end_inset
is invertible.
\layout Subsection
Decompositions
\layout Standard
In many applications it is useful to decompose a matrix using other representati
ons.
There are several decompositions supported by SciPy.
\layout Subsubsection
Eigenvalues and eigenvectors
\layout Standard
The eigenvalue-eigenvector problem is one of the most commonly employed
linear algebra operations.
In one popular form, the eigenvalue-eigenvector problem is to find for
some square matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
scalars
\begin_inset Formula $\lambda$
\end_inset
and corresponding vectors
\begin_inset Formula $\mathbf{v}$
\end_inset
such that
\begin_inset Formula \[
\mathbf{Av}=\lambda\mathbf{v}.\]
\end_inset
For an
\begin_inset Formula $N\times N$
\end_inset
matrix, there are
\begin_inset Formula $N$
\end_inset
(not necessarily distinct) eigenvalues --- roots of the (characteristic)
polynomial
\begin_inset Formula \[
\left|\mathbf{A}-\lambda\mathbf{I}\right|=0.\]
\end_inset
\layout Standard
The eigenvectors,
\begin_inset Formula $\mathbf{v}$
\end_inset
, are also sometimes called right eigenvectors to distinguish them from
another set of left eigenvectors that satisfy
\begin_inset Formula \[
\mathbf{v}_{L}^{H}\mathbf{A}=\lambda\mathbf{v}_{L}^{H}\]
\end_inset
or
\begin_inset Formula \[
\mathbf{A}^{H}\mathbf{v}_{L}=\lambda^{*}\mathbf{v}_{L}.\]
\end_inset
With it's default optional arguments, the command
\series bold
linalg.eig
\series default
returns
\begin_inset Formula $\lambda$
\end_inset
and
\begin_inset Formula $\mathbf{v}.$
\end_inset
However, it can also return
\begin_inset Formula $\mathbf{v}_{L}$
\end_inset
and just
\begin_inset Formula $\lambda$
\end_inset
by itself (
\series bold
linalg.eigvals
\series default
returns just
\begin_inset Formula $\lambda$
\end_inset
as well).
\layout Standard
In addtion,
\series bold
linalg.eig
\series default
can also solve the more general eigenvalue problem
\begin_inset Formula \begin{eqnarray*}
\mathbf{Av} & = & \lambda\mathbf{Bv}\\
\mathbf{A}^{H}\mathbf{v}_{L} & = & \lambda^{*}\mathbf{B}^{H}\mathbf{v}_{L}\end{eqnarray*}
\end_inset
for square matrices
\begin_inset Formula $\mathbf{A}$
\end_inset
and
\begin_inset Formula $\mathbf{B}.$
\end_inset
The standard eigenvalue problem is an example of the general eigenvalue
problem for
\begin_inset Formula $\mathbf{B}=\mathbf{I}.$
\end_inset
When a generalized eigenvalue problem can be solved, then it provides a
decomposition of
\begin_inset Formula $\mathbf{A}$
\end_inset
as
\begin_inset Formula \[
\mathbf{A}=\mathbf{BV}\boldsymbol{\Lambda}\mathbf{V}^{-1}\]
\end_inset
where
\begin_inset Formula $\mathbf{V}$
\end_inset
is the collection of eigenvectors into columns and
\begin_inset Formula $\boldsymbol{\Lambda}$
\end_inset
is a diagonal matrix of eigenvalues.
\layout Standard
By definition, eigenvectors are only defined up to a constant scale factor.
In SciPy, the scaling factor for the eigenvectors is chosen so that
\begin_inset Formula $\left\Vert \mathbf{v}\right\Vert ^{2}=\sum_{i}v_{i}^{2}=1.$
\end_inset
\layout Standard
As an example, consider finding the eigenvalues and eigenvectors of the
matrix
\begin_inset Formula \[
\mathbf{A}=\left[\begin{array}{ccc}
1 & 5 & 2\\
2 & 4 & 1\\
3 & 6 & 2\end{array}\right].\]
\end_inset
The characteristic polynomial is
\begin_inset Formula \begin{eqnarray*}
\left|\mathbf{A}-\lambda\mathbf{I}\right| & = & \left(1-\lambda\right)\left[\left(4-\lambda\right)\left(2-\lambda\right)-6\right]-\\
& & 5\left[2\left(2-\lambda\right)-3\right]+2\left[12-3\left(4-\lambda\right)\right]\\
& = & -\lambda^{3}+7\lambda^{2}+8\lambda-3.\end{eqnarray*}
\end_inset
The roots of this polynomial are the eigenvalues of
\begin_inset Formula $\mathbf{A}$
\end_inset
:
\begin_inset Formula \begin{eqnarray*}
\lambda_{1} & = & 7.9579\\
\lambda_{2} & = & -1.2577\\
\lambda_{3} & = & 0.2997.\end{eqnarray*}
\end_inset
The eigenvectors corresponding to each eigenvalue can be found using the
original equation.
The eigenvectors associated with these eigenvalues can then be found.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.3.1}
preview false
\end_inset
\layout Subsubsection
Singular value decomposition
\layout Standard
Singular Value Decompostion (SVD) can be thought of as an extension of the
eigenvalue problem to matrices that are not square.
Let
\begin_inset Formula $\mathbf{A}$
\end_inset
be an
\begin_inset Formula $M\times N$
\end_inset
matrix with
\begin_inset Formula $M$
\end_inset
and
\begin_inset Formula $N$
\end_inset
arbitrary.
The matrices
\begin_inset Formula $\mathbf{A}^{H}\mathbf{A}$
\end_inset
and
\begin_inset Formula $\mathbf{A}\mathbf{A}^{H}$
\end_inset
are square hermitian matrices
\begin_inset Foot
collapsed true
\layout Standard
A hermition matrix
\begin_inset Formula $\mathbf{D}$
\end_inset
satisfies
\begin_inset Formula $\mathbf{D}^{H}=\mathbf{D}.$
\end_inset
\end_inset
of size
\begin_inset Formula $N\times N$
\end_inset
and
\begin_inset Formula $M\times M$
\end_inset
respectively.
It is known that the eigenvalues of square hermitian matrices are real
and non-negative.
In addtion, there are at most
\begin_inset Formula $\min\left(M,N\right)$
\end_inset
identical non-zero eigenvalues of
\begin_inset Formula $\mathbf{A}^{H}\mathbf{A}$
\end_inset
and
\begin_inset Formula $\mathbf{A}\mathbf{A}^{H}.$
\end_inset
Define these positive eigenvalues as
\begin_inset Formula $\sigma_{i}^{2}.$
\end_inset
The square-root of these are called singular values of
\begin_inset Formula $\mathbf{A}.$
\end_inset
The eigenvectors of
\begin_inset Formula $\mathbf{A}^{H}\mathbf{A}$
\end_inset
are collected by columns into an
\begin_inset Formula $N\times N$
\end_inset
unitary
\begin_inset Foot
collapsed true
\layout Standard
A unitary matrix
\begin_inset Formula $\mathbf{D}$
\end_inset
satisfies
\begin_inset Formula $\mathbf{D}^{H}\mathbf{D}=\mathbf{I}=\mathbf{D}\mathbf{D}^{H}$
\end_inset
so that
\begin_inset Formula $\mathbf{D}^{-1}=\mathbf{D}^{H}.$
\end_inset
\end_inset
matrix
\begin_inset Formula $\mathbf{V}$
\end_inset
while the eigenvectors of
\begin_inset Formula $\mathbf{A}\mathbf{A}^{H}$
\end_inset
are collected by columns in the unitary matrix
\begin_inset Formula $\mathbf{U}$
\end_inset
, the singular values are collected in an
\begin_inset Formula $M\times N$
\end_inset
zero matrix
\begin_inset Formula $\mathbf{\boldsymbol{\Sigma}}$
\end_inset
with main diagonal entries set to the singular values.
Then
\begin_inset Formula \[
\mathbf{A=U}\boldsymbol{\Sigma}\mathbf{V}^{H}\]
\end_inset
is the singular-value decomposition of
\begin_inset Formula $\mathbf{A}.$
\end_inset
Every matrix has a singular value decomposition.
Sometimes, the singular values are called the spectrum of
\begin_inset Formula $\mathbf{A}.$
\end_inset
The command
\series bold
linalg.svd
\series default
will return
\begin_inset Formula $\mathbf{U}$
\end_inset
,
\begin_inset Formula $\mathbf{V}^{H}$
\end_inset
, and
\begin_inset Formula $\sigma_{i}$
\end_inset
as an array of the singular values.
To obtain the matrix
\begin_inset Formula $\mathbf{\Sigma}$
\end_inset
use
\series bold
linalg.diagsvd.
\series default
The following example illustrates the use of
\series bold
linalg.svd
\series default
.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.3.2}
preview false
\end_inset
\layout Subsubsection
LU decomposition
\layout Standard
The LU decompostion finds a representation for the
\begin_inset Formula $M\times N$
\end_inset
matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
as
\begin_inset Formula \[
\mathbf{A}=\mathbf{PLU}\]
\end_inset
where
\begin_inset Formula $\mathbf{P}$
\end_inset
is an
\begin_inset Formula $M\times M$
\end_inset
permutation matrix (a permutation of the rows of the identity matrix),
\begin_inset Formula $\mathbf{L}$
\end_inset
is in
\begin_inset Formula $M\times K$
\end_inset
lower triangular or trapezoidal matrix (
\begin_inset Formula $K=\min\left(M,N\right)$
\end_inset
) with unit-diagonal, and
\begin_inset Formula $\mathbf{U}$
\end_inset
is an upper triangular or trapezoidal matrix.
The SciPy command for this decomposition is
\series bold
linalg.lu
\series default
.
\layout Standard
Such a decomposition is often useful for solving many simultaneous equations
where the left-hand-side does not change but the right hand side does.
For example, suppose we are going to solve
\begin_inset Formula \[
\mathbf{A}\mathbf{x}_{i}=\mathbf{b}_{i}\]
\end_inset
for many different
\begin_inset Formula $\mathbf{b}_{i}$
\end_inset
.
The LU decomposition allows this to be written as
\begin_inset Formula \[
\mathbf{PLUx}_{i}=\mathbf{b}_{i}.\]
\end_inset
Because
\begin_inset Formula $\mathbf{L}$
\end_inset
is lower-triangular, the equation can be solved for
\begin_inset Formula $\mathbf{U}\mathbf{x}_{i}$
\end_inset
and finally
\begin_inset Formula $\mathbf{x}_{i}$
\end_inset
very rapidly using forward- and back-substitution.
An initial time spent factoring
\begin_inset Formula $\mathbf{A}$
\end_inset
allows for very rapid solution of similar systems of equations in the future.
If the intent for performing LU decomposition is for solving linear systems
then the command
\series bold
linalg.lu_factor
\series default
should be used followed by repeated applications of the command
\series bold
linalg.lu_solve
\series default
to solve the system for each new right-hand-side.
\layout Subsubsection
Cholesky decomposition
\layout Standard
Cholesky decomposition is a special case of LU decomposition applicable
to Hermitian positive definite matrices.
When
\begin_inset Formula $\mathbf{A}=\mathbf{A}^{H}$
\end_inset
and
\begin_inset Formula $\mathbf{x}^{H}\mathbf{Ax}\geq0$
\end_inset
for all
\begin_inset Formula $\mathbf{x}$
\end_inset
, then decompositions of
\begin_inset Formula $\mathbf{A}$
\end_inset
can be found so that
\begin_inset Formula \begin{eqnarray*}
\mathbf{A} & = & \mathbf{U}^{H}\mathbf{U}\\
\mathbf{A} & = & \mathbf{L}\mathbf{L}^{H}\end{eqnarray*}
\end_inset
where
\begin_inset Formula $\mathbf{L}$
\end_inset
is lower-triangular and
\begin_inset Formula $\mathbf{U}$
\end_inset
is upper triangular.
Notice that
\begin_inset Formula $\mathbf{L}=\mathbf{U}^{H}.$
\end_inset
The command
\series bold
linagl.cholesky
\series default
computes the cholesky factorization.
For using cholesky factorization to solve systems of equations there are
also
\series bold
linalg.cho_factor
\series default
and
\series bold
linalg.cho_solve
\series default
routines that work similarly to their LU decomposition counterparts.
\layout Subsubsection
QR decomposition
\layout Standard
The QR decomposition (sometimes called a polar decomposition) works for
any
\begin_inset Formula $M\times N$
\end_inset
array and finds an
\begin_inset Formula $M\times M$
\end_inset
unitary matrix
\begin_inset Formula $\mathbf{Q}$
\end_inset
and an
\begin_inset Formula $M\times N$
\end_inset
upper-trapezoidal matrix
\begin_inset Formula $\mathbf{R}$
\end_inset
such that
\begin_inset Formula \[
\mathbf{A=QR}.\]
\end_inset
Notice that if the SVD of
\begin_inset Formula $\mathbf{A}$
\end_inset
is known then the QR decomposition can be found
\begin_inset Formula \[
\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{H}=\mathbf{QR}\]
\end_inset
implies that
\begin_inset Formula $\mathbf{Q}=\mathbf{U}$
\end_inset
and
\begin_inset Formula $\mathbf{R}=\boldsymbol{\Sigma}\mathbf{V}^{H}.$
\end_inset
Note, however, that in SciPy independent algorithms are used to find QR
and SVD decompositions.
The command for QR decomposition is
\series bold
linalg.qr
\series default
.
\layout Subsubsection
Schur decomposition
\layout Standard
For a square
\begin_inset Formula $N\times N$
\end_inset
matrix,
\begin_inset Formula $\mathbf{A}$
\end_inset
, the Schur decomposition finds (not-necessarily unique) matrices
\begin_inset Formula $\mathbf{T}$
\end_inset
and
\begin_inset Formula $\mathbf{Z}$
\end_inset
such that
\begin_inset Formula \[
\mathbf{A}=\mathbf{ZT}\mathbf{Z}^{H}\]
\end_inset
where
\begin_inset Formula $\mathbf{Z}$
\end_inset
is a unitary matrix and
\begin_inset Formula $\mathbf{T}$
\end_inset
is either upper-triangular or quasi-upper triangular depending on whether
or not a real schur form or complex schur form is requested.
For a real schur form both
\begin_inset Formula $\mathbf{T}$
\end_inset
and
\begin_inset Formula $\mathbf{Z}$
\end_inset
are real-valued when
\begin_inset Formula $\mathbf{A}$
\end_inset
is real-valued.
When
\begin_inset Formula $\mathbf{A}$
\end_inset
is a real-valued matrix the real schur form is only quasi-upper triangular
because
\begin_inset Formula $2\times2$
\end_inset
blocks extrude from the main diagonal corresponding to any complex-valued
eigenvalues.
The command
\series bold
linalg.schur
\series default
finds the Schur decomposition while the command
\series bold
linalg.rsf2csf
\series default
converts
\begin_inset Formula $\mathbf{T}$
\end_inset
and
\begin_inset Formula $\mathbf{Z}$
\end_inset
from a real Schur form to a complex Schur form.
The Schur form is especially useful in calculating functions of matrices.
\layout Standard
The following example illustrates the schur decomposition:
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.3.6}
preview false
\end_inset
\layout Subsection
Matrix Functions
\layout Standard
Consider the function
\begin_inset Formula $f\left(x\right)$
\end_inset
with Taylor series expansion
\begin_inset Formula \[
f\left(x\right)=\sum_{k=0}^{\infty}\frac{f^{\left(k\right)}\left(0\right)}{k!}x^{k}.\]
\end_inset
A matrix function can be defined using this Taylor series for the square
matrix
\begin_inset Formula $\mathbf{A}$
\end_inset
as
\begin_inset Formula \[
f\left(\mathbf{A}\right)=\sum_{k=0}^{\infty}\frac{f^{\left(k\right)}\left(0\right)}{k!}\mathbf{A}^{k}.\]
\end_inset
While, this serves as a useful representation of a matrix function, it
is rarely the best way to calculate a matrix function.
\layout Subsubsection
Exponential and logarithm functions
\layout Standard
The matrix exponential is one of the more common matrix functions.
It can be defined for square matrices as
\begin_inset Formula \[
e^{\mathbf{A}}=\sum_{k=0}^{\infty}\frac{1}{k!}\mathbf{A}^{k}.\]
\end_inset
The command
\series bold
linalg.expm3
\series default
uses this Taylor series definition to compute the matrix exponential.
Due to poor convergence properties it is not often used.
\layout Standard
Another method to compute the matrix exponential is to find an eigenvalue
decomposition of
\begin_inset Formula $\mathbf{A}$
\end_inset
:
\begin_inset Formula \[
\mathbf{A}=\mathbf{V}\boldsymbol{\Lambda}\mathbf{V}^{-1}\]
\end_inset
and note that
\begin_inset Formula \[
e^{\mathbf{A}}=\mathbf{V}e^{\boldsymbol{\Lambda}}\mathbf{V}^{-1}\]
\end_inset
where the matrix exponential of the diagonal matrix
\begin_inset Formula $\boldsymbol{\Lambda}$
\end_inset
is just the exponential of its elements.
This method is implemented in
\series bold
linalg.expm2
\series default
.
\layout Standard
The preferred method for implementing the matrix exponential is to use scaling
and a Padé approximation for
\begin_inset Formula $e^{x}$
\end_inset
.
This algorithm is implemented as
\series bold
linalg.expm
\series default
.
\layout Standard
The inverse of the matrix exponential is the matrix logarithm defined as
the inverse of the matrix exponential.
\begin_inset Formula \[
\mathbf{A}\equiv\exp\left(\log\left(\mathbf{A}\right)\right).\]
\end_inset
The matrix logarithm can be obtained with
\series bold
linalg.logm
\series default
.
\layout Subsubsection
Trigonometric functions
\layout Standard
The trigonometric functions
\begin_inset Formula $\sin$
\end_inset
,
\begin_inset Formula $\cos$
\end_inset
, and
\begin_inset Formula $\tan$
\end_inset
are implemented for matrices in
\series bold
linalg.sinm, linalg.cosm,
\series default
and
\series bold
linalg.tanm
\series default
respectively.
The matrix sin and cosine can be defined using Euler's identity as
\begin_inset Formula \begin{eqnarray*}
\sin\left(\mathbf{A}\right) & = & \frac{e^{j\mathbf{A}}-e^{-j\mathbf{A}}}{2j}\\
\cos\left(\mathbf{A}\right) & = & \frac{e^{j\mathbf{A}}+e^{-j\mathbf{A}}}{2}.\end{eqnarray*}
\end_inset
The tangent is
\begin_inset Formula \[
\tan\left(x\right)=\frac{\sin\left(x\right)}{\cos\left(x\right)}=\left[\cos\left(x\right)\right]^{-1}\sin\left(x\right)\]
\end_inset
and so the matrix tangent is defined as
\begin_inset Formula \[
\left[\cos\left(\mathbf{A}\right)\right]^{-1}\sin\left(\mathbf{A}\right).\]
\end_inset
\layout Subsubsection
Hyperbolic trigonometric functions
\layout Standard
The hyperbolic trigonemetric functions
\begin_inset Formula $\sinh$
\end_inset
,
\begin_inset Formula $\cosh$
\end_inset
, and
\begin_inset Formula $\tanh$
\end_inset
can also be defined for matrices using the familiar definitions:
\begin_inset Formula \begin{eqnarray*}
\sinh\left(\mathbf{A}\right) & = & \frac{e^{\mathbf{A}}-e^{-\mathbf{A}}}{2}\\
\cosh\left(\mathbf{A}\right) & = & \frac{e^{\mathbf{A}}+e^{-\mathbf{A}}}{2}\\
\tanh\left(\mathbf{A}\right) & = & \left[\cosh\left(\mathbf{A}\right)\right]^{-1}\sinh\left(\mathbf{A}\right).\end{eqnarray*}
\end_inset
These matrix functions can be found using
\series bold
linalg.sinhm
\series default
,
\series bold
linalg.coshm
\series default
, and
\series bold
linalg.tanhm
\series default
.
\layout Subsubsection
Arbitrary function
\layout Standard
Finally, any arbitrary function that takes one complex number and returns
a complex number can be called as a matrix function using the command
\series bold
linalg.funm
\series default
.
This command takes the matrix and an arbitrary Python function.
It then implements an algorithm from Golub and Van Loan's book
\begin_inset Quotes eld
\end_inset
Matrix Computations
\begin_inset Quotes erd
\end_inset
to compute function applied to the matrix using a Schur decomposition.
Note that
\emph on
the function needs to accept complex numbers
\emph default
as input in order to work with this algorithm.
For example the following code computes the zeroth-order Bessel function
applied to a matrix.
\layout Standard
\begin_inset Include \verbatiminput{examples/scipy/example10.4.4}
preview false
\end_inset
\layout Section
Statistics
\layout Standard
SciPy has a tremendous number of basic statistics routines with more easily
added by the end user (if you create one please contribute it).
All of the statistics functions are located in the sub-package
\series bold
stats
\series default
and a fairly complete listing of these functions can be had using
\family typewriter
info(stats)
\family default
\emph on
.
\layout Subsection
Random Variables
\layout Standard
There are two general distribution classes that have been implemented for
encapsulating continuous random variables and discrete random variables.
Over 80 continuous random variables and 10 discrete random variables have
been implemented using these classes.
The list of the random variables available is in the docstring for the
stats sub-package.
A detailed description of each of them is also located in the files continuous.l
yx and discrete.lyx in the stats sub-directories.
\layout Section
Interfacing with the Python Imaging Library
\begin_inset OptArg
collapsed true
\layout Standard
Interfacing with the PIL
\end_inset
\layout Standard
If you have the Python Imaging Library (PIL) installed, SciPy provides some
convenient functions that make use of it's facilities particularly for
reading, writing, displaying, and rotating images.
In SciPy an image is always a two- or three-dimensional array.
Gray-scale, and colormap images are always two-dimensional arrays while
RGB images are three-dimensional with the third dimension specifying the
channel.
\layout Standard
Commands available include
\layout Itemize
fromimage --- convert a PIL image to a Numeric array
\layout Itemize
toimage --- convert Numeric array to PIL image
\layout Itemize
imsave --- save Numeric array to an image file
\layout Itemize
imread --- read an image file into a Numeric array
\layout Itemize
imrotate --- rotate an image (a Numeric array) counter-clockwise
\layout Itemize
imresize --- resize an image using the PIL
\layout Itemize
imshow --- external viewer of a Numeric array (using PIL)
\layout Itemize
imfilter --- fast, simple built-in filters provided by PIL
\layout Itemize
radon --- simple radon transform based on imrotate
\layout Section
Some examples
\layout Subsection
Least squares fit
\layout Standard
\begin_inset ERT
status Open
\layout Standard
\backslash
lstinputlisting{examples/scipy/least_squares_fit.py}
\end_inset
\layout Subsection
Roots of a polynomial in 1d
\layout Standard
\begin_inset ERT
status Open
\layout Standard
\backslash
lstinputlisting{examples/scipy/roots1d.py}
\end_inset
\layout Subsection
Bessel functions
\layout Standard
\begin_inset ERT
status Open
\layout Standard
\backslash
lstinputlisting{examples/scipy/bessel.py}
\end_inset
\the_end
