Numerical Method With Sage
Numerical Method With Sage
f(x
k
)
f
(x
k
)
, k = 0, 1, 2, . . .
An implementation of the Newtons method algorithm is presented
in the next code. As an example, we use the algorithm to nd the
root of the equation x
2
3 = 0. The function newton method is used
to generate a list of the iteration points. Sage contains a preparser
that makes it possible to use certain mathematical constructs such as
f(x) = f that would be syntactically invalid in standard Python. In
the program, the last iteration value is printed and the iteration steps
are tabulated. The accuracy goal for the root 2h is reached when
f(x
n
h)f(x
n
+ h) < 0 . In order to avoid an innite loop, the maxi-
mum number of iterations is limited by the parameter maxn.
def newton_method(f, c, maxn, h):
f(x) = f
iterates = [c]
j = 1
while True:
c = c - f(c)/derivative(f(x))(x=c)
iterates.append(c)
if f(c-h)*f(c+h) < 0 or j == maxn:
break
6
Figure 1. Output of the Newton iteration.
j += 1
return c, iterates
f(x) = x^2-3
h = 10^-5
initial = 2.0
maxn = 10
z, iterates = newton_method(f, initial, maxn, h/2.0)
print "Root =", z
html.table(([i, c.n(digits=7), f(c).n(digits=5),
(f(c-h)*f(c+h)).n(digits=4)] for i, c in enumerate(iterates)),
header=["$n$", "$x_n$", "$f(x_n)$", "$f(x_n-h)f(x_n+h)$"])
3.2. Computational Methods of Linear Algebra. Sage oers a
good platform for practicing both symbolic and numerical linear alge-
bra. The software packages specialized in computational linear algebra
that are contained in Sage include LAPACK, LinBox, IML, ATLAS,
BLAS and GSL. Both the NumPy and SciPy libraries have subpackages
for algorithms that utilize NumPys fast arrays for matrix operations.
Also, the GSL library can be used via Cython. In most of the examples
of this chapter, we use native Sage matrices that suit most needs.
Let A be a matrix over the real double eld (RDF) as dened in the
next code. The matrix function accepts the base ring for the entries
and the dimensions of the matrix as its parameters. We can compute
various values associated with the matrix, such as the determinant, the
rank and the Frobenius norm:
Sage: A = matrix(RDF, 3, [1,3,-3, -3,7,-3, -6,6,-2])
Sage: A.determinant()
-32.0
Sage: A.rank()
3
Sage: A.norm()
12.7279220614
7
The function A.inverse() returns the inverse of A, should it exist.
Otherwise Sage informs that the matrix must be nonsingular in order
to compute the inverse.
Sage: A.inverse()
3x
0
cos (x
1
x
2
)
1
2
x
2
0
81(x
1
+ 0.1)
2
+ sin x
2
+ 10.6
e
x
0
x
1
+ 20x
2
+
103
3
,
where x = (x
0
, x
1
, x
2
). In this program the Jacobian matrix J
f
(x) is
computed symbolically and its inverse numerically. As a result, the
program produces a table of the iteration steps and an interactive 3D
plot that shows the steps in a coordinate system.
x0, x1, x2 = var(x0 x1 x2)
f1(x0, x1, x2) = 3*x0 - cos(x1*x2) - (1/2)
f2(x0, x1, x2) = x0^2 - 81*(x1 + 0.1)^2 + sin(x2) + 10.6
12
f3(x0, x1, x2) = e^(-x0*x1) + 20*x2 + (10*pi - 3)/3
f(x0, x1, x2) = (f1(x0,x1,x2), f2(x0,x1,x2), f3(x0,x1,x2))
j = jacobian(f, [x0,x1,x2])
x = vector([3.0, 4.0, 5.0]) # Initial values
data = [[0, x, n(norm(f(x[0], x[1], x[2])), digits=4)]]
for i in range(1,8):
x = vector((n(d) for d in x - j(x0=x[0], x1=x[1],
x2=x[2]).inverse()*f(x[0], x[1], x[2])))
data.append([i, x, norm(f(x[0], x[1], x[2]))])
# HTML Table
html.table([(data[i][0], data[i][1].n(digits=10),
n(data[i][2], digits=4)) for i in range(0,8)],
header = ["$i$", "$(x_0,x_1,x_2)$", "$norm(f)$"])
# 3D Picture
l = line3d([d[1] for d in data], thickness=5)
p = point3d(data[-1][1], size=15, color="red")
show(l + p)
Figure 5. The program produces a table showing the
iteration steps of the Newtons method.
3.5. Nonlinear tting of multiparameter functions. Given the
data (x
j
, y
j
), j = 1, . . . , m, we wish to t y = f(x, ) into a model,
where = (
1
, ...,
p
) by minimizing the object function
13
Figure 6. An interactive 3D plot shows the iteration
steps in a coordinate system. The plot is made with the
Jmol application integrated in Sage.
s() =
m
j=1
(y
j
f(x
j
, ))
2
.
The minimization may encounter the usual diculties: the minimum
need not be unique and there may be several local minima, each of
which could cause the algorithm to stop prematurely. In the next
example the function minimize uses the Nelder-Mead Method from
the scipy.optimize package.
Let = (
1
,
2
,
3
). Consider the model function
f(x, ) =
1
e
x
+
2
e
3
x
.
The data points used in this example are generated randomly by
deviating the values of the model function.
14
from numpy import *
def fmodel(lam, x):
return lam[0]*exp(-x) + lam[1]*exp(-lam[2]*x)
def fobj(lam, xdata, ydata):
return linalg.norm(fmodel(lam, xdata) - ydata)
xdata = arange(0, 1.15, 0.05)
lam = [0.2, 1.5, 0.7]
y = fmodel(lam, xdata)
# The generation of the data points
ydata = y*(0.97+0.05*random.rand(y.size))
# Initial values
lam0 = [1, 1, 1]
y0 = fobj(lam0, xdata, ydata)
# The minimization of the object function
lam = minimize(fobj, lam0, args=(xdata, ydata), algorithm=simplex)
yfinal = fobj(lam, xdata, ydata)
# Plot of the datapoints and the model function
fit = plot(fmodel(lam, x), (x, 0, 1.5), legend_label="Fitted curve")
datapoints = list_plot(zip(xdata, ydata), size=20,
legend_label="Data points")
html("\n\n$\\text{Object function values: start = %s, final = %s}$\n"
%( n(y0, digits=5), n(yfinal, digits=5)))
show(fit + datapoints, figsize=5, gridlines=True,
axes_labels=("xdata", "ydata"))
3.6. Polynomial Approximation. The problem of nding (n1)th
order polynomial approximation for a function g on the interval [r
1
, r
2
]
leads to the minimization of the expression
f(c
1
, ..., c
n
) =
r
2
r
1
(g(x)
n
k=1
c
k
x
nk
)
2
dx
with respect to the parameter vector (c
1
, ..., c
n
) . In order to nd the
optimal value of the parameter vector, we consider the critical points
15
Figure 7. The algorithm used in the program returns
a report on the success of the optimization. The plot
shows the data points and the model function in the same
coordinate system.
where gradient vanishes i.e. the points where
f
c
i
= 0 , i = 1, ..., n.
For the purpose of illustration, consider the case when g(x) = e
x
and
n = 2, 3, 4. The equations
f
c
i
= 0 lead to the requirement
n
k=1
c
k
r
2nkj+1
2n k j + 1
r
2
r
1
=
r
2
r
1
g(x)x
nj
dx ,
16
which is an n n linear system of equations for the coecients c
k
. In
the code below the integrals on the right hand side are evaluated in
terms of the function numerical integral.
from numpy import arange, polyval, zeros, linalg
f(x) = e^x
interval = [-1, 1]
nmax = 3
data = zeros((nmax, nmax))
r1, r2 = interval[0], interval[1]
for n in range(2, nmax+1):
a, b, c = zeros((n, n)), zeros(n), zeros(n)
for j in range(1, n+1):
for k in range(1, n+1):
a[j-1, k-1] = (r2^(2*n-k-j+1) - r1^(2*n-k-j+1))/(2*n-k-j+1)
b[j-1] = numerical_integral(f*x^(n-j), r1, r2)[0]
c = linalg.solve(a,b)
h = (r2-r1)/40
xs = arange(r1, r2+h, h)
y1 = [f(xi) for xi in xs]
y2 = polyval(c, xs)
err = abs(y2-y1)
maxer = max(err)
# Use trapezoidal rule to compute error
int1 = h*(sum(err) - 0.5*(err[0] + err[-1]))
int2 = h*(sum(err^2) - 0.5*(err[0]^2 + err[-1]^2))
# Plots
eplot = plot(f(x), (x, r1, r2), color="black")
polyplot = plot(polyval(c, x), (x, r1, r2), color="red", figsize=3)
epoly = eplot + polyplot
errplot = plot(abs(f(x)-polyval(c, x)), (x, r1, r2), figsize=3)
# Output text and graphics
html("<hr>$n=%s:$"%n)
html.table([["$%s$"%latex(matrix(a).n(digits=4)),
"$%s$"%latex(vector(b).column().n(digits=4)),
"$%s$"%latex(vector(c).column().n(digits=4))]],
header=["$a$", "$b$", "$c$"])
html("$\\text{Abs. error = } %s\qquad\qquad\\text{L2 error = }
%s$"%(maxer, int2))
html.table([["$\\text{Approximation (n = %s)}$"%n,
"$\\text{Abs. error function (n = %s)}$"%n],
17
[epoly, errplot]], header=True)
Figure 8. The picture shows the (n 1)th order poly-
nomial approximation for the function e
x
on the interval
[1, 1] in the cases of n = 2 and n = 3.
18
4. Concluding remarks
During its initial years of development, the Sage project has grown
to an environment which oers an attractive alternative for the com-
mercial packages in several areas of computational mathematics. For
the purpose of scientic computation teaching, the functionalities of
Sage are practically the same as those of commercial packages. While
free availability to instructional purposes is a very signicant advan-
tage, there are also other important factors from the learners point of
view:
(1) The Python language can be used also for many other purposes
not tied with the scientic computing. A wide selection of ex-
tensions and other special libraries are available in the Internet.
(2) The support of advanced data structures and support of object-
oriented data types and modular program structure is available.
(3) There is an active users forum.
It is likely that the Sage environment in education will become more
popular on all levels of mathematics education from junior high school
to graduate level teaching at universities. The support of symbolic
computation via Maxima and various numerical packages are notewor-
thy in this respect. For purposes of teaching scientic computing, the
Sage environment and the modules it contains form an excellent option.
References
[ADV] M. S. Andersen, J. Dahl, and L. Vandenberghe: CVXOPT: A
Python package for convex optimization, http://abel.ee.ucla.edu/cvxopt.
[BBSE] R. Bradshaw, S. Behnel, D. S. Seljebotn, G. Ewing, et al.: The
Cython compiler, http://cython.org.
[BF] R. L. Burden and J. D. Faires: Numerical analysis. 2002
[CdB] S. D. Conte and C. de Boor: Elementary numerical analysis: An al-
gorithmic approach. Third ed. McGraw-Hill Book Co., New York-Toronto,
Ont.-London 1965 x+278 pp.
[DGPS] W. Decker, G.-M. Greuel, G. Pfister and H. Sch onemann:
Singular A computer algebra system for polynomial computations,
http://www.singular.uni-kl.de.
[GAP] GAP Groups, Algorithms, and Programming, The GAP Group,
http://www.gap-system.org.
[H] M. T. Heath: Scientic computing: An Introductory Survey. Second ed.
McGrawHill 2002.
[His] Sage Reference v5.2: History and License, The Sage Development Team,
2012, http://www.sagemath.org/doc/reference/history and license.html.
[Hun] J. D. Hunter: Matplotlib: A 2D Graphics Environment. Com-
puting in Science & Engineering, Vol. 9, No. 3. (2007), pp. 90-95,
doi:10.1109/MCSE.2007.55.
19
[Jmol] Jmol: an open-source Java viewer for chemical structures in 3D,
http://www.jmol.org.
[K] J. Kiusalaas: Numerical methods in engineering with Python. Second
edition. Cambridge University Press, New York, 2010.
[L] H. P. Langtangen: A Primer on Scientic Programming With Python,
Springer, 2009, ISBN: 3642024742, 9783642024740
[MF] J. H. Mathews and K. D. Fink: Numerical methods using MATLAB,
Third ed. 1999, Prentice Hall, Inc., Englewood Clis, NJ.
[Mol] C. B. Moler: Numerical computing with MATLAB. Society for Industrial
and Applied Mathematics, Philadelphia, PA, 2004. xii+336 pp. ISBN: 0-
89871-560-1
[Num] Numerical Computing with Sage, Release 5.2, The Sage Development Team,
2012, http://www.sagemath.org/pdf/numerical sage.pdf
[O] T. E. Oliphant: Python for Scientic Computing, Computing in Science
& Engineering 9, 90 (2007).
[PARI] PARI/GP, Bordeaux, 2012, http://pari.math.u-bordeaux.fr.
[PFTV] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P.
Flannery: Numerical recipes. The art of scientic computing. Third edi-
tion. Cambridge University Press, Cambridge, 2007. xxii+1235 pp. ISBN:
978-0-521-88068-8
[PG] F. Prez and B. E. Granger: IPython: A System for Interactive Scientic
Computing, Computing in Science & Engineering 9, 90 (2007).
[R] R: A Language and Environment for Statistical Computing, R Core Team,
R Foundation for Statistical Computing, Vienna, Austria, ISBN: 3-900051-
07-0, http://www.R-project.org.
[Ras] A. Rasila: Introduction to numerical methods with Python language, part
1, Mathematics Newsletter / Ramanujan Mathematical Society 14: 1 and 2
(2004), 1 -15. http://www.ramanujanmathsociety.org/
[Sage] William A. Stein et al.: Sage Mathematics Software (Version 5.2), The
Sage Development Team, 2012, http://www.sagemath.org.
[SageTeX] Dan Drake et al.: The Sage-
TeX Package, 2009, ftp://tug.ctan.org/pub/tex-
archive/macros/latex/contrib/sagetex/sagetexpackage.pdf.
[SciPy] E. Jones, T. Oliphant, P. Peterson, et al.: SciPy: Open source
scientic tools for Python, http://www.scipy.org/.
[T] S. Tosi: Matplotlib for Python Developers, From technologies to solutions,
2009, Packt Publishing.
[Tac] J. E. Stone: The Tachyon 3D Ray Tracer,
Sage Reference v5.2, The Sage Development Team,
http://www.sagemath.org/doc/reference/sage/plot/plot3d/tachyon.html.
[TLN] A. Tveito, H. P. Langtangen, B. F. Nielsen, and X. Cai: Elements
of scientic computing. Texts in Computational Science and Engineering, 7.
Springer-Verlag, Berlin, 2010. xii+459 pp. ISBN: 978-3-642-11298-0
E-mail address: [email protected], [email protected]
Department of Mathematics and Statistics, University of Turku