(1D ENGINEERS

(p g
lw)
&

Namir C. Shammas

© C & C++
source code
included on disk
This Book is the Property of
Kenneth Neeld
Date Purchased 5 /%é
Book No. _ ss
 C/C++ Mathematical
Algorithms for Scientists
& Engineers
Other McGraw-Hill Titles of Interest

BAKER * C Mathematical Functions Handbook

BAKER * C Tools for Scientists and Engineers
BAKER * More C Tools for Scientists and Engineers
DORFMAN ¢ C++ by Example: Object-Oriented Analysis, Design & Programming
DORFMAN, NEUBERGER * C++ Memory Management
HANCOCK « The C Primer
HOLUB * C + C++: Programming with Objects in C and C++
RANADE ¢ The Elements of C Programming Style
RANADE, ZAMIR * C++ Primer for C Programmers
RAO * C++ and the OOP Paradigm
SHAMMAS « Visual C++ Generic Programming
SHAMMAS ¢ Visual Basic: Easy Windows Programming
SHAMMAS *« Windows Batch File Programming
SMITH * Concepts of Object-Oriented Programming
SMITH * C++ Applications Guide
TERRIBLE * Practical C++
WATSON ¢ Portable User Interface Programming in C++
WATSON * C++ Power Paradigms

In order to receive additional information on these or any other

McGraw-Hill titles, in the United States please call 1-800-822-8158.
In other countries, contact your local McGraw-Hill representative.
 C/C++ Mathematical
Algorithms for Scientists
& Engineers

Namir C. Shammas

McGraw-Hill, Inc.
New York San Francisco WashingtonSacer Auckland rete
Caracas Lisbon London Ma ari Me peek
Montreal New Delhi ee Juan abate
Sydney Pe Toronnie
Product or brand names used in this book may be trade names or
trademarks. Where we believe that there may be proprietary claims to such
trade names or trademarks, the name has been used with an initial capital or
it has been capitalized in the style used by the name claimant. Regardless of
the capitalization used, all such names have been used in an editorial
manner without any intent to convey endorsement of or other affiliation
with the name claimant. Neither the author nor the publisher intends to
express any judgment as to the validity or legal status of any such
proprietary claims.

Library of Congress Cataloging-in-Publication Data

Shammas, Namir Clement, 1954—

C/C++ mathematical algorithms for scientists and engineers / by
Namir C. Shammas.
De em:
ISBN 0-07-912004-0 (pbk.)
1. Numerical analysis—Data processing. 2. Mathematical
statistics—Data processing. 3. Computer algorithms. 4. C
(Computer program language) 5. C++ (Computer program language)
I. Title.
QA297.S457 1995
519.4'0285'513—dc20 95-7444
CIP

Copyright © 1995 by McGraw-Hill Inc. Printed in the United States of

America. Except as permitted under the United States Copyright Act of
1976, no part of this publication may be reproduced or distributed in any
form or by any means, or stored in a data base or retrieval system, without
the prior written permission of the publisher.

123456789 DOC/DOC 98765

ISBN 0-07-912004

The sponsoring editor for this book was Jennifer Holt DiGiovanna. The
executive editor was Robert E. Ostrander, the book editor was Aaron G.
Bittner, and the director of production was Katherine G. Brown. This book
was set in ITC Century Light. It was composed in Blue Ridge Summit, Pa.

Printed and bound by R.R. Donnelley & Sons Company, Crawfordsville,

Indiana..

Information contained in this work has been obtained by McGraw-

Hill, Inc. from sources believed to be reliable. However, neither
McGraw-Hill nor its authors guarantee the accuracy or
completeness of any information published herein and neither
McGraw-Hill nor its authors shall be responsible for any errors,
omissions, or damages arising out of use of this information. This
work is published with the understanding that McGraw-Hill and
its authors are supplying information but are not attempting to
render engineering or other professional services. If such services
are required, the assistance of an appropriate professional should
be sought. MH94
9120040
 gr
?
’
¢ 2

=
— oon
= oo - ee
_ es ———

1 y
7 ry

. ,
a a
i
a
, Janet Kay

Lome Grpant on >

Cheper |. Gewiteen>
I ‘
2 Tie UCU

Ty Gore Vado oc et @
Th cera orem ERiee wimon & os
.
the (2 ov Gee Bete
ote @
a
es oe tapes
The Cas See Comp
tea Cutan

het Te

eee >. Somigg Menive | pact

7 wart Aww
gst 2
y tigerica « j

Tia Nive at ie?co ;

to ee So “a

o oe. o> ot
‘2 i.

a a s 2y¢ed
tae BH Ohy
Thee wt ies cm iis

wi i-« ~
Gavi WOR
6 7. os os =
= 659i
(ha OP ster

re te? ge ; 4

= oe

ines & =
 Digitized by the Internet Archive
in 2021 with funding from
Kahle/Austin Foundation

https://archive.org/details/ccmathematicalaloO00sham
 Contents

Introduction xi

Chapter 1. Simultaneous Linear Equations af

The GLOBAL.H File

The General Vectors and Matrices
The Gauss-Jordan Elimination Method
The Gauss-Siedel Method
The LU Decomposition Method
The C Source Code ND
NNN
The C++ Source Code
The C Test Program 24
The C++ Test Program 28

Chapter 2. Solving Nonlinear Equations 33

The Bisection Method 34

Newton’s Method 34
The Richmond Method 36
The Combined Method 37
Newton’s Multiroot Method 37
Deflating Polynomial Method 38
The Lin-Bairstow Method 39
Solving Multiple Nonlinear Equations 40
Newton’s Method for Multiple Equations 41
The C Functions 41

Chapter 3. Interpolation 61

The Lagrangian Interpolation 61

The Barycentric Interpolation 62
Newton’s Divided Difference Interpolation 64
The Newton Difference Method
The Cubic Spline Interpolation Method 65
The C Source Code 66
Chapter 4. Numerical Differentiation 75
The Forward/Backward Difference Method 75
The Central Difference Method 76
The Extended Central Difference Method 77
The C Source Code 77

Chapter 5. Numerical Integration 89

Simpson’s Method 89
Simpson’s Alternate Extended Rule 90
The Gaussian Quadrature Methods 91
Gauss-Legendre Quadrature 91
The Gauss-Laguerre Quadrature 92
The Gauss-Hermite Quadrature 93
The Gauss-Chebyshev Quadrature 93
The Romberg Method 93
The C Source Code 94

Chapter 6. Solving Ordinary Differential Equations 105

The Runge-Kutta Method 105
The Runge-Kutta-Gill Method 106
The Runge-Kutta-Fehlberg Method 107
The C Source Code 108

Chapter 7. Optimization 119

The Bisection Method 119
Newton’s Method 120
The Golden Section Search Method 121
The Quadratic Interpolation Method 121
The Cubic Interpolation Method 122
The Simplex Method 123
Newton’s Method for Sequential Optimization 125
The C Source Code 125
The Test Program 134

Chapter 8. Basic Statistics 139

The Mean and Standard Deviation 139
The Confidence Intervals 140
The First Four Moments 140
Testing Sample Means 141
The C Source Code 142
The C++ Source Code 147
The C Test Program 151
The C++ Test Program 154

Chapter 9. The ANOVA Tests 159

The One-Way ANOVA 159
The Two-Way ANOVA 162
 The Two-Way ANOVA with Replication 164
The Latin-Square ANOVA 167
The Analysis of Covariance 169
The C Source Code 172
The C++ Source Code 188

Chapter 10. Linear Regression 203

Basic Linear Regression 203
Linearized Regression 205
The Confidence Interval for Projections 206
The Confidence Interval for Regression Coefficients 206
The Regression Algorithms 207
The Automatic Best Fit 208
The C Source Code 208
The C++ Source Code 220

Chapter 11. Multiple and Polynomial Regression 231

Multiple Regression 231
The C Source Code 234
The C++ Source Code 243
The C Test Program 250
The C++ Test Program 254
Polynomial Regression 259
The C Source Code 259
The C++ Source Code 268
The C Test Program 275
The C++ Test Program 278

Chapter 12. The Functions Library 283

The STATLIB Library 284
The Normal Distribution 284
The Inverse Normal Distribution 284
The Student-t Distribution 285
The Inverse Student-t Distribution 285
The F Distribution 285
The Inverse F Distribution 286
The Mathematical Functions Library 290
The Combination and Permutation Functions 290
The Gamma and Beta Functions 290
The Error Function 290
The Sine and Cosine Integral Functions 291
The Laguerre, Hermite, and Chebyshev Polynomials 291
The Bessel Functions 292

Index 299
About the Author 305
Disk Instructions 308
 wtion
rorieolagh dive AVCAA ya W-ow’? off]
‘ Ud =
AYQMA exraune-sthed ol f
rn@imrc? io dlegtara at!
leewted Co ret COlpoange (ialt as sertue? D ofT
The C Eearroe Co ‘3
oO emude «<9 et)

MjAM 5. Meroe S wreetrTem nolessipel yennid 01 wigat

pa
Papsee~ 4 he
Ginga s tow nere Emteedee Alo an threo

ena ot holaamgon eee

- Tie O@a0-Meu Qpereere
he fue.
ae Ceca
The Bhetoned eine 4 a0 eins
” PheG Sree Cade
eg a :
aang
_‘s v afine
Chiappe© eeatonementr

Tee Pe go ay ee on% nl A via.

Bxxe 7 obws ms
ive jo JG Metmag
lena: ty :
+3 (ay i sian hee a
re @ CecSe 7” ita
= i end corge W! 7 ode
rain alee
f we: a)
Saat ort af
Yy A eee"
iat seeder
- 7 “on iat *

pa 5 Wins
i
‘ard! enohan ett Sr

ruihhed termom
ctotvdinel® fe
no netted — :

notated
yavdils enoliau hae
neater nolurn tee bee Nolnen
enoltacni® ao) tae 2
Ao te
arttion 4 largely orto) base
hot verte scanty Dive Palani!
anolioturt

' ee - .

i vir Pa Of — ;

om it 5) > aPeare
 Introduction

For many decades FORTRAN has monopolized statistical- and numerical-analysis

computing. As C and C++ gain popularity, many find them fairly suitable for the
same tasks. This book presents source code in both of these languages, taking ad-
vantage of the fact that C and C++ compilers are typically bundled together by ma-
jor software vendors.
The book contains 12 chapters that cover popular numerical analysis and statisti-
cal topics. While the book provides C source code for all chapters, it offers the C++
version for the cases where using C++ classes offers a special advantage.
This book also focuses on the algorithms involved in implementing the various
methods. Just about every chapter discusses the algorithms involved in the methods
presented in that chapter. As for the equations behind the methods, they are of sec-
ondary importance. There are many books that discuss the equations behind the var-
ious numerical and statistical methods. However, there are not that many books that
present pseudo-code for the algorithms. This book does, allowing you to use the
pseudo-code in developing libraries in other programming languages, such as BASIC
and Pascal.
Chapter 1 discusses simultaneous linear equations. The chapter presents a library
for general vectors and matrices and then discusses popular methods for solving lin-
ear equations. The chapter includes both C and C++ source code.
Chapter 2 looks at solving nonlinear equations. The text discusses various meth-
ods for solving the roots of functions with single and multiple variables.
Chapter 3 focuses on interpolation use methods including Lagrangian interpola-
tion, the barycentric interpolation, Newton’s divided difference interpolation, the
Newton difference method, and the Cubic Spline method.
Chapter 4 discusses numerical differentiation and presents the forward/backward
difference method, the central difference method, and the extended central differ-
ence method. These methods offer you nontrivial tools for estimating the derivative
of a function.
Chapter 5 looks at popular numerical integration methods such as Simpson's
Method, the Gaussian Quadrature Methods, and the Romberg method. These meth-
ods offer tools for calculating the finite and infinite integrals for functions.

xi
xii Introduction

Chapter 6 focuses on solving ordinary differential equations. The chapter dis-

cusses the popular Runge-Kutta method and two of its spinoff methods, namely, the
Runge-Kutta-Gill and the Runge-Kutta-Fehlberg method. These spinoff methods of-
fer improved accuracy for the solutions.
Chapter 7 discusses the optimization of single-variable and multivariable func-
tions. The chapter discusses methods such as the Golden Section Search method, in-
terpolation methods, and the Simplex method.
Chapter 8 looks at basic statistics that include the mean, standard deviation, the
confidence intervals for the mean and standard deviation, and the first four mo-
ments. The chapter also looks at testing sample means to determine whether or not
they are significantly different. The chapter includes both C and C++ source code.
Chapter 9 discusses the various ANOVA tests. These tests include the one-way
ANOVA, the two-way ANOVA, the two-way ANOVA with replication, the latin-square
ANOVA, and the analysis of covariance. The chapter includes both C and C++ source
code.
Chapter 10 presents a library that supports linear regression. The chapter covers
basic linear regression, linearized regression, the confidence interval for projections,
and the confidence interval for regression coefficients. The text also discusses auto-
matic best fit for linearized regression. The chapter includes both C and C++ source
code.
Chapter 11 discusses multiple and polynomial regression. The text looks at calcu-
lating the regression intercept, slopes, standard errors for the slopes, projections,
the confidence intervals for the slopes, and performing various Student-t tests for
the slopes and correlation coefficient. The chapter includes both C and C++ source
code.
Chapter 12 presents libraries for common statistical and mathematical functions.
The statistical functions library includes the Normal distribution, Student-T distribu-
tion, the F distribution, and their inverses. The mathematical functions library in-
cludes functions such as the combination function, the Gamma function, the error
function, and the Laguerre functions. |
I hope that this book offers you a time-saving library for common methods in nu-
merical analysis and in statistics.
Happy Programming!

Namir Shammas
 Chapter

Simultaneous Linear Equations

Solving simultaneous linear equations is among the most popular operations in nu-
merical analysis as well as other scientific, statistical, and engineering fields. This
chapter will present selected methods in solving linear equations in both C and C++.
The chapter will present and discuss the following topics:

# The general-purpose header file GLOBAL.H

=# The C and C++ files which support general vectors and matrices
=» The Gauss-Jordan elimination method
= The Gauss-Seidel method
= The LU decomposition method
= The C and C++ source code for the above methods
=# The C and C++ test programs

The chapter presents both C and C++ listings for the above methods that solve si-
multaneous linear equations and perform miscellaneous matrix operations. The first
sections in this chapter discuss general-purpose source code files used in many
chapters in this book.

The GLOBAL.H File

The header file GLOBAL.H contains a set of declarations which are used in many list-
ings of this book. Listing 1.1 shows the contents of file GLOBAL.H, which is used by
both C and C+ listings.
2 Chapter One

Listing 1.1 The source code for the GLOBAL.H header file.
#ifndef _GLOBAL_H_
#define _GLOBAL_H_

#define TRUE 1
#define FALSE 0
#define EPSILON 1.0e-15
#define BAD RESULT -1.0e+30
#define MISSING_DATA -1.0e+30

#define SQR(x) ((x) * (x))

#FAOeEInNe CUBE (3c) (35) 9% (ae) * Ge)
#define ONEOVER(x) (1 / (x))
#define RECIPROCAL(x) (1 / (x))

typedef int Boolean;

enum boolean { false, true };

enum answer { no, yes, dont_care };

#endif

Listing 1.1 defines constants, macro-based functions, and data types commonly
used by the programs in this book. The items declared include Boolean-related con-
stants and data types, and some macro-based functions that calculate the square,
cube, and reciprocal values of numbers.
It is important to point out that the file GLOBAL.H is the only file used by both C
and C++ programs in this book. The C programs use the .H and .C files whereas the
C++ programs use the .HPP and .CPP files.

The General Vectors and Matrices

The implementation of many numerical analysis and statistical algorithms requires

the use of vectors and matrices to store input, output, and calculated data. This sec-
tion presents the C and C++ listings which provide simple implementations of dy-
namic vectors and matrices. Let’s first look at C code. Listing 1.2 shows the source
code for the ARRAYS.H header file. Listing 1.3 contains the source code for the AR- .
RAYS.C implementation file.

Listing 1.2 The source code for the ARRAYS.H header file.
#ifndef _ARRAYS_H_
#define _ARRAYS_H_

struct VectTag {
double* pData;
int maxSize;
i

struct MatTag {
double* pData;
int maxRows;
int maxCols;
nF

struct IntVectTag {
int* pData;
int maxSize;
1:
 Simultaneous Linear Equations 3

typedef struct MatTag Matrix;

typedef struct VectTag Vector;
typedef struct IntVectTag IntVector;

int newMat (Matrix* Mat, int maxRows, int maxCols);

void deleteMat (Matrix* Mat) ;
int newVect (Vector* Vect, int maxSize);
void deleteVect (Vector* Vect) ;
int newIntVect (IntVector* Vect, int maxSize);
void deleteIntVect (IntVector* Vect) ;
int checkRowCol (Matrix Mat, int row, int col);
int checkIndex(Vector Vect, int index);

#define MAT(Mat, row, col) (Mat.pData[(row) + Mat.maxRows * (col)])

#define VEC(Vect, index) (Vect.pData[ (index) ] )

#tendif

The header file in Listing 1.2 declares the structures VectTag, MatTag, and IntVect-
Tag to support dynamic double-type vectors, dynamic double-type matrices, and dy-
namic int-type vectors. Each structure declares the member pData as the pointer to
the dynamic array. In addition, these structures declare one or more members to store
the number of elements. The header file also uses a typedef statement to create the
types Matrix, Vector, and IntVector as aliases to the structures MatTag, VectTag, and
IntVectTag, respectively.
The header file declares a set of functions which create, remove, and verify the
indices for each type of array. In addition, the file declares the macros MAT and
VEC which offer a shorthand form for accessing the elements of a matrix or a vec-
tor. Notice that the macro VEC works with both double-type and int-type vectors,
since their supporting structures declare data members which have common
names.
Listing 1.3 contains the source code for the ARRAYS.C implementation file.

Listing 1.3 The source code for the ARRAYS.C implementation file.

#include "arrays.h"
#include <stdio.h>

int newMat (Matrix* Mat, int maxRows, int maxCols)

{
maxRows = (maxRows < 1) ? 1 : maxRows;
mMaxColse——(maxColse< 1) a? ls maxColls:
Mat->maxRows = maxRows;
Mat->maxCols = maxCols;
Mat->pData = NULL;
Mat->pData = (double*) malloc(maxRows * maxCols *
sizeof (double) );
return (Mat->pData != NULL) ? 1 : 0;
}
void deleteMat (Matrix* Mat)
{
if (Mat->pData)
free (Mat->pData) ;
Mat->pData = NULL;
Mat->maxCols 0;
Mat->maxRows Meith0;

}
4 Chapter One

Listing 1.3 (Continued)

int newVect (Vector* Vect, int maxSize)

{
maxSize = (maxSize < 1) ? 1 : maxSize;
Vect->maxSize = maxSize;
Vect->pData = NULL;
Vect->pData = (double*) malloc(maxSize * sizeof (double) );
return (Vect->pData != NULL) ? 1: 0;
}

void deleteVect (Vector* Vect)

it
if (Vect->pData)
free (Vect->pData) ;
Vect->pData = NULL;
Vect->maxSize = 0;
}

int newIntVect (IntVector* Vect, int maxSize)

{
maxSize = (maxSize < 1) ? 1 : maxSize;
Vect->maxSize = maxSize;
Vect->pData = NULL;
Vect->pData = (int*) malloc(maxSize * sizeof(int));
return (Vect->pData != NULL) ? 1: 0;
}

void deleteIntVect (IntVector* Vect)

{
if (Vect->pData)
free (Vect->pData) ;
Vect->pData = NULL;
Vect->maxSize = 0;
}

int checkRowCol (Matrix Mat, int row, int col)

il
return (row >= 0 && col >>= 0 &&
row < Mat.maxRows && col < Mat.maxCols) ? 1 : 0;
}

int checkIndex(Vector Vect, int index)

{
return (index >= 0 && index < Vect.maxSize) ? 1: 0;
}

Listing 1.3 declares the following C functions:

is The function newMat creates a new dynamic matrix which is associated with
the argument of parameter Mat. The parameters maxRows and MaxCols spec-
ify the number of rows and columns of the dynamic matrix. The function stores
the arguments for these parameters in the data members maxRows and max-
Cols, respectively. The function returns 1 if successful, and yields 0 when oth-
erwise.

2 . The function deleteMat deletes the dynamic matrix associated with the argument
of parameter Mat. The function also assigns NULL, 0, and 0 to the members
pData, maxCols, and maxRows, respectively.
 Simultaneous
Linear Equations 5

3. The function checkRowCol verifies that the arguments for parameter Row and Col
are non-negative integers that are less than the values of members Mat.maxRows
and Mat.maxCols. The function yields 1 if the above condition is true, and returns 0
when otherwise.
. The function newVect creates a new dynamic double-type vector which is associ-
ated with the argument of parameter Vect. The parameter maxSize specifies the
number of elements in the dynamic vector. The function stores the argument for
the above parameter in the data member maxSize. The function returns 1 if suc-
cessful and yields 0 when otherwise.

. The function deleteVect deletes the dynamic vector associated with the argument
of parameter Vect. The function also assigns NULL and 0 to the members pData
and maxSize, respectively.

. The function checkIndex verifies that the argument for parameter index is a non-
negative integer that is less than the value of member Vect.maxSize. The function
yields 1 if the above condition is true, and returns 0 when otherwise.

. The function newlnVect and deleteIntVect are similar to functions newVect and
deleteVect. The main difference is that functions newIntVect and deleteIntVect
work with int-type vectors.

Let’s now look at the C++ version of the arrays. Listing 1.4 shows the source code
for the ARRAYS.HPP header file. This header file declares classes for the matrix,
double-type vector, and int-type vector.

Listing 1.4 The source code for the ARRAYS.HPP header file.
#ifndef _ARRAYS_HPP_
#define _ARRAYS HPP_

class Vector
{
public:
Vector(int MaxSize)
{ pData = new double[maxSize = MaxSize]; }
~Vector
()
( dellete | ~pbata;}
void ReSize(int NewSize)
{
delete [] pData;
pData = new double[maxSize = NewSize];
}
double& operator [] (int index)
{ return pData[index]; }
double& operator () (int index)
{ return pData[index]; }
int getSize()
{ return maxSize; }
int checkiIndex(int index)
{@ return (index S= 0! && andex < maxSize) ? des 0; }

protected:
int maxSize;
6 Chapter One

Listing 1.4 (Continued)

double* pData;
Tie

class IntVector
{
public:
IntVector(int MaxSize = 1)
{ pData = new int[maxSize = MaxSize]; }
~IntVector ()
{ delete [] pData; }
void ReSize(int NewSize)
ib
delete [] pData;
pData = new int[maxSize = NewSize];
i
int& operator [ ] (int index)
{ return pData[index]; }
int& operator () (int index)
{ return pData[index]; }
int checkIndex(int index)
{ return (index >= 0 && index < maxSize)
int getSize()
{ return maxSize; }

protected:
int maxSize;
int* pData;
si

class Matrix
{
public:
Matrix(int MaxRows = 1, int MaxCols = 1)
{
maxCols = MaxCols;
maxRows = MaxRows;
pData = new double[maxRows * maxCols];
}
~Matrix()
{ delete [] pData; }
void ReSize(int NewRows, int NewCols)
{
delete [] pData;
maxCols = NewCols’
maxRows = NewRows;
pData = new double[maxRows * maxCols];
}
double& operator () (int row, int col)
{ return pData[row + maxRows * col]; }
int checkRowCol(int row, int col)
{ return (row >= 0 && row < maxRows &&
col >= 0 && col < maxCols) ? 1
int getRows( )
{ return maxRows; }
int getCols()
{ return maxCols; }

protected:
int maxRows;
int maxCols;
double* pData;
}e

#endif
 Simultaneous Linear Equations 7

Listing 1.4 declares the classes Vector, IntVector, and Matrix. The classes Vector
and IntVector are very similar and differ only in the type of the vector element.
Therefore, I'll only discuss the class Vector. This class declares a constructor, de-
structor, a set of member functions, and a set of data members. The simplicity of the
above classes allows me to use inline definition of the member functions. The class
Vector declares the data members pData and maxSize. The member functions of
class Vector include the element-access operators [] and () and the functions get-
Size and checkIndex. The operators access an element of the vector by specifying an
index. I chose to offer both [] and () operators for this purpose.
The ARRAYS.HPP header file also declares the class Matrix which supports dy-
namic matrices. The class declares a constructor, destructor, a set of member func-
tions, and a set of data members. The member functions include the operator (), and
functions getRows, getCols, and checkRowCol. The class has no operator [] because
the C++ compiler does not allow it to have two indices. The class declares the data
members pData, maxRows, and maxCols.

The Gauss-Jordan Elimination Method

The subject of solving simultaneous linear equations has occupied mathematicians for
along time. Today, we have a repertoire of numerous methods which vary in approach
and level of sophistication. This section discusses one of the popular methods for
solving linear equations, namely, the Gauss-Jordan elimination method. This method
is generally as efficient as other methods and is stable. The algorithm implemented in
this chapter uses full pivoting in the process of reducing the set of linear equations.

The Gauss-Seidel Method

The Gauss-Seidel method is an iterative technique which solves a set of linear equa-
tions by providing an initial guess and then refining that guess. Each iteration cycle
calculates the updated guess for each variable in terms of the current values of the
other variable. The algorithm employs the updated values for a variable within an it-
eration, instead of waiting for the current iteration to complete.

The LU Decomposition Method

The LU decomposition method solves linear equations by performing two main
steps. The first step transforms the main matrix into upper/lower triangular matri-
ces. The second step solves for a specific solution vector using back substitution.
You can repeat the second steps as many times needed to solve for different solu-
tion vectors.

The C Source Code

Let’s look at the C source code which implements the above linear equation algo-
rithms as well as other algorithms which manipulate matrices. Listing 1.5 shows the
source code for the MATVECT.H header file.
8 Chapter One

Listing 1.5 The source code for the MATVECT.H header file.
/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 8/9/94
Q module for basic vect and array operations:

Add matrices
Subtract matrices
Multiply matrices
44+Solve a set of linear equations using
++ the
Gauss-Jordan method
Solve a set of linear equations using the
LU decomposition method
Solve a set of linear equations using the
Gauss-Seidel method
ue Obtain the inverse of a matrix
+ Obtain the determinant of a matrix
bil
#ifndef _MATVECT_H_
#def ine _MATVECT_H_

#inc lude "global.h"

#inc lude "arrays.h"

#def ine MATVECT_EPSILON 1.0e-15

#def ine MATVECT_BAD_RESULT -1.0e+30

enum MatErrType { matErr_None, matErr_Size, matErr_ Singular,

matErr_IllConditioned, matErr_IterLimit };

enum MatErrType CopyMat (Matrix MatB, Matrix MatA,

int numRows, int numCols) ;

enum MatErrType AddMat (Matrix MatC, Matrix MatA, Matrix MatB,

int numRows, int numCols) ;

enum MatErrType SubMat (Matrix MatC, Matrix MatA, Matrix MatB,

int numRows, int numCols) ;
enum MatErrType MulMat (Matrix MatC, Matrix MatA, Matrix MatB,
int numRowsA, int numColsA,
int numRowsB, int numColsB) ;

enum MatErrType GaussJordan(Matrix A, Matrix B,

int numRows, int numCols) ;

enum MatErrType LUDecomp(Matrix A, IntVector Index,

int numRows, int* rowSwapFlag) ;

void LUBackSubst (Matrix A, IntVector Index,

int numRows, Vector B);

enum MatErrType GaussSeidel (

Matrix A, Vector B, Vector X,
int numRows, int maxIter,
double epsl, double eps2);

void LUInverse (Matrix A, Matrix InvA,

IntVector Index, int numRows) ;

enum MatErrType MatInverse(Matrix A, int numRows) ;

doub le LUDeterminant (Matrix A, int numRows,
int rowSwapFlag) ;

doub le MatDeterminant (Matrix A, int numRows) ;

#end sits
 Simultaneous Linear Equations 9

Listing 1.5 declares two constants, the enumerated type MatErrType, and a col-
lection of functions. The enumerated type lists the types of errors associated with
solving linear equations. Many of the functions declared in the header file return val-
ues of the MatErrType enumerated type. The functions use the structures Matrix,
Vector, and IntVector for the parameters that pass matrices, vectors, and integer
vectors, respectively.
The listing contains a set of C functions which perform basic matrix manipulation.
These functions include:

af The function CopyMat copies matrix MatA into matrix MatB. The parameters
numRows and numCols specify the number of rows and columns to copy.
. The function AddMat adds matrices MatA and MatB, and stores the result in ma-
trix MatC. The parameters numRows and numCols specify the number of rows
and columns to add.
. The function SubMat subtracts matrices MatA and MatB, and stores the result in
matrix MatC. The parameters numRows and numCols specify the number of rows
and columns to subtract.
. The function MultMat multiplies matrices MatA and MatB, and stores the result in
matrix MatC. The parameters numRows and numCols specify the number of rows
and columns to multiply.

The header file MATVECT.H contains the following functions which solve for lin-
ear equations:

if The function GaussJordan implements the Gauss-Jordan method. The parameter

A is the matrix of coefficients. The parameter B is the constants matrix. The pa-
rameter numRows specifies the number of rows and columns in matrix A, and the
number of rows in matrix B. The parameter numCols specifies the number of
columns in matrix B and represents the number of solution sets required.
. The function LUDecomp supports the LU decomposition of a matrix of coeffi-
cients, represented by parameter A. The parameter numRows specifies the num-
ber of rows and columns. The parameter Index is an integer vector which obtains
the row indices. The parameter rowSwapFlag is a pointer to an int-type value that
is a row-swap flag.
. The function LUBackSubst complements function LUDecomp by solving for a so-
lution vector. The parameter A presents the LU matrix. The parameter Index is
the vector of row indices. The parameter numRows specifies the number of rows
and columns. The parameter B represents the constant vector (as input) and so-
lution vector (as output).
. The function GaussSeidel implements the Gauss-Seidel method. The parameter A
represents the matrix of coefficients. The parameter B is the constants vector.
The parameter X is the solution vector. The parameter numRows specifies the
number of rows and columns. The parameters epsl is the small value which is
compared with the diagonal elements to determine if the matrix A is singular. The
parameter eps2 is the small value used in testing for convergence.
10 Chapter One

5. The function LUInverse yields the inverse matrix based of an LU matrix produced
by function LUDecomp. The parameter A represents the LU matrix of coeffi-
cients. The parameter InVA represents the inverse matrix of A. The parameter In-
dex is the index of rows. The parameter numRows specifies the number of rows
(which is also equal to the number of columns).
. The function MatInverse yields the inverse of a matrix. The parameter A repre-
sents the input matrix and the output inverse matrix. The parameter numRows
specifies the number of rows (which is also equal to the number of columns).
. The function LUDeterminant returns the determinant of a matrix. The parameter
A is the matrix generated by function LUDecomp. The parameter numRows spec-
ifies the number of rows (which is also equal to the number of columns). The pa-
rameter rowSwapFlag is the row-swap flag supplied by function LUDecomp.
. The function MatDeterminant returns the determinant of a matrix. The parame-
ter A is the input matrix. The parameter numRows specifies the number of rows
(which is also equal to the number of columns).

Listing 1.6 The source code for the MATVECT.C implementation file.
/*

C module for basic vector and array operations

il

#include "matvect.h"
#include <math.h>

void swapint(int* ail, amt* 12)

{
antictr
ee a a
*i2;
t;
}

void swapDouble(double* dl, double* d2)

{
double t;
£ =e Fil ¢
Fal. = dais
“G2 = "03
}

enum MatErrType CopyMat (Matrix MatB, Matrix MatA,

int numRows, int numCols)
{
int row, col;
if (!(checkRowCol(MatA, numRows, numCols) &&
checkRowCol(MatB, numRows, numCols) ))
return matErr_Size;

for (row = 0; row < numRows; row+t+t)

for .(col.= 0; col. < numCols; .col++)
MAT(MatB, row, col) = MAT(MatA, row, col);

return matErr_None;
}
enum MatErrType AddMat (Matrix MatC, Matrix MatA, Matrix MatB,
int numRows, int numCols)
{
 Simultaneous Linear Equations 11

int row, col;

if (!(checkRowCol(MatA, numRows, numCols) &&

checkRowCol (MatB, numRows, numCols) &&
checkRowCol(MatC, numRows, numCols) &&
numCols == numRows) )
return matErr_Size;

for (row = 0; row < numRows; row++)

Fons (COL = 0), colm< numColie- cols+)
MAT(MatC, row, col) = MAT(MatA, row, col) +
MAT(MatB, row, col);
return matErr_None;
}
enum MatErrType SubMat (Matrix MatC, Matrix MatA, Matrix MatB,
int numRows, int numCols)
{
int now, col;

if (!(checkRowCol(MatA, numRows, numCols) &&

checkRowCol(MatB, numRows, numCols) &&
checkRowCol(MatC, numRows, numCols) &&
numCols == numRows) )
return matErr_Size;

for (row = 0; row < numRows; row++)

for (col = 0; col < numCols; col++)
MAT(MatC, row, col) = MAT(MatA, row, col) -
MAT(MatB, row, col);

return matErr_None;
}

enum MatErrType MulMat (Matrix MatC, Matrix MatA, Matrix MatB,

int numRowsA, int numColsA,
int numRowsB, int numColsB)

int row, col;

if (!checkRowCol (MatA, numRowsA, numColsA) )

return matErr_Size;

if (!checkRowCol(MatB, numRowsB, numColsB) )

return matErr_Size;

if (nmumColsA != numRowsB)
return matErr_Size;

for (row = 0; row < numRowsB; row++) {

for (col = 0; col < numColsA; col++)
MAT(MatC, row, col) = 0;
for (col = 0; col < numColsA; col++)
MAT(MatC, row, col) += MAT(MatA, row, col) *
MAT (MatB, row, col);
}
return matErr_None;
}
enum MatErrType GaussJordan(Matrix A, Matrix B,
int numRows, int numCols)
{
int* rowIndex;
int* colIndex;
int* pivotIndex;
Ines aye ke wil, ms
inte POWs, CoOL:
double large, z, oneOverPiv;
12 Chapter One

Listing 1.6 (Continued)

if (!checkRowCol(A, numRows, numCols) )

return matErr_Size;

rowIndex = (int*) malloc(numRows * sizeof(int));

colIndex = (int*) malloc(numRows * sizeof(int));
pivotIndex = (int*) malloc(numRows * sizeof(int));

/* initialize the row and column indices */

for (i = 0; i < numRows; i++) {
rowIndex[i] = i;
Golindex [x ji .= 25
}

/* initialize the pivot index array */

for (i = 0; i < numRows; i++)
pivotindex[i] = -1;

for (i = 0; 3 < numRows; i++). f

large = 0;
for (j = 0; j < numRows; j++) {
if (pivotIndex[j] != 0)
for (k = 0; k < numRows; k++) {
if (pivotiIndex[k] == -1) {
i£ (fabs (MAT(A, 3, K)) >= Large) {
large = fabs(MAT(A, j, k));
row = ais
col- = k;
}
}
else if (pivotiIndex[k] > 0) {
free (rowindex) ;
free (coliIndex) ;
free (pivotindex) ;
return matErr_Singular;
}
}
}
PivotiIndex[col] += 1;
ip {row f= col), if
for (n = 0; n < numRows; n++)
swapDouble(&MAT(A, row, n), &MAT(A, col, n));
for (n = 0; n < numCols; n++)
swapDouble(&MAT(B, row, n), &MAT(B, col, n));
}
rowlndex[i] = row;
colindex[i] = col;
if (fabs (MAT(A, col, col)) < 1.e-10)
return matErr_Singular;
oneOverPiv = 1 / MAT(A, col, col);
MAT (A, col, col) = 4;
for (n = 0; n < numRows; n++)
MAT(A, col, n) *= oneOverPiv;
for (n = 0; n < numCols; n++)
MAT(B, col, n) *= oneOverPiv;
for (m = 0; m < numRows; m++)
af {(m° i= col)
Z = MAD (Am, cede:
MAT(A, m, col) = i;
for (n = 0; n < numRows; n++)
MAT (A; mj mh) —= MAT{(A, col, ay =e
for (n = 0; n < numCols; n++)
MAT (B, mm; 2) —, MAT(B,, col, sae oze
 Simultaneous
Linear Equations 13

for (nm = numRows - 1; n >= 0; n-—=) {

if (rowIndex[n] != colIndex[n])
for (k = 0; k < numRows; k++)
swapDouble(&MAT(A, k, rowIndex[n]),
&MAT(A, k, colIndex[n]));
}
free (rowIndex) ;
free (colIndex) ;
free (pivotIndex) ;
return matErr_None;
}

enum MatErrType LUDecomp (Matrix A, IntVector Index,

int numRows, int* rowSwapFlag)
{
AMepae aipe dk, elMase
double large, sum, z, 22;
double* scaleVect;

scaleVect = (double*) malloc(A.maxCols * sizeof(double)

);
/* initialize row interchange flag */
*rowSwapFlag = 1;
/* loop to obtain the scaling element */
for (i = 0; i < numRows; i++) {
large = 0);
fom (j= 0Rea, — numRows) myc) A
Z2 falos (MAI (Ayes es)
lange: = (z2e> large) 2922 ; large;
}
/* no non-zero large value? then exit with an error code */
aise (Gkevqers S= 0) <i
free (scaleVect) ;
return matErr_Singular;
}
scaleVect[i] = 1 / large;
}
form (3 = 0k = numRowsi) a4),
ope (hat OR sl <= g/t ae) ey
sum = MAT(A, i, Jj);
sone (is) Ol ie Ke sky ieee)
Sum == MAD (Ay a, ie)" * MAmICAA Mo” 3h
MAT CAP aetrs})) p= sume
}
large = 0;
for (1 = 3; i < numRows; i++) {
sum = MAT(A, 2, 3)>
Fore (ke =NORe kel << Ae ikea)
sum —= MAT(A,, 27k) = MAM(A, kK; 3);
MAUA eee |)! essen
Zz = scaleVect[i] * fabs(sum);
te (ee= large)!
large = Z;
iMax = i;
}
}
neon alee Mase) fi
for (kK = 0; k < numRows; k++) {
Zz = MAT(A, iMax, k);
MAT (AV—iMax, kK) = MATI(A, a7 k)i
NATCAC el = ze
}
*rowSwapFlag *= -1;
scaleVect[iMax] = scaleVect[j];
}
Index.pData[j] = iMax;
14 Chapter One

Listing 1.6 (Continued)

if (MAT(A, j, j) == 0)
MAT(A, j, 3) = MATVECT_EPSILON;
if (j != numRows) {
A ee ee
for (i = j+1; i < numRows; i++)
MATA, 1, £3 sais
}
}
free(scaleVect) ;
return matErr_None;
}
void LUBackSubst (Matrix A, IntVector Index,
int numRows, Vector B)
{
ine pk: «ji, bax Oka oad
double sum;

for (i = 0; i < numRows; i++) {

idx = Index.pData[i];
sum = B.pData[idx];
B.pData[idx] = B.pData[i];
22 (reo p14
for (3 =f ages ++)
sum -= MAT(A, i, j) * B.pData[jl];
else if (sum != 0)
Kis ae
B.pData[i] = sum;
}
for (i,.=.numRows)—ole.4>—)0? ==)
sum = B.pData[i];
for (j = i+1; j < numRows; j++)
sum -= MAT(A, i, j) * B-.pData[j];
B.pData[i] = sum / MAT(A, i, i);
}
}

enum MatErrType GaussSeidel (

Matrix A, Vector B, Vector X,
int numRows, int maxiter,
double epsi, double eps2)

enum opType { opContinue, opConverge,

opSingular, opError};
Vector Xold;
double denom, sum, dev, devMax;
ant 2, 3,0 Beer 083
enum opType operType = opContinue;

newVect (&Xold, numRows) ;

/* normalize matrix A and vector B */
for (i = 0; i < numRows; i++) {
denom = MAT(A, i, i);
if (denom < eps1)
return matErr_Singular;
B.pData[i] /= denom;
for (j = 0; j < numRows; j++)
MAT(A, i, j) /= denom;
}

/* perform Gauss-Seidel iteration */

while (operType == opContinue) {
for (i = 0; i < nmumRows; i++) {
Xold.pData[i] = X.pData[i];
 Simultaneous Linear Equations

X.pData[i] = 0;
for (j = 0; j < numRows; jtt)
Benya eae a)
X.pData[i] -= MAT(A, i, j) * X.pData[j];
X.pData[i] += B.pData[i];
}

/* check for the convergence */

devMax = fabs(Xold.pData[0] - X.pData[0]) / X.pData[0];
for (i = 1; i < numRows; i++) {
dev = fabs (Xold.pData[i] - X.pData[i]) / X.pData[i];
devMax = (dev > devMax) ? dev : devMax;
}
if (devMax <= eps2)
operType = opConverge;
else {
iter++;
if (iter > maxIter)
operType = opError;
}
}
deleteVect (&Xold) ;

switch(operType) {
case opConverge:
return matErr_None;

case opSingular:
return matErr_Singular;

case opError:
return matErr_IterLimit;

default:
return matErr_None;

}
void LUInverse(Matrix A, Matrix InvA,
IntVector Index, int numRows)

Vector colVect;
sqher chy es

newVect (&colVect, numRows) ;

for (j = 0; j < numRows; jt+) {
for (i = 0; i < numRows; i++)
colVect.pData[i] = 0;
colVect.pData[j] = 1;
LUBackSubst
(A, Index, numRows, colVect) ;
for (i = 0; i < numRows; i++)
MAT(InvA, i, j) = colVect.pData[i];
}
deleteVect (&colVect) ;
}
enum MatErrType MatInverse(Matrix A, int numRows)
{
Vector colVect;
IntVector Index;
teed: ys
int rowSwapFlag;
enum MatErrType err;

newVect (&colVect, numRows) ;

newlIntVect (&Index, numRows) ;
err = LUDecomp(A, Index, numRows, &rowSwapFlag) ;
16 Chapter One

Listing 1.6 (Continued)

if (err != matErr_None)
return err;

for (j, = 0; j < numRows; 3+F) {

For (i1.= 0; i < numRows; i++)
colVect.pData[i] = 0;
colVect. Data[j] = 1;
LUBackSupst
(A, Index, numRows, colVect);
for (i = 0; i < numRows; i++)
MAT(A, i, 3) = GobVect .pData [ia ]>
}

deleteVect (&colVect) ;
deleteIntVect (&Index) ;
return matErr_None;
}
double LUDeterminant (Matrix A, int numRows,
int rowSwapFlag)
{
double result = (double) rowSwapFlag;
ties by

for (i = 0; i < numRows; i++)

result *= MAT(A, 35° 2);

return result;
}
double MatDeterminant (Matrix A, int numRows)
{
IntVector Index;
oha)soe Pes
int rowSwapFlag;
enum MatErrType err;
double result;

newlintVect (&Index, numRows) ;

err = LUDecomp(A, Index, numRows, &rowSwapFlag) ;
if (err != matErr_None)
return MATVECT_BAD RESULT;

result = (double) rowSwapFlag;

for (i = 0; i < numRows; i++)
result, = MAM(A 2,739 5

deleteIntVect (&Index) ;
return result;
}

Listing 1.6 contains the definitions of the functions declared in the MATVECT.H
header files. I will discuss the source code for the function which solves the linear
equations. These functions are:

1. The function GaussJordan implements the Gauss-Jordan method. The function

uses local pointers to create and access dynamic arrays of indices. The function
uses a main for loop to process each column. The main loop contains a nested for
loop which searches for a pivot element and then divides a row by that element.
The main loop then reduces the rows, except the row with the pivot element.
2. The function GaussSeidel implements the Gauss-Seidel method. The function
normalizes the elements of the matrix A and the vector B by dividing them by the
diagonal elements of matrix A. After this step, the function uses a while loop to
 Simultaneous Linear Equations 17

refine the guesses. Each iteration refines the guesses for the variables and then
tests for convergence.
3. The function LUDecomp performs the LU decomposition on matrix A. The func-
tion uses a set of nested loops to obtain the largest matrix element, used for scal-
ing all other matrix elements. The function then uses other nested for loops to
search for pivot elements and interchange rows. The operations transform the in-
put matrix A into an LU matrix.
4. The function LUBackSubst performs the back substitution to solve for a constant
vector. The function uses a for loop for forward substitution, and then uses an-
other for loop for backward substitution.

The C++ Source Code

Let’s look at the C++ source code. Listing 1.7 shows the source code for the
MATVECT.HPP header file. This file declares the class MatrixOp to encapsulate the
various matrix operations. The member functions of class MatrixOp use the classes
Matrix, Vector, and IntVector to represent matrices, vectors, and integer vectors, re-
spectively. The class MatrixOp does not declare any data members and connects with
the classes Matrix, Vector, and IntVector only through the parameters of the member
functions. These member functions parallel the functions in file MATVECT.H.

Listing 1.7 The source code for the MATVECT.HPP header file.
/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 8/9/94

C++ module for basic vect and array operations:

Add matrices
Subtract matrices
Multiply matrices
+t Solve a set of linear equations using
++ the
Gauss-Jordan method
+ Solve a set of linear equations using the
LU decomposition method
+ Solve a set of linear equations using the
Gauss-Seidel method
+ Obtain the inverse of a matrix
+ Obtain the determinant of a matrix
cae

#ifndef _MATVECT_H_
#define _MATVECT_H_

#include "global.h"
#include "“arrays.hpp"

#define MATVECT_EPSILON 1.0e-15

#define MATVECT_BAD RESULT -1.0e+30

class MatrixOp
{
public:
enum MatErrType { matErr_None, matErr_Size, matErr_Singular,
matErr_IllConditioned, matErr_IterLimit };
18 Chapter One

Listing 1.7 (Continued)

void Copy(Matrix& MatA, Matrix&

MatB) ;
MatErrType AddMat (Matrix& aMat,
Matrix& MatB,
int numRows, int numCols) ;
MatErrType SubMat (Matrix& MatA, Matrix& MatB,
int numRows, int numCols);
MatErrType MulMat (Matrix& MatA, Matrix& MatB,
Matrix& MatC,
int numRowsA, int numColsA,
int numRowsB, int numColsB) ;
MatErrType GaussJordan(Matrix& A, Matrix& B,
int numRows, int numCols);
MatErrType LUDecomp(Matrix& A, IntVector& Index,
int numRows, int& rowSwapFlag) ;
void LUBackSubst (Matrix& A, IntVector& Index,
int numRows, Vector& B);
MatErrType GaussSeidel (Matrix& A,
Vector& B, Vectoré& X,
int numRows, int maxIter,
double epsl, double eps2);
void LUInverse(Matrix& A, Matrix& InvA,
IntVector& Index, int numRows) ;
MatErrType MatInverse(Matrix& A, int numRows) ;
double LUDeterminant (Matrix& A, int numRows, int rowSwapFlag) ;
double MatDeterminant (Matrix& A, int numRows);

}F

#endif

Listing 1.8 contains the source code for the MATVECT.CPP implementation file.
The implementation in this listing is similar to that in Listing 1.6. The main differ-
ences are:

1. The member functions use the functions Matrix::operator(), Vector::operator{],

IntVector::operator[] to access the elements of the matrices and vectors. Using
the operators () to access the elements of matrices makes the C++ code more
readable than its C counterpart which uses the macro MAT.
2. The member functions create and remove dynamic arrays using the C++ opera-
tors new and delete.

Listing 1.8 The source code for the MATVECT.CPP implementation file.
/ *

C++ module for basic vector and array operations

ty

#include "matvect.hpp"
#include <math.h>

void swapInt(int& il, int& i2)

{
rite tere
6 tls
Eos ee
mA See
}
void swapDouble(double& dl, double& d2)
{
double t;
6 = ails
 Simultaneous Linear Equations 19

dip sae
d2— ser
}

void MatrixOp: :Copy(Matrix& MatA, Matrix& MatB)

{
int tow, col;
for (row = 0; row < MatA.getRows(); row++)
for (col = 0; col < MatA.getRows(); col++)
MatA(row, col) = MatB(row, col);
}
MatrixOp: :MatErrType MatrixOp: :AddMat (
Matrix& MatA, Matrix& MatB,
int numRows, int numCols)
{
Int, Bow, Col;

if (!(MatA.checkRowCol (numRows, numCols) &&

MatB.checkRowCol (numRows, numCols) &&
numCols != numRows) )
return matErr_Size;

for (row = 0; row < numRows; row++)

for (col = 0; col < numCols; col++)
MatA(row, col) += MatB(row, col);

return matErr_None;
}
MatrixOp::MatErrType MatrixOp: :SubMat (
Matrix& MatA, Matrix& MatB,
int numRows, int numCols)
{
int. SOw, COL.

if (! (MatA.checkRowCol(numRows, numCols) &&

MatB.checkRowCol(numRows, numCols) &&
numCols != numRows) )
return matErr_Size;

for (row = 0; row < numRows; row++)

for (col = 0; col < numCols; coll++)
MatA(row, col) -= MatB(row, col);

return matErr_None;
I
MatrixOp::MatErrType MatrixOp: :MulMat (
Matrix& MatA,
Matrix& MatB,
Matrix& MatcC,
int numRowsA, int numColsA,
int numRowsB, int numColsB)
ni
it. row, Col, k-:

if (! (MatA.checkRowCol (numRowsA, numColsA) &&

MatB.checkRowCol (numRowsA, numColsA) &&
numColsA == numRowsB) )
return matErr_Size;

for (row = 0; row < numRowsB; row++) {

for (col = 0; col < numColsA; col++)
MatA(row, col) = 0;
fom (ie = 02 kee numColsA. ks-2)
MatA(row, stet) += MatB(row, k) * MatC(k, col);
}
return matErr_None;
20 Chapter One

Listing 1.8 (Continued)

MatrixOp::MatErrType MatrixOp: :GaussJordan (

Matrix& A, Matrix& B,
int numRows, int numCols)

int* rowIndex = new int[A.getRows()];

int* colIndex = new int[A.getCols()];
int* pivotIndex = new int[A.getCols()];
Brite, dey sien. Ke, oid tate
Hie Meow, (Coll,
double large, z, oneOverPiv;

if (!A.checkRowCol (numRows, numCols) )

return matErr_Size;

/* initialize the row and column indices */

for (i = 0; i < numRows; i++) {
rowilndex[i] = i;
GoLingdex [aii = 24.
}

/* initialize the pivot index array */

for (1 = 0; i < numRows; it+)
pivotIndex[i] = -1;

for (i = 0% a2 < numRows; i++) {

large = 0;
for (j = 0; j < numRows; j++)
if (pivotIndex[j] != 0)
for (k = 0; k < numRows; k++) {
if (pivotIndex[k] == —-1) {
Le (Eabs(Ay, K)))) s= Large) 4
large = fabs(A(j, k));
How = ae
Col, = ky
}
}
else if (pivotIndex[k] > 0) {
delete [] colIndex;
delete [] rowIndex;
delete [] pivotIndex;
return matErr_Singular;

pivotindex[col] += 1;
if (row != col)
for (n = 0; n < numRows; n++)
swapDouble(A(row, n), A(col, n));
for (n = 0; n < numCols; n++)
swapDouble(B(row, n), B(col, n));
}
rowilndex[i] = row;
colIndex[i] = col:
if (fabs(A( col, seolyjr= 1.e=f0)y 4
delete [] colIndex;
delete [] rowIndex;
delete [] pivotIndex;
return matErr_Singular;
}
oneOverPiv = 1 / A(col, col);
AGoL, col) = dy
for (n = 0; n < numRows; n++)
A(col, n) *= oneOverPiv;
for (n = Or n < numCols; n+)
 Simultaneous Linear Equations 21

B(col, n) *= oneOverPiv;
for (m = 0; m < numRows; m++)
ie (me l= col) 4
Za Ai(m, 1eow)r
Aim, col) = 1s
for (n = 0; n < numRows; n++)
Atm, mis= AlGol, nm) * 2.
Eo (i = 0) an < numColls!> fa)
Bi) — BCC Ole al) ce
}
}
for (n = numRows - 1; n >= 0; n--) {
if (rowIndex[n] != colIndex[n] )
for (k = 0; k < numRows; k++)
swapDouble(A(k, rowIndex[n]),
A(k, colIndex[n]));
}
delete [] colIndex;
delete [] rowlIndex;
delete [] pivotIndex;
return matErr_None;

}
MatrixOp: :MatErrType MatrixOp: :LUDecomp (
Matrix& A,
IntVector& Index,
int numRows, int& rowSwapFlag)

{
The ah Gal Me aliteber
double large, sum, Zz, Z2;
Vector scaleVect (A.getCols());

/* initialize row interchange flag */

rowSwapFlag = 1;
/* loop to obtain the scaling element */
for (a) = 40); <2 <enumRows; Vz++)
targe = 0
for (7) = 0593 < numRows; j++)
22 = fabs(A(i, j));
tanges=— (220 Varge) 7 22.5) Larges
}
/* no non-zero large value? then exit with an error code */
Lt (Large == 0)
return matErr_Singular;
scaleVect[i] = 1 / large;
if
for (ji = 0; 3 < numRows; j++) ¢{
For. (ca: Os ae <i, Asha) ak
sum = A(i, j);
for (k-=.0 Kk <e25 k++)
sume== A(a, ky = Atk, ac
A(i, j) = sum;
}
flange. = 0;
fou {a = 37 2 < numRows; att) 4
Sume= Aa, a)iz
for (k = 07 ki <.95 K++)
SUM pee aCe eek)! eam AK a) 7
Aa ao = sum;
Z = scaleVect[i] * fabs(sum);
if (z >= large) {
large = Zz;
iMax .=—al>
22 Chapter One

Listing
1.8 (Continued)

}
she (3) Wes atuiewe)y Af
for (k = 0; k < numRows; k++) {
Z = A(iMax, k));
AuGi Mas, 1K) = Ant Kylee
A(j, kK) =e
}
rowSwapFlag *= -1;
scaleVect[iMax] = scaleVect[j];
}
Index[j] = iMax;
LE (AG == 10))
A(j, j) = MATVECT_EPSILON;
if (j != numRows) {
bach RIN (aly apie
for (i = j+1; i < numRows; i++)
Diy ey) Lee
}
}
return matErr_None;
}
void MatrixOp: :LUBackSubst (Matrix& A, IntVector& Index,
int numRows, Vectoré& B)
{
ahoh ee yeah oh geal
double sum;

for (i = 0; i < numRows; i++) {

idx = Index[i];
sum = B[idx];
B[idx] = B[il;
if (k > -1)
fox (= ks 4 < eo)
sum —= A(a> 9) Bla;
else if (sum != 0)
Vee a8r
B[i] = sum;
i
for (i = numRows - 1; i >= 0; i--) {
sum B[il;
for (j = i+1; j < numRows; j+t)
sum== Aiea) es “Blade
Bia] = sone as a)
}
}

MatrixOp::MatErrType MatrixOp: :GaussSeideli (

Matrix& A,
Vector& B, Vector& X,
int numRows, int maxIter,
double epsl, double eps2)

enum opType { opContinue, opConverge,

opSingular, opError};
Vector Xold(numRows) ;
double denom, dev, devMax;
Lee, Ny oLeere=.0s
enum opType operType = opContinue;

/* normalize matrix A and vector B */

for (i = 0; i < numRows; i++) {
denom = A(i, i);
if (denom < eps1)
 Simultaneous Linear Equations 23

return matErr_Singular;
B[i] /= denom;
for (j = 0; 3 < numRows; j++)
AiG sess Cenonre
}

/* perform Gauss-Seidel iteration */

while (operType == opContinue) {
for (i = 0; i < numRows; i++) {
Xoldiiat== Xi;
Sl = lO;
for (j = 0; j < numRows; j++)
Le (ah as)
xePat = A, Sa) Lael
le) ees Bale
}

/* check for the convergence */

devMax = fabs(Xold[0] - X[0]) / X[0];
for (2 = i; 2 < numRows; 2+) {
dev = fabsi(Xoldiia.)) — sca) e7) elas
devMax = (dev > devMax) ? dev : devMax;
}
if (devMax <= eps2)
operType = opConverge;
else {
iter++;
if (iter > maxIter)
operType = opError;
}
}

switch(operType) {
case opConverge:
return matErr_None;

case opSingular:
return matErr_Singular;

case opError:
return matErr_IterLimit;
default:
return matErr_None;

void MatrixOp: :LUInverse(Matrix& A, Matrix& InvA,

IntVector& Index, int numRows)
{
Vector colVect (numRows) ;
int i, 3;
for (j = 0; j < numRows; j++) f{
Ors sia 0; i < numRows; i++)
colVect[i] = 0;
colVect[j] = 1;
LUBackSubst (A, Index, numRows, colVect) ;
for (i = 0; i < numRows; i++)
EnvA (i; 95) =<colVect [oaks

}
MatrixOp: :MatErrType MatrixOp: :MatInverse(Matrix& A, int numRows)
{
Vector colVect (numRows) ;
IntVector Index (numRows) ;
PM PLE
 Chapter One

Listing 1.8 (Continued)

int rowSwapFlag;
MatErrType err;

err = LUDecomp(A, Index, numRows, rowSwapFlag) ;

if (err != matErr_None)
return err;

for (Gp = 07 4 < numkows; J++) 1X

for (i = 0; i < numRows; i++)
colvect[i] = 0;
ColVvece|al = se
LUBackSubst
(A, Index, numRows, colVect);
for (i = 0; i < numRows; i++)
Aa, 3) =scolVect fal>
}

return matErr_None;
}
double MatrixOp: :LUDeterminant (Matrix& A, int numRows,
int rowSwapFlag)
{
double result = (double) rowSwapFlag;
aigahe, ok
for (i = 0; i < numRows; i++)
resule *=—A(2 2s
return result;
}
double MatrixOp: :MatDeterminant (Matrix& A, int numRows)
{
IntVector Index (numRows) ;
pO glywe
int rowSwapFlag;
MatErrType err;
double result;

err = LUDecomp(A, Index, numRows, rowSwapFlag) ;

if (err != matErr_None)
return MATVECT_BAD RESULT;

result = (double) rowSwapFlag;

for (i = 0; i < numRows; i++)

resule *] Ai, 2)

return result;

The C Test Program

Let’s look at the C test program TSMAT.C which tests the various functions that solve
linear equations. To compile the test program you need to include the files TSMAT.C,
MATVECT.C, and ARRAYS.C in your project file (which depends on the type of C++
compiler you are using). Listing 1.9 shows the source code for the TSMAT.C program
file. The program performs the following main tasks:

= Creates the matrices A and B, and the vectors B2 and X.

= Assigns values to the matrices A and B.
= Displays the values in matrices A and B in the form of linear equations.
 Simultaneous Linear Equations 25

Invokes the function GaussJordan to solve the linear equations.

Displays the solution vector stored in matrix B.
Initializes the matrix A and the vectors B2 and X to test the Gauss-Seidel method.
Displays the values in matrices A and B2 in the form of linear equations.
Invokes the function GaussSeidel to solve the linear equations. The vector X con-
tains the solution vector.
Displays the solution vector in vector X.
Initializes the matrix A and the vector B2 to test the LU decomposition method.
Displays the values in matrices A and B2 in the form of linear equations.
Invokes the functions LUDecomp and LUBackSubst to solve the linear equations.
The vector B2 contains the solution vector.
Displays the solution vector in vector B2.

Listing 1.9 The source code for the TSMAT.C program file.
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include "global.h”
#include "arrays.h"
#include "matvect.h"

void pressAnyKey (void) ;

int main()
{
Matrix A;
Matrix B;
Vector B2;
Vector x;
IntVector Index;
int numRows = 5;
int numCols = als
int row, col;
int dFlag;
enum MatErrType Err;

newMat (&A, 10, 10);

newMat(&B, 10, 10);
newVect (&B2, 10);
newVect (&X, 10);
newlntVect (&Index, 10);

printf("Testing the Gauss-Jordan method\n") ;

printf£("to solve the following set of equations:\n");
for (row = 0; row < numRows; row++)
for (col = 0; col < numRows; col++)
MARCA, z:ow, <col) = "1;

MAT(A, 1, 4) = =1;
MATA 2), So) = dee
MATCA, Sy, 92) = —i;
MAT(A, 4, 1) = =—1;

MAT(B, 0, 0) = 15;
MAT (Bi, 0) = 5;
MAT(B. 27.0) = Tz
26 Chapter One

Listing 1.9 (Continued)

MAT (Bea 3). 0)! =2 9!

MAT (BA4, 0) = 11;

for (row = 0; row < numRows; row++) {

for (col = 0; col < numRows; col++) {
Bin GOdm > Q))is if
tf. (MAT (A, erow.) Col), >= 10)
Print i setley ct" MAT (Al acow, col), cols. 1).
else
printf(" - lg X%d", -MAT(A, row, col), col + 1);
} else
DELnel(Uslgexsa", eMATI(AVS rowieiCol)s, eGo a1) >
}
primed! S=—eloqNns eMAT(B, eacows® Oi)u)is
}
Err = GaussJordan(A, B, numRows, numCols);
if (Err == matErr_None) {
printf("\nSolution vector is:\n");
for (row = 0; row < numRows; row++)
printf ("X[%d] = %lg\n", row + 1, MAT(B, row, 0));
}
else
printf("Error in solving equations\n") ;

pressAnyKey
() ;

printf("Testing the Gauss-Seidel method\n") ;

printf("to solve the following set of equations:\n");
MAT (Ay 07 0) = LO)
MAT (Al 07. 1)h =e
MAT(A, 0, 2) = 2;

MAT (Aj 1h, 0) e=" he

MAT(A, 1, 1) = 10;
MATA, 2 2c=. 2

MAT. (Ayere,, 0) =a
MAT (Ayer oye eos
MATICAL. 27,5 2)" =s 20

B2.pData[0] = 12;
B2.pData[1] = 12;
B2.pData[2] = 12;

X.pData[0] = 0;
X.pData[1] = 0;
x.pData[(2] = 0;

numRows = 3;
for (row = 0; row < numRows; row++) {
for (col = 0; col < numRows; col++) {
DE (etoile. Oe
if (MAT(A, xow, col)m>=20)
printf£(" + tlg X%d", MAT(A, row, col), col + 1);
else
printf(" - %lg X%d", -MAT(A, row, col), col + 1);
} else
printf("%tlg X%d", MAT(A, row, col), col + 1);
}
printt(" = slg\n", B2.pData[row]);
}
Err = GaussSeidel(A, B2, X, numRows, 50, 0.01, 0.0001);
if (Err == matErr_None) {
 Simultaneous
Linear Equations 27

printf("\nSolution vector is:\n");

for (row = 0; row < numRows; row++)
printf ("X[%d] = %lg\n", row + 1, X.pData[row]);
}
else
printf("Error in solving equations\n") ;

pressAnyKey
() ;

printf("Testing the LU decomposition method\n") ;

printf("to solve the following set of equations:\n");
numRows = 5;
for (row = 0; row < numRows; row++)
for (col = 0; col < numRows; col++)
MAT(A, “row; col) = 1;

MAT(A, 1, 4) = -1;
MAT(AV 2 eS) a= 1;
MAT (AG 93); -2) = SL
MAT(A, 4, 1) = -1;
B2.pData[0] = 15;
B2.pData[1] = 5;
B2.pData[2] = 7;
B2.pData[3] = 9;
B2.pData[4] = 11;

for (row = 0; row < numRows; rowt+) {

for (col = 0; col < numRows; col++) {
aie (COA a0) at
Le se(MAT(A prow eiGoL) >= (0)
print£(" + $lg X%d", MAT(A, row, col), col: 1);
else
printf(" - %tlg X%d", -MAT(A, row, col), col + 1);
} else
DLrint£l( "Sig xXtdepeMAT (Ay row, col); col +1);
}
printf(" = %lg\n", B2.pData[row]
);
}
Err = LUDecomp(A, Index, numRows, &dFlag) ;
if (Err == matErr_None) {
LUBackSubst (A, Index, numRows, B2);
printf("\nSolution vector is:\n");
for (row = 0; row < numRows; row+t+)
printf("X[%d] = %lg\n", row + 1, B2.pData[row]);
}
else
printf("Error in solving equations\n") ;

pressAnyKey
() ;

deleteMat
(&A) ;
deleteMat (&B) ;
deleteVect (&B2) ;
deleteVect (&X) ;
deleteIntVect (&Index) ;
return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
puts("\n\n") ;
}
28 Chapter One

Figure 1.1 shows the output of program TSMAT.EXE.

Testing the Gauss-Jordan method

to solve the following set of equations:
tO dilee = X2 xs 1 x4 1x5 15
X1 X2 1eexS x4 x5
X1 X2 LEX x4 X5
X1 X2 1 x3 x4 X5
X1 X2 xs x4 X5 HT
He

Solution vector is:

X[1] af
X[2]
X[3]
X[4]
X[5] i
nin
fe

Press any key to continue...

Testing the Gauss-Seidel method

to solve the following set of equations:
WO) si HES Ser tne See ie} 2
exit 0) Xo eS 12
eX XO Fo On xs 12

Solution vector is:

X[1]
x2
xsl

Press any key to continue...

Testing the LU decomposition method

to solve the following set of equations:
iE eile es X2 i @celwen ib pes X5 LS
‘besa X2 X3 + x4 x5
xe X2 x3 = x4 X5
pal X2 X3 + x4 x5 i

1 x1 X2 X3 + x4 x5

Solution vector is:

X[1] 1
X[2]
X[3]
X[4]
X[5] Figure 1.1 The output of program
TSMAT.EXE.
Press any key to continue...

The C++ Test Program

Listing 1.10 shows the source code for the TSMAT.CPP program file. This program is
the C++ version of TSMAT.C. To compile the program you need to include the files
TSMAT.CPP and MATVECT.CPP in your project files. The C++ program performs
the same tasks and produces the same output as its C counterpart. The main differ-
ence between the two versions is that Listing 1.10 uses the classes Matrix, Vector,
and IntVector to create matrices, vectors, and integer vectors. Since each one of
these classes has a constructor and a destructor, the C++ program creates and re-
moves these arrays using fewer statements than the C version. Moreover, the C++
program uses the operator () to access the elements of the matrices and vectors.
 Simultaneous Linear Equations 29

Listing 1.10 The source code for the TSMAT.CPP program file.

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include "global.h"
#include "matvect.hpp"

void pressAnyKey (void) ;

int main()
{
Matrix MatA(10, 10);
Matrix B(10, 10);
Vector B2(10);
Vector x (0);
MatrixOp MatOp;
IntVector Index(10) ;
int numRows = 5Sir
Ties TUIMC ols) = lke
Mee GOW, eCOle.
int dFlag;
MatrixOp::MatErrType Err;

printf ("Testing the Gauss-Jordan method\n") ;

printf("to solve the following set of equations: \n");
Mata(0>—0) = 1;
MatA(0, 1) i r
Maen(0, 2)- = ,
MatA(0, 3) = ,
MatA(0, 4) = PRPRPPR

MatA(1,
MatA(1, |
MatA(1,
MatA(1, PREP
MatA(1, oO iiT} | h~
BPWNYP

MatA(2,
MatA(2,
MatA(2, nou PRR
MatA(2,
MatA(2, BWNRO

je) ii} ey

ic ia) Be > is W |

I
PPP

for (row = 0; row < numRows; row++) {

for (col = 0; col < numRows; colt++) {
dete Wen b= 60))s wf
if (MatA(row, col) >= 0)
printf(" + lg X%d", MatA(row, col), col + 1);
30 Chapter One

Listing 1.10 (Continued)

else
printf£(" - lg X%d", -MatA(row, col), col + 1);
} else
print£("%lg X%d", MatA(row, col), col + 1);
}
printf(" = lg\n", B(row, 0));
}
Err = MatOp.GaussJordan(MatA, B, numRows, numCols);
Vf (Exr == MatrixOp::matErr_None) {
printf("\nSolution vector is:\n");
for (row = 0; row < numRows; row++)
PLrIntl ("X($d) = slg\n";, row + 1; Birow, 0))'>
}
else
printf("Error in solving equations\n");

pressAnyKey
() ;

printf ("Testing the Gauss-Seidel method\n") ;

printf("to solve the following set of equations:\n");
MatA(0, 0) = 10;
MatA(0, 1) = 1;
Mata(0, 2) = 1;

MatA(1, 0) = 1;
MatA(1, 1) = 10;
MataA(i, 2) = 1;

Mata(2, 0) = 1;
Mata (2) d)e=s te
MatA(2, 2) = 10;

B2(0) = 12;
Bo eae Le
Phe) le

0) ee
Xi(L y= 0
Ki(2)) ans

numRows = 3;
for (row = 0; row < numRows; row++) {
for (col = 0; col < numRows; col++) {
I £e (cole > 10)/ 26
if (MatA(row, col) >= 0)
print£(" + %lg xX%d", MatA(row, col), col + 1);
else
printf(" - %*lg X%d", -MatA(row, col), col + 1);
} else
printf("%lg X%d", MatA(row, col), col + 1);
}
printf£(" = $lg\n", B2 (row)
);
}
Err = MatOp.GaussSeidel (MatA, B2, X, numRows, 50, 0.01, 0.0001);
if (Err == MatrixOp::matErr_None) {
printf("\nSolution vector is:\n");
for (row = 0; row < numRows; row++)
printf("X(%d) = lg\n", row + 1, X(row));
}
else
printf("Error in solving equations\n") ;

pressAnyKey
() ;

printf ("Testing the LU decomposition method\n") ;

printf("to solve the following set of equations:\n");
 Simultaneous Linear Equations 31

numRows = 5
MatA(0, 0) 1;
Mata((0), 1) = 1;
MatA(O 2 = 1
MatA(O) 3)" 1.
MatA(0, 4) 1;

Matai 0s) ds
MatA(l, i) =.1;
Matat(l, 2) = 1;
MataA(l, 3) = iL;
MatA(l, 4) = -1;

MatA(2, 0) = 1;
MatAi2) 1) -=" 1.
MatA(2) 2) ees le
MatA(2, 3) = -1;
Mata (2), <4) = as

Matas). 0) = 1;
MatA(3; 1) = 1
MatA(3, 2) = -1;
Matas; 3) = i
Mata(3, 4) = 1;

MatA(4, 0) = 1;
MatA(47 2) = =i;
MatA(4, 2) = 1;
MatA(4, 3) = 1;
MatA(4, 4) = 1;

B2(0) = 15%
ZAG Ly) es Or
iF) Se TG
B2i(3')) = 9
B24) = 115

for (row = 0; row < numRows; row+t+) {

for (col = 0; col < numRows; col++) {
TE (Coles s0) at
if(MatA(row, col) >= 0)
printf(" + lg X%d", MatA(row, col), col + 1);
else
printf(" - lg X%d", -MatA(row, col), col + 1);
} else
printf ("%lg X%d", MatA(row, col), col + 1);
}
Printi(. = slg\n", B2 (cow)
)
}
Err = MatOp.LUDecomp(MatA, Index, numRows, dFlag) ;
if (Err == MatrixOp::matErr_None) {
MatOp.LUBackSubst (MatA, Index, numRows, B2);
printf("\nSolution vector is:\n");
for (row = 0; row < numRows; row++)
printf ("X(%d) = %lg\n", row + 1, B2[row]);
}
else
printf("Error in solving equations\n") ;
pressAnyKey
() ;
return 0;
}
void pressAnyKey ()
{
printf("\nPress any key to continue...");
getchar();
puts("\n\n");
}
(hers saedi, eucenuiuret>

, . . >
~~ + eRe
7 k ©. 16) Slap’
of if i@ AI
s & _ Sw we) Clase
7 7 “ee )F @ook. A) ae
tu, ee Vela yve= © view
{
oe P Vince 21 ie 40 chee
: A =D Eka
are 4 MiG tite Tin © Seabee, pees re eae
‘fl of e me 4e.2>7 So= ‘ if * tL: .leae
i © ‘ Sie ueror pes wh, =~ ‘oe
fers lye * re © SrA: Shee . :
A ema "~ te a RM
= ww ~ 1. Siem nme =mh
ie
« : . oo
: ote be wah
pepe ey fy weit eyinticeaie > (F)
enmee pi PH of —, :
3
errs = (tg he wure-Srghe] atta"
erian s ame ee Lalbootag Wel et Otis
. ve i@a@
> &
= 4 es :

. : a
@wWs \
4 i= —

) ewes sonueit» Seaver

ie+ los ( eeetias MP ae
<
— 10 ae Shee ort ATONT
+ (Dee .worjado@ "ney ele © Sas San

‘ * Gon Setar ere oF - “Siang ne

wl
j« 7 > werlarat .' Bee soe tani
h oe Gy nite +YeanF

» ie Pos Gra atk tt erent) hwonddits yc ees @

) twice Det lice «o a
ts + er ert Alt¢) p=2eto ast), ce
‘fees sotows “ST ipiodad* sige
ww mw EReuen > eet _ en
Le B a ‘a Lp a (ep) biA) yng *

ad. 208 de verte ine 82. Jones") Sone

 Chapter

Solving Nonlinear Equations

This chapter looks at a group of popular algorithms involved in solving for the roots
of single and multiple nonlinear equations. Most of the methods presented in this
chapter solve for a single real root of a single nonlinear function. The chapter does
include methods which solve for the multiple roots of a nonlinear function and a
polynomial, as well as solve for the real roots of multiple nonlinear equations. The
chapter discusses the following topics:

= Overview of methods
= The Bisection method
=» Newton’s method
# The Richmond method
=» The combined method
= Brent’s method
# The deflating polynomial method
= The Lin-Bairstow method
=» Newton’s method for solving two nonlinear functions
=» Newton’s method for solving multiple nonlinear functions

This chapter discusses various methods that solves for the roots of a diverse class
of functions. The simplest case is the one which deals with finding the single root of
a single nonlinear function. The following equation represents the general form of
such a function:
fea).=:0 (2.1)

33
34 Chapter Two

Equation 2.1 represents a single nonlinear function with one variable. The first set of
methods presented in this chapter solve for the value or values of x which conform
to equation 2.1. The implementation of the C functions in this chapter allow for func-
tion f(x) to have an array of fixed parameters. Thus, the C functions solve for the fol-
lowing form of function f:

f(x, p) = 0 (2.2)
The symbol p represents an array of one or more parameters which remain fixed for
a particular case, yet vary from one case to another. This approach allows you to use
the C functions to solve a wide range of mathematical functions, including polynomials.
The chapter also presents methods which are dedicated in solving for the roots of
polynomials. In addition, the chapter presents methods which solve for two or more
nonlinear equations. The method which solves for the two roots of two nonlinear
equations uses the following forms:

F(x, y) =0 (2.3)

G(x, y) =0 (2.4)
The method solves for the values of variables x and y which fulfill equations 2.3
and 2.4.

The Bisection Method

The Bisection method solves for the root of a function f(x) by selecting an initial in-
terval which contains the root and then narrowing that interval down to the desired
tolerance level. Here is the algorithm for the Bisection method:
Given: the interval [A, B] which contains the root of function f(x). Also given the
tolerance T which is used to specify the width of the final root-containing interval.

1 Repeat the following steps until |A — BI < T:

1.1 Set guess G = Gee

1.2 If f(G) * f(B) > 0, set B = G, otherwise set A = G.

2 Return the guess as G or as Sail

The above algorithm shows the simplicity of the Bisection method. The Bisection
method is a slow-converging method, but it is not affected by the slope of the func-
tion f(x), as is the case with many of the methods presented in this chapter. Conse-
quently, the Bisection method is slow yet reliable! Figure 2.1 depicts the guesses for
the root in the Bisection method.

Newton’s Method

The most popular method for solving for a single root of a single function is Newton’s
method. This method is, in general, quite suitable for its convergence and computa-
tion efforts. Here is the algorithm for Newton’s method:
 Solving Nonlinear Equations 35

Figure 2.1. The quesses for the root in the bisection method.

Given: the initial guess x for the root of function f(x). Also given: the tolerance T
which specifies the minimum guess refinement which ends the iteration.

1 Calculate guess refinement

Go
>= 0)’
where f'(x) is the first derivative of function f(x).
2 Refine guess x by setting x = x —D.
3 If|D| > T then resume at step 1. Otherwise, return x as the refined guess for the
root. See Figure 2.2 on next page.

Newton’s method requires the calculation (or estimation) of the first derivative of
the function. The above algorithm assumes that you can calculate the derivative
without extensive computational overhead. Here is the algorithm for the version of
Newton’s method which estimates the derivative using the function f(x):
Given: the initial guess x for the root of function f(x). Also given: the tolerance T
which specifies the minimum guess refinement which ends the iteration.

1 If Ix! > 1, seth = 0.01 * x. Otherwise, set h = 0.01.

(f(x+h) — f(x-h))
2 Calculate guess refinement D =
(2*h*fi)
3 Refine guess x by setting x = x — D.
4 If IDI > T then resume at step 1. Otherwise, return x as the refined guess for the
root.
36 Chapter Two

Figure 2.2. The successive quesses for the root in Newton's method.

The previous algorithm uses a small increment for the current guess to estimate
the slope of the function at the guess.

The Richmond Method

The Richmond method has a higher convergence order than Newton’s method. How-
ever, this advantage does come at a price—the Richmond method requires the first
and second derivatives of the function f(x). As with Newton’s method you can im-
plement the algorithm for the Richmond method using either a direct evaluation of
the function’s derivatives or using numerical approximations of the function’s deriv-
atives. Here is the algorithm for the Richmond method:
Given: the initial guess x for the root of function f(x). Also given: the tolerance T
which specifies the minimum guess refinement which ends the iteration.

1 Calculate guess refinement D =

f(x) * fi) where f'(x) and
Ce? =05 +i) tf")
f"'(x) are the first and second derivatives of function f(x), respectively .
2 Refine guess x by setting x = x —D.
3 If|D| > T then resume at step 1. Otherwise, return x as the refined guess for the
root.

Here is the algorithm which uses the approximations to the first and second de-
rivatives:
 Solving Nonlinear Equations 37

Given: the initial guess x for the root of function f(x). Also given: the tolerance T
which specifies the minimum guess refinement that ends the iteration.

1 If|Ixl > 1, set h= 0.01 * x. Otherwise, set h = 0.01.

D Set id] = es
ee ee A
2*h
3 Set fa2 = (f(x+ h) -2 ee eae

f(x) * fdl :
4 Calculate guess refinement D =
(fd1? — 0.5 * f(x) * fd2)
5 Refine guess x by setting x = x — D.
6 If|DI > T then resume at step 1. Otherwise, return x as the refined guess for the
root.

Use the above algorithm if calculating the derivatives is more time consuming than
calculating the values of the function at x, x + h, and x—h.

The Combined Method

It is possible to devise a method which combines different basic root-seeking algo-

rithms. For example, you can combine the Bisection and the Newton methods to
take advantage of the strong points of both methods. Such a combination works well
when you are dealing with a function that has a small slope in the vicinity of the root.
Here is the algorithm for the Combined method:
Given: the interval [A, B] which contains the root of function f(x), the initial guess
x, and the tolerance T.

1 Refine guess x using Newton’s method.

2 If refined guess is not in interval [A, B] perform steps 3 and 4.
3 Calculate a new refined guess using the Bisection method and the interval [A, B].
4 Shrink the interval [A, B].
5 If guess refinement > T, resume at step 1. Otherwise return x as the sought root.

I developed the above algorithm in 1980. While researching the algorithms for this
book I found Brent’s method, which uses a similar approach.

Newton’s Multiroot Method

Newton’s method essentially solves for one root. It is possible to incorporate this
method (or any other root-solving method) in a more sophisticated algorithm which
obtains all of the real roots of a function. I developed this algorithm using a pseudo-
deflation technique. This technique avoids zooming in on roots already obtained by
dividing the function f(x) by the product of x minus a previously determined root.
Consequently, the modified function steers away from previous roots as they be-
come points in which the modified function possesses infinite values. Since we are
dealing with discontinuities in the modified function, use this method with care and
38 Chapter Two

test it before applying it to the kind of functions you want to solve. Another caveat is
the fact that the accuracy of the roots suffers as you obtain more roots. If your initial
tolerance is small enough, you will most likely obtain roots with acceptable accuracy.
Here is the algorithm for the Multiple-root method:
Given: the function f(x), the maximum number of roots N, the maximum number
of iterations M, the tolerance factor T, the array of roots R, and the root counter
Nroot.

ie Setec— hi).
2 Set iter = 0.
3 Set h=0.01 * x if lxl > 1. Otherwise, set h = 0.01.
4 Set fl = f(x), f2 =f(x + h), and f3 = f(x—h).
5 If Nroot > 1, perform step 7.
6 For i=0 to Nroot — 1, divide fl, f2, and f3 by (x - R@)).
& artnet
( Calculate root refinement as D = (f2 — £3)

8 Update root x using x = x —D.

9 Increment iter.
10 If iter <M and IDI > T, resume at step 1.
11 If IDI < T perform the following steps:
11.1 Store new root using R(NRoot) = x
11.2 Increment NRoot
11.3 Update guess x to search for the next root.
12 Ifiter <= M AND Nroot < N, resume at step 1.
13 Return the roots in array R.

Deflating Polynomial Method

The previous algorithms which I presented make no assumptions about the form of
function f(x). This section handles the case of applying Newton’s method to solving
the real roots of polynomials with real coefficients. The method is also known as the
Birge-Vieta method and incorporates Newton’s method with a polynomial deflating
step to create the reduced polynomial. In other words, the method solves for the var-
ious roots in cycles. The initial cycle solves for a root of the polynomial you specify.
The next cycle solves for the root of the polynomial after the reduction of its original
polynomial. The subsequent cycles repeat the same tasks. Here is the algorithm for
the Birge-Vieta method:
Given: a polynomial of order N and with coefficients A[0] through A[n — 1], the
maximum iteration limit M, the initial root guess x, and the array of roots Root.

1 Set iter = 0 and NRoot = 0.

2 Repeat the next steps while iter < Mand N > 1.
3 Increment iter.
4 Setz=x.
5 Set B[n- 1] = A[n—-1] and C[n—- 1] = A[n—-1.
6 Fori=n-—2 down to 0 repeat the following:
 Solving Nonlinear Equations 39

6.1 Set Bi] = Afi] + z * Bfi+ 1]

6.2 Set C[i] = B[i] + z * C[i + 1].
7 Set B[O] = A[0] + z * B[1].
8 Set
B[O]
D = ——-:
ae Gi)
9 Setx=x-—-D.
10 If |D! <= T perform the following:
10.1 Set iter = 1
10.2 Decrement N
10.3 Set Root(Nroot) = x
10.4 Increment Nroot
10.5 Fori=0 to N—-1 set A[i] = B[i+ 1]
10.6 If N is equal to 2 perform the following tasks:
10.6.1 Decrement N
10.6.2 Set Root(NRoot) = — A[0]
10.6.3 Increment Nroot.
11 Return the results in array Root and counter NRoot.

The Lin-Bairstow Method

The deflation polynomial method solves for only the real roots of a polynomial. A
method developed by Bairstow and refined by Lin is able to solve for all real and
complex roots of a polynomial with real coefficients. The basic approach works as
follows: the method iteratively determines the coefficients of a quadratic polynomial
to be factored out of the main polynomial. The method then easily calculates the roots
of the quadratic polynomial. Next, the method repeats the above general steps with
the deflated polynomials until all the roots are obtained. Here is the algorithm for the
Lin-Bairstow method:
Given: a polynomial with order N, the polynomial coefficients A[1] to A[N+1} where
A[1] is the coefficient associated with the term that has the highest power, the toler-
ance factor T, and the array of roots R (R has the fields real, image, and .isComplex).

1 If A[1] is not 1, divide coefficients A[N + 1] down to A[1] by A[1].

2 Set root counter Nroot to 0.
3 Repeat the next steps until N < 2.
3.1 Set Al =1 and Bl=1
dla) ySet B[O] = 0,-DiO} = 0, B[1] = 1, and D[1] =1
3.1.2 Fori=2toN + 1 repeat the following steps:
3.1.2.1 Set Bi] = Afi] — Al * Bfi- 1]- Bl * Bfi- 2]
3.1.2.2 Set D[i] = Bi] — Al * Dfi- 1] -—B1 * Dfi—2];
3.1.3 Set D1 = D[N - 1]? — (D[N] — BIN]) * D[N - 2]
(B[N] * D[N — 1] — B[N + 1] * D[N - 2})
3.1.4 Set A2 =
D1
(BIN + 1] * DIN - 1] — D[y] « Bin] —- B[N]?))
3.1.5 Set B2 =
D1
40 Chapter Two

3.1.6 Add A2 to Al
3.1.7 Add B2 to Bl
3.2 If |A2| > T or |B2I > T resume at step 3.1.1
3.3 SetDl = Al * Al—4*Bl
3.4 If D1 < 0 then calculate the following:
SQRTCID1I)
3.4.1 Set D2 =
2
3.4.2 Set D3 = =

3.4.3 Set Root(Nroot).isComplex to true

3.4.4 Set Root(Nroot).real = D8
3.4.5 Set Root(Nroot).imag = D2
3.4.6 Set Root(Nroot+1).isComplex to true
3.4.7 Set Root(Nroot+1).real = D3
3.4.8 Set Root(Nroot+1).imag = —D2
3.5 If D1 >= 0 then calculate the following:
3.5.1 Set Root(Nroot).isComplex to false

3.5.2 Set Root(Nroot).real = ae

3.5.3 Set Root(Nroot).imag = 0

3.5.4 Set Root(Nroot+1).isComplex to false
—(D2 + Al)
3.5.5 Set Root(Nroot+1).real =
2
3.5.6 Set Root(Nroot+1).imag = 0
3.6 Add 2 to Nroot
3.7 Decrement N by 2
3.8 If N >= 2 then set Afi] = B[i] fori=1toN+1.
4 If N is 1 perform the following tasks:
4.1 Set Root(Nroot).isComplex to false
4.2 Set Root(Nroot).real = —B[2]
4.3 Set Root(Nroot).image = 0.

Solving Multiple Nonlinear Equations

You can extend Newton’s method to solve two or more equations. In the case of 3 or
more equations you need to use the routines which solve for linear equations. These
routines allow you to refine the guesses for the multiple roots. Here are the algo-
rithms for solving two nonlinear equations F(x, y) = 0 and G(x, y) = 0:
Given: the equations F(x, y) = 0, and G(x, y) = 0, the initial guesses x and y, and
the tolerance factor T.

Set F = F(x, y).

Set G = G(x, y).
Calculate the increments hx and hy for x and y, respectively.
Set Fx = approximate derivative of function F(x, y) at x.
CO
OP
We Set Fy = approximate derivative of function F(x, y) at y.
 Solving Nonlinear Equations 41

Set Gx = approximate derivative of function G(x, y) at x.

Set Gy = approximate derivative of function G(x, y) at y.
Set Jacobian matrix J = Fx * Gy — Fy * Gx.
(F « Gy -—G * Fy)
co
&
CONSet refinement for x, diffX =
4
(G * Fx — F * Gx)
10 Set refinement for y, diffY =
J
11 Set x = x - diffX and y = y — diffY.
12 If IdiffX! > T and IdiffYl > T resume at step 1.
13 Return the roots x and y for the functions F(x, y) and G(x, y).

Newton’s Method for Multiple Equations

You can expand the last algorithm to solve for the multiple real roots of multiple non-
linear equations. Solving the roots of two functions is a special and simple case of
solving for the roots of multiple equations. In the case of three or more nonlinear
functions, you proceed in using C functions that solve for simultaneous linear equa-
tions. These C functions assist in calculating the refinements for the roots’ guesses
in each iteration. Here is the algorithm for Newton’s method:
Given: N nonlinear functions F'(x), where x is an array of variables, and the toler-
ance factor T.

1 Copy the values of array x into array x0.

2 Store the values of functions F(x) in array FO.
3 Calculate the matrix, M, of approximate derivates for each function and for each
variable.
4 Solve for the simultaneous linear equations M D = FO, where D is the array of
guess refinements.
5 Update the array x using the values in array D.
6 If any absolute value in array D exceeds the value of T, resume at step 1.
7 Return the roots in array x.

The C Functions
After presenting the algorithms for the various root-seeking methods, let me offer
you the C source code which implements these methods. Listing 2.1 shows the source
code for the ROOT.H header file. Listing 2.2 shows the source code for the ROOT.C
implementation file.

Listing 2.1 The source code for the ROOT.H header file.

#ifndef _ROOT_H_
#define _ROOT_H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/3/94

Module: C root-finding routines which support the following

42 Chapter Two

Listing 2.1 (Continued)

general methods for a single real function:

Bisection method
Tangent method
Newton's method
Richmond's method
Brent's method
Combined method (Bisection and Newton)
Lin-Bairstow method for polynomials
Newton's method for two equations
eeeeetst
++ Newton's method for multiple equations

a)
#include "global.h"
#include "matvect.h"

#define ROOT_EPS 1.0e-30

typedef struct polyRootTag {

double real;
double imag;
Boolean .isComplex;
} polyRoot;

int Bisection(double low, double high, double tolerance,

double* root, double* params,
double (*f) (double, double*));

int NewtonApprox(double *root, double tolerance,

int maxIter, double* params,
double (*f) (double, double*));

int NewtonExact (double *root, double tolerance,

int maxIter, double* params,
double (*f) (double, double*),
double (*fderiv) (double, double*));

int RichmondApprox(double *root, double tolerance,

int maxIter, double* params,
double (*f) (double, double*));

int RichmondExact (double *root, double tolerance,

int maxIter, double* params,
double (*f£) (double, double*),
double (*flstDeriv) (double, double*),
double (*f2ndDeriv) (double, double*));

int Combined(double low, double high, double tolerance,

double* root, int maxIter, double* params,
double (*f) (double, double*));

int Brent (double low, double high, double tolerance,

double* root, int maxIter, double* params,
double (*f) (double, double*));

int NewtonMultiRoots(double *roots, int* numRoots, int maxRoots,

double tolerance, int maxIter, double* params,
double (*f) (double, double’));

int DeflatePolyRoots(double* coeff, double initGuess,

double* roots, int* numRoots,
int polyOrder, int maxIter,
double tolerance) ;

int LBPolyRoots(double* coeff, polyRoot* roots,

int polyOrder, double tolerance) ;
 Solving Nonlinear Equations 43

int Newton2Functions(double *rootX, double* rooty,

double tolerance, int maxIter,
double* paramsxX, double* paramsY,
double (*fx) (double, double, double*),
double (*fy) (double, double, double*));

int NewtonSimNLE(Vector X, int numEqns, double tolerance,

int maxIter, double (*f[]) (double*));

#endif

Listing 2.1 declares the structure polyRoot to support solving the real and com-
plex roots of polynomials using the function LBPolyRoots. The header file declares
the following C functions:

The function Bisection implements the Bisection method. The function has para-
meters which allow you to specify the root-containing interval, the tolerance fac-
tor, the refined root guess, the array of parameters mentioned in equation 2.1, and
the pointer to the solved function.
The function NewtonApprox implements the version of Newton’s method which
approximates the first derivative. The function NewtonApprox has parameters
which specify the initial guess for the root (and also report the final guess refine-
ment), the tolerance factor, the maximum number of iterations, the parameters of
the solved function, and the pointer to the solved function.
The function NewtonExact implements the version of Newton’s method which
uses the derivative of the solved function. The function NewtonExact has parame-
ters which specify the initial guess for the root (and also report the final guess re-
finement), the tolerance factor, the maximum number of iterations, the parameters
of the solved function, the pointer to the solved function, and the pointer to the
first derivative of the solved function.
The function RichmondApprox implements the version of Richmond’s method
which approximates the first and second derivatives. The function Richmond-
Approx has parameters which specify the initial guess for the root (and also re-
port the final guess refinement), the tolerance factor, the maximum number of
iterations, the parameters of the solved function, and the pointer to the solved
function.
The function RichmondExact implements the version of Richmond's method which
uses the derivatives of the solved function. The function RichmondExact has pa-
rameters which specify the initial guess for the root (and also report the final guess
refinement), the tolerance factor, the maximum number of iterations, the parame-
ters of the solved function, the pointer to the solved function, the pointer to the
first derivative of the solved function, and the pointer to the second derivative of
the solved function.
The function Combined implements the Combined method. The function Combined
has parameters which specify the root-containing range, the initial guess for the
root (and also report the final guess refinement), the tolerance factor, the maxi-
mum number of iterations, the parameters of the solved function, and the pointer
to the solved function.
44 Chapter Two

The function Brent implements Brent’s method. The function has parameters which
specify the root-containing range, the initial guess for the root (and also report the
final guess refinement), the tolerance factor, the maximum number of iterations,
the parameters of the solved function, and the pointer to the solved function.
The function NewtonMultiRoots implements the method that uses Newton's algo-
rithm to solve for the multiple roots of a function. The function NewtonMultiRoot
has parameters which reports the calculated roots, report the number of roots
found, specify the maximum roots to find, specify the tolerance factor, specify the
maximum number of iterations, specify the parameters of the solved function, and
supply the pointer to the solved function.
The functionDeflatePolyRoots implements the deflating polynomial method (a.k.a
the Birge-Vieta method). The function has parameters which pass the roots of the
polynomial, the initial guess, the pointer to the array of roots, the pointer to the
number of roots, the polynomial order, the maximum iterations, and the tolerance
factor.
The function LPPolyRoots implements the Lin-Bairstow method. The function has
parameters which include the pointer to the array of polynomial coefficient, pointer
to the array of polyRoot structures (which report the results back to the function
caller), the polynomial order, and the tolerance factor. The arrays of the coeffi-
cients and solved roots must have a number of elements which exceeds the value
of the polynomial order by at least 1.
The function Newton2Functions solves for the two roots of two nonlinear equa-
tions, using a special version of Newton’s method. The function has parameters
which supply the initial guess for the variable x and y (and also report the latest
guess refinement), the tolerance factor, the maximum number of iterations, the
parameters for function F(x, y), the parameters for function G(x, y), the pointer
to function F(x, y), and the pointer to function G(x, y).
The function NewtonSimNLE implements the method which solves for multiple
nonlinear equations. The function has parameters which supply the array of initial
guesses (and obtain the sought roots), the number of nonlinear equations, the tol-
erance factor, the maximum number of iterations, and the pointer to the array of
solved functions.

The above functions return a TRUE or FALSE value (defined in the header file
GLOBAL.H) to reflect the success or failure of their operations.
Listing 2.2 shows the source code for the ROOT.C implementation file.

Listing 2.2 The source code for the ROOT.C implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "root.h"

maetine SIGN(2) (((a), == 0" oe 1 sw)

int Bisection(double low, double high, double tolerance,

double* root, double* params, -
 Solving Nonlinear Equations 45

double (*f) (double, double%*))

LEO (*#E)) (low, params) * \(*£) (nigh; =parans) "> 0)

return FALSE;

while (fabs(high - low) > tolerance) {

/* update guess */
*root = (low + high) / 2);
1£= ((*£) (*xoot,,params)) * (*£) (high, params) > 0)
hrgh==" ~7oo0te
else
Lowl= *rooit>
}
return TRUE;
}

int NewtonApprox(double *root, double tolerance,

int maxIter, double* params,
double (*f) (double, double’) )

int, stere= 0:
double h, diff;

do {
hee (fabs(*root) > 1) AO alse Poot p00 de
/* calculate guess refinement */
Gutt —— 2 he (ae (*rootm. params | //
((*£) (*root + h, params) - (*f£) (*root - h, params)
);
/* update guess */
*TFook —— waist.
iter++;
} while (iter <= maxIter && fabs(diff) > tolerance) ;
return (fabs(diff) <= tolerance) ? TRUE : FALSE;
}
int NewtonExact (double *root, double tolerance,
int maxIter, double* params,
double (*f£) (double, double*),
double (*fDeriv) (double, double*) )

int Leer = 0!
double diff;

do {
/* calculate guess refinement */
Gilt Sab root,sparams)! / -(*fbexriw) (*root, parame);
/* update guess */
*root -= diff;
itert++;
} while (iter <= maxIter && fabs(diff) > tolerance) ;
return (fabs(diff) <= tolerance) ? TRUE : FALSE;
}
int RichmondApprox(double *root, double tolerance,
int maxIter, double* params,
double (*f) (double, double*) )
{
‘te eer =. 10
Goulbilteshhy eGikttemmclee pees ec ca

dou
he (RAO SesOOt) aa laa sO Ou a StcOotea: (OL Oil:
le (ae COO sa line claceunsSu)ine [uel Ga lat) = 37)
£2) = (*£)\(*Loot, params); [ah nis (bre) rad
£3 = (*£) (*root + h, params); fA EASE 27)
iSULN =e (Bins je Maatyy 0/0 a)Je J dsARS) ees)
COZ se Geen Zee Ga ere IL) SOR Gin) in fie te (5)
46 Chapter Two

Listing 2.2 (Continued)

/* calculate guess refinement */

dure = £1 * sfdl o/s (SOR(EAL)s = 050 setL =e Lda)
/* update guess */
*root -= diff;
iter++;
} while (iter <= maxIter && fabs(diff) > tolerance);
return (fabs(diff) <= tolerance) ? TRUE : FALSE;
}
int RichmondExact (double *root, double tolerance,
int maxIter, double* params,
double (*f) (double, double’),
double (*flstDeriv) (double, double*),
double (*f2ndDeriv) (double, double*) )
{
shoe pmetehe (0)
double Gifttiyeti, tal, =td2);

do {
£I= 9(F2£) a(t, spaxrams))i
fdl = (*flstDeriv) (*root, params) ;
fd2 = (*f2ndDeriv) (*root, params) ;
/* calculate guess refinement */
Gift = fies std (SOR (ECL) anOmS em fleet clo)
/* update guess */
*root == ditt.
iter++;
} while (iter <= maxIter && fabs(diff) > tolerance) ;
return (fabs(diff) <= tolerance) ? TRUE : FALSE;
}
int Combined(double low, double high, double tolerance,
double* root, int maxIter, double* params,
double (*f) (double, double*) )
{
int tter = 0);
double h, diff;

do {
iter++;
bh =) (fabs (*root)e> 1) 20.01 Froote<s OL OL e
/* calculate guess refinement */
aLff = 2° * he* \(*£)\(sncot params) ay
((*£) (*root + h, params) - (*£) (*root - h, params)
);
*root -= diff;
/* check if Newton's method yields a refined guess
outside the range [low, high] */
if (*root) < How’ || > *noot = uhi'gh)y {
/* apply Bisection method for this iteration */
*root = (low + high) / 2;
if ((*f£)(*root, params) * (*f) (high, params) > 0)
high = *root:
else
low = *root;
}
} while (iter <= maxIter && fabs(diff) > tolerance) ;
return (fabs(diff) <= tolerance) ? TRUE : FALSE;
}
int Brent(double low, double high, double tolerance,
double* root, int maxIter, double* params,
double (*f) (double, double’) )
{
const double SMALL = 1.0e-7; /* epsilon */
 Solving Nonlinear Equations 47

const double VERY_SMALL = 1.0e-10; /* near zero */

int jeer s=_L:
doublerays by, Cc, La, Eb,aec:
double d, e, tol, smalll, small2;
double p, q, r, Ss, xm;

a = Low;
b =— hich:
@ = ach:
fa (*£) (low, params) ;
fb (*£) (high, params) ;
GEG: = —£b;

/*
check that the guesses contain the root */
ae (((GzeY 2 zley)) > (0)
return FALSE; /* bad guesses */

/* start loop to refine the guess for the root */

while (iter++ <= maxIter) {
ain (CE ete) 0) Meg
Ci= vals
Eov= fa:
e = b = a;
d= e;
}
Teetctabsi(re) s< tabs: (£b)))
a =-—b;
DESC
Gh= ai;
1aey, E> Baloyp
Ebc=sfc;,
fog=nta
}
tol = 2 * SMALL * fabs(b) + tolerance / 2;
Sai ES (= Fo) ff AB
if (fabs(xm) <= tol || fabs(fb) <= VERY_SMALL) {
*root = b;
return TRUE;
}
if (fabs(e) >= tol && fabs(fa) > fabs(fb)) {
/* perform the inverse quadratic interpolation */
SP=2tb 8/ Rea
1f (fabsiia —"c) <= VERY SMALL) {
je) ca AY te bran to) tie
eh SS dk = Sih
}
else {
Geastae/ fc}
gt Daley Je auc
GS eS (RS say tS or a (ep = aay = ley 1 fee aha eh
Cte (CT ls) i ae(al) Ee Se)
}
/* determine if improved guess is inside
the range */
qr=e(pe<— 0) 2g sic;
p = fabs(p);
Sma w= ex BCE abs (tol, * iG)rs
Srializit= tabs (e¢*) q);
1£ (2.* po< ((smalli < small2) ? smalll : smali2)) {
/* interpolation successful */
e ="d:
d=p/q;
}
else {
/* use bisection because the interpolation
48 Chapter Two

Listing 2.2 (Continued)

did not succeed */

ad = xm;
em= "d's
}
}
else {
/* use bisection because the range
is slowly decreasing */
aq = sm;
er=" as
}
/* copy most recent guess to variable a */
i = Joy
TRU =) Snelelt
/* evaluate improved guess for the root */
it (tabei(d)=stol)
b += d;
else
b += (am 0) ? tabs Geol) = =fabs (tol);
Eb V=" (AE)(b; spaxams))7
}
return FALSE;
}

int NewtonMultiRoots(double* roots, int* numRoots, int maxRoots,

double tolerance, int maxIter, double* params,
double (*f£) (double, double*) )

int Deer, LF
double hy idzit, EL, i2> (£3, soot,

*numRoots = 0;
rook, = *roects*

ido £
Teer =" Oe
do {
h= (€absi(snootjec 45) se) 0/015 + eect es OR 0
Bul Sea alolohat "= Waly Boyshiarsnits)))*
£2 = (*£) (root, params);
£3 = (*£) (root + h, params);
if (*numRoots > 0) {
for (2 = 0; 1 < “numRoots; 2+) 4
fi (= (root — he — * (roots +s);
£2) J="(mooty— F(roots es)!s
£3. f= (EoOoe tous) (hOOtS era) ie
}
}
/* calculate guess refinement */
GLEE = 292 = E267 Ces sek)
/* update guess */
ROOT == Girk
iter++;
} while (iter <= maxIter && fabs(diff) > tolerance);
if (fabs(diff) <= tolerance) {
*(roots + *numRoots) = root;
*numRoots += 1;
ne (root < 0)
Foot, *= 0.95
else if (root > 0)
root. t= 1,.05>
else
root = 0.05;
 Solving Nonlinear Equations

} while (iter <= maxIter && *numRoots < maxRoots);

return (*numRoots > 0) ? TRUE = FALSE:
}
int DeflatePolyRoots(double* coeff, double initGuess,
double* roots, int* numRoots,
int polyOrder, int maxIter,
double tolerance)

double* a;
double* b;
double* c;
double diff;
double z, x = initGuess;
aioe, 215
mhe, ahkere! = Ap
int n = polyOrder + 1;
*numRoots = 0;

/* allocate dynamic coefficients */

a = (double*)malloc(n * sizeof(double)
);
1£ ('a) /* allocation failed? */
return FALSE;
b = (double*)malloc(n * sizeof (double) );
Eh) Ee alehocctly
Ome hailed gy
free (a);
return FALSE;
}
c = (double*)malloc(n * sizeof (double) ) ;
if (!c) { /* allocation failed? */
free(a);
free(b);
return FALSE;
}
for (Gi = (0); eee meeet)
(al . a4)) = M(Coetee Ae at).

while (iter++ <= maxIter && n> 1) {

pe aXe
*(b +n - 1) M +
@ se 3 I]

(cos m=" 1)
Oran) Tle eo {
Pa a aN)) Re (al a) ia a, *(b + i + 1);
(CC eae) een (1 Pactes SO)! eae GET AE IS: ik
}
71D =e alee (Sor ceeds)ie
lheae ils) ales al)
x == Gute;
if (fabs(diff) <= tolerance) {
iter = 1; /* reset iteration counter */
not;
*(roots + *numRoots) = x;
*numRoots += 1;
/* update deflated roots */
ore (ee OED Re Sar vat tr)
(a) ste BU) ae (sy oF et Saale
i= get the Mast root (*/
te (1. =S'2 ef
gaa
*(roots + *numRoots) = -(*a);
*numRoots += 1;

}
}
/* deallocate dynamic arrays */
free(a)j;
50 Chapter Two

Listing 2.2 (Continued)

free(b);
free(c);
return TRUE;
}
int LBPolyRoots(double* coeff, polyRoot* roots,
int polyOrder, double tolerance)
/*

solves for the roots of the following polynomial

y = Cocfe(O) = coctEli]) K+ Costii2) x°2> 42.4 coed imi exon

Parameters:

coeff must be an array with at least polyOrder+1 elements.

roots output array of roots

polyOrder order of polynomial

tolerance tolerance of solutions

const double SMALL = 1.0e-8;

double* a;
double* b;
double* d;
double alfal, alfa2;
double betal, beta2;
double deltal, delta2, delta3;
leoYege cag lipuel os
int count;
int n = polyOrder;
nooks, sail =a a ale
ine m2 = nm + 2s
int arraySize = n2 * sizeof (double) ;

/* is the coefficient of the highest term zero? */

if (fabs(coeff[0]) < SMALL)
return FALSE;
/* allocate dynamic coefficients */
a = (double*) malloc(arraySize) ;
if (!a) /* allocation failed? */
return FALSE;
b = (double*) malloc(arraySize) ;
Lf ('b) ff /* alllocataion, tas led 4
free(a);
return FALSE;
}
d = (double*) malloc(arraySize) ;
if (!d) { /* allocation failed? */
free(a);
free (b) ;
return FALSE;
}

for (i = 0; i < nl; i++)

a[nl-i] = coeff[il;

/* is highest coeff not close to 1? */

if (fabs(a[1] - 1) > SMALL) {
/* adjust coefficients because a[1] != 1*/
for, (1 = 25 2 <pm2* a+)
ali] ¥="alill;
afl] = 1.0;
 Solving Nonlinear Equations 51

}
/* initialize root counter */
Gounites
=) OF

do {
/*

start the main Lin-Bairstow iteration loop

initialize the counter and guesses for the
coefficients of quadratic factor:

Pix) = x2 + altel * x + beta

anf -

ated a>
betal = 51

do {

biog e= 0)
d[0] = 0;
ley ee ake
lira i

for* (2 °=925 aay = Nick ORS 2 ae steko Fel

bila) = "aii elbaieers bifiq| = -betaly = bik];
Ge) = [eal alte ads ra ailei—abetals adiiiki!-
}
JS pels
k = 'n "2;
clewlhecml S Stouticllispi\)) = (Wells) 2 dolieu) < llicil
aliteale=) (olmier ecg ee ifm ee ia Tin = "delivals
beta2)>= (bin) *"d(g] = dinl * bin] + SOR(binm]))y / deltal;
alfal += alfa2;
betal += beta2;
} while (fabs(alfa2) > tolerance ||
fabs (beta2) > tolerance) ;

deltal = SOR(alfal) - 4 * betal;

cee (Kelepereul <2 ())) %

/* imaginary roots */
delta2 = sqrt(fabs(deltal)) / 2;
deltas = =alftal, / 2.
LOT «(aOR
<< Die Sek
roots[count+i].isComplex = TRUE;
roots[count+i].real = delta3;
roots[count+i].imag = SIGN(i) * delta2;
i
}
else {
delta2 = sqrt(deltal);
/* roots are real */
fom (a = Oe) 4.54.26 a) ot
roots[count+i].isComplex = FALSE;
roots[count+i].real = (SIGN(i) * delta2 - alfal) / 2;
roots[count+i].imag = 0;
}
}
/* update root counter */
count += 2;

/* reduce polynomial order */

n= 2);
ni == 2:
n2 -= 2 t

/* for n >= 2 calculate coefficients of

52 Chapter Two

Listing 2.2 (Continued)

the new polynomial */

neem = 2)
form (a = de ae mas 44)
ala) = Bisa
} while (n >= 2);

LE (m == 1) { /* obtain last sangle reak root +*/

roots[count].isComplex = FALSE;
roots[count].real = -b[2];
roots[count].imag = 0;
}

/* deallocate dynamic arrays */

free(a);
free(b);
free(d);

return TRUE;
}
int Newton2Functions(double *rootX, double* rootyY,
double tolerance, int maxIter,
double* paramsX, double* paramsY,
double (*fx) (double, double, double*),
double (*fy) (double, double, double*) )

double Jacob;
double f£x0, fy0;
double hX, hy;
double Gifix, ditty;
double £xy, f£xx, fyy, fyx;
double x = *rootxX;
double y = *rootyY;
hayes GAC ehe ee y

do {
hx = (als) SEL)? OSOI = OO, A ae;
hYwel abeliy) | eNO One ts) ORGale ese.
Ex0% =) (PEX) (x, YY; PakamsxX):
-fy0 = (*fy) (x, y, paramsY) ;
fxx = ((*f£x) (x + hX, y, paramsxX) -
(*£x) (= HX, yy paramsx)) "2" 7) ix;
fyx = ((*fy)(x + hX, y, paramsY) -
(*fvy)(x =Phey ype paramsy)) 2) (his
fxy = ((*Ex) Geely + bY; paramsx)) =
(*£x) (x, y = hY, paramsx)) /~ 2 7 by:
fyy = ((*fy)
(x, y + BY, paramsy) -
(*fy) (x, y - hY, paramsY)) / 2 / hyY;
Tacob) =* Ea *) Lyyeeeticy. Se hye >
Gi£EX = (fxOo** fyy = fy0rs Ey)" 7 Jacob:
Gpity = (fy0. * Exe = ext * thy) vaces:
se S= AL FEX?
3 ame fs
Wisah
} while (iter++ <= maxIter &&
(fabs (diffX) > tolerance ||
fabs(diffY) > tolerance)
);

*rootx I x
*rootY = y;
return (fabs(diffX) <= tolerance &&
fabs (diffY) <= tolerance) ? TRUE : FALSE;

}
int NewtonSimNLE(Vector X, int numEgns, double tolerance,
 Solving Nonlinear Equations 53

int maxIter, double (*f[]) (double*) )

Vector Xdash;
Vector Fvector;
IntVector index;
Matrix J;
ime, J, monelter, ater = Ol
int rowSwapFlag;
double h;

newMat(&J, numEqns, numEqns) ;

newVect (&Xdash, numEqns) ;
newVect (&Fvector, numEqns) ;
newlIntVect (&index, numEqns) ;

dont
iter++;
// copy the values of array X into array Xdash
for (1 = 0; 2. < numBqns; i++)
NAGI DGEISIa” a) Waele 9 ag
// calculate the array of function values
for (2 = 05 4 < numbgns ++)
VEC (Fvector, i)= (*f£[i]) (X.pData);
// calculate the J matrix
for (2 =. 0% 2 < numegqns; 24+) {
fom (5) = OF) 7) < numbgqnsi art) Ff
// calculate increment in variable number j
He= iGralas (VEC (re g)))) > 1) 20.0 VCC. ae) O0d
VEC (Xdash, j) += h;
MAT(J, i, j) = ((*£[i]) (Xdash.pData) - VEC(Fvector, i)) / h;
// restore incremented value
VEC (Xdash) a9) = VHC (Xx, §3)))+
}
}
// solve for the guess refinement vector
LUDecomp(J, index, numEqns, &rowSwapFlag) ;
LUBackSubst (J, index, numEqns, Fvector) ;
// clear the more-iteration flag
moreIter = 0;
// update guess and test
for convergence
for (1 = 0; & < i++) numEgns;
{
VEC(X, i) -= VEC(Fvector, i);
if (fabs(VEC(Fvector, i)) > tolerance)
moreiIter = 1;
}
if
(morelIter)
morelter = (iter > maxiter) ? 0 : 1;
} while (morelIter);
/* delete dynamic arrays */
deleteMat (&J) ;
deleteVect (&Xdash) ;
deleteVect (&Fvector) ;
deleteIntVect (&index) ;
return 1 - morelIter;
i

Listing 2.2 implements the various root seeking functions. These implementations
are based on the algorithms that I presented earlier. The implementation of the var-
ious C functions incorporate iteration counters to prevent endless looping. In addi-
tion, several C functions use local dynamic arrays created and removed using the
standard C functions malloc and free.
Let’s look at a test program which applies the root-seeking functions declared in
Listing 2.1. Listing 2.3 shows the source code for the TSROOT.C program. To compile
54 Chapter Two

the test program you need to include the files TSROOT.C, ROOT.C, MATVECT.C,
and ARRAYS.C in your project file.
The program tests the following C functions:

1 . The function Bisection in solving for the following equation:

e— 3x7 =0 (2.5)
The test specifies the root-containing interval of [2, 4].
. The function NewtonApprox to solve equation 2.5. The test specifies the initial
root of 3, the tolerance factor of 10-*, and the maximum number of 50 iterations.
. The function NewtonExact to solve equation 2.5. The test specifies the initial
root of 3, the tolerance factor of 10-*, and the maximum number of 50 iterations.
The call to function NewtonExact also includes the pointer to the C function
fl_1stDeriv. The latter function implements the first derivative of equation 2.5.
. The function RichmondApprox to solve equation 2.5. The test specifies the initial
root of 3, the tolerance factor of 10-°, and the maximum number of 50 iterations.
The function RichmondExact to solve equation 2.5. The test specifies the initial
root of 3, the tolerance factor of 10°, and the maximum number of 50 iterations.
The call to function NewtonExact also includes the pointers to the C functions
fl_lstDeriv and fl_2ndDeriv. These functions implement the first and second
derivatives of equation 2.5, respectively.
. The function Combined to solve equation 2.5. The test specifies the root-con-
taining interval of [2, 4], the tolerance factor of 10°, and the maximum number
of 50 iterations.
. The function Brent to solve equation 2.5. The test specifies the root-containing
interval of [2, 4], the tolerance factor of 10-*, and the maximum number of 50:
iterations.
. The function NewtonMultiRoots to solve the multiple roots of equation 2.5. The
test supplies 2 as the initial guess for the first root. The program displays the
three roots of equation 2.5.
. The function DeflatePolyRoots to solve for the real roots of the following
polynomial:

x? — 23x? + 62x — 40 =0 (2.6)

The program displays the real roots of equation 2.6 which are 1, 2, and 20.
10. The function LBPolyRoots to solve for the roots of equation 2.6.
Li, The function Newton2Functions to solve for the roots of the following equations:
x*+y?-1=0 (2.0)

x4 y?+05=0 (2.8)
The test uses the initial guesses of 1 and 3 for x and y, respectively. The test also
specifies a tolerance factor of 10-8 and a maximum of 50 iterations.
 Solving Nonlinear Equations 55

12. The function NewtonSimNLE to solve for the roots of the following equations:

See =O (2.9)

2
exe =a) (2.10)
The test uses the initial guess of -0.16 and 2.7 for x, and x,, respectively.

Listing 2.3 The source code for the TSROOT.C program.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "“arrays.h"
#include "root.h"

double f1(double x, double* params);

double £1_1stDeriv(double x, double* params) ;
double f1_ 2ndDeriv(double x, double* params) ;
double £X(double, double, double%*);
double £fY (double, double, double*);

double f£1(double*) ;
double ££2 (double*);

void p ressAnyKey (void) ;

main ()
{
cons t SMALL = 1.e-10;
double low, high, root;
double tolerance = 1.0e-8;
double params[10];
double roots [10];
double paramsxX[5];
double paramsY[5];
int numRoots;
int maxIter = 50;
double rootX, rooty;
int alg
polyRoot polyRoots[20];
double coeff[10];
int polyOrder;

double (*f££[10]) (double*) ;

Vector Xguess;
Blagls numEqns = 2;
Ee0 [0 Sr ae:caLe
naIs ji j=) B£2):

newVect (&Xguess, 10);

VEC (Xquess, 0) = -0.16;
VEC (RMS ak) Sen ag:

low =e

high = Ale

params[0] = 3;
printf ("**** Bisection method ****\n");
printf ("Solving exp(x)
x * x= O\n",
- params[0]);
#lg *
printf("Guess interval is [%lg,%lg]\n", low, high) ;
es (Bisection(low, high, tolerance, &root, params, f1))
print£("Root = Sig\n\n", root);
56 Chapter Two

Listing 2.3 (Continued)

else
printf("Failed to obtain root\n\n");

printf("**** Newton's method (Approx. Derivative) ****\n");

params[0] = 3;
root = 3;
print £ (Solvang vexpi(s2)) 0 Signs a Fee = ON params Olly:
printf("Initial guess is %lg\n", root);
if (NewtonApprox(&root, tolerance, maxIter, params, f1))
PrEIntE( "Root = —silig haa moot)
else
printf("Failed to obtain root\n\n");

printf("**** Newton's method (Supplied Derivative) ****\n");

params[0] = 3;
Poot ="3;
print£ ("Solving exp(x) = lg * x * x = O\n", params/[0]));
printf("Initial guess is %lg\n", root);
if (NewtonExact (&root, tolerance, maxIter, params,
fll eee Deca.) p)
PLrANeEL (TROOG — slg Nima eo ties
else
printf("Failed to obtain root\n\n");

pressAnyKey
() ;

printf("**** Richmond's method (Approx. Derivative) ****\n");

params[0] = 3;
LOO, = oe
Printé ("Solving tig exp(x))
a =)
x = On" params! Ol)
print£ ("Initial guess is
$lg\n", root);
if (RichmondApprox(&root, tolerance, maxIter, params, f1))
printh("Roet = Slg\n\ny LOoe)-
else
printf ("Failed to obtain root\n\n");

printf("**** Richmond's method (Supplied Derivative) ****\n");

params[0] = 3;
roor = 3
printf ("Solving exp(x) = tlg * x * x = O\n", params [0];
Prinke ("Initial guess: ts 419 \n" 9 root):
if (RichmondExact(&root, tolerance, maxIter, params,
£1, £islstbDeriv, ££ 2nddexrzv,)))
PLinté (Root = slogin\n",, Loot) |
else
printf ("Failed to obtain root\n\n");

Low = 2
brgh = 45
params[0] = 3;
printf ("**** Combined method ****\n") ;
print£ ("Solving exp(x) — tlg * x * x = O\m", params[0i]))-
printf ("Guess interval is [%lg,%lg]\n", low, high);
if (Combined(low, high, tolerance, &root, maxIter, params, f1))
printf£("Root = t$lg\n\n", root);
else
printf("Failed to obtain root\n\n");

low = 2;
high = 4;
params[0] = 3;
print£("**** Brent's method ****\n");
print£ ("Solving exp(x) = tlig * x * x = 0\n", paramsi0])>
printf("Guess interval is [%lg,%lg]\n", low, high);
if (Brent(low, high, tolerance, &root, maxIter, params, f1))
 Solving Nonlinear Equations 57

Drinte("Roet = Slg\n\n™, Look):

else
printf("Failed to obtain root\n\n");

pressAnyKey
() ;

printf ("**** Newton's method (Multiple Roots) ****\n");

params[0] = 3;
HOOeS [Ol = 25
Diinie (We Sonving (expo) sont sce eon Ome params. |0)1)))>
printf("Initial guess is %lg\n", roots[0]);
if (NewtonMultiRoots(roots, &numRoots, 10, tolerance,
maxiIter, params, f1)) {
for (i = 0; i < numRoots; i++)
pEintn ("Root + sdl=stigin™, “a+ 27 roots ial!) >
puechar (o\n") «
}
else
printf ("Failed to obtain root\n\n");

pressAnyKey
() ;

printf ("**** Deflation method (Polynomial Roots) ****\n");

polyOrder = 3;
coeff[0] = -40;
CoeLt [ij =. 62%
coeff[2] = -23;
Coectes |" — ai,
for (a= —polyorderaas >=" 0) p=)
if (i > 1 && fabs(coeff[i]) > SMALL)
Print’ (*tligks Kester "y coctE [a], 2);
else if (1 == 1 && fabs(coeff[i]) > SMALL)
print£("slg * X +", coef£[i]);
else if (i == 0 && fabs(coeff[i]) > SMALL)
Prantci ("sig", "coere [at i)
}
Pete ee a=
if (DeflatePolyRoots(coeff, 1.1, roots, &numRoots,
polyOrder, maxIter, tolerance)) {
for (i = 0; i < numRoots; i++)
Prints Root Sds= sig\n oi + i, * (noes + —1))).;
}
else
printf("Failed to obtain root\n\n");
pues ("\n™);

printf ("**** Lin-Bairstow method (Polynomial Roots) ****\n");

polyOrder = 3;
coeff[0] = -40;
coeff{1i] = 62;
cocker? | =*=23;
Cocer (si =i;
for (a.)= polyOrder>, i >= "0; 2——) {
if (i > 1 && fabs(coeff[i]) > SMALL)
DoImntceC sla * Xetder, scoetE lil 7 L)s
else if (i == 1 && fabs(coeff[i]) > SMALL)
DEIN e(wslg * Kt Coekt lai )iy
else if (i == 0 && fabs(coeff[i]) > SMALL)
printf("%lg", coeff[il);
}
pret: (=O Nr")

if (LBPolyRoots(coeff, polyRoots, polyOrder, tolerance) )

for (a= 0 3 < polyoOrder?, i++) {
if (polyRoots[i] .isComplex)
printf("Root # %d = (%1g) +i (%1g)\n", i+ 1,
58 Chapter Two

Listing 2.3 (Continued)

polyRoots[i].real, polyRoots[i].imag) ;
else
printf("Root # %d = %lg\n", i + 1, polyRoots[i].real);
}
else
printf("Error using the Lin-Bairstow method\n") ;
PUES!
S\i

pressAnyKey
() ;

paramsx[0] = -1;
paramsY[0] = 0.5;
Trootx = I
FOOLY = 3
printf ("**** Newton's method for 2 Equations ****\n");
print£ ("Solving x * x + y * y + %lg = O\n", paramsx([0i));
Print£ ("Solving x * x = yes vy + $16 = 0\n";, params iO;
DrintL£ ("Inreraligqguess Lom Kis tlgin™, z00bx) >
printf("Initial guess for Y is %lg\n", rooty);
if (Newton2Functions(&rootX, &rootY, tolerance,
maxiter, paramsX, paramsyY,
pas HENS Oi
printf("Root X Slig\n" ; -LOOtx);
PLIncE("Root Y¥ tiig\n\n"; LOOtY));
}
else
printf("Failed to obtain roots\n\n");

pressAnyKey
() ;

printf("***** Newton's method for multiple equations*****\n") ;

printf("Solving for:\n");
PELneL("ET(X0, XL) = XE * exp(xX0) = 2\n™) >
PiewnE£( E2(X0 XE) = KOZ x= AN)
for (2 °= 0 2 < numeqns. 2+)
printf("Initial guess for X%d = %lg\n", i, VEC(Xguess, Es
NewtonSimNLE(Xguess, numEqns, tolerance, 30, ff);
printf ("Roots are: \n");
for (1 = 0; i < numEqns; i++)
printf ("X%d = %lg\n", i, VEC(Xguess, i));

pressAnyKey
() ;

/* delete dynamic vector */

deleteVect (&Xguess) ;

return 0;
}
double f1(double x, double* params)
{
return exp(x) - params[0] * x * x;
}

double f1_lstDeriv(double x, double* params)

{
return exp(x) - 2 * params [0] ** =F
}
double f1_2ndDeriv(double x, double* params)
{
return exp(x) - 2 * params[0];
}

double £X(double x, double y, double* p)

{
 Solving Nonlinear Equations 59

return x * xX 4 vy * vy = pol:
}
double fY(double x, double y, double* p)
{
Eetum x * x = y * y + plo;
}
double ff1(double* X)
{
return X[1] * exp(X[0]) - 2;
}

double ff2(double* X)
il
return X[0] * X[0] + X[1] - 4;
}

void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
pues (" \n\n")>
}

Figure 2.3 shows the output of the sample program.

-~A+) Bi Secttom Method ~*~

Solvang fexpisc) =) 3 aoa ee (0
Guess interval is [2,4]
ROOEE—) SS 505

**x*x*x Newton's method (Approx. Derivative) ****

eyo \yalhave pmeb.qo}i(p-@))) > 6) 02 5 eo as eal)
Initial guess is 3
Roots= 3 47/3608

x**x*x* Newton's method (Supplied Derivative) ****

Soltyinic; texcp (acer ok Fs See 0)
Initial guess is 3
Roots = 3473308

Press any key to continue...

****x Richmond's method (Approx. Derivative) ****

Solvang exp) = 36% =x * x = 0
Initial guess is 3
Roots = 3.77079

***x* Richmond's method (Supplied Derivative) ****

Solvang exp) = 3 * ste =O
Initial guess is 3
ROOtl= S-75508

‘Combined method *+***

SolvamcmexD (6) — Sy aoe ok
Guess interval is [2,4]
Root = 93- 73308

#*** Brent's method ****

Solving exp(x) - 3 * x * x =
Guess interval is [2,4]
Root 3.735.085

Press any key to continue...

Figure 2.3 The output of program TSROOT.EXE.

60 Chapter Two

***x* Newton's method (Multiple Roots) ****

Solving exp Go) wowace Seo O
Initial guess is 2
Root # 1 = 0.910008
Root # 2 = -0.458962
Root # 3 = 3.73308

Press any key to continue...

**** Deflation method (Polynomial Roots) ****

ee RNS SR Oe eA Oe)
Rootc# 1 = 2
Root. # 2 = 2
Root #3 >= 20

*x*x*x Tin-Bairstow method (Polynomial Roots) ****

XS eH 23 XQ 62k He — 200 = 20
Root, Hf dies Tt
Root # 2 = 2
Root # 3 = 20

Press any key to continue...

***x*x Newton's method for 2 Equations ****

Solvang! x test. yet Nya =O
Solving ts 5s ay var Oho nO)
Initial guess for X is 1
Initial guess for Y is 3
Root X = 0.5
Root Y = 0.866025

Press any key to continue...

*x*x*x* Newton's method for multiple equations*****

Solving for:
EL CSOF EXT) (=) x ee expx0) = 2
E2Q(KO;, XL) = XO*2 0 4
Initial guess for XO = -0.16
Initial guess for X1 Pra
Roots are:
XO = =0.599125
ile 3264005

Press any key to continue...

Figure 2.3 (Continued)

 Chapter

Interpolation

Tables represent finite values of observed or calculated data that are based on two
or more arrays. Often, you need to calculate, or interpolate if you prefer, values that
are not listed in the tables but are in the range of data in that table. This chapter dis-
cusses popular methods for interpolating the functions in the form of y = f(x). The
chapter covers the following methods:

=» The Lagrangian interpolation

=» The Barycentric interpolation
# The equidistant Barycentric interpolation
=» The Newton divided difference method
=» The Newton difference method
# The Cubic spline method

Most of the interpolation methods are based on the Taylor expansion of a function
f(x) about a specific value x,. Each method manipulates the Taylor expansion dif-
ferently to yield the interpolation equation.

The Lagrangian Interpolation

The Lagrangian interpolation is among the most popular interpolation methods. How-
ever, the method is not the most efficient, since it does not save intermediate results
for subsequent calculations. Here is the equation for the Lagrangian interpolation:

61
62 Chapter Three

kiVote x) Vi <= xy we
+ (3.1)
iG eee) IGeate eo
Voltas) sx) VIG). es)
[CX — Xp)---(& — X,)] Peay (a)
ee =)

Here is the algorithm for the Lagrangian interpolation (Figure 3.1):

Given: the arrays Xarr and Yarr (which contains x and y values), the number of el-
ements in each of these arrays, N, and the interpolating value of x, Xint.

1 Set Yint = 0.
2 Repeat the next steps for l= 0 to N-1:
2.1 Set product P = Yarr[I]
2.2 Repeat the next step for J=0to N-1:
: P * (Xint — Xarr[J])
ers ffrom alJ, itlatthen se (fie= Rane
2.2.1 IfI differs anu

2.3 Add P to Yint.

3 Return the interpolated value Yint.

The Barycentric Interpolation

The above algorithm for the Lagrangian method indicates that you need to recalcu-
late all of the terms for each distinct value of x. It is possible to rearrange the equa-

BS

Figure 3.1. A sample case of Lagrangian interpolation (the thin curve is the interpolating polynomial).
 Interpolation 63

tion for the Lagrangian method to define weights which do not depend on the in-
terpolated value of x. Here is the algorithm for the Barycentric method:
Given: the arrays Xarr, Yarr, and Wtarr (which represent arrays of x, y, and inter-
polation weights), the calculate-weights flag (cwF lag), the number of elements in
the above arrays (N), and the interpolated value of x (Xint).

1 Set sum1 and sum2 to 0.

2 If cwFlag is true repeat the next steps for! =0 to N—-1:
2.1 Set product P to 1
2.2 Repeat the following step for J = 0 to N-1:
2.2.1 Ifl and J are not equal, set P = P * (Xarr[I] — Xarr[J])

2.3 Set Wtarr[I] = os

3 Repeat the next steps forl =0toN-1:

; wtArr{[I]
ll SOU Ses
x (Xint — Xarr{l})
3.2 Set suml = sum1 + Yarr[I] * diff
3.3 Set sum2 = sum2 + diff.

4 Set Yint = Um.

sum2
5 Return Yint as the interpolated value of y.

If the array for x contains equidistant data, then the Barycentric algorithm can be
simplified as shown next:
Given: the arrays Xarr and Yarr (which represent arrays of x and y), the number of
elements in the above arrays (N), and the interpolated value of x (Xint).

Seti = 0) w = tik 0} m= Nel s.= 0,.t=.0,x0:= XarnjO]; and h = Xarr[1] -

Xarr[0].
2 Repeat the next steps:
2.1 Set diff = Xint — x0
2.2 If diff is very small, set diff to an epsilon value
t+w
28 Ci SS
" diff
_ s+w* Yarr{i]
2.4 Sets = ih Giguere

2.5 If mis not equal to zero, perform the next steps. Otherwise, resume at step 3
2.0.1 SeCtw=w*m
2.5.2 Decrement m
2.5.3 Decrement k
Ww
ZO S et W reK

2.5.5 Set x0 =x0+h

2.5.6 Increment i
2.5.7 Resume at step 2.
64 Chapter Three

3 Set Yint = a

4 Return Yint as the interpolated value of y.

Newton’s Divided Difference Interpolation

Newton's divided difference uses the Taylor expansion in performing the interpolation.
The divided differences approximate the various derivatives and allow the method to
work with non-equidistant data. The divided differences of the first level approximate
the first derivatives. The divided difference of the second level approximate the second
derivatives, and so on. Each level of differences has one value less than the preceding
level. Here is a sample difference table which also shows the arrays of x and y data.

X Yo Do ye Dom De
x, y, OD, D, D,’
% Yo D, D,
X,; Y3 Ds
X, V4

where D, = —-—, and D,? = =>,

VY, mA Yo) (By ¥ De)
Df and (D? iy DY)
corer
(X, — Xp) (X, — Xp) (X, — Xp)
above difference table is an upper triangular matrix. You can store this kind of ma-
trix in a one-dimensional array and save space. The following divided difference al-
gorithm uses such an array to store the table of differences:
Given: the arrays Xarr, Yarr, and Table (which store the x, y, and difference table
data), the number of elements (N) in each of the above arrays, the build-table flag
(buildFlag), and the interpolated value of x, Xint.

1 If buildFlag is true perform the following steps:

1.1 ForI=0 to N—1, copy Yarr[I] into Table[I]
1.2 For Il =0 to N —2 perform the next step:
Table[J — 1] — Table[J]
1.2.1 For J=N-1downtolI+ 1, set Table[J] = (Xanr(Fo — 1) = Rae

2 Set Yint = Table[N — 1].

3 For I =N-—2 down to 0, set Yint = Table[I] + (Xint — Xarr[I]) * Yint.
4 Return Yint as the interpolated value of y.

The Newton Difference Method

A special case of the divided difference method emerges when the values of the in-
dependent variable are equidistant. This order in the independent variable simplifies
the calculations involved. Here is the algorithm for the Newton difference method:
Given: the arrays Yarr and Table (which store the y and difference table data), the
number of elements (N) in each of the above arrays, the initial value of x (XO), the
increment in the x values (h), the build-table ee (buildFlag), and the interpolated
value of x (Xint).
 Interpolation 65

1 If buildFlag is true perform the following steps:

1.1 For I =0 to N —1, copy Yarr[I] into Table[]]
1.2 For I = 0 to N —2 perform the next step:

Ihe
we Ohi ane baowown
n tofol 1+ 1,tstcce
set able
Table[J]= ee
Gard ee
ab

2 Set Yint = Table[N — 1].

3 For 1 = N-—2 down to 0, set Yint = Table[I] + (Xint — (XO + 1 * h)) * Yint.
4 Return Yint as the interpolated value of y.

The Cubic Spline Interpolation Method

The Cubic spline method enjoys a great deal of smoothness for interpolations that in-
volve data that varies significantly. The Cubic spline method estimates the second
derivatives at the points of reference and then uses these derivatives in the interpo-
lation. Here is the algorithm for calculating the array of second derivatives:
Given: the arrays Xarr, Yarr, Deriv, and H (which represent the data for the x, y,
second derivatives, and increments), and the number of elements (N) in each of
these arrays.

1 ForI=1toN-—1 perform the following tasks:

1.1 Set H[I] = Xarr[I] — X{I- 1]
(Yarr[I] — Yarr[I — 1])
1.2 Set Deriv[I] =
H{I]
2 ForI= 1 to N -2 repeat the following tasks:
2.1 Set J=I-landK=I1+1
2.2 Set BiJ] =2

2.3 Set C[J] =

AIK}
(H[I] + H[K])
2.4 Set A[J] = 1-C[J]
6 * (Deriv[K] — Deriv[I])
2.5 Set Deriv[J] =
(H[I] + H[K])
3 Solve for the second derivatives using the arrays A, B, and C, the arrays of the
subdiagonal, diagonal, and superdiagonal elements, respectively. The array De-
riv represents the constants vector.

Once you obtain the derivatives, you can then apply the following interpolation al-
gorithm:
Given: the arrays Xarr, Yarr, Derive, and H (which represent the data for the x, y,
second derivatives, and increments), the number of elements (N) in each of these
arrays, and the interpolated value of x, Xint.

1 Set I=0.
2 Locate subinterval containing Xint by incrementing the value in I while (Xint <
Xarr[I] or Xint > Xarr[I+1]) andI < (N- 1).
3 ExitifI>=N-1.
66 Chapter Three

4 Set J=1+1.
5 Set Deltal = Xint — Xarr[I].
6 Set Delta2 = Xarr[J] — Xint.
7 Set Vint Deriv[I — 1] * delta2*3 — Deriv[I] * deltal
“3
ri 6 * Hid] 6 * HJ]
= +

Yarr[J] = Deriv[I] * H[J] Ace Nee)

HD) ite fanaa gs * elta 1 +

Yarr[I]_ Deriv{I — 1] * H[J] raid

AD) 7 Ba a elta2.

8 Return Yint as the interpolated y.

The C Source Code

Let’s look at the C source code which implements the above interpolation methods.
Listing 3.1 shows the source code for the INTERP.H header file. Listing 3.2 contains
the source code for the INTERP.C implementation file.

Listing 3.1 The source code for the INTERP.H header file.
#ifndef _INTERP_H_
#define _INTERP_H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/2/94

Module: C interpolation routines which support:

Lagrange interpolation
Barycentric interpolation
Equidistant-points Barycentric interpolation
Newton divided difference interpolation
4++ Newton divided difference interpolation for
+++
equidistant points
a Cubic spline interpolation
ee/

#include <math.h>

#define INTERP_EPS 1.0e-30

double Lagrange(double* xarr, double* yarr, int n, double x);

double Barycentric(double* xarr, double* yarr, double* wt,
int calcWtFlag, int n, double x);
double ED_Barycentric(double* xarr, double* yarr,
int n, double x);

double NewtonDivDiff(double* xarr, double* yarr,

double* diffTable, int buildMatFlag,
int n, double x);
double NewtonDiff (double x0, double h, double* yarr,
double* diffTable, int buildMatFlag,
int n, double x);

int getDerivatives(double* xarr, double* yarr, int n,

double* deriv, double* h,
double tolerance) ;
 Interpolation 67

double Spline(double* xarr, double* yarr, double* deriv,

double* h, int n, double x);

int tridiagonal(int n, double* a, double* b, double* c,

double* d, double epsilon);

#endif

Listing 3.1 declares the following C functions:

. The function Lagrange performs the Lagrangian interpolation. The function returns
the interpolated value of y and has parameters which represent the arrays of x and
y, the number of elements in each of these arrays, and the interpolated value of x.
. The function Barycentric performs the Barycentric interpolation. The function
returns the interpolated value of y and has parameters which represent the arrays
of x, y and weights, the number of elements in each of these arrays, and the in-
terpolated value of x.
. The ED_Barycentric function performs the equidistant Barycentric interpolation.
The function returns the interpolated value of y and has parameters which repre-
sent the arrays of x and y, the number of elements in each of these arrays, and the
interpolated value of x.
. The NewtonDivDiff function performs the divided difference interpolation. The
function returns the interpolated value of y and has parameters which represent
the arrays of x, y, and the difference table, the build-table flag, the number of el-
ements in each of these arrays, and the interpolated value of x.
. The NewtonDiff function performs the difference interpolation. The function re-
turns the interpolated value of y and has parameters which represent the first value
of x, the increment in x, the arrays of y and the difference table, the build-table flag,
the number of elements in each of these arrays, and the interpolated value of x.
. The getDerivatives function which obtains the second derivatives used by the
function Spline. The function returns an integer value which indicates its success
or failure. The parameters of the functions represent the arrays x and y, the num-
ber of elements in each of these arrays, the array of derivatives, the array of dif-
ferences, and the tolerance factor.
. The function tridiagonal solves the tridiagonal matrix created in function get-
Derivatives. The function has parameters which represent the number of linear
equations, the subdiagonal, diagonal, superdiagonal elements, the constants vec-
tor, and the tolerance factor.
. The Spline function performs the Cubic spline interpolation. The function returns
the interpolated value of y and has parameters which represent the arrays x, y,
the derivatives, and the increments, the number of elements in each of these ar-
rays, and the interpolated value of x.
Listing 3.2 The source code for the INTERP.C implementation file.
#include <stdlib.h>
#include <math-.h>
#include "global.h"
68 Chapter Three

Listing 3.2 (Continued)

#include "interp.h"

double Lagrange(double* xarr, double* yarr, int n, double x)

{
double prod, yint = 0;
VS FR ay

/* loop for each term */

for (2 = OP-wS ns ae) 4
/* initialize term with yint[i] */
DEO p= (cre ee 4)
/* build each term */
£05, (G) = Open nF tats)
a (a eae
prod %= (x =] (xarre + 3) -/
(iF (xxar 9: 9) SF (xarme fig) Diz

yint += prod;
}
return yint;
}

double Barycentric(double* xarr, double* yarr, double* wt,

int calcWtFlag, int n, double x)
{
double prod, diff, suml = 0, sum2 = 0;
Sat Sa ape

if (calcWtFlag) {
form, (P4=00 iS ne ee
prod. =)
for (3 = OF f < me +s)
gE (iel= gi)
prod *= (*(xarr + i) - *(xarr + j));
(we +“ ay =" "7 preds

}
/* loop for each term */
Fort) (27= (eh a= es ye
Giff =e (wEi + 2) fo (ee = |* (are 2)
suml += *(yarr + i) * diff;
sum2 += diff;
}
return suml / sum2;
}

double ED_Barycentric(double* xarr, double* yarr,

int n, double x)
{
double h, x0, GifE; w, s; t=
Bae Fy Kp a;

te
w = 1;
kK 2 O¢
m=n- i;
s = @;
t = 0;
xO = *xarr;
bus, * (xarr + 1) by

fan te shen
aiff = x = x0;
if (fabs(diff) < INTERP_EPS)
 Interpolation 69

diff = INTERP_EPS;
Eeera Ww / Gace es
SSa=w, pyar ba) dale ©
cine (Gil We (ayy ee
w *= (double)m--;
ko
w /= (double)k;
x0 +=-h;
i++;
}
else
break;
}
return s) / t;
}

double NewtonDivDiff(double* xarr, double* yarr,

double* diffTable, int buildMatFlag,
int n, double x)
{
double yint;
ashy sy weal

if (buildMatFlag) {
fom (a = OF a < me a)
*(ALLETabhke +12), = * (vars + 2)5

for (a = 0h <— (eS ee as) Ei

HE@ (ia) =a tae cle ei —i-=2)
*(adiffTable + j) = (*(diffTable + j - 1) -
*(diffTable + j)) /
(CG eaaig cise Sp = 1h Srey) = sel(Gechaiq se oi)ie
}
}
/* initialize interpolated value */
yint = *(diffTable + n - 1);

or (Gr ii} m= 2s ae >=. Or, t——)

yint I A(QvetTable + 1+) + Ge = *(xarr + 2)) * yint;

return yint;

double NewtonDiff(double x0, double h, double* yarr,

double* diffTable, int buildMatFlag,
int n, double x)
{
double yint;
ialepa oly Enc

if (buildMatFlag) {
Ore (ae = Ole | ee re ite)
*(diffTable + i) = *(yarr + i);

OSM = Ole ae <a — ire ae) Af

HOue (qa ae le ye 7 ==)
~“(dititable + 3), = (*(aditfTable + 3 — 1) =
*(diffTable + j)) /
(Gah (GINS 5 38)
1;
}

/* initialize interpolated value */

70 Chapter Three

Listing 3.2 (Continwed)

yint = *(diffTable + n - 1);

Bor (ea ea Ae Ore a)

yint = *(dstttable + 2) + (& = (&0 + 2 * h)) * yan;

return yint;
}

int getDerivatives(double* xarr, double* yarr, int n,

double* deriv, double* h,
double tolerance)

double* a;
double* b;
double* c;
IM, ky, a, ds eeesuibta:

a = (double*)malloc(n * sizeof(double)
);
b = (double*)malloc(n * sizeof(double)
);
c = (double*)malloc(n * sizeof(double)
);

for (itn ip el See)”

gos = chy
(hp Se ae) Sn (scars So) (Kara gk:
Ai(derar + 1) = ((varr=e a) or iGvarr be s))), 2 (ngeeeaie
}

for (4 S21; 2 <— (ma = Lye zee) 4

ger ae as WY;
Khare
7 ll
Soe Ps) a= Ts
iG) =e" (hee Ky (ee eee ie eis
a (ele Saal) ee RUE (Got SS. sle)ir
Fi(deniv + 3) = 6 * (#(derave+ kk) =) * (dernv a),
(Gail(laWeree cb) me col(Lnlents aie) ie
}

/* solve for the second derivatives */

result = tridiagonal(n - 2, a, b, c, deriv, tolerance) ;

/* deallocate the dynamic arrays */

free(a);
free (b) ;
free(c);

return result;
}

double Spline(double* xarr, double* yarr, double* deriv,

double* h, int n, double x)
{
BI ll, pals = CLs
double deltal, delta2, y;

/* locate subinterval containing x */

While ((x < *(xarr + 1) 1
xO> *(xarr + 2 4.2) )) &&
ee ra rey)
i++;

36 2 S= (mn = 2)
return BAD RESULT;
 Interpolation 71

ah tft oe
deltal = x - *(xarr + i);
delta2g= "(kart +7)" =) ac;
y = (*(deriv + i - 1) * delta2 * delta2 * delta2) /
(Ya Gales e ip) es
(* (deriv + i) * deltal * deltal * deltal) /
QO (Ta He 5) nats
(* (yarx—+-j) / *(h + 3) - *(deriv + i)* *(h + j) / 6) *
deltal +
Gacyenee 538) fs aa) (deri oake=s Ih) oe (i a) 6 )\a
delta2;

return y;
}
int tridiagonal(int n, double* a, double* b, double* c,
double* d, double epsilon)
{
int isSingular;
That ales

isSingular = (*b < epsilon) ? TRUE : FALSE;

/* carry out LU factorization */

Eom (i = as a <n as VasSinguliar-, a+)
meNEOT), = + {alt 2) f *(b sua =, 1);
rie meh) Arlo) ay = aa(ey <a me greerssay Gives lin
/* determine if diagonal element is too small */
isSingular = (*(b + i) < epsilon) ? TRUE : FALSE;
Fide i) = id eae + 4) (as a" 1),
}

Le (laisSingwular))
/* carry out backward substitution */
(aoe rieS D)Sea eee ra Ss A) seh OF (be cea em) =
fom (4 = a = 25) a= 1 —)
ENCCN tonal) tai (Cle reets) ee Cut ao) ee ECL ci) fi h(i 2) 5
return TRUE;
}
else
return FALSE;

Listing 3.2 shows the source code for the C functions declared in Listing 3.1.
These functions conform to the algorithms which I presented earlier. Some of the
functions use local dynamic arrays which are created and removed using the stan-
dard functions malloc and free.
Listing 3.3 shows the source code for the test program TSINTERP.C. The pro-
gram declares arrays xarr and yarr which contain the data used in testing the var-
ious kinds of interpolating methods. The program tests the following functions:

1. The function Lagrange at x = 2.5.

2. The function Barycentric at x = 2.5 and 3.5. This test uses the array table to store
the table of data for the second call to function Barycentric.
3. The function ED_Barycentric at x = 2.5 and 3.5.
4. The function NewtonDivDiff at x = 2.5 and 3.5. This test uses the array table to
store the table of data for the second call to function NewtonDivDiff.
72 Chapter Three

5. The function NewtonDiff at x = 2.5 and 3.5. This test uses the array table to store
the table of data for the second call to function NewtonDiff.
6. The function Spline at x = 2.5, 3.5, and 1.5. This test involves calling the function
getDerivatives to first obtain the derivatives required for the interpolation. The
calls to function Spline also use the arrays deriv and h to store the derivatives and
increments.
Figure 3.2 shows the output of the test program.

Listing 3.3 The source code for the test program TSINTERP.C.
#include <stdio.h>
#include <math.h>
#include "global.h"
#include "interp.h"

void pressAnyKey (void) ;

int main()
{
double xarr[] Wy ere 2 Ds Oe,
double yarr[] Dieta ieee ee
doublLevtableii = OFs0; OO wOlats
double deriv mst 107.20), On Ope On a.
doubler hii, = 0, Or Ope OL) ar
int n = sizeof(xarr) / sizeof (double) ;
double x, y;

saga ee
y = Lagrange(xarr, yarr, nN, X);
printf ("Lagrange interpolation\n") ;
joncatialnseg
(uMNey Elise) ee Calle awl = pee a¥e))

mee 2 ar
VY = Barycentric(xarc, yarr, table, TRUE, m=)
printf("Barycentric interpolation\n") ;
PEsMeen ME (SIE ie =) Sale aoe wy)
tes S'S «
y = Barycentric(xarr, yarr, table, FALSE, n, x);
printf ("Barycentric interpolation\n") ;
js go uaa A (el es) es PEN ATL Ten, ay)

Rie= aZh..bs
y = ED _Barycentric(xarr, yarr, nm, x);
printf ("Equidistant Barycentric interpolation\n") ;
DrintLl ("£(tLE) = SLe\m yy Sc, ay)
K~= 43). 5
y= ED Barycentric(xarr, Verma, x) >
printf ("Equidistant Barycentric interpolation\n") ;
Pea rates (i EC Se) a LS a ea id

Ses 2 Eu
y = NewtonDivDiff(xarr, yarr, table, TRUE, n, x);
printf ("Newton divided difference interpolation\n") ;
Print h( (SLE) Ses, Se yvahe
Bisse Sy
y = NewtonDivDiff(xarr, yarr, table, FALSE, n, x);
printf ("Newton divided difference interpolation\n") ;
Drincl( EL (Sle) = ele sc. wie

Sh Se ae oe
y = NewtonDiff(xarr[0], xarr[1] - xarr[0],
yarr, table, TRUE, n, x)&
 Interpolation 73

printf ("Newton difference interpolation\n");

Drie OE SEE) = Se \n a sy)
So SSDs
y = NewtonDiff(xarr[0], xarr[1] - xarr[0],
yarr, table, FALSE, n, x);
printf ("Newton difference interpolation\n");
joveniayere
(Wie (Kniss) fe Enalse ial, se SA)g

getDerivatives(xarr, yarr, n, deriv, h, 1.0E-15);

Sao ee
y = Spline(xarr, yarr, deriv,-h, n, x);
printf ("Cubic spline interpolation\n");
jorqmaere (Waa (CCelay yy ea Sele \Gate, Sie, aye)9
ne =) Syalsye
y = Spliune(xarr, yarr, deriv, h, n, x);
printf ("Cubic spline interpolation\n") ;
jousalinhere
(ee (Caia)), a. RSIeNeatll Se, ay))p
23 Alaleye
Ves Spline(xarn. vari. Gerilvy,) hee mine )5
printf("Cubic spline interpolation\n") ;
forsee (aE ME = Cale, Se. 4¥A)2

pressAnyKey
() ;

return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
puési("\n\n")!;

Lagrange interpolation
£(2.500000) = 6.250000
Barycentric interpolation
£(2.500000) = 6.250000
Barycentric interpolation
£(S= 500000) S= 11222'500,00
Equidistant Barycentric interpolation
£(2.500000) 6.250000
Equidistant Barycentric interpolation
2S. 500000) =e 2e2 50000
Newton divided difference interpolation
£(2.500000) = 6.250000
Newton divided difference interpolation
E(B 500000) s— 12 2250000
Newton difference interpolation
£(2.500000) = 6.250000
Newton difference interpolation
£(3.500000) = 12.250000
Cubic spline interpolation
£(2.500000) = 6.232143
Cubic spline interpolation
£ (Si 500000) = La.232143
Cubic spline interpolation
£(1.500000) = 2.276786

Press any key to continue...

Figure 3.2 The output of program TSINTERP.EXE.

 ‘> "
~ o aloe rain!
.

; — na tab? owl ches eh «hind OMe tee me iy. See

fe Hala i = +Hyias emniQaa= (pee .

i ee ae SM cin dds 4 Cprralinn : Wanieetieiiae the oan

|g piveniveg® de
_s- eS ae
“SO jabs i 42. om ON (eV RRTES
Syeedt ue iw pa be
rs peer ae Ce ca

ay (pans petogznind
ieeie rei ta
Baducq -i0@, bid
ieee eR! ia
Piwtiwinn ulmiee
e -tist~ — . i ahaa ret zz
mi4 +a wat aril. nee.
; S. ct al

Moye | i=.’
tT 2 5
- = sol ,

‘4

‘ . ga j ‘a -
' oe ——— — eae
(a0 Ty
H worse #0
‘ Pg = ° Ws Os AS Thal

| - (wis. oy
! — Fol aves
na Goel = er
} Api «et @. a ilenlt
t : ongetr4
ert tt 44 @oa, io be
\ use tos
{inte met eee (eh
yo * { ae :.¢ inet?
) dye Ay ‘Orde eeey ih bain
ey a u

*j.hgee ie > oe EEE

; : ——winit © @ %6
‘ — ey

tiie te © (Cee
alee Pt peu smh oat A ib
cuseth, ® 2 40m pee a
lg dessaa ettiqge
Se
"7 t va
\) el qh eet”
% ae é 4y/, (tu

‘ ee | ot gaceen

oo
_— — tio 80
>»?
BLY bad a i eee .-

a “f
 Chapter

Numerical Differentiation

This chapter looks at three kinds of methods used to approximate the first four der-
ivates of a function. These methods are:

# The Forward/Backward difference method

= The Central difference method
= The Extended Central difference method

The chapter presents two sets of C functions for the above three methods. The
first set uses arrays of function values, whereas the second set uses pointers to func-
tions. All the above methods assume that the values of the independent variable
are equidistant. Figure 4.1 provides a graphic representation of a simple two-point
slope approximation.

The Forward/Backward Difference Method

The Forward/Backward difference method uses the following equations to approxi-

mate the first four derivatives of function f(x):

"@) a
5
G3l, + Sts)
(4.1)
Al

PGs (CARESS : Ai, —f,) (4.2)

on?
=
—bf, 0 i+ 18f, 1 — 24f,2 iJ
(
+ 14f, 3 — 3f 4) (4.3)
Pies)
2h?

75
76 Chapter Four

Chrer26h=o4rtirreer
1) = e (4.4)

The Forward method uses 6 function values indexed 0 through 5.

The Central Difference Method

The Central difference method uses the following equations to approximate the first
four derivatives of function f(x):

(2
' ze Gh, + f)
(4.5)

1, =A ai
f(x) = Ae ae he
(4.6)

(x)
"(x)= 2 fs
(-f., + 2f , = 2f, + f,)
2 (4.7)
4.7
pee aie eA heat an
ht

The Central difference method uses 5 function values indexed —2 through 2.

fx]
approximated ~ exact slope
slope

Figure 4.1. A simple two-point approximation for a slope.

 Numerical Differentiation 77

The Extended Central Difference Method

The Extended Central difference method uses the following equations to approxi-
mate the first four derivatives of function f(x):
(i, shee ok —1,)
Te (x) = ix (4.9)
(esis + 16f , — 30f, + 16f, — f,)
(x)
"'@)=—— oe
1___* + (4.10)
4.10

fie) zs
i
(f;
= Cll—2 +
IB —l
SINGH,
:
Se i
1 zs 2
= ti3) (4.11)

8h?

f(x) = (le peel see Ok 56f, Se eye bra) c= f,) (4.12)

6h*

The Extended Central difference method uses 7 function values indexed —3

through 3.

The C Source Code

Let’s look at the C source code which implements the above numerical derivation
methods. Listing 4.1 shows the source code for the DERIV.H header file. Listing 4.2
contains the source code for the DERIV.C implementation file.

Listing 4.1 The source code for the DERIV.H header file.

#ifndef _DERIV_H_
#define _DERIV_H_
*
: Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 8/5/94

Module: C routines which support the calculations for the

first and second derivatives.

ey

#include <math.h>

#define DERIV_DEFAULT_INCR 0.01

#define DERIV_BAD_RESULT -1.0e+30

double FBDiffDerivl(double* yArr, double incr,

diate at. VyiOTes)) 5
double FBDiffDeriv2(double* yArr, double incr,
imiee tm, amt yOids«)) >
double FBDiffDeriv3(double* yArr, double incr,
ries hy eeditate Cy Ok Cse))-
double FBDiffDeriv4(double* yArr, double incr,
TTitew Mec, syOlicscp=
double CDiffDeriv1(double* yArr, double incr,
Tite) edie ead eeeOc) 7
double CDiffDeriv2(double* yArr, double incr,
78 Chapter Four

Listing 4.1 (Continued)

suiake ial int yOIdx);

double CDiffDeriv3 (double* yArr, double incr,
date da) Sete ey OWee)i =
double CDiffDeriv4(double* yArr, double incr,
Ime in, EEA Obes);
double XCDiffDerivl1(double* yArr, double incr,
Imtenm, Int yvOEGsx) ;
double XCDiffDeriv2(double* yArr, double incr,
Intan, ame vOcese)ie
double XCDiffDeriv3 (double* yArr, double incr,
Into, ant yOdds):
double XCDiffDeriv4 (double* yArr, double incr,
Ime i, int yOIdx);

/* versions with user-defined functions */

double FBUd£Derivi1 (double x, double incr,
double (*£x) (double) ) ;
double FBUdfDeriv2 (double x, double incr,
double (*fx) (double) );
double FBUdfDeriv3 (double x, double incr,
double (*f£x) (double) );
double FBUd£Deriv4 (double x, double incr,
double (*fx) (double) );
double CUdfDerivi(double x, double incr,
double (*fx) (double) );
double CUdfDeriv2 (double x, double incr,
double (*f£x) (double) );
double CUdfDeriv3 (double x, double incr,
double (*fx) (double) );
double CUdfDeriv4(double x, double incr,
double (*fx) (double) );
double XCUdfDerivi(double x, double incr,
double (*fx) (double) );
double XCUdfDeriv2 (double x, double incr,
double (*£x) (double) );
double XCUdfDeriv3 (double x, double incr,
double (*fx) (double) );
double XCUdfDeriv4(double x, double incr,
double (*fx) (double) );

#endif

Listing 4.1 declares the following set of C functions:

1. The functions FBDiffDerivl through FBDiffDeriv4 implement equations 4.1

through 4.4 using arrays of function values. Each C function has parameters
which pass the y values, the x increment value, the number of elements in the ar-
ray y, and the index of array y which specifies the central element (index 0 as far
as the difference equations are concerned).
. The functions CDiffDeriv1 through CDiffDeriv4 implement equations 4.5 through
4.8 using arrays of function values. Each C function has parameters which pass
the y values, the x increment value, the number of elements in the array y, and
the index of array y which specifies the central element (index 0 as far as the dif-
ference equations are concerned).
. The functions XCDiffDerivl through XCDiffDeriv4 implement equations 4.9
through 4.12 using arrays of function values. Each C function has parameters
which pass the y values, the x increment value, the number of elements in the ar-
 Numerical Differentiation 79

ray y, and the index of array y which specifies the central element (index 0 as far
as the difference equations are concerned).
. The functions FBUdfDerivl through FBUdfDeriv4 implement equations 4.1
through 4.4 using pointers to a function. Each C function has parameters which
pass the value of the x, the increment value, and the pointer to the C function
which implements the targeted mathematical function.
. The functions CUdfDeriv1 through CUdfDeriv4 implement equations 4.5 through
4.8 using pointers to a function. Each C function has parameters which pass the
value of the x, the increment value, and the pointer to the C function which im-
plements the targeted mathematical function.
. The functions XCUdfDerivl through XCUdfDeriv4 implement equations 4.5
through 4.8 using pointers to a function. Each C function has parameters which
pass the value of the x, the increment value, and the pointer to the C function
which implements the targeted mathematical function.
Listing 4.2 The source code for the DERIV.C implementation file.
#include <stdlib.h>
#include "deriv.h"
#include "global.h"

double FBDiffDerivl1l(double* yArr, double incr,

SHORE Gol, hole: AyAdlekeb:d))
{
// verify arguments for parameters n and yOIdx
Te Arce =— NUL LN) |omemSen | |eyiO Dc <a Ol]
VO LG (eS) |e r= 2.0)
return DERIV_BAD RESULT;

return (-3 * *(yArr + yOIdx) +

4 * *(yArr + yOIdx + 1) -
*(yArr + yOIdx + 2)) / 2 / incr;
}

double FBDiffDeriv2(double* yArr, double incr,

inten, sant) y00dsx)
{
// verify arguments for parameters n and y0Idx
ie (Goes Sa iNinAm ||| sm << ey ||| ydOetcbs <= toy ||)
yOidx > (n = 4) || iner == 0)
return DERIV_BAD_ RESULT;

return (2) * *(yArr + v0 Eds) ==

5 * *(yArr + yOIdk + 1) +
4 * *(yArr + yOIdx + 2) -
*(yArr + yOIdx + 3)) / (SQR(incr));
}
double FBDiffDeriv3(double* yArr, double incr,
int n, int y0OIdx)
{
// verify arguments for parameters n and y0Idx
Te GyAr == NUL | eme< 05) |i yOrdx <0 ||
yOIdx > (n - 5) || incr == 0)
return DERIV_BAD_RESULT;

return (-5 * *(yArr + yOIdx) +

18 * *(yArr + yOIdx + 1)
24 * *(yArr + yOIdx + 2)
14 * *(yArr + yOIdx + 3)
3 * *(yArr + yOIdx + 4)) Se
Sal
80 Chapter Four

Listing 4.2 (Continwed)

(SOR (inex) * inex);

}
double FBDiffDeriv4(double* yArr, double incr,
ilsahe Soh, slhokel av0)siehse))
{
// verify arguments for parameters n and y0OIdx
Ine (Geter = ydpine, ||| ga < (5 ||) Adecb< << (0), |]
nAOitche es liek = G)) |||) peeves = 1(0)))
return DERIV_BAD_RESULT;

return (3) * **(yArr + yOIdx) =

14 * *(yArr + yOIdx + 1) +
26 * *(yArr + yOldx 4+ 2) =
24 * * (yArem +) vOGdx ++ 3) +
a Ay Aten ct OMe) ha A) a
2 * *(yArr + yOIdx + 5))
(SOR(2ner), * SOR.(anex)
);
}

double CDiffDerivl(double* yArr, double incr,

int n, int y0OIdx)
if
// verify arguments for parameters n and y0OIdx
she Gigsvares Se iNGiney | leiater< 3) ||| ai@itsbs at ||
sOisbs SS (Gal Se) e||i| clinere = 0)
return DERIV_BAD RESULT;

return (*(yArr + yOIdx + 1) =

*k(yArr + yOLdx = 1)) /°2 / incr;
}
double CDiffDeriv2(double* yArr, double incr,
int ay, ant VvOrdx)
{
// verify arguments for parameters n and y0OIdx
Lf (yaArr == NULT /|)| ene < 93 1 Omesa |
yOLess Sa —F 2) e til mci = s0)
return DERIV_BAD_RESULT;

return (*(yArr + yOIdx - 1) -

2 * *(yArr y0iieac)i ay
+ (yes sh vOtds + 1) 7) (SOR (mens ii
}
double CDiffDeriv3 (double* yArr, double incr,
Vat. mM, int yOeds)
{
// verify arguments for parameters n and yOIdx
if (vArE == NULA ae Se |) iOnickermeia a
vOtdx > (n — 3) |||) amen =— 0)
return DERIV_BAD RESULT;

( =H * *(vArr + yOLids = 2) +
2 * *(yvArr + yOIdx = 1) =
2 * *(yArr + yOIdx + 1) +
(yvArr + vOTds 2) eyaee a/
( SOR(iner) * anexr);
}
double CDiffDeriv4(double* yArr, double incr,
int n, int y0OIdx)
{
// verify arguments for parameters n and y0Idx
LE (yArr == NUL || a <"5 | Olds <taec|
yOIdx > (mn = 3) || iner ==1'0)
return DERIV_BAD_RESULT;
 Numerical Differentiation 81

return (*(yArr + yOIdx - 2) -

Be (yA 4 V0 dsr sii) ae
6 * *(yArr + yOIdx) -
Zl i (Nene eet Oitebre 3 Hil) es
yaa Se slOiiebe ts 22)\)\ 77
(SOR (aner) * SOR(ancxr)
);
}

double XCDiffDerivl1(double* yArr, double incr,

Hae aly) siale Si)aneb‘<))
{
// verify arguments for parameters n and y0OIdx
se (wae SS UA ||) mm << S ||| saduesbe «< 2. |||
wAOB. S Ga = sh) |||| aes =O)
return DERIV_BAD_RESULT;

(oQiewiae! ap yiDaneb > 2) =

Sh ro eslGyieane) ae SiDimobee 1M) ate
Se we(ygigie cp yOatobieecp ls) =
wi(yetae a> QAQIEEKS as Ay) ale) / atiavebels

double XCDiffDeriv2(double* yArr, double incr,

int n, int y0OIdx)
{
// verify arguments for parameters n and y0OIdx
Ane (qytee Sone, |||b ser <= Eyellil syoadebe erm
siAditisbe SS Ga S S}))|))||) aleve S90)
return DERIV_BAD_ RESULT;

return (-1 * *(yArr + yOIdx - 2) +

16 * *(yArr + yOIdx - 1) -
310) % *(yAre ey Onis)
16, * Ay(yAwe ey Olds g47—))
col(vasiona se yyOaiebss 8 AN) a 6 a (USO Ie(Sealehe)2
}

double XCDiffDeriv3 (double* yArr, double incr, int n, int y0OIdx)

{
// verify arguments for parameters n and yOIdx
ase (Owe == turn || || aay SPN yack ee eh |
WOitebe > Ga 9) | || lsaverat == 0)
return DERIV_BAD_RESULT;

return (4 (yAGe SS y0id< — 3) =

8 * *(yArr + yOIdx - 2) +
13 VAT. 1 y.0lds— i) i=
LB) fon (Gig Nigras See ayOuiGbiewe ail))y es
8 * *(yArr + yOIdx + 2) <=
i pyAriota een O Coch Si) w/a Gen CUBBI (mcr);
}

double XCDiffDeriv4(double* yArr, double incr,

imt m, ant yOids)
{
// verify arguments for parameters n and y0Idx
ie (Qawere SS ume || seh eS {lh ardaluiebs <2" (|
yOldx > (n-- 4) || incr == 0)
return DERIV_BAD_RESULT;

mewn (= an (ty Arr (+ OTC —— 0S) ee

1D (yer + y Oca eee)
Shoe iy ealvgseatammes ave KGbg i)
Sees. Five Sy OTs) s—
39 *% *(yArr + yOTda: + 1)) +
1292" (yarn - yOldx + 2)" =
ReyAser st VOI ey 3) 7/16) /
82 Chapter Four

Listing 4.2 (Continued)

(SQR(iner) * SQR(incr));
}

double FBUdfDerivl (double x, double incr,

double (*fx) (double) )
{
incr = (incr ==0) 2 DERIV DEFAULTLINGR = inex:

Hetumn (3) (etx) Ga) +

4 * (*£x)(x + incr) -
(FEx\iGe 4. 2°* incr) ef e2 ff ance;
}
double FBUdf£Deriv2 (double x, double incr,
double (*fx) (double) )
{
iner = i(anen == 10)) 2 DERTV DEBAULEINCR a: anes

iehebeaay (PS iCtig) (ere))s

5 a ix 2 ines) +
4% (Ess)
o (ee 2 4 Sinex) =
(FEx)(Ce +S * amex) )) / (SOR (anier),) =
}

double FBUd£Deriv3 (double x, double incr,

double (*fx) (double) )
{
iner = (iner == 0) 2? DERTIV_ DEFAULT _INCR 7 anes;

return (55% (*Esa)i(ee)

18 * (*fx)(x. + incr) -
24. * (*£x) (x + 2 * ancr) +
14 * (*fx)(x + 3 * incr) -
3° (Fix )iee 4 ain]er) 7) 27) CUBE(amacs)
}

double FBUdfDeriv4 (double x, double incr,

double (*£x) (double) )
{
incr = (incr == 0) ? DERIV_DEFAULT_INCR : incr;

return: (3° * Es) (x)) =

6 eT (eth ie I (re << incr) +
26 -* Csr) 2) ena
PPR eA rad) (al(re + 3 *iner) +
ti * i(* fx)te + 4 * incr) -
2 * (*ix)pGe + 5S? aner)y 7
(SOR(incr) * SQR(incr));
}
double CUdfDerivl (double x, double incr,
double (*fx) (double) )
{
iner = (incr == 0) ? DERIV_DEFAULT_INCR s inex;

return ((*fx)(x + incr) -

(*£) (= nes)! 2° feamexs
}
double CUdfDeriv2 (double x, double incr,
double (*fx) (double) )
{
incr = (incr == 0) ? DERTV DEFAULT SENCR 2 ine;

return ((*£x)(x = 1nex)) =

Zoi «6* Eee) see
 Numerical Differentiation 83

(scares ancrz)))P / (SOR(anexr))i>

}

double CUdfDeriv3 (double x, double incr,

double (*fx) (double) )
{
incr y= (aner *== 0) §? DERTVZDEPAULTEINER, 3 inex:

Petite (AES)!(se 2 ar ea) ee

DP (AEs)!xe ners)
2 Fis) (Gxt Pes) ss
CPEs ie eee 2a incr) ) e/a 2 ay ACUBIEN (eereT) i=
}

double CUdfDeriv4 (double x, double incr,

double (*f£x) (double) )
if
iners= (ance == 0) 2 DERTVIDEFAURT
AINGR, sine:

mec UETIn (((Ct £30) ) (ee Ses) y=

Ae * 0+ fe)(se = a nicas)) +
Cet tan) Gc) ee
4 * (*£x)(x + incr) +
(*£5) (x92) * anex)) ji 7
(SOR(iner) * SQR(incr));
}

double XCUdfDerivi1 (double x, double incr,

double (*fx) (double) )
{
iner = (incr == 0) ? DERIV_DEFAULT_INCR : incr;

return ((*f£x)(x - 2 * iner) -

[SH (Khap’)(orey =e tia elrg) aes
8 * (*fx)(x + incr) -
(ha ait @:crs 2 <2 ansrere) Na ale 7) Gingvonar
}
double XCUdfDeriv2 (double x, double incr,
double (*f£x) (double) )
{
iner = (1ner ==0)) > DERIV
DEFAULT _INCR 3) anex-

return (-(*fx) (x - 2 * incr) +

16 * (*£x)(x = iner) =
SiO i ema ies), late
Ge een (atesc)) (Gc ae ie)
ocheac)) (Gia ca Qe es “siaenc)) ly ae) TSONSN GalieKoha)
}

double XCUd£Deriv3 (double x, double incr,

double (*fx) (double) )
{
Inch = (aner == 0)... DEREV_DEFAULT_INCR. & incr;

Heburm (C(x)(sc = 3.9* “aner)) =

Seats Css eee SCI) >
iS (SEs )(cxey = ane) —
1B) (Ese ner)” -F
Bh Moe (Garros)
(Se) tay ah aes Sitiaong))
(CPE Rees rcs) \P / 8B 9/
(SOR (mex) *) anen)|
}
double XCUdfDeriv4 (double x, double incr,
double (*fx) (double) )
{
84 Chapter Four

Listing 4.2 (Continued)

ier = (inner ==)0)) 2 DERIV. DEFAULT. INGR. = mer;

meturn (=(*£x)(x = 3° * aner) +

NOL (caiob.)
(5g came?) er atiakela) | ea
39 * (*fx)(x - incr) +
56 * (*fx) (x) =
39) * \(*iisc)) ) (ge tabaci) eee
12 * (*£x)(x + 2 * adner) =
(2b)Go Ses Ginnie) 97 6) 7/
(SOR: (anes) 5* SOR (Gmex))) ;
}

Listing 4.2 implements the C functions declared in Listing 4.1. Notice that the C
functions which use arrays of data verify the arguments for the number of array ele-
ments and the parameter yOIdx. This verification makes sure that there are enough
elements in the array and that the value of parameter yOldx is appropriate.
Listing 4.3 shows the source code for the test program TSDERIV.C. The program
tests the C functions declared in Listing 4.1. The program creates an array of y val-
ues for x ranging from 0.5 to 2.5. The listing also defined the test function as x*— 1.
The program then performs the following tasks:

t Tests the Forward/Backward difference formula for x = 1 and using yOIdx = 5.

The program calculates and displays the values of the first four derivatives using
functions FBDiffDeriv1 through FBDiffDeriv4.
. Tests the Forward/Backward difference formula for x = 1. The program calculates
and displays the values of the first four derivatives using functions FBUdfDerivl
through FBUdffDeriv4.
. Tests the Central difference formula for x = 1 and using yOIdx = 5. The program
calculates and displays the values of the first four derivatives using functions
CDiffDerivl through CDiffDeriv4.
. Tests the Central difference formula for x = 1. The program calculates and dis-
plays the values of the first four derivatives using functions CUdfDeriv1 through
CUdffDeriv4.
. Tests the Central difference formula for x = 1 and using yOldx = 5. The program
calculates and displays the values of the first four derivatives using functions
XCDiffDerivl through XCDiffDeriv4.
. Tests the Central difference formula for x = 1. The program calculates and dis-
plays the values of the first four derivatives using functions XCUdfDeriv1 through
XCUdffDeriv4.

To compile the test program you need to include files DERIV.C and TSDERIV.C in
your project file. Figure 4.2 shows the output of program TSDERIV.EXE.

Listing 4.3 The source code for the test program TSDERIV.C.

#include <stdio.h>
#include <math.h>
#include "global.h"
#include "deriv.h"
 Numerical Differentiation 85

#define ARRAY SIZE 20

void pressAnyKey (void) ;

double fx(double x)
{
/* return 1 - 5 * x - SOR(x) + SOR(SOR(x)); */
return SOR(SOR(x)) - 1;
}

int main()
{
double yArr[ARRAY_SIZE] ;
double x;
double deltaX = 0.1;
Mahe st, avAdanebrts

oq Satie
fom (i = 0s < ARRAY SIZE T+) {
a (Giaweia Stall) ee AeGeir
x += deltax;
}

printf("Using Forward/Backward difference formula\n") ;

x = 01:
VOM = 5)
printf ("Using diffence values\n");
joreaunmyers
((WayAl (Exile) = “Esaltep iat 5 Ste,
FBDiffDerivil(yArr, deltaX, ARRAY_SIZE, y0OIdx));
gone tate (Mayet (Sa. Salo Nae x,
FBDiffDeriv2(yArr, deltaX, ARRAY_SIZE, yOIdx));
pict £ (My (Sig) = Sienna sc,
FBDiffDeriv3(yArr, deltaX, ARRAY_SIZE, yOIdx));
asta (ty UO CSG) ela Nn) St,
FBDiffDeriv4(yArr, deltaX, ARRAY_SIZE, yOIdx));
printf("Using user-defined equation\n") ;
Nossa tee (Maye (eich) eelicy a e,
FBUdfDerivl(x, deltaX, fx));
joucabmjona (Cava (sao) im) SouloiNalllees Grey
FBUdfDeriv2 (x, deltaX, fx));
(oOrebiate te(ys ope eS licn ia kay
FBUd£Deriv3(x, deltaX, fx));
(ehqulimhesa((Wie OU Wen) Mea Calle Nga. Sis
FBUdf£Deriv4 (x, deltaX, f£x));

pressAnyKey
() ;

printf("Using Central difference formula\n") ;

ere Oi
VOldx = 5.
printf("Using diffence values\n") ;
Ditnee (ya Slg)) = Silko \ne, x,
CDiffDerivl(yArr, deltaX, ARRAY_SIZE, yOIdx));
DuIMemOnyL (Sig) = Slo \a x,
CDiffDeriv2(yArr, deltaX, ARRAY_SIZE, yOIdx));
Dirinte(ty! (sig) = wslg\n", x,
CDiffDeriv3(yArr, deltaX, ARRAY_SIZE, yOIdx));
Piesemitecy ee ees eee hn" 7 3c,
CDiffDeriv4(yArr, deltaX, ARRAY_SIZE, yOIdx));
printf ("Using user-defined equation\n") ;
printf ("y'(%lg) = tig\n", x,
CUdfDeriv1(x, deltaX, £x));
Pence y sdig) = eign", x,
CUdfDeriv2(x, deltaX, fx));
PoIsleratede(et
yas NSO) = boloNm SG,
CUd£Deriv3(x, deltaX, fx));
printk("y'
0!) (Slig)<= Sig\n"7 x,
86 Chapter Four

Listing 4.3 (Continued)

CUdfDeriv4(x, deltaX, fx));

pressAnyKey
() ;

printf("Using Extended Central difference formula\n") ;

x = 2/0
sid Kobo ley
printf("Using diffence values\n");
pranehiCry (SlGg)h Saag hn". exe
XCDiffDerivl(yArr, deltaX, ARRAY _SIZE, yOIdx) );
pent (*y. (Gig) = tigi") ss,
XCDiffDeriv2(yArr, deltaX, ARRAY SIZE, yOIdx));
Perel (ye a Niche ecp Nt xe
XCDiffDeriv3(yArr, deltaX, ARRAY _SIZE, yOIdx) );
PrIneh( "yi" (Siig) = slg\a=, ss,
XCDiffDeriv4(yArr, deltaX, ARRAY SIZE, yOIdx) );
printf("Using user-defined equation\n") ;
DENCE Vy eG) GN es
XCUdfDerivil(x, deltaX, £x));
DMD y Os ig) f= slg \n eos,
XCUd£Deriv2(x, deltaX, fx));
PEINCE (Sy ASG). = Sag hae ese,
XCUd£Deriv3 (x, deltaX, £x));
jovaub eens (larg US GE Ken i catopGenin, te
XCUdfDeriv4 (x, deltaX, fx));

pressAnyKey() ;

return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
pues (™ \ni\n") +
}

Using Forward/Backward difference formula

Using difference values
y'(1) = 3.914
pO? ha) ee aS,
y'''(1) = 24
SO DS (1) ees 2
Using user-defined equation
pes5) laa Jp
yee i 1.) ed 8
Pelee (Lies 28
Sat aARAN Ugh ag ee?

Press any key to continue...

Using Central difference formula

Using difference values
y'(1) = 4.04
pg
SO 2ah Yd a4 0B
2
anete ee?!
es ee
Using user-defined equation
y'(1) = 4.04
ye MiCL) 208
alasUo8B ahd |
Si een 24

Figure 4.2 The output of program TSDERIV.EXE.

 Numerical Differentiation 87

Press any key to continue...

Using Extended Central difference formula

Using difference values
y'(1) = 4
ya (1) = 12
yr et) eos
yo 2 (y= 24
Using user-defined equation
y'(1) = 4
ye PC) =a
y'''(1) = 24
ai 0 UI (at) ect Ne)

Press any key to continue...

Figure 4.2 (Continued)

 Chapter

Numerical Integration

This chapter looks at popular methods for numerical integration. These methods ei-
ther use arrays of values (with the option of using direct function evaluations) or use
function evaluations. The chapter discusses the following methods:

= Simpson’s method
# Simpson’s alternate method
=» Gauss-Legendre quadrature
=» Gauss-Laguerre quadrature
= Gauss-Hermite quadrature
= Gauss-Chebyshev quadrature
= Romberg’s method

Simpson’s Method
The most popular numerical integration method is Simpson’s method (Figure 5.1).
In fact, there are several versions of the Simpson’s method which use points with
equidistant values for the independent variable. The most popular Simpson methods
are the one- third and three-eighths rules. The equation of the one-third rule re-
quires at least three values of y and grows into odd numbers (5, 7, 9, and so on) of y
values. The equation for the Simpson’s rule is:

h (y, + 4y, + Ya)

[tco ax = 3 (5.1)

89
90 Chapter Five

x 0 Xqt h x_+?h x + 3h x f 4h xt 5h

Figure 5.1. Simpsons rule.

where h is the increment in the independent variable. The general form for the above
equation is:

[tco ee h (Wy,1 + 4y,2 + 2y.3 +...

- + N-2
2Vy5 + N-l4Vn, *+ Yu)
YN (5.2)

Here is the algorithm for Simpson’s one-third rule:

Given: the array Yarr with an odd number of elements (N) and the increment h.

1 Set sumEven = 0 and sumOdd = 0.

2 Forl= 1 to N —2 add Yarr[I] to sumEven.
3 ForI=2 to N—3 add Yarr[I] to sumOdd.
4 Set Area = +*(iarr(0) + 4* sumEven + 2 * sumOdd + Yarr[N — 1]).

5 Return Area as the value of the integral.

You can use the above algorithm with direct function evaluations by replacing Yarr{I]
with the expression (X0 +h *I).

Simpson’s Alternate Extended Rule

Let’s look at another Simpson’s rule which offers better results than the one-third
(and three-eighths) rules. Here is the equation for the alternate extended rule:
 Numerical Integration 91

(5.3)
niGlG, +r0t) agate 40 inet tie O aah At, +4 oie bOfes ,+ 17f,)
[fG9 ax = -2
48
Here is the algorithm for using the Simpson’s alternate extended rule:
Given: an array Yarr of y values, the number of array elements (N), and the incre-
ment in the independent variable, h.

1 Set Sum = 0.
2 For1=0 to N—-1 add Yarr{I] to Sum.
3 Set Sum = h/48 * (48 * Sum — 31 * Yarr[O] + 11 * Yarr[1] — 5 * Yarr[2] + Yarr[3] —
Yarr[N — 4] —5 * Yarr[N — 3] + 11 * Yarr[N — 2] — 31 * Yarr[N — 1]).
4 Return Sum as the numerical integral.

You can use the above algorithm with direct function evaluations by replacing
Yarr[I] with the expression (X0 + h * I).

The Gaussian Quadrature Methods

The Gaussian quadrature methods represent a different way of calculating integrals

that enjoy high accuracy. Rather than sampling the function value are regular values
of the independent variable, the Gaussian quadrature zoom in on critical function
values. These methods obtain the sought integral based on the product of the criti-
cal function values and special weights. The Gaussian quadratures can serve ob-
served data if these data can be sampled at the critical values. Another key point to
keep in mind regarding Gaussian quadratures is that they have specific limits of in-
tegration. You can still use some of these methods to integrate over your own range
of x values by mapping the values of your x values (call it the real-world x) with the
Gaussian quadrature’s x values.

Gauss-Legendre Quadrature
The Gauss-Legendre quadrature uses the Legendre polynomial to obtain the weights
and critical points which evaluate an integral in the of -1 to 1. You can use the Gauss-
Legendre quadrature to evaluate the integral for the range [A, B] by using the fol-
lowing equations:
fico sie (BSA) (ey iGc)h ae some pac f(a) (6.4)

x =A+(B-AG+)) (5.5)

where z, is the critical quadrature value in the range —1 to 1, x, is the real world value
which corresponds to z,. Equation 5.4 shows the weights c, through c,. The value of
n in equation 5.4 is the order of the Legendre polynomial used to precalculate the
values of z, and c,. The C source code in this chapter uses values for z, and c, based
92 Chapter Five

on the Legendre polynomial order of 6. This means that the Gauss-Legendre quad-
rature evaluates an integral with 6 points. Because integrals may cover a wide range
of x values and/or require enhanced accuracy, I present an algorithm which applies
the Gauss-Legendre quadrature over subintervals. Here is the algorithm:
Given: the range of x, A, and B for the integration, the number of subintervals (N)
for the range [A, B], and the function f(x).

Initialize the array Xk to store the critical x values of the quadrature.

Initialize the array Ak to store the weights.
Set Area = 0.
(B-A)
Sep
‘ N
Set xA =A.
On
o> For I = 1 to N repeat the next steps:
WHrH
re
6.1 Set Sum = 0
6.2 Set xB = xA+H
6.3 For J = 0 to 5 repeat the next steps:

Goulet [xa+ H)* (Xk{J] + 1)

6.3.2 Add Ak{J] * f{(X) to Sum

6.4 Add (2 * Sum]to Area

6.5 Set xA = xB.

7 Return Area as the result of the numerical integration.

The Gauss-Laguerre Quadrature

The Gauss-Laguerre quadrature evaluates the integral of e*f(x) between 0 and in-
finity using the Laguerre polynomials. The method calculates the approximation for
the integral by summing the product of quadrature weights and the values of function
f(x) at critical points of x. Here is the equation for the Gauss-Laguerre quadrature:

[tc dx = A, f(x) (5.6)

To obtain the integral of a custom function, calling g(x), you need to write that
function as f(x) = g(x) e* and then use f(x) in the integration process. The number
of added terms (each term is the product of the weight and function value) depends
on the order of the Laguerre polynomial used in approximating the integral. Here is
the algorithm for the Gauss-Laguerre quadrature.
Given: the function f(x).

1 Set Sum = 0.
2 Initialize the array Xk to store the critical x values of the quadrature.
3 Initialize the array Ak to store the weights.
4 ForI=0 to 5 add Ak{I] * f(Xk[I]) to Sum.
5 Return Sum as the area for e* f(x) between 0 and infinity.
 Numerical Integration 93

The Gauss-Hermite Quadrature

The Gauss-Hermite quadrature evaluates the integral of exp(—x”) f(x) between mi-
nus and plus infinity using the Hermite polynomials. The method calculates the ap-
proximation for the integral by summing the product of quadrature weights and the
values of function f(x) at critical points of x. Here is the equation for the Gauss-
Laguerre quadrature:

[tco ax = 2 A, fx) (6.7)

To obtain the integral of a custom function, calling g(x), you need to write that
function as f(x) = g(x) exp(x?) and then use f(x) in the integration process. The
number of added terms (each term is the product of the weight and function value)
depends on the order of the Hermite polynomial used in approximating the integral.
Here is the algorithm for the Gauss-Hermite quadrature.
Given: the function f(x).

1 Set Sum = 0.
2 Initialize the array Xk to store the critical x values of the quadrature.
3 Initialize the array Ak to store the weights.
4 For l= 0 to 5 add Ak{I] * f(Xk[I]) to Sum.
5 Return Sum as the area for e* f(x) between 0 and infinity.

The Gauss-Chebyshev Quadrature

The Gauss-Chebyshev quadrature evaluates the integral of f(x)/V (1 — x) between
—1 to 1 using the Chebyshev polynomials. The method calculates the approximation
for the integral by summing the values of function f(x) at critical points of x. Here is
the equation for the Gauss-Laguerre quadrature:

[tc dx = (win) ZC) (5.8)

Here is the algorithm for the Gauss-Chebyshev quadrature:
Given: the function f(x) and the number of points N used to evaluate the interval.

1 Set Sum = 0.
2 For I= 1 to N repeat the following steps:
21 Set X ores
.1 Set X = cos ON

2.2 Add X to Sum.

T ‘ f(x)
3 Return IN Sua as the area of function Va) :

The Romberg Method

The Romberg method is an iterative method that calculates the area by successive
refinements. The method calculates a column of area estimates using the Trape-
94 Chapter Five

zoidal rule. The method then uses the so-called Richardson extrapolation to refine
the first set of area estimates. Here is the algorithm for the Romberg method:
Given: the integration limits A and B, the tolerance factor T, the maximum num-
ber of iterations N, and the integrated function f(x).

1 Set operation flag to goOn.

2 Set number of iteration, Iter, to 0.
3 Set number of steps M to 1.
4 Set H=B-A.

De sett = FAR
ata(x
(f(A) + f(B)) he

6 Set T[0] = h * fA.

7 Set Area = T[O].
8 Repeat the next steps while the operation flag is goOn:
8.1 Increment nIter
8.2 Halve the value of H and double the value of M
8.3 Set cl = T[0]
8.4 Set J=OandI=M-1
8.5 Increment J
8.6 Set fnArr{I — 1] = f(A +1* H)
8.7 If > 1 Set fnArr[l — 2] = fnArr aie
8.8 Setl=I-2
8.9 If >= 1 resume at step 8.5
8.10 Set T[O] = fA
8.11 ForI = 1 to M-—1 Add fnArr{[I — 1] to T[0]
8.12 Set T[0] = T[0] *H

9.18:
3.13 Farle
For oS aleriacnteT|
I= 2 to niter+ 1 set |= (4°d-D-)

8.14 If |T[nIter] — Area)| <= T then set operation flag to opConverge

8.15 If nIter >= N then set operation flag to opReachedMax
8.16 Set Area = T[nIter].
9 Return Area as the integral.

The C Source Code

Let’s look at the C source code which implements the integration methods that I pre-
sented earlier in this chapter. Listing 5.1 shows the source code for the INTEGRAL.H
header file. Listing 5.2 shows the source code for the INTEGRAL.C implementation
file.

Listing 5.1 The source code for the INTEGRAL.H header file.
#ifndef _INTEGRAL_H_
#define _INTEGRAL_H_

/*

Copyright (c) 1995 Namir C. Shammas

 Numerical Integration 95

Version 1.0 Date 8/5/94

Module: C routines which support the following

numerical integrations methods:

+ Simpson's rule
+ Gaussian quadrature
+ Romberg's method

The library offers two sets of functions: one for

arrays and the other for user-defined functions.

sag

#include <math.h>
#include "global.h"

#define INTEGRAL_BAD_RESULT -1.0e+30

#define MAX_ROMBERG_TABLE 52

double Simpson(int numElem, double* yArr, double h);

double £Simpson(double xFirst, double xLast, int n,
double (*fx) (double) );
double AltExtSimpson(int numElem, double* yArr, double h);
double fAltExtSimpson(double xFirst, double xLast, int n,
double (*fx) (double) );
double GaussLegendreQuadrature (
double xFirst, double xLast,
int nSubIntervals,
double (*fx) (double) );
double GaussLaguerreQuadrature(double (*fx) (double) );
double GaussHermiteQuadrature(double (*f£x) (double) );
double GaussChebyshevQuadrature(int n, double (*fx) (double) );

Boolean Romberg(double xA, double xB, double tolerance,

int MaxIter, double (*f£x) (double),
double* area);
#endif

Listing 5.1 declares the following C functions:

. The function Simpson calculates the integral using Simpson’s one-third rule with
an array of function values. The function returns the value of the area and has pa-
rameters which specify the number of array elements, the array of function val-
ues, and the increment in x.
. The function fSimpson calculates the integral using Simpson’s one-third rule with
direct function evaluation. The function returns the value of the area and has pa-
rameters which specify the first and last value of x, the number of subintervals,
and the pointer to the integrated function.
. The function AltExtSimpson calculates the integral using Simpson’s alternate ex-
tended rule with an array of function values. The function returns the value of the
area and has parameters which specify the number of array elements, the array of
function values, and the increment in x.
. The function fAltExtSimpson calculates the integral using Simpson’s alternate
extended rule with direct function evaluation. The function returns the value of
the area and has parameters which specify the first and last value of x, the num-
ber of subintervals, and the pointer to the integrated function.
96 Chapter Five

. The function GaussLegendreQuadrature implements the Gauss-Legendre quad-

rature. The function returns the value of the integral and has parameters which
specify the integration range, the number of subintervals, and the pointer to the
integrated function.
. The function GaussLaguerreQuadrature implements the Gauss-Laguerre quadra-
ture. The function returns the value of the integral and has a single parameter
which is the pointer to the integrated function.
. The function GaussHermiteQuadrature implements the Gauss-Hermite quadra-
ture. The function returns the value of the integral and has a single parameter
which is the pointer to the integrated function.
. The function GaussChebyshevQuadrature implements the Gauss-Chebyshev quad-
rature. The function returns the value of the integral and has parameters which
specify the pointer to the integrated function and the number of subintervals.
. The function Romberg implements the Romberg integration. The function re-
turns an integer value which represents the success or failure of the function. The
function has parameters which define the integration range, tolerance factor,
maximum number of iteration, pointer to the integrated function, and the pointer
to the calculated area.

Listing 5.2 The source code for the INTEGRAL.C implementation file.
#include <stdlib.h>
#include <math.h>
#include "integral.h"
#include "global.h"

double Simpson(int numElem, double* yArr, double h)

{
double sumEven = 0;
double sumOdd = 0;
double sum;
int. 1
Boolean isOdd = ((numElem % 2) != 0) ? TRUE : FALSE;

if (numElem < 3)
return INTEGRAL
BAD RESULT;

/* decrement number of element if value is even */

if (!isOdd)
numE1lem--;

/* add even terms */

for (2 = 2s a. < (number = ee a= 2))
sumEven += yArr[il];

/* add odd terms */

for (i = 25.1, <» (numElem = 2)') 2 +572)
sumOdd += yArr[il];

/* calculate integral */
sum = h / 3 * (yArr[0] + 4 * sumEven +
2 * sumOdd + yArr[numElem - 1));

/* adjust for even number of observations */

LE (iieOdad)
numElem++;
sum += h / 2 * (yArr[numElem - 2] +
 Numerical Integration 97

yArr[numElem - 1]);
}
return sum;
}
double fSimpson(double xFirst, double xLast, int n,
double (*fx) (double) )
{
double h;
double x;
double sumEven = 0;
double sumOdd = 0;
double sum;
ant a
Boolean isOdd = ((n % 2) != 0) ? TRUE : FALSE;

his “(Gah <S “8)))

return INTEGRAL
BAD RESULT;

/* increment n if it is even */
if (!isOdd)
1@'G ear

/* calculate increment */
hea (haste xmas) 7 (ne = dh)

/* add even terms */

Secs Sashiheshe Saillalp
ore (ay Salo sk <2 Gey SANG kh ce
sumEven += (*f£x) (x);
od Be PhS Inle
}

/* add odd terms */

reo Signibatshe, ee it ale
fOte(a = 925 a <) (ie 2) a = 2)
sumOdd += (*£x) (x);
xo+S2
ee he
}
/* calculate integral */
sum = h / 3 * ((*£x) (xFirst) + 4 * sumEven +
2° sumOdd + (= £x) (xLast)));
return sum;
}
double AltExtSimpson(int numElem, double* yArr, double h)
{
double sum = 0;
mibghe ashes
Boolean isOdd = ((numElem % 2) != 0) ? TRUE : FALSE;

if (numElem < 10)

return INTEGRAL
BAD RESULT;

/* decrement number of element if value is even */

if (!isOdd)
numElem--;

/* add terms */
for (i = 0; i < numElem; i++)
sum += yArr[i];

/* calculate integral */
sum = h / 48 *
(48 * sum -
See SyAr (Oi) +
11 * yArr[1] =
98 Chapter Five

Listing 5.2 (Continwed)

5 * yArr[2] +
yArr[3] +
yArr[numElem - 4] -
5 * yArr[numElem - 3] +
11 * yArr[numElem - 2] -
31 * yArr[numElem - 1]);

/* adjust for even number of observations */

Pel tsOdd), ei
numE]1em++;
sum += h / 2 * (yArr[numElem - 2] +
yArr[numElem - 1]);
}
return sum;
}
double fAl1tExtSimpson(double xFirst, double xLast, abil 1aly,
double (*£x) (double) )
{
double h;
double x;
double sum = 0;
aimaten aise
Boolean isOdd = ((n % 2) != 0) ? TRUE FALSE;

cei (ray) << “AL0)))

return INTEGRAL
BAD RESULT;

/* increment n if it is even */
if (!isOdd)
Robt

/* calculate increment */
h = (xLast - xFirst) / (n - 1);

/* add terms */
x = xEiest.
for (a = Oly 3. < ms eee) 4
sum += (*£x) (x);
pede ee, Males
}
/* calculate integral */
sum = h / 48 *
(48 * sum -
S0 Fa ese) acs ey)
dak (ESE Geese to ae
5 MMSE rst ae i! ee
(*E)(kEirst + Sie*igh)) +
(*f2c) Gxbasitee= 3: * hh)
5 * (* iss) (Gehast =) 2% yes
Il * (*£x) (sbast3= hye
Si Ge Px ecsie)ye

return sum;
}
double GaussLegendreQuadrature (
double xFirst, double xLast,
int nSubIntervals,
double (*fx) (double) )
{
double xA, xB, xJ, h, hDiv2;
double sum, area = 0;
double Xk[6] = { -0.9324695142, -0.6612093865,
=0.2386191861, 0.2386191861,
 Numerical Integration 99

06612093865, 089324695142 };
double Ak[6] = { 0.1713244924, 0.3607615730,
0.4679139346, 0.4679139346,
OLS 607615730, OL 171s 244924 }-
int n = nSubIntervals; /* just a local alias for parameter */
arate |i calire

ho (xbast — xRirst) / n>

KA T= xa ei
BO taal = lies cress dae)
sum = 0's
i cA ee hare
IgIDBIN T= Nel // 2
/* obtain area of sub-interval */
EOE = sOne ee On ie) at
xO XA hDiv2 * (Xz) 1);
sume+= Aki) (efx )a(sa)) >
}
area += hDiv2 * sum;
DON et eye
}

return area;
}
double GaussLaguerreQuadrature(double (*fx) (double) )
{
double sum = 0;
double Xk[6] = { 0.22284660, 1.18893210,
QEII2ZT3 6638 7D aD L43 575
9..83746742), 15.98287398 };
double Ak[6] { 0.45896467, 0.41700083,
O13 37133357 102701039920),
0.00026102, 0.00000090 };
rts, Ls

fom (a= OF 2 < 67 +s)

sum Gh="Aloie= aC eEsoN
(ck Pasi

return sum;
}
double GaussHermiteQuadrature(double (*fx) (double) )
{
double sum = 0;
double Xk[6] i =2..S
50G0C9 Tie =e 3358491007
-0.43607741, 0.43607741,
1233584907, 2-385060497, };
double Ak[6] {10 00453001, 0.257 0673 2;
0.72462960, 0.72462960,
Ones 7067s2, 1000453 001, t};
strate) ls

for) (a= 40 a. < 16s. 2+)

Ssumet= Aki] * (FE) (xXk{s 1);

return sum;
i
double GaussChebyshevQuadrature(int n, double (*fx) (double) )
{
double sum = 0;
double x;
double pi = 4 * atan(1);
tits aie

for(i = te 4 <Sssnwea Df
RIGGS (ome) Ue aL) ep ase eae) 7
100 Chapter Five

Listing 5.2 (Continwed)

sum += (*f£x) (x);

}
return pi / n * sum;
}
double intPow(int base, int exponent)
{
Huighe™ ab Seay IEE
double pow = 1;

for (i = 1; i <= exponent; i++)

pow *= (double) base;
return pow;
}
Boolean Romberg(double xA, double xB, double tolerance,
int MaxIter, double (*£x) (double),
double* area)

enum opFlagType { goOn, converge, reachedMax };

double T[MAX_ROMBERG_TABLE] ;
double fnArr[MAX_ROMBERG
TABLE - 2];
double fA, h, cl;
int a gp miter, mSteps;
enum opFlagType opFlag;

opFlag = goOn;
niter = 0;
nSteps = 1;
h = xB - xA;
EA SSPE) (cA) ae) (fax) (8) a) 2
Ye POl)) Fao “EAS
*area = T[0];

while (opFlag == goOn) {

/* divide the step size by 2 */
niter++;
Lay fem PA,
nSteps *= 2;
el =O iOies

/* get new area estimate */

I= 0%
i = nSteps - 71;
do {
Int?
farce ee (Wisc)! (eA aoe ey
LE Sab
fnArr[i —- 2] = fnArr[nSteps / 2 - j - 1];
eR eet
} while (i >= 1);

LO] pe LAC
for (i = 1; i. <= (mSteps = 1); a+)
T[O] += fnArr[i-1];
1) Oy EE tale
/* perform Richardson's extrapolation */
for (i = 23). de<=) (niterts A) aes f
TLis1) (=9Tlae = 2)het (TIL = 2) = a) oy
(intPow(4, i- 1) - 1);
}

/* verify convergence */
if (fabs(T[nIter] - *area) <= tolerance)
opFlag = converge;
 Numerical Integration 101

else if (nIter >= MaxIter)

opFlag = reachedMax;

*area = T[nIter];
}
return (opFlag == converge) ? TRUE : FALSE;
}

Listing 5.2 implements the C functions declared in Listing 5.1. The functions that
implement the various versions of Simpson’s rules check if the number of elements
are odd or even. The functions handle the case of even number of array elements by
using only the smaller odd number of elements. Then, the function adds the extra
point using the trapezoid rule.
Listing 5.3 shows the source code for the test program TSINTEG.C. The listing de-
clares a number of test C functions and performs the following tasks:

1. Initializes the 100 elements of array yArr with values of x ranging from 0 to 1 in in-
crements of 0.01. The program uses a for loop which invokes the local function fx.
2. Tests the function Simpson to calculate and display the area of f(x) = e* — 3x? for x
= (to 1. This test uses the array yArr as one of the arguments to function Simpson.
3. Tests the function fSimpson to calculate and display the area of f(x) = e* — 3x?
for x = 0 to 1. This test uses the pointer to the C function fx as one of the argu-
ments to function fSimpson.
4. Tests the function AltExtSimpson to calculate and display the area of f(x) =
e* — 3x? for x = 0 to 1. This test uses the array yArr as one of the arguments to
function AItExtSimpson.
5. Tests the function fAltExtSimpson to calculate and display the area of f(x) =
ex — 3x? for x = 0 to 1. This test uses the pointer to the C function fx as one of the
arguments to function fAItExtSimpson.
6. Tests the function GaussLegendreQuadrature to calculate and display the area
under f(x) = e* — 3x? for x = 0 to 1. The program repeats the test with 1, 2, and
10 subintervals.
7. Tests the function Romberg to calculate and display the area under f(x) =
e* — 3x? for x = 0 to 1. This test uses the pointer to the C function fx as one or
the argument. The call to function Romberg obtains the result using the address
of variable area which is passed as the last argument to the function.
8. Tests the function GaussLaguerreQuadrature to calculate and display the area
under function f(x) = 1. The call to the function GaussLaguerreQuadrature has
the pointer to the C function fx2 as its sole argument.
9. Tests the function GaussHermiteQuadrature to calculate and display the area
under function f(x) = x”. The call to the function GaussHermiteQuadrature has
the pointer to the C function fx3 as its sole argument.
10. Tests the function GaussChebyshevQuadrature to calculate and display the area
under function f(x) = 1. The call to the function GaussChebyshevQuadrature
has the pointer to the C function fx4 as its sole argument.
102 Chapter Five

To compile the test program you need to include the files INTEGRAL.C and TSIN-
TEG.C in your project file. Figure 5.2 shows the output of the program TSINTEG.EXE.

Listing 5.3 The source code for the test program TSINTEG.C.
#include <stdio.h>
#include <math.h>
#include "global.h"
#include "integral.h"

#define ARRAY_SIZE 100

void pressAnyKey (void) ;

double fx(double x)
{
Beturn esis) ea Ss tex
}
double £x2(double x)
{
return 1;
}
double £x3(double x)
{
reburnr sey a
}
double fx4(double x)
{
return 1;
}
int main()
{
double yArr[ARRAY_SIZE + 1];
double deltaX = 0.01;
double =0" = @;
double x1 = 1;
double x = x0;
double area;
matahope eal ee

for(i = 0; i <= ARRAY_SIZE; i++) {

NidGane|
EI) es ids.dlG:e)
x += deltaX;
}

printf("Testing Simpson's rule\n");

printf("Area from %lg to %lg = %lg\n", x0, xl,
Simpson (ARRAY_SIZE + 1, yArr, deltaxX));
printf ("Area from tlg to 41g =, tlg\n", x0, x1,
fSimpson (x0, xi, ARRAY SIZE + 1, fx)?
printf("\nTesting Simpson's Alternative extended rule\n");
printf("Area from tlg to tlg = tig\n", xO, x1,
AltExtSimpson(ARRAY_SIZE + 1, yArr, deltaX));
printf("Area from %lg to %lg = %lg\n", x0, x1,
fALtExtSimpson(x0, x1, ARRAY_SIZE + 1, £x));
printf("\nTesting Gauss-Legendre Quadrature\n") ;
nm = Ls
printf ("Area from %lg to lg = lg (%d subintervals)\n",
SCO sae
GaussLegendreQuadrature(x0,
xl, n, £35) 5) mF
n= 2;
printf("Area from lg to %*lg = %lg (%d subintervals)
\n",
 Numerical Integration 103

ziOleeexalls,
GaussLegendreQuadrature(x0, x1, n, fx), n);
Tinescons Ole
printf("Area from lg to %lg = lg (%d subintervals)
\n",
(8, sally,
GaussLegendreQuadrature(x0, x1, n, £x), n);

printf("\nTesting Romberg's method\n") ;

if (Romberg(x0, x1, 0.001, 6, fx, &area))
print£t ("Area from tlg to %ig = Slg\n", x0, x1,
area);
else
printf("Function Romberg failed! \n");

pressAnyKey
();

printf("Testing the Gauss-Laguerre method\n");

Pinte ("Area from 0 to antinity a= sligin
GaussLaguerreQuadrature(fx2)
);
printf("Testing the Gauss-Hermite method\n") ;
Pra vAreas trom ONto) anirnetya—— sig \ialy
GaussHermiteQuadrature(f£x3)
);
printf("Testing the Gauss-Chebyshev method\n") ;
printf("Area from -1 to +1 = %lg\n",
GaussChebyshevQuadrature(10, f£x4));
return 0;
}
void pressAnyKey
()
{
printf£("\nPress any key to continue...");
getchar();
TOUAC Sie at) jae
y

Testing Simpson's rule

Area from 0 to 1 = 0.718282
Beas tcom OF tor le=s0.., 7182182

Testing Simpson's Alternative extended rule

Area from 0 to 1 0.718282
Area from 0 to 1 = 0.718282

Testing Gauss-Legendre Quadrature

Area from 0 to 1 = 0.718282 (1 subinterval)
Area from 0 to 1 = 0.718282 (2 subintervals)
Area from 0 to 1 = 0.718282 (10 subintervals)

Testing Romberg's method

Area from 0 to 1 = 0.718406

Testing the Gauss-Laguerre method

Area from 0 to infinity = 1
Testing the Gauss-Hermite method
Area from 0 to infinity = 0.886227
Testing the Gauss-Chebyshev method
Area from -1 to +1 = 3.14159
Press any key to continue...

Figure 5.2 The output of test program TSINTEG.EXE.

OF ‘ ‘ey i} Leumi rqial

Lig ne iy oe)
Why Salt
rro pe as ' ope ;f erke no-saena neers ree ‘ah

a TA" Fe ° a ' mm ge » pif Bi? ara ava! \Biniiwe

‘ a, me
iataean cP Mae ae Ee =e
~ aaa ar < eeil ee © giantess ge
BR
o
ie
f
feared 43 0 ,1°O.0 Se

specie nll i AN CE OTwt9:8mu Me ee

me gt
om : 7° :- Lae atin
Mi yineaten ‘tome? “Sts
- eres esitxdampell +
all pee will owe
; ><i , aia o UE or
o ot4
Fiat hate inde
0) otietee pss
a Gee ai. -*sfeuy

ah ihc beuskeaoa: deme

. a hose eels Bto eeei s now
Se ed
eks enb ekpua rpo ali n
Site lraftr

winger? OF vant qu

™“
—
oa

- ;
— ae
eee
i<ti¢ geese
TEAS)? ae F

duke, we ev! “aarp ly @

: aga ¥ @48
‘ A cpevyt

7.3 ani
ems 4) > <t¥
7

A wider 4
7 av ay

= &
nd

nm To
r

yg

Vog

ai
 Chapter

Solving Ordinary
Differential Equations

Numerical analysis provides valuable tools for solving ordinary differential equations
(ODE), especially those (and they are many!) equations that cannot be solved an-
alytically. The ODE methods fall into three categories of sophistication: simple, mod-
erate, and advanced. This chapter looks at popular methods which offer moderate
solution levels. You will learn about the following methods:

# The fourth-order Runge-Kutta method

=# The Runge-Kutta-Gill method
=» The Runge-Kutta-Fehlberg method

The chapter presents C source code for applying the above methods for solving
both single and multiple differential equations.

The Runge-Kutta Method

The third-order and fourth-order Runge-Kutta methods are among the popular meth-
ods which solve differential equations, yielding results that generally have accept-
able accuracy. The method starts with an initial set of values of x and y from which
subsequent values of y are calculated as a solution for the differential equation dy/dx
= f(x,y). Here are the equations involved in the fourth-order Runge-Kutta method:

Cert 2k, 2k ok,)

OS 0) a a (6.1)

k, =hiGg,, y,) (6.2)

105
106 Chapter Six

k= hal hee
Did n 9° Yn
FZ
5)
(6.3)
:

h k,
k= hil, + 97 Ynt =| (6.4)

k,=h (x, +h, y,, + ks) (6.5)

The variable h is the increment in x used to solve the differential equation. Here is
the algorithm for the Runge-Kutta method:
Given: the initial values x and y (namely, x0 and y0), the increment value H, the
array Yarr which stores the solution, the number of array elements N, and the func-
tion f(x, y).

Set x = x0 and y = y0.

Repeat the subsequent tasks for 1 =0to N-1.
Set kl =H * f(x, y).
H kl
Set
ec k2 =H *f (x+—Bye ——h
WDHre
Lr, |

5 seks =Het(x+ ys 2)
2 2
6 Set k4 =H * f(x+H,y+k8).
(kl + 2 * (k2 + k8) + k4)
7 Sety=yrt
6
8 Set Yarr[]] = y.
9 Add H to x.

You can easily modify the above algorithm to solve for an array of differential equa-
tions. To do this, use arrays for the variables H, x0, yO, x, y, kl, k2, k3, k4, and f(x, y).

The Runge-Kutta-Gill Method

The Runge-Kutta-Gill method offers better control on the round-off errors of the
fourth-order Runge-Kutta method. Here are the equations which define the Runge-
Kutta-Gill method:

Yau = Jn tb Kt KJ/6 + + G3 (6.6)

k, =hf(,, y,) (620)

K,
ra =n A(x
n a i? Yn +p) (6.8)
;

ky =ht(x, +2, y+ 05k, +64 Kp] (6.9)

 Solving Ordinary Differential Equations 107

Keni+h, yance. kc. k.) (6.10)

where c, = (1 -1/V2)/3, c, = (1 + 1/V2)/3, c, =-0.5 + 1/V2, ¢, = 1- 1/V2, ee

=1/V32, and c, = 1+ 1/V2. Here is the algorithm for the Runge-Kutta-Gill method:
Given: the initial values x and y (namely, x0 and y0), the increment value H, the
array Yarr which stores the solution, the number of array elements N, and the func-
tion f(x, y).

1 Set x = x0 and y = y0.

1
Zeoet Cl = \/9? 02 = 14 Cl, C8 =1—-Cl, C4=—0.5 + Cl, and C5 = C1.

3 Repeat the subsequent tasks for I = 0 to N— 1.

4 Set kl = H * f(x, y).

5 setka=Het(x+ Hy i)

6 Sot ka =Het(x+ By +04 +k + 03k]

7 Setk4=H*f(x +H, y+ C5 *k2 + C2 *k8).

grSeten- y + (kl + k4) . (C3 + k2 + C2 * KS) |
6 3
9 Set Yarr[I] = y.
10 Add H to x.

You can easily modify the above algorithm to solve for an array of differential equa-
tions. To do this, use arrays for the variables H, x0, yO, x, y, kl, k2, k3, k4, and f(x, y).

The Runge-Kutta-Fehlberg Method

The Runge-Kutta-Fehlberg method offers another refinement for the Runge-Kutta
method. This refinement supplies you with even better accuracy than the Runge-
Kutta-Gill method. The price, of course, includes more computational effort. The
Runge-Kutta-Fehlberg method uses the same notion as the Gill modification, except
the Fehlberg modifications are more elaborate and use 5 increment factors (kl
through k5) to update the value of y. Here is the algorithm for the Runge-Kutta-
Fehlberg method:
Given: the initial values x and y (namely, x0 and y0), the increment value H, the
array Yarr which stores the solution, the number of array elements N, and the func-
tion f(x, y).

1 Set x = x0 and y = y0.

2 Repeat the subsequent tasks for 1=0to N-1.
3 Set kl =H * f(x, y).

4 seta =Hat(x+ By i)
108 Chapter Six

5 Set
k3 = "ohade y+ 2 sQd+3*12))

- 1932 7200 7296 k3).

ss ie Het(x+ Ne ion «S107 Bot. 3)
7 Set k5= on i
439
deeryaen | uf8*k2+
3680
513 Pie 3 er
845
“104 * s)al)
25
25 1408
meses 2197 iesk5
B:Behiycs ay pebrhco
ae Sey ee 5
9 Set Yarr{[I] =
10 Add H to x.

You can easily modify the above algorithm to solve for an array of differential
equations. To do this, use arrays for the variables H, x0, yO, x, y, kl, k2, k3, k4, k5,
and f(x, y).

The C Source Code

Let’s look at the C source code which implements the above methods to solve single
and multiple differential equations. Listing 6.1 shows the source code for the ODE.H
header file. Listing 6.2 shows the source code for the ODE.C implementation file.

Listing 6.1 The source code for the ODE.H header file.
#ifndef _ODE_H_
#define _ODE_H_

Ha
Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 8/4/94

Module: C routines which support the following ODE methods:

+ Runge-Kutta fourth-order method

+ Runge-Kutta-Gill method
+ Runge-Kutta-Fehlberg method

The library offers routines which integrate a single or

a vector of ODEs for each of the above methods

yu

#include <math.h>

void RungeKutta4 (double x0, double y0, double deltaX,

double* yArr, int numElem,
double (*fx) (double, double)
);

void RungeKuttaGill (double x0, double y0, double deltax,

double* yArr, int numElem,
double (*fx) (double, double)
);

void RungeKuttaFehlberg(double x0, double yO,

double deltaX, double* yArr,
int numElem,
double (*fx) (double, double));
 Solving Ordinary Differential Equations 109

int numElem,
double (*fx[]) (double, double*));

void VectRungeKutta4 (double x0, double* y0, double deltax,

double* yl, int numYVar,
double (*fx[]) (double, double*));

void VectRungeKuttaGill (double x0, double* y0, double deltax,

double* yl, int numyYVar,
double (*fx[]) (double, double*));

void VectRungeKuttaFehlberg(double x0, double* y0,

double deltaX, double* yl,
int numyYVar,
double (*fx[]) (double, double’*));

#endif

Listing 6.1 declares the following C functions which implement the above methods
for single and multiple differential equations. The main difference between the two
sets of C functions is that the ones which solve multiple differential equations only
calculate the y values for the next x value. Therefore, you need to include these C
functions in a loop to obtain a series of y values. Here are the declared C functions:

i The function RungeKutta4 implements the fourth-order Runge-Kutta method.

The function has parameters which specify the starting x and y values, the incre-
ment in x, the array of y solutions, the number of array elements, and the pointer
to the differential equation. The function fills the specified number of array ele-
ments with the solution of the specified differential equation.
. The function RungeKuttaGill implements the Runge-Kutta-Gill method. The
function has parameters which specify the starting x and y values, the increment
in x, the array of y solutions, the number of array elements, and the pointer to the
differential equation. The function fills the specified number of array elements
with the solution of the specified differential equation.
. The function RungeKuttaFehlberg implements the Runge-Kutta-Fehlberg method.
The function has parameters which specify the starting x and y values, the incre-
ment in x, the array of y solutions, the number of array elements, and the pointer
to the differential equation. The function fills the specified number of array ele-
ments with the solution of the specified differential equation.
. The function VectRungeKutta4 implements a vector-version of the fourth-order
Runge-Kutta method. The function has parameters which specify the arrays
starting x and y values, the increment in x, the array of y solutions, the number of
differential equations, and the pointer to the array of differential equations. The
C function stores the values of y for the next x value in the parameter yl.
. The function VectRungeKuttaGill implements a vector-version of the fourth-order
Runge-Kutta-Gill method. The function has parameters which specify the arrays
starting x and y values, the increment in x, the array of y solutions, the number of
differential equations, and the pointer to the array of differential equations. The
C function stores the values of y for the next x value in the parameter yl.
110 Chapter Six

6. The function VectRungeFehlberg implements a vector-version of the fourth-order

Runge-Kutta-Fehlberg method. The function has parameters which specify the
arrays starting x and y values, the increment in x, the array of y solutions, the
number of differential equations, and the pointer to the array of differential equa-
tions. The C function stores the values of y for the next x value in the parameter yl.

Listing 6.2 The source code for the ODE.C implementation file.

#include <stdlib.h>
#include "mode.h"
#include "global.h"

void RungeKutta4(double x0, double y0, double deltax,

double* yArr, int numElem,
double (*fx) (double, double) )

double x, y, halfDelta;
double k1, k2, k3, k4;
sbayop aie

if (numElem < 1)
return;

x = x0?
y = yO;
halfDelta = deltaX / 2;
for (= 20s. ae <emum
Blemisy sto) eet
ki = deltaxs * (*£x) (x, wr
k2 = deltaX * (*fx)(x + halfDelta, y + k1 / 2);
k3 = deltaX * (*fx)(x + halfDelta, y + k2 / 2);
k4 = deltaX * (*fx)(x + deltaX, y + k3);
yea (KL te 2 (k2 Se 3) eke) os
Gaiveng ES Be)! Sie
x += deltaX;

}
void RungeKuttaGill (double x0, double y0, double deltax,
double* yArr, int numElem,
double (*fx) (double, double) )

double x, y, halfDelta;
double k1, k2, k3, k4;
double cl = 1 / sqrt(2);
double e2)= 1°+ cl;
double c3 = 1 —- cl;
double c4 = -0.5 + cl;
double c5 = -cl;
eat amie

if (numElem < 1)
return;

x = x0;
y = y0;
halfDelta = deltaX / 2;
for (i = 0; i < numElem; i++) {
kl = deltaX * (*fx) (x, y);
k2 = deltaX * (*fx)(x + halfDelta, y + kl /.2);
k3 = deltaX * (*fx)(x + halfDelta, y + c4 * kl + c3 * k2);
k4 = deltaX * (*£x) (x + deltak,y + cb.2 °k2"+ 162 ** 3);
yeow= (kl + ka) f ce (E34 Kk) S48 kee Se
 Solving Ordinary Differential Equations 111

A A(yAsers 4208) = “yi

x += deltaX;

}
void RungeKuttaFehlberg(double x0, double y0, double deltax,
double* yArr, int numElem,
double (*fx) (double, double) )

double x, y;
double) Ki k27) 37 k4, k5?
Int 1;

if (numElem < 1)
return;

ean Oye
y = y0;
for (i = 0; i < numElem; i++) {
k1 ISIN SED.G ty ((osbt)) (rep aia)7
k2 Gel taxak*@ (E50) (sc + deltax / 4, yuki / 4):
ke="delitakx * S(Atsa) Gc + Bo/ Se deltaX,
y ae Owon “a tkiLet 3. *. ik2') )i5
k4 I dettax * (*fsxc))
(x + 2s Celta,
Vrs 2/297. eke
TA00R/ ZEST *) 12m + 72916.0/ 29 7] tare KS)
k5 = deltaX * (*£x) (x + deltax,
Vie 96/2 UGres ki —) Be xe ke!
S680m/ SLs +e ks mn OAD ey AOA aca) us
Vi = 2 5e/
2 LO Kiet OiGr/ 2 565s Kaa Oe (AOA ee kan k5/ 5)
A(VATIG + 2) = yz
x += deltaX;

}
void VectRungeKutta4 (double x0, double* y0, double deltax,
double* yl, int numyYVar,
double (*fx[]) (double, double®*) )

double x, *y, halfDelta;

qaoubler Aki *k27 ee ks “kee
Ants ale

if (numYVar < 1)
return;

// allocate local dynamic arrays

k1 = malloc(numYVar * sizeof (double) ) ,
k2 = malloc(numYVar * sizeof(double) );
k3 = malloc(numYVar * sizeof(double) );
k4 = malloc(numYVar * sizeof(double) );
y = malloc(numYVar * sizeof (double) ) ;

= SCO
for (i = 0; i < numYVar; i++)
wives a) AQP ak)
halfDelta = deltax / 2;
fox as = Of) a < MUMMY Vary a+)
(kL + a), = dellttax * (*£x(1])
(x, y)F
for (i°= 0; i < numYVar; i++)
yp eo ah esas CIS a a) 7/2
for (a = Os a. < numyVvar; 2++)
(2 + 1) = deltax * (*£x[il)(x + halfdbelta, y);
fom (i = Oj; 2 <= numYVar; 1++)
SoC yePaty) MimeCWO) ted) eke IK) 8a) / 2.
112 Chapter Six

Listing 6.2 (Continued)

for (f= 107 & < numyVare 24+)

*(k3. + 2) = deltaX * (*£x2]))(% + HalfDelta, yy);
for (i = 0; i < numYVar; i++)
olOScecera 8)Yaa eaTNO) oe ott) cas ar oes} tet tls)i
form (= Of 2S numMVvar 14+)
*(ke + 3) = deltas (FM fsca
|) Gee deltax; 4)
for (2 = 0; 2 < numyVar. Tt)
a(yl + 4) = *(yO + i) + (* (el + a) |
ok AED oa) a RS aa yee
PGA ee a) he Gy
// deallocate dynamic arrays
free(k1) ;
free(k2);
free(k3);
free(k4);
free(y);
}
void VectRungeKuttaGill(double x0, double* y0, double deltax,
double* yl, int numyYVar,
double (*f£x[]) (double, double*) )

double x, *y, halfDelta;

Goubilkes <isly etka Sco Suk
double el = ihy/ sqre (2);
double «2 = 1 + el;
double «3 = 1 = el;
double® c4 = -—0.5 +: .cl;
double c5 = -cl;
Miqieme 1

Le (numyMvians —< 1)
return;

// allocate local dynamic arrays

k1 = malloc(numYVar
( * sizeof (double) ) ,
k2 = malloc(numyYyVar * sizeof(double) );
k3 = malloc(numYVar * sizeof(double) );
k4- = malloc(numYVar * sizeof (double) ); ’
y = malloc(numYVar * sizeof(double) );

eee 49
for (i = 0; i < numYVar; i++)
muy Si) aan OO eae)ay
halfDelta = deltaX / 2;
for (1 = O+ 2 < numYVary 247)
* (kL + 1) Ssideltax * (isc
ii Geary ey,
for (1 = Of 4 < numyVars) i++)
ey + a) f= Cea Sa eee
for) (as = Oi. do UM Velo metre)
*(k2 + 1) = deltaX * (*£x[i]))(x + halfDelta, vy);
for (1 = 0; 1 < numYVar; 1++)
*(y +o) = *(yO + 1) +.e4 * *(kL + a) # GS * *(k2 see
for (1 = 0; 2 < numYVars a++4)
*(k3 + 4) = deltax © (*fx[21)(x + hallfibpedita, 3)%
for(i = O07; 2 < numYVers i+)
*(y + 1) = *(yO+ 2) + Gb * (KBs 2 aaa ea ee ma
for (i = 0; 2 < numYVar; i++)
*(k4 + 4) = deltax * (*£Ex[i]) (x + deltax, y);
for (i = 0; i < numYVar; i++)
(vl + 1) = *(yO + 1) + (* (KD # 2) ¥ * (kA 4+ 2)) / 6 &
(CCM NGK2 Pa) ay eZ Seed eae
// deallocate dynamic arrays
free(kl);
 Solving Ordinary Differential Equations 113

free (k2);
free(k3);
free (k4);
bree (iyi;
}
void VectRungeKuttaFehlberg(double x0, double* y0,
double deltaX, double* yl,
int numyYVar,
double (*fx[]) (double, double’*) )

double so, yy:

Goublier tiie, stk AicSe eA eekbis
Werte es

if (numYVar < 1)
return;

// allocate local dynamic arrays

k1 = malloc (numYVar sizeof (double) ) ‘
k2 = malloc (numYVar sizeof (double) );
k3 = malloc (numyVar sizeof (double) );
k4 = malloc (numYVar sizeof (double) ) /
k5 = malloc (numYVar Ae
EE
oh sizeof (double) ) ,
y = malloc(numYVar * sizeof (double) );

x= x0:
for (i = 0; i < numYVar; i++)
Byatt 2) =) Gy O) ea) iy
Rome (ge = Or a << numy Vater acts)
* (ela) =
deilltaxe (real)
(a7 sys
for (zx = 0; i < numYVar; 2++)
R(y +o) = * (yO eileen
(KL aly 74;
for (2 = 05 1 < numYVans, i++)
A(e2 + 2) = delitax (+ £21)(x + deltax / 47 3
7);
for (i = 0; i < numYVar; i++)
22 ie Boia) eat EAGAN) Ge ak) “aS SSI ASR Mor (arial San) Viel et BLA etlGloeyaeic ety) ipJere
for (1 = 0; i < numYVar: i++)
- (koteba) e=ndedizax, ) (etx [Fail\) (Ge 3.../8) 4s eaeditax ys ay)
for (2 = Ol 2 < numYVar; i++)
Bi(sy7 ety cle) RN COME) mete OS QT pa Oi ae CIE ean) a
7200./2197 * *(k2 4 i) +
TANS
sy) PET ee USS) do BENG
for (2 = 0; i < mumYVar; i++)
A(kter e=deltax ss (tx) (xt t2./ 13 =* delltax;, y)'>
EON an = Ope ei oemYeVelts me tet)
CA((ie Eb SSE)E cer (QUAD eis BUN ae CUNY ea oe NG UPC a)
Sieg (keer) ee
BGS Oey alk ieaes Keon)
SAS ia Oe ae(Kae) is
for (i = 0; i < numyYVar; i++)
E(K5 + 2) = deltax * (*fx[a])(xk + deltax, y);
for (2 7="0F 2 < numyYVary 2++)
(yee) eet (710) ely ect Dro O es * ((kL +e) eer
AAO LSS WAS) Gl 22h as J19)) ae
DON wie ©) Aeon (ice tact) ae
EId(nds) eae) Wh Sip

// deallocate dynamic arrays

free (k1);
free(k2);
free (k3);
free(k4);
free (k5);
free(y);
114 Chapter Six

Listing 6.2 implements the C functions declared in Listing 6.1. Notice that the
functions which solve multiple differential equations use for loop to update the co-
efficients k1 through k5 and other intermediate values. In addition, the functions use
local dynamic arrays to store the arrays of intermediate values. The C functions cre-
ate and remove these temporary dynamic arrays using the standard C functions mal-
loc and free, respectively.
Let’s look at the test program. Listing 6.3 contains the source code for the test pro-
gram TSODE.C. The listing declares a set of C functions used to test the various
methods for solving differential equations. The program performs the following tasks:

1. Tests the function RungeKutta4 to solve the differential equation dy/dx = —xy?.
The program supplies (2, 1) as the starting point, and uses the increment of
0.001. The test uses a do-while loop to call function RungeKutta4 while x is less
than 3. Thus, the solution enjoys a very good level of accuracy. The program
stores the data for y in the array yArr and displays only the last value of that ar-
ray in each iteration.
. Tests the function RungeKuttaGill in a manner very similar to testing function
RungeKutta4.
. Tests the function RungeKuttaFehlberg in a manner very similar to testing func-
tion RungeKutta4.
. Tests the function VectRungeKutta4 to solve the two differential equations dy/dx
= y,, and dy/dx = y, + x. The test supplies the initial values for x, y,, and y, as 0,
1, and —1, respectively. The test uses an increment in x of 0.2, and loops while the
value of x is less than 2.
. Tests the function VectRungeKuttaGill in a manner very similar to testing func-
tion VectRungeKutta4.
. Tests the function VectRungeKuttaFehlberg in a manner very similar to testing
function VectRungeKutta4.

To compile the program include files ODE.C and TSODE.C in your project file.
Figure 6.1 shows the output of program TSODE.EXE. This output contains the solu-
tions to the differential equations used in file TSODE.C.

Listing 6.3 The source code for the test program TSODE.C.
#include <stdio.h>
#include <math.h>
#include "global.h"
#include "ode.h"

#define ARRAY SIZE 100

#define NUM_SIMUL 2

void pressAnyKey (void) ;

double deriv(double x, double y)

{
returm =x * y **y;
}
 Solving Ordinary Differential Equations 115

double fl1(double x, double* y)

{
igewebhad “Gi Go al)\y
}
double £2(double x, double* y)
{
return *y + x;
}
int main()
{
double yArr[ARRAY_SIZE];
double x00 = 2;
double y00O = 1;
double x0;
double xn = 3;
double y0;
double deltaX = 0.001;

double (*f£x[NUM_SIMUL]) (double, double*);

double yArr0 [NUM_SIMUL] ;
double yArri1[NUM_SIMUL] ;
Geyaite odes

printf("Testing the Runge-Kutta method\n") ;

x0 = x00;
yO = y00;
Daneel ("x = tg. Vv = silo Nee x0) 70)
do {
RungeKutta4(x0, yO, deltaX, yArr, ARRAY_SIZE, deriv);
x0 += ARRAY SIZE * deltaX;
yO = *(yArr + ARRAY SIZE - 1);
primes ("sx = Slog, eye ee \ia" 7) x0, yO) >
} while (x0 < xn);

pressAnyKey
();

printf ("Testing the Runge-Kutta-Gill method\n") ;

Oi ScO10),
yO = y00;
jonernah
one wed = Cyilrep, aye => eile walle >0) a0)ie
do {
RungeKuttaGill(x0, yO, deltaX, yArr, ARRAY_SIZE, deriv);
x0 += ARRAY SIZE * deltaX; :
yO = *(yArr + ARRAY SIZE — 1);
joneniahea (Ube 8 Syiltes ai Kore wall Sid0)e Sedo)TE
} while (x0 < xn);

pressAnyKey
() ;

printf ("Testing the Runge-Kutta-Fehlberg method\n") ;

x0. = x00
yO = y00;
PeINnEE(ss= Slg. ve= sig\ni", x0, v0) 5
do {
RungeKuttaFehlberg(x0, yO, deltaX, yArr, ARRAY_SIZE, deriv);
x0) += ARRAY SiZB * deltax;
vO = *(vyArr + ARRAY. SIZE — i)>
Pe eta e te Sian ane oN “ee Ol 70,)u2
} while (x0 < xn);

pressAnyKey
() ;

printf (
"Testing the Runge-Kutta method to solve two ODEs\n");
SQ)
= i0'e
*VArrO) = 1;
116 Chapter Six

Listing 6.3 (Continued)

e(yArcO’ +L) = =2;

deltaX = 0.2;
(sense MONI) Peed
(fix11]) = £2;
DBE (x =) Silicn ey = Sig, v2 SseeLoNn
Ob yearsOey yeni sa le) oie
do. {
VectRungeKutta4(x0, yArr0, deltaX, yArrl, NUM_SIMUL, ffx);
x0 += deltaX;
DeLee (re Soy ye = Slig 7 v2 =) Soe
B10 a anya
NselecaeDg tn (Yi NGoslla il)
for (i = 0; i < NUM_SIMUL; i++)
Slay Arr O) eet) Pn FiyeAraredl tom 8)
} while (x0 < 2);

pressAnyKey() ;

printf (
"Testing the Runge-Kutta-Gill method to solve two ODEs\n");
xO) = Oe
tang: \rajal Ome Le
* (yArrO) ae ok) = a
deltaX = 0.2;
(aise) ((O))})) == aeibe
(EES Sre2s
PEINtE( "= Big, y= Slope. tL ota,
OF. My Are O!, Ayan Oi ss) his
do {
VectRungeKuttaGill(x0, yArr0, deltaX, yArrl, NUM_SIMUL, ffx);
x0 += deltaX;
prince (** = 26 yl = cave selon:
xO) SyArEL, (AipyArrii ssl)re
for (i = 0; i < NUM_SIMUL; i++)
CVA. oe el) se) Aa a ee eth)
} while (x0 < 2);

pressAnyKey
() ;

print
é(
"Testing the Runge-Kutta-Fehlberg method to solve two ODEs\n");
x0 =O);
*yArr0 = 2
Pi(yArLO 2) te
deltaX = 0.2;
(f££x[0]) = £1;
(fis ll j= 82;
printi("« x = tig, yi = slg, yo alone,
x0, *SyArro; *(yvArrOee J )0)s
do {
VectRungeKuttaFehlberg(x0, yArr0O, deltax,
yArr1, NUM_SIMUL, ffx);
x0 += deltaX;
printé("s. = tlg, yk = sige v2 = slain,
ROP eyArels. * (yare Sys
for (i = 0; i < NUM_SIMUL; i++)
(vA £2.) See (vArrd spe
} woille™ (x0°<"2))+

pressAnyKey
() ;

return 0;
}
void pressAnyKey()
{
 Solving Ordinary Differential Equations 117

printf("\nPress any key to continue...");

getchar();
Puls (CUNT) Es
}

Testing the Runge-Kutta method

=2; | al
ke
- 829876
- 704225
. 607903
oS L9L5
-470588
-420168
23 18072
342466
312012
ODINIDUORWNPR
ts kl
Ke
Rash
Hatake
te WNNNNNNNNN
kK
~

Press any key to continue...

Testing the Runge-Kutta-Gill method

= ee ke I al
0.829876
0.704225
0.607903
Ors aos
0.470588
0 -420168
0 -378072
0 - 342466
WCOIDAURWNE
0 ~ 3221012
Re
SS
4 N o Kea KKK 2 85714
KK
IKK
WNHONNNNNNN
ES
sr
Te
tS

Press any key to continue...

Testing the Runge-Kutta-Fehlberg method

= ae

0.829876
0.704225
0.607903
Ons 30925
0.470588
0.420168
0 -378072
0 - 342466
0 5 sO
O©DIDUOBWNHP
|
te ND KG
Me
KR
Hi 2 85714
WNHONNNNNNDN
cs
K~.

Press any key to continue...

Testing the Runge-Kutta method to solve two ODEs

= yVies ieey2 = =i
. OL 820067 7 y20 = -0-79 8.667
ORGS Oiler 2 =), 539253
0.585462, y2 =0.363355
OnEN KKM
KS OS53a 7428 ye -0.111907
Ee ~543068y2e= Oe 75183
kK“
~
es
iow 0.610636, y2 0.509436
0.750869, y2 0.904267
ORO TA227 ayi2— ib. 3) 552
anFrN KKK ees On Ade yee ele oo Lal
PPPPPHTPRPPR
By OO nly, Va) ae Oa OU)
2aa
Do
oo SS
PSS
PLS)
Sy)
(er
to)
te
SPSS
| tT) 223678,
{Vite ¥2 =) 3.45699

Figure 6.1 The output of program TSODE.EXE.

118 Chapter Six

Press any key to continue...

Testing the Runge-Kutta-Gill method to solve two ODEs

x = 0, yl= iy, Sal
x = 02, wal =
0.820067, y2 = -0.798667
x 0.681071, y2 -0.589253
x oreows 0.585462, y2 = -0.363355
x kG PPP 0.537428,
kg
ke y2 -0.111907
x 2543068, v2 = 0.175183
x 0.610636,
"oll y2 0.509436
x 0.750869, y2 = 0.904267
x = Wasim. x72) = il sivlsysy)
x FPN
an eS
OM ANar eee tO Aas
x LSPA 2) S AAG a
x AlDesOuSey a) =e 25699
NNRPRPRPRPRO

Press any key to continue...

Testing the
Runge-Kutta-Fehlberg method to solve two ODEs
ey 2. = i
«2 , = 0.820067, y2 = -0.798664
-4, = 026810727 y2 -0.589247
-6 ' 0.585465, y2 -0.363345
18 , iKKK OS538 743865) v2 -0.111892
ke bh DASUS27 Van =) Ost S204
tiath0.610658,
eo y2 0.509465
0.750902, y2 0.904306
On97T7469> wa =. 37 557
anrN 1.30743), y2 = 1.94218
Ke - PPePPPHPRPPR
SI O22 V2 OOO
esNNFERPPROGDOCO
CS NS) < as air 2 SOU 2a ye Sea oe dee

Press any key to continue...

Figure 6.1 (Continued)

 Chapter

Optimization

Optimization involves finding the values of variables which yield a minimum or max-
imum function value. This chapter looks at methods which minimize functions with
single and multiple variables. You will learn about the following methods:

= The bisection method for minimizing a single-variable function.

= Newton’s method for minimizing a single-variable function.
# The Golden Section search method for minimizing a single-variable function.
= The quadratic interpolation method for minimizing a single-variable function.
= The cubic interpolation method for minimizing a single-variable function.
= The flexible simplex method for minimizing a multivariable function.
= Newton’s method for sequential optimization of a multivariable function.

The Bisection Method

The bisection method for finding a minimum works much like the bisection method
which locates a root of a function. The method starts with an interval that contains
the minimum and then halves that interval to zoom in on the minimum value. Here
is the algorithm for the bisection method:
Given: The interval [A, B], which contains the minimum value for function f(x), the
first derivative of the minimized function, f'(x), and the tolerance level (Tol).

1 Set Fa = f(A).
2 Set Fb = f(B).
3 Repeat the next steps until |A — BI > Tol:

pcan).
:E)
119
120 Chapter Seven

DC 5G)
3.3 if f'(C) * f'(A) > 0 then Set A = C and Fc = Fa, else set B = C and Fc = Fb.
4 Return the minimum value as C.

Figure 7.1 depicts an iteration of the bisection method.

Newton’s Method
You can use Newton’s root seeking method to find a minimum, maximum, and saddle
point since the derivative of the targeted function is 0 at these points. Here is the al-
gorithm for Newton’s method:
Given: The minimized function f(x), the initial guess for the minimum (X), the tol-
erance level (Tol), and the maximum number of iterations (N).

1 Set Iter = 0.
2 Repeat the following steps until Idiffl < Tol or Iter > N:
2.1 If [Xl > 1 then set H = 0.01 * X, else set H= 0.01
2.2 Set Fm = f(X — H)
2.3 Set FO = f(X)
2.4 Set Fp = f(X + H)
(Fp — Fm)
2.5 Set FirstDeriv = (2*

Next Interval

C Current Interval
__ a a ae SF )

Figure 7.1 An iteration of the bisection method.

 Optimization 121

2.6 Set SecondDeriv = Eee Ew)

(H * H)
FirstDeriv
a et. ditt =
SecondDeriv
2.8 Subtract diff from X
2.9 Increment Iter.
3 Return X as the minimum if Iter < N. Otherwise return an error code.

The above version of the algorithm approximates the first and second derivatives
of the targeted function.

The Golden Section Search Method

The Golden Section Search method uses the same basic principle as the bisection
method in locating the minimum in an interval. The Golden Section Search method
uses an interval reduction factor that is based on the Fibonacci numbers, instead of
selecting the mean interval value. Here is the algorithm for the Golden Section
Search method:
Given: The interval [A, B], which contains the minimum value for the function f(x),
the tolerance level (Tol), and the maximum number of iterations (N).

1 SetIter
= 0.
5-1
2 Sett= HVDay
2
3 SetC=A+(1-t) * (B-A).
4 Set Fc = f(C).
5 Set D=B-(1-t) * (B-A).
6 Set Fd = f(D).
7 Repeat the next steps until IB
— Al > Tol and Iter < N:
7.1 Increment Iter++
7.2 IfFc < Fd then set B=D, D=C,C =A+ (1 -t) * (B—A), Fd = Fc, and Fc
= f(C); else set A= C; C=D, D=B- (1 -t) * (B=A), and Fc = Fc = Fd,
and Fd = f(D).

8 Return —
B) as the minimum if Iter < N. Otherwise return an error code.

Figure 7.2 depicts an iteration of the Golden Search method.

The Quadratic Interpolation Method

The quadratic interpolation method attempts to locate the minimum of a function by
zooming in on the minimum, instead of reducing the interval that contains the mini-
mum. The method uses three guesses for the minimum for the interpolation. Here is
the algorithm for the quadratic interpolation method:
Given: the points A, B, and C, which are initial guesses for the minimum of func-
tion f(x), the tolerances for the X values (Xtol), the tolerance for the function (Ftol),
and the maximum number of iterations (N).
122 Chapter Seven

Figure 7.2 An iteration of the Golden Search method.

1 Set Iter = 0.
2 Set IterFlag to false.
3 Set Fa = f(A).
4 Set Fb = f(B).
5 Set Fc = f(C).
6 Repeat the following steps until ([terFlag is true or |Chasis —Ec) See
LastFc
Iter> N:
6.1 Increment Iter
6.2 Set LastFc = Fc
6.3 Set X=
0.5* (B*B-C*C) *Fa+(C*C—Ax*
A) *Fb+(A*A-—B*B)) * Fe)
((B—C) * Fa + (C—A) * Fb + (A—B) * Fc)
6.4 Set Fx = f(X)
6.5 Test the following conditions:
6.5.1 If X < C and Fx < Fc then set B = C, C = X, Fb = Fe, and Fc = Fx
6.5.2 Else If X > C and Fx > Fc then set B = x and Fb = Fx
6.5.3 Else If X < C and Fx > Fc then set A = X and Fa = Fx
6.5.4 Else set A = C, C = X, Fa = Fc, and Fc = Fx
6.6 If |A—Cl < Xtol or IC — BI < Xtol then set IterFlag to true.
7 Return X as the minim if Iter <= N. Otherwise, return an error code.

The Cubic Interpolation Method

The cubic interpolation method attempts to locate the minimum of a function by
zooming in on the minimum, instead of reducing the interval which holds the mini-
 Optimization 123

mum. The method uses two guesses for the minimum, and estimates the slope at
these points to perform the interpolation. Here is the algorithm for the cubic inter-
polation method:
Given: The points A and B, which are initial guesses for the minimum of the func-
tion f(x), the tolerances for the X values (Xtol), the tolerance for the first derivative
(Gtol), and the maximum number of iterations (N).

1 Set Iter = 0.
2 Set Fa = f(A).
3 Set Fb = f(B).
4 Set Ga = f'(A).
5 Set Gb = f'(B).
6 Repeat the next steps until |B — Al < Xtol or Gmin < Gtol and Iter > N:
6.1 Increment Iter

6.2 Set w= 2
(B — A) * Fa- Fb) + Ga + Gb

6.3 Setv = V (w* w-—Ga

* Gb)
A+(B-A) *(-(Gb+v-w)
6.4 Set X=
(Gb
— Ga + 2 * v))
6.5 Set Fx = f(X)
6.6 Set Gx = f'(X)
6.7 If Ga < 0 and (Gx > 0 or Fx > Fa) then Set B = X, Fb = Fx, and Gb = Gx, else
set A =X, Fa = Fxyand Ga= Gx
6.8 If |Gal > |GblI then set Gmin = Gb, else set Gmin = Ga.
7 Return X as the minimum if Iter <= N. Otherwise, return an error code.

The Simplex Method

The simplex method allows you to find the minimum for a function with multiple
variables. The method obtains the minimum for a function with N variables by ex-
amining the function values at N+1 points. The method locates the points with the
best and worst values, and then attempts to replace the worst-value point with a bet-
ter point. This replacement process involves expanding and contracting the simplex
near the worst-value point to determine a better replacement point. The iterations
of the method shrink the multidimensional simplex around the minimum point. Fig-
ure 7.3 depicts the iterations of the simplex method.
Here is the algorithm for the simplex method:
Given: N variables that evaluate function f(X). Also given are the matrix X that
contains N + 1 rows and N columns, the array of function values Y (containing N + 1
values), the tolerance factor (Tol), and the maximum number of iterations M. Also,
the reflection, contraction, and expansion factors are given. The reflection factor is
a positive value that may be unity. The expansion factor is greater than 1 (usually 2).
The contraction factor is a value between 0 and 1 (usually 0.5).

1 Set Iter
= 0.
2 Set P=N +1.
124 Chapter Seven

Figure 7.3 The iterations of the Simplex method.

3 Create the dynamic arrays X1, X2, and Centroid to contain N elements.
4 Calculate the function values for array Y.
5 Set convergence flag to false.
6 Repeat the next steps while Iter < M and convergence flag is false:
6.1 Increment Iter
6.2 Find the indices of the best and worst points
6.3 Calculate the array of centroids for the various points, except the worst
point. Store the array in Centroid
6.4 Calculate reflected point: for i = 0 to N—1, set X1[i] = (1 + reflectionFac-
tor) * Centroid[i] — reflectionFactor * X[worstl, i]
6.5 Set Y1 = function value at point X1
6.6 If Yl < best Y perform the following tasks:
6.6.1 Calculate expanded point: for i = 0 to N — 1, set X2[i] = (1 + expan-
sionFactor) * X1[i] — expansionFactor * Centroid[i]
6.6.2 Set Y2 = function value at point X2
6.6.3 If Y2 < best Y then replace worst point X with point X2. Otherwise,
replace worst point X with point X1. Resume at step 7
6.7 If Yl >= best Y test if Y1 is not greater than the values of array Y (except
the worst Y) replace the worst X with X1 and resume at step 7. Otherwise,
perform the next steps:
6.7.1 If Yl > worst Y replace worst X with point X1
6.7.2 Calculate contracted point: for i = 0 to N — 1, set X2[i] = contraction-
Factor * X[{worstl, i] + (1 — contractionFactor) * Centroid{i]
6.7.3 Set Y2 = function value at point X2
 Optimization 125

6.7.4 If Y2 <= worst Y replce worst X by point X2. Otherwise for i= 0 to N

—1, set X[i] = (X[i] + best X)/2.
7 Calculate array Y for current matrix X.
8 Set Ymean = mean value of array Y.
9 Set sum = 0.
10 Fori=0 to N —1, add (Y[i] = Ymean)? to sum.
S ;
LS Oe eh < tolerance, set convergence flag to true. Otherwise, set the con-

vergence flag to false.

12 Move best X and Y to index 0.
13 Delete the dynamic arrays X1, X2, and Centroid.

Newton’s Method for Sequential Optimization

It is possible to extend a single-variable optimization method, such as Newton’s
method, to work with multiple variables. The extended method performs repeated
sequential optimization on the variables, one at a time. At the end of each cycle the
method tests for convergence.

The C Source Code

Let’s look at the C source code which implements the optimization library. Listing 7.1
shows the source code for the OPTIM.H header file. The file declares the following
functions:

= The function BisectionMin applies the bisection method to return the minimum
value of a function. The function has parameters that specify the interval con-
taining the minimum, the tolerance, and the pointer to the targeted single-vari-
able function.
. The function NewtonMin uses Newton’s method to return the minimum value of
a function. The function has parameters that specify the initial guess for the min-
imum, the tolerance, the maximum number of iterations, and the pointer to the
targeted single-variable function.
. The function GoldenSearchMin applies the Golden Section Search method to re-
turn the minimum value of a function. The function has parameters that specify
the interval that contains the minimum, the tolerance, the maximum number of
iterations, and the pointer to the targeted single-variable function.
. The function QuadIntMin uses the quadratic interpolation method to yield the
minimum value of a function. The function has parameters that specify the initial
three guesses for the minimum, the tolerance for the independent variable, the
tolerance for the function values, the maximum number of iterations, and the
pointer to the targeted single-variable function.
. The function CubeIntMin uses the cubic interpolation method to return the min-
imum value of a function. The function has parameters that specify the initial two
126 Chapter Seven

guesses for the minimum, the tolerance for the independent variable, the toler-
ance for the slope values, the maximum number of iterations, and the pointer to
the targeted single-variable function.
6. The function CalcYSimplex calculates the values of the function for a given data
matrix. The function has parameters that specify the independent variable’s data
matrix, the array of calculated function values, the number of variables, and the
pointer to the multivariable function.
7. The function Simplex applies the simplex method to locate the minimum value
for a multivariable function. The function has parameters that specify the inde-
pendent variables’ data matrix, the array of calculated function values, the num-
ber of variables, the tolerance, the reflection factor, the expansion factor, the
contraction factor, and the pointer to the multivariable function. If the argument
for the reflection factor, the expansion factor, or the contraction factor is zero,
the function assigns a default value for that factor.
8. The function NewtonMultiMin sequentially minimizes a multivariable function.
The function has parameters that pass the initial guess for the minimum point,
the number of variables, the tolerance level, the maximum number of iterations,
and the pointer to the multivariable function. The function returns TRUE if it
finds a minimum. Otherwise, the function yields FALSE. The parameter x yields
the point for the sought minimum function value.

Listing 7.2 shows the source code for the OPTIM.C library file. The file contains
the implementation of the functions declared in the header file OPTIM.H. The im-
plementation is based on the algorithms that I presented earlier in this chapter. The
implementation file contains the C function getYTest and ExNewtonMin, which are
local to the library. The function getYTest calculates a new point for the function
Simplex. The function ExNewtonMin is called by function NewtonMultiMin to opti-
mize a single variable.

Listing 7.1 The source code for the OPTIM.H header file.
#ifndef _OPTIM_H_
#define _OPTIM_H_

#include "global.h"
#include "arrays.h"

#define OPTIM_BAD RESULT -1.0E+30

double BisectionMin(double Xa, double Xb,

double tolerance, double (*f) (double)
);

double NewtonMin(double X, double tolerance,

int MaxIter, double (*f) (double) );

double GoldenSearchMin(double Xa, double Xb,

double tolerance, int maxIter,
double (*f) (double) );

double QuadIntMin(double Xa, double Xb, double Xc,

double Xtol, double Ftol,
int maxIter,
double (*f) (double)
);
 Optimization 127

double CubicIntMin(double Xa, double Xb,

double Xtol, double Gtol,
int maxIter,
double (*f) (double));

void CalcYSimplex (Matrix x, Vector y, int numVars,

double (*fx) (double®*) );

void Simplex(Matrix x, Vector y, int numVars, double tolerance,

int maxIter, double (*fx) (double*));

int NewtonMultiMin(Vector x, int numVars, double tolerance,

int maxIter, double (*fx) (double*));
#endif

Listing 7.2 The source code for the OPTIM.C library file.
#include "global.h"
#include "arrays.h"
#include "optim.h"
#include <math.h>

double slope(double x, double (*f) (double) )

if
doubler bs =) (fabs) =a se OnOwe tac 7210.01:
casegmy (eae)ex Geta) = (Ose) hes Seley WA VP Mate
}

double BisectionMin(double Xa, double Xb,

double tolerance, double (*f) (double) )
{
double Xc;
double Fa, Fb, Fc;

Fa (rake (OC
Bbe=) (2 )n(Xb))y
do {
Kegae (Xai ce Xen oe
Ker=) (7) (Xe) i;
Heel ODe (KC mE) mS ODel xc) mec 0)! of
Xa = Xc;
Fe = Far

else {
oy a p.Moip
BiG =" Bilols
}
} while (fabs(Xb - Xa) > tolerance) ;
return Xc;
}
double NewtonMin(double X, double tolerance,
int maxIter, double (*f) (double) )
{
double h, diff;
double Fm, FO, Fp;
double firstDeriv, secondDeriv;
int iter = 0;

do {
/* caluclate increment */
heeenefabs(2)) = iy ee 0. On ex Ss) OR 01;
/* calculate function values at X-h, X, and Xth */
Pm. = 2084) (X#—sh) +;
FO (*£) (X);
128 Chapter Seven

Listing 7.2 (Continued)

19) Se Wey i@:6 A Ia)))9

/* calculate the first derivative */
fiestperay = (Eps — Fm) / 25)/ he
/* calculate the second derivative */
secondDeriv = (Fp - 2 * FO + Fm) / SQR(h);
/* calculate the guess refinement */
aiff = firstDeriv / secondDeriv;
/* refine the guess */
X= dit fe
iter++; /* the increment iteration counter */
} while (fabs(diff) > tolerance &&
iter < maxIter);

return (iter <= maxIter) ? X : OPTIM_BAD RESULT;

double GoldenSearchMin(double Xa, double Xb,

double tolerance, int maxIter,
double (*f) (double) )

double Xc, Xd, Fc, Fd;

double oneMinusTau = 1 - (sqrt(5) - 1) / 2;
int Lter = 0;

Xc = Xa + oneMinusTau * (Xb - Xa);

BG =e (AE) (Xe);
Xd = Xb - oneMinusTau * (Xb - Xa);
Fd = (*£) (Xd);
do {
iter++;
ieee hes< Ba)
Xb = Xd;
Xd.= xXe-
Xc = Xa + oneMinusTau * (Xb - Xa);
Fd = Fc;
Fos=) ((*£)\(Xe));
}
else {
Xa = Xc;
Xo = Xd
Xd = Xb - oneMinusTau * (Xb - Xa);
Fo = Fd:
Fd = (*f) (Xd);
}
} while (fabs(Xb - Xa) > tolerance && iter < maxIter) ;
return (iter <= maxIter) ? (Ka + Xb) / 2 : OPTIM_BAD RESULT;
}

double QuadIntMin(double Xa, double Xb, double Xc,

double Xtol, double Ftol,
int maxIter,
double (*f) (double) )

double Fa, Fb, Fe, Lastic, x, Fx;

int iter = (0);
int ok = FALSE;

Fa = a Bont
Fb = (*£) (Xb)
Fe = (*£) (Xe);

do {
 Optimization 129

iter++;
lastFc = Fc;
xo Oh Omer
(
(SQR(Xb) - SQR(Xc)) * Fa +
(SOR(Xc) - SQR(Xa)) * Fb +
(SQOR(Xa) - SOR(Xb)) * Fe
i) ff
(
(Xb - Xc) * Fa +
(ee Xa) sb bre
(Xia) soe:
VG
Fx = (*£) (x);
aia (Gs <s Ooh Ute) Bose KS Une) Af
Sie Ce
XC, =, Xi?
igo) => Waterd
BiG = hs
}
elceaut (co XCUSe Pao >) Be)
Ry Fes
Pisa=. Bsc:

elseat (sc = Xe && Poo Beye,

Ka=
Fa. = Fx;
}
else {
Xa = Xc;
Coins Ks
Fa = Fc;
ee SS abiF
}
if (fabs(Xa - Xc) < Xtol ||
fabs (Xc - Xb) < Xtol)
ok = TRUE;

} while (!(ok ||
fabs((lastFc - Fc)/lastFc) < Ftol &&
iter > maxIter));
return (iter <= maxIter) ? x : OPTIM_BAD_RESULT;
}

double CubicIntMin(double Xa, double Xb,

double Xtol, double Gtol,
int maxIter,
double (*f£) (double) )

double Fa, Fb, Fx;

double Ga, Gb, Gx;
double w, x, v, h, Gmin;
int ter = 0)

Fa I (*£) (Xa);
Fb U (Ff) (Xb) 7
/* calculate slope at Xa */
Ga = slope(Xa, f£);
/* calculate slope at Xb */
Gb = slopel(Xb, -£));
do {
iter++;
w= 3 / (Xb = Xa) * (Pa = Fb) + Ga + Gb;
v = sqrt(SQR(w) - Ga * Gb);
130 Chapter Seven

Listing 7.2 (Continued)

x = Xa + (Xb = Xa) *
(1 - (Gb +v-w) / (Gb - Ga + 2 * v));
Beage= (7) (Ge)i
/* calculate slope at x */
Gx = slope(x, £);
ne (Ga <0) Ge (Gx >= 0) | Px = Fa) {
Xo =) Sts
Ebes Fescr
Gb = Gx;
}
else {
Xa = X;
Ra = Ex
Ga = Gx;
}
Gmin = (fabs(Ga) > fabs(Gb)) ? fabs(Gb) : fabs(Ga);
} while (!(fabs(Xb - Xa) < Xtol ||
Gmin < Gtol ||
iter > maxIter));
return (iter <= maxIter) ? x : OPTIM_BAD RESULT;
}

void CalcYySimplex (Matrix x, Vector y, int numVars,

double (*fx) (double*) )
{
int numPoints = numVars + 1;
bihaWets sere lon
Vector Xarr;

newVect (&Xarr, numVars) ;

/* calculate the values for y[] using sumX[] */

fom (2,= 05 & < numPornts i++) ft
for (qj -= 07 9 = numVarst) 5)
VEC (arr, yi)) = MATS ak, walliy
VEGI, 4) = (iss) arr pbata);
}

deleteVect (&Xarr) ;

void Simplex(Matrix x, Vector y, int numVars, double tolerance,

double reflectFact, double expandFact,
double contractFact, int maxIter,
double (*f£x) (double*) )

const double EPS = 1.0e-8;

const double defReflectFact = 1.0;
const double defExpandFact = 2. 0 i
const double defContractFact = O55;

int numIter = 0;
int numPoints = numVars + 1;
int J, 2, Worstl, basti;
Boolean goOn = TRUE;
Boolean flag;
double’ vi, v2) x0);
double yMean, sum;
Vector X1, X2, Centroid;

newVect (&X1, numVars) ;

newVect (&X2, numVars) ;
newVect (&Centroid, numVars) ;
 Optimization 131

/* check Simplex modification factors */

if (reflectFact < EPS)
reflectFact = defReflectFact;
if (expandFact < EPS)
expandFact = defExpandFact;
if (contractFact < EPS)
contractFact = defContractFact;

/* calculate the values for y[] using X1 */

for (1 = 0; 1 < numPoints; i++)—{
for (7) = 03; 3 <-numVars; j++)
WADCNCGL, ah) WAUR(Ger alpe sili
WEE (vip) = (4:E5c) (Xl Data)
}

while ((++numIter < maxIter) && goOn) {

/* find worst and best point */
worstI = 0;
bestL = 0;
for (1 = 137i < numPoints; i++)
iE (VEC (Gye) eee VEC (Vie best)
besti = a;
else if (VEC(y, i) > VEC(y, worstT) )
worst = i;

/* calculate centroid (exclude worst point) */

for (i = 0; i < numVars; i++) {
VEC (Gentroid) ) 110
for “(j= 0; 7) < numPoints; 3++)
lide (Cj WOrss teal))
VEC (Centroid, i) += MAT(x, j, i);
VEC (Centroid, i) /= numVars;
}
/* calculate reflected point */
for (i = 0; i < numVars; i++)
VEC (Xi) 2) = "(Gees refilectract) * VEC (Centrord,, 1)
reflectFact * MAT(x, worstI, i);
Vile £x(XiepData)i

LE a(yi = VEC(ypebestr))) {
/* calculate expanded point */
form (a =sO0e — numVars +. a+)
VEC (2a) =)
(“twexpandkact let VEC (ky," 4) =
expandFact * VEC(Centroid, i);
y2 = £x(X2.pData) ;
Pee (yee VEG (We bestia) )met
/* replace worst point by X2 */
for (i = 0; i < numVars; i++)
MAT (sc cworstd, a)e = VECKX2, a)i
}
else {
/* replace worst point by X1 */
for (1 = 0; a < numVars; i++)
MWAENGSs {hfepaisnelk, al) = [email protected]) a9)'r
}
}
else {
flag = TRUE;
for (1 = 07 1 < numPoints; i++)
if (1 t= worstL && yl <= VEC(y, 2)) £
flag = FALSE;
break;
}
DERE Lage
132 Chapter Seven

Listing 7.2 (Continwed)

LE (vi VEG worst) imc

/* replace worst point by X1 */
for (1 = 0; i < numVars; i++)
MUNIN Ge5 Wonetshediy bees wane. aL))5
VEC (y, worstI) = yl;
}
/* calculate contracted point */
for (4 = Of a < numVars; i++)
VEC (X2, i) = contractFact * MAT(x, worsti, i) +
(2 Contrackhach)) a VEGiGentroscraeni.
y2 = £x(X2.pData);
LE (v2) VEC; worst)!
fx Store bestx a xl +7
for (1 = 0; a2 < numVars; i++)
WEKCIO.LS SE VNR (Oe, NeYeveteak at))i5
for j= Of F< mumPounts- sage) ft
for (i = 0; i < numVars; i++)
NUE er ayes SOE OP Swe Wale hee? 4. SIN) es
MAT (x; di,= L))e
}
}
else {
/* replace worst point by XK2 */
for (i = 0; i < numVars; i++)
MAT (x, WORSE, 25) = VEGC.(2)seab)ie
}
}
else {
/* replace worst point by X1 */
for (2 = 0; 2 < numVares a6)
MAT (Gc) WORS ti, ee) en VELOX eee ye
}
}
/* calculate the values for y[] using X1 */
for (4. = O07 a <= numPoints: a++){
for (j = 0; j < numVars; j++)
WAICI@:GLS sp) Smiaun(Ge, aie syle
VEC(y, 2) = (fx)
(i spbata)- }
/* calculate mean y */
yMean = 0.;
for (i = 0; i < numPoints; i++)
yMean += VEC(y, i);
yMean /= numPoints;
/* calculate deviation fron mean y */
sum = 0.;
for (i = 0; i < numPoints; i++)
sum += SOQR(VEC(y, i) - yMean);
/* test convergence */
goOn = (sqrt(sum / numPoints) > tolerance) ? TRUE : FALSE;

/* find the best point */

bestI = 0;
for (i = 1; i < numPoints; i++)
LE (VEC(y, 1) = VEC(y, bestr))
bestI = i;
if, {bestr T= 0), {
/* move best point to index 0 */
for (i = 0; i < numVars; i++) {
x0 = MAT(x, 0, i);
MAT Gc, ©, 2) = MAB (sx) “bestr> 2)
MAT (x, bestl, 2)) = x0;
 Optimization 133

Ae SS Wie Or
VEC(y, 0) = VEC(y, besti) >
WAHCIOW, leeeheir)) = wvaille
}

deleteVect (&X1) ;
deleteVect (&X2) ;
deleteVect (&Centroid) ;

int ExNewtonMin(Vector X, double tolerance,

double* diff, int maxIter, int index,
double (*f£) (double*) )

double h;
double Fm, FO, Fp;
double firstDeriv, secondDeriv;
wate shecre =) (0
double xOld = VEC(X, index);
double x = x0Old;

do {
/* calculate increment */
ig) = (Graloey(Rey ss ah) 7 Oo Wal st se 3 We Onls
/* calculate function values at x-h, X, and xth */
VEC(X, andex) = x = h;
Fm = (*f£) (X.pData) ;
VEG, andex)/ <= sc 4 ih
Epes (LEX. pbData),
VEC (X, index) = x;
BOs (At (cx pDatay)
/* calculate the first derivative */
Evnst Dery = (Sperm) vie ae whe
/* calculate the second derivative */
secondDeriv = (Fp - 2 * FO + Fm) / SQR(h);
/* calculate the guess refinement */
*diff = firstDeriv / secondDeriv;
/* refine the guess */
Se a ts htseran
VEC(S,, index) = sc:
iter++; /* the increment iteration counter */
} while (fabs(*diff) > tolerance &&
iter < maxIter);

if (iter <= maxIter)

return TRUE;
else {
VEC(X;, Index) = xOld;
return FALSE;

int NewtonMultiMin(Vector x, int numVars, double tolerance,

int maxIter, double (*f£x) (double®*) )
{
tp i, ks
double diff;
int ok;

do {
ok = FALSE;
fos (a = OF a < numVars; 24+)
if (ExNewtonMin(x, tolerance, &diff, maxiIter,
134 Chapter Seven

Listing 7.2 (Continwed)

iO eee) a RUE) at

if (diff > tolerance)

ok = TRUE;
} else
return FALSE;
} while (ok);
}

The Test Program

Let’s look at a test program that tests the C functions in the optimization library. List-
ing 7.3 contains the source code for the TSOPTIM.C test program. The program lo-
cates the minimum of the following functions:

To) Crete tl

f(x, y) = (x - 0.5)? + (y- 0.15)? + 1

and

f£,(x, y, Z) = (x- 0.5)? + (y-0.15)? + @-1)? +1

The mathematical functions f(x), f,(x, y), and f,(x, y, z) are implemented using
the C functions f, f2, and f8, respectively. The program performs the following tests:

Al It tests the function BisectionMin to get the minimum of function f(x). The pro-
gram supplies the interval [1, 4] and specifies a tolerance level of 10°.
. It tests the function NewtonMin to get the minimum of function f(x). The program
supplies the initial guess of 4, a maximum of 30 iterations, and specifies a toler-
ance level of 10>. ;
. It tests the function GoldenSearchMin to get the minimum of function f(x). The
program supplies the interval [1, 5], a maximum of 30 iterations, and specifies a
tolerance level of 10°.
. It tests the function QuadIntMin to get the minimum of function f(x). The pro-
gram supplies the guesses 1, 2, and 5, the X tolerance level of 10°, the function
tolerance level of 10~’, and a maximum of 30 iterations.
oO . It tests the function Simplex to get the minimum of function f,(x, y). The program
supplies the data matrix, displays the initial values of the data matrix and the
function values, solves for the minimum, and then displays the final values of the
data matrix and the function values.
. It tests the function NewtonMultiMin to get the minimum of the function f,(x, y,
z). The program supplies the array of initial guesses, the tolerance level, the max-
imum number of iterations, and the number of variables. The program displays the
initial guess, calls function NewtonMultiMin, and then displays the result.

Figure 7.4 shows the output of program TSOPTIM.EXE. You need to include the
files ARRAYS.C, OPTIM.C, and TSOPTIM.C in the project file.
 Optimization 135

Listing 7.3. The source code for the TSOPTIM.C test program.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "optim.h"

double fx(double x)
{
return SOR(x - 3) + 1;
}

double £x2(double* x)
{
double y0 ScnOn) a 0.55
double yl = x[1] — Ores

cAcieunain, SO) ta BAO cal caylee S aly

}

double £x3(double* x)
{
detbile y0 = x0] — 0.5;
doubiletyt*=px1l] = 0.15;
doubilety2 = (2) — 1-10;

metumn yO) * yO yie* “yin ye yo

void pressAnyKey
() ;

main ()
{
double Xa, Xb, Xc, X;
double Xtol, Ftol, Gtol;
double maxIter = 30;
Matrix mat;
Vector Y;
Vector Xarr;
int i, j, numVars, numPoints;

printf ("****Testing the Bisection method****\n") ;

Ran alte
>
Goji UA
tole = 17.'0e=5;
printf("Initial interval is [%g, %g]\n", Xa, Xb);
X = BisectionMin(Xa, Xb, Xtol, £x);
printf ("Optimum at %lg\n", X);

pressAnyKey
() ;

printf ("****Testing Newton's method****\n") ;

x = 5;
Ktol =) ise=8);
printf("Initial guess is %g\n", X);
X = NewtonMin(X, Xtol, maxIter, £x);
printf("Optimum at %lg\n", X);

pressAnyKey
() ;

printf ("****Testing the Golden Search method****\n");

Xal= 1;
Xb nee 5 5
136 Chapter Seven

Listing 7.3 (Continued)

Stole = 1 5e-8';
printe ("Initial iamterval vs [tq, soii\\n", Ka, Xodje
X = GoldenSearchMin(Xa, Xb, Xtol, maxIter, fx);
printf("Optimum at %lg\n", X);

pressAnyKey
() ;

printf ("****Testing the Quadratic Interpolation method****\n") ;

Thal eS A
5 oad ae
iG =) Si:
xXtol = 1.e-8;
Peo le; 1.0e—7r
printf("Initial guesses are %g, %g, and %lg\n", Xa, Xb, Xc);
xX = QuadinteMin (Ka, Xb; Xc, Xtol, Froll) maxtter, fx);
Diminie (Optamum save olic Nines) >

pressAnyKey
() ;

printf ("****Testing the Cubic Interpolation method****\n") ;

Nal ey le
KD a= 5
Xtol = 1.e-8;
Gtol = 1.0e-2;
printf("Initial guesses are %g and %lg\n", Xa, Xb);
x = CubileintMin(xXa, Xi, Xtol, Gtol, maxtter, f=);
printf("Optimum at %lg\n", X);

pressAnyKey
() ;

numVars = 2;
numPoints = numVars + 1;
newMat (&mat, numPoints, numVars) ;
newVect (&Y, numPoints);
MaAmM(maty, (0,7 \O)i =r 5)
MAR nat, 0), 1s) eae Olr

MAD (mate, 1 10) ee Se

MA (nicltee) 02 ee se Ok

MAT mat; 27) O)i=" 3.10)"

MAT (niat, 2, 4) =] 05

Prints (("**** see ESE) Testing Simpliex Methods ssw ts = AN e,

printf("Initial guesses for coordinates:\n");
CalcYySimplex(mat, Y, numVars, £x2);
for (3 = 0% 3 <= numVars> 34h) 4
pramcL("Y
[td] = SLE Naa VEG a)ee
for (i = 0; i < numVars; i++)
Printt ('xttd, sd) = SLEW ap Lp MAD (meateyamay) eles
}

Simplex(mat, Y¥, numVars, 1: 0e=7, 0., O02, Of, LOOO) iex2)-

printf("Final coordinates: \n");
for (j = OF <= numVats?; aes) 4
joa ohob Ciba leToll ee aeSiew atime WaKeKare ab O Yr
for. (i = 0» &.< numVars; 14+)
prinkil("xX[sad,
3a] = SslF\n") |, 2 NAR Giae, a, eee
}

pressAnyKey
() ;

newVect (&Xarr, 3);

 Optimization 137

NMC ae pe VON vest Iie

WEG (Xan 2) = 0);
Vine Kaw 2) =" Ole
Xtol = 1.0e-5;
maxiter = 20;
numVars = 3;
Prater (WArNte takes Testing Newton's Extended Method: **** =* +42 4am).
printf("Initial guesses for coordinates: \n");
for (a-= OF 1 < numVars; i++)
PEIMCE ("xX [td] = Slig\n", atl, VEGCXarnr, =)))-;
if (NewtonMultiMin(Xarr, numVars, Xtol,
Maxcaleer, £°5)) 0
printf ("Minimum at:\n");
for (i = 0; i < numVars; i++)
Drante (xT sa) =F Slg\iw sy ai, VECXarse, 4)iF
}
else
printf ("Newton's method failed!\n\n");

pressAnyKey
() ;

deleteMat (&mat) ;
deleteVect (&Y);
deleteVect (&Xarr) ;
return 0;

void pressAnyKey()
{
printf("Press any key to continue...");
getchar();
Dstt
h (uN);

****xTesting the Bisection method****

Initial interval is [1, 4]
Optimum at 3
Press any key to continue...

*x*x*xTesting Newton's method****

Initial guess is 5
Optimum at 3
Press any key to continue...

***k*Testing the Golden Search method****

Datel aniervyal as) ik, 35]
Optimum at 3
Press any key to continue...

****Testing the Quadratic Interpolation method****

Initial guesses are 1, 2, and 5
Optimum at 3
Press any key to continue...

****Testing the Cubic Interpolation method****

Initial guesses are 1 and 5
Optimum at 3
Press any key to continue...

Figure 7.4 The output of the TSOPTIM.EXE test program.

138 Chapter Seven

KkeKK Kk KKK KKK Testing Simplex Method Kak KK KKK KKK

Initial guesses for coordinates:

Y[0] = 5.422500
X[0,0] 1.500000
PEH0) Sik 2.000000
Malslie=— 9622500)
Pell 0) Tl
Ww
i -1.500000
xt la) -2.000000
Mio = 87 7.972500
X[2,0] 3.000000
2 aes 1.000000
Final coordinates:
ALON a= OOOO
nou
|low
X[0,0] 0.500334
xoOr al OREO 255
allel ele OOOO
X[1,0] iesi 0.499769

Pxaletareales) 0.150559
Y[2] = 1.000000
X[(2,0] ist
0.499876
M27 = O49 534
Press any key to continue...

ARKKEREASZAX Testing Newton's Extended Method ********

xx
Initial guesses for coordinates:
refill = 0)
X[2] = 0
X (3) -= 0
Minimum at:
Lee OS
XP (OS
X[3] = 1
Press any key to continue...

Figure 7.4 (Continued)

 Chapter

Basic Statistics

This chapter and the next three ones discuss popular statistical algorithms. This
chapter discusses C and C++ functions which perform basic statistics, including the
following:

=» The mean and standard deviation statistics

# The confidence intervals for the mean and standard deviation
= The first four moments
= Statistical testing for the mean

The Mean and Standard Deviation

The mean and standard deviation statistics represent the simplest and perhaps the
most common statistics about data. Here are the equations for calculating the mean
and standard deviation:

Tee (8.1)

Oe rim)
O= | Ge (8.2)

where n is the number of observations and x is the observed variable. Here is the al-
gorithm for obtaining the mean and standard deviation:
Given: the array Xarr, which contains N observations.
1 Set SumX = 0 and SumX2 = 0.
2 ForI =0to N—1 repeat the next steps:
2.1 Add Xarr[I] to SumX

139
140 Chapter Eight

2.2 Add Xarr[I] squared to SumX2.

3 Set Mean = _

(SumX2 — SumX * SumX\N)

4 Set Sdev =
New ;
5 Return Mean and Sdev as the mean and standard deviation values.

The Confidence Intervals

The mean and standard deviations calculated for a sample data are estimates of the
general population statistics. You can calculate the confidence interval for the range
of the mean value using the following equation:

Share
ete ee (8.3)
Ma
where mis the calculated mean for the sample, s is the calculated standard deviation
for the sample, t is the student-t probability distribution function for n—1 degrees of
freedom and 1 — @ probability, and nis the number of observations in the data sample.
Regarding the confidence interval of the standard deviation, here are the equa-
tions which specify the lower and upper limits:

—1) <2
Lower limit = eee (8.4)
Xn- 1; o/2

= fyse .
Upper limit = i (8.5)
Xn-1;1-o/2

where y is the Chi-Square probability distribution function.

The First Four Moments

Basic statistical analysis provides the first four moments along with the moment
coefficients of skewness and kurtosis to offer a bit more insight about the distrib-
ution of a data sample. The first moment is the mean value. The second moment
is the variance. The coefficient of skewness measures the lack of symmetry in a
distribution. The coefficient of kurtosis is the relative peakness or flatness of a
distribution. Here are the equations which calculate the statistics that I men-
tioned in this section:

$= aes (8.6)

m,=— Ix2-x? (8.7)

 Basic Statistics 141

Ms =pills yeaa aa
ae ak KX"
meen
+ 2x (8.8)

sOaye aE i SCORE Se
nN 1
(8.9)

Moment coefficient of skewness y, = ee (8.10)

m,'

Moment coefficient of kurtosis y, = —; (Sr)

m2

where &, is the moment coefficient of skewness and x, is the moment coefficient of
kurtosis.
Here is the algorithm for calculating the first four moments and their associated
coefficients:
Given: the array Xarr, which contains N observations.

1 Set SumX = 0, SumX2 = 0, SumX3 = 0, and SumX4 = 0.

2 ForI=0 to N—1 repeat the next steps:
2.1 Add Xarr{I] to SumX
2.2 Add Xarr[I] squared to SumX2
2.3 Add Xarr[I] cubed to SumX3
2.4 Add Xarr{I] raised to the fourth power to SumX4.

3 Set M1 = am,

4 Set M2 = SumX2/N — M12.

5 Set M3 = a -(2 * M1 « Sumx2) +2 «M12

6 Set M4 = Si -(4 * M1 + sumxs) +(2 « MI? + SumX2 -3 * MI)

7 Set Gammal = ere

8 Set Gamma2 = or

9 Return M1, M2, M3, M4, Gammal1, and Gammaz2 as the sought statistics.

Testing Sample Means

Statisticians. offer many tests which compare the estimated values of the means
and standard deviations of samples. This section looks at the test which compares
two means of two data samples. The test is based on calculating the student-t value
and comparing it with a tabulated value. If the calculated value exceeds the tabu-
lated value, then you do not accept the hypothesis that the means are equal. By
142 Chapter Eight

contrast, if the calculated value does not exceed the tabulated value, then you do
not reject the hypothesis that the means are equal. Here are the equations which
perform the test:

: (Cr t= d)

V CEeT)

a
ae 1
eae
1

ara (2x? —n, m, + Zy’—n, m,)

: (oer)

where m, and m, are the means for the x and y variables, d is the tested difference
in the means, n, and n, are the number of observations for variables x and y, re-
spectively.

The C Source Code

Let’s look at the C source code which implements the calculations for the basic sta-
tistics that I discussed earlier in this chapter. Listing 8.1 contains the source code for
the BASTAT.H header file. Listing 8.2 shows the source code for the BASTAT.C im-
plementation file.

Listing 8.1 The source code for the BASTAT.H header file.

#ifndef _BASTAT_H_
#define _BASTAT _H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/14/94

Module: C routines which support the following

basic statistics:

Mean and standard deviation

Confidence interval for the mean
Confidence interval for the std. deviation
First four moments
+++++
Testing the means of two samples
ull

#define BASTAT_EPS 1.0e-30

struct basicStatTag {
int hasMissingData;
int countData;
double sum;
double sumX;
double sumX2;
double sumX3;
 Basic Statistics 143

double sumx4;
double missingCode;
double mean;
double sdev;
TRE

typedef struct basicStatTag basicStat;

void initializeBasicStat (basicStat* basStat,

int HasMissingData,
double missingCode) ;

int getMeanSdev(double* x, int numData,

basicStat* basStat) ;

int meanCI (basicStat* B, double probability,

double* meanXLow, double* meanXHi) ;

int sdevCI (basicStat* B, double probability,

double* sdevXLow, double* sdevXHi) ;

int moments (basicStat* B,

double* meanX, double* varianceX,
double* moment3, double* moment4,
double* skewnessCoeff, double* kurtosisCoeff) ;

int meanTest (basicStat* Bl, basicStat* B2,

double testedDifference, double probability,
double* calcT, double* tableT, int* passTest) ;
#endif

Listing 8.1 declares the structure basicStatTag and its typedefed alias basicStat.
The structure contains fields which represent the statistical summations as well as
the mean and standard deviation. In addition, the structure has the fields hasMiss-
ingData and missingCode to support missing data. The C functions declared in the
file use the basicStat structure to manage the data and statistical results.
The header file declares the following C functions:

. The function initializeBasicStat initializes a basicStat structure accessed by the

pointer-parameter basStat. The function has two additional parameters which al-
low you to specify the status and value for missing data. You need to call function
InitializeBasicStat to initialize and reset the values in a basicStat structure.
. The function getMeanSdev calculates the mean and standard deviation for data
accessed by the array-parameter x. The parameter numData specifies how many
elements of array x should be included in the calculations. You can repeatedly call
function getMeanSdev to process a large number of observations in batches. This
approach empowers you to use relatively small arrays to gradually read and
process numerous observations. The function returns a Boolean value to indicate
its success or failure.
. The function meanClI calculates the confidence interval for the mean. The func-
tion has parameters which pass a pointer to basicStat structure, specify the con-
fidence probability, and pass the pointers to the low and high mean values.
. The function sdevClI calculates the confidence interval for the standard deviation.
The function has parameters which pass a pointer to basicStat structure, specify
144 Chapter Eight

the confidence probability, and pass the pointers to the low and high standard de-
viation values.
. The function moment calculates the four moments and their related coefficients.
This function has parameters which specify the pointer to the basicStat structure
(which provides the needed statistical summations) and the pointers to the mo-
ments and their coefficients. To use this function, you need to first call Initialize-
BasicStat and getMeanSdev (at least once).
. The function meanTest performs the test for the mean of two samples. The func-
tion uses the parameter B1 and B2 as the pointers to the two basicStat structures
which already contain the summations and basic statistical values. The function
has additional parameters that specify the tested mean difference, the test prob-
ability, the pointer to the calculated student-t value, the pointer to the tabulated
student-t value, and the pointer to the test outcome flag.

Listing 8.2 The source code for the BASTAT.C implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "bastat.h"
#include "statlib.h"

void initializeBasicStat (basicStat* B,

int HasMissingData,
double MissingCode)

B->hasMissingData = HasMissingData;
B->missingCode = MissingCode;
B->countData = 0;
B->sum = 0;
B->sumxX = 0
B->sumx2
B->sumx3
B->sumx4 |
ee
At
so
est)
Cer
B->mean
B->sdev [YeeittoO

int getMeanSdev(double* x, int numData, basicStat* B)

{
ane 2
double xx, x2;

if (numData > 1) {
B->countData += numData;
if (B->hasMissingData) {
for (i = 0; i < numData; i++)
exes (ese
if (xx > B->missingCode) {
xo = SORE s
B->sum++;
B->sumX += xx;
B->sumX2 += x2;
B->sumx3 += xx * x2;
B->sumxX4 += SOR(x2);

else {
for (i = 0; i < numData; it+) {
 Basic Statistics 145

BOR (SE ays)=

x2 SOR (xx) ;
B->sum++;
B=>sumxk F= 3x;
B->sumX2 += x2;
B=>Sumxs -=ssce. 502)-
B->sumX4 += SQR(x2);
}
}
if (B->sum > 1) {
B->mean = B->sumX / B->sum;
B->sdev = sqrt((B->sumxX2 = SOR(B->sumX) /
B->sum) / (B->sum - 1.0));
ij
else
return FALSE;
}
return FALSE;
i
int meanCI(basicStat* B, double probability,
double* meanXLow, double* meanXHi)
/* calculate the confidence interval of the mean value */
{
double tableT, df, delta, p;

if (B->sum < 2)
return FALSE;
Gi =—B=>sumy =) 1 A0r
sietsuen (ors @loailog lena; eel)
p = 0.50 = probability / 200.0;
else
p = 0.50 = probabilaty / 2.0;
table = Ting (py, GL);
delta = tableT * B->sdev / sqrt(B->sum);
*meanXHi = B->mean + delta;
*meanXLow = B->mean - delta;
return TRUE;

int sdevCI (basicStat* B, double probability,

double* sdevXLow, double* sdevXHi)
/* calculate the confidence interval of the
standard deviation value. */
{
double df, p;

if (B->sum < 2)
return FALSE;
aE = B->sum — 1.0;
ite) (probabidarty > 1210))
p = 0.50 - probability / 200.0;
else
p = 0250 = probability 7 220);
*sdevXLow = B->sdev * sqrt(df / ChilInv(p, df£));
*sdevxXHi = B->sdev * sqrt(df / ChilInv((1- p), qaé£));
return TRUE;

int moments (basicStat* B,

double* meanX, double* variancexX,
double* moment3, double* moment4,
double* skewnessCoeff, double* kurtosisCoeff)
/* Procedure to calculate the first four moments and
the coefficients of skewness and kurtosis. */
146 Chapter Eight

Listing 8.2 (Continued)

{
double meanSqrd;

if (B->sum < 2)
return FALSE;

*meanX = B->sumX / B->sum;

meanSqrd = SQR(*meanX) ;
*varianceX = B->sumX2 / B->sum - meanSqrd;
*moment3 = B->sumX3 / B->sum - 3.0 /
B->sum * *meanX * B->sumX2 +
2.0 * *meanX * meanSqrd;
*moment4 = B->sumxX4 / B->sum - 4.0 /
B->sum * *meanX * B->sumX3 +
.0 / B->sum * meanSqrd * B->sumX2 -
-O * SQR(meanSqrd) ;
*skewnessCoeff IWo *moment3 / pow(*variancexX, 1.5);
*kurtosisCoeff = *moment4 / SOR(*varianceX) ;
return TRUE;
}
int meanTest (basicStat* Bl, basicStat* B2,
double testedDifference, double probability,
double* calcT, double* tableT, int* passTest)
/* test means of two samples*/
{
double dit, p,, weactornl bacton2)) nil nz

if (Bl->sum < 2 || B2->sum < 2)

return FALSE;

nl = Bl->sum;
n2 = B2->sum;

if (probability > 1.0)

D = 0.50) — probability / 200 R0F
else
» = 0.50 — probability / 2.0;

af ="nil +
ne = 2.10%
Factory —
scree O) Joie tO) oy) aos
factor2 =
sqrt((Bl->sumX2 - SQR(B1l->sumxX)/nl +
B2->sumX2 - SQR(B2->sumX)/n2) / df);
*calcT = (Bl->sumX/nl - B2->sumX/n2 - testedDifference)
/ (factor1,* fLactorzZ) +:
*tablet =Tinvip, ae)i-
*passTest = (fabs(*calcT) <= *tableT) ? TRUE : FALSE;
return TRUE;

Listing 8.2 shows the definitions of the C functions declared in the header file BA-
STAT.H. The code for these functions implements the algorithms that I described
earlier. The implementation includes the header file STATLIB.H to use the function
TInv, which returns the inverse Student-t p.d.f. value. The functions in the imple-
mentation functions return values via pointer-parameters. The functions’ return val-
ues are TRUE or FALSE, which reflect their success or failure.
Keep in mind that you need to call the functions in a certain sequence—you need
to first initialize a basicStat structure, then accumulate data in that structure before
calling the functions that calculate the confidence interval, calculate the first mo-
ments, and test the sample means.
 Basic Statistics 147

The C++ Source Code

Let’s now focus on the C++ code which supports the basic statistics. Listing 8.3
shows the source code for the BASTAT.HPP header file. Listing 8.4 contains the
source code for the BASTAT.CPP implementation file.

Listing
8.3 The source code for the BASTAT.HPP header file.

#ifndef _BASTAT HPP_

#define _BASTAT
HPP _

fx

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/14/94

Module: C++ class which supports the following

basic statistics:

+ Mean and standard deviation

+ Confidence interval for the mean
+ Confidence interval for the std. deviation
+ First four moments
+ Testing the means of two samples
ay

#include "global.h"

class BasicStat
{
public:
BasicStat()
{ intevalize(HALSE, .0)es }
BasicStat (int HasMissingCode, double& MissingCode)
{ initialize(HasMissingCode, MissingCode); }

void initialize(int HasMissingCode, double MissingCode) ;

int getMeanSdev(double* x, int numData) ;
int meanCI (double probability,
double& meanXLow,
double& meanXHi) ;
int sdevCI (double probability,
double& sdevXLow,
double& sdevXHi) ;
int moments (double& meanX, double& varianceX,
double& moment3, double& moment4,
double& skewnessCoeff, double& kurtosisCoeff) ;
int meanTest (BasicStat& B,
double testedDifference, double probability,
double& calcT, double& tableT, int& passTest) ;
int getCountData()
{ return countData; }
double getSum()
{ return sum; }
double getMean()
{ return mean; }
double getSdev()
{ return sdev; }

protected:
int hasMissingData;
double missingCode;
int countData;
148 Chapter Eight

Listing 8.3 (Continwed)

double sum;
double sumX;
double sumxX2;
double sumX3;
double sumx4;
double mean;
double sdev;
Fe

#endif

Listing 8.3 declares the class BasicStat which encapsulates the data members and
member functions that support the basic statistics described in this chapter. The
class incorporates data members that are equivalent to the fields of the C structure
basicStat. In addition, the class encapsulates the member functions which initialize
the data, update the statistical summations, and perform various tests. The class Ba-
sicStat has the advantage over the C version in that it has constructors that are au-
tomatically invoked to initialize the class instances.
The member functions of class BasicStat resemble the C function declared in List-
ing 8.1. The main difference between the declarations of the C and C++ functions are

1. The C++ functions use reference parameters to pass results via arguments.
2. The C++ functions do not include a parameter for a structure comparable to the
basicStat structure, since all member functions have automatic access to the data
members of the class.

Listing 8.4 The source code for the BASTAT.CPP implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "bastat.hpp"
#include "sStatlib.h"

void BasicStat::initialize(int HasMissingData,

double MissingCode)
{
hasMissingData = HasMissingData;
missingCode = MissingCode;
countData = 0;
sum =. 0;
sumX
= 0;
sumxX2 = OF
sumx3 0;
sumx4 oul OF

mean = 0;
sdev = 0;
}

int BasicStat::getMeanSdev(double* x, int numData)

{
int as
double sec; x2);

if (numData > 1) {
countData += numData;
if (hasMissingData) {
 Basic Statistics 149

for (i = 0; i < numData; i++)

SxS Ni(e a)
if (xx > missingCode) {
2 = (SOR (sax);
sum++;
SUMS = exexes
sumxX2 += x2;
SUNXS = exon x
sumxX4 += SOQR(x2);
}
}
else {
for (i = 0; i < numData; i++) {
396 SS (Ge se Aly
x2 = SOR(xx);
sum++;
sumxX += 36%;
sumX2 += x2;
SumK3 += xx *) x2);
sumxX4 += SQR(x2);
}

ate Aion s Wy) ff

mean = sumX / sum;
sdev sqrt ((sumxX2 - SQR(sumX) / sum) / (sum - 1.0));
}
else
return FALSE;
}
return FALSE;

int BasicStat::meanCI (double probability,

double& meanXLow, double& meanXHi)
/* calculate the confidence interval of the mean value */
{
double tableT, df, delta, p;

eC SUM <2.)
return FALSE;
af = sume— ie0;
4£ (probability > 1.0)
p = 0.50 - probability / 200.0;
else
P= OL S00 = probability. 7/2510)
tableT = Tiny (p, de);
delta = tableT * sdev / sqrt(sum);
meanXHi = mean + delta;
meanXLow = mean - delta;
return TRUE;

int BasicStat::sdevCI (double probability,

double& sdevXLow, double& sdevXHi)
/* calculate the confidence interval of the
standard deviation value. */
{
double df, p;

af (sum < 2))

return FALSE;
af =" sum) = 2.0);
if \(probabalaty > 1.0)
p= 0250 = probability / 200.0);
150 Chapter Eight

Listing 8.4 (Continued)

else
p = 0.50 - probability / 2.0;
sdevXLow = sdev * sqrt(df / ChilInv(p, df));
sdevxXHi = sdev * sqrt(df / Chiinv((1 - p), dt£));
return TRUE;

int BasicStat:
:moments (
double& meanX, double& varianceX,
double& moment3, double& moment4,
double& skewnessCoeff, double& kurtosisCoeff)
/* Procedure to calculate the first four moments and
the coefficients of skewness and kurtosis. */
{
double meanSqrd;

2 (sum <02))
return FALSE;

meanX = sumX / sum;

meanSqrd = SQR(meanxX);
varianceX = sumX2 / sum - meanSqrd;
moment3 = sumX3 / sum - 3.0 /
sum * meanX * sumX2 +
2.0 * meanX * meanSqrd;
moment4 = sumX4 / sum - 4.0 /
sum * meanX * sumxX3 +
6.0 / sum * meanSqrd * sumX2 -
3.0 * SQR(meanSqrd) ;
skewnessCoeff = moment3 / pow(varianceX, 1.5);
kurtosisCoeff = moment4 / SOR(varianceX) ;
return TRUE;
}

int BasicStat: :meanTest (BasicStat& B,

double testedDifference, double probability,
double& calcT, double& tableT, int& passTest)
/* test means of two samples*/
{
Gouble di; ‘p, factorl, factor2;, nk; nZ-
if (sum < 2 || B.sum < 2)
return FALSE;
nl = sum;
n2 = B.sum;

L& (probability > 1-0)

Dp = 0.50 = probability / 20020>
else
p=. .0 50 — probabalarys 7 920

GE =jnt + n2)= 2.0;

factor] = sqre(l0 7 nl 42.0 Y n2)e
factor2 = sqrt((sumX2 - SQR(sumX)/n1l +
B.sumX2 - SQR(B.sumxX)/n2) / df);
calcT = (sumX/nl - B.sumX/n2 - testedDifference)
/ (€actori * factor2);
tableT = TiInv(p, df);
passTest = (fabs(calcT) <= tableT) ? TRUE : FALSE;
return TRUE;
 Basic Statistics 151

Listing 8.4 contains the definitions of the member functions of class BasicStat. The
code for these functions is somewhat similar to the counterpart C functions. Notice
that the C++ member functions access the data members directly and without the
use of a pointer to a structure. Also notice that the member functions return results
via reference parameters which do not use the access operator *.

The C Test Program

Let’s look at the C test program. Listing 8.5 shows the source code for the TSBA-
STAT.C test program. The program performs the following tasks:

ie Declares arrays of different sizes, as well as variables. Among these variables are
the basicStat-type structures B, B1, B2, and B3.
2. Initializes the structure B by calling the function initializeBasicStat.
3. Stores random numbers (in the range of 0 to 99) in the array x.
. Calculates and displays the mean and standard deviation values for the array x.
This task involves calling the function getMeanSdev with the arguments x,
SMALL_ARR, and the address of structure B. This task displays the sought sta-
tistics by accessing the fields mean and sdev of the structure B.
. Repeats steps 3 and 4 twice to add more data to the fields of structure B.

. Calculates and displays the confidence interval for the mean value. This task in-
volves calling the function meanCl.
. Calculates and displays the confidence interval for the !SD value. This task in-
volves calling the function sdevCl.
. Calculates and displays the first four moments and their related coefficients.
This task involves calling the function moments.
. Stores an unequal number of random numbers in the arrays x1 and x2. Both ar-
rays contain random numbers in the range of 0 to 99.

10. Tests the means of arrays x1 and x2. This task involves initializing the struc-
tures B1 and B2, followed by adding data using function getMeanSdey, and
then finally calling function meanTest to test the means of the unpaired sam-
ples. The program displays the outcome of the statistical test of the two
means.
Lae Stores random numbers in the array x3. The array contains random numbers
in the range of 0 to 1999. The array x3 contains the same number of data as
array X2.

ee Tests the means of arrays x3 and x2. This task involves initializing the structure
B3, followed by adding data using function getMeanSdev, and then finally calling
function meanTest to test the means of the paired samples. The program dis-
plays the outcome of the statistical test of the two means.
152 Chapter Eight

Listing 8.5 The source code for the TSBASTAT.C test program.

#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include "bastat.h"
#include "global.h"

void pressAnyKey (void) ;

#define SMALL ARR 10

#define BIG_ARR1 50
#define BIG_ARR2 70

main ()
if
seYateneede
double x[SMALL_ARR] ;
double x1[BIG_ARR1];
double x2[BIG_ARR2] ;
double x3[BIG_ARR2];
basieStat B, Bi 82), BS
double meanLow, meanHi;
double sdevLow, sdevHi;
double meanX, varX, moment3, moment4;
double skewnessCoeff, kurtosisCoeff;
double probab = 95.0;
double calcT, tableT;
int passTest;

/* initialize basic stat structure */

initializeBasicStat(&B, FALSE, 0);
/* initialize array */
randomize ();
for (i = 0; i < SMALL_ARR; i++)
x[{i] = random(100);
/* get mean and sdev */
getMeanSdev(x, SMALL ARR, &B);
print£ ("********* Mean and Std. Deviation *********x**\y") >
printf("Mean = %lg\n", B.mean) ;
printf ("Std. deviation = %lg\n", B.sdev);

for (i = 0; i < SMALL_ARR; i++)

x[{i] = random(100));
/* get mean and sdev */
getMeanSdev(x, SMALL_ARR, &B);
PLINnGk(e sess ase Moan and Std. Devaataom, = eer eset x tN)
printf("Mean = %lg\n", B.mean);
printf("Std. deviation = %lg\n", B.sdev);

for (i = 0; i < SMALL _ARR; i++)

x[i] = random(100) ;
/* get mean and sdev */
getMeanSdev(x, SMALL ARR, &B);
printf ("********* Mean and Std. Deviation ***********\n") >
printf("Mean = %lg\n", B.mean) ;
printf("Std. deviation = %lg\n\n", B.sdev);

pressAnyKey
() ;

meanCI(&B, probab, &meanLow, &meanHi) ;

printf("At lg %%, the confidence interval for ",
printf("the mean is\n");
printf("%lg to %lg\n\n", meanLow, meanHi) ;
 Basic Statistics 153

sdevCI(&B, probab, &sdevLow, &sdevHi);

printf("At %lg %%, the confidence interval for ", probab);
printf("the std. deviation is\n");
printf("%lg to %lg\n\n", sdevLow, sdevHi);

moments(&B, &meanX, &varxX, &moment3, &moment4,

&skewnessCoeff, &kurtosisCoeff) ;
PLint£ (1****x*xe* First Four Moments ********x*x*\n") ;
printf("Mean = %lg\n", meanX) ;
printf ("Variance = %lg\n", varX);
printf("Moment 3 = %$lg\n", moment3) ;
printf("Moment 4 = %lg\n", moment4);
printf("Skewness coeff. %$lg\n", skewnessCoeff) ;
printf("Kurtosis coeff. i] %lg\n", kurtosisCoeff£) ;

pressAnyKey () ;
/* assign data to array x1 */
randomize();
for (1 = 0; i < BIG_ARR1; i++)
x1[i] = random(100);

/* assign data to array x2 */

randomize();
for (i = 0; i < BIG_ARR2; i++)
x2[i] = random(100);

printf ("**** Testing mean values of unpaired data ****\n");

/* initialize basic stat structures */
initializeBasicStat(&B1, FALSE, 0);
initializeBasicStat(&B2, FALSE, 0);
/* calc mean and sdev */
getMeanSdev
(x1, BIG_ARR1, &B1);
getMeanSdev
(x2, BIG_ARR2, &B2);
meanTest(&B1l, &B2, 0.0, probab,
&calcT, &tableT, &passTest) ;
printf ("Calculated student-t = %lg\n", calcT);
printf ("Tabulated student-t = %$lg\n", tableT);
if (passTest)
printf("Cannot reject that means are equal\n\n");
else
printf("Cannot reject that means are different\n\n");

/* assign data to array x3 */

randomize();
for. (a =) 0% 1. < BIG ARR2) ++)
x3[i] = random(2000) ;

printf("**** Testing mean values of paired data ****\n");

/* initialize basic stat structure */
initializeBasicStat
(&B3, FALSE, 0);
/* calc mean and sdev */
getMeanSdev (x3, BIG_ARR2, &B3);
meanTest(&B2, &B3, 0.0, probab,
&calcT, &tableT, &passTest);
printf ("Calculated student-t = %lg\n", calcT);
printf ("Tabulated student-t = %lg\n", tableT) ;
if (passTest)
printf("Cannot reject that means are equal\n\n");
else
printf("Cannot reject that means are different\n\n") ;

pressAnyKey
() ;

return 0;
154 Chapter Eight

Listing 8.5 (Continued)

void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar() ;
puts ("\n\n");
}

To compile and run this program you need to include the files BASTAT.C, TSBA-
STAT.C, and STATLIB.C in your project file. Figure 8.1 shows a sample output of pro-
gram TSBASTAT. EXE.

The C++ Test Program

Listing 8.6 shows the source code for the TSBASTAT.CPP test program. The pro-
gram essentially performs the same tasks as its C counterpart. The main difference
is that the C++ test program creates the instances B, B1, B2, and B3 of class Basic-

se aeweene aecle ae 3C Mean and Seay Deviation KKK

KKK KEK

Mean = 41.1
Std. deviation = 37.4417
KKK KKK Mean and Std. Deviation MEKEKEK
ERE TE

Mean = 40.85
Std. deviation = 32.4058
KKK KKK Mean and Std. Deviation Re ee ee ae ee Ke ae ae

Mean = 44.9667
Std. deviation = 29.8357

Press any key to continue...

At 95 %, the confidence interval for the mean is

33.8232 to Soea10n
At 95 %, the confidence interval for the std. deviation is
23% 7595 co4 0g
KEKEKKREKKR First Four Moments KREKKKKKKEKEK

Mean = 44.9667
Variance = 860.499
Moment 3 = -337.983
Moment 4 = 1.19484e+06
Skewness coeff. = -0.0133897
Kunptosis CoekL. =p 52565
Press any key to continue...
**** Testing mean values of unpaired data ****
Calculated student-t = -0.0185904
Tabulated student-t = 1.98071
Cannot reject that means are equal
**** Testing mean values of paired data ****
Calculated student-t = -13.7776
Tabulated student-t = 1.97774
Cannot reject that means are different

Press any key to continue...

Figure 8.1 The output of program TSBASTAT.C.

 Basic Statistics 155

Stat, and sends various kinds of C++ messages to these instances. These messages
invoke the associated member functions to perform a required task.
To compile and run this program you need to include the files BASTAT.CPP, TS-
BASTAT.CPP, and STATLIB.CPP in your project file. The C++ test program shows an
output that is similar to that of the C version.

Listing 8.6 The source code for the TSBASTAT.CPP test program.

#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include "bastat.hpp"
#include "global.h"

void pressAnyKey (void) ;

#define SMALL ARR 10

#define BIG_ARR1 50
#define BIG_ARR2 70

main ()
{
clas abe
double x[SMALL_ARR];
double x1[BIG_ARR1];
double x2 [BIG_ARR2];
double x3 [BIG_ARR2];
BasieState 8) )5 1582 Bor
double meanLow, meanHi;
double sdevLow, sdevHi;
double meanX, varX, moment3, moment4;
double skewnessCoeff, kurtosisCoeff;
double probab = 95.0;
double calcT, tableT;
int passTest;

/* initialize basic stat structure */

B.initialize(FALSE, 0);
/* initialize array */
randomize() ;
for (i = 0; i < SMALL_ARR; i++)
x[i] = random(100);
/* get mean and sdev */
B.getMeanSdev (x, SMALL_ARR) ;
printf ("********* Mean and Std. Deviation ***********\n") ;
printf("Mean = %lg\n", B.getMean());
printf£("Std. deviation = %lg\n", B.getSdev());

for (i = 0; i < SMALL ARR; i++)

x[i] = random(100);
/* get mean and sdev */
B.getMeanSdev (x, SMALL_ARR) ;
print£("********* Mean and Std. Deviation ***********\n") ;
printf("Mean = %lg\n", B.getMean());
printf("Std. deviation = %lg\n", B.getSdev());

fom (2 =) 0-2 < SMALL ARR a+)

x[i] = random(100) ;
/* get mean and sdev */
B.getMeanSdev
(x, SMALL_ARR) ;
printf ("********* Mean and Std. Deviation ***********\n") ;
printf("Mean = %lg\n", B.getMean());
156 Chapter Eight

Listing 1.3 (Continued)

printf("Std. deviation = %lg\n\n", B.getSdev());

pressAnyKey() ;

B.meanCI (probab, meanLow, meanHi) ;

printf("At lg %%, the confidence interval for ", probab);
printf("the mean is\n");
printf£("$lg to %lg\n\n", meanLow, meanHi) ;

B.sdevCI (probab, sdevLow, sdevHi) ;

printf("At %lg %%, the confidence interval for ", probab) ;
printf("the std. deviation is\n");
print£("%lg to %lg\n\n", sdevLow, sdevHi) ;

B.moments (meanX, varX, moment3, moment4,

skewnessCoeff, kurtosisCoeff) ;
DrInta (is AatteA AT Bast Bout Moments, ***********\")\s
printf("Mean = %lg\n", meanX);
printf("Variance = %lg\n", varxX);
printf("Moment 3 = %lg\n", moment3) ;
printf£("Moment 4 = %lg\n", moment4) ;
printf("Skewness coeff. = %lg\n", skewnessCoeff) ;
printf ("Kurtosis coeff. = lg\n", kurtosisCoeff) ;

pressAnyKey () ;
/* assign data to array x1 */
randomize() ;
for (i = 0; i < BIG_ARR1; i++)
x1[i] = random(100);

/* assign data to array x2 */

randomize ();
Lor) ((ue=s0r a =< BICLARR2 - ++)
x2[i] = random(100);

printf("**** Testing mean values of unpaired data ****\n");

/* initialize basic stat structures */
Bl.initialize(FALSE, 0);
B2.initialize(FALSE, 0);
/* calc mean and sdev */
B1l.getMeanSdev (x1, BIG_ARR1);
B2.getMeanSdev (x2, BIG_ARR2);
Bl.meanTest(B2, 0.0, probab,
calcT, tableT, passTest);
printf ("Calculated student-t = %lg\n", calcT);
printf ("Tabulated student-t = %lg\n", tableT);
if (passTest)
printf("Cannot reject that means are equal\n\n");
else
printf("Cannot reject that means are different\n\n");

/* assign data to array x3 */

randomize() ;
for (i = 0; i < BIG_ARR2; i++)
x3[i] = random(2000) ;

printf("**** Testing mean values of paired data ****\n");

/* initialize basic stat structure */
B3.initialize(FALSE, 0);
/* calc mean and sdev */
B3.getMeanSdev (x3, BIG_ARR2) ;
B3.meanTest(B2, 0.0, probab,
calcT, tableT, passTest);
 Basic Statistics 157

printf ("Calculated student-t = $lg\n", calcT);

printf ("Tabulated student-t = %lg\n", tableT);
if (passTest)
printf("Cannot reject that means are equal\n\n");
else
printf("Cannot reject that means are different\n\n");
pressAnyKey () ;
return 0;
}

void pressAnyKey
()
{
printf("\nPress any key to continue...");
getchar();
PUES (S\N)
}
Mh eofeilere wink

: 79G Ge @herviuiay ’ ay 4 i@ .
‘ * LAB, Taj
— ae
‘| 7a
;
ae ‘SO¢ing
“ao - _

am |

or
(

”
-_
——
 Chapter

The ANOVA Tests

Statistics offer various kinds of analysis of variance (ANOVA) tests which empower
you to determine whether or not changes in data are statistically significant. In this
chapter you will learn about the methods, C source code, and C++ source code for
the following ANOVA tests:

=» The one-way ANOVA

=» The two-way ANOVA with no replications
=s The two-way ANOVA with replications
The Latin square ANOVA
The analysis of covariance ANOCOV

The One-Way ANOVA

The one-way ANOVA tests whether the observed differences among a set of sample
means can be attributed to chance or whether these variations indicate the actual
differences among the corresponding population means. The null hypothesis for the
one-way ANOVA is that the population means are all equal. The one-way ANOVA in-
volves calculating several items which are:
The mean value for each of the N samples:

2X,
m, = eee Nand) 5 te o (9.1)
i

The standard deviation for each of the N samples:

159
160 Chapter Nine

as
Ng ooeeereeneee amente (9.2)
: nt)
The sum of observations for each of the N samples:
Sum, = Xx, (9.3)

The sum of squares:

xax,? - (22x,,)”
TSS = (9.4)
=n,

The treatment sum of squares:

>[(2x,,)” / nj] = (22x,)”

TrSS = a (9.5)
1

The error sum of squares:

ESS = TSS — TrSS (9.6)
The treatment degrees of freedom:
df, =N-1 (9.7)
The error degrees of freedom:
df, = Zn,—N (9.8)

The total degrees of freedom:

df, = df, + df, (9.9)

The treatment mean square:

TrSS
DMS = ——— (9.10)
df,

The error mean square:

ESS
EMS = —— (9.11)
df,
The F ratio:
MS
Ra (9.12)
EMS
The above statistics are typically arranged and displayed as follows:
 The ANOVA Tests 161

SS df MS F
Treatments TrSS df : TrMS F
Error ESS df,2 EMS
Total TSS df.3

Here is the algorithm for the one-way ANOVA:

Given: the matrix Xmat, which contains N columns and M rows.

1 For 1=0 to N—-1 set Sum[I] = 0, SumX{[I] = 0, and SumX2[I] = 0.

2 For 1=0 to M-—1 repeat the following:
2.1 For J = 0 to N-1 repeat the following:
2.1.1 Increment Sum[J]
2.1.2 Add Xmat{I, J] to SumX[J]
2.1.3 Add Xmat([I, J] squared to SumX2[J].
3 Set TS = 0, TSS = 0, and GrandTotal = 0.
4 For 1 =0 to N—1 repeat the following steps:
SumX[I]
4.1 Set Mean{I] =
Sum[{]]

42 Set Sdev[i] = (“sa )

_
Gumi)
(Sum{i] — 1)
4.3 Add SumX{I] to TS
4.4 Add SumX2[I] to TSS
4.5 Add Sum{I] to GrandTotal.
: TSS = TS42
Set TSS =
GrandTotal ©
6 Set TrSS = 0.
7 ForI =0to N—-1 add SumX{[I]*2/Sum[]] to TrSS.
roo ls72
preh a GrandTotal °
9 Set ESS
= TSS — TrSS.
10 Set dfl
=N-—-1.
11 Set df2
= GrandTotal — dfl — 1.
12 Set df3
= dfl + df2.
TrSS
13 Set TrMS = ——.
afi

ESS
14 SetEMS ==:
ap

TrMS
15;SetF= EMS ©
162 Chapter Nine

The Two-Way ANOVA

The two-way ANOVA with no replications analyzes the total variability of a set of
data into components which can be attributed to diverse sources of variation. The
two-way ANOVA examines the row effects and the column effects independently.
The two-way ANOVA requires calculating several statistics given by the following
equations:
The column and row sums:

RS, = 2x, fori=1 tor

CS, = 2X; forj=1-to¢

The sums of squares for the rows and columns:

ys (2x,)? (22x,,)?
RSS =
c rc

x (2x)? (2Ex,)?
CSS
iG LG

The total sum of squares:

=zx,? ~ (2Ex,,)?
TSS =
EG

The error sum of squares:

ESS = TSS — RSS — CSS

The degrees of freedom for the row effect, the column effect, and the error:

df, =
df, =C— 1)
df, = @- D(e-)

The F ratios for the row and column:

 The ANOVA Tests 163

The above statistics are typically arranged and displayed as follows:

SS df F
Row RSS df, EG
Column CSS df, Er
Error ESS df,
Total TSS

Here is the algorithm for the two-way ANOVA with no replication:

Given: the matrix Xmat with R rows and C columns.

1 ForIl=0to C—1 repeat the following steps:

1.1 Set ColSumX{I] = 0
1.2 Set ColSumX2[]] = 0.
2 For] =0to R-1 repeat the following steps:
2.1 Set RowSumX{I] = 0
2.2 Set RowSumX2|]] = 0.
3 Set GrandSumX = 0.
ww Set GrandSumX2 = 0.
5 ForI =0 to R—-1 repeat the next steps:
5.1 For J = 0 to C—1 repeat the next steps:
5.1.1 Add Xmat|I, J] to RowSumX[I]
5.1.2 Add Xmat[lI, J] to ColSumX{J]
5.1.3 Add Xmat[I, J] squared to RowSumX2[]I]
5.1.4 Add Xmat[I, J] squared to ColSumX2[J].
6 For 1 =0to R-1 repeat the next steps:
6.1 Add RowSumX{I] to GrandSumX
6.2 Add RowSumX2[I] to GrandSumX2.
7 SetRC=R#*C.
8 Set TSS = GrandSumX2 =a ;

9 Set RSS = 0.
10 For Il=0 to R—1 Add RowSumX{[I]*2/C to RSS.
GrandSumxX*2
11 Subtract ince We from RSS.

12 Set CSS = 0.
N

18 For]=0to C-1add — UE TOSS.

A

14 Subtract SS
umX*2 ‘som CSS.
15 Set ESS = TSS — RSS — CSS.
16 Set RowDf = R - 1.
17 Set ColDf = C-1.
18 Set ErrDf = RowDf * ColDf.
19 Set TotalDf = RC - 1.
164 Chapter Nine

ae
wDf
PAO) Seve la = 3g

=bi)
(cam)
ColbDf
Zl set FZ = 7 ESS_\

ErrDf

The Two-Way ANOVA with Replication

The two-way ANOVA with replication allows you to have multiple observations for
each row and column effect. The multiple samples allow this kind of ANOVA test to
study the interaction effect which may appear to the unique combination of the row
and column effects. The two-way ANOVA requires calculating several statistics given
by the following equations:
The cell summations matrix:

LS; = 2X for k = 1 to number of replicates m

The row and column sums:

RS.j= 22K, fOr Kees LTO =n bore

CS, = XIX, fonk=altom,
1 =1)tor

The grand sum and sum of squares:

(Cie XXEX,

TSS LEEK,”

The sums of squares for the rows, columns, interactions, and error:

RS? :
RSS = set fy G38
rm T cm

CS? 2
CSS = fe) Gales
cm rem

LS.2 :
ISS = s3( )--@ _nss_oss
m rem

LS?
ESS = SS — ZE| —m
The degrees of freedom for the row effect, the column effect, interaction, and
the error:
 The ANOVA Tests 165

df,=r-1
dif="e—1
Clay (Tord) (Geoui)
df, = rc@m
- 1)

The F ratios for the row, column, and interaction:

The above statistics are typically arranged and displayed as follows:

SS df F
Row RSS df, te
Column Cosa, die E
Interaction ISS df, Ibe
Error ESS aby
Total TSS

Here is the algorithm for the two-way ANOVA with replication:

Given: the data matrix Xmat with R rows, C columns, and M replicates.

1 ForI=0to R—-1, set RowSumX{[I] = 0.

2 ForIl=0 to C — 1, set ColSumX[I] = 0.
3.1 For J = 0 to C — 1 repeat the next step:
3.1.1 Set CellSum[I, J] = 0.
4 Set GrandSumX = 0.
5 Set GrandSumxX2 = 0.
6 For] =0 to R—1 repeat the next step:
i
6.1 Set rowIndex = M

6.2 For J = 0 to C —1 repeat the next step:

6.2.1 Add X[I,J] to CellSum[rowIndex, J] = 0
166 Chapter Nine

6.2.2 Add X{I,J] to RowSumX[rowIndex]

6.2.3 Add X{I,J] to ColSumX[J]
6.2.4 Add X{I,J] squared to GrandSumX2.
R
fi Set IK= SS.
M

8 Set NRow = K and NCol = C.

9 For I= 0 to C—1 Add ColSumX{I] to GrandSumX.
10 Set RC = NRow * NCol * M.
Set TSS = GrandSumxX2 Pr
= renee A
2 .
1a!

12 Set RSS = 0.

13 Foal Otomo iad RowsSumX{I]*2

A
to RSS.
(NCol * M)
um x NA

14 Subtract a from RSS.

15 Set CSS 10)

ColSumX{I]*2
16 For I =O;
= 0 to NCo iil add SaaS
Gikew= i) to CSS.

GrandSumxX*2
ile Subtract a. Ree from CSS.

18 Set Sum = 0.
19 For I = 0 to NRow repeat the following steps:
19.1 For J = 0 to NCol repeat the following steps:
19.1.1 Add CellSum{I, J]*2 to Sum.

Set ISS = Sum _RSS— Css _ Sean 2 .

NN

Sum
iw)bt
iw)SS Set ESS = GrandSumxX2 —

GrandSumxX2 — GrandSumX*2
Set TSS =
RC ,
Set RowDf = NRow — 1.
Set ColDf = NCol— 1.
Set InteractDf = RowDf * ColDf.
Set TotalDf = RC — 1.
Ww
&
Do bo Set
we)NID
bw
WH oP ErrorDf = NRow * NCol * (M—1).
RSS i cme
iw)co Set RowF = (
ESS RowDf /
CSS 2 ran |
boNe} Set ColF = (
ESS ColDf

30 Set InteractF = (ISS

( ErrorDf
ESS InteractDf /”
 The ANOVA Tests 167

The Latin-Square ANOVA

The Latin-Square ANOVA tests for the effects of three factors A, B, and C. The Latin-
Square ANOVA uses a data table with an equal number of rows and columns. Here is
an example of a 5 by 5 Latin Square experimental design:

B, B, B, B, B;
A Cy Cy C, C, C,
A, C, Cs C, C, C,
eee CMC CG,
ee Cer Come Ces 47. C.
bce
5 3
Ge4 Ge.5 9G,1 oC 9
a

You can call the effect A the row effect, the effect B the column effect, and the effect
C the treatment effect. The ANOVA table calculates the following statistics:
The sums of effects A, B, and C:

SA, = 2X, forf=1 tor

SB, = 2x, fori =a tor
SC, = 2X,

The grand sum and sum of square:

Ge 2x,
Tio rdx,?

The sum of squares for the three effects, the sum of squares for the error, and the
total sum of squares:

SA2 2
TSA = cam = i
if. iia

— 2{ g SB2

If it
9

SC? |
TSC == if
Di aERsatbo

ESS = TSS — TSA — TSB — TSC

Pp,

TSS = SS - =
The degrees of freedom for the effects and for the error:
Adf=r—1
Bdf=r— 1
168 Chapter Nine

Cdf=r-1
Errdf = (7 —1)(r-2)

The F ratios for the three effects:

mee (=2)TSA
~ ESS

_ (@-2)TSB
iS ESS

_ @-2)TSC
oe ESS
The above statistics are typically arranged and displayed as follows:

Effect SS df F
A TSA Adf FA
B TSB Bdf FB
C TSG Cdf FC
Error ESS ErrDf
Total TSS

Here is the algorithm for the Latin-Square ANOVA:

Given: the matrix Xmat with R rows and columns, the indexing matrix Map which
specifies how the factor C appears in the matrix Xmat.

1 Forl=0 to R—-1 repeat the following steps:

1.1 Set ColSumX{[I] = 0
1.2 Set RowSumX{I] = 0
1.3 Set TrtSumX{I] = 0.
2 Set GrandSumX = 0.
3 Set GrandSumxX2 = 0.
4 Set N=0.
5 For!I=0to R—1 repeat the following steps:
5.1 Increment N
5.2 For J = 0 to R- 1 repeat the following steps:
5.2.1 Add Xmat[lI, J] to ColSumX{J]
5.2.2 Add Xmat[lI, J] to RowSumX{]]
5.2.3 Set K = Map{I, J]
5.2.4 Add X to TrtSumX[K]
5.2.5 Add Xmat(I, J] squared to GrandSumX2
5.3 Add RowSumX{I]] to GrandSumX.
6 Set RSS = 0, CSS = 0, and TrtSS = 0.
7 Forl=0toR-1.
7.1 Add RowSumX([I] squared to RSS
7.2 Add ColSumX[I] squared to CSS
7.3 Add TrtSumX{I] squared to TrtSS.
 The ANOVA Tests 169

B cere GrandSumx
N

9 Set RSS = EEE ey

N

(ecenee 2 N
ae
TESTS:
1 Ser Iisa] =—G.

12 Set ESS = GrandSumX2 — RSS — CSS — TrtSS — G.

13 Set TSS = GrandSumX2 - G.
14 Set RowDf = N-—1.
15 Set ColDf =N-—-1.
16 Set TrtbDfi =N =1,
17 Set ErrorDf = (N —- 1) * (N—2).
18 Set TotalDf =N * N-1.

19 Set
et RowF
Row 2 ((N-2 )*
RSS
gg

PO Set ColF VON2 ee Css

EN COT ON Aa faa
21 Set TrtF = (N-2)+# TrtSS
. CE ae
The Analysis of Covariance
The analysis of covariance (ANOCOV) examines the effects of variable separately
from the effect of a second variable. The test assumes that the second variable can
be measured. The analysis of covariance assumes that there are k population, each
with n, number of observations. Thus, each population may have a varying number
of observations. The ANOCOV statistics calculate the following items:
The sums and sums of squares:

Sx, = 2X, OV 1e—alenone

Se =e
isos =
zn,

(x) (22x;,) 2
ASSx = |
y” 2S 1

WSSx = TSSx — ASSx

The degrees of freedom:
df, =k-1
df, = 2n,—-k
170 Chapter Nine

The mean squares and F statistic

ASSx
AMSx = at

Vie tee
df,

AMSx
B= MSE
You can generate the above items for the variable y by replacing Xj with Vij
The sum of products:

22x;; ry i (2x;;) (2y,,)

TSPi=
=n. 1

r
(2x,) 7 (ZEx,,) (22y,,)
ASP = ae oS SS
=n.

WSP = TSP — ASP

The residual sums of squares:

TSSy — TSP’2
TSSy = ee
ee TSSx
W. 7 ice WSSy — WSP’2
eta Eta NAB
as WSSx
ASSy* = TSSy* — WSSy*
The residual degrees of freedom:

df, =k-1
df, = 2n,-k-1

The residual mean squares and F statistics:

A
AMSy”% = ASSy”

WSSy%
WMS}*4
y =df,—

_ AMSy*
~ WMSy*
 The ANOVA Tests 171

Here is the algorithm for ANOCOV:

Given: the matrix Xmat with K sets of X and Y pairs of observations. Each set has
N{I] observations, and the maximum number of observations in any set in NData.

1 Set N =2 * K * NData.
2 For i = 0 to N — 1 set Sum[I], SumX[I], SumX2[I], SumY{[I], SumY2[I], and
SumXY[I] to 0.
3 Set GrandSum = 0.
4 Set ASSx = 0.
5 Set ASSy = 0.
6 Set ASP = 0.
7 For I = 0 to NData — 1 repeat the following:
7.1 For J = 0 to K—1 repeat the following:
(Aw seri = 2%) ll
7.1.2 Setjx =jY-1
7.1.3 If Xmat[I, J] is not missing perform the next steps:
7.1.3.1 Set x = Xmat[I, jX]
7.1.3.2 Set y = Xmat(I, jY]
7.1.3.3 Increment Sum[{J]
7.1.3.4 Add x to SumX{J]
7.1.3.5 Add x squared to SumxX2[J]
7.1.3.6 Add y to SumY{J]
7.1.3.7 Add y squared to SumY2[J]
7.1.3.8 Add x * y too SumXY[I].
ee) Set dSumX, dSumY, dSumX2, dSumY2, dSumXY = 0.
For J = 0 to K—1 repeat the next steps:

9.1 Adq SUMAUI"2 4, asx

Sum[J]
SumyY[J]*2
9.2 Add to ASSy
Sum[J]
SumX[J] * SumY[J]
9.3 Add to ASP
Sum{[J]
9.4 Add Sum[J] to GrandSum
9.5 Add SumX{J] to dSumxX
9.6 Add SumX2[J] to dSumX2
9.7 Add SumY[J] to dSumY
9.8 Add SumY2[J] to dSumY2
9.9 Add SumXY[J] to dSumXyY.
_ dSumX2 — dSumX*2
10
Pets GrandSum
dSumY2 — dSumY“*2
dd
Se SNe GrandSum
© A
12 SG EE Sa eG
GrandSum
172 Chapter Nine

ASSy — dSumY“*2
13 Set ASSy =
GrandSum
14 Set WSSx = TSSx — ASSx.
15 Set WSSy = TSSy — ASSy.
16 Set Dfl =K-—-1.
17 Set Dfg = Dfl.
18 Set Df2 = GrandSum — K.
i= Set Df4 = Df2 — 1.
Set TSP = dSumXY — dSumX * dSumy
20
GrandSum
dSumX * dSumY
21 Set ASP =
GrandSum
22 Set WSP = TSP — ASP.
ASSx
23 Set AMSx = Df

24 Set AMSy = ASSy

Dfl —
Wssx
25 Set WMSx =
Df2 ~
WSSy
26 Set WMSy =
bee
AMSx
Set Fx =
WMSx
AMSy
Set
Fy =
WMSy —
TSSy — TSP*2
Set TSSyhat =
TSSxX
WSSy — WSP42
Set WSSyhat =
WSSx
Set ASSyhat = TSSyhat — WSSyhat.
. : ASSyhat
Set AMSyhat = DB

WSSyhat
Set WMSyhat =
Df4
_ AMSyhat
34 Set F
~ WMSyhat °

The C Source Code

Let’s look at the C source code which implements the various kinds of ANOVA statistics
that I present in this chapter. Listing 9.1 shows the source code for the ANOVA.H
header file. Listing 9.2 shows the source code for the ANOVA.C implementation file.
 The ANOVA Tests 173

Listing 9.1 The source code for the ANOVA.H header file.

#ifndef _ANOVA_H_
#define _ANOVA_H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/14/94

ANOVA statistics:

+ one-way ANOVA
+ two-way ANOVA without replication
+ two-way ANOVA with replication
+ Latin-square ANOVA
+ Analysis of covariance ANOCOV

eed

#include "arrays.h"

#define ANOVA_EPS 1.0e-30

struct ANOVAItag {
int NVar;
double* sum; //[ANOVA1_MAX VAR] ;
double* sumX; //[ANOVA1_MAX VAR];
double* sumX2; //[ANOVA1_MAX_VAR] ;
double Treatment_SS;
double Treatment_df;
double Treatment_MS;
double Error_SS;
double Error_df;
double Error_MS;
double Total_SS;
double Total_df;
double ANOVA1_F;
int hasMissingData;
double missingCode;
}7

typedef struct ANOVAltag ANOVAI1rec;

struct ANOVA2tag {
int Num_Row;
int Num_Col;
double* ColSumxX; // [ANOVA2_MAX_VAR] ;
double* ColSumxX2; // [ANOVA2_MAX_VAR] ;
double* RowSumx; //[ANOVA2_MAX DATA];
double* RowSumx2; // [ANOVA2_MAX DATA];
double GrandSumx;
double GrandSumxX2;
double Row_SS;
double Row_df;
double Row_F;
double Col_SS;
double Col_df;
double Col_F;
double Error_SS;
double Error_df;
double Total_SS;
double Total_df;
ie
typedef struct ANOVA2tag ANOVA2rec;
174 Chapter Nine

Listing 9.1 (Continued)

struct ANOVA2Rtag {
int NRe plicate;
int Num_Row;
ant. Num Coil ;
double* ColSumx; // [ANOVA2R_MAX
VAR] ;
/* “doubl e CellSum[ANOVA2R_MAX
DATA] [ANOVA2_MAX VAR]; */
double* RowSumxX; //[ANOVA2_MAX_VAR] ;
double GrandSumx;
double GrandSumX2 ;
double Row_SS;
double Row_df;
double Row_F;
double Coicss-
double Colmdé;
double Come
double Interact_SS;
double Interact_df;
double Interact_F;
double laparatone ISiSi5
double Error ar:
double Total ss
double Total_df;
hi

typedef s truct ANOVA2Rtag ANOVA2Rrec;

struct LatinSqrtag {
int N;
double* ColSumxX; //[LTNSQR_MAX DATA];
double* RowSumxX; // [LTNSQR_MAX DATA];
double* TrtSumx; //[LTNSQR_MAX DATA];
double Num;
double GrandSumx ;
double GrandSumX2 ;
double Row_SS;
double Row_df;
double Row_F;
double Coilass;
double Coilsdé-
double Gols
double Hiia meeSiSys
double Hy sd a ar
double iDrsitas Bts
double EEror_ Ss?
double Brom. car:
double Total ss;
double Total_df;
3

typedef s truct LatinSqrtag LatinSqr;

struct ANOCOVtag {
int Num_Set;
int has MissingData;
double* sum; //[ANOCOV_MAX VAR];
double* sumx; //[ANOCOV_MAX VAR];
double* sumY; // [ANOCOV_MAX_VAR];
double* sumX2; //[ANOCOV_MAX VAR];
double* sumY2; //[ANOCOV_MAX VAR];
double* sumxXY; // [ANOCOV_MAX VAR];
double missingCode;
double GrandSum;
double Dini.
double Df2;
 The ANOVA Tests 175

double Df3;
double Df4;
double ASSx;
double ASP;
double ASSy;
double ASSyhat;
double AMSyhat;
double WSSx;
double WSP;
double WSSy;
double WSSyhat;
double WMSyhat;
double TSSx;
double TSP;
double TSSy;
double TSSyhat;
double ANOCOV_F;
double AMSx;
double WMSx;
double AMSy;
double WMSy;
double Fx;
double Fy;
};

typedef struct ANOCOVtag ANOCOVrec;

void ANOVA1(ANOVAl1rec* r,
Matrix DataMat,
int NData, int NVar,
int HasMissingData,
double MissingCode) ;

void ANOVA2 (ANOVA2rec* r,

Matrix DataMat,
int NData, int NVar);

void ANOVA2R(ANOVA2Rrec* r,
Matrix DataMat,
int NData, int NVar, int NumReplicates) ;

void Latin(LatinSqr* r,
Matrix DataMat,
int* Map, int NData);

void ANOCOV(ANOCOVrec* r,
Matrix DataMat,
int NData, int NumSets,
int HasMissingData,
double MissingCode) ;

#endif

Listing 9.1 declares a set of structures which support the different ANOVA statis-
tics. The listing also declares shorthand typedefs for these structures. The latter
shorthand identifiers are ANOVAlrec, ANOVA2rec, ANOVA2Rrec, LatinSqr, and
ANOCOVrec. These identifiers support the one-way ANOVA, two-way ANOVA, two-
way ANOVA with replication, Latin-Square ANOVA, and ANOCOV, respectively.
The header file declares the following C functions:

1. The function ANOVAI supports the one-way ANOVA. The function has parameters
which specify the pointer to an ANOVAtIrec structure, a Matrix structure which
176 Chapter Nine

accesses the dynamic data, the number of observations, the number of variables,
the missing-code flag, and the missing-code value.
2. The function ANOVA2 supports the two-way ANOVA. The function has parame-
ters which specify the pointer to an ANOVA2rec structure, a Matrix structure
which accesses the dynamic data, the number of observations, and the number of
variables.
3. The function ANOVA2R supports the two-way ANOVA with replication. The func-
tion has parameters which specify the pointer to an ANOVA2Rrec structure, a
Matrix structure which accesses the dynamic data, the number of observations,
the number of variables, and the number of replicates.
4. The function Latin which supports the Latin-Square ANOVA. The function has
parameters which specify the pointer to a LatinSqr structure, a Matrix structure
which accesses the dynamic data, the number of observations, the array which
maps the indices, and the number of replicates.
5. The function ANOCOV supports the analysis of covariance. The function has pa-
rameters which specify the pointer to an ANOCOVrec structure, a Matrix struc-
ture which accesses the dynamic data, the number of observations, the number of
sets, the missing-code flag, and the missing-code value.

Listing 9.2 The source code for the ANOVA.C implementation file.

#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "anova.h"
#include "statlib.h"

void ANOVA1 (ANOVAirec* r,

Matrix DataMat,
int NData, int NVar,
int HasMissingData,
double MissingCode)

= 7 Eo? Sag(7
double x;
double TS, TSS, GrandTotal;
double* Mean = (double*)malloc(NVar * sizeof (double) );
double* Sdev = (double*)malloc(NVar * sizeof (double) );

/* create and initialize statistical summations */

r->sum = (double*)malloc(NVar * sizeof (double) ) ;
r->sumX = (double*)malloc(NVar * sizeof(double));
r->sumX2 = (double*)malloc(NVar * sizeof(double));
Bor (i = 0; 4 < NVax; 3e¥) 4
r->sum[j] = 0;
r->sumX[j] = 0;
o->simnx.
[3] = 05
}
r->hasMissingData = HasMissingData;
r->missingCode = MissingCode;

/* update statistical summations */

r->NVar = NVar;
for Uz = 0; i < NData; i++) {
for (j = 0; j < NVar; j++) {
 The ANOVA Tests 177

x = MAT(DataMat, i, j);
if (!(r->hasMissingData && x <= r->missingCode)) {
r->sum[j]++;
r->sumX[j] += x;
r->sumxX2[j] += SOR(x);
}
}
}

/* perform one way ANOVA calculations */

Ansys “Ole
meas SOs
GrandTotal = 0;
fora (= Ora er NVaiy at)
Mean[i] = r->sumX[i] / r->sum[i];
Sdev[i] = sqrt((r->sumX2[i] - SOR(r->sumX[i]) /
r->sum(i]) / (r->sum[i] - 1));
TS += r->sumX[i];
TSS += r->sumX2 [i];
GrandTotal += r->sum[i];
}
r->Total_SS = TSS - SOR(TS) / GrandTotal;
6 > Tneatinen
ta Sici— 0k
for (1 = 0% a < xs=SNVary i++)
r->Treatment_SS += SQR(r->sumX[i]) / r->sum[i];
r->Treatment_SS -= SOR(TS) / GrandTotal;
r->Error_SS = r->Total_SS — r->Treatment_SS;
r->Treatment_df = r->NVar —- 1;
r->Error_df = GrandTotal - r->Treatment_df - 1;
r->Total_df = r->Error_df + r->Treatment_df;
r->Treatment_MS = r->Treatment_SS / r->Treatment_df;
r->Error_MS = r->Error_SS / r->Error_df;
r->ANOVA1_F = r->Treatment_MS / r->Error_MS;

/* deallocate dynamic arrays */

free(r->sum);
free (r->sumX) ;
free (r->sumxX2) ;
free (Mean) ;
free (Sdev) ;
}
void ANOVA2 (ANOVA2rec* r,
Matrix DataMat,
int NData, int NVar)

rib@¥ios ais 2
int nCol, nRow;
double RowCol;
double x, sx;

r->RowSumX = (double*)malloc(NData * sizeof(double) );

r->RowSumX2 = (double*)malloc(NData * sizeof (double) ) ;
r->ColSumX = (double*)malloc(NVar * sizeof (double) );
r->ColSumxX2 = (double*)malloc(NVar * sizeof(double) );

/* initialize statistical summations */

fore = Of jlo NVar; 3+4) 1
r->ColSumx[j] = 0;
r->ColSumx2[j] = 0;
}

for {j= 07 3 < NData;, 34+) £

r->RowSumX[j] = 0;
r->RowSumxX2[j] = 0;
178 Chapter Nine

Listing 9.2 (Continued)

r->Num_Row = NData;
r->Num_Col = NVar;

/* update statistical summations */

r->GrandSumx = 0;
r->GrandSumx2 = 0;
r->Num_Col = NVar;
r->Num_Row = NData;
Por ay = Ul a Se INDeteels) tt)
Poe. (She We 5) — ierey aan) 4
x = MAT(DataMat, i, Jj);
sx = SORE) >
r->RowSumX[i] += x;
¥=->ColSumxX[g] += %;
r->RowSumX2[i] += sx;
E-SsColsumxag) += Sss5
}

Hor Wee Or tea Natta salt) met

r->GrandSumX += r->RowSumX[i];
r->GrandSumx2 += r->RowSumX2 [i];
}
/* perform two-way ANOVA calculations */
nCol = r->Num_Col;
nRow = r->Num_Row;
RowCol = r->Num_Row * r->Num_Col;
r->Total_SS = r->GrandSumx2 - SQR(r->GrandSumxX) / RowCol;
r->Row_SS = 0;
for (1 = 0; ai < nRow; i++)
r->Row_SS += SQR(r->RowSumX[i]) / nCol;
r->Row_SS -= SQR(r->GrandSumx) / RowCol;
r=SCel_ss =. 0G
for (i = On = mol a5)
r->Col_SS += SOR(r->ColSumxX[i]) / nRow;
r->Col_SS -= SQR(xr->GrandSumxX) / RowCol;
¢->Error_SS = r->Total_SS — r->Row_SS — r->Col_Ss;
r->Row_df = r->Num_Row - 1;
r->Col_df = r=->Num_Col — 1;
yoShrromc ae = w-SRow_Gl. * ©—=Col det,
r->Total_df = RowCol.— 1;
r->Row_F = (r->Row_SS / r->Row_df) /
(x->Error_SS / r->Error_df);
r=SCol F = (r=sGollss / x-SGei-de)/
(r=>Error_SS / r->Error_dE£);

/* delete dynamic arrays */

free (r->RowSumx) ;
free (r->RowSumX2) ;
free (r->ColSumx) ;
free (r->ColSumxX2) ;
}
void ANOVA2R(ANOVA2Rrec* r,
Matrix DataMat,
int NData, int NVar, int NumReplicates)

double x, sx;
int nCol, nRow;
double RowCol, sum;
Site ci,. “ty, kk, ROE
Matrix CellSum;

/* initialize statistical summations */

r->ColSumxX = (double*)malloc(NVar * sizeof(double));
r->RowSumX = (double*)malloc(NData * sizeof(double));
 The ANOVA Tests 179

newMat (&CellSum, NData, NVar);

fon (jj 0= Of 7 < NVare ae)
r->ColSumx[j] = 0;
Lor (je =O. ay <a NDaitcace ach)
r->RowSumX[j] = 0;
Ore (at =e Olen as <a ONVretis oise-fs)
for (j = 0; 3 < NData; j++)
MAT\GedTSum, i, 3) = 0;

r->Num_Row = NData;
r->Num_Col = NVar;
r->GrandSumxX = 0;
r->GrandSumx2 = 0;

/* update statistical summations */

r->NReplicate = NumReplicates;
for (a. = (0 -sas — NDatalys ace) kt
iRow = i / r->NReplicate;
fon Gg. = 0; a) NVaxy ai)
x = MAT (DataMat, 2, 3)+
Sxe =r iSOR (Gc)
MAT(CellSum, iRow, j) += x;
r->RowSumxX[iRow] += x;
r->ColSumX[j] += x;
r->GrandSumxX2 += sx;

3;
k = NData / r->NReplicate;
r->Num_Row = k;
r->Num_Col = NVar;
for (7°= 0% 3 -< NVare a++)
r->GrandSumX += r->ColSumX[j];

/* perform two-way ANOVA calculations */

neo =" L—>Num Col.
nRow = r->Num_Row;
RowCol = r->Num_Row * r->Num_Col * r->NReplicate;
r->Total_SS = r->GrandSumx2 - SOR(r->GrandSumx) / RowCol;
r->Row_SS = 0;
for (1 = 0; i < nRow; i++)
r->Row_SS +=
SQR(xr->RowSumxX[i]) /
(x->Num_Col * r->NReplicate) ;
r->Row_SS -= SOR(r->GrandSumxX) / RowCol;
r>Coliss = (0'-
Lome (te = Ol ee <— nCols a+)
r->Col_SS += SQR(xr->ColSumX[i]) /
(x->Num_Row * r->NReplicate) ;
r->Col_SS -= SOQR(r->GrandSumxX) / RowCol;
sum = 0;
hore(ea= O- a <i nRow; ++)
fone (gq =" 3 < mnCol; 34+)
sum += SOR(MAT(CellSum, i, j));
r->Interact_SS / r->NReplicate = sum
- r->Row_SS -
r->Col_SS - SOR(r->GrandSumxX) / RowCol;
r->Error_SS = r->GrandSumx2 - sum / r->NReplicate;
r->Total_SS = r->GrandSumx2 - SQR(r->GrandSumX) / RowCol;
r->Row_df = r->Num_Row - 1;
¥->Col df = r=SNum_Col = 1;
r->Interact_df£ = r->Row_df * r->Col_df;
r=>Total_df = RowCol = 1;
r->Error_df = r->Num_Row * r->Num_Col *
(r->NReplicate - 1);
r->Row_F = (r=->Row_SS / r->Error_SS) *
(r->Error_df / r->Row_df) ;
P= Colne —>Col.SS 7 r-SsErron ss) *
180 Chapter Nine

Listing 9.2 (Continued)

(r->Error_df / r->Col_df);
r->Interact_F = (r—Sintberact Sor %—->mrror sss) *
(x->Error_df / r->Interact_df) ;

/* remove dynamic arrays */

free (r->ColSumxX) ;
free (r->RowSumX) ;
deleteMat (&Cel1Sum) ;
}
VOU) Lattin
ba tinScne tat,
Matrix DataMat,
int* Map, int NData)
{
stale BB ale hee
double x, sx, gj;

/* initialize statistical summations 7)

r->ColSumxX = (double*)malloc(NData * sizeof (double) );
r->RowSumx = (double*)malloc(NData * sizeof (double) ) ;
r->TrtSumx = (double*)malloc(NData * sizeof (double) );

for (2 = O-e a) < NDatarerat+) {

r=->ColSumx[i] = 0;
r->RowSumX[i] = 0;
4, r->TrtSumx[i] = 0;
}
r->GrandSumx = 0;
r->GrandSumx2 = 0;
r->Num = 0;

/* update statistical summations */

r->N = NData;
/* Obtain sums of columns, rows and treatments */
fora (0 i < r->N; i++)
r->Num++ i

see (a) Oy Sa] SEEN Gh )) {

x = MAT(DataMat, i, j);
Sx — SOR (x);
r->ColSumX[j] += x;
r->RowSumX[i] += x;
k = *(Map + i + NData * ay)§
r->TrtSumxX[k] += x;
r->GrandSumxX2 += sx;
}
r->GrandSumX += r->RowSumX[i];
}
/* perform Latin Squares ANOVA */
r->Row_SS I 0;
He-COL_
SS I 0;
i Pal hl eS) 0;
Ora «= O)= aL < r->N; i++)
r->Row_SS += SQR (r->RowSumX
[i]
G=>Col ss += SOR (r->ColSumX
[i]
r=STLt SS 4= SOR (r->TrtSumx
[i]
}
g =SOR(r->GrandSumxX / r->Num);
r- >Row_SS = r->Row_SS / r=->Num —- g;
r=>Col_Ss = r->Col_ss / -SNum = gy;
rake
Mig Gl apc fo pee r->Trt_SS / r->Num — g;
r->Error_Ss r->GrandSumX2 = G=>ROW_SS =
r->Col_SS — r=sTrt_Ss — g;
r->Total_SS r->GrandSumxX2 Gi
r->Row_df = r->Num - 1;
 The ANOVA Tests 181

r->Col_df = r->Num - 1;
gedaan ichs = sapayibii ly9 Np
r->Error_df = (r->Num - 1) * (xr->Num - 2);
r->Total_df = SQOR(r->Num) - 1;
©->Row_F’ = (x->Num = 2) * r=SRow_SS / r->Error_Ss;
r->Col_F = (x->Num — 2) * r->Col_SS / r->Error_SS;
r=>Trt Fr = (r=->Num — 2) * r=>Trt_SS / r->Error_Ss;

/* deallocate dynamic arrays */

free(r->ColSumx) ;
free (r->RowSumx) ;
free (r->TrtSumxX) ;
}

void ANOCOV(ANOCOVrec* r,
Matrix DataMat,
int NData, int NumSets,
int HasMissingData,
double MissingCode)

cig Sly. Sih. gaP.@re ai ae

double x, y;
double dSumxX, dSumy, dSumxX2;
double dSumY2, dSumXyY;
int N = 2 * NumSets;

/* initialize statistical summations */

r->sum = (double*)malloc(N * sizeof(double) );
r->sumxX = (double*)malloc(N * sizeof(double) );
r->sumxX2 = (double*)malloc(N * sizeof(double) );
r->sumY = (double*)malloc(N * sizeof(double) );
r->sumY2 (double*)malloc(N * sizeof (double) ) ,
r->sumxY = (double*)malloc(N * sizeof (double) ),

LOR s(t =' Ol a5 <Nieeeaee ae) af

r->sum[i] = 0
r->sumxX[i] = 0;
r->sumxX2[i] = 0;
f—>sumy [2] =) 0);
r->sumY2[i] = 0;
ro>sumxy [a] = 0 ,

}
r->GrandSum = 0;
¥=>ASSx = 0;
©r->SASSy = 0;
r->ASP = 0;
r->hasMissingData = HasMissingData;
r->missingCode = MissingCode;

/* update statistical summations */

r->Num_Set = NumSets;
for (1 = 0; i < NData; i++)
for (aj) = 0 P< cao NumiSety eg t+ it
SINE 4 Meee Safa ela
IPS ape wee
x = MAT(DataMat, i, JX);
if (!(xr->hasMissingData && x <= r->missingCode) ) {
y = MAT(DataMat, i, JY);
r->sum[j]++;
r->sumxX[j] += x;
r->sumX2[j] += SQR(x);
r->sumY[j] += y;
r->sumY2[j] += SOR(y);
r->sumxY[j] += x * y;
182 Chapter Nine

Listing 9.2 (Continued)

/* carry out ANOCOV calculations */

daSumx = 0;
dSumy = 0;
dSumxX2 = 1
dSumy2 = /
dSumxy = (s)
> 0;
(2s)
igen (5) = Op a) seep alse)
r->ASSx += SOR(r->sumX[j]) / r->sum[j];
r->ASSy += SQOR(r->sumY[j]) / r->sum[j];
r->ASP += r->sumX[j] * r->sumY[j] / r->sum[j];
r->GrandSum += r->sum[j];
dSumX += r->sumX[jl];
daSumxX2 += r->sumX2[j];
dSumYy += r->sumY[j];
daSumy2 += r->sumy2[j];
dSumXY += r->sumXY[jl;
}
r->TSSx = dSumxX2 -— SQR(dSumxX) / r->GrandSum;
r->TSSy = dSumy2 - SQR(dSumy) / r->GrandSum;
r->ASSx -= SQR(dSumX) / r->GrandSum;
r->ASSy -= SQR(dSumY) / r->GrandSum;
r->WSSx = r->TSSx -— r->ASSx;
r->WSSy = r©r->TSSy -— r->ASSy;
r->Df1l = r->Num_Set - 1;
r->Df£3 = r->Df1;
r->Df2 = r->GrandSum —- r->Num_Set;
=> Die aie Deere
r->TSP = dSumxy - dSumX * dSumY / r->GrandSum;
r->ASP -= dSumx * dSumY / r->GrandSum;
r->WSP = r->TSP — r->ASP;
r->AMSx = r->ASSx / r->Dfl1;
r->AMSy = r->ASSy / r->Df1;
r->WMSx = r->WSSx / r->Df2;
r->WMSy = r->WSSy / r->Df2;
r->FxX = r->AMSx / r->WMSx;
r->Fy = r->AMSy / r->WMSy;
r-sTSSsyhat = r->TSsSy — SOR (r=STSP) 7 s=STsSsx;,
r->WSSyhat = r->WSSy - SQR(r->WSP) / r->WSSx;
r->ASSyhat = r->TSSyhat - r->WSSyhat;
r->AMSyhat ©->ASSyhat / xr->D£3;
r->WMSyhat r->WSSyhat / r->Df4;
r->ANOCOV_F = r->AMSyhat / r->WMSyhat;

/* delete dynamic arrays */

free (r->sum) ;
free(r->sumx) ;
free (r->sumX2) ;
free (r->sumyY) ;
free (r->sumyY2) ;
free (r->sumXY) ;

Listing 9.2 implements the C function declares in the ANOVA.H header file. The
implementation follows the algorithms that I presented earlier in this chapter. The
functions create and remove local dynamic arrays using the standard C functions
malloc and free. The functions also access the fields of the various ANOVA-related
structures using the -> operator.
Let’s look at the test program. Listing 9.3 shows the source code for the TSANOVA.C
test program. The program tests the various ANOVA-related functions using matri-
 The ANOVA Tests 183

ces of data. The program assigns different data for testing each C function. Figure 9.1
shows the output of program TSANOVA.EXE. To compile the C test program you
need to include the files ARRAYS.C, ANOVA.C and TSANOVA.C in your project file.

Listing 9.3 The source code for the TSANOVA.C test program.

#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include “anova.h"
#include "global.h"
#include "arrays.h"

void pressAnyKey (void) ;

#define SMALL ARR 10

#define BIG_ARR1 50
#define BIG _ARR2 70

main ()
{
Matrix mat;
int map[25];
ANOVAI1ree rl;
ANOVA2rec r2;
ANOVA2Rrec r3;
LatinSqr r4;
ANOCOVrec r5;

MAT(mat, 0, 0) = 88;
MAT(mat, 1, 0) = S))p
MAT (mat, 2, 0) = 96;
MAM(matk, es5 0). = 68;
MAT(mat, 4, 0) = 85;
MAT (meat, 5; 6) = 2 * MISSING_DATA;
MAT (mat, 6; O) = 2 * MISSING_DATA;

MAT(mat, 0, 1) = 78;
MAT(mat, 1, 1) = 62>
MAri(mate 2, Lie 98;
MAT (mat, 3; 2) = 83;
MAT (mat, 4, 2) = Gis
MAT (mat, 5, 2) = 88;
MAT(mat, 6, 1) = 2 * MISSING _DATA,;

MAT(mat, 0, 2) = 80;
MAT (mat, 1, 2) = 61;
Maninmat, 2, 2) = GA
MAR (mat, 3,2) = 92;
MAT (mat, 4, 2) = TES
MAT (mst. 5S, 2) = ori
MAGM(Matr 6, 2)) = Tie

MAT(mat, O, 3) = rales
MAT (nat; 1, 3) = 65;
MAG (mat, e2y. 3) = 90;
MAT (mat; 3, 37 = 46;
MAT (mat, 4, 3) = 2 * MISSING_DATA;
MAT(mat, 5, 3) = 2 * MISSING DATA;
MAT(mat, 6, 3) = 2 * MISSING DATA;

ANOVA1 (&r1, mat, OY, TRUE, MISSING_DATA) ;

POL es mie eS one-way ANOVA KKKKKKKKEKK\ YN") >

printf ("Treatment SS $lg\n", r1l.Treatment_SS) ;

printf("Treatment df $lg\n", r1.Treatment_dE£) ;
printf("Treatment MS = %lg\n", r1l.Treatment_MS) ;
184 Chapter Nine

Listing 9.3 (Continued)

Dreiobe (“Error SS = Sigqin", =rl. ernroress):>

Prager hrrom dl =—slqun", miaErrorldt) >
Drinti("Exror MS = slg\n", vil. Error
MS) -
Prance ("Total SS = slg\n"; rl -Totallss);
Doimtn( "Total di =<sslg\n", ci. Total ide)
DEINE ("hh = Sig\n" ol ANOVAL &)i-

pressAnyKey
() ;

MAT(mat, 0, 0) ie
MAT(mat, 1, 0) Dye
MAR (mat, 2, 0) ran4
MAT (mat, 0, 2) = Ge
MAT(mat, 1, 1) = 4;
MAT (mat, 2, L)h=.6%
MAT(mat, 0, 2) = 8;
MAT(mat, 1, 2) 4;
MAT (mat, 2) 2). = Sis
MAT(mat, 0, 3) oF
MAT(mat, 1, 3) 4;
MAT (Maite). ope) ee
Print ("KKK AKKKK KKK two-way ANOVA GSN Natta lhl yyJo

ANOVA2 (&r2, mat, 3, 4);

printf("Row SS = %lg\n", r2.Row_SS);
printf ("Row df = %1g\n", r2.Row_df) ;
printf£("Row F = %lg\n", r2.Row_F);
printL("Col SS = Sig\n", m2 COlESS i.
Printé ("Col dt = Sig\n", 2 Covmde;) =
DrELnte ("Col B= tig\n™, 12 Collar).
printi ("Error SS = Slgq\n", r2.Error_SS) ;
PHInte ("Error d= = Slig\n", E2-ETrror_ dF)
primes (otal Sse="slo\n"s r2.Total_SS) ;
printf("Total df = %1lg\n", r2asTotal dé);

pressAnyKey
() ;

MAT(mat, 0, 0) = 2;
MAR (mak, 1, 0) <= 125i
MAT(mat, 2, 0) = 1;
MAT(mat, 3, 0) = 1;
MAT(mat, 4, 0) = 1.5;
MAT (mate Si. (0i)) =e)

MAT(mat, 0, 1) = ite
WuweRtiekee ile El) eel aay
MAM (martes Oa =" ii
MAD (matce 357 1.) =) 1
MAT(mat, 4, 1) = 1;
MAT (mat, 5) «Li = 2 .5\

MAT(mat, 0, 2) = -0.5;
MA (Inet, lt) = (Ons
MAT(mat, 2, 2) = -1;
MAT(mat, 3, 2) = 0;
MAT (mat 4.2) =.
MAT (mat, 5,02) = 1;

MAT (mat. 0; Ss) = 15%

MAT(mat, 1, 3) = 1.5;
MAT (mat, 2, 3s = 1.
MAT (matin 3.32 =-0i
MAT(mat, 4, 3) = 0.5;
MAT (MR i, 25), ots
 The ANOVA Tests 185

printf ("*** two-way ANOVA with replication ***\n");

ANOVAZR (&E3, mat, 6, 4; 2)¢
printf("Row SS = %$lg\n", r3.Row_SS);
printf ("Row df = %$lg\n", r3.Row_df);
printf("Row F = %lg\n", r3.Row_F);
prune (MCoImSSe=— sign 3. Cole Ss)r
DEineEe(MColmd t=. tigi 61S Col edt)ir
pina (MColg Pe=) tLo\nt m3Col. EI.
printf("Interaction SS = %$lg\n", r3.Interact_SS) ;
Drinte("anteraccion dt = Sig\n™ —r3\cinteractdf))>
printf("Interaction F = %lg\n", r3.Interact_F);
DIME nCerrerorr SSe=) Slg\n, 63 \hGEors SS)!
roraiiakent (Waveraoue Yohe =) trllfo pall, heeyyinkaialonas
(ols))
PrIntL(eTotalLvss SIGN" 7 ES LOtal Ss)
printf("Total df Sil Nee Se Otel)

pressAnyKey
() ;

map[0] = 0;
mapiil]" =" 2;
map[2] = 1;
map[3] = 3;
map[4] = 1;
map(5] = 3;
map[6] = 0;
map [7 ]*=) 2;
map[8]°= 2;
map[9] = 0;
map[10] = 3;
Map hii) = 2;
mapll2)) =s3:
Maplisi =) Ly
Map hi4i = 2
map 5) =) 0)

MAT (mat, 0, 0) =" 26.46;

MAT (mati, (0; )0 =) 296i;
MAT (mat, (0%) 2) = 27 .o2s
MAT (mat, Opes) = 20 9 ber
MAS (a tele) eG
i NUE aly cill)) =, PA tears
MAT (mat, 12) e=— 20.2Ole
1AW NAMGarren SL 2S))\e se eile
MAT (Mac, 2) pao 29-54;
MAG(Mat 27) 2 7 OO
TWN (Sone e,9 92 8) sae OH eS lee
MAT (mat, 254) = 27.90
MAG (matin Syl) =) 2.9). Sis
MAT (Mat cian), = 28.03%
MAT (matey) 3), 2)) = 292535
MAR (mat, 8,13) = 26.54

PLAints ("ee AAAA RAH Latin Square ANOVA KKK KEK KK EKER DM)»

Latin(&r4, mat, map, 4);

printf("Row SS = %lg\n", r4.Row_SS) ;
printf ("Row df = lg\n", r4.Row_dE£);
printf("Row F = %lg\n", r4.Row_F);
pranee ("Col Ss Sillig\n",, 24..Col Ss);
DEINEL ("Col df Sign) ra Col_d&) 7
prance (Col Fo=.slig\n", ©r4.CcColLor);
printf("Treatment SS = %lg\n", r4.Trt_SS);
printf("Treatment df = %lg\n", r4.Trt_df);
printf("Treatment F = %lg\n", r4.Trt_F);
printf("Error SS = %lg\n", r4.Error_SS);
printf("Error df = tlg\n", r4.Error_df);
186 Chapter Nine

Listing 9.3 (Continued)

printf£("Total SS = %lg\n", n4) Total sss)

print£ ("Total df = ¢lg\n", r4.Total_df);

pressAnyKey
() ;

MAT(mat, 0, 0) = 3;
MAT(mat, 1, 0) = 2;
MAT (mat, 2, 0) "==2;
MAT(mat, 3, 0) = 2;

MAT(mat, 0, 1) = 10;
MAT (mat, 27 1) =s8)s
MAT (mat, 62 .2)esas
MAT (mat, 3), al) St;

MAT (mat, ©, 2) = 4;
MAI (mat, 2) ase
MAT (mat, 2, 2) = 3;
MAR (mat, Sy, 2)! = 57

MAT(mat, 0, 3) = 12;
MAT(mat, 1, 3) = 12;
WONT Gites PSI) Se allOye
MAT (mat, SiS) =. LS

MAT (mat, OF 64) =",

MAT (matty, ee
MAT (mat, (2p) no
MATT (maiege 65/4) ales

MAT (matey 0; 651) S60

MAT (nat ale 5) = Se
MAT (mata) heels
MAT (mete 3:2 5) = iy
pong ayAETE (MES eA Seka See recite sas ANOCOV REKKEKKEKE VY") *

ANOCOV(&xr5, mat, 4, 3, FALSE, 0);

printf("D£f among means = %lg\n", r5.Df1);
DINERDE ewithin groups = sig\n™, r5.DE2) ;
PreIner( TASS — sliq\n",. f5 ASS) =
printf("ASP = %lg\n", r5.ASP);
pine ( VASSVe= tlo\n" , osASSy) >
printf ("WSSx = %lg\n", r5.WSSx) ;
Ppranti("WSPe= Sig\n",, 5 .WSP) >
printf("WSSy = %lg\n", r5.WSSy) ;
popqgucnan (WUESISs a cao Filye Naki salsyadNeisp<))
Prsnce (TSP — Sigin" . Loe lsh) >
PEwniat (Soy one i hon ho Sy).
printf ("------- Residuals statistics ---------
printf("Df among means = %lg\n", r5.Df£3);
prince ("DE within. groups = Slq\n", 15. Di4y;
printf ("ASSy* = %lg\n", r5.ASSyhat) ;
printf ("AMSy* = %lg\n", r5.AMSyhat) ;
print£ ("WSSy* = tig\n", r5.WSSyhat);
printf ("WMSy* $lg\n", r5.WMSyhat) ;
prince ("PSsSy~ = tlg\n", «5. TSsvhat).
DeLee ime =) SC NL) eno pe
DLIINcE(*Hy =" slg\ne;, Bo. PVs
print£("F = tlg\n", r5.ANOCOV_F);

pressAnyKey
() ;

return 0;
}
void pressAnyKey()
{
 The ANOVA Tests 187

printf("\nPress any key to continue...");

getchar();
pues (EN) iy

KRREKKKKKKKKKKK one-way ANOVA KKEKKKKKKEKK

Treatment SS = 930.438
Treatment df = 3
Treatment MS = 310.146
Enror SS 315992516
Error df 18
Error MS 199). 97/6
Total SS = 4530
Total df Dik
Eisele 55092

Press any key to continue...

KKKKKKEKKKKKKK two-way ANOVA KaKKKKKKKKK

Row SS = 26
Row df 2
Row f= iis
Col = 3
Col
Col
EEror SS = 6.06667
Brror GE 6
Totadess 36
Tokaweate= al

Press any key to continue...

*** two-way ANOVA with replication ***

ROW. Soe =) 5202033
Row Gf = 92
Row F = 11.4762
Col SS = 4.61458
Colpdie——3
Coven == Osis
Interaction SS = 2).72917
Interaction df 6
Interaction F = 2.07937
BEron SSe= 2025
idngretone” (ola | i)?}
Total SS 14.9896
Mo tallic fer=10 3

Press any key to continue...

KKKKKKKKKEKEK Latin Square ANOVA KKKKKKKKKKEKKE

Row SS 0.14175
Row df = 3
Row, f= 5.58071
ColLssea Oso i15
Col dei ="3
Col be=— 3). 8243
Treatment SS = 21.4387
Treatment df = 3
Treatment F = 844.043

BP Eee ee = 00008 Figure9.1 The output of the

Error df 6
Poeeaticeaeaa 9604 TSANOVA.EXE program.
Total df LS
188 Chapter Nine

Press any key to continue...

KEKE KKK KKK EKSK ANOCOV KKK

Df among means = 2
Df within groups = 9
ASSx = 9.5
ASP = 20.75
BSSV= 255). 667
WSSx = 7.5
WSP = 6.25
WSSy = 16.5
TSSse = 7
TSP = 27
ToOSves 6667

Residuals statistics
Df among means = 2
Df within groups = 8
ASSy* = 17.4926
AMSy* 8.74632
WSSy> = Le 2 Oy,
WMSy* = 1.41146
LSS SS Mise Tisks!
Bee 0455
= 04555
F = 6.19666 Figure 9.1 (Continwed)
Press any key to continue...

The C++ Source Code

Let’s look at the C++ source code that implements the ANOVA statistics. Listing 9.4
contains the source code for the ANOVA.HPP header file. Listing 9.5 shows the
source code for the ANOVA.CPP implementation file.

Listing 9.4 The source code for the ANOVA.HPP header file.

#ifndef _ANOVA_HPP_
#define _ANOVA_HPP_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/14/94

C++ classes which supports the following

ANOVA statistics:

+ one-way ANOVA
+ two-way ANOVA without replication
+ two-way ANOVA with replication
+ Latin-square ANOVA
+ Analysis of covariance ANOCOV
et |

#include "global.h"
#include "arrays.hpp"

class ANOVAlclass
{
public:
ANOVA1 (Matrix& DataMat,
int NumVar, int NumData,
 The ANOVA Tests 189

int HasMissingData FALSE,

double MissingCode ONE.

double Treatment_SS;
double Treatment_df;
double Treatment_MS;
double Error_SS;
double Error_df;
double Error_MS;
double Total_SS;
double Total_df;
double ANOVA1_F;

protected:
int NVar;
double* sum;
double* sumX;
double* sumX2;
int hasMissingData;
double missingCode;
Mp

class ANOVA2class
{
public:
void ANOVA2 (Matrix& DataMat,
int NumRows, int NumCols) ;

double Row_SS;
double Row_df;
double Row_F;
double Col_SS;
double Col_df;
double Col_F;
double Error_SS;
double Error_df;
double Total_SS;
double Total_df;

protected:

int Num_Row;
int Num_Col;
double* ColSumx;
double* ColSumxX2;
double* RowSumXx;
double* RowSumX2;
double GrandSumX;
double GrandSumX2;
a
class ANOVA2Rclass
{
public:
void ANOVA2R(Matrix& DataMat,
int NumRows, int NumCols,
int NumReplicates) ;

double Row_SS;
double Row_df;
double Row_F;
double Col_SS;
double Col_df;
double Col_F;
double Interact_SS;
double Interact_df;
double Interact_F;
double Error_SS;
190 Chapter Nine

Listing 9.4 (Continued)

double Error_df;
double Total_SS;
double Total_df;

protected:

int NReplicate;
int Num_Row;
int Num_Col;
double* ColSumx; // [ANOVA2R_MAX
VAR];
double* RowSumx; // [ANOVA2_MAX VAR];
double GrandSumxX;
double GrandSumx2;
ay
class LatinSqrclass
{
public:
void Latin(Matrix& DataMat, int* Map, int NData);

double Row_SS;
double Row_df;
double Row_F;
double Col_SS;
double Col_df;
double Col_F;
double Trt_SS;
double Trt_df;
double Trt_F;
double Error_SS;
double Error_df;
double Total_Ss;
double Total_df;

protected:
int N;
double* ColSumx; //[LTNSQR_MAX DATA];
double* RowSumx; //[LTNSQR_MAX DATA];
double* TrtSumx; // [LTNSQR_MAX DATA];
double Num;
double GrandSumx;
double GrandSumx2;
195

class ANOCOVclass
{
public:
void ANOCOV(Matrix& DataMat, int NData, int NumSets,
int HasMissingData, double MissingCode) ;

double Df1;
double D£2;
double Df3;
double Df4;
double ASSx;
double ASP;
double ASSy;
double ASSyhat;
double AMSyhat;
double WSSx;
double WSP;
double WSSy;
double WSSyhat;
double WMSyhat;
double TSSx;
double TSP;
 The ANOVA Tests 191

double TSSy;
double TSSyhat;
double ANOCOV_F;
double AMSx;
double WMSx;
double AMSy;
double WMSy;
double Fx;
double Fy;

protected:
int Num_Set;
int hasMissingData;
double* sum; // [ANOCOV_MAX VAR];
double* sumxX; // [ANOCOV_MAX_VAR] ;
double* sumY; //[ANOCOV_MAX_VAR];
double* sumX2; // [ANOCOV_MAX_VAR] ;
double* sumY2; // [ANOCOV_MAX_VAR] ;
double* sumXY; // [ANOCOV_MAX VAR];
double missingCode;
double GrandSum;
he

#endif

Listing 9.4 declares the classes ANOVAI class, ANOVA2class, ANOVA2Rclass, Latin-
Sarclass, and ANOCOVclass to support the one-way ANOVA, two-way ANOVA, two-
way ANOVA with replication, Latin-Square ANOVA, and ANOCOV, respectively. Each
class declares a public member function, a set of public data members, and a group
of protected data members. The member function performs the ANOVA test, while
the public data members give you direct access to the results of the ANOVA.

Listing 9.5 The source code for the ANOVA.CPP implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "anova.hpp"

ANOVA1class:
:ANOVA1 (Matrix& DataMat,
int NumVar, int NumData,
int HasMissingData,
double MissingCode)

arate ie ge
double x;
double TS, TSS, GrandTotal;
double* Mean = new double[NVar];
double* Sdev = new double[NVar] ;
/* initialize statistical summations */

NVar = NumVar;
// allocate dynamic arrays
sum = new double[NVar] ;
sumX = new double[NVar] ;
sumX2 = new double[NVar];
for (j = 0; 3 =< Nvar; j+#) {
sum[j] = 0;
sumX[j] = 0;
sumxX2[j] = 0;
192 Chapter Nine

Listing 9.5 (Continued)

hasMissingData = HasMissingData;
missingCode = MissingCode;

/* update statistical summations */

for (i = 0; i < NumData; i++)
for (7 = 0; 3 =< NVar; 3+) {
x = DataMat(a,. 3a)
if (! (hasMissingData && x <= missingCode)) {
sum[j]++;
sumX[j] += x;
sumX2[j] += SQR(x);
}
}

/* perform one way ANOVA calculations */

TS =) 0s
TSS = 0;
GrandTotal OF
for (2°= 0.92 <—NVare a+) {
Mean[i] = sumX[i] / sum[i];
Sdev[i] = sqrt((sumX2[i] - SOR(sumX[i]) /
sum[i]) / (sum[i] - 1));
TS += sumX[il];
TSS += sumxX2[i];
GrandTotal += sum[i];
}
Total_SS = TSS - SQR(TS) / GrandTotal;
Treatment_SS = 0;
for? (2°=" Oy a) < NVare a+)
Treatment_SS += SQR(sumX[i]) / sum[i];
Treatment_SS -= SQR(TS) / GrandTotal;
Error_SS = Total_SS - Treatment_SS;
Treatment_df = NVar - 1;
Error_df = GrandTotal - Treatment_df - 1;
Total_df = Error_df + Treatment_df;
Treatment_MS = Treatment_SS / Treatment_df;
Error_MS = Error_SS / Error_df;
ANOVA1_F = Treatment_MS / Error_MS;
// remove dynamic arrays
delete [] Mean;
delete [] Sdev;
delete [] sum;
delete [] sumx;
delete [] sumx2;
}
void ANOVA2class:
:ANOVA2 (Matrix& DataMat,
int NumRows, int NumCols)
{
ant 2, as
double x, sx;
int nCol, nRow;
double RowCol;

// create dynamic arrays

ColSumX = new double[NumCols];
ColSumX2 = new double[NumCols];
RowSumX = new double[NumRows] ;
RowSumX2 = new double [NumRows] ;
// initialize summations
for (j = 0; j < NumCols; j++) {
ColSumx[j] = 0;
ColSumx2[j] = 0;
 The ANOVA Tests 193

for (j = 0; j < NumRows; jt+) {

RowSumX[j] = 0;
RowSumX2[j] = 0;
}
Num_Row = NumRows;
Num_Col = NumCols;

/* update statistical summations */

GrandSumX = 0;
GrandSumx2 = 0;
for (1 = 0; 1 < Num_Row; i++)
formaig, = Ona <eNUumMeColl > at) f
x = DataMat(i, j);
coc SOR (5x)i>
RowSumX[i] += x;
ColSumxX[j] += x;
RowSumX2[i] += sx;
ColSumX2[j] += sx;
}

for (i = 0; i < Num

_Row; i++) {
GrandSumX += RowSumX[i];
GrandSumxX2 += RowSumxX2[i];
}

/* perform two-way ANOVA calculations */

nCol = Num coll;
nRow = Num_Row;
RowCol = Num_Row * Num_Col;
Total_SS = GrandSumxX2 - SQR(GrandSumxX) / RowCol;
Row_SS = 0;
fom (ie=" 021 < nRows prt)
Row_SS += SQR(RowSumX[i]) / nCol;
Row_SS -= SQR(GrandSumxX) / RowCol;
Comass=— 0;
for (2 ="0F 2, <— nol a4++)
Col_SS += SQR(ColSumX[i]) / nRow;
Col_SS -= SQR(GrandSumX) / RowCol;
ErrorSs = Totaless — Row_Ss = Coil_Ss;
Row_df = Num_Row - 1;
Collect = Numicol— 1
Bn rOrmeCie = Rowen =) COW Gr.
Totaldf "= Rowcol = 1;
Row_F = (Row_SS / Row_df) /
(EErcoressi/ Bion at);
Cole = m(Colscs / Colsdey
(ExvoraSS if) Epo cit),

// remove dynamic arrays

delete [] ColSumx;
delete [] ColSumX2;
delete [] RowSumX;
delete [] RowSumxX2;
}

void ANOVA2Rclass: :ANOVA2R(Matrix& DataMat,

int NumRows, int NumCols,
int NumReplicates)

amt ae lp pak ROWS

double x, sx;
int nCol, nRow;
double RowCol, sum;

/* initialize statistical summations */

ColSumxX = new double[NumCols];
194 Chapter Nine

Listing 9.5 (Continued)

RowSumX = new double [NumRows] ;

Matrix CellSum(NumRows, NumCols) ;

for (j = 0; 3 < NumCols; j++)

Colsumxgi) = 10F
fon (7 = 0; 37 < NumRows; 7-2)
RowSumX[j] = 0;
for (i = 0; i < NumRows; i++)
fom (9 = 07 5) = NumGols; 44+)
Celi Sum (ay, sae = Ol

Num_Row = NumRows;
Num_Col = NumCols;
NReplicate = NumReplicates;
GrandSumX = 0;
GrandSumx2 = 0;

/* update statistical summations */

for (i = 0; 2 < NumRows; i++) {
iRow = i / NReplicate;
Fon (5 = 0; 3 < NumCols; j++) {
x = DataMat(i, j);
sx = SOR(x);
CellSum(iRow, j) += x;
RowSumX[iRow] += x;
ColSumX[j] += x;
GrandSumxX2 += sx;

}
k = NumRows / NReplicate;
Num_Row = k;
Num_Col = NumCols;
for (j = 0; j < Num_Col; j++)
GrandSumX += ColSumxX[j];

/* perform two-way ANOVA calculations EU

nCol = Num_Col;
nRow = Num_Row;
RowCol = Num_Row * Num Col * NReplicate;
Total_SS = GrandSumx2 - SQR(GrandSumxX) / RowCol;
Row SS. =" 0;
for (ai = 0) as <= nRow;s a?)
Row_SS += SQR(RowSumX[i]) /
(Num_Col * NReplicate);
Row_SS -= SQR(GrandSumx) / RowCol;
€ol_ss = 0F
for (2 = OF 02 = mote ies)
Col_SS += SQR(ColSumx[i]) /
(Num_Row * NReplicate);
Col_SS -= SQR(GrandSumx) / RowCol;
sum = 0;
for (2 = Ove < nRow; actd)
£ox (5) = 0; 3 <= mCol; 52-4)
sum += SQR(CellSum(i, j));
Interact_SS sum / NReplicate =
- Row_SS -
Col_SS - SQR(GrandSumxX) / RowCol;
Error_SS = GrandSumx2 - sum / NReplicate;
Total_SS = GrandSumx2 - SQR(GrandSumxX) / RowCol;
Row_df = Num_Row - 1;
Col_df = Num_Col —- 1;
Interact_df = Row_df * Col _df;
Totaldt = RowColl = 1;
Error_df = Num_Row * Num_Col *
 The ANOVA Tests 195

(NReplicate - 1);
Row_F = (Row_SS
/ Error SS) *
(Error_df / Row_df);
Colmes = ((Coless. 7 Eirronsss) ss
(ier Ey MCOdmcir)ir
Tneeract f= Winteract.SSe/ Error. Ss) +
(Exror
di / Interact
d£) ;
// xemove dynamic arrays
delete [] ColSumx;
delete [] RowSumx;
}

void LatinSqrclass::Latin(Matrix& DataMat, int* Map,

int NData)
{
Tigh, “kee yl apelee
double x, sx, g;
/* initialize statistical summations */
ColSumX = new double[NData];
RowSumxX = new double[NData];
TrtSumX = new double[NData];

for (a) =_Oe a < Nee)

Colsum<|2] =" 0);
RowSumX[i] = 0;
Te Sumi | al) =) Ole
}
GrandSumxX = 0;
GrandSumx2 = 0;
Num = 0;

/* update statistical summations */

N = NData;
/* Obtain sums of columns, rows and treatments */
for (i = 0% a < oN a+)
Num++ ;
Bore a) OR a) << NPE Ghee) oat
x = DataMat(i, j);
Sx =) SOR(x)F
ColSumX[j] += x;
RowSumX[i] += x;
k = *(Map + i + N * 3);
TrtSumX[k] += x;
GrandSumX2 += sx;
}
GrandSumX += RowSumX[i];
}
/* perform Latin Squares ANOVA */
Row_SS = 0;
CodessS— a0!
Pre SS esas
Ore =sOle a <Nis acta)
Row_SS += SQR(RowSumX[i]) ;
Col_SS += SOR(ColSumX[i]) ;
Trt_SS += SOR(TrtSumX[i]);
}
g = SQR(GrandSumxX / Num) ;
Row_SS = Row_SS / Num — g;
Colsss ="Col ssi/* Num <a;
Ria CNY Th bral atts S/N bse aS enh
Error_SS = GrandSumX2 -— Row_SS -
CoisS. =r SS e=1g7
Totalss = (GrandSumXx2. = g;
Row df = Num — 1;
Coledt = Nun’ —" 1;
196 Chapter Nine

Listing 9.5 (Continued)

iiiete ole my ANjblag) See

Hrror_ dt = (Num = 1) * (Num = 2);
POG ciel o—) SOR (ING) ieee
ROwLE) = (Num = 2) 9* sRowlss) / shrroruss;
Cole = (Num =) 2) s+eColss) /\Erroxsss;
Uureie ce. (UN(Ein = ))) eI, 7) apahetogs/S)-

delete [] ColSumx;
delete[] RowSumx;
delete [] TrtSumxX;
}
void ANOCOVclass:
: ANOCOV (
Matrix& DataMat,
int NData, int NumSets,
int HasMissingData,
double MissingCode)

aie a, el een Nee NDatasts NumSets.

double x, y;
double dSumxX, dSumY, dSumx2;
double dSumy2, dSumxXyY;

/* initialize statistical summations */

sum = new double[N];
sumX = new double[N];
sumxX2 = new double[N];
sumY = new double[N];
sumY2 = new double[N];
sumXY = new double[N];

fore (Cie = Ove: < Np a) 4

Siamese =s (Ol
sumx [a] = 0;
sumx2 [i] = 0;
Suma sede = OF
sumyv2 [a] = 0
sumxyY [i] 0 /
}
GrandSum = 0;
RSS = 0
ASSy = 0;
ASP = 0;
hasMissingData = HasMissingData;
missingCode = MissingCode;

/* update statistical summations */

Num_Set = NumSets;
for (1 = 0; i < NData; i++)
for (j = 0; 3 < Num_Set; j++) {
| Ware ye charley
ee el,
x = DataMat(i, 3X);
if (!(hasMissingData && x <= missingCode)) {
y = DataMat(i, JY);
sum[j]++;
sumX[j] += x;
sumX2[j] += SOR(x);
sumY[j] += y;
sumY2[j] += SQR(y);
sumxy[j] += x * y;
 The ANOVA Tests 197

/* carry out ANOCOV calculations */

dSumx = ,
asumy = oo
dsumx2 i

dSumy2
aSumxy Ors
SFO)

fonw(g ou 0;
MoM j < Num_Set; j++) {
ASSx += SQR(sumX[j]) / sum[j];
ASSy += SQR(sumY[j]) / sum[j];
ASP += sumX[j] * sumY[j] / Sum[j];
GrandSum += sum[j];
dSumX += sumX[j];
dSumX2 += sumX2[j];
dSumy += sumY[j];
dSumYy2 += sumY2[j];
dSumXY += sumXY[j];
}
TSSx = dSumX2 -— SQR(dSumX) / GrandSum;
TSSy = dSumyYy2 - SOR(dSumY) / GrandSum;
ASSx -= SQR(dSumxX) / GrandSum;
ASSy -= SQR(dSumY) / GrandSum;
WSSx = TSSx - ASSx;
WSSy = TSSy —- ASSy;
Df£1 = Num_Set - 1;
IDNES) -S. iDhails
Df£2 = GrandSum —- Num_Set;
DEL T= DE2Z =" ik;
TSP = dSumxXyY - dSumxX * dSumy / GrandSum;
ASP -= dSumX * dSumY / GrandSum;
WSP = TSP — ASP;
AMSx = ASSx / Df1;
AMSy = ASSy / Df1;
WMSx = WSSx / D£2;
WMSy = WSSy / Df2;
Fx = AMSx / WMSx;
Fy = AMSy / WMSy;
TSSyhat = TSSy - SOR(TSP) / TSSx;
WSSyhat = WSSy - SQR(WSP) / WSSx;
ASSyhat = TSSyhat - WSSyhat;
AMSyhat = ASSyhat / Df£3;
WMSyhat = WSSyhat / Df4;
ANOCOV_F = AMSyhat / WMSyhat;

delete [] sum;
delete [] sumx;
delete [] sumx2;
delete [] sumyY;
delete [] sumy2;
delete [] sumxXyY;

Listing 9.5 implements the member functions declared in Listing 9.4. The imple-
mentation uses the algorithms that I presented earlier in this chapter. The member
functions create and remove dynamic arrays using the operators new and delete.
The member functions access the data members without the use of pointers or
structures.
Listing 9.6 shows the source code for the TSANOVA2.CPP test program. The pro-
gram performs the same tasks as its C counterpart. The main difference is that the
C++ version uses classes and sends C++ messages to the instances of these classes.
198 Chapter Nine

These messages carry out the statistical calculations. To compile the C++ test program
you need to include the files ANOVA.CPP and TSANOVA.CPP in your project file.

Listing 9.6 The source code for the TSANOVA2.CPP test program.

#include <stdlib.h>
#include <stdio.h>
#include "anova.hpp"
#include "global.h"

void pressAnyKey (void) ;

main ()
{
Matrix mat(10, 10);
int map[25];
ANOVAIclass ri;
ANOVA2class r2;
ANOVA2Rclass 1r3;
LatinSqrclass r4;
ANOCOVclass r5;

mat(0, O}} = 88;

mat(l, oj = 89>
mat (2, 0) =996;
mat(3, 0) = 68;
mat (4, OF = 85
mat (5, 0) = 2 * MESSING DATA;
mat(6, 0) = 2 * MISSING DATA;

Mae (0; woe = TS

mae (i, A) = 625
mat (2,7 2) =- 983
mate, 2) = 83);
matta, 1) = 61;
Met siek) =) 88;
mat(6, 1) = 2 * MISSING DATA;

mat(0, 2) = 80;
mat (ds, 42) = 61>
ah CDA) = hee
mac(s, 2) = 92>
mat(4, 2) = 78s
mat(S, 2)0= 54;
Matte, a= Ths

mat(0, 3) = 71;
MAE(L, 3) = 65s
mat(2, 3) = 90;
mak(S, 3) = 46%
mat(4, 3) = 2 * MISSING DATA;
mat(5, 3) = 2 * MISSING DATA;
mat(6, 3) = 2 * MISSING DATA;

r1.ANOVA1 (mat, 4, 7, TRUE, MISSING_DATA) ;

PLint£ ("* KH KKKKKKAKE one-way ANOVA BS ERIS RE Vey ie

printf ("Treatment SS = $lg\n", ri.Treatment_SS) ;

printf("Treatment df = $lg\n", r1.Treatment_df);
printf("Treatment MS = %lg\n", r1.Treatment_MS);
printf("Error SS = %lg\n", rl.Error_SS);
print£("Error df = $lg\n", rl-Error_daf);
printf("Error MS = $lg\n", r1.Error_MS);
print£ ("Total SS’="Sig\n*, £1. otal Ss) =
printf ("Total afisutiavn", wi Wotal die
printf("F = lg\n", r1.ANOVA1_F);
 The ANOVA Tests 199

pressAnyKey
() ;

mich: (Ole O))) == aaa.

Mati, ‘O)}) = 2s
Atel te((2prO))) tee
Meus (0) 1) = 16>
nmat(L, y= 4<
matt2z, 1) = 6;
mat(0, 2) = 8:
mMat(l, 2) = 4;
Matt. (2), °2)) 5;
mat(0, 3) 7;
mat(l, 3) = 4;
Mae (245) = Se

isda (APRs Ae SH two-way ANOVA RRA RK KK KAK Vy) .

r2.ANOVA2 (mat, 3, 4);

Peintke (SRow SS) = tilg\n". 22. Row ss) >
Printer (“Row di = tig\n", 2. Row de);
printf ("Row F = %lg\n", r2.Row_F);
jonmalsahene (LGN Sis) Arle] Nm, ne?) eo sisi ie
osetia Com sci —— eagle toa COlmuclis)i
jongamolens ((Uieeyly hal ses Kyle ial Gah) a@royl iahep
jonmeumiore (Minhsigeia Sou pile Naa! 5 nae) Siaresctonaunsts)))) 2
jhersohore (Wharaaonas Che — “ale \wal) 5 see) siahesctonau else)
Disinte (Total SS =: slg wees Total assyr
Dist nem Total ds = elo). Total dt)

pressAnyKey
();

Mieaie(O), 10) — rr
Mictte(ales Oh) 9 Sie
ict, 10) — ee
miehel(Sl, OD)s Se ale
Piette (20)! lees
mats, 0) = *L>

Merte(Oip d\n Ale

meet) cL)eew 35s
mati GLb) t= 0;
MAES. (Lj) tHehe
matei(4y, 91) —seler
matteSiy, ©Lee ey

Mmae(Olp 2) earlSis
mat(l, 2d» = 0..5;
Maei(2 7, 2) = aa;
mMakisin 2) = 0
mati4a, 2) = ls
tebe (Sy, 2) ae le

Mictt=((Ohue shal. iy
Meets 1h...
mati2, 3) = =-1;
mars, 0s) = Ol
Marta. oS) = Ul5%
meets, s) = L;

printf ("*** two-way ANOVA with replication ***\n");

r3).ANOVA2R (mat, 6, 4, 2);
printf("Row SS = %lg\n", r3'.Row_SS);
Drinte("Row df = $ig\n", x13 Row de);
printf("Row F = %lg\n", r3.Row_F) ;
praner ("Col Ss SL Nid. COU _SS.) 7
Preamiri "Cod Ge iT) Silg\m pes EOL AE ).;
Deinte( Gol fy — sig\n"j ers Col _F) ;
printf("Interaction SS = %lg\n", r3.Interact_SS) ;
200 Chapter Nine

Listing 9.6 (Continwed)

printf£("Interaction df = $lg\n", r3.Interact_dEf);

printé ("Interaction F = $ilg\n", r3.iInteract_F) >;
Price "hirronr SS e alog\n", ro Error SS) +
(Oferighess (Winiarona teh cx Valle Nia’, hash aiSnsiaore
“ohi) iy
Drier ("Total SS = slig\n"™) 3 Total Ss).
DEIMcLe( "Total dae tlg\n") roetTotaloldt)

pressAnyKey
();

map [0]
map[1] =
map[2] =
map [3]
map[4] =
map[5] =
map[6] =
map[7] =
map[8] =
map[9] = ONNOWEWENO
map Loy =
map ili] =
map[12] =
mapa =
map[14] =
map[15] = ONRFPWEREWs~

mat(0, 0) I Bom aor

mat (0, BP i PRS). (sulle
matiiG 2) = 217.82.
mat (OFS )ip= 29) 45.
Mere GemON =e Ole
Malte Glen) == 2 oOSor
mate, 2) =) 2'6)..4 Gis
mat( 3S) = 2913);
Dichter) n= Or ais
eho, (Ape: Sh) eee ONO) E
Mat (2, 2) = 29.31
MeeC2 panS)) eet 2a Ole
makits:, WO) P="29 15)
Mavi3 ae) = eer Ost
Mert ((3:An |) a= Sisrs
mat3), 3) 2 6S

Print ("#*RRKKKKKKK Latin Square ANOVA PII Ree Rea as I aN Ee

r4.Latin(mat, map, 4);

printf("Row SS = %lg\n", r4.Row_SS);
printf("Row df = %lg\n", r4.Row_df);
printf("Row F = %lg\n", r4.Row_F);
DEanti("col SS = Sig Wn", 24.Col ss).
Printé ("Col dé = siig\n eas 2Col de);
prints ("Col F = Sig\nty recor.)
printf("Treatment SS = $lg\n", r4.Trt_SS);
printf("Treatment df = $lg\n", r4.Trt_df);
printf("Treatment F = $lg\n", r4.Trt_F);
printf("Error SS $lg\n", r4.Error_SS);
printf("Error df Sig\niy TA. Boro dibs
printf("Total ss $lg\n", r4.Total_SS);
printf("Total df Sig\n", m4 totals dt).

pressAnyKey
() ;

mat (0, 0) = 3);

mat(1, 0) 23
mat. (25 0!) Saad
mat(3, 0) = 2;
 The ANOVA Tests 201

nee (Ny aly). Sa abOr

Taye (GE lB) ea) 13h
Macaw)
Metts, 1) = ais;

mat(0, “2 "=" 4:
MicitalGleeee ye = Shs
inait(27) 2) p= eos
ENE (Si, CA Ey asp

mat(0;, 3)) = 12;

ene (5 S})) tes aloes
anes (2), 2) es alle
nineye(Z), S}))) T=) alec

mere e4))) =) he
fore (il 4h) ete
Matta. 4)! =) 3
mati(3i; 4) = L;

mat(0, 5) = 6;
mat (i, 5) = 5;
Mabie, 5) =—.o7
Macks.) =

WETICE (Uk REE A EAA AK ANOCOV FER KIX ARCA K 2G Na) :

r5.ANOCOV(mat, 4, 3, FALSE, 0);

printf("Df among means = %lg\n", r5.Df1);
Diane (UD witha igmOupse— slg iat 5. Dia lh;
prance ("ASSx = Silo \n eo ASSxX) >
Dremnice (ASP) =esilg Nima Mose ASP) ts
preantr ("ASSy = Sig\n"725-ASSy))>
Deinte ("WSSx = Sighn™, %5.WSSx);
prant£("WSP = tlg\n", ©£5.WSP) ;
PLM ("WOSve=s eliotn! pe SmNSGy) ¢
josesaers((Wunstepemay Caio Gell sesiihcisy.<i:
panic (UPS P= pel an eS SP) ie
Prine (MUSSve—eeag NG) aco LOS) pe
printf ("------- Residuals statistics --------
printf("D£ among means = %lg\n", r5.Df3);
printf("Df within groups = %lg\n", r5.Df£4);
PLrintke (MVASSy* = Silg\n"; xSe.ASSsyhat) ,
printf ("AMSy* = %lg\n", r5.AMSyhat) ;
printf ("WwSSy* $lg\n", r5.WSSyhat) ;
printf ("WMSy* = %lg\n", r5.WMSyhat) ) ;
primer "sSyve = slg\n", 25. TSSyhat ) /
(oeaiell
ea (Minbe waite GANS alse) ae
paimGe Gb yee Sg \n"l, bey)!
pEeinte(“Fe= Slg\n", +15,.ANOCOV_EF) ;

pressAnyKey
() ;

return 0;
}
void pressAnyKey
()
{
printf("\nPress any key to continue...");
getchar();
puts GN);
}
e287 6U9Ha

<

+;
: cas

Mite
Liar
i < aT th
4 ‘ny
"MLS
5
»*
a, 1 piEary
Lage
ry Wy
e1g
‘<—— e
aes |
td
pe 2) thy

a |

othe 39
roan
' a
at ie

we

eu 4 vu
2-72F *
clue

a
’ 7 ee |
ba§ as ig

a fel

s
 Chapter

10
Linear Regression

Linear regression empowers you to examine the relationship between two variables.
The basic linear regression calculations yield the best line which passes through the
observations. By applying mathematical transformations to the variables you can
also obtain the best fit for different kinds of curves. This chapter looks at the follow-
ing aspects of linear regression:

= Basic linear regression

= Linearized regression
= The confidence interval for the project values
# The confidence interval for the regression coefficients
= Testing the values of the regression coefficients
a The automatic best method

Basic Linear Regression

The basic linear regression examines the correlation between an independent vari-
able, x, and a dependent variable y. Linear regression attempts to find the slope and
intercept for the best line which passes through the observed values of x and y. This
simple relation is given by the following equation:

y=A+ Bx LOEB)

Where A is the intercept and B is the slope. To calculate the slope for the best line
use the following equation:

R= (zx, y, = (2x,) (Zy,))

10.2
(n=x,? — (2x,)*) : )

203
204 Chapter Ten

Figure 10.1 A sample linear regression.

The next equation calculates the intercept A:

_ @y,-B Xx)
A =e ae (10.3)

The coefficient of determination, R?, indicates the percent (as a fraction) of the
variation between variables x and y which is explained by equation 10.1. When R? is
1, you have a perfect fit and equation 10.1 fully explains the relation between vari-
able x and y. By contrast, when R? is 0, there is no relation between the two variables.
The next equation shows how to calculate the value for R?:
_ (Ady, + B&xy, - (Zy,)?/n]
R? (10.4)
: [Zy,? — (2y,)*/n]
The ANOVA associated with linear regression calculates the following items:
1. The variances:

S(X*) = Xx? -n@x)? (10.5)

S(?) = Zy?-n(y,)? (10.6)

S(XY) = Ixy, — n(2x,) (Ly,) (10.7)

2. The sum of squares for y, due to A + Bm,, due to B, and the residuals:

SSY=Ey2 - (10.8)
 Linear Regression 205

gga - 2y0" n
(10.9)

SSB = b S(XY) (10.10)

SSD = S(¥2) -b S(XY) (10.11)
where m, is the mean value for x.

3. The degrees of freedom for Y, due to A + Bm,, due to B, and the residuals:

Of Yen. (10.12)

dfA = 1 (10.13)

fe eall (10.14)

ResDf = n-2 C1045)

4. The mean squares for the slope B and for the residuals:

MSB = b S(XY) (10.16)

SSD
Seypee= (n — 2) (10.17)

Linearized Regression
Equation 10.1 represents the simplest relationship between two variables. Often, we
already know or strongly suspect that a pair of variables are related in a nonlinear
fashion that can be linearized by applying the appropriate transformations. When
this nonlinear relation can be linearized, you can use the above equations on the
transformed data. Thus for example, if you have variables x and y related in the fol-
lowing power function:

ya Ae (10.18)

Then by applying the In function to both sides of equation 10.5 you can linearize
tee

In(y) = In(A) + B In(x) (10.19)

You can regard equation 10.19 as:

vy =A +B x (10.20)
where y' is In(y), A' is In(A), and x' is In(x). Thus, applying linear regression to In(y)
and In(x) yields the intercept In(A) and the slope B.
You can apply the above method with other functions, such as the square, square
root, and reciprocal, to obtain a linear fit.
206 Chapter Ten

7‘ y*(upper)
+ y=A+Bx

y (lower)

Figure 10.2 A sample depiction of the confidence interval band for the projected values of the dependent
variable.

The Confidence Interval for Projections

Once you calculate the slope and intercept for the best line, you can use equation
10.1 to calculate the projected values of y for given values of x (and vice versa). What
about the confidence interval for the projected value of y? Here is the equation for
calculating the confidence interval for y:
= 2 :

Yo + tise J[p2 + oral (10.21)

where y, is the projected value of y, t,, 5... 1s the student-t value for n — 2 degrees of
freedom, and o is the risk probability (1 — @ is the confidence probability).

The Confidence Interval for Regression Coefficients

The slope and intercept values represent estimates for the mean value for the real
slope and intercept. You can calculate the confidence interval for the slope and in-
tercept using the following equations:

Pe= n-2;0/2
Cake Jl 2Bee
( pba We
n a S(X°) ) (10 22)

5 2
Bote tees
n-2;0/2
payee ae
SCK?) |
2
(10 23)

You can also use equation 10.22 and 10.23 to test whether the slope and intercept
statistically differ from specific values. As for the correlation coefficient (which is
 Linear Regression 207

merely the square root of the determination coefficient) you can use the next in-
equality to test if the value of the correlation coefficient differs from 0:

V M2)
IRI SS > thon (10.24)
VCR)
The Regression Algorithms
Let’s look at the algorithms involved in linear (and linearized) regression. Here is the al-
gorithm for performing the linear regression. This process includes initializing the sta-
tistical summations, accumulating data in these summations, and calculating the
regression coefficients and regression ANOVA statistics:
Given: the matrix Xmat with N observations, the indices Xindex and Yindex
(which select the X and Y variables from matrix Xmat), the transformation functions
fx, fy, and invfy.

1 Set Sum, SumX, SumX2, SumY, SumY2, and SumXY to 0.

2 For I= 0 to N—1 repeat the next steps:
2.1 Set Xr = Xmat(I, Xindex]
2.2 Set Yr = Xmat[I, Yindex]
2.3 If Xr or Yr represents missing data resume at step 2
2.4 Set Xr = fx(Xr)
2.5 Set Yr = fy(yr)
2.6 Increment Sum
2.7 Add Xr * Yr to SumXY
2.8 Add Xr to SumX
2.9 Add Yr to SumY
2.10 Add Xr squared to SumX2
2.11 Add Yr squared to SumY2.

Set MeanX = ue :
Sum

Set MeanY = a?
um
Set SdevX = V ((SumXX — SumX?/Sum)/(Sum — 1)).
Set SdevY = V ((SumYY — SumY?/Sum)/(Sum — 1)).
Set Slope = (SumXY — MeanX * MeanY * Sum)/SdevX?/(Sum — 1).
O1 Set Intercept = MeanY — Slope * MeanX.
COND
Ne} Set R2 = (Slope * SdevX/SdevyY)?.
10 Set RegSS = (SumXY — SumY * MeanX)?/(SdevX? * (Sum — 1)).
ti Set TotalSS = SumYY — SumY?/Sum.
12 Set ResidualSS = TotalSS — Regss.
13 Set Residualdf = Sum — 2.
14 Set S = V (ResidualSS\Residualdf).
15 Set StdErSlope = S/V (SumXX — SumX?/Sum).
16 Set StdErIntercept = S * V (SumXX/Sum/SdevX?/(Sum — 1)).
208 Chapter Ten

17 Set Regdf = 1.
18 Set Totaldf = Sum - 1.
19 Set S2=S*S.
RegSS
20 P= Se
39

The Automatic Best Fit

Since computers are ideal for systematic number crunching, you can use them to
systematically fit a set of data with various kinds of curves. Each curve requires a
distinct combination of transformation functions. Moreover, each curve yields a set
of regression coefficients and determination coefficient. The best curve fit is the one
which produces the highest value for the determination coefficient. In this chapter I
present the source code for the automatic best fit which sorts the results of 64 curve
fits in order of descending determination coefficient values. If you apply square root,
logarithm, and reciprocal functions, then you need to make sure that the examined
data contain only positive values.

The C Source Code

Let’s look at the C source code which implements the regression analysis and re-
lated tests and calculations. Listing 10.1 shows the source code for the LINREG.H
header file. Listing 10.2 shows the source code for the LINREG.C implementation
file.

Listing 10.1 The source code for the LINREG.H header file.

#ifndef _LINREG_H_
#define _LINREG H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/18/94

C module which performs the following linear

regression operations:

+ Basic linear regression

+ Confidence interval for regression coefficients
+ Testing regression coefficients
+ Automatic best linearized fit
ay

#include "arrays.h"

#define LINREG
MAX FUNCTIONS 64

/* ANOVA table record structure */

struct LinRegANOVAtag {
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_Ss;
double Total_df;
double Total_SS;
double S2;
 Linear Regression 209

double LINREG_F;
Ie
typedef struct LinRegANOVAtag LinRegANOVA;

/* Linear regression record structure */

struct LinRegtag {
int Xindex;
int Yindex;
int hasMissingData;
double (*fx) (double) ;
double (*fy) (double) ;
double (*Invfy) (double) ;
double missingCode;
double sumxX;
double sumxXx;
double sum;
double sumyY;
double sumYY;
double sumXY;
/* basic statistics */
double Meanx;
double Meany;
double Sdevx;
double SdevyY;
double Slope;
double Intercept;
double R2;
/* Regression results */
double StdErSlope;
double StdErIntercept;
LinRegANOVA ANOVA;
yi

typedef struct LinRegtag LinRegrec;

struct BestLinRegtag {
int fxIndex;
double R2;
double Slope;
double Intercept;
ae

typedef struct BestLinRegtag BestLRrec;

void InitializeLinReg(LinRegrec* r,
double (*Fx) (double),
double (*Fy) (double),
double (*InvFy) (double),
int XIndex, int YIndex,
int HasMissingData,
double MissingCode) ;

void sumLinReg(Matrix DataMat, LinRegrec* r, int NData) ;

void LinReg(LinRegrec* r);

void YhatCI (LinRegrec* r, double xr, double probability,

double* yHat, double* yHi, double* yLow);

void LinRegCoefCI (LinRegrec* r, double probability,

double* slopeHi, double* slopeLow,
double* intHi, double* intLow) ;

void LR_Slope_T_test(LinRegrec* r, double probability,

double testValue,
210 Chapter Ten

Listing 10.1 (Continued)

double* calcT, double* tableT,

int* passTest) ;

void LR_Int_T_test(LinRegrec* r, double probability,

double testValue,
double* calcT, double* tableT,
int* passTest) ;

void LR_R2_ T Test (LinRegrec* r, double probability,

double* calcT, double* tableT,
int* passTest);

int BestFit (Matrix DataMat,

int NData, int numTrnsFx,
int Xindex, int Yindex,
double (*fx[LINREG_MAX FUNCTIONS] ) (double),
double (*fy[LINREG_MAX FUNCTIONS] ) (double),
BestLRrec* r);

#endif

Listing 10.1 shows the declaration of the constant LINREG_MAX_ FUNCTIONS,

the typedefed structures LinRegANOVA, LinRegrec, and BestLinReg. The above
constant specifies the total number of functions examined by the automatic best fit
function. The structure LinRegANOVA stores the fields for the regression ANOVA.
The structure LinReg stores the fields for the linearized regression. The structure
BestLinReg stores the data for the current fit.
The listing declares the following C functions:

. The function InitializeLinReg initializes the LinRegrec structure accessed by

pointer r. The function includes additional parameters which specify the pointers
to the transformation functions, the indices for the regression variables, the miss- -
ing-code flag, and the missing-code value. You must call this function at least once
to initialize the LinRegrec structure. Additional calls to this function (with the ad-
dress of the same LinRegrec structure) help you reset the targeted regression
structure and prepare it for another round of calculations.
. The function sumLinReg updates the statistical summations. The function has pa-
rameters which specify the data matrix, the pointer to the LinRegrec structure,
and the number of observations. You must call this function with enough data at
least once. You can call this function multiple times to accumulate a large number
of observations. Keep in mind that in this case, the data matrix should pass data
with the same number of variables.
The function LinReg performs the linear regression calculations. The function has
a single parameter which is a pointer to a LinRegrec structure. This structure
passes data to and from the function LinReg. You must call this function before
calling the next five functions, since these functions expect the accessed LinRe-
grec structure to contain regression results.
The function YhatCl calculates the projected y value and its confidence interval.
The function has parameters which access a LinRegrec structure, the value of x
 Linear Regression 211

to project, the probability for the confidence interval, and pointers to the pro-
jected y, lower interval value of y, and higher interval value of y.
. The LinRegCoefCl calculates the confidence intervals for the regression slope
and intercept. The function has parameters which access a LinRegrec structure,
the probability for the confidence intervals, and pointers to the low and high lim-
its of the slope and intercept.
. The function LR_Slope_T_test tests the value of the slope. The function has pa-
rameters which access a LinRegrec structure, the probability for the confidence
intervals, the tested slope value, and pointers to the calculated student-t value,
the tabulated student-t value, and the Boolean flag. The latter flag indicates
whether or not the tested value is accepted.
. The function LR_Int_T_test tests the value of the intercept. The function has pa-
rameters which access a LinRegrec structure, the probability for the confidence
intervals, the tested intercept value, and pointers to the calculated student-t
value, the tabulated student-t value, and the Boolean flag. The latter flag indi-
cates whether or not the tested value is accepted.
. The function LR_R2_T_test tests whether the value of R (or R?) is 0. The function
has parameters which access a LinRegrec structure, the probability for the test,
and pointers to the calculated student-t value, the tabulated student-t value, and
the Boolean flag.
. The function BestFit performs the automatic best fit. The function has parame-
ters which specify the data matrix, number of observations, the number of trans-
formation functions, the index of variable x, the index of variable y, the array of
transformation functions for x, the array of transformation functions for y, and
the array of BestLRrec structure. The latter array passes the results of the auto-
matic best fit to the caller. The function sorts these results in the order of de-
scending values of R?.

Listing 10.2 The source code for the LINREG.C implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "linreg.h"
#include "statlib.h"

void InitializeLinReg(LinRegrec* r,
double (*Fx) (double),
double (*Fy) (double),
double (*InvFy) (double),
int XIndex, int YIndex,
int HasMissingData,
double MissingCode)
/* initialize statistical summations and data range */
{
r->sumX = 0;
r->sumxXX 0;
r->sum = ll
o
r->sumY = 0;
r->sumYY Oi
r->sumxY oil oO
212 Chapter Ten

Listing 10.2 (Continued)

r->hasMissingData = HasMissingData;
r->missingCode = MissingCode;
Pasi = Fst
resfy = Fy;
r->Invfy = InvFy;
r->Xindex = XIndex;
r->Yindex = YIndex;
}
void sumLinReg(Matrix DataMat, LinRegrec* r, int NData)
/* update statistical summations */
{
aie, als
double xr, yr;
double (*ffx) (double) ;
double (*ffy) (double) ;

iE ge) Remoer
ffy = r->fy;
eOiegn (te =O eee UNDalai net 3)
xr = MAT(DataMat, i, r->Xindex) ;
yr = MAT(DataMat, i, r->Yindex) ;
if (!(r->hasMissingData &&
((xr <= r->missingCode)| |
(yr <= r->missingCode)))) {
/* transform x and y data */
Sime = i(k iEheed)) (rae) p
yr = (*ffy) (yr);
/* Update summations */
r->sum += 1;
B-SSuUMRY Sac es yao
r->sumxk += xr;
r->sumxXX += SQR(xr);
r->sumyY += yr;
r->sumYY += SOR(yr);

}
void LinReg(LinRegrec* r)
/* calculate regression coefficients and related results */
{
double S;
r->MeanX = r->sumX / r->sum;
r->MeanY = r->sumY / r->sum;
r->SdevX = sqrt((r->sumxXX - SQR(r->sumX) / r->sum) /
(x->sum — 1));
r->SdevY = sqrt((r->sumYY - SOR(r->sumY) / r->sum) /
(r=>sum - 1));
r->Slope = (r->sumXY - r->MeanX * r->MeanY * r->sum) /
SQR(r->SdevxX) / (r->sum - 1);
r->Intercept = r->MeanY - r->Slope * r->MeanX;
r->R2 = SQR(r->Slope * r->SdevX / r->SdevyY);
r->ANOVA.Reg_SS = SQR(r->SsumXY - r->sumY * r->MeanX) /
(SQR(xr->SdevxX) * (r->sum - 1));
r->ANOVA.Total_SS = r->sumYY - SQOR(r->sumyY) / r->sum;
r->ANOVA.Residual_SS = r->ANOVA.Total_SS - r->ANOVA.Reg_SS;
r->ANOVA.Residual_df = r->sum - 2;
S = sqrt(r->ANOVA.Residual_SS / r->ANOVA.Residual_df) ;
r->StdErSlope = S / sqrt(r->sumXX - SQR(r->sumX) / r->sum);
r->StdEriIntercept = S * sqrt(r->sumXX / r->sum /
SQR(r->SdevX) / (r=->sum - 1));
r->ANOVA.Reg_df = 1; _
r->ANOVA.Total_df = r->sum - 1;
 Linear Regression 213

> ANOVAR Sam ons Oy

r->ANOVA.LINREG_F = r->ANOVA.Reg_SS / r->ANOVA.S2;
}
void YhatCI(LinRegrec* r, double xr, double probability,
double* yHat, double* yHi, double* yLow)
/* calculate projections and their confidence interval */
{
double Df, deltaY, p, tableT;
double (*ff£x) (double);
double (*iffy) (double) ;

fix = r=>tx-
iffy = r->Invfy;

ni (probabiiity > 1)
p= 0.5) — probability / 200);
else
Des 0n Si probabalertyss/ 2
DE = r=>sum =" 2):
table Te = Witry (pp Dts)
<r =" ("£fx) (xr); (* transform «rn */

deltaY = sqrt(SQR(xr - r->MeanX) /

(SQR(r->sumX) * (r->sum - 1))+1/
(r->sum+1)) * sqrt(r->ANOVA.S2) * tableT;
*yHat = r->Intercept + r->Slope * xr;
*yHi= (*iffy) (*yHat + deltay);
*yLow = (*iffy) (*yHat - deltaY);
*yHat = (*i£f£y) (*yHat) ;
}

void LinRegCoefCI(LinRegrec* r, double probability,

double* slopeHi, double* slopeLow,
double* intHi, double* intLow)
/* calculate confidence interval for slope and intercept ay
{
double Df, tableT, diff, p;

if (probability > 1)
p= 055 = probability. /2.005
else
De=20 5) — probabadentye s/ 52)
DE = r->sum - 2;
tableT = TInv(p, Df);
diff = tableT * r->StdErSlope;
*slopeHi = r->Slope + diff;
*slopeLow = r->Slope - diff;

diff = tableT * r->StdErIntercept;

*intHi = r->Intercept + diff;
*intLow = r->Intercept - diff;
}
void LR_Slope_T
test (LinRegrec* r, double probability,
double testValue,
double* calcT, double* tableT,
int* passTest)
/* compare slope value with a tested value
Hypothesis tested is HO: Slope = testValue */
{
double Df, p;

if (probability > 1)
p= 0.5 — probability 7 200;
else
De 0).5 = probabadlatyey, (2
DE = r—->sum - 2;
214 Chapter Ten

Listing 10.2 (Continued)

*tableT = TiInv(p, D£);

*calcT = (r->Slope - testValue) / r->StdErSlope;
*passTest = (fabs(*calcT) <= *tableT) ? TRUE : FALSE;
}

void LR_Int_T_test(LinRegrec* r, double probability,

double testValue,
double* calcT, double* tableT,
int* passTest)
/* compare intercept value with a tested value
Hypothesis tested is HO: Intercept = testValue */
{
double Df, p;

af (probability > 1)
p = 0.5>=> probability / 200;
else
p= 0S —sprobabaliity 72.
DE = r—Ssum = 2;
Seablen = iMinvi(DE, Dp)+
*calcT = (r->Intercept - testValue) / r->StdErIntercept;
*passTest = (fabs(*calcT) <= *tableT ) ? TRUE : FALSE;
}

void LR_R2_T
Test (LinRegrec* r, double probability,
double* calcT, double* tableT,
int* passTest)
/* Procedure to test hypothesis HO : R*2 = 0 */
{
double p, Df;

if (probability > 1)
pe= 025) — probabs kitty 9/9 200%
else
p = 0.5 - probability / 2;
DE = E->sum — 2;
*tablen = Tinv(p, DE);
*calict = sqri(s=SR2 * DE ff (Ll = e=SR2)
)i-
*passTest = (*calcT <= *tableT) ? TRUE : FALSE;
}

int BestFit (Matrix DataMat,

int NData, int numTrnsFx,
int Xindex, int Yindex,
double (*fx[LINREG_MAX FUNCTIONS] ) (double),
double (*fy[LINREG_MAX FUNCTIONS] ) (double),
BestLRrec* r)
/* find the best model to fit the data */
{
double sum, sumX, sumXX, sumY, sumYY, sumxXY;
double MeanxX, MeanY, SdevxX, SdevyY;
double x, y;
int i, j, offset, inOrder;
BestLRrec temp;

if (Xindex == Yindex ||
NData < 3 ||
numTrnsFx < 2)
return FALSE;

/* iterate over the functions */

for (1 = OF 2 < MumTrnshic 14h) 4
/* initialize summations */
sum = 0;
sumX = 0;
 Linear Regression 215

sumXX = 0;
Sumy =) 0);
sumyy = 0’;
sumxXY = 0;
/* process the observations */
for (j = 0; 3 < NData; j++) ¢£
x = MAT(DataMat, j, Xindex);
y = MAT(DataMat, j, Yindex);
22 Se (Gio
ieliPele)
y = (*fylil) (y);
sum++;
sumX += x;
sumY += y;
sumXX += SOR(x);
sumYY += SQR(y);
SUM) Soe om yi
}
/* calculate the results */
MeanX = sumX / sum;
MeanY = sumY / sum;
SdevX = sqrt((sumXX - SQR(sumX) / sum) /
(suml— =l)!);
SdevY = sqrt((sumYY - SQR(sumY) / sum) /
(sum - 1));
(r+1)->Slope = (sumXY - MeanX * Meany * sum) /
SOR(SdevxX) / (sum - 1);
(r+1)->Intercept = MeanY - (r+i)->Slope * Meanx;
(r+i)->R2 = SQR((r+i)->Slope * Sdevx / SdevyY);
(r+i)->fxIndex = i;
}
/* sort the results in descending order */
offset = numTrnsFx;
do {
orfset = (offset * 78) “/ 11;
Offset = (Cfisee == 10) tt Fs oteset:
inOrder = TRUE;
fon (2 =10- = (numirnshs = Ofiset)i; a+) 4
j = 1 + offset;
/* swap elements? */
if ((r+i)->R2 < (r+j)->R2) {
inOrder = FALSE;
temp = *(r + i);
cream ere Eh)
(eer) =" tempr
}
}
} while (! (offset == 1 && inOrder == TRUE));
return TRUE;

Listing 10.2 defines the functions declared in the header file LINREG.H. The func-
tions implement the algorithm and equations presented earlier in this chapter. These
functions rely on parameters, which point to regressions-related structures Lin-
Regrec and BestLRrec, to access the statistical results.
Listing 10.3 shows the source code for the TSLINREG.C test program. The pro-
gram performs the following tasks:

1. Creates the matrix mat and initializes it with values. The matrix has 10 rows and
5 columns. The initializing data occupies 7 rows and 2 columns (that is, the ma-
trix stores data for 2 variables).
216 Chapter Ten

2. Initializes the variable r which has the LinRegrec structure type. This task involves
calling the function InitializeLinReg. The arguments for this function include the
address of structure r, the functions fx, fy, ify, the indices for the x and y variables
(0 and 1), the FALSE missing code flag, and the missing code value of 0.
. Accumulates the selected data of matrix mat in the statistical summations of
structure r. This task calls function sumLinReg and passes the arguments mat,
&r, and 7 (the number of observations).
. Performs the basic linear regression calculations by calling the function LinReg.
The argument for this call is the address of the structured variable r.
. Displays the regression results. These results include the number of points, the
value of R?, the slope, the intercept, and the ANOVA table components.
. Obtains and displays the 95% confidence interval for the regression coefficients.
This task calls function LinRegCoefCl.
. Tests if the slope is significantly different from —1.5. This task calls the function
LR_Slope_T_test and displays the results and inference drawn from the test.
. Tests if the intercept is significantly different from 25. This task calls the func-
tion LR_Int_T_test and displays the results and inference drawn from the test.
. Tests if the value of R? is significantly different from 0. This task calls the func-
tion LR_R2_T_test and displays the results and inference drawn from the test.
10. Stores a new set of data in the matrix mat.
11. Stores a set of function pointers in the function-pointer arrays fxArr and fyArr.
12. Performs the automatic best fit by calling the function BestFit. The arguments
for this function call are mat, 5, numF'x, 0, 1, fxArr, fyArr, and bestR2. The latter -
argument is the array of BestLRrec structures.
13. Displays the results of the automatic curve fit.

To compile the C test program you need to include the files ARRAYS.C,
STATLIB.C, LINREG.C, and TSLINREG.C in your project file. Figure 10.1 shows the
output of the TSLINREG.EXE program.

Listing 10.3 The source code for the TSLINREG.C test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "linreg.h"
#include "global.h"
#include "arrays.h"

#define MAX FX 4

void pressAnyKey (void) ;

double fx(double) ;
double fy(double) ;
double ify(double) ;

double linear (double) ;

double 1n(double) ;
 Linear Regression 217

main ()
{
Matrix mat;
LinRegrec r;
double (*fxArr[MAX_FX]) (double) ;
double (*fyArr[MAX_FX]) (double) ;
double probability;
double slopeHi, slopeLow;
double intHi, intLow;
double testValue, calcT, tableT;
BestLRrec bestR2 [MAX_FX];
int numFx = MAX FX;
ent is
int passTest;

newMat(&mat, 10, 5);

MAT(mat, 0, 0) = 2;
MAT (mat) Oj; 2) =—245)5;
MAM (mat, 1, 0). =4)-
MAT (mat... jl Lb) =) Lon S:
MAT (mat, 2), 0!) =o
EAU atiet Emr) geet) ela
MAT(mat, 3, 0) = 8;
MAT (Maley. 3%,. el es aeae
MAT(mat, 4, .0) = 10;
MAT (matty «475 14) = elDr
MAT (mat, 5, 0) = 12;
MAT (mat ye 55 die =) di
MAT(mat, 6, 0) = 14;
MAT(mat, 6, 1) = 5;

InitializeLinReg(&r, fx, fy, ify, 0, 1, FALSE, 0);

sumLinReg(mat, &r, 7);
LinReg(&r) ;
PLEINEE ("**RAXRRAARARS Linear Regression RE KR EEE RE NTO) s

printf("Number of points: %lg\n", r.sum);

[RSS AN UREA ole iat) amined)ee
printf("Slope = %lg\n", r.Slope);
printf("Intercept = %lg\n", r.Intercept) ;
PEINCE (“FFXAAAAAAA KE * ANOVA EREEKEKK KERN") 5

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS) ;

printf("Regression df = %lg\n", r.ANOVA.Reg_df) ;
printf("Residual SS = %lg\n", r.ANOVA.Residual_SS);
printf("Residual df = %lg\n", r.ANOVA.Residual_df) ;
printf("Total SS = %$lg\n", r.ANOVA.Total_SS) ;
PuLnte (“Total dt = slg\n",, r ANOVA. Total_dtf) ;
printf("S*2 = %lg\n", r.ANOVA.S2) ;
printf("F = $lg\n", r.ANOVA.LINREG_F) ;
pressAnyKey
();

probability = 95;
LinRegCoefCI(&r, probability, &slopeHi, &slopeLow,
&intHi, &intLow);
printf("At %lg %% probability\n", probability) ;
printf("Range for slope is %lg to %lg\n", slopeLow, slopeHi) ;
printf("Range for intercept is lg to %lg\n", intLow, intHi);

pressAnyKey
() ;

testValue = -1.5;
LR_Slope_T test(&r, testValue, probability,
&calcT, &tableT, &passTest);
printf("At lg %% probability\n", probability) ;
Drance( Hs SLg L=reslg; wWusrclope;, testValue) ;
if (passTest)
218 Chapter Ten

Listing 10.3 (Continued)

printf("cannot be rejected\n") ;
else
printf("cannot be accepted\n") ;

pressAnyKey
() ;
testValue = 25.0;
LR_Int_T_test(&r, probability, testValue,
&calcT, &tableT, &passTest) ;
printf("At %lg %% probability\n", probability);
printf("HO: %lg ?=? lg, ", r.Intercept, testValue) ;
if (passTest)
printf("cannot be rejected\n") ;
else
printf("cannot be accepted\n");

pressAnyKey
();
LR_R2_T_Test(&r, probability, &calcT, &tableT,
&passTest) ;
printf("At %lg %% probability\n", probability) ;
Prantél( “HO Sl gauss no tmOu weet ee.)
if (passTest)
printf("cannot be rejected\n") ;
else
printf("cannot be accepted\n") ;

pressAnyKey
() ;
PLANtEL ("**RAKKKAKKKKES Automatic Best Fit RRR RERK ERK A ERE REN TM) s

MAT (mata) 07, an0)i= =) LO

MAT (mat, OF 2)! = 223026;
MAT(mat, 1, 0) = 25;
MAT (mat, Lb, 2) = 3.2189)
MAT(mat, 2, 0) = 30;
MAT (matte, 2 1)) es A Oe, =
MAM (mat, 93, 0) = 35)
MAT (ate 3) cL) = si 5 555.7
MAT(mat, 4, 0) = 100;
MAT(mat, 4, 1) = 4.6052;
fxArr[0] = linear;
fyArr[0] = linear;
fxArr[1] = linear;
fyArr [1] = dnr
te dN avalay To ial
fyArr[2] = linear;
fxArr[3] = ln;
fyArr[3] = ln;

BestFit(mat, 5, numFx, 0, 1, fxArr, fyArr, bestR2);

for (i = 0; i < numFx; i++) {
printf("Function number %d\n", bestR2[i] .fxIndex) ;
Printi("R*2 = sig\n™) sbestR2Z
[2 llR2)i-
printf("Slope = %lg\n", bestR2[i].Slope) ;
printf("Intercept = %lg\n\n", bestR2[i] .Intercept) ;
BOS (Ci((cstea0)! ae ===)
pressAnyKey () ;
}
deleteMat (&mat) ;
return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
 Linear Regression 219

pues ("\n\n" jr
}
double fx(double x)
{
return x;
}
double fy(double x)
{
TAN ehaal 5:6)
}
double ify(double x)
{
CECUTT Ser
}
double linear(double x)
{
return x;
}
double 1ln(double x)
{
return log(x);
}

KKKKKKKKEKKKKK Linear Regression KEK KKKKKEKEK

Number of points: 7
Ro = 0987656
Slope = -1.58929
Intercept = 26.9286
Pieri ie,oe ence ie a ie eeSe aie ANOVA KKKKKKKRKKK

Regression SS 282.893
Regression df alk
Resicucie SS u— ms jos .oul
Residual df =
Total SS = 286.429
Total dis =) 16
Sate Oh OAs
B= 4005051

Press any key to continue...

At 95 % probability
Range for slope is -1.79359 to -1.38498
Range for intercept is 25.1012 to 28.7559
At 95 % probability
HO: -1.58929 ?=? -1.5, cannot be rejected
Press any key to continue...
At 95 % probability
HO: 26.9286 ?=? 25, cannot be accepted

Press any key to continue...

At 95 % probability
HO: 0.987656 is not 0, cannot be accepted

Press any key to continue...

KEKE KKK KEKE Automatic Best Fit KKKK KEK KKK KEKE

Function number 2
i
a
Slope = 1
Intercept = -4.55519e-06

Figure 10.3 The output of the test program TSLINREG.EXE.

220 Chapter Ten

Function number 3
RA? = 10598334
Slope = 0.299307
Intercept = 0.181942

Function number 0
R*2 = 0.872967
Slope = 0.0220953
Intercept = 2.53283

Function number 1
OP S10) 5e/SWKSS}
Slope = 0.00628679
Intercept = 0.953095

Press any key to continue...

Figure 10.3 (Continued)

The C++ Source Code

Let’s look at the C++ source code which implements the linear regression algorithms
and equations that I presented earlier in this chapter. Listing 10.4 shows the source
code for the LINREG.HPP header file. Listing 10.5 shows the source code for the
LINREG.CPP implementation file.

Listing 10.4 The source code for the LINREG.HPP header file.

#ifndef _LINREG_HPP_
#define _LINREG_HPP_
/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 6/18/94

C++ module which performs the following linear

regression operations:

Basic linear regression

Confidence interval for regression coefficients
Testing regression coefficients
+ Automatic best linearized fit
++
xy

#include "arrays.hpp"

#define LINREG_MAX FUNCTIONS 64

/* ANOVA table record structure */

class LinRegANOVA {
public:
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_SS;
double Total_df;
double Total_SS;
double S82;
double LINREG_F;
3

/* Linear regression record structure */

class LinReg {
 Linear Regression 221

public:
LinReg() {}

void Initialize(double (*Fx) (double),

double (*Fy) (double),
double (*InvFy) (double),
int XIndex, int YIndex,
int HasMissingData,
double MissingCode) ;

double getSlope()
{ return Slope; }
double getIntercept()
{ return Intercept; }
double getR2()
{ return R2; }
double getSum()
{ return sum; }

void sumLinReg(Matrix& DataMat, int NData);

void DoLinReg();

void YhatCI(double xr, double probability,

double& yHat, double& yHi, double& yLow) ;

void LinRegCoefCI (double probability,

double& slopeHi, double& slopeLow,
double& intHi, double& intLow);

void LR_Slope_T_test(double probability, double testValue,

double& calcT, double& tableT,
int& passTest) ;

void LR_Int_T_test (double probability, double testValue,

double& calcT, double& tableT,
int& passTest) ;

void LR_R2_T_
Test (double probability,
double& calcT, double& tableT,
int& passTest) ;

LinRegANOVA ANOVA;

protected:
int Xindex;
int Yindex;
int hasMissingData;
double (*fx) (double) ;
double (*fy) (double) ;
double (*Invfy) (double) ;
double missingCode;
double sumX;
double sumXX;
double sum;
double sumyY;
double sumYY;
double sumXY;
[roagicstatistscs */
double MeanxX;
double Meany;
double SdevxX;
double SdevY;
double Slope;
double Intercept;
double R2;
/* Regression results */
222 Chapter Ten

Listing 10.4 (Continued)

double StdErSlope;
double StdErintercept;
pe
class BestLinReg {
public:
int fxIndex;
double R2;
double Slope;
double Intercept;
};

int BestFit (Matrix& DataMat,

int NData, int numTrnsFx,
int Xindex, int Yindex,
double (*fx[LINREG_MAX FUNCTIONS] ) (double),
double (*fy[LINREG_MAX FUNCTIONS] ) (double),
BestLinReg* r);

#endif

Listing 10.4 declares three classes: LinRegANOVA, LinReg, and BestLinReg. The
first and last classes are practically structures whose data members store various re-
gression-related information. The class LinRegANOVA contains data members
which store the regression ANOVA statistics.
The class LinReg has data members and member functions. Most of the data mem-
bers are protected. The class declares the public data member ANOVA which is an
instance of class LinRegANOVA. This kind of declaration allows you easy access to
the ANOVA table components. By contrast, you need to invoke member functions to
access the number of observations, as well as the regression slope, intercept, and R? .
value. The class also declares member functions which allow you to initialize the sta-
tistical summations, accumulate data, perform the linear regression calculations,
and perform various tests and post-regression calculations. These member functions
resemble the C functions declared in the LINREG.H header file. The member func-
tions differ from the C function in two main ways. First, these are no pointers to a re-
gression-structure in the member functions. Second, the member functions use
reference parameters, instead of pointers, to pass results, via parameters, back to
the caller.
The header file declares the class BestLinReg to store the data for the best curve
fit. The header file declares the function BestFit which is very similar to the C func-
tion BestFit in the LINREG.H header file.

Listing 10.5 The source code for the LINREG.CPP implementation file.

#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "linreg.hpp"
#include "statlib.h"

void LinReg::Initialize(double (*Fx) (double),

double (*Fy) (double) ,.
double (*InvFy) (double),
int XIndex, int YIndex,
int HasMissingData,
 Linear Regression 223

double MissingCode)
/* initialize statistical summations and data range */
{
sumX = 0;
sumxX = 0;
sum = 0;
sumY = 0;
sumyy = 0);
sumxY = 0;
hasMissingData = HasMissingData;
missingCode = MissingCode;
Ee— =Ps
fy = Fy;
Invfy = InvFy;
Xindex = XIndex;
Yindex = YIndex;
y
void LinReg::sumLinReg(Matrix& DataMat, int NData)
/* update statistical summations */
{
att i;
double xr, yr;
double (*ffx) (double) ;
double (*ffy) (double) ;

ieee ES Tego
ify = fy;

for (i = 0; i < NData; i++) {

xr = DataMat(i, Xindex) ;
yr = DataMat(i, Yindex) ;
if (!(hasMissingData &&
((xr <= missingCode) ||
(yr <= missingCode)))) {
/* transform x and y data */
sare a h(E Seg) (Grea)"5
yro=(*£E£Y) (ys);
/* Update summations */
sum += 1;
SUMXY +=) xe * yx;
sumX += xr;
sumXX += SOR(xr) ;
sumY += yr;
sumYY += SQR(yr);

void LinReg: :DoLinReg()

/* calculate regression coefficients and related results */
{
double S;
MeanX = sumX / sum;
MeanY = sumY / sum;
SdevX = sqrt((sumxXX - SQR(sumX) / sum) /
(sum - 1));
Sdevy = sqrt((sumYY - SQR(sumY) / sum) /
(sum = 1) );
Slope = (sumXY -
MeanX * MeanY * sum) /
SQR(SdevxX) / (sum - 1);
Intercept = MeanY - Slope * Meanx;
R2 = SQR(Slope * SdevxX / SdevyY);
ANOVA.Reg_SS = SQR(sumXY - sumY * MeanX) /
(SQR(SdevxX) * (sum - 1));
ANOVA.Total_SS = sumYY - SQR(sumY) / sum;
224 Chapter Ten

Listing 10.5 (Continued)

ANOVA.Residual_SS = ANOVA.Total_SS - ANOVA.Reg_SS;

ANOVA.Residual_df = sum - 2;
S = sqrt (ANOVA.Residual_SS / ANOVA.Residual_df) ;
StdErSlope = S / sqrt(sumXX - SQR(sumX) / sum);
StdErIntercept = S * sqrt(sumxXX / sum /
SOR (Sdevx) / (sum — 1));
ANOVA.Reg_df = i
ANOVA. Total_df sum — 1;
ANOVA.S2 = S * ;
ANOVA. LINREG_F ih
I
tap ANOVA.Reg_SS / ANOVA.S2;
}

void LinReg::YhatCI(double xr, double probability,

double& yHat, double& yHi, double& yLow)
/* calculate projections and their confidence interval */
{
double Df, deltaY, p, tableT;
double (*ffx) (double);
double (*iffy) (double) ;

eee aoe
Itty = Inviy;

ee (prObababluartey: y-. le)

p = 0.5>= probabilaty / 200;
else
Dp = 0.5 — probability / 2;
D£ = sum - 2;
table? = Tinv(p, DE£));
Re = (* Ef) (x) transform sac 4/
deltaY =sqrt(SQR(xr - MeanX) /
(SQOR(sumX) * (sum - 1))+1/
(sum+1)) * sqrt (ANOVA.S2) * tableT;
yHat = Intercept + Slope * xr;
yHi= (*iffy) (yHat + deltay);
yLow = (*iffy) (yHat - deltaY);
yHat = (*iffy) (yHat);
}

void LinReg:
:LinRegCoefCI(double probability,
double& slopeHi, double& slopeLow,
double& intHi, double& intLow)
/* calculate confidence interval for slope and intercept */
{
double Df, tableT, diff, p;

if (probability > 1)
p = 0.5 - probability / 200;
else
p = 0.5 = probability / 2;
DE = sum —-2;
tableT = TInv(p, Df);
diff = tableT * StdErSlope;
slopeHi = Slope + diff;
slopeLow = Slope - diff;

diff = tableT * StdErIntercept;

intHi = Intercept + diff;
intLow = Intercept - diff;
}

void LinReg::LR_Slope_T_test
(double probability,
double testValue,
double& calcT, double& tableT,
 Linear Regression 225

int& passTest)
/* compare slope value with a tested value
Hypothesis tested is HO: Slope = testValue */
{
double Df, p;

if (probability > 1)
p = 0.5 —- probability / 200;
else
p= 0 5e— probabalityv/) 25
DE V= "sum = 2;
tableD y= slimy. Die)
calcT = (Slope - testValue) / StdErSlope;
passTest = (fabs(calcT) <= tableT) ? TRUE : FALSE;
}
void LinReg:
:LR_Int_T_test(double probability,
double testValue,
double& calcT, double& tableT,
int& passTest)
/* compare intercept value with a tested value
Hypothesis tested is HO: Intercept = testValue */
{
double Df, p;

if (probability > 1)
Dp = 0.5 = probability / 200;
else
= 025) —) probabmlatye/ 2;
Dr = sum — 27
tablet. = Tinw(DE Ap);
calcT = (Intercept - testValue) / StdErIntercept;
passTest = (fabs(calcT) <= tableT ) ? TRUE : FALSE;
}

void LinReg::LR_R2_T Test(double probability,

double& calcT, double& tableT,
int& passTest)
/* Procedure to test hypothesis HO : R*2 = 0 */
{
double p, Df;

tf (probabaltty > 1!)

De=) 0.5 e— prebabiiliaty 7 200%
else
Dea Oso —sprobabilatity 7 )2
DE = ‘sum = 25
tabten = Tinv(p, DL);
Calicihe—soqrinn(Re ~. Di /) “(ls — 32)
passTest = (calcT <= tableT) ? TRUE : FALSE;
}
int BestFit (Matrix& DataMat,
int NData, int numTrnsFx,
int Xindex, int Yindex,
double (*fx[LINREG_MAX FUNCTIONS] ) (double) ,
double (*fy[LINREG_MAX FUNCTIONS] ) (double),
BestLinReg* r)
/* find the best model to fit the data */
{
double sum, sumX, sumXX, sumY, sumYY, sumXY;
double MeanX, MeanY, SdevxX, SdevyY;
double x, y;
int i, ji, offset, inOrdex;
BestLinReg temp;

if (Xindex == Yindex | |
NData < 3 ||
226 Chapter Ten

Listing 10.5 (Continued)

numTrnsFx < 2)
return FALSE;

/* iterate over the functions */

for (2 = 0; 2 < numTrmsrx; i++) 6
/* initialize summations */
sum = 0);
sumX 0; =
SUM N= Oj;
sumyY = 0;
SUMY Ve Ol
Sumy = (OF
/* process the observations */
for a) = Obes SeNDatttares net
x = (*fx[i]) (DataMat(j, Xindex) );
y = (*fy[i]) (DataMat(j, Yindex));
sumt++;
SUMx eS Sc;
sumY += y;
sumxXX += SOQR(x);
sumYY += SQR(y);
SuMKY += x * y;.
}
/* calculate the results */
MeanX = sumX / sum;
MeanY = sumY / sum;
SdevX = sqrt((sumXX - SQR(sumX) / sum) /
(sum) 5
SdevY = sqrt((sumYY - SQR(sumY) / sum) /
(sum <= 1)));
(r+i)->Slope = (sumxXY - MeanX * MeanY * sum) /
SOR(SdevxX) / (sum - 1);
(r+i)->Intercept = MeanY - (r+i)->Slope * Meanx;
(r+1)->R2 = SOR((r+i)->Slope * SdevX / SdevyY);
(r+1)->fxIndex = i;
}
/* sort the results in descending order */
offset = numTrnsFx;
do {
GO£tset = KOfEset. a8) 7 ade
OfESet = (ortset ==) 0) irra offset;
inOrder = TRUE;
for (i = Os) te <. (munis Hee Ott Sein) yaar)
J] = = + offset:
/* swap elements? */
if ((r+i)->R2 < (r+j)->R2) {
inOrder = FALSE;
Eemp Sethe)
CU Gee caladh)) sae “Kets “ay))iG
Aro +) ol) emo.
}
:
} while (! (offset == 1 && inOrder == TRUE));
return TRUE;
}

Listing 10.5 implements the member functions of class LinReg and the function
BestFit. The implementation of these C++ member functions and function resemble
their C counterpart in file LINREG.C. The C++ code uses the member function Ma-
trix::operator( ) (declared in file ARRAYS.HPP) to access the elements of the data
matrices.
 Linear Regression 227

Listing 10.6 shows the source code for the TSLINREG.CPP test program. The pro-
gram performs the same task as its C counterpart. The main differences between the
two versions is that the C++ program uses instances of classes and sends C++ mes-
sages to these instances to perform most of the calculation.
To compile the C++ test program you need to include the files STATLIB.CPP, LIN-
REG.CPP, and TSLINREG.CPP in the project file.

Listing 10.6 The source code for the TSLINREG.CPP test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "linreg.hpp"
#include "global.h"

#define MAX FX 4

void pressAnyKey (void) ;

double fx(double) ;
double fy(double) ;
double ify(double) ;

double linear (double) ;

double 1n(double) ;

main()
{
Matrix mat(10, 5);
LinReg r;
double (*fxArr[MAX_FX]) (double) ;
double (*fyArr[MAX_FX]) (double) ;
double probability;
double slopeHi, slopeLow;
double intHi, intLow;
double testValue, calcT, tableT;
BestLinReg bestR2 [MAX_FX];
int numFx = MAX_FX;
Siqhoweely
int passTest;

mael(O, O)y SA
mene, 1) ee ANS iya
mate(l, Oi) t=. Ar
nuenen (ily Vail) Me alo) aeio
mat (2), 0) = 65
Dacia Lees. De
mattis, 0) = 8;
nan (3), Mab) able
mat(4, 0) = 10;
Mae (Ai ky) aL Bie
rqiteyeniS\y, 40) rans
ieteuen(S\/\ JI), (Sars
mat(6; 0) *= 14.
Ticvts (Gyms) = 57

PeinEivaluze(fx, Lyi) tay 0), hk, BALSE, (0))i;

r.sumLinReg(mat, 7);
r.DoLinReg() ;
ivi sreralte (AR A ere ce ee Linear Regression KKK KKK KKK) >

printf("Number of points: %lg\n", r.getSum());

DEInNGe("R*2 = slg\n" 7) LegetR2\())i5
printf£("Slope = %lg\n", r.getSlope());
printf("Intercept = %lg\n", r.getIntercept());
228 Chapter Ten

Listing 10.6 (Continued)

pobantyay yay (We AeA spa Itt tet 3 ANOVA ERERRAKA RK Vay p

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS) ;

printf("Regression df = %lg\n", r.ANOVA.Reg_df) ;
printf ("Residual SS = %lg\n", r.ANOVA.Residual_SS) ;
printf("Residual df = %$lg\n", r.ANOVA.Residual_df) ;
Prine
Le("Total (SSe= ssi Wie a sANOVA
Lo tall aos) i
peInte ("Total di =) silog\n") reANOVAL total df);
Dramas ("S.2° = slight), c. ANOVA S2)is
printf£("F = lg\n", r.ANOVA.LINREG_F) ;

pressAnyKey
();

probability = 95)
r.LinRegCoefCI (probability, slopeHi, slopeLow,
intHi, intLow);
printf("At %lg %% probability\n", probability) ;
printf("Range for slope is %lg to %lg\n", slopeLow, slopeHi);
printf("Range for intercept is %lg to %lg\n", intLow, intHi);

pressAnyKey() ;

testValue = -1.5;
r.LR_Slope_T_
test (probability, testValue,
calcT, tableT, passTest) ;
printf("At lg %% probability\n", probability) ;
printf£("HO: lg ?=? lg, ", r.getSlope(), testValue) ;
if (passTest)
printf("cannot be rejected\n") ;
else
printf("cannot be accepted\n");

pressAnyKey
() ;

testValue = 25.0;
r.LR_Int_T_test (probability, testValue,
calcT, tableT, passTest);
printf("At %lg %% probability\n", probability);
printf("HO: lg ?=? lg, ", r.getIntercept(), testValue) ;
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;

pressAnyKey
() ;

r.LR_R2_T_
Test (probability, tableT, calcT,
passTest) ;
printf("At lg %% probability\n", probability) ;
PHAIMEL (HOS Silo as not 10), ee get)
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;
pressAnyKey () ;

PLIANeL
("As AeA Aes Hee eS At omatae Beste chante he eee ee ek eee)
mat(0, 0) = 10;
MenteOl, eae) ors Or
mat(1, 0) = 25;
matitl 7.1) = 3.2189).
mat (2; 0) = 30
mat(2, 1) = 3.4012;
mat(3, <0) = 35;
Meats. = ao Dos
mat(4, 0) 100;
mat(4, 1) = 4.6052;
 Linear Regression 229

fxArr[0] = linear;
fyArr[0] = linear;
fxArr[1] = linear;
rang wena |(ili) 3 Abate
seideaia
74) =! allele
fyArr[2] = linear;
FxArr [3], = in
fyArr [3] = In;

BestFit(mat, 5, numFx, 0, 1, fxArr, fyArr,

for (a= OF < numer ee) et
printf("Function number %d\n", bestR2[i].fxIndex) ;
Preiner(“RO2T= ss gi") beste [usak2))ey
printf("Slope = %lg\n", bestR2[i].Slope) ;
printf("Intercept = %lg\n\n", bestR2[i].Intercept) ;
fee (citi) eee —— sO)
pressAnyKey
() ;
}
return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar () ;
puts ("\n\n");
}

double fx(double x)
{
return x;
}
double fy(double x)
{
return x;
}
double ify(double x)
{
return x;
}
double linear(double x)
{
return x;
}
double 1ln(double x)
{
return log(x);
}
muleeatien| ners J

SreeniT «°701>2ee4
rsee2lh me [Ol riAg*
oe ispenee = titre!
he. aA @ ALS SEaet
’ ' _ 7 eve a ‘ante 53] sme
arc eo * : ean Pest |
Ovi a Mya’ | WY sayaqetiag * 12) Rew)
B's Le VUES 2? PRM sale. i ~
ot ot pip* 7 or mee) £52 ed seen
ae teaine yee HA! 0 \eeone 6, vee
o . y oo Upeeew feed wae > 0? dt ort
a one "yhdareas “nib? wehwin
ol sue eS ecie
o> 9168
(2) tase
. “ople > Paviggiag
0 P Chi kaze *erets, © ‘ ee aor ja. * .¢

: ra ' a faa tskome

aru Ue iat ina Fae hanne
went eeee ee
fees
ee) n5 ne
[8 ° sine reas WAG VEG & seat, ow
= 69 au eum
; 7 7= J ¥
a OY
- -
oo 219 7) wtsihe <u Aneel a
4.
¥ 7
ye sa hb ee ee st-4lp Lf is
a3. _ >. FCA Oey! i ;
. Te _— 7
or tet oe ops =e}?

wart

=
Fug

'

oa 7 4 —

6 = ae
we 5
—_

x
 Chapter

11
Multiple and Polynomial Regression

This chapter looks at multiple and polynomial regression as extensions to linear re-
gression that I presented in chapter 10. The chapter presents the C and C++ source
code that implement the following aspects of multiple and polynomial regression:

# The coefficient of determination

# The regression intercept and slopes
= The standard error for the regression slopes
= The critical student-t values for the regression slopes
= The regression ANOVA table

Multiple Regression
Multiple regression correlates multiple independent variables with a single depen-
dent variable, as shown in the following equation:

NP SING Os ee (11.
where X, through X_ are the independent variables (or mathematical transformations
of such variables), Y is the dependent variable, A, is the intercept, and A, through A,
are the regression slopes for the variables X, through X,, respectively (Figure 11.1).
To obtain the intercept and slope for equation 11.1, a program needs to initialize sta-
tistical summations for the observed data. This process generates a set of simultaneous
linear equations that the program must solve to obtain the regression coefficients.
Here is the algorithm for obtaining the statistical summations:
Given: the matrix of observed data Mat, the array of indices Idx, the number of ob-
servations NData, and the number of independent variable N. The array of indices
Idx selects which columns of the matrix Mat participate in the regression. The ele-
231
232 Chapter Eleven

y=A+Bx+Cx*+Dx

Figure 11.1. A sample polynomial regression.

ment Idx[{0] selects the column that represents the variable Y. The element Idx[1] se-
lects the column that represents the variable X,, the element Idx[2] selects the col-
umn that represents the variable X,, and so on. Also given is the matrix of
summations A with rows and columns that have indices 0 to N. Finally, the solution
vector B with indices ranging from 0 to N is also given.

1 Create an array T with indices ranging from 0 to N.

2 Assign zeros to the elements of matrix A and vector B. Also assign zero to
sumYY (sum of Y squared).
3 For i= 0 to NData-— 1 repeat the following steps:
3.1 Forj =0to N-1 set T[j] = Mat@, Idx{j])
3.2 Set Yt = T[0]
3.3 Set T[0] =1
3.4 Add Yt to sumYY
3.5 For k = 0 to N repeat the following steps:
3.5.1 Add T[k] * Yt to B[k]
3.5.2 For j= 0 to N add T[j] * T[k] to A[j,k].
4 Remove dynamic array T.

Here is the algorithm that indicates how to obtain the regression results from the
statistical summation matrix A and the solution vector B.
Given: the summation matrix A, the solution vector B, and the number of inde-
pendent variables N.
 Multiple and Polynomial Regression 233

Set sum = A[0, O].

Set sumY = B[0].
For i= 1 to N repeat the following steps:
; Bl] — sumY * A(0, i]
ee At 0
sum
3.2 Forj = 1 toirepeat the next steps:
A[0, i] * A[O,j] es
seek SUOLAC, << == [rome |, 1|
sum
3.2.2 Copy Afj, i] into Aj, i].
sumY * sumY
Subirach<——_ ——fromisum ays
sum
sumY
Set sumY =
sum ~
For i= 1 to N repeat the following steps:
Diet Vst
6.1 Set A[0, i] =
sum
6.2 Set Mean{i] = A[O, i]
6.3 Set StdError[i] = Ali, i].
Form the correlation matrix by using a for loop with i = 1 to N that repeats the
following steps:
Afi, 0]
fT eset Ali, 0) =
(sumYY * StdError[i])

(ce viory =) tOd—liset

Ali.) =
Alj, i]
(StdError[i] * StdError[j])
7.3 Set Afi, i] = 1.
8 Complete the symmetric part of the matrix by using a for loop in which i = 2 to
N, which repeats the following steps:
8.1 Forj=1toi-—1, set Afi,j]= A[j, il.
Start solving the simultaneous equations by using a for loop with j = 1 to N:
9.1 Set Diag = Aj, j]
9:2. Set Alj;j] = 1
Aj, K]
9,0 For k= Ito N, set. Alf, k| =——
Diag
9.4 For k = 1 to N repeat the following statements if} does not equal k:
9.4.1 Set temp = A[k, j]
9.4.2 Set A[k, j] =0
9.4.3 Fori=1 to N subtract A[j, i] * temp from A[k, i].
10 Set R2 = 0.
it Set Intercept = 0.
12 For i= 1 to N repeat the following steps:
12.1 Set B[i] = 0
12.2 Forj = 1 to N add A{j, 0] * A[j, i] to Bhi]
234 Chapter Eleven

12.3 Add B[i] * A[i, 0] to R2

sumYY
12.4 Set B[i] = B[i] * (
StdError{i]
12.5 Add B[i] * A[0, i] to Intercept.
13 Set Intercept = sumY — Intercept.
14 Set DegF = sum—N-1.
sumYY
15 Set MS == ((0) Sey,
) * DesF

16 Fori=1 to N repeat the following steps:

16.1 Set StdErSlopefi] = (MS * AG, i)

StdError{[i]

16.2 Set TCalc[i] = SE

RS .
sum
17 Set R2adj = 1 - (1 — R2) *
DegF
18 Set Total_df =N + 1.
19 Set Total_SS = sumYY.
20 Set Reg_df=N.
21 Set S2 = MS.
22 Set Reg_SS = sumYY — Set S2.
23 Set Residual_df = DegF.
24 Set Residual_SS = S2 * DegF.
DegF , __Ra
25 pet =
N (i =R2),

The items calculated in steps 18 to 25 form the entries in the regression ANOVA
table. The item R2adj represents the adjusted value for the coefficient of correla-
tion. The adjustment takes into account the number of terms and number of ob-
servations. You can then compare adjusted value with similar ones obtained from
other multiple regressions with different number of terms and number of obser-
vations.

The C Source Code

Listing 11.1 shows the source code for the MLREG.H header file. This file declares two
structures, namely, MLR_ANOVAtag and MLRegtag. The structure MLR_ANOVAtag,
as the name might suggest, contains fields that support the ANOVA table for the mul-
tiple regression. The second structure, MLRegtag, contains fields that support the
multiple regression calculations. The structure contains the following groups of fields:

s The field numTerms stores the number of independent variables involved in the
multiple regression.
# The field Index is an IntVector structure that stores the indices for the dependent
variable (at index 0) and the independent variables (index 1 for X1, index 2 for X2,
and so on).
 Multiple and Polynomial Regression 235

# The field fx is an array of function pointers that specify the transformations ap-
plied to each variable. The element fx[0] is a pointer to the function that trans-
forms the dependent variable Y. The element fx[1] is a pointer to the function that
transforms the independent variable X1. The element fx[2] is a pointer to the func-
tion that transforms the independent variable X2, and so on.
= The field invfy is the pointer to the function that performs the inverse transforma-
tion on the dependent variable Y.
= The fields sum, sumY, and sumYY store the number of observations, sum of Y, and
sum of Y squared, respectively.
= The fields A and S are matrix-type structures that store the various statistical
summations of data.
® The fields TCalc, StdErSlope, B, and Mean are vector-type structures that store
the calculated student-t values for the regression slopes, the standard errors for
the regression slopes, the regression slopes, and the main values for the indepen-
dent variables, respectively.
= The fields hasMissingData and missingCode are the missing-code flag and missing-
code value, respectively.
# The field Inverted_S is a flag that determines whether or not the field S contains
an inverted matrix.
= The field Intercept stores the intercept of the multiple regression.
# The fields R2 and R2adj store the coefficient of correlation and its adjusted value,
respectively.
# The field ANOVA stores the ANOVA table for the multiple regression.

The header file contains typedef statements that create shorted aliases, MLR_
ANOVA and MLRegrec, for the above structures.
The header file declares the following functions:
1. The function InitializeMLR initializes an MLRegrec structure to prepare it for a
new set of calculations. The parameter r is the pointer to the initialized MLRegrec
structure. The parameter NumTerms specifies the number of independent vari-
ables. The parameter Fx passes the array of function pointers for the dependent
variable (at index 0) and the independent variables (at indices 1 and upward).
The parameter InvFy passes the pointer to the function that performs the inverse
transformation for the dependent variable. The parameter VarsIndex specifies
the indices that select the regression variables. The element at index 0 specifies
the index for the dependent variable. The element at index 1 specifies the index
for variable X1. The element at index 2 specifies the index for variable X2, and so
on. The parameter HasMissingData specifies the missing-code flag. The parame-
ter MissingCode indicates the numeric value of the missing code. If the argument
to parameter HasMissingData is 0, the argument for parameter MissingCode can
be any value. YOU MUST call function InitializeMLR at least once before you per-
form multiple regression analysis.
2. The function CalcSumMLR updates the statistical summations. The parameter
DataMat is a matrix-type structure that supplies the observed data. The arguments
236 Chapter Eleven

for parameter VarsIndex, in function InitializeMLR, specifies which columns of

parameter DataMat enter in the regression. The parameter r is the pointer to the
MLRegrec structure that stores the statistical summations. The parameter NData
indicates the number of observations to process. You can call function CalcSumMLR
more than once before calling the next function.
. The function CalcMultiReg calculates the regression slopes, intercept, ANOVA,
and other related data. The parameter r is the pointer to the structure MLRegrec
that holds the regression matrices and vectors. You must call function CalcMulti-
Reg before calling the remaining functions listed below.
. The function yHatCl calculates the projected value of the dependent variable,
and its confidence interval, for a given combination of independent variables. The
parameter r is the pointer to the MLRegrec structure that contains the results of
multiple regression. The parameter probability specifies the confidence level (ex-
pression as a percent or as a decimal). The parameter X specifies the array that
stores the independent variables. The element X[{1] stores the value for X1. The
element X[2] stores the value for X2, and so on. The parameters yHat, yHatLow,
and yHatHigh are pointers to double-type variables that pass the value for the
projected value and its confidence interval.
. The function MultiRegCoefClI calculates the confidence intervals for the regres-
sion slopes. The parameter r is the pointer to the MLRegrec structure that con-
tains the results of multiple regression. The parameter probability specifies the
confidence level (expressed either as a percent or as a decimal). The parameter
slopeHi specifies the array of upper limits for the regression slope. The element
slopeHi[1] corresponds to the slope for variable X1. The element slopeHi[2] cor-
responds to the slope for variable X2, and so on. The parameter slopeLow speci-
fies the array of lower limits for the regression slope. The element slopeLow{1]
corresponds to the slope for variable X1. The element slopeLow[2] corresponds to
the slope for variable X2, and so on.
. The function MLR_Slope_T_test tests the value of a specific regression slope. The
parameter r is the pointer to the MLRegrec structure that contains the results of
multiple regression. The parameter probability specifies the confidence level (ex-
pressed either as a percent or as a decimal). The parameter testValue specifies
the tested value of the targeted regression slope. The parameter termNum is the
index of the tested regression slope. The parameter calcT is a pointer to a double-
type variable that receives the calculated Student-t value. The parameter tableT
is a pointer to a double-type variable that receives the tabulated Student-t value.
The parameter passTest is the pointer to the in-type variable that reports
whether the outcome of the test is positive or negative.
. The function MLR_R2_T test tests whether or not the value of the correlation co-
efficient is not 0. The parameter r is the pointer to the MLRegrec structure that
contains the results of multiple regression. The parameter probability specifies
the confidence level (expressed either as a percent or as a decimal). The param-
eter calcT is a pointer to a double-type variable that receives the calculated Stu-
dent-t value. The parameter tableT is a pointer to a double-type variable that
 Multiple and Polynomial Regression 237

receives the tabulated Student-t value. The parameter passTest is the pointer to
the in-type variable that reports whether the outcome of the test is positive or
negative.
8. The function deleteMLRegrec deletes the dynamic data associated with an ML-
Regrec-type structure. You need to call this function at least once, after you are
done with the multiple regression analysis.
Listing 11.2 shows the source code for the MLREG.C implementation file. The
code in that file is based on the algorithms that I presented earlier. The code also in-
clude statements that manage missing data.

Listing 11.1 The source code for the MLREG.H header file.
#ifndef _MLREG
H_
#define _MLREG
H_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 7/1/94

C module that performs the following multiple linear

regression operations:

+ Basic multiple regression

+ Confidence interval of slopes
+ Student-t test for regression slopes
+ Student-t test for regression coefficient
y/

#include "arrays.h"

#define MLREG MAX TERM 20

#define MLREG BIG 1.0E+30

struct MLR_ANOVAtag {
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_SS;
double Total_df;
double Total_SS;
double S2;
double F;
3

typedef struct MLR_ANOVAtag MLR_ANOVA;

struct MLRegtag {
int numTerms;
IntVector Index;
double (*f£x[MLREG
MAX TERM+1]) (double) ;
double (*invfy) (double) ;
/* summation block */
double sum;
double sumyY;
double sumyYY;
Matrix A;
Matrix S;
Vector TCalc;
Vector StdErSlope;
Vector B;
Vector Mean;
Chapter Eleven

Listing 11.1 (Continued)

int hasMissingData;
double missingCode;
int Inverted_S;
double Intercept;
double R2;
double R2adj;
MLR_ANOVA ANOVA;
‘i
typedef struct MLRegtag MLRegrec;

void InitializeMLR(MLRegrec* r,
int NumTerms,
double (*Fx[MLREG_MAX_TERM+1]) (double),
double (*InvFy) (double),
IntVector XIndex,
iGae NAGMClErs,
int HasMissingData,
double MissingCode) ;

void CalcSumMLR (Matrix DataMat, MLRegrec* r, int NData);

void CalcMultiReg(MLRegrec* r);

void yHatCI (MLRegrec* r, double probability,

double* X,
double* yHatLow,
double* yHat,
double* yHatHigh) ;

void MultiRegCoefCI (MLRegrec* r, double probability,

double* slopeHi,
double* slopeLow) ;

void MLR_Slope_T_test (MLRegrec* r, double probability,

double testValue, int termNum,
double* calcT, double* tableT,
int* passTest) ;

void MLR_R2_T
Test (MLRegrec* r, double probability,
‘ double* calcT, double* tableT,
int* passTest) ;

void deleteMLRegrec (MLRegrec* r);

#fendif

Listing 11.2 The source code for the MLREG.C implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "mlreg.h"
#include "statlib.h"

void InitializeMLR(MLRegrec* r,
int NumTerms,
double (*Fx[MLREG_MAX_TERM+1]) (double),
double (*InvFy) (double),
IntVector XIndex,
int YIndex,
int HasMissingData,
double MissingCode)
/* initialize summations and range values */
 Multiple and Polynomial Regression 239

phahc; Fw 9 ale, tain

r->numTerms = (NumTerms > MLREG_MAX TERM) ?

MLREG_MAX TERM : NumTerms;
n = numTerms + 1;

newlntVect (&r->Index, n);

newMat (&r->A, n, n);
newMat (&r->S, n, n);
newVect (&r- =TCaley jn);
newVect (&r- >StdErSlope, n);
newVect (&r- Spy 1);
newVect (&r- >Mean, n);

for (i = 0; i <= r->numTerms; i++) {

for (j = 0; j <= r->numTerms; j++)
MAT (> Aya, eg))) =) Ole
VECK(x=>B, 1) "= 0);
}
for (1 = 0; 2 <= r->numTErms; i++) {
ae fe [ol] a axe [alice
VEC (r->Index, i+1) = VEC(XIndex, i);
}
VEC (r->Index, 0) = YIndex;
r->invfy = InvFy;
r->sumYY = 0;
r->sum = 0;
r->hasMissingData = HasMissingData;
r->missingCode = MissingCode;
}
void CalcSumMLR (Matrix DataMat, MLRegrec* r, int NData)
{
EP jig KP
int n = r->numTerms + 1;
Vector T;
double Yt;
Hm tn OKs
newVect (&T, n);

Bone (iy SOR ny xe Wypyeheewe SERS) oH

/* copy data to local variables */
/ <5 X* VEC (DO) = MAT (DataMaty, a VEC(z=—>Index, 0) ); */
for (j = 0; j <= r->numTerms; j++)
VEC(T, 3) = MAT(DataMat, i, VEC(r-Index, j));

// are there any possible missing data? */

if (r->hasMissingData) {
/* search for missing data */
OK = TRUE;
4) g= LO
while (j <= r->numTerms && OK)
if (VEC(T, j) > r->missingCode)
Saeioy
else {
OK = FALSE;
continue;
}
// found missing data?
Lite OK) eo
continue;
}

Le (OK) {
/* now transform the data */
/ tars transtomm TL) )/
Chapter Eleven

Listing 11.2 (Continued)

for (j = 0; j <= r->SnumTerms; j++)

VEC (2) ai) =) (A= Sect )RGEIG(, aye

Nae = wanel((ae), (0)

WAKER Oye =. Ibe
r->sum++;
r->sumYY += SOR(Yt);
for (k = 07 ki <= r-snumterme ta) a
VECita=>By. 6) = VEC kK) eave
for (j = 0; j <= r->numTerms; j++)
MAINES
= wy 30k) t= VEC a) + VinCiCi aus

}
}
deleteVect
(&T) ;
}

void CalcMultiReg(MLRegrec* r)
{
double Diag, tempo, DegF, MS;
lS laws jae Lee

r->sum = MAT (r=-SA, 0, 0);

r->sumY = VEC(r->B, 0);
/* Form the Si Matrix “*/
for (i = 1 ai <= s-Ssnumferms; 2++)) {
MAT (r-SA, i, 0) = VEC(r=-sB, 2) = r= >sumy *
MAT(r->A, 0, i) / r->sum;
for (gj = Ly a °<=] 27 ag+4)) x
MAT (R=SAy 3, SO) ==) MAT (6=SAn 10) 8) =
MAT(r->A, 0, 3) / r->sum;
/* Make a copy of the S matrix */
MAT (r=>S, 3) 2) = MAM(=SA, ay 2)

}
/* Clear S inversion flag */
r->Inverted_S = FALSE;

r->sumYY -= SQR(r->sumY) / r->sum;

r->sumY /= ¥Y->sum;
for (2 = 15 %& <= xn-SnumTerms? ach) 4
MAT (r=>A, 0, 2) /= x=>sum;
/* Make copies of the mean vector */
VEC (r->Mean, i) = MAT(r->A, 0, i);
VEC (r->StdErSlope, i) = MAT(r->A, i, i);
}

/* Form correlation matrix */

for (1 = 1) 1 <= r=Snumlerms; i1++) 4
MAT(r->A, i, 0) /= sqrt(r->sumYY * VEC(r->StdErSlope,
for (3) = hp og <7. gee)
MAT(r->A, j, i) /= sqrt (VEC(r->StdErSlope, i) *
VEC (r->StdErSlope, j));
MAT (S=—Say i, ay = Le
}

/* Complete symmetric part of the matrix */

for (i = 2: i <= r=-SnumTerms; i++) {
for (34= Ley SS wee ee)
MAT(r->A, i, J) = MAT(r=sA, 3, 1);
}

/* Start solving the simultaneous equations */

for (j = 1; j <= r->numTerms; jt+t) {
 Multiple and Polynomial Regression 241

Diag = MAT(r=sA, 4, 3);

MAT (1s =>A\, wales) elas

for (k = 1; k <= r->numTerms; k++)

MAT(r->A, j, k) /= Diag;
for (k = 1; k <= r->numTerms; k++)
eee (Sse)
tempo = MAT(r->A, k, j);
MAC (a Alri yl) ee 10)!
for (1 = 1; i <= r->numTerms; i++)
MAT(r->A, k, i) -= MAT(r->A, j, i) * tempo;
}
}
B-SR2 y= 20);
r->Intercept = 0;
for (i = 1; i <= r->numTerms; i++) {
VEC\(s=>Bi, 2)e= 05
for (j = 1; j <= r-<numTerms; j++)
VEC (r=>B, 1) += MAT(xr->A, 3, 0) * MAT(r=>A, 3, 1);
=> RO) VEC (a= Byes) MAM (ie SA 0)))5
VEC (xr->B, i) *= sqrt(r->sumYY / VEC(xr->StdErSlope, i));
r->Intercept += VEC(r->B, i) * MAT(r->A, 0, i);
}
r->Intercept = r->sumY - r->Intercept;
DegF = r->sum - r->numTerms - 1;
MS = (1 — r->R2) * r->sumYY / DegF;
for (i = 1; i <= r->numTerms; i++) {
VEC (x—->StdirSlope, 1) ==) sarki (MSP =sMAT (>A Pe 3) 7
VEC (r->StdErSlope, i));
VEC (r->TCalc, i) = VEC(r->B, i) / VEC(r->StdErSlope, i);
}
wae >R2adj = 1 - (1 - r->R2) * r->sum / DegF;
ae >ANOVA.Total_df = r->numTerms + 1;
= >ANOVA.Total_SS = r->sumYY;
— >ANOVA.Reg_df = r->numTerms;
Se >ANOVA.S2 = MS;
— >ANOVA.Reg_SS = r->sumYY - r->ANOVA.S2;
= >ANOVA.Residual_df = DegF;
ee >ANOVA.Residual_SS = r->ANOVA.S2 * DegF;
= >ANOVA.F = DegF / r->numTerms * r->R2 / (1 —- r->R2);
}
void yHatCI (MLRegrec* r, double probability,
double* X,
double* yHatLow,
double* yHat,
double* yHatHigh)

double diff, p, df, tableT, traceMat, pivot, tempo;

sligiey sane eile a oie
Matrix XX;
Matrix prodMat;
int n = r->numTerms + 1;

newMat (&XX, n, n);

newMat (&prodMat, n, n);

ie (probability > 1)
p = 0.5 - probability / 200;
else
De= 01.5 — probabilaty / 2);
df = r->sum -— (r->numTerms + 1);
tablent = "Tinv(p, Gt) 7

if (!r->Inverted_S) { /* Invert matrix S */

r->Inverted_S = TRUE;
242 Chapter Eleven

Listing 11.2 (Continued)

for (g = 1; j <= r->numTerms; j++) {

pivot = MAT (RSS ai ook
MAT (eS) ane eels
for (k = 1; k <= r->numlerms; k++)
MAT (c=>S) ges) f= pivot:

for (k = 1; k <= r->numTerms; k++)

chee t(d => afl yh at
tempo = MAT(r->S, k, j);
MAT (m=Ss, k, gi)! = 105
for (m = 1; m <= r->numTerms; m++)
MAT (r->S, k, m) -= MAT(r->S, j, m) * tempo;

}
/* Calculate yHat */
*yHat = r->Intercept;
for (i = 1; i <= r->numTerms; i++)
eV Hates WECGg eS ieel)e Exo) dally

/* Form standardized vector */

for (2. = 0; » <= 2-Snumlermss a+)
X[i] -= VEC(r->Mean, i);

fx, Form) S64 maierasc o7/)

for (k = 1; k <= xr-SnumTerms; k++) {
for (j = 1; j <= r->numTerms; j++)
MAT NK, Ske) -h=— til ee ale
}

/* Multiply S_Inverse and XX' */

for (i = 1; i <= r->numTerms; i++) {
for (7 = Ly 3) <= s=SnunTermes Fite
MAT (prodMat, i, j) = 0;
for (k = 1; k <= r->numTerms; k++)
MAT (prodMat, 11, 3) t=) MAT (s—SSi aa, Jc) *
MAT (XX, k, Jj);

}
/* Calculate trace of prodMat */
traceMat = iL.-
for (i = 1; i <= r->numTerms; i++)
traceMat *= MAT(prodMat, i, i);

aiff = tableT * sqrt(r->ANOVA.S2 * (1 + 1/r->sum + traceMat)

);
*yHatLow = *yHat - diff;
*yHatHigh = *yHat + diff;

*yHat = (*r->invfy) (*yHat) ;

*yHatLow = (*r->invfy) (*yHatLow) ;
*yHatHigh = (*r->invfy) (*yHatHigh) ;

deleteMat (&XX) ;
deleteMat (&prodMat) ;

}
void MultiRegCoefCI (MLRegrec* r, double probability,
double* slopeHi,
double* slopeLow)
{
double diff, p, df, tableT;
inte
 Multiple and Polynomial Regression 243

Zf (probability > 41)

p = 0.5 - probability / 200;
else
p = 0.5 = probability / 2;
df = r->sum - (r->numTerms + 1);
tabler =" Tinv (p;, dé) >
for (j = 1; j <= r->numTerms; j++) {
diff = tableT * VEC(r->StdErSlope, j);
slopeHi[j] = VEC(r->B, j) + diff;
slopeLow[j] = VEC(r->B, j) - diff;
}
}
void MLR_Slope_T_test(MLRegrec* r, double probability,
double testValue, int termNum,
double* calcT, double* tableT,
int* passTest)
{
double p, df;

Re (probabilttiy >) i)
p= 0.5 - probability / 200;
else
p= 055>— probabiiaty / 2
agf = r->sum - (r->numTerms + ib)B
*tableT = TiInv(p, df);
*calcT = (VEC(r->B, termNum) —- testValue) /
VEC (r->StdErSlope, termNum) ;
*passTest = (fabs(*calcT) <= *tableT) ? TRUE FALSE;
}

void MLR_R2_T_
Test (MLRegrec* r, double probability,
double* calcT, double* tableT,
int* passTest)
/* Procedure to test hypothesis HO Eto ay
{
double p, df;

LE (probabilzty > 1)
DE 0'.5) — probabmlaty./ 2010;
else
pu 055) =) probabsa lity 2;
df = r->sum - (r->numTerms + Ly;
*tabvleD = Mn (oy, dt) +
TO e— Shineha | Ae Gie 7/\( eT as > 2)!)
*passTest = (*caleT <= *tableT ) ? FALSE TRUE;
}
void deleteMLRegrec(MLRegrec* r)
{
deleteiIntVect (&r->Index) ;
deleteMat (&r->A) ;
deleteMat (&r->S) ;
deleteVect (&r->TCalc);
deleteVect (&r->StdErSlope) ;
deleteVect (&r->B) ;
deleteVect (&r->Mean) ;

The C++ Source Code

Now let’s look at the C++ code for the multiple regression library. Listing 11.3 shows
the source code for the MLREG.HPP header file. The file declares the classes MLR_
244 Chapter Eleven

ANOVA and MLReg. The class MLR_ANOVA contains public data members that rep-
resent the components of an ANOVA table for the multiple regression. The class ML-
Reg contains public data members and member functions that support the multiple
regression. The member functions in class MLReg are very similar to the C functions
in file MLREG.H. Likewise, the data members in the class are similar to their coun-
terparts in the C structure MLRegtag, located in file MLREG.H. There are some mi-
nor differences in the declarations of some member functions and their counterpart
C functions. The member functions yHatCI, MLR_Slope_T_test, and MLR_R2_T_test
use reference parameters instead of pointer-type parameters to pass information to
the caller. Also, the member function CalcSumMLR declares a reference to the ma-
trix-type parameter instead of declaring that parameter to pass a copy of its data.

Listing 11.3 The source code for the MLREG.HPP header file.
#ifndef _MLREG HPP_
#define _MLREG HPP _

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 7/1/94

C++ module which performs the following multiple linear

regression operations:

Basic multiple regression

Confidence interval of slopes
Student-t test for regression slopes
+4 Student-t test for regression
+++ coefficient
7.

#include "arrays.hpp"

#define MLREG
MAX TERM 20
#define MLREG BIG 1.0E+30

class MLR_ANOVA
{
public:
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_SS;
double Total_df;
double Total_SS;
double S2;
double F;
ae

class MLReg
{
public:

void InitializeMLR(int NumTerms,

double (*Fx[MLREG_MAX _TERM+1]) (double),
double (*InvFy) (double),
IntVector& VarsIndex,
int HasMissingData,
double MissingCode) ;

void CalcSumMLR(Matrix& DataMat, int NData) ;

 Multiple and Polynomial Regression 245

void CalcMultiReg() ;
void yHatCI (double probability,
double* X,
double& yHatLow,
double& yHat,
double& yHatHigh) ;
void MultiRegCoefCI (double probability,
double* slopeHi,
double* slopeLow) ;
void MLR_Slope_T_test (double probability,
double testValue, int termNum,
double& calcT, double& tableT,
int& passTest) ;
void MLR_R2_T_Test (double probability,
double& calcT, double& tableT,
int& passTest);

public:
int numTerms;
double sum;
Vector TCalc;
Vector StdErSlope;
Vector B;
Vector Mean;
double Intercept;
double R2;
double R2adj;
MLR_ANOVA ANOVA;

protected:
IntVector Index;
double (*fx[MLREG_MAX
TERM+1]) (double) ;
double (*invfy) (double) ;
/* summation block */
double sumyY;
double sumyYY;
Matrix A;
Matrix S;
int hasMissingData;
double missingCode;
int Inverted_S;
ihe

#endif

Listing 11.4 contains the source code for the MLREG.CPP implementation file.
The file defines the member functions of class MLReg. The implementation of these
member functions are similar to their counterparts C functions in file MLREG.C. You
will notice that the statements are generally shorter since C++ class provides auto-
matic access to their data members.

Listing 11.4 The source code for the MLREG.CPP implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include_ "mlreg.hpp"
#include "statlib.h"

void MLReg: :InitializeMLR (

int NumTerms,
double (*Fx[MLREG_
MAX TERM+1]) (double),
246 Chapter Eleven

Listing 11.4 (Continued)

double (*InvFy) (double),

IntVector& VarsIndex,
int HasMissingData,
double MissingCode)

aiding 1) MatP

numTerms = (NumTerms > MLREG_

MAX TERM) ?
MLREG_MAX TERM : NumTerms;
int n = numTerms + 1;

Index.ReSize(n) ;
A.ReSize(n, n);
S.ReSize(n, n);
TCalc.ReSize(n);
StdErSlope.ReSize(n) ;
B.ReSize(n);
Mean.ReSize(n) ;

/* initialize summations */
for (1 = 0; 2 <= numTerms; i++) {
for (3 = 07 9 <= numTerms') 374+)
ANG asl) en Ole
Bl Lae -0;
eee [(Gh | = sabe (8 i)3
Index[i] = VarsIndex[i];
}
invfy = InvFy;
sumvy = 0;
hasMissingData = HasMissingData;
missingCode = MissingCode;
}
void MLReg::CalcSumMLR(Matrix& DataMat, int NData)
{
LANES: eS by)pa 2
int n = numTerms + 1;
Vector TTrnsf(n);
Vector Torig(n);
double Yt;
int OK;

for (1 = OF; 2 < NDatas a++) {

/* copy data to local array Torig */
for (j ='0; 3 <= numTerms; j++) {
k = Index[j];
Torig[j] = DataMat(i, k);
}

/* set ok Elag */
OK = TRUE;

/* are there any possible missing data? */

if (hasMissingData) {
/* search for missing data */
OF
while (j <= numTerms && OK)
if (Torig[j] > missingCode)
chest
else
OK = FALSE;

LE (OK) f
 Multiple and Polynomial Regression 247

/* now transform the data */

j= First transtorm Torag|] anto, TTrms£p] +7
for (j = 0; j <= numTerms; j++)
Tirnseigi) =" (ALR) (lorig ial)
Ve =e Ttrnset 10>
MUTANStelROM) ele Ole
sumYY += SQR(Yt);
for (k = 0; k <= numTerms; k++) {
Bik) = Jimskey Ss We
for (j = 0; j <= numTerms; j++)
IN|, ie)) Fas Uisnerakspel
aitl) a Wilanrarove
he) [teil

}
void MLReg: :CalcMultiReg()
{
double Diag, tempo, DegF, MS;
iialcn inyea tailor

sum = A(0, 0);

sumY = B[0];
/* Form the A and S Matrices */
for (i = 1; i <= numTerms; i++) {
NG 0)\) = Bla — sumy *.A0) 2) 9/7 ‘sume
fon (yj = 1503 <= ett)
Gy 2) == AO ty) AION, 3) / sum;
/* Make a copy of the matrix A */
Si, 2) = AG ae

}
/* Clear the matrix S inversion flag */
Inverted_S = FALSE;

sumYY -= SQR(sumY) / sum;

sumY /= sum;
for (2 = 1. 2 <= numrerms i++) ¢{
A(0O, i) /= sum;
/* Make copies of the mean and std error vectors */
Mean[i] = A(0, i);
StdErSlopela|| = A(z, 2)ir
}

/* Form correlation matrix */

for (2, = ls 2 <= numTerms)? i++) {
Ai, OO) /= sqre (SuNYY *sStdEnSlopell
1]))) ;
FOr (Healey yee st)
A(j, i) /= sqrt(StdErSlope[i] * StdErSlope[j]) ;
Nal, aly eke 1

If

/* Complete symmetric part of the matrix */

for (1 = 2; i <= numTerms; i++)
fom (7 = 1; 3) <a)
Aa) =H Aa 2h

/* Start solving the simultaneous equations */

hom (a = Ls 3 <="numtermns; 72>)
Diag = Aye au
Ao 3) (Sry

for (k = ie.k <= numTerms; k++)

BiG, ae =F Daag;
248 Chapter Eleven

- Listing 11.4 (Continued)

for (k = 1; k <= numTerms; k++) {

2E (9) Y= kK)
tempo = A(k, j);
A(k, j) = 0;
for (1 = 1; i <= numTerms; i++)
A(k, 1) == A(j, i) * tempo;
}
}
}
R2=s0
Intercept = 0;
for (i0= 1792 <= numferms: i++) {
BL) 7= 0;
for (j = 1; j <= numTerms; j++)
BEL PAai. 0) Aa ie
R29+= 8B fa eer (is 0) >
B[i] *= sqrt(sumyy / StdErSlope[i]);
Intercept += B[i] * A(0O, i);
}
Intercept = sumY — Intercept;
DegF = sum —- numTerms - 1;
MS = (1 - R2) * sumYY / DegF;
for (a °=) 0 a <=" numtermsis ++)
StdErSlope[i] = sqrt(MS * A(i, i) / StdErSlope[i]);
Teale[il = Bli} / StdbeSlope (it;
}
R2adj = 1 - (1 — R2) * sum / DegF;
ANOVA. Total_df I numTerms + 1;
ANOVA.Total_SS = sumYY;
ANOVA.Reg_df = numTerms;
ANOVA.S2 = MS;
ANOVA.Reg_SS = sumYY - ANOVA.S2;
ANOVA.Residual_df = DegF;
ANOVA.Residual_SS = ANOVA.S2 * DegF;
ANOVA.F = DegF / numTerms * R2 / (1 - R2);
}
void MLReg: :yHatCI (double probability,
double* X,
double& yHatLow,
double& yHat,
double& yHatHigh)

double diff, p, df, tableT, traceMat, pivot, tempo;

Bosh a, ie Sa aie
int n = numTerms + 1;
Matrix XX(n, n);
Matrix prodMat(n, n);
Vector Xcopy(n) ;

if (probability > 1)
p= 0.5 =) probabiiaitys/2200F
else
p = 0.5 —- probability / 2;
df = sum - (numTerms + 1);
tableT = TInv(p, dé£);

if (!Inverted_S) { /* Invert matrix S */

Inverted_S = TRUE;
for (j = 1; j <= numTerms; jt+) {
pivot = Siz, 3)
Wr
i

For ak =. 2 k <= numTerms; k++)

S(j7 &) Se ky) ¥ pivot;
 Multiple and Polynomial Regression 249

for (k = 1; k <= numTerms; k++)

eine (ce ee fe years
tempo = S(k, j);
S(k,, 3))) = 0;
for (m = 1; m <= numTerms; m++)
S(k, m) =="S(j, m) * tempo;
}

}
/* Calculate yHat */
yHat = Intercept;
jsope (ab Fs aly SU Ee albinafehayey alee)
Xcopy[i] = (*fx[i]) (X[il]l);
yHat += B[i] * Xcopy[il];
}

/* Form standardized vector */

for (i = 1; i <= numTerms; i++)
Xcopy[i-1] -= Mean[i];

{> Form, XX" matrix */

for (k = 1; k <= numTerms; k++) {
fore (j= 1-3 <= numlerms 7-4)
XX(j, k) = 0;
}

for (k = 1; k <= numTerms; k++) {

for (j = 1; j <= numTerms; j++)
XX(j, k) = XX(j, k) + Xcopy[j-1] * Xcopy[k-1];
}

/* Multiply S_Inverse and XX' */

fom (ue = 1s) 2 <= numbers; i++) &€
for (qj = 1) a0 <= numterms + j++)) 4
prodMat(i, j) = 0;
for (k = 1; k <= numTerms; k++)
jenaeyelirehe
(sy, ap)! sree SE, s)) SY 2:0-cUSe) Sh)

}
/* Calculate trace of prodMat */
traceMat = 1.;
Om (ie = doe e<= num brerms:, at)
traceMat *= prodMat(i, i);

diff = tableT * sqrt(ANOVA.S2 * (1 + 1/sum + traceMat));

yHatLow = yHat - diff;
yHatHigh = yHat + diff;

yHat = (*invfy) (yHat) ;

yHatLow = (*invfy) (yHatLow) ;
yHatHigh = (*invfy) (yHatHigh) ;
}
void MLReg: :MultiRegCoefCI
(double probability,
double* slopeHi,
double* slopeLow)

double diff, p, df, tableT;

inteae

if (probability > 1)
p = 0.5 - probability / 200;
else
D = 0).59 — probability 1. 2)°
df = sum —- (numTerms + 1);
tablet = Tinv(p, dt):
250 Chapter Eleven

Listing 11.4 (Continued)

Lom, (cs 1; j <= numTerms; j++) {

diff tableT * StdErSlope[j];
slopeHi[j] = B[j] + diff;
slopeLow[j] = B[j] - diff;
}
}
void MLReg: :MLR_Slope_T_test(double probability,
double testValue, int termNum,
double& calcT, double& tableT,
int& passTest)

double p, df;

if (probability > 1)
p = 0.5 - probability / 200;
else
Dp = 0.5e— probability. 2:
df = sum — (numTerms + 1);
tableT = Tinv(p, df);
calcT = (B[termNum] - testValue) /
StdErSlope[termNum] ;
passTest = (fabs(calcT) <= tableT) ? TRUE : FALSE;
}

void MLReg::MLR_R2_T_Test(double probability,

double& calcT, double& tableT,
int& passTest)
/* test hypothesis HO)» R°2 = 0 */
{
double p, df;

DE (probability > 1)
D = 0.5 — probability 7) 200%
else
p = 0.5 = probability / 2;
af = sum —- (numTerms + 1);
tableT = TInv(p, df);
GalieT = sqri(R2 stsdte/\(1ei— RZ)
passTest = (calcT <= tableT ) ? FALSE : TRUE;

The C Test Program

Let’s look at the test program for the C version of the multiple regression library.
Listing 11.5 shows the source code for the TSMLREG.C test program. This program
performs the following tasks:

1. Declares a variety of variables to conduct the diverse calculations.

Initializes the array of function pointers, fx, using the C function linear. This is a
default initialization that is purely optional.
3. Allocates the dynamic space for the matrix mat and the array of indices Index.
4. Assigns the test data to the elements of matrix mat.
5. Assigns the values to the variables that store the number of data and number of
terms (that is, the number of independent variables).
 Multiple and Polynomial Regression 251

6. Assigns the transformation functions to the first three elements of array fx.
7. Assigns the indices for the regression variables to the elements of array Index.
8. Initializes the MLRegrec structure r by calling the function InitializeMLR. The ar-
guments for the function call are the address of structure r, the variable numTerms,
the array fx, the pointer to function linear (which is the name of the function), the
array Index, the value FALSE, and the value 0. The latter two arguments indicate
that the data matrix will have no missing observations.
. Updates the statistical summations by calling function CalcSumMLR. The argu-
ments for this function call are the matrix mat, the address of structure r, and
the variable numData.
10. Calculates the regression slopes and other results by calling the function Calc-
MultiReg. The argument for this function call is the address of structure r.
it. Displays the regression results that include the coefficient of correlation, the in-
tercept, the regression slopes, and the ANOVA table components.
12. Calculates and displays the confidence intervals, at 95% confidence, for the re-
gression slopes. This task involves calling function MultiRegCoefCI. The pro-
gram uses the arrays slopeHi and slopeLow to obtain the sought confidence
intervals.
13. Tests the values for the two regression slopes at 95% confidence. The program
calls function MLR_Slope_T_test to determine if the first and second slopes are
not that significantly different from the values of 0.5 and 0.01, respectively. The
program displays the outcome of the test.
14, Tests whether or not the value of the correlation coefficient is 0 (at 95% confi-
dence). The program calls function MLR_R2_T_test and displays the outcome of
the test.
15. Calculates the projected value of the dependent variable and its confidence in-
terval (at 95% confidence) for the first row of data. The program calls function
yHatCI and passes the arguments &r, probability, Xarr, &yHatLow, &yHat, and
&yHatHigh. The program then displays the value of the dependent variables, the
projected value, and the confidence interval for the projection.
16. Deletes the dynamic data of the matrix mat, vector Index, and structure r.

Figure 11.2 shows the output of the test program. The project file for the test pro-
gram should include the files STATLIB.C, ARRAYS.C, MLREG.C, and TSMLREG.C.

Listing 11.5 The source code for the TSMLREG.C test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "mlreg.h"
#include "global.h"
#include "arrays.h"
252 Chapter Eleven

Listing 11.5 (Continued)

#define MAX FX 3

void pressAnyKey (void) ;

double linear(double) ;

main ()
{
Matrix mat;
MLRegrec r;
IntVector Index;
double (*fx[MLREG_MAX_TERM+1])
(double) ;
double probability;
double testValue, calcT, tableT;
int numFx = MAX FX;
int i, numTerms, numData;
int passTest;
/* initialize array fx with default values */
for (i = 0; i <= MLREG
MAX TERM; i++)
fx[i] = linear;

newMat (&mat, 10, 10);

newIntVect (&Index, 10);
/*

MAT (mat (0), 10) "=" 162%

MAT (matt, LO) =" 20%
MAT (mat) 240010) 5= 223%,
MAT (mate S700) eeu:
MAT (mat, 427.0)" = 6:72
MAT(mat, 5, 0) = 169;
MAT\(maty (6, O)® = Sie
MAT(mat; 7, 0) = 192;
MAT(mat, 8, 0) = 116;
MAT (mat, 9, 0) = 55;
MA Gnas, m0 0) =o 5 ie
MAT (mat, 11,0)" ="232"
MAT(mat, 12, 0) = 144;
MAT (mat, 13, 0) ="103;
MAT(mat, 14, 0) = 212;

MAR inat, (0, y= 274

MAT (mat, 1, 1) = 180;
MAT (masts 02), 0h ) ye=eS Tbr
MAT (mat, 3, Ly =- 205
MAT(mat, 4, 1) = 86;
MAT (matt,) 5. 1) e526 5i5
MAT(mat, 6, 1) = 98;
MAT Gna Gear 1)s 330i
MAT(mat, 8, 1) = 195;
MAT (mat; 971)" 53'5
MAT(mat, 10, 1) = 430;
MAT (mat, LL, 1) = 372;
MAT (mat, 12, 1) = 236;
MAT (mat, 137° 1)" = 157
MAT (mat, 14, 1) = 370

MAT(mat, 0, 2) = 2450;
MAT (mat, 1, 2) = 3254;
MAT (mat, 2, 2) = 3802:
MAT(mat, 3, 2) = 2838;
MAT(mat, 4, 2) = 2347;
MAT (mat, 5, 2) = 3782"
MAT(mat, 6, 2) = 3008;
MAT (mat; 7, 2) = 2450+
MAT (IAt; =o, 2) = cise.
 Multiple and Polynomial Regression 253

MAT (matt, 9, 2) = 2560;

MAT (mat, 10, 2) = 4020;
MAT(mat, 11, 2) = 4427;
MAT(mat, 12, 2) = 2660;
MAT(mat, 13, 2) = 2088;
MAT(mat, 14, 2) = 2605;
ip
MAT (mat, 0, 20)—= 3:2)
MAT(mat eel 0) =) 14.8
MAT (mat 02. 0) = 7.2;
MAT (mate 130 10) = 38 55)
MATT (Mel tye 0) =) ads

MAT (mat, 0.8 ek lek

MiNemGgeve, ii. al} PRE
MAT(mat, 2, 1) ele
MAT(mat, 3, 1) 4;
Wu NaaNbeens -, 7 My) eS ey6

MAT (Matis OF. Ziv diy

MAT(mat, 1, 2) 25
MAN mat, 276 2). =e3+
MAT(mat, 3, 2) Ae
MAT(mat, 4, 2) = 5;

numData = 5;
numTerms = 2;

£x[0] = linear;
£x[i] = linear;
fx[2] = linear;
fx (3) = Lineany

VEC (Index, 0) = 1;
VEC (index, 1) 7= 2)
InitializeMLR(&r, numTerms, fx, linear, Index, 0, 0, 0);
CalcSumMLR(mat, &r, numData) ;
CalcMultiReg(&r) ;
PLAnNtE ("AAA K KKK KK Linear Regression KAKKKKKEKE\ DM) »

printf("Number of points: %lg\n", r.sum);

PEInte (Ree —s shone ea wRie
printf ("Adjusted R*2 = %lg\n", r.R2adj);
printf("Intercept = %lg\n", r.Intercept) ;
for (i = 1; i <= numTerms; i++)
Drier (SlOpemshOr d= slg \mi ea) VC (a, Bye Lyi)
PLAInNtE ("FARRAR RKKKKS ANOVA REKKKRKKEKE\ TH") 2

/*

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS) ;

printf("Regression df = %lg\n", r.ANOVA.Reg_df);
printf("Residual SS = %lg\n", r.ANOVA.Residual_SS) ;
printf("Residual df = %lg\n", xr.ANOVA.Residual_df) ;
printf("Total SS = %lg\n", r.ANOVA.Total_SS) ;
printf("Total df = %lg\n", r.ANOVA.Total_df) ;
printf ("S*2 = %lg\n", r.ANOVA.S2);
printf£("F = %lg\n", xr.ANOVA.LINREG_F) ;
pressAnyKey
() ;

probability = 95;
LinRegCoefCI(&r, probability, &slopeHi, &slopeLow,
&intHi, &intLow);
printf("At lg %% probability\n", probability) ;
printf("Range for slope of %lg to %lg\n", slopeLow, slopeHi) ;
printf("Range for intercept is %lg to %lg\n", intLow, intHi);

pressAnyKey
() ;

testValue = -1.5;
254 Chapter Eleven

Listing 11.5 (Continwed)

LR_Slope_T_test(&r, probability, testValue

&calcT, &tableT, &passTest) ;
printf("At %lg %% probability\n", probability) ;
printf("HO: lg ?=? lg, ", r.Slope, testValue);
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;

pressAnyKey
();
testValue = 25.0;
LR_Int_T_test(&r, probability, testValue,
&calcT, &tableT, &passTest);
printf("At %lg %% probability\n", probability) ;
print£("HO: lg ?=? lg, ", r.Intercept, testValue) ;
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;

pressAnyKey
();
LR_R2_T Test(&r,
probability, &calcT, &tableT,
&passTest) ;
printf("At lg %% probability\n", probability) ;
princi (HOS tig 2s not Oy ae eR)
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n");
a
pressAnyKey
() ;
deleteMat (&mat) ;
deleteIntVect (&Index) ;
deleteMLRegrec(&r);
return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
Pues
\waA is
}
double linear(double x)
{
return x;
}

The C++ Test Program

Let’s now look at the test program that tests the C++ version of the multiple regres-
sion library. Listing 11.6 shows the source code for the TSMLREG.CPP test program.
This program performs the following tasks:

1. Declares a variety of variables to conduct the diverse calculations. The declara-

tion includes the object r, which is an instance of class MLReg.
iw). Initializes the array of function pointers, fx, using the function linear.
we). Reallocates the dynamic space for the matrix mat and the array of indices Index.
 Multiple and Polynomial Regression 255

KEKKKKKKKK KKK Multiple Regression KaKKK

KK KEK

Number of points: 15
R“2 = 0.9989
Adjusted R*2 = 0.9987
Intercept = 3.45261
Slope for X1 = 0.496005
Slope for X2 = 0.00919908
KKKKKK KK KKKAKK ANOVA KKKKKKKKKEK

Regression SS = 53896.9
Regression df = 2
Residual SS = 56.8836
Residual df = 12
Total SS = 53901.6
Totaliak = 3
Saa—ee4 OS
Ee 5167.9 aid)

Press any key to continue...

At 95 % probability
Range for slope of X1 is 0.48281 to 0.5092
Range for slope of X2 is 0.0070892 to 0.011309

Press any key to continue...

At 95 % probability
HO: 0.496005 ?=? 0.5, cannot be rejected
At 95 % probability
HO: 0.00919908 ?=? 0.01, cannot be rejected

Press any key to continue...

At 95 % probability
HO: 0.998945 is not 0, cannot be rejected

Press any key to continue...

At 95 % probability given:
6) 274
X[2] 2450
¥* = 161.896
Range for Y* is 156.994 to 166.797

Press any key to continue...

Figure 11.2 The output of the program TSMLREG.EXE.

This task involves sending the C++ message ReSize to each of the matrix mat and
the array Index.
4. Assigns the test data to the elements of matrix mat.
5. Assigns the values to the variables that store the number of data and number of
terms (that is, the number of independent variables).
6. Assigns the transformation functions to the first three elements of array fx.
7. Assigns the indices for the regression variables to the elements of array Index.
8. Initializes the object r by sending it the message InitializeMLR. The arguments for
this message are the variable numTerms, the array fx, the pointer to function lin-
ear (which is the name of the function), the array Index, the value FALSE, and
256 Chapter Eleven

the value 0. The latter two arguments indicate that the data matrix will have no
missing observations.
. Updates the statistical summations by sending the C++ message CalcSumMLR
to the object r. The arguments for this message are the matrix mat and the vari-
able numData.
10. Calculates the regression slopes and other results by sending the C++ message
CalcMultiReg to the object r.
ie Displays the regression results that include the coefficient of correlation, the in-
tercept, the regression slopes, and the ANOVA table components.
12. Calculates and displays the confidence intervals, at 95% confidence, for the re-
gression slopes. This task involves sending the C++ message MultiRegCoefCI to
the object r. The program uses the arrays slopeHi and slopeLow to obtain the
sought confidence intervals.
13. Tests the values for the two regression slopes at 95% confidence. The program
sends the C++ message MLR_Slope_T_test to object r in order to determine if
the first and second slopes are not that significantly different from the values of
0.5 and 0.01, respectively. The program displays the outcome of the test.
14. Tests whether or not the value of the correlation coefficient is 0 (at 95% confi-
dence). The program sends the C++ message MLR_R2_T_test to object r and
then displays the outcome of the test.
15. Calculates the projected value of the dependent variable and its confidence in-
terval (at 95% confidence) for the first row of data. The program sends the C++
message yHatClI to object r and then passes the argument’s probability, Xarr,
yHatLow, yHat, and yHatHigh. The program then displays the value of the depen-
dent variables, the projected value, and the confidence interval for the projection.

The project file for the test program should include the files STATLIB.CPP, ML-
REG.CPP, and TSMLREG.CPP.

Listing 11.6 The source code for the TSMLREG.CPP test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "mlreg.hpp"
#include "global.h"
#include "arrays.hpp"

void pressAnyKey (void) ;

double linear(double) ;

main ()
{
Matrix mat;
MLReg r;
IntVector Index;
double (*fx[MLREG_MAX TERM+1]) (double) ;
double probability;
double testValue, calcT, tableT;
 Multiple and Polynomial Regression 257

double slopeHi[10], slopeLow[10], Xarr[10];

double yHat, yHatLow, yHatHigh;
int i, numTerms, numData;
int passTest;

/* initialize array fx with default values */

for (i = 0; i <= MLREG
MAX TERM; i++)
fx[i] = linear;

mat.ReSize(15, 3);
Index .ReSize(3) ;

mat(0, 0) = 162;
mate (ae 0) me 207
Melts(27 m0) ie aS
Maes) 20) =o dhs
mat(4, 0) = 67;
mat(5, 0) = 169;
mat(6, 0) = 81;
mati(7, 10), = 192’
mat(8, 0) = 116;
mat(9, 0) = 55;
mat(10, 0) = 252;
matte (dda O))) = 25.217
mat(12, 0) = 144;
Malte (2S 0)—s LOSI
Metta (lO)! = 22)

mat(0, 1) = 274;
Meare (lo ot) = E80}
iieeN(Zi, Ib ey ehy/eye
ameyel(Sie al) ee BOlsyr
mat(4, 1) = 86;
mae(5, 1)" = 265i;
Ma (Gis) =a B's
mate, 1), = 330)
mats, 1)) = 195;
mat (95. 1) = 53;
mat(10, 1) = 430;
iere(Gll, wh) te ehyeic
mat(12, 1) = 236%
mats (13)! = Sis
mat(i4, 1) = 370);

mats(0), 2) = 2450);
Mati, 2) = 3254.
mMae(2,, 2) = 3802;
mat(3, 2) = 2838;
mat(4, 2) = 2347;
mat (5,02) =a Sinko eve
mat(6, 2) = 3008;
mat(7, 2) = 2450;
Mae(S a a eses LS:
mat(9, 2)) = 2560;
mat(10, 2) = 4020;
mat(ii, 2) = 44277,
mat(12, 2) = 2660;
mat(13, 2) = 2088;
Mat (14,42) "=" 2605;

numData = 15;
numTerms = 2;

£x[0] linear;
fx[1] linear;
fx(2]|-= Linear;
258 Chapter Eleven

Listing 11.6 (Continued)

Index[0] = 0;
index[1] = i;
Index[2] = 2;

r.InitializeMLR(numTerms, fx, linear, Index, FALSE, 0);

r.CalcSumMLR(mat, numData) ;
r.CalcMultiReg() ;
PLAntEs ("****R HRA A ARES Linear Regression KEKKKKKKKE\Y");

printf("Number of points: %lg\n", r.sum);

DoPMuE(h 2 = Socom, iano)»
printf ("Adjusted R*2 = %6.41f\n", r.R2adj);
printf("Intercept = %lg\n", r.Intercept) ;
for (i = 1; i <= numTerms; i++)
PEL Ei SSiope etc sai Nt sh ete ice ee
PLAntEf ("***KARKAAKKER ANOVA **********\n") >

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS);

printf("Regression df = %lg\n", r.ANOVA.Reg_df);
printf("Residual SS = $%lg\n", r.ANOVA.Residual_SS) ;
printf("Residual df = %$lg\n", r.ANOVA.Residual_df);
DEIneEE("Total SS = sig\n", »-ANOVA- Total Ss)i>
printf("Total df = %lg\n", r.ANOVA.Total_df) ;
prime ("S42 = Aho \m% a. ANOVA 252i
prince’ ("*F = Sig\n", x£.ANOVA.F) ;

pressAnyKey
() ;

probability = 95;
r.MultiRegCoefCI (probability, slopeHi, slopeLow) ;
printf("At lg %% probability\n", probability);
for (i = 1; i <= numTerms; i++)
printf("Range for slope of X%d is %lg to %lg\n",
i, slopeLow[i], slopeHi[i]);

pressAnyKey
() ;

for (i = 1; i <= numTerms; i++) {

switch (i) {
case 1:
testValue 0...
break;

case 2:
testValue = 0.01;
break;
}
r.MLR_Slope_T_test (probability, testValue, i,
calcT, tableT, passTest);
printf ("At %lg %% probability\n", probability);
Print£(*HO: Slg 2=? tig, ", xr.Bli], testValue)>
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n");
}
pressAnyKey
() ;

r.MLR_R2_T_ Test (probability, calcT, tableT, passTest) ;

printf ("At lg %% probability\n", probability) ;
PHINEL ("Hs Big-is moles a; 2. er Raye
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;
 Multiple and Polynomial Regression 259

pressAnyKey
() ;

for (i = 1; i <= numTerms; i++)

Karri = mat(O, 2);

r.yHatCI (probability, Xarr, yHatLow, yHat, yHatHigh) ;

printf£("At %lg %% probability given:\n", probability);
for (i = 1; i <= numTerms; i++)
Prine e("Xisdieo= SLg\n", ay, Kasra):
Drinkn(@y> = silg\n") vat):
printf("Range for Y* is %lg to %lg\n",
yHatLow, yHatHigh) ;

pressAnyKey
() ;

return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
puts("\n\n");
}
double linear(double x)
{
return x;
}

Polynomial Regression
Polynomial regression is a special case of multiple regression. Instead of dealing with
multiple variables, you correlate two variables using a polynomial model. Thus poly-
nomial regression takes the independent variable X and creates additional pseudo-
variables whose values are based on the integer powers of X. Therefore creating a
library for polynomial regression is very easy when you start with a library that sup-
ports multiple regression. Thus, the multiple regression variables X1, X2, X3, and so
on, correspond to X, X?, X?, and so on, in polynomial regression.

The C Source Code

Listing 11.7 shows the source code for the POLYREG.H header file. This file declares
two structures, namely, PolyReg_ANOVAtag and PolyRegtag. The structure PolyReg_
ANOVAtag, as the name might suggest, contains fields that support the ANOVA table
for the polynomial regression. The second structure, PolyRegtag, contains fields that
support the polynomial regression calculations. The structure contains the following
groups of fields:

= The field numTerms stores the number of independent variables involved in the
polynomial regression.
= The fields xIndex and yIndex store the indices for the independent variable X and
the dependent variable Y, respectively.
= The fields fx and fy are function pointers that specify the transformations applied
to each variable.
260 Chapter Eleven

= The field invfy is the pointer to the function that performs the inverse transforma-
tion on the dependent variable Y.
The fields sum, sumY, and sumYY store the number of observations, sum of Y, and
sum of Y squared, respectively.
The fields A and S are matrix-type structures that store the various statistical sum-
mations of data.
The fields TCalc, StdErSlope, B, and Mean are vector-type structures that store
the calculated Student-t values for the regression slopes, the standard errors for
the regression slopes, the regression slopes, and the mean values for the indepen-
dent variables, respectively.
The fields hasMissingData and missingCode are the missing-code flag and missing-
code value, respectively.
The field Inverted_S is a flag that determines whether or not the field S contains
an inverted matrix.
The field Intercept stores the intercept of the polynomial regression.
The fields R2 and R2adj store the coefficient of correlation and its adjusted value,
respectively.
The field ANOVA stores the ANOVA table for the polynomial regression.
The header file contains typedef statements that create shorted aliases, PolyReg_
ANOVA and PolyRegrec, for the above structures.
The header file declares the following functions:

1. The function InitializePolyReg initializes a PolyRegrec structure to prepare it for

a new set of calculations. The parameter r is the pointer to the initialized PolyRe-
grec structure. The parameter NumTerms specifies the order of the polynomial.
The parameters Fx and Fy pass function pointers for the independent variable X
and the dependent variable Y. The parameter InvFy passes the pointer to the
function that performs the inverse transformations for the dependent variable.
The parameters XIndex and YIndex specify the indices that select the regression
variables. The parameter HasMissingData specifies the missing-code flag. The pa-
rameter MissingCode indicates the value of the missing code. If the argument to
parameter HasMissingData is 0, the argument for parameter MissingCode can be
any value. YOU MUST call function InitializePolyReg at least once before you per-
form polynomial regression analysis.
. The function CalcSumPolyReg updates the statistical summations. The parame-
ter DataMat is a matrix-type structure that supplies the observed data. The argu-
ments for parameters XIndex and YIndex in function InitializePolyReg specify
which columns of parameter DataMat enter in the regression. The parameter r is
the pointer to the PolyRegrec structure that stores the statistical summations.
The parameter NData specifies the number of observations to process. You can
call function CalcSumPolyReg more than once.
. The function CalcPolyReg calculates the regression slopes, intercept, ANOVA,
and other related data. The parameter r is the pointer to the structure PolyRegrec
 Multiple and Polynomial Regression 261

that holds the regression matrices and vectors. You must call function CalcPolyReg
before calling the remaining functions listed below.
4. The function PolyRegYHatCI calculates the projected value of the dependent
variable and its confidence interval for a given value of the independent variable.
The parameter r is the pointer to the PolyRegrec structure that contains the re-
sults of polynomial regression. The parameter probability specifies the confi-
dence level (expressed either as a percent or as a decimal). The parameter X
specifies the value of the independent variable. The parameters yHat, yHatLow,
and yHatHigh are pointers to double-type variables that pass the value for the
projected value and its confidence interval.
5. The function PolyRegCoefCI calculates the confidence intervals for the regression
slopes. The parameter r is the pointer to the PolyRegrec structure that contains the
results of polynomial regression. The parameter probability specifies the confi-
dence level (expressed either as a percent or as a decimal). The parameter slopeHi
specifies the array of upper limits for the regression slope. The element slopeHi[1]}
corresponds to the slope for variable X. The element slopeHi[2] corresponds to the
slope for X?, and so on. The parameter slopeLow specifies the array of lower limits
for the regression slope. The element slopeLow[1] corresponds to the slope for
variable X. The element slopeLow[2] corresponds to the slope for X?, and so on.
6. The function PolyReg_Slope_T_test tests the value of a specific regression slope.
The parameter r is the pointer to the PolyRegrec structure that contains the re-
sults of polynomial regression. The parameter probability specifies the confi-
dence level (expression as a percent or as a decimal). The parameter testValue
specifies the tested value of the targeted regression slope. The parameter
termNum is the index of the tested regression slope. The parameter calcT is a
pointer to a double-type variable that receives the calculated Student-t value.
The parameter tableT is a pointer to a double-type variable that receives the tab-
ulated Student-t value. The parameter passTest is the pointer to the in-type vari-
able that reports whether the outcome of the test is positive or negative.
7. The function PolyReg_R2_T_test tests whether or not the value of the correlation
coefficient is not 0. The parameter r is the pointer to the PolyRegrec structure
that contains the results of polynomial regression. The parameter probability
specifies the confidence level (expressed as a percent or as a decimal). The pa-
rameter calcT is a pointer to a double-type variable that receives the calculated
Student-t value. The parameter tableT is a pointer to a double-type variable that
receives the tabulated Student-t value. The parameter passTest is the pointer to
the in-type variable that reports whether the outcome of the test is positive or
negative.
8. The function deletePolyRegrec deletes the dynamic data associated with a Poly-
Regrec-type structure. You need to call this function at least once, after you are
done with the polynomial regression analysis.

Listing 11-8 shows the source code for the POLYREG.C implementation file. The
code in that file is based on the algorithms that I presented earlier. The code also in-
clude statements that manage missing data.
262 Chapter Eleven

Listing 11.7 The source code for the POLYREG.H header file.

#ifndef _POLYREG_H_
#define _POLYREG_H_
/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 1/18/95

C module that performs the following polynomial

regression operations:

+ Basic multiple regression

+ Confidence interval of slopes
+ Student-t test for regression slopes
+ Student-t test for regression coefficient
ways

#include "arrays.h"

#define POLYREG_BIG 1.0E+30

struct PolyReg_ANOVAtag {
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_Ss;
double Total_df;
double Total_SS;
double S2;
double F;
ie

typedef struct PolyReg_ANOVAtag PolyReg_ANOVA;

struct PolyRegtag {
int numTerms;
int xIndex;
int yIndex;
double (*fx) (double) ;
double (*fy) (double) ;
double (*invfy) (double) ;
/* summation block */
double sum;
double sumyY;
double sumYY;
Matrix A;
Matrix S;
Vector TCalc;
Vector StdErSlope;
—
Vector 3%
Vector Mean;
int hasMissingData;
double missingCode;
int Inverted_S;
double Intercept;
double R2;
double R2adj;
PolyReg_ANOVA ANOVA;
Te

typedef struct PolyRegtag PolyRegrec;

void InitializePolyReg(PolyRegrec* r,
int NumTerms,
double (*Fx) (double),
double (*Fy) (double),
 Multiple and Polynomial Regression

double (*InvFy) (double),

int XIndex,
int YIndex,
int HasMissingData,
double MissingCode) ;

void CalcSumPolyReg (Matrix DataMat, PolyRegrec* r, int NData);

void CalcPolyReg(PolyRegrec* r);

void PolyRegYHatCI (PolyRegrec* r, double probability,

double X,
double* yHatLow,
double* yHat,
double* yHatHigh) ;

void PolyRegCoefCI (PolyRegrec* r, double probability,

double* slopeHi,
double* slopeLow) ;

void PolyReg_Slope_T_test(PolyRegrec* r, double probability,

double testValue, int termNum,
double* calcT, double* tableT,
int* passTest) ;

void PolyReg_R2_T
Test (PolyRegrec* r, double probability,
double* calcT, double* tableT,
int* passTest) ;

void deletePolyRegrec (PolyRegrec* r) ;

#endif

Listing 11.8 The source code for the POLYREG.C implementation file.

#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "polyreg.h"
#include "statlib.h"

void InitializePolyReg(PolyRegrec* r,
int NumTerms,
double (*Fx) (double),
double (*Fy) (double),
double (*InvFy) (double),
int XIndex,
int YIndex,
int HasMissingData,
double MissingCode)

alhahee SMAneSir nal.

r->numTerms = NumTerms;
n = r->numTerms + 1;

newMat (&r->A, n, n);

newMat (&r->S, n, n);
newVect (&r->TCalc, n);
newVect (&r->StdErSlope, n);
newVect (&r->B, n);
newVect (&r->Mean, n);

iS(orge (Cale ape Gills eg aan) a

/* initialize summations */
rong (ae ON ales eal ees)
Chapter Eleven

Listing 11.8 (Continued)

MAT (r->A, i, 3) = 0;
WANN Ge=sas}, Gl) = iF
}
r->xIndex = XIndex;
r->yIndex = YIndex;
igs Sie S Ipsies
ee ys
r->invfy = InvFy;
r->sumyY = 0;
r->hasMissingData = HasMissingData;
r->missingCode = MissingCode;
}
void CalcSumPolyReg (Matrix DataMat, PolyRegrec* r, int NData)
{
SME BN Si Lee
int n = r->numTerms + 1;
double T[2];
double Yt;
double Xt;
int OK;

for (i = 0; i < NData; i++) {

T[O] = MAT(DataMat, i, r->yIndex) ;
T[1] = MAT(DataMat, i, r->xIndex) ;

pF sel ok ilag */
OK = TRUE;

/* are there any possible missing data? */

if (r->hasMissingData) {
/* search for missing data */
j= Oly
while (j < 2 && OK)
if (T[j] > r->missingCode)
Bthr
else
OK = FALSE;

ue (OK) {
/* now transform the data */
TROT = (sae > faci T Odie
Til = (Fn=Sty) ur iyi

Vase nO
Ree EET
r->sumYY += SOR(Yt);
for (k = 0; k < n; kt+) \{
VEC (r->B, k) += Yt * pow(Xt, (double)k);
for (a = 10s) sj e< saree)
MAT(r->A, j, k) += pow(Xt, (double)j + k);

}
void CalcPolyReg(PolyRegrec* r)
{
double Diag, tempo, DegF, MS;
a a ig by Boi lt
int n = r->numTerms + 1;

r->sum = MAT(r->A, 0, 0);

r->sumY = VEC(r->B, 0);
 Multiple and Polynomial Regression 265

/* Form the A and S Matrices */

Om (Ce = eee es) ot
MAT(r->A, i, 0) = VEC(r->B, i) - r->sumY *
MAT(r->A, 0, i) / r->sum;
eRe (5) =" als ) SS ah ae)
MAT (r->A, ji, i), == MAT (r=SA7 0), a) *
MAT(r->A, 0, j) / xr->sum;
/* Make a copy of the matrix A */
MAR Crass, ey, 22)) = MA (AT i
}
}

/* Clear the matrix S inversion flag */

r->Inverted_S = FALSE;

r->sumYY -= SQR(r->sumY) / r->sum;

r->sumY /= r->sum;
Ore. sulin Te Sem ales)
MAT (r=>A, “0, 2)°/= xc->sum;
/* Make copies of the mean and std error vectors */
VEC (r->Mean, i) = MAT(r->A, 0, i);
VEC (r->StdErSlope, i) = MAT(r->A, i, i);
}

/* Form correlation matrix */

Or pas =e oa aes gee) AG
MAT(r->A, i, 0) /= sqrt(r->sumYY * VEC(r->StdErSlope, cu)ies
fom (= yy, <a et)
MAT(r->A, j, i) /= sqrt(VEC(r->StdErSlope, i) *
VEC (r->StdErSlope, j));
MAT (=>), “1,7 ah) = a0);
}
/* Complete symmetric part of the matrix */
Oise (=) Dip) e< ee)
oral (Ge eb eseee al? 5)-v20))
HU Gale Ne aby, ai) WN
Ta SIN Sy al)

/* Start solving the simultaneous equations */

Pore (j= ey <omieats))t
Diag s= MAT tr SAG a, a) F
MAT (Ge—> Aig salh = ales
for ({k = 1; & =m; k++)
MAT(r->A, j, k) /= Diag;

form(k = die mip) k++)

we (7 =k) {
tempo = MAT(r->A, k, a
MAT (x=>A, k, 3) = 0%
scone” (tn “=> A sie Sema ai 3))
MAT(r->A, k, i) -= MAT(r->A, Jj, i) * tempo;
}
}
r->R2 = 0;
r->Intercept = 0;
for We ia ea a e< NF i++) {
VEC (2=>B,) 2) = 0;
Rove =) Ls a Seas 34s)
VEC (r->B, i) += MAT(r->A, j, 0) * MAT(r->A, j, 2) 5
Y—->R2 += VEC(r->B, i) * MAT(r->A, i, 0);
VEC (r->B, i) *= sqrt(r->sumYY / VEC(r->StdErSlope, SL)ar
r->Intercept += VEC(r->B, i) * MAT(r->A, 0, i);
}
r->Intercept = r->sumY - r->Intercept;
DegF = r->sum - r->numTerms - 1;
MS = (2) = r=sR2) * 2-Ssumyy / Degk;
266 Chapter Eleven

Listing 11.8 (Continued)

OvemCte =e <omle eter

VEC (r->StdErSlope, i) = sqrt(MS * MAT(r->A, i, i) /
VEC (r->StdErSlope, i));
VEC (r->TCalc, i) = VEC(r->B, i) / VEC(r->StdErSlope, Eh)
}
r=>R2adj = 1 - (1 — r->R2) * r->sum / DegF;
r->ANOVA.Total_df = xr->numTerms + 1;
r->ANOVA.Total_SS = r->sumYY;
r->ANOVA.Reg_df = r->numTerms;
r->ANOVA.S2 = MS;
r->ANOVA.Reg_SS = r->sumYY - r->ANOVA.S2;
r->ANOVA.Residual_df = DegF;
r->ANOVA.Residual_SS = r->ANOVA.S2 * DegF;
r->ANOVA.F = DegF / r->numTerms * r->R2 / (1 - r->R2);
}
void PolyRegYHatCI (PolyRegrec* r, double probability,
double X,
double* yHatLow,
double* yHat,
double* yHatHigh)

double diff, p, df, tableT, traceMat, pivot, tempo;

cba ne eegpee Lee. we
Matrix XX;
Matrix prodMat;
Vector Xpow;
int n = r->numTerms + 1;

newMat (&XX, n, n);

newMat (&prodMat, n, n);
newVect (&Xpow, n);

if (probability > 1)
Bea Ob — probabriaty e/ e200;
else
p = 0.5 - probability / 2;
df = r->sum - (r->numTerms + 1);
tableT = TiInv(p, df);

if (!r->Inverted_S) { /* Invert matrix S */

r->Inverted_S = TRUE;
for “4 *=s13 aaseny Gite)
pivot = MAR(r=>S; a7, 3)/7
MAT (r->S, she 3) = 1;
form ct eilss lee<oris te)
MAT(T=SS) 37K) f= DEVOE;

for dik ioelewice< nee ke)

i ik tes aie lt
tempo = MAT(r->S, k, j);
MAT (r=—>S, kya) =10-
for (m= Ly m </mzemst)
MAT(r->S, k, m) -= MAT(r->S, j, m) * tempo;
}

}
/* Calculate yHat */
*vHat = r=-sIntercepe,
X= (*r=-SEx) (ye
for (2 = Le 2 < ne see
VEC (Xpow, i) = pow(X, i);
*yHat += VEC(r->B, i) * VEC(Xpow, i);
 Multiple and Polynomial Regression 267

/* Form standardized vector */

fore (i =o en; i++)
VEC (Xpow, i) -= VEC(r->Mean, i);

i= BoOrm xx Xk" inatr is 7/7.

fore (=e ee< mi tic)
Fore (ty, aay oy) =<) ta gees)
MATIC Paik) = 0s

form (ike = ek <"n-) ka)

eC (Gieg BLY oles relr) Spe.)
MAX es), SK) ot=— VEC pow, bal * ovinC(oow), ik)i+

/* Multiply S_Inverse and XxX' */

form (2 °= "lien <> (ist
ioe (a), iedbyahafess Waifs siecna)\e ahi
MAT(prodMat, i, j) = 0;
sHolian (Ge = SAO eee Goth Mateo)
MAT (prodMat, i, j) += MAT(r->S, i, k) *
MAT (XX, k, J);

}
/* Calculate trace of prodMat */
traceMat = 1.;
Ori spades gece wy) ++)
traceMat *= MAT(prodMat, i, i);

diff = tableT * sqrt(r->ANOVA.S2 * (1 + 1/r->sum + traceMat)

);
*yHatLow = *yHat —- diff;
‘vHatHigh = *yHat + diff;

SyHate = (*r-Sinvyty) (*yHat)) >

*yHatLow = (*r->invfy) (*yHatLow) ;
*yHatHigh = (*r->invfy) (*yHatHigh) ;

deleteMat (&XX) ;
deleteMat (&prodMat) ;
deleteVect (&Xpow) ;

}
void PolyRegCoefCI (PolyRegrec* r, double probability,
double* slopeHi,
double* slopeLow)

double diff, p, df, tableT;

Lakes [Re
int n = r->numTerms + 1;

if (probability > 1)
DE OMS orcobabsti sf 20107
else
Dy—2 02S) Stprobabilatysy/i 2;
agf = r->sum —- (r->numTerms + 1);
Eabwvetle— in (po, AL) 7
fom! Gia LW nyiok) it
aiff = tableT * VEC(r->StdErSlope, j);
Slope a] = VEC (t—SB ea) + Ge ti;
sllopelowlj] = VEC(r=-sB) 7) = ditt;
}
}

void PolyReg_Slope_T_test(PolyRegrec* r, double probability,

double testValue, int termNum,
double* calcT, double* tableT,
int* passTest)
{
double p, df;
268 Chapter Eleven

Listing 11.8 (Continued)

if (probability > 1)
Dp = 0.5 = probabudeiey, 7/200)
else
p=) 0.5) = probabmlity 7/2
df = r->sum - (r->numTerms + 1);
*tableD = Tino, de);
*calcT = (VEC(r->B, termNum) - testValue) /
VEC (r->StdErSlope, termNum) ;
*passTest = (fabs(*calcT) <= *tableT) ? TRUE : FALSE;
}
void PolyReg_R2_T_
Test (PolyRegrec* r, double probability,
double* calcT, double* tableT,
int* passTest)
j-wteste hypothesis) HON Ro2 = O47,
if

double p, df;

if (probability > 1)
p= 08S = probabailecy 7/5200).
else
De OL Se —sprobabileteyea/. 2e
af = r->sum - (r->numTerms + 1);
table = TWinw (qo, (athr
AcaleT: = sqrt (r=>R2 * di 7 (lL = £=SR2)))>
*passTest = (*callcT <= *tableT ) ? FALSE =: TRUE;
}

void deletePolyRegrec(PolyRegrec* r)
{
deleteMat (&r->A);
deleteMat (&r->S) ;
deleteVect (&r->TCalc);
deleteVect (&r->StdErSlope) ;
deleteVect (&r->B) ;
deleteVect (&r->Mean) ;
}

The C++ Source Code

Now let’s look at the C++ code for the multiple regression library. Listing 11.9 shows
the source code for the POLYREG.HPP header file. The file declares the classes
PolyReg_ANOVA and PolyReg. The class PolyReg_ ANOVA contains public data
members that represent the components of an ANOVA table for the polynomial re-
gression. The class PolyReg contains public data members and member functions
that support the polynomial regression. The member functions in class PolyReg are
very similar to the C functions in file POLYREG.H. Likewise, the data members in the
class are similar to their counterparts in structure PolyRegtag in file POLYREG.H.
There are some minor differences in the declarations of some member functions and
their counterpart C functions. The member functions PolyRegYHatCI, PolyReg_
Slope_T_test, and PolyReg_R2_T_test use reference parameters instead of pointer-
type parameters to pass information to the caller. Also, the member function Calc-
SumPolyReg declares a reference to the matrix-type parameter instead of declaring
that parameter to pass a copy of its data.
 Multiple and Polynomial Regression 269

Listing 11.9 The source code for the POLYREG.HPP header file.

#ifndef _POLYREG HPP_

#define _POLYREG HPP_

/*

Copyright (c) 1995 Namir C. Shammas

Version 1.0 Date 1/18/95

C++ module which performs the following polynomial

regression operations:

+ Basic multiple regression

+ Confidence interval of slopes
+ Student-t test for regression slopes
+ Student-t test for regression coefficient
“if

#include "arrays.hpp"

#define POLYREG_BIG 1.0E+30

class PolyReg_ANOVA {
public:
double Reg_df;
double Reg_SS;
double Residual_df;
double Residual_SS;
double Total_df;
double Total_SS;
double S2;
double F;
EG
class PolyReg
{
Publics

void InitializePolyReg(int NumTerms,

double (*Fx) (double),
double (*Fy) (double),
double (*InvFy) (double),
int XIndex,
int YIndex,
int HasMissingData,
double MissingCode) ;

void CalcSumPolyReg (Matrix DataMat, int NData);

void CalcPolyReg() ;

void PolyRegYHatCI (double probability,

double X,
double& yHatLow,
double& yHat,
double& yHatHigh) ;

void PolyRegCoefCI (double probability,

double* slopeHi,
double* slopeLow) ;

void PolyReg_Slope_T_
test (double probability,
double testValue, int termNum,
double& calcT, double& tableT,
int& passTest) ;
270 Chapter Eleven

Listing 11.9 (Continued)

void PolyReg_R2_T_
Test (double probability,
double& calcT, double& tableT,
int& passTest) ;

int numTerms;
int xIndex;
int yIndex;
double (*f£x) (double) ;
double (*fy) (double) ;
double (*invfy) (double) ;
/* summation block */
double sum;
double sumyY;
double sumYY;
Matrix A;
Matrix S;
Vector TCalc;
Vector StdErSlope;
Vector B;
Vector Mean;
int hasMissingData;
double missingCode;
int Inverted_S;
double Intercept;
double R2;
double R2adj;
PolyReg_ANOVA ANOVA;
Le

#endif

Listing 11.10 The source code for the POLYREG.CPP implementation file.
#include <stdlib.h>
#include <math.h>
#include "global.h"
#include "polyreg.hpp"
#include "statlib.h"

void PolyReg: :InitializePolyReg (

int NumTerms,
double (*Fx) (double),
double (*Fy) (double),
double (*InvFy) (double),
int XIndex,
int YIndex,
int HasMissingData,
double MissingCode)

af gareys ie a

numTerms = NumTerms;
n = numTerms + 1;

A.ReSize(n, n);
S.ReSize(n, n);
TCalc.ReSize(n);
StdErSlope.ReSize(n) ;
B.ReSize(n) ;
Mean.ReSize(n) ;

for (i = 07 2'< ne ier)”

/* initialize summations */
 Multiple and Polynomial Regression 271

fora) (Ge) Oe <i at)

(Ge i= 0);
Bil = 0;
y
xIndex = XIndex;
yindex = YIndex;
Boco= Bs
fy = Fy;
invfy = InvFy;
sumyYy = 0;
hasMissingData = HasMissingData;
missingCode = MissingCode;
}
void PolyReg: :CalcSumPolyReg (Matrix DataMat, int NData)
{
sialic als air, aalep
int n = numTerms + 1;
double T[2];
double Yt;
double Xt;
int OK;

for (i 0; i < NData; i++) {

T[0] DataMat (i, yIndex) ;
sufi DataMat(i, xIndex) ;

LeeSseEnok £lag 27,

OK = TRUE;

/* are there any possible missing data? */

if (hasMissingData) {
/* search for missing data */
7) = Op
while (j < 2 && OK)
if (T[j] > missingCode)
att
else
OK = FALSE;

Ee (OK Ie
/* now transform the data */
BLOT = Ge) TO)
re en Go ten) G1 (bale)ce

Yas = GH OMls
4c ES UNlialils
sumYY += SOR(Yt);
Tor Wke= Ok =< ni k+ey f
B([k] += Yt * pow(Xt, double(k));
LO (ey =. 0} 3) eerie) et)
A(j, k) += pow(Xt, double(j) + k);

}
void PolyReg: :CalcPolyReg()
{
double Diag, tempo, DegF, MS;
ite canes) URon
int n = numTerms + 1;

sum = A(0, 0);

sumy = B[0];
/* Form the A and S Matrices */
272 Chapter Eleven

Listing 11.10 (Continwed)

for (4. =] i. 2 <ne sae

Ata, 0) = Bly] =tsumy < AG; a) fh sum?
for (3 = 1; 3 <= a7 a+) 4
AG ei) a-= AOR a) * AO; 3) 7 sum;
/* Make a copy of the matrix A */
SHETy i) = A(g, tA
}
}

/* Clear the matrix S inversion flag */

Inverted_S = FALSE;
sumyYY -= SQR(sumY) / sum;
sumy /= sum;
for (1 = 13 i < me a++)
BXO, 2) f= sums
/* Make copies of the mean and std error vectors */
Mean[i] = A(0, i);
StdErSlopefil’ = A(a, 2)
}

/* Form correlation matrix */

Fore (2. =F depeche aa) 1
A(z, ©)” 7] sqrt(sumyy * StdExrSlopelia
|);
fom (Geer see tar alice)
A(j, i) /= sqrt(StdErSlope[i] * StdErSlope[j]);
Aa Lye ae te
}

/* Complete symmetric part of the matrix */

forets = 25) ae <omj) i+)
itera M(Gfome WU G/B SE SETS,
ANE, 3) = AGe ADs

/* Start solving the simultaneous equations */

Ome spi=! Ay Sy retreat
Diag = A(j;, 5)F
AG 3H ols
fore(K = le ki ne ke+)
Ayes) (= Diag;

fox (k = 1; k = n> KE)

Ee (gue)
tempo = A(k, j);
A(k, 3) = 0;
for (a = Us Baee mie isa)
Ak, 2) <=="A0Ge 3) Seerempe;
}
}
R2 = 0;
Intercept = 0;
for (1 = byob <a
Bla = 0;
Bor (30 = 135) 3) Ss ne ae)
Bil += ACs, OC) * AG:
R2 += B[i] * A(i, 0);
B[i] *= sqrt(sumyY / StdErSlope[i]);
Intercept += B[i] * A(0, i);
}
Intercept = sumY - Intercept;
DegF = sum — numTerms - 1;
MS = (1 -— R2) * sumYY / DegF;
bor) (hs sa sear
StdErSlope[i] = sqrt(MS * A(i, i) / StdErSlope[i]);
TCalc[i] = B[i]) / StdErSlope[il];
 Multiple and Polynomial Regression 273

}
R2adj = 1 - (1 -— R2) * sum / DegF;
ANOVA.Total_df = numTerms + 1;
ANOVA.Total_SS = sumYY;
ANOVA.Reg_df = numTerms;
ANOVA.S2 = MS;
ANOVA.Reg_SS = sumYY —- ANOVA.S2;
ANOVA.Residual_df = DegF;
ANOVA.Residual_SS = ANOVA.S2 * DegF;
ANOVA.F = DegF / numTerms * R2 / (1 - R2);
}

void PolyReg: :PolyRegYHatCI (double probability,

double X,
double& yHatLow,
double& yHat,
double& yHatHigh)

double diff, p, df, tableT, traceMat, pivot, tempo;

stale eBAd melee enone bules
int n = numTerms + 1;
Matrix XX(n, n);
Matrix prodMat(n, n);
Vector Xpow(n) ;

if (probability > 1)
p = 0.5 - probability / 200;
else
p = 0.5 — probability / 2;
df = sum —- (numTerms + 1);
tablet = Tinv (p, GE£);

if (!Inverted_S) { /* Invert matrix S */

Inverted_S = TRUE;
EOrs (ap f= et Fe eae
pivot S (ai, ai
SIGs sa)! Sly
ngepan {Gey aR NS <x, fein) Neen)
S(j, k) /= pivot;
for (k= Uy keam- k++)
oS (Giemii— af wd) At
tempo = S(k, j);
Sky a) = 10);
for (m = 1; m < n; m++)
S(k, m) —= S(j, m) * tempo;
}

}
/* Calculate yHat */
yHat = Intercept;
X = (*£x) (X);
POIs (eae beet <The p++) sf
Xpow[i] = pow(X, i);
yHat += B[i] * Xpowl[il;
}
/* Form standardized vector */
forms (ae= 15 i<j mF at++)
Xpow[i] -= Mean[i];

/* Form X X' matrix */

for (k= Ue k< me k++)
EO (a= ele oe tt)
XX(j, k) = 0;
274 Chapter Eleven

Listing 11.10 (Continued)

fom (k = iy k < meek)

Lor (3; = Ls 3) nie +)
XX(j, k) += Xpow[j] * Xpow[k];
/* Multiply S_Inverse and XX' */
ort (i. =) > de mare re)
for! (qi = Le a) any ae) 4
ProOdMat(a eaeai c= Ol
for (k = dk <m, kay)
jouaereitiene
(i, 3))\) = TSG, i) eo Odes SIN)

}
/* Calculate trace of prodMat */
traceMat = 1.;
fom. (4 = Ip 2 < nz i++)
traceMat *= prodMat(i, 1);

aiff = tableT * sqrt(ANOVA.S2 * (1 + 1/sum + traceMat)

);
yHatLow = yHat — diff;
ViatHigh = sylat. + ditt;

VHat = (sanvety) (vyiat) >

yHatLow = (*invfy) (yHatLow) ;
yHatHigh = (*invfy) (yHatHigh) ;

}
void PolyReg: :PolyRegCoefCI (double probability,
double* slopeHi,
double* slopeLow)

double diff, p, df, tableT;

biol eae
int n = numTerms + 1;

tee (probabrizty > 1)

Dp = 0.5 ="probabilzty 7 2007
else
p= 085 —probabiliaty 7) 2%
af = sum —- (numTerms + 1);
tableT = TiInv(p, df);
for (f= 2s poe ae) et
dadiff = tableT * StdErSlope[jl];
slopeHi[j] = B[j] + diff;
slopeLow[j] = B[j] - diff;
}
}
void PolyReg::PolyReg_Slope_T_test(double probability,
double testValue, int termNum,
double& calcT, double& tableT,
int& passTest)

double p, df;

if (probability > 1)
Dp = 0.5 = probabilwiey 7 200-
else
p= 0.5 = probabilaiy 7 25
df = sum — (numTerms + 1);
tablet = PTinv(p, at):
calcT = (B[termNum] - testValue) / StdErSlope[termNum] ;
passTest = (fabs(calcT) <= tableT) ? TRUE : FALSE;
 Multiple and Polynomial Regression 275

void PolyReg::PolyReg_R2_T
Test (double probability,
double& calcT, double& tableT,
int& passTest)
/* test hypothesis HO : R“2 = 0 */
{
double p, df;
ie (orobabmlity > 1°)
p= O85 = probability / 200);
else
Pe= 0. Ses DrObabilaty / 2
af = sum — (numTerms + 1);
tabler = Tinv(p, dé);
Galle Sormtiik2s ice j/a(liy — R2))\ i
passTest = (calcT <= tableT ) ? FALSE : TRUE;

The C Test Program

Let’s look at the test program for the C version of the multiple regression library.
Listing 11.11 shows the source code for the TSPOLYRE.C test program. This pro-
gram performs the following tasks:

. Declares a variety of variables to conduct the diverse calculations.

. Initializes the array of function pointers, fx, using the C function linear.
. Allocates the dynamic space for the matrix mat.
. Assigns the test data to the elements of matrix mat.
He.
DH
WO
FP
o Assigns the values to the variables that store the number of data and number of
terms (that is, the number of independent variables).
6. Assigns the indices for the regression variables to the variables Xindex and YIndex.
7. Initializes the PolyRegrec structure r by calling the function InitializePolyReg.
The arguments for the function call are the address of structure r, the variable
numTerms, the pointer to function linear (which appears three times), the vari-
able XIndex, the variable YIndex, the value FALSE, and the value 0. The latter
two arguments indicate that the data matrix will have no missing observations.
. Updates the statistical summations by calling function CalcSumPolyReg. The ar-
guments for this function call are the matrix mat, the address of structure r, and
the variable numData.
. Calculates the regression slopes and other results by calling the function Calc-
PolyReg. The argument for this function call is the address of structure r.
10: Displays the regression results that include the coefficient of correlation, the in-
tercept, the regression slopes, and the ANOVA table components.
Bh Calculates and displays the confidence intervals, at 95% confidence, for the
regression slopes. This task involves calling function PolyRegCoefCI. The
program uses the arrays slopeHi and slopeLow to obtain the sought confi-
dence intervals.
12. Tests the values for the two regression slopes at 95% confidence. The program
calls function PolyReg_Slope_T_test to determine if the first and second slopes
276 Chapter Eleven

are not that significantly different from the values of 0 and 1, respectively. The
program displays the outcome of the test.
13. Tests whether or not the value of the correlation coefficient is 0 (at 95% confi-
dence). The program calls function PolyReg_R2_T_test and displays the out-
come of the test.
14. Calculates the projected value of the dependent variable and its confidence in-
terval (at 95% confidence) for the first row of data. The program calls function
PolyRegYHatCI and passes the arguments &r, probability, Xarr, &yHatLow,
&yHat, and &yHatHigh. The program then displays the value of the dependent
variables, the projected value, and the confidence interval for the projection.
16. Deletes the dynamic data of the matrix mat, vector Index, and structure r.

Figure 11.3 shows the output of the test program. The project file for the test pro-
gram should include the files STATLIB.C, ARRAYS.C, POLYREG.C, and TSPOLYRE.C.

Listing 11.11 The source code for the TSPOLYRE.C test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "polyreg.h"
#include "global.h"
#include "“arrays.h"

void pressAnyKey (void) ;

double linear(double) ;

main()
{
Matrix mat;
PolyRegrec r;
int XIndex, YIndex;
double probability;
double testValue, calcT, tableT;
double yHat, yHatLow, yHatHigh;
double slopeLow[10] ;
double slopeHi[10];
double X;
int i, numTerms, numData;
int passTest;

newMat (&mat, 11, bo

MAT (maith yOy Oia Gs

MAT (mat, ih; Opies disse
MAT (mati OjM= ease
MAT(mat, 3, 0) = 1.54;
MAT (mat, 4, 0) = 2.0;
MAT (matsnSe 0) (= 2. 255
MAT (mat, 6, 0) = 2.60!
MAT (mat, 7, 0) = 2.95;
MAT (mat, 8, 0) = 3.25%
MAT(mat, 9, 0) = 3.54;
MAT (mat, 10, 0) = 4.0.

MAT (mat,' 0; ZT) = L20F

MAT (matt, > yy ais
 Multiple and Polynomial Regression 277

MAD nae he De
MAT (nate es eee Less
MAT (mat, 4, 1) = 1). 4.
UPN Gree Wey a0) ea alesse
MATIN (Mcte nO pal) endl Ose
PAGE eh ey Ho; 1) eae ae VES
MAT (Maton eo =. 0.85
Mmarimat,, 3 2) =. 1.9;
MAT (mat Or 1) =) 20)

numData = 11;
numTerms = 2;
YIndex = 0;
XIndex = 1;

InitializePolyReg(&r, numTerms, linear, linear, linear,

XIndex, YIndex, FALSE, 0);
CalcSumPolyReg(mat, &r, numData) ;
CalcPolyReg(&r) ;
PLintf (VARA AKA AK KARE Polynomial Regression KKKKKKKKKK\Y") >

printf ("Number of points: %lg\n", r.sum);

DEtneE( (RO? = $6.4 i \m" oR?)
printf ("Adjusted R*2 = %6.41f\n", r.R2adj);
printf("Intercept = %lg\n", r.Intercept) ;
Mow) (c= alee mum ermsis | ate)
Dmintr ("Slope for Xtdsv= silig\n",, 2, VEC(r.By) 2))));
jopcelea hone (MEF esta atts cis ANOVA REKKKKKKKK\ YH") »

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS) ;

printf("Regression df = %lg\n", r.ANOVA.Reg_df) ;
printf("Residual SS = %lg\n", r.ANOVA.Residual_SS) ;
printf ("Residual df = %lg\n", r.ANOVA.Residual_df) ;
printét ("Total SS = tig\n", x£-ANOVA.Total_Ss);
printf("Total df = %lg\n", r.ANOVA.Total_dEf) ;
peinti( tS. 2 .— siig\ie i. ANOVA.S2,))
print£("F = %lg\n", r.ANOVA.F) ;

pressAnyKey
() ;

probability = 95;
PolyRegCoefCI(&r, probability, slopeHi, slopeLow) ;
printf("At lg %% probability\n", probability) ;
for (2) = 1) 2 <= numlerms, a+44)
printf("Range for slope of X%d is %lg to %lg\n",
i, slopeLow[i], slopeHi[i]);

pressAnyKey
();

for (1 = 1; 1 <= numTerms; i++) {

switch (i) {
case 1:
testValue = 0.0;
break;

case 2:
testValue = 1.0;
break;
}
PolyReg_Slope.T_test(&r, probability, testValue, zs
&calcT, &tableT, &passTest);
printf("At %lg %% probability\n", probability) ;
Drintl("HO: tligue=- tig, “;, VEC(x.B, i), testValue);
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n") ;
278 Chapter Eleven

Listing 11.11 (Continued)

pressAnyKey
() ;

PolyReg_R2_T Test(&r, probability, &calcT, &tableT, &passTest) ;

printf("At %lg %% probability\n", probability);
PiEinee("HOs Slosvsenot 10; VaR)
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n");

pressAnyKey
() ;

X= MAT (mat, 15. 0))>

PolyRegYHatCI(&r, probability, X, &yHatLow, &yHat, &yHatHigh) ;

printf("At lg %% probability given:\n", probability) ;
DEINE ("x= elon" , 22c)hs
DrEIntl (yo se silig\n pe yiat))
printf("Range for Y* is %lg to %lg\n",
yHatLow, yHatHigh) ;

pressAnyKey
() ;

deleteMat (&mat) ;
deletePolyRegrec (&r) ;

return 0;
}
void pressAnyKey
()
{
printf("\nPress any key to continue...");
getchar();
puts ("\n\n"));
}

double linear(double x)
{
return x;
}

The C++ Test Program

Let’s now look at the test program that tests the C++ version of the multiple regres-
sion library. Listing 11.12 shows the source code for the TSPOLYRE.CPP test pro-
gram. This program performs the following tasks:

1K Declares a variety of variables to conduct the diverse calculations. The declara-

tion includes the object r that is an instance of class PolyReg.
. Reallocates the dynamic space for the matrix mat. This task involves sending the
C++ message ReSize to the matrix mat.
3. Assigns the test data to the elements of matrix mat.
4. Assigns the values to the variables that store the number of data and number of
terms (that is, the number of independent variables).
5. Assigns the indices for the regression variables to the variables XIndex and YIndex.
6. Initializes the PolyReg object r by sending it the message InitializePolyReg. The
arguments for the message are the variable numTerms, the pointer to function
 Multiple and Polynomial Regression 279

KKEKKKKKKKKKKK Polynomial Regression KEKKKKKKEKK

Number of points: 11
R*2 = 0.9965
Adjusted R*2 0.9952
Intercept = - - 23662
Slope for X1 fH) 0.305664
Slope for X2 I 0.903263
KEKKKKKKKKKKE ANOVA KEK KKEKK KKK

Regression SS = 10.1032
Regression df = 2
Residual SS = 0.0354071
Residual df = 8
Dotan oo = Oe 077
Total dii= 33
So27 = 0200442589
ip! ce asia iekes)

Press any key to continue...

At 95 % probability
Range for slope of X1 is -1.27278 to 1.88411
Range for slope of X2 is 0.379381 to 1.42715

Press any key to continue...

At 95 % probability
HO: 0.305664 ?=? 0, cannot be rejected
At 95 % probability
HO: 0.903263 ?=? 1, cannot be rejected

Press any key to continue...

At 95 % probability
HO: 0.996497 is not 0, cannot be rejected

Press any key to continue...

At 95 % probability given:
XC ey alg ake}
Newel SoGay,
Range for ¥Y~ is 1.222748 to 1.54205

Press any key to continue...

Figure 11.3 The output of the program TSPOLYRE.EXE.

linear (which appears three times), the variable XIndex, the variable YIndex, the
value FALSE, and the value 0. The latter two arguments indicate that the data
matrix will have no missing observations.
7. Updates the statistical summations by sending the C++ message CalcSumPolyReg
to the object r. The arguments for this message are the matrix mat and the vari-
able numData.
8. Calculates the regression slopes and other results by sending the C++ message
CalcPolyReg to the object r.
9. Displays the regression results that include the coefficient of correlation, the in-
tercept, the regression slopes, and the ANOVA table components.
280 Chapter Eleven

10. Calculates and displays the confidence intervals, at 95% confidence, for the re-
gression slopes. This task involves sending the C++ message PolyRegCoefCI to
the object r. The program uses the arrays slopeHi and slopeLow to obtain the
sought confidence intervals.
11. Tests the values for the two regression slopes at 95% confidence. The program
sends the C++ message PolyReg_Slope_T_test to object r in order to determine
if the first and second slopes are not that significantly different from the values
of 0 and 1, respectively. The program displays the outcome of the test.
12. Tests whether or not the value of the correlation coefficient is 0 (at 95% confi-
dence). The program sends the C++ message PolyReg_R2_T_ test to object r
and then displays the outcome of the test.
13. Calculates the projected value of the dependent variable and its confidence in-
terval (at 95% confidence) for the first row of data. The program sends the C++
message yHatClI to object r and then passes the arguments &r, probability, X,
yHatLow, yHat, and yHatHigh. The program then displays the value of the depen-
dent variables, the projected value, and the confidence interval for the projection.
The project file for the test program should include the files STATLIB.CPP,
POLYREG.CPP and TSPOLYRE.CPP.

Listing 11.12 The source code for the TSPOLYRE.CPP test program.
#include <stdlib.h>
#include <time.h>
#include <stdio.h>
#include <math.h>
#include "polyreg.hpp"
#include "global.h"
#include "arrays.hpp"

void pressAnyKey (void) ;

double linear (double) ;

main ()
{
Matrix mat;
PolyReg r;
int XIndex, YIndex;
double probability;
double testValue, calcT, tableT;
double yHat, yHatLow, yHatHigh;
double slopeLow[10] ;
double slopeHi[10];
double X;
int i, numTerms, numData;
int passTest;

mat.ReSize(11, 2);

mat(O, 0) = 1.0;
mat(1,, 0) = 1.18;
mat(2, 0) = 1.48;
mat (3, 0) = 1.54;
mat(4, 0) = 2.0;
mat(5, 0) = 2.25;
mat(6, 0) = 2.60;
 Multiple and Polynomial Regression 281

MeLte (he Ol) es 2 Se

jireen Hehe (ON > Soars
mats (9). 0) = sia54
mat(10, 0) = 4.0;

mats (Ol) 0}
Miieute (ley isl) ieeee clare
Mavteieaye do) ue bree
Tree (she) APR BRA sto
riveKey (be ly) =e aba dlp
mati(S, 0) = 15>
Ma ti6 ee — ele. Gi
BRULr. HE Ea bear
Mat. Cosme Lees el. Bic
ioe (Cl ab) ea eee)
rishen((GO) Gili) Gat t= Oo.

numData = 11;
numTerms = 2
YIndex = 0;
XIndex = 1

r.InitializePolyReg(numTerms, linear, linear, linear,

XIndex, YIndex, FALSE, 0);
r.CalcSumPolyReg(mat, numData) ;
r.CalcPolyReg() ;
PEINtE ("***x***xexX**X POlynomial Regression **********\n") ;
printf ("Number of points: %lg\n", r.sum);
Prince ("RA~2 o=) S64 oN eR)
DreInt. (Adywusted R\2s—= sO-40t\n" . rt. R2ady))>
printf ("Intercept = %$lg\n", r.Intercept) ;
for (i = 1; i <= numTerms; i++)
DEIntce ("Slope form Kedeae elo Nie ree
|al)
printf ( WKEKKKKEKKKKKEKKEK ANOVA RREKKKKKER\ NW) :

printf("Regression SS = %lg\n", r.ANOVA.Reg_SS) ;

printf("Regression df = %lg\n", r.ANOVA.Reg_df);
printf("Residual SS = %lg\n", r.ANOVA.Residual_SS) ;
printf("Residual df = %lg\n", r.ANOVA.Residual_df) ;
print£ ("Total SS = %ig\n", r.ANOVA.Total_SS) ;
printf("Total df = %lg\n", r.ANOVA.Total_df) ;
Drint£(*S*2 = tig\n", x..ANOVA.S2) 7
printf("F = %lg\n", r.ANOVA.F) ;

pressAnyKey
() ;

probability = 95;
r.PolyRegCoefCI (probability, slopeHi, slopeLow) ;
printf("At lg %% probability\n", probability) ;
for (i = 1; i <= numTerms; i++)
printf£("Range for slope of X%d is %lg to %lg\n",
i, slopeLow[i], slopeHi[i]);

pressAnyKey
() ;

Bom ia) ee <= numTermsi: i++) {

swatoh (2) -{
case 1:
testValue = 0.0;
break;

case 2:
testValue = 1.0;
break;
}

xr.PolyReg_Slope_T_test (probability, testValue, i,

calcT, tableT, passTest) ;
282 Chapter Eleven

Listing
11.12 (Continued)

printf("At %lg %% probability\n", probability) ;

print’ ("HOs Sigi?=? Sig, ") >. Biill, testvalue):
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n");
}
pressAnyKey
() ;

r.PolyReg_R2_T_Test (probability, calcT, tableT, passTest) ;

printf("At %lg %% probability\n", probability) ;
PLANE (MAO Fi geais: Not Oi wy, ea aR)
if (passTest)
printf("cannot be rejected\n");
else
printf("cannot be accepted\n");

pressAnyKey
() ;

xX = mat(1,.0);

r.PolyRegYHatCI (probability, X, yHatLow, yHat, yHatHigh) ;

printf("At %lg %% probability given:\n", probability);
joshHe (ES.¢ =, tale Nua s ao.o
We
printf("Y* = %lg\n", yHat);
printf("Range for Y* is lg to $%lg\n",
yHatLow, yHatHigh) ;

pressAnyKey
() ;

return 0;
}
void pressAnyKey()
{
printf("\nPress any key to continue...");
getchar();
puts("\n\n");
}

double linear(double x)
{
return x;
}
 Chapter

12
The Functions Library

This chapter presents libraries of statistical and mathematical functions. These func-
tions include the following:

=# The normal probability distribution function and its inverse function

# The Student-t probability distribution function and its inverse function
# The Chi-square probability distribution function and its inverse function
# The Snedecor F probability distribution function and its inverse function
# The factorial function
# The combination function
=» The permutation function
» The gamma function
# The beta function
# The error function
# The error integral function
= The sine integral function
= The cosine integral function
= The Laguerre function
=» The Hermite function
= The Chebyshev function
« The Bessel functions of the first and second kind

283
284 Chapter Twelve

The STATLIB Library

The first library of functions that I present is STATLIB. This library has eight special
functions that calculate the main four probability distribution functions and their in-
verses. Listing 12.1 shows the source code for the header file STATLIB.H. Listing
12.2 contains the source code for the file STATLIB.C. I mention these files here es-
pecially because the implementation file contains the values of the constants used in
the various statistical functions that I present next.
When using the statistical functions with C++ programs, you need to save an ex-
act copy of the file STATLIB.C as file STATLIB.CPP. Both files can use the same
header file STATLIB.H without getting error messages from the compiler or linker.

The Normal Distribution

The normal distribution function implemented in the statistical library calculates the
normal integral between minus infinity and a specified value x. The equations used
for the calculations are:

QG@)=Rik <0
=1-RifR20 (12.1)

R= iG), te bt? bot pat) (122)

1
i (+b, xl) (12.3)

e(—x?/2)
f(x) aVem (12.4)
12.4

where b, through b, are constants whose values appear in Listing 12.2.

The Inverse Normal Distribution

The inverse normal distribution function implemented in the statistical library uses
the following equations:

OaP,(t)
SSS (12.5)
3

Pi) Cy + Cp bee (12.6)

P(t) =1+d,t+d,t? + d,t? (12.7)

t= Jeo(&) for0<Q<0.5 (12.8)

 The Functions Library 285

4 [on{ os )0" seo el

Where c, through c, and d, through d, are constants which appear in Listing 12.2.

The Student-t Distribution

The Student-t distribution function implemented in the statistical library calculates

the distribution integral between minus infinity and a specified value x. The equa-
tions used for the calculations are:

T(x,n) = 1-Q(t) (12.9)

V(14+x)/(2n) (12.10)

where n is the degrees of freedom.

The Inverse Student-t Distribution

The inverse Student-t distribution function implemented in the statistical library

uses the following equations:

FO) VEO eo) ae

T'(an) =q+ + + + (21)
n n? n? n*

q= Q's) (12.12)
where g,(q) through g,(q) are polynomials which contribute in approximating the
value of the inverse Student-t distribution. Listing 12.2 contains the definition of
these polynomials. The variable n represents the degrees of freedom.

The F Distribution
The statistical library approximates the F distribution using the following equations:
f
F(x, n,, n,) =1-o( | (12.13)
2

pees eee (12.14)

Np Bt

k k -
{2 = & + 5 =| (12.15)
286 Chapter Twelve

where k = S,and nl and n2 are the degrees of freedom.

The Inverse F Distribution

The statistical library approximates the inverse F distribution using the following
equations:

HelORE aS i (12.16)

7 | (B? —AAC))
r=(-B+ (he C210

A o7 ae (12.18)

B=-2a8 (12.19)

C= f?-_q? (12.20)

C= ak (12.21)
Ny

1-k
b= ny, (12.22)

t=1-8 (12.23)

A=1-a (12.24)

q = Q'(x) (12.25)

where k = =,and n, and n, are degrees of freedom.

Listing 12.1 The source code for the header file STATLIB.H.
#ifndef _STATLIB
H_
#define _STATLIB_H_

double Q(double x);

double QInv(double x);
double T(double x, double df);
double TInv(double x, double df);
double ChiInv(double x, double df);
double F(double x, double df1, double df2);
double FInv(double x, double dfl1, double df2);

#endif
 The Functions Library 287

Listing 12.2 The source code for the implementation file STATLIB.C.
#include <math.h>
#include "statlib.h"
#include "global.h"

double Q(double x)
/* Function that returns the "Normal"
probability distribution integral */
{
const twoPi = 6.283185308;

double result;
double sum; /* summation */
double xp; /* x raised to an integer power */
double tempo;
/* Array of coefficients */
double b[5] = { 0.319381530, -0.356563782,
1.781477937, -1.821255978,
1.330274429 };
Emits ae

tempo = 1,0) 5/ a(eL0n + 0n2376419" =e £abs (sc):)l;

/* Initialize summation */
sum = 0.0;
ago) t= AWS (O)r
/* Loop to obtain summation term */
ona (Cal) “(Oa Sy Ee ey
xp *= tempo; /* Update power factor */
SUM p+= Deen tax
}
/* Calculate result */
result (exp(=x"* x / 2.70) / sqrt (twoPi) * sum);
nesulto= (x >=)000)) 20) — result) 7 result.
return result;
}
double QInv(double x)
/* calculate the inverse normal. */
{
double result;
double sumi, sum2; /* Two summations used */
double tempo, xp;
rgiew ali?
/* First and second coefficient array */
Coupler ciz imitans > 57 ORG Uac oar
Om OOS 28%, OF. jis
couble ai4i =f 1.0) 912432788, 0. 1892697 = 0-'0013:081}7

/* Check limits of x */
Eaa(G< =) (040)
=? 0), 00015:
im (Gis “ss=P al 310)
x = 0.9999;
ance (exe <— 10)5'5))
tempo = sqrt(log(1.0/SQR(x)));
else
tempo = sqrt(log(1.0/SQR(1.0 - x)));

/* Initialize summations */
suml = 0.0;
sum2 = 0.0;
xp) = 107
/* Start loop to calculate summations */
288 Chapter Twelve

Listing 12.2 (Continwed)

Oa (ee oA otic)
sumly—se[2 + "xp
Syerny) dhe islf[Stull 2 Sige)
xp *= tempo; /* Update power factor */
}
/* Calculate the result */
result = tempo - suml / sum2;
result = (x > 0.5) ? -result : result;
return result; /* Return sought value */
}
double T(double x, double df)
/* Function will return the probability of
obtaining a student-t statistic, x, at
df degrees of freedom. *9/)

double xt;

xe se ~ (COR O25 a/ aa)

Sqre(t- 0. SOR (Gc)ip e/a 2. On / Ge)
return (1.0 - Q(xt));
}

double TInv(double x, double df)

/* double will return the value of student-t statistic for a
given probability, x, and df degrees of freedom. “oy
{
double sum, xp, xq;
double Pwr[10]; /* Array of powers of xq */
double term[5]; /* Array of terms */
eDritoe ine

(= Cheeks Tamms motescs 7,

LE) (2 <= 050)
x = 0.0001;
ies (tie ke 0)
be = (SINE

xq = QInv(x);
Pwr[l] = xq>
/* Loop to obtain the array of powers */
for (G4 ts525) 3 <= 9s) ie)
Pwr [i] = Pwr[i=1] * xq;

/* Calculate the four terms */

term(TJ_ = 0.25 * (Pwr[3] +5 Pwrilil!);
term[2] = (5*Pwr[5] + 16*Pwr[3] + 3*Pwr[1])/96;
term[3] = (3*Pwr[7] + 19*Pwr[5] + 17*Pwr[3] -
15*Pwr[1])/384;
term[4] = (79*Pwr[9] + 776*Pwr[7] + 1482*Pwr[5] -
1920*Pwr[3] - 945*Pwr[1]) /92160.0;
/* Initialize summation and power factor */
sum = xq;
Be) indie
/* Loop to add terms */
fork (aba Uv Lt RSS Se aye
xp *= df; /* Update df power factor */
sum += term[i] / xp;
}
return sum;
}

double Chi(double x, double df)

/* Function will return the probability of obtaining a
 The Functions Library 289

chi-squared statistic, x, at df degrees of freedom. */

double k, xq;

Ks= 26.0) /99..00/ ‘aE;

xq = (pow ( (ac/dé)y7 0233333) — (1.0 = 1))/ sqrti(k);
return (1.0 - Q(xq));
}
double ChiInv(double x, double df)
/* double will return the value of chi-squared statistic for a */
/* given probability, x, and df degrees of freedom. */

{
double k, tempo, xq;

/* Check limits of x */
Bhs (eee =a O)e.0))
OO OO Mus
2 fe (x >= 0)
x = 0.9999;

Ke t= 2.007 eo ROr
xq = QInv(x);
tempo = 1.0" = kK / dE + xq * ‘sqrre(k / d£));
tempo = df * CUBE(tempo) ;
return tempo; /* Return sought value */
}
double F(double x, double df1, double df2)
/* Function will return the probability of obtaining an
F statistic, x, at df degrees of freedom. mf

{
double f1, £2, k;

koe Oe 29! OF
f= POW (EX) OS S55 Smee ele a Kae) SCE) Ui(Ge ka Clie)
£2°= sqrt(k / dil pow(x, 0.666667) * k / dé2);
return <1.0 © (Eley EZ)
}

double FInv(double x, double df1, double df2)

/* Function will return the value of the F statistic for a
given probability, x, and df degrees of freedom. fh
{
double alpha, beta, gamma, delta,
Ave ByiGr Us eresu ite, 20cs

77 Check jaimits of es 27)

fee 1O!.10' )
x = 0, 0001;
ate Wire Sr ale) 4)
x — 10.9999)

k=" 2).0) £ 970;

xq = QiInv (x);
alpha =e Kiss Gei2yie
beta = (1 — k /dilys
gamma = 1/210) —. (laa keel).
delta = 1.0). — (tien dr2)>
A = SQR(alpha) - SQR(xq) * delta;
B = =2..0 * alpha * beta;
C = SQR(beta) - SQR(xq) * gamma;

Tesuler= (= J06* iB ce sarie( soe) Bee Ole A eC) ) 7 (2.0) * A);

return CUBE(result) ;
290 Chapter Twelve

The Mathematical Functions Library

In this section I present the mathematical functions library, which contains popular
math functions. Listing 12.3 shows the source code for the MATHLIB.H header file.
Listing 12.4 contains the source code for the MATHLIB.H header file. The library
supports the factorial, combination, permutation, gamma, beta, error, sine integral,
cosine integral, Laguerre, Hermite, Chebyshev, and Bessel functions. The imple-
mentation of most of the above functions uses an iterative and/or approximative ap-
proach. The implementations of the factorial, combination, permutation, Laguerre,
Hermite, and Chebyshev functions yield exact values for these functions. By con-
trast, the implementations of the remaining functions are approximate, and based on
working with infinite series polynomials that approximate the functions.

The Combination and Permutation Functions

The implementation of the factorial function uses a for loop that evaluates the facto-
rial function which is defined by the following equation:

n}=n(n-1)(M-2)...21 (12.26)

The following equations define the combination and permutation functions, re-
spectively:

m!
Gee SS 12.27
7 (oo) in ( )

m!
ae 12.28
= Cn esi! ( )

The Gamma and Beta Functions

The implementation of the gamma uses a series expansion given by the following
equation:

1
———
We) = ic,x*
CX (12.2 9)

for k = 1 to 26

where c, through c,, are constants shown in Listing 12.4.

The beta function uses the gamma function to yield a value, as shown below:

T(z) [Cw
B(z,w) = hie) TN (12.30)
(z+ w)

The Error Function

The error function has several approximations. The mathematical library MATHLIB
uses the following one:
 The Functions Library 291

erf(x) = 1-(a,t + a,t? + a,t? + a,t4 + a,t?) e¥ (12.31)

opel ance 1 se t (12.32)

(1 + 0.32759 x)
where y = x” and a, through a, are constants shown in Listing 12.4.

The Sine and Cosine Integral Functions

The mathematical library calculates the values for the sine and cosine integrals us-
ing the following series expansions:
(eliexe |
SiCx) = > onan) (12.33)

:
Ci(x) ae In
=t+n@) +> ea
ere
(On Gn]
Gigs (12.34)

where T is the Euler constant. The above summations are taken for n = 1 and upward
(until the absolute value of the evaluated term falls below a minimum limit).p;

The Laguerre, Hermite, and Chebyshev Polynomials

The mathematical library offers functions to evaluate the Laguerre, Hermite, and
Chebyshev polynomials. The code uses loops to calculate these polynomials based
on recursive formulas. The recursive equations for the Laguerre polynomial are:

L)(x) = 1 (12.35)

L,@&=1-x (12.36)

Lj@ = 0+ 2n-x)L,@) -n? L@&) (12737)

The recursive equations for the Hermite polynomial are:

Hy) =1 (12.38)

H, (x) = 2x (12.39)

H.i@X) = 2xH,, Cx) + 2nH,_,@) (12.40)

As for the Chebyshev polynomial, the mathematical library uses the following non-
recursive equation to evaluate the polynomial:

TGs) = cos(n cos x) (12.41)

292 Chapter Twelve

The Bessel Functions

The mathematical library implements the Bessel functions of the first and second
kinds. The following equation provides a series expansion for the Bessel function of
the first kind:

h(a (-z2/4)*Se
a) (12.42)
TG= G2) > TirGekeDI
The above summation is taken for k = 1 and upward (until the absolute value of
the evaluated term falls below a minimum limit).
The mathematical library evaluates the Bessel function of the second kind using
the following equation:
‘ [J,Cx) cos(nm) — J_(x)]
YC) (12.43)
~~ sin(nt)

Listing 12.3 The source code for the MATHLIB.H header file.

#ifndef _MATHFUN_H_
#define _MATHFUN_H_

double factorial(int n);

double combination(int m, int n);
double permutation(int m, int n);
double gamma(double x);
double beta(double z, double w);
double erf(double x);
double Ei(double x);
double Si(double x);
double Ci(double x);
double Laguerre(double x, int n);
double Hermite(double x, int n);
double Chebyshev(double x, int n);
double BesselJ(double z, int v);
double BesselY(double z, int v);

#endif

Listing 12.4 The source code for the MATHLIB.C implementation file.

#include <math.h>
#include "mathlib.h"

double factorial(int n)
{
double result = 1;
unsigned i;

tf (es = 0),
for (i = Of 2 <= ne ree)
result *= (2 0) WV;

return result;
}
double combination(int m, int n)
{
double factM = 1;
 The Functions Library 293

double factMN = 1;
double factN = 1;
aigghes a8

ie (en Sm ||| << © ||) ae = ©)

return -1; /* error code */

pr caleulat ey (ml — (7) ie

for (3. = Of oe<= (m=) n)ie a+)
factMN *= (i > 0) ? (double)i

factM = factMN;
/* calculate m! */
Oi mi(ae = ms ry ss! a ye)
factM *= (double)i;

/* caleulate: mil) *7/,

form (a= 105) Lime aa)
factN *= (i > 0) ? (double)i

return factM / factMN / factN;

}
double permutation(int m, int n)
{
double factM = 1;
double factMN = 1;
wrt, as

sem (Git = ra || eat <= ©, ||| va <= 0)

return -1; /* error code */

// calculate (m - n)!
for (a = ie 2 <=) (ml Seniesa s+)
factMN = (19 Oe (double):
factM = factMN;
// calculate m!
for (4 =m =n + 1; 2 <= mz i++)
factM *= (double)i;

return factM / factMN;

}
double gamma(double x)
{
double c[27] =
{ 0.0, 1.000000000,
ORS57 7215166497 O55 87 80 115),
-4.2002635034e-2, 0.16665386113,
-4.21977345555e-2, ~IMOZL ILS A 8e—S%,
7.21894324666e-3, =i LOSt6/ 598 59te=37
-2.152416741149e-4, 1.280502823882e-4,
-2.01348547807e-5, -1.2504934821e-6,
1.1330272320e-6, =2 05633841 7e-7,
6.1160950e-9, 5 O00 20075e—9),
-1.1812746e-9, 1.043427e-10,
7.7823e-12, -3.6968e-12,
5.1le-13, -2.06e-14, -5.4e-15, 1.4e-16, le-16 };

double sum = 0;
shige; she

Hometime 26), i> Ul a)

sum = (sum + e[il) * x;

return 1 / sum;
}
double beta(double z, double w)
{
294 Chapter Twelve

Listing 12.4 (Continued)

return gamma(z) * gamma(w) / gamma(z + w);

}
double erf (double x)
{
Gdouble t = 1 f (Glee 0.32759. * ae)
double a[6] = { 0, 0.254829592, =-0.284496736,
1.424143741, -1.453152027,
1.061405429 };
double sum = 0;
2m ays

forme -= Sse O ps =)
sum = (sum + a[i]) * €;

return 1 - sum / exp(x * x);

}

double Ei(double x)
{
const double tolerance = 1.e-8;
double sum = 0;
double term;
double pow = x;
double fact = 1;
Sat, yay =a AG

do {
term = pow / nm / fact;
sum += term;

/* update components of the next term */

oe 7
fact *= (double)n;
pow *= x

} while (fabs(term) > tolerance) ;

return 0.5772156649 + log(x) + sum;
i
double Si(double x)
{
const double tolerance = 1.e-8;
double sum = 0;
double term;
double pow = x;
double chs = 1;
Pte) = eel,

do {
term = chs * pow / {2 * m + i) / facterralt2 © 3a, C4 ta)
sum += term;

/* update components of the next term */

Tiras

chs = -chs;
pow *= x * x;

} while (fabs(term) > tolerance);

return sum;

}
double Ci(double x)
{
const double tolerance = 1.e-8;
double sum = 0;
 The Functions Library 295

double term;
double pow = x * x;
double chs = -1;
Int. no= ae
do {
Germi— Chis DOW W/ m(2uetn)) eetactordaln(2 a Aa 1!)ir
sum += term;

/* update components of the next term */

relapse

chs = -chs;
pow *= x * se;

} while (fabs(term) > tolerance) ;

return 0.5772156649 + log(x) + sum;
}
double Laguerre(double x, int n)
{
double LO = 1;
double Li = 1 - x;
double L2;
ant a = 2%

ath |{Gaye cea eyp)

return -1;

ane (Gar =)
return LO;
GuuGKey SIGE {Gall = ah)
return L1;
else {
while (i <n) {
C2 (ol eto a =) Tend ec TPO) Tg)
LO = Li;
Li = 12
i++;

}
return L2;

}
double Hermite(double x, int n)
{
double HO = 1;
double H1 PH geo er
double H2;
rte, ae = 2s

eleteem (rat <7!)

return -1;

sian ==" 10!)

return HO;
elise af (n == 1)
return H1;
else {
while (i <n) {
RPA RS 2) Wo e's OSE Se ee al LSS he
EO =a
Hi =" HF
L++;
}
return H2;
296 Chapter Twelve

Listing 12.4 (Continued)

double Chebyshev (double x, int 7)

{
double TO ave
double T1 x;
double 12;
Ghat: ah Ei AG

sien (a0)
return Zales

ale (Gel se {0)))

return TO;
else 1£ (n == 1)
return Diy
else {
while (i < n) {
2 eo ee Lee Oe
TOD a ras
A ea Ds
i++;
}
return 72;

}
double Legrendre(double x, double n)
{
double LO Ly
double L1 ah
double L2;
THiS 2

nine Gey =< (0)))

return -1;

if (n == 0)
return LO;
else if (a= 1)
return Dil:
else {
while (i <n) {
Tic) (2 ae a) ee a 1) ie (ee
LO = Lil;
L1 = 12;
i++;

}
return L2;

}
double BesselJ(double z, int v)
{
const double tolerance = 1.0e-8;
double sum = 0;
double term;
okey ak — f6hr

do {
term = pow =—z, * 2/24, (double)i) /
factorial(i) / gamma(v + i + 1);
i++;
} while (fabs (term) > tolerance) ;

return pow(z / 2, v) * sum;

 The Functions Library 297

double BesselY(double z, int v)

{
double pi = 4 * atan(l1);

return (BesselJ(z, v) * cos(v * pi) - BesselJ(z, -v))

ji ‘ssaliaulyy +2 Gest)p
 [dyot)) Fiewsed Sida
}
is'a
se * Sg elcid

_dageet img ‘Late ‘Yat ieaert) (Res

rita vials

“« Ul

7
’

u
 index

() operator, 7, 18 B
[] operator, 7 Backward difference method, 75-76
Barycentric function, 67, 71
A Barycentric interpolation, 62-64
AddMat function, 9 BASTAT.H, 142-143
AltExtSimpson function, 95, 101 BASTAT.C, 144-146
analysis of covariance (see ANOCOY) BASTAT.CPP, 148-150
analysis of variance (see ANOVA tests) BASTAT.HPP, 147-148
ANOGOV, 169-172 BestFit function, 211, 216
ANOCOV function, 176 Birge-Vieta method, 38-39
ANOVA tests, 159-201 Bisection function, 43, 54
Latin-Square, 167-169 Bisection method, 34, 119-120
one-way, 159-161 combined with Newton’s method, 37
source code, BisectionMin function, 125, 134
ANOVA.C, 176-182 Brent function, 44, 54
ANOVA.CPP, 191-197 Brent’s method, 37
ANOVA.H, 173-175
ANOVA.HPP, 188-191
TSANOVA.C, 183-187 C
TSANOVA2.CPP, 198-201 CalcMultiReg function, 236, 251, 256
two-way with replication, 164-166 CalcPolyReg function, 260
two-way, 162-164 CalecSumMLR function, 235-236, 251, 256
ANOVA.C, 176-182 CalcSumPolyReg function, 260, 275, 279
ANOVA.CPP, 191-197 CalcYSimplex function, 126
ANOVA.H, 173-175 CDiffDerivl function, 78, 84
ANOVA.HPP, 188-191 CDiffDeriv2 function, 78, 84
ANOVAI function, 175 CDiffDeriv3 function, 78, 84
ANOVA2 function, 176 CDiffDeriv4 function, 78, 84
ANOVA2R function, 176 Central difference method, 76
ARRAYS.C, 3-4 checkIndex function, 5
ARRAYS.H, 2-3 checkRowCol function, 5, 7
ARRAYS.HPP, 5-6 Combined function, 43, 54

299
300 Index

confidence interval, 140 Forward difference method, 75-76

for projections, 206 fSimpson function, 95, 101
for regression coefficients, 206-207 functions (see specific names of)
CopyMat function, 9 functions library, 283-297
CubelIntMin function, 125 MATHLIB, 290-297
Cubic interpolation, 122-123 STATLIB, 284-289
Cubic spline interpolation, 65-66
CUdfDerivl function, 79, 84
CUdfDeriv2 function, 79, 84 G
CUdfDeriv3 function, 79, 84 GaussChebyshevQuadrature function, 96, 101
Gauss-Chebyshev quadrature method, 93
CUdfDeriv4 function, 79, 84
GaussHermiteQuadrature function, 96, 101
Gauss-Hermite quadrature method, 93
D GaussJordan function, 9, 16
DeflatePolyRoots function, 44, 54 Gauss-Jordan method, 7
deleteIntVect function, 5 GaussLaguerreQuadrature function, 96, 101
deleteMat function, 4 Gauss-Laguerre quadrature method, 92
deleteMLRegrec function, 237 GaussLegendreQuadrature function, 96, 101
deletePolyRegrec function, 261 Gauss-Legendre quadrature method, 91-92
deleteVect function, 5 GaussSeidel function, 9, 16
DERIV.C, 79-84 Gauss-Seidel method, 7
DERIV.H, 77-78 Gaussian gradrature method, 91
differential equations, 105-118 getCols function, 7
Runge-Kutta method, 105-106 getDerivatives function, 67
Runge-Kutta-Fehlberg method, 107-108 getMeanSdev function, 143
Runge-Kutta-Gill method, 106-107 getRows function, 7
source code, getYTest function, 126
ODE.G, 110-113 GLOBAL.H, 1-2
ODE.H, 108-109 GoldenSearchMin function, 125, 134
TSODE.C, 114-117 Golden Section Search method, 121

E I
ED_Barycentric function, 67, 71 InitializeBasicStat function, 148
equations InitializeLinReg function, 210, 216
differential, 105-118 InitializeMLR function, 235-236, 251, 255
linear, 1-31 InitializePolyReg function, 260, 275, 278
nonlinear, 33-60 INTEGRAL.C, 96-101
ExNewtonMin function, 126
INTEGRAL.H, 94-95
Extended Central difference method, 77
INTERP.C, 67-71
INTERP.H, 66-67
F interpolation, 61-73
fAltExtSimpson function, 95, 101 Barycentric, 62-64
FBDiffDeriv1 function, 78, 84 Cubic, 122-123
FBDiffDeriv2 function, 78, 84 Cubic spline, 65-66
FBDiffDeriv3 function, 78, 84 Lagrangian, 61-62
FBDiffDeriv4 function, 78, 84 Newton’s difference, 64-65
FBUdfDeriv1 function, 79, 84 Newton’s divided difference, 64
FBUdfDeriv2 function, 79, 84 quadratic, 121-122
FBUdfDeriv3 function, 79, 84 source code,
FBUdfDeriv4 function, 79, 84 INTERP.H, 66-67
Fibonacci numbers, 121 INTERP.C, 67-71
first four moments, 140-141 TSINTERP.C, 72-73
 Index 301

IntVector class, 6 M
IntVector::operator[ ]function, 18 MatDeterminant function, 10
MATHLIB library, 290-297
L Bessel functions, 292-297
Lagrange function, 67, 71 beta functions, 290
Chebyshev polynomial, 291
Lagrangian interpolation, 61-62
combination functions, 290
Latin function, 176
error functions, 290-291
LBPolyRoots function, 44, 54
gamma functions, 290
Lin-Bairstow method, 39-40
Hermite polynomial, 291
linear equations, 1-31
Laguerre polynomial, 291
Gauss-Jordan method, 7
permutation functions, 290
Gauss-Seidel method, 7
sine and cosine integral functions, 291
LU decomposition method, 7
source code,
source code,
MATHLIB.C, 292-297
ARRAYS.C, 3-4 MATHLIB.H, 292
ARRAYS.H, 2-3 MATHLIB.C, 292-297
ARRAYS.HPP, 5-6 MATHLIB.H, 292
GLOBAL.H, 1-2 MatInverse function, 10
MATVECT.C, 10-16 matrices, 2-7
MATVECT.CPP, 18-24 Matrix class, 7
MATVECT.H, 7-8 Matrix::operator( ) function, 18
MATVECT.HPP, 17-18 MATVECT.C, 10-16
TSMAT.C, 25-27 MATVECT.CPP, 18-24
TSMAT.CPP, 29-31 MATVECT.H, 7-8
vectors and matrices, 2-7 MATVECT.HPP, 17-18
linear regression, 203-229 mean and standard deviation, 139-140
algorithms, 207-208 meanCl function, 148
basic, 203-205 meantTest function, 144
confidence interval, 206-207 MLREG.C, 238-243
linearized, 205 MLREG.CPP, 245-250
source code, MLREG.H, 237-238
LINREG.C, 211-215 MLREG.HPP, 244-245
LINREG.CPP, 222-226 MLR_R2_T_test function, 236, 251, 256
LINREG.H, 209-210 MLR_Slope_T_test function, 236, 251, 256
LINREG.HPP, 220-222 moment function, 144
TSLINREG.C, 216-219 multiple regression, 231-259
TSLINREG.CPP, 227-229 source code,
LinReg function, 210 MLREG.C, 238-243
LINREG.C, 211-215 MLREG.CPP, 245-250
LINREG.CPP, 222-226 MLREG.H, 237-238
LINREG.H, 209-210 MLREG.HPP, 244-245
LINREG.HPP, 220-222 TSMLREG.C, 251-254
LinRegCoefCl function, 211, 216 TSMLREG.CPP, 256-259
LR_Int_T_test function, 211, 216
MultiRegCoefCl function, 236, 251, 256
MultMat function, 9
LR_R2_T_test function, 211, 216
LR_Slope_T_test function, 211, 216
LU decomposition method, 7 N
LUBackSubst function, 9, 17 newlnVect function, 5
LUDecomp function, 9, 16 newMat function, 4
LUDeterminant function, 10 Newton2Functions function, 44, 54
LUInverse function, 9 NewtonApprox function, 43, 54
302 Index

NewtonDiff function, 67, 72 O

NewtonDivDiff function, (ONG (all ODE.C, 110-113
NewtonExact function, 43, 54 ODE.H, 108-109
NewtonMin function,
Aisi
125,
d
134 operators
NewtonMultiMin function, 126, 134 (), 7, 18
NewtonMultiRoots function, 44, 54 t] 7 :
Newton's difference interpolation, 64-65
OPTIM.C, 127-134
Newton's divided difference interpolation,
OPTIM.H, 126-127
64 optimization, 119-138
NewtonSimNLE function, 44, 55
Bisection method, 119-120
Newton’s method, 34-36, 41, 120-121, 125
Cubic interpolation method, 122-123
combined with Bisection method, 37
Golden Section Search method, 121
Newton’s multiroot method, 37-38
Newton’s method, 120-121
newVect function, 5
Newton’s method for sequential, 125
nonlinear equations, 33-60
OPTIM.C, 127-134
Birge-Vieta method, 38-39
quadratic interpolation method, 121-122
Bisection method, 34
simplex method, 123-125
Brent’s method, 37
source code,
combining Bisection and Newton’s methods,
OPTIM.H, 126-127
37
TSOPTIM.C, 135-137
Lin-Bairstow method, 39-40
Newton’s method, 34-36, 41
Newton’s multiroot method, 37-38 P
Richmond method, 36-37 polynomial deflating, 38-39
solving multiple, 40-41 polynomial regression, 259-282
source code, source code,
ROOT.C, 44-53 POLYREG.C, 263-268
ROOT.H, 41-48 POLYREG.CPP, 270-275
TSROOT.C, 55-60 POLYREG.H, 262-263
numerical differentiation, 75-87 POLYREG.HPP, 269-270
Backward method, 75-76 TSPOLYRE.C, 276-278
Central method, 76 TSPOLYRE.CPP, 280-282
Extended Central method, 77 POLYREG.C, 263-268
Forward method, 75-76 POLYREG.CPP, 270-275
source code, POLYREG.H, 262-263
DERIV.C, 79-84 POLYREG.HPP, 269-270
DERIV.H, 77-78 PolyRegCoefCl function, 261, 268, 275, 280
TSDERIV.C, 84-87 PolyRegYHatCl function, 261, 268
numerical integration, 89-103 PolyReg_R2_T_test function, 261, 268, 276,
Gauss-Chebyshev quadrature method, 93 280
Gauss-Hermite quadrature method, 93 PolyReg_Slope_T_test function, 261, 268,
Gauss-Laguerre quadrature method, 92 275, 280
Gauss-Legendre quadrature method, 91-92
Gaussian quadrature method, 91
Romberg method, 93-94
QuadIntMin function, 125, 134
Simpson’s alternate extended rule, 90-91
quadratic interpolation, 121-122
Simpson’s method, 89-90
source code,
INTEGRAL.C, 96-101
INTEGRAL.H, 94-95 regression
TSINTEG.C, 102-103 linear, 203-229
 Index 303

multiple, 231-259 MATVECT.H, 7

polynomial, 259-282 MATVECT.HPP, 17-18
RichmondApprox function, 43, 54 MLREG.C, 238-243
RichmondExact function, 43, 54 MLREG.CPP, 245-250
Richmond method, 36-37 MLREG.H, 237-238
Romberg function, 96 MLREG.HPP, 244-245
Romberg method, 93-94 ODE.C, 110-113
ROOT.C, 44-53 ODE.H, 108-109
ROOT.H, 41-43 OPTIM.C, 127-134
RungeKutta4 function, 109, 114 OPTIM.H, 126-127
RungeKuttaFehlberg function, 109, 114 POLYREG.C, 263-268
RungeKuttaGill function, 109, 114 POLYREG.CPP, 270-275
Runge-Kutta method, 105-106 POLYREG.H, 262-263
Runge-Kutta-Fehlberg method, 107-108 POLYREG.HPP, 269-270
Runge-Kutta-Gill method, 106-107 ROOT.C, 44-53
ROOT.H, 41-43
STATLIB.C, 287-289
S STATLIB.H, 286
sdevCl function, 143
TSANOVA.C, 183-187
simplex function, 126, 134
TSANOVA2.CPP, 198-201
simplex method, 123-125
TSBASTAT.C, 152-154
Simpson function, 95, 101 TSBASTAT.CPP, 155-157
Simpson’s alternate extended rule, 90-91 TSDERIV.C, 84-87
Simpson’s method, 89-90 TSINTEG.C, 102-103
source code TSINTERP.C, 72-73
ANOVA.C, 176-182 TSLINREG.C, 216-219
ANOVA.CPP, 191-197 TSLINREG.CPP, 227-229
ANOVA.H, 173-175 TSMAT.C, 25-27
ANOVA.HPP, 188-191 TSMAT.CPP, 29-31
ARRAYS.C, 3-4 TSMLREG.C, 251-254
ARRAYS.H, 2-3 TSMLREG.CPP, 256-259
ARRAYS.HPP, 5-6 TSODE.C, 114-117
BASTAT.C, 144-146 TSOPTIM.C, 135-137
BASTAT.H, 142-143 TSPOLYRE.C, 276-278
BASTAT.CPP, 148-150 TSPOLYRE.CPP, 280-282
BASTAT.HPP, 147-148 TSROOT.C, 55-60
DERIV.C, 79-84 Spline function, 67, 72
DERIV.H, 77-78 statistics, 139-157
GLOBAL.H, 1-2 ANOVA, 159-201
INTEGRAL.C, 96-101 BASTAT.CPP, 148-150
INTEGRAL.H, 94-95 confidence interval, 140, 206
INTERP.C, 67-71 first four moments, 140-141
INTERP.H, 66-67 mean and standard deviation, 139-140
LINREG.C, 211-215 source code,
LINREG.CPP, 222-226 BASTAT.C, 144-146
LINREG.H, 209-210 BASTAT.H, 142-143
LINREG.HPP, 220-222 BASTAT.HPP, 147-148
MATHLIB.C, 292-297 TSBASTAT.C, 152-154
MATHLIB.H, 292 TSBASTAT.CPP, 155-157
MATVECT.C, 10-16 STATLIB library, 284-289
MATVECT.CPP, 18-24 F distribution, 285-286
304 Index

STATLIB library continued TSMLREG.C, 251-254

inverse F distribution, 286 TSMLREG.CPP, 256-259
inverse normal distribution function, 284- TSODE.C, 114-117
285 TSOPTIM.C, 135-137
inverse student-t distribution function, TSPOLYRE.C, 276-278
285 TSPOLYRE.CPP, 280-282
normal distribution function, 284 TSROOT.C, 55-60
source code,
STATLIB.C, 287-289
STATLIB.H, 286
Vv
Vector::operator| ] function, 18
student-t distribution function, 285
Vector class, 7
STATLIB.C, 287-289
vectors, 2-7
STATLIB.H, 286
VectRungeF ehlberg function, 110, 114
SubMat function, 9
VectRungeKutta4 function, 109, 114
sumLinReg function, 210, 216
VectRungeKuttaGill function, 109, 114

T
tridiagonal function, 67
xX
XCDiffDeriv1 function, 78, 84
TSANOVA.C, 183-187
XCDiffDeriv2 function, 78, 84
TSANOVA2.CPP, 198-201
XCDiffDeriv3 function, 78, 84
TSBASTAT.C, 152-154
XCDiffDeriv4 function, 78, 84
TSBASTAT.CPP, 155-157
XCUdfDeriv1 function, 79, 84
TSDERIV.C, 84-87
XCUdfDeriv2 function, 79, 84
TSINTEG.C, 102-103
XCUdfDeriv3 function, 79, 84
TSINTERP.C, 72-73
XCUdfDeriv4 function, 79, 84
TSLINREG.C, 216-219
TSLINREG.CPP, 227-229
TSMAT.C, 25-27 Y
TSMAT.CPP, 29-31 YhatCI function, 210, 236
ABOUT THE AUTHOR
Namir C. Shammas (Richmond, Virginia) is the author of more than 20 books on
programming topics for McGraw-Hill, Prentice-Hall, Wiley, and M&T. His articles and
columns have appeared in such journals as Computer Language, Dr. Dobb’s, and BYTE.
Shammas holds an M.Sc. in chemical engineering from the University of Michigan, Ann
Arbor.
 = of ee 4 r ‘4 hy
he! e

{ >" , * i "ri. A-\in?

rervewe
‘ | ivTT fee

me
Mi ‘+ 7

:
;
Gay
a: ¥ $¢ar L- i> whe abil ry
fie
iws 2
‘2 itekood OS ant arom Yeyolten ot a .
bebewnlatis eft TAM Byrn eat? table
AVES iad Sot “fT nghirturod marge
ek RS Te vaheylat 1 onond athens

»
M ad SCI tierce] fant =
; ‘nto 2. (every
Se A.4 Ripe i (pet, OE
a (Cri “arti a
(iit, ears 6 >
AVE aa? wes
UIA Wi (emcees J
" yo i = Ot oe ae
 DISK WARRANTY
This software is protected by both United States copyright law and international copyright treaty
provision. You must treat this software just like a book, except that you may copy it into a
computer in order to be used and you may make archival copies of the software for the sole
purpose of backing up our software and protecting your investment from loss.

By saying “just like a book,” McGraw-Hill means, for example, that this software may be used by
any number of people and may be freely moved from one computer location to another, so long
as there is no possibility of its being used at one location or on one computer while it also is being
used at another. Just as a book cannot be read by two different people in two different places at
the same time, neither can the software be used by two different people in two different places at
the same time (unless, of course, McGraw-Hill’s copyright is being violated).

LIMITED WARRANTY
McGraw-Hill takes great care to provide you with top-quality software, thoroughly checked to
’ prevent virus infections. McGraw-Hill warrants the physical diskette(s) contained herein to be
free of defects in materials and workmanship for a period of sixty days from the purchase date. If
McGraw-Hill receives written notification within the warranty period of defects in materials or
workmanship, and such notification is determined by McGraw-Hill to be correct, McGraw-Hill
will replace the defective diskette(s). Send requests to:

McGraw-Hill, Inc.
Customer Services
PO. Box 545
Blacklick, OH 43004-0545

The entire and exclusive liability and remedy for breach of this Limited Warranty shall be limited
to replacement of defective diskette(s) and shall not include or extend to any claim for or right to
cover any other damages, including but not limited to, loss of profit, data, or use of the software,
or special, incidental, or consequential damages or other similar claims, even if McGraw-Hill has
been specifically advised of the possibility of such damages. In no event will McGraw-Hill’
liability for any damages to you or any other person ever exceed the lower of suggested list price
or actual price paid for the license to use the software, regardless of any form of the claim.

McGRAW-HILL, INC. SPECIFICALLY DISCLAIMS ALL OTHER WARRANTIES,

EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, ANY IMPLIED
WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Specifically, McGraw-Hill makes no representation or warranty that the software is fit for any
particular purpose and any implied warranty of merchantability is limited to the sixty-day
duration of the Limited Warranty covering the physical diskette(s) only (and not the software)
and is otherwise expressly and specifically disclaimed.

This limited warranty gives you specific legal rights; you may have others which may vary from
state to state. Some states do not allow the exclusion of incidental or consequential damages, or
the limitation on how long an implied warranty lasts, so some of the above may not apply to you.
 Disk Instructions

The diskette packaged with this book contains the code for all the sample programs listed in the
text. The diskette is double-density, and is formatted for IBM-compatible computers.

WARNING: The programs on the diskette cannot be run without suitable C or C++ compiler
software, for example the C/C++ compiler from Borland. If you don’t have a C/C++ compiler,
you will be able to edit the code files, but you will not be able to run them.

IMPORTANT
Read the Disk Warranty terms on the previous page before opening the disk envelope. Open-
ing the envelope constitutes acceptance of these terms and renders this entire book-disk
package unreturnable except for material defect.
ee Diskio Asoprapary Saks "

“ork |
ee ‘Namir ie; SE Ea F,

Mal ie
Slei PIN870569-X E
 $45.00 U.S.A.

——— Programming mo

Cott
MATHEMATICAL ALGORITHMS
FOR SCIENTISTS AND ENGINEERS

Discover the object-oriented advaritages

of C++ for mathematical analysis with
this handy guide
Analyze complex numerical data and get the answers you need fast
with this time-saving toolbox of powerful C and C++ mathematical
routines. All of the programs are written in C, with C++ extensions
provided for those cases where using the C++ classes offers a special
advantage. The code is compatible with a variety of platforms, including
UNIX, Windows®, and DOS.
Author and programming expert Namir Shammas covers a wide range
of popular numerical analysis and statistical topics:
® Solving linear and nonlinear equations
® Interpolation use methods
© Differentiation—tools for estimating th
® Integration—tools for calculating the-fi

ultivariable functions

®™ Various types of ANOVA tables

© Linear regression
© Multiple and polynomial regression
In addition to providing libraries for common mathematical and
statistical functions, Shammas includes pseudo-code for the algorithms
used to implement the various methods, allowing you to execute these
algorithms in other programming languages, such as BASIC and Pascal.

Bundled disk contains

P/N 057099-xX all of the mathematical
PART OF routines explained FI
in the book
ISBN 0-07?-912004-0
90000 Cover Design: Holberg Design, York, PA

McGraw-Hill, Inc.
Serving the Need for Knowledge
1221 Avenue of the Americas
New York, NY 10020
4° 760079" 1200 45