Scribd
Upload
37 views6 pages

Is Positive Definite.: 7.6 The Conjugate Gradient Method Assumption: Definition: Inner Production of Vectors

The document summarizes the conjugate gradient method, an algorithm for solving certain linear systems of equations. It defines key terms like quadratic forms, gradients, and A-orthogonal vectors. The conjugate gradient method works by finding search directions that are conjugate to previous directions. Preconditioning can make the method less sensitive to rounding errors by solving an equivalent system where the matrix is scaled to be better conditioned.

Uploaded by

Prabesh Shrestha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
37 views6 pages

Is Positive Definite.: 7.6 The Conjugate Gradient Method Assumption: Definition: Inner Production of Vectors

The document summarizes the conjugate gradient method, an algorithm for solving certain linear systems of equations. It defines key terms like quadratic forms, gradients, and A-orthogonal vectors. The conjugate gradient method works by finding search directions that are conjugate to previous directions. Preconditioning can make the method less sensitive to rounding errors by solving an equivalent system where the matrix is scaled to be better conditioned.

Uploaded by

Prabesh Shrestha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

7.

6 The Conjugate Gradient Method


Assumption: , is positive definite.
Definition: Inner production of vectors and is .
Let be positive definite, .

Definition: A quadratic form is a scale, quadratic function of a vector with the form

where is a matrix, and are vectors, and is a scalar constant.

Example. Let [ ], [ ] The solution to is [ ]

Figure: The graph of a quadratic form . The minimum point of this


surface is the solution to

1
The gradient of a quadratic form is defined to be

[ ]

If is symmetric, reduces to
Therefore, the solution to is a critical point of

Theorem. The vector is a solution to the positive definite linear system if and only if produces
the minimal value of
Proof Let and be fixed vectors and a real number variable.

So
Define
Then assumes a minimal value when

The minimum occurs when ̂

2
̂ ̂

For any vector , we have ̂ unless


Suppose satisfies then for any Thus minimizes

On the other hand, suppose that is a vector minimizes Then for any vector , ̂
Thus This implies that
The Method of Steepest Descent
a) Start with an arbitrary initial guess to the solution to
b) Let .
Compute

Remark: the gradient of is . The direction of greatest decrease in the value of


is
c)

d) Repeat the above process.


3
Remark: The Method of Steepest Descent does not lead to fastest convergence.

Definition: A set of nonzero vectors that satisfy


is said to be A-orthogonal.

Theorem. Let be an A-orthogonal set of nonzero vectors associated with the positive definite
matrix , and let be arbitrary. Define

for Then, assuming exact arithmetic,

Conjugate Gradient Method


Theorem. The residual vectors where , for a conjugate direction method, satisfy the
equation

Algorithm:
a) Start with an arbitrary initial guess to the solution to
Set

4
b) for

Preconditioning
Motivation: When is ill-conditioned, conjugate gradient method is highly sensitive to rounding errors.
Instead of solving directly, consider to solve ̃ ̃ ̃, where ̃ ,̃ and ̃ .
Here is a nonsingular matrix.

To see and ̃ ̃ ̃ are equivalent,


̃̃

Preconditioned Conjugate Gradient Method


Since ̃ and ̃ ̃ ̃̃
Let ̃ and

5
A) Start with an arbitrary initial guess to the solution to
Set

B) for

̃
̃

You might also like