Lecture 3 – Numerical Solutions to CE Problems

Solving System of Linear Equations

A. Solutions of Systems of Linear Non-Homogeneous Equations (mxm)

Definition 1: A linear equation is said to be homogeneous when its constant part is zero. For
example both of the following are homogeneous:
3x+2y=0
x-5y+z=0
Definition 2: The following equation, on the other hand, is not homogeneous because its
constant part does not equal zero:
2x-y+z=3
x-2y+3z = 5

1. Solving Small Numbers of Equation

There are several methods appropriate for solving small (n ≤ 3) sets of simultaneous equations.
These are the graphical method, Cramer’s rule, and the elimination of unknowns.

1.1 Graphical Method

A graphical solution is obtainable for two equations by plotting them on Cartesian coordinates
with one axis corresponding to x1 and the other to x2. Because we are dealing with linear
systems, each equation is a straight line. This can be easily illustrated for the general equations.

a11x1 + a12x2 = b1
a21x1 + a22x2 = b2

Example: Use the graphical method to solve

3x1 + 2x2 = 18
−x1 + 2x2 = 2
Solution:

1.2 Elimination of Unknowns

The elimination of unknowns by combining equations is an algebraic approach that can
be illustrated for a set of two equations.
a11x1 + a12x2 = b1 (Eq. 1)
a21x1 + a22x2 = b2 (Eq. 2)

The basic strategy is to multiply the equations by constants so that one of the unknowns will be
eliminated when the two equations are combined. The result is a single equation that can be
solved for the remaining unknown. This value can then be substituted into either of the original
equations to compute the other variable. For example, the Eq. 1 might be multiplied by a21 and
Eq. 2 by a11 to give
a11a21x1 + a12a21x2 = b1a21
a21a11x1 + a22a11x2 = b2a11

Subtracting Eq. 1 to Eq. 2 will, therefore, eliminate the x1 term from the equations to yield

a22a11x2 − a12a21x2 = b2a11 − b1a21 . Solve for x2 then substitute the value of x2 in finding x1

Example: Use the elimination of unknowns to solve

3x1 + 2x2 = 18
−x1 + 2x2 = 2

1.3 Comparison Method

We can use the comparison method, a procedure to solve for the unknowns in a set of
independent equations:

We set up the equations so the left-hand sides have the same unknown variable. Thus, the
left-hand sides are the same. Then we equate the right-hand sides and solve.
Steps;
1. write each equation with x as the subject.
2. Equate the obtained value of x from each equation to get the value of y.
3. Substitute the value of y to any of the equation.

Example: From the equations 2x + 3y = 13 and 7x - 4y = 2. Solve for x and y

1.4 Substitution Method

The substitution method is used to solve systems of linear equation by finding the exact
values of x and y which correspond to the point of intersection.
The idea is to pick one of the two given equations and solve for either of the
variables, x or y. The result from the first step will be substituted into the other equation. The
effect will be a single equation with one variable which can be solved as usual.

Example: From the equations 2x + 3y = 13 and 7x - 4y = 2. Solve for x and y using substitution
method.

1.5 Cramer’s Rule

This rule states that each unknown in a system of linear algebraic equations may be
expressed as a fraction of two determinants with denominator D and with the numerator obtained
from D by replacing the column of coefficients of the unknown in question by the constants b1,
b2, . . . , bn.

Example. Find the value of x, y and z using Cramer’s Rule

2x + 1y + 1z = 3
1x – 1y – 1z = 0
1x + 2y + 1z = 0
a. Find the coefficient matrix (left-hand side) and answer column (right-hand side) or the
augmented matrix

b. Find the determinant of coefficient matrix

D=
c. Formed Dx by replacing the x-column values with the answer column. Similarly with Dy and
Dz which results to;

Dx= Dy= Dz=

d. Evaluating each determinant, then find the value of x, y and z using;

2. Gaussian Elimination
The goals of Gaussian elimination are to make the upper-left corner element a 1, use
elementary row operations to get 0s in all positions underneath that first 1, get 1s for leading
coefficients in every row diagonally from the upper-left to lower-right corner, and get 0s beneath
all leading coefficients.
Basically, you eliminate all variables in the last row except for one, all variables except
for two in the equation above that one, and so on and so forth to the top equation, which has all
the variables. Then you can use back substitution to solve for one variable at a time by plugging
the values you know into the equations from the bottom up.
Example. Find the value of x, y and z using Gaussian Elimination
2x + 1y + 1z = 3
1x – 1y – 1z = 0
1x + 2y + 1z = 0
We can use the following procedure;
1. Set a11 into “1”
2. Set a21, a31 to 0s underneath the 1 in the first column (a11).
3. Set a22 into “1”
4. Set a32, to 0 underneath the 1 in the second column (a22).
5. Use backward substitution to solve for one variable at a time.

Resulting to;

3. Gauss- Jordan Elimination

It is a method of solving a linear system of equations. This is done by transforming the
system's augmented matrix into reduced row-echelon form by means of row operations.
Example: Find the value of x, y and z using Gauss-Jordan Elimination
2x + 1y + 1z = 3
1x – 1y – 1z = 0
1x + 2y + 1z = 0

Properties of a matrix in reduced row-echelon form:

1. Each row contains only zeros until the first non-zero element, which must be 1.
2. As the rows are followed from top to bottom, the first non-zero number occurs further
to the right than in the previous row.
3. The entries above and below the first 1 in each row must all be 0.
Resulting to;
4. LU Decomposition

L U decomposition of a matrix is the factorization of a given square matrix into two triangular
matrices, one upper triangular matrix and one lower triangular matrix, such that the product of
these two matrices gives the original matrix
A square matrix A can be decomposed into two square matrices L and U such that A = L U
where U is an upper triangular matrix formed as a result of applying Gauss Elimination Method
on A; and L is a lower triangular matrix with diagonal elements being equal to 1.

Steps for L U Decomposition

Given a set of linear equations, first convert them into matrix form A X = C where A is the
coefficient matrix, X is the variable matrix and C is the matrix of numbers on the right-hand side
of the equations.
Now, reduce the coefficient matrix A, i.e., the matrix obtained from the coefficients of variables
in all the given equations such that for ‘n’ variables we have an nXn matrix, to row echelon form
using Gauss Elimination Method. The matrix so obtained is U.
To find L, we have two methods. The first one is to assume the remaining elements as some
artificial variables, make equations using A = L U and solve them to find those artificial
variables.
The other method is that the remaining elements are the multiplier coefficients because of which
the respective positions became zero in the U matrix. (This method is a little tricky to understand
by words but would get clear in the example below)
Now, we have A (the nXn coefficient matrix), L (the nXn lower triangular matrix), U (the nXn
upper triangular matrix), X (the nX1 matrix of variables) and C (the nX1 matrix of numbers on
the right-hand side of the equations).
The given system of equations is A X = C. We substitute A = L U. Thus, we have L U X = C.
We put Z = U X, where Z is a matrix or artificial variables and solve for L Z = C first and then
solve for U X = Z to find X or the values of the variables, which was required.
Example: Solve the following system of linear equations using LU decomposition.
X1+X2+X3 = 1
4X1+3X2-X3 = 6
3X1+5X2+3X3 = 4
A= X= and C= such that AX = C

Now consider matrix A and convert it in row echelon form using Gaussian elimination method.

We get

Hence we get, L =

Now, we assume Z= and solve L Z = C.

So, we have Z1 = 1, 4Z1 +Z2 = 6 and 3Z1-2Z2+Z3 = 4

Solving for the value of z, Z1 = 1, Z2 = 2 and Z3 = 5

Now we should UX = Z

Therefore, we get X1+X2+X3 = 1, -X2-5X3 = 2 and -10X3 = 5

Thus, the solution to the given system of linear equations is X1 = 1, X2 = 0.5 and X3 = -0.5

Hence the matrix X =

5. Jacobi Method
Two assumptions made on Jacobi Method:
1. The system given by

Has a unique solution.

2. The coefficient matrix has no zeros on its main diagonal, namely, a11, a22, ann, are nonzeros.
To begin, solve the 1st equation for x1, the 2nd equation for x2 and so on to obtain the rewritten
equations:
Choose the initial guess =

and substitute these values of x1 into the right-hand side of the rewritten equations to obtain the
first approximation. After this procedure has been completed, one iteration has been performed.
In the same way, the second approximation is formed by substituting the first approximation’s x-
values into the right-hand side of the rewritten equations. By repeated iterations, you will form a
sequence of approximations that often converges to the actual solution

Example; Apply the Jacobi method

5x1-2x2+3x3 = -1
-3x1+9x2+x3 = 2
2x1-x2-7x3 = 3
Continue the iterations until two successive approximations are identical when rounded to three
significant digits. (Refer to the table below for the possible number of iterations)

Because the last two columns are identical, you can conclude that to three significant digits the
solution is x1 = 0.186, x2 = 0.331 and x3 = -0.423

For the system of linear equations given in Example 1, the Jacobi method is said to converge.
That is, repeated iterations succeed in producing an approximation that is correct to three
significant digits. As is generally true for iterative methods, greater accuracy would require more
iterations.

6. Gauss Seidel Method

You will now look at a modification of the Jacobi method called the Gauss-Seidel method,
named after Carl Friedrich Gauss (1777–1855) and Philipp L. Seidel (1821–1896). This
modification is no more difficult to use than the Jacobi method, and it often requires fewer
iterations to produce the same degree of accuracy.
With the Jacobi method, the values of x1 obtained in the nth approximation remain unchanged
until the entire (n+1)th approximation has been calculated. With the Gauss-Seidel method, on the
other hand, you use the new values of each x1 as soon as they are known. That is, once you have
determined x1 from the first equation, its value is then used in the second equation to obtain the
new x2. Similarly, the new x1 and x2 are used in the third equation to obtain the new x3 and so on.
Example; Apply the Gauss-Seidel method use initial guess as x1 =0 , x2=0, x3=0

5x1-2x2+3x3 = -1
-3x1+9x2+x3 = 2
2x1-x2-7x3 = 3

Will result to;

Neither of the iterative methods presented in this section always converges. That is, it is possible
to apply the Jacobi method or the Gauss-Seidel method to a system of linear equations and obtain
a divergent sequence of approximations. In such cases, it is said that the method diverges.

Example of Divergence
Apply the Jacobi method to the system
x1-5x2 = -4
7x1-x2 = 6 , use x1 and x2 equal to zero as initial guess.

Resulting to;

Exercises. Solve the following system of linear equations using;

1. Graphical, elimination, comparison and substitution method.
x-3y = 16
3x-2y = 13
2. Cramer’s Rule
x+2y +3z = -5
3x+y-3z = 4
-3x+4y+7z = -7
3. Gaussian, Gauss-Jordan, and LU decomposition method.
-3x+2y-6z = 6
5x+7y-5z = 6
x+4y-2z = 8
4. Gauss Seidel use initial guess as x1=1, x2=0 and x3=1. Jacobi method as x1,x2 and x3 = 0
2x1+6x2-3x3=5
x1-2x2+5x3=12
5x1+2x2-x3 = 6
B. Solutions of Systems of Linear Non-Homogeneous Equations (mxn)

Matrix form of a linear system of equations. The matrix form of a system of m linear equations
in n unknowns

To determine if a system of m linear equations in n unknowns is consistent or inconsistent i.e. if

it has a solution or not? The answer is given by the following fundamental theorem.
A system AX = B of m linear equations in n unknowns is consistent if and only if the coefficient
matrix and the augmented matrix of the system have the same rank.

Definition of rank (r) of a matrix

Let say we have a (mxm) matrix, if the determinants of mxm matrix is non-zero, the rank is m,
otherwise find the determinant of all the next order (m-1) sub-matrices. If the determinant of all
the m-1 sub matrices are not zero rank is (m-1) while if determinant are all zero rank is
automatically (m-2)
Another way in finding the rank of a matrix, is to transform the matrix to its row-echelon form
and count the number of non-zero rows.

Example; Find the rank of the given matrix.

det A = 0, Recall properties of determinants. Since det A = 0, find the determinant of (2x2) sub
matrices

det = -2 det = -1 det = 0 (Do the same to all 2x2)

Since the determinant of all 2x2 sub matrices are not all zero therefore the rank is 2.

or by transforming the given matrix to its row-echelon form.

Let say,

Resulting to where the rank is 2 because there are 2 non-zero rows.

C. Solutions of Systems of Linear Homogeneous Equations (mxn)

One of the principle advantages to working with homogeneous systems over non-homogeneous
systems is that homogeneous systems always have at least one solution, namely, the case where
all unknowns are equal to zero. Such a case is called the trivial solution to the homogeneous
system.
For instance, looking at this system
 2x+3y+z = 0
x-y-2z = 0

we see that if x = 0, y = 0, and z = 0, then all three equations are true.

D. Solutions of Systems of Linear Homogeneous Equations (mxm)

Example;

x+y+z=0

x-y+2z = 0

3x-y+5z=0

Transforming the given matrix to its row-echelon form resulting to;

Then find the value of x, y and z, as z = z then y = ½ z and x = -3/2z

Exercises; Solve the following homogeneous system of equations.

1. 4x+y+z = 0
3x-2y+3z = 0
2x-y-z =0

2. 3x+2y-z=0
2x-2y+4z = 0