COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 1

Second Midterm Exam

Sofía Alejandra De La Hoz Escorcia

Skarleys Daniela Hernandez Movilla

Docente:

Jose David Posada Aguilar

Universidad del Norte

2021
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 2

Second Midterm Exam

Solving linear systems with GPU

Solving linear systems of equations has always been a valued approach to solve real

life problems in numerous domains such as physics, biology, geometry, etc.

Many approaches use an implicit time integration that is unconditionally stable to

solve the stability problems with explicit integration methods. Articulated rigid bodies are

often simulated using a Lagrange multiplier or an impulse-based method.

However, computer graphics processors (GPUs) with their high computational

throughput theoretically offer a tremendous speed-up by offering the possibility to execute

special GPU programs also known as kernels.

It should be added that in now a days different advances have been carried out in the

computer field for the solution of linear systems by implementing Computer graphics

processors (GPU), which is the objective of this investigative work, present some results of

these advances, with their respective explanation and proposed solution in a simple program

or script example.

First at all, we must understand that linear systems equations help us to solve divers

problems, from those that occur in our daily lives to problems that happen in engineering,

physics, mathematics, economics, and other sciences, like electromagnetics, fluid mechanics,

and quantum mechanics, which use large dense linear systems.

The structure of the linear system of equations: Ax = b, challenge the investigators to

make a system as efficiently and accurately as possible. To solve this, is possible to

implemented two classes of methods: direct and iterative methods. The direct methods use a
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 3

finite sequence of operations to solve the problem while the iterative methods use an initial

guess of the result and generate approximate solutions and tries to make the iterations

converge.

I must be said that the classical approach for solving a linear system using iterative

methods consists in Jacobi, Gauss-Seidel and SOR methods that are well known are

presented in many textbooks. In comparison, direct methods generally involve the

decomposition of matrices followed by the successive resolution of triangular systems.

Different methods of decomposition exist such as QR factorization, Cholesky, LDLT or LU.

Iterative methods: Jacobi

Scheme of the algorithm of the solution of the iterative methods, Jacobi to solve linear

systems of equations.

In each step the components of the new vector x are calculated through the previous

vector z using the following formula:

Inputs: A, b, tol, smax, where A is a square matrix of order n, b is a vector of length n.

Output: vector x and number of iterations performed s.

Iterative method: Gauss-Seidel

As well as the Jacobi methos, the structure of the Gauss-Seidel has a similar structure

in which the components of the new vector x are calculated through the previous vector z

using the following formula:

If 𝐴 ∈ 𝑀𝑛 (ℝ) is a strictly dominant matrix in the diagonal sense, then for any choice

of the initial approximation 𝑥̅ (0) , both the Jacobi and Gauss-Seidel methods converge to the

unique solution of the linear system 𝐴𝑥̅ = 𝑏̅.

The following code solves by the same Gauss-Seidel method but in the C program.
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 5

Iterative method: SOR

Unlike the Jacobi method, this scheme is fully sequential. However, it can be

efficiently implemented without storing the solution of the previous step, with a saving of

memory storage.
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 6

Direct method: QR factorization

We can compute the reduced QR factorization with the following (somewhat more

practical and almost Matlab implementation of the) classical Gram-Schmidt algorithm.

The classical Gram-Schmidt algorithm is not ideal for numerical calculations since it

is known to be unstable. But, the Gram-Schmidt algorithm yields an existence proof for the

QR factorization.

Direct method: Cholesky

The Cholesky decomposition or Cholesky factorization is a decomposition of a

Hermitian, positive-definite matrix into the product of a lower triangular matrix and its

conjugate transpose.

The Cholesky decomposition of a Hermitian positive-definite matrix A is a

decomposition of the form 𝐴 = [𝐿][𝐿]𝑇 , where L is a lower triangular matrix with real and

positive diagonal entries, and 𝐿𝑇 denotes the conjugate transpose of L. Every Hermitian

positive-definite matrix (and thus also every real-valued symmetric positive-definite matrix)

has a unique Cholesky decomposition.

Every symmetric, positive definite matrix A can be decomposed into a product of a

unique lower triangular matrix L and its transpose: 𝐴 = 𝐿𝐿𝑇

These are the basis of Cholesky Decomposition Algorithm:

// CPP program to decompose a matrix using

// Cholesky Decomposition

#include <bits/stdc++.h>

using namespace std;

const int MAX = 100;

void Cholesky_Decomposition(int matrix[][MAX],

int n)

int lower[n][n];

memset(lower, 0, sizeof(lower));

// Decomposing a matrix into Lower Triangular

for (int i = 0; i < n; i++) {

for (int j = 0; j <= i; j++) {

int sum = 0;

if (j == i) // summation for diagonals

{
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 8

for (int k = 0; k < j; k++)

sum += pow(lower[j][k], 2);

lower[j][j] = sqrt(matrix[j][j] -

sum);

} else {

// Evaluating L(i, j) using L(j, j)

for (int k = 0; k < j; k++)

sum += (lower[i][k] * lower[j][k]);

lower[i][j] = (matrix[i][j] - sum) /

lower[j][j];

// Displaying Lower Triangular and its Transpose

cout << setw(6) << " Lower Triangular"

<< setw(30) << "Transpose" << endl;

for (int i = 0; i < n; i++) {

// Lower Triangular

for (int j = 0; j < n; j++)

cout << setw(6) << lower[i][j] << "\t";

cout << "\t";

// Transpose of Lower Triangular

for (int j = 0; j < n; j++)

cout << setw(6) << lower[j][i] << "\t";

cout << endl;

// Driver Code

int main()

int n = 3;

int matrix[][MAX] = { { 4, 12, -16 },

{ 12, 37, -43 },

{ -16, -43, 98 } };

Cholesky_Decomposition(matrix, n);

return 0;

Direct method: 𝑳𝑫𝑳𝑻

The 𝐿𝐷𝐿𝑇 factorization is given by equation 𝐴 = 𝐿𝐷𝐿𝑇 .

where A is an N ×N symmetric squared matrix, L is unit lower triangular and D is

diagonal. For simplicity and also because pivoting is not used, the assumption that D is

diagonal has been made, although D can also be block-diagonal, with blocks of size 1 × 1 or

2×2, when pivoting is applied. The matrix A can also be factorized as 𝐴 = 𝑈𝐷𝑈𝑇 , where U is

unit upper triangular. The algorithm for the lower triangular case (L) can straightforwardly be
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 10

extended to the upper triangular case (U) and therefore only the development of the former is

presented.

Direct method: LU

LU is a method to Decomposition for Solving Linear Equations in which Ax=b is the

same to Lx=b, where L is a lower triangular matrix.

The code for the forward substitution algorithm to solve Lx=b is:
COMPUTATIONAL SOLUTIONS OF ENGINEERING PROBLEMS 11

Referencias

Apellidos, n. s. (Año). Título del artículo. Título del diario, Páginas desde - hasta.

Rémy, A (2015). Solving dense linear systems on accelerated multicore architectures, 1- 20.

Weber, D., Bender, J., Schnoes, M., Stork, A., Fellner, D. (2013). Efficient GPU Data

Structures and Methods to Solve Sparse Linear Systems in Dynamics Applications,1-34.

Molero, J. M., Garzón, E. M., García, I., Quintana-Ortí., E. S., Plazo, A. Solución de

Múltiples Sistemas Lineales en GPUs, 1-4.

Bogdan, O., Tudorel, A., Raluca, M. D. Improving the performance of the linear systems

solvers using CUDA, 1-10.

Becker, D., Faverge, M., Dongarra, J. J. (2011). Towards a Parallel Tile LDL Factorization

for Multicore Architectures.1-11.

Lang, N., Hermann, M., Saak, J. (2014). On the benefits of the 𝐿𝐷𝐿𝑇 factorization for large-

scale differential matrix equation solvers.1-37.