Using Model Checking with Symbolic Execution to

Verify Parallel Numerical Programs

Stephen F. Siegel
1
Anastasia Mironova
2
George S. Avrunin
1
Lori A. Clarke
1
1
University of Massachusetts Amherst
2
University of Utah
ISSTA 2006
Portland, Maine, July 1720, 2006
1
Parallel numerical programs
(x
1
, . . . , x
n
)
P
(y
1
, . . . , y
m
)
scientic computation

simulations of physical phenomena

climate modeling

molecular dynamics
matrix algorithms

solving systems of linear equations

matrix factorization

reducing to normal forms

The Problem:
Parallel numerical programs are very dicult to get right.
2
Parallel numerical programs
(x
1
, . . . , x
n
)
P
(y
1
, . . . , y
m
)
scientic computation

simulations of physical phenomena

climate modeling

molecular dynamics
matrix algorithms

solving systems of linear equations

matrix factorization

reducing to normal forms

The Problem:
Parallel numerical programs are very dicult to get right.
2
Parallel numerical programs
(x
1
, . . . , x
n
)
P
(y
1
, . . . , y
m
)
scientic computation

simulations of physical phenomena

climate modeling

molecular dynamics
matrix algorithms

solving systems of linear equations

matrix factorization

reducing to normal forms

The Problem:
Parallel numerical programs are very dicult to get right.
2
Parallel numerical programs
(x
1
, . . . , x
n
)
P
(y
1
, . . . , y
m
)
scientic computation

simulations of physical phenomena

climate modeling

molecular dynamics
matrix algorithms

solving systems of linear equations

matrix factorization

reducing to normal forms

The Problem:
Parallel numerical programs are very dicult to get right.
2
Matrix multiplication: sequential version
double A[N][L], B[L][M], C[N][M];
.
.
.
int i,j,k;
for (i=0; i<N; i++)
for (j=0; j<M; j++) {
C[i][j] = 0.0;
for (k=0; k<L; k++)
C[i][j] += A[i][k]*B[k][j];
}
3
Matrix multiplication: parallel version (master-slave)
int rank,nprocs,i,j,numsent,sender,row,anstype;
double buffer[L], ans[M];
MPI_Status status;
MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
if (rank==0) { /* I am the master */
numsent=0;
for (i=0; i<nprocs-1; i++) {
for (j=0; j<L; j++)
buffer[j] = A[i][j];
MPI_Send(buffer, L, MPI_DOUBLE, i+1,
i+1, MPI_COMM_WORLD);
numsent++;
}
for (i=0; i<N; i++) {
MPI_Recv(ans, M, MPI_DOUBLE, MPI_ANY_SOURCE,
MPI_ANY_TAG, MPI_COMM_WORLD, &status);
sender = status.MPI_SOURCE;
anstype = status.MPI_TAG-1;
for (j=0; j<M; j++)
C[anstype][j] = ans[j];
if (numsent<N) {
for (j=0; j<L; j++)
buffer[j] = A[numsent][j];
MPI_Send(buffer, L, MPI_DOUBLE, sender,
numsent+1, MPI_COMM_WORLD);
numsent++;
}
else MPI_Send(buffer, 1, MPI_DOUBLE, sender,
0, MPI_COMM_WORLD);
}
} else { /* I am a slave */
while (1) {
MPI_Recv(buffer, L, MPI_DOUBLE, 0,
MPI_ANY_TAG, MPI_COMM_WORLD, &status);
if (status.MPI_TAG==0) break;
row = status.MPI_TAG-1;
for (i=0; i<M; i++) {
ans[i] = 0.0;
for (j=0; j<L; j++)
ans[i] += buffer[j]*B[j][i];
}
MPI_Send(ans, M, MPI_DOUBLE, 0,
row+1, MPI_COMM_WORLD);
}
}
adapted from
Using MPI
by
William Gropp
Ewing Lusk
Anthony Skjellum
4
Why parallel numerical programs are dicult to get right
usual reasons concurrent programming is dicult

parallelism adds complexity

nondeterminism

deadlocks

race conditions
additional sources of nondeterminism from MPI
problem of test oracles

in scientic computation, often dont know correct result for a given

test input, so cant tell if the observed result is correct

analytical solutions rarely exist

sequential program may only work on small test cases

oating-point arithmetic diers from real arithmetic

5
Why parallel numerical programs are dicult to get right
usual reasons concurrent programming is dicult

parallelism adds complexity

nondeterminism

deadlocks

race conditions
additional sources of nondeterminism from MPI
problem of test oracles

in scientic computation, often dont know correct result for a given

test input, so cant tell if the observed result is correct

analytical solutions rarely exist

sequential program may only work on small test cases

oating-point arithmetic diers from real arithmetic

5
Why parallel numerical programs are dicult to get right
usual reasons concurrent programming is dicult

parallelism adds complexity

nondeterminism

deadlocks

race conditions
additional sources of nondeterminism from MPI
problem of test oracles

in scientic computation, often dont know correct result for a given

test input, so cant tell if the observed result is correct

analytical solutions rarely exist

sequential program may only work on small test cases

oating-point arithmetic diers from real arithmetic

5
Why parallel numerical programs are dicult to get right
usual reasons concurrent programming is dicult

parallelism adds complexity

nondeterminism

deadlocks

race conditions
additional sources of nondeterminism from MPI
problem of test oracles

in scientic computation, often dont know correct result for a given

test input, so cant tell if the observed result is correct

analytical solutions rarely exist

sequential program may only work on small test cases

oating-point arithmetic diers from real arithmetic

5
Current methods used to discover and correct problems in
parallel numeric programs
1. testing

only a tiny fraction of inputs can be tested

nondeterminism limits eectiveness

oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear

insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing

only a tiny fraction of inputs can be tested

nondeterminism limits eectiveness

oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear

insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing

only a tiny fraction of inputs can be tested

nondeterminism limits eectiveness

oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear

insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing

only a tiny fraction of inputs can be tested

nondeterminism limits eectiveness

oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear

insertion of barriers
6
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version

for a given conguration

reduces the problem of verifying the correctness of a parallel

numerical program to the problem of verifying the correctness of a
sequential numerical program
uses symbolic execution to model oating-point computation
uses model checking

requires translation of program into input language of model checker

veries equivalence over all executions

produces a trace if program is incorrect

7
How do we model oating-point computation?
one double-precision oating-point variable has 2
64
possible states
abstraction?
Input: symbolic constants x
0
, x
1
, . . .
Output: symbolic expressions in the x
i
+
0.0
x
0
x
4
+

x
1
x
6
=
+
+
0.0 x
1
x
6
x
0
x
4
0.0 + (x
0
x
4
) + x
1
x
6
= (0.0 + (x
0
x
4
)) + x
1
x
6
8
How do we model oating-point computation?
one double-precision oating-point variable has 2
64
possible states
abstraction?
Input: symbolic constants x
0
, x
1
, . . .
Output: symbolic expressions in the x
i
+
0.0
x
0
x
4
+

x
1
x
6
=
+
+
0.0 x
1
x
6
x
0
x
4
0.0 + (x
0
x
4
) + x
1
x
6
= (0.0 + (x
0
x
4
)) + x
1
x
6
8
How do we represent symbolic expressions?
Value numbering
place all symbolic expressions in an expression table

every expression has a unique ID number

in the model. . .

replace all oating-point values with ID numbers

replace all oating-point operations with symbolic operations

to evaluate x + y:

is x + y already in the table?

if yes, return its ID number

if no, create new table entry and return new ID number

9
How do we represent symbolic expressions?
Value numbering
place all symbolic expressions in an expression table

every expression has a unique ID number

in the model. . .

replace all oating-point values with ID numbers

replace all oating-point operations with symbolic operations

to evaluate x + y:

is x + y already in the table?

if yes, return its ID number

if no, create new table entry and return new ID number

9
How do we represent symbolic expressions?
Value numbering
place all symbolic expressions in an expression table

every expression has a unique ID number

in the model. . .

replace all oating-point values with ID numbers

replace all oating-point operations with symbolic operations

to evaluate x + y:

is x + y already in the table?

if yes, return its ID number

if no, create new table entry and return new ID number

9
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_ _
=
_ _
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_ _
=
_ _
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
0 0
0 0
_
=
_
0.0 0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
0 0
0 0
_
=
_
0.0 0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
11 0
0 0
_
=
_
0.0+x
0
x
4
0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
11 0
0 0
_
=
_
0.0+x
0
x
4
0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 0
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 0
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
0.0
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 15
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
0.0+x
0
x
5
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 15
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
0.0+x
0
x
5
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
0 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
0.0 0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
19 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
0.0+x
2
x
4
0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
19 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
0.0+x
2
x
4
0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
21 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
(0.0+x
2
x
4
)+x
3
x
6
0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
21 0
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
(0.0+x
2
x
4
)+x
3
x
6
0.0
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
21 23
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
(0.0+x
2
x
4
)+x
3
x
6
0.0+x
2
x
5
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
21 23
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
(0.0+x
2
x
4
)+x
3
x
6
0.0+x
2
x
5
_
10
i e
i
interpretation
0 (L, 0.0) 0.0
1 (L, 1.0) 1.0
2 (X, 0) x
0
3 (X, 1) x
1
4 (X, 2) x
2
5 (X, 3) x
3
6 (X, 4) x
4
7 (X, 5) x
5
8 (X, 6) x
6
9 (X, 7) x
7
10 (*, 2, 6) x
0
x
4
11 (+, 0, 10) 0.0+x
0
x
4
12 (*, 3, 8) x
1
x
6
i e
i
interpretation
13 (+, 11, 12) (0.0+x
0
x
4
)+x
1
x
6
14 (*, 2, 7) x
0
x
5
15 (+, 0, 14) 0.0+x
0
x
5
16 (*, 3, 9) x
1
x
7
17 (+, 15, 16) (0.0+x
0
x
5
)+x
1
x
7
18 (*, 4, 6) x
2
x
4
19 (+, 0, 12) 0.0+x
2
x
4
20 (*, 5, 8) x
3
x
6
21 (+, 19, 20) (0.0+x
2
x
4
)+x
3
x
6
22 (*, 4, 7) x
2
x
5
23 (+, 0, 22) 0.0+x
2
x
5
24 (*, 5, 9) x
3
x
7
25 (+, 23, 24) (0.0+x
2
x
5
)+x
3
x
7
A =
_
2 3
4 5
_
=
_
x
0
x
1
x
2
x
3
_
B =
_
6 7
8 9
_
=
_
x
4
x
5
x
6
x
7
_
C =
_
13 17
21 25
_
=
_
(0.0+x
0
x
4
)+x
1
x
6
(0.0+x
0
x
5
)+x
1
x
7
(0.0+x
2
x
4
)+x
3
x
6
(0.0+x
2
x
5
)+x
3
x
7
_
10
The path correspondence problem
the programs may contain branches on expressions that involve the
symbolic variables

if (x
0
= 0) {. . .} else {. . .}
only want to compare the result of an execution path in the parallel
program to the result of a corresponding path in the sequential
program
11
The path correspondence problem
the programs may contain branches on expressions that involve the
symbolic variables

if (x
0
= 0) {. . .} else {. . .}
only want to compare the result of an execution path in the parallel
program to the result of a corresponding path in the sequential
program
11
Path conditions and domains
enumerate all paths through the sequential program

keeping track of the path condition for each path

y =
_

_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j

i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Path conditions and domains
enumerate all paths through the sequential program

keeping track of the path condition for each path

y =
_

_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j

i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Path conditions and domains
enumerate all paths through the sequential program

keeping track of the path condition for each path

y =
_

_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j

i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Modeling conditional statements
To model the statement if (x
0
= 0) {. . .} else {. . .}
p true; /* path condition */
.
.
.
b (p, x
0
= 0);
if (b = 1) {
if (choose()) {
b 1; p p (x
0
= 0);
} else {
b 0; p p (x
0
= 0);
}
}
if (b = 1) { . . . } else { . . . }
(p, q) =
_

_
1 if p q
0 if p q
1 if dont know
for boolean-valued symbolic
expressions p, q
13
The method
1. construct symbolic model M
seq
of sequential program

input: x, output: y, path condition: p

2. construct symbolic model M
par
of parallel program

input: x, output: y

, path condition: p

using same symbolic table

3. create composite model M:

p true; M
seq
; M
par
; assert(y = y

);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:

a trace through M
seq

a trace through M
par

the values of p, y, and y

14
The method
1. construct symbolic model M
seq
of sequential program

input: x, output: y, path condition: p

2. construct symbolic model M
par
of parallel program

input: x, output: y

, path condition: p

using same symbolic table

3. create composite model M:

p true; M
seq
; M
par
; assert(y = y

);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:

a trace through M
seq

a trace through M
par

the values of p, y, and y

14
The method
1. construct symbolic model M
seq
of sequential program

input: x, output: y, path condition: p

2. construct symbolic model M
par
of parallel program

input: x, output: y

, path condition: p

using same symbolic table

3. create composite model M:

p true; M
seq
; M
par
; assert(y = y

);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:

a trace through M
seq

a trace through M
par

the values of p, y, and y

14
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand

exact equality of expressions

2. IEEE

x + y = y + x

1x = x = x1

x + 0 = x = 0 + x
3. Real

all of IEEE

(x + y) + z = x + (y + z)

(xy)z = x(yz)

. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand

exact equality of expressions

2. IEEE

x + y = y + x

1x = x = x1

x + 0 = x = 0 + x
3. Real

all of IEEE

(x + y) + z = x + (y + z)

(xy)z = x(yz)

. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand

exact equality of expressions

2. IEEE

x + y = y + x

1x = x = x1

x + 0 = x = 0 + x
3. Real

all of IEEE

(x + y) + z = x + (y + z)

(xy)z = x(yz)

. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand

exact equality of expressions

2. IEEE

x + y = y + x

1x = x = x1

x + 0 = x = 0 + x
3. Real

all of IEEE

(x + y) + z = x + (y + z)

(xy)z = x(yz)

. . .
15
Preliminary experimental results
implemented as extension to SPIN
wrote our own simple symbolic algebra library and light-weight
theorem-prover
found bug in one example (Jacobi iteration)
matrix
multiplication
Gaussian
elimination
Jacobi
iteration
Monte
Carlo
processes 6 6 17 9
path domains 1 13327 4 4
symbolic expressions 2202 247656 8239 1232
input vector size 200 36 1333 99
output vector size 100 36 36 1
states (10
3
) 4443 16114 6295 3112
memory (MB) 217 801 362 279
time (s) 506 3224 9846 738
16
Related Work
1. Ball and Rajamani. Automatically validating temporal safety
properties of interfaces. SPIN 2001.
2. Khurshid, Pasareanu, and Visser. Generalized symbolic execution for
model checking and testing. TACAS 2003.
3. Pasareanu and Visser. Verication of Java programs using symbolic
execution and invariant generation. SPIN 2004.
4. Elmas, Tasiran, and Qadeer. VYRD: verifying concurrent programs
by runtime renement-violation detection. PLDI 2005.
17
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .

. . .over all inputs

. . .over all possible executions

can be largely automated
conservative
produces detailed counterexample if equivalence cannot be veried
Weaknesses of method
models are constructed by hand (for now)
state explosion

often requires small bounds on conguration

possibility of spurious error report

precision depends upon close correspondence between numerical

operations in sequential and parallel programs
18
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_

b00 b01
b10 b11

b00 b01
b10 b11

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
00
a
01

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
10
a
11

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
a
10
a
11

_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
10
c
11

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
20
a
21

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
_
a
30
a
31

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
30
a
31

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
30
a
31

_
b
00
b
01
b
10
b
11
_
=
_
c
30
c
31

Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
_
c
30
c
31

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
c
20
c
21

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
20
c
21

19
Master-slave matrix multiplication
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_

b00 b01
b10 b11

b00 b01
b10 b11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
00
a
01

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
10
a
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11

_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
a
00
a
01

_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01

Slave 2
_
a
10
a
11

_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11

_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11

_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

_
c
10
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

_
a
20
a
21

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21

_
b
00
b
01
b
10
b
11
_
=
_
c
20
c
21

_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

_
c
20
c
21

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

_
a
30
a
31

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
30
a
31

_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
30
a
31

_
b
00
b
01
b
10
b
11
_
=
_
c
30
c
31

_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

_
c
30
c
31

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11

20
Master-slave matrix multiplication: execution 2
Master
_

_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_

_
_
b
00
b
01
b
10
b
11
_
=
_

_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_

_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Gaussian elimination
Step 1 Locate the leftmost column of A that does not consist entirely of
zeros, if one exists. The top nonzero entry of this column is the
pivot.
Step 2 Interchange the top row with the pivot row, if necessary, so that
the entry at the top of the column found in Step 1 is nonzero.
Step 3 Divide the top row by pivot in order to introduce a leading 1.
Step 4 Add suitable multiples of the top row to all other rows so that all
entries above and below the leading 1 become zero.
Repeat.
21
Gaussian elimination
transforms a matrix to its reduced row-echelon form:
x =
_
x
0
x
1
x
2
x
3
_
y =
_
y
0
y
1
y
2
y
3
_
y =
_

_
_
0 0
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x3/x2
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 0
0 1
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x1/x0
0 0
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
_
1 0
0 1
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
22
Gaussian elimination
transforms a matrix to its reduced row-echelon form:
x =
_
x
0
x
1
x
2
x
3
_
y =
_
y
0
y
1
y
2
y
3
_
y =
_

_
_
0 0
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x3/x2
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 0
0 1
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x1/x0
0 0
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
_
1 0
0 1
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
22