05 15slides
05 15slides
climate modeling
molecular dynamics
matrix algorithms
matrix factorization
climate modeling
molecular dynamics
matrix algorithms
matrix factorization
climate modeling
molecular dynamics
matrix algorithms
matrix factorization
climate modeling
molecular dynamics
matrix algorithms
matrix factorization
nondeterminism
deadlocks
race conditions
additional sources of nondeterminism from MPI
problem of test oracles
nondeterminism
deadlocks
race conditions
additional sources of nondeterminism from MPI
problem of test oracles
nondeterminism
deadlocks
race conditions
additional sources of nondeterminism from MPI
problem of test oracles
nondeterminism
deadlocks
race conditions
additional sources of nondeterminism from MPI
problem of test oracles
oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear
insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing
oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear
insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing
oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear
insertion of barriers
6
Current methods used to discover and correct problems in
parallel numeric programs
1. testing
oracle problem
2. parallel debuggers
3. rewriting code in the hope that the problem will disappear
insertion of barriers
6
Our contribution
A method to verify parallel numerical programs.
veries that the parallel program is functionally equivalent to a
trusted sequential version
x
1
x
6
=
+
+
0.0 x
1
x
6
x
0
x
4
0.0 + (x
0
x
4
) + x
1
x
6
= (0.0 + (x
0
x
4
)) + x
1
x
6
8
How do we model oating-point computation?
one double-precision oating-point variable has 2
64
possible states
abstraction?
Input: symbolic constants x
0
, x
1
, . . .
Output: symbolic expressions in the x
i
+
0.0
x
0
x
4
+
x
1
x
6
=
+
+
0.0 x
1
x
6
x
0
x
4
0.0 + (x
0
x
4
) + x
1
x
6
= (0.0 + (x
0
x
4
)) + x
1
x
6
8
How do we represent symbolic expressions?
Value numbering
place all symbolic expressions in an expression table
to evaluate x + y:
to evaluate x + y:
to evaluate x + y:
if (x
0
= 0) {. . .} else {. . .}
only want to compare the result of an execution path in the parallel
program to the result of a corresponding path in the sequential
program
11
The path correspondence problem
the programs may contain branches on expressions that involve the
symbolic variables
if (x
0
= 0) {. . .} else {. . .}
only want to compare the result of an execution path in the parallel
program to the result of a corresponding path in the sequential
program
11
Path conditions and domains
enumerate all paths through the sequential program
_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j
i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Path conditions and domains
enumerate all paths through the sequential program
_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j
i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Path conditions and domains
enumerate all paths through the sequential program
_
f
1
(x) if p
1
(x)
f
2
(x) if p
2
(x)
.
.
.
.
.
.
f
n
(x) if p
n
(x)
each p
i
determines a path domain D
i
= {x | p
i
(x)}
D
i
D
j
= if i = j
i
D
i
is the whole input space
Solution to path correspondence problem:
1. discover path conditions/domains automatically
2. for each domain D
i
: compare symbolic results of sequential and
parallel programs for all inputs in D
i
12
Modeling conditional statements
To model the statement if (x
0
= 0) {. . .} else {. . .}
p true; /* path condition */
.
.
.
b (p, x
0
= 0);
if (b = 1) {
if (choose()) {
b 1; p p (x
0
= 0);
} else {
b 0; p p (x
0
= 0);
}
}
if (b = 1) { . . . } else { . . . }
(p, q) =
_
_
1 if p q
0 if p q
1 if dont know
for boolean-valued symbolic
expressions p, q
13
The method
1. construct symbolic model M
seq
of sequential program
input: x, output: y
, path condition: p
p true; M
seq
; M
par
; assert(y = y
);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:
a trace through M
seq
a trace through M
par
14
The method
1. construct symbolic model M
seq
of sequential program
input: x, output: y
, path condition: p
p true; M
seq
; M
par
; assert(y = y
);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:
a trace through M
seq
a trace through M
par
14
The method
1. construct symbolic model M
seq
of sequential program
input: x, output: y
, path condition: p
p true; M
seq
; M
par
; assert(y = y
);
4. use model checker to verify the assertion in M can never be violated
The model checker returns either
Yes: the property holds, or
No + counterexample:
a trace through M
seq
a trace through M
par
14
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand
x + y = y + x
1x = x = x1
x + 0 = x = 0 + x
3. Real
all of IEEE
(x + y) + z = x + (y + z)
(xy)z = x(yz)
. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand
x + y = y + x
1x = x = x1
x + 0 = x = 0 + x
3. Real
all of IEEE
(x + y) + z = x + (y + z)
(xy)z = x(yz)
. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand
x + y = y + x
1x = x = x1
x + 0 = x = 0 + x
3. Real
all of IEEE
(x + y) + z = x + (y + z)
(xy)z = x(yz)
. . .
15
Numerical Issues
dierent symbolic expressions are equivalent over real numbers
example: ((x
3
+ x
1
) + x
2
) + x
0
and ((x
0
+ x
1
) + x
2
) + x
3
solution: provide dierent modes
1. Herbrand
x + y = y + x
1x = x = x1
x + 0 = x = 0 + x
3. Real
all of IEEE
(x + y) + z = x + (y + z)
(xy)z = x(yz)
. . .
15
Preliminary experimental results
implemented as extension to SPIN
wrote our own simple symbolic algebra library and light-weight
theorem-prover
found bug in one example (Jacobi iteration)
matrix
multiplication
Gaussian
elimination
Jacobi
iteration
Monte
Carlo
processes 6 6 17 9
path domains 1 13327 4 4
symbolic expressions 2202 247656 8239 1232
input vector size 200 36 1333 99
output vector size 100 36 36 1
states (10
3
) 4443 16114 6295 3112
memory (MB) 217 801 362 279
time (s) 506 3224 9846 738
16
Related Work
1. Ball and Rajamani. Automatically validating temporal safety
properties of interfaces. SPIN 2001.
2. Khurshid, Pasareanu, and Visser. Generalized symbolic execution for
model checking and testing. TACAS 2003.
3. Pasareanu and Visser. Verication of Java programs using symbolic
execution and invariant generation. SPIN 2004.
4. Elmas, Tasiran, and Qadeer. VYRD: verifying concurrent programs
by runtime renement-violation detection. PLDI 2005.
17
Conclusion
Strengths of method
can establish the correctness of a parallel numerical program. . .
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
b00 b01
b10 b11
b00 b01
b10 b11
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
00
a
01
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
10
a
11
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
a
10
a
11
_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
10
c
11
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
20
a
21
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
_
a
30
a
31
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
30
a
31
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
30
a
31
_
b
00
b
01
b
10
b
11
_
=
_
c
30
c
31
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
_
c
30
c
31
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
c
20
c
21
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
20
c
21
19
Master-slave matrix multiplication
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
19
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
b00 b01
b10 b11
b00 b01
b10 b11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
00
a
01
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
a
10
a
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11
_
b
00
b
01
b
10
b
11
_
=
_
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
a
00
a
01
_
b
00
b
01
b
10
b
11
_
=
_
c
00
c
01
Slave 2
_
a
10
a
11
_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
10
a
11
_
b
00
b
01
b
10
b
11
_
=
_
c
10
c
11
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
_
c
10
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
_
a
20
a
21
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
20
a
21
_
b
00
b
01
b
10
b
11
_
=
_
c
20
c
21
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
_
c
20
c
21
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
_
a
30
a
31
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
30
a
31
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
a
30
a
31
_
b
00
b
01
b
10
b
11
_
=
_
c
30
c
31
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
_
c
30
c
31
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
01
c
11
20
Master-slave matrix multiplication: execution 2
Master
_
_
a
00
a
01
a
10
a
11
a
20
a
21
a
30
a
31
_
_
_
b
00
b
01
b
10
b
11
_
=
_
_
c
00
c
01
c
10
c
11
c
20
c
21
c
30
c
31
_
_
Slave 1
_
_
b
00
b
01
b
10
b
11
_
=
_
Slave 2
_
_
b
00
b
01
b
10
b
11
_
=
_
20
Gaussian elimination
Step 1 Locate the leftmost column of A that does not consist entirely of
zeros, if one exists. The top nonzero entry of this column is the
pivot.
Step 2 Interchange the top row with the pivot row, if necessary, so that
the entry at the top of the column found in Step 1 is nonzero.
Step 3 Divide the top row by pivot in order to introduce a leading 1.
Step 4 Add suitable multiples of the top row to all other rows so that all
entries above and below the leading 1 become zero.
Repeat.
21
Gaussian elimination
transforms a matrix to its reduced row-echelon form:
x =
_
x
0
x
1
x
2
x
3
_
y =
_
y
0
y
1
y
2
y
3
_
y =
_
_
_
0 0
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x3/x2
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 0
0 1
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x1/x0
0 0
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
_
1 0
0 1
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
22
Gaussian elimination
transforms a matrix to its reduced row-echelon form:
x =
_
x
0
x
1
x
2
x
3
_
y =
_
y
0
y
1
y
2
y
3
_
y =
_
_
_
0 0
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0 x
3
= 0
_
0 1
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x3/x2
0 0
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 0
0 1
_
if x
0
= 0 x
2
= 0 x
1
= 0
_
1 x1/x0
0 0
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
_
1 0
0 1
_
if x
0
= 0 x
3
x
2
(x
1
/x
0
) = 0
22