BIOENG 1330/2330

BIOMEDICAL IMAGING
FALL 2015

Lecture 4
09/10/2015

Sowmya Aggarwal
Ker-Jiun Wang
University of Pittsburgh

1
Briefly cover the following

- Program Flow Control

- Matrix/Vector Algebra

- Gaussian Elimination

- Curve Fitting

2
Program Flow Control in MATLAB

1) Logical Operator
Conditional Mathematical Symbol MATLAB Symbol
Relational Operators
Equal = ==
Not equal ≠ ~=
Less than < <
Greater than > >
Less than or equal ≤ <=
Greater than or equal ≥ >=

Logical operators
And AND & or &&
Or OR | or||
Not NOT ~
3
Program Flow Control in MATLAB - Branching – If, for, while, switch

1) if loop 2) for loop

3) while loop 4) switch loop

4
Program Flow Control in MATLAB - Branching – If, for, while, switch

1) if loop 2) for loop

3) while loop 4) switch loop

5
Program Flow Control in MATLAB - Branching – If, for, while, switch

1) if loop 2) for loop

3) while loop 4) switch loop

6
Program Flow Control in MATLAB - Branching – If, for, while, switch

1) if loop 2) for loop

3) while loop 4) switch loop

7
Parallel Programming Example - Parfor

Compare with
‘for’ using tic
toc before and
after the loop

Exit loops early – put ‘break’ command within the loop to get out of ‘for’ or ‘while’ loops

break

8
Briefly cover the following

- Program Flow Control

- Matrix/Vector Algebra

- Gaussian Elimination

- Curve Fitting

9
Scalar
• A quantity (variable), described by a single real
number
Image Histogram (Single MRI Slice)

e.g. Intensity of each

Counts
voxel in an MRI scan

Intensity

Single MRI Slice

Voxel
Linear Algebra & Matrices, MfD 2010
 Vector
• Not a physics vector (magnitude, direction)
Intensity Plot (for the same (y,z) coordinates)

EXAMPLE:

Intensity of voxel
Slice # along x - axis

 x1 
 x 2
Then:
VECTOR= A “column” of numbers  
representing intensity values along
X-axis  xn
3D MRI Image (T1
Linear Space)& Matrices, MfD 2010
Algebra
 Matrix Definition – MATLAB Conventions

1) Vector A– m x 1 array, m = 4 Vector transpose, AT = Row Vector, 1 x n , n=4

MATLAB MATLAB
A = [1;2;3;4]; A = A’;

2) m x n matrix, m=3, n=4 This is the sequence

MATLAB MATLAB indexes array
A = [1 4 7 10;2 5 8 11;3 6 9 12]; elements

3) Diagonal Matrix, 4 x 4
(Square Matrix)
MATLAB
A = [1 0 0 0;0 6 0 0;0 0 11 0; 0 0 0 16];

4) Identity Matrix
MATLAB
A = [1 0 0 0;0 1 0 0;0 0 1 0; 0 0 0 1];

This example is upper triangular matrix MATLAB

5) Triangular Matrix A = [1 5 9 13;0 6 10 14;0 0 11 15; 0 0 0 16];
12
 Matrix Algebra

1) Matrix Addition: MATLAB: A+B = C

2) Matrix Multiplication: MATLAB: A*B = C

13
 Matrix Algebra

1) Matrix Addition: MATLAB: A+B = C

2) Matrix Multiplication: MATLAB: A*B = C

14
Matrix Algebra

1) Matrix Inverse, A-1 2) Determinant of a matrix, D = |A|

1
A-1 = (adjoint of A)
det 𝐴

A-1A = AA-1 =1

3) Eigen Vectors and Eigen Values 4) System of Equations, Ax = b

A x = b

15
Reference material: Numerical Recipes in C – The Art of Scientific Computing, William H Press et. el.
Matrix Algebra

1) Matrix Inverse, A-1 2) Determinant of a matrix, D = |A|

1
A-1 = (adjoint of A)
det 𝐴

A-1A = AA-1 =1

3) Eigen Vectors and Eigen Values 4) System of Equations, Ax = b

A x = b

16
Reference material: Numerical Recipes in C – The Art of Scientific Computing, William H Press et. el.
Matrix Algebra

1) Matrix Inverse, A-1 2) Determinant of a matrix, D = |A|

1
A-1 = (adjoint of A)
det 𝐴

A-1A = AA-1 =1

3) Eigen Vectors and Eigen Values 4) System of Equations, Ax = b

A x = b

17
Reference material: Numerical Recipes in C – The Art of Scientific Computing, William H Press et. el.
Matrix Algebra

1) Matrix Inverse, A-1 2) Determinant of a matrix, D = |A|

1
A-1 = (adjoint of A)
det 𝐴

A-1A = AA-1 =1

3) Eigen Vectors and Eigen Values 4) System of Equations, Ax = b

A x = b

18
Reference material: Numerical Recipes in C – The Art of Scientific Computing, William H Press et. el.
19
20
21
MATLAB Function Description
det(A) Computes determinant of a square matrix, A

diag(A) Creates diagonal of a square matrix, A

Dot(x,y) Computes dot product of two vectors, x and y

eig(A) Computes eigen vectors and eigen values

eye(n) Created identity matrix, of size n x n

inv(A) Computes inverse of a matrix, A

length(x) Computes length of a vector, x

max(x), min(x) Determines maximum and minimum value in a vector

rank(A) Estimates the number of linearly independent rows/cols of matrix, A

sort(A) Sorts elements of an array, A, in ascending order. More options available.

ones(n,m), zeros(n,m) Creates matrix of ones and zeros respectively of size n x m

eye(n) Created identity matrix, of size n x n

linspace Creates equally spaced elements of a vector

logspace Creates equally spaced elements of a vector on a log10 scale

repmat Replicates array

mesh Creates surface plot

plot(X,Y) Plots curves in a plane using axes data, X and Y 22

Briefly cover the following

- Program Flow Control

- Matrix/Vector Algebra

- Gaussian Elimination

- Curve Fitting

23
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.

24
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.

25
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.

26
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.

Back
Substitution
can begin
here

27
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.

Gauss
Reduced Jordanian
Row
Echelon form
28
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method

• Transforms the linear system, Ax = b, into a solvable “triangular” system.
• Systematically eliminates unknowns from the equations until only a equation with
only one unknown is left.
• Algorithm: For an mxn matrix A and m x 1 vector, b:
o To transform A to upper triangular form, begin transforming a row(ith row) below
pivot (kth) row, to eliminate kth coefficient on ith row. For this: Algorithm
o Multiply pivot(kth)row by a constant, λ = Aik/Akk ,and subtract it from ith row Checkpoints
o Overall, numerically both A and b are to be transformed as:
λ = Aik/Akk
Aij  Aij – λAkj , j = k, k + 1, … , n First loop
bi  bi – λbk

• This results into a triangular system, that’s solved by Backward-Substitution.

Numerically: xn  bn / Ann
 n  New loop: Determine xi for ith equation
xi   bi   Aij x j  / Aii , i  n  1, ,1
 j i 1 
Reference material: Numerical Recipes in C – The Art of Scientific Computing, William H Press et. el. 29
 Matrix Algebra – Solving a System of Equations, Ax = b

Gaussian Elimination Method: Coding Illustration

% Gauss Elimination m x n matrix
% User defined linear equation Ax = b, where A is m x n matrix, b is m x 1 [A|B]
vector
% We create an augmented matrix m x (n+1) by appending the vector b with A
% Let rows = m, cols = n+1 for the augmented matrix, Aug
% Example : A = [4 -2 1; -2 4 -2; 1 -2 4];
% b = [11 -16 17]';

Aug = [A b]; % create the augmented matrix from A and b

[rows cols] = size(Aug);

% Check for singularity

if det(A)==0 || det(A)<norm(A,Inf) || cond(A) > 50
error('system of equations has infinitely many solutions or no
solutions');
end

npivots = rows - 1; % number of initial pivot equations

nlambda = npivots; % number of lambdas required for first pivot equation

for i = 1:npivots % iterating over the pivot row

for j = i+1:rows % iterating over the rows to be transformed
lambda = Aug(j,i)/Aug(i,i);
Aug(j,:) = Aug(j,:) - lambda.*Aug(i,:); Loop 1
end
end

Can you code loop 2?

30
Briefly cover the following

- Program Flow Control

- Matrix/Vector Algebra

- Gaussian Elimination

- Curve Fitting

31
 Curve Fitting – Interactive
Approximate a function (eg. polynomial/spline) that explains the data
Steps:
1) Load some data at the MATLAB® command line.
load hahn1
Useful Tip: Avoid overfitting
2) Open the Curve Fitting app. Enter:
cftool

3) In the Curve Fitting app, select X Data and Y Data.

Curve Fitting app creates a default interpolation fit to the data.

4) Choose a different model type using the fit category drop-down list, e.g., select Polynomial, Fourier.

5) Try different fit options for your chosen model type.

Select File > Generate Code.
Curve Fitting app creates a file in the Editor containing MATLAB code to recreate all fits and plots in your
interactive session.

Create a fit using the fit function, specifying the variables and a model type (in this case rat23 is the model type).
f = fit( temp, thermex, 'rat23' )

6) Plot your fit and the data.

plot( f, temp, thermex )
f( 600 ) 32
33
34
35
36
Curve Fitting – Approximate a function (eg. polynomial/spline) that explains the data
MATLAB COMMAND FUNCTION USAGE

polyfit Fits a polynomial to an array of values a = polyfit(x,y,n)

Evaluates a polynomial at an array of
polyval values
y = polyval(a,xnew)
Interpolates cubic spline to arrays of
spline coordinates
Y = spline(x,y,X)

interpl Interpolates between pairs of coordinates Y = interpl(x,y,X)

polyarea Calculates the area of a polygon polyarea(x,y)
fft Determines Fourier transform F = fft(y,N)
ifft Determines inverse Fourier y = ifft(F,N)

Why fitting is also useful? To interpolate value at a previously

unknown location (coordinates)

Useful Tip: Avoid overfitting

37
Curve Fitting – Command line
MATLAB FUNCTION USAGE Where
COMMAND

Function

y(x) = a1xn + a2xn-1 + … + anx + an+1

Fits a polynomial to an x is input data, independent variable

polyfit array of values
a = polyfit(x,y,n) y is the output/dependent variable
ai are the coefficients of the polynomial
n degree of polynomial

So, a = polyfit(x,y,n) gets us array of ‘a’

To evaluate the above equation after

determining coefficients, a, we
Evaluates a polynomial determine the value of y for a new data
polyval at an array of values
y = polyval(a,xnew)
entry, xnew:

y = polyval(a,xnew)

38
39
 See this on slideshow

Convolution: Rectangular Function Convolution: Gaussian Function

Reference: Wolfram 40
Curve Fitting – Approximate a function (eg. polynomial/spline) that explains the data
MATLAB FUNCTION USAGE Where
COMMAND

Determines Fourier y is the real function

fft transform
F = fft(y,N)
N number of sampled data points
Shift zero-frequency rearranges the outputs of fft, fft2, and fftn by
fftshift component to center F1 = fftshift(F) moving the zero-frequency component to the
of spectrum center of the array
F is the fourier transform of y
Determines inverse
ifft Fourier
y = ifft(F,N) y can be reconstructed from its F
N is the original number of sampled points

41
Curve Fitting – Approximate a function (eg. polynomial/spline) that explains the data
MATLAB FUNCTION USAGE Where
COMMAND

Determines Fourier y is the real function

fft transform
F = fft(y,N)
N number of sampled data points
Shift zero-frequency rearranges the outputs of fft, fft2, and fftn by
fftshift component to center F1 = fftshift(F) moving the zero-frequency component to the
of spectrum center of the array
F is the fourier transform of y
Determines inverse
ifft Fourier
y = ifft(F,N) y can be reconstructed from its F
N is the original number of sampled points

42
MATLAB Demonstration: Apply Laplace Filter to an Image
MATLAB FUNCTION USA Where
COMMAND
GE
fft2 Determines Fourier transform F = fft2(y) y is the 2D real function
ifft2 Determines inverse Fourier y = ifft2(F) F is the fourier transform of y

%% Load Image and apply Spatial Transforms

% Change directory to matlab's inbuilt images
cd('C:\Program Files\MATLAB\R2015a\toolbox\images\imdata') %check version of matlab

% Load image of moon

f = imread('moon.tif');
% show original image
imshow(f)
% Convert data to float
f = double(f);
kernel = fspecial('laplacian',0);

%apply filter
tic
g = f - imfilter(f, kernel, 'replicate');
toc
%view results
figure, imshowpair(f,g,'montage');

43
44
% Now Apply Transforms to images in Frequency Domain
% fourier transform of the image
padsize= =[546 368];

F = fft2(f,padsize(1),padsize(2));

% fourier transform of the filter

kernel = fspecial('laplacian',0);
H = fft2(kernel,padsize(1),padsize(2));

% convolution process
tic
G = F .* H;
toc

g = real(ifft2(G));
figure, imshowpair(f,g,'montage');

45
Time elapsed in spatial convolution

- 0.195813 seconds

Time Elapsed in Frequency Domain Convolution

- 0.012137 seconds

Why?

46
 Convolution
in Spatial Domain

Convolution
in Frequency
Domain

47
HMM (Hidden Markov model)

At each time step t,

the only thing that affects future states 1 2

is the current state t

P(t+1 = k | “whatever happened so far”) = K …

P(t+1 = k | 1, 2, …, t, x1, x2, …, xt) =
P(t+1 = k | t)

48
 Generating a hidden state sequence by MHH
model
Observable state: x = x1……xN, Hidden state:  = 1, ……, N
Given a HMM, we can generate a sequence of hidden states of length n as
follows:
1 1 1 … 1
1. Start at state 1 according to prob a01 a02 2 2 2 … 2
2. Emit observable state x1 according to 0 … … … …
prob e1(x1) K K K … K

3. Go to state 2 according to prob a12 e2(x1)

4. … until emitting xn x1 x2 x3 xn

49
Likelihood of a Hidden States Sequence
1 1 1 … 1
Given an observable state sequence x = x1……xN
and a hidden state sequence  = 1, ……, N, 2 2 2 … 2

… … … …
K K K … K
To find how likely this scenario is:
(given our HMM)
x1 x2 x3 xK
P(x, ) = P(x1, …, xN, 1, ……, N)
= P(xN | N) P(N | N-1) ……P(x2 | 2) P(2 | 1) P(x1 | 1) P(1)
= a01 a12……aN-1N e1(x1)……eN(xN)

50
The Viterbi Algorithm 1 1 1 1
…

GIVEN x = x1x2……xN 2 2 2 … 2

Find  = 1, ……, N, … … … …

to maximize P[ x,  ] K K K … K

x1 x2 x3 xK
* = argmax P[ x,  ]
Maximizes a01 e1(x1) a12……aN-1N eN(xN) Given that we end up in
state k at step i,
maximize product to the
Dynamic Programming! left and right
Vk(i) = max{1… i-1} P[x1…xi-1, 1, …, i-1, xi, i = k]
= Prob. of most likely sequence of states ending at state
i = k
51
The Viterbi Algorithm 1 1 1 1
…
Inductive assumption: 2 2 2 … 2
Given that for all states k, and for a fixed position i, … … … …
Vk(i) = max{1… i-1} P[x1…xi-1, 1, …, i-1, xi, i = k] K K K K
…

What is Vl(i+1)? x1 x2 x3 xK

From definition,
Given that we end up in
Vl(i+1) = max{1… i}P[ x1…xi, 1, …, i, xi+1, i+1 = l ] state k at step i,
= max{1… i}P(xi+1, i+1 = l | x1…xi, 1,…, i) P[x1…xi, 1,…, i] maximize product to the
left and right
= max{1… i}P(xi+1, i+1 = l | i ) P[x1…xi-1, 1, …, i-1, xi, i]
= maxk [P(xi+1, i+1 = l | i=k) max{1… i-1}P[x1…xi-1,1,…,i-1, xi,i=k]]
= maxk [ P(xi+1 | i+1 = l ) P(i+1 = l | i=k) Vk(i) ]
= el(xi+1) maxk akl Vk(i)
52
The Viterbi Algorithm 1 1 1 1
…
Input: x = x1……xN
2 2 2 … 2
Initialization:
V0(0) = 1 (0 is the imaginary first position) 0 … … … …
Vk(0) = 0, for all k > 0 K K K … K

Iteration: x1 x2 x3 xK

Vj(i) = ej(xi)  maxk akj Vk(i – 1)

Ptrj(i) = argmaxk akj Vk(i – 1) Given that we end up in
state k at step i,
maximize product to the
Termination: left and right
P(x, *) = maxk Vk(N)

Traceback:
N* = argmaxk Vk(N)
i-1* = Ptri (i) 53
The Viterbi Algorithm
Similar to “aligning” a set of states to a sequence
x1 x2 x3 ………………………………………..xN
Computational complexity:
State 1
Time: Space: 2
O(K2N) O(KN) Vj(i)

K
Underflows are a significant problem
P[ x1,…., xi, 1, …, i ] = a01 a12……ai e1(x1)……ei(xi)
These numbers become extremely small – underflow
Solution: Take the logs of all values

Vl(i) = log ek(xi) + maxk [ Vk(i-1) + log akl ]

54
The Viterbi Algorithm
Example
The Noisy Channel Model

• Search through space of all possible sentences.

• Pick the one that is most probable given the waveform.
55
The Viterbi Algorithm
MATLAB Example
trans = [0.95,0.05;
0.10,0.90];
emis = [1/6 1/6 1/6 1/6 1/6 1/6;
1/10 1/10 1/10 1/10 1/10 1/2];

[seq,states] = hmmgenerate(100,trans,emis);
estimatedStates = hmmviterbi(seq,trans,emis);

[seq,states] = ...
hmmgenerate(100,trans,emis,...
'Statenames',{'fair';'loaded'});
estimatesStates = ...
hmmviterbi(seq,trans,emis,...
'Statenames',{'fair';'loaded'});
56
Gram-Schmidt procedure

• given independent vectors , , G-S procedure finds

orthonormal vectors s.t.

• thus, is an orthonormal basis for

• rough idea of method: first orthogonalize each vector w.r.t. previous

ones; then normalize result to have norm one

57
Gram-Schmidt procedure
•step1.a

•step1.b

•step2.a

•step2.b

•step3.a

•step3.b

•etc.
58
Gram-Schmidt procedure
• For we have

• Written in matrix form: , where , , ,

• Called QR decomposition (or factorization) of A. and R is upper

triangular & invertible 59
Gram-Schmidt procedure
function [Q,R]=gschmidt(V)
% Input: V is an m by n matrix of full rank m<=n
% Output: an m-by-n upper triangular matrix R
% and an m-by-m unitary matrix Q so that A = Q*R.
[m,n]=size(V);
R=zeros(n);
R(1,1)=norm(V(:,1));
Q(:,1)=V(:,1)/R(1,1);
for k=2:n
R(1:k-1,k)=Q(:,1:k-1)’*V(:,k);
Q(:,k)=V(:,k)-Q(:,1:k-1)*R(1:k-1,k);
R(k,k)=norm(Q(:,k));
Q(:,k)=Q(:,k)/R(k,k);
end

Or simply type [Q,R] = qr(A)

In MATLAB

60
61