Data Acquisition, Representation

and Reconstruction
Jian Huang, CS 594, Spring 2002

This set of slides references slides developed by Profs.
Machiraju (Ohio State), Torsten Moeller (Simon Fraser) and
Han-Wei Shen (Ohio State).
Acquisition Methods
X-Rays
Computer Tomography (CT or CAT)
MRI (or NMR)
PET / SPECT
Ultrasound
Computational
X-Rays
photons produced by an electron beam
similar to visible light, but higher energy!
X-Rays - Physics
Associated with inner shell electrons
As the electrons decelerate in the target through
interaction, they emit electromagnetic radiation in
the form of x-rays.
patient between an x-ray source and a film ->
radiograph
cheap and relatively easy to use
potentially damaging to biological tissue
X-Rays - Visibility
bones contain heavy atoms -> with many electrons,
which act as an absorber of x-rays
commonly used to image gross bone structure and
lungs
excellent for detecting foreign metal objects
main disadvantage -> lack of anatomical structure
all other tissue has very similar absorption
coefficient for x-rays
X-Rays - Images
CT or CAT - Principles
Computerized (Axial) Tomography
introduced in 1972 by Hounsfield and Cormack
natural progression from X-rays
based on the principle that a three-dimensional
object can be reconstructed from its two
dimensional projections
based on the Radon transform (a map from an n-
dimensional space to an (n-1)-dimensional space)
CT or CAT - Methods
measures the attenuation of X-rays from many
different angles
a computer reconstructs the organ under study in
a series of cross sections or planes
combine X-ray pictures from various angles to
reconstruct 3D structures
CT - Reconstruction: FBP
Filtered Back Projection
common method
uses Radon transform and Fourier Slice Theorem
f(x,y)
|
y
x
s
g
|
(s)
G
|
()
u
F(u,v)
Spatial Domain Frequency Domain
CT - Reconstruction: ART
Algebraic Reconstruction Technique
iterative technique
attributed to Gordon
Reconstructed
model
Actual Data
Slices
Projection
Back-
Projection
Initial Guess
CT - FBP vs. ART
Computationally cheap
Clinically usually 500
projections per slice
problematic for noisy
projections
Still slow
better quality for
fewer projections
better quality for
non-uniform project.
guided reconstruct.
(initial guess!)
FBP
ART
CT - 2D vs. 3D
Linear advancement (slice by slice)
typical method
tumor might fall between cracks
takes long time
helical movement
5-8 times faster
A whole set of trade-offs
CT or CAT - Advantages
significantly more data is collected
superior to single X-ray scans
far easier to separate soft tissues other than
bone from one another (e.g. liver, kidney)
data exist in digital form -> can be analyzed
quantitatively
adds enormously to the diagnostic information
used in many large hospitals and medical centers
throughout the world
CT or CAT - Disadvantages
significantly more data is collected
soft tissue X-ray absorption still relatively similar
still a health risk
MRI is used for a detailed imaging of anatomy
MRI
Nuclear Magnetic Resonance (NMR) (or Magnetic
Resonance Imaging - MRI)
most detailed anatomical information
high-energy radiation is not used, i.e. save
based on the principle of nuclear resonance
(medicine) uses resonance properties of protons
MRI - polarized
all atoms (core) with an
odd number of protons
have a spin, which leads to
a magnetic behavior
Hydrogen (H) - very
common in human body +
very well magnetizing
Stimulate to form a
macroscopically
measurable magnetic field
MRI - Signal to Noise Ratio
proton density pictures - measures H
MRI is good for tissues, but not for bone
signal recorded in Frequency domain!!
Noise - the more protons per volume unit, the
more accurate the measurements - better SNR
through decreased resolution
PET/SPECT
Positron Emission Tomography
Single Photon Emission Computerized Tomography
recent technique
involves the emission of particles of antimatter by
compounds injected into the body being scanned
follow the movements of the injected compound
and its metabolism
reconstruction techniques similar to CT - Filter
Back Projection & iterative schemes
Ultrasound
the use of high-frequency sound (ultrasonic) waves
to produce images of structures within the human
body
above the range of sound audible to humans
(typically above 1MHz)
piezoelectric crystal creates sound waves
aimed at a specific area of the body
change in tissue density reflects waves
echoes are recorded
Ultrasound (2)
Delay of reflected signal and amplitude
determines the position of the tissue
still images or a moving picture of the inside of
the body
there are no known examples of tissue damage
from conventional ultrasound imaging
commonly used to examine fetuses in utero in
order to ascertain size, position, or abnormalities
also for heart, liver, kidneys, gallbladder, breast,
eye, and major blood vessels
Ultrasound (3)
by far least expensive
very safe
very noisy
1D, 2D, 3D scanners
irregular sampling -
reconstruction problems
Computational Methods (CM)
Computational Field Simulations
Computational Fluid Dynamics -
Flow simulations
Computational Chemistry -
Electron-electron interactions,
Molecular surfaces
Computational Mechanics -
Fracture
Computational Manufacturing -
Die-casting
CM - Approach
(Continuous) physical model
Partial/Ordinary Differential Equation (ODE/PDE)
e.g. Navier-Stokes equation for fluid flow
e.g. Hosted Equations:
e.g. Schrdinger Equation - for waves/quantum
Continuous solution doesnt exist (for most part)
Numerical Approximation/Solution
1. Discretize solution space - Grid generation explicit
2. Replace continuous operators with discrete ones
3. Solve for physical quantities
b x a B b f A a f x g f
xx
< < = = = , ) ( , ) ( : ) (
CM - Methods
Grid Generation
non-elliptical methods:
algebraic, conformal, hyperbolic, parabolic, biharmonic
elliptical methods (based on elliptical PDEs)
Numerical Methods
Newton
Runge-Kutta
Finite Element
Finite Differences
Time Varying
CM - Solutions (Structured)
PDE usually constrained/given at boundary
map from computational to physical space
polar maps, elliptical, non-elliptical
structured grids
computational parametric
physical
f
x
t
]) [ ( t x g f
xx
=
CM - Solutions (Unstructured)
usually scattered data set
Delaunay Triangulation
Element Size Optimization
start with initial tetrahedral grid
interactively insert grid points
insertion guided by curvature and distance to surface
Advancing Front Method
start with boundary
advance boundary towards inside until filled
CM - Grid Types (2)
Multiblock structured grids
multiple structured grids
connected not necessarily structured
hybrid grids
structured + unstructured
chimera grids
multiple structured grids
partially overlapping
hierarchical grids
generated by quad-tree and octree like subdivision
schemes (AKA embedded or semi-structured grids)
CM - Grid Examples
CM - Structured vs. Unstructured
consider discretization points as
samples, points, cells, or voxels
Structured
Addressing - Cell [i,j,k] provides location of neighbors
Boundaries of volume - Easy to determine
Unstructured
No addressing mechanism - adjacency list required
Cannot determine the boundaries easily
Cells never of same size
Cells are hexahedrons, tetrahedrons, curved patches
Synthetic Methods
3D Discretization Techniques Voxelization
Scan Conversion of Geometric Objects
Planes / Triangles
Cylinders
Sphere
Cone
NURBS, Bezier patches
Synthetic Methods
Solid Textures
Hyper Texture - 3D
Textures
Fur
Marble
Hair
Turbulent flow
3D Regular grid has
texture values
Grid Types
uniform rectilinear regular curvilinear
Structured Grids:
regular irregular hybrid curved
Unstructured Grids:
Data Representation
Regular
Contour Stacks
Raster
RLE
raster Point list
RLE
Data Representation (2)
Polygon mesh, Curvilinear, Unstructured
Vertex list
Adjacency list of cells (not for curvilinear)
No convenient structure
Compressed Grids - RLE, JPEG, Wavelets
Multi-resolution Grids
Data Characteristics
Large Data Sets
Modest Head 256
3
= 16Mbytes !!
Visible Human - MRI (256
2
) + CT (512
2
) + photo
Male - 15GB (1mm slice dist.); Female - 40GB (0.33mm)
Noisy - Ultrasound (How about CFD, CAGD ?)
Band-limited - Textured Images (Mandrill)
w < wc
FFT
Frequency
0
w
c

Noisy Data set
Data Characteristics
Limited Dynamic Range - values between min, max
populated with unequal probability
Non-Uniform Spatial Occupancy -
Quadtree/Octree
Rectangular spatial subdivision
Data Objects
Data Object: dataset
(representation of information)

Structures: how the information is organized
- Topology
- Geometry

Attributes: store the information we want
to visualize. e.g. function values
Data Objects: structures
Topology:
- Invariant under geometric transformation
(rotation, translation, scaling etc)
- Topological structures: Cells

Geometry:
- The instantiation of the topology
- Geometric structures: cells with
positions in 3D space
Cell Types for Unstructured Grid
(a) vertex
(b) Polyvertex (c) line (d) polyline (e) triangle
(e) Quadrilateral (e) Polygon (f) Tetrahedron
(f) Hexahedron
And more (I am too tired drawing now )
Data Attributes
The information stored at each vertex of the
cell
Scalars: temperature, pressure, etc
Vector: velocity
Normal: surface directions
Texture coordinates: graphics specific
Tensors: matrices
Where are we now ...
Data Object: dataset
(representation of information)

Structures: how the information is organized
- Topology
- Geometry

Attributes: store the information we want
to visualize. e.g. function values
Remember:
Interpolation (1)
Visualization deals with discrete data
(a) vertex
(b) Polyvertex (c) line (d) polyline (e) triangle
(e) Quadrilateral (e) Polygon (f) Tetrahedron
(f) Hexahedron
Values defined only at cell vertices
Color Mapping
R G B
Values at vertices Color lookup Result
table
v1
v2
v3
...
Interpolation (2)
We often need information at positions other than cell
vertices
Interpolation: compute data
from known points
p
P = ?
10 13
9 12
Interpolation (3)
Three essential information:
Cell type
Data values at cell vertices
Parametric coordinates of the point p




D = E Wi * di
i=0
n-1
di
di: cell point value
Wi: weight (E wi = 1)
D: interpolated result
Interpolation (4)
Parametric Coordinates:
Used to specify the location of a point within a cell
D = E Wi * di
i=0
n-1
W0 = (1-r)
W1 = r
(a) line
r = 0
r = 1
0 <= r <=1
d0
d1
r
Interpolation (5)
(b) Triangle
s
r
p0 p1
p2
r=0
s=0
r+s = 1 (why?)
W0 = 1-r-s
W1 = r
W2 = s
Why?
Interpolation (6)
(C) Pixel
p0
p1
p2 p3
r
s
(s,t)
s=0
r=0 r =1
s=1
W0 = (1-r)(1-s)
W1 = r(1-s)
W2 = (1-r)s
W3 = rs
Why?
This is also called bi-linear interpolation
Interpolation (7)
(D) Polygon
p0
p1
p2
p3
p4
p5
Wi = (1/ri) / E(1/ri)
2
2
r3
Weighted distance function
Interpolation (8)
(D) Tetrahedron
r
s
t
W0 = 1-r-s-t
W1 = r
W2 = s
W3 = t
p1
p0
p2
p3
Interpolation (9)
(D) Cube (voxel)
r
t
s
W0 = (1-r)(1-s)(1-r)
W1 = r(1-s)(1-t)
W2 = (1-r)s(1-t)
W3 = rs(1-r)
W4 = (1-r)(1-s)t
W5 = r(1-s)t
W6 = (1-r)st
W7 = rst
p0
p1
p2 p3
p4 p5
p6 p7
Interpolation (10)
The interpolation function can be used to calculate the
geometric position as well.

That is, given (r,s,t), calculate the global coordinates

Local to global coordinate transformation:


P = E Wi * Pi
i=0
n-1
Interpolation (11)
How to get (r,s,t) ?

Line, Pixel, Cube are all trivial

Triangle, Tetrahedron can be solved analytically

Qudrilateral or Hexahedra need numerical method

P = E Wi * Pi
i=0
n-1
Known: P, Pi
Unkown: Wi (i.e. r,s,t)
Contours
3 7
10 4
C = 6
Interpolation
Given:
Needed:
2D
1D
Given:
Needed:
The Need for Interpolation
Interpolation is needed throughout the
visualization process
Rendering?
Extracting iso-contours?
Shading?
Texture mapping?

Anytime when resampling is needed
General Process
Original function
Sampled function
Reconstructed
Function
Acquisition
Re-sampled function
Resampling
How? - Convolution
Spatial Domain:
Mathematically:
f(x)*h(x) =
( ) ( )dt t x g t f
}

Frequency Domain:
( ) ( ) e e H F
Evaluated at discrete
points (sum)
Multiplication:
Convolution:
Reconstruction
Mathematically:
f(x)*h(x) = (Ef[i])*h(x)
Ef[i]
h(x)
General Process - Frequency Domain
Acquisition
Resampling
Original function
Sampled function
Reconstructed
Function
Re-sampled function
Pre-Filtering
Pre-Filtering
Reconstruction
Original function
Band-limited function
Sampled
Function
Reconstructed function
Ideal Reconstruction with
Sinc function
Spatial Domain:
convolution is exact
Frequency Domain:
cut off freq. replica
( ) ( ) 0 = x f x f
r
( )
( )
x
x
x
t
t
=
sin
Sinc
-0. 4
-0. 2
0
0. 2
0. 4
0. 6
0. 8
1
-25 -20 -15 -10 -5 0 5 10 15 20 25
0.65
0.7
0.75
0.8
0.85
0.9
0.95
0 0.05 0.1 0.15 0. 2 0. 25 0.3 0.35 0.4 0.45 0.5
Reconstructing Derivatives
Spatial Domain:
convolution is exact
Frequency Domain:
cut off freq. replica
( ) ( ) 0 =
'
x f x f
d
r
( )
( ) ( )
2
sin cos
Cosc
x
x
x
x
x
t
t

t
=
-1.5
-1
-0.5
0
0.5
1
1.5
-25 -20 -15 -10 -5 0 5 10 15 20 25
Cosc(t)
0.65
0.7
0.75
0.8
0.85
0.9
0.95
0 0.05 0.1 0.15 0. 2 0. 25 0.3 0.35 0.4 0.45 0.5
Possible Errors
Post-aliasing
reconstruction filter passes frequencies beyond the
Nyquist frequency (of duplicated frequency spectrum) =>
frequency components of the original signal appear in
the reconstructed signal at different frequencies
Smoothing
frequencies below the Nyquist frequency are attenuated
Ringing (overshoot)
occurs when trying to sample/reconstruct discontinuity
Anisotropy
caused by not spherically symmetric filters
How Good? = Error
Spatial Domain:
local error
asymptotic error
numerical error
Frequency Domain:
global error
visual appearance
blurring
aliasing
smoothing
Approximation
Theory/Analysis
Signal Processing
Sources of Aliasing
Non-bandlimited signal
Low sampling rate (below Nyquist)
Non perfect reconstruction
Reconstruction Kernels
stop band pass band
Smoothing
error
Postaliasing
error
Ideal filter
filter
The spatial extent of
reconstruction kernels,
or interpolation basis
functions, depend on
the cut-off frequency
as well.
Reconstruction Kernels
Nearest Neighbor (Box)
0
0.2
0.4
0.6
0.8
1
-6 -4 -2 0 2 4 6
0
0.2
0.4
0.6
0.8
1
-6 -4 -2 0 2 4 6
Triangular func
Sinc
Gaussian
+ many others
Spatial d. Frequency d.
Higher Dimensions
An-isotropic Filters
(radially symmetric)
separable filters
( ) ( ) ( ) y h x h y x h = ,
( ) ( )
2 2
, y x h y x h + =
Interpolation (an example)
Very important; regardless of algorithm
expensive => done very often for one image
Requirements for good reconstruction
performance
stability of the numerical algorithm
accuracy
Nearest
neighbor
Linear
Put Things in Perspective
In visualization, need to use continuous space functions.
But can only work with discrete data
So, lets reconstruct from discrete data to continuous space
(convolution) and resample
Interpolation is doing the same thing. Computing one data
point in the resulting function, say, at x
1
.
So, which reconstruction kernel (basis function) does
linear/bilinear/tri-linear interpolations use?