SPRING 2017 1

MEDICAL IMAGE COMPUTING (CAP 5937)

LECTURE 7: Medical Image Segmentation (I)

(Radiology Applications of Segmentation, and Thresholding)

Dr. Ulas Bagci

HEC 221, Center for Research in Computer
Vision (CRCV), University of Central Florida
(UCF), Orlando, FL 32814.
[email protected] or [email protected]
 2

Outline
• Introduction to Medical Image Segmentation, type of
segmentation methods, and definitions
– Recognition & Delineation
• Simplest Segmentation Method(s): Thresholding
– Otsu Thresholding
– Parametric Method
– PET Image Thresholding Methods
• ITM (Iterative Thresholding Method)
 3

Motivation for Image Segmentation

In the last 20 years the computer vision and medical imaging
communities have produced a number of useful algorithms for
localizing object boundaries in images.
 4

Motivation for Image Segmentation

• Content based image retrieval
• Machine Vision
• Medical Imaging applications (tumor delineation,..)
• Object detection (face detection,…)
• 3D Reconstruction
• Object/Motion Tracking
• Object-based measurements such as size and shape
• Object recognition (face recognition,…)
• Fingerprint recognition,
• Video surveillance
• …
 5

Segmentation Tools in Radiology Applications

• 3D views to visualize structural information and spatial
anatomic relationships is a difficult task, which is usually
carried out in the clinician’s mind.
 6

Segmentation Tools in Radiology Applications

• 3D views to visualize structural information and spatial
anatomic relationships is a difficult task, which is usually
carried out in the clinician’s mind.
• Image-processing tools provide the surgeon with interactively
displayed 3D visual information.
 7

Segmentation Tools in Radiology Applications

Credit: Kaus, et al. Radiology 2001.

 8

Segmentation Tools in Radiology Applications

• Determination of the volumes of abdominal solid organs and focal lesions
has great potential importance (liver, spleen, …).
• Monitoring the response to therapy and the progression of neoplastic
disease and preoperative examination of living liver donors are the most
common clinical applications of volume determination.

(credit: Farraher, et al.

Radiology 2005)
 9

Segmentation Tools in Radiology Applications

• Gross Tumor Volume in CT/MRI
• Metabolic Tumor Volume in PET/SPECT/
– Surgery/Therapy Planning
• Planning Tumor Volume (PTV)
– Tumor characterization
• Texture Extraction requires
segmentation to be done
• Shape analysis
 Credit: Manniesing, et al, 10
Radiology 2008

Segmentation Tools in Radiology Applications

• There is a strong interest in automatic and reproducible
techniques for detection and quantification of vascular
disease
• A first step toward an effective vessel analysis tool is
segmentation of the vasculature.
axial coronal sagittal

MIP: maximum intensity

Projection image of cerebral vessels (in CTA)
 11

Segmentation Tools in Radiology Applications

• MR volumetry of the
hippocampus can help
distinguish patients with
AD (Alzheimer’s
Disease) from elderly
controls with a high
degree of accuracy
(80%–90%).
 12

Segmentation Tools in Radiology Applications

• MR volumetry of the
hippocampus can help
distinguish patients with
AD (Alzheimer’s
Disease) from elderly
controls with a high
degree of accuracy
(80%–90%).

amygdala
hippocampus

Credit: Colliot et al, Radiology 2008.

 13

Image Segmentation
Definition: Partitioning a picture/image into distinctive subsets is
called segmentation.
 14

Image Segmentation
Definition: Partitioning a picture/image into distinctive subsets is
called segmentation.

Segmentation of an image entails the division or

separation of the image
into regions of similar attribute.
 15

Image Segmentation
Definition: Partitioning a picture/image into distinctive subsets is
called segmentation.

Segmentation of an image entails the division or

separation of the image
into regions of similar attribute.

The most basic attributes:

-intensity
-edges
-texture
-other features…
 16

Image Segmentation
Definition: Partitioning a picture/image into distinctive subsets is
called segmentation.
Purpose: To extract object information
and represent this as a
hard/fuzzy geometric
structure.
Recognition: Determining the object’s
whereabouts in the scene.
(humans > computer)
Delineation: Determining the object’s
spatial extent and
composition in the scene.
(computers > humans)
 17

Recognition - Example

Model is induced No Model is induced

(slice credit: J. Kim et al,

Signal Processing 2007)
 18

Approaches to Recognition
• Model-based
• Knowledge-based - Non-interactive
• Atlas-based

• Human-assisted - Interactive
 19

Approaches to Recognition
• Model-based
• Knowledge-based - Non-interactive
• Atlas-based

• Human-assisted - Interactive

- They all originate from human knowledge.

- Their relative efficacy is unknown.
 20

Approaches to Delineations
pI (purely image-based) approaches
• Rely mostly on information available in the given image
only.
• Recognition: manual
 21

Approaches to Delineations
pI (purely image-based) approaches
• Rely mostly on information available in the given image
only.
• Recognition: manual

SM (shape model-based) approaches

• Employ models to codify object family shape info.
• Recognition: model-based/manual
 22

Approaches to Delineations
pI (purely image-based) approaches
• Rely mostly on information available in the given image
only.
• Recognition: manual

SM (shape model-based) approaches

• Employ models to codify object family shape info.
• Recognition: model-based/manual

Hybrid approaches
• Combine among pI and SM approaches.
• Recognition: model-based, automatic.
 23

Classification of Methods
Boundary-based (BpI):
• optimum boundary
• active boundary
• live wire
• level sets
 24

Classification of Methods
Boundary-based (BpI):
• optimum boundary
• active boundary
• live wire
• level sets
Region-based (RpI):
• clustering – kNN, CM, FCM
• graph cut
• fuzzy connectedness
• MRF
• watershed
• optimum partitioning
• (Mumford-Shah)
 25

Classification of Methods
Boundary-based (BpI): SM Approaches
• optimum boundary • manual tracing
• active boundary • live wire
• live wire • active shape/appearance
• level sets • M-reps
• atlas-based
Region-based (RpI):
• clustering – kNN, CM, FCM
• graph cut
• fuzzy connectedness
• MRF
• watershed
• optimum partitioning
• (Mumford-Shah)
 26

Classification of Methods
Boundary-based (BpI): SM Approaches
• optimum boundary • manual tracing
• active boundary • live wire
• live wire • active shape/appearance
• level sets • M-reps
• atlas-based
Region-based (RpI):
Hybrid Approaches
• clustering – kNN, CM, FCM
• graph cut • BpI + BpI
• fuzzy connectedness • RpI + RpI
• MRF • BpI + RpI
• watershed • BpI + SM
• optimum partitioning • RpI + SM
• (Mumford-Shah) • SM + SM
 27

Classification of Methods
pI Approaches

+ Where image info is good,

accuracy is good;
- Bad where it is poor/absent;
- Need recognition help;
+ Can determine degree of
match of model to image
well;
- Lack obj shape &
geographic info;
 28

Classification of Methods
SM Approaches

- Even where image info is

good, accuracy suffers;
+ Where bad, model helps;
+ Can help in recognition;
- Need best match info;

+ Good models embody obj

shape & geographic info;
 29

Purely Image Based Segmentation Methods

 30

Thresholding – Simple Segmentation

• Image binarization
– mapping a scalar image I into a binary image J

(
0 if I(x, y) < T
J(x, y) =
1 otherwise.
 31

Thresholding – Simple Segmentation

• Image binarization
– mapping a scalar image I into a binary image J

(
0 if I(x, y) < T
J(x, y) =
1 otherwise.
 32

Thresholding – Simple Segmentation

Brighter objects

Darker objects
 33

Thresholding – Simple Segmentation

Brighter objects
DIFFICULTIES
1. The valley may be so broad that
it is difficult to locate a
significant minimum
Darker objects 2. Number of minima due to type
of details in the image
3. Noise
4. No visible valley
5. Histogram may be multi-modal
 34

Example: CT Scan
 35

Example: CT Scan
 36

Example: CT Scan
 37

Example: CT Scan
 38

Example: CT Scan
 39

Thresholding Methods
• Huang
• Intermode
• Isodata
• Li
• MaxEntropy
• Mean
• MinError
• Otsu
• Percentile
• RenyiEntropy
• Moments
 40

Thresholding Methods
• Huang
• Intermode
• Isodata
• Li
• MaxEntropy
• Mean
• MinError
• Otsu
• Percentile
• RenyiEntropy
• Moments
 41

Thresholding Methods
• Huang PET Imaging
• Intermode Fixed Thresholding
• Isodata Adaptive Thresholding
• Li Iterative Thresholding
• MaxEntropy
• Mean
• MinError
• Otsu (non-parametric)
• Percentile
• RenyiEntropy
• Moments
 42

Otsu Thresholding
• Definition: The method uses the grey-value histogram of the
given image I as input and aims at providing the best
threshold in the sense that the “overlap” between two
classes, set of object and background pixels, is minimized
(i.e., by finding the best balance).
 43

Otsu Thresholding
• Definition: The method uses the grey-value histogram of the
given image I as input and aims at providing the best
threshold in the sense that the “overlap” between two
classes, set of object and background pixels, is minimized
(i.e., by finding the best balance).

• Otsu’s algorithm selects a threshold that maximizes the

2
between-class variance b . In the case of two classes,
2 2 2 2
b = P1 (µ1 µ) + P2 (µ2 µ) = P1 P2 (µ1 µ2 )
 44

Otsu Thresholding
• Definition: The method uses the grey-value histogram of the
given image I as input and aims at providing the best
threshold in the sense that the “overlap” between two
classes, set of object and background pixels, is minimized
(i.e., by finding the best balance).

• Otsu’s algorithm selects a threshold that maximizes the

2
between-class variance b . In the case of two classes,
2 2 2 2
b = P1 (µ1 µ) + P2 (µ2 µ) = P1 P2 (µ1 µ2 )
• where P1 and P2 denote class probabilities, and μi the means
of object and background classes.
 45

Otsu Thresholding
• Definition: The method uses the grey-value histogram of the
given image I as input and aims at providing the best
threshold in the sense that the “overlap” between two
classes, set of object and background pixels, is minimized
(i.e., by finding the best balance).

u
u
X
P1 = p(i)
u
ı=0
GX
max

P2 = p(i)
ı=u+1
 46

Otsu Thresholding
• Definition: The method uses the grey-value histogram of the
given image I as input and aims at providing the best
threshold in the sense that the “overlap” between two
classes, set of object and background pixels, is minimized
(i.e., by finding the best balance).

u u
X
X
P1 = p(i) µ1 = ip(i)/P1
ı=0
ı=0
GX
max
GX
max
µ2 = ip(i)/P2
P2 = p(i) ı=u+1
ı=u+1 CLASS MEANS
 47

Otsu Thresholding-Algorithm

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 48

Otsu Thresholding-Algorithm

2
b = P1 (µ1 µ)2 + P2 (µ2 µ)2 = P1 P2 (µ1 µ2 ) 2

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 49

Otsu Thresholding-Algorithm

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 50

Otsu Thresholding-Algorithm

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 51

Otsu Thresholding-Algorithm

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 52

Otsu Thresholding-Algorithm

P1 P2

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 53

Otsu Thresholding-Algorithm

P1 P2 optimal

cI (u) 1 cI (u)
c indicates cumulative histogram, and P1 and P2
can be approximated well with cumulative density function.
 54

Parametric Method for Optimal Thresholding

• Assuming again a two-class problem and assuming that the
distribution of gray levels for each class can be modeled by a
normal distribution with mean and variance
 55

Parametric Method for Optimal Thresholding

• Assuming again a two-class problem and assuming that the
distribution of gray levels for each class can be modeled by a
normal distribution with mean and variance

• the overall normalized intensity histogram can be written as

the following mixture probability density function:
 56

Parametric Method for Optimal Thresholding

• Assuming again a two-class problem and assuming that the
distribution of gray levels for each class can be modeled by a
normal distribution with mean and variance

• the overall normalized intensity histogram can be written as

the following mixture probability density function:

where P1 and P2 are class probabilities. The optimal threshold

(T) can be found as solving the quadratic equation à
 57

Parametric Method for Optimal Thresholding

 58

Parametric Method for Optimal Thresholding

In case, variances of both classes are equal, then->

 59

Parametric Method for Optimal Thresholding

In case, variances of both classes are equal, then->

 60

Thresholding methods for PET Image

Segmentation
• Due to the nature of PET images (i.e., low resolution with high
contrast), thresholding-based methods are suitable
– because the local or global intensity histogram usually provides a
sufficient level of information for separating the foreground (object of
interest) from the background. (Foster, Bagci, et al., CBM 2014)
 61

Thresholding methods for PET Image

Segmentation
• Due to the nature of PET images (i.e., low resolution with high
contrast), thresholding-based methods are suitable
– because the local or global intensity histogram usually provides a
sufficient level of information for separating the foreground (object of
interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Adaptive Iterative

Thresholding Thresholding Thresholding
 62
Fixed Thresholding Methods

• Due to the nature of PET images (i.e., low resolution with high
contrast), thresholding-based methods are suitable
– because the local or global intensity histogram usually provides a
sufficient level of information for separating the foreground (object of
interest) from the background. (Foster, Bagci, et al., CBM 2014)
 63

Thresholding methods for PET Image

Segmentation
• Due to the nature of PET images (i.e., low resolution with high
contrast), thresholding-based methods are suitable
– because the local or global intensity histogram usually provides a
sufficient level of information for separating the foreground (object of
interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Adaptive Iterative

Thresholding Thresholding Thresholding

Phantom Image Quality

Based metrics based
Adaptive Thresholding 64
 65

Thresholding methods for PET Image

Segmentation
• Due to the nature of PET images (i.e., low resolution with high
contrast), thresholding-based methods are suitable
– because the local or global intensity histogram usually provides a
sufficient level of information for separating the foreground (object of
interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Adaptive Iterative

Thresholding Thresholding Thresholding

Phantom Image Quality

Based metrics based
 66

Iterative Thresholding Method (ITM)

S/B: Source to background ratio.

The method is based on calibrated

threshold-volume curves at varying
S/B ratio acquired by phantom
measurements using spheres of known
volumes.
 67

Iterative Thresholding Method (ITM)

S/B: Source to background ratio.

The method is based on calibrated

threshold-volume curves at varying
S/B ratio acquired by phantom
measurements using spheres of known
volumes.
 68

Iterative Thresholding Method (ITM)

S/B: Source to background ratio.

The method is based on calibrated

threshold-volume curves at varying
S/B ratio acquired by phantom
measurements using spheres of known
volumes.

The measured S/B ratios of the

lesions are then estimated from
PET images, and their volumes are
iteratively calculated using the
calibrated S/B-threshold-volume curves
 69

Iterative Thresholding Method (ITM)

S/B: Source to background ratio.

The method is based on calibrated

threshold-volume curves at varying
S/B ratio acquired by phantom
measurements using spheres of known
volumes.

The measured S/B ratios of the

lesions are then estimated from
PET images, and their volumes are
iteratively calculated using the
calibrated S/B-threshold-volume curves

The resulting PET volumes are then

compared with the known sphere volume
and CT volumes of tumors that served
as gold standards.
 70

ITM Example Result on PET Images/Lung

 71

Another Example for PET Thresholding

ITM for tumor segmentation/FDG PET

 72

Another Example for PET Thresholding

 73

Further Thresholding Example – CT Bones

 74

Further Thresholding Example – CT Bones

 75

Head-Neck CT – Thresholding for Skull

Modeling

Segmentation of the skull and the mandibula in CT images using thresholding. (a) Original CT
image of the head. (b) Result with a threshold value of 276 Hounsfield units. The segmented bony
structures are represented in color. (c) 3D rendering of the skull shows a congenital growth
deficiency of the mandibula in this 8-year-old patient. This information was used preoperatively to
plan a repositioning of the mandibula.
(Slice Credit: P.Seutens)
 76

Multiple Thresholds – MRI Thresholding

Thresholding can be done interactively and separates the image into different
regions. Valleys in the histogram indicate potentially useful threshold values
Credit: Toeonies, K.
 77

Summary of today’s lecture

• Introduction into the Medical Image Segmentation
• Recognition and Delineation concepts in Segmentation
• Simplest Segmentation method: Thresholding
– Otsu
– Parametric method for optimal thresholding
– PET Image thresholding
• ITM, fixed thresholding, etc.
 78

Slide Credits and References

• Jayaram K. Udupa, MIPG of University of Pennsylvania, PA.
• P. Suetens, Fundamentals of Medical Imaging, Cambridge
Univ. Press.

• Foster, B., et al. CBM, Review paper, 2014.

• Kaus, et al. Radiology 2001.
• Toeonies, K., Medical Image Analysis.
• Farraher, et al., Radiology 2005
• Zaidi, H., Quantitative Analysis in Nuclear Medicine Imaging.
• Bailey et al. Positron Emission Tomography, Springer.
• Dawood, M., et al. Correction Techniques in Emission
Tomography