3D Image Reconstruction and Analysis in Medicine

Contour-Based Segmentation Laboratory

Chunliang Wang
Mehdi Astaraki

I. Deformable Models and Level Set; General Definitions

Deformable models are a famous class of image segmentation methods which are very useful
for medical image segmentation. These methods combine different source of information
including geometric, physic and estimation theory.
The fundamental approach of these methods is as follows:
First, we have to define an energy functional. This energy term should reflect some information
about image data which is either related to image statistics (region-based) or edges of the image
(edge-based). This term is called 𝐸𝑒𝑥𝑡𝑒𝑟𝑛𝑎𝑙 . The energy should include a regularization term to
make the segmented boundaries smooth; this term called 𝐸𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 . Sometimes, based on the
problem, some constraints will also add to the energy functional such as shape constraint
(𝐸𝑐𝑜𝑛𝑠𝑡𝑟𝑎𝑖𝑛𝑡 ).
𝐸𝑒𝑥𝑡𝑒𝑟𝑛𝑎𝑙 + 𝐸𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 + 𝐸𝑐𝑜𝑛𝑠𝑡𝑟𝑎𝑖𝑛𝑡
Then, we set an initial contour (surface or even seeds) on image plane and set this initialization
as an argument of the energy functional. By using the concept of variational calculus, we
minimize the energy functional with respect to the argument and this process lead the contour
to evolve on the image.
In a general view, we can divide these methods into two major categories: parametric and non-
parametric. In parametric methods, we parameterize the contour (surface) e.g. with B-splines.
1
𝐸𝑠𝑛𝑎𝑘𝑒 = ∫ {𝐸𝑖𝑛𝑡 (𝑉(𝑠)) + 𝐸𝑒𝑥𝑡 (𝑉(𝑠)) + 𝐸𝑐𝑜𝑛𝑠𝑡 (𝑉(𝑠))}𝑑𝑠
𝑠=0

However, there exist some difficulties with this approach. They are very sensitive to
initialization; they cannot follow the topological changes such as merging and splitting; and
different parametric methods lead to different solutions. To solve the mentioned problems,
some methods were introduced such as Gradient Vector Flow and Dynamic Directional
Gradient Vector Flow.
In the non-parametric methods, the concept of “Level Set” function is utilized. In this method,
we map the information from image plane to a smooth function (Level Set). When we set the
initial contour on the image, we can consider that we divided the image into two parts: outside
of the contour and inside of the contour. Also, the contour itself is the edge between inside and
outside regions. Therefore, we map the image in a way which the inside part represented by
negative values, the outside by positive values and the zero-level set is related to the contour
boundary.

1
 3D Image Reconstruction and Analysis in Medicine

𝜑(𝐱, 𝑡) < 0 𝐱 ∈ Ω
{ ̅
𝜑(𝐱, 𝑡) > 0 𝐱 ∈ Ω
𝜑(𝐱, 𝑡) = 0 𝐱 ∈ Ω𝜕 = Γ(t)

By this function, we can follow the contour evolution by only finding the zero values of the level set
function.
As we look at the whole process more carefully, we can find that the level set based method is exactly
a PDE equation, because we try to evolve a function through the time on an image (we have to variable,
spatial and temporal). The whole evolution function can be written as:
𝜕𝜑
+ 𝑉. ∇𝜑 = 0
𝜕𝑡
This equation can be read as: level set function (𝜑) will be changed though the time spatially with the
speed (energy) of V.
Regarding this formula, there are some notation:
∇𝜑
• Normal vectors can be calculated as: 𝐍 = |∇𝜑|
∇𝜑
• The evolution should be done only toward normal vector of each point: 𝑣𝑛 = 𝑉. |∇𝜑|
∇𝜑
• Curvature of the contour (surface) can be calculated as: 𝜅 = −∇. (|∇𝜑|)
• The boundaries of the level set function can be obtained by: 𝛿(𝜑)|∇𝜑| where 𝛿 is Dirac delta
function.
• We can use Heaviside function to find inside and outside of the function:
𝐻(𝑥) ≡ 1 𝑖𝑓 𝑥 > 0
{
𝐻(𝑥) ≡ 0 𝑖𝑓 𝑥 < 0
• Integral over the contour (surface) for a quantity like 𝑃(𝑥, 𝑡) is presented by:

∫ 𝑝(𝑥, 𝑡)𝛿(𝜑)|∇𝜑|𝑑𝐱
𝐑𝑛
• “Reinitialization” process helps the contour to hold its zero level after each iteration and prevent
it from being too flat or too sharp.
• Level set function 𝜑(𝐱, 𝑡) defines over the whole image but in order to speed up the evolution
process, the computation will often be done in a narrowband around the zero-level set.

II. Laboratory Instruction

Task 1) Constant Energy
The simplest energy function (V) you can define is a constant speed. This means regardless of the image
data, the level set will evolve toward the normal vector over the image.
∇𝜑
𝑣1 = 𝛼(𝑥)
|∇𝜑|
The effect of curvature term is to increase the evolution speed where the zero-level set is sharp and
decrease it where the zero level set is smooth. Therefore, it will smooth the whole zero level set.
∇𝜑
𝑣2 = µ(𝑥)∇. ( )
|∇𝜑|

2
 3D Image Reconstruction and Analysis in Medicine

By combining these two energy terms, we have:

∇𝜑 ∇𝜑
𝑉 = 𝛼(𝑥) − µ(𝑥)∇. ( )
|∇𝜑| |∇𝜑|
This means we have an evolution equation like:

𝜕𝜑 ∇𝜑 ∇𝜑
+ (𝛼(𝑥) − µ(𝑥)∇. ( )) . ∇𝜑 = 0
𝜕𝑡 |∇𝜑| |∇𝜑|

Try to implement this equation.

Task 2) Region-based energy

In this part, we are going to add a region-based energy term into the previous task. The energy
functional we are supposed to work on, named as “Chan-Vese (CV)”. The formulation is as
follow:

𝐸(𝜑) = ∫ 𝐹(𝐼(𝑥), 𝜑(𝑥))𝑑𝑥 + 𝛼 ∫ 𝛿(𝜑(𝑥))‖∇𝜑(𝑥)‖ 𝑑𝑥

Where

𝐹(𝐼(𝑥), 𝜑(𝑥)) = 𝐻(𝜑(𝑥))(𝐼(𝑥) − 𝑣)2 + (1 − 𝐻(𝜑(𝑥)))(𝐼(𝑥) − 𝑢)2

U and v are two parameters updated at each iteration as:

∫ (1 − 𝐻(𝜑(𝑥))) . 𝐼(𝑥)𝑑𝑥
𝑢=
∫(1 − 𝐻(𝜑(𝑥)))𝑑𝑥

∫ 𝐻(𝜑(𝑥)). 𝐼(𝑥)𝑑𝑥
𝑣=
∫ 𝐻(𝜑(𝑥))𝑑𝑥
Therefore, the evolution equation will be:
𝜕𝜑
(𝑥) = 𝛿(𝜑(𝑥))((𝐼(𝑥) − 𝑣)2 − (𝐼(𝑥) − 𝑢)2 )
𝜕𝑡
I correspond to the main image and x refers to coordinates (pixels).
First, try to find out how this equation works.
Then, implement this on the provided framework.

Task 3) Edge-based energy

Similar to previous task, here we are going to add another energy term to the task one but this
time we will use an edge-based energy.
Energy criterion:

𝐸(Г) = ∫ 𝑔(Г(𝑞))‖Г′ (𝑞)‖ 𝑑𝑞

3
 3D Image Reconstruction and Analysis in Medicine

1
𝑔(𝐼) =
1+ ‖∇(𝐺 ∗ 𝐼)‖2

I(.) correspond to the image intensity, Г is the curve and G is a gaussian filter of variance 1.
Evolution equation:
𝜕𝜑
(𝑥) = 𝑔(𝐼(𝑥))‖∇𝜑(𝑥)‖(𝑐 + 𝑘) + ∇𝑔(𝐼(𝑥))∇𝜑(𝑥)
𝜕𝑡
∇𝜑(𝑥)
Where 𝑘 = 𝑑𝑖𝑣(‖∇𝜑(𝑥)‖)

Again, find out how this energy works and then implement it in the framework.

Task 4) High-level segmentation

For this task on, you should use a software named “mialab” developed by Dr. Chunliang Wang.
Please download it. Open the software by clicking on “MiaLab” icon.
Open the 2D image “testdisk.bmp” and choose “Threshold based method”. Set the lower
threshold to 128 and the upper threshold to maximum. The red mask now covers all pixels
whose intensity are in between these two values. Choose next step, click and drag the mouse
cursor inside the disk to draw a seed bubble. Go next step again and click the “start” button,
the green mask gives the segmentation result. (Do the red and green masks match each other?)
Task 5) Get a feeling of the active contour’s movement
Do the same segmentation on “testdisk15.bmp”. (Observe, do the red and green masks still
match each other?) Change the smoothing factor to 0.1 and rerun the segmentation (you need
go back to step 2 and then step 3 again to reset the seed region). (What happens and why?)
Keep the smoothing factor at 0.5, decrease the lower threshold and rerun the segmentation.
Also try to increase the lower threshold. (At what point, the segmentation starts failing? No
need to be precise) Try to change the size and location of the seed bubble and rerun the test.
(Dose the size or location matter?)
To better understand the movement of the active contour, you could imagine that the red
colored pixels are hydrophilic molecules while the rest pixels are hydrophobic molecules, and
the smoothing factor controls the surface tension of the water bubble.

Task 6) Can level set stop leaking?

Try to use MiaLab to segment the bigger disk in also “TestLeak.bmp”, “TestLeak12.bmp”
and “TestLeak15.bmp”. Can it do it and why?
Task 7) Brain Segmentation
Use the “open mhd file” option to open the “T1_1mm_brain.mhd” file. Try to segment the
white matter of the brain. Try to increase/ decrease the smoothing factor and observe the
changes.

4
 3D Image Reconstruction and Analysis in Medicine

In step 2, enable “show blocking region”, and draw a blocking bubble in the brain stem. Re-
run the segmentation.
Try to find a way to segment the whole brain (so-called skull stripping). (What are the means
to prevent leaking?)
Task 8) Abdominal Artery Segmentation
Load aorta.mhd and try to segment out the abdominal arteries using the threshold-based
method. You can also try to segment the kidneys using the interactive tool. (Compare the result
with the result from Task 4, which is better and why).
Task 9) Multi-structure segmentation
The purpose of this task is to segment several bony structures in one framework.
Open the software by clicking on “MiaLab” icon. Load the data by click on “Open DataSet”/
open Nifti file and then select the file “fracture.nii”. From top menu choose “WorkFlow” and
click on “Bone segmentation and print”. Here, you can set different labels on different
structures; these seeds work as initial surfaces. After that, click on “Start Segmentation”; this
will use watershed segmentation method which leads to over-segmenting the region.
After it gives back the results in different color, click “Next” to use threshold-based level set
method for each of the structure. In this step, you have to find the intensity range and set initial
seeds for each structure separately.
This process, generate a final mask in which each segmented region will be labeled by a
different integer. The final mask and the mesh grid (that can be useful for 3D printing) will be
automatically generated as “fracture_mask.nii” and “fracture_mask.sed”in the same path of the
original file.

Good Luck!