DEPARTMENT OF BIOMEDICAL ENGINEERING

BM3652 – MEDICAL IMAGE PROCESSING NOTES

(2 marks QA and Part B QAs)

VI SEMESTER- BME (R-2021)

UNIT IV REGISTRATION AND VISUALISATION

Registration–Rigid body transformation, principal axes registration, and feature based. Visualisation
- Orthogonal and perspective projection in medicine, Surface based rendering, Volume visualization
in medical image. Explain the significance of registration of various imaging modalities and appraise
the concepts of image visualization in healthcare using Matlab
1. What is image registration? Mention its primary use.
The image registration is the technique used to align multiple scenes into a single integrated
image. It is used to bring the better visualization for the image understanding.

2. Give suitable example for image registration.

The position of a patient in an MR scanner during a head scan is governed by the shape of
the receiver coil and the gradients, whereas patient pose in a CT or PET scanner is governed
by the patient’s principal axes. Combined modalities like PET/CT solve this problem.

3. Write short note on registration paradigms.

Registration algorithms determine a volume transformation from identifying common
features in two coordinate systems. These features can be intrinsic (that is features that are
inherent to the image data) and extrinsic. Extrinsic features are usually markers attached to
the patient; these markers, usually called fiducial markers, are identified in both frames of
reference.

4. What is fusion of images?

Fusion of images – that is, finding a common frame of reference for both images – is
therefore a vital task for all kinds of diagnostic imaging. Besides fusing images for tasks such
as comparison of image series, we can co-register the coordinate system of an external device
such as a robot or a LINAC.

5. What are the purposes of registration algorithms?

Registration algorithms optimize parameters of rigid motion and internal degrees-of-freedom
[dof] (for deformable registration) until a measure that compares images is found to be
optimal.

6. Categorize the registration algorithms?

Registration algorithms can also be categorized by the type of image data or frames of
reference they operate on.
 Intra- and intermodal registration
 Rigid and non-rigid registration
7. What is Intra model registration? Mention its advantages.
In intra modal registration, we are registering images that stem from the same modality; an
example is CT-to-CT registration of volumes acquired at different times. Such a procedure is
extremely helpful when doing time series evaluation, for instance when tracking the effect of
chemo- or radiotherapy on tumor growth.

8. What is Inter model registration? Mention its advantages.

When fusing image data from different modalities, we are dealing with intermodal
registration. A classical example is MR/CT fusion; an example of MR/CT fusion using a
similarity measure named normalized mutual information (NMI) in AnalyzeAVW. In intra
modal registration, it may be sufficient to define a measure that compares the voxel gray
values ρ at identical positions in the base and match volume.

9. What are Rigid and non-rigid registrations?

 Rigid registration refers to algorithms that are confined to finding an affine transformation
in three or six dof. However, tissue may change its shape, and the relative position of
inner organs may vary.
 In non-rigid registration, the affine transformation is only a first step in alignment of
volumes. The fine-tuning is handled by adding additional, internal dof, which handle the
deformation as well.

10. Define Merit function and mention its uses.

The merit function is used to measure the optimize parameters to determine the rigid motion
and degrees-of-freedom of registration algorithms. The merit function depends on the
specific registration task.

11. What is Rigid Body transformation? Give its mathematical representation.

Rigid - body transformation is primarily based on translation and rotation operations. Two
images of equal dimensions are registered by applying a pixel - by - pixel transformation
consistently throughout the image space. In three dimensions, a rigid transformation-based
mapping of a point vector x to x′ is defined by
x′ = Rx + t
Where R is a rotation matrix and t is translation vector.
12. Draw the translation and rotation operations of a 3 - D rigid transformation.
The translation and rotation operations of a 3 - D rigid transformation is shown below

13. Define affine transformation.

Affine transformation is a special case of rigid - body transformation that includes
translation, rotation, and scaling operations. If the two image volumes to be registered are not
at the same scale, a scaling parameter in each dimension has to be added as
x′ = ax
y′ = by
z′ = cz
Where a, b and c are the scaling parameters along x, y and z - directions.
The affine transformation can be expressed as:
x′ = Ax
14. What is Principal axes registration?
Principal axes registration (PAR) can be used for global matching of two binary volumes
such as segmented brain volumes from CT, MR, or PET images.
Let us represent a binary segmented B(x, y, z) as
B(x, y, z) = 1 if (x, y, z) is in the object
B(x, y, z) = 0 if (x, y, z) is not in the object
15. Define Inertia matrix.
The inertia matrix is diagonal when computed with respect to the principal axes. Thus, the
centroid and the principal axes provide a method to completely describe the orientation of an
arbitrary volume. The method can resolve six degrees of freedom of an object, including
three rotations and three translations. Furthermore, it provides a precise way of comparing
the orientations of two binary volumes through rotation, translation, and scaling.
16. What are the operations used for registration with two volumes in PAR method.

17. What are IPAR? Mention its advantage over conventional PAR.
An iterative PAR (IPAR) method is used for registering MR and PET brain images. The
advantage of the IPAR over the conventional principal axes registration method is that the IPAR
method can be used with partial volumes. This procedure assumes that the field of view (FOV) of
a functional image such as PET is less than the full brain volume, while the other volume (MR
image) covers the entire brain.

18. State the uses of image landmarks and feature based registration.
Rigid and non-rigid transformations have been used in image landmarks (points) and features -
based medical image registration. Once the corresponding landmarks or features are identified
from source and target image spaces, a customized transformation can be computed for
registering the source image into the target image space.
19. Classify the algorithms used for point based registration.
There are two simple algorithms used for point - based registration of source and target images.
 Similarity Transformation for Point - Based Registration
 Weighted Features - Based Registration

20. What is image visualization? Mention its common techniques.

Visualization is the art and science of conveying information to a human observer by means of an
image; we already learned about the most important and widespread visualization techniques in
medical imaging – windowing, which makes information hidden in the full intensity depth of the
image visible, and reformatting, which allows for exploring a data cube or volume in arbitrary
directions.

21. What is rendering? Give an example.

The rendering is the method to generate a 2D image from 3D data; every camera and every x-ray
machine does this actually. It is therefore straightforward to mimic the behavior of these devices
mathematically, how we can derive the most important property of an imaging device – the
distance of the viewpoint (or x-ray focus) from the 3D scene to be imaged.

22. Write the projection operator matrix which is used for rendering.
The projection operator is shown below if one assumes that the origin of all rays that project the
3D scene to the 2D imaging plane lies at infinity. When setting f, the distance of the observer to
the projection plane, to infinity,

23. What is orthogonal projection? Mention its advantage.

The Orthogonal projection which is computationally more efficient than perspective rendering, is
the common geometry for rendering. However, want to be able to look at our object from
different viewpoints. In real life, we have two possibilities to look at something from a different
point of view. We can change our position, or we can change the position of the object.

24. Define Raycasting.

Raycasting can simulate this behavior if we sum up all gray values ρ in the path of the ray passing
through a volume. This is, mathematically speaking, a line integral. If we want to simulate a
 camera, the situation is similar, but a different physical process takes place. Rather than
penetrating the object, a ray of electromagnetic radiation is reflected. Therefore we have to
simulate a ray that terminates when hitting a surface, and which is weakened and reflected;

25. Give the types of rendering algorithm.

The rendering algorithms are classified as,
 Volume rendering
 Surface rendering

26. What is Volume rendering?

A ray that passes an object and changes its initial intensity or color during this passage is defined
by a volume rendering algorithm. Such an algorithm does not know about surfaces but draws all
of its information from the gray values in the volume. It is an intensity-based algorithm.

27. What is Surface rendering?

If we simulate a ray that terminates when hitting a surface, we are dealing with a surface
rendering algorithm. The local gradient in the surrounding of the point where the ray hits the
surface determines the shading of the corresponding pixel in the image plane. The drawback lies
in the fact that a surface rendering algorithm requires segmentation

28. State the volume rendering techniques with its transfer function.
The MIP (Maximum Intensity Projections) and the DRR (Digitally Rendering Radiographs) are
simple volume rendering techniques; in volume rendering, a transfer function determines the final
intensity of the rendered pixel. The two transfer functions are,

Where ρ is the intensity of voxels located at positions {x}, which is the set of all voxels within the
path of the ray.

29. State the main problem of Raycasting.

The main problem of raycasting lies in the fact that it is computationally expensive, especially
when switching to a perspective view model. Various refinements to improve the performance of
raycasting do exist, for instance shear-warp rendering. But we can also use our old pal, the
projection operator P to render images. This method is called splat rendering.
30. Differentiate Volume rendering and Surface rendering.

Volume based rendering Surface based rendering

1. It uses the gray values in the image to 1. Do not use the gray values in the image to
assign a gray value or a color to the assign a gray value or a color to the
rendered pixel. rendered pixel, but we encode the properties
2. Segmentation is not required of a surface element to the gray value.
2. Some sort of “soft” segmentation is
required

31. What is Depth Shading (DS)? Give its mathematical relationship.

The Depth shading (DS) is the distance of a voxel to the imaging plane determines its gray value.
The transfer function for depth shading is given as

Again, {x} is the set of voxels lying in the beams path, and x Render Plane is the end point of the ray.
The voxel with the greatest distance to the image plane defines a surface.

32. What is Lambertian Shading? State the Lambert’s law for optical properties of surface.
The simple model is actually Lambertian shading; it is based on Lambert’s law, which states that
the intensity of reflected light from a diffuse surface is proportional to the cosine of the viewing
angle.

33. State the usefulness of surface rendering.

The usefulness of surface rendering is,
 Fast rendering
 Collision detection
 Finite element modelling
 Rapid prototyping
 Part B – Question and Answers
1. Explain Rigid-body transformation in detail with necessary expressions. (13)
Rigid - Body Transformation - Based Global Registration:
The second category of multimodality image registration methods treats the brain volume as
a rigid body for global registration. Levy et al. presented a principal axes transformation - based 3
- D registration method, which requires no operator interaction. This method was used in PET –
PET, MR – MR and MR – PET brain image registration. PET scans, because of their limited field
of view, often do not cover the entire brain, while MR scans are usually obtained for the entire
brain volume. In such cases, the PET volume does not match the MR volume. The principal axes
transformation method therefore does not provide accurate registration of MR and PET volumes.
An iterative principle axes registration method was developed by Dhawan et al. to register MR
brain images with PET scans covering the partial volume of the brain.
2. Explain the following in detail with necessary expressions. (8+5)
(i) Registration Paradigms
In general, registration algorithms determine a volume transformation from identifying
common features in two coordinate systems. These features can be intrinsic (that is features
that are inherent to the image data) and extrinsic. Extrinsic features are usually markers
attached to the patient; these markers, usually called fiducial markers, are identified in both
frames of reference, but cause considerable additional clinical effort and are usually not
available for retrospective studies. Registration algorithms can also be categorized by the
type of image data or frames of reference they operate on.
(1) Intra- and intermodal registration
In intramodal registration, we are registering images that stem from the same modality; an
example is CT-to-CT registration of volumes acquired at different times. Such a procedure
is extremely helpful when doing time series evaluation, for instance when tracking the effect
of chemo- or radiotherapy on tumor growth. Here, a number of CT images are taken at
different points in time. The one quantity of interest is tumor size, which can be identified
easily provided a sufficient uptake of contrast agent. But with varying orientation of the
patient, segmentation of the tumor and, above all, assessment of the direction of tumor
growth or shrinkage becomes extremely cumbersome. By registration of the volumes, this
task is greatly simplified since all slices can be viewed with the same orientation. When
fusing image data from different modalities, we are dealing with intermodal registration. A
classical example is MR/CT fusion; an example of MR/CT fusion using an similarity
measure named normalized mutual information (NMI) in AnalyzeAVW can be found in
Figure 7.3. It is evident that different merit functions are necessary for this type of
application. In intramodal registration, it may be sufficient to define a measure that
compares the voxel gray values ρ at identical positions in the base and match volume. In
intermodal registration, such an approach is bound to fail since the physical principle for the
imaging modalities is different; therefore, there is no reason why a bright voxel in one
volume should correspond to a bright voxel in the other volume. Our already well-known
pig is actually the result of a series of experiments for multi-modal image registration. In
total, the cadaver was scanned using T1, T2 and proton density sequences in MR as well as
multislice CT and CBCT; after registration of those volumes, a single slice was selected and
the images were blended.
(2) Rigid and non-rigid registration
Rigid registration refers to algorithms that are confined to finding an affine transformation in
three or six dof; affine transformations were extensively handled in Chapter 6. However,
tissue may change its shape, and the relative position of inner organs may vary. Even the
simple registration example shown in Figure 7.3 reveals this. In the registered transversal
and coronal slices, we see that bony structures coincide very well. The soft tissue does not –
this is simply due to the fact that the poor little piglet was decapitated before we scanned it.
The head was sealed in a body bag, but in the case of the CT scan, it was lying in a plastic
tub, whereas in the case of the MR, it was tucked into a head coil. The soft tissue was
therefore deformed, and while the NMI-based registration algorithm of AnalyzeAVW does
its job well (since it perfectly aligns the rigid anatomical structures), it cannot compensate
for those deformations. In non-rigid registration, the affine transformation is only a first step
in alignment of volumes. The fine-tuning is handled by adding additional, internal dof,
which handle the deformation as well. A non-rigid registration algorithm is therefore not a
completely different approach to the registration problem; it relies on the same merit
functions and optimization schemes, and it can be intra- or intermodal. What is added is
actually a model that governs the deformation behavior. In other words, we need additional
assumptions on how image elements are allowed to migrate. The formalism to describe this
displacement is a deformation field. Each image element is displaced by a vector that
indicates the direction and amount of displacement.
(ii) Merit functions
In general, merit functions in registration provide a measure on the similarity of images;
therefore, these functions are also referred to as similarity measures or cost functions. In
general, these have the following properties:
• They yield an optimum value if two images are aligned in an optimal manner; it is
therefore evident that a merit function for intramodal registration may have completely
different properties than an merit function for a special intermodal registration
problem.
• The capture (or convergence) range of the merit function should be as wide as
possible. In other words, the merit function has to be capable of distinguishing images
that are only slightly different from those that are completely different, and a well-
defined gradient should exist for a range of motion as large as possible.
• Merit functions for image fusion can be defined as intensity-based, or as gradient-
based.
A number of intensity-based merit functions were defined, and most of them are basically
statistical measures for giving a measure of mutual dependence between random variables.
In these cases, the intensity values ρ are considered the random variables, which are
inspected at the location of image elements which are assumed to be the same. This directly
leads to a very straightforward and simple measure, the sum of squared differences:

where N is the total number of pixels. The inner mechanics of this measure are evident; if
two gray values ρ differ, the squared difference will be non-zero, and the larger the
difference for all image elements, the larger the sum of all squared differences will be. It is
evident that this measure will work best for completely identical images with the same
histogram content, which only differ by a spatial transform. A more versatile merit function
can be defined by using Equation 6.16; in this case, the correlation coefficient of paired gray
values ρBase (x, y, z) and ρMatch(x, y, z) is computed. Pearson’s cross-correlation assumes a
linear relationship between the paired variables; it does not assume these to be identical.
This is a clear progress over MSSD given in Equation 7.1. Still it is confined by definition to
intramodal registration problems. And, to add some more real life problems, one can usually
not assume that the gray values ρ in two intramodal images follow a linear relationship.
 Another possibility to compare intramodal images is to take a look at the difference image
Idiff of two images, and to assess the disorder in the result. Such a measure is pattern
intensity, given here only for the 2D case:

d is the diameter of the local surrounding of each pixel; if this surrounding is rather
homogeneous, a good match is assumed. This measure, which has gained some popularity
in 2D/3D registration, assesses the local difference in image gray values of the difference
image Idiff within a radius r, thus giving a robust estimate on how chaotic the difference
image looks like. It is very robust, but also requires an excellent match of image histogram
content; furthermore, it features a narrow convergence range and therefore can only be used
if one is already pretty close to the final registration result. The internal scaling factor σ is
mainly used to control the gradient provided by MPI.

3. Enumerate Principal Axes Registration in detail. Give a suitable example. (13)

Principal axes registration (PAR) can be used for global matching of two binary volumes
such as segmented brain volumes from CT, MR, or PET images.
Let us represent a binary segmented B(x, y, z) as
B(x, y, z) = 1 if (x, y, z) is in the object
B(x, y, z) = 0 if (x, y, z) is not in the object
 Given two volumes, V1 and V2, for registration, the PAR method provides the following
operations:
1. Translate the centroid of V1 to the origin.
2. Rotate the principal axes of V1 to coincide with the x-, y-, and z- axes.
3. Rotate the x, y, and z axes to coincide with the principal axes of V2.
4. Translate the origin to the centroid of V2.
Finally, the volume V2 is scaled to match the volume V1 using the scaling factor Fs

Probabilistic models can be constructed by counting the occurrence of a particular binary sub
volume that is extracted from the registered volumes corresponding to various images.
Let M (x, y, z) be a function describing the model providing the spatial probability
distribution of the sub volume, then

Where n is the total number of data sets in the model and Si(x, y, z) is the ith sub volume.

4. Explain the feature based registration and its types. (13)

Rigid and non-rigid transformations have been used in image landmarks (points) and features -
based medical image registration. Once the corresponding landmarks or features are identified
from source and target image spaces, a customized transformation can be computed for
registering the source image into the target image space.
(1) Similarity Transformation for Point - Based Registration
Let us assume that x and y respectively, the corresponding points (represented as vectors) in
the source and target image spaces belonging to the source X and target Y images. A non-
rigid transformation T ( x ) for registering the source image into the target image space can be
defined by a combination of rotation, translation, and scaling operations to provide x ′ from
x as (41)
 Where wi are the weighting factors representing the confidence in the specific landmark
(point) or feature correspondence and N is the total number of landmarks.
The following algorithm can be implemented to register the source image into the target
image space:
1. Set s = 1.
2. Find r through the following steps:
(a) Compute the weighted centroid of the body representing the set of landmarks (points)
in each space As

(2) Weighted Features - Based Registration

Different optimization functions can be designed to improve the computation of parameters of
transformation for registration of the source image into the target image space. In the above
example, an error function was used for minimization to achieve the transformation
parameters. In many registration methods, geometrical features or landmarks are used to
register 3 - D images. Defining a transformation T on x as in Equation 12.23 , a disparity
function can be designed as
 Where { X i } for i = 1, 2, 3, … ,
N s represents a set of corresponding data shapes in x and y spaces.
The transformation T must minimize the disparity function to register the source image into
the target image space utilizing the correspondence of geometrical features.
An iterative algorithm can be implemented for registration as;
1. Determine the parameters for a rigid or non-rigid - body transformation T using the
above algorithm.
2. Initialize the transformation optimization loop for k = 1 as

5. Explain the following in detail with necessary expressions. (8+5)

(i) Principal Axes Registration (PAR)
Refer the Answer of Q.No.3

(ii) Iterative Principal Axes Registration (IPAR)

Dhawan et al. developed an iterative PAR (IPAR) method for registering MR and PET brain
images. The advantage of the IPAR over the conventional principal axes registration method is
that the IPAR method can be used with partial volumes. This procedure assumes that the field of
view (FOV) of a functional image such as PET is less than the full brain volume, while the other
volume (MR image) covers the entire brain.
Let V1 and V2 represent two volumes to be registered. The IPAR method can be implemented
using the following steps :
 1. Find the full dynamic range of the PET data and select a threshold, T, which is about 20% of
the maximum gray - level value. Threshold each PET slice to avoid streaking artifacts and
overestimation of the brain regions, such that all pixels with gray - level values equal to or less
than the threshold T are set to zero. Extract binary brain regions using a region - growing method
on the thresholded PET slice data.
2. Threshold and extract binary brain regions from the MR data using a region growing method.
3. Stack two - dimensional (2 - D) binary segmented MR slices and interpolate as necessary to
obtain cubic voxel dimensions using a shape - based interpolation algorithm. This is referred to as
3 - D binary MR data.
4. Stack 2 - D binary segmented PET slices and interpolate as necessary to obtain a cubic voxel
dimension to match the voxel dimension of binary MR data using a shape - based interpolation
algorithm. This is referred to as 3 – D binary PET data.
5. D e fi ne a FOV box, FOV(0), as a parallelepiped from the slices of the interpolated binary
PET data to cover the PET brain volume. Because of the smaller FOV of the PET scanner, voxels
outside the binary PET brain volume will have zero values.
6. Compute the centroid and principal axes of the binary PET brain volume as described above.
7. Add n slices to the FOV(0) box (of binary PET brain volume) on the top and the bottom such
that the augmented FOV( n ) box will have the same number of slices as the binary MR data. The
added slices are spaced equally on the top and bottom. The FOV(n) box will now be shrunk
gradually back to its original size, FOV(0), through an iterative process. Initially, the whole MR
brain volume is used in computing the transformation parameters using the augmented FOV(n)
box. In each iteration, the FOV (n) box is reduced in size and the binary MR data is trimmed
accordingly to provide the current MR brain volume. This volume is used to compute the centroid
and principal axes parameters to obtain a new transformation for matching the PET FOV box to
the current MR brain. The iterative process continues until the FOV box reaches its original size
to provide the final transformation parameters to register MR and PET data. Figure 12.6 shows
three iterations of the IPAR algorithm for registration of two volumes. The IPAR algorithm is as
follows.
For i = n to 0:
A. Compute the centroid and principal axes of the current binary MR brain volume. This
binary MR brain volume is gradually trimmed at each step by the transformed FOV( i ) box.
B. Transform the augmented FOV( i ) box according to the space of the MR slices as follows.
Let c PET and c MR be the centroids, (xg, yg, zg), of the binary PET and MR data, and let E
PET and E MR be the normalized eigenvector matrices of the binary PET and MR data; then
for any point, x PET , its position, x MR , in the MR space can be calculated as

The normalized eigenvector matrix of a binary volume will rotate the 0 standard x-, y-, and z-
axes parallel to the principal axes of the volume. If the centroid of the volume is first translated to
the origin, then after rotation by the eigenvector matrix the x-, y-, and z- axes will be coincident
with the principal axes of the volume.
The PET data is registered with the MR data through the process of the required translations and
rotations. The rotation angles are determined as described above. To express all points, (x PET,
y PET, z PET), of the PET data in the coordinate space of the MR data, the following steps must
be performed:
 1. Translate the centroid of the binary PET data to the origin.
2. Rotate the principal axes of the binary PET data to coincide with the x-, y-, and z- axes
3. Rotate the x-, y-, and z- axes to coincide with the MR principal axes.
4. Translate the origin to the centroid of the binary MR data.
C. Remove all voxels of the binary MR brain that lie outside the transformed FOV( i ) box.
This is the new binary MR brain volume.
The final transformation parameters for registration of MR and PET data are obtained from the
last iteration.
8. Interpolate the gray - level PET data to match the resolution of MR data, to prepare the PET
data for registration with MR data.
9. Transform the gray - level PET data into the space of the MR slices using the last set of MR
and PET centroids and principal axes. Extract the slices from the transformed gray - level PET
data that match the gray - level MR image.
The IPAR algorithm allows registration of two 3 - D image data sets, in which one set does not
cover the entire volume but has the sub volume contained in the other data set. Figures 12.7 a – c
show the MR (middle rows) and PET (bottom rows) brain.

6. Derive the orthogonal projections which are used for image visualization in medicine and
explain. (13)
The basic idea of rendering is to generate a 2D image from 3D data; every camera and every x-
ray machine does this actually. It is therefore straightforward to mimic the behavior of these
devices mathematically, and we have already learned in Section 7.6 how we can derive the most
important property of an imaging device – the distance of the viewpoint (or x-ray focus) from the
3D scene to be imaged. However, one can simplify the projection operator from Equation 7.8 if
one assumes that the origin of all rays that project the 3D scene to the 2D imaging plane lies at
infinity. When setting f, the distance of the observer to the projection plane, to infinity.
 This projection images everything onto the x-y plane; the eye point is located at infinity in the
direction of the z-axis. This is an orthogonal projection: all rays hitting the object are parallel. The
term infinite only refers to the fact the viewer is located at a large distance from the object. It
therefore depends on the size of the object viewed. Orthogonal projection, which is
computationally more efficient than perspective rendering, is the common geometry for
rendering. We do, however, want to be able to look at our object from different viewpoints. In
real life, we have two possibilities to look at something from a different point of view. We can
change our position, or we can change the position of the object. When rendering an object, we
can do the same. We can either apply a volume transform V on every voxel of our object or apply
the projection PV x or we can apply another volume transform V to the projection operator:
V P x.

7. Explain the following in detail with necessary expressions. (7+6)

(i) Orthogonal and Perspective projection in medicine
Refer the answer of Q. No. 6

(ii) Raycasting
The projection operator is very helpful, but not very intuitive. Let’s go back to the idea of
simulating a camera, or an x-ray machine. Such a device records or emits rays of electromagnetic
radiation (for instance light). The most straightforward approach is to simulate an x-ray. X-rays
emerge from the anode of the x-ray tube and pass through matter. In dependence of the density
and radio opacity of the object being imaged, the x-ray is attenuated. The remaining intensity
produces a signal on the detector. Raycasting can simulate this behavior if we sum up all gray
values ρ in the path of the ray passing through a volume. This is, mathematically speaking, a line
integral. If we want to simulate a camera, the situation is similar, but a different physical process
takes place. Rather than penetrating the object, a ray of electromagnetic radiation is reflected.
Therefore we have to simulate a ray that terminates when hitting a surface, and which is
weakened and reflected; the amount of light hitting the image detector of a camera after reflection
is defined by the objects surface properties. These properties are defined by lighting models,
shading and textures. From these basic considerations, we can derive several properties of these
rendering algorithms:
• Volume rendering: A ray that passes an object and changes its initial intensity or color during
this passage is defined by a volume rendering algorithm. Such an algorithm does not know about
surfaces but draws all of its information from the gray values in the volume. It is an intensity-
based algorithm.
• Surface rendering: If we simulate a ray that terminates when hitting a surface, we are dealing
with a surface rendering algorithm. The local gradient in the surrounding of the point where the
ray hits the surface determines the shading of the corresponding pixel in the image plane. The
drawback lies in the fact that a surface rendering algorithm requires segmentation.
 Raycasting is an image-driven technique – each pixel in the imaging plane is being assigned since
it is the endpoint of a ray by definition. This is a huge advantage of raycasting since round-off
artifacts as the ones we encountered in Example 6.6.1 cannot occur. Every pixel in the imaging
plane has its own dedicated ray. If the voxels are large in comparison to the resolution of the
image, discretization artifacts may nevertheless occur. In such a case, one can interpolate between
voxels along the path of the ray by reducing the increment of the ray. Performance does, of
course, suffer from this.

8. What are rendering and explain the following in detail. (6+7)

(i) Volume rendering
Let’s stick to volume rendering first; if we simulate an x-ray tube and simplify the attenuation
model in such a manner that we just project the most intense voxel, we are dealing with
maximum intensity projection (MIP). Figure 8.3 shows such a rendering. While the method
sounds extremely simple, it is astonishingly efficient if we want to show high contrast detail in
a volume. The appearance is somewhat similar to an x-ray with contrast agent. Example 8.5.2
implements a very simple algorithm for MIP rendering; as usual in our sample scripts, the
resolution of the volume to be used is coarse, but it shows the basic principle of raycasting. In
this implementation, a straight line parallel to the z axis is drawn from each pixel on the image
plane to the boundary of the volume. The most intense pixel in the path of this straight line is
finally saved on the image plane. Example 8.5.2 also introduces another important component
of rendering, which is intensity clipping. In order to keep the image tidy from a gray film that
stems, for instance, from non-zero pixels of air surrounding the object, it is recommendable to
introduce a minimum rendering threshold – if a voxel does not show high intensity, it is
omitted. Intensity clipping by introducing a rendering threshold must not be mistaken for the
segmentation method named thresholding, where a binary volume is constructed by omitting
voxels below or above a given threshold. Nevertheless, we have already seen the effects of
introducing a rendering threshold in Figure 5.2, where the problems of segmentation based on
thresholding were illustrated by using a rendering threshold.
If we simply sum up the voxels encountered by the ray, we end up with a render type that is
called summed voxel rendering. It is the simplest type of a DRR which we already know from
Section 7.7.2. Figure 8.4 shows such a summed voxel rendering; it is not exactly a simulation
of an x-ray image since the exponential attenuation of the x-ray is not taken into account. It is
nevertheless a very good approximation, and besides 2D/3D registration, DRRs are widely
used in clinical radiation oncology for the computation of so called simulator images. The
MIP and the DRR are simple volume rendering techniques; in volume rendering,
a transfer function determines the final intensity of the rendered pixel. The two transfer
function we encountered so far are
 (ii) Surface rendering
In Figure 8.7, we make a step from volume rendering to surface rendering. The main
difference of surface rendering compared to volume rendering methods lies in the fact that we
do not use the gray values in the image to assign a gray value or a color to the rendered pixel,
but we encode the properties of a surface element to the gray value. An example already
mentioned is depth shading – the distance of a voxel to the imaging plane determines its gray
value. An example can be found in Figure 8.7.
The transfer function for depth shading is given as

Again, { x}is the set of voxels lying in the beams path, and x Render Plane is the end
point of the ray. The voxel with the greatest distance to the image plane defines a surface; it is
also the first voxel which lies above a rendering threshold encountered by the ray. Its distance
gives the gray value of the pixel in the rendering plane the ray aims at. What is remarkable
about Equation 8.4 when comparing it to the fact that ρ, the intensity of the voxel, does not
play a role here since depth shading is, in fact, a surface rendering technique.

9. Explain the surface based rendering in detail with necessary expressions and diagrams. (13)
Refer the answer od Q. No. 8 (ii) and