TRILATERAL FILTERING FOR BIOMEDICAL IMAGES

Wilbur C.K. Wong† , Albert C.S. Chung† , Simon C.H. Yu‡

†
Department of Computer Science, The Hong Kong University of Science and Technology, HK.
‡
Department of Diagnostic Radiology and Organ Imaging, The Prince of Wales Hospital, HK.

ABSTRACT 2. FUNDAMENTAL AND METHOD

Filtering is a core operation in low level computer vision. It In this section, we present the fundamental and the method for-
is a preliminary process in many biomedical image processing ap- mulation of the new filtering technique. Section 2.1 introduces
plications. Bilateral filtering has been applied to smooth biomed- bilateral filtering, as the preliminary of our trilateral filtering. The
ical images while preserving the edges. However, to avoid over- formulation of the trilateral filter is given in Section 2.2. In Section
smoothing structures of sizes comparable to the image resolutions, 2.3, the methodology to extract the local structural information is
a narrow spatial window has to be used. This leads to the ne- discussed.
cessity of performing more iterations in the filtering process. In
this paper, we propose a novel filtering technique namely trilat- 2.1. Bilateral Filtering
eral filter, which can achieve edge-preserving smoothing with a
narrow spatial window in only a few iterations. The experimental Bilateral filtering, representing a large class of non-linear filters
results have shown that our novel method provides greater noise proposed by Tomasi et al. [6], is a non-iterative and local approach
reduction than bilateral filtering and smooths biomedical images to edge-preserving smoothing. A filtered image is obtained by
without over-smoothing ridges and shifting the edge locations, as replacing the intensity value of each pixel with an average value
compared to other noise reduction methods. weighted by the geometric and photometric similarities between
neighboring pixels within a spatial window. The bilateral filtering
can be summarized in the following discrete formulation:
1. INTRODUCTION 1


 
 



I
∗ (

x) = I ξ · c ξ,

x · s I ξ , I (

x) , (1)
k (

x)


ξ∈N
Filtering is a preliminary process in many biomedical image pro-

x

cessing applications. It is a fundamental operation in low level

computer vision, which is aimed at restoring a noise-corrupted im- where
x and ξ
are spatial coordinates, I
is the noisy image, I
∗ is
age to its noiseless counterpart. Any post-processing tasks, such the filtered image, N
x defines the spatial window around the pixel
as visualization and segmentation, may benefit from the reduction x, k (

at
x) is the normalization constant which assures the weights
of noise. Diffusion equations with scalar-valued and tensor-valued c (·) · s (·) are added up to 1 within N
x , the functions c and s mea-
diffusivities have been applied to magnetic resonance (MR) imag- sure the geometric and photometric similarities between neighbor-
ing and 3D rotational angiography (RA) [1], [2], [3] for edge- hoods respectively. For the definitions of the two functions, see
preserving smoothing. MR angiography (MRA) denoising with [6].
adaptive filtering in the Fourier domain has been proposed in [4]. The concept of bilateral filtering has been investigated in a
Non-linear noise reduction techniques in computed tomography recent publication. Barash [7] revealed that bilateral filtering is
(CT) imaging have been investigated in [5]. a non-iterative method only if a wide spatial window is used (15
In this paper, we present a novel filtering method for bio- pixels in each dimension). However, a wide spatial window may
medical images, namely trilateral filtering. The method works over-smooth sharp ridges and gutters in the image as indicated in
along the same lines as bilateral filtering [6], integrating geomet- [8]. Therefore, it is necessary to strike a balance between the size
ric, photometric and local structural similarities, to achieve edge- of the spatial window and the number of iterations needed to be
preserving smoothing. It is simple to implement and is applicable performed in bilateral filtering.
to multi-dimensional signals. It uses a narrow spatial window (3
pixels in each dimension) and takes only a few iterations (3 itera-
2.2. Trilateral Filtering for Biomedical Images
tions in all the experiments conducted in this work) in the smooth-
ing process. Sharp ridges and gutters are commonly found in biomedical im-
This paper is organized as follows: Section 2 briefly intro- ages, such as nested vessels in digital subtraction angiography
duces the bilateral filter, as the fundamental of our trilateral filter- (DSA) and 3D angiography, and folded gray and white matters
ing. The formulation of the trilateral filter follows. In Section 3, in brain MR imaging. Therefore, a narrow spatial window, say, 3
sensitivity analyses of the new method as well as experimental re- pixels in each dimension, should be used in order to avoid over-
sults on 3D numerical phantoms, 2D and 3D clinical datasets and smoothing structures of sizes comparable to the image resolutions.
a 3D MR imaging phantom are presented. Conclusions are drawn This leads to the necessity of performing more iterations in the
in Section 4. filtering process.

0-7803-8388-5/04/$20.00 ©2004 IEEE 820

 1
In this paper, we propose a novel filtering method for biomed-

Mapping Function m
ical images, which works along the same lines as the bilateral fil-
ter, it takes not only the geometric and photometric similarities
0.5
into account, but also, the local structural similarity to smooth the
images with a narrow spatial window while preserving the edges.
Local structural information is used to determine inhomogeneity
0
in the images. On one hand, low-pass filtering is performed in 0 0.5 A* 1
the homogeneous regions. On the other hand, smoothing along (a) (b) (c)
edges is achieved by considering the geometric, photometric and
local structural orientation similarities between neighboring pixels Fig. 1. Local signal amplitude regularization. (a) Mapping func-
in the inhomogeneous regions. We found that this new approach tion m as a function of A
∗x ∈ [0, 1], q = 4, with five different val-
provides greater noise reduction than bilateral filtering with a 3- ues of p: 0.1 (leftmost curve), 0.3, 0.5, 0.7 and 0.9; (b) a portion
pixel-width spatial window (cf. Section 3). of a slice image from a numerical phantom; and (c) regularized
Because of the use of the three similarities in the filtering pro- local signal amplitudes a, p = 0.3 and q = 4 in the regularization
cess, we name this novel method trilateral filtering. The trilateral process. It is evident that a → 0 (dark) in the background region,
filtering is expressed as follows: while a → 1 (bright) if the pixel at
x is located near the edges
of the bright structure. Therefore, it can be used to determine the
1

(t) 
  
I
(t+1) (

x) = I ξ · w ξ,

x, t , (2) inhomogeneity in an image
k (

x)


ξ∈N

x

where in which it is demonstrated that the use of the orientation tensor

gives more reliable local structural information than the use of the
   




Hessian matrix. The vectors ê1 and ê2 are the eigenvectors cor-
w ξ, x, t = (1 − a (

x)) · c ξ, x +
responding to the first and second smallest eigenvalues in the de-
    

 D−1  composition.

The Frobenius norm of the orientation tensor (i.e.,
a (



x) · c ξ, x · s I
(t) ξ
, I
(t) (

x) ·


di ξ, x , (3) D 2
λ
i=1 i , where λi , i = 1, 2, . . . , D, are the eigenvalues of
i=1
the tensor in a D-dimensional space) defines the local signal am-
t is a time variable, a (
x) ∈ [0, 1] is the regularized local signal plitude A
x ∈ [0, +∞). Regularization is achieved by a mapping
amplitude (cf. Section 2.3) of the pixel at
x, a → 0 in the homo- function m, which maps the normalized local signal amplitude
geneous regions, the functions c and s are defined as in the bilateral A
∗x ∈ [0, 1] to [0, 1], as follows:
filter, D is the dimensionality of the image I
and the function di
measures the similarity of the rank i local structural orientation x) = m (A
∗x , p, q) =
a (

between the pixels at ξ
and
x. Rank 1 orientation refers to the (A
∗x · (1 − p))q
, (6)
principal direction of a linear structure, whereas rank 2 orientation (A
∗x · (1 − p))q + ((1 − A
∗x ) · p)q
refers to one of the principal directions of a planar structure (cf.
Section 2.3). The function di is defined as follows: where p ∈ [0, 1] and q are positive constants. Figure 1(a) shows
   the mapping function m as a function of A
∗x , q = 4, with five
  δ 2 ξ
−

x, êi different values of p. Figure 1(b) shows a portion of a slice image
di ξ, x = exp −


, (4) from a numerical phantom (see Section 3 for details). Figure 1(c)
2σ 2 shows the regularized local signal amplitudes a, p = 0.3 and q =
4 in the regularization process. It is evident that a → 0 (dark)
where in the background region, while a → 1 (bright) if the pixel at
x
  is located near the edges of the bright structure. Therefore, the

u ·
v 
u,
v ) = 1 − 
δ (
, (5) regularized local signal amplitudes a can be used to determine the
u 
v  

inhomogeneity in an image.
êi is the direction of rank i orientation and the parameter σ is cho-
sen based on the desired amount of orientation discrepancy filter- 3. RESULTS AND DISCUSSION
ing between neighborhoods. δ → 0 if the angle between
u and
v
is 0◦ / 180◦ , therefore, di → 1 if the pixel at ξ
locates along the A 64×64×64 voxels numerical phantom has been built to evaluate
direction êi with respect to the pixel at
x. the performance of the trilateral filter. The phantom is shown in
In other words, the trilateral filter produces a smoothed image Figure 2(a), which is a pipe (5 voxels in diameter) in a 3D space
by weighting the intensity value of each pixel within the narrow with its centerline aligned with a cubic B-spline curve. In order
spatial window. Weights are either defined by the geometric sim- to study the capability to smooth an image with a narrow spatial
ilarity, c (·), in the homogeneous regions or defined trilaterally by window (3×3×3 voxels), we have applied the trilateral filter (TF)
the geometric, photometric and local structural orientation similar- and the bilateral filter (BF) to the numerical phantom at signal level
equals 5. The signal level is defined as follows:
ities, c (·) · s (·) · i di (·), in the inhomogeneous regions.
Q
2.3. Extraction of Local Structural Information Signal Level = , (7)
σN
The local structural information is obtained from the eigen decom- where Q = 255 is the intensity value of the noiseless pipe and σN
position of orientation tensors as described in a previous work [9], is the standard deviation of an additive white Gaussian noise. The

821
 Increase in SNR Increase in CNR
20 40

15
30
10

dB
dB
20
5 AF AF
EED EED
TF 10
0 TF
20 NEX 20 NEX
(a) Surface model (b) Original (c) 20 NEX −5 0
1 5 Signal Level 9 1 5 Signal Level 9

(a) (b)
Increase in MSE Deviation in Gradient Direction
−10 60
EED AF
−15 TF 50
EED
20 NEX TF
−20 40
20 NEX

Degree
dB
−25 30

−30 20
(d) AF (e) EED (f) TF
−35 10
Fig. 2. Numerical phantom. (a) 3D surface model; (b) a portion −40 0
1 5 9 1 5 9
of a slice image from the phantom; smoothed images with (c) 20 Signal Level Signal Level

NEX, (d) AF (e) EED and (f) TF (c) (d)

Fig. 3. Results of noise sensitivity analyses. Increases in (a) SNR,

signal level 5 implies σN = 255/5 = 51. We found that there is (b) CNR and (c) MSE; and (d) deviation in gradient direction at
93.96% decrease in the mean square error (MSE) for TF, in con- different signal levels
trast to 59.20% decrease in MSE for BF, using a single iteration.
These results have shown that TF provides better image denoising
than BF, using a narrow (3-pixel-width) spatial window. In addition to the numerical phantom, we have applied TF to
Furthermore, we have conducted four noise sensitivity analy- 2D and 3D biomedical datasets and a 3D MR imaging phantom.
ses, in which the increases in (a) signal-to-noise ratio (SNR), (b) Figure 4 shows filtered images of a 2D DSA obtained with BF and
contrast-to-noise ratio (CNR), (c) MSE and (d) deviation in gradi- TF. Figure 5 shows denoised slice images of a 3D RA produced
ent direction after the application of different denoising methods by AF, EED and TF. Figure 6 shows smoothed slice images of a
are studied. SNR and CNR are defined in the following: 3D T1 MR imaging phantom from BrainWeb1 obtained with AF,
EED, BF and TF. It is evident that: TF gives a greater noise re-
Q¯P duction than BF in the vascular regions, see Figures 4(d)-(f), and
SNR = , (8)
σB smooths the DSA image without over-smoothing ridges, as found
in the BF denoised image, pointed by the arrows in Figures 4(b)
 2 and 4(e); TF produces a smoothed 3D RA image without shifting
Q¯P − Q¯B the edge locations as compared to EED, indicated by the arrow
CNR = , (9)
σB in Figure 5(e); and TF gives smoother solution in homogeneous
regions, while produces sharper edges than the other methods as
where Q¯P and Q¯B are the mean intensity values of the pipe and illustrated in the T1 MR imaging phantom (see Figure 6).
the background respectively, and σB is the sample standard de-
viation of the noise in the background. Deviation in gradient di-
rection is calculated according to Equation 5, taking the truth gra- 4. CONCLUSION
dient vector and the computed gradient vector from the denoised
image as arguments. We have compared TF with three other de- We have presented a novel denoising method for biomedical im-
noising methods, viz. 20 NEX (averaging), adaptive filtering (AF) ages, namely the trilateral filter. The method integrates the ge-
[4] and edge-enhancing anisotropic diffusion (EED) [3], where AF ometric, photometric and local structural similarities to filter the
and EED are edge-preserving noise reduction techniques. images. It replaces the intensity value at each pixel with an aver-
Figure 2(b) shows a portion of a slice image from the phantom age value weighted by the three similarities between neighboring
at signal level equals 5. Figures 2(c)-(f) show the filtered images pixels within a narrow spatial window.
obtained with 20 NEX, AF, EED and TF respectively. Figure 3 In order to evaluate the performance of the trilateral filter,
shows the results of the noise sensitivity analyses. Increases are we have conducted several experiments on 3D numerical phan-
measured in dB according to this formula: toms, 2D and 3D biomedical datasets and a 3D MR imaging phan-
tom. The experimental results have shown that our novel method
New Value produces greater noise reduction than the bilateral filtering and
Increase in dB = 20 · log10 . (10)
Old Value smooths the images without over-smoothing ridges and shifting
the edge locations, as compared to other edge-preserving noise re-
It is noted that TF gives the largest amount of reduction in MSE
duction methods.
and the greatest increase in both SNR and CNR amongst the other
methods. Further, TF provides better gradient direction restoration 1 A simulated brain database from Montréal Neurological Institute,
than 20 NEX if the signal level ≥ 5. McGill University (www.bic.mni.mcgill.ca/brainweb/).

822
 (a) Original (b) BF (c) TF

(a) Original (b) AF (c) EED

Original
AF
EED
TF

(d) Original (e) BF (f) TF

Fig. 4. DSA. (a) Original image; denoised images with (b) BF and
(c) TF; and (d)-(f) closeup of the images (a)-(c)

In this paper, we have demonstrated an application of the trilat- 1 20 Position 40

eral filter. We expected that this new method will not be restricted
(d) TF (e) Profile
to biomedical images. It could be applicable to other image types,
e.g., natural scenes, artwork or spatial-temporal images. Fig. 5. 3D RA. (a) A slice image from the dataset; filtered images
with (b) AF, (c) EED and (d) TF; and (e) intensity profile of the
5. REFERENCES images (a)-(d) at the scanline defined in (a)

[1] G. Gerig, O. Kübler, R. Kikinis, and F. A. Jolesz, “Nonlinear

anisotropic filtering of MRI data,” TMI, vol. 11, no. 2, pp.
221–232, 1992.
[2] K. Krissian, “Flux-based anisotropic diffusion applied to en-
hancement of 3-D angiogram,” TMI, vol. 21, no. 11, pp. 1440–
1442, 2002.
[3] E. Meijering, W. Niessen, J. Weickert, and M. Viergever,
“Diffusion-enhanced visualization and quantification of vas-
cular anomalies in three-dimensional rotational angiography:
Results of an in-vitro evaluation,” MedIA, vol. 6, no. 3, pp.
215–233, 2002.
[4] C. F. Westin, L. Wigström, T. Loock, L. Sjöqvist, R. Kiki-
nis, and H. Knutsson, “Three-dimensional adaptive filtering
in magnetic resonance angiography,” JMRI, vol. 14, pp. 63–
71, 2001. (a) Original (b) Closeup (c) AF
[5] G. Fernández, H. Bischof, and R. Beichel, “Nonlinear filters
on 3D CT imaging - bilateral filter and mean shift filter,” in
Computer Vision Winter Workshop, 2003.
[6] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and
color images,” in ICCV, 1998, pp. 839–846.
[7] D. Barash, “A fundamental relationship between bilateral fil-
tering, adaptive smoothing, and the nonlinear diffusion equa-
(d) EED (e) BF (f) TF
tion,” PAMI, vol. 24, no. 6, pp. 844–847, 2002.
[8] P. Choudhury and J. Tumblin, “The trilateral filter for high Fig. 6. T1 MR imaging. (a) An original slice image; (b) closeup
contrast images and meshes,” in Eurographics Symposium on of the slice image; smoothed images with (c) AF, (d) EED, (e) BF
Rendering, 2003. and (f) TF
[9] W. C. K. Wong, A. C. S. Chung, and S. C. H. Yu, “Local
orientation smoothness prior for vascular segmentation of an-
giography,” 2003, Submitted to ECCV 2004.

823