Extraction of Spots in Biological Images Using Multiscale Products
Extraction of Spots in Biological Images Using Multiscale Products
www.elsevier.com/locate/patcog
Extraction of spots in biological images using multiscale
products
Jean-Christophe Olivo-Marin
Laboratoire dAnalyse dImages Quantitative, Institut Pasteur, 25 rue du Dr Roux, 75724 Paris, Cedex 15, France
Received 18 July 2000; accepted 6 June 2001
Abstract
We present a new method to detect and count bright spots in uorescence images coming from biological immu-
nomicroscopy experiments. It is based on the multiscale product of subband images resulting from the a trous wavelet
transform decomposition of the original image, after thresholding of non-signicant coecients. The multiscale corre-
lation of the ltered wavelet coecients, which allows to enhance multiscale peaks due to spots while reducing noise,
combines information coming from dierent levels of resolution and gives a clear and distinctive chacterization of
the spots. Results are presented for the analysis of typical immunouorescence images. ? 2002 Pattern Recognition
Society. Published by Elsevier Science Ltd. All rights reserved.
Keywords: Spot detection; Multiscale analysis; Undecimated wavelet transform; Microscopy; Biology
1. Introduction
Immunomicroscopy images are routinely used in
biology and molecular medicine to detect the presence
of particular biological objects (protein, bacteria, or-
ganelle) or to examine the distribution of antigens in
cellular systems [1,2]. The uorescence-labelled bio-
logical objects appear generally as small specic spots
superimposed on an unspecic background. In the bio-
logical sciences context of this work, we dene a spot
as an object which is relatively small and compact and
has no clear border, and the intensity of which is both
diuse and higher than that of its immediate neighbour-
hood. The number, position and distribution of the spots
are of major importance for biologists to understand the
biological information encoded by the image document.
To get quantitative data that is reliable and statistically
signicant, it is necessary to perform the analysis on a
large number of images so as, moreover, to minimise
the biological variability of the samples.
i=1
W
i
(x; y):
3. Spot detection
The presented a trous wavelet transform gives a
multiresolution representation of images consisting of
approximation images which display the image with
increasingly coarser resolution as the scale itself in-
creases, and of detail planes which show the objects
whose size is adapted to the resolution of the lter at
each scale. There is an inherent adaptiveness of the
analysis to the object size since, with the support of the
convolution lter increasing with the scale of analysis
(see Table 1), the lter smoothes out the response of
too narrow objects at a given scale. At the rst level
of analysis, the support of the lter is such that the de-
tail image has signicant coecients at those locations
where pixel-sized signicant features are present (see
Fig. 1b). When going down in resolution, the lter sup-
port increases in size and signicant coecients in the
detail images correspond more and more to signicant
features of increasing spatial dimension (see Figs. 1(c)
(d)). However, it is very dicult to pick up the interest-
ing features from the analysis of one detail image only.
This is because relevant coecients are embedded into
non-specic background detail coecients.
To overcome the limitation of data coming from a sin-
gle image and to distinguish important wavelet coe-
cients from non-relevant ones, we take advantage of the
multiresolution representation provided by the a trous
wavelet transform. As mentioned in the introduction,
spots are features in the image that are small compared to
the global image, but indeed relatively large when anal-
ysed locally. We assume that spots are features of inter-
est represented by a small number of coecients which
are large and correlated across levels. Following results
rst demonstrated in the case of additive Gaussian white
noise [11], it has been shown that in the case of images
contaminated by additive correlated Gaussian noise, lo-
cal maxima in wavelet planes tend to propagate across
scales when they are due to signicant discontinuities in
the image, while they do not if caused by noise [21,22].
We therefore design a multiscale spatial ltering scheme
Table 1
Length of wavelet lter support
Scale 1 2 3 4 5
Support 5 9 17 33 65
that results in wavelet coecients that have high values
in the presence of a spot and characterise it unambigu-
ously, whereas they have non-signicant values for the
background or for large structures. To that goal, we com-
pute a correlation image P
J
(x; y) which is dened at each
location (x; y) by the direct spatial multiscale product of
the wavelet coecient images at adjacent scales in the a
trous representation:
P
J
(x; y) =
J
i=1
W
i
(x; y);
where J is the deepest level at which the correlation is
computed.
We subsequently use the fact that the product of sig-
nicant coecients across scales at the location (x; y)
results in a signicant value of P
J
(x; y) only if the local
maxima propagate down to the considered scale. Obvi-
ously, if the local maxima die at some intermediate scale,
this one small coecient in the product will be sucient
to decrease the value of P
J
(x; y) signicantly. The key
point here is that the wavelet coecients at large scales
are signicant only in the vicinity of an important fea-
ture while they are close to zero elsewhere. On the other
hand, for a given feature, the support of its interval of
relevance decreases at small scales. The spatial ltering
method can therefore be interpreted as a process by which
wavelet coecient images at large scales are used to give
a coarse estimation of possible spots positions. This es-
timation is then rened by supplementing data coming
from ner scales only at those spatially ltered locations.
To increase further the eciency of the method,
we have found that before computing the multiscale
correlation image, it is desirable to select the most
signicant wavelet coecients and to reduce the inu-
ence of non-signicant noisy coecients by applying
a threshold-based denoising to the wavelet coecients.
Given an input image of the form
Y =f + n;
where Y is the observation, f the noise-free data and n
an additive Gaussian noise, we want to compute, from
the wavelet transformation of Y; W
Y
=W
f
+ W
n
, an
estimate
W
Y
where coecients due to noise are replaced
by zero. Assuming that noise is stationary and that the
correlation between two noise realisations depends on
their relative distance only, we have the following result
[21,22] that, for a given resolution level i, the variance of
the wavelet coecients of a correlated noise W
n
i
depends
only on that resolution level i:
E(W
n
i
)
2
=
2
i
:
From this, we can dene a thresholding strategy that
makes use of the k hard thresholding technique to de-
ne a scale-dependent threshold t
i
[12,13]. The wavelet
1992 J.-C. Olivo-Marin / Pattern Recognition 35 (2002) 19891996
Fig. 1. Spot detection in a synthetic noisy image: (a) Synthetic noisy image with additive Gaussian white noise, SNR is 3:3 dB;
(b)(d) detail wavelet images at scales 1, 2 and 3; (e) detected spots.
coecients W
Y
i
are therefore transformed according to
the following rule:
t
hard
(W
Y
i
; t
i
) =
_
W
Y
i
W
Y
i
t
i
0 W
Y
i
t
i
with t
i
=k
i
, where
i
is the standard deviation of the
noisy wavelet coecients at scale i and a usual choice
is k =3 [12]. A robust estimation of
i
is obtained from
the MAD estimate [15], and is given by
i
= =0:67;
where is the median absolute deviation of the wavelet
coecients at scale i.
The coecient correlation image is used subsequently
to obtain a direct characterisation of the spots. The algo-
rithm compares the values in the correlation image to a
predetermined detection level l
d
in order to discriminate
between spots and background. A spot is accepted only
J.-C. Olivo-Marin / Pattern Recognition 35 (2002) 19891996 1993
Fig. 2. Spot detection in immunomicroscopy images. Left column: original images. All the biological objets were labelled with
Rhodamine or FITC and images acquired with a cooled CCD camera. Right column: processed images showing the spots detected
by the proposed method: (a) transferrin vesicles in neuron (eld of view 50 m); (b) detected vesicles; (c) Shigella bacteria in HeLa
cells (eld of view 50 m); (d) detected bacteria; (e) endosomes in BHK cells (eld of view 25 m); (f ) detected endosomes.
at positions where the correlation coecient is above l
d
,
according to
P
J
(x; y) =
_
255 |P
J
(x; y)| l
d
;
0 otherwise:
A further characterization of spot shape can be accom-
plished with the help of geometrical criteria like elliptic-
ity to discriminate e.g. round spots amongst the features
resulting from the multiscale processing.
4. Experimental results
In order to test the performance of the method, we
have applied it to a synthetic image and to a number of
real-world dierent types of immunomicroscopy images
showing various types of bright spots.
4.1. Synthetic image
Fig. 1(a) shows a synthetic image composed of nine
rows of nine dierent-sized round spots with hard and
1994 J.-C. Olivo-Marin / Pattern Recognition 35 (2002) 19891996
smooth edges and diameters ranging from 9 to 1 pixels,
with added Gaussian noise up to a signal-to-noise ratio
of 3:3 dB. The goal is to detect in the noisy image all the
spots with size above three pixels, that is to say spots
in rows 1, 2 and 57. Figs. 1(b)(d) show the detail
wavelet images corresponding to levels 2, 4 and 8, respec-
tively (contrast has been adjusted for display purposes).
Fig. 1(e) shows the result of the detection algorithm ap-
plied on a three-level wavelet representation, with k =3
and l
d
=1:0, where the detected spots are represented
as binary areas. When using single detail images, it can
be noticed that the spots are still embedded into noise
and that not all the spots are simultaneously present
in one image in a usable way. Conversely, when using
the proposed algorithm and the combination of the
detail images, the result of the detection is very accurate,
as all the spots but one in rows 1, 2 and 57 are correctly
detected.
4.2. Immunomicroscopy images
Fig. 2(a) shows the image of low light level uo-
rescence microscopy of a mouse hippocampal neuron
after endocytosis of transferrin coupled to Rhodamine
[1]. Fig. 2(b) shows a uorescence microscopy image
of a eld of HeLa cells infected by FITC-labelled Shigella
bacteria [2]. The image in Fig. 2(c) is part of a video-
microscopy sequence and presents a low light level uo-
rescence video-microscopy image of Rhodamine-labelled
endosomes in BHK cells. These images are charac-
terised by an increasing degree of diculty to see and
individualise the spots and also by an increasing amount
of acquisition and background noise.
The vesicles, bacteria and endosomes detected by the
proposed method are shown as binary areas in Fig. 2(b),
(d) and (f ), respectively. In all the examples, the ltering
is applied on a three-level wavelet decomposition of the
original images, with k =3 and l
d
=1:0. In the images
presented, isolated single pixels have been removed. To
demonstrate the performance of the method, the results
have been checked against those provided by a human
observer and are summarised in Table 2. They are given
in terms of false positive, false negative and debatable
spots, which represent spots which should not have been
detected, spots which were missed and spots which could
be accepted or not depending on the biological context.
The detectivity provided by our method is very high
and well in accordance with the human classication, as
can be seen from Table 2 and the result images. The
accuracy of the method is also well reected by the small
number of FN and FP detections. In images 2(a) and (c),
the bright spots on focus are correctly detected and only a
very small number of dimmer ones are missing. On image
2(e), the number of FP and FN detections is slightly
higher but this can be explained by the relatively high
number of aggregated rather than isolated spots. Also, at
Table 2
Results of spot detection
a
Image N
a
N
h
FP FN N
d
Vesicles 241 235 6 2 9
Bacteria 248 241 8 3 14
Endosomes 316 290 18 8 26
a
N
a
denotes the number of spots detected by the algorithm
and N
h
the number of spots detected by a human observer. FP
and FN stand for the false positive and false negative whereas
N
d
denotes the number of debatable spots.
some positions in all three images, the algorithmoutputs a
detection even though spots are not well dened because
they are out of the focal plane. Whether these spots should
be validated depends mostly on the biological context.
When imaging xed cells using a narrow depth-of-view
objective, it is most desirable to discard blurry spots in
order to avoid unspecic detections. Conversely, if one
is to follow spots moving in a 3D space (like in image
2(e)), it is important, for establishing the continuity of
the trajectories, to validate even those spots which are
blurry because they are temporarily out of the focal plane
of the imaging system.
5. Conclusion
We have presented a new method to detect bright spots
in biological immunomicroscopy images using an un-
decimated wavelet transform and the ltering of wavelet
coecients. The key point of the method is its ability to
combine multiscale information elements by coecient
correlation which gives a very eective detection of the
spots. We have demonstrated that it has good detectivity
and specicity properties. This method represents an im-
provement over previous ones in that it can be applied to
dierent kinds of images without the need of specifying
application-related parameters. We have demonstrated
its eectiveness with example images coming from bio-
logical microscopy applications.
6. Acknowledgements
The author wishes to thank A. H emar (CNRS 5541,
Universit e Bordeaux 2, Bordeaux), G. Tran van Nhieu
(INSERM U 389, Institut Pasteur, Paris) and C. Murphy
(Biological Chemistry Laboratory, University of Ionnina,
Greece) for providing him with the immunomicroscopy
example images. This work was initiated while the au-
thor was with the Cell Biophysics Programme, European
Molecular Biology Laboratory, Heidelberg.
J.-C. Olivo-Marin / Pattern Recognition 35 (2002) 19891996 1995
Appendix A
For reference, we give a short description of the 1D
a trous wavelet transform, following the presentation
done in Ref. [14]. Extension to 2D is straightforward and
facilitated in terms of computational load by assuming
separability in x and y.
We consider sampled data {A
0
(k)} dened as the
scalar product at pixel k of a function f(x) with a scal-
ing function (x) corresponding to a low-pass lter. The
scaling function is chosen to satisfy the dilation equation
1
2
_
x
2
_
=
l
h(l)(x l);
where h is a discrete low-pass lter associated to the scal-
ing function (x). At level j, the smoothed approxima-
tion A
j
(k) at pixel x is given by the scalar product
A
j
(k) =
1
2
j
_
f(x);
_
x k
2
j
__
which can be written as the convolution
A
j
(k) =
l
h(l)A
j1
(k + 2
j1
l):
This last equality makes clear the increasing gap
between pixels when computing the convolution, hence
the name a trous (with holes).
The wavelet transform of f(x) is dened by the dif-
ference between two consecutive approximations
W
j
(k) =A
j
(k) A
j1
(k)
which can be written as
W
j
(k) =
1
2
j
_
f(x);
_
x k
2
j
__
;
where the wavelet (x) associated to the scaling function
(x) is dened by
1
2
_
x
2
_
=(x)
1
2
_
x
2
_
:
References
[1] A. H emar, J.-C. Olivo, E. Williamson, R. Sarich, C.
Dotti, Dendroaxonal transcytosis of Transferrin in cultured
hippocampal and sympathetic neurons, J. Neuroscience
17 (23) (1997) 90269034.
[2] G. Dumenil, J.-C. Olivo, S. Pellegrini, M. Fellous,
P.J. Sansonneti, G. Tran Van Nhieu, Interferon inhibits
a src-mediated pathway necessary for Shigella-induced
cytoskeletal rearrangements in epithelial cells, J. Cell Biol.
143 (4) (1998) 10031012.
[3] N. Otsu, A threshold selection method from gray-level
histograms, IEEE Trans. Syst., Man Cybern. 9 (1979)
377393.
[4] P. Serra, Image Analysis and Mathematical Morphology,
Academic Press, New York, 1982.
[5] X. Bright, E.B. Steel, Two-dimensional top hat lter
for extracting spots and spheres from digital images, J.
Microsc. 146 (2) (1987) 191200.
[6] X. Breen, G.H. Joss, K.L. Williams, Locating objects of
interest within biological objects: the top hat box lter, J.
Comp. Ass. Microsc. 3 (2) (1991) 97102.
[7] R.P. Blanford, S.L. Tanimoto, Bright-spot detection in
pyramids, Comput. Vision Graphics Image Process. 43
(1988) 133149.
[8] J.-C. Olivo, Automatic detection of spots in biological
images by a wavelet-based selective ltering technique,
Proceedings of IEEE third International Conference
on Image Processing, ICIP96, Lausanne, Switzerland,
September 1996, pp. III: 311314.
[9] Y. Xu, J.B. Weaver, D.M. Healy, J. Lu, Wavelet
transform domain lters: a spatially selective noise
ltration technique, IEEE Trans. Image Processing 3 (6)
(1994) 747758.
[10] A. Rosenfeld, M. Thurston, Edge and curve detection for
visual scene analysis, IEEE Trans. Comput. 20 (5) (1971)
562659.
[11] S. Mallat, S. Zhong, Characterization of signals from
multiscale edges, IEEE Trans. Pattern Anal. Machine
Intell. 14 (7) (1992) 710732.
[12] J.-L. Starck, F. Murtagh, A. Bijaoui, Multiresolution
support applied to image ltering and restoration, Graph.
Models Image Process. 57 (5) (1995) 420431.
[13] F. Murtagh, J.-L. Starck, Image processing through
multiscale analysis and measurement noise modeling, Stat.
Comput. 10 (2000) 95103.
[14] J.-L. Starck, F. Murtagh, A. Bijaoui, Image processing
and Data Analysis: the Multiscale Approach, Cambridge
Press, Cambridge, 2000.
[15] B.M. Sadler, A. Swami, Analysis of multiscale products
for step detection and estimation, IEEE Trans. Inform.
Theory 45 (3) (1999) 10431051.
[16] S. Mallat, A theory for multiresolution signal decom-
position: the wavelet representation, IEEE Trans. Pattern
Anal. Machine Intell. 11 (7) (1989) 674693.
[17] I. Daubechies, Orthogonal bases of compactly supported
wavelets, Commun. Pure Appl. Math. XLI (1988)
909996.
[18] J.-C. Olivo, Automatic threshold selection using the
wavelet transform, CVGIP: Graph. Models Image Process
56 (3) (1994) 205218.
[19] J. Deubler, J.-C. Olivo, A wavelet-based multiresolu-
tion method to automatically register images, J. Math.
Imaging Vision 7 (3) (1997) 199209.
[20] M. Holschneider, R. Kronland-Martinet, J. Morlet, A
real-time algorithm for signal analysis with the help of
the wavelet transform, in: J.M. Combes, A. Grossmann,
Ph. Tchamitchian (Eds.), Wavelets: Time-Frequency
Methods and Phase Space, Springer, Berlin, pp.
286297, 1989.
[21] I.M. Johstone, B.W. Silverman, Wavelet threshold
estimators for data with correlated noise, J. Roy. Statist.
Soc. Ser. B 59 (1997) 319351.
[22] M. Jansen, A. Bultheel, Multiple wavelet threshold
estimation by generalized cross validation for images with
correlated noise, IEEE Trans. Image Processing 8 (7)
(1999) 947953.
1996 J.-C. Olivo-Marin / Pattern Recognition 35 (2002) 19891996
About the AuthorJEAN-CHRISTOPHE OLIVO-MARIN received a Ph.D. in 1989 and the Habilitation a Diriger des Recherches
in 1998 from the Institut dOptique Th eorique et Appliqu ee, University of Paris-Orsay, France. He is in charge of the Quantitative
Image Analysis Group at the Pasteur Institute, Paris. Previous to that, he was a sta scientist at the European Molecular Biology
Laboratory, Heidelberg, from 1990 to 1998. His research interests are in image processing and computer vision applied to biological
image analysis, with special emphasis in multiresolution processing, image segmentation and video microscopy sequence analysis.
Dr. Olivo-Marin is a member of IEEE, SPIE and the Pattern Recognition Society.