Pattern Recognition Letters 31 (2010) 2474–2488

Contents lists available at ScienceDirect

Pattern Recognition Letters

journal homepage: www.elsevier.com/locate/patrec

Identifying touching and overlapping chromosomes using the watershed

transform and gradient paths
Petros Karvelis a,b, Aristidis Likas a, Dimitrios I. Fotiadis b,⇑
a
Department of Computer Science, University of Ioannina, GR 45110 Ioannina, Greece
b
Unit of Medical Technology and Intelligent Information Systems, Department of Materials Science and Engineering, University of Ioannina, GR 45110 Ioannina, Greece

a r t i c l e i n f o a b s t r a c t

Article history: Automation of chromosome analysis has long been considered as a difficult task. However the advent of
Received 23 December 2009 Multiplex Fluorescence In Situ Hybridization (M-FISH) made the analysis of chromosomes much easier.
Available online 17 August 2010 Nevertheless, the chromosomes in an M-FISH image do very often partially occlude each other; hence,
Communicated by T. Vasilakos
their segmentation is not trivial and requires the application of a dedicated procedure. In this paper a
method is presented for the segmentation of touching and overlapping groups of chromosomes in M-FISH
Keywords: images. Initially, the watershed transform is applied and the image is decomposed into watershed
Chromosome classification
regions. Next, gradient paths starting from points of high concavity are computed for each produced
Watershed transform
Karyotyping
region. Finally, adjacent regions are merged producing the final chromosome areas. To validate our
Multiplex Fluorescent In Situ Hybridization method a benchmark database of 183 M-FISH images has been used. The proposed algorithm resulted
in a 90.6% success rate for touching chromosomes and 80.4% for overlapping groups of chromosomes.
Ó 2010 Elsevier B.V. All rights reserved.

1. Introduction vector, where each element of the vector represents the magnitude
of the dye at that pixel of the image, Fig. 2. This technique not only
Chromosomes are structures that contain the genetic informa- facilitates the detection of subtle chromosomal aberrations (Veld-
tion of cells. In a normal, nucleated human cell, there are 46 chro- man et al., 1997), but also makes the analysis of chromosome
mosomes represented in the clinical routine by a structure called images easier; both for human inspection and computerized anal-
the karyotype. The karyotype shows the complete set of chromo- ysis. However, in practice, fluorophore absorption is not binary and
somes organized into 22 classes (each of which consists of a there is significant overlap between each of the fluorophore
matching pair of two homologous chromosomes) and two sex absorptions along with variability in signal strength. This leads to
chromosomes, XX in females or XY in males (Thompson et al., a non-trivial classification problem, especially in the context of
1991). Producing a karyotype of a cell is of practical importance touching or overlapping regions (Schwartzkopf et al., 2005).
since it greatly facilitates the detection of abnormalities in the Many attempts have been made to automate parts of the chro-
chromosome structure as shown in Fig. 1. Normally, the procedure mosome M-FISH image analysis procedure (Schwartzkopf et al.,
of assigning each chromosome to a class (karyotyping) is based on 2005; Sampat et al., 2005; Wang and Castleman, 2005; Wang
the visual scanning of chromosome images by experts (biologists and Dandpat, 2006; Karvelis et al., 2008). However, chromosome
or cytogeneticists) (Thompson et al., 1991). This visual inspection images are inherent with the partial occlusion and touching of
is a time consuming and expensive process. Hence automated chromosomes, as shown in Fig. 3. This is one of the major factors
image chromosome analysis is still an important problem. hindering automatic analysis. Spectrum based methods use a
A technique was developed in the mid 90s to stain chromo- pixel-by-pixel classifier to classify each pixel of the M-FISH image
somes with multiple colours so that each chromosome class and this information may be sufficient to segment touching and
appears with a distinct colour (Speicher et al., 1996). In this tech- overlapping chromosomes (Schwartzkopf et al., 2005). However
nique all chromosomes are labelled with five fluorophores. Also a the measured fluorescence at a pixel may be the combination of
DNA stain, called DAPI (40 ,6-diamidino-2-phenylindole), is used fluorescence in a neighbouring region leading many times to
to stain all the chromosomes with the same colour. The fluoro- misclassification errors. These factors make the pixel spectral
phores attach to specific sequences of DNA, thus each pixel of information of touching or overlapping chromosomes unreliable.
the new multispectral image is represented as a five-dimensional Hence the spectral information alone cannot separate the touching
and overlapping chromosomes efficiently.
⇑ Corresponding author. Tel.: +30 26510 08803; fax: +30 2651007092. On the other hand there is a variety of geometric separation
E-mail address: [email protected] (D.I. Fotiadis). based methods proposed in the literature for greyscale

0167-8655/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved.
doi:10.1016/j.patrec.2010.08.002
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2475

We can conclude that when only the spectral information is

used, the segmentation accuracy relies on the pixel-by-pixel classi-
fication accuracy. On the contrary, the geometry based methods as-
sume that chromosome shape alone is sufficient for the purpose of
separation. Thus both, geometry and spectral information, has to
be merged in order to achieve better segmentation results for M-
FISH chromosome images.
In this paper we present a novel segmentation method that
tackles the problem of touching–overlapping group of chromo-
somes. Initially, the method uses the watershed transform to seg-
ment the DAPI image into watershed regions. The watershed
transform has been widely used for the separation of touching/
overlapping groups of objects from images (Beucher, 1992; Vincent
and Soille, 1991; Malpica et al., 1997; Chen et al., 2003). In our case
we propose the recursive application of the watershed transform
to each watershed region. However there exist difficult cases of
touching as also of overlapping groups of chromosomes that need
separation. For this reason we use a geometry method such as the
‘‘gradient paths” to split each group of touching–overlapping chro-
mosomes. However we do not compute the gradient paths using
the intensity of pixels of the DAPI image, but we propose the com-
putation of paths in the M-FISH image using pixels with high mul-
tichannel gradient magnitude values. This computation proves to
be more efficient than the computation of the gradient path on
the DAPI image since there are cases of touching or overlapping
groups of chromosomes where the gradient path on the DAPI im-
age is difficult to compute since the chromosomes are difficult to
disentangle. Finally, after path computation, a region adjacency
graph is computed and a region merging algorithm is used to
merge all regions. In Section 2 the methods are presented describ-
ing in detail all the steps. Section 3 presents the results of the
methodology. The discussion of the results is also presented in this
section. Finally the conclusions of our work are presented in
Section 4.

2. Method

The proposed method consists of three stages as it is shown in

Fig. 4: (a) the recursive watershed transform computation, (b) the
computation of each gradient path and (c) the region merging pro-
cess. The first stage consists of a number of steps. The first step is
the conversion of the initial DAPI chromosome image to binary. In
the second step, the Euclidean distance transform of the binary im-
age is computed. The watershed transform is applied in the next
step and an initial estimation of the segmented chromosome areas
Fig. 1. (a) M-FISH chromosome image of a woman missing three chromosomes
from classes 7, 15, 19, and (b) Karyotype of the M-FISH image: 43XX, -7,-15,-19. is obtained. The watershed transform is further applied separately
to every segmented area until no more new areas are created. The
first step of the second stage is the computation of the high concav-
chromosome images (Ji, 1989, 1994; Agam and Dinstein, 1997; Rit-
ity points along the boundary of each chromosome area. Next, all
ter and Schreib, 2001). The main idea of these methods is that they
gradient paths are computed and the binary chromosome area is
split the chromosome groups into segments and then they try to
split along the gradient path. All gradient paths are computed
combine these segments into chromosomes. Valley searching tech-
using the multichannel gradient magnitude. In the final stage a
niques (Ji, 1989, 1994) attempt to find a ‘‘pale path” of grey values
recursive region merging procedure is applied as follows. A region
corresponding to a separation between touching–overlapping
adjacency graph is computed and also each region is classified
groups of chromosomes. Initially, all high concavity points (cut-
independently using a region Bayes classifier. Then we merge all
points) are detected along the boundary of chromosomes. Next, a
neighbouring regions that share the same class. The identification
heuristic search is performed to detect the minimum density path
of the overlapping chromosomes takes place in the final step.
between touching chromosomes. The chromosome group is split
by the pale path and the segments are combined to form separate
chromosomes. Agam and Dinstein (1997) used concave points to 2.1. Recursive watershed segmentation
construct all the possible separation lines. In their work, they
determined potential chromosomes using rectangle hypothesis In the first step, the DAPI chromosome image is converted to
testing. However this hypothesis does not always hold because of binary using a well known automated threshold selection process
the existence of bended chromosomes that are touching or over- (Otsu, 1979). Using the DAPI channel an initial estimation of the re-
lapping to each other and thus a straight line cannot split exactly gions of the M-FISH image is produced. The threshold operation at
the chromosomes. grey level l partitions the pixel values of an image into two classes
2476 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 2. An M-FISH image and its channels. (a) Cy 5.5 fluorophore, (b) Cy 5 fluorophore, (c) Spectrum Green fluorophore, (d) Spectrum Orange fluorophore, (e) Texas Red
fluorophore, (f) DAPI fluorophore, and (e) M-FISH image.

K0 and K1 (representing background and object respectively), i.e., In order to apply the watershed transform (WT) (Vincent and
K0 = {1, 2, . . . , l} and K1 = {l + 1, l + 2, . . . , L}, where L is the total Soille, 1991) to the image B it is common to first compute the dis-
number of grey levels in the image. An optimal threshold l* can tance transform (DT) (Malpica et al., 1997; Chen et al., 2003). Given
be determined by minimizing the following criterion function: an m  n binary image B, its distance transform is a map that as-

signs to each on-pixel (p1) (with coordinates (x1, y1)) the distance
l ¼ arg min r2B ðlÞ; to the nearest off-pixel (p2) (with coordinates (x2, y2)). The distance
l qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
metric used is the Euclidean distance D ¼ ðx1  x2 Þ2 þ ðy1  y2 Þ2 .
where r2B ðlÞ is the between-class variance for the threshold value l
The distance transform provides important information for the
(Otsu, 1979).
application of the watershed algorithm (Beucher, 1992). The num-
After the computation of the threshold l* the binary image B can
ber of regional minima of the negative distance transform consti-
be computed:
tutes indication of the number of areas that will be segmented
  
0 if DAPIðx; yÞ 6 l ; by the WT. However a common problem is that the distance trans-
Bðx; yÞ ¼  form contains a large number of such minima leading the WT to
1 if DAPIðx; yÞ > l :
over segment the initial image. On the other hand the greyscale
An example of the application of the threshold operation to a DAPI reconstruction (Vincent, 1993) of the negative distance transform
image is shown in Fig. 5. suppresses all minima whose depth are lower than or equal to a
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2477

Fig. 3. Touching and overlapping group of chromosomes. (a) Three chromosomes that are touching each other, (b) two chromosomes that overlap.

threshold h 2 R Thus we apply this procedure in order to alleviate greyscale images such as the G-banded chromosome images (Sum-
the over-segmentation problem. An alternative for the elimination ner et al., 1971). It is based on two assumptions: (a) where chro-
of the over-segmentation effect could be the Gaussian blur of the mosomes touch the cluster boundary tends to form an acute
gradient image (Gauch, 1999), however the choice of the width angle and (b) at points where chromosomes touch, the optical den-
of the Gaussian kernel is a key parameter for these approaches. sity is relatively low. The detection of the paths is computed via a
The next step is the application of the WT. The watershed trans- search algorithm. The search begins at a cut-point and proceeds in
form is a popular segmentation method originated in the field of the direction of the normal vector. A cut-point is a boundary point
mathematical morphology. The image is considered as a topo- at which the boundary is highly concave. It then proceeds until an-
graphical relief, where the height of each point is related to its grey other boundary point is found as follows: At the current point a list
level. Imaginary rain falls on the terrain and water begins to rise of candidates is found as it is shown in Fig. 7(a). A new trace point
filling the different catchment basins. The watersheds are the lines is found by choosing the candidate with the smallest intensity va-
separating the catchment basins that form. lue. Finally, the searching direction is updated every d points to al-
In our case we apply the watershed method using the negative low the path to follow the shape of its trace points, as it is shown in
distance transform. The watershed algorithm produces a tessella- Fig. 7(b). The path that starts from the cut-point and ends to a
tion of the image into regions, these regions are called watershed boundary point was called a pale path.
regions and depicted in Fig. 6(a) and (b). Whereas several methods The pale paths (Ji, 1989, 1994) were used to cut only touching
start with an over-segmentation of the image and iteratively merge groups of chromosomes without addressing the case of overlap-
regions based on some measures of similarity (Haris et al., 1998), ping chromosomes. Moreover, these studies computed the pale
our method introduces a new region splitting technique based on paths only for greyscale images. Using a low intensity path the sep-
the watershed transform. All the steps of our method – which do aration of touching chromosomes is feasible, but fails particularly
not require any a priori knowledge – are recursively applied to in overlapping cases. Indeed as it is shown in Fig. 8(a) a pale path
every watershed area until no more new areas are produced. The does not exist for the case of the overlapping group of chromo-
result of the recursive watershed transform is shown Fig. 6(c)–(f). somes since the intensity of the overlapping region is homogenous
and relatively high.
2.2. Computation of gradient paths In this work, we propose a modification of the pale path ap-
proach in order to achieve separation of touching and overlapping
The idea of paths has been introduced in early 90s (Ji, 1989, chromosome groups in coloured M-FISH images. This modification
1994) in order to separate touching groups of chromosomes for uses the multichannel gradient of the M-FISH image (Karvelis et al.,
2478 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 4. Flowchart of our method.

2007; Drewniok, 1994). The basic idea is the following: instead of Schreib, 2001) since local maxima of the curvature indicate candi-
leading the path to follow low intensities pixels, the path now fol- date positions of the cut-points. For each point of the boundary (ci:
lows pixels of high multichannel gradient magnitude values. The i = 1, . . . , PB) we consider the triangle that is defined from the three
computation of the multichannel gradient magnitude is based on points cik, ci, ci+k (k = 3) and compute the angle a(i) defined by the
the five channel coloured M-FISH image. This gives the advantage triangle:
that the paths follow high gradient magnitude pixel values and
 
these high values occur when chromosomes touch or overlap. ðci  cik Þ  ðciþk  ci Þ
aðiÞ ¼ arccos  sgn½detð ci  cik ciþk  ci Þ:
The path that begins from a cut-point and follows pixels of high kci  cik k  kciþk  ci k
gradient magnitude values of the M-FISH image until it reaches a
boundary point is now called a gradient path. In Fig. 8, we demonstrate the steps for the computation of the
To compute the cut-points we first extract the boundary from cut-points in a group of touching and overlapping chromosomes.
the binary image B. Suppose that the pixels of the boundary of a After the binarization of the chromosome group (Fig. 8(a) and
segmented region define the set (c1, c2, . . . , cPB) where ci, ci+1 are (b)) the curvature of the boundary points (Fig. 8(c)) is computed
successive points of the boundary and PB the number of pixels of and is illustrated in Fig. 8(d). All the cut-points are automatically
the region boundary. In order to compute the cut-points we com- computed by choosing the boundary points that exceed an angle
pute the curvature of the boundary (Ji, 1989, 1994; Ritter and threshold: a(i) P 210°, i = 1, . . . , PB. The red points in Fig. 8(d)
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2479

The directional derivative of I(x, y) for a direction n is computed

as follows:
2 3 2 3
rI 1  n Ix1 Iy1
@I 6 .. 7 6 . .. 7
¼6 .
7¼6 . 7
. 5 ¼ J  n;
@n 4 5 4 .
rIm  n Ixm Iym
where Ixi and Iyi 1 6 i 6 m are the derivatives of the ith component in
the x and y direction respectively.
Next the direction n which corresponds to the maximum of the
directional derivative is found by maximizing the Euclidean norm
kJ  nk2 ¼ ðJ  nÞT ðJ  nÞ ¼ nT ðJ T JÞn.
The maximum of this norm nT(JTJ)n is given by the maximum
eigenvalue of the matrix (JTJ) (Karvelis et al., 2008). The symmetric
matrix (JTJ) can be written as:
2 3
Pm
x 2 P
m
6 ðIi Þ Ixi  Iyi 7
6 i¼1 i¼1 7
JT J ¼ 6 m 7;
4P x y Pm
y 2 5
I i  Ii ðIi Þ
i¼1 i¼1

where finite differences (e.g. Sobel operators (Karvelis et al., 2008))

can be used to compute the directional derivatives Ixi and Iyi
1 6 i 6 m in the x and y directions respectively.
Next we proceed to compute the gradient path. The initial
direction of the gradient path is set as the bisector of the angle
aðiÞ ¼ \cik ; ci ; ciþk at the starting points cik, ci, ci+k, as it is shown
in Fig. 9(a) and (b), where ci is the initial cut-point. The computa-
tion of the gradient path proceeds as follows: we choose from
the pixel-candidates the one that has the maximum gradient value.
We then proceed to the next pixel updating the current search
direction every d = 3 points until we reach a boundary point. Final-
ly, we delete points, of the binary image, along the gradient path.
We present the computation of the gradient path for a touching–
overlapping chromosome group in Fig. 9(c) and (d) and in
Fig. 9(e) the final regions produced from the binary image by cut-
ting along the gradient paths.

Fig. 5. The thresholding procedure for a greyscale DAPI image. (a) The DAPI image 2.3. Region merging
and (b) the binary image.
The purpose of this stage is to connect regions that have been
split by gradient paths. In our case we call a region small if it con-
and (e) illustrate the cut-points that exceed this angle threshold. As tains less than 25 pixels. This step was implemented by computing
we observe in Fig. 8(e), several candidate cut-points are computed. for each region of the binary image, the Region Adjacency Graph
To overcome this problem, the neighbouring candidate cut-points (Haris et al., 1998) (RAG), where two nodes (representing two dis-
are automatically grouped and from each group of candidate cut- tinct regions) are connected if the corresponding regions are adja-
points we extract the one that has the maximum angle as it is illus- cent. An example of a RAG is shown in Fig. 10.
trated in Fig. 8(f). Then a Region Bayes Classifier (RBC) (Karvelis et al., 2007; Land-
The next step of our method is the computation of the multi- grebe, 1980) was employed in order to classify all the regions of
channel gradient magnitude, as the gradient path will follow pixels the watershed area as follows. Suppose a region R = {z1, z2, . . . , zw}
having high multichannel gradient values. The multichannel gradi- consists of w pixels. Each vector zi 2 R5 ; i ¼ 1; . . . ; w measures the
ent magnitude is computed as follows. Assume a multichannel im- intensity of each of the five M-FISH channels. Also let xj,
age Iðx; yÞ : Z2 ! Rm for an M-FISH image and a direction n defined j = 1, . . . , NC the chromosome classes, where NC = 24.
by the angle u: The likelihood of the region R is computed as (Landgrebe, 1980):
2 3 Y
w
I1 ðx; yÞ
6 7 pðRjxi Þ ¼ pðz1 ; z2 ; . . . ; zw jxj Þ ¼ pðzi jxj Þ
6 I1 ðx; yÞ 7 i¼1
6 7 !w !
Iðx; yÞ ¼ 6 7;
6
6
.. 7
7 1 1X w
t 1
4 . 5 ¼ exp  ðzk  lj Þ Rj ðzk  lj Þ ;
ð2pÞ5=2 jRj j1=2 2 k¼1
Im ðx; yÞ
" #
cos u where lj is the five-component mean vector of class j, Rj is the 5  5
n¼ ; covariance matrix of class j, |Rj| and R1 are the determinant and
sin u j
inverse respectively. The mean vectors and the covariance matrixes
where Ii(x, y), i = 1, . . . , 5 are the components (channels) of the for each class are computed by a training phase, from an annotated
M-FISH image. set of M-FISH images as follows:
2480 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 6. An example of the application of the recursive watershed transform for a DAPI chromosome image. (a) DAPI image, (b) 1 iteration, (c) 2 iteration, (d) 3 iteration (e) 4
iteration and (f) 5 iteration.

1
Nj
X a
lj ¼ zk ; j ¼ 1; . . . ; NC ; Perpendicular to Current
Nj k¼1;zj 2xj search direction search
direction
Nj
1 X
Cj ¼ ðzk  lj Þ  ðzk  lj ÞT ; j ¼ 1; . . . ; NC ;
Nj  1 k¼1;zj 2xj

where Nj the number of pixels of the jth class.

Using Bayes theorem (Karvelis et al., 2007; Fukunaga, 1990):
Pixel
Candidates
pðRjxj Þ  Pðxj Þ
Pðxj jRÞ ¼ P24 ;
j¼1 pðRjxj Þ  Pðxj Þ

we can classify a region R to class j having maximum a posterior

probability (Fukunaga, 1990):

Pðxj jRÞ > Pðxq jRÞ 8q ¼ 1; . . . ; 24; q – j: b Current search

direction
The prior probabilities are estimated from the same set of anno-
tated images as follows:
Pn
Nq Pn-d
Pðxq Þ ¼ P24 ; q ¼ 1; . . . ; NC :
i¼1 N q New search direction
after d=3 path points
For each small region R:(# pixels of R 6 25) of the watershed
area we use the RAG to select the neighbours NR of R: NR = Fig. 7. Pale path computation: (a) candidates for the next path point and (b) update
{R1, R2, . . . , Rk}. Let xi the class of region Ri e NR, i = 1, . . . , k and of path’s direction after d = 3 points.
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2481

Fig. 8. The computation of the cut-points for a touching–overlapping group of chromosomes. (a) The DAPI watershed area, (b) the binary image, (c) the boundary of the group
of chromosomes overimposed on the M-FISH image (the yellow point depicts the first boundary point), (d) the curvature along the boundary points with red points are
depicted the cut-points that exceed the angle threshold, (e) the cut-points (red points) overimposed on the M-FISH image, (f) the groups of the candidate cut-points and (g)
the final computed cut-points. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
2482 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 9. The computation of the gradient paths. (a) The initial search direction and the perpendicular of the search direction, (b) a gradient path reaching the other side of the
boundary (the green points depict the points of the gradient path), (c) all the gradient paths computed for all the cut-points of the chromosome group, and (d) the binary
image after the binary image has been cut by the gradient paths.
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2483

Fig. 10. The Region Adjacency Graph after the gradient paths split the binary image. (a) The M-FISH image for a touching group of chromosomes (the thin white line depicts
the boundary of the binary image), (b) the gradient paths overimposed on the M-FISH image, and (c) the Region Adjacency Graph after the gradient paths split the
chromosomes.

Fig. 11. Region merging of the binary image. (a) Initial region adjacency graph and the classes of each of the regions of the binary image: the small region R3 is merged with
region R1, since the posterior probability P(x1|R3) P P(xi|R3), i = {1, 4}, (b) the region R1 and R4 are merged since they share the same class x1 and (c) the final region merging
result.
2484 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 12. Two overlapping chromosome cases. The proposed method identifies them correctly. CASE A: Regions R2 and R4 are identified as an overlap since the region R3 is
connected with two regions of the class x23. CASE B: Regions R1 and R4 are identified as an overlap since the region R2 is connected with two regions of the class x1.

Table 1 Table 2
Number of touching and overlapping chromosomes in the M-FISH database. Comparison of our work with other works presented in the literature for the touching
group of chromosomes.
Touching chromosomes Overlapping chromosomes
Schwartzkopf Ji (1989) The proposed
Total number 1178 189
et al. (2005) method
Separation accuracy Accuracy (%) 77 84.2 90.6
Dataset description #Images 183 183 183
#Touches 720 1178 1178
XR = {x1, x2, . . . , xi, . . . , xk} the set of these classes. Then we com-
pute the posterior probabilities P(xi|R) of region R for xi e XR and
region R is merged to region Rj whose class xj has maximum pos-
terior: XR = {x1, x2, . . . , xi, . . . , xk}
3. Results and discussion
Pðxj jRÞ  Pðxi jRÞ 8xi 2 XR :
3.1. Dataset
An example of small region merging is shown in Fig. 11(a) and (b).
The next step of our method is to classify all the regions (includ- To validate our method we used the ADIR M-FISH database
ing the merged ones) of the binary image using the Region Bayes (Schwartzkopf et al., 2005) which is a database of M-FISH images.
Classifier described previously. Then all regions that are adjacent The database consists of 183 multispectral M-FISH images. Each
and share the same class are connected. Finally, the RAG is com- image has 517  645 pixels of 8-bit resolution. The database con-
puted and the procedure is repeated until no more regions are tains five-channel image sets recorded at different wavelengths.
connected. In addition, a DAPI image file is included for each M-FISH image.
The final step of our method is the identification of the overlap- The dataset includes also a classification map (karyotype image),
ping chromosomes. The key idea in this step is that when two stored as an image file established by experienced cytogeneticists.
chromosomes overlap, a cross shaped object is formed. In this case This image is labelled so that the grey level of each pixel represents
our method splits the binary image in way that one region of the its class number (chromosome class). In addition, background pix-
image separates two regions that share the same class. This is illus- els are 0, and pixels in a region of overlap are 1. This data file
trated in Fig. 12. Thus the final step of our method is to identify serves as ‘‘ground truth” to test the accuracy of our method.
these overlapping cases by checking for each region of the binary As a ground truth for the touching chromosomes, we used the
image whether it has two neighbouring regions of the same class. binary image produced by the DAPI image to identify the cases
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2485

Fig. 13. Examples of three different cases of bended chromosomes. The proposed method handles them successfully after the region merging stage.

of touching chromosomes in an M-FISH image. For each object pro- Table 3

duced by the binarization procedure we determined the cases of Comparison of our work with other works presented in the literature for the
touching. Finally as a ground truth for the cases of overlapping overlapping group of chromosomes.
we used the characterized karyotype image of the M-FISH database Schwartzkopf The proposed
since an overlapping region is represented in that image by pixels et al. (2005) method
having the value of 1. The number of touches and overlaps in the Separation accuracy Accuracy (%) 34 80.4
M-FISH database is shown in Table 1. Dataset description #Images 183 183
#Overlaps 189 189

3.2. Touching chromosomes

The separation accuracy for the touching group of chromo- 3.3. Overlapping chromosomes
somes was measured by our method. A correct separation occurs
when two or more touching chromosomes are segmented The separation accuracy for the overlapping group of chromo-
correctly. The results of our method for the touching groups of somes is also measured. The results of our method for the overlap-
chromosomes are shown in Table 2. We have also compared ping groups of chromosomes are shown in Table 3.
our method with the method of pale paths (Ji, 1989) for the We have described a novel method for the separation of touch-
touching groups of chromosomes as the method of pale paths ing and overlapping groups of M-FISH chromosome images. Our
cannot handle overlapping cases. In order to compute the pale method is based on the recursive application of the watershed
path we have used the DAPI image since the pale path uses a transform and the computation of gradient paths for each wa-
greyscale image. tershed area. A region merging stage is finally applied to merge re-
It is interesting to mention the robust behaviour of our method gions that have been wrongly split by the gradient paths. Our
in the case of isolated bended chromosomes. It is common in the method is evaluated using an M-FISH chromosome image database
M-FISH chromosome database to find cases where isolated and an overall separation accuracy of 90.6% and 80.4% for the
chromosomes bend, as shown in Fig. 13. For these cases, cut-points touching and overlapping groups of chromosomes respectively
are found and gradient paths begin to split the chromosome into has been found.
two regions. However the region merging stage merges these In fact, only one method has been presented in the literature for
regions to form one chromosome again. the separation of M-FISH images testing its ability to separate
2486 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

Fig. 14. Six examples of touching and overlapping groups of chromosomes where straight lines between cut-points cannot separate correctly the chromosomes.

touching and overlapping groups of chromosomes for the whole (B) The paths have been appropriately modified in order to sep-
M-FISH database (Schwartzkopf et al., 2005). Our method uses arate overlapping and touching groups of chromosomes in
the information from all the channels (the five channel M-FISH im- M-FISH images by computing the multichannel gradient
age including the DAPI image) whereas Schwartzkopf et al. (2005) and the running of the path through high gradient magni-
use only the information provided by the five channel M-FISH tude values. Fig. 15 illustrates some examples of the compu-
image. tation of the pale path (Ji, 1989, 1994) using the DAPI image
To best of our knowledge the pale paths were able to separate versus the gradient path which uses the multichannel gradi-
only touching groups of chromosomes without handling overlap- ent magnitude of the M-FISH image.
ping chromosomes (Ji, 1989, 1994). In this paper, we expand the
idea of the paths in order to address also the case of overlapping While previous methods were based only on region merging
groups. More specifically we introduce the gradient paths which techniques (Karvelis et al., 2008), the proposed method utilizes re-
more effectively segment not only touching but also overlapping gion merging and region splitting techniques to correctly segment
groups of chromosomes for the M-FISH images. The gradient path the chromosome touching and/or overlapping groups. More specif-
is superior to other proposed splitting techniques for two reasons: ically the proposed method starts with an initial number of seg-
mented regions and recursively split or merge the regions until
(A) Unlike other methods (Agam and Dinstein, 1997), we do not no more regions are produced. Fig. 13 is an indicative example of
assume that a path is a straight line between two cut-points. how the region merging step corrects erroneous separations of
Fig. 14 depicts some examples of touching and overlapping the region splitting step. Although, the region splitting technique
chromosome groups, none of which can be split by a straight separated the chromosome into two distinct regions, region merg-
line without fragmenting a chromosome. Such cases usually ing corrected this error by merging the regions again to form one
happen where more than two chromosomes are involved in chromosome.
a group of touching–overlapping or one of the chromosomes There is a tradeoff between the choice of this angle threshold
is bent. and the number of cut-points. If we choose a smaller threshold a
 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488 2487

Fig. 15. Pale path versus gradient path. Three examples where the pale path fails to separate correctly the chromosomes while the gradient path correctly separates them.

larger number of cut-points is computed. However, the grouping of ping chromosome groups are highly likeable to appear in a chro-
neighbouring cut-points and the selection of those with the maxi- mosome image making time consuming and difficult the effort of
mum angle (among their neighbours) result in the same final cut- the cytogeneticist. Therefore, the efficient treatment of this prob-
points. On the other hand the choice of a larger angle threshold will lem is very important for the biologists. Since our method provides
result to a smaller number of cut-points and thus less gradient sufficiently accurate results in identifying touching and overlap-
paths will be computed. Thus, the separation of the touching or ping chromosomes, it is expected to be a very useful tool that will
overlapping group of chromosomes will not be feasible. In our greatly facilitate the task of the cytogeneticist.
experiments we have heuristically determined the best value for
this parameter.
Tables 2 and 3 show a comparison between the proposed study 4. Conclusions
and the method which proposed by Schwartzkopf et al. (2005).
While the number of overlapping cases is the same, the number A method for the separation of touching and overlapping groups
of touches differs between the two methods. This is done due to of chromosomes in M-FISH images was presented. The method is
the different approaches employed by the two methods for the based on the recursive application of the watershed transform
determination of touching groups of chromosomes. In our case and the splitting of groups of chromosomes by multichannel gradi-
we used the binary image produced by the DAPI image to identify ent paths. These paths split these groups of chromosomes from
the number of touching chromosomes in an M-FISH image points of high concavity and follow pixels of high multichannel
whereas Schwartzkopf et al. (2005) has manually chosen the num- gradient magnitude. Then a region merging stage is applied to
ber of touches. In general, it is difficult to compare the two meth- merge neighbouring regions producing the final chromosome
ods directly since they are not handling the same number of areas. The computation of the gradient path based on multichannel
touching chromosomes. However our method is employed in the gradient values instead of the computation of the pale path which
same M-FISH database and the number of touches is higher than is based on low intensity values proves to be more robust. Also the
that reported in Schwartzkopf et al. (2005). We have also com- gradient paths can now handle overlapping cases of chromosomes
pared our method with the method of pale paths (Ji, 1989, 1994) whereas the pale paths could only handle touching groups of chro-
for the touching groups of chromosomes as the method of pale mosomes. As for future work an unsupervised method will be
paths cannot handle overlapping cases. Touching and/or overlap- developed for the classification of M-FISH images. In this way we
2488 P. Karvelis et al. / Pattern Recognition Letters 31 (2010) 2474–2488

avoid the use of already characterized M-FISH images in order to Landgrebe, D., 1980. The development of a spectral-spatial classifier for earth
observational data. Pattern Recognition 12 (3), 165–175.
train the Bayes classifier.
Malpica, N., Solorzano, C., Vaquero, J., Santos, A., Vallcorba, I., Sagredo, J., Pozo, F.,
1997. Applying watershed algorithms to the segmentation of clustered nuclei.
Cytometry 28, 289–297.
References Otsu, N., 1979. A threshold selection method for gray-levels histograms. IEEE Trans.
Pattern Anal. Machine Intell. 13, 583–598.
Agam, G., Dinstein, I., 1997. Geometric separation of partially overlapping non rigid Ritter, G., Schreib, G., 2001. Using dominant points and variants for profile
objects applied to automatic chromosome classification. IEEE Trans. Pattern extraction from chromosomes. Pattern Recognition 34, 923–938.
Anal. Machine Intell. 19, 1212–1222. Sampat, M., Bovik, A., Aggarwal, J., Castleman, K., 2005. Supervised parametric and
Beucher, S., 1992. The watershed transformation applied to image segmentation. non-parametric classification of chromosome images. Pattern Recognition 38,
Scanning Microscope. 6, 299–314. 1209–1223.
Chen, Q., Yang, X., Petriu, E., 2003. Watershed segmentation for binary images with Schwartzkopf, W., Bovik, A., Evans, B., 2005. Maximum-likelihood techniques for
different distance transforms. In: 3rd IEEE Internat. Workshop on Haptic, Audio joint segmentation–classification of multispectral chromosome images. IEEE
and Visual Environments and their Applications, 20–21 September 2003, Trans. Med. Imaging 24, 1593–1610.
Ottawa, Canada. Speicher, M., Ballard, S., Ward, D., 1996. Karyotyping human chromosomes by
Drewniok, C., 1994. Multi-spectral edge detection – Some experiments on data from combinatorial multi-fluor FISH. Nature Genet. 12, 368–375.
Landsat-TM. Internat. J. Remote Sens. 15, 3743–3765. Sumner, T., Evans, J., Buckland, A., 1971. A new technique for distinguishing
Fukunaga, K., 1990. Introduction to Statistical Pattern Recognition. Academic Press, between human chromosomes. Nature New Biol. 232, 31–32.
San Diego. Thompson, M., McInnes, R., Willard, H., 1991. Genetics in Medicine. Saunders,
Gauch, J., 1999. Image segmentation and analysis via multiscale gradient watershed Ontario.
hierarchies. IEEE Trans. Image Process. 8 (1), 69–79. Veldman, T., Vignon, C., Schröck, E., Rowley, J., Ried, T., 1997. Hidden chromosome
Haris, K., Efstratiadis, S., Maglaveras, N., Katsaggelos, A., 1998. Hybrid image abnormalities in hematological malignancies detected by multicolor spectral
segmentation using watersheds and fast region merging. IEEE Trans. Image karyotyping. Nature Genet. 15, 406–410.
Process. 7 (12), 1684–1699. Vincent, L., Soille, P., 1991. Watershed in digital spaces: An efficient algorithm based
Ji, L., 1989. Intelligent splitting in the chromosome domain. Pattern Recognition 22 on immersion simulations. IEEE Trans. Pattern Anal. Machine Intell. 13, 583–
(5), 519–532. 598.
Ji, L., 1994. Fully automatic chromosome segmentation. Cytometry 17, 196–208. Vincent, L., 1993. Morphological grayscale reconstruction in image analysis:
Karvelis, P., Tzallas, A., Fotiadis, D., Georgiou, I., 2008. A multichannel watershed- Applications and efficient algorithms. IEEE Trans. Image Process. 2, 176–201.
based segmentation method for multispectral chromosome classification. IEEE Wang, Y., Castleman, K., 2005. Normalization of multicolour fluorescence in situ
Trans. Med. Imaging 27 (5), 697–708. hybridization (M-FISH) images for improving colour karyotyping. Cytometry
Karvelis, P., Tzallas, A., Fotiadis, D., Georgiou, I., 2007. Region based segmentation 64, 101–109.
and classification of multispectral chromosome images. In: 20th IEEE Wang, Y., Dandpat, A., 2006. A hybrid approach of using wavelets and fuzzy
International Symposium on Computer-Based Medical Systems (CBMS), 20– clustering for classifying multispectral florescence in situ hybridization images.
27 June 2007, Maribor, Slovenia. Int. J. Biomed. Imaging, 1–11.