3. IMAGE SEGMENTATION 3.

1 SEGMENTATION
In computer vision, segmentation refers to the process of partitioning a digital image into multiple segments (sets of pixels, also known as super pixels). The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics. The result of image segmentation is a set of segments that collectively cover the entire image, or a set of contours extracted from the image (see edge detection). Each of the pixels in a region is similar with respect to some characteristic or computed property, such as color, intensity, or texture. Adjacent regions are significantly different with respect to the same characteristics.

Thresholding Edge finding Binary mathematical morphology Gray-value mathematical morphology In the analysis of the objects in images it is essential that we can distinguish between the

objects of interest and "the rest." This latter group is also referred to as the background. The techniques that are used to find the objects of interest are usually referred to as segmentation techniques - segmenting the foreground from background. In this section we will two of the most common techniques thresholding and edge finding and we will present techniques for improving the quality of the segmentation result. It is important to understand that: 1. There is no universally applicable segmentation technique that will work for all images, and, 2. No segmentation technique is perfect.

THRESHOLDING:
This technique is based upon a simple concept. A parameter threshold is chosen and applied to the image a [m, n] as follows: called the brightness

This version of the algorithm assumes that we are interested in light objects on a dark background. For dark objects on a light background we would use:

The output is the label "object" or "background" which, due to its dichotomous nature, can be represented as a Boolean variable "1" or "0". In principle, the test condition could be based upon some other property than simple brightness (for example, If (Redness {a [m, n]} >= red), but the concept is clear. The central question in thresholding then becomes: how do we choose the threshold ? While there is no universal procedure for threshold selection that is guaranteed to work on all images, there are a variety of alternatives.

EDGE FINDING:
Thresholding produces a segmentation that yields all the pixels that, in principle, belong to the object or objects of interest in an image. An alternative to this is to find those pixels that belong to the borders of the objects. Techniques that are directed to this goal are termed edge finding techniques.From our discussion on mathematical morphology, specifically Esq. , and, we see that there is an intimate relationship between edges and regions.

BINARY MATHEMATICAL MORPHOLOGY

The various algorithms that we have described for mathematical morphology in Section 9.6 can be put together to form powerful techniques for the processing of binary images and gray level images. As binary images frequently result from segmentation processes on gray level images, the morphological processing of the binary result permits the improvement of the segmentation result.

Salt-or-pepper filtering - Segmentation procedures frequently result in isolated "1" pixels

in a "0" neighborhood (salt) or isolated "0" pixels in a "1" neighborhood (pepper). The appropriate neighborhood definition must be chosen as in Figure 3. Using the lookup table formulation for Boolean operations in a 3 x 3 neighborhood that was described in association with Figure 43, salt filtering and pepper filtering are straightforward to implement. We weight the different positions in the 3 x 3 neighborhood as follows:

For a 3 x 3 window in a [m, n] with values "0" or "1" we then compute:

The result, sum, is a number bounded by 0 <= sum <= 511. Salt Filter The 4-connected and 8-connected versions of this filter are the same and are given by the following procedure: i) Compute sum ii) If ((sum == 1) c [m, n] = 0 Else c [m, n] = a [m, n]

Pepper Filter - The 4-connected and 8-connected versions of this filter are the following procedures: 4-connected 8-connected i) Compute sum i) Compute sum ii) If ( (sum == 170) ii) If ( (sum == 510) c[m,n] = 1 c[m,n] = 1 Else Else c[m,n] = a[m,n] c[m,n] = a[m,n] Isolate objects with holes - To find objects with holes we can use the following procedure which is illustrated in Figure 8. i) ii) iii) iv) Segment image to produce binary mask representation Compute skeleton without end pixels Use salt filter to remove single skeleton pixels Propagate remaining skeleton pixels into original binary mask.

a) Binary image b) Skeleton after salt filter c) Objects with holes Figure 8: Isolation of objects with holes using morphological operations. The binary objects are shown in gray and the skeletons, after application of the salt filter, are shown as a black overlay on the binary objects. Note that this procedure uses no parameters other then the fundamental choice of connectivity; it is free from "magic numbers." In the example shown in Figure 58, the 8-connected definition was used as well as the structuring element B = N8. Filling holes in objects - To fill holes in objects we use the following procedure which is illustrated in Figure 9. i) Segment image to produce binary representation of objects

ii) iii) iv) v)

Compute complement of binary image as a mask image Generate a seed image as the border of the image Propagate the seed into the mask - eq. Complement result of propagation to produce final result

a) Mask and Seed images Figure 9: Filling holes in objects.

b) Objects with holes filled

The mask image is illustrated in gray in Figure 9a and the seed image is shown in black in that same illustration. When the object pixels are specified with a connectivity of C = 8, then the propagation into the mask (background) image should be performed with a connectivity of C = 4, that is, dilations with the structuring element B = N4. This procedure is also free of "magic numbers." Removing border-touching objects - Objects that are connected to the image border are not suitable for analysis. To eliminate them we can use a series of morphological operations that are illustrated in Figure 10. i) ii) iii) iv) Segment image to produce binary mask image of objects Generate a seed image as the border of the image Propagate the seed into the mask - eq. Compute XOR of the propagation result and the mask image as final result

a) Mask and Seed images Figure 10: Removing objects touching borders.

b) Remaining objects

The mask image is illustrated in gray in Figure 10a and the seed image is shown in black in that same illustration. If the structuring element used in the propagation is B = N4, then objects are removed that are 4-connected with the image boundary. If B = N8 is used then objects that 8connected with the boundary are removed. Exo-skeleton - The exo-skeleton of a set of objects is the skeleton of the background that contains the objects. The exo-skeleton produces a partition of the image into regions each of which contains one object. The actual skeletonization is performed without the preservation of end pixels and with the border set to "0." The procedure is described below and the result is illustrated in Figure 11. i) ii) iii) Segment image to produce binary image Compute complement of binary image Compute skeleton using eq. i+ii with border set to "0"

Figure 11: Exo-skeleton.

Touching objects - Segmentation procedures frequently have difficulty separating slightly touching, yet distinct, objects. The following procedure provides a mechanism to separate these objects and makes minimal use of "magic numbers." The exo-skeleton produces a partition of the image into regions each of which contains one object. The actual skeletonization is performed without the preservation of end pixels and with the border set to "0." The procedure is illustrated in Figure 12. i) ii) iii) iv) v) Segment image to produce binary image Compute a "small number" of erosions with B = N4 Compute exo-skeleton of eroded result Complement exo-skeleton result Compute AND of original binary image and the complemented exo-skeleton

a) Eroded and exo-skeleton images Figure 12: Separation of touching objects.

b) Objects separated (detail)

The eroded binary image is illustrated in gray in Figure 12a and the exo-skeleton image is shown in black in that same illustration. An enlarged section of the final result is shown in Figure 12b and the separation is easily seen. This procedure involves choosing a small, minimum number of erosions but the number is not critical as long as it initiates a coarse separation of the desired objects. The actual separation is performed by the exo-skeleton which, itself, is free of "magic numbers." If the exo-skeleton is 8-connected than the background separating the objects will be 8-connected. The objects, themselves, will be disconnected according to the 4-connected criterion.