Image Processing
Image Processing
Neighbourhood Processing
3.1 Introduction
We have seen in chapter 2 that an image can be modied by applying a particular function to each
pixel value. Neighbourhood processing may be considered as an extension of this, where a function
is applied to a neighbourhood of each pixel.
The idea is to move a mask: a rectangle (usually with sides of odd length) or other shape over
the given image. As we do this, we create a new image whose pixels have grey values calculated
from the grey values under the mask, as shown in gure 3.1. The combination of mask and function
Mask
Pixel at position
Pixel at position
is called a lter. If the function by which the new grey value is calculated is a linear function of all
the grey values in the mask, then the lter is called a linear lter.
A linear lter can be implemented by multiplying all elements in the mask by corresponding
elements in the neighbourhood, and adding up all these products. Suppose we have a mask
as illustrated in gure 3.1. Suppose that the mask values are given by:
57
58 CHAPTER 3. NEIGHBOURHOOD PROCESSING
! "#$% &'! (&
)*
+#! % &,! &#
&-
.&' ! /&,$% .&'&'! .&'(&
A diagram illustrating the process for performing spatial ltering is given in gure 3.2.
Spatial ltering thus requires three steps:
1. position the mask over the current pixel,
2. form all products of lter elements with the corresponding elements of the neighbourhood,
3. add up all the products.
This must be repeated for every pixel in the image.
Allied to spatial ltering is spatial convolution. The method for performing a convolution is the
87
adding. Using the
$
same as that for ltering, except that the lter must be rotated by
and
before multiplying and
notation as before, the output of a convolution with a
mask for a single pixel is
0 0 $ 5
65
132 ! 4 2 !
3.1. INTRODUCTION 59
Product of neighbourhood
Mask
with mask
Pixel
Neighbourhood
Input image
Output pixel
Output image
Figure 3.2: Performing spatial ltering
60 CHAPTER 3. NEIGHBOURHOOD PROCESSING
Note the negative signs on the indices of . The same result can be achieved with
$
0 0 $
5
5
132 !
4 2 !
7
Here we have rotated the image pixels by ; this does not of course aect the result. The
importance of convolution will become apparent when we investigate the Fourier transform, and
the convolution theorem. Note also that in practice, most lter masks are rotationally symmetric,
so that spatial ltering and spatial convolution will produce the same output.
An example: One important linear lter is to use a mask and take the average of all nine
values within the mask. This value becomes the grey value of the corresponding pixel in the new
image. This operation may be described as follows:
where is grey value of the current pixel in the original image, and the average is the grey value of
the corresponding pixel in the new image.
To apply this to an image, consider the image obtained by:
>> x=uint8(10*magic(5))
x =
our working will thus consist only of nine values. We shall see later how to obtain 25 values in the
output.
Consider the top left neighbourhood of our image x:
>> mean2(x(1:3,1:3))
ans =
111.1111
which can be rounded to 111. Now we can move to the second neighbourhood:
>> mean2(x(1:3,2:4))
ans =
108.8889
and this can be rounded either down to 108, or to the nearest integer 109. If we continue in this
manner, the following output is obtained:
3.2 Notation
It is convenient to describe a linear lter simply in terms of the coecients of all the grey values of
pixels within the mask. This can be written as a matrix.
The averaging lter above, for example, could have its output written as
62 CHAPTER 3. NEIGHBOURHOOD PROCESSING
Pad with zeros. We assume that all necessary values outside the image are zero. This gives us
all values to work with, and will return an output image of the same size as the original, but
may have the eect of introducing unwanted artifacts (for example, edges) around the image.
filter2(filter,image,shape)
and the result is a matrix of data type double. The parameter shape is optional, it describes the
method for dealing with the edges:
>> a=ones(3,3)/9
a =
>> filter2(a,x,same)
ans =
filter2(filter,image,valid) applies the mask only to inside pixels. The result will
always be smaller than the original:
>> filter2(a,x,valid)
ans =
The result of same above may also be obtained by padding with zeros and using valid:
>> x2=zeros(7,7);
>> x2(2:6,2:6)=x
x2 =
0 0 0 0 0 0 0
0 170 240 10 80 150 0
0 230 50 70 140 160 0
0 40 60 130 200 220 0
0 100 120 190 210 30 0
0 110 180 250 20 90 0
0 0 0 0 0 0 0
>> filter2(a,x2,valid)
64 CHAPTER 3. NEIGHBOURHOOD PROCESSING
>> filter2(a,x,full)
ans =
The shape parameter, being optional, can be omitted; in which case the default value is same.
There is no single best approach; the method must be dictated by the problem at hand; by
the lter being used, and by the result required.
We can create our lters by hand, or by using the fspecial function; this has many options
which makes for easy creation of many dierent lters. We shall use the average option, which
produces averaging lters of given size; thus
>> fspecial(average,[5,7])
>> c=imread(cameraman.tif);
>> f1=fspecial(average);
>> cf1=filter2(f1,c);
We now have a matrix of data type double. To display this, we can do any of the following:
use mat2gray to scale the result for display. We shall discuss the use of this function later.
>> figure,imshow(c),figure,imshow(cf1/255)
Notice how the zero padding used at the edges has resulted in a dark border appearing around
the image. This is especially noticeable when a large lter is being used. If this is an unwanted
artifact of the ltering; if for example it changes the average brightness of the image, then it may
66 CHAPTER 3. NEIGHBOURHOOD PROCESSING
Separable lters
Some lters can be implemented by the successive application of two simpler lters. For example,
since
the
averaging lter can be implemented by rst applying a averaging lter, and then
applying a averaging lter to the result. The averaging lter is thus separable into two
smaller lters. Separability can result in great time savings. Suppose an lter is separable
into two lters of size
and
and . The application of an lter requires
additions for each pixel in the image. But the application of an
multiplications,
lter only
requires multiplications and additions. Since this must be done twice, the total number of
multiplications and additions are and respectively. If is large the savings in eciency
can be dramatic.
All averaging lters are separable; another separable lter is the laplacian
high pass lter if it passes over the high frequency components, and reduces or eliminates low
frequency components,
low pass lter if it passes over the low frequency components, and reduces or eliminates high
frequency components,
3.4. FREQUENCIES; LOW AND HIGH PASS FILTERS 67
averaging lter is low pass lter, as it tends to blur edges. The lter
The resulting values are close to zero, which is the expected result of applying a high pass lter to
a low frequency component. We shall see how to deal with negative values below.
High pass lters are of particular value in edge detection and edge enhancement (of which we
shall see more in chapter 8). But we can provide a sneak preview, using the cameraman image.
>> f=fspecial(laplacian)
f =
>> cf=filter2(f,c);
>> imshow(cf/100)
>> f1=fspecial(log)
f1 =
>> cf1=filter2(f1,c);
>> figure,imshow(cf1/100)
The images are shown in gure 3.5. Image (a) is the result of the Laplacian lter; image (b) shows
the result of the Laplacian of Gaussian (log) lter.
In each case, the sum of all the lter elements is zero.
68 CHAPTER 3. NEIGHBOURHOOD PROCESSING
We have seen that for image display, we would like the grey values of the pixels to lie between 0
and 255. However, the result of applying a linear lter may be values which lie outside this range.
We may consider ways of dealing with values outside of this displayable range.
Make negative values positive. This will certainly deal with negative values, but not with val-
ues greater than 255. Hence, this can only be used in specic circumstances; for example,
when there are only a few negative values, and when these values are themselves close to zero.
Clip values. We apply the following thresholding type operation to the grey values produced by
the lter to obtain a displayable value :
if
if
if
This will produce an image with all pixel values in the required range, but is not suitable if
there are many grey values outside the 0255 range; in particular, if the grey values are equally
spread over a larger range. In such a case this operation will tend to destroy the results of the
lter.
highest value is
Scaling transformation. Suppose the lowest grey value produced by the lter if
. We can transform all values in the range
and the
to the range 0255 by
255
we can write the equation of the line as
and applying this transformation to all grey levels produced by the lter will result (after
any necessary rounding) in an image which can be displayed.
As an example, lets apply the high pass lter given in section 3.4 to the cameraman image:
mat2gray function automatically scales the matrix elements to displayable values; for any matrix
, it applies a linear transformation to to its elements, with the lowest value mapping to 0.0, and
the highest value mapping to 1.0. This means the output of mat2gray is always of type double.
The function also requires that the input type is double.
>> figure,imshow(mat2gray(cf2));
To do this by hand, so to speak, applying the linear transformation above, we can use:
>> maxcf2=max(cf2(:));
>> mincf2=min(cf2(:));
>> cf2g=(cf2-mincf2)/(maxcf2-mncf2);
The result will be a matrix of type double, with entries in the range . This can be be viewed
with imshow. We can make it a uint8 image by multiplying by 255 rst. The result can be seen in
gure 3.6.
We can generally obtain a better result by dividing the result of the ltering by a constant before
displaying it:
>> figure,imshow(cf2/60)
Figure 3.6: Using a high pass lter and displaying the result
Unsharp masking
The idea of unsharp masking is to subtract a scaled unsharp version of the image from the original.
In practice, we can achieve this aect by subtracting a scaled blurred image from the original. The
schema for unsharp masking is shown in gure 3.7.
Scale for
Original Subtract
display
Suppose an image x is of type uint8. The unsharp masking can be applied by the following
sequence of commands:
>> f=fspecial(average);
>> xf=filter2(f,x);
>> xu=double(x)-xf/1.5
>> imshow(xu/70)
3.5. EDGE SHARPENING 71
The last command scales the result so that imshow displays an appropriate image; the value may
need to be adjusted according to the input image. Suppose that x is the image shown in gure 3.8(a),
then the result of unsharp masking is given in gure 3.8(b). The result appears to be a better image
than the original; the edges are crisper and more clearly dened.
To see why this works, we may consider the function of grey values as we travel across an edge,
as shown in gure 3.9.
As a scaled blur is subtracted from the original, the result is that the edge is enhanced, as shown
in graph (c) of gure 3.9.
We can in fact perform the ltering and subtracting operation in one command, using the
linearity of the lter, and that the lter
)
where is a constant chosen to provide the best result. Alternatively, the unsharp masking lter
may be dened as
so that we are in eect subtracting a blur from a scaled version of the original; the scaling factor
may also be split between the identity and blurring lters.
72 CHAPTER 3. NEIGHBOURHOOD PROCESSING
The unsharp option of fspecial produces such lters; the lter created has the form
where is an optional parameter which defaults to 0.2. If the lter is
>> p=imread(pelicans.tif);
>> u=fspecial(unsharp,0.5);
>> pu=filter2(u,p);
>> imshow(p),figure,imshow(pu/255)
Figure 3.10(b), appears much sharper and cleaner than the original. Notice in particular the rocks
and trees in the background, and the ripples on the water.
Although we have used averaging lters above, we can in fact use any low pass lter for unsharp
masking.
high boost
original
low pass
74 CHAPTER 3. NEIGHBOURHOOD PROCESSING
where is an amplication factor. If , then the high boost lter becomes an ordinary high
pass lter. If we take as the low pass lter the averaging lter, then a high boost lter will
have the form
where . If we put
, we obtain a ltering very similar to the unsharp lter above, except
will produce an image similar to that in gure 3.8. The value 80 was obtained by trial and error to
produce an image with similar intensity to the original.
We can also write the high boost formula above as
high boost
original
low pass
original
original
high pass
original
high pass
Best results for high boost ltering are obtained if we multiply the equation by a factor so that
or
%
If we take
, the formula becomes
original low pass
original
low pass
If we take
% we obtain
hb1 =
>> hb2=1.25*id-0.25*f
hb2 =
If each of the lters hb1 and hb2 are applied to an image with filter2, the result will have enhanced
edges. The images in gure 3.11 show these results; gure 3.11(a) was obtained with
>> x1=filter2(hb1,x);
>> imshow(x1/255)
(a) High boost ltering with hb1 (b) High boost ltering with hb2
Of the two lters, hb1 appears to produce the best result; hb2 produces an image not very much
crisper than the original.
76 CHAPTER 3. NEIGHBOURHOOD PROCESSING
neighbourhood:
>> cmax=nlfilter(c,[3,3],max(x(:)));
The nlfilter function requires three arguments: the image matrix, the size of the lter, and the
function to be applied. The function must be a matrix function which returns a scalar value. The
result of this operation is shown in gure 3.12(a).
A corresponding implementation of the minimum lter is:
>> cmin=nlfilter(c,[3,3],min(x(:)));
Note that in each case the image has lost some sharpness, and has been brightened by the
maximum lter, and darkened by the minimum lter. The nlfilter function is very slow; in
3.6. NON-LINEAR FILTERS 77
general there is little call for non-linear lters except for a few which are dened by their own
commands. We shall investigate these in later chapters.
Non-linear ltering using nlfilter can be very slow. A faster alternative is to use the colfilt
function, which rearranges the image into columns rst. For example, to apply the maximum lter
to the cameraman image, we can use
>> cmax=colfilt(c,[3,3],sliding,@max);
The parameter sliding indicates that overlapping neighbourhoods are being used (which of course
is the case with ltering). This particular operation is almost instantaneous, as compared with the
use of nlfilter.
To implement the maximum and minimum lters as rank-order lters, we may use the Matlab
function ordfilt2. This requires three inputs: the image, the index value of the ordered results to
choose as output, and the denition of the mask. So to apply the maximum lter on a mask,
we use
>> cmax=ordfilt2(c,9,ones(3,3));
>> cmin=ordfilt2(c,1,ones(3,3));
A very important rank-order lter is the median lter, which takes the central value of the ordered
list. We could apply the median lter with
>> cmed=ordfilt2(c,5,ones(3,3));
However, the median lter has its own command, medfilt2, which we discuss in more detail in
chapter 5.
Other non-linear lters are the geometric mean lter, which is dened as
where is the lter mask, and its size; and the alpha-trimmed mean lter, which rst orders
the values under the mask, trims o elements at either end of the ordered list, and takes the mean
of the remainder. So, for example, if we have a mask, and we order the elements as
and trim o two elements at either end, the result of the lter will be
"
Both of these lters have uses for image restoration; again see chapters 5 and 6.
Exercises
1. The array below represents a small greyscale image. Compute the images that result when
the image is convolved with each of the masks (a) to (h) shown. At the edge of the image use
a restricted mask. (In other words, pad the image with zeroes.)
78 CHAPTER 3. NEIGHBOURHOOD PROCESSING
20 20 20 10 10 10 10 10 10
20 20 20 20 20 20 20 20 10
20 20 20 10 10 10 10 20 10
20 20 10 10 10 10 10 20 10
20 10 10 10 10 10 10 20 10
10 10 10 10 20 10 10 20 10
10 10 10 10 10 10 10 10 10
20 10 20 20 10 10 10 20 20
20 10 10 20 10 10 20 10 20
-1 -1 0 0 -1 -1 -1 -1 -1 -1 2 -1
(a) -1 0 1 (b) 1 0 -1 (c) 2 2 2 (d) -1 2 -1
0 1 1 1 1 0 -1 -1 -1 -1 2 -1
-1 -1 -1 1 1 1 -1 0 1 0 -1 0
(e) -1 8 -1 (f) 1 1 1 (g) -1 0 1 (h) -1 4 -1
-1 -1 -1 1 1 1 -1 0 1 0 -1 0
2. Check your answers to the previous question with Matlab.
3. Describe what each of the masks in the previous question might be used for. If you cant do
this, wait until question 5 below.
4. Devise a
)
Apply all the lters listed in question 1 to this image. Can you now see what each lter does?
6. Apply larger and larger averaging lters to this image. What is the smallest sized lter for
which the whiskers cannot be seen?
7. Read through the help page of the fspecial function, and apply some of the other lters to
the cameraman image, and to the mandrill image.
8. Apply dierent laplacian lters to the mandrill and cameraman images. Which produces the
best edge image?
9. Is the median lter separable? That is, can this lter be implemented by a
lter
followed by a lter?
10. Repeat the above question for the maximum and minimum lters.
11. Apply a
)
$
5
3.6. NON-LINEAR FILTERS 79
12. Matlab also has an imfilter function, which if x is an image matrix (of any type), and f is
a lter, has the syntax
imfilter(x,f);
It diers from filter2 in the dierent parameters it takes (read its help le), and in that the
output is always of the same class as the original image.
(a) Use imfilter on the mandrill image with the lters listed in question 1.
(b) Apply dierent sized averaging lters to the mandrill image using imfilter.
(c) Apply dierent laplacian lters to the mandrill image using imfilter. Compare the
results with those obtained with filter2. Which do you think gives the best results?
13. Display the dierence between the cmax and cmin images obtained in section 3.6. You can do
this with
>> imshow(imsubtract(cmax,cmin))
What are you seeing here? Can you account for the output of these commands?
14. Using the tic and toc timer function, compare the use of nlfilter and colfilt functions.
15. Use colfilt to implement the geometric mean and alpha-trimmed mean lters.
16. Can unsharp masking be used to reverse the eects of blurring? Apply an unsharp masking
lter after a averaging lter, and describe the result.
80 CHAPTER 3. NEIGHBOURHOOD PROCESSING