Proceedings of the 2018 IEEE

International Conference on Robotics and Biomimetics

December 12-15, 2018, Kuala Lumpur, Malaysia

Image denoising with Weighted

ORientation-Matched filters(WORM)

Debajyoti Karmaker∗ , Ingo Schiffner‡ , Michael Wilson∗ and Mandyam V. Srinivasan∗†

∗
The Queensland Brain Institute, University of Queensland,St Lucia, QLD, Australia
† The School of Information Technology and Electrical Engineering, University of Queensland, Australia
‡ School of Natural Sciences, Bangor University, Gwynedd, Wales, UK

Email: {d.karmaker,m.wilson,m.srinivasan}@uq.edu.au and [email protected]

Abstract—Real world signals commonly exhibit slow vari- II. R ELATED W ORKS
ations or oscillations, punctuated with rapid transients. For
example, images typically have smooth regions interrupted by There is substantial amount of work available on various
edges or abrupt changes in contrast. These abrupt changes are image denoising techniques. Denoising techniques can be
often the most interesting parts of the data perceptually, as well broadly categorized into two approaches: (A) spatial filtering
as in terms of the information that they provide. Some of the methods and (B) transform domain filtering methods.
high frequency content represents the important abrupt changes
in image intensity that are associated with real edges of objects
in the image. However, some of the high-frequency content also A. Spatial Filtering
comprises the noise that is present in the image. We wish to
retain this edge information, while removing the noise. In this Spatial filters are widely used till days before edge detec-
paper, we present a dynamic filtering process where the dynamic tion algorithms are applied. These methods have less compu-
mask is oriented to match the local gradients and its weights are tational complexity which is most suitable as a pre-processing
proportional to the magnitude of the local gradients. technique. These methods remove noise by convolving the
Keywords—Image denoising, Non-local filters, Spatial filtering, original image with a mask (sliding window). Spatial filters
Noise removal, Edge detection. can be further sub divided into two categories: (1) Linear filters
and (2) non-linear filters.

I. I NTRODUCTION 1) Linear Filters: One form of linear filtering is the average

filter or mean filter. A mean filter acts on an image by
The use of Unmanned Aerial Vehicles (UAVs) is becom- smoothing it; that is, it reduces the intensity variation between
ing more and more pervasive every year, rendering current adjacent pixels. The mean filter is nothing but a simple sliding
ground-based control systems inadequate in avoiding mid-air window spatial filter that replaces the center value in the
collisions. Most airborne creatures, such as birds or flying window with the average of all the neighbouring pixel values
insects, are extremely adept at avoiding collisions with their including itself.
conspecifics or other moving objects in their environments.
How they achieve this is largely unknown. In the past decade, Arguably the most widely used linear filter applied before
researchers have started to gain insights into how birds and edge detection is Gaussian smoothing [6], a 2D convolution
insects control their flight speed [1], avoid obstacles [2] and operator - used to smooth images and reduce the noise con-
perform smooth landings [3]. This has drawn considerable tained in an image. The technique is very much similar to
attention from roboticists, who are challenged with similar the mean filter, but it uses a special kernel that represents the
problems in the design of guidance systems for unmanned shape of a Gaussian (bell-shaped). Gaussian smoothing makes
aerial vehicles. However, studying bird flight is not simple: one use of the 2D distribution as a point-spread function, and this
first has to be able to collect accurate data from flying animals, is accomplished by convolution.
with sufficient temporal and spatial resolution. This is best Another variation of linear filters are the adaptive filters.
achieved using high-speed cameras and stereo or multi-camera Adaptive filters are adept at denoising images with abrupt
setups. While high-speed cameras are now becoming more and changes in intensity. This kind of filter can handle irregularity
more affordable, the need for efficient image restoration meth- in a signal with little a prior knowledge about the signal to be
ods has grown with the use of high-speed video cameras as the processed [7]. The Least Mean Square (LMS) adaptive filter
exposure time for high-speed cameras is limited by the frame works well for images corrupted with salt and pepper type
rate, which limits the SNR. Thus, pre-processing the images noise and it does a better denoising job compared to the mean
to get denoised version of the images is often the first step filter.
conducted before the images data is analysed. Natural images
mostly contain additive random noise which can be modelled The most important advantages of these above-mentioned
as a Gaussian. Speckle noise [4] is observed in ultrasound filters are their high speed and their limited computational
images whereas Rician noise [5] affects MRI images. The complexity. An accompanying disadvantage is that linear filters
scope of the paper is to focus on a smoothing technique to tend to blur sharp edges, destroy lines and other fine image
denoise the images acquired by high speed cameras to help details, and perform poorly in the presence of signal-dependent
detecting edges, often filmed in poor conditions. noise.

978-1-7281-0377-8/18/$31.00 © 2018 IEEE 1022

1
 " 

) " 

*

& 

 

& 

 

)



*

) *

 
 
 


 & 

 & 
*
)  
 





     
 
)




 
)  

 
*  

 
%

 
 

 



$  
   

  

 
 # 

  ! 

+

#


"






# "


 
 
' (

 

 
' (    
 

 "


+

#


!


# 
 
'( 





 



  #   
  $  
   

 


 


Fig. 1: The figure contains an system overview (top diagram) and detail (lower diagram)

2) Non-linear Filters: Largely linear filters eliminate noise The principle idea behind wavelet transform is to break up
to a reasonable level but they achieve this at the expense of a signal into different frequency components. Next, each
blurring images. A variety of nonlinear median- type filters section is analysed with a resolution that matches its scale.
such as weighted median [8], rank conditioned rank selection The effectiveness of this method lies in its capability of
[9], and relaxed median [10] have been proposed to handle the representing the signal in few transform coefficient values.
limitations of linear filters. Wavelets provide some advantages over Fourier transforms.
For example, they do a good job in approximating signals
The median filter also engages the mask approach similar with sharp spikes or signals with discontinuities. The wavelet
to the mean filter. The center pixel under the mask is replaced equation produces different wavelet families like Daubechies,
with the median value of the pixel values that belong inside the Haar, Coiflets, etc. [12]. But these methods have high run
mask. As the median value is not significantly influenced by time complexity and also depend on the cut-off frequency and
an outlier in a neighbourhood, the median filter is more robust the filter function behaviour. Furthermore, they may produce
compared to the mean filter. For the same reason, the median artificial frequencies in the processed image.
filter also performs significantly better while preserving the
sharp edges.
III. I MAGE D ENOISING USING W EIGHTED
The spatial median filter is another variation of non- OR IENTATION -M ATCHED FILTERS (WORM)
linear filters. In this filter, the median value is computed by
computing the spatial depth between a point and a set of points Figure 1 describes the pyramidal system architecture of the
in a neighbourhood. The central pixel inside a mask is judged proposed image denoising technique. Given an image S, the
to be corrupted or not based on these spatial depth values. image is first filtered with a low-pass filter, which removes the
The central pixel will remain unchanged if the pixel is not high spatial frequencies whilst preserving the low-frequency
corrupted. components of the image. Concurrently, an edge mask W (see
Figure 1, Operation EM) is generated from the input image
There are various implementations of weighted median S. To create the edge mask, we apply a low pass filter and a
filters (WMF) available. The weighted median filters give more high pass filter to the rows of the image S. The low pass filter
weight to some values within the window. The centre weighted extracts the low-frequency components (Horizontal Approxi-
median filter is an extension of the weighted median filter mation) and the high pass filter extracts the high-frequency
where weight is given to the central value of a window and components (Horizontal Detail). We then apply a high pass
is thus easier to design and implement than other versions of filter to the columns of the Horizontal Approximation, which
weighted median filters. yields a horizontally smoothed Vertical Detail (VD) of the
original Horizontal Approximation. Similarly, by applying a
B. Transform Domain Filtering low pass filter to the columns of the Horizontal Detail, we
obtain a vertically smoothed Horizontal Detail (HD). In effect,
Amongst various methods denoising under transform do- this is a rapid way of low-and high-pass filtering the image
main filtering, the most popular is wavelet transform [11]. S in two dimensions that speeds up computation by taking

1023
2
 2
 2  
WL−1 (2x + m, 2y + n) + WL−1 (2x − m, 2y − n)
PL (x, y) = LP F OL−1 (2x + m, 2y + n) (1)
m=−2 n=−2
2

2
  
WL−1 (2x + m) + WL−1 (2x − m)
PL (x) = LP F OL−1 (2x + m) (2)
m=−2
2

advantage of the separability of the kernels of the low and

high-pass filters. The results are shown in Figure 2.

Fig. 4: Edge mask

(a) Horizontal details (b) Vertical details

the low-pass filtered output (LPFO) and downsampled by a
Fig. 2: Horizontal and Vertical details factor of two to produce the input for the next pyramid level.
Unlike normal convolution - where the spatial filter is constant
irrespective of image location, here, the filter is dynamically
Next, the Laplacians Δ2 V D and Δ2 HD are computed generated from the edge mask corresponding to each image
from the VD and HD images. We then construct Vertical Detail location. At each image location the filter is oriented to match
Slopes (VDS) by finding the gradients of the VD image at the local image gradient, and its weight(gain) is proportional
the zero crossings of Δ2 V D along the y axis, Similarly, we to the magnitude of the local gradient. It is a Weighted,
build the Horizontal Detail Slopes (HDS) by scanning the zero ORientation-Matched filter, which we acronymize as WORM.
crossings of the HD image along the x axis. Figure 3, shows the The edge-masked image is then down-sampled by a factor of
detected zero crossings, labeled according to the magnitudes two in the horizontal and vertical dimensions, to yield an edge-
of the slopes. The slopes are represented in a heat map where masked image that contains 1/4 as many pixels as the original
red represents the strongest slopes. image.
The processes of dynamic convolution and downsampling
is represented by expressions 1. where PL refers to the input
image for the Lth level of the pyramid. The 2x and 2y terms
achieve the desired downsampling. The double summations
refer to the 5*5 mask that is applied across the rows and
columns. For simplicity, we can consider the one-dimensional
version of the same expression shown in equation 2.
When x = 2, the expression 2 can be written as

WL−1 (2) + WL−1 (6)

(a) Horizontal zero crossings (b) Vertical zero crossings PL (2) = LP F OL−1 (2)
2
WL−1 (3) + WL−1 (5)
Fig. 3: Horizontal and Vertical zero crossings + LP F OL−1 (3)
2
WL−1 (4) + WL−1 (4)
+ LP F OL−1 (4)
Finally, the edge mask (W) is created by taking the magni- 2
tudes of the slope values from HDS and VDS using equation WL−1 (5) + WL−1 (3)
+ LP F OL−1 (5)
Wij = HDSij 2 + V DS 2 which represents the magnitude 2
ij
WL−1 (6) + WL−1 (2)
of the local two-dimensional gradient of the image. Figure 4 + LP F OL−1 (6)
shows the weight values of the mask, again plotted as a heat 2
map.
In effect, we are reducing the resolution to generate the next
Once the edge mask (W) is generated, it is convolved with level of the image. However, simple downsampling (discarding

1024
3
alternate rows and columns) can create problems, such as
aliasing, and loss of useful information. Smoothing (low-
pass-filtering) the image prior to downsampling would solve
the aliasing problem. This smoothing process is illustrated in
Figure 3 for one dimension. The intensity value at pixel 2 in
the level L+1 is obtained by calculating a weighted sum of
(a) (b)
the intensity values at pixels 2,3,4,5 and 6 from the low pass
filtered output (LPFO) in level L, as shown in Figure 5. (This
process obviously discards the border pixels in the level L+1). Fig. 7: a) Original image b) Salt & pepper noise added version

    
 



 

      
 


 
  

  

 

  



         
 


Fig. 5: Pixel intensity contribution from the previous level

 
  

  

 

A low pass filter of this kind acts to (a) reduce the

noise and (b) minimize the effects of aliasing that could
arise from the down-sampling process. A unique feature of   


this filter, however, is that it varies from location to location
depending upon the local structure of the image, and allows the
filter to selectively enhance local high contrast edges without
diminishing their contrast in the way a standard low-pass filter  
  

  

 
does. This is achieved by using the edge mask to control the
weights of the low-pass filter, as shown in Figure 6. The mask
weights on either side of the center of the kernel of the low
pass filter are made symmetrical,   


2  as indicated by the term
W (2x+m)+ W (2x−m)
m=−2 2 in equation 2, to ameliorate
the effects of noise.
 
  

  

 
       
 


Fig. 8: High pass filter output shows that the noise reduces as
the level increases


























version of the original image, using the equation described














       
 
 above, for processing at the next level. Additionally, a high
pass filter output is also shown at each level in Figure 8 .
Fig. 6: Dynamic weights assignment This high pass filter is required for two reasons. Firstly, the
noise reduces significantly as the level increases - as can be
seen from the high pass filter outputs (figure 8). Going too far
To illustrate the entire process described above, we take an down the pyramid level will eventually get rid of some true
image (Figure 7a) and add some salt and pepper noise using the edges. For this reason, visual examination of the high pass
imnoise function from Matlab. The parameter noise density(d) filter output - allows us to determine the coarsest level of the
of imnoise was set to 0.10. This affected approximately 10% pyramid that we need to construct. For the particular example
pixels of the original image of dimension 256x128 as shown shown in Figure 8, level 3 is visually chosen to be the coarsest
in figure 7b. The noisy image is now passed through a low level, as some of the true edges have already disappeared at
pass filter and the edge mask (W) is generated from the level 4 and the input image (unfiltered) at level 3 is nominated
image as shown in Figure 8. The symbol represents the as the denoised version of the original noisy image. Secondly,
dynamic convolution and down sampling process described we can apply a threshold on the final high pass filtered output
above, which produces an edge-enhanced and down-sampled to further reduce the noise and then by combining outputs of

1025
4
the thresholded high pass and low pass filters (at the highest Original Signal
level of the pyramid), we produce a denoised version of the 300
200
original noisy image.
100
0
20 40 60 80 100 120
Our Method SNR: 16.85 dB
300
IV. E XPERIMENTAL R ESULTS
200
100
To test our smoothing technique, we take one row of an 0
image as shown in Figure 9a and 9c. We add salt and pepper 20 40 60 80 100 120
noise by using the imnoise function of Matlab to the image 300
Wavelet SNR: 16.85 dB
as shown in figure 9b with the parameter noise density(d) 200
set to 0.10 and pick the same row shown in Figure 9d. 100
We compare our method of smoothing and denoising with 0
other popular image smoothing and denoising methods, as 20 40 60 80 100 120
Median SNR: 15.03 dB
shown in Figure 10. The signal to noise ratio is computed 300
by SN RDB = −20∗log10 norm(abs(original−new))
norm(original) . 200
100
0
20 40 60 80 100 120
Gaussian SNR: 13.24 dB
300
200
100
0
(a) (b) 20 40 60 80 100 120
300 Savitzky Golay SNR: 12.88 dB
300
200 200
Intensity

100
100
0
0 20 40 60 80 100 120
20 40 60 80 100 120
Row index Average SNR: 12.09 dB
300
(c) 200
300
100
200 0
Intensity

20 40 60 80 100 120
100
Moving Average SNR: 10.04 dB
300
0
20 40 60 80 100 120 200
Row index
100
(d) 0
20 40 60 80 100 120

Fig. 9: a) Single row from the original image shown in red b)

Fig. 10: Comparison of different smoothing techniques where
Same row selected from the Salt & pepper noise added version
y axis represents intensity value and x axis represents row
of the same image c) Single row from the original image where
index
the y axis represents intensity value and the x axis represents
row index d) Single row from the noisy image where the y
axis represents intensity value and the x axis represents row
index
V. C ONCLUSION
The image denoising technique described here is not lim-
In order to test our denoising method on images captured ited to help with edge detection. It can be applied to a variety
by high speed cameras, we apply our method to one of our of problems in computer vision and medical image analysis.
previous works on object detection [13] using interframe dif- The main advantage of our denoising method is that it makes
ferencing. Figure 11 illustrates the benefits of object detection use of the high speed and the limited computational complexity
[13] when it is performed on images that have initially been of the WORM filters to reduce the noise. Most importantly the
denoised with our denoising method. Here we also show, how use of the dynamic mask overcomes the problem of blurring
our denoising method applied to the same dataset [13] affects the edges.
detection accuracy. To test robustness of our denoising method,
we added two different types of noise (i) salt and pepper noise
(SPN)and (ii) Gaussian white noise (GWN) to the images by ACKNOWLEDGMENT
using the imnoise function of Matlab, with various settings
for parameter noise density(d) for SPN and the parameter This work was co-funded by the Australian Research
mean (m) for GWN. Table I evaluates the performance of our Council and by Boeing Research & Technology Australia
denoising method by comparing the detection accuracies with through a Linkage Project Grant (LP 130100483), and an ARC
and without denoising, for a range of noise densities. Distinguished Outstanding Researcher Award (DP140100914).

1026
5
 TABLE I: Detection accuracy with various types and densities of noise added to the images
Noise density (m/d) 0.1 0.2 0.3 0.4 0.5
Noise type SPN GWN SPN GWN SPN GWN SPN GWN SPN GWN
Without Denoising 90% 92% 85% 86% 73% 69% 54% 48% 35% 32%
With Denoising 93% 94% 91% 89% 79% 75% 65% 62% 44% 37%

Fig. 11: (a) Logic map for k=20 on raw frames 79 & 78 (b) Logic map for k=20 on denoised frames 79 & 78 (c) Centroids of
different logic maps on raw frames 79 & 78 (d) Centroids of different logic maps on denoised frames 79 & 78

R EFERENCES Restoration,” IEEE Transactions on Image Processing, vol. 3, no. 2,

pp. 192–206, mar 1994.
[1] I. Schiffner and M. V. Srinivasan, “Direct Evidence for Vision-based
Control of Flight Speed in Budgerigars,” Scientific Reports, vol. 5, p. [10] A. B. Hamza, P. L. Luque-Escamilla, J. Martı́nez-Aroza, and R. Román-
10992, 2015. Roldán, “Removing noise and preserving details with relaxed median
filters,” Journal of Mathematical Imaging and Vision, vol. 11, no. 2,
[2] H. D. Vo, I. Schiffner, and M. V. Srinivasan, “Anticipatory manoeuvres pp. 161–177, 1999.
in bird flight,” Sci Rep, vol. 6, p. 27591, Jun 2016.
[11] L. A. Barford, R. S. Fazzio, and D. R. Smith, “An introduction to
[3] M. V. Srinivasan, “Visual control of navigation in insects and its wavelets,” Hewlett-Packard Labs, Bristol, UK, Tech. Rep. HPL-92-124,
relevance for robotics,” Current Opinion in Neurobiology, vol. 21, no. 4, vol. 2, no. 2, pp. 1–29, 1992.
pp. 535–543, 2011.
[12] “Wavelet Toolbox Reference.” [Online]. Available:
[4] H. Guo, J. E. Odegard, M. Lang, R. A. Gopinath, I. W. Selesnick, https://au.mathworks.com/products/wavelet.html
and C. S. Burrus, “Wavelet based speckle reduction with application
to SAR based ATD/R,” in Proceedings - International Conference on [13] D. Karmaker, I. Schiffner, M. Wilson, and V. Mandyam, “The bird
Image Processing, ICIP, vol. 1. IEEE Comput. Soc. Press, 1994, pp. gets caught by the WORM : tracking multiple deformable objects in
75–79. noisy environments using Weight ORdered logic Maps,” in International
Symposium on Visual Computing (ISVC). Springer-Verlag, 2018 in
[5] R. D. Nowak, “Wavelet-based Rician noise removal for magnetic press.
resonance imaging,” IEEE Transactions on Image Processing, vol. 8,
no. 10, pp. 1408–1419, 1999.
[6] M. Lindenbaum, M. Fischer, and A. Bruckstein, “On Gabor’s contri-
bution to image enhancement,” Pattern Recognition, vol. 27, no. 1, pp.
1–8, jan 1994.
[7] J. N. Lin, X. Nie, and R. Unbehauen, “Two-Dimensional LMS Adaptive
Filter Incorporating a Local-Mean Estimator for Image Processing,”
IEEE Transactions on Circuits and Systems II: Analog and Digital
Signal Processing, vol. 40, no. 7, pp. 417–428, jul 1993.
[8] R. Yang, L. Yin, M. Gabbouj, J. Astola, and Y. Neuvo, “Optimal
weighted median filters under structural constraints,” 1993 IEEE In-
ternational Symposium on Circuits and Systems, vol. 942, no. 2, pp.
942–945.
[9] K. E. Barner, “Rank Conditioned Rank Selection Filters for Signal

1027
6