Noise Reduction and Object Enhancement in Passive Millimeter Wave Concealed

Weapon Detection
Seungsin Lee†, Raghuveer Rao† and Mohamed-Adel Slamani‡
†
Rochester Institute of Technology, Rochester, NY 14623.
‡
ITT Industries, Rome NY 13441.
dimension and multiscale edge representation obtained by
ABSTRACT undecimated wavelet transforms in the spatial dimension.
This paper is organized as follows. Temporal denoising is
Passive MM wave offers the advantage of penetration for described in Section 2. Section 3 briefly reviews multiscale
concealed weapon detection. Its ability to penetrate through edge representation of images. Wavelet transform based
fog, smoke, clothing etc. makes it an attractive candidate to denoising and object enhancement are described in Section
look for weapons concealed underneath a person’s 4. Experimental results are presented in Section 5. Finally,
clothing. The sensor technology has advanced to a point concluding remarks are given in Section 6.
where it is possible to generate real time video sequences.
However, noise and blur are still severe problems. The 2. MOTION-COMPENSATED FILTERING
work reported here investigates the problem of
simultaneous noise suppression and object enhancement in Denoising of the video sequences can be achieved spatially
passive millimeter wave video sequences. The basis for the or temporally. Whereas spatial denoising filters such as
approach is provided by undecimated wavelet transforms in Wiener filter or even median filters cause blurring of image
the spatial dimension and motion compensated filtering in details, temporal approaches can reduce the blurring effect
the temporal dimension. The paper presents the underlying on edges [9]. In these methods, noise reduction is
principles of the approach as well as experimental results. performed along several consecutive frames. The simplest
temporal filtering is frame averaging, where the pixels at
1. INTRODUCTION the same coordinates in the frames are averaged without
further processing. This method is efficient in static parts of
Passive millimeter wave (MMW) sensors measure the the frames but may produce smearing in the moving areas
apparent temperature through emitted or reflected energy or spatial blurring when there are sudden temporal
from sources. The output of the sensors is a function of the variations. Motion-compensated filtering makes use of the
emissivity of the objects in the MMW spectrum of the motion trajectory of a pixel to reduce the blurring effect
receiver. The MMW sensors have advantages such as all caused by the motion of an object between the frames. The
weather operation, ability to penetrate fog, smoke, clothing methods assume that the variation of the pixel gray levels
or foliage, and day or night operation [2]. These abilities over any motion trajectory is due to noise (Figure 1). Thus,
make MMW sensors attractive in military application such applying a 1-D denoising filter to pixels along the
as detecting targets and guiding missiles as well as civilian estimated motion trajectory can reduce the noise. In
application such as weather forecasting. Detection of addition to the temporal methods, adaptive filtering can be
concealed weapons is one of the appealing applications. applied for more robust denoising.
Generally, metal objects such as guns have low emissivity In this paper, we exploited motion-compensated
and high reflectivity at MMW frequencies. With this filtering for temporal noise filtering. To do this we choose
property and its ability to penetrate clothing, it offers the 5 frames from N - 2 to N + 2. where Nth frame is the frame
potential to detect concealed weapons underneath a to be denoised. Because of the low resolution, we
person's clothing. Moreover, advanced sensor technology compared pixels of the frames to estimate the motion
makes it possible to generate real time video sequences trajectory instead of using blocks. Also we only implement
from the MMW sensor outputs. However, noise and low the comparison within 8 neighbors of the next or previous
contrast are severe problems in the video sequences. frame because the movement of the sequence is usually not
In this paper, we address the problem of the much abrupt. Motion trajectory has been estimated by
simultaneous noise suppression and object enhancement in comparing a pixel with the pixels in eight neighbors of the
passive MMW video sequences. The basis for the approach pixel at the same coordinate in the previous frame and the
is provided by motion compensation in the temporal next frame. The errors between those pixels are computed,

0-7803-7622-6/02/$17.00 ©2002 IEEE I - 509 IEEE ICIP 2002

and the pixel with minimum error is chosen. Hence, we 1 § x y ·
ψ 2k ( x, y ) = ψ¨ , ¸ , k = 1, 2 . (2)
have 5 pixels to be averaged along the estimated motion j
2j © 2j 2j ¹
trajectory. Chosen pixels from 5 frames are assumed to The dyadic wavelet transform of a function f(x, y) at (x, y)
represent the same pixels when there is no noise. These are is defined as
then averaged. W2k f ( x, y ) = f *ψ 2k ( x, y ) , k = 1, 2
j (3) j

x1 Mallat and Hwang [6] proved that a vector at point (x, y)

x2 composed by the dyadic wavelet transform pairs represents
the gradient of f ( x, y ) smoothed by θ ( x, y ) . That is,
x3
∇ 2 j f ( x, y )  ª¬W21j f ( x, y ) ,W22j f ( x, y ) º¼
1
∇ (θ 2 j ∗ f ) ( x, y )
Frame 1 Frame 2 Frame 2
pixel: x1 pixel: x2 = x1+ n pixel: x3 = x2+ n =
(4)
22 j
1
= 2 j ∇f ∗θ 2 j ( x, y )
Figure 1. Motion-compensated filtering along the motion 2
trajectory. Where n is noise. Denoised pixel value is The original function f ( x, y ) can be reconstructed from
approximated by averaging 3 pixels along the trajectory. W2kj f ( x, y ) with the inverse wavelets χ 2k j whose Fourier
transform satisfies
3. MULTISCALE EDGE REPRESENTATION VIA ∞

WAVELET TRANSFORM ¦ ª¬Ψ ( 2 ω

1 j
x , 2 j ω y ) Χ1 ( 2 j ω x , 2 j ω y )
j =−∞ (5)
Combination of spatial approach with temporal methods + Ψ ( 2 ω x , 2 ω y ) Χ ( 2 ω x , 2 ω y ) º¼ = 1
2 j j 2 j j

can reduce noise further. For the spatial method we The reconstruction formula is
employed the wavelet transform denoising method ∞

proposed by Bronnikov and Duifhuis [1]. The method is f ( x, y ) = ¦ (W f ( x, y ) ∗ χ

j =−∞
1
2j
1
2j
+ W22j f ( x, y ) ∗ χ 22j ) (6)
based on multiscale edge representation of images using
Detailed algorithms for the wavelet transform and
wavelet transform in the Mallat and Hwang's framework
inverse transforms are found in many other documents such
[5]. Unlike spatial filters such as an averaging filter or
as [6-8]. We utilize the Mallat's algorithm to compute
Weiner filter, the wavelet transform method provides more
undecimated fast wavelet transforms with the quadratic
flexibility and selectivity with less blurring effect.
spline wavelet whose Fourier transform is defined as
Furthermore, the wavelet transform approach provides a
sin 3 (ω x / 2 ) sin (ω y / 2 )
3
way to enhance objects in low contrast images. The method Ψ1 ( 2ω x , 2ω y ) = 4i exp ( −iω x ) (7)
(ω x / 2 ) (ω y / 2 )
3 3
has been used effectively in enhancing medical images [4],
x-ray imaging and tomography [1], and astronomical Impulse responses of the corresponding filters are found in
images [3] to denoise and enhance contrast in the images. [6, 7].
Object enhancement of the individual frame of the Multiscale edge information of the image can be
low contrast video sequence can also be performed from obtained by the magnitude and angle of the gradient
the multiscale edge representation. Such representation can
be obtained through undecimated wavelet transform j {
ρ 2 f ( x, y )  ª¬W21 f ( x, y )º¼ + ª¬W22 f ( x, y ) º¼
j
2
j }
2 1/ 2
(8)
proposed by Mallet and Hwang [5, 6]. They showed that
ªW22j f ( x, y ) º
the singularities in 1 or 2-D signals can be obtained by the θ 2 f ( x, y )  arctan « ». (9)
¬«W2 j f ( x, y ) ¼»
j

dyadic wavelet transforms. Let a function θ ( x, y ) be a

1

smoothing function. The smoothing function has the For each scale 2j, a set of local maxima of gradient points
integral one over x and y, and converges to zero when x and (xi, yi) collected by
y is infinity. The function is also supposed to be
differentiable. Then, the corresponding wavelets ψ 1 ( x, y ) G2 j ( f ) =
and ψ 2 ( x, y ) are defined by °­ ρ 2 j f ( xi , yi ) has local maximum at °½
® ª¬( xi , yi ) , ∇ 2 j f ( xi , yi ) ¼º | ¾
1 ∂
ψ ( x , y ) = φ ( x, y ) , ¯° ( xi , yi ) along the dicrection θ 2 j f ( xi , yi ) ¿
°
∂x (10)
(1)
∂
ψ 2 ( x, y ) = φ ( x, y ) represent the edge information at a scale j. For a J-level 2-
∂y D wavelet transformation, the collection
Let ψ 2k j ( x, y ) be the dilation of ψ ( x ) by a factor 2j
{S 2j
f ( x, y ) , ª¬G2 j ( f ) º¼
1≤ j ≤ J } (11)

I - 510
where S2 f ( xi , yi ) is the discrete low-pass approximation
J Enhancement of the contrast of an image is now
of f ( x, y ) at the coarsest scale 2 , is called the multiscale
J performed with the denoised edges G 2 ( f ) by transforming
j

edge representation of the image f ( x, y ) . the magnitude ρ 2 f ( x, y ) at the edge point (x, y). There are
j

Mallat and Zhong [6], and Mallat and Hwang [5] two transforming methods for the enhancement, which are
showed that the original image can be estimated from the the linear and non-linear transformations [1, 3]. The linear
multiscale edge representation by the iterated projection transform is performed by multiplying the multiscale
onto the convex sets (POCS) methods. However, we use all gradients by a constant factor. The method can be modified
the available details as well as the multiscale edge to use different multiplication factors for different scales.
representations to reconstruct denoised and enhanced On the contrary, multiplication factors of non-linear
images. This modified approach removes the need of transfer function methods are variable with respect to the
iterated projection on the convex sets and increases the magnitude of the gradient. In this paper we used the non-
computational speed. linear transform
°­ ª g º ½°
G 2 j ( f ) = ® « k − ( k − 1) » × g | ∀g ∈ G2 j ( f ) ¾ (14)
4. DENOISING AND OBJECT ENHACEMENT max g
¯° ¬ ¼ ¿°
USING THE MULTISCALE EDGE
REPRESENTATION where, k is a constant and g is a gradient at an edge.
Equation (14) reveals that the multiplication factor
becomes small, when the gradient is large.
With wavelet transformation, higher frequency components
Transformed edges by (13) and (14) are used to
are generally located in finer scales, the edges in each scale
reconstruct the denoised and enhanced image using (6). As
are represented the set G2 ( f ) in (10). Therefore, most of
j
mentioned above, instead of using iterated method with
the noise will be observed at edge points in the fine scales only the edge information, other gradient information is
because noise generally exhibits high-frequency used to reconstruct the enhanced image.
characteristics. By suppressing the noisy edges in the fine
scales, noise can be reduced without removing true edges. 5. EXPERIMENTAL RESULTS
Denoising with wavelet transforms can be performed by
two ways, either using the magnitude (modulus based In this section, we present examples of the denoised and
method) or the angle information (phase based method) of enhanced frames from the video sequences captured by the
the gradient on the multiscale edge representation [1]. The passive millimeter wave array sensor. The sensor used here
modulus-based methods use information about the is a passive millimeter-wave camera manufactured by Trex
magnitude of the gradient. It is supposed that noise Enterprises. The structure of the camera is based on a
typically has a small gradient than true edges. Thus, noise phased radiometer array. The output of the camera is
can be removed by eliminating the gradients whose converted into a 60 x 75 pixel digital image frame . Output
magnitudes are smaller than a given threshold. For the video sequence has been preprocessed by the following
phase based algorithms, an observation that the gradients of procedures;
true edges have similar angles in a small neighborhood is 1) Gain and offset calibration.
used. A gradient is considered as noise if its angle is totally 2) Linear interpolation from adjacent good pixels to correct
different from the neighbors. We adopted the modulus isolated dead pixels.
based method. A problem with the modulus based methods 3) A previously saved background image was then
is that if the same threshold is applied to all the scales, it subtracted from each frame in real time to remove the
may remove true edges as well as noisy edges in lower effects of background objects such as buildings, cars etc.
scales because the differences between the magnitude of 4) Pixel values of the original data represent a value in a
noisy and true edges become smaller in lower scales. To scale of -90 to –70 in this example. The values were
avoid this problem, we apply a variable threshold linearly mapped into [0, 1].
t j = k × max ª¬ ρ 2 f ( x, y ) º¼ , 0 < k < 1
j (12) Figure 2 shows two frames taken from the sample
Applying this method, we assume that only video sequence. In the frames, the concealed gun does not
show clearly because of noise and low contrast. To reduce
G 2 j ( f )
the noise in the scene, we first applied motion–
(13) compensated filtering. The images in Figure 3 show
{ }
= ª¬( xi , yi ) , ∇ 2 j f ( xi , yi ) ¼º | ρ 2 j f ( xi , yi ) > t j , ∀i ∈ I 2 j ( f ) denoised frames by the motion compensated filtering. The
have the true edges and other values are considered as frames were then spatially denoised and enhanced by the
noise and removed. wavelet transform methods. The frames were decomposed
with 4 decomposition level and edges in the fine scales
were detected using the magnitude and angles of the

I - 511
gradient of the multiscale edge representation. The as true objects. Techniques are currently being investigated
threshold for denoising was 15% of the maximum gradient to address this problem.
at each scale. Figure 4 shows the final results, which show
the contrast enhanced and denoised frames. One can notice ACKNOWLEDGEMENT
the image of a pistol in the chest of the subject is more
apparent in the enhanced frames than in the original This work is sponsored in part by the National Institute of
frames. Justice under an Air Force Research Laboratory, Rome,
contract to support detection of concealed weapons.
Additional support was provided by NYSTAR. Trex
Enterprises supplied the passive millimeter wave video
sequences.

REFERENCES

Figure 2. Original frames [1] A. V. Bronnikov and G. Duifhuis, "Wavelet-Based

Image Enhancement in X-Ray Imaging and
Tomography," Applied Optics, vol. 37, no. 20, pp.
4437-4448, 1998.
[2] L. A. Klein, Millimeter-Wave and Infrared Multisensor
Design and Signal Processing. Boston: Artech House,
1997.
[3] J. Lu and J. Dennis M. Healy, "Contrast Enhancement
Via Multiscale Gradient Transformation," pp. 482-486,
Figure 3. Denoised frames by averaging along the motion 1994.
trajectory [4] J. Lu, J. Dennis M. Healy, and J. B. Weaver, "Contrast
Enhancement of Medical Images Using Multiscale Edge
Representation," Optical Engineering, vol. 33, no. 7, pp.
2151-2161, 1994.
[5] S. Mallat and W. L. Hwang, "Singularity Detection and
Processing with Wavelets," IEEE Transactions on
Information Theory, vol. 38, no. 2, pp. 617-643, 1992.
[6] S. Mallat and S. Zhong, "Characterization of Signals
from Multiscale Edges," IEEE Transactions on Pattern
Figure 4. Denoised and enhanced frames by wavelet approach Analysis and Machine Intelligence, vol. 14, no. 7, pp.
710-732, 1992.
[7] R. M. Rao and A. S. Bopardikar, Wavelet Transforms :
6. CONCLUSION Introduction to Theory and Applications. Reading, MA:
Addison-Wesley, 1998.
With the abilities of penetration and high reflectance off [8] G. Strang and T. Nguyen, Wavelets and Filter Banks.
Wellesley, MA: Wellesley-Cambridge Press, 1997.
metal surfaces, MMW sensors become attractive tools for
[9] A. M. Tekalp, Digital Video Processing. Upper Saddle
detection of concealed weapons. Moreover, sensor River, NJ: Prentice-Hall, Inc., 1995.
technologies now make it possible to generate the video
sequence from the sensor output in real time. However, the
output video sequences have problems related to noise and
blurriness. We have adopted undecimated wavelet
transform and motion-compensated filtering to solve to the
problems. It was possible to reduce the noise in the video
sequences without much blurring of edges. Additionally,
objects in the low-contrast frames could be enhanced using
the multiscale edge representation obtained by the wavelet
transform. The paper also provided experimental results of
the algorithms. For further research, there is one more
noticeable problem with the video sequence, which is that
ghost images due to grating lobes and side lobes exist in the
frames. The ghost images are generated because of the
phased array structure of the MMW sensor. The algorithm
in the paper enhanced the unwanted ghost images as well

I - 512