IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL.

32, 2023 737

Fast Guided Median Filter

Kazu Mishiba , Member, IEEE

Abstract— Faster computation of a weighted median (WM)

filter is impeded by the construction of a weighted histogram for
every local window of data. Since the calculated weights vary for
each local window, it is difficult, using a sliding window approach,
to construct the weighted histogram efficiently. In this paper,
we propose a novel WM filter that overcomes the difficulty of
histogram construction. Our proposed method achieves real-time
processing for higher resolution images and can be applied to
multidimensional, multichannel, and high precision data. The
weight kernel used in our WM filter is the pointwise guided
filter, which is derived from the guided filter. The use of kernels
based on the guided filter avoids gradient reversal artifacts
and shows a higher denoising performance than the Gaussian
kernel based on the color/intensity distance. The core idea of
the proposed method is a formulation that allows the use of
histogram updates with a sliding window approach to find
the weighted median. For high precision data we propose an
algorithm based on a linked list that can reduce the memory
requirements of storing histograms and the computational cost
of updating them. We present implementations of the proposed
method that are suitable for both CPU and GPU. Experimental Fig. 1. Flash/no-flash denoising with WM filters. (a) Guide image. (b) Fil-
results show that the proposed method indeed realizes faster tered image with the FWM-O(r ) filter, whose weights are based on the
computation than conventional WM filters and is capable of Gaussian kernel. (c) Zoomed-in result of (b). (d) Input image. (e) Filtered
filtering multidimensional, multichannel, and high precision data. image with the GPU-O(r ) filter, whose weights are based on the guided filter
This is an approach which is difficult to achieve with conventional kernel. (f) Zoomed-in result of (e).
methods.
Index Terms— Weighted median filter, guided filter. include flash/no-flash denoising [6], image upsampling [7],
depth map enhancement [8], and cost volume aggregation in
stereo matching [9], [10], [11].
I. I NTRODUCTION The results of joint filters depend on the filter kernel that
is used as well as the guide image. Methods that use the
S MOOTHING filters are one of the most basic tools
in image processing. Their wide range of applications
includes image processing for visible light, medical image
Gaussian kernel based on color/intensity distance, like the
bilateral filter, have known problems with gradient reversal
analysis [1], [2] and seismic data processing [3], [4]. artifacts on the edges [12]. On the other hand, the guided filter
Edge-preserving smoothing is also an important technique kernel [12] allows for fast operation while avoiding gradient
for better smoothing. Edge-preserving smoothing filters can reversal artifacts.
be divided into two groups: Filters in the first group calculate Joint filtering can be applied to weighted median (WM)
their coefficients from the input image. An example is the filters as well as weighted average filters. WM filters
bilateral filter [5], which calculates a weighted average using have edge-preserving properties and robustness against out-
filter coefficients derived from a Gaussian distribution of liers. Due to these properties, joint WM filters are used
spatial and color/intensity distances. Filters in the second in various image processing applications like optical flow
group calculate their coefficients from the guide image. Joint estimation [13], disparity refinement [14], [15], and rain
filters are often used when the input image has unreliable edge removal [16]. The use of the Gaussian kernel based on
information, and a guide image is available that has reliable color/intensity distance can cause edge reversal artifacts in
edge information. Examples of the application of joint filters WM filters. Using a guided filter kernel can avoid this (Fig. 1).
The heavy computational cost of the WM filter is an obsta-
Manuscript received 8 June 2022; revised 27 November 2022; cle to applications that require real-time performance. A fast
accepted 19 December 2022. Date of publication 5 January 2023; date computation algorithm for WM filters is needed. A possible
of current version 10 January 2023. This work was supported by JSPS
KAKENHI under Grant 20K11887. The associate editor coordinating the approach to accelerate the computation is to adopt acceleration
review of this manuscript and approving it for publication was Dr. Marta approaches used in the unweighted median (UWM) filter. The
Mrak. core idea of acceleration in the UWM filter is the sliding win-
The author is with the Department of Electrical and Electronic Engineering,
Tottori University, Tottori 680-8550, Japan (e-mail: [email protected]). dow, which takes advantage of overlapping filter windows by
Digital Object Identifier 10.1109/TIP.2022.3232916 using neighboring pixels to accelerate histogram construction.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
738 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

As described in [17], it is almost impossible to apply methods The remainder of this paper is organized so that Section II
that accelerate UWM filters to WM filters. The WM filter reviews the related work on faster computation for UWM
requires the construction of a weighted histogram. Since the and WM filters. We define the weighted median used in this
weights spatially vary for each local window, the sliding paper in Section III. The proposed WM filter is presented in
window approach cannot be applied directly to construct the Section IV, and its extensions to multidimensional, multichan-
weighted histogram. Several acceleration methods have been nel, and high precision data are presented in Section V. The
proposed for WM filters using ideas different from those of implementation details of our method are shown in Section VI.
UWM filters. Ma et al. [14] proposed a constant time WM Section VII shows the experimental results over high resolu-
filter: An input image is projected onto a 3D space, and tion, multidimensional, multichannel, and high precision data.
then an edge-preserving filter is applied to each 2D slice. Finally, we conclude our paper in Section VIII.
The advantage of this method is that any filter kernel can
be used, including bilinear and guided filter kernels. Since
II. A R EVIEW OF R ELATED W ORK
an edge-preserving filter is applied to all 2D slices, the
computational cost is not low enough to realize real-time pro- This section focuses on the study of UWM and WM filters
cessing. Zhang et al. [17] proposed an innovative acceleration in achieving faster computation in image processing.
method for WM filters. They use a joint histogram with two
dimensions: one of the dimensions is pixel value and the other
is feature. Faster computation is achieved by combining the A. Median Filter
joint histogram with a median tracking technique and a data The median filter, which we also call the UWM filter,
structure that provides instant access to the data. This method replaces the pixel value with the median value in a window
achieves real-time processing for relatively small grayscale centered at each pixel. There are two categories of methods
images with small window sizes. Filter kernels that can be for calculating median values. The first category is histogram-
used in this method are limited to those calculated from the based methods, and the second category is sort-based methods.
color/intensity difference between two pixels. This includes The histogram-based methods first construct a histogram
the Gaussian kernel based on color/intensity distance. This of the pixels in the window and then search for their
means that this method cannot use a guided filter kernel, which median. For faster processing, faster histogram construction
calculates weights from multiple pixels. A fast WM filter using and faster median searches have been studied. To accelerate
a guided filter kernel is a challenging task that has not yet been the construction of histograms, many methods use a slid-
achieved. ing window approach that exploits the redundancy of the
In this paper, we propose a fast guided median filter overlapping regions of the windows of adjacent pixels. The
weighted by a pointwise guided filter kernel. Like the guided sliding window approach has two categories: The first is
filter kernel proposed in [12], the pointwise guided filter called a O(r ) sliding window approach and the latter a O(1)
kernel can prevent gradient reversal artifacts. The core idea sliding window approach. This is because the computational
of the proposed method is a formulation that allows for the complexities with respect to window radius r are O(r ) and
use of histogram updates with a sliding window approach O(1), respectively. Huang et al. [18] proposed a UWM filter
to find the weighted median. Compared to the conventional that uses the O(r ) sliding window approach to construct a
fast WM filters, the proposed method is not only faster but histogram efficiently. The histogram of a horizontal sliding
also applicable to multidimensional, multichannel, and high window can be updated efficiently by removing the elements
precision data. Until recently, conventional methods were used of an excluded column resulting from the sliding window and
to process various types of data. Examples include 8-bit color then adding the elements of a new column included also, from
image processing; multidimensional data such as video and the sliding window. To accelerate the median search, Huang
light field images; multichannel data such as multispectral et al. also used a technique referred to as median tracking [17].
images; and high precision data such as medical images, HDR Median tracking records changes in the number of pixels
images, and depth sensor data. The conventional methods have below the median value while the histogram is updating and
been insufficient for this data due to their heavy computational starts the median search for the next window from the median
cost and enormous memory requirements. This has lead to a value of the previous window. Huang’s method is extendable
need to reduce the data size, e.g., by using downsampling. in the temporal dimension too, so that it can be used as a
However, this leads to degraded accuracy in execution. Despite UWM filter for video [19]. The computational cost increases
the situation, there has been little discussion about using WM as the window size increases in the O(r ) sliding window
filtering for multidimensional, multichannel, or high precision approach. However, the computational cost is independent of
data. the window size in the O(1) approach [20], [21]. In the O(1)
The main contributions of this research are: approach, the histogram of a horizontal sliding window is
1) We propose an accelerated computation algorithm for updated by respectively subtracting and adding the histograms
the WM filter whose kernel is based on the guided filter. of the column excluded and the column included by the
Faster computation is achieved by formulating a WM sliding window. To accelerate the median search, a coarse-
filter to which a sliding window method can be applied. level histogram that accumulates only the higher order bits of
2) The proposed filter can be applied to multidimensional, pixels is used to reduce the number of bins to search for and
multichannel, and high precision data. update. SIMD operations are used to accelerate the addition
MISHIBA: FAST GUIDED MEDIAN FILTER 739

and subtraction between histograms. The O(1) sliding window it is computationally expensive. In this paper, we propose a
approach is also used in the arc-distance median filter [22]. fast WM filter based on the guided filter kernel.
The sort-based methods first sort the pixel values in the win-
dow and then search for their median. Similar to the histogram- III. A W EIGHTED M EDIAN F ORMULATION
based methods, the sorting-based methods use redundancy in In this section, we define the weighted median formulation
the window’s overlapping region of adjacent pixels and this used in this paper. Several formulations of weighted medians
accelerates the finding of the median. Chaudhuri [23] used have been proposed [28], [29], [30]. The weighted median
the O(r ) sliding window approach to compute rank orders in formulation is based on the formulation proposed in [14].
a window. This can be used for min, max, or median filters. Let f x ∈ F be a pixel value at x in a single channel
When updating ordered elements through a sliding window, input image f , where F = {i ∈ Z| f min ≤ i ≤ f max } and
the elements of the excluded column are removed, and the f min , f max ∈ Z are the minimum and maximum values that
elements of the windows newly included column are inserted f x can take, respectively. In [14], the weighted median f x⋆ in
using the principle of mergesort. Sánchez and Rodríguez [24] a local window centered at pixel x is defined as:
proposed a UWM filter that combines a sorting algorithm, j f max
1 X
based on the complementary cumulative distribution function, f x⋆ = min j
X
s.t. Hx (i) ≥ Hx (i), (1)
with the O(1) sliding window approach. Adams [25] proposed j∈F
i= f min
2
i= f min
fast UWM filters using separable sorting networks. The key
idea for the accelerating median search is to share most of where Hx is a local weighted histogram and a value at bin i
the sorting tasks between adjacent pixels. Sort-based meth- is calculated by
wx,y f yδ (i),
X
ods work faster than histogram-based methods for small to Hx (i) = (2)
medium window sizes or high precision data. y∈x
where x is a set of pixels inside the local window, wx,y is
B. The Weighted Median Filter a weight calculated from the affinity between pixels x and y,
and
The WM filter replaces the pixel value of each pixel with (
the weighted median value in a window centered at each δ 1 ( f x = i)
f x (i) = (3)
pixel. The weights are calculated from an input image itself 0 ( f x ̸ = i).
or from a guide image. Since the weights vary from window
to window, it is difficult to accelerate the construction of In this paper, the weighted median formulation is extended
the weighted histogram using a sliding window approach as as follows. A weighted cumulative histogram up to bin i can
is used in UWM filters. Ma et al. [14] proposed the first be expressed as
O(1) time WM filter. This method projected a 2D input i i
f yδ ( j).
X X X
image into a 3D space, where the extended coordinate was a Hx↓ (i) = Hx ( j) = wx,y (4)
histogram bin of the input image. By applying an O(1) edge- j= f min y∈x j= f min
preserving filter to 2D slices with the same bin, the method The weights wx,y are often calculated from the coefficients
achieved O(1) time computation. Since filtering is repeated of a smoothing filter. Since the sum of the coefficients of a
for the number of bins, it is difficult to perform real-time smoothing filter is 1, the following holds:
processing for high precision images. Zhang et al. [17] used f max
a joint histogram with the O(r ) sliding window approach to Hx↓ ( f max ) =
X
wx,y
X
f yδ ( j) =
X
wx,y = 1. (5)
accelerate the construction of a weighted histogram. Since the
y∈x j= f min y∈x
joint histogram had a large number of bins, data traversal
was time-consuming. To solve this problem, a data structure From this, the following weighted median with smoothing
that allows fast traversal was introduced. The large memory filter coefficients can be derived:
requirement of the joint histogram made it difficult to apply f x⋆ = min i s.t. Hx↓ (i) ≥ 0.5. (6)
the O(1) sliding window approach and to deal with mul- i∈F
↓
tichannel and high precision data. Zhao et al. [26] applied To find its solution, we track the change in Hx (i) with i.
the bilateral grid data structure [27] to WM filtering. Their Suppose all the weights are non-negative. When tracking from
↓
method achieved real-time processing on 2D images when i = f min with increasing i, the solution is i where Hx (i)
the sampling rate of the bilateral grid was high. Since their becomes 0.5 or higher for the first time. When tracking from
method was based on the bilateral grid, it had the same ↓
i = f max with decreasing i, the solution is i just before Hx (i)
problems that the bilateral grid had. For example, using a small becomes less than 0.5 for the first time. By generalizing this
window size required fine sampling, which incurred large observation, we define the weighted median tracked from a
memory and computational costs. Due to memory limitations, starting bin k as follows:
processing high resolution, high-dimensional, multichannel, ↓ ↓
 min i s.t. Hx (i) ≥ 0.5 (Hx (k) < 0.5)

and high precision data requires coarse sampling, leading to
significant degradation of results. f x⋆ = i>k,i∈ F
1 + max i s.t. Hx↓ (i) < 0.5 (Hx↓ (k) ≥ 0.5).
Zhang’s method and Zhao’s method cannot use the guided i≤k,i∈F
filter kernel. Ma’s method can use the guided filter kernel, but (7)
740 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

This weighted median formulation is used for the proposed time using the separable summed area table (SSAT) or the one
method. pass summed area table (OP-SAT) [32].
When the weights are non-negative, the solutions of Using the multipoint guided filter kernel the histogram
(6) and (7) are the same. On the other hand, when the weights Hx (i) is,
contain negative values, the solutions are not necessarily the
same. As is seen in the next section, the weights used in the Hx (i) = µ(axδ (i))gx + µ(bδx (i)), (14)
proposed method may contain negative values. where
1
g y f yδ (i) − µ(gx )µ( f xδ (i))
P
IV. T HE FAST G UIDED M EDIAN F ILTER axδ (i) =
|x | y∈x
, (15)
The construction of a histogram Hx (i) shown in (2) impedes vx
δ δ δ
fast computation of WM filters. Section IV-A discusses the bx (i) = µ( f x (i)) − ax (i)µ(gx ). (16)
problems of the computation of Hx (i) in the WM filter
To use a sliding approach, we attemptPto rewrite these
using the guided filter kernel. Section IV-B presents the δ
equations using histograms. Let Fx (i) = y∈x f y (i) be a
proposed method to solve this problem. The proposed method δ δ
efficiently obtains the weighted median using the median histogram of f at pixel x. ax (i) and bx (i) can be rewritten
tracking technique [17] and the sliding window approach using Fx (i) as
δ
y∈x g y f y (i) − µ(gx )Fx (i)
shown in Sec. IV-C.
P
axδ (i) = , (17)
vx |x |
A. Problems With the Guided Filter Kernel 1
bδx (i) = Fx (i) − axδ (i)µ(gx ). (18)
For the histogram-based WM filter process, the weighted |x |
cumulative histograms are updated as follows:
These equations reveal three problems that arise when the
Hx↓ (i) = Hx↓ (i − 1) + Hx (i). (8) multipoint guided filter kernel is used in WM filtering. First,
δ
y∈x g y f y (i) in (17) cannot be expressed using a histogram
P
As this formula shows, fast computation of Hx (i) is the key of f . The naive calculation of this term is computationally
to achieving faster WM filtering. In the UWM filter, the inefficient. Second, the computation of the mean value of axδ (i)
histogram can be updated efficiently using a sliding approach. and bδx (i) is time consuming. Since axδ (i) and bδx (i) depends
This subsection discusses the difficulties of using the same on i, SSAT and OP-SAT cannot be used to calculate their
approach with the guided filter kernel. mean values. Third, since many histograms are required, the
Let gx ∈ R be a pixel value at pixel x in a single channel calculation is inefficient and requires a lot of memory. For
guide image g. The guided filter [12] assumes that the filtering the calculation of the mean values of axδ (i) and bδx (i), it is
output ex at x can be estimated by a linear transformation of necessary to keep histograms for all pixels in x .
g in a local window z centered at pixel z:
e x = az g x + bz , (9) B. Our Approach
where Instead of the multipoint guided filter kernel, the proposed
1 P
g y f y − µ(gz )µ( f z ) method uses a pointwise guided filter kernel, which uses
|z | y∈z
az = , (10) pointwise modeling [31]. The filtering result is
vz
bz = µ( f z ) − az µ(gz ), (11) ex = ax gx + bx = ax (gx − µ(gx )) + µ( f x ). (19)
µ(·⋆ ) is the mean value of · in ⋆ , |⋆ | is the number of This equation can also be formulated as
elements of ⋆ , and v⋆ = σ⋆ + ϵ where σ⋆ is a variance of g X X
in ⋆ and ϵ is a regularization parameter. The output value ex ex = cx g y f y + dx fy, (20)
is estimated in multiple windows, which is called multipoint y∈x y∈x
modeling [31]. In the rest of this paper, we call the guided where
filter proposed in [12] the multipoint guided filter. The final
gx − µ(gx ) 1
estimate is calculated as the average of multiple estimates. cx = , dx = − µ(gx )cx . (21)
vx |x | |x |
ex = µ(ax )gx + µ(bx ). (12)
The weight in the pointwise guided filter can be expressed as
The weight of the multipoint guided filter is 1

(gx − µ(gx ))(g y − µ(gx ))

X  (gx − µ(gz ))(g y − µ(gz ))
 wx,y = 1+ . (22)
wx,y =
1
1+ . |x | vx
|x |2 vz Unlike the multipoint guided filter, the weight of the pointwise
{z|x,y∈z }
(13) guided filter is not symmetric, i.e., wx,y ̸ = w y,x .
Using (20), Hx (i) can be expressed as
Here it is noted that the weights may contain negative values.
g y f yδ (i) + dx f yδ (i).
X X
The summation of the elements in the local window needed Hx (i) = cx (23)
in the calculation of mean values can be computed in O(1) y∈x y∈x
MISHIBA: FAST GUIDED MEDIAN FILTER 741

Algorithm 1 Search Weighted Median

Fig. 2. (a) 2D window and column windows. (b) Column window update
in the O(1) sliding window approach.

column window of length 2r + 1. Here r is the window radius

(2)
of Ws,t .
The median tracking technique can reduce the computation
time for the median search. This technique starts the calcula-
⋆ in the window W (2) from k which is the ending
tion of f s,t s,t
⋆
bin used for the calculation of f s−1,t in the previous window
(2) ⋆
Ws−1,t . Here, k is f s−1,t when the median value is searched
(2) ⋆
in ascending order in Ws−1,t , and f s−1,t − 1 when the median
Here, gx f xδ (i) can be replaced with value is searched in descending order. This method works
( because the median value does not change much between
gx ( f x = i) neighboring pixels and this aids in the search for the median.
gxδ (i) = (24)
0 ( f x ̸ = i). Since the weights used in the proposed method can be negative
values, the search from f min and the search using median
Let G x (i) = y∈x g δy (i) be a histogram of f weighted by
P
tracking may have different solutions, as mentioned in Sec. III.
g. Then Since it rarely occurs, it makes little visual difference. Less
than 0.1 % of the pixels have different solutions in the
Hx (i) = cx G x (i) + dx Fx (i) (25)
30 natural color images used in the experiment in Sec. VII-C.
which is that histogram Hx (i) can be calculated as a weighted The sliding window approach allows for efficient calculation
sum of the histograms G x (i) and Fx (i). Equation (25) is the of F, G, f ↓ , and g ↓ . When sliding a filter window to the right,
(2) (2)
core result of this research. A fast WM filter using a guided window Ws,t overlaps much of the region of window Ws−1,t .
− +
Let s = s−r −1 and s = s+r . To obtain the data in window
filter kernel can be achieved because all the elements used
(2) (1)
to compute Hx can be computed fast. G x and Fx can be Ws,t , the sliding window approach adds data in window Ws + ,t
obtained efficiently using a sliding window approach. Since (2) (1)
to data in window Ws−1,t and removes data in window Ws − ,t
the weights cx and dx are independent of i, they can be (2)
from data also in window Ws−1,t (see Fig. 2 (a)). The O(r )
computed efficiently using SSAT or OP-SAT. and O(1) sliding window approaches differ in this update
process.
C. Efficient Median Search The algorithm for the proposed weighted median filter
Let W = {F, G, f ↓ , g ↓ , k} be the local window at the pixel based on the O(r ) sliding window approach is shown in
(2)
of interest Algorithm 2. The data in Ws,t is obtained by adding the
P x, where F and G are Pthe histograms of W , and (1) (2) (1)
f ↓ = kj= fmin F( j) and g ↓ = kj= fmin G ( j) are values at pixels in Ws + ,t to Ws−1,t and removing the pixels in Ws − ,t
the tracking bin k of the cumulative histogram of F and G, (2)
from Ws−1,t . The algorithm for adding pixels is shown in
respectively. Equation (8) can be rewritten as Algorithm 3. The computational complexity of the histogram
update in the O(r ) sliding window approach is proportional
Hx↓ (i) = cx (g ↓ + G (i)) + dx ( f ↓ + F(i)). (26)
to the filter radius. Given an image with a pixel bit depth of n
The pseudocode for the median search of the proposed method bits, 2(2n + 1)(n + 2 log2 (2r + 1)) + n bits are required to store
is shown in Algorithm 1. the data in the window. f ↓ and g ↓ need n + 2 log2 (2r + 1)
For fast computation of the WM filter, the proposed method bits each, F and G need 2n (n + 2 log2 (2r + 1)) bits each, and
uses the median tracking technique and the sliding window k needs n bits.
approach. Here we will discuss the case where the input image The algorithm for the proposed weighted median filter
and the guide image are both 2D grayscale images of size based on the O(1) sliding window approach is shown in
(2) (2)
M × N . Let Ws,t be a 2-dimensional square window of size Algorithm 4. The data in Ws,t is obtained by merging the
(1) (1) (2) (1)
2r + 1 centered at pixel (s, t) and Ws,t be a 1-dimensional data in Ws + ,t into Ws−1,t and separating the data in Ws − ,t
742 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

Algorithm 2 O(r ) Sliding Window Approach Algorithm 5 Merge Windows

Algorithm 3 Add Pixels to Window

Algorithm 4 O(1) Sliding Window Approach

the next row, the separation process adjusts f (2)↓ and g (2)↓
(2) (1)
in Ws−1,t without updating k (1) in Ws − ,t , e.g., f (2)↓ ←
f (2)↓ (1) (1)
+ F (k ) instead of f (1)↓ ← f (1)↓ + F (1) (k (1) ). The
computational complexity of the histogram update in the O(1)
sliding window approach is proportional to the bit depth of
the image. Since column windows are used in addition to the
main window W (2) , (2(n +2 log2 (2r +1))(2n +1)+n)(M +1)
bits are required to store the data of all the windows. While
the O(1) sliding window approach has an advantage over
the O(r ) sliding window approach in terms of computational
complexity due to the window size, it needs additional memory
and computation for the column windows.

V. E XTENSIONS TO THE WM F ILTER

(2) The proposed method can be extended to multidimensional,
from Ws−1,t . To achieve this update, column windows must multichannel, and higher precision data.
be stored and updated. The number of column windows to be
stored is equal to the image width N . Before the merging
(1) A. The Multidimensional Extension
process, the column window Ws + ,t is obtained by sliding
(1) The proposed method can easily be applied to multidi-
Ws + ,t−1 vertically one pixel to add and remove pixels (see
Fig. 2 (b)). The algorithm for merging two windows is shown mensional images because SSAT and OP-SAT used in the
in Algorithm 5. F (2) and G (2) are updated by adding the values calculation of cx and dx and the sliding window approach
of each bin of F (1) and G (1) , respectively. Since k (1) and k (2) used in histogram updating can be straightforwardly extended
can be different, first k (1) is updated to be equal to k (2) , then to multidimensional images.
f (1)↓ and g (1)↓ are added to f (2)↓ and g (2)↓ , respectively.
(1) (2)
The updated Ws + ,t is used again to update Ws−1,t+1 in the B. Using Multichannel Guide Images
(2) (2)
next row. Since k (2)
in Ws−1,t and Ws−1,t+1 are expected to Like various filters, a multichannel input image is processed
(1) (2)
be close, Ws + ,t updated with k (2) in Ws−1,t is expected to for each channel. Since cx and dx are commonly used for all
(2) channels of an input image, they need to be calculated only
have a small update when merged with Ws−1,t+1 . The process
(1) (2)
once.
of separating Ws − ,t from Ws−1,t is similar to the opposite In the case when the guide image is multichannel, the
process of merging but is partially different. Since k (1) in pointwise guided filter can be extended in the same way as
(1)
Ws − ,t has been updated to a value suitable for merging in the multipoint guided filter. Let gx ∈ Rm be a vector whose
MISHIBA: FAST GUIDED MEDIAN FILTER 743

entries are the pixel value of each channel at pixel x in the This problem is solved by having the histogram in a linked
guide image g, where m denotes the number of channels of list of high precision data in ascending order. Both O(r ) and
the guide image. The output of the pointwise guided filter with O(1) sliding window methods can use this solution. This paper
a multichannel guide image is expressed as discusses the O(1) sliding window approach to sorted linked
X X lists. Adding elements to the linked list is like the merge
e x = c⊤
x g y f y + dx fy, (27) process of a mergesort, although slightly different. Elements
y∈x y∈x
with the same bin are not inserted into the list but are added
where ·⊤ denotes the transpose of the matrix, to the existing elements. Please for a moment assume that
the linked list of the histogram of a column window has
gx − µ(gx ) 1
cx = Vx−1 , dx = x µ(gx ),
− c⊤ (28) already been sorted during processing in the previous row.
|x | |x | Then, adding one pixel to the histogram of the column window
Vx = 6x + ϵ I , 6x ∈ Rm×m is a covariance matrix of g in x , merges one element into the linked list. Adding the histogram
I is the identity matrix, and µ(gx ) ∈ Rm is a vector whose of the column window to the histogram of the main window
entries are the mean values of each channel of g in x . Using merges two sorted lists. When removing elements of the
(27), the histogram Hx is derived in the same way as for the histogram of the column window from the histogram of the
single channel as follows: main window, bins whose value becomes zero are removed
from the list. The computation time for updating the linked list
Hx (i) = c⊤
x Gx (i) + dx Fx (i), (29)
is proportional to the length of the linked list. As the window
where Gx is a weighted histogram of f and Gx (i) ∈ Rm is a size increases, the computation time tends to increase because
vector with values for each channel bin. the possibility of having values at various bins increases.
We analyze the behavior of the regularization parameter ϵ,
which controls the smoothing effect, when using multichannel VI. T HE I MPLEMENTATION D ETAILS
guide images. Consider the case where all channels of g are the In this section, we describe CPU and GPU implementations
same. That is, gx = gx 1m , where 1m ∈ Rm is a vector, whose of the proposed method. In general, a CPU is composed of
entries are all 1. Here, the same output is wanted as when fewer cores and more cache memory than a GPU. Due to
the single channel g is used as the guide image. Substituting this difference in composition, the suitable sliding window
gx = gx 1m into (27), we obtain approach differs between CPU and GPU, which will be
ex = αx (gx − µ(gx )) + µ( f x ), (30) discussed in Sec. VI-A. In the implementation of the proposed
method, the O(r ) sliding window approach is used for the
where GPU implementation as shown in Sec. VI-B, and the O(1)
 
sliding window approach is used for the CPU implementation
1 X
αx = 1⊤ −1
m Ṽx 1m
 g y f y − µ(gx )µ( f x ) , (31) as shown in Sec. VI-C. We also describe the implementation
|x | for high precision data in Sec. VI-D.
y∈x

and Ṽx = σx 1m 1⊤
m +ϵ I . Using the Sherman-Morrison formula,
we obtain A. Sliding Window Approach for CPU and GPU
 ϵ −1 The O(1) sliding window approach is superior to the O(r )
m Ṽx 1m = σx +
1⊤ .
−1
(32) sliding window approach in computational time. However,
m
while this is true for single-threaded environments, it is
As can be seen by comparing with (19), ϵ becomes relatively
not necessarily true for multi-threaded environments. In the
small as the number of channels increases. To obtain the same
O(1) sliding window approach, each row cannot be computed
level of smoothing effect when the guide image has multiple
independently because the data of the column window updated
channels compared to when the guide image has a single
in the previous row is needed. In a parallel implementation
channel, ϵ should be multiplied by the number of channels.
of the O(1) sliding window approach, an input image is
The same is true for the multipoint guided filter kernel.
divided into multiple blocks, and these blocks are processed
in parallel. A small number of divisions is ineffective in
C. High Precision Extensions terms of parallelism while a high number of divisions causes
Theoretically, the proposed method described so far can a lot of overhead, which is mainly in the construction of
be applied to high precision data. However, high precision the histograms of column windows. For fast computation,
data causes an explosive increase in the size of the histogram, it is necessary to store the data of the column windows in
which leads to practical problems. One is the requirement for the cache memory, which needs a large amount of cache
a large amount of memory to store histograms and another is memory. The O(1) sliding window approach is therefore
the increased computational cost of updating the histograms. suitable for CPUs but not for GPUs. [34] shows that the CPU
In a UWM filter, using the ordinal transform solves these implementation outperforms the GPU implementation in terms
problems because the transform can reduce the number of bins of computational speed in the UWM filter implementation
in the histogram [33]. On the other hand, the proposed method using the O(1) sliding window approach. Unlike the O(1)
cannot use the ordinal transform because (25) is not invariant sliding window approach, the O(r ) sliding window approach
to the transform. does not have the overhead of parallelization because each
744 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

row is computed independently. Also, it does not require a Algorithm 6 GPU Kernel Implementation
large cache for execution because it does not need the data
of the column windows. Therefore, the O(r ) sliding window
approach is suitable for processors with many cores and a
small cache, such as GPUs.

B. GPU Implementation With O(r ) Sliding Window Approach

In this paper, the NVIDIA’s CUDA platform is used to
implement the proposed method on GPUs.
In the O(r ) sliding window approach on GPU, cx and dx
are first calculated using SSAT, and then the weighted median
is calculated by sliding a window vertically for each image
column. The reason for sliding vertically instead of horizon-
tally is to achieve coalesced memory access for efficient pixel
loading.
High parallelism and efficient memory access are required
to achieve high-speed processing. Since the data of the sliding
window are accessed frequently, they should be stored in
on-chip memory, such as the L1 cache or shared memory.
One viable way to achieve high parallelism is to allocate
one thread for each image column. This approach, however,
achieves low parallelism. Since each thread stores the data
of the sliding window, running a small number of threads
consumes most of the on-chip memory in one thread block.
Therefore, the number of threads running per thread block is
small. Furthermore, the window update is inefficient because
it is not parallelized.
In the proposed implementation, one thread block is allo-
cated for each image column. Each thread block has the A strategy for multi-threaded processing is to divide an
same number of threads as the width of a local window. image into vertical, horizontal, or both partitions and assign
When a local window slides, each thread loads a pixel in each thread to process each partial image. Note that this
parallel and updates F, G, f ↓ and g ↓ in parallel using atomic strategy incurs the overhead of initializing the histograms of
operations. The median tracking process is performed by one a 2D window and column windows for each partial image.
of the threads in a thread block. The pseudocode for the In our implementation, we simply divide an image into T
GPU kernel is shown in Algorithm 6. Except when using vertical partitions, where T is the number of threads used. The
a small filter radius, this implementation uses more threads optimal division method and number depend on the image size,
than the implementation of allocating one thread for each window size, CPU cache size, etc. Consideration of a more
image column. In practical applications, this implementation appropriate division is beyond the scope of this paper.
is effective because a large filter radius is often used.
D. O(1) Sliding Window on CPU for High Precision Data
C. CPU Implementation With O(1) Sliding With high precision data, the O(1) sliding window approach
Window Approach is used on CPU with the linked list-based histogram as
described in Sec. V-C. This implementation searches for a
Unlike the O(r ) sliding window on GPUs, we compute median from f min instead of using median tracking because
the weighted median while computing cx and dx using OP- it takes time to find specific elements from a linked list.
SAT, because both computations are one-pass algorithms. For SIMD operations are not used because they cannot update
multichannel input, cx and dx are saved in the first channel the histogram for the linked list effectively. However, multi-
processing and loaded in the subsequent channels processing threaded processing is used to process each partial image in
without recalculating cx and dx . parallel.
For high-speed computation, we use SIMD operations and
multi-threaded processing. Since the histogram update is sim-
VII. E XPERIMENTATION
ply the addition and subtraction of elements between the
same bins of histograms, it can be accelerated by using A. Experimental Setup
SIMD operations. In our implementation, we use Intel AVX2 The specification of the computer conducting the experi-
instructions, which have 256-bit vector integer instructions. ment was a Core i7-8700 @ 3.20 GHz CPU with 6 cores
We assign 32 bits to each bin of a histogram, so we can update (12 threads) and 64 GB of RAM, and an NVIDIA GeForce
8 bins in parallel with a single instruction. GTX 1060 @ 1.51 GHz graphics card with 1152 CUDA cores
MISHIBA: FAST GUIDED MEDIAN FILTER 745

and 3 GB of RAM. We used the three types of the proposed TABLE I

methods shown in Sec. VI. We name each implementation as AVERAGE MSE (×100) ON N OISE R EDUCTION FOR D ISPARITY I MAGE
follows: GPU-O(r ) (Sec. VI-B), CPU-O(1) (Sec. VI-C), and
List-O(1) (Sec. VI-D). The proposed method was compared
with the following conventional methods: The UWM filter,
the constant time WM filter (CT-O(1)) [14] and a hundred
times or faster WM filter (FWM-O(r )) [17]. All methods
were implemented in the C++ programming language. The
OpenMP application programming interface was used for
parallel computation of the CPU implementation. The CUDA
toolkit was used for the GPU implementation. CT-O(1) was
coded using CUDA by the author. It employs the multipoint
guided filter [12] for calculating weights. The code of FWM-
O(r ) is provided by the author and is slightly modified for
parallelization using OpenMP. The default parameters that
were set in the code of FWM-O(r ) were used. I.e., the weights
were calculated as
!
(gx − g y )2
wx,y = exp − , (33)
2σ 2
where σ is a parameter.
Fig. 3. Zoomed up of denoising results of estimated disparity in dataset table.
B. Noise Reduction Performance (a) Estimated disparity in initial estimation process of [36]. (b) UWM filter.
(c) FWM-O(r ) filter. (d) CT-O(1) filter. (e) GPU-O(r ) filter. (f) Ground
In this experiment, noise reduction performance is verified. truth.
One scenario in which WM filters are used is to remove noise
contained in a disparity image using a color image as a guide
image. We performed denoising of a disparity image estimated
from a 4D light field image. We used 16 scenes in the
Additional category of the dataset [35]. Each scene consisted
of a 4D color light field image with a spatial resolution of
512×512 and an angular resolution (the number of viewpoints)
of 9×9. First, an 8-bit disparity image of the central viewpoint
was estimated using the initial estimation process in the
disparity estimation method [36]. Next, we denoised the initial Fig. 4. Examples of test images. (a) Color image. (b) Depth image.
estimation result using the GPU-O(r ) filter with the color
image of the central viewpoint of the light field image as a
guide image. Finally, the mean squared errors (MSEs) were C. Computation Time of Natural Images
obtained using the ground truth of the disparity image provided In this experiment, the computation time of each method
in the dataset. The filters and their parameters used in the was evaluated for grayscale and color images (8 bits/channel)
experiment were as follows: The UWM filter with rs = 14, with different image sizes and window sizes. The dataset [37]
the FWM-O(r ) filter with rs = 16 and σ = 25.5, the CT- was used. It contained high resolution natural color images
O(1) filter with rs = 24 and ϵ = 0.512 , and the GPU-O(r ) of assorted sizes. Thirty images of size 2040 × 1356 were
filter with rs = 18 and ϵ = 2.552 . Here rs was the spatial selected, resized, and converted to grayscale for the experi-
radius of a filter window. For a fair comparison, parameters ment. Figure 4 (a) shows one of the test images.
were adjusted for the best results in terms of MSEs. The computation times for three different image sizes
The MSEs are shown in Table I. The WM filters showed (640 × 480, 1024 × 768, and 1920 × 1080) with window
higher denoising performance compared to the UWM filter. size 15 are shown in Table II. The CPU-O(1) filter and
The CT-O(1) and GPU-O(r ) filters, which used kernels based GPU-O(r ) filter were clearly faster than the conventional
on the guided filter, showed higher denoising performance than methods. List-O(1) was suitable for high precision data and
the FWM-O(r ) filter, which used the Gaussian kernel based on not for 8-bit natural images, but it was competitive with the
color/intensity distance. Fig. 3 shows how the CT-O(1) filter FWM-O(r ) filter and faster than the CT-O(1) filter. For color
and the GPU-O(r ) filter helped recover the structural details. images, the computation times of all methods increased signifi-
Compared to GPU-O(r ), the CT-O(1) filter showed higher cantly because the guide images were also color images. When
performance. This may be due to the CT-O(1) filter using the FWM-O(r ) filter uses a color guide image, the image
the kernel based on multipoint modeling, while the GPU-O(r ) requires quantization into 256 different values, which increases
filter used the kernel based on pointwise modeling. Multipoint the computation time as overhead. The significant increase
modeling has better estimation performance than pointwise in computation time of the CT-O(1) filter and our proposed
modeling because of its redundant estimation [31]. methods, both of which are the guided filter-based methods,
746 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

TABLE II
AVERAGE C OMPUTATION T IMES ( IN M ILLISECONDS ) W ITH R ESPECT TO
D IFFERENT I MAGE S IZES FOR G RAYSCALE AND C OLOR I MAGES

was due to the inverse matrix operations shown in (28). Given

these reasons, despite the difficulty of efficient computation
for large color images, the GPU-O(r ) filter achieved near
real-time processing speed. Of particular emphasis is the fact
that both the CT-O(1) and GPU-O(r ) filters use kernels based
on the guided filter with high noise reduction performance,
but the GPU-O(r ) filter is more than 50 times faster than the
CT-O(1) filter.
The computation times for images of size 1920 × 1080 with
different window sizes are shown in Fig. 5. The computation
times of the CT-O(1) and CPU-O(1) filters were nearly
constant for increasing the window size. The slight increase
in computation time with increasing window size in the CPU-
O(1) filter is due to the increase in overhead caused by
parallelization. The computation time of the List-O(1) filter
increased as the window size increased. This is because the
length of the linked list tends to become longer as the window Fig. 5. Average computation times with respect to different window radii
for (a) grayscale and (b) color images.
size increases, as mentioned in Sec. V-C. The computation
time of the FWM-O(r ) and GPU-O(r ) filters increased as
the window size increased. Despite O(r ) time algorithm, the
GPU-O(r ) filter was faster than other O(1) time algorithms.

D. Application to High Precision Data

In this experiment, we verify the behavior of our proposed
methods for high precision data. We used 16-bit depth images
in the dataset [38]. Thirty images of size 1280 × 720 were
selected. To measure the computation time for different bits,
we created 8-, 10-, 12-, and 14-bit data by quantizing the
original data. Figure 4 (b) shows one of the test images. The
window size was set to 15.
The computation times with different bit depths are shown
in Fig. 6. The GPU-O(r ) filter was not able to manage 14-bit
or higher data because the histograms could not be stored on
the shared memory of the GPU. The CPU-O(1) filter was able Fig. 6. Average computation times for each precision.
to manage 16-bit data. Its computation time for 16-bit data was
about 600 times longer than one in the List-O(1) filter. Most
of the computation time was spent on adding and subtracting does not depend on the distribution of the histogram for its
histograms with 216 bins. The List-O(1) filter operated at near computation speed.
real-time speed for 16-bit data. Unlike the results for 8-bit
grayscale images in Sec. VII-C, the computation time of the
E. Application to Multichannel Images
List-O(1) filter was shorter than that of the CPU-O(1) filter
for the 8-bit data in this experiment. Because the histogram In this experiment, the behavior of the proposed methods
distribution of the depth data used in the experiment was for multichannel data is verified. The University of Pavia
sparse, the linked list of the depth data was shorter than that dataset [39] was used, which is a multispectral image of
of natural images. The List-O(1) filter runs faster the sparser size 610 × 340 with 103 bands. Salt-and-pepper noise with
the distribution of the histogram is, while the CPU-O(1) filter a probability of 0.05 was added to each channel.
MISHIBA: FAST GUIDED MEDIAN FILTER 747

TABLE III
AVERAGE PSNR S OF I MAGES F ILTERED W ITH D IFFERENT C HANNEL R ADII rc ON N OISE R EDUCTION FOR M ULTISPECTRAL I MAGE

TABLE IV
AVERAGE MSE S OF I MAGES F ILTERED W ITH D IFFERENT A NGULAR R ADII ra ON D ISPARITY R EFINEMENT FOR L IGHT F IELD I MAGE

We performed multispectral image denoising using the

GPU-O(r ) filter and varied the number of channels as a guide
image. The number of channels to be used as a guide image
was determined by the channel radius rc . Filtering for the
i-th channel with rc used channels from i − rc to i + rc
except i as a guide image. The number of channels of the
guide image to be used was expressed as 2rc . The reason
why the i-th channel was not used for the guide image was
that if the guide image was the same as the input image,
it would have been difficult to efficiently remove noise in
the input image. As described in Sec. V-B, ϵ was adjusted Fig. 7. Denoising results of a multispectral image (band 80). (a) Original
image. (b) Noisy image. (c) UWM filter. (d) FWM-O(r ) filter. (e) CT-O(1)
for the number of channels in the guide image as follows: filter. (f) GPU-O(r ) filter with single channel guide. (g), (h), (i) and (j) are
ϵ = 2 rc ϵ̂. We set the spatial window radius to 1 and ϵ̂ = 552 . GPU-O(r ) filter with rc = 1, 3, 5 and 7, respectively.
For comparison, the following methods were applied to each
channel for denoising: The UWM filter with rs = 1, the FWM-
O(r ) filter with rs = 1 and σ = 510, the CT-O(1) filter with
rs = 1 and ϵ = 2552 , and the GPU-O(r ) filter with rs = 1 and
ϵ = 5102 . Parameters were adjusted for best results in terms
of the average peak signal-to-noise ratios (PSNRs).
The PSNRs of the filtered images are shown in Table III.
Fig. 8. Noisy input channels. (a) Band 78. (b) Band 79. (c) Band 80.
The FWM-O(r ), CT-O(1) and GPU-O(r ) filters gave the best (d) Band 81. (e) Band 82.
results using large σ and ϵ. If the pixel of interest is affected by
noise, the weights computed from that pixel result in improper
smoothing. While the use of large σ and ϵ reduces the effect
in the experiments in Sec. VII-B was used. Disparities of
of noise in the guide image, their kernels approach the mean
all viewpoint estimates were obtained by using the initial
filter kernel. As a result, they are almost identical to the UWM
estimation process of the disparity estimation method [36].
filter. As can be seen from (13), the multipoint guided filter
The disparities of all viewpoints could be regarded as 8-bit
kernel used in the CT-O(1) filter considers overlapping local
4D data of size 512 × 512 × 9 × 9. Next, the initial estimation
windows. Therefore, the range of pixels used to calculate the
results were denoised using the GPU-O(r ) filter using the 4D
output is 2rs . This leads to a lower PSNR for the CT-O(1)
color light field image as a guide image. The angular radius
filter with rs = 1 compared to the UWM filter with rs = 1.
ra of the filter varied from 0 to 4. An ra = 0 meant that the
As shown in Fig. 7, these methods degrade luminance changes
disparity of each viewpoint was filtered independently. I.e.,
that are not noise in the subject. In the GPU-O(r ) filter with a
it is equivalent to applying a 2D filter instead of a 4D filter.
multichannel guide, the use of more channels achieved better
The spatial window radius was set to 20 and ϵ̂ = 2.552 . For
performance. Figure 8 shows the input channels of bands 78 to
comparison, the following methods were applied to each view
82. The luminance changes that the subject has commonly
for denoising: The UWM filter with rs = 14, the FWM-O(r )
appear in multiple channels at the same coordinates, while the
filter with rs = 16 and σ = 25.5, and the CT-O(1) filter with
luminance changes due to noise appear only in some channels
rs = 24 and ϵ = 2.552 . The parameters were adjusted for the
at the same coordinates. The use of multiple channels may lead
best results in terms of MSEs.
to appropriate weights based on similarities between channels.
The MSEs of filtered images with different angular radii are
shown in Table IV. Figure 9 shows the refinement results of
F. Application to Multidimensional Data the upper left viewpoint. As can be seen by comparing the
In this experiment, the behavior of the proposed methods result of ra = 0 with the other results, using the 4D filter
is verified for multidimensional data. Denoising of a 4D instead of the 2D filter (ra = 0) significantly improved the
light field image was performed. The same dataset used refinement performance.
748 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 32, 2023

Fig. 9. The denoising results for the upper left viewpoint of estimated disparity in the dataset antinous. (a) The estimated disparity in the initial estimation
process of [36]. (b) UWM filter. (c) FWM-O(r ) filter. (d) CT-O(1) filter. (e) GPU-O(r ) filter with ra = 0. (f) GPU-O(r ) filter with ra = 4. (g) Ground truth
disparity.

VIII. C ONCLUSION [11] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and

M. Gelautz, “Fast cost-volume filtering for visual correspondence
In this paper, we proposed a novel fast WM filter with the and beyond,” in Proc. IEEE Conf. (CVPR), Jun. 2011,
pointwise guided filter kernel. The proposed method can effi- pp. 3017–3024.
ciently update the weighted histogram, which is the bottleneck [12] K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397–1409,
in the real-time processing of WM filters. Fast computation Jun. 2013.
is achieved by combining a sliding window approach with [13] D. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estimation and
a median tracking technique. While conventional methods their principles,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
struggle to handle multidimensional, multichannel, and high Jun. 2010, pp. 2432–2439.
[14] Z. Ma, K. He, Y. Wei, J. Sun, and E. Wu, “Constant time weighted
precision images, the proposed method can manage them median filtering for stereo matching and beyond,” in Proc. IEEE Int.
without data reduction. The experimentation results showed Conf. Comput. Vis., Dec. 2013, pp. 49–56.
that the proposed method realizes faster computation than [15] W. Wu, L. Li, and W. Jin, “Disparity refinement based on segment-tree
conventional WM filters. Furthermore, the proposed method, and fast weighted median filter,” in Proc. IEEE Int. Conf. Image Process.
(ICIP), Sep. 2016, pp. 3449–3453.
which calculates weights based on the guided filter kernel, [16] Z. Shi, Y. Li, C. Zhang, M. Zhao, Y. Feng, and B. Jiang, “Weighted
has a higher noise reduction performance than the conven- median guided filtering method for single image rain removal,”
tional method which calculates weights using the Gaussian EURASIP J. Image Video Process., vol. 2018, no. 1, pp. 1–8,
May 2018.
kernel.
[17] Q. Zhang, L. Xu, and J. Jia, “100+ times faster weighted median
filter (WMF),” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
R EFERENCES Jun. 2014, pp. 2830–2837.
[18] T. Huang, G. Yang, and G. Tang, “A fast two-dimensional median
[1] M. Hilts and C. Duzenli, “Image filtering for improved dose resolution filtering algorithm,” IEEE Trans. Acoust., Speech, Signal Process.,
in CT polymer gel dosimetry,” Med. Phys., vol. 31, no. 1, pp. 39–49, vol. ASSP-27, no. 1, pp. 13–18, Feb. 1979.
Jan. 2004.
[19] T.-S. Le, N.-T. Do, and K. Hamamoto, “Speed up temporal median
[2] C. P. Behrenbruch, S. Petroudi, S. Bond, J. D. Declerck, F. J. Leong,
filter and its application in background estimation,” in Proc. IEEE RIVF
and J. M. Brady, “Image filtering techniques for medical image post-
Int. Conf. Comput. Commun. Technol., Res., Innov., Vis. Future (RIVF),
processing: An overview,” Brit. J. Radiol., vol. 77, pp. 126–132,
Nov. 2016, pp. 175–180.
Dec. 2004.
[3] S. Gan, S. Wang, Y. Chen, X. Chen, and K. Xiang, “Separa- [20] D. Cline, K. B. White, and P. K. Egbert, “Fast 8-Bit median filter-
tion of simultaneous sources using a structural-oriented median filter ing based on separability,” in Proc. IEEE Int. Conf. Image Process.,
in the flattened dimension,” Comput. Geosci., vol. 86, pp. 46–54, Sep. 2007, p. 281.
Sep. 2016. [21] S. Perreault and P. Hebert, “Median filtering in constant time,”
[4] Y. Chen, S. Zu, Y. Wang, and X. Chen, “Deblending of IEEE Trans. Image Process., vol. 16, no. 9, pp. 2389–2394,
simultaneous source data using a structure-oriented space-varying Sep. 2007.
median filter,” Geophys. J. Int., vol. 222, no. 3, pp. 1805–1823, [22] M. Storath and A. Weinmann, “Fast median filtering for phase or
Jun. 2020. orientation data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3,
[5] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and pp. 639–652, Mar. 2018.
color images,” in Proc. 6th Int. Conf. Comput. Vis., Jan. 1998, [23] B. B. Chaudhuri, “An efficient algorithm for running window pel gray
pp. 839–846. level ranking in 2-D images,” Pattern Recognit. Lett., vol. 11, no. 2,
[6] G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, pp. 77–80, Feb. 1990.
and K. Toyama, “Digital photography with flash and no-flash [24] R. M. Sánchez and P. A. Rodríguez, “Bidimensional median
image pairs,” ACM Trans. Graph., vol. 23, no. 3, pp. 664–672, filter for parallel computing architectures,” in Proc. IEEE Int.
Aug. 2004. Conf. Acoust., Speech Signal Process. (ICASSP), Mar. 2012,
[7] J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, “Joint pp. 1549–1552.
bilateral upsampling,” ACM Trans. Graph., vol. 26, no. 3, p. 96, [25] A. Adams, “Fast median filters using separable sorting networks,” ACM
Jul. 2007. Trans. Graph., vol. 40, no. 4, pp. 1–11, Aug. 2021.
[8] I. Eichhardt, D. Chetverikov, and Z. Janko, “Image-guided ToF depth [26] H. Zhao, D. Gao, M. Wang, and Z. Pan, “Real-time edge-aware weighted
upsampling: A survey,” Mach. Vis. Appl., vol. 28, nos. 3–4, pp. 267–282, median filtering on the GPU,” Comput. Graph., vol. 61, pp. 11–18,
2017. Dec. 2016.
[9] K.-J. Yoon and I. S. Kweon, “Adaptive support-weight approach for [27] J. Chen, S. Paris, and F. Durand, “Real-time edge-aware image process-
correspondence search,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, ing with the bilateral grid,” ACM Trans. Graph., vol. 26, no. 3, p. 103,
no. 4, pp. 650–656, Apr. 2006. Jul. 2007.
[10] Q. Yang, “Stereo matching using tree filtering,” IEEE Trans. [28] B. I. Justusson, “Median filtering: Statistical properties,” in Two-
Pattern Anal. Mach. Intell., vol. 37, no. 4, pp. 834–846, Dimensional Digital Signal Prcessing II (Topics in Applied Physics).
Apr. 2015. Berlin, Germany: Springer, 1981, pp. 161–196.
MISHIBA: FAST GUIDED MEDIAN FILTER 749

[29] J. Nieweglowski, M. Gabbouj, and Y. Neuvo, “Weighted medians— [38] A. Clapes, J. C. S. J. Junior, C. Morral, and S. Escalera, “ChaLearn
Positive Boolean functions conversion algorithms,” Signal Process., LAP 2020 challenge on identity-preserved human detection: Dataset and
vol. 34, no. 2, pp. 149–161, Nov. 1993. results,” in Proc. 15th IEEE Int. Conf. Autom. Face Gesture Recognit.
[30] G. R. Arce, “A general weighted median filter structure admitting (FG), Nov. 2020, pp. 801–808.
negative weights,” IEEE Trans. Signal Process., vol. 46, no. 12, [39] University of Pavia Dataset. Accessed: Mar. 17, 2022.
pp. 3195–3205, Dec. 1998. [Online]. Available: http://www.ehu.ews/ccwintoco/index.php?title=
[31] V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, “From local kernel to Hypespectral_Remote_Sensing_Scenes
nonlocal multiple-model image denoising,” Int. J. Comput. Vis., vol. 86,
no. 1, pp. 1–32, Jul. 2009.
[32] N. Fukushima et al., “Efficient computational scheduling of box and
Gaussian FIR filtering for CPU microarchitecture,” in Proc. Asia–Pacific
Signal Inf. Process. Assoc. Annu. Summit Conf., Nov. 2018, pp. 875–879.
[33] B. Weiss, “Fast median and bilateral filtering,” ACM Trans. Graph.,
vol. 25, no. 3, pp. 519–526, 2006.
[34] O. Green, “Efficient scalable median filtering using histogram-based
operations,” IEEE Trans. Image Process., vol. 27, no. 5, pp. 2217–2228,
May 2018. Kazu Mishiba (Member, IEEE) received the B.E.,
[35] K. Honauer, O. Johannsen, D. Kondermann, and B. Goldluecke, M.E., and Ph.D. degrees from Keio University,
“A dataset and evaluation methodology for depth estimation on 4D Yokohama, Japan, in 2004, 2006, and 2011, respec-
light fields,” in Proc. Asian Conf. Comput. Vis., Taipei, Taiwan, 2016, tively. In 2006, he joined FUJIFILM Company Ltd.
pp. 19–34. He became an Assistant Professor at Keio University
[36] K. Mishiba, “Fast depth estimation for light field cameras,” IEEE Trans. in 2011. He is currently an Associate Professor
Image Process., vol. 29, pp. 4232–4242, 2020. at Tottori University. His research interests include
[37] E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image image processing and computer vision.
super-resolution: Dataset and study,” in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit. Workshops (CVPRW), Jul. 2017, pp. 1122–1131.