100+ Times Faster Weighted Median Filter (WMF)
Qi Zhang
†
Li Xu
‡
Jiaya Jia
†
†
The Chinese University of Hong Kong
‡
Image & Visual Computing Lab, Lenovo R&T
zhangqi@cse.cuhk.edu.hk xulihk@lenovo.com leojia@cse.cuhk.edu.hk
http://www.cse.cuhk.edu.hk/leojia/projects/fastwmedian/
Abstract
Weighted median, in the form of either solver or filter,
has been employed in a wide range of computer vision solu-
tions for its beneficial properties in sparsity representation.
But it is hard to be accelerated due to the spatially varying
weight and the median property. We propose a few efficient
schemes to reduce computation complexity from O(r
2
) to
O(r) where r is the kernel size. Our contribution is on a
new joint-histogram representation, median tracking, and a
new data structure that enables fast data access. The effec-
tiveness of these schemes is demonstrated on optical flow
estimation, stereo matching, structure-texture separation,
image filtering, to name a few. The running time is largely
shortened from several minutes to less than 1 second. The
source code is provided in the project website.
1. Introduction
There is no need to explain how common and elementary
local filters are in computer vision. In this paper, we
study and retrofit a fundamentally useful tool, i.e., weighted
median, considering its importance along several major
lines of applications. As a local operator, weighted median
can effectively filter images while not strongly blurring
edges. More importantly, it mathematically corresponds to
global optimization [14], which produces resu lts with fully
explainable connections to global energy minimization.In
many computer vision problems, including stereo matching,
optical flow estimation, and image denoising, applying
local weighted median suitably is equivalent to employing
solvers that involve global L
1
sparse regularizers, which is
rather common now in sparsity pursuit in a lot of tasks.
Unweighted median filter was accelerated in previous
work. But it is almost impossible to apply these methods to
weighted median because of the spatially varying weights
for each local window. These varying weights hamper
simplifying computation in a straightforward way. This is
why many previous computer vision systems still rely on
a direct implementation of weighted median, which has to
spend several minutes even using small windows on a one-
megapixel image.
On the other hand, there have been several schemes to
speed up weighted average, such as bilateral filter [20, 16,
4, 2, 1, 9, 6, 24, 18], domain transform filter [8], and guided
filter [10]. Weighted median is different from them due to
a few special characteristics. 1) The filtering kernel is not
separable. 2) It cannot be approximated by interpolation o r
down-sampling. 3) There is no iterative solution. Taking
prior techniques for accelerating bilateral filter as an exam-
ple, Gaussian weights are generally assumed, which may
not be satisfied in weighted median.
Our contribution in this p aper lies on a few algorithms
to accelerate weighted median. The novelty attributes
to three main techniques making respective use of data-
weight distribu tion by a new joint histogram to dynamically
allocate weights and pixels, a median tracking algorithm
to reduce time of seeking median by leveraging color
coherency of images, and data structure sparsity to reduce
the data access cost. They are put together to construct a
practically workable system with O(r) complexity where
r is the kernel size. It is notable that this complexity is
only associated with a small constant, empirically effective
to shorten running time.
1.1. Related Work
Median Filter While there is little work accelerating
weighted median, simpler unweighted filter finds several
solutions. Huang [11] proposed a slidin g window approach
leveraging histograms to compute median in O(r) time,
which is further accelerated to O(log r) with the distributed
histograms [21]. Constant time median algorithms [17, 5]
were also proposed, focusing on reducing histogram update
complexity. Kass and Solomon [12] introduced another
constant time algorithm based on integral of smoothed local
histograms.
Two main issues make these methods not directly usable
2014 IEEE Conference on Computer Vision and Pattern Recognition
1063-6919/14 $31.00 © 2014 IEEE
DOI 10.1109/CVPR.2014.362
2824
2014 IEEE Conference on Computer Vision and Pattern Recognition
1063-6919/14 $31.00 © 2014 IEEE
DOI 10.1109/CVPR.2014.362
2830
2014 IEEE Conference on Computer Vision and Pattern Recognition
1063-6919/14 $31.00 © 2014 IEEE
DOI 10.1109/CVPR.2014.362
2830