Eulerian Video Magnification for Revealing Subtle Changes in the World
Hao-Yu Wu
1
Michael Rubinstein
1
Eugene Shih
2
John Guttag
1
Fr
´
edo Durand
1
William Freeman
1
1
MIT CSAIL
2
Quanta Research Cambridge, Inc.
(a) Input
(b) Magnified (c) Spatiotemporal YT slices
time
y
time
y
Figure 1: An example of using our Eulerian Video Magnification framework for visualizing the human pulse. (a) Four frames from the
original video sequence (face). (b) The same four frames with the subject’s pulse signal amplified. (c) A vertical scan line from the input (top)
and output (bottom) videos plotted over time shows how our method amplifies the periodic color variation. In the input sequence the signal
is imperceptible, but in the magnified sequence the variation is clear. The complete sequence is available in the supplemental video.
Abstract
Our goal is to reveal temporal variations in videos that are diffi-
cult or impossible to see with the naked eye and display them in
an indicative manner. Our method, which we call Eulerian Video
Magnification, takes a standard video sequence as input, and ap-
plies spatial decomposition, followed by temporal filtering to the
frames. The resulting signal is then amplified to reveal hidden in-
formation. Using our method, we are able to visualize the flow
of blood as it fills the face and also to amplify and reveal small
motions. Our technique can run in real time to show phenomena
occurring at temporal frequencies selected by the user.
CR Categories: I.4.7 [Image Processing and Computer Vision]:
Scene Analysis—Time-varying Imagery;
Keywords: video-based rendering, spatio-temporal analysis, Eu-
lerian motion, motion magnification
Links: DL PDF WEB
1 Introduction
The human visual system has limited spatio-temporal sensitivity,
but many signals that fall below this capacity can be informative.
For example, human skin color varies slightly with blood circu-
lation. This variation, while invisible to the naked eye, can be ex-
ploited to extract pulse rate [Verkruysse et al. 2008; Poh et al. 2010;
Philips 2011]. Similarly, motion with low spatial amplitude, while
hard or impossible for humans to see, can be magnified to reveal
interesting mechanical behavior [Liu et al. 2005]. The success of
these tools motivates the development of new techniques to reveal
invisible signals in videos. In this paper, we show that a combina-
tion of spatial and temporal processing of videos can amplify subtle
variations that reveal important aspects of the world around us.
Our basic approach is to consider the time series of color values at
any spatial location (pixel) and amplify variation in a given tempo-
ral frequency band of interest. For example, in Figure 1 we auto-
matically select, and then amplify, a band of temporal frequencies
that includes plausible human heart rates. The amplification reveals
the variation of redness as blood flows through the face. For this
application, temporal filtering needs to be applied to lower spatial
frequencies (spatial pooling) to allow such a subtle input signal to
rise above the camera sensor and quantization noise.
Our temporal filtering approach not only amplifies color variation,
but can also reveal low-amplitude motion. For example, in the sup-
plemental video, we show that we can enhance the subtle motions
around the chest of a breathing baby. We provide a mathematical
analysis that explains how temporal filtering interplays with spatial
motion in videos. Our analysis relies on a linear approximation re-
lated to the brightness constancy assumption used in optical flow
formulations. We also derive the conditions under which this ap-
proximation holds. This leads to a multiscale approach to magnify
motion without feature tracking or motion estimation.
Previous attempts have been made to unveil imperceptible motions
in videos. [Liu et al. 2005] analyze and amplify subtle motions and
visualize deformations that would otherwise be invisible. [Wang
et al. 2006] propose using the Cartoon Animation Filter to create
perceptually appealing motion exaggeration. These approaches fol-
low a Lagrangian perspective, in reference to fluid dynamics where
the trajectory of particles is tracked over time. As such, they rely