USER-ORIENTED STEREO VIDEO REFOCUSING BY COMPUTATIONAL
CINEMATOGRAPHIC MODEL
Wenjing Geng, Dapeng Du, Tongwei Ren and Gangshan Wu
State Key Laboratory for Novel Software Technology
Nanjing University, Nanjing 210023, China
jenngeng@gmail.com, dudp.nju@gmail.com, {rentw, gswu}@nju.edu.cn
ABSTRACT
Refocusing, as a most popular photographic technique, is
widely welcomed in both photography and cinematography.
Currently, most refocusing effects in movies are implemented
manually using professional devices or computer graphics
methods. Few solutions are designed for the videos captured
by common users. In this paper, we propose a user-oriented
method to facilitate the video refocusing for daily life, with
an extra requirement for only stereo cameras which are quite
popular today. Given a stereo video, one can select the
desired focusing part in a single frame, and the user selected
intention will be tracked along with the timeline. Then the
depth-of-field (DoF) based blurring effect can be generated
by scene depth estimation based on the stereo techniques
and the proposed cinematographic models in computational
photography. The performance of the refocused videos
compares favourably with the ones generated by digital single
lens reflex (DSLR). To evaluate the proposed method, we
conduct the experiments on several challenging stereo video
datasets. The quantitative and subjective evaluations both
show that the proposed method can achieve attractive and
highly aesthetic performance with few user interactions.
Index Terms— Photography, video refocusing, stereo
camera, computational cinematographic model
1. INTRODUCTION
Knowing how to make the parts of your image you want
sharp and the parts you want to be out of focus, is a great
artistic tool to create great images, as well as videos [1,
2]. The emergence on stereoscopic equipment breeds large
amount of stereo media in recent years. With the increasing
binocular handheld devices, such as Fujifilm 3D camera,
Huawei honor and the upcoming dual-lens smartphones, 3D
videos have quietly stepped into people’s daily life, which
is no longer a kind of media living in the cinemas or 3D
TVs. Precisely because of the widely use of stereo devices,
it is a fair chance of achieving computational photography
without the help of professional camera devices, such as
DSLR. It is noted that, refocusing, as a most popular
…
…
Timeline for user interaction
Fig. 1. Interactive video refocusing facilitated to ordinary
users. From top to down are the stereoscopic videos, user
multiple interactions along with the video timeline, and the
refocused results generated by the proposed method.
photographic and cinematographic technique, is pursued by
many shutterbugs and is also applied in most movies to create
romantic and aesthetic atmosphere [3]. However, depending
on the professional equipment and specialized knowledge
would make video refocusing extravagant for ordinary users.
Inspired by building a bridge between the refocusing and the
binocular video captured by widely used 3D equipment, and
making 3D videos much more vivid and interesting, a user-
oriented stereo video refocusing framework is proposed in
this paper aiming at generating attractive refocusing results
specified by user intention and re-edited by the presented
cinematographic model. A user can also perform multiple
interactions to select different focus planes for each frame to
create the rack focusing effect, as shown in Fig. 1.
Some previous works involving video refocusing [4, 5]
have achieved satisfactory refocusing results by designing
customized equipment for acquiring depth more accurately.
The extra overhead of setting hardware limits the popu-
larization of these techniques and generates bottlenecks.
Besides, the categories of the experimental videos are less
than abundance and detached from the shoot scenes of life.
Some other methods [6, 7] calculated depth by utilizing
the parameters of the cameras or the smartphones, and the
movements during the acquisition of the video. These
approaches suffer from the disadvantages of stipulating
restricted conditions. Furthermore, the refocused continuity
should also be considered in the scheme of video refocusing,