DE PESSEMIER et al.: REBUFFERING INTERRUPTIONS USER’S QOE DURING MOBILE VIDEO WATCHING 49
In the context of heterogeneous mobility, it is also inves-
tigated how network hand-overs can be optimized and made
seamless, allowing the user to have the best possible experience.
To improve this handover process across multiple link-lay er
access technologies, a modified Android user terminal using
the IEEE 802.2 1 framework has been proposed [20]. The as-
sessment of the handover process via an experim ental test bed
showed th at under the proposed solution the handover delay and
packet loss are significantly lower than th e ones resulting from
the normal operation.
Besides, various researchers have s tudied the QoE of mobile
media sessions in cellular d ata networks. Based on experim ents
in WCDMA networks, a predictive QoE model for multimedia
applications was proposed [21]. A stepwise regression analysis
revealed the m ost relevant factors for the QoE: the number of
transmission errors, b uffering occurrences, and coding profile.
Moreover the study pointed t o the importance of the buffering
duration and frequency. Exp eriments in UMTS networks found
that the effect of the RTT (round-trip time) and bandwidth are
very perceivable by the users while browsing web pages [22].
The same study showed that the i nitial startup time of streaming
video is crucial, independent of the quality of the streaming. The
test subjects were also very sensitive to any rebuffering that oc-
curs after the streaming has started, and rated the overall quality
regardless of the video quality after the rebuffering interruption.
Other studies considered the rebuffering length and re-
buffering frequency as the properties that have the greatest
impact on QoE. If interruption is unavoidable, a single re-
buffering is a better solution than repeated rebuffering events
[23]. Other subjective tests showed that also a single rebuffering
interruption can reduce the users’ QoE considerably [24]. A
very recent study compared the impact of initial delays vs. in-
terruptions and found that the latter should always be avoided,
even at the cost of increasing the initial waiting time due to
prebuffering [25]. Regrettably, these studies do not evaluate
how much time can be spent on the rebuffering of m obile
video before this becomes unacceptable for the user, or in other
words the acceptability regarding the initial loading time and
the rebuffering interruptions during video playback. In t his
paper, we therefore investigate the acceptability of rebuffering
interruptions during mobile video watching and provide a
model that estim ates this acceptability considering the initial
loading ti me and rebuffering interruptions.
To evaluate Q oE in the context of mo bile applications o r ser-
vices, both traditional test beds with controlled pa ram eters or
living lab exper iments in the fi eld can be set up . Test beds allow
for transparent, rigorous, and replicable testing of new tech-
nologies, scientific theories, and tools regarding the quantifica-
tion and optimization of the QoE. Co nversely, living lab exper-
iments are less transparent and predefined but aim to provide
more natural settings fo r studying QoE by in volvin g the users
in the innovation process [26]. Although these living lab exper-
iments are an extension towards more n atural and realistic re-
search test environments [27], a stro ng tradition ex ists in exper-
imental research taking place in controlled laboratory settings.
Research using this kind of test beds makes it possible to inves-
tigate the relative i nfluence of particular isolated parameters on
users’ quality perceptions. Therefore, the test presented in this
paper has been carried out in such a controlled environm ent test
room.
III. T
EST SETUP
A. Aims of the Study
The main aim of this study is to investigate the influence of re-
buffering in terrup tions on Quality of Experience during mobile
video watching. More concretely, we investigate if and how the
test subjects’ QoE is influenced by the number of reb uffering in -
terruptions in six t echnical scenarios com binin g three simulated
connection types and two video qualities. We investigate the in-
fluence of t he objective measu res mentioned in Table III on dif-
ferent m easures of QoE, as dependent variables in our study.
These include the overall experi ence rating, and the evaluation
of both the overall technical quality, as well as specificQoE
features, being interruptions, loading time, and fluidity (some-
times also r efer red to as fluentness or smoothness of the video
playback). By the term QoE feature, we refer to ‘a perceivable,
recognized and namable characteristic of the individual’s expe-
rience of a service which contributes to its quality ’ [28]. We
included these specific features to investigate their relative im-
portance and concrete evaluation by test subjects and because
previous research pointed to their importance. Finally, we inves-
tigate possible differences in term s of the acceptability of video
playback interruption s due to rebufferings. Following the defi-
nition given in [29], acceptability refers to a ‘binary measure to
locate the threshold of minimum acceptable quality that fulfills
user quality expectations and needs for a certain application or
system’. In addition, we also take into account the importance
of speci fic QoE features related to mobile video watching be-
fore and after the actual test. Furtherm ore, we complem ent the
test subjcts’ ratings with qualit ative feedback on expectations
and assum ed importance of features and influencing factors, in-
dicated by test subjects themselves. We now describe t he tech-
nical and experimental set-up in m ore d etail.
B. Procedure
The test consisted of three successive phases:
1) Phase 1: Pre-Questionnaire: Before the actual exper-
iment started, participants were asked to fill in a traditional
paper questionnaire consisting of closed and open questions.
The questionnaire inquired after their socio-demographic char-
acteristics, type and connection p ossibi lit ies of their cur ren t
mobile phone, and experien ces and habits (in term s of viewing
frequency, ranging from never to several times a day) regarding
the watching of video content on a mobile phone. Next, by
means of the first open question, the test subjects with prior
experience were asked to specify which characteristics (QoE
features) related to mobile videowatchingtheypersonally
consider to be essential for having a good experience. ‘Open’ in
this context means that n o pre-de fined answer categories were
given and that test subjects wereabletoexpressthemselvesin
their ow n words. Thereupon, the test subjects who had no prior
experience with mobile video watching were able to indicate
what according to them m ight influence their experience. By
means of this second open question, we wanted to gain more
insight in the test subjects’ exp ectations with regard to possible