3
current states (user actions, objects states, etc.) on the newly
connected Client.
3) Social VR Flows: Next, we identify two flows to de-
scribe how the Social VR Plugin exchanges data with both the
Asset Server and Proxy Server to achieve social VR workflow.
Asset Exchange Flow: This flow carries both VR-required
3D models and panoramic videos that are exchanged between
the Asset Client and Asset Server. Considering their applicabil-
ity and functionalities, the two different assets require different
network services.
VR-required 3D models, no matter the 3D environment
or 3D avatar model, must be prepared in advance to ensure
that they can be rendered correctly during runtime. Thus,
prior to enjoying social VR, the Asset Client always requests
the nearest edge servers to aggressively download necessary
models. Particularly, Asset Clients who are willing to share
self-built avatars or environments must further upload their
own models to the Asset Server.
When referring to panoramic videos, it requires social VR to
provide video-streaming services to various the Clients in real
time. Serving as the streaming client, the Asset Client requests
and retrieves panoramic video clips from the Asset Server.
However, for live streaming service, one of the Asset Clients
would become the live source with necessary equipment (e.g.
panoramic camera) facilitated.
Social Sync Flow: This flow is designed for transferring
rendering commands to synchronize user actions across the
network. As discussed previously, all avatars and objects are
managed by Control Servers so that rendering commands
cannot be directly exchanged between Clients. Thus, for a
typical user action sync flow, we propose to adopt a server-
authoritative communication model. The Social Syncer opens
persistent connections to the Control Server to send and
receive rendering commands. Rendering commands that are
always serialized from local Clients will be invoked on the
Control Server’s corresponding avatar instance first. Then, the
Control Server will invoke such commands on all connected
Clients to synchronize user actions.
However, VR’s strict latency requirement make it unsuitable
to employ the server-authoritative model for delay-sensitive
actions like head rotations and position movements. Compared
with ordinary actions like hand gestures, delay-sensitive ac-
tions directly change most parts of the VR final frame that
account for motion-to-photon latency. Previous research [3]
indicates that even minor motion-to-photon latency (larger
than 25 ms) can induce motion sickness. It is obvious that
network latency introduced server-authoritative model can
lead to significant motion-to-photon latency that makes users
uncomfortable. To hide the network latency, delay-sensitive
actions should be invoked first on local Clients and then
forwarded to Control Servers and other Clients.
III. IMPLEMENTATION AND EXPERIMENTS
A. Implementation
We implement MSVR as a client/server prototype to eval-
uate its basic performance. We leverage the Google VR SDK
to implement a Unity-based client to provide an immersive
VR experience and achieve the avatar-controlling feature that
renders avatars and actions by tracking user inputs from both
the headset and controller.
To realize the scene-sharing feature, we implement the asset
client as a GameObject in Unity. If the scene is rendered from
3D model, we build the model as an AssetBundle in Unity. In
this scenario, the asset client only needs to download corre-
sponding AssetBundles from an HTTP server and loads them
on demand. If the scene is decoded from panoramic videos,
we use the GvrVideoPlayerTexture (provided by Google VR
SDK) to stream panoramic videos and render them on the
inner surface of a large sphere texture to create the 360-degree
playback effect.
As for the social-interaction feature, we exploit Pomelo
framework
5
to achieve the social sync flow. The social syncer
is also implemented as a GameObject in Unity that opens
persistent connections to NodeJS servers using Pomelo API.
B. Microbenchmark
We first evaluate the MSVR’s system performance and
overhead. To further understand the architecture better, we
analyze the performance of each key feature (i.e. avatar-
controlling, scene-sharing, and social-interaction). We run the
MSVR client on the latest Google high-end smartphone PIX-
EL, which runs Android 7.1.2 with Google Daydream enabled.
The client communicates with both asset servers and control
servers via an 802.11b/n/g AP.
To verify how the scene complexity would affect the per-
formance, we build three scenes: Room
6
, City
7
and Nature
8
with different complexities in MSVR by importing open-
source 3D models from Unity Asset Store. We use BRISQUE
[4], a well-known non-reference image quality access metric
ranging from 0 to 100, to identify scene complexity. A
lower BRISQUE value indicates a higher scene complexity.
Since the scene decoded from panoramic video is currently
equivalent as video streaming playback that does not present
VR-specific performance, we do not evaluate such scenes in
our microbenchmark.
1) System performance: We first evaluate the basic VR-
specific and end-to-end performance of the MSVR.
Rotation Latency: The VR headset’s near-eye display and
immersive experience make users more sensitive to any visual
lag during watching. Thus, the VR system asks for extremely
low motion-to-photon latency, (less than 25ms is recom-
mended) to provide a comfortable user experience. Note that
existing Daydream headsets only track users’ head rotation
as motion input. We define and measure the rotation latency
as the elapsed time from when the latest head rotation is
received to the final frame is ready to be displayed. We utilize
smartphone’s rotation to simulate the user’s head rotation.
Table I lists different scenes’ average rotation latencies when
rotating around 45 degrees. We observe that higher scene
complexity contributes to longer rotation latency. Considering
5
http://pomelo.netease.com/
6
https://www.assetstore.unity3d.com/en/#!/content/85732
7
https://www.assetstore.unity3d.com/en/#!/content/65679
8
https://www.assetstore.unity3d.com/en/#!/content/52977