The
Journal
of
Systems
and
Software
86 (2013) 367–
376
Contents
lists
available
at
SciVerse
ScienceDirect
The
Journal
of
Systems
and
Software
jo
u
rn
al
hom
epage:
www.elsevier.com/locate/jss
A
high
performance
inter-domain
communication
approach
for
virtual
machines
Yuebin
Bai
a,c,∗
, Yao
Ma
a
, Cheng
Luo
b
, Duo
Lv
a
,
Yuanfeng
Peng
a
a
State
Key
Laboratory
of
Software
Development
Environment,
Beihang
University,
Beijing
100191,
China
b
Department
of
Computer
Science,
The
University
of
Tokyo,
Tokyo,
Japan
c
Science
and
Technology
on
Information
Transmission
and
Dissemination
in
Communication
Networks
Laboratory,
Shijiazhuang
050081,
China
a
r
t
i
c
l
e
i
n
f
o
Article
history:
Received
28
September
2011
Received
in
revised
form
15
August
2012
Accepted
23
August
2012
Available online 1 September 2012
Keywords:
Virtualization
Inter-VM
communication
Overhead
Communication
path
a
b
s
t
r
a
c
t
In
virtualization
technology
field,
researches
mainly
focus
on
strengthening
the
isolation
barrier
between
virtual
machines
(VMs)
that
are
co-resident
within
a
single
physical
machine.
At
the
same
time,
there
are
many
kinds
of
distributed
communication-intensive
applications
such
as
web
services,
transaction
processing,
graphics
rendering
and
high
performance
grid
applications,
which
need
to
communicate
with
other
virtual
machines
at
the
same
platform.
Unfortunately,
current
inter-VM
communication
method
cannot
adequately
satisfy
the
requirement
of
such
applications.
In
this
paper,
we
present
the
design
and
implementation
of
a
high
performance
inter-VM
communication
method
called
IVCOM
based
on
Xen
virtual
machine
environment.
In
para-virtualization,
IVCOM
achieves
high
performance
by
bypass-
ing
some
protocol
stacks
and
privileged
domain,
shunning
page
flipping
and
providing
a
direct
and
high-performance
communication
path
between
VMs
residing
in
the
same
physical
machine.
But
in
full-virtualization,
IVCOM
applies
a
direct
communication
channel
between
domain
0
and
Hardware
Vir-
tualization
based
VM
(HV
2
M)
and
can
greatly
reduce
the
VM
entry/exit
operations,
which
has
improved
the
HV
2
M
performance.
In
the
evaluation
of
para-virtualization
consisting
of
a
few
of
benchmarks,
we
observe
that
IVCOM
can
reduce
the
inter-VM
round
trip
latency
by
70%
and
increase
throughput
by
up
to
3
times,
which
prove
the
efficiency
of
IVCOM
in
para-virtualized
environment.
In
the
full-virtualized
one,
IVCOM
can
reduce
90%
VMX
transition
operations
in
the
communication
between
domain
0
and
HV
2
M.
© 2012 Elsevier Inc. All rights reserved.
1.
Introduction
Virtual
machines
(VMs)
technology
was
introduced
in
the
1960s,
and
reached
prominence
in
the
early
1970s.
It
can
cre-
ate
virtual
machines
which
can
provide
function
and
performance
isolation
across
applications
and
services
that
share
a
common
hardware
platform.
At
the
same
time,
VMs
can
improve
the
system-
wide
utilization
efficiency
and
provide
lower
overall
operation
cost
of
the
system.
With
the
advent
of
low-cost
minicomput-
ers
and
personal
computers,
the
need
for
virtualization
declined
(Figueiredo
et
al.,
2005).
As
growing
interest
in
improving
the
uti-
lization
of
computing
resources
through
server
consolidation,
VM
technology
is
regaining
the
spotlight
again
and
is
widely
used
in
many
fields.
Now,
there
are
many
virtual
machine
monitors
(VMMs)
such
as
VMware
(Sugerman
et
al.,
2001),
Virtual
PC,
UML,
and
Xen
(Barham
et
al.,
2003).
Among
them,
Xen
develops
a
tech-
nique
known
as
para-virtualization
(Whitaker
et
al.,
2002),
which
offers
virtualization
with
low-overhead
and
has
attracted
much
attention
from
both
the
academic
VM
and
the
enterprise
market.
∗
Corresponding
author.
Tel.:
+86
10
8233
9020.
E-mail
addresses:
byb@buaa.edu.cn (Y.
Bai),
mayao@cse.buaa.edu.cn
(Y.
Ma).
However,
para-virtualized
approach
has
its
intrinsic
shortcoming
that
it
has
to
modify
the
OS
kernel
on
it
to
recuperate
the
proces-
sor’s
virtualization
holes.
To
implement
full-virtualization
(Adams
and
Agesen,
2006)
on
x86
platform,
some
processor
manufactur-
ers
propose
their
own
hardware
assisted
technologies
to
support
full
virtualization
such
as
Intel’s
VT
technology,
and
AMD’s
Pacifica
technology.
In
spite
of
the
recent
advance
in
the
VM
technology,
virtual
network
performance
remains
a
major
challenge
(Menon
et
al.,
2006).
Some
researches
done
by
Menon
et
al.
(2005)
show
that
Linux
guest
domain
has
far
lower
network
performance
than
native
Linux
in
the
scenarios
of
inter-VM
communication.
The
communi-
cation
performance
between
two
processes
in
their
own
VMs
on
the
same
physical
machine
is
even
worse
than
expected,
which
is
mainly
due
to
the
virtualization
technology’s
central
character-
istic
of
isolation.
For
example,
a
distributed
HPC
application
may
have
two
processes
running
in
different
VMs
that
need
to
com-
municate
using
messages
over
MPI
libraries.
Another
example
is
network
transaction.
In
order
to
satisfy
a
client
transaction
request,
a
web
service
running
in
one
VM
may
need
to
communicate
with
a
database
server
which
is
running
in
another
VM.
Even
routine
inter-
VM
communication,
such
as
file
transfers,
may
need
to
cross
the
isolation
barrier
frequently.
In
the
examples
above,
we
would
like
0164-1212/$
–
see
front
matter ©
2012 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jss.2012.08.054