xiv PREFACE
tion to run commodity operating systems on scalable multiprocessors. In 1999, VMware released
VMware Workstation 1.0 [45], the first commercial virtualization solution for x86 processors.
At the time, researchers and commercial entities started building virtual machines solu-
tions for desktops and servers. A few years later, the approach was introduced to mobile plat-
forms. Disco, VMware Workstation, VMware ESX Server [177], VirtualPC [130], Xen [27],
Denali [182], and Cells [16], were all originally designed for architectures that did not provide
support for virtualization. ese different software systems each took a different approach to work
around the limitations of the hardware of the time. Although processor architectures have evolved
to provide hardware support for virtualization, many of the key innovations of that era such as
hosted architectures [162], paravirtualization [27, 182], live migration [51, 135], and memory
ballooning [177], remain relevant today, and have a profound impact on computer architecture
trends.
Clearly, the virtualization tide has turned, to the point that it is once more a central driver of
innovation throughout the industry, including system software, systems management, processor
design, and I/O architectures. As a matter of fact, the exact quote from Goldberg’s 1974 paper
would have been equally timely 30 years later: Intel introduced its first-generation hardware sup-
port for virtual machines in 2004. Every maintained virtualization solution, including VMware
Workstation, ESX Server, and Xen, quickly evolved to leverage the benefits of hardware support
for virtualization. New systems were introduced that assumed the existence of such hardware sup-
port as a core design principle, notably KVM [113]. With the combined innovation in hardware
and software and the full support of the entire industry, virtual machines quickly became central
to IT organizations, where they were used among other things to improve IT efficiency, sim-
plify provisioning, and increase availability of applications. Virtual machines were also proposed
to uniquely solve hard open research questions, in domains such as live migration [51, 135] and
security [73]. Within a few years, they would play a central role in enterprise datacenters. For
example, according to the market research firm IDC, since 2009 there are more virtual machines
deployed than physical hosts [95].
Today, virtual machines are ubiquitous in enterprise environments, where they are used to
virtualize servers as well as desktops. ey form the foundation of all Infrastructure-as-a-Service
(IAAS) clouds, including Amazon EC2, Google CGE, Microsoft Azure, and OpenStack. Once
again, the academic community dedicates conference tracks, sessions, and workshops to the topic
(e.g., the annual conference on Virtual Execution Environments (VEE)).
ORGANIZATION OF THIS BOOK
is book focuses on the core question of the necessary architectural support provided by hardware
to efficiently run virtual machines. Despite the focus on architectural support in current architec-
tures, some historical perspective is necessary to appropriately frame the problem. Specifically,
this includes both a theoretical framework, and a description of the systems enabling virtualiza-
tion despite the lack of architectural support in hardware. As is often the case, theory defines