![](https://csdnimg.cn/release/download_crawler_static/87202091/bg3.jpg)
524 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’16) USENIX Association
over its hardware counterpart. We can address scalabil-
ity by adopting the scale-out mod el, wher e the capac-
ity of the load balancer can be improved by increasing
the number of machines in the system: through ECMP
forwarding, traffic can be evenly distributed across all
machines. Availability and reliability are enhanced as
the system provides N+1 redundancy. By controlling
the entire system ourselves, we can quickly add, test,
and deploy new features. Meanwhile, deployment of the
load balancers themselves is greatly simplified: the sys-
tem uses o nly existing servers inside the clusters. We
can also divide services between multiple shards of load
balancers in the same cluster in order to achieve perfor-
mance isola tion.
Despite all the be nefits, the design an d implementation
of a software network load balancer are highly complex
and challenging. First, each individual machine in the
system must provide high throughput. Let N be the num-
ber of machines in the system and T be the maximum
throughput of a single machine. The maximum capac-
ity of the system is bounded by N × T . If T is not high
enough, it will be uneconomical for the system to pro-
vide enough capacity for all services [22]. The system as
a whole must also provide connection persistence: pack-
ets belonging to the same connection should always be
directed to the same service endpoint. This ensures qual-
ity of service as clusters are very dynamic and failures
are quite common [23, 40].
This paper presents Maglev, a fast and reliable soft-
ware network load balancing system. Maglev h as been a
critical component of Google’s frontend serving infras-
tructure since 2008, and currently serves almost all of
Google’s incoming user traffic. By exploiting recent ad-
vances in high-speed server networking techniques [18,
41, 3 5, 31], each Maglev machine is able to achieve line-
rate throughput with small packets. Through consistent
hashing and connection tracking, Maglev p rovides reli-
able packet delivery despite frequent changes and unex-
pected failures. While some of the techniques described
in this paper have existed for years, this paper shows how
to build an operational system using these techniques.
The major contributions of this paper are to: 1) present
the design and implementation of Maglev, 2) share ex-
periences of op erating Maglev at a global scale, and 3)
demonstrate the capability of Maglev through extensive
evaluations.
2 System Overview
This section provides an overview of how Maglev works
as a network load balancer. We give a brief introduction
to Google’s frontend serving architecture, followed by a
description of how the Maglev system is configured.
Figure 2: Maglev packet flow.
2.1 Frontend Serving Architecture
Maglev is deployed in Google’s frontend-serving loca-
tions, including clusters of varying sizes. For simplicity,
we only f ocus on the setup in the sma ller clusters in this
paper, and briefly describe the larger cluster setup below.
Figure 2 shows an overview of Google’s frontend serving
architecture in the small cluster setup.
Every Google service has one or more Virtual IP ad-
dresses (VIPs). A VIP is different from a physical IP in
that it is not assigned to a specific network interface, but
rather served by multiple service endpoints behind Ma-
glev. Maglev associates each VIP with a set of service
endpoints and announces it to the router over BGP; the
router in turn announces the VIP to Google’s backbone.
Aggregations of the VIP networks are announced to the
Internet to make them globally accessible. Maglev han-
dles both IPv4 and IPv6 traffic, and all the discussion
below applies equally to both.
When a user tries to access a Google service served on
www.google.com, her browser first issues a DNS query,
which gets a response (possibly cached) from one of
Google’s au th oritative DNS servers. The DNS server as-
signs the user to a nearby frontend location taking into
account both her geo lo cation and the current load at each
location, and returns a VIP belonging to the selected lo-
cation in response [16]. The browser will then try to es-
tablish a new connectio n with the VIP.
When the router receives a VIP packet, it forwards
the packet to one of the Maglev machines in the clus-
ter through ECMP, since all Maglev machines announce
the VI P with the same cost. Wh en the Maglev machine
receives the packet, it selects an endpo in t from the set
of service endpoints associated with the VIP, and encap-
sulates the packet u sing Generic Routing Encapsulation
(GRE) with the outer IP header destined to the endpoint.
When the packet arrives at the selected service end-
point, it is decapsulated and consumed. The response,
when ready, is put into an IP packet with the source ad-
dress being the VIP and the destination address being