AP
AP
AP
AP
Anycast
Client
Anycast
Client
Anycast
Target
Anycast
Target
AP
Anycast Proxy
Unicast (Tunnel/NAT)
Native IP Anycast
Figure 1: Proxy Architecture: the client packets
reaching the proxies through native IP anycast are
tunnelled to the targets
anycast addresses, and they do not interact with each other
3
.
In the remainder of this document, for simplicity of exposi-
tion, we assume a single PIAS infrastructure.
The basic idea of PIAS, illustrated in Figure 1, is very sim-
ple. Router-like boxes, hereon referred to as anycast proxies
(AP or simply proxies), are deployed at various locations in
the Internet, for example at POPs (Point of Presence) of dif-
ferent ISPs. These proxies advertise the same block of IP
addresses, referred to as the anycast prefix , into the rout-
ing fabric (BGP, IGP). As such, the proxies are reachable by
native IP anycast—a packet transmitted to the anycast prefix
will reach the closest proxy. However, these proxies are not
the actual anycast target destinations(AT)
4
. Rather, true
to their name, they proxy packets that reach them via na-
tive IP anycast to the true target destinations using unicast
IP. This proxying can take the form of lightweight tunnels
or NAT. NAT allows for backwards compatibility with the
protocol stack at target hosts, but increases processing at the
proxy.
This novel combination of native IP anycast with tunnelling
to the unicast addresses of the targets allows PIAS to fulfill
three critical design goals and drives the rest of the system
design. First, it allows for efficient use of the address space as
all the IP addresses in the prefix advertised by the proxies can
be used by different anycast groups. In fact, PIAS does one
better. It identifies an anycast group by the full transport
address (TA), i.e. IP address and TCP/UDP port, thus al-
lowing thousands of anycast groups per IP address. Second, it
solves the IP routing scaling problem by allowing many any-
cast groups to share a single address prefix and hence, fulfills
goal 7. Finally, it relieves targets from the burden of inter-
acting with the routing substrate. They can join an anycast
group by registering with a nearby proxy that is discovered
using native IP anycast. This fulfills goal 6.
The reader may notice two suspicious claims in the last
paragraph. First, we claim to ease deployment by running
unicast at the target instead of anycast, and yet the proxies
still must run anycast. So, how is this an improvement? The
benefit is that the difficult work of deploying IP anycast is
borne by the anycast provider once, and amortized across
many anycast groups. Second, we claim to improve scaling
by allowing thousands of IP anycast groups to share a single
IP address prefix. All we’ve really done, however, is to move
the scaling problem from the IP routing domain to the PIAS
infrastructure domain. This is quite intentional. As we argue
3
Indeed, a single operator could deploy multiple distinct
PIAS infrastructures as a way to scale.
4
the members of the anycast group; hereon referred to as
anycast targets or simply targets
later on, the scaling issues are much easier to deal with in the
overlaythaninIProuting.
PIAS offers two primitives to the members of an anycast
group, which involve sending messages to a nearby proxy:
• join(IP
A
:port
A
,IP
T
:port
T
,options): this message instructs
the proxy to forward packets addressed to the anycast
group identified by the TA IP
A
:port
A
to the joining
node’s unicast TA IP
T
:port
T
.Theoptions may spec-
ify additional information such as the selection criteria
(load balance etc.), delivery semantics (scoping etc.), or
security parameters needed to authenticate the target
host. These are discussed later.
• leave(IP
A
:port
A
,IP
T
:port
T
,options): this message in-
forms the proxy that the target identified by TA IP
T
:port
T
has left the group IP
A
:port
A
. options are the security
parameters.
The join and leave messages are transmitted to the anycast
address IP
A
(that belongs to the anycast prefix) at some well-
known port that is dedicated to receiving registration mes-
sages. This means that no extra configuration is required for
a target to discover a nearby proxy.
Note that we don’t specify a “create group” primitive. For
the purpose of this paper, we assume that the first join essen-
tially results in the creation of the group. In practice, a sub-
scriber to the service would presumably have entered into a
contract with the anycast service provider, which would have
resulted in the assignment of anycast TAs to that subscriber.
The subscriber would also have obtained authentication in-
formation using which targets may join the group. While the
issues surrounding this sort of group creation are important,
they are not central to the PIAS architecture, and we don’t
discuss them further.
3.1 The Join Anycast Proxy (JAP)
A target may leave a group either through the leave prim-
itive, or by simply falling silent (for instance, because the
target is abruptly shut off or loses its attachment to the In-
ternet). This means that the Join AP (JAP—the nearby
proxy with which the target registers; shown in figure 2) must
monitor the health of its targets, determine when they are no
longer available, and treat them as having left the group. The
proximity of the JAP to the target makes it ideal for this.
The JAP must also inform zero or more other anycast prox-
ies (APs) of the target(s) that have registered with it. This is
because not all APs may be JAPs for a given group (that is,
no target joined through them), but anycast clients (ACs)
may nevertheless send them packets destined for the group.
A proxy that receives packets directly from a client is referred
to as the Ingress AP (IAP)
5
for the client. Note that the
client-IAP relation is established using native IP anycast. As
an IAP, the proxy must know how to forward packets towards
a target; even though the IAP may not explicitly know of the
target.
One possible way to achieve this would have the JAP spread
information about targets associated with it to all proxies.
This allows the IAP to tunnel packets directly to clients (as in
Figure 1). However, such an approach would hamper PIAS’s
ability to support a large number of groups. In fact, Figure 1
is conceptual—PIAS’s approach for spreading group infor-
mation is described in the next section and the actual paths
taken by packets are shown in Figure 2.
5
in figure 1 the proxies in the client-target path are IAPs