2/73
'U705
+705'0$
,%6ZLWFK
:RUNHU7KUHDGV
0HPRU\
6WRUH
7UDQVDFWLRQ
/D\HU
ŜŜŜ
5'0$
+70
2/73
'U705
+705'0$
+70
1HWZRUN
NH\YDOXHRSV
0DFKLQH
0DFKLQH1
Figure 1. ThearchitectureoverviewofDrTM+R.
running n worker threads atop n cores; each worker thread
executes and commits a transaction at a time.
Approach Overview. Like prior work [52, 54], DrTM+R
comprises of two independent components: transaction layer
and memory store, as shown in Figure 1.DrTM+Rlever-
ages a hybrid OCC and locking scheme using HTM and
RDMA (§4). On each machine, DrTM+R utilizes an OCC-
based scheme to provide transaction support. Like typical
OCC, DrTM+R separates execution from commit by first
tracking the read/write sets of a transaction in the execu-
tion phase, validating the read set in the validation phase
and finally committing the updates in the commit phase.
The last two phases require atomicity, which DrTM+R guar-
antees using HTM transactions. DrTM+R exposes a parti-
tioned global address space such that remote records and lo-
cal records are explicitly distinguished using their address
identifiers. DrTM+R follows a local execution model. To
access remote data in a transaction, DrTM+R first fetches
remote records into the hosting machine, makes necessary
updates and then sends the records back to the remote ma-
chine. Remote accesses in DrTM+R are mainly done using
one-sided RDMA operations for efficiency. As there is no
apparent way to guarantee the atomicity between accessing
a record in a remote machine and validating the record in the
local machine, DrTM+R introduces a remote locking phase
(§4.4) before the validation and commit phases, which lever-
ages one-sided RDMA operations to lock remote records.
The high availability of DrTM+R is guaranteed by effi-
cient replication of database records before fully committing
a transaction (§5). This is achieved by leveraging a revised
commit protocol (§5.1). DrTM+R leverages ZooKeeper [20]
to reach an agreement on the current configuration among
surviving machines. Inspired by FaRM [17], an RDMA-
base protocol is used to manage leases, detect failures, and
coordinate recovery(§5.2). Thanks to the fast interconnect,
DrTM+R can detect a failure in a very short time with high
accuracy. During recovery, DrTM+R first reconfigures the
cluster and then recovers the state of crashed machines by
leveraging surviving machines.
&RPPLW3KDVH([HFXWLRQ3KDVH
7;/D\HU
0HPRU\
6WRUH
&/RFN
&9DOLGDWH
&8SGDWH
&8SGDWH
&8QORFN
&9DOLGDWH
/56
/:6
556
5:6
/RFDO5HDG
/RFDO:ULWH
5HPRWH5HDG
5HPRWH:ULWH
5:6
5:6
5:6
556
/56
/:6
0HPRU\
6WRUH
EXIIHU EXIIHU
1HWZRUN
UHSRUW
WRXVHU
VHULDOL]DWLRQSRLQW
556
556
556
5:6
5HDG6HW
:ULWH6HW
5HPRWH
/56
/:6
5HDG6HW
:ULWH6HW
/RFDO
5HDG:ULWH
ZLWKLQ+707;
5'0$5($'
5'0$:5,7(
5'0$&$6
+70
7LPH
Figure 2. The concurrency control protocol in DrTM+R.
4. Supporting Distributed Transactions
DrTM+R uses an HTM-friendly optimistic concurrency
control (OCC) protocol to provide transaction support within
a s ingle machine and further adopts an RDMA-friendly op-
timistic concurrency control (OCC) protocol to coordinate
accesses to remote records for distributed transactions.
4.1 HTM/RDMA-friendly OCC Protocol
Since an HTM transaction provides strong atomicity and
one-sided RDMA operations are cache-coherent, the prior
system [54] leverages them to bridge the HTM and two-
phase locking (2PL) protocol for distributed transactions.
However, it requires priori knowledge of read/write sets of
transactions for proper locking to implement the 2PL-like
protocol. Unfortunately, this is not always the case for some
general transaction workloads, like TPC-C [46]
3
and TPC-
E[47], which have dependent transactions.
DrTM+R addresses this limitation by designing a hy-
brid OCC and locking protocol to provide strictly serial-
izable transactions. The main observation is that the read-
/write of an transaction will be known after the execution
phase in OCC, due to its separation of execution from com-
mit. Hence, DrTM+R has all read/write sets known after the
execution phase. However, any RDMA operation inside an
HTM transaction will unconditionally cause an HTM abort
and thus we cannot directly access remote records through
RDMA within an HTM transaction. To this end, DrTM+R
adjusts the traditional OCC protocol by distinguishing oper-
ations on local and remote records, which will be protected
using HTM and RDMA-based locking mechanisms respec-
tively.
3
DrTM leverages transaction chopping to address this issue for TPC-C