Abstract—Typical Byzantine fault tolerance algorithms
require the application requests to be executed sequentially,
which may severely limit the throughput of the system
considering that modern CPUs are equipped with multiple
processing cores. In this paper, we present the design and
implementation of a Byzantine fault tolerance framework for
software-transactional-memory based applications that aims to
maximize concurrent processing while preserving strong replica
consistency. The approach is based on the idea of committing
concurrent transactions according to the total order of the
requests that triggered the transactions. A comprehensive
performance evaluation is carried out to characterize the
effectiveness and limitations of this approach.
Index Terms—Byzantine fault tolerance, software
transactional memory, distributed systems, concurrent
computing, performance evaluation
I. INTRODUCTION
Byzantine fault tolerance (BFT) [1], [2] appears to be a
powerful technology to enhance the trustworthiness of
distributed applications. In the past decade, we have seen
significant advancement of both the efficiency and robustness
of BFT algorithms [1]–[5]. However, typical BFT algorithms
require the application requests to be executed sequentially,
which may severely limit the throughput of the system
considering that modern CPUs are equipped with multiple
processing cores. This issue has been addressed by a number
of researchers [3], [4], [6], [7]. The primary approach is to
enable concurrent execution of requests that do no involve
conflicting operations. However, to enable concurrent
execution, it is assumed that the application semantics is
already known. This inevitably increases the design cost of
such BFT solutions and limits their reusability for other
applications.
In this paper, we present the design and implementation of
a BFT framework for software-transactional-memory based
applications that aims to maximize concurrent processing
without requiring the knowledge of application semantics.
The approach is based on the idea of committing concurrent
transactions according to the total order of the requests that
triggered the transactions. In essence, the dependency
between different requests is discovered dynamically (and
automatically) by the software-transactional-memory runtime.
Non-conflicting requests can be processed concurrently with
Manuscript received April 10, 2012; revised May 8, 2012.
Honglei Zhang and Wenbing Zhao are with the Department of Electrical
and Computer Engineering, Cleveland State University, Cleveland, OH
44115, USA (e-mail: wenbing@ieee.org.)
the only constraint that the commit order must respect the
total order of the corresponding requests. Some of the
conflicting requests that have been processed concurrently
may have to be aborted and retried. A comprehensive
performance evaluation is carried out to characterize the
effectiveness and limitations of this approach.
II. RELATED WORK
The primary approach to increasing the performance of
BFT systems is by exploiting application semantics. In PBFT
[1], Castro and Liskov noted that read-only requests can be
delivered without the need of total ordering. In BASE [2], it
was recognized that a BFT system can be made more robust
(to minimize deterministic software errors) by adopting a
common abstract specification for the service to be replicated.
A conformance wrapper for each distinct implementation is
then developed to ensure that it behaves according to the
common specification. Furthermore, an abstraction function
and one of its inverses are needed to map between the
concrete state of each implementation and the common
abstract state.
In [3], Kotla and Dahlin proposed to exploit application
semantics for higher throughput by parallelizing the execution
of independent requests. They outlined a method to determine
if a request is dependent on any pending request using
application specific rules. In [4], Distler and Kapitza further
extended Kotla and Dahlin’s work by introducing a scheme to
execute a request on only a selected subset of replicas. This
scheme assumes that the state variables accessed by each
request are known, and that the state object distribution and
object access are uniform.
In prior work [6], [7], we proposed to rely on deeper
application semantics to not only enable more requests (such
as those that are commutative) to be executed concurrently,
but also minimize the number of Byzantine agreement steps
used in an application (particularly for session-oriented
applications).
This paper takes a drastically different approach from those
mentioned above. Rather than resorting to the application
semantics, which may be expensive to acquire accurately and
hard to reuse, we rely on the use of software transactional
memory to dynamically capture the dependency of concurrent
operations automatically. This approach is inspired by the
work of Brito, Fetzer, and Felber [8], where a similar idea was
used to ensure multithreaded execution for actively-replicated
event stream processing systems. Our work applies the idea in
a different context (i.e., Byzantine fault tolerance instead of
crash fault tolerance) and furthermore, we carry out detailed
experiments and analysis on the level of concurrency that can
Honglei Zhang and Wenbing Zhao
Journal of Future Computer and Communication, Vol. 1, No. 1, June 2012
47
Concurrent Byzantine Fault Tolerance for
Software-Transactional-Memory Based Applications