allocate and deallocate for memory management,
and terminate to exit.
Despite the simple environment model, the binaries
provided by DARPA for the CGC have a wide range of
complexity. They range from 4 kilobytes to 10 megabytes
in size, and implement functionality ranging from simple
echo servers, to web servers, to image processing libraries.
DARPA has open-sourced all of the binaries used in the
competition thus far, complete with proof-of-concept exploits
and write-ups about the vulnerabilities [24].
Because the simple environment model makes it feasible to
accurately implement and evaluate (on a large scale) binary
analysis techniques, we use the DARPA CGC samples as our
dataset for the comparative evaluations in this paper.
C. Comparative Analysis of CGC Binaries
Offensive binary analyses use different underlying
techniques to reason about the application that is being
processed. For example, they may analyze data over different
domains or utilize different levels of interaction with the
application being tested. In the next two sections, we survey
the current state of the art, and choose several analyses
for in-depth evaluation in the rest of the paper. We focus
specifically on analyses whose goals are to identify and
exploit flaws in binary software (for example, memory safety
violation identification using symbolic execution), as opposed
to the more general binary analysis techniques on which
those are based (in this case, symbolic execution itself).
III. BACKGROUND: STATIC VULNERABILITY DISCOVERY
Static techniques reason about a program without executing
it. Usually, a program is interpreted over an abstract domain.
Memory locations containing bits of ones and zeroes contain
other abstract entities (at the familiar end, this might simply
be integers, but, as we explain below, these can include more
abstract constructs). Additionally, program constructs such as
the layout of memory, or even the execution path taken, may
be abstracted as well.
Here, we split static analyses into two paradigms: those
that model program properties as graphs (i.e., a control-flow
graph) and those that model the data itself.
Static vulnerability identification techniques have two main
drawbacks, relating to the trade-offs discussed in Section II-A.
First, the results are not replayable: detection by static analysis
must be verified by hand, as information on how to trigger
the detected vulnerability is not recovered. Second, these
analyses tend to operate on simpler data domains, reducing
their semantic insight. In short, they over-approximate: while
they can often authoritatively reason about the absence of
certain program properties (such as vulnerabilities), they
suffer from a high rate of false positives when making
statements regarding the presence of vulnerabilities.
A. Recovering Control Flow
The recovery of a control-flow graph (CFG), in which
the nodes are basic blocks of instructions and the edges are
possible control flow transfers between them, is a pre-requisite
for almost all static techniques for vulnerability discovery.
Control-flow recovery has been widely discussed in the
literature [21], [33], [34], [50], [58], [59]. CFG recovery
is implemented as a recursive algorithm that disassembles
and analyzes a basic block (say, B
a
), identifies its possible
exits (i.e., some successor basic block such as B
b
and B
c
)
and adds them to the CFG (if they have not already been
added), connects B
a
to B
b
and B
c
, and repeats the analysis
recursively for B
b
and B
c
until no new exits are identified.
CFG recovery has one fundamental challenge: indirect jumps.
Indirect jumps occur when the binary transfers control flow
to a target represented by a value in a register or a memory
location. Unlike a direct jump, where the target is encoded
into the instruction itself and, thus, is trivially resolvable, the
target of an indirect jump can vary based on a number of
factors. Specifically, indirect jumps fall into several categories:
Computed. The target of a computed jump is determined by
the application by carrying out a calculation specified by
the code. This calculation could further rely on values
in other registers or in memory. A common example
of this is a jump table: the application uses values in a
register or memory to determine an index into a jump
table stored in memory, reads the target address from
that index, and jumps there.
Context-sensitive. An indirect jump might depend on the
context of an application. The common example is
qsort() in the standard C library – this function takes
a callback that it uses to compare passed-in values. As
a result, some of the jump targets of basic blocks inside
qsort() depend on its caller, as the caller provides
the callback function.
Object-sensitive. A special case of context sensitivity is
object sensitivity. In object-oriented languages, object
polymorphism requires the use of virtual functions, often
implemented as virtual tables of function pointers that
are consulted, at runtime, to determine jump targets.
Jump targets thus depend on the type of object passed
into the function by its callers.
Different techniques have been designed to deal with
different types of indirect jumps, and we will discuss the
implementation of several of them in Section VII. In the
end, the goal of CFG recovery is to resolve the targets of as
many of these indirect jumps as possible, in order to create
a CFG. A given indirect jump might resolve to a set of
values (i.e., all of the addresses in a jump table, if there are
conditions under which their use can be triggered), and this
set might change based on both object and context sensitivity.
Depending on how well jump targets are resolved, the CFG
recovery analysis has two properties:
Soundness. A CFG recovery technique is sound if the set
of all potential control flow transfers is represented in
the graph generated. That is, when an indirect jump is
resolved to a subset of the addresses that it can actually
target, the soundness of the graph decreases. If a potential