![](https://csdnimg.cn/release/download_crawler_static/14926861/bg10.jpg)
RELIABLE CODE
14 IEEE SOFTWARE | WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE
against hardware faults, but it can-
not do the same for the software. If
there is a bug in the code, the same
bug will obviously affect both execu-
tions, which then provides no protec-
tion at all. This is also the scenario
that led to the failure of the maiden
flight of the Ariane 5 launch system
in June 1996.
5
The same software
was running on both the main CPU
and backup CPU, which meant that
a single floating-point exception er-
ror could crash both, causing the
costly loss of the vehicle.
Another method for using redun-
dancy in the design of software sys-
tems, called N-version programming,
was popular for a while, but it also
failed to deliver the promised in-
crease in reliability. The thought was
to have multiple independent teams
write software in parallel and com-
pare the results of the executions of
all systems to detect inconsistencies
caused by bugs. As Knight and Leve-
son
6
pointed out in the late 1980s,
all teams working on the design still
begin from a common set of require-
ments and are likely to make similar
types of mistakes. Splitting a team
into N subteams can also lead to an
increase in cost and loss of produc-
tivity as well as create tension among
them. That is, how does one avoid
cross contamination of key findings
during development, for example, if
one team discovers significant omis-
sions in the design requirements all
teams work from?
Self-Checking Code
Although all of this sounds discourag-
ing, there is, in fact, an effective tech-
nique for using redundancy to improve
system reliability, although we often
do not think of it in quite that way.
The method is to increase the use of
assertions throughout a code base. As
many software developers will tell you,
these assertions are redundant and can
safely be removed after testing. Do
they help to improve reliability? The
sobering news, at least for those devel-
opers, is that it can be shown that the
number of assertions that are retained
in the code after testing correlates
strongly with postrelease fault density.
More assertions means fewer faults.
It is as simple as that. This was first
shown in a study done by Microsoft
researchers, who studied postrelease
faults in the Office suite of tools and
compared it with the assertion density
of the failing code.
7
For the mission code that is devel-
oped at NASA/Jet Propulsion Labo-
ratory, we require that the average
assertion density for each module is
2% or more. This means that 2% of
the code performs self-checks at key
steps in the computations performed
to make sure that integrity is main-
tained, even in anomalous execution
scenarios. The use of assertions thus
provides a form of software redun-
dancy that can indeed make a system
more reliable.
What About Testing?
I have not mentioned testing in the dis-
cussion of reliability so far. Some say
that, since the cyclomatic complexity
metric measures the number of paths
in the control-flow graph of a func-
tion, this gives the number of tests that
are required to test all of those paths.
Fewer required tests should, then,
make it possible to test code more rig-
orously. This ignores, though, the fact
that most of the complexity in a pro-
gram execution does not originate in
the structure of the control-flow graph
but in the data that are processed. A
single path in the control-flow graph
can be executed in an astronomi-
cal number of different ways, which
means that a single test of each path in
the control-flow graph has little chance
of revealing all of the bugs that may
hide on that path.
A function with a lower cyclomatic
complexity does not necessarily re-
quire fewer tests to vet thoroughly.
All code is written to satisfy some set
of requirements and provide a desired
functionality. We can spread that
functionality across multiple modules
and functions to create a large assem-
bly of interconnected and mutually
dependent small functions, but that
will rarely be the right way to struc-
ture the code, and neither will it be
able to simplify testing.
Requirements Testing
To put it most succinctly: tests should
not be derived from the structure
of a control-flow graph but from
the requirements that prompted the
creation of the code. To check if a
requirement is met, the tester or devel-
oper should define a suite of tests that
can evaluate the adequacy of the code
to handle a range of cases, including
expected cases, boundary cases, and
even cases that outright violate the
design assumptions. If, after running
that suite of tests for all requirements,
it turns out that parts of the code
base were not reached, this indicates
a flaw in either the requirements or in
the code. The most common situation
is that the requirements are found to
be incomplete and fail to cover things
that the code must be able to handle.
It can, however, also be that the code
itself is redundant and that parts of it
serve no legitimate purpose to satisfy
the requirements.
Code coverage metrics serve to
check the adequacy of a test suite for
a specific set of requirements. Adding
test cases to an existing test suite that
serve only to improve the coverage
metric, without actually testing any-
thing, is common practice, but it can-
not improve a system’s reliability.