![](https://csdnimg.cn/release/download_crawler_static/9296971/bg4.jpg)
4
Google® Inc. researchers conducted a
two-year study of memory errors in
Google‟s server fleet (see Google Inc.,
“DRAM Errors in the Wild: A Large-Scale
Field Study,” in the References section).
All of the servers in the study were
protected by ECC, in the form of either
SECDED or Chipkill. Researchers found
that ECC fixed most of the memory
errors. In that study, Chipkill technology
was more effective in correcting errors
by a factor of 4 to 10 over the less-
powerful SECDED codes.
Researchers observed more than
8 percent of DIMMS and about
one-third of the machines in the
study were affected by
correctable (DCE) errors per year.
Researchers observed more than 8
percent of DIMMS and about one-third of
the machines in the study were affected
by correctable (DCE) errors per year, and
DIMMs averaged nearly 4,000
correctable errors per year. The study
found FIT rates (failures in time per
billion device hours) of 25,000 to 70,000
per Mbit. The annual percentage of
detected, uncorrected errors (DUE) was
1.3 percent per machine and 0.22
percent per DIMM.
DCEs can be handled by ECC and
therefore are largely transparent to the
running application. As the Google paper
concludes, the DUE rate makes a crash-
tolerant layer indispensable for large-
scale server farms.
The Google research studied whether
errors could be used to detect future
errors. It found that DCEs in DIMMs
provide a strong indication of future
DCEs and DUEs. For the analysis, the
researchers sorted the servers into six
platform groups, defined by motherboard
and memory generation. For each tested
platform, they found that in 85 percent
of cases, a DCE was followed by at least
one more DCE in the same month, making
the various platforms from 13 to 90
times more likely to have DCEs later in
the month compared to an average
month. DCEs in the previous month also
increased the probability of DCEs. DUEs
were 27 to 400 times more likely in a
month with DCEs, and from 9 to 47 times
more likely if there had been DCEs the
previous month.
The research also found similar, though
much slighter, correlations among
failures in DIMMs in the same system,
with systems that experienced failures
on any DIMMs more likely to soon have
failures on other DIMMs.
The Google researchers analyzed
external factors related to errors. They
found that memory utilization and age of
DIMMs strongly correlated with memory
errors. Correctable error rates increased
almost logarithmically as a function of
utilization levels. The incidence of
For each tested platform, they
found that in 85 percent of
cases, a DCE was followed by at
least one more DCE in the same
month,
correctable errors increased with DIMM
age, while the incidence of uncorrectable
errors decreased with age (due to
replacements of DIMMs with those
errors). DIMM capacity and operating
temperature, two factors previously
thought to affect errors, did not correlate
with error. The study found no evidence
of worse error behavior among newer
generation, higher-capacity DIMMs.
Within the range of temperatures of the
tested production systems, temperature
also did not correlate with errors.
Intel Xeon Processor E7 Family: Reliability, Availability, and Serviceability
More advanced ECC technologies can
increase reliability. These include
IBM® Chipkill™ technology, which can
correct and detect more error bits in
a single memory word than
traditional ECCs. Chipkill can correct
up to eight adjacent error bits, and
thus can correct a broken four or
eight-bit-wide DRAM chip. Chipkill is
frequently combined with sparing, so
that if a memory chip fails (or
exceeds a threshold of correctable
bit errors), a spare memory chip is
used to replace it. Chipkill technology
lets servers withstand errors that
could crash a system with less
advanced error correction. Although
IBM introduced the technology, today
other vendors have equivalent
technologies. Sun Microsystems, now
a subsidiary of Oracle Corporation,
calls its approach Extended ECC, and
HP calls its approach Chipspare. Intel
introduced Single Device Data
Correction (SDDC), in the 1990‟s
which uses ECC to protect against
single x4 or x8 DRAM device failure.
Intel also provides a more advanced
memory correction technology called
Enhanced Double Device Data
Correction (DDDC), which allows
recovery from two sequential DRAM
failures on the memory DIMMs, as
well as recovery from a subsequent
single-bit soft error on the DIMM.
ADVANCED RAS AND THE INTEL
XEON PROCESSOR E7 FAMILY
The Intel Xeon Processor E7
Family
The Intel Xeon processor E7 family
delivers top-of-the-line performance
for the most data-demanding
workloads and mission-critical
applications. Built on the 32 nm Intel
process technology, the Intel Xeon
processor E7 family offers an
increased core count of up to 10
cores and 20 hardware threads in
either four- or eight-socket
configurations. For massive