PRIMS: Making NVRAM Suitable for Extremely Reliable Storage
†
Kevin M. Greenan
kmgreen@cs.ucsc.edu
Ethan L. Miller
elm@cs.ucsc.edu
Storage Systems Research Center
University of California, Santa Cruz
Abstract
Non-volatile byte addressable memories are becoming
more c ommon, and are increasingly used for critical data
that must n ot be lost. However, existing NVRAM-based file
systems do not include features that gu ard against file sys-
tem corruption or NVRAM corruption. Furthermore, most
file systems check consistency only after the system has al-
ready crashed. We are designing PRIMS to address these
problems by providing file storage that can survive mul-
tiple errors in NVRAM, whether caused by errant operat-
ing system writes or by media corruption. PRIMS uses an
erasure-encoded log structure to store pe rsistent metadata,
making it possible to periodically verify the correctness of
file system operations while achieving throughput rates of
an order of magnitude higher than page-pro tection during
small writes. It also checks integrity on every operation and
performs on-line scans of the e ntire NVRAM to ensure that
the file system is consistent. If errors are found, PRIMS
can correct them using file system logs and extensive error
correction information. While PRIMS is designed for relia-
bility, we expect it to have excellent performance, thanks to
the a bility to do word-aligned rea ds and writes in NVRAM.
1 Introduction
Byte-addre ssab le, non-volatile memory (NVRAM) tech-
nologies such as magnetoresistive random acc e ss mem-
ory (MRAM) and phase-change memory (PRAM) have re-
cently emerged as viable competitors to Flash RAM [1, 2].
These relatively low capacity technologies are perfect for
permane nt metadata storage, and can greatly improve file
system performance, reliability and power consumption.
Unfortu nately, due to the inc reased chance of data corrup-
tion, storing permanent structures in NVRAM is generally
regarded as unsafe, particularly whe n compared to disk.
The simplicity of most memory access interfaces makes er-
roneous writes more likely, resulting in data corruption—
it is far easier to man ipulate structures in memory tha n on
disk. Such behavior is comm on in OS kernels, in which
buggy code can issue err oneous wild writes that ac c iden-
†
This research was funded in part by NSF-0306650, the Dept. of
Energy-funded Petascale Data Storage Institute, and by SSRC industrial
partners.
tally overwrite memory used by another module or applica-
tion.
The goal of PRIMS (Persistent, Reliable In-Memory
Storage) is to provide reliable storage in NVRAM with-
out hindering the access speed of byte-addressable mem-
ory. Given the limitations of current in-memory reliabil-
ity mecha nisms, we believe that a log-based scheme using
software erasure codes is the most effective way to ensure
the consistency of persistent, memory-resid e nt data. We
present a log-based approach that has the ability to detect
and corre ct errors at multiple byte-granularity without using
page-b a sed access control or specialized hardware support.
PRIMS consists of a single, erasure-encoded log structure
that is used to detect and correct hardware erro rs, software
errors an d file system inc onsistencies.
2 Motivation
Modern operating systems protect critical regions of
memory using access control bits in the paging stru ctures.
While page-level access control is an effective tool for pre-
venting wild writes in write caches, it is not the best so-
lution for pro te c ting small, persistent structures in byte-
addressable memory because every protected write r equires
a TLB flush and two structu re modifications to m ark a page
as read-write an d read-only. During periods of freq uent
small writes, these permission changes have a dramatic ef-
fect o n performance.
Disk interfaces also decrease the likelihood of wild
writes by re quiring access through device dr ivers contain-
ing complex I/O routines. The probability of rogue code
accidentally corru pting disk blocks while evading the con-
trolled device drive r interface is extremely low. This strict
I/O interface greatly improves data reliability with respect
to software errors, but hinders perform ance o n low-latency
media, such as NVRAM.
In addition to software errors, hardware errors such as
random bit flips and ce ll wear may occur on the media,
leading to data corruption. Hardware-based error correction
schemes require a specialized controller and resolve er rors
beyond the correction capability by rebooting the system.
Obviously, rebooting is not an option when protecting per-
sistent data in memory; thus, a more robust scheme is neces-
sary. Hardware-based error correction is also computed in-
dependent of any software implementation; as a result, wild