NVM驱动的数据库设计指南：兼顾速度与挑战
需积分: 10 34 浏览量
更新于2024-09-08
收藏 510KB PDF 举报
随着非易失性内存(NVM)技术的发展，其与易失性存储设备（如DRAM）在性能上的显著差异对数据库管理系统(DBMS)的设计产生了重大影响。长久以来，人们普遍认为非易失性存储速度远低于动态随机访问内存(DRAM)，但新型NVM的出现，其读写速度几乎接近DRAM且支持细粒度操作，这使得传统的设计假设变得不再适用。
本教程旨在概述如何在NVM硬件环境变化的背景下构建新一代DBMS。首先，我们将回顾NVM领域的最新进展，这些发展为DBMS设计提供了新的可能性。我们会深入讨论之前关于设计NVM数据库系统的研究成果，分析其中的经验教训。
在设计层面，我们重点关注以下几个关键点：
1. **硬件兼容性**：理解NVM特性的兼容性需求，如何使其无缝集成到现有的数据库架构中，包括缓存机制、一致性模型等。
2. **持久性和事务处理**：由于NVM提供了低延迟和高可用性，设计时需要重新思考数据的持久化策略以及如何处理ACID(原子性、一致性、隔离性、持久性)事务。
3. **性能优化**：NVM的特性如何影响查询优化、索引设计以及I/O操作的效率，如何利用NVM的快速读写特性提升数据库操作的响应速度。
4. **数据持久性与一致性**：探讨如何在NVM的固态特性下保证数据的一致性，以及在电源故障或系统崩溃后如何恢复数据完整性。
5. **故障恢复与容错**：由于NVM的耐用性，需要重新评估故障恢复策略，如何利用NVM的特性降低数据丢失的风险。
6. **安全性**：NVM可能引入新的安全挑战，如何在保持性能的同时，确保数据加密和隐私保护。
7. **开放研究问题**：最后，我们提出一些尚未解决的难题，如NVM对现有DBMS设计的适应性研究、新型存储接口和协议的开发等，并分享可能的解决方案思路。
设计基于NVM的数据库管理系统是一项极具挑战的任务，但也充满了机遇。通过结合最新的硬件趋势和技术革新，我们可以期待更高效、可靠、适应性强的数据库解决方案。
How to Build a Non-Volatile Memory
Database Management System
Joy Arulraj Andrew Pavlo
Carnegie Mellon University Carnegie Mellon University
jarulraj@cs.cmu.edu pavlo@cs.cmu.edu
ABSTRACT
The difference in the performance characteristics of volatile (DRAM)
and non-volatile storage devices (HDD/SSDs) influences the design
of database management systems (DBMSs). The key assumption
has always been that the latter is much slower than the former. This
affects all aspects of a DBMS’s runtime architecture. But the arrival
of new non-volatile memory (NVM) storage that is almost as fast
as DRAM with fine-grained read/writes invalidates these previous
design choices.
In this tutorial, we provide an outline on how to build a new
DBMS given the changes to hardware landscape due to NVM. We
survey recent developments in this area, and discuss the lessons
learned from prior research on designing NVM database systems.
We highlight a set of open research problems, and present ideas for
solving some of them.
1. INTRODUCTION
DBMSs have always dealt with the trade-offs between volatile and
non-volatile storage devices. In order to retain modifications after
a loss of power, the DBMS must write that data to a non-volatile
device, such as a SSD or HDD. Such devices only support slow,
bulk data transfers as blocks. Contrast this with volatile DRAM,
where a DBMS can quickly read and write a single byte from these
devices, but all data is lost once power is lost.
Non-volatile memory (NVM)
1
offers an intriguing blend of these
two storage mediums. NVM is a broad class of technologies, in-
cluding phase-change memory [
67
], memristors [
73
], and STT-
MRAM [
36
], that provide low latency reads and writes on the same
order of magnitude as DRAM, but with persistent writes and large
storage capacity like a SSD [24].
Researchers have been discussing the possibility of building a
DBMS for NVM for decades [
31
]. Although battery-backed DRAM
has existed for some time, it has physical form, cost, and availability
limitations that prevent it from being widely adopted. And until re-
cently, it looked as if NVM would suffer the same fate. We contend,
however, that there are three recent developments that make it look
like we are finally at the point where NVM will become available.
First is that the industry has agreed to standard definitions of NVM
1
NVM is also referred to as storage-class memory or persistent memory.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
SIGMOD’17, May 14-19, 2017, Chicago, IL, USA
© 2017 ACM. ISBN 978-1-4503-4197-4/17/05. . . $15.00
DOI: http://dx.doi.org/10.1145/3035918.3054780
technologies and form factors [
8
,
10
]. The second change is that
both Linux [
3
,
1
] and Microsoft [
52
,
48
] have announced support
for NVM in their respective kernels. Lastly, Intel has announced
that there are new instructions coming in their next ISA update that
are specifically for flushing data from CPU caches to NVM [
69
,
9
].
All of these portend the soon arrival of NVM. But it is unclear at
this point how to best leverage these new technologies in a DBMS.
There are several aspects of NVM that make existing DBMS archi-
tectures inappropriate for them [
33
]. For example, disk-oriented
DBMSs (e.g., Oracle RDBMS, IBM DB2, MySQL) are predicated
on using block-oriented devices for durable storage that are slow
at random access. As such, they maintain an in-memory cache for
blocks of tuples and try to maximize the amount of sequential reads
and writes to storage. In the case of memory-oriented DBMSs (e.g.,
VoltDB, MemSQL), they contain components for overcoming the
volatility of DRAM. Such components may be unnecessary in a
system with byte-addressable NVM with fast random access.
Tutorial Outline and Goals:
This tutorial is a comprehensive
discussion of how to design and build a DBMS for NVM. Our
focus is on systems that contain a hybrid architecture comprised
of DRAM, NVM, and HDD/SSDs (as opposed to one with only
NVM [
14
]), as such platforms will likely be the most common in
the near-term. The goal of our tutorial is to provide an overview
of the major design decisions in a DBMS implementation that are
affected by NVM. We will discuss issues related to both on-line
transaction processing (OLTP) and on-line analytical processing
(OLAP) systems, as well as hybrid (HTAP) DBMSs that seek to
support both workloads in a single platform.
We will first present an overview of NVM technologies (Sec-
tion 2), highlighting their differences both with each other and
compared to DRAM/SSDs. We will then discuss the three areas of
a DBMS architecture that are affected the most by NVM. First, we
discuss the core components, such as memory management, that
are shared throughout the DBMS (Section 3). We will then discuss
how the DBMS builds off of these components to support storage
and recovery mechanisms (Section 4). Then we describe how to
build on top of this storage layer to perform query optimization
and execution for NVM-resident databases (Section 5). Lastly, we
conclude with a summary of the lessons that we have learned in
building a DBMS for NVM.
This tutorial differs from previous presentations on NVM [
77
]
because we go beyond storage management and talk about the entire
internal DBMS stack. We will couch all of the above topics in the
context of the
Peloton
[
4
,
63
] DBMS. Peloton is an open-source
HTAP DBMS that we are building that is designed from the ground-
up to use NVM. Our intended audience are developers, researchers,
and practitioners with knowledge of DBMS internals. They do not
need any in-depth background or experience with NVM.
1753
下载后可阅读完整内容，剩余5页未读，立即下载
2017-09-29 上传
2024-08-23 上传
2021-08-08 上传
2021-03-27 上传
2021-12-19 上传
2021-08-07 上传
2020-12-13 上传
点击了解资源详情
闫玮
- 粉丝: 0
- 资源: 5
我的内容管理
展开
