HBase权威指南：深入理解大数据存储

需积分: 19 13 浏览量更新于2024-07-26 收藏 12.14MB PDF 举报

“HBase: The Definitive Guide by Lars George” 《HBase: The Definitive Guide》是由Lars George编写的关于HBase的权威指南，旨在深入理解和掌握这款基于Apache Hadoop的分布式列式数据库。这本书详细介绍了HBase的核心概念、设计原理以及实际操作技巧，为读者提供了全面的学习资源。 HBase是构建在Hadoop之上的开源数据库，特别适合处理海量的非结构化和半结构化数据。它的主要特点是高度可扩展性、实时读写能力和水平扩展能力，这使得它在大数据领域中被广泛应用。HBase的设计灵感来源于Google的Bigtable，但针对开源社区进行了优化和改进。在本书中，作者Lars George详细阐述了以下关键知识点： 1. **HBase架构**：HBase基于行的存储模型，每个表由行键（Row Key）、列族（Column Family）、列（Column）和时间戳（Timestamp）组成。HBase利用Region Server来管理表的分区，通过Zookeeper进行集群协调。 2. **数据模型**：理解HBase的数据模型对于有效利用其特性至关重要。包括如何设计合理的行键，以及如何利用列族和列来组织数据。 3. **安装与配置**：书中详细指导了如何在各种环境中安装和配置HBase，包括单机模式、伪分布式模式和完全分布式模式。 4. **操作与管理**：涵盖了创建、删除、修改表，以及数据导入导出、备份恢复等日常运维任务。 5. **性能优化**：讨论了如何调整HBase的配置参数以提高性能，包括内存管理、磁盘I/O和并行处理策略。 6. **查询与索引**：HBase支持基于行键的查询，书中还介绍了如何通过Secondary Indexes和 Bloom Filters来优化查询效率。 7. **故障排查与监控**：讲解了如何监控HBase集群的健康状态，以及如何诊断和解决常见问题。 8. **应用程序开发**：提供了使用Java API和HBase Shell操作HBase的实例，以及如何在应用程序中集成HBase。 9. **案例研究**：通过实际案例展示了HBase在不同领域的应用，如Web日志分析、物联网数据存储等。 10. **未来展望**：探讨了HBase的发展趋势和新特性，如HBase与Spark的结合，以及与Hadoop生态系统其他组件的交互。《HBase: The Definitive Guide》是一本深入了解和实践HBase的必备书籍，无论你是初学者还是经验丰富的开发者，都能从中受益匪浅。通过学习本书，你将能够有效地利用HBase处理大规模数据存储和检索的需求。

Foreword

The HBase story begins in 2006, when the San Francisco-based startup Powerset was

trying to build a natural language search engine for the Web. Their indexing pipeline

was an involved multistep process that produced an index about two orders of mag-

nitude larger, on average, than your standard term-based index. The datastore that

they’d built on top of the then nascent Amazon Web Services to hold the index inter-

mediaries and the webcrawl was buckling under the load (Ring. Ring. “Hello! This is

AWS. Whatever you are running, please turn it off!”). They were looking for an alter-

native. The Google BigTable paper

had just been published.

Chad Walters, Powerset’s head of engineering at the time, reflects back on the

experience as follows:

Building an open source system to run on top of Hadoop’s Distributed Filesystem (HDFS)

in much the same way that BigTable ran on top of the Google File System seemed like a

good approach because: 1) it was a proven scalable architecture; 2) we could leverage

existing work on Hadoop’s HDFS; and 3) we could both contribute to and get additional

leverage from the growing Hadoop ecosystem.

After the publication of the Google BigTable paper, there were on-again, off-again dis-

cussions around what a BigTable-like system on top of Hadoop might look. Then, in

early 2007, out of the blue, Mike Cafarela dropped a tarball of thirty odd Java files into

the Hadoop issue tracker: “I’ve written some code for HBase, a BigTable-like file store.

It’s not perfect, but it’s ready for other people to play with and examine.” Mike had

been working with Doug Cutting on Nutch, an open source search engine. He’d done

similar drive-by code dumps there to add features such as a Google File System clone

so the Nutch indexing process was not bounded by the amount of disk you attach to

a single machine. (This Nutch distributed filesystem would later grow up to be HDFS.)

Jim Kellerman of Powerset took Mike’s dump and started filling in the gaps, adding

tests and getting it into shape so that it could be committed as part of Hadoop. The

first commit of the HBase code was made by Doug Cutting on April 3, 2007, under

the contrib subdirectory. The first HBase “working” release was bundled as part of

Hadoop 0.15.0 in October 2007.

* “BigTable: A Distributed Storage System for Structured Data” by Fay Chang et al.

Not long after, Lars, the author of the book you are now reading, showed up on the

#hbase IRC channel. He had a big-data problem of his own, and was game to try HBase.

After some back and forth, Lars became one of the first users to run HBase in production

outside of the Powerset home base. Through many ups and downs, Lars stuck around.

I distinctly remember a directory listing Lars made for me a while back on his produc-

tion cluster at WorldLingo, where he was employed as CTO, sysadmin, and grunt. The

listing showed ten or so HBase releases from Hadoop 0.15.1 (November 2007) on up

through HBase 0.20, each of which he’d run on his 40-node cluster at one time or

another during production.

Of all those who have contributed to HBase over the years, it is poetic justice that Lars

is the one to write this book. Lars was always dogging HBase contributors that the

documentation needed to be better if we hoped to gain broader adoption. Everyone

agreed, nodded their heads in ascent, amen’d, and went back to coding. So Lars started

writing critical how-tos and architectural descriptions inbetween jobs and his intra-

European travels as unofficial HBase European ambassador. His Lineland blogs on

HBase gave the best description, outside of the source, of how HBase worked, and at

a few critical junctures, carried the community across awkward transitions (e.g., an

important blog explained the labyrinthian HBase build during the brief period we

thought an Ivy-based build to be a “good idea”). His luscious diagrams were poached

by one and all wherever an HBase presentation was given.

HBase has seen some interesting times, including a period of sponsorship by Microsoft,

of all things. Powerset was acquired in July 2008, and after a couple of months during

which Powerset employees were disallowed from contributing while Microsoft’s legal

department vetted the HBase codebase to see if it impinged on SQLServer patents, we

were allowed to resume contributing (I was a Microsoft employee working near full

time on an Apache open source project). The times ahead look promising, too, whether

it’s the variety of contortions HBase is being put through at Facebook—as the under-

pinnings for their massive Facebook mail app or fielding millions of of hits a second on

their analytics clusters—or more deploys along the lines of Yahoo!’s 1k node HBase

cluster used to host their snapshot of Microsoft’s Bing crawl. Other developments in-

clude HBase running on filesystems other than Apache HDFS, such as MapR.

But plain to me though is that none of these developments would have been possible

were it not for the hard work put in by our awesome HBase community driven by a

core of HBase committers. Some members of the core have only been around a year or

so—Todd Lipcon, Gary Helmling, and Nicolas Spiegelberg—and we would be lost

without them, but a good portion have been there from close to project inception and

have shaped HBase into the (scalable) general datastore that it is today. These include

Jonathan Gray, who gambled his startup streamy.com on HBase; Andrew Purtell, who

built an HBase team at Trend Micro long before such a thing was fashionable; Ryan

Rawson, who got StumbleUpon—which became the main sponsor after HBase moved

on from Powerset/Microsoft—on board, and who had the sense to hire John-Daniel

Cryans, now a power contributor but just a bushy-tailed student at the time. And then

xvi | Foreword

剩余553页未读，继续阅读

仲景武

粉丝: 12
资源: 4

HBase权威指南：深入理解大数据存储

HBase summit document

Architecting_HBase_Applications_201608

Architecting HBase Applications

SpringBoot使用Hbase增删改查

非关系型数据库有哪些？

nosql 的数据库模型

NoSQL数据库有哪些主要的数据模型？各自有什么特点？

NoSQL数据库有哪几种主要类型，各自特点是什么？

nosql 数据类型

非关系型数据库有哪些

最新资源