HBase权威指南：探索Java开源NoSQL数据库

需积分: 19 57 浏览量更新于2024-07-28 收藏 12.14MB PDF 举报

"Hbase-the Definitive Guide.pdf" HBase是一种基于Java开发的NoSQL数据库，专注于提供高并发、高可扩展性和实时读写能力，特别适合处理大规模数据。作为Apache软件基金会的重要项目，HBase在大数据领域扮演着至关重要的角色，尤其是在淘宝等拥有海量数据的公司中作为底层存储系统。 HBase的设计理念源自Google的Bigtable论文，它构建于Hadoop分布式文件系统（HDFS）之上，保证了数据的高可用性和容错性。HBase的核心特性包括： 1. **列族存储**：HBase是列式存储的，这意味着数据按列族组织。列族是预定义的一组列，允许用户根据需要存储和查询特定列，从而提高查询效率。 2. **稀疏性**：HBase表格可以非常稀疏，只存储有值的单元格，这使得它非常适合存储大量可能为空的数据。 3. **行键与时间戳**：每个数据单元格都有一个行键（Row Key）和时间戳，行键用于唯一标识一行，时间戳则记录单元格的版本，支持多版本数据。 4. **分布式架构**：HBase采用Master-Slave架构，由一个HMaster节点负责元数据管理和 Region Server的调度，Region Server负责实际的数据存储和服务。Region Server会将数据分区并分布在集群的不同节点上，实现水平扩展。 5. **实时读写**：HBase支持亚秒级的读写操作，对于实时数据分析和快速响应的应用场景非常有利。 6. **强一致性**：HBase提供了单行事务的强一致性保证，但不支持跨行或跨列族的事务。 7. **MapReduce集成**：与Hadoop紧密集成，可以通过MapReduce进行批量数据处理和分析。 8. **ZooKeeper依赖**：HBase依赖ZooKeeper进行分布式协调，确保集群的稳定运行和故障恢复。 9. **索引和查询优化**：虽然HBase主要面向随机读取，但它可以通过创建二级索引来优化查询性能。此外，通过使用Filter和Scan API，用户可以定制复杂的查询逻辑。 10. **社区支持和生态**：HBase拥有活跃的开发者社区和丰富的生态系统，包括各种工具和库，如HBase Shell、Phoenix（SQL接口）、Ambari监控等。《HBase：权威指南》这本书详细介绍了如何设计、部署和管理HBase集群，涵盖了从基本概念到高级特性的方方面面，是学习和掌握HBase不可或缺的参考资料。书中不仅讲解了如何处理PB级别的数据，还分享了最佳实践和案例研究，帮助读者深入理解HBase在大数据解决方案中的作用。

Foreword

The HBase story begins in 2006, when the San Francisco-based startup Powerset was

trying to build a natural language search engine for the Web. Their indexing pipeline

was an involved multistep process that produced an index about two orders of mag-

nitude larger, on average, than your standard term-based index. The datastore that

they’d built on top of the then nascent Amazon Web Services to hold the index inter-

mediaries and the webcrawl was buckling under the load (Ring. Ring. “Hello! This is

AWS. Whatever you are running, please turn it off!”). They were looking for an alter-

native. The Google BigTable paper

had just been published.

Chad Walters, Powerset’s head of engineering at the time, reflects back on the

experience as follows:

Building an open source system to run on top of Hadoop’s Distributed Filesystem (HDFS)

in much the same way that BigTable ran on top of the Google File System seemed like a

good approach because: 1) it was a proven scalable architecture; 2) we could leverage

existing work on Hadoop’s HDFS; and 3) we could both contribute to and get additional

leverage from the growing Hadoop ecosystem.

After the publication of the Google BigTable paper, there were on-again, off-again dis-

cussions around what a BigTable-like system on top of Hadoop might look. Then, in

early 2007, out of the blue, Mike Cafarela dropped a tarball of thirty odd Java files into

the Hadoop issue tracker: “I’ve written some code for HBase, a BigTable-like file store.

It’s not perfect, but it’s ready for other people to play with and examine.” Mike had

been working with Doug Cutting on Nutch, an open source search engine. He’d done

similar drive-by code dumps there to add features such as a Google File System clone

so the Nutch indexing process was not bounded by the amount of disk you attach to

a single machine. (This Nutch distributed filesystem would later grow up to be HDFS.)

Jim Kellerman of Powerset took Mike’s dump and started filling in the gaps, adding

tests and getting it into shape so that it could be committed as part of Hadoop. The

first commit of the HBase code was made by Doug Cutting on April 3, 2007, under

the contrib subdirectory. The first HBase “working” release was bundled as part of

Hadoop 0.15.0 in October 2007.

* “BigTable: A Distributed Storage System for Structured Data” by Fay Chang et al.

Not long after, Lars, the author of the book you are now reading, showed up on the

#hbase IRC channel. He had a big-data problem of his own, and was game to try HBase.

After some back and forth, Lars became one of the first users to run HBase in production

outside of the Powerset home base. Through many ups and downs, Lars stuck around.

I distinctly remember a directory listing Lars made for me a while back on his produc-

tion cluster at WorldLingo, where he was employed as CTO, sysadmin, and grunt. The

listing showed ten or so HBase releases from Hadoop 0.15.1 (November 2007) on up

through HBase 0.20, each of which he’d run on his 40-node cluster at one time or

another during production.

Of all those who have contributed to HBase over the years, it is poetic justice that Lars

is the one to write this book. Lars was always dogging HBase contributors that the

documentation needed to be better if we hoped to gain broader adoption. Everyone

agreed, nodded their heads in ascent, amen’d, and went back to coding. So Lars started

writing critical how-tos and architectural descriptions inbetween jobs and his intra-

European travels as unofficial HBase European ambassador. His Lineland blogs on

HBase gave the best description, outside of the source, of how HBase worked, and at

a few critical junctures, carried the community across awkward transitions (e.g., an

important blog explained the labyrinthian HBase build during the brief period we

thought an Ivy-based build to be a “good idea”). His luscious diagrams were poached

by one and all wherever an HBase presentation was given.

HBase has seen some interesting times, including a period of sponsorship by Microsoft,

of all things. Powerset was acquired in July 2008, and after a couple of months during

which Powerset employees were disallowed from contributing while Microsoft’s legal

department vetted the HBase codebase to see if it impinged on SQLServer patents, we

were allowed to resume contributing (I was a Microsoft employee working near full

time on an Apache open source project). The times ahead look promising, too, whether

it’s the variety of contortions HBase is being put through at Facebook—as the under-

pinnings for their massive Facebook mail app or fielding millions of of hits a second on

their analytics clusters—or more deploys along the lines of Yahoo!’s 1k node HBase

cluster used to host their snapshot of Microsoft’s Bing crawl. Other developments in-

clude HBase running on filesystems other than Apache HDFS, such as MapR.

But plain to me though is that none of these developments would have been possible

were it not for the hard work put in by our awesome HBase community driven by a

core of HBase committers. Some members of the core have only been around a year or

so—Todd Lipcon, Gary Helmling, and Nicolas Spiegelberg—and we would be lost

without them, but a good portion have been there from close to project inception and

have shaped HBase into the (scalable) general datastore that it is today. These include

Jonathan Gray, who gambled his startup streamy.com on HBase; Andrew Purtell, who

built an HBase team at Trend Micro long before such a thing was fashionable; Ryan

Rawson, who got StumbleUpon—which became the main sponsor after HBase moved

on from Powerset/Microsoft—on board, and who had the sense to hire John-Daniel

Cryans, now a power contributor but just a bushy-tailed student at the time. And then

xvi | Foreword

剩余553页未读，继续阅读

codedancer

粉丝: 0
资源: 5

HBase权威指南：探索Java开源NoSQL数据库

hadoop练习-mytest.rar

titan-hbase-0.5.0.jar

phoenix-5.0.0.3.1.4.0-315.tar.gz

HBase.The.Definitive.Guide.2nd.Edition

[HBase.The.Definitive.Guide].Lars.George.文字版

Hadoop- The Definitive Guide, 3rd Edition.pdf

Hbase权威指南(HBase: The Definitive Guide)

HBase The Definitive Guide

Hadoop The Definitive Guide 2nd Edition.pdf

[Hadoop权威指南(第2版)].(Hadoop：The.Definitive.Guide).文字版.pdf

最新资源