HBase权威指南：打造大规模NoSQL数据库集群

需积分: 19 120 浏览量更新于2024-07-29 收藏 12.14MB PDF 举报

"HBase.The.Definitive.Guide.Lars.George.文字版" 《HBase：权威指南》是由Lars George编写的关于HBase的详细技术文档，它深入介绍了如何利用HBase在分布式环境中构建大规模的结构化数据存储系统。这本书是针对O'Reilly出版社出版的，通常以高质量的技术内容和实用指导而闻名。 HBase是一种基于Apache Hadoop的数据存储系统，属于NoSQL（Not Only SQL）非关系型数据库。NoSQL数据库强调灵活性、可扩展性和高性能，尤其适合处理海量、分布式的数据。与传统的关系型数据库不同，HBase不依赖于固定的表结构，而是采用行键、列族、时间戳和版本等概念来组织数据。在HBase中，数据被分布在大量的廉价服务器上，形成一个分布式集群。这种架构使得HBase能够处理PB级别的数据，并提供高并发读写操作。HBase充分利用了Hadoop的分布式文件系统HDFS（Hadoop Distributed File System），确保数据的容错性和高可用性。该书的内容可能包括以下几个主要方面： 1. **HBase基础知识**：介绍HBase的基本概念、架构以及工作原理，包括Region服务器、Master节点、ZooKeeper的作用等。 2. **数据模型**：详细讲解HBase的行、列族、列和时间戳的概念，以及如何设计数据模型以满足各种查询需求。 3. **安装与配置**：指导读者如何在不同的环境下部署和配置HBase，包括单机模式、伪分布式和完全分布式模式。 4. **API使用**：涵盖Java API和HBase Shell，演示如何创建表、插入数据、查询数据以及进行批量操作。 5. **性能优化**：讨论如何调整HBase的参数以提升系统性能，包括Region大小、Compaction策略和BlockCache的使用。 6. **监控与故障排查**：介绍监控工具和指标，帮助管理员识别和解决HBase中的问题。 7. **高级特性**：涵盖多版本控制、过滤器、 Coprocessors（协处理器）、复杂查询和其他进阶功能。 8. **案例研究**：通过实际应用场景，展示HBase在互联网、大数据分析、实时监控等领域的应用。 9. **最佳实践**：分享来自作者和社区的经验，提供在生产环境中使用HBase的最佳实践和建议。 10. **未来展望**：探讨HBase的发展趋势和技术演进，如与Spark、Hive等其他大数据工具的集成。通过阅读《HBase：权威指南》，读者可以全面了解HBase的各个方面，从基础到高级，从理论到实践，从而在自己的项目中有效地利用HBase处理大规模数据。

Foreword

The HBase story begins in 2006, when the San Francisco-based startup Powerset was

trying to build a natural language search engine for the Web. Their indexing pipeline

was an involved multistep process that produced an index about two orders of mag-

nitude larger, on average, than your standard term-based index. The datastore that

they’d built on top of the then nascent Amazon Web Services to hold the index inter-

mediaries and the webcrawl was buckling under the load (Ring. Ring. “Hello! This is

AWS. Whatever you are running, please turn it off!”). They were looking for an alter-

native. The Google BigTable paper

had just been published.

Chad Walters, Powerset’s head of engineering at the time, reflects back on the

experience as follows:

Building an open source system to run on top of Hadoop’s Distributed Filesystem (HDFS)

in much the same way that BigTable ran on top of the Google File System seemed like a

good approach because: 1) it was a proven scalable architecture; 2) we could leverage

existing work on Hadoop’s HDFS; and 3) we could both contribute to and get additional

leverage from the growing Hadoop ecosystem.

After the publication of the Google BigTable paper, there were on-again, off-again dis-

cussions around what a BigTable-like system on top of Hadoop might look. Then, in

early 2007, out of the blue, Mike Cafarela dropped a tarball of thirty odd Java files into

the Hadoop issue tracker: “I’ve written some code for HBase, a BigTable-like file store.

It’s not perfect, but it’s ready for other people to play with and examine.” Mike had

been working with Doug Cutting on Nutch, an open source search engine. He’d done

similar drive-by code dumps there to add features such as a Google File System clone

so the Nutch indexing process was not bounded by the amount of disk you attach to

a single machine. (This Nutch distributed filesystem would later grow up to be HDFS.)

Jim Kellerman of Powerset took Mike’s dump and started filling in the gaps, adding

tests and getting it into shape so that it could be committed as part of Hadoop. The

first commit of the HBase code was made by Doug Cutting on April 3, 2007, under

the contrib subdirectory. The first HBase “working” release was bundled as part of

Hadoop 0.15.0 in October 2007.

* “BigTable: A Distributed Storage System for Structured Data” by Fay Chang et al.

Not long after, Lars, the author of the book you are now reading, showed up on the

#hbase IRC channel. He had a big-data problem of his own, and was game to try HBase.

After some back and forth, Lars became one of the first users to run HBase in production

outside of the Powerset home base. Through many ups and downs, Lars stuck around.

I distinctly remember a directory listing Lars made for me a while back on his produc-

tion cluster at WorldLingo, where he was employed as CTO, sysadmin, and grunt. The

listing showed ten or so HBase releases from Hadoop 0.15.1 (November 2007) on up

through HBase 0.20, each of which he’d run on his 40-node cluster at one time or

another during production.

Of all those who have contributed to HBase over the years, it is poetic justice that Lars

is the one to write this book. Lars was always dogging HBase contributors that the

documentation needed to be better if we hoped to gain broader adoption. Everyone

agreed, nodded their heads in ascent, amen’d, and went back to coding. So Lars started

writing critical how-tos and architectural descriptions inbetween jobs and his intra-

European travels as unofficial HBase European ambassador. His Lineland blogs on

HBase gave the best description, outside of the source, of how HBase worked, and at

a few critical junctures, carried the community across awkward transitions (e.g., an

important blog explained the labyrinthian HBase build during the brief period we

thought an Ivy-based build to be a “good idea”). His luscious diagrams were poached

by one and all wherever an HBase presentation was given.

HBase has seen some interesting times, including a period of sponsorship by Microsoft,

of all things. Powerset was acquired in July 2008, and after a couple of months during

which Powerset employees were disallowed from contributing while Microsoft’s legal

department vetted the HBase codebase to see if it impinged on SQLServer patents, we

were allowed to resume contributing (I was a Microsoft employee working near full

time on an Apache open source project). The times ahead look promising, too, whether

it’s the variety of contortions HBase is being put through at Facebook—as the under-

pinnings for their massive Facebook mail app or fielding millions of of hits a second on

their analytics clusters—or more deploys along the lines of Yahoo!’s 1k node HBase

cluster used to host their snapshot of Microsoft’s Bing crawl. Other developments in-

clude HBase running on filesystems other than Apache HDFS, such as MapR.

But plain to me though is that none of these developments would have been possible

were it not for the hard work put in by our awesome HBase community driven by a

core of HBase committers. Some members of the core have only been around a year or

so—Todd Lipcon, Gary Helmling, and Nicolas Spiegelberg—and we would be lost

without them, but a good portion have been there from close to project inception and

have shaped HBase into the (scalable) general datastore that it is today. These include

Jonathan Gray, who gambled his startup streamy.com on HBase; Andrew Purtell, who

built an HBase team at Trend Micro long before such a thing was fashionable; Ryan

Rawson, who got StumbleUpon—which became the main sponsor after HBase moved

on from Powerset/Microsoft—on board, and who had the sense to hire John-Daniel

Cryans, now a power contributor but just a bushy-tailed student at the time. And then

xvi | Foreword

剩余553页未读，继续阅读

jasonqucccc

粉丝: 2
资源: 9

HBase权威指南：打造大规模NoSQL数据库集群

[HBase.The.Definitive.Guide].Lars.George.文字版

HBase 权威指南（HBase The Definitive Guide）

HBase_The_Definitive_Guide

浅谈HBASE数据结构设计.pdf

HBase权威指南：第一版精要

HBase企业应用开发实战：英版解析

HBase权威指南：Hadoop集群部署与HBase数据库详解

探索HBase：权威指南

apache hbase

基于freeRTOS和STM32F103x的手机远程控制浴室温度系统设计源码

最新资源