HBase实战：深度探索大数据存储

5星 · 超过95%的资源需积分: 13 132 浏览量更新于2024-07-26 收藏 7.86MB PDF 举报

"Hbase in Action 是一本由 Nick Dimiduk 和 Amandeep Khurana 合著的英文原版书籍，被业界视为《Hbase 权威指南》的进阶读物，由 Manning 出版社出版。本书的技术编辑是 Mark Henry Ryan。" 在深入探讨《Hbase in Action》这本书的内容之前，我们先了解下 Hbase 的基本概念。Hbase 是一个分布式的、基于列族的开源数据库，运行在 Apache Hadoop 上，提供高吞吐量的数据访问。它设计用于处理大规模数据集，能够存储PB级别的数据，并且支持实时查询。Hbase 提供了强一致性的读写操作，是大数据领域中实时分析的关键组件。《Hbase in Action》这本书旨在帮助读者深入理解 Hbase 的核心概念和技术，提高在实际项目中的应用能力。作者 Nick Dimiduk 和 Amandeep Khurana 都是 Hbase 领域的专家，他们通过实践经验分享了如何有效地使用 Hbase 来管理大规模数据。书中可能会涵盖以下主题： 1. **Hbase 架构**：包括 Region Server、Master Server、Zookeeper 在 Hbase 中的角色，以及如何通过 Region 分区来优化数据分布。 2. **数据模型**：介绍列族（Column Family）、列（Column）、行（Row）和时间戳的概念，以及如何设计高效的数据模型。 3. **API 使用**：讲解 Java API 和 Shell 命令行工具的使用，以及如何进行数据的插入、查询、更新和删除操作。 4. **性能优化**：讨论 Hbase 的读写性能优化策略，如 Region 分裂、BlockCache、Compaction 等。 5. **监控与故障排查**：介绍如何监控 Hbase 集群的健康状况，以及在遇到问题时如何进行诊断和修复。 6. **扩展与集成**：涵盖 Hbase 与其他系统（如 Hadoop、Hive、Spark）的集成，以及如何进行水平扩展以满足更大的数据需求。 7. **实战案例**：可能包含实际业务场景下的应用示例，帮助读者将理论知识转化为实际操作能力。技术编辑 Mark Henry Ryan 的参与确保了书中的技术内容准确无误，对于想要深入了解 Hbase 的开发者和数据工程师来说，这是一本不可多得的参考书籍。通过阅读本书，读者不仅可以掌握 Hbase 的基础操作，还能了解到如何在复杂环境中管理和优化 Hbase 集群，提升数据处理效率。在大数据时代，对 Hbase 的深入理解和应用将对个人和企业的数据战略产生重大影响。

preface

I got my start with

ase in the fall of 2008. It was a young project then, released only

in the preceding year. As early releases go, it was quite capable, although not without its

fair share of embarrassing warts. Not bad for an Apache subproject with fewer than 10

active committers to its name! That was the height of the No

SQL

hype. The term No

SQL

hadn’t even been presented yet but would come into common parlance over the next

year. No one could articulate why the idea was important—only that it was important—

and everyone in the open source data community was obsessed with this concept. The

community was polarized, with people either bashing relational databases for their fool-

ish rigidity or mocking these new technologies for their lack of sophistication.

The people exploring this new idea were mostly in internet companies, and I came

to work for such a company—a startup interested in the analysis of social media con-

tent. Facebook still enforced its privacy policies then, and Twitter wasn’t big enough to

know what a Fail Whale was yet. Our interest at the time was mostly in blogs. I left a

company where I’d spent the better part of three years working on a hierarchical data-

base engine. We made extensive use of Berkeley

, so I was familiar with data tech-

nologies that didn’t have a

SQL

engine. I joined a small team tasked with building a

new data-management platform. We had an

SQL

database stuffed to the gills with

blog posts and comments. When our daily analysis jobs breached the 18-hour mark,

we knew the current system’s days were numbered.

After cataloging a basic set of requirements, we set out to find a new data technol-

ogy. We were a small team and spent months evaluating different options while main-

taining current systems. We experimented with different approaches and learned

PREFACE

xvi

firsthand the pains of manually partitioning data. We studied the

CAP

theorem and

eventual consistency—and the tradeoffs. Despite its warts, we decided on

ase, and

we convinced our manager that the potential benefits outweighed the risks he saw in

open source technology.

I’d played a bit with Hadoop at home but had never written a real MapReduce job.

I’d heard of

ase but wasn’t particularly interested in it until I was in this new posi-

tion. With the clock ticking, there was nothing to do but jump in. We scrounged up a

couple of spare machines and a bit of rack, and then we were off and running. It was a

NET

shop, and we had no operational help, so we learned to combine bash with rsync

and managed the cluster ourselves.

I joined the mailing lists and the

IRC

channel and started asking questions. Around

this time, I met Amandeep. He was working on his master’s thesis, hacking up

ase

to run on systems other than Hadoop. Soon he finished school, joined Amazon, and

moved to Seattle. We were among the very few HBase-ers in this extremely Microsoft-

centric city. Fast-forward another two years…

The idea of

ase in Action was first proposed to us in the fall of 2010. From my

perspective, the project was laughable. Why should we, two community members,

write a book about

ase? Internally, it’s a complex beast. The Definitive Guide was still

a work in progress, but we both knew its author, a committer, and were well aware of

the challenge before him. From the outside, I thought it’s just a “simple key-value

store.” The

API

has only five concepts, none of which is complex. We weren’t going to

write another internals book, and I wasn’t convinced there was enough going on from

the application developer’s perspective to justify an entire book.

We started brainstorming the project, and it quickly became clear that I was wrong.

Not only was there enough material for a user’s guide, but our position as community

members made us ideal candidates to write such a book. We set out to catalogue the

useful bits of knowledge we’d each accumulated over the couple of years we’d used

the technology. That effort—this book—is the distillation of our eight years of com-

bined

ase experience. It’s targeted to those brand new to

ase, and it provides

guidance over the stumbling blocks we encountered during our own journeys. We’ve

collected and codified as much as we could of the tribal knowledge floating around

the community. Wherever possible, we prefer concrete direction to vague advice. Far

more than a simple

FAQ

, we hope you’ll find this book to be a complete manual to

getting off the ground with

ase.

ase is now stabilizing. Most of the warts we encountered when we began with the

project have been cleaned up, patched, or completely re-architected.

ase is

approaching its 1.0 release, and we’re proud to be part of this community as we

approach this milestone. We’re proud to present this manuscript to the community in

hopes that it will encourage and enable the next generation of HBase users. The sin-

gle strongest component of

ase is its thriving community—we hope you’ll join us in

that community and help it continue to innovate in this new era of data systems.

ICK

IMIDUK

PREFACE

xvii

If you’re reading this, you’re presumably interested in knowing how I got involved

with

ase. Let me start by saying thank you for choosing this book as your means to

learn about

ase and how to build applications that use

ase as their underlying

storage system. I hope you’ll find the text useful and learn some neat tricks that will

help you build better applications and enable you to succeed.

I was pursuing graduate studies in computer science at

Santa Cruz, specializing

in distributed systems, when I started working at Cisco as a part-time researcher. The

team I was working with was trying to build a data-integration framework that could

integrate, index, and allow exploration of data residing in hundreds of heterogeneous

data stores, including but not limited to large

RDBMS

systems. We started looking for

systems and solutions that would help us solve the problems at hand. We evaluated

many different systems, from object databases to graph databases, and we considered

building a custom distributed data-storage layer backed by Berkeley

. It was clear

that one of the key requirements was scalability, and we didn’t want to build a full-

fledged distributed system. If you’re in a situation where you think you need to build

out a custom distributed database or file system, think again—try to see if an existing

solution can solve part of your problem.

Following that principle, we decided that building out a new system wasn’t the best

approach and to use an existing technology instead. That was when I started playing

with the Hadoop ecosystem, getting my hands dirty with the different components in

the stack and going on to build a proof-of-concept for the data-integration system on

top of

ase. It actually worked and scaled well!

ase was well-suited to the problem,

but these were young projects at the time—and one of the things that ensured our

success was the community.

ase has one of the most welcoming and vibrant open

source communities; it was much smaller at the time, but the key principles were the

same then as now.

The data-integration project later became my master’s thesis. The project used

ase at its core, and I became more involved with the community as I built it out. I

asked questions, and, with time, answered questions others asked, on both the mailing

lists and the

IRC

channel. This is when I met Nick and got to know what he was work-

ing on. With each day that I worked on this project, my interest and love for the tech-

nology and the open source community grew, and I wanted to stay involved.

After finishing grad school, I joined Amazon in Seattle to work on back-end distrib-

uted systems projects. Much of my time was spent with the Elastic MapReduce team,

building the first versions of their hosted

ase offering. Nick also lived in Seattle,

and we met often and talked about the projects we were working on. Toward the end

of 2010, the idea of writing HBase in Action for Manning came up. We initially scoffed

at the thought of writing a book on

ase, and I remember saying to Nick, “It’s gets,

puts, and scans—there’s not a lot more to

ase from the client side. Do you want to

write a book about three

API

calls?”

But the more we thought about this, the more we realized that building applications

with

ase was challenging and there wasn’t enough material to help people get off the

剩余361页未读，继续阅读

bluekeyv

粉丝: 201
资源: 3

HBase实战：深度探索大数据存储

HBase in Action(中文)

HBase in action

HBase in Action

hbase实战（HBase in Action）

HBase实战(HBase In Action)

HBase in Action：英文原版解读

HBase in Action：深入理解与应用

HBase in Action by Nick Dimiduk and Amandeep Khurana

《HBase in Action》：实践指南与英文原著详解

hbase in action 英文版

最新资源