HBase in Action：深入理解与应用 - CSDN文库

4星 · 超过85%的资源需积分: 35 111 浏览量更新于2024-07-27 收藏 8.99MB PDF 举报

"Manning.HBase.in.Action 是一本由Nick Dimiduk和Amandeep Khurana合著，Michael Stack作序的书籍，专注于介绍Apache HBase这一分布式、高性能的NoSQL数据库。本书由Manning出版社出版，旨在提供深入理解和实际操作HBase的详细指南。" 在《HBase in Action》这本书中，作者们详细探讨了HBase的核心概念和技术，它是建立在Hadoop生态系统之上，专门用于处理大规模数据存储的数据库。HBase是基于列族的，提供了实时的读写访问，这使得它在大数据领域中成为一种强大的工具，特别适合处理半结构化和非结构化的数据。书中的内容可能涵盖了以下几个关键知识点： 1. **HBase概述**：介绍HBase的基本架构，包括其与Hadoop的关系，以及HDFS（Hadoop分布式文件系统）如何支持HBase的数据存储。 2. **数据模型**：详细解释HBase的数据模型，包括行、列族、列和时间戳的概念，以及如何通过这些元素来组织和查询数据。 3. **表设计**：讨论如何进行有效的表设计，以优化查询性能和数据分布，包括分区策略和索引的使用。 4. **操作HBase**：涵盖创建、修改和删除表，以及插入、查询和更新数据的基本命令和API，可能包括Java API和HBase shell。 5. **数据读写**：深入讲解HBase的数据读取和写入机制，包括RegionServer的角色，以及HBase如何实现高可用性和容错性。 6. **监控和调优**：介绍如何监控HBase集群的性能，并进行必要的调优，以确保系统的稳定性和高效运行。 7. **扩展性和集群管理**：讨论如何随着数据量的增长来扩展HBase集群，包括Region分裂和负载均衡策略。 8. **安全与权限控制**：讲解如何在HBase中实现用户认证、授权和数据加密，以保障数据的安全性。 9. **应用案例**：书中可能会包含一些真实世界的使用场景和案例研究，展示HBase在不同行业的实际应用，如日志分析、物联网(IoT)数据存储等。 10. **最佳实践**：总结在使用HBase时应遵循的最佳实践，帮助读者避免常见的陷阱并提升开发效率。通过本书，读者不仅可以学习到HBase的基本概念和技术，还能获得在实际项目中运用HBase的宝贵经验。无论是开发者、数据工程师还是系统管理员，都能从中受益，提升自己在大数据处理和存储领域的专业能力。

xv

preface

I got my start with HBase in the fall of 2008. It was a young project then, released only

in the preceding year. As early releases go, it was quite capable, although not without its

fair share of embarrassing warts. Not bad for an Apache subproject with fewer than 10

active committers to its name! That was the height of the No

SQL hype. The term NoSQL

hadn’t even been presented yet but would come into common parlance over the next

year. No one could articulate why the idea was important—only that it was important—

and everyone in the open source data community was obsessed with this concept. The

community was polarized, with people either bashing relational databases for their fool-

ish rigidity or mocking these new technologies for their lack of sophistication.

The people exploring this new idea were mostly in internet companies, and I came

to work for such a company—a startup interested in the analysis of social media con-

tent. Facebook still enforced its privacy policies then, and Twitter wasn’t big enough to

know what a Fail Whale was yet. Our interest at the time was mostly in blogs. I left a

company where I’d spent the better part of three years working on a hierarchical data-

base engine. We made extensive use of Berkeley

DB, so I was familiar with data tech-

nologies that didn’t have a SQL engine. I joined a small team tasked with building a

new data-management platform. We had an MS SQL database stuffed to the gills with

blog posts and comments. When our daily analysis jobs breached the 18-hour mark,

we knew the current system’s days were numbered.

After cataloging a basic set of requirements, we set out to find a new data technol-

ogy. We were a small team and spent months evaluating different options while main-

taining current systems. We experimented with different approaches and learned

Download from Wow! eBook <www.wowebook.com>

PREFACE

xvi

firsthand the pains of manually partitioning data. We studied the CAP theorem and

eventual consistency—and the tradeoffs. Despite its warts, we decided on HBase, and

we convinced our manager that the potential benefits outweighed the risks he saw in

open source technology.

I’d played a bit with Hadoop at home but had never written a real MapReduce job.

I’d heard of

HBase but wasn’t particularly interested in it until I was in this new posi-

tion. With the clock ticking, there was nothing to do but jump in. We scrounged up a

couple of spare machines and a bit of rack, and then we were off and running. It was a

.

NET shop, and we had no operational help, so we learned to combine bash with rsync

and managed the cluster ourselves.

I joined the mailing lists and the

IRC channel and started asking questions. Around

this time, I met Amandeep. He was working on his master’s thesis, hacking up HBase

to run on systems other than Hadoop. Soon he finished school, joined Amazon, and

moved to Seattle. We were among the very few HBase-ers in this extremely Microsoft-

centric city. Fast-forward another two years…

The idea of

HBase in Action was first proposed to us in the fall of 2010. From my

perspective, the project was laughable. Why should we, two community members,

write a book about

HBase? Internally, it’s a complex beast. The Definitive Guide was still

a work in progress, but we both knew its author, a committer, and were well aware of

the challenge before him. From the outside, I thought it’s just a “simple key-value

store.” The

API has only five concepts, none of which is complex. We weren’t going to

write another internals book, and I wasn’t convinced there was enough going on from

the application developer’s perspective to justify an entire book.

We started brainstorming the project, and it quickly became clear that I was wrong.

Not only was there enough material for a user’s guide, but our position as community

members made us ideal candidates to write such a book. We set out to catalogue the

useful bits of knowledge we’d each accumulated over the couple of years we’d used

the technology. That effort—this book—is the distillation of our eight years of com-

bined

HBase experience. It’s targeted to those brand new to HBase, and it provides

guidance over the stumbling blocks we encountered during our own journeys. We’ve

collected and codified as much as we could of the tribal knowledge floating around

the community. Wherever possible, we prefer concrete direction to vague advice. Far

more than a simple

FAQ, we hope you’ll find this book to be a complete manual to

getting off the ground with HBase.

HBase is now stabilizing. Most of the warts we encountered when we began with the

project have been cleaned up, patched, or completely re-architected. HBase is

approaching its 1.0 release, and we’re proud to be part of this community as we

approach this milestone. We’re proud to present this manuscript to the community in

hopes that it will encourage and enable the next generation of HBase users. The sin-

gle strongest component of

HBase is its thriving community—we hope you’ll join us in

that community and help it continue to innovate in this new era of data systems.

NICK DIMIDUK

Download from Wow! eBook <www.wowebook.com>

PREFACE

xvii

If you’re reading this, you’re presumably interested in knowing how I got involved

with HBase. Let me start by saying thank you for choosing this book as your means to

learn about HBase and how to build applications that use HBase as their underlying

storage system. I hope you’ll find the text useful and learn some neat tricks that will

help you build better applications and enable you to succeed.

I was pursuing graduate studies in computer science at

UC Santa Cruz, specializing

in distributed systems, when I started working at Cisco as a part-time researcher. The

team I was working with was trying to build a data-integration framework that could

integrate, index, and allow exploration of data residing in hundreds of heterogeneous

data stores, including but not limited to large

RDBMS systems. We started looking for

systems and solutions that would help us solve the problems at hand. We evaluated

many different systems, from object databases to graph databases, and we considered

building a custom distributed data-storage layer backed by Berkeley

DB. It was clear

that one of the key requirements was scalability, and we didn’t want to build a full-

fledged distributed system. If you’re in a situation where you think you need to build

out a custom distributed database or file system, think again—try to see if an existing

solution can solve part of your problem.

Following that principle, we decided that building out a new system wasn’t the best

approach and to use an existing technology instead. That was when I started playing

with the Hadoop ecosystem, getting my hands dirty with the different components in

the stack and going on to build a proof-of-concept for the data-integration system on

top of

HBase. It actually worked and scaled well! HBase was well-suited to the problem,

but these were young projects at the time—and one of the things that ensured our

success was the community.

HBase has one of the most welcoming and vibrant open

source communities; it was much smaller at the time, but the key principles were the

same then as now.

The data-integration project later became my master’s thesis. The project used

HBase at its core, and I became more involved with the community as I built it out. I

asked questions, and, with time, answered questions others asked, on both the mailing

lists and the

IRC channel. This is when I met Nick and got to know what he was work-

ing on. With each day that I worked on this project, my interest and love for the tech-

nology and the open source community grew, and I wanted to stay involved.

After finishing grad school, I joined Amazon in Seattle to work on back-end distrib-

uted systems projects. Much of my time was spent with the Elastic MapReduce team,

building the first versions of their hosted

HBase offering. Nick also lived in Seattle,

and we met often and talked about the projects we were working on. Toward the end

of 2010, the idea of writing HBase in Action for Manning came up. We initially scoffed

at the thought of writing a book on

HBase, and I remember saying to Nick, “It’s gets,

puts, and scans—there’s not a lot more to HBase from the client side. Do you want to

write a book about three API calls?”

But the more we thought about this, the more we realized that building applications

with

HBase was challenging and there wasn’t enough material to help people get off the

Download from Wow! eBook <www.wowebook.com>

剩余361页未读，继续阅读

wrangrongli115

粉丝: 3
资源: 32

最新资源