Apache HBase参考指南：入门与数据模型解析

需积分: 1 67 浏览量更新于2024-07-09 收藏 14.1MB PDF 举报

"Apache HBase参考指南，英文原版，涵盖了HBase的基础配置、升级路径、数据模型等核心概念。" Apache HBase是一款基于列族的分布式数据库，属于Apache Hadoop项目的一部分，专为处理大数据设计。它提供了低延迟的数据访问，并能够支持海量数据存储。在大数据领域，HBase常用于实时查询和分析大规模数据。该参考指南分为多个部分，首先介绍了HBase的基本概念，包括入门、版本兼容性以及升级策略。在"Getting Started"章节中，读者可以学习如何搭建单机版和分布式版的HBase环境。配置文件是HBase运行的关键，指南详细阐述了配置文件的结构和基本要求，如`hbase-site.xml`和`hbase-env.sh`。 "HBase run modes"章节讲解了独立模式和分布式模式的区别，独立模式适用于开发和测试，而分布式模式则适合大规模生产环境。确认安装正确并理解默认配置是运维HBase的基础，指南提供了检查安装和配置的例子。 "Dynamic Configuration"章节讨论了如何在运行时动态调整HBase的配置，这对于应对不断变化的系统需求非常重要。升级章节则涵盖了不同HBase版本间的兼容性和升级路径，包括回滚和升级策略。此外，指南还深入介绍了HBase的数据模型，这是理解其工作原理的关键。数据模型包括概念视图和物理视图，涉及命名空间、表、行、列族和单元格等基本元素。每个单元格可以有多个版本，支持时间戳，而且HBase的排序规则也是数据操作中的一个重要方面。 "Column Metadata"章节讨论了列的元数据，这些信息用于定义列族的属性，如压缩、编码和块大小等，对于优化存储和查询性能至关重要。这份Apache HBase参考指南是学习和操作HBase的宝贵资源，无论是新手还是经验丰富的开发者，都能从中获取有价值的信息。通过深入理解这些概念和实践，可以更好地利用HBase处理大数据场景下的复杂任务。

hbase(main):003:0> describe 'test'

Table test is ENABLED

test

COLUMN FAMILIES DESCRIPTION

{NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',

NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE

'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',

REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'f

alse', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',

PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true',

BLOCKSIZE

Ê=> '65536'}

1 row(s)

Took 0.9998 seconds

5. Put data into your table.

To put data into your table, use the put command.

hbase(main):003:0> put 'test', 'row1', 'cf:a', 'value1'

0 row(s) in 0.0850 seconds

hbase(main):004:0> put 'test', 'row2', 'cf:b', 'value2'

0 row(s) in 0.0110 seconds

hbase(main):005:0> put 'test', 'row3', 'cf:c', 'value3'

0 row(s) in 0.0100 seconds

Here, we insert three values, one at a time. The first insert is at row1, column cf:a, with a value

of value1. Columns in HBase are comprised of a column family prefix, cf in this example,

followed by a colon and then a column qualifier suffix, a in this case.

6. Scan the table for all data at once.

One of the ways to get data from HBase is to scan. Use the scan command to scan the table for

data. You can limit your scan, but for now, all data is fetched.

hbase(main):006:0> scan 'test'

ROW COLUMN+CELL

Êrow1 column=cf:a, timestamp=1421762485768,

value=value1

Êrow2 column=cf:b, timestamp=1421762491785,

value=value2

Êrow3 column=cf:c, timestamp=1421762496210,

value=value3

3 row(s) in 0.0230 seconds

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through the quickstart using standalone mode, you can re-configure HBase

to run in pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs

completely on a single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper)

runs as a separate process. Previously in standalone mode, all these daemons ran in a single jvm

process, and your data was stored under hbase.tmp.dir. In this walk-through, your data will be

stored in in HDFS instead, assuming you have HDFS available. This is optional; you can skip the

HDFS configuration to continue storing your data in the local filesystem.



Hadoop Configuration

This procedure assumes that you have configured Hadoop and HDFS on your local

system and/or a remote system, and that they are running and available. It also

assumes you are using Hadoop 2. The guide on Setting up a Single Node Cluster in

the Hadoop documentation is a good starting point.

1. Stop HBase if it is running.

If you have just finished quickstart and HBase is still running, stop it. This procedure will create

a totally new directory where HBase will store its data, so any databases you created before will

be lost.

2. Configure HBase.

Edit the hbase-site.xml configuration. First, add the following property which directs HBase to

run in distributed mode, with one JVM instance per daemon.

Ê <name>hbase.cluster.distributed</name>

Ê <value>true</value>

</property>

Next, add a configuration for hbase.rootdir so that it points to the address of your HDFS

instance, using the hdfs://// URI syntax. In this example, HDFS is running on the localhost at

port 8020.

Ê <name>hbase.rootdir</name>

Ê <value>hdfs://localhost:8020/hbase</value>

</property>

You do not need to create the directory in HDFS; HBase will do this for you. If you create the

directory, HBase will attempt to do a migration, which is not what you want.

Finally, remove the configuration for hbase.tmp.dir.

3. Start HBase.

Use the bin/start-hbase.sh command to start HBase. If your system is configured correctly, the

jps command should show the HMaster and HRegionServer processes running.

4. Check the HBase directory in HDFS.

If everything worked correctly, HBase created its directory in HDFS. In the configuration above,

it is stored in /hbase/ on HDFS. You can use the hadoop fs command in Hadoop’s bin/ directory to

list this directory.

$ ./bin/hadoop fs -ls /hbase

Found 7 items

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/.tmp

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/WALs

drwxr-xr-x - hbase users 0 2014-06-25 18:48 /hbase/corrupt

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/data

-rw-r--r-- 3 hbase users 42 2014-06-25 18:41 /hbase/hbase.id

-rw-r--r-- 3 hbase users 7 2014-06-25 18:41 /hbase/hbase.version

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/oldWALs

5. Create a table and populate it with data.

You can use the HBase Shell to create a table, populate it with data, scan and get values from it,

using the same procedure as in shell exercises.

6. Start and stop a backup HBase Master (HMaster) server.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses two ports (16000 and 16010 by default). The port

offset is added to these ports, so using an offset of 2, the backup HMaster would use ports 16002

and 16012. The following command starts 3 backup servers using ports 16002/16012,

16003/16013, and 16005/16015.

$ ./bin/local-master-backup.sh start 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

Since HBase version 1.1.0, HMaster doesn’t use region server ports, this leaves 10 ports (16020 to

16029 and 16030 to 16039) to be used for RegionServers. For supporting additional

RegionServers, set environment variables HBASE_RS_BASE_PORT and

HBASE_RS_INFO_BASE_PORT to appropriate values before running script local-

regionservers.sh. e.g. With values 16200 and 16300 for base ports, 99 additional RegionServers

can be supported, on a server. The following command starts four additional RegionServers,

running on sequential ports starting at 16022/16032 (base ports 16020/16030 plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

8. Stop HBase.

You can stop HBase the same way as in the quickstart procedure, using the bin/stop-hbase.sh

command.

2.4. Advanced - Fully Distributed

In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world

scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs

剩余859页未读，继续阅读

紧到长卜帅

粉丝: 53
资源: 3

Apache HBase参考指南：入门与数据模型解析

apache_hbase_reference_guide(官网).pdf

apache_hbase_reference_guide-3.0.0-SNAPSHOT

apache_hbase_reference_guide_1.1.2

hbase详解，apache_hbase_reference_guide

Aapache_hbase_reference_guide

Apache HBase ™ Reference Guide.zip

apache hbase reference guide pdf

apache hbase reference guide

Apache HBase ™ Reference Guide

Apache HBase权威参考指南：英文PDF

最新资源