Apache HBase 3.0入门与配置详解

需积分: 10 170 浏览量更新于2024-07-16 收藏 14.05MB PDF 举报

Apache HBase Reference Guide 是一份详细的文档，专为那些寻求理解和使用 Apache HBase 的用户提供指导。HBase 是一个开源的分布式、可扩展的大数据存储系统，旨在支持在大规模硬件集群上处理海量数据（数十亿行X数百万列），特别适合需要实时、随机读写的场景，如在线数据分析和日志处理。它借鉴了 Google 的 Bigtable 数据存储模型，通过在 Hadoop 和 HDFS 的基础上实现类似的功能。该指南分为多个章节，涵盖了从安装与配置到高级概念和技术细节的方方面面。首先，用户可以通过 Preface 了解背景和目标，接着是 Getting Started 部分，帮助读者快速上手独立运行 HBase。核心内容包括： 1. **配置文件管理**：从基础配置文件介绍开始，解释了 HBase 配置的重要性，并探讨了不同运行模式（单机模式和分布式模式）的设置。 2. **安装与验证**：详述如何安装和配置 HBase，确保系统的正确运行。 3. **配置示例与关键配置**：提供了示例配置以及必须关注的关键配置项，这对于性能优化至关重要。 4. **升级和兼容性**：介绍了版本号体系和回滚策略，以及升级路径的选择。 5. **HBase Shell**：讲解了 shell 的使用，包括非交互模式、脚本编写、命令文件读取以及传递 VM 参数等，这有助于管理和操作数据。 6. **数据模型**：深入剖析了 HBase 的概念模型，包括命名空间、表、行、列族、单元格等基本元素，以及它们之间的关系和操作。 7. **数据模型操作**：涉及版本管理、排序顺序等高级特性，展示了数据如何在 HBase 中高效地存储和查询。这份参考手册对于任何希望在实际环境中使用 HBase 或对其内部机制有深入了解的开发人员和管理员来说，都是一个不可或缺的工具。通过阅读和实践其中的内容，用户可以更好地利用 HBase 在大数据处理场景中的优势。

hbase(main):003:0> describe 'test'

Table test is ENABLED

test

COLUMN FAMILIES DESCRIPTION

{NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',

NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE

'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',

REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'f

alse', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',

PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true',

BLOCKSIZE

Ê=> '65536'}

1 row(s)

Took 0.9998 seconds

5. Put data into your table.

To put data into your table, use the put command.

hbase(main):003:0> put 'test', 'row1', 'cf:a', 'value1'

0 row(s) in 0.0850 seconds

hbase(main):004:0> put 'test', 'row2', 'cf:b', 'value2'

0 row(s) in 0.0110 seconds

hbase(main):005:0> put 'test', 'row3', 'cf:c', 'value3'

0 row(s) in 0.0100 seconds

Here, we insert three values, one at a time. The first insert is at row1, column cf:a, with a value

of value1. Columns in HBase are comprised of a column family prefix, cf in this example,

followed by a colon and then a column qualifier suffix, a in this case.

6. Scan the table for all data at once.

One of the ways to get data from HBase is to scan. Use the scan command to scan the table for

data. You can limit your scan, but for now, all data is fetched.

hbase(main):006:0> scan 'test'

ROW COLUMN+CELL

Êrow1 column=cf:a, timestamp=1421762485768,

value=value1

Êrow2 column=cf:b, timestamp=1421762491785,

value=value2

Êrow3 column=cf:c, timestamp=1421762496210,

value=value3

3 row(s) in 0.0230 seconds

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through the quickstart using standalone mode, you can re-configure HBase

to run in pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs

completely on a single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper)

runs as a separate process. Previously in standalone mode, all these daemons ran in a single jvm

process, and your data was stored under hbase.tmp.dir. In this walk-through, your data will be

stored in in HDFS instead, assuming you have HDFS available. This is optional; you can skip the

HDFS configuration to continue storing your data in the local filesystem.



Hadoop Configuration

This procedure assumes that you have configured Hadoop and HDFS on your local

system and/or a remote system, and that they are running and available. It also

assumes you are using Hadoop 2. The guide on Setting up a Single Node Cluster in

the Hadoop documentation is a good starting point.

1. Stop HBase if it is running.

If you have just finished quickstart and HBase is still running, stop it. This procedure will create

a totally new directory where HBase will store its data, so any databases you created before will

be lost.

2. Configure HBase.

Edit the hbase-site.xml configuration. First, add the following property which directs HBase to

run in distributed mode, with one JVM instance per daemon.

Ê <name>hbase.cluster.distributed</name>

Ê <value>true</value>

</property>

Next, add a configuration for hbase.rootdir so that it points to the address of your HDFS

instance, using the hdfs://// URI syntax. In this example, HDFS is running on the localhost at

port 8020.

Ê <name>hbase.rootdir</name>

Ê <value>hdfs://localhost:8020/hbase</value>

</property>

You do not need to create the directory in HDFS; HBase will do this for you. If you create the

directory, HBase will attempt to do a migration, which is not what you want.

Finally, remove the configuration for hbase.tmp.dir.

3. Start HBase.

Use the bin/start-hbase.sh command to start HBase. If your system is configured correctly, the

jps command should show the HMaster and HRegionServer processes running.

4. Check the HBase directory in HDFS.

If everything worked correctly, HBase created its directory in HDFS. In the configuration above,

it is stored in /hbase/ on HDFS. You can use the hadoop fs command in Hadoop’s bin/ directory to

list this directory.

$ ./bin/hadoop fs -ls /hbase

Found 7 items

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/.tmp

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/WALs

drwxr-xr-x - hbase users 0 2014-06-25 18:48 /hbase/corrupt

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/data

-rw-r--r-- 3 hbase users 42 2014-06-25 18:41 /hbase/hbase.id

-rw-r--r-- 3 hbase users 7 2014-06-25 18:41 /hbase/hbase.version

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/oldWALs

5. Create a table and populate it with data.

You can use the HBase Shell to create a table, populate it with data, scan and get values from it,

using the same procedure as in shell exercises.

6. Start and stop a backup HBase Master (HMaster) server.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses two ports (16000 and 16010 by default). The port

offset is added to these ports, so using an offset of 2, the backup HMaster would use ports 16002

and 16012. The following command starts 3 backup servers using ports 16002/16012,

16003/16013, and 16005/16015.

$ ./bin/local-master-backup.sh start 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

Since HBase version 1.1.0, HMaster doesn’t use region server ports, this leaves 10 ports (16020 to

16029 and 16030 to 16039) to be used for RegionServers. For supporting additional

RegionServers, set environment variables HBASE_RS_BASE_PORT and

HBASE_RS_INFO_BASE_PORT to appropriate values before running script local-

regionservers.sh. e.g. With values 16200 and 16300 for base ports, 99 additional RegionServers

can be supported, on a server. The following command starts four additional RegionServers,

running on sequential ports starting at 16022/16032 (base ports 16020/16030 plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

8. Stop HBase.

You can stop HBase the same way as in the quickstart procedure, using the bin/stop-hbase.sh

command.

2.4. Advanced - Fully Distributed

In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world

scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs

剩余856页未读，继续阅读

ddttoop

粉丝: 44
资源: 16

Apache HBase 3.0入门与配置详解

apache_hbase_reference_guide_1.1.2

HbaseReferenceBook-Hbase参考指南英文版

Apache HBase ™ Reference Guide

apache_hbase_reference_guide(官网).pdf

hbase详解，apache_hbase_reference_guide

apache_hbase_reference_guide-3.0.0-SNAPSHOT

Aapache_hbase_reference_guide

Apache HBase ™ Reference Guide.zip

apache hbase reference guide pdf

apache hbase reference guide

最新资源