Apache HBase参考指南：入门与数据模型

需积分: 5 118 浏览量更新于2024-06-24 收藏 11.1MB PDF 举报

"Apache HBase 参考指南" Apache HBase 是一个分布式的、基于列族的NoSQL数据库，它构建在Hadoop文件系统(HDFS)之上，提供了高度可伸缩性和实时读写能力。这份参考指南是针对Apache HBase 3.0.0-SNAPSHOT版本编写的，由Apache HBase团队提供，涵盖了从入门到高级配置和设计原则的广泛内容。 **入门与配置** 1. **快速入门 - 单独运行HBase**：这部分介绍了如何在本地环境中快速搭建并运行HBase，是初学者入门的基础。 2. **Apache HBase配置**：HBase的配置是其高效运行的关键，包括配置文件、基本前提条件、运行模式（独立和分布式）以及安装后的验证。 3. **默认配置**：详述了HBase启动时的默认配置参数，帮助理解系统的行为。 4. **动态配置**：讨论了如何在运行时更改配置，以适应不断变化的需求。 **升级** 1. **HBase版本号和兼容性**：介绍了HBase的版本管理策略和不同版本间的兼容性，对于维护和升级非常重要。 2. **升级路径**：提供了从一个版本升级到另一个版本的具体步骤和注意事项。 **HBase Shell** 1. **Apache HBase Shell**：HBase的命令行接口，用于交互式操作数据库。 2. **用Ruby脚本化**：允许通过Ruby语言编写脚本来操作HBase。 3. **非交互式模式**：介绍如何在非交互模式下运行Shell，适用于自动化任务。 4. **在OSScripts中使用Shell**：在操作系统脚本中调用HBase Shell命令的方法。 5. **从命令文件读取HBase Shell命令**：允许将多个命令存储在一个文件中，一次性执行。 6. **向Shell传递VM选项**：调整JVM参数以优化Shell性能。 7. **Shell技巧**：分享了一些提高Shell使用效率的技巧。 **数据模型** 1. **概念视图**：解释了HBase的核心概念，如表、行、列族和单元格。 2. **物理视图**：描述了数据在磁盘上的实际存储方式。 3. **命名空间**：HBase中的逻辑分区，用于组织表。 4. **表**：数据存储的基本单元，由行和列族组成。 5. **行**：数据的水平分组，由键值对组成。 6. **列族**：数据的垂直分组，包含一组相关的列。 7. **单元格**：数据的最小存储单元，具有特定的时间戳。 8. **数据模型操作**：列族和单元格上的操作，如读写和删除。 9. **版本**：每个单元格可以有多个版本，用于数据的历史记录。 10. **排序顺序**：HBase中的数据默认按行键排序。 11. **列元数据**：关于列族和列的附加信息。 **ACID特性与模式设计** 1. **ACID**：HBase支持事务的一致性、隔离性、持久性和原子性，保证数据的正确性。 2. **HBase和模式设计**：探讨如何根据业务需求设计有效的HBase表结构，包括列族选择、分区策略等。这份参考指南是HBase开发者和管理员的重要参考资料，涵盖了从基础操作到高级特性的全面内容，有助于用户深入理解和充分利用HBase的功能。

hbase(main):008:0> disable 'test'

0 row(s) in 1.1820 seconds

hbase(main):009:0> enable 'test'

0 row(s) in 0.1770 seconds

Disable the table again if you tested the enable command above:

hbase(main):010:0> disable 'test'

0 row(s) in 1.1820 seconds

9. Drop the table.

To drop (delete) a table, use the drop command.

hbase(main):011:0> drop 'test'

0 row(s) in 0.1370 seconds

10. Exit the HBase Shell.

To exit the HBase Shell and disconnect from your cluster, use the quit command. HBase is still

running in the background.

Procedure: Stop HBase

1. In the same way that the bin/start-hbase.sh script is provided to conveniently start all HBase

daemons, the bin/stop-hbase.sh script stops them.

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through quickstart standalone mode, you can re-configure HBase to run in

pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs completely on a

single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate

process: in standalone mode all daemons ran in one jvm process/instance. By default, unless you

configure the hbase.rootdir property as described in quickstart, your data is still stored in /tmp/. In

this walk-through, we store your data in HDFS instead, assuming you have HDFS available. You can

$ ./bin/hadoop fs -ls /hbase

Found 7 items

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/.tmp

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/WALs

drwxr-xr-x - hbase users 0 2014-06-25 18:48 /hbase/corrupt

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/data

-rw-r--r-- 3 hbase users 42 2014-06-25 18:41 /hbase/hbase.id

-rw-r--r-- 3 hbase users 7 2014-06-25 18:41 /hbase/hbase.version

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/oldWALs

5. Create a table and populate it with data.

You can use the HBase Shell to create a table, populate it with data, scan and get values from it,

using the same procedure as in shell exercises.

6. Start and stop a backup HBase Master (HMaster) server.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default).

The port offset is added to these ports, so using an offset of 2, the backup HMaster would use

ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports

16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

$ ./bin/local-master-backup.sh 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

However, the base ports for additional RegionServers are not the default ports since the default

ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base

ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a

HMaster or backup HMaster, on a server. The following command starts four additional

RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300

plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

8. Stop HBase.

You can stop HBase the same way as in the quickstart procedure, using the bin/stop-hbase.sh

command.

2.4. Advanced - Fully Distributed

In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world

scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs

one or more HBase daemon. These include primary and backup Master instances, multiple

ZooKeeper nodes, and multiple RegionServer nodes.

This advanced quickstart adds two more nodes to your cluster. The architecture will be as follows:

Table 1. Distributed Cluster Demo Architecture

Node Name Master ZooKeeper RegionServer

node-a.example.com yes yes no

node-b.example.com backup yes yes

node-c.example.com no yes yes

This quickstart assumes that each node is a virtual machine and that they are all on the same

network. It builds upon the previous quickstart, Pseudo-Distributed Local Install, assuming that the

system you configured in that procedure is now node-a. Stop HBase on node-a before continuing.



Be sure that all the nodes have full access to communicate, and that no firewall

rules are in place which could prevent them from talking to each other. If you see

any errors like no route to host, check your firewall.

Procedure: Configure Passwordless SSH Access

node-a needs to be able to log into node-b and node-c (and to itself) in order to start the daemons.

The easiest way to accomplish this is to use the same username on all hosts, and configure

password-less SSH login from node-a to each of the others.

1. On node-a, generate a key pair.

While logged in as the user who will run HBase, generate a SSH key pair, using the following

command:

$ ssh-keygen -t rsa

If the command succeeds, the location of the key pair is printed to standard output. The default

name of the public key is id_rsa.pub.

2. Create the directory that will hold the shared keys on the other nodes.

On node-b and node-c, log in as the HBase user and create a .ssh/ directory in the user’s home

directory, if it does not already exist. If it already exists, be aware that it may already contain

other keys.

3. Copy the public key to the other nodes.

Securely copy the public key from node-a to each of the nodes, by using the scp or some other

secure means. On each of the other nodes, create a new file called .ssh/authorized_keys if it does

not already exist, and append the contents of the id_rsa.pub file to the end of it. Note that you

also need to do this for node-a itself.

$ cat id_rsa.pub >> ~/.ssh/authorized_keys

4. Test password-less login.

If you performed the procedure correctly, you should not be prompted for a password when

you SSH from node-a to either of the other nodes using the same username.

5. Since node-b will run a backup Master, repeat the procedure above, substituting node-b

everywhere you see node-a. Be sure not to overwrite your existing .ssh/authorized_keys files, but

concatenate the new key onto the existing file using the >> operator rather than the > operator.

Procedure: Prepare node-a

node-a will run your primary master and ZooKeeper processes, but no RegionServers. Stop the

RegionServer from starting on node-a.

1. Edit conf/regionservers and remove the line which contains localhost. Add lines with the

hostnames or IP addresses for node-b and node-c.

Even if you did want to run a RegionServer on node-a, you should refer to it by the hostname the

other servers would use to communicate with it. In this case, that would be node-a.example.com.

This enables you to distribute the configuration to each node of your cluster any hostname

conflicts. Save the file.

剩余661页未读，继续阅读

大数据张老师

粉丝: 956
资源: 33

Apache HBase参考指南：入门与数据模型

apache hbase reference guide pdf

apache hbase reference guide

apache_hbase_reference_guide(官网).pdf

apache_hbase_reference_guide_1.1.2

hbase详解，apache_hbase_reference_guide

apache_hbase_reference_guide-3.0.0-SNAPSHOT

Aapache_hbase_reference_guide

Apache HBase ™ Reference Guide.zip

Apache HBase ™ Reference Guide

Apache HBase权威参考指南：英文PDF

最新资源