Apache HBase 3.0官方指南：配置与数据模型详解

需积分: 10 129 浏览量更新于2024-07-19 收藏 11.18MB PDF 举报

Apache HBase Reference Guide 是一份由 Apache HBase 团队编写的官方文档，适用于 HBase 版本 3.0.0-SNAPSHOT。该文档详细介绍了 HBase 的各个方面，旨在帮助用户快速上手并深入理解其工作原理。以下是一些关键知识点的概述： 1. 简介 (Chapter 1): 介绍了 HBase 的基本概念，包括其分布式、NoSQL 数据存储系统的特点，以及它如何处理大规模数据，并支持高并发读写操作。 2. 快速入门 (Chapter 2): 提供了一个简明的步骤指南，针对 Standalone 模式下的 HBase 安装和配置，帮助读者迅速建立一个本地环境进行实践。 3. 配置管理 (Chapters 3-9): 配置文件的管理和设置是核心内容，包括默认配置、示例配置和重要配置项的解释，以及动态配置和升级策略。 4. 安装与验证 (Chapter 6): 如何运行 HBase 并确认其正确安装，包括启动模式（Standalone 和 Distributed）的区别。 5. Shell工具 (Chapters 10-19): 着重于 HBase Shell 的使用，如非交互模式、脚本支持、命令文件读取、VM选项传递以及一些实用技巧。 6. 数据模型 (Chapter 20-27): 介绍了 HBase 的数据模型，包括概念视图（数据在内存和磁盘中的组织）、物理视图、命名空间、表、行、列族和单元格等核心组件。 7. 数据操作 (Chapter 28-31): 数据模型的操作，如版本管理、排序规则、列元数据以及数据的合并和查询。 8. ACID属性 (Chapter 32): 说明 HBase 如何实现基本的 ACID（原子性、一致性、隔离性和持久性）事务处理原则，尽管其设计重点在于大量分布式数据处理的性能。 9. schema设计 (Chapter 135): 关注如何根据业务需求设计合理的数据模型和表结构，确保数据的有效管理和查询效率。 10. 升级与兼容性 (Chapters 11-12): 说明了不同版本之间的兼容性以及如何进行平稳的版本升级和回滚策略。这份文档对于初学者和有经验的 HBase 用户都是宝贵的参考资料，不仅提供了详尽的配置指导，还深入剖析了 HBase 的内部机制，是理解和优化 HBase 系统不可或缺的工具。无论是新手想要快速入门，还是高级开发者寻求技术细节，都能从中找到所需的信息。

hbase(main):008:0> disable 'test'

0 row(s) in 1.1820 seconds

hbase(main):009:0> enable 'test'

0 row(s) in 0.1770 seconds

Disable the table again if you tested the enable command above:

hbase(main):010:0> disable 'test'

0 row(s) in 1.1820 seconds

9. Drop the table.

To drop (delete) a table, use the drop command.

hbase(main):011:0> drop 'test'

0 row(s) in 0.1370 seconds

10. Exit the HBase Shell.

To exit the HBase Shell and disconnect from your cluster, use the quit command. HBase is still

running in the background.

Procedure: Stop HBase

1. In the same way that the bin/start-hbase.sh script is provided to conveniently start all HBase

daemons, the bin/stop-hbase.sh script stops them.

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through quickstart standalone mode, you can re-configure HBase to run in

pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs completely on a

single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate

process: in standalone mode all daemons ran in one jvm process/instance. By default, unless you

configure the hbase.rootdir property as described in quickstart, your data is still stored in /tmp/. In

this walk-through, we store your data in HDFS instead, assuming you have HDFS available. You can

$ ./bin/hadoop fs -ls /hbase

Found 7 items

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/.tmp

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/WALs

drwxr-xr-x - hbase users 0 2014-06-25 18:48 /hbase/corrupt

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/data

-rw-r--r-- 3 hbase users 42 2014-06-25 18:41 /hbase/hbase.id

-rw-r--r-- 3 hbase users 7 2014-06-25 18:41 /hbase/hbase.version

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/oldWALs

5. Create a table and populate it with data.

You can use the HBase Shell to create a table, populate it with data, scan and get values from it,

using the same procedure as in shell exercises.

6. Start and stop a backup HBase Master (HMaster) server.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default).

The port offset is added to these ports, so using an offset of 2, the backup HMaster would use

ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports

16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

$ ./bin/local-master-backup.sh 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

However, the base ports for additional RegionServers are not the default ports since the default

ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base

ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a

HMaster or backup HMaster, on a server. The following command starts four additional

RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300

plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

8. Stop HBase.

You can stop HBase the same way as in the quickstart procedure, using the bin/stop-hbase.sh

command.

2.4. Advanced - Fully Distributed

In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world

scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs

one or more HBase daemon. These include primary and backup Master instances, multiple

ZooKeeper nodes, and multiple RegionServer nodes.

This advanced quickstart adds two more nodes to your cluster. The architecture will be as follows:

Table 1. Distributed Cluster Demo Architecture

Node Name Master ZooKeeper RegionServer

node-a.example.com yes yes no

node-b.example.com backup yes yes

node-c.example.com no yes yes

This quickstart assumes that each node is a virtual machine and that they are all on the same

network. It builds upon the previous quickstart, Pseudo-Distributed Local Install, assuming that the

system you configured in that procedure is now node-a. Stop HBase on node-a before continuing.



Be sure that all the nodes have full access to communicate, and that no firewall

rules are in place which could prevent them from talking to each other. If you see

any errors like no route to host, check your firewall.

Procedure: Configure Passwordless SSH Access

node-a needs to be able to log into node-b and node-c (and to itself) in order to start the daemons.

The easiest way to accomplish this is to use the same username on all hosts, and configure

password-less SSH login from node-a to each of the others.

1. On node-a, generate a key pair.

While logged in as the user who will run HBase, generate a SSH key pair, using the following

command:

$ ssh-keygen -t rsa

If the command succeeds, the location of the key pair is printed to standard output. The default

name of the public key is id_rsa.pub.

2. Create the directory that will hold the shared keys on the other nodes.

On node-b and node-c, log in as the HBase user and create a .ssh/ directory in the user’s home

directory, if it does not already exist. If it already exists, be aware that it may already contain

other keys.

3. Copy the public key to the other nodes.

Securely copy the public key from node-a to each of the nodes, by using the scp or some other

secure means. On each of the other nodes, create a new file called .ssh/authorized_keys if it does

not already exist, and append the contents of the id_rsa.pub file to the end of it. Note that you

also need to do this for node-a itself.

$ cat id_rsa.pub >> ~/.ssh/authorized_keys

4. Test password-less login.

If you performed the procedure correctly, you should not be prompted for a password when

you SSH from node-a to either of the other nodes using the same username.

5. Since node-b will run a backup Master, repeat the procedure above, substituting node-b

everywhere you see node-a. Be sure not to overwrite your existing .ssh/authorized_keys files, but

concatenate the new key onto the existing file using the >> operator rather than the > operator.

Procedure: Prepare node-a

node-a will run your primary master and ZooKeeper processes, but no RegionServers. Stop the

RegionServer from starting on node-a.

1. Edit conf/regionservers and remove the line which contains localhost. Add lines with the

hostnames or IP addresses for node-b and node-c.

Even if you did want to run a RegionServer on node-a, you should refer to it by the hostname the

other servers would use to communicate with it. In this case, that would be node-a.example.com.

This enables you to distribute the configuration to each node of your cluster any hostname

conflicts. Save the file.

剩余671页未读，继续阅读

tangyaya8

粉丝: 46
资源: 7

Apache HBase 3.0官方指南：配置与数据模型详解

apache_hbase_reference_guide_1.1.2

HbaseReferenceBook-Hbase参考指南英文版

Apache HBase ™ Reference Guide

import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.*; import org.apache.hadoop.hbase.client.*; import org.apache.hadoop.hbase.util.Bytes;

hbase org.apache.hadoop.hbase.tool.LoadIncrementalHFiles 命令

使用org.apache.hadoop.hbase.client进行HBase连接时，如何在失败后，不进行重试

最新资源

import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.; import org.apache.hadoop.hbase.client.; import org.apache.hadoop.hbase.util.Bytes;