Apache HBase 3.0.0官方指南：配置详解与数据模型

需积分: 10 164 浏览量更新于2024-07-19 收藏 11.11MB PDF 举报

Apache HBase 3.0.0-SNAPSHOT官方参考指南是针对这个流行的开源分布式NoSQL数据库系统的重要文档。该指南详细介绍了HBase的基本架构、配置选项、运行模式、安装验证、升级策略以及其核心概念和技术细节。 1. **概述** (Introduction): 本部分提供了一个简洁的入门介绍，帮助读者理解HBase作为列式存储的非关系型数据库在大数据场景中的角色，以及它如何支持高吞吐量的数据读写操作。 2. **快速入门** (QuickStart): 指导用户在单机模式（Standalone）下搭建并运行HBase，以便快速体验其基本功能。 3. **配置管理** (Apache HBase Configuration): 从配置文件的结构、基本设置到高级配置选项，如默认配置和示例配置，这部分深入剖析了如何定制HBase以适应特定环境需求。 4. **环境准备** (Basic Prerequisites): 描述了运行HBase所需的最低硬件和软件要求，包括操作系统、Java环境等。 5. **运行模式** (HBaSe Runmodes): 区分了Standalone模式（适合开发测试）和Distributed模式（生产环境部署），阐述了它们之间的区别和适用场景。 6. **安装与验证** (Running and Confirming Your Installation): 提供了安装步骤和检查是否成功启动的详细方法。 7. **默认配置与重要配置** (Default Configuration & The Important Configurations): 强调了一些关键配置项，如数据块大小、内存分配、数据复制策略等，这些对性能和可靠性至关重要。 8. **动态配置** (Dynamic Configuration): 讨论了如何在运行时动态调整配置参数，以适应不断变化的需求。 9. **升级管理** (Upgrading): 包括HBase版本号和兼容性，以及升级路径建议，确保平稳过渡到新版本。 10. **HBase Shell**: 介绍HBase shell，它是与HBase交互的主要工具，涵盖了非交互模式、脚本支持、命令文件读取以及传递VM选项等内容。 11. **数据模型** (Data Model): 分析了HBase的概念性视图（如表、行、列族和单元格）、物理存储结构以及数据模型操作，如版本控制和排序规则。 12. **ACID特性** (ACID): 解释了HBase如何实现基本的ACID事务保障，尽管它的设计初衷更偏向于大规模分布式处理。 13. **HBase与schema设计** (HBase and Schema Design): 关注数据库设计，包括创建和管理表结构，以及如何利用HBase的特点进行灵活的schema设计。这份指南是HBase 3.0.0-SNAPSHOT的权威参考资料，对于HBase的新手和资深用户来说都是不可或缺的学习资料，它详细地覆盖了从安装到运维、再到数据管理和性能优化的各个环节。通过阅读和实践这份指南，用户可以深入理解和掌握HBase的核心原理和实用技巧。

hbase(main):008:0> disable 'test'

0 row(s) in 1.1820 seconds

hbase(main):009:0> enable 'test'

0 row(s) in 0.1770 seconds

Disable the table again if you tested the enable command above:

hbase(main):010:0> disable 'test'

0 row(s) in 1.1820 seconds

9. Drop the table.

To drop (delete) a table, use the drop command.

hbase(main):011:0> drop 'test'

0 row(s) in 0.1370 seconds

10. Exit the HBase Shell.

To exit the HBase Shell and disconnect from your cluster, use the quit command. HBase is still

running in the background.

Procedure: Stop HBase

1. In the same way that the bin/start-hbase.sh script is provided to conveniently start all HBase

daemons, the bin/stop-hbase.sh script stops them.

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through quickstart standalone mode, you can re-configure HBase to run in

pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs completely on a

single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate

process: in standalone mode all daemons ran in one jvm process/instance. By default, unless you

configure the hbase.rootdir property as described in quickstart, your data is still stored in /tmp/. In

this walk-through, we store your data in HDFS instead, assuming you have HDFS available. You can

$ ./bin/hadoop fs -ls /hbase

Found 7 items

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/.tmp

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/WALs

drwxr-xr-x - hbase users 0 2014-06-25 18:48 /hbase/corrupt

drwxr-xr-x - hbase users 0 2014-06-25 18:58 /hbase/data

-rw-r--r-- 3 hbase users 42 2014-06-25 18:41 /hbase/hbase.id

-rw-r--r-- 3 hbase users 7 2014-06-25 18:41 /hbase/hbase.version

drwxr-xr-x - hbase users 0 2014-06-25 21:49 /hbase/oldWALs

5. Create a table and populate it with data.

You can use the HBase Shell to create a table, populate it with data, scan and get values from it,

using the same procedure as in shell exercises.

6. Start and stop a backup HBase Master (HMaster) server.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default).

The port offset is added to these ports, so using an offset of 2, the backup HMaster would use

ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports

16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

$ ./bin/local-master-backup.sh 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

However, the base ports for additional RegionServers are not the default ports since the default

ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base

ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a

HMaster or backup HMaster, on a server. The following command starts four additional

RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300

plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

8. Stop HBase.

You can stop HBase the same way as in the quickstart procedure, using the bin/stop-hbase.sh

command.

2.4. Advanced - Fully Distributed

In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world

scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs

one or more HBase daemon. These include primary and backup Master instances, multiple

ZooKeeper nodes, and multiple RegionServer nodes.

This advanced quickstart adds two more nodes to your cluster. The architecture will be as follows:

Table 1. Distributed Cluster Demo Architecture

Node Name Master ZooKeeper RegionServer

node-a.example.com yes yes no

node-b.example.com backup yes yes

node-c.example.com no yes yes

This quickstart assumes that each node is a virtual machine and that they are all on the same

network. It builds upon the previous quickstart, Pseudo-Distributed Local Install, assuming that the

system you configured in that procedure is now node-a. Stop HBase on node-a before continuing.



Be sure that all the nodes have full access to communicate, and that no firewall

rules are in place which could prevent them from talking to each other. If you see

any errors like no route to host, check your firewall.

Procedure: Configure Passwordless SSH Access

node-a needs to be able to log into node-b and node-c (and to itself) in order to start the daemons.

The easiest way to accomplish this is to use the same username on all hosts, and configure

password-less SSH login from node-a to each of the others.

1. On node-a, generate a key pair.

While logged in as the user who will run HBase, generate a SSH key pair, using the following

command:

$ ssh-keygen -t rsa

If the command succeeds, the location of the key pair is printed to standard output. The default

name of the public key is id_rsa.pub.

2. Create the directory that will hold the shared keys on the other nodes.

On node-b and node-c, log in as the HBase user and create a .ssh/ directory in the user’s home

directory, if it does not already exist. If it already exists, be aware that it may already contain

other keys.

3. Copy the public key to the other nodes.

Securely copy the public key from node-a to each of the nodes, by using the scp or some other

secure means. On each of the other nodes, create a new file called .ssh/authorized_keys if it does

not already exist, and append the contents of the id_rsa.pub file to the end of it. Note that you

also need to do this for node-a itself.

$ cat id_rsa.pub >> ~/.ssh/authorized_keys

4. Test password-less login.

If you performed the procedure correctly, you should not be prompted for a password when

you SSH from node-a to either of the other nodes using the same username.

5. Since node-b will run a backup Master, repeat the procedure above, substituting node-b

everywhere you see node-a. Be sure not to overwrite your existing .ssh/authorized_keys files, but

concatenate the new key onto the existing file using the >> operator rather than the > operator.

Procedure: Prepare node-a

node-a will run your primary master and ZooKeeper processes, but no RegionServers. Stop the

RegionServer from starting on node-a.

1. Edit conf/regionservers and remove the line which contains localhost. Add lines with the

hostnames or IP addresses for node-b and node-c.

Even if you did want to run a RegionServer on node-a, you should refer to it by the hostname the

other servers would use to communicate with it. In this case, that would be node-a.example.com.

This enables you to distribute the configuration to each node of your cluster any hostname

conflicts. Save the file.

剩余661页未读，继续阅读

千与

粉丝: 351
资源: 13

Apache HBase 3.0.0官方指南：配置详解与数据模型

apache_hbase_reference_guide_1.1.2

HbaseReferenceBook-Hbase参考指南英文版

2018 Apache HBase 技术实战专刊

hbase详解，apache_hbase_reference_guide

apache_hbase_reference_guide.pdf

apache_hbase_reference_guide(官网).pdf

Aapache_hbase_reference_guide

apache-atlas-3.0.0-SNAPSHOT-bin.tar.gz

Idp_S_Mysql2Hbase-1.0-SNAPSHOT-jar-with-dependencies

xmljava系统源码-springboot_hbase_kafka:springboot_hbase_kafka

最新资源