Apache HBase 3.0.0-SNAPSHOT 完整指南：配置、升级与数据模型详解

需积分: 9 108 浏览量更新于2024-07-16 收藏 13.14MB PDF 举报

Apache HBase Reference Guide 是一份详尽的指南，专为3.0.0-SNAPSHOT版本的HBase提供深入的介绍和操作指南。该文档涵盖了广泛的主题，旨在帮助用户从入门到精通HBase技术。 **1. Introduction (简介)** 本部分介绍了HBase的基本概念，包括其作为NoSQL数据库的角色，以及在大数据存储和处理中的应用。它强调了HBase的可扩展性和高吞吐量特性，适用于大规模数据集的实时查询和处理。 **2. Quick Start - Standalone HBase (快速入门 - 单机模式)** 这部分为初次接触HBase的用户提供了简单的安装和配置步骤，通过单机模式演示如何启动和运行HBase服务。 **3. Configuration Files (配置文件)** 这部分详细阐述了HBase的配置管理，包括系统级别的默认配置、示例配置和重要配置参数，强调了理解配置对性能和安全性的影响。 **4. Prerequisites (基本前提条件)** 讲解了运行HBase所需的最低系统要求，如Java环境、内存设置以及操作系统支持等。 **5. HBase Run Modes (运行模式)** 区分了Standalone（单机）和Distributed（分布式）两种运行模式，讨论了它们之间的差异和适用场景。 **6-7. Installation and Verification (安装与确认)** 指南指导用户安装HBase，并提供了验证安装成功的步骤，确保环境设置正确无误。 **8-10. Upgrading HBase (升级)** 这部分涵盖升级策略，包括版本号识别、回滚机制和升级路径选择，确保平稳地进行版本更新。 **11. The HBase Shell (HBase命令行工具)** 深入解析HBase Shell，不仅介绍了它的基本用法，还涵盖了非交互模式、脚本化操作、配置选项和一些实用技巧。 **12-21. Data Model (数据模型)** 这部分详细阐述了HBase的数据模型，包括概念视图、物理视图、命名空间、表、行、列族、单元格等核心组件，以及数据操作和版本控制。 **22. Sort Order (排序规则)** 讨论了HBase中数据的排序方式，对于处理大量数据时的查询效率至关重要。 **23. Column Metadata (列元数据)** 解释了列元数据的重要性，包括定义列族属性和数据类型，以及如何影响数据存储和查询。总结来说，这份指南是HBase技术开发人员和管理员不可或缺的参考材料，提供了从基础安装配置到高级数据管理的全方位指导。无论是初次接触者还是经验丰富的开发者，都可以从中找到所需的信息来提升HBase的运用水平。

hbase(main):002:0> list 'test'

TABLE

test

1 row(s) in 0.0180 seconds

=> ["test"]

Now use the describe command to see details, including configuration defaults

hbase(main):003:0> describe 'test'

Table test is ENABLED

test

COLUMN FAMILIES DESCRIPTION

{NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',

NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE

'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',

REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'f

alse', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',

PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true',

BLOCKSIZE

Ê=> '65536'}

1 row(s)

Took 0.9998 seconds

5. Put data into your table.

To put data into your table, use the put command.

hbase(main):003:0> put 'test', 'row1', 'cf:a', 'value1'

0 row(s) in 0.0850 seconds

hbase(main):004:0> put 'test', 'row2', 'cf:b', 'value2'

0 row(s) in 0.0110 seconds

hbase(main):005:0> put 'test', 'row3', 'cf:c', 'value3'

0 row(s) in 0.0100 seconds

Here, we insert three values, one at a time. The first insert is at row1, column cf:a, with a value

of value1. Columns in HBase are comprised of a column family prefix, cf in this example,

followed by a colon and then a column qualifier suffix, a in this case.

6. Scan the table for all data at once.

One of the ways to get data from HBase is to scan. Use the scan command to scan the table for

data. You can limit your scan, but for now, all data is fetched.

To exit the HBase Shell and disconnect from your cluster, use the quit command. HBase is still

running in the background.

Procedure: Stop HBase

1. In the same way that the bin/start-hbase.sh script is provided to conveniently start all HBase

daemons, the bin/stop-hbase.sh script stops them.

$ ./bin/stop-hbase.sh

stopping hbase....................

2. After issuing the command, it can take several minutes for the processes to shut down. Use the

jps to be sure that the HMaster and HRegionServer processes are shut down.

The above has shown you how to start and stop a standalone instance of HBase. In the next sections

we give a quick overview of other modes of hbase deploy.

2.3. Pseudo-Distributed Local Install

After working your way through quickstart standalone mode, you can re-configure HBase to run in

pseudo-distributed mode. Pseudo-distributed mode means that HBase still runs completely on a

single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate

process: in standalone mode all daemons ran in one jvm process/instance. By default, unless you

configure the hbase.rootdir property as described in quickstart, your data is still stored in /tmp/. In

this walk-through, we store your data in HDFS instead, assuming you have HDFS available. You can

skip the HDFS configuration to continue storing your data in the local filesystem.



Hadoop Configuration

This procedure assumes that you have configured Hadoop and HDFS on your local

system and/or a remote system, and that they are running and available. It also

assumes you are using Hadoop 2. The guide on Setting up a Single Node Cluster in

the Hadoop documentation is a good starting point.

1. Stop HBase if it is running.

If you have just finished quickstart and HBase is still running, stop it. This procedure will create

a totally new directory where HBase will store its data, so any databases you created before will

be lost.

2. Configure HBase.

Edit the hbase-site.xml configuration. First, add the following property which directs HBase to

run in distributed mode, with one JVM instance per daemon.



Running multiple HMaster instances on the same hardware does not make

sense in a production environment, in the same way that running a pseudo-

distributed cluster does not make sense for production. This step is offered for

testing and learning purposes only.

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers,

which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-

master-backup.sh. For each backup master you want to start, add a parameter representing the

port offset for that master. Each HMaster uses two ports (16000 and 16010 by default). The port

offset is added to these ports, so using an offset of 2, the backup HMaster would use ports 16002

and 16012. The following command starts 3 backup servers using ports 16002/16012,

16003/16013, and 16005/16015.

$ ./bin/local-master-backup.sh start 2 3 5

To kill a backup master without killing the entire cluster, you need to find its process ID (PID).

The PID is stored in a file with a name like /tmp/hbase-USER-X-master.pid. The only contents of

the file is the PID. You can use the kill -9 command to kill that PID. The following command

will kill the master with port offset 1, but leave the cluster running:

$ cat /tmp/hbase-testuser-1-master.pid |xargs kill -9

7. Start and stop additional RegionServers

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one

HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same

system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh

command allows you to run multiple RegionServers. It works in a similar way to the local-

master-backup.sh command, in that each parameter you provide represents the port offset for

an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030.

Since HBase version 1.1.0, HMaster doesn’t use region server ports, this leaves 10 ports (16020 to

16029 and 16030 to 16039) to be used for RegionServers. For supporting additional

RegionServers, set environment variables HBASE_RS_BASE_PORT and

HBASE_RS_INFO_BASE_PORT to appropriate values before running script local-

regionservers.sh. e.g. With values 16200 and 16300 for base ports, 99 additional RegionServers

can be supported, on a server. The following command starts four additional RegionServers,

running on sequential ports starting at 16022/16032 (base ports 16020/16030 plus 2).

$ .bin/local-regionservers.sh start 2 3 4 5

To stop a RegionServer manually, use the local-regionservers.sh command with the stop

parameter and the offset of the server to stop.

$ .bin/local-regionservers.sh stop 3

剩余801页未读，继续阅读

四火流年

粉丝: 8
资源: 1

Apache HBase 3.0.0-SNAPSHOT 完整指南：配置、升级与数据模型详解

apache_hbase_reference_guide(官网).pdf

apache_hbase_reference_guide-3.0.0-SNAPSHOT

apache_hbase_reference_guide_1.1.2

import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.*; import org.apache.hadoop.hbase.client.*; import org.apache.hadoop.hbase.util.Bytes;

hbase org.apache.hadoop.hbase.tool.LoadIncrementalHFiles 命令

使用org.apache.hadoop.hbase.client进行HBase连接时，如何在失败后，不进行重试

最新资源

import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.; import org.apache.hadoop.hbase.client.; import org.apache.hadoop.hbase.util.Bytes;