没有合适的资源？快使用搜索试试~ 我知道了~
首页Apache impala-3.2 Guide.pdf
Apache impala-3.2 Guide.pdf
需积分: 9 506 浏览量
更新于2023-05-26
评论
收藏 3.88MB PDF 举报
Impala是Cloudera公司主导开发的新型查询系统，它提供SQL语义，能查询存储在Hadoop的HDFS和HBase中的PB级大数据。已有的Hive系统虽然也提供了SQL语义，但由于Hive底层执行使用的是MapReduce引擎，仍然是一个批处理过程，难以满足查询的交互性。相比之下，Impala的最大特点也是最大卖点就是它的快速。 [1]
资源详情
资源评论
资源推荐
Apache Impala Guide
| Contents | ii
Contents
Introducing Apache Impala...................................................................................14
Impala Benefits................................................................................................................................................... 14
How Impala Works with Apache Hadoop.........................................................................................................14
Primary Impala Features.....................................................................................................................................15
Impala Concepts and Architecture.......................................................................15
Components of the Impala Server......................................................................................................................15
The Impala Daemon............................................................................................................................... 15
The Impala Statestore............................................................................................................................. 16
The Impala Catalog Service................................................................................................................... 16
Developing Impala Applications........................................................................................................................ 17
Overview of the Impala SQL Dialect.................................................................................................... 17
Overview of Impala Programming Interfaces........................................................................................18
How Impala Fits Into the Hadoop Ecosystem................................................................................................... 18
How Impala Works with Hive............................................................................................................... 18
Overview of Impala Metadata and the Metastore..................................................................................19
How Impala Uses HDFS........................................................................................................................ 19
How Impala Uses HBase....................................................................................................................... 19
Planning for Impala Deployment..........................................................................19
Impala Requirements.......................................................................................................................................... 20
Supported Operating Systems................................................................................................................ 20
Hive Metastore and Related Configuration............................................................................................20
Java Dependencies.................................................................................................................................. 20
Networking Configuration Requirements...............................................................................................20
Hardware Requirements..........................................................................................................................21
User Account Requirements...................................................................................................................21
Guidelines for Designing Impala Schemas........................................................................................................ 22
Installing Impala..................................................................................................... 23
What is Included in an Impala Installation........................................................................................................24
Managing Impala....................................................................................................24
Post-Installation Configuration for Impala.........................................................................................................24
Configuring Impala to Work with ODBC......................................................................................................... 26
Configuring Impala to Work with JDBC...........................................................................................................26
Configuring the JDBC Port.................................................................................................................... 26
Choosing the JDBC Driver.................................................................................................................... 26
Enabling Impala JDBC Support on Client Systems.............................................................................. 27
Establishing JDBC Connections.............................................................................................................28
Notes about JDBC and ODBC Interaction with Impala SQL Features................................................. 28
Kudu Considerations for DML Statements............................................................................................29
Upgrading Impala...................................................................................................29
| Contents | iii
Upgrading Impala............................................................................................................................................... 29
Impala Upgrade Considerations......................................................................................................................... 30
Grant REFRESH Privilege to Impala Roles with SELECT or INSERT Privilege when Upgrading
to Impala 3.0..................................................................................................................................... 30
List of Reserved Words Updated in Impala 3.0.................................................................................... 30
Decimal V2 Used by Default in Impala 3.0.......................................................................................... 30
Behavior of Column Aliases Changed in Impala 3.0............................................................................ 30
Default PARQUET_ARRAY_RESOLUTION Changed in Impala 3.0................................................ 30
Enable Clustering Hint for Inserts......................................................................................................... 31
Deprecated Query Options Removed in Impala 3.0.............................................................................. 31
Fine-grained Privileges Added in Impala 3.0........................................................................................ 31
refresh_after_connect Impala Shell Option Removed in Impala 3.0..................................................... 31
Return Type Changed for EXTRACT and DATE_PART Functions in Impala 3.0.............................. 31
Port Change for SHUTDOWN Command.............................................................................................31
Default Setting Changes......................................................................................................................... 31
Starting Impala....................................................................................................... 32
Starting Impala from the Command Line.......................................................................................................... 32
Modifying Impala Startup Options.................................................................................................................... 33
Configuring Impala Startup Options through the Command Line.........................................................33
Checking the Values of Impala Configuration Options.........................................................................35
Startup Options for impalad Daemon.................................................................................................... 35
Startup Options for statestored Daemon................................................................................................ 35
Startup Options for catalogd Daemon....................................................................................................35
Impala Tutorials..................................................................................................... 36
Tutorials for Getting Started.............................................................................................................................. 36
Explore a New Impala Instance............................................................................................................. 36
Load CSV Data from Local Files.......................................................................................................... 41
Point an Impala Table at Existing Data Files........................................................................................ 43
Describe the Impala Table......................................................................................................................45
Query the Impala Table..........................................................................................................................45
Data Loading and Querying Examples.................................................................................................. 46
Advanced Tutorials.............................................................................................................................................48
Attaching an External Partitioned Table to an HDFS Directory Structure............................................48
Switching Back and Forth Between Impala and Hive...........................................................................51
Cross Joins and Cartesian Products with the CROSS JOIN Operator...................................................51
Dealing with Parquet Files with Unknown Schema.......................................................................................... 53
Download the Data Files into HDFS..................................................................................................... 54
Create Database and Tables................................................................................................................... 54
Examine Physical and Logical Schema................................................................................................. 55
Analyze Data...........................................................................................................................................57
Impala Administration........................................................................................... 66
Setting Timeout Periods for Daemons, Queries, and Sessions..........................................................................67
Increasing the Statestore Timeout.......................................................................................................... 67
Setting the Idle Query and Idle Session Timeouts for impalad............................................................. 67
Setting Timeout and Retries for Thrift Connections to the Backend Client.......................................... 68
Cancelling a Query................................................................................................................................. 68
Using Impala through a Proxy for High Availability........................................................................................ 69
Overview of Proxy Usage and Load Balancing for Impala...................................................................69
Choosing the Load-Balancing Algorithm.............................................................................................. 69
Special Proxy Considerations for Clusters Using Kerberos.................................................................. 70
| Contents | iv
Special Proxy Considerations for TLS/SSL Enabled Clusters...............................................................71
Example of Configuring HAProxy Load Balancer for Impala.............................................................. 72
Managing Disk Space for Impala Data..............................................................................................................74
Impala Security....................................................................................................... 75
Security Guidelines for Impala.......................................................................................................................... 76
Securing Impala Data and Log Files................................................................................................................. 76
Installation Considerations for Impala Security.................................................................................................77
Securing the Hive Metastore Database.............................................................................................................. 77
Securing the Impala Web User Interface...........................................................................................................77
Configuring TLS/SSL for Impala...................................................................................................................... 78
Using the Command Line...................................................................................................................... 78
Using TLS/SSL with Business Intelligence Tools.................................................................................79
Specifying TLS/SSL Minimum Allowed Version and Ciphers.............................................................79
Enabling Sentry Authorization for Impala.........................................................................................................79
The Sentry Privilege Model................................................................................................................... 79
Starting the impalad Daemon with Sentry Authorization Enabled........................................................83
Using Impala with the Sentry Service................................................................................................... 83
Using Impala with the Sentry Policy File..............................................................................................86
Setting Up Schema Objects for a Secure Impala Deployment.............................................................. 87
Debugging Failed Sentry Authorization Requests................................................................................. 88
The DEFAULT Database in a Secure Deployment...............................................................................88
Impala Authentication.........................................................................................................................................88
Enabling Kerberos Authentication for Impala....................................................................................... 88
Enabling LDAP Authentication for Impala........................................................................................... 91
Using Multiple Authentication Methods with Impala........................................................................... 94
Configuring Impala Delegation for Hue and BI Tools..........................................................................94
Auditing Impala Operations............................................................................................................................... 95
Durability and Performance Considerations for Impala Auditing......................................................... 96
Format of the Audit Log Files............................................................................................................... 96
Which Operations Are Audited..............................................................................................................97
Viewing Lineage Information for Impala Data................................................................................................. 97
Impala SQL Language Reference.........................................................................98
Comments............................................................................................................................................................98
Data Types.......................................................................................................................................................... 99
ARRAY Complex Type (Impala 2.3 or higher only)............................................................................ 99
BIGINT Data Type...............................................................................................................................102
BOOLEAN Data Type......................................................................................................................... 103
CHAR Data Type (Impala 2.0 or higher only)....................................................................................105
DECIMAL Data Type (Impala 3.0 or higher only).............................................................................106
DOUBLE Data Type............................................................................................................................ 111
FLOAT Data Type............................................................................................................................... 112
INT Data Type......................................................................................................................................114
MAP Complex Type (Impala 2.3 or higher only)............................................................................... 114
REAL Data Type.................................................................................................................................. 118
SMALLINT Data Type........................................................................................................................ 119
STRING Data Type.............................................................................................................................. 120
STRUCT Complex Type (Impala 2.3 or higher only).........................................................................122
TIMESTAMP Data Type..................................................................................................................... 128
TINYINT Data Type............................................................................................................................ 134
VARCHAR Data Type (Impala 2.0 or higher only)............................................................................135
Complex Types (Impala 2.3 or higher only)....................................................................................... 137
Literals...............................................................................................................................................................168
| Contents | v
Numeric Literals................................................................................................................................... 168
String Literals........................................................................................................................................170
Boolean Literals.................................................................................................................................... 171
Timestamp Literals............................................................................................................................... 171
NULL.................................................................................................................................................... 172
SQL Operators.................................................................................................................................................. 173
Arithmetic Operators............................................................................................................................ 173
BETWEEN Operator............................................................................................................................ 176
Comparison Operators.......................................................................................................................... 177
EXISTS Operator..................................................................................................................................178
ILIKE Operator.....................................................................................................................................181
IN Operator........................................................................................................................................... 182
IREGEXP Operator.............................................................................................................................. 185
IS DISTINCT FROM Operator............................................................................................................186
IS NULL Operator................................................................................................................................188
IS TRUE Operator................................................................................................................................ 189
LIKE Operator...................................................................................................................................... 189
Logical Operators..................................................................................................................................190
REGEXP Operator................................................................................................................................193
RLIKE Operator....................................................................................................................................194
Impala Schema Objects and Object Names..................................................................................................... 195
Overview of Impala Aliases.................................................................................................................196
Overview of Impala Databases............................................................................................................ 198
Overview of Impala Functions.............................................................................................................198
Overview of Impala Identifiers............................................................................................................ 200
Overview of Impala Tables.................................................................................................................. 201
Overview of Impala Views.................................................................................................................. 203
Impala SQL Statements.................................................................................................................................... 207
DDL Statements....................................................................................................................................207
DML Statements................................................................................................................................... 209
ALTER DATABASE Statement.......................................................................................................... 209
ALTER TABLE Statement.................................................................................................................. 210
ALTER VIEW Statement.....................................................................................................................224
COMMENT Statement......................................................................................................................... 225
COMPUTE STATS Statement............................................................................................................. 226
CREATE DATABASE Statement....................................................................................................... 234
CREATE FUNCTION Statement.........................................................................................................236
CREATE ROLE Statement (Impala 2.0 or higher only)..................................................................... 242
CREATE TABLE Statement................................................................................................................242
CREATE VIEW Statement.................................................................................................................. 257
DELETE Statement (Impala 2.8 or higher only)................................................................................. 258
DESCRIBE Statement.......................................................................................................................... 260
DROP DATABASE Statement............................................................................................................ 272
DROP FUNCTION Statement............................................................................................................. 274
DROP ROLE Statement (Impala 2.0 or higher only)..........................................................................276
DROP STATS Statement..................................................................................................................... 276
DROP TABLE Statement.....................................................................................................................281
DROP VIEW Statement....................................................................................................................... 282
EXPLAIN Statement............................................................................................................................ 283
GRANT Statement (Impala 2.0 or higher only).................................................................................. 286
INSERT Statement................................................................................................................................290
INVALIDATE METADATA Statement..............................................................................................299
LOAD DATA Statement...................................................................................................................... 301
REFRESH Statement............................................................................................................................ 304
REFRESH AUTHORIZATION Statement.......................................................................................... 306
REFRESH FUNCTIONS Statement.................................................................................................... 307
剩余891页未读，继续阅读
安全验证
文档复制为VIP权益，开通VIP直接复制
信息提交成功
评论0