快速入门指南:Hive实战教程与优化详解

需积分: 21 26 下载量 36 浏览量 更新于2024-07-18 收藏 3.27MB PDF 举报
Hive简明教程是一本专为理解和快速应用Hive设计的实用指南,它分为三个部分。第一部分重点关注Hive的日常使用,适合非技术背景的分析人员。这部分详细介绍了Hive的基础语法,如数据类型、数据定义语言(DDL)用于创建、查看、管理数据库和表,以及数据操作语言(DML)用于加载数据、导出数据和执行基本查询。 在数据定义部分,作者指导读者如何创建和管理Hive的数据库,包括创建普通表、分区表和桶表,以及相应的查看、修改和删除操作。此外,还介绍了数据加载和数据管理操作,比如使用LOAD DATA INPATH语句导入数据,以及如何使用INSERT INTO和COPY命令进行数据操作。 第二部分是关于Hive执行原理和优化技巧,这对于想要编写高效Hive语句的技术人员来说至关重要。这部分内容将深入探讨Hive的工作机制,帮助读者理解其查询处理流程,从而能够优化查询性能。 最后,第三部分则针对技术细节爱好者,提供了关于Hive的深层次技术解析,涵盖了查询语言(HiveQL)的更高级特性,如SELECT语句的复杂筛选和聚合功能,以及WHERE子句的使用。这部分内容旨在为技术人员提供坚实的技术基础,以便他们能够灵活且高效地利用Hive进行复杂的数据分析。 在整个教程中,作者淳月宾通过实例和清晰的解释,确保读者无论是在日常数据分析还是技术学习上都能得到充分的支持。Linux公社(www.linuxidc.com)作为该教程的发布平台,提供了丰富的Linux和大数据相关资源,包括Hadoop、Python、Java等技术,为读者构建了一个全面的学习环境。此外,Linux公社还设有多个专题栏目,如Ubuntu、Fedora、Hadoop等,便于用户根据兴趣深入探索。通过阅读这本书,无论是初学者还是进阶者,都能找到所需的信息,提升Hive技能。
2019-09-01 上传
Table of Contents About the Tutorial ····································································································································· i Audience ··················································································································································· i Prerequisites ············································································································································· i Disclaimer & Copyright ······························································································································ i Table of Contents ····································································································································· ii . 1. INTRODU CTION ································ ································ ································ ···················· 1 1 Hadoop ···················································································································································· 1 What is Hive? ··········································································································································· 2 Features of Hive ······································································································································· 2 Architecture of Hive ································································································································· 2 Working of Hive ······································································································································· 4 . 2. HIVE INSTALLATION ································ ································ ································ ·············· 6 6 Step 1: Verifying JAVA Installation ··········································································································· 6 Step 2: Verifying Hadoop Installation ······································································································· 8 Step 3: Downloading Hive ······················································································································ 15 Step 4: Installing Hive ····························································································································· 15 Step 5: Configuring Hive ························································································································· 16 Step 6: Downloading and Installing Apache Derby ················································································· 17 Step 7: Configuring Metastore of Hive ···································································································· 19 Step 8: Verifying Hive Installation ·········································································································· 20 . 3. HIVE DATA TYPES ································ ································ ································ ················ 22 Column Types ········································································································································· 22 Literals ··················································································································································· 24 Null Value ··············································································································································· 24 Complex Types ······································································································································· 24

结合下面hive元数据信息,生成hive建表语句,并且加上comment,注意day是分区 dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 apk 应用包名 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 app_name 应用名称 string day string 入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 app_url 平台详情页链接 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 cate 应用所属分类 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 other 其他 string day string 入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 region 平台名称 string day stri

2023-06-10 上传