Apache Hive入门精要

4星 · 超过85%的资源 需积分: 10 20 下载量 4 浏览量 更新于2024-07-20 1 收藏 1.87MB PDF 举报
"Apache Hive Essentials" 是一本关于Hive的入门书籍,旨在介绍大数据和Hive的基本概念,以及如何设置和使用Hive环境。 在大数据领域,Hive扮演着一个重要的角色。它是一个基于Hadoop的数据仓库工具,允许用户通过SQL(HQL,Hive查询语言)对存储在Hadoop分布式文件系统(HDFS)中的大量非结构化或半结构化数据进行分析和处理。这本书首先回顾了大数据的发展历程,解释了大数据的概念,同时对比了关系型数据库和NoSQL数据库与Hadoop之间的差异。此外,书中还讨论了批处理、实时处理和流处理等不同数据处理模式。 Hadoop生态系统是Hive运作的基础,其中包含了许多关键组件,如HDFS、MapReduce、YARN等。Hive作为一个轻量级的接口,使得没有Hadoop背景的用户也能方便地处理大数据。书中详细介绍了Hive的概览,包括其架构、功能和优点。 在设置Hive环境部分,作者指导读者如何从Apache官方网站下载并安装Hive,同时也涵盖了通过供应商提供的包进行安装的方法。对于云环境,书中提到了如何在云端启动Hive。此外,读者还将学习如何使用Hive的命令行界面(Hive CLI)和Beeline,以及集成开发环境(IDE),以更高效地进行Hive操作。 数据定义和描述是Hive操作的核心。书中详细阐述了Hive的数据类型,包括如何进行数据类型的转换。Hive数据定义语言(DDL)用于创建、修改和删除数据库对象,如数据库、内部表和外部表。Hive的分区功能使得数据管理更加高效,而桶则提供了基于特定列的预排序,有助于提高查询性能。 书中的每一章节都包含了丰富的实例和总结,以帮助读者更好地理解和应用所学知识。无论是初学者还是有经验的Hadoop开发者,都能从这本书中受益,提升对Hive的理解和使用技能。读者可以通过书中的链接获取示例代码,并参与到反馈、提问和讨论中,以深化学习体验。

ALTER DATABASE test1 RENAME TO test2; NoViableAltException(254@[]) at org.apache.hadoop.hive.ql.parse.HiveParser.alterDatabaseStatementSuffix(HiveParser.java:9043) at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatement(HiveParser.java:7647) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:4337) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2494) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1420) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:220) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:74) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:67) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:616) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) FAILED: ParseException line 1:21 cannot recognize input near 'test1' 'RENAME' 'TO' in alter database statement

2023-05-24 上传

[root@zhaosai ~]# hive Logging initialized using configuration in jar:file:/opt/programs/apache-hive-1.2.2-bin/lib/hive-common-1.2.2.jar!/hive-log4j.properties Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) ... 8 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) ... 14 more Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory NestedThrowables:

2023-06-09 上传

结合下面hive元数据信息,生成hive建表语句,并且加上comment,注意day是分区 dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 apk 应用包名 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 app_name 应用名称 string day string 入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 app_url 平台详情页链接 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 cate 应用所属分类 string day string入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 other 其他 string day string 入库日期 org.apache.hadoop.hive.ql.io.orc.OrcSerde serialization.format 1 hdfs://DSbigdata/hiveDW/dwd_exten_crawl/appmarket_appinfo org.apache.hadoop.hive.ql.io.orc.OrcInputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat dwd_weibo_crawl NULL appmarket_appinfo GN线应用市场 2021-01-07 15:07:29 region 平台名称 string day stri

2023-06-10 上传