Apache Oozie 4.0.1与Hadoop 2.3.0安装教程

4星 · 超过85%的资源 需积分: 15 36 下载量 117 浏览量 更新于2024-09-11 收藏 970KB DOC 举报
本文档详细介绍了如何在Hadoop 2.3.0环境中安装Oozie 4.0.1。首先,提供了Oozie下载的官方镜像链接,推荐从Apache官方网站的Quick Start页面获取,而不是直接访问外部网站。下载完成后,用户需要解压oozie-4.0.1.tar.gz文件,并进入目录进行配置。 配置过程中,作者特别强调了针对Java版本的选择,由于使用的是Java 1.7,因此在pom.xml文件中将<javaVersion>和<targetJavaVersion>设置为1.7。关于Hadoop版本,尽管尝试过2.2.0但未成功,因此保持为原定的2.3.0。HBase、Hive、Pig和Sqoop的版本也进行了相应的设置,确保与Oozie兼容。在编译阶段,使用mvn clean package assembly:single-DskipTests命令,如果遇到网络问题,可能需要多次尝试。 编译完成后,生成的oozie-4.0.1-distro.tar.gz文件被复制到了/usr/local/oozie目录,这里定义为OOZIE_HOME,即Oozie的工作目录。接下来的步骤包括配置Oozie,例如修改oozie-site.xml等核心配置文件,以及启动Oozie Server和协调器服务,以便完成Hadoop工作流的管理和调度。 为了确保顺利安装和运行,读者可以参考提供的环境变量设置,尤其是软件版本的部分,这有助于解决可能遇到的问题。此外,文档中还附有Oozie目录结构图,以便用户了解安装后的文件组织。整体而言,本文档为在Hadoop 2.3.0环境中部署Oozie 4.0.1提供了详尽的指导,对开发人员和运维人员在实际操作中有很高的实用价值。
2023-07-25 上传

Stdoutput Warning: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Stdoutput Please set $ACCUMULO_HOME to the root of your Accumulo installation. Exit code of the Shell command 1 <<< Invocation of Shell command completed <<< java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410) at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55) at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141) Caused by: org.apache.oozie.action.hadoop.LauncherMainException at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:76) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104) at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:63) ... 16 more Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.ShellMain], exit code [1] Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice0/user/admin/oozie-oozi/0000118-230724100647793-oozie-oozi-W/shell-af10--shell/action-data.seq Stopping AM Callback notification attempts left 0 Callback notification trying http://nn.hdfs.offline:11000/oozie/callback?id=0000118-230724100647793-oozie-oozi-W@shell-af10&status=FAILED Callback notification to http://nn.hdfs.offline:11000/oozie/callback?id=0000118-230724100647793-oozie-oozi-W@shell-af10&status=FAILED succeeded Callback notification succeeded

2023-07-25 上传