大数据Hadoop2.0集群搭建_头歌hadoop开发环境搭建答案

需积分: 20 161 浏览量更新于2023-03-16 评论收藏 5.94MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源评论

资源推荐

Hadoop2.0 安装手册目录

第 1 章安装 VMWare Workstation 10 ................................................................................................................. 4

第 2 章 VMware 10 安装 CentOS 6 ...................................................................................................................10

2.1 CentOS 系统安装 .....................................................................................................................................10

2.2 安装中的关键问题 ..................................................................................................................................13

2.3 克隆 HadoopSlave .................................................................................................................................... 17

2.4 windows 中安装 SSH Secure Shell Client 传输软件 ................................................................................19

第 3 章 CentOS 6 安装 Hadoop .........................................................................................................................23

3.1 启动两台虚拟客户机 ..............................................................................................................................23

3.2 Linux 系统配置.........................................................................................................................................24

3.2.1 软件包和数据包说明 ...................................................................................................................... 25

3.2.2 配置时钟同步 .................................................................................................................................. 25

3.2.3 配置主机名 ...................................................................................................................................... 26

3.2.5 使用 setup 命令配置网络环境 ...................................................................................................... 27

3.2.6 关闭防火墙 ......................................................................................................................................29

3.2.7 配置 hosts 列表 ...............................................................................................................................30

3.2.8 安装 JDK ...........................................................................................................................................31

3.2.9 免密钥登录配置 ..............................................................................................................................32

3.3 Hadoop 配置部署 ....................................................................................................................................34

3.3.1 Hadoop 安装包解压 ........................................................................................................................34

3.3.2 配置环境变量 hadoop-env.sh .........................................................................................................34

3.3.3 配置环境变量 yarn-env.sh ...............................................................................................................35

3.3.4 配置核心组件 core-site.xml ............................................................................................................35

3.3.5 配置文件系统 hdfs-site.xml ............................................................................................................35

3.3.6 配置文件系统 yarn-site.xml ............................................................................................................36

3.3.7 配置计算框架 mapred-site.xml ....................................................................................................... 37

3.3.8 在 master 节点配置 slaves 文件 .................................................................................................... 37

3.3.9 复制到从节点 ................................................................................................................................. 37

3.4 启动 Hadoop 集群 ................................................................................................................................... 37

3.4.1 配置 Hadoop 启动的系统环境变量 ............................................................................................38

3.4.2 创建数据目录 ...............................................................................................................................38

3.4.3 启动 Hadoop 集群 ...........................................................................................................................38

第 4 章安装部署 Hive ..........................................................................................................................................44

4.1 解压并安装 Hive ......................................................................................................................................44

4.2 安装配置 MySQL ......................................................................................................................................45

4.3 配置 Hive ..................................................................................................................................................45

4.4 启动并验证 Hive 安装 .............................................................................................................................46

第 5 章安装部署 HBase ........................................................................................................................................49

5.1 解压并安装 HBase ...................................................................................................................................49

5.2 配置 HBase ...............................................................................................................................................50

5.2.1 修改环境变量 hbase-env.sh .........................................................................................................50

5.2.2 修改配置文件 hbase-site.xml .......................................................................................................50

5.2.3 设置 regionservers ........................................................................................................................ 51

第 1 章安装 VMWare Workstation 10

5.2.4 设置环境变量 ............................................................................................................................... 51

5.2.5 将 HBase 安装文件复制到 HadoopSlave 节点 ............................................................................ 51

5.3 启动并验证 HBase ................................................................................................................................... 51

第 6 章安装部署 Mahout .....................................................................................................................................54

6.1 解压并安装 Mahout ................................................................................................................................54

6.2 启动并验证 Mahout ................................................................................................................................ 55

第 7 章安装部署 Sqoop ...................................................................................................................................... 57

7.1 解压并安装 Sqoop ................................................................................................................................... 57

7.2 配置 Sqoop ........................................................................................................................................58

7.2.1 配置 MySQL 连接器 ......................................................................................................................58

7.2.2 配置环境变量 ..................................................................................................................................58

7.3 启动并验证 Sqoop ...................................................................................................................................59

第 8 章安装部署 Spark ........................................................................................................................................61

8.1 解压并安装 Spark ....................................................................................................................................61

8.2 配置 Hadoop 环境变量 ........................................................................................................................... 62

8.3 验证 Spark 安装 ....................................................................................................................................... 62

第 9 章安装部署 Storm .........................................................................................................................................66

安装 Storm 依赖包 ...................................................................................................................................66

9.1 安装 ZooKeeper 集群 ..................................................................................................................................66

9.1.1 解压安装 ..........................................................................................................................................66

9.1.2 配置 ZooKeeper 属性文件 ............................................................................................................... 67

9.1.3 将 Zookeeper 安装文件复制到 HadoopSlave 节点 .......................................................................68

9.1.3 启动 ZooKeeper 集群 .......................................................................................................................68

9.2 安装 Storm ..................................................................................................................................................69

9.2.1 解压安装 .........................................................................................................................................69

9.2.2 修改 storm.yaml 配置文件 .............................................................................................................. 70

9.2.3 将 Storm 安装文件复制到 HadoopSlave 节点 .............................................................................. 70

9.2.4 启动 Storm 集群 .............................................................................................................................. 70

9.2.5 向 Storm 集群提交任务 ................................................................................................................. 71

第 10 章安装部署 Kafka ......................................................................................................................................... 73

10.1. 安装 Kafka ......................................................................................................................................... 73

10.1.1 下载 Kafka 安装文件 ..................................................................................................................... 73

10.2. 配置 Kafka ......................................................................................................................................... 73

10.3. 启动 Kafka ......................................................................................................................................... 74