centos7安装hadoop伪分布式
时间: 2023-04-28 08:04:01 浏览: 169
1. 首先安装Java环境,Hadoop需要Java环境支持,可以使用yum命令安装:
```
yum install java-1.8.0-openjdk-devel.x86_64
```
2. 下载Hadoop安装包,可以从官网下载最新版本:
```
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.3.0/hadoop-3.3.0.tar.gz
```
3. 解压Hadoop安装包:
```
tar -zxvf hadoop-3.3.0.tar.gz
```
4. 配置Hadoop环境变量,在/etc/profile文件中添加以下内容:
```
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
```
然后执行以下命令使环境变量生效:
```
source /etc/profile
```
5. 配置Hadoop伪分布式模式,修改以下配置文件:
- /usr/local/hadoop/etc/hadoop/hadoop-env.sh
```
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.el7_9.x86_64
```
- /usr/local/hadoop/etc/hadoop/core-site.xml
```
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
```
- /usr/local/hadoop/etc/hadoop/hdfs-site.xml
```
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/usr/local/hadoop/hadoop_data/hdfs/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/hadoop/hadoop_data/hdfs/datanode</value>
</property>
</configuration>
```
- /usr/local/hadoop/etc/hadoop/mapred-site.xml
```
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
```
- /usr/local/hadoop/etc/hadoop/yarn-site.xml
```
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>localhost</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>1</value>
</property>
</configuration>
```
6. 格式化Hadoop文件系统,执行以下命令:
```
hdfs namenode -format
```
7. 启动Hadoop服务,执行以下命令:
```
start-all.sh
```
8. 查看Hadoop服务状态,执行以下命令:
```
jps
```
如果输出以下内容,则表示Hadoop服务已经启动成功:
```
12345 NameNode
23456 DataNode
34567 ResourceManager
45678 NodeManager
56789 Jps
```
9. 访问Hadoop Web界面,打开浏览器,输入以下地址:
```
http://localhost:9870/
```
如果能够正常访问,则表示Hadoop伪分布式安装成功。
阅读全文