CentOS7下下Hadoop3.2.1集群的安装与部署(下)集群的安装与部署(下)
在《CentOS7下Hadoop3.2.1集群的安装与部署(上)》中我们我们完成了对Hadoop集群的安装与启动。接下来,重点对
HDFS和Yarn的HA配置进行介绍。
HDFS高可用高可用
在上述Hadoop集群搭建完成之后,若要启用HA还需要对hdfs-site.xml和core-site.xml两个文件进行一点额外的配置。
hdfs-site.xml
dfs.nameservices
hadoop-cluster
dfs.ha.namenodes.hadoop-cluster
nn1,nn2
dfs.namenode.rpc-address.hadoop-cluster.nn1
NN1:8020
dfs.namenode.rpc-address.hadoop-cluster.nn2
NN2:8020
dfs.namenode.http-address.hadoop-cluster.nn1
NN1:50070
dfs.namenode.http-address.hadoop-cluster.nn2
NN2:50070
dfs.namenode.shared.edits.dir
qjournal://JN1:8485;JN2:8485;JN3:8485/hadoop-cluster
dfs.namenode.name.dir
file:///usr/local/hadoop-3.2.1/data/namespace,file:///home/hadoop/name
Path on the local filesystem where the NameNode stores
dfs.datanode.data.dir
file:///usr/local/hadoop-3.2.1/data/dataspace,file:///home/hadoop/data
Path on the local filesystem where the DataNode stores Data
dfs.journalnode.edits.dir
/home/hadoop/journaldata
dfs.ha.automatic-failover.enabled.hadoop-cluster
true
dfs.client.failover.proxy.provider.hadoop-cluster
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.fencing.methods
sshfence
dfs.ha.fencing.ssh.private-key-files
/home/hadoop/.ssh/id_rsa
dfs.ha.fencing.ssh.connect-timeout
30000
core-site.xml
fs.defaultFS
hdfs://hadoop-cluster
ha.zookeeper.quorum
hadoop34:2181,hadoop39:2181,hadoop40:2181
启用启用HDFS故障自动转移故障自动转移
评论0