centos7 快速搭建hadoop集群脚本

以下是一个简单的脚本，可以快速在 CentOS 7 上搭建 Hadoop 集群：

#!/bin/bash
# Hadoop Cluster Setup Script

# Set hostname
echo "Setting hostname..."
hostnamectl set-hostname hadoop-master
echo "127.0.0.1 hadoop-master" >> /etc/hosts

# Install Java
echo "Installing Java..."
yum install -y java-1.8.0-openjdk-devel

# Download Hadoop
echo "Downloading Hadoop..."
curl -O https://apache.org/dist/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz

# Extract Hadoop
echo "Extracting Hadoop..."
tar -xzvf hadoop-3.3.1.tar.gz
mv hadoop-3.3.1 /usr/local/hadoop
rm hadoop-3.3.1.tar.gz

# Configure Hadoop
echo "Configuring Hadoop..."
echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk" >> /usr/local/hadoop/etc/hadoop/hadoop-env.sh
echo "export HADOOP_HOME=/usr/local/hadoop" >> /etc/profile
echo "export PATH=$PATH:/usr/local/hadoop/bin:/usr/local/hadoop/sbin" >> /etc/profile
source /etc/profile

# Configure Hadoop Cluster
echo "Configuring Hadoop Cluster..."
cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml
sed -i 's/localhost/hadoop-master/' /usr/local/hadoop/etc/hadoop/core-site.xml
sed -i 's/localhost/hadoop-master/' /usr/local/hadoop/etc/hadoop/hdfs-site.xml
sed -i 's/localhost/hadoop-master/' /usr/local/hadoop/etc/hadoop/yarn-site.xml
echo "export HDFS_NAMENODE_USER=root" >> /usr/local/hadoop/etc/hadoop/hadoop-env.sh
echo "export HDFS_DATANODE_USER=root" >> /usr/local/hadoop/etc/hadoop/hadoop-env.sh
echo "export HDFS_SECONDARYNAMENODE_USER=root" >> /usr/local/hadoop/etc/hadoop/hadoop-env.sh
echo "export YARN_RESOURCEMANAGER_USER=root" >> /usr/local/hadoop/etc/hadoop/yarn-env.sh
echo "export YARN_NODEMANAGER_USER=root" >> /usr/local/hadoop/etc/hadoop/yarn-env.sh

# Start Hadoop Cluster
echo "Starting Hadoop Cluster..."
/usr/local/hadoop/sbin/start-dfs.sh
/usr/local/hadoop/sbin/start-yarn.sh

# Configure Hadoop Cluster on other nodes
echo "Configure Hadoop Cluster on other nodes..."
echo "hadoop-slave1" >> /etc/hosts
echo "hadoop-slave2" >> /etc/hosts

# Done
echo "Hadoop Cluster Setup Complete!"

这个脚本包括了以下步骤：

1. 设置主机名和 hosts 文件。
2. 安装 Java。
3. 下载和解压 Hadoop。
4. 配置 Hadoop。
5. 配置 Hadoop 集群。
6. 启动 Hadoop 集群。
7. 在其他节点上配置 Hadoop 集群。

可以将这个脚本保存为 `setup-hadoop.sh`，并在每个节点上运行它。注意，在运行脚本之前，需要将每个节点的 hostname 和 IP 地址添加到 `/etc/hosts` 文件中。