共计 3669 个字符，预计需要花费 10 分钟才能阅读完成。
简单记录 CentOS 7.4 下 Hadoop 3.0 单机安装教程。
相关下载
hive 下载地址
http://mirrors.hust.edu.cn/apache/hive/stable-2/apache-hive-2.3.2-bin.tar.gz
hadoop 下载地址
https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.0.0/hadoop-3.0.0.tar.gz
1. hive 安装目录 /opt/hive/apache-hive-2.3.2-bin
2. hadoop 安装目录 /opt/hadoop/hadoop-3.0.0
3. JDK 安装目录 /usr/java/jdk1.8.0_65
4. 环境变量配置
export JAVA_HOME=/usr/java/jdk1.8.0_65
export HADOOP_HOME=/opt/hadoop/hadoop-3.0.0
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS=”-Djava.library.path=${HADOOP_HOME}/lib”
export HIVE_HOME=/opt/hive/apache-hive-2.3.2-bin
export HIVE_CONF_DIR=${HIVE_HOME}/conf
export CLASS_PATH=.:${JAVA_HOME}/lib:${HIVE_HOME}/lib:$CLASS_PATH
export PATH=.:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:${HIVE_HOME}/bin:$PATH
5. 让环境变量生效
source /etc/profile
6. vim /opt/hadoop/hadoop-3.0.0/etc/hadoop/core-site.xml 修改
<configuration>
<!– 指定 HDFS 老大（namenode）的通信地址 –>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<!– 指定 hadoop 运行时产生文件的存储路径 –>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/tmp</value>
</property>
</configuration>
7. vim /opt/hadoop/hadoop-3.0.0/etc/hadoop/hdfs-site.xml 修改增加以下内容
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/opt/hadoop/hdfs/name</value>
<description>namenode 上存储 hdfs 名字空间元数据 </description>
</property>
<property>
<name>dfs.data.dir</name>
<value>/opt/hadoop/hdfs/data</value>
<description>datanode 上数据块的物理存储位置 </description>
</property>
<!– 设置 hdfs 副本数量 –>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
8. SSH 设置免密码登录
ssh-keygen -t dsa -P ” -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys
9. 启动命令
9.1 初始化
cd /opt/hadoop/hadoop-3.0.0
./bin/hdfs namenode -format
9.2 启动命令
./sbin/start-dfs.sh
9.3 停止命令
./sbin/stop-dfs.sh
错误处理
Starting namenodes on [localhost]
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Starting datanodes
ERROR: Attempting to operate on hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
Starting secondary namenodes [bogon]
ERROR: Attempting to operate on hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.
处理 1
$ vim sbin/start-dfs.sh
$ vim sbin/stop-dfs.sh
两处增加以下内容
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
处理 2
$ vim sbin/start-yarn.sh
$ vim sbin/stop-yarn.sh
两处增加以下内容
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
10. 验证安装
http://192.168.50.48:9870/dfshealth.html#tab-overview
Hadoop2.3-HA 高可用集群环境搭建 http://www.linuxidc.com/Linux/2017-03/142155.htm
Hadoop 项目之基于 CentOS7 的 Cloudera 5.10.1（CDH）的安装部署 http://www.linuxidc.com/Linux/2017-04/143095.htm
Hadoop2.7.2 集群搭建详解（高可用）http://www.linuxidc.com/Linux/2017-03/142052.htm
使用 Ambari 来部署 Hadoop 集群（搭建内网 HDP 源）http://www.linuxidc.com/Linux/2017-03/142136.htm
Ubuntu 14.04 下 Hadoop 集群安装 http://www.linuxidc.com/Linux/2017-02/140783.htm
CentOS 6.7 安装 Hadoop 2.7.2 http://www.linuxidc.com/Linux/2017-08/146232.htm
Ubuntu 16.04 上构建分布式 Hadoop-2.7.3 集群 http://www.linuxidc.com/Linux/2017-07/145503.htm
CentOS 7 下 Hadoop 2.6.4 分布式集群环境搭建 http://www.linuxidc.com/Linux/2017-06/144932.htm
Hadoop2.7.3+Spark2.1.0 完全分布式集群搭建过程 http://www.linuxidc.com/Linux/2017-06/144926.htm
更多 Hadoop 相关信息见 Hadoop 专题页面 http://www.linuxidc.com/topicnews.aspx?tid=13
本文永久更新链接地址 ：http://www.linuxidc.com/Linux/2018-02/150857.htm
