Hadoop掉坑记录

启动Hadoop时,以root用户操作yarn和hdfs服务报错,原因是缺少YARN_RESOURCEMANAGER_USER、HDFS_NAMENODE_USER等环境变量。解决办法是在start-dfs.sh和start-yarn.sh等脚本顶部添加对应的用户变量定义,然后重新启动服务,并用jps和netstat验证。文章还列出了Namenode、Datanode、Yarn等组件的常用端口。

作者:zhuge···预计阅读 9 分钟·674 阅读·0 评论
Hadoop掉坑记录

ERROR: Attempting to operate on yarn resourcemanager as root

ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.

Starting nodemanagers

ERROR: Attempting to operate on yarn nodemanager as root

ERROR: but there is no YARN_NODEMANAGER_USER defined. Aborting operation.


ERROR: Attempting to operate on hdfs namenode as root

ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.

Starting datanodes

ERROR: Attempting to operate on hdfs datanode as root

ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.


将start-dfs.sh,stop-dfs.sh两个文件顶部添加以下参数


#!/usr/bin/env bash
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root


还有,start-yarn.sh,stop-yarn.sh顶部也需添加以下:


#!/usr/bin/env bash
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

./start-dfs.sh

./start-yarn.sh

jps查看

netstat -aptn确认端口


http://ip:9870

http://ip:8088

Namenode 端口:

9871 dfs.namenode.https-address The namenode secure http server address and port.

9870 dfs.namenode.http-address The address and the base port where the dfs namenode web ui will listen on.

9820 fs.defaultFS 指定HDFS运行时nameNode地址

Secondary NN 端口:

9869 dfs.namenode.secondary.https-address The secondary namenode HTTPS server address and port

9868 dfs.namenode.secondary.http-address The secondary namenode HTTPS server address and port

Datanode 端口:

9867 dfs.datanode.ipc.address The datanode ipc server address and port.

9866 dfs.datanode.address The datanode server address and port for data transfer.

9865 dfs.datanode.https.address The datanode secure http server address and port

9864 dfs.datanode.http.address The datanode http server address and por

Yarn 端口

8088 yarn.resourcemanager.webapp.address http服务端口

相关文章

评论

加载中...