java - Hadoop 身份验证与 Kerberos 错误

Question

我正在尝试使用以下方法在 HDFS 中创建文件:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

为此，我添加了如下配置:

Configuration configuration = new Configuration();

configuration.set("fs.hdfs.impl",
        org.apache.hadoop.hdfs.DistributedFileSystem.class.getName()
);
configuration.set("fs.file.impl",
        org.apache.hadoop.fs.LocalFileSystem.class.getName()
);

OutputStream fileout1 = new FileOutputStream("CONF_before.XML");
configuration.writeXml(fileout1);

configuration.addResource(new Path("/etc/hive/conf.cloudera.hive/hdfs-site.xml"));
configuration.addResource(new Path("/etc/hive/conf.cloudera.hive/core-site.xml"));
OutputStream fileout = new FileOutputStream("CONF_after.XML");
configuration.writeXml(fileout);
FileSystem hdfs = FileSystem.get(configuration);

Path out_path = new Path(hdfs.getWorkingDirectory() + "/OD.xml");
OutputStream os = hdfs.create(out_path);

当我运行这段代码时，OutputStream os = hdfs.create(out_path) 中出现错误:

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled.  Available:[TOKEN,KERBEROS]

但如果我将 core-site.xml 添加到项目工件并在服务器上运行它，则不会出现错误。

两种情况下的输出配置相同。 core-site.xml 的相关部分是:

 <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hadoop.security.authorization</name>
    <value>false</value>
  </property>
  <property>
    <name>hadoop.rpc.protection</name>
    <value>authentication</value>
  </property>

知道为什么会这样吗？谢谢!

Answer 1

尝试将其添加到 hdfs-site.xml

<property>
  <name>ipc.client.fallback-to-simple-auth-allowed</name>
  <value>true</value>
</property>