hadoop - 客户端无法通过 :[TOKEN, KERBEROS 进行身份验证]

Question

我正在使用 YarnClient 以编程方式开始一项工作。我正在运行的集群已经过 kerberos 化。

法线贴图减少通过“yarn jar examples.jar wordcount ...”提交的工作。

我尝试以编程方式提交的作业没有。我收到此错误:

14/09/04 21:14:29 ERROR client.ClientService: Error happened during application submit: Application application_1409863263326_0002 failed 2 times due to AM Container for appattempt_1409863263326_0002_000002 exited with exitCode: -1000 due to: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "yarn-c1-n1.clouddev.snaplogic.com/10.184.28.108"; destination host is: "yarn-c1-cdh.clouddev.snaplogic.com":8020; .Failing this attempt.. Failing the application. 14/09/04 21:14:29 ERROR client.YClient: Application submission failed

代码看起来像这样:

ClientContext context = createContextFrom(args);
YarnConfiguration configuration = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(configuration);
ClientService client = new ClientService(context, yarnClient, new InstallManager(FileSystem.get(configuration)));
LOG.info(Messages.RUNNING_CLIENT_SERVICE);
boolean result = client.execute();

我想也许添加一些东西的效果是:

yarnClient.getRMDelegationToken(new Text(InetAddress.getLocalHost().getHostAddress()));

也许可以减轻我的痛苦，但这似乎也无济于事。任何帮助将不胜感激。

Answer 1

好吧，经过数小时又数小时的时间，我们已经弄清楚了。对于所有后代的编码人员，永远被 hadoop 缺乏文档所困扰:

您必须通过调用获取凭据从 UserGroupInformation 对象中获取 token 。然后，您必须在 ContainerLaunchContext 上设置 token 。