- 使用 Spring Initializr 创建 Spring Boot 应用程序
- 在Spring Boot中配置Cassandra
- 在 Spring Boot 上配置 Tomcat 连接池
- 将Camel消息路由到嵌入WildFly的Artemis上
本文整理了Java中org.apache.hadoop.yarn.ipc.YarnRPC
类的一些代码示例,展示了YarnRPC
类的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。YarnRPC
类的具体详情如下:
包路径:org.apache.hadoop.yarn.ipc.YarnRPC
类名称:YarnRPC
[英]Abstraction to get the RPC implementation for Yarn.
[中]获取纱线的RPC实现的抽象。
代码示例来源:origin: apache/incubator-gobblin
private static void getJhToken(Configuration conf, Credentials cred) throws IOException {
YarnRPC rpc = YarnRPC.create(conf);
final String serviceAddr = conf.get(JHAdminConfig.MR_HISTORY_ADDRESS);
LOG.debug("Connecting to HistoryServer at: " + serviceAddr);
HSClientProtocol hsProxy =
(HSClientProtocol) rpc.getProxy(HSClientProtocol.class, NetUtils.createSocketAddr(serviceAddr), conf);
LOG.info("Pre-fetching JH token from job history server");
Token<?> jhToken = null;
try {
jhToken = getDelegationTokenFromHS(hsProxy, conf);
} catch (Exception exc) {
throw new IOException("Failed to fetch JH token.", exc);
}
if (jhToken == null) {
LOG.error("getDelegationTokenFromHS() returned null");
throw new IOException("Unable to fetch JH token.");
}
LOG.info("Created JH token: " + jhToken.toString());
LOG.info("Token kind: " + jhToken.getKind());
LOG.info("Token id: " + Arrays.toString(jhToken.getIdentifier()));
LOG.info("Token service: " + jhToken.getService());
cred.addToken(jhToken.getService(), jhToken);
}
代码示例来源:origin: Qihoo360/XLearning
protected void serviceStart() throws Exception {
Configuration conf = new XLearningConfiguration();
YarnRPC rpc = YarnRPC.create(conf);
initializeWebApp(conf);
InetSocketAddress address = conf.getSocketAddr(
XLearningConfiguration.XLEARNING_HISTORY_BIND_HOST,
XLearningConfiguration.XLEARNING_HISTORY_ADDRESS,
conf.get(XLearningConfiguration.XLEARNING_HISTORY_ADDRESS, XLearningConfiguration.DEFAULT_XLEARNING_HISTORY_ADDRESS),
conf.getInt(XLearningConfiguration.XLEARNING_HISTORY_PORT, XLearningConfiguration.DEFAULT_XLEARNING_HISTORY_PORT));
server =
rpc.getServer(HSClientProtocol.class, protocolHandler, address,
conf, jhsDTSecretManager,
conf.getInt(XLearningConfiguration.XLEARNING_HISTORY_CLIENT_THREAD_COUNT,
XLearningConfiguration.DEFAULT_XLEARNING_HISTORY_CLIENT_THREAD_COUNT));
// Enable service authorization?
if (conf.getBoolean(
CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION,
false)) {
server.refreshServiceAcl(conf, new ClientHSPolicyProvider());
}
server.start();
this.bindAddress = conf.updateConnectAddr(XLearningConfiguration.XLEARNING_HISTORY_BIND_HOST,
XLearningConfiguration.XLEARNING_HISTORY_ADDRESS,
conf.get(XLearningConfiguration.XLEARNING_HISTORY_ADDRESS, XLearningConfiguration.DEFAULT_XLEARNING_HISTORY_ADDRESS),
server.getListenerAddress());
LOG.info("Instantiated HistoryClientService at " + this.bindAddress);
super.serviceStart();
}
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager
@Test
public void testRMStartWithDecommissionedNode() throws Exception {
String excludeFile = "excludeFile";
createExcludeFile(excludeFile);
YarnConfiguration conf = new YarnConfiguration();
conf.set(YarnConfiguration.RM_NODES_EXCLUDE_FILE_PATH,
excludeFile);
MockRM rm = new MockRM(conf) {
protected ClientRMService createClientRMService() {
return new ClientRMService(this.rmContext, scheduler,
this.rmAppManager, this.applicationACLsManager, this.queueACLsManager,
this.getRMContext().getRMDelegationTokenSecretManager());
};
};
rm.start();
YarnRPC rpc = YarnRPC.create(conf);
InetSocketAddress rmAddress = rm.getClientRMService().getBindAddress();
LOG.info("Connecting to ResourceManager at " + rmAddress);
ApplicationClientProtocol client =
(ApplicationClientProtocol) rpc
.getProxy(ApplicationClientProtocol.class, rmAddress, conf);
// Make call
GetClusterNodesRequest request =
GetClusterNodesRequest.newInstance(EnumSet.allOf(NodeState.class));
List<NodeReport> nodeReports = client.getClusterNodes(request).getNodeReports();
Assert.assertEquals(1, nodeReports.size());
rm.stop();
rpc.stopProxy(client, conf);
new File(excludeFile).delete();
}
代码示例来源:origin: ch.cern.hadoop/hadoop-yarn-common
@Test
public void testUnknownCall() {
Configuration conf = new Configuration();
conf.set(YarnConfiguration.IPC_RPC_IMPL, HadoopYarnProtoRPC.class
.getName());
YarnRPC rpc = YarnRPC.create(conf);
String bindAddr = "localhost:0";
InetSocketAddress addr = NetUtils.createSocketAddr(bindAddr);
Server server = rpc.getServer(ContainerManagementProtocol.class,
new DummyContainerManager(), addr, conf, null, 1);
server.start();
// Any unrelated protocol would do
ApplicationClientProtocol proxy = (ApplicationClientProtocol) rpc.getProxy(
ApplicationClientProtocol.class, NetUtils.getConnectAddress(server), conf);
try {
proxy.getNewApplication(Records
.newRecord(GetNewApplicationRequest.class));
Assert.fail("Excepted RPC call to fail with unknown method.");
} catch (YarnException e) {
Assert.assertTrue(e.getMessage().matches(
"Unknown method getNewApplication called on.*"
+ "org.apache.hadoop.yarn.proto.ApplicationClientProtocol"
+ "\\$ApplicationClientProtocolService\\$BlockingInterface protocol."));
} catch (Exception e) {
e.printStackTrace();
}
}
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common
@Override
public T run() {
return (T) rpc.getProxy(protocol, serverAddress, conf);
}
});
代码示例来源:origin: org.apache.apex/apex-engine
public YarnClientHelper(Configuration conf)
{
// Set up the configuration and RPC
this.conf = conf;
this.rpc = YarnRPC.create(conf);
}
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common
public Server getServer(Class protocol, Object instance,
InetSocketAddress addr, Configuration conf,
SecretManager<? extends TokenIdentifier> secretManager,
int numHandlers) {
return getServer(protocol, instance, addr, conf, secretManager, numHandlers,
null);
}
代码示例来源:origin: ch.cern.hadoop/hadoop-yarn-client
private boolean tryCloseProxy(
ContainerManagementProtocolProxyData proxy) {
proxy.activeCallers--;
if (proxy.scheduledForClose && proxy.activeCallers < 0) {
LOG.info("Closing proxy : " + proxy.containerManagerBindAddr);
cmProxy.remove(proxy.containerManagerBindAddr);
try {
rpc.stopProxy(proxy.getContainerManagementProtocol(), conf);
} finally {
this.notifyAll();
}
return true;
}
return false;
}
代码示例来源:origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager
rm.start();
YarnRPC rpc = YarnRPC.create(conf);
InetSocketAddress rmAddress = rm.getClientRMService().getBindAddress();
LOG.info("Connecting to ResourceManager at " + rmAddress);
ApplicationClientProtocol client =
(ApplicationClientProtocol) rpc
.getProxy(ApplicationClientProtocol.class, rmAddress, conf);
rpc.stopProxy(client, conf);
new File(excludeFile).delete();
代码示例来源:origin: io.hops/hadoop-yarn-common
@Test
public void testUnknownCall() {
Configuration conf = new Configuration();
conf.set(YarnConfiguration.IPC_RPC_IMPL, HadoopYarnProtoRPC.class
.getName());
YarnRPC rpc = YarnRPC.create(conf);
String bindAddr = "localhost:0";
InetSocketAddress addr = NetUtils.createSocketAddr(bindAddr);
Server server = rpc.getServer(ContainerManagementProtocol.class,
new DummyContainerManager(), addr, conf, null, 1);
server.start();
// Any unrelated protocol would do
ApplicationClientProtocol proxy = (ApplicationClientProtocol) rpc.getProxy(
ApplicationClientProtocol.class, NetUtils.getConnectAddress(server), conf);
try {
proxy.getNewApplication(Records
.newRecord(GetNewApplicationRequest.class));
Assert.fail("Excepted RPC call to fail with unknown method.");
} catch (YarnException e) {
Assert.assertTrue(e.getMessage().matches(
"Unknown method getNewApplication called on.*"
+ "org.apache.hadoop.yarn.proto.ApplicationClientProtocol"
+ "\\$ApplicationClientProtocolService\\$BlockingInterface protocol."));
} catch (Exception e) {
e.printStackTrace();
}
}
代码示例来源:origin: org.apache.hadoop/hadoop-mapreduce-client-common
@Override
public MRClientProtocol run() {
return (MRClientProtocol) rpc.getProxy(HSClientProtocol.class,
hsAddress, conf);
}
});
代码示例来源:origin: alibaba/mpich2-yarn
/**
* Constructor, connect to Resource Manager
*
* @throws IOException
*/
public ApplicationMaster() throws IOException {
super(ApplicationMaster.class.getName());
// Set up the configuration and RPC
conf = new MPIConfiguration();
rpc = YarnRPC.create(conf);
dfs = FileSystem.get(conf);
}
代码示例来源:origin: com.github.jiayuhan-it/hadoop-yarn-common
public Server getServer(Class protocol, Object instance,
InetSocketAddress addr, Configuration conf,
SecretManager<? extends TokenIdentifier> secretManager,
int numHandlers) {
return getServer(protocol, instance, addr, conf, secretManager, numHandlers,
null);
}
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-client
private boolean tryCloseProxy(
ContainerManagementProtocolProxyData proxy) {
proxy.activeCallers--;
if (proxy.scheduledForClose && proxy.activeCallers < 0) {
if (LOG.isDebugEnabled()) {
LOG.debug("Closing proxy : " + proxy.containerManagerBindAddr);
}
cmProxy.remove(proxy.containerManagerBindAddr);
try {
rpc.stopProxy(proxy.getContainerManagementProtocol(), conf);
} finally {
this.notifyAll();
}
return true;
}
return false;
}
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common
@Override
public T run() {
return (T) YarnRPC.create(conf).getProxy(protocol, rmAddress, conf);
}
});
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-nodemanager
Server createServer() {
Configuration conf = getConfig();
YarnRPC rpc = YarnRPC.create(conf);
if (UserGroupInformation.isSecurityEnabled()) {
secretManager = new LocalizerTokenSecretManager();
}
Server server = rpc.getServer(LocalizationProtocol.class, this,
localizationServerAddress, conf, secretManager,
conf.getInt(YarnConfiguration.NM_LOCALIZER_CLIENT_THREAD_COUNT,
YarnConfiguration.DEFAULT_NM_LOCALIZER_CLIENT_THREAD_COUNT));
// Enable service authorization?
if (conf.getBoolean(
CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION,
false)) {
server.refreshServiceAcl(conf, NMPolicyProvider.getInstance());
}
return server;
}
代码示例来源:origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager
YarnRPC rpc = YarnRPC.create(conf);
InetSocketAddress rmAddress = rm.getClientRMService().getBindAddress();
LOG.info("Connecting to ResourceManager at " + rmAddress);
ApplicationClientProtocol client =
(ApplicationClientProtocol) rpc.getProxy(
ApplicationClientProtocol.class, rmAddress, conf);
Assert.assertTrue(nodeToLabels.get(node2).containsAll(Arrays.asList("y")));
rpc.stopProxy(client, conf);
rm.close();
代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-common
conf.set(YarnConfiguration.IPC_RPC_IMPL, HadoopYarnProtoRPC.class
.getName());
YarnRPC rpc = YarnRPC.create(conf);
String bindAddr = "localhost:0";
InetSocketAddress addr = NetUtils.createSocketAddr(bindAddr);
Server server = rpc.getServer(ContainerManagementProtocol.class,
new DummyContainerManager(), addr, conf, null, 1);
server.start();
ApplicationClientProtocol proxy = (ApplicationClientProtocol) rpc.getProxy(
ApplicationClientProtocol.class, NetUtils.getConnectAddress(server),
conf);
代码示例来源:origin: ch.cern.hadoop/hadoop-yarn-common
@Override
public T run() {
return (T) rpc.getProxy(protocol, serverAddress, conf);
}
});
代码示例来源:origin: org.apache.twill/twill-yarn
public Hadoop20YarnAMClient(Configuration conf) {
super(ApplicationConstants.AM_CONTAINER_ID_ENV);
this.amrmClient = new AMRMClientImpl(containerId.getApplicationAttemptId());
this.amrmClient.init(conf);
this.nmClient = new Hadoop20YarnNMClient(YarnRPC.create(conf), conf);
}
Schema.org、Goodrelations-vocabulary.org 和 Productontology.org 之间有什么关系? Schema.org 告知,“W3C schema.org
大家好,我想知道包 org.ietf、org.omg、org.w3c 和 org 是如何实现的.xml 已进入 "official" Java classes ? 例如,默认 JDK 不会包含 Apa
首先,我试图用来自 Schema.org 的属性定义数据库表,例如,例如,我有一个名为“JobPosting”的表,它或多或少具有与 http://schema.org/JobPosting 中定义的
我有一个 org.w3c.dom.Document 被 org.dom4j.io.DOMReader 解析。 我想通过 org.w3c.dom.Element 搜索 dom4j DOM 文档。 比方说
我正在将我的应用程序部署到 Tomcat 6.0.20。 应用程序使用 Hibernate 作为 Web 层的 ORM、Spring 和 JSF。 我还从 main() 方法制作了简单的运行器来测试
我有一个使用 hibernate > 4 的 gradle 项目。如果我在 Apache tomcat 中运行我的 war 文件,我不会收到任何错误。但是当我在 Wildfly 8.2 中部署它时,出
我正在尝试将 JaCoCo 添加到我的 Android 以覆盖 Sonar Qube。但是在运行命令 ./gradlew jacocoTestReport 时,我收到以下错误。 Task :app:
如何在 emacs 组织模式中格式化日期? 例如,在下表中,我希望日期显示为“Aug 29”或“Wed, Aug 29”而不是“” #+ATTR_HTML: border="2" rules="all
我想使用 org 模式来写一本技术书籍。我正在寻找一种将外部文件中的现有代码插入到 babel 代码块中的方法,该代码块在导出为 pdf 时会提供很好的格式。 例如 #+BEGIN_SRC pytho
用作引用:https://support.google.com/webmasters/answer/146750?hl=en 您会注意到在“产品”下有一个属性类别,此外页面下方还有一个示例: Too
我读了这个Google doc .它说我们不使用列表中的产品。 那么对于产品列表(具有多页的类似产品的类别,如“鞋子”),推荐使用哪种模式? 我用这个: { "@context": "htt
我目前在做DBpedia数据集,想通过wikidata实现schema.org和DBpedia的映射。因此我想知道 schema.org 和 wikidata 之间是否存在任何映射。 最佳答案 我认为
我爱org-tables ,我用它们来记录各种事情。我现在正在为 Nix 记录一些单行代码(在阅读了 Domen Kožar 的 excellent guide 后,在 this year's Eur
如果看一下 Movie在 schema.org 中输入,actor 和 actors 属性都是允许的(actor 取代 actors)。但是 author 和 contributor 属性没有等效项。
我们有一些餐厅有多个地点或分支机构。我想包含正确的 Schema.org 标记,但找不到任何允许列出多个餐厅的内容。 每家餐厅都有自己的地址、电子邮件、电话和营业时间,甚至可能是“分店名称”。 两个分
我在一个页面中有多个综合评分片段。 有没有办法让其中之一成为默认值?将显示在搜索引擎结果中的那个? 谢谢大家! 更新:该网页本质上是品牌的页面。它包含品牌评论的总评分及其产品列表(每个产品的总评分)。
我提到了一些相关的职位,但并没有解决我的问题。因为我正在使用maven-jar-plugin-2.4 jar。 我正在使用JBoss Developer Studio 7.1.1 GA IDE,并且正
网站的根页面(即 http://example.com/ )的特殊之处在于它是默认的着陆页。它可能包含许多不同的对象类型。 它可能被认为是一个网站,或者一个博客等... 但它是否也应该被标记为给定对象
我想将一些文本放入一个 org 文件中,当我将内容导出到其中一种目标类型(在本例中为 HTML)时,该文件不会发布。有什么方法可以实现这个目标吗? 最佳答案 您可能想要使用 :noexport: 标签
org-mode 是否有一个键绑定(bind)可以在编号/项目符号列表项之间移动,就像您可以对标题一样? 喜欢的功能: org-forward-heading-same-level 大纲下一个可见标题
我是一名优秀的程序员,十分优秀!