- 使用 Spring Initializr 创建 Spring Boot 应用程序
- 在Spring Boot中配置Cassandra
- 在 Spring Boot 上配置 Tomcat 连接池
- 将Camel消息路由到嵌入WildFly的Artemis上
本文整理了Java中org.apache.samza.job.yarn.YarnContainer
类的一些代码示例,展示了YarnContainer
类的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。YarnContainer
类的具体详情如下:
包路径:org.apache.samza.job.yarn.YarnContainer
类名称:YarnContainer
暂无
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
/**
* Returns the Id of the Samza container that corresponds to the provided Yarn {@link ContainerId}
* @param containerId the Yarn ContainerId
* @return the id of the Samza container corresponding to the {@link ContainerId} that is pending launch
*/
private String getPendingSamzaContainerId(ContainerId containerId) {
for (String samzaContainerId: state.pendingYarnContainers.keySet()) {
YarnContainer yarnContainer = state.pendingYarnContainers.get(samzaContainerId);
if (yarnContainer != null && yarnContainer.id().equals(containerId)) {
return samzaContainerId;
}
}
return null;
}
代码示例来源:origin: org.apache.samza/samza-yarn
@Override
public void onContainerStarted(ContainerId containerId, Map<String, ByteBuffer> allServiceResponse) {
log.info("Received a containerStart notification from the NodeManager for container: {} ", containerId);
String samzaContainerId = getPendingSamzaContainerId(containerId);
if (samzaContainerId != null) {
// 1. Move the container from pending to running state
final YarnContainer container = state.pendingYarnContainers.remove(samzaContainerId);
log.info("Samza containerId:{} has started", samzaContainerId);
state.runningYarnContainers.put(samzaContainerId, container);
// 2. Invoke the success callback.
SamzaResource resource = new SamzaResource(container.resource().getVirtualCores(),
container.resource().getMemory(), container.nodeId().getHost(), containerId.toString());
clusterManagerCallback.onStreamProcessorLaunchSuccess(resource);
} else {
log.info("Got an invalid notification from YARN for container: {}", containerId);
}
}
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
state.pendingYarnContainers.put(samzaContainerId, new YarnContainer(container));
代码示例来源:origin: org.apache.samza/samza-yarn
state.pendingYarnContainers.put(samzaContainerId, new YarnContainer(container));
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
@Override
public void onContainerStarted(ContainerId containerId, Map<String, ByteBuffer> allServiceResponse) {
log.info("Received a containerStart notification from the NodeManager for container: {} ", containerId);
String samzaContainerId = getPendingSamzaContainerId(containerId);
if (samzaContainerId != null) {
// 1. Move the container from pending to running state
final YarnContainer container = state.pendingYarnContainers.remove(samzaContainerId);
log.info("Samza containerId:{} has started", samzaContainerId);
state.runningYarnContainers.put(samzaContainerId, container);
// 2. Invoke the success callback.
SamzaResource resource = new SamzaResource(container.resource().getVirtualCores(),
container.resource().getMemory(), container.nodeId().getHost(), containerId.toString());
clusterManagerCallback.onStreamProcessorLaunchSuccess(resource);
} else {
log.info("Got an invalid notification from YARN for container: {}", containerId);
}
}
代码示例来源:origin: org.apache.samza/samza-yarn
/**
* Returns the Id of the Samza container that corresponds to the provided Yarn {@link ContainerId}
* @param containerId the Yarn ContainerId
* @return the id of the Samza container corresponding to the {@link ContainerId} that is pending launch
*/
private String getPendingSamzaContainerId(ContainerId containerId) {
for (String samzaContainerId: state.pendingYarnContainers.keySet()) {
YarnContainer yarnContainer = state.pendingYarnContainers.get(samzaContainerId);
if (yarnContainer != null && yarnContainer.id().equals(containerId)) {
return samzaContainerId;
}
}
return null;
}
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
@Override
public void onStartContainerError(ContainerId yarnContainerId, Throwable t) {
log.error(String.format("Yarn Container: %s could not start.", yarnContainerId), t);
String samzaContainerId = getPendingSamzaContainerId(yarnContainerId);
if (samzaContainerId != null) {
YarnContainer container = state.pendingYarnContainers.remove(samzaContainerId);
log.info("Failed Yarn Container: {} had Samza ContainerId: {} ", yarnContainerId, samzaContainerId);
SamzaResource resource = new SamzaResource(container.resource().getVirtualCores(),
container.resource().getMemory(), container.nodeId().getHost(), yarnContainerId.toString());
log.info("Invoking failure callback for container: {}", yarnContainerId);
clusterManagerCallback.onStreamProcessorLaunchFailure(resource, new SamzaContainerLaunchException(t));
} else {
log.info("Got an invalid notification for container: {}", yarnContainerId);
}
}
代码示例来源:origin: org.apache.samza/samza-yarn
/**
* Given a lookupContainerId from Yarn (for example: containerId_app_12345, this method returns the SamzaContainer ID
* in the range [0,N-1] that maps to it.
* @param lookupContainerId the Yarn container ID.
* @return the samza container ID.
*/
//TODO: Get rid of the YarnContainer object and just use Container in state.runningYarnContainers hashmap.
//In that case, this scan will turn into a lookup. This change will require changes/testing in the UI files because
//those UI stub templates operate on the YarnContainer object.
private String getIDForContainer(String lookupContainerId) {
String samzaContainerID = INVALID_YARN_CONTAINER_ID;
for(Map.Entry<String, YarnContainer> entry : state.runningYarnContainers.entrySet()) {
String key = entry.getKey();
YarnContainer yarnContainer = entry.getValue();
String yarnContainerId = yarnContainer.id().toString();
if(yarnContainerId.equals(lookupContainerId)) {
return key;
}
}
return samzaContainerID;
}
代码示例来源:origin: org.apache.samza/samza-yarn
@Override
public void onStartContainerError(ContainerId yarnContainerId, Throwable t) {
log.error(String.format("Yarn Container: %s could not start.", yarnContainerId), t);
String samzaContainerId = getPendingSamzaContainerId(yarnContainerId);
if (samzaContainerId != null) {
YarnContainer container = state.pendingYarnContainers.remove(samzaContainerId);
log.info("Failed Yarn Container: {} had Samza ContainerId: {} ", yarnContainerId, samzaContainerId);
SamzaResource resource = new SamzaResource(container.resource().getVirtualCores(),
container.resource().getMemory(), container.nodeId().getHost(), yarnContainerId.toString());
log.info("Invoking failure callback for container: {}", yarnContainerId);
clusterManagerCallback.onStreamProcessorLaunchFailure(resource, new SamzaContainerLaunchException(t));
} else {
log.info("Got an invalid notification for container: {}", yarnContainerId);
}
}
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
/**
* Given a lookupContainerId from Yarn (for example: containerId_app_12345, this method returns the SamzaContainer ID
* in the range [0,N-1] that maps to it.
* @param lookupContainerId the Yarn container ID.
* @return the samza container ID.
*/
//TODO: Get rid of the YarnContainer object and just use Container in state.runningYarnContainers hashmap.
//In that case, this scan will turn into a lookup. This change will require changes/testing in the UI files because
//those UI stub templates operate on the YarnContainer object.
private String getIDForContainer(String lookupContainerId) {
String samzaContainerID = INVALID_YARN_CONTAINER_ID;
for(Map.Entry<String, YarnContainer> entry : state.runningYarnContainers.entrySet()) {
String key = entry.getKey();
YarnContainer yarnContainer = entry.getValue();
String yarnContainerId = yarnContainer.id().toString();
if(yarnContainerId.equals(lookupContainerId)) {
return key;
}
}
return samzaContainerID;
}
代码示例来源:origin: org.apache.samza/samza-yarn_2.11
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
ContainerId yarnContainerId;
PrintWriter printWriter = resp.getWriter();
String containerIdParam = req.getParameter(YARN_CONTAINER_ID);
ContainerHeartbeatResponse response;
resp.setContentType(APPLICATION_JSON);
boolean alive = false;
try {
yarnContainerId = ContainerId.fromString(containerIdParam);
for (YarnContainer yarnContainer : yarnAppState.runningYarnContainers.values()) {
if (yarnContainer.id().compareTo(yarnContainerId) == 0) {
alive = true;
break;
}
}
if (!alive) {
heartbeatsExpiredCount.inc();
}
response = new ContainerHeartbeatResponse(alive);
printWriter.write(mapper.writeValueAsString(response));
} catch (IllegalArgumentException e) {
LOG.error("Container ID {} passed is invalid", containerIdParam);
resp.sendError(HttpServletResponse.SC_BAD_REQUEST, e.getMessage());
}
}
}
代码示例来源:origin: org.apache.samza/samza-yarn
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
ContainerId yarnContainerId;
PrintWriter printWriter = resp.getWriter();
String containerIdParam = req.getParameter(YARN_CONTAINER_ID);
ContainerHeartbeatResponse response;
resp.setContentType(APPLICATION_JSON);
boolean alive = false;
try {
yarnContainerId = ContainerId.fromString(containerIdParam);
for (YarnContainer yarnContainer : yarnAppState.runningYarnContainers.values()) {
if (yarnContainer.id().compareTo(yarnContainerId) == 0) {
alive = true;
break;
}
}
if (!alive) {
heartbeatsExpiredCount.inc();
}
response = new ContainerHeartbeatResponse(alive);
printWriter.write(mapper.writeValueAsString(response));
} catch (IllegalArgumentException e) {
LOG.error("Container ID {} passed is invalid", containerIdParam);
resp.sendError(HttpServletResponse.SC_BAD_REQUEST, e.getMessage());
}
}
}
我正在关注Apache Samza website上的hello-samza教程,并想按此处所述添加REST服务:http://samza.apache.org/learn/tutorials/lat
您好,由于以下错误 ,我的 samza 作业容器经常失败: Exception from container-launch. Container id: container_1540535314451
如果您使用 Samza 的 OutgoingMessageEnvelope使用此格式发送消息: public OutgoingMessageEnvelope(SystemStream systemSt
使用 Samza KeyValueStore接口(interface),如何检索具有公共(public)键前缀的所有文档?键是字符串,RocksDb 将是底层存储。 下面使用 range 方法的方法是
我正在尝试设置 Apache Samza 和 Kafka 环境。我在尝试运行模块时遇到了一些问题。 我让 Kafka 正常工作,但我无法让 Samza 工作。我已经安装了两个 Debian Jeesy
我目前正在编写一个 Samza 脚本,它只会从 Kafka 主题获取数据并将数据输出到另一个 Kafka 主题。我写了一个非常基本的 StreamTask 但是在执行时我遇到了错误。 错误如下: Ex
本文整理了Java中org.apache.samza.zk.ZkCoordinationUtilsFactory类的一些代码示例,展示了ZkCoordinationUtilsFactory类的具体用法
本文整理了Java中org.apache.samza.zk.ZkProcessorLatch类的一些代码示例,展示了ZkProcessorLatch类的具体用法。这些代码示例主要来源于Github/S
本文整理了Java中org.apache.samza.zk.ZkUtils类的一些代码示例,展示了ZkUtils类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven
本文整理了Java中org.apache.samza.zk.ZkLeaderElector类的一些代码示例,展示了ZkLeaderElector类的具体用法。这些代码示例主要来源于Github/Sta
本文整理了Java中org.apache.samza.zk.ZkKeyBuilder类的一些代码示例,展示了ZkKeyBuilder类的具体用法。这些代码示例主要来源于Github/Stackover
本文整理了Java中org.apache.samza.zk.ZkJobCoordinator类的一些代码示例,展示了ZkJobCoordinator类的具体用法。这些代码示例主要来源于Github/S
本文整理了Java中org.apache.samza.zk.ZkBarrierForVersionUpgrade类的一些代码示例,展示了ZkBarrierForVersionUpgrade类的具体用法
本文整理了Java中org.apache.samza.zk.ZkDistributedLock类的一些代码示例,展示了ZkDistributedLock类的具体用法。这些代码示例主要来源于Github
本文整理了Java中org.apache.samza.zk.ZkBarrierListener类的一些代码示例,展示了ZkBarrierListener类的具体用法。这些代码示例主要来源于Github
本文整理了Java中org.apache.samza.zk.ZkUtilsMetrics类的一些代码示例,展示了ZkUtilsMetrics类的具体用法。这些代码示例主要来源于Github/Stack
本文整理了Java中org.apache.samza.zk.ZkCoordinationUtils类的一些代码示例,展示了ZkCoordinationUtils类的具体用法。这些代码示例主要来源于Gi
本文整理了Java中org.apache.samza.zk.ZkJobCoordinatorFactory类的一些代码示例,展示了ZkJobCoordinatorFactory类的具体用法。这些代码示
本文整理了Java中org.apache.samza.config.ZkConfig类的一些代码示例,展示了ZkConfig类的具体用法。这些代码示例主要来源于Github/Stackoverflow
本文整理了Java中org.apache.samza.zk.ZkMetadataStore类的一些代码示例,展示了ZkMetadataStore类的具体用法。这些代码示例主要来源于Github/Sta
我是一名优秀的程序员,十分优秀!