gpt4 book ai didi

amazon-web-services - DSE Spark 独立集群启动应用程序 'remote Akka client disassociated' 错误

转载 作者:行者123 更新时间:2023-12-04 08:10:34 25 4
gpt4 key购买 nike

我在 DataStax Enterprise 4.7 (DSE) 上使用 Spark 1.2.1 作为 3 个节点(AWS vpc 服务器)的独立集群。
当从主节点向它启动应用程序时,它通过了第一阶段,但在第二阶段遇到了“远程 Akka 客户端已解除关联”错误。
我还收到“要求删除不存在的执行程序 0”错误。

  • 没有 yarn 。
  • 试图将 AKKA 超时设置为 6000,没有任何改变。
  • 所有端口都已设置,集群似乎通过 Spark Web UI 正常运行。

  • 难道是 超时 问题?

    ERROR 2015-07-09 12:59:24 org.apache.spark.scheduler.TaskSchedulerImpl: Lost executor 1 on 1xx.xx.xx.x1: remote Akka client disassociated WARN 2015-07-09 12:59:24 org.apache.spark.scheduler.TaskSetManager: Lost task 6.0 in stage 1.0 (TID 19, 1xx.xx.x.x1): ExecutorLostFailure (executor 1 lost) WARN 2015-07-09 12:59:24 akka.remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@1xx.xx.x.x1:38145] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. ERROR 2015-07-09 12:59:24 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 1 ERROR 2015-07-09 12:59:24 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 1 [Stage 1:=====================================================> (5 + 0) / 12]ERROR 2015-07-09 12:59:32 org.apache.spark.scheduler.TaskSchedulerImpl: Lost executor 2 on 1xx.xx.xx.x2: remote Akka client disassociated WARN 2015-07-09 12:59:32 akka.remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@1xx.xx.xx.x2:33914] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. WARN 2015-07-09 12:59:32 org.apache.spark.scheduler.TaskSetManager: Lost task 0.1 in stage 1.0 (TID 20, 1xx.xx.xx.x2): ExecutorLostFailure (executor 2 lost) ERROR 2015-07-09 12:59:32 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 2 ERROR 2015-07-09 12:59:32 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 2 [Stage 1:====================================================================================> (8 + -2) / 12]ERROR 2015-07-09 13:01:03 org.apache.spark.scheduler.TaskSchedulerImpl: Lost executor 3 on 1xx.xx.xx.x3: remote Akka client disassociated WARN 2015-07-09 13:01:03 akka.remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@1xx.xx.xx.x3:58630] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. WARN 2015-07-09 13:01:03 org.apache.spark.scheduler.TaskSetManager: Lost task 1.1 in stage 1.0 (TID 23, 1xx.xx.xx.x3): ExecutorLostFailure (executor 3 lost) ERROR 2015-07-09 13:01:03 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 3 ERROR 2015-07-09 13:01:03 org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend: Asked to remove non-existent executor 3 [Stage 1:====================================================================================> (8 + -3) / 12

    最佳答案

    我尝试更改 AKKA 设置、端口等,但最终的解决方案是在新的和干净的 AWS 环境中重新开始 - 3 个新服务器并重新安装 DSE 系统。

    :/

    关于amazon-web-services - DSE Spark 独立集群启动应用程序 'remote Akka client disassociated' 错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31319147/

    25 4 0
    Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
    广告合作:1813099741@qq.com 6ren.com