- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
Tomcat 版本:7.0.47
我有一个使用 Apache Spark 的网络应用程序。我的 Web 应用程序充当 Apache Spark 驱动程序。
当远程独立 spark 集群不可用时,Spark 上下文将关闭并显示日志 org.apache.spark.util.Utils - Shutdown hook called
越早发生这种情况,Tomcat 也开始正常关闭。我在tomcat中能看到的唯一日志是[exec] Result: 50
当 spark 调用 it's shut down hook 时,Tomcat 关闭的原因是什么?
Spark 日志
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/downloads/spark-1.4.1-bin-hadoop2.6/lib/spark-assembly-1.4.1-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/downloads/spark-1.4.1-bin-hadoop2.6/lib/spark-examples-1.4.1-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/12/09 17:11:23 INFO SparkContext: Running Spark version 1.4.1
15/12/09 17:11:24 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/12/09 17:11:24 WARN Utils: Your hostname, pesamara-mobl-vm1 resolves to a loopback address: 127.0.0.1; using 10.30.9.107 instead (on interface eth0)
15/12/09 17:11:24 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
15/12/09 17:11:25 INFO SecurityManager: Changing view acls to: pes
15/12/09 17:11:25 INFO SecurityManager: Changing modify acls to: pes
15/12/09 17:11:25 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(pes); users with modify permissions: Set(pes)
15/12/09 17:11:26 INFO Slf4jLogger: Slf4jLogger started
15/12/09 17:11:27 INFO Remoting: Starting remoting
15/12/09 17:11:27 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@10.30.9.107:55740]
15/12/09 17:11:27 INFO Utils: Successfully started service 'sparkDriver' on port 55740.
15/12/09 17:11:27 INFO SparkEnv: Registering MapOutputTracker
15/12/09 17:11:27 INFO SparkEnv: Registering BlockManagerMaster
15/12/09 17:11:27 INFO DiskBlockManager: Created local directory at /tmp/spark-30d61b03-0b1c-4250-b68e-c2404c7884a8/blockmgr-3226ed7e-f8e5-40a2-bfb1-ffabb51cd0e0
15/12/09 17:11:28 INFO MemoryStore: MemoryStore started with capacity 491.5 MB
15/12/09 17:11:28 INFO HttpFileServer: HTTP File server directory is /tmp/spark-30d61b03-0b1c-4250-b68e-c2404c7884a8/httpd-7f2572c2-5677-446e-a80a-6f9d05ee2891
15/12/09 17:11:28 INFO HttpServer: Starting HTTP Server
15/12/09 17:11:28 INFO Utils: Successfully started service 'HTTP file server' on port 45047.
15/12/09 17:11:28 INFO SparkEnv: Registering OutputCommitCoordinator
15/12/09 17:11:28 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/12/09 17:11:28 INFO SparkUI: Started SparkUI at http://10.30.9.107:4040
15/12/09 17:11:29 INFO FairSchedulableBuilder: Created default pool default, schedulingMode: FIFO, minShare: 0, weight: 1
15/12/09 17:11:29 INFO AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@localhost2:7077/user/Master...
15/12/09 17:11:29 WARN AppClient$ClientActor: Could not connect to akka.tcp://sparkMaster@localhost2:7077: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost2:7077
15/12/09 17:11:29 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@localhost2:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: localhost2: unknown error
15/12/09 17:11:49 INFO AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@localhost2:7077/user/Master...
15/12/09 17:11:49 WARN AppClient$ClientActor: Could not connect to akka.tcp://sparkMaster@localhost2:7077: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost2:7077
15/12/09 17:11:49 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@localhost2:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: localhost2: unknown error
15/12/09 17:12:09 INFO AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@localhost2:7077/user/Master...
15/12/09 17:12:09 WARN AppClient$ClientActor: Could not connect to akka.tcp://sparkMaster@localhost2:7077: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost2:7077
15/12/09 17:12:09 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@localhost2:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: localhost2: unknown error
15/12/09 17:12:29 ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.
15/12/09 17:12:29 WARN SparkDeploySchedulerBackend: Application ID is not initialized yet.
15/12/09 17:12:29 INFO SparkUI: Stopped Spark web UI at http://10.30.9.107:4040
15/12/09 17:12:29 INFO DAGScheduler: Stopping DAGScheduler
15/12/09 17:12:29 INFO SparkDeploySchedulerBackend: Shutting down all executors
15/12/09 17:12:29 INFO SparkDeploySchedulerBackend: Asking each executor to shut down
15/12/09 17:12:29 ERROR OneForOneStrategy:
java.lang.NullPointerException
at org.apache.spark.deploy.client.AppClient$ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(AppClient.scala:160)
at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
at org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:59)
at org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
at org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at org.apache.spark.deploy.client.AppClient$ClientActor.aroundReceive(AppClient.scala:61)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
15/12/09 17:12:29 INFO AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@localhost2:7077/user/Master...
15/12/09 17:12:29 WARN AppClient$ClientActor: Could not connect to akka.tcp://sparkMaster@localhost2:7077: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost2:7077
15/12/09 17:12:29 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@localhost2:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: localhost2: unknown error
15/12/09 17:12:29 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 54184.
15/12/09 17:12:29 INFO NettyBlockTransferService: Server created on 54184
15/12/09 17:12:29 INFO BlockManagerMaster: Trying to register BlockManager
15/12/09 17:12:29 INFO BlockManagerMasterEndpoint: Registering block manager 10.30.9.107:54184 with 491.5 MB RAM, BlockManagerId(driver, 10.30.9.107, 54184)
15/12/09 17:12:29 INFO BlockManagerMaster: Registered BlockManager
15/12/09 17:12:30 ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext
at org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:103)
at org.apache.spark.SparkContext.getSchedulingMode(SparkContext.scala:1503)
at org.apache.spark.SparkContext.postEnvironmentUpdate(SparkContext.scala:2007)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:543)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at org.apache.spark.examples.sql.SparkContextTest.main(SparkContextTest.java:32)
15/12/09 17:12:30 INFO SparkContext: SparkContext already stopped.
java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext
at org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:103)
at org.apache.spark.SparkContext.getSchedulingMode(SparkContext.scala:1503)
at org.apache.spark.SparkContext.postEnvironmentUpdate(SparkContext.scala:2007)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:543)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at org.apache.spark.examples.sql.SparkContextTest.main(SparkContextTest.java:32)
15/12/09 17:12:30 INFO DiskBlockManager: Shutdown hook called
15/12/09 17:12:30 INFO Utils: path = /tmp/spark-30d61b03-0b1c-4250-b68e-c2404c7884a8/blockmgr-3226ed7e-f8e5-40a2-bfb1-ffabb51cd0e0, already present as root for deletion.
15/12/09 17:12:30 INFO Utils: Shutdown hook called
15/12/09 17:12:30 INFO Utils: Deleting directory /tmp/spark-30d61b03-0b1c-4250-b68e-c2404c7884a8/httpd-7f2572c2-5677-446e-a80a-6f9d05ee2891
15/12/09 17:12:30 INFO Utils: Deleting directory /tmp/spark-30d61b03-0b1c-4250-b68e-c2404c7884a8
最佳答案
如果您使用 IP 地址代替主机名,则在 SparkConf 中设置主机时使用 spark://hostname:7077 作为 spark 主机。
我遇到了同样的问题,通过在 spark master 中使用主机名而不是使用 IP 地址解决了这个问题。
问候,火锅
关于tomcat - Apache Spark 导致 Tomcat 正常关闭,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34180278/
我正在使用的网站上有一个非 Canvas 导航。关闭 Canvas 导航的默认状态是关闭的,这在移动网站上运行良好,因为您可以打开它并选择您的链接,但在桌面上关闭它并打开它会隐藏用户的信息,我希望它是
我有一个 NSViewController 是这样连接的: 在底部 viewController 中,我尝试使用 self.dismiss(self) 关闭它,但是,它会产生此错误: [General
我昨天制作了一个扩展的 JQuery 搜索框,它的作用就像一个魅力!但是,我在创建一个脚本时遇到问题,当用户单击搜索框时,它会关闭。 这是我的 JQuery: function expandSearc
我一辈子都无法在 API V3 中一次只显示一个信息窗口。我需要一个在下一次开放之前关闭。还希望在 map 上的任何地方关闭 infoWindow onclick。这是否在初始化函数中? 这是我的完整
关闭和清理套接字的正确方法是什么? 我在辅助线程中运行 io_service,我需要关闭与主线程的连接: void closeConnection() { ioc.post([&socket]
我的 Selenium 测试看起来像这样:客户选择金融产品,填写一些必要的数据,并在打印预览中显示条款/协议(protocol)文档(根据本地法律的要求)。打印/关闭打印预览对话框后,客户输入更多数据
我目前正在从 android 网站了解 Navigation Drawer,我正在使用他们的示例 http://developer.android.com/training/implementing-
尝试通过 expo 在模拟器上运行 react-native 应用程序时出现此错误。 Couldn't start project on Android: Error running adb: adb
方法一 function transform(ar) { var alStr = []; for(var i=0; i
我想按以下方式自定义我的抽屉导航: 我希望在抽屉打开时显示一个图标,在抽屉关闭时显示另一个图标,而不是将菜单图标稍微向左滑动的当前默认动画。 关于我在哪里可以找到类似内容的任何想法/线索? 我做了一些
我们刚刚从 0.6.2 或 0.7 升级了我们的 dropwizard 版本,发现 .yml 文件中的很多配置都发生了变化。尽管我们能够弄清楚其中的大部分,但我们无法弄清楚如何关闭“requestLo
从 celery 2.4.5 升级后,我开始让 celery 随机关闭。 我在 centOS 机器上使用 celery 3.0.12、boto 2.6 和 amazon sqs 和 django 1.
我试图包含一些语句来指导用户更多地了解文件无法打开或关闭的原因。文件在写入模式下无法打开的一些可能情况是什么?无法关闭怎么办? FILE *fp; if(!(fp = fopen("testing",
我有一个DLL,可以访问数据库并从存储在配置文件中的应用程序设置中读取连接字符串。然后,引用此DLL的应用程序将需要在其配置文件中为此配置设置设置值。 我遇到的问题是,生成的配置代码会通过Defaul
我将 UIDatePicker 添加为 UITextField 的输入 View UIDatePicker *oBirth; NSDateFormatter *dateFormat; _edit
我有以下代码: SecondViewController *secondView = [[SecondViewController alloc] initWithNibName:@"SecondVie
Closed. This question does not meet Stack Overflow guidelines。它当前不接受答案。 想改善这个问题吗?更新问题,以便将其作为on-topic
通常,按下 option 键关闭窗口会关闭应用程序中的所有窗口。在我的应用程序中,我希望它仅关闭与用户正在关闭的窗口相关的窗口。我怎样才能做到这一点?我可以为所有窗口实现 windowShouldCl
我有一个 NSWindow,它托管一个已连接到脚本处理程序的 WebView。 现在,当用户单击 WebView 上的控件上的按钮时,它会调用我的对象上的 Objective C 方法。 在这种特定情
我想根据 MBP 上的相机使用情况自动化个人工作流程。 基本上我想知道是否任何 的摄像头(内置或 USB)已打开或关闭,因此我可以运行我将创建的程序或脚本。 我认为如果我需要轮询相机状态也可以,但基于
我是一名优秀的程序员,十分优秀!