apache-spark - 在 Yarn 上启动 Spark 时在资源管理器上抛出 "java.io.IOException: Connection reset by peer"

Question

我正在尝试在集群上使用 yarn 客户端模式启动 Spark 作业。我已经用 yarn 尝试过 spark-shell，我可以启动该应用程序。但是，我还希望能够从 Eclipse 运行驱动程序，同时使用集群运行任务。我还向 HDFS 添加了 spark-assembly jar，并通过将 (HADOOP_CONF_DIR env variable) 添加到 eclipse 来指向它，尽管我不确定这是否是解决此问题的最佳方法。

我的应用程序确实在集群上启动(正如我在资源管理器的监视器中看到的那样)它“成功”完成，但没有任何结果返回给驱动程序。我在 Eclipse 控制台中看到以下异常:

WARN  10:11:08,375  Logging.scala:71 -- Lost task 0.0 in stage 1.0 (TID 1, vanpghdcn2.pgdev.sap.corp): java.lang.NullPointerException 
        at org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1.apply(ExistingRDD.scala:56) 
        at org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1.apply(ExistingRDD.scala:55) 
        at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686) 
        at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686) 
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70) 
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) 
        at org.apache.spark.scheduler.Task.run(Task.scala:70) 
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745) 

ERROR 10:11:08,522  Logging.scala:75 -- Task 0 in stage 1.0 failed 4 times; aborting job 
INFO  10:11:08,538  SparkUtils.scala:67 --           SparkContext stopped 

我在 ResourceManager 日志中得到以下信息:

2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:1024, vCores:1>, Priority: 0, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 clusterResource=<memory:24576, vCores:24>
2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:1024, vCores:1>, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 
2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.041666668 absoluteUsedCapacity=0.041666668 used=<memory:1024, vCores:1> cluster=<memory:24576, vCores:24>
2015-10-22 10:10:42,604 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn3.pgdev.sap.corp:41419 for container : container_1445462013958_0012_01_000001 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Clear node set for appattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Storing attempt: AppId: application_1445462013958_0012 AttemptId: appattempt_1445462013958_0012_000001 MasterContainer: Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:1024, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from SCHEDULED to ALLOCATED_SAVING 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from ALLOCATED_SAVING to ALLOCATED 
2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching masterappattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting up container Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:1024, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] for AM appattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Command to launch container container_1445462013958_0012_01_000001 : {{JAVA_HOME}}/bin/java,-server,-Xmx512m,-Djava.io.tmpdir={{PWD}}/tmp,'-Dspark.eventLog.dir=','-Dspark.driver.port=57819','-Dspark.app.name=Sparca Application','-Dspark.executor.memory=1g','-Dspark.master=yarn-client','-Dspark.executor.id=driver','-Dspark.externalBlockStore.folderName=spark-10391661-8d35-40d9-8242-fe79bdc19d2d','-Dspark.fileserver.uri=<a href="http://10.161.43.118:57820','-Dspark.driver.appUIAddress=http://10.161.43.118:4040','-Dspark.driver.host=10.161.43.118','-Dspark.eventLog.enabled=false','-Dspark.yarn.jar=hdfs://vanpghdcn1.pgdev.sap.corp:8020/data/spark-assembly-1.4.0-hadoop2.6.0.jar','-Dspark.cores.max=6',-Dspark.yarn.app.container.log.dir=">http://10.161.43.118:57820','-Dspark.driver.appUIAddress=http://10.161.43.118:4040','-Dspark.driver.host=10.161.43.118','-Dspark.eventLog.enabled=false','-Dspark.yarn.jar=hdfs://vanpghdcn1.pgdev.sap.corp:8020/data/spark-assembly-1.4.0-hadoop2.6.0.jar','-Dspark.cores.max=6',-Dspark.yarn.app.container.log.dir=<LOG_DIR>,org.apache.spark.deploy.yarn.ExecutorLauncher,--arg,'10.161.43.118:57819',--executor-memory,1024m,--executor-cores,1,--num-executors ,2,1>,<LOG_DIR>/stdout,2>,<LOG_DIR>/stderr 
2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Create AMRMToken for ApplicationAttempt: appattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Creating password for appattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,640 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Done launching container Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:1024, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] for AM appattempt_1445462013958_0012_000001 
2015-10-22 10:10:42,640 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from ALLOCATED to LAUNCHED 
2015-10-22 10:10:43,613 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from ACQUIRED to RUNNING 
2015-10-22 10:10:48,176 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1445462013958_0012_000001 (auth:SIMPLE) 
2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AM registration appattempt_1445462013958_0012_000001 
2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs IP=10.165.28.145    OPERATION=Register App Master   TARGET=ApplicationMasterService RESULT=SUCCESS  APPID=application_1445462013958_0012 APPATTEMPTID=appattempt_1445462013958_0012_000001 
2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from LAUNCHED to RUNNING 
2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from ACCEPTED to RUNNING 
2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from NEW to ALLOCATED 
2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container    TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000002 
2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1445462013958_0012_01_000002 of capacity <memory:2048, vCores:1> on host vanpghdcn3.pgdev.sap.corp:41419, which has 2 containers, <memory:3072, vCores:2> used and <memory:5120, vCores:6> available after allocation 
2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000002, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:2048, vCores:1>, Priority: 1, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:1024, vCores:1>, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 clusterResource=<memory:24576, vCores:24>
2015-10-22 10:10:48,633 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:3072, vCores:2>, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 
2015-10-22 10:10:48,633 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used=<memory:3072, vCores:2> cluster=<memory:24576, vCores:24>
2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from NEW to ALLOCATED 
2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container    TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000003 
2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1445462013958_0012_01_000003 of capacity <memory:2048, vCores:1> on host vanpghdcn2.pgdev.sap.corp:36064, which has 1 containers, <memory:2048, vCores:1> used and <memory:6144, vCores:7> available after allocation 
2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000003, NodeId: vanpghdcn2.pgdev.sap.corp:36064, NodeHttpAddress: vanpghdcn2.pgdev.sap.corp:8042, Resource: <memory:2048, vCores:1>, Priority: 1, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:3072, vCores:2>, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 clusterResource=<memory:24576, vCores:24>
2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:5120, vCores:3>, usedCapacity=0.20833333, absoluteUsedCapacity=0.20833333, numApps=1, numContainers=3 
2015-10-22 10:10:48,820 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.20833333 absoluteUsedCapacity=0.20833333 used=<memory:5120, vCores:3> cluster=<memory:24576, vCores:24>
2015-10-22 10:10:53,253 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn3.pgdev.sap.corp:41419 for container : container_1445462013958_0012_01_000002 
2015-10-22 10:10:53,255 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 
2015-10-22 10:10:53,256 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn2.pgdev.sap.corp:36064 for container : container_1445462013958_0012_01_000003 
2015-10-22 10:10:53,257 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 
2015-10-22 10:10:53,643 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from ACQUIRED to RUNNING 
2015-10-22 10:10:53,830 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from ACQUIRED to RUNNING 
2015-10-22 10:10:58,282 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: checking for deactivate of application :application_1445462013958_0012 
2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Updating application attempt appattempt_1445462013958_0012_000001 with final state: FINISHING, and exit status: -1000 
2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from RUNNING to FINAL_SAVING 
2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Updating application application_1445462013958_0012 with final state: FINISHING 
2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from RUNNING to FINAL_SAVING 
2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Updating info for app: application_1445462013958_0012 
2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from FINAL_SAVING to FINISHING 
2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from FINAL_SAVING to FINISHING 
2015-10-22 10:11:08,453 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: application_1445462013958_0012 unregistered successfully. 
2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from RUNNING to COMPLETED 
2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000002 in state: COMPLETED event:FINISHED 
2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000002 
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000002 of capacity <memory:2048, vCores:1> on host vanpghdcn3.pgdev.sap.corp:41419, which currently has 1 containers, <memory:1024, vCores:1> used and <memory:7168, vCores:7> available, release resources=true 
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=<memory:3072, vCores:2> numContainers=2 user=hdfs user-resources=<memory:3072, vCores:2>
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000002, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:2048, vCores:1>, Priority: 1, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:3072, vCores:2>, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 cluster=<memory:24576, vCores:24>
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used=<memory:3072, vCores:2> cluster=<memory:24576, vCores:24>
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:3072, vCores:2>, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 
2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000002 on node: host: vanpghdcn3.pgdev.sap.corp:41419 #containers=1 available=<memory:7168, vCores:7> used=<memory:1024, vCores:1> with event: FINISHED 
2015-10-22 10:11:08,704 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8050: readAndProcess from client 10.161.43.118 threw exception [java.io.IOException: Connection reset by peer] 
java.io.IOException: Connection reset by peer 
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method) 
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) 
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) 
        at sun.nio.ch.IOUtil.read(IOUtil.java:197) 
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) 
        at org.apache.hadoop.ipc.Server.channelRead(Server.java:2603) 
        at org.apache.hadoop.ipc.Server.access$2800(Server.java:136) 
        at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1481) 
        at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:771) 
        at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:637) 
        at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:608) 
2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from RUNNING to COMPLETED 
2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000003 in state: COMPLETED event:FINISHED 
2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000003 
2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000003 of capacity <memory:2048, vCores:1> on host vanpghdcn2.pgdev.sap.corp:36064, which currently has 0 containers, <memory:0, vCores:0> used and <memory:8192, vCores:8> available, release resources=true 
2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=<memory:1024, vCores:1> numContainers=1 user=hdfs user-resources=<memory:1024, vCores:1>
2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000003, NodeId: vanpghdcn2.pgdev.sap.corp:36064, NodeHttpAddress: vanpghdcn2.pgdev.sap.corp:8042, Resource: <memory:2048, vCores:1>, Priority: 1, Token: Token { kind: ContainerToken, service: 10.165.28.143:36064 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:1024, vCores:1>, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 cluster=<memory:24576, vCores:24>
2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.041666668 absoluteUsedCapacity=0.041666668 used=<memory:1024, vCores:1> cluster=<memory:24576, vCores:24>
2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:1024, vCores:1>, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 
2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000003 on node: host: vanpghdcn2.pgdev.sap.corp:36064 #containers=0 available=<memory:8192, vCores:8> used=<memory:0, vCores:0> with event: FINISHED 
2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from RUNNING to COMPLETED 
2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: Unregistering app attempt : appattempt_1445462013958_0012_000001 
2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000001 in state: COMPLETED event:FINISHED 
2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000001 
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000001 of capacity <memory:1024, vCores:1> on host vanpghdcn3.pgdev.sap.corp:41419, which currently has 0 containers, <memory:0, vCores:0> used and <memory:8192, vCores:8> available, release resources=true 
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=<memory:0, vCores:0> numContainers=0 user=hdfs user-resources=<memory:0, vCores:0>
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: <memory:1024, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 cluster=<memory:24576, vCores:24>
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.0 absoluteUsedCapacity=0.0 used=<memory:0, vCores:0> cluster=<memory:24576, vCores:24>
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 
2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000001 on node: host: vanpghdcn3.pgdev.sap.corp:41419 #containers=0 available=<memory:8192, vCores:8> used=<memory:0, vCores:0> with event: FINISHED 
2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Application finished, removing password for appattempt_1445462013958_0012_000001 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from FINISHING to FINISHED 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from FINISHING to FINISHED 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application Attempt appattempt_1445462013958_0012_000001 is done. finalState=FINISHED 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=Application Finished - Succeeded  TARGET=RMAppManager RESULT=SUCCESS  APPID=application_1445462013958_0012 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: Application application_1445462013958_0012 requests cleared 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Application removed - appId: application_1445462013958_0012 user: hdfs queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1445462013958_0012 user: hdfs leaf-queue of parent: root #applications: 0 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1445462013958_0012,name=Sparca Application,user=hdfs,queue=default,state=FINISHED,trackingUrl=http://vanpghdcn1:8088/proxy/application_1445462013958_0012/,appMasterHost=10.165.28.145,startTime=1445533842429,finishTime=1445533868349,finalStatus=SUCCEEDED,memorySeconds=109990,vcoreSeconds=67,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0\, vCores:0>,applicationType=SPARK 
2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Cleaning master appattempt_1445462013958_0012_000001 
2015-10-22 10:11:10,719 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Null container completed... 
2015-10-22 10:11:10,925 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Null container completed... 

值得一提的是，10.161.43.118 是我运行 eclipse 的机器。我的测试应用程序只是将 csv 读入数据帧并进行计数。

Answer 1

你的文件有多大？
这表示，某种客户端网络问题暂时导致主机和端点之间的通信问题。
如果文件大小足够大，那么通过 TCP channel 传输数据需要很长时间，并且连接不能长时间保持事件状态并被重置。希望，这个答案有帮助。