个人中心
文章发布
退出
作者热门文章
- html - 出于某种原因，IE8 对我的 Sass 文件中继承的 html5 CSS 不友好？
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景？
26
4
我正在尝试从此处 (Learning Spark book) 运行示例，但出现以下错误:
16/10/07 01:15:26 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
我的工作开始于:$SPARK_HOME/bin/spark-submit --classcom.oreilly.learningsparkexamples.mini.java.WordCount ./target/learning-spark-mini-example-0.0.1.jar ./README.md ./wordcounts
请指教，为什么会这样？
完整日志:
mini-complete-example$ $SPARK_HOME/bin/spark-submit --class com.oreilly.learningsparkexamples.mini.java.WordCount ./target/learning-spark-mini-example-0.0.1.jar ./README.md ./wordcounts
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/10/07 01:15:23 INFO SparkContext: Running Spark version 2.0.0
16/10/07 01:15:23 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/10/07 01:15:24 INFO SecurityManager: Changing view acls to: eDS
16/10/07 01:15:24 INFO SecurityManager: Changing modify acls to: eDS
16/10/07 01:15:24 INFO SecurityManager: Changing view acls groups to:
16/10/07 01:15:24 INFO SecurityManager: Changing modify acls groups to:
16/10/07 01:15:24 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(eDS); groups with view permissions: Set(); users with modify permissions: Set(eDS); groups with modify permissions: Set()
16/10/07 01:15:24 INFO Utils: Successfully started service 'sparkDriver' on port 63851.
16/10/07 01:15:24 INFO SparkEnv: Registering MapOutputTracker
16/10/07 01:15:24 INFO SparkEnv: Registering BlockManagerMaster
16/10/07 01:15:24 INFO DiskBlockManager: Created local directory at /private/var/folders/yw/zl5hc321387g3sz2fg3l01980000gq/T/blockmgr-0fb2af5a-8662-4d78-88c8-8e0608f35ff3
16/10/07 01:15:24 INFO MemoryStore: MemoryStore started with capacity 366.3 MB
16/10/07 01:15:24 INFO SparkEnv: Registering OutputCommitCoordinator
16/10/07 01:15:24 INFO Utils: Successfully started service 'SparkUI' on port 4040.
16/10/07 01:15:24 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.1.17:4040
16/10/07 01:15:24 INFO SparkContext: Added JAR file:/Users/eDS/dev/learning-spark/mini-complete-example/./target/learning-spark-mini-example-0.0.1.jar at spark://192.168.1.17:63851/jars/learning-spark-mini-example-0.0.1.jar with timestamp 1475795724857
16/10/07 01:15:24 INFO Executor: Starting executor ID driver on host localhost
16/10/07 01:15:24 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 63852.
16/10/07 01:15:24 INFO NettyBlockTransferService: Server created on 192.168.1.17:63852
16/10/07 01:15:24 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.1.17, 63852)
16/10/07 01:15:24 INFO BlockManagerMasterEndpoint: Registering block manager 192.168.1.17:63852 with 366.3 MB RAM, BlockManagerId(driver, 192.168.1.17, 63852)
16/10/07 01:15:24 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.1.17, 63852)
16/10/07 01:15:25 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 145.5 KB, free 366.2 MB)
16/10/07 01:15:25 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 16.3 KB, free 366.1 MB)
16/10/07 01:15:25 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.1.17:63852 (size: 16.3 KB, free: 366.3 MB)
16/10/07 01:15:25 INFO SparkContext: Created broadcast 0 from textFile at WordCount.java:31
16/10/07 01:15:25 INFO FileInputFormat: Total input paths to process : 1
16/10/07 01:15:25 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
16/10/07 01:15:25 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
16/10/07 01:15:25 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
16/10/07 01:15:25 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
16/10/07 01:15:25 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
16/10/07 01:15:25 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
16/10/07 01:15:25 INFO SparkContext: Starting job: saveAsTextFile at WordCount.java:46
16/10/07 01:15:25 INFO DAGScheduler: Registering RDD 3 (mapToPair at WordCount.java:39)
16/10/07 01:15:25 INFO DAGScheduler: Got job 0 (saveAsTextFile at WordCount.java:46) with 2 output partitions
16/10/07 01:15:25 INFO DAGScheduler: Final stage: ResultStage 1 (saveAsTextFile at WordCount.java:46)
16/10/07 01:15:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
16/10/07 01:15:25 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 0)
16/10/07 01:15:25 INFO DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[3] at mapToPair at WordCount.java:39), which has no missing parents
16/10/07 01:15:25 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.9 KB, free 366.1 MB)
16/10/07 01:15:25 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.7 KB, free 366.1 MB)
16/10/07 01:15:25 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.1.17:63852 (size: 2.7 KB, free: 366.3 MB)
16/10/07 01:15:25 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1012
16/10/07 01:15:25 INFO DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[3] at mapToPair at WordCount.java:39)
16/10/07 01:15:25 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks
16/10/07 01:15:25 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0, PROCESS_LOCAL, 5479 bytes)
16/10/07 01:15:25 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, partition 1, PROCESS_LOCAL, 5479 bytes)
16/10/07 01:15:26 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
16/10/07 01:15:26 INFO Executor: Running task 1.0 in stage 0.0 (TID 1)
16/10/07 01:15:26 INFO Executor: Fetching spark://192.168.1.17:63851/jars/learning-spark-mini-example-0.0.1.jar with timestamp 1475795724857
16/10/07 01:15:26 INFO TransportClientFactory: Successfully created connection to /192.168.1.17:63851 after 73 ms (0 ms spent in bootstraps)
16/10/07 01:15:26 INFO Utils: Fetching spark://192.168.1.17:63851/jars/learning-spark-mini-example-0.0.1.jar to /private/var/folders/yw/zl5hc321387g3sz2fg3l01980000gq/T/spark-5adda737-293c-483e-bdbe-f8fa7c171211/userFiles-2a2cf77d-5794-4006-a125-5df94550cbf8/fetchFileTemp6160431711224595115.tmp
16/10/07 01:15:26 INFO Executor: Adding file:/private/var/folders/yw/zl5hc321387g3sz2fg3l01980000gq/T/spark-5adda737-293c-483e-bdbe-f8fa7c171211/userFiles-2a2cf77d-5794-4006-a125-5df94550cbf8/learning-spark-mini-example-0.0.1.jar to class loader
16/10/07 01:15:26 INFO HadoopRDD: Input split: file:/Users/eDS/dev/learning-spark/mini-complete-example/README.md:66+66
16/10/07 01:15:26 INFO HadoopRDD: Input split: file:/Users/eDS/dev/learning-spark/mini-complete-example/README.md:0+66
16/10/07 01:15:26 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/10/07 01:15:26 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 1385 bytes result sent to driver
16/10/07 01:15:26 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 307 ms on localhost (1/2)
16/10/07 01:15:26 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[Executor task launch worker-0,5,main]
java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/10/07 01:15:26 INFO SparkContext: Invoking stop() from shutdown hook
16/10/07 01:15:26 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/10/07 01:15:26 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
16/10/07 01:15:26 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/10/07 01:15:26 INFO TaskSchedulerImpl: Cancelling stage 0
16/10/07 01:15:26 INFO SparkUI: Stopped Spark web UI at http://192.168.1.17:4040
16/10/07 01:15:26 INFO DAGScheduler: ShuffleMapStage 0 (mapToPair at WordCount.java:39) failed in 0.364 s
16/10/07 01:15:26 INFO DAGScheduler: Job 0 failed: saveAsTextFile at WordCount.java:46, took 0.448155 s
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1450)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1438)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1437)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1437)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1659)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1871)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1884)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1904)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1219)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply(PairRDDFunctions.scala:1161)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply(PairRDDFunctions.scala:1161)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopDataset(PairRDDFunctions.scala:1161)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$4.apply$mcV$sp(PairRDDFunctions.scala:1064)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$4.apply(PairRDDFunctions.scala:1030)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$4.apply(PairRDDFunctions.scala:1030)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:1030)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$1.apply$mcV$sp(PairRDDFunctions.scala:956)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$1.apply(PairRDDFunctions.scala:956)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopFile$1.apply(PairRDDFunctions.scala:956)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:955)
at org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1.apply$mcV$sp(RDD.scala:1440)
at org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1.apply(RDD.scala:1419)
at org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1.apply(RDD.scala:1419)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1419)
at org.apache.spark.api.java.JavaRDDLike$class.saveAsTextFile(JavaRDDLike.scala:549)
at org.apache.spark.api.java.AbstractJavaRDDLike.saveAsTextFile(JavaRDDLike.scala:45)
at com.oreilly.learningsparkexamples.mini.java.WordCount.main(WordCount.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/10/07 01:15:26 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerJobEnd(0,1475795726327,JobFailed(org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.AbstractMethodError: com.oreilly.learningsparkexamples.mini.java.WordCount$1.call(Ljava/lang/Object;)Ljava/util/Iterator;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:124)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:))
16/10/07 01:15:26 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/10/07 01:15:26 INFO MemoryStore: MemoryStore cleared
16/10/07 01:15:26 INFO BlockManager: BlockManager stopped
16/10/07 01:15:26 INFO BlockManagerMaster: BlockManagerMaster stopped
16/10/07 01:15:26 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/10/07 01:15:26 INFO SparkContext: Successfully stopped SparkContext
16/10/07 01:15:26 INFO ShutdownHookManager: Shutdown hook called
16/10/07 01:15:26 INFO ShutdownHookManager: Deleting directory /private/var/folders/yw/zl5hc321387g3sz2fg3l01980000gq/T/spark-5adda737-293c-483e-bdbe-f8fa7c171211
mini-complete-example$
最佳答案
我遇到了同样的错误:
ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
和
java.lang.AbstractMethodError: ...
首先，在以下路径中检查 spark 版本:
%SPARK_HOME%/jars/spark-core_x.xx-y.y.y.jar
并修改pom.xml文件如下图:
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_x.xx</artifactId>
<version>y.y.y</version>
<scope>provided</scope>
</dependency>
希望对像我这样的人有所帮助。
关于apache-spark - Apache 星火 : ERROR Executor -> Iterator，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/39907104/
26
4
0
reqwest v0.9 将 serde v1.0 作为依赖项，因此实现 converting serde_json errors into reqwest error . 在我的代码中，我使用 se
我有这个代码: let file = FileStorage { // ... }; file.write("Test", bytes.as_ref()) .map_err(|e| Mu
我只是尝试用angular-cli创建一个新项目，然后运行服务器，但是它停止并显示一条有趣的消息:Error: No errors。 我以这种方式更新了(希望有帮助):npm uninstall -g
我从我的 javascript 发送交易 Metamask 打开传输对话框 我确定 i get an error message in metamask (inpage.js:1 MetaMask -
这个问题在这里已经有了答案: How do you define custom `Error` types in Rust? (3 个答案) How to get a reference to a
我想知道两者之间有什么大的区别 if let error = error{} vs if error != nil?或者只是人们的不同之处，比如他们如何用代码表达自己？ 例如，如果我使用这段代码: u
当我尝试发送超过 50KB 的图像时，我在 Blazor 服务器应用程序上收到以下错误消息 Error: Connection disconnected with error 'Error: Serv
我有一个error-page指令，它将所有异常重定向到错误显示页面 我的web.xml: [...] java.lang.Exception /vi
我有这样的对象: address: { "phone" : 888, "value" : 12 } 在 WHERE 中我需要通过 address.value 查找对象，但是在 SQL 中有函数
每次我尝试编译我的代码时，我都会遇到大量错误。这不是我的代码的问题，因为它在另一台计算机上工作得很好。我尝试重新安装和修复，但这没有帮助。这是整个错误消息: 1>------ Build starte
在我的代码的类部分，如果我写一个错误，则在不应该的情况下，将有几行报告为错误。我将'| error'放在可以从错误中恢复的良好/安全位置，但是我认为它没有使用它。也许它试图在某个地方恢复中间表情？ 有
我遇到了 csv 输入文件整体读取故障的问题，我可以通过在 read_csv 函数中添加 "error_bad_lines=False" 来删除这些问题来解决这个问题。 但是我需要报告这些造成问题的文
在 Spring 中，验证后我们在 controller 中得到一个 BindingResult 对象。 很简单，如果我收到验证错误，我想重新显示我的表单，并在每个受影响的字段上方显示错误消息。 因此
我不知道出了什么问题，因为我用 Java 编程了大约一年，从来没有遇到过这个错误。在一分钟前在 Eclipse 中编译和运行工作，现在我得到这个错误: #A fatal error has been
SELECT to_char(messages. TIME, 'YYYY/MM/DD') AS FullDate, to_char(messages. TIME, 'MM/DD
我收到这些错误: AnonymousPath\Anonymized.vb : error BC30037: Character is not valid. AnonymousPath\Anonymiz
我刚刚安装了 gridengine 并在执行 qstat 时出现错误: error: commlib error: got select error (Connection refused) erro
嗨，我正在学习 PHP，我从 CRUD 系统开始，我在 Windows 上安装了 WAMP 服务器，当我运行它时，我收到以下错误消息。 SCREAM: Error suppression ignore
我刚刚开始一个新项目，我正在学习核心数据教程，可以找到:https://www.youtube.com/watch?v=zZJpsszfTHM 我似乎无法弄清楚为什么会抛出此错误。我有一个名为“Exp
当我使用 Jenkins 运行新构建时，出现以下错误: "FilePathY\XXX.cpp : fatal error C1853: 'FilePathZ\XXX.pch' precompiled
我是一名优秀的程序员,十分优秀！