个人中心
gpt4 book ai didi

scala - java.lang.ClassNotFoundException:类scala.runtime.Nothing运行scoobi WordCount示例时

转载 作者：行者123 更新时间：2023-12-02 21:51:37 25 4
gpt4 key购买 nike

我正在尝试从“快速入门”页面运行字数示例

import com.nicta.scoobi.Scoobi._
import Reduction._

object WordCount extends ScoobiApp {
  def run() {
    val lines = fromTextFile(args(0))

    val counts = lines.mapFlatten(_.split(" "))
      .map(word => (word, 1))
      .groupByKey
      .combine(Sum.int)
    counts.toTextFile(args(1)).persist
  }
}

在内存模式下使用时，它工作正常，但是在尝试本地模式(或群集模式)时，我失败并显示以下错误: 
[WARN] LocalJobRunner - job_local_0001 <java.lang.RuntimeException: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found>java.lang.RuntimeException: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1439)
        at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.com$nicta$scoobi$impl$mapreducer$ChannelOutputFormat$$mkTaskContext$1(ChannelOutputFormat.scala:63)
        at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat$$anonfun$getContext$1.apply(ChannelOutputFormat.scala:75)
        at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat$$anonfun$getContext$1.apply(ChannelOutputFormat.scala:75)
        at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189)
        at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91)
        at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.getContext(ChannelOutputFormat.scala:75)
        at com.nicta.scoobi.impl.mapreducer.ChannelOutputFormat.write(ChannelOutputFormat.scala:43)
        at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5$$anonfun$write$1.apply(OutputChannel.scala:137)
        at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5$$anonfun$write$1.apply(OutputChannel.scala:135)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at com.nicta.scoobi.impl.plan.mscr.MscrOutputChannel$$anon$5.write(OutputChannel.scala:135)
        at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply$mcV$sp(OutputChannel.scala:201)
        at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply(OutputChannel.scala:201)
        at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel$$anonfun$reduce$1.apply(OutputChannel.scala:201)
        at scala.Option.getOrElse(Option.scala:120)
        at com.nicta.scoobi.impl.plan.mscr.GbkOutputChannel.reduce(OutputChannel.scala:200)
        at com.nicta.scoobi.impl.mapreducer.MscrReducer$$anonfun$reduce$1.apply(MscrReducer.scala:55)
        at com.nicta.scoobi.impl.mapreducer.MscrReducer$$anonfun$reduce$1.apply(MscrReducer.scala:52)
        at scala.Option.foreach(Option.scala:236)
        at com.nicta.scoobi.impl.mapreducer.MscrReducer.reduce(MscrReducer.scala:52)
        at com.nicta.scoobi.impl.mapreducer.MscrReducer.reduce(MscrReducer.scala:33)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:164)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:572)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:309)
Caused by: java.lang.ClassNotFoundException: Class scala.runtime.Nothing$ not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1350)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1437)
        ... 25 more

[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/3757970833182018747_-1642337927_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.combiners-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/1307074498433974065_910223079_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.mappers-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-624792843022440048_-470268278_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TG23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-7527273518266336656_-470264434_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TK23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-7162952586058180219_-470259629_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TP23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/-1228551315878554095_-470253863_156372685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.metadata.TV23
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/6598684265640022340_1943382592_156373685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/dist-objs/scoobi.reducers-step1 of 1
[INFO] TrackerDistributedCacheManager - Deleted path /tmp/scoobi-root/WordCount$-1124-035523--1298047809/step1 of 1/archive/1699308645513763631_1905624154_156371685/file/tmp/scoobi-root/WordCount$-1124-035523--1298047809/env/a88809af-334b-499e-bafc-1a2ebeffdfbd
[INFO] MapReduceJob - Map 100%    Reduce   0%
[error] (run-main) com.nicta.scoobi.impl.exec.JobExecException: MapReduce job 'job_local_0001' failed! Please see http://localhost:8080/ for more info.
com.nicta.scoobi.impl.exec.JobExecException: MapReduce job 'job_local_0001' failed! Please see http://localhost:8080/ for more info.
        at com.nicta.scoobi.impl.exec.MapReduceJob.report(MapReduceJob.scala:80)
        at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$reportMscr$1.apply(HadoopMode.scala:157)
        at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$reportMscr$1.apply(HadoopMode.scala:154)
        at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:54)
        at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:53)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
        at scala.collection.AbstractTraversable.map(Traversable.scala:105)
        at com.nicta.scoobi.impl.exec.HadoopMode$Execution.runMscrs(HadoopMode.scala:133)
        at com.nicta.scoobi.impl.exec.HadoopMode$Execution.execute(HadoopMode.scala:115)
        at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeLayer$1.apply(HadoopMode.scala:105)
        at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeLayer$1.apply(HadoopMode.scala:104)
        at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:61)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
        at scala.collection.AbstractTraversable.map(Traversable.scala:105)
        at com.nicta.scoobi.impl.exec.HadoopMode.com$nicta$scoobi$impl$exec$HadoopMode$$executeLayers$1(HadoopMode.scala:68)
        at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:91)
        at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:84)
        at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:61)
        at scalaz.syntax.IdOps$class.$bar$greater(IdOps.scala:15)
        at scalaz.syntax.ToIdOps$$anon$1.$bar$greater(IdOps.scala:78)
        at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:52)
        at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:48)
        at com.nicta.scoobi.impl.Persister.persist(Persister.scala:44)
        at com.nicta.scoobi.impl.ScoobiConfigurationImpl.persist(ScoobiConfigurationImpl.scala:355)
        at com.nicta.scoobi.application.Persist$class.persist(Persist.scala:33)
        at p.WordCount$.persist(scoobi-test.scala:6)
        at com.nicta.scoobi.application.Persist$PersistableList.persist(Persist.scala:151)
        at p.WordCount$.run(scoobi-test.scala:14)
        at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply$mcV$sp(ScoobiApp.scala:80)
        at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply(ScoobiApp.scala:75)
        at com.nicta.scoobi.application.ScoobiApp$$anonfun$main$1.apply(ScoobiApp.scala:75)
        at com.nicta.scoobi.application.LocalHadoop$class.runOnLocal(LocalHadoop.scala:41)
        at p.WordCount$.runOnLocal(scoobi-test.scala:6)
        at com.nicta.scoobi.application.LocalHadoop$class.executeOnLocal(LocalHadoop.scala:35)
        at p.WordCount$.executeOnLocal(scoobi-test.scala:6)
        at com.nicta.scoobi.application.LocalHadoop$$anonfun$onLocal$1.apply(LocalHadoop.scala:29)
        at com.nicta.scoobi.application.InMemoryHadoop$class.withTimer(InMemory.scala:71)
        at p.WordCount$.withTimer(scoobi-test.scala:6)
        at com.nicta.scoobi.application.InMemoryHadoop$class.showTime(InMemory.scala:79)
        at p.WordCount$.showTime(scoobi-test.scala:6)
        at com.nicta.scoobi.application.LocalHadoop$class.onLocal(LocalHadoop.scala:29)
        at p.WordCount$.onLocal(scoobi-test.scala:6)
        at com.nicta.scoobi.application.Hadoop$class.onHadoop(Hadoop.scala:60)
        at p.WordCount$.onHadoop(scoobi-test.scala:6)
        at com.nicta.scoobi.application.ScoobiApp$class.main(ScoobiApp.scala:75)
        at p.WordCount$.main(scoobi-test.scala:6)
        at p.WordCount.main(scoobi-test.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
[trace] Stack trace suppressed: run last compile:runMain for the full output.
[INFO] Task - Communication exception: java.lang.InterruptedException
        at java.lang.Object.wait(Native Method)
        at org.apache.hadoop.mapred.Task$TaskReporter.run(Task.java:648)
        at java.lang.Thread.run(Thread.java:679)

java.lang.RuntimeException: Nonzero exit code: 1
        at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:runMain for the full output.
[error] (compile:runMain) Nonzero exit code: 1
[error] Total time: 9 s, completed Nov 24, 2013 3:55:30 AM

从github( https://github.com/NICTA/scoobi/tree/SCOOBI-0.7.3/examples/wordCount)运行非常相似的示例确实可行。

有任何想法吗？

编辑
我根据 scoobi quick start中的说明运行了样本
使用sbt命令运行示例: 
sbt compile
sbt "run-main mypackage.myapp.WordCount input-files output"

没有关于如何或在何处提供参数(例如外部 jar 的位置)的引用。

最佳答案

已安装的scala版本和build.sbt中指定的版本必须相同。一旦相同，scala库将自动包含在类路径中。

关于scala - java.lang.ClassNotFoundException:类scala.runtime.Nothing运行scoobi WordCount示例时，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/20210167/

25 4 0
文章推荐： hadoop - 从其他Blob容器在HDInsight中创建Hive表时出错:路径不合法
文章推荐： hadoop - Hadoop链作业-如果输入文件未退出，则跳过第二个作业
文章推荐： hadoop - 是否可以从Hadoop中的同一reducer写入HBase和多个文件
文章推荐： java - Hadoop获取文件未找到异常
行者123
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com