mongodb - 在 Hadoop 中提交 MapReduce 作业时找不到 com.mongodb.hadoop.MongoOutputFormat

Question

我遵循本教程 http://www.mongodb.org/display/DOCS/Hadoop+Quick+Start构建 mongodb-hadoop。我尝试构建 Treasury Yield 示例(我的 Hadoop 版本是 0.20.2。)，但是当我提交 MapReduce 作业时出现以下错误:

12/10/24 14:01:09 INFO util.MongoTool: Created a conf: 'Configuration: core-default.xml, core-site.xml, mongo-defaults.xml, mongo-treasury_yield.xml' on {class com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig} as job named '<unnamed job="" mongotool="">'
12/10/24 14:01:10 INFO util.MongoTool: Mapper Class: class com.mongodb.hadoop.examples.treasury.TreasuryYieldMapper
12/10/24 14:01:10 INFO util.MongoTool: Setting up and running MapReduce job in foreground, will wait for results. {Verbose? true}
12/10/24 14:01:10 INFO util.MongoSplitter: Calculate Splits Code ... Use Shards? false, Use Chunks? true; Collection Sharded? true
12/10/24 14:01:10 INFO util.MongoSplitter: Creation of Input Splits is enabled.
12/10/24 14:01:10 INFO util.MongoSplitter: Sharding mode calculation entering.
12/10/24 14:01:10 INFO mapred.JobClient: Running job: job_201208171702_0054
12/10/24 14:01:11 INFO mapred.JobClient: map 0% reduce 0%
12/10/24 14:01:17 INFO mapred.JobClient: Task Id : attempt_201208171702_0054_m_000116_0, Status : FAILED

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mongodb.hadoop.MongoOutputFormat 
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809) 
    at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:193) 
    at org.apache.hadoop.mapred.Task.initialize(Task.java:413) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288) 
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.MongoOutputFormat 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:264) 
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762) 
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807) 
... 4 more

我无法让国债 yield 示例运行...任何帮助将不胜感激。

Answer 1

我遇到了同样的问题。请确保您已将所需的 jar 文件(mongo-hadoop-core_cdh3u3-1.0.0.jar、mongo-2.10.1.jar)复制到 hadoop lib 文件夹中。之后，您将需要重新启动 hadoop 节点以将其放入其类路径中。然后它应该运行。

它对我有用。