hadoop - 使用 Apache Giraph 时出错

Question

我在 Apache Giraph 中运行 SimpleOutDegreeCountComputation 程序时收到以下错误。我使用 JsonLongDoubleFloatDouble 作为输入格式。

$HADOOP_HOME/bin/hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleOutDegreeCountComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/Simpleout -w 1 Warning: $HADOOP_HOME is deprecated.

15/05/18 12:56:04 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 15/05/18 12:56:04 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one. Exception in thread "main" java.lang.IllegalStateException: checkClassTypes: vertex value types not assignable, computation - class org.apache.hadoop.io.LongWritable, VertexInputFormat - class org.apache.hadoop.io.DoubleWritable at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381) at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228) at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:222) at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:160) ^[[Ahduser@deepa-Inspiron-N5050:/usr/local/hadoop/bin$ $HADOOP_HOME/bin/hadoop jGIRAPH_HOME/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleOutDegreeCountComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/Simpleout -w 1

Answer 1

正如日志所说你的计算，例如SimpleOutDegreeCountComputation 需要顶点 ID 的 LongWritable 格式，而 hdfs 中的数据是 DoubleWritable。因此，有两个选择:更改 SimpleOutDegreeCountComputation 以使用 DoubleWritable 或将数据从 DoubleWritable 转换为 LongWritable。