hadoop - slf4j 在 Amazon EMR 上与 Mahout 进行多重绑定(bind)

Question

我在 Amazon EMR 上运行 mahout 作业并收到以下异常:

ArrayUtil.oversize(II)I
attempt_201311181700_0002_m_000000_0: SLF4J: Class path contains multiple SLF4J bindings.
attempt_201311181700_0002_m_000000_0: SLF4J: Found binding in [jar:file:/home/hadoop/lib/slf4j-log4j12-1.7.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201311181700_0002_m_000000_0: SLF4J: Found binding in [jar:file:/mnt/var/lib/hadoop/mapred/taskTracker/hadoop/jobcache/job_201311181700_0002/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201311181700_0002_m_000000_0: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
attempt_201311181700_0002_m_000000_0: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: org.apache.lucene.util.ArrayUtil.oversize(II)I
attempt_201311181700_0002_m_000000_1: SLF4J: Class path contains multiple SLF4J bindings.
attempt_201311181700_0002_m_000000_1: SLF4J: Found binding in [jar:file:/home/hadoop/lib/slf4j-log4j12-1.7.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201311181700_0002_m_000000_1: SLF4J: Found binding in [jar:file:/mnt/var/lib/hadoop/mapred/taskTracker/hadoop/jobcache/job_201311181700_0002/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201311181700_0002_m_000000_1: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
attempt_201311181700_0002_m_000000_1: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: org.apache.lucene.util.ArrayUtil.oversize(II)I
Exception in thread "main" java.lang.IllegalStateException: Job failed!
    at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateCollocations(CollocDriver.java:238)
    at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateAllGrams(CollocDriver.java:187)
    at org.apache.mahout.vectorizer.DictionaryVectorizer.createTermFrequencyVectors(DictionaryVectorizer.java:184)
    at clustering.AmazonClusteringDriver.main(AmazonClusteringDriver.java:122)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:187)

我排除了 mahout 依赖中的 slf4j 依赖；但是，它并不能解决问题。那么，问题出在哪里？

Answer 1

这是一个非常错误的地方问这个。

你应该在 Mahout 开发者邮件列表上问这个问题。

https://cwiki.apache.org/confluence/display/MAHOUT/Mailing+Lists,+IRC+and+Archives#MailingLists%2CIRCandArchives-MahoutUserList