java - 错误 : java. lang.IllegalArgumentException:即使使用变通方法，比较方法也违反了其一般约定

Question

我已经花了两天时间来解决这个错误，即使我尝试了几个 stackoverflow 帖子“-Djava.util.Arrays.useLegacyMergeSort=true”中建议的解决方法，但它也不起作用。

这是我的命令及其返回错误的详细信息:

命令:

hadoop jar CloudBrush.jar -Djava.awt.headless=true -Djava.util.Arrays.useLegacyMergeSort=true -reads /Ec10k -asm Ec10k_Brush -k 21 -readlen 36

错误:

    Error: java.lang.IllegalArgumentException: Comparison method violates its general contract!
    at java.util.TimSort.mergeHi(TimSort.java:895)
    at java.util.TimSort.mergeAt(TimSort.java:512)
    at java.util.TimSort.mergeCollapse(TimSort.java:437)
    at java.util.TimSort.sort(TimSort.java:241)
    at java.util.Arrays.sort(Arrays.java:1512)
    at java.util.ArrayList.sort(ArrayList.java:1454)
    at java.util.Collections.sort(Collections.java:175)
    at Brush.VerifyOverlap$VerifyOverlapReducer.reduce(VerifyOverlap.java:252)
    at Brush.VerifyOverlap$VerifyOverlapReducer.reduce(VerifyOverlap.java:1)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Exception in thread "main" java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
    at Brush.VerifyOverlap.run(VerifyOverlap.java:381)
    at Brush.BrushAssembler.buildOverlap(BrushAssembler.java:326)
    at Brush.BrushAssembler.run(BrushAssembler.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at Brush.BrushAssembler.main(BrushAssembler.java:913)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

这是比较器:

    class OvelapSizeComparator implements Comparator {
        public int compare(Object element1, Object element2) {
            OverlapInfo obj1 = (OverlapInfo) element1;
            OverlapInfo obj2 = (OverlapInfo) element2;
            if ((int)(obj1.overlap_size - obj2.overlap_size) >= 0) {
                return -1;
            } else {
                return 1;
            }
        }
    }

Answer 1

真正的问题是您的 OvelapSizeComparator [sic] 有缺陷。如果两个对象的 overlap_size 值相等，则在应返回 0 时返回 -1。如果它们不相等，它会返回一个符号不正确的值。

要修复它，请替换为:

if ((int)(obj1.overlap_size - obj2.overlap_size) >= 0) {
   return -1;
} else {
   return 1;
}

...有了这个:

return obj1.overlap_size - obj2.overlap_size;