java - IntStream导致将数组元素错误地设置为0(JVM Bug，Java 11)

Question

在下面的P类中，test方法似乎返回相同的false:

import java.util.function.IntPredicate;
import java.util.stream.IntStream;

public class P implements IntPredicate {
    private final static int SIZE = 33;

    @Override
    public boolean test(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        for (int i = 1; i < SIZE; i++) {
            state[i] = state[i - 1];
        }
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(new P()).count();
        System.out.println(count);
    }
}

但是，将 P类与 IntStream组合在一起，方法 test可以(错误地)返回 true。
上面的 main方法中的代码产生一些正整数，例如 716208。
每次执行后结果都会更改。
之所以会出现这种 意外行为，是因为在执行过程中 int数组 state[]可以设置为零。
如果是测试代码，例如

if (seed == 0xf_fff0){
    System.out.println(Arrays.toString(state));
} 

将插入到 test方法的尾部，然后程序将输出类似于 [1048560, 1048560, 1048560, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]的行。
问题:为什么将int数组 state[]设置为零？
我已经知道如何避免这种行为:仅将 int[]替换为 ArrayList。
我检查了:

带有OpenJDK Runtime Environment(内部版本15.0.1 + 9-18)的

Windows 10 +和debian 10+)OpenJDK 64位服务器VM(内部版本15.0.1 + 9-18，混合模式，共享)

debian 9 + OpenJDK运行时环境AdoptOpenJDK(内部版本13.0.1 + 9)OpenJDK 64位服务器VM AdoptOpenJDK(内部版本13.0.1 + 9，混合模式，共享)

Answer 1

可以用一个更简单的示例来重现该问题，即:

class Main {
    private final static int SIZE = 33;

    public static boolean test2(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        for (int i = 1; i < SIZE; i++) {
            state[i] = state[i - 1];
        }
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(Main::test2).count();
        System.out.println(count);

    }
}

该问题是由 JVM优化标志引起的，该标志允许对循环(即 -XX:+AllowVectorizeOnDemand)进行向量化(SIMD)。可能是由于在具有相交范围(即 state[i] = state[i - 1];)的同一数组上应用矢量化而引起的。如果 JVM将(对于 IntStream.range(0, 0x0010_0000)的某些元素)优化循环，则可能会重现类似的问题:

   for (int i = 1; i < SIZE; i++)
       state[i] = state[i - 1];

变成:

    System.arraycopy(state, 0, state, 1, SIZE - 1);

例如:

class Main {
    private final static int SIZE = 33;

    public static boolean test2(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        System.arraycopy(state, 0, state, 1, SIZE - 1);
        if(seed == 100)
           System.out.println(Arrays.toString(state));
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(Main::test2).count();
        System.out.println(count);
    }
}

输出:

[100, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

新更新:01/01/2021
我已将电子邮件发送给涉及该标志 -XX:+AllowVectorizeOnDemandand的实现/集成的开发人员之一，收到以下答复:

It is known that part of AllowVectorizeOnDemand code is broken.

There was fix (it excluded executing broken code which does incorrectvectorization) which was backported into jdk 11.0.11:

https://hg.openjdk.java.net/jdk-updates/jdk11u-dev/rev/69dbdd271e04

If you can, try build and test latest OpenJDK11u fromhttps://hg.openjdk.java.net/jdk-updates/jdk11u-dev/

从第一个链接，可以阅读以下内容:

@bug 8251994@summary Test vectorization of Streams$RangeIntSpliterator::forEachRemaining@requires vm.compiler2.enabled & vm.compMode != "Xint"

@run main compiler.vectorization.TestForEachRem test1@run main compiler.vectorization.TestForEachRem test2@run main compiler.vectorization.TestForEachRem test3@run main compiler.vectorization.TestForEachRem test4

从关于该错误的JIRA story的注释中，可以看到:

I found the cause of the issue. To improve a chance to vectorize aloop, superword tries to hoist loads to the beginning of loop byreplacing their memory input with corresponding (same memory slice)loop's memory Phi :http://hg.openjdk.java.net/jdk/jdk/file/8f73aeccb27c/src/hotspot/share/opto/superword.cpp#l471

Originally loads are ordered by corresponding stores on the samememory slice. But when they are hoisted they loose that ordering -nothing enforce the order. In test6 case the ordering is preserved(luckily?) after hoisting only when vector size is 32 bytes (avx2) butthey become unordered with 16 (avx=0 or avx1) or 64 (avx512) bytesvectors.(...)

I have simple fix (use original loads ordering indexes) but looking onthe code which causing the issue I see that it is bogus/incomplete -it does not help cases listed for JDK-8076284 changes:

https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-April/017645.html

Using unrolling and cloning information to vectorize is interestingidea but as I see it is not complete. Even if pack_parallel() methodis able created packs they are all removed by filter_packs() method.And additionally the above cases are vectorized without hoisting loadsand pack_parallel - I verified it. That code is useless now and Iwill put it under flag to not run it. It needs more work to be useful.I reluctant to remove the code because may be in a future we will havetime to invest into it.

这可能可以解释为什么当我比较带有和不带有标记 -XX:+AllowVectorizeOnDemand的版本的程序集时，我注意到带有标记的版本包含以下代码:

   for (int i = 1; i < SIZE; i++)
       state[i] = state[i - 1];

(我提取了一种名为 hotstop的方法以方便在程序集中查找它)，具有:

00000001162bacf5: mov    %r8d,0x10(%rsi,%r10,4)
0x00000001162bacfa: mov    %r8d,0x14(%rsi,%r10,4)
0x00000001162bacff: mov    %r8d,0x18(%rsi,%r10,4)
0x00000001162bad04: mov    %r8d,0x1c(%rsi,%r10,4)
0x00000001162bad09: mov    %r8d,0x20(%rsi,%r10,4)
0x00000001162bad0e: mov    %r8d,0x24(%rsi,%r10,4)
0x00000001162bad13: mov    %r8d,0x28(%rsi,%r10,4)
0x00000001162bad18: mov    %r8d,0x2c(%rsi,%r10,4)  ;*iastore {reexecute=0 rethrow=0 return_oop=0}
                                             ; - AAAAAA.Main::hotstop@15 (line 21)

在我看来，这类似于循环 unrolling，从另一方面来看， java.util.stream.Streams$RangeIntSpliterator::forEachRemaining方法仅在带有标志的版本的汇编中显示。