java - Tomcat 因 JDK 中的 native 代码为 malloc_consolidate 而崩溃

Question

当 JavaThread“Finalizer”在 JVM 中运行时，生产环境中的 tomcat 服务器经常崩溃，hs_err_pid 在下方。

此外，错误来自 libc.so 中 malloc_consolidate 的 Java native 代码。

# Problematic frame:
# C  [libc.so.6+0x75f85]  malloc_consolidate+0xf5
#
---------------  T H R E A D  ---------------

Current thread (0x00007f2c6018f000):  JavaThread "Finalizer" daemon [_thread_in_native, id=11112, stack(0x00007f2c3f1f2000,0x00007f2c3f2f3000)]


Stack: [0x00007f2c3f1f2000,0x00007f2c3f2f3000],  sp=0x00007f2c3f2f13e0,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x75f85]  malloc_consolidate+0xf5

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 2177  java.util.zip.Inflater.end(J)V (0 bytes) @ 0x00007f2c50e3d301 [0x00007f2c50e3d2c0+0x41]
J 3502 C2 java.util.zip.ZipFile.close()V (223 bytes) @ 0x00007f2c514371c4 [0x00007f2c51436900+0x8c4]
J 7022 C2 java.util.zip.ZipFile.finalize()V (5 bytes) @ 0x00007f2c511b43e0 [0x00007f2c511b43a0+0x40]
J 11603% C2 java.lang.ref.Finalizer$FinalizerThread.run()V (55 bytes) @ 0x00007f2c528d6f78 [0x00007f2c528d6cc0+0x2b8]
v  ~StubRoutines::call_stub

Heap:
 PSYoungGen      total 2318848K, used 222178K [0x0000000720000000, 0x00000007c0000000, 0x00000007c0000000)
  eden space 2016256K, 5% used [0x0000000720000000,0x0000000726d4fbe0,0x000000079b100000)
  from space 302592K, 36% used [0x000000079b100000,0x00000007a1ca8fe0,0x00000007ad880000)
  to   space 289792K, 0% used [0x00000007ae500000,0x00000007ae500000,0x00000007c0000000)
 ParOldGen       total 2586624K, used 522596K [0x00000005e0000000, 0x000000067de00000, 0x0000000720000000)
  object space 2586624K, 20% used [0x00000005e0000000,0x00000005ffe59390,0x000000067de00000)
 Metaspace       used 148258K, capacity 154058K, committed 154200K, reserved 1183744K
  class space    used 18157K, capacity 19308K, committed 19328K, reserved 1048576K

Internal exceptions (10 events):
Event: 61.804 Thread 0x00007f2c2c0c6800 Exception <a 'java/io/IOException'> (0x0000000798129018) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 61.816 Thread 0x00007f2c2c0c6800 Exception <a 'java/io/IOException'> (0x0000000798304680) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 61.816 Thread 0x00007f2c2c0c6800 Exception <a 'java/io/IOException'> (0x0000000798305b90) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 62.015 Thread 0x00007f2c2c0bb800 Exception <a 'java/io/IOException'> (0x0000000720db4188) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 62.015 Thread 0x00007f2c2c0bb800 Exception <a 'java/io/IOException'> (0x0000000720db47e0) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 62.046 Thread 0x00007f2c2c0bb800 Exception <a 'java/io/IOException'> (0x0000000720dbc2f0) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 62.046 Thread 0x00007f2c2c0bb800 Exception <a 'java/io/IOException'> (0x0000000720dbd800) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 62.082 Thread 0x00007f2c606f8800 Exception <a 'java/lang/InternalError'> (0x0000000721cfe9b8) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 735]
Event: 62.082 Thread 0x00007f2c606f8800 Exception <a 'java/lang/InternalError'> (0x0000000721cfeda8) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jni.cpp, line 735]
Event: 62.082 Thread 0x00007f2c606f8800 Exception <a 'java/lang/InternalError'> (0x0000000721cfeda8) thrown at [/HUDSON3/workspace/8-2-build-linux-amd64/jdk8u121/8372/hotspot/src/share/vm/prims/jvm.cpp, line 1394]

VM Arguments:
jvm_args: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heap -Djava.util.Arrays.useLegacyMergeSort=true -Xms3072M -Xmx7680M -Dsun.zip.disableMemoryMapping=true 

Answer 1

-Dsun.zip.disableMemoryMapping=true 应该解决这个问题，zip 实现在 JDK 9 中得到改进，崩溃表明，在终结器尝试关闭文件句柄期间，它仍然没有被释放。升级 JDK9 应该可以避免这个问题 - https://jdk9.java.net/download/