java - 解释核心转储文件堆栈

Question

我的应用程序导致 JVM 随机崩溃。

我已经收集了核心转储并进行了查看，但我需要一些帮助，因为我对核心转储的经验很少。

在 15 个或更多线程中，这 2 个引起了我的注意:

线程 1(线程 0xbb8f5b70 (LWP 27584)):

Thread 1 (Thread 0xbb8f5b70 (LWP 27584)):
#0 0x00a6b430 in __kernel_vsyscall ()
#1 0x0020bb11 in raise () from /lib/libc.so.6
#2 0x0020d3ea in abort () from /lib/libc.so.6
#3 0x0024b9d5 in __libc_message () from /lib/libc.so.6
#4 0x00251e31 in malloc_printerr () from /lib/libc.so.6  <---- THIS LINE
#5 0x00254823 in _int_free () from /lib/libc.so.6 
#6 0x0078a068 in _dlerror_run () from /lib/libdl.so.2
#7 0x00789d7c in dlsym () from /lib/libdl.so.2
#8 0x01397fa0 in os::dll_lookup(void*, char const*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#9 0x0136aa07 in NativeLookup::lookup_style(methodHandle, char*, char const*, int, bool, bool&, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#10 0x0136abf8 in NativeLookup::lookup_entry(methodHandle, bool&, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#11 0x0136b107 in NativeLookup::lookup_base(methodHandle, bool&, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#12 0x0136b1f7 in NativeLookup::lookup(methodHandle, bool&, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#13 0x0119a32b in InterpreterRuntime::prepare_native_call(JavaThread*, methodOopDesc*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#14 0xf477e12b in ?? ()
#15 0xf4776387 in ?? ()
#16 0xf4776387 in ?? ()
#17 0xf4776387 in ?? ()
#18 0xf4776387 in ?? ()
#19 0xf4776387 in ?? ()
#20 0xf4776387 in ?? ()
#21 0xf4776387 in ?? ()
#22 0xf4776387 in ?? ()
#23 0xf4776387 in ?? ()
#24 0xf47765fb in ?? ()
#25 0xf4773459 in ?? ()
#26 0x011a0915 in JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#27 0x01396769 in os::os_exception_wrapper(void (*)(JavaValue*, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#28 0x0119f58f in JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#29 0x0119f7ca in JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#30 0x0119f8eb in JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#31 0x0121b4c9 in thread_entry(JavaThread*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#32 0x014b8d49 in JavaThread::thread_main_inner() () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#33 0x014b8ea3 in JavaThread::run() () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#34 0x0139d999 in java_start(Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#35 0x0037ea49 in start_thread () from /lib/libpthread.so.0
#36 0x002c3aee in clone () from /lib/libc.so.6

线程 11(线程 0xf77736c0 (LWP 27570)):

Thread 11 (Thread 0xf77736c0 (LWP 27570)):
#0 0x00a6b430 in __kernel_vsyscall ()
#1 0x00385019 in __lll_lock_wait () from /lib/libpthread.so.0
#2 0x0038043e in _L_lock_731 () from /lib/libpthread.so.0
#3 0x0038034a in pthread_mutex_lock () from /lib/libpthread.so.0
#4 0x007b9b2d in _dl_open () from /lib/ld-linux.so.2
#5 0x00789c3b in dlopen_doit () from /lib/libdl.so.2
#6 0x007b5ba6 in _dl_catch_error () from /lib/ld-linux.so.2
#7 0x0078a03c in _dlerror_run () from /lib/libdl.so.2
#8 0x00789b71 in dlopen@@GLIBC_2.1 () from /lib/libdl.so.2
#9 0x0139e13e in os::dll_load(char const*, char*, int) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#10 0x0136a17f in NativeLookup::lookup_critical_style(methodHandle, char*, char const*, int, bool) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#11 0x0136a72d in NativeLookup::lookup_critical_entry(methodHandle) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#12 0x0143719c in SharedRuntime::generate_native_wrapper(MacroAssembler*, methodHandle, int, BasicType*, VMRegPair*, BasicType) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#13 0x0142476f in AdapterHandlerLibrary::create_native_wrapper(methodHandle, int) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#14 0x0101c5a9 in CompileBroker::compile_method(methodHandle, int, int, methodHandle, int, char const*, Thread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#15 0x0100aa18 in SimpleCompPolicy::method_invocation_event(methodHandle, JavaThread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#16 0x01009d5b in NonTieredCompPolicy::event(methodHandle, methodHandle, int, int, CompLevel, nmethod*, JavaThread*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#17 0x011983dd in InterpreterRuntime::frequency_counter_overflow_inner(JavaThread*, unsigned char*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#18 0x0119a982 in InterpreterRuntime::frequency_counter_overflow(JavaThread*, unsigned char*) () from /usr/local/thirdparty/java/j2sdk/jre/lib/i386/server/libjvm.so
#19 0xf477e525 in ?? ()
#20 0xf47ccbec in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?) <---- AND THIS LINE

有人可以帮我理解这里的关键部分吗？

此核心转储是近 3 个月前生成的，重现步骤有点不一致。

额外问题:

当堆栈条目显示 0x00789b71 in dlopen@@GLIBC_2.1 () from/lib/libdl.so.2

@@ 是什么意思？有人可以解释一下吗？ Google 没有给我任何东西 - 我认为它假定 @ 为特殊字符并完全忽略它们。

Answer 1

Among 15 or more threads these 2 caught my attention:

线程1是崩溃的根本原因；我认为线程 11 没有任何问题。

线程 1 崩溃，因为 glibc 检测到堆损坏，并在将其报告给/dev/tty 后调用 abort() ...

不幸的是，堆损坏的本质是它通常会在完全不相关的代码中导致崩溃。标准建议是在 Valgrind 下运行程序，但该建议通常不适用于 Java 程序。

What does the @@ in dlopen@@GLIBC_2.1 mean?

这意味着正在调用一个函数dlopen，该函数在libdl中有多个版本，默认版本是使用GLIBC_2版本制作的.1 正在被调用。

您可以阅读有关 GNU 符号版本控制的更多信息 here .