java - 关闭 tomcat 服务器时无法加载 org.apache.hadoop.util.ShutdownHookManager

Question

我有一个在 tomcat 上运行的简单 Web 应用程序。此 Web 应用程序将文件读​​写到 HDFS。

我面临的问题是每次使用 ./bin/shutdown.sh 停止服务器时。我收到无法加载 hadoop shutdownHookManager 异常。

我确定 hadoop-common(包含 ShutDownManager)在 tomcat 类路径中。

谁能帮帮我？

我得到的异常:

Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [com.ebay.kernel.calwrapper.CalTransactionHelper$1] (value [com.ebay.kernel.calwrapper.CalTransactionHelper$1@3d5a1c5f]) and a value of type [com.ebay.kernel.calwrapper.CalTransactionHelper.Stack] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop 
INFO: Stopping ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load org.apache.hadoop.util.ShutdownHookManager$2.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Exception in thread "Thread-9" java.lang.NoClassDefFoundError: org/apache/hadoop/util/ShutdownHookManager$2
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.ShutdownHookManager$2
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1714)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  ... 2 more

Answer 1

我在构建的 Maven Mojo 中遇到了同样的问题。我认为问题是由在您的 Web 应用程序中加载 hadoop 类的上下文类加载器引起的。 WebAppClassLoader 类加载器看不到这些定义。当发生 JVM 关闭时，正在执行关闭 Hook 并调用 ShutdownHookManager 但它已加载到现在已销毁的上下文类加载器中，这就是问题所在。

将 hadoop jar 放在 tomcat 的 lib 目录中可能会有所帮助。或者在启动 tomcat 时设置一些 *_CLASSPATH java 选项。这样 WebAppClassLoader 就会有你的类的定义。

否则，您可以在销毁上下文之前手动执行 hadoop 的关闭 Hook 并忽略上述异常。