java - tomcat 中有 100 个线程 TIMED_WAITING，导致它在线程总数超过 200 时停止

Question

最近我们的一个生产 tomcat 服务器变得没有响应，因为 tomcat 的繁忙线程激增到 200。当我们在重新启动之前进行线程转储时，我们有 100 个线程处于 TIMED_WAITING 状态，就像这 3 个线程:

""http-bio-7007"-exec-241" daemon prio=10 tid=0x00002aaab107b000 nid=0x59df waiting on condition [0x0000000051239000]
java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x0000000580d877d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
   at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
   at java.lang.Thread.run(Thread.java:662)

""http-bio-7007"-exec-237" daemon prio=10 tid=0x00002aaab186e000 nid=0x596d waiting on condition [0x000000004d1f9000]
java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x0000000580d877d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
   at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
   at java.lang.Thread.run(Thread.java:662)

""http-bio-7007"-exec-236" daemon prio=10 tid=0x00002aaab1118000 nid=0x596c waiting on condition [0x000000004e50c000]
java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x0000000580d877d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
   at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
   at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
   at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
   at java.lang.Thread.run(Thread.java:662)

我们有 4 个应用程序的线程池(例如 pool-4-thread-20 等)，每个线程池也有 20 个线程，所以我不确定这 100 个线程在哪个阻塞队列上等待？我们正在使用带 hibernate 功能的 c3P0 连接池，这似乎不是造成这种情况的原因。

知道 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject 是什么吗？

Answer 1

当我们修复泄漏由 c3p0 管理的数据库连接的代码时，这个问题就得到了修复。在我们的代码中很少有流程在关闭 finally block 中的实体管理器之前没有在 catch block 中专门调用 rollback() ，因此在异常情况下连接不会返回池并且异常频率很高(超过大小超时间隔内的池)然后所有其他进程线程将堆积起来以获得连接。