java - 解决 Windows Server 上的高 CPU 使用率问题

Question

我们有基于 J2EE 的 Web 应用程序。我们的生产环境间歇性地面临高 CPU 使用率 (80-90%)。我们无法在我们的 QA 环境中复制它。

生产环境:Windows 2012 Server(64位)、JDK 1.8(64位)

为了进行故障排除，我们采取了线程转储。它显示总共 215 个线程。

111 threads are in WAITING status
34 threads are in RUNNABLE status
67 threads are in TIMED_WAITING status
3 threads are in BLOCKED status

我们如何找到哪些线程导致 CPU 使用率过高？

线程转储

2016-03-01 11:07:52 Full thread dump Java HotSpot(TM) 64-Bit Server VM(25.5-b02 mixed mode):

"Thread-739969" - Thread t@807668 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <229fc226> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"Thread-739968" - Thread t@807667 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <5a578345> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"Thread-739963" - Thread t@807662 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <20851ef7> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"Thread-739962" - Thread t@807661 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <35061b89> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"Thread-739960" - Thread t@807659 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <3491a931> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"Thread-739959" - Thread t@807658 java.lang.Thread.State: WAITINGat java.lang.Object.wait(Native Method)

• waiting on <6395c27d> (a org.apache.solr.util.ConcurrentLRUCache$CleanupThread) atjava.lang.Object.wait(Object.java:502) atorg.apache.solr.util.ConcurrentLRUCache$CleanupThread.run(ConcurrentLRUCache.java:604)

Locked ownable synchronizers:

• None

"httpShardExecutor-3-thread-57429-processing-{core=actionscomments,replica=core_node1,ShardRequest.shards=s1.example.com:8983//solr//actionscomments|s1r1.example.com:8983//solr//actionscomments,node_name=10.120.112.31:8983_solr, shard=shard1,collection=actionscomments,ShardRequest.urlList=[http:////s1r1.example.com:8983//solr//actionscomments,http:////s1.example.com:8983//solr//actionscomments]}" - Thread t@807648java.lang.Thread.State: RUNNABLE atjava.net.SocketInputStream.socketRead0(Native Method) atjava.net.SocketInputStream.read(SocketInputStream.java:150) atjava.net.SocketInputStream.read(SocketInputStream.java:121) atorg.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)atorg.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)atorg.apache.http.impl.io.SocketInputBuffer.isDataAvailable(SocketInputBuffer.java:95)atorg.apache.http.impl.AbstractHttpClientConnection.isStale(AbstractHttpClientConnection.java:310)atorg.apache.http.impl.conn.ManagedClientConnectionImpl.isStale(ManagedClientConnectionImpl.java:158)atorg.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:433)atorg.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)atorg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)atorg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)atorg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)atorg.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)atorg.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)atorg.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)atorg.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)atorg.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)atorg.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:246)atorg.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:221)atorg.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183)at java.util.concurrent.FutureTask.run(FutureTask.java:266) atjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266) atorg.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:

• locked (a java.util.concurrent.ThreadPoolExecutor$Worker)

Answer 1

使用visualvm识别消耗 CPU 的线程。可以在 DZone 上找到一篇关于解释线程转储的好文章:how-analyze-java-thread-dumps .

这应该允许您识别线程，也许您已经可以直接识别问题本身。