个人中心
文章发布
退出
作者热门文章
- Java 双重比较
- java - 比较器与 Apache BeanComparator
- Objective-C 完成 block 导致额外的方法调用？
- database - RESTful URI 是否应该公开数据库主键？
27
4
我的网络应用程序在晚上运行作业!遇到问题!它占用了大量内存!
我用命令查找哪个函数占用了java资源!
结果是:
[tomcat@uhzd006525 ~]$ jstack 2365 |grep 93f -A 30
- parking to wait for <0x00000007eac93f68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:131)
at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:281)
at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:62)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:176)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:172)
at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:100)
at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:212)
at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:199)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:456)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at com.trendata.spider.PageGetter.getPageContent(PageGetter.java:262)
at com.trendata.spider.PageGetter.getTaobaoContent(PageGetter.java:376)
at com.trendata.taobao.MbpBkDataCreator.getBkMbpValue(MbpBkDataCreator.java:48)
at com.trendata.taobao.MbpBkDataCreator.getIntoStores(MbpBkDataCreator.java:106)
at com.trendata.service.impl.OddJobsServiceImpl.getLast7DaysIntoStore(OddJobsServiceImpl.java:448)
at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.trendata.service.interceptor.MethodCacheInterceptor.invoke(MethodCacheInterceptor.java:32)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy144.getLast7DaysIntoStore(Unknown Source)
at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
--
"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000000d46000 nid=0x93f runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000000d48000 nid=0x940 runnable
"VM Periodic Task Thread" prio=10 tid=0x00002aaabc059800 nid=0x94b waiting on condition
JNI global references: 342
"Thread-114" prio=10 tid=0x000000002281e800 nid=0xf4a waiting on condition [0x000000004c7d3000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000790f5dc40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
"Thread-113" prio=10 tid=0x0000000022624800 nid=0xf48 waiting on condition [0x000000004c6d2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000790f5dc40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
"Thread-112" prio=10 tid=0x00000000225d9800 nid=0xf37 waiting on condition [0x000000004c5d1000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000790f5dc40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
--
"GC task thread#0 (ParallelGC)" prio=10 tid=0x000000001f287000 nid=0x7d30 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x000000001f288800 nid=0x7d31 runnable
"VM Periodic Task Thread" prio=10 tid=0x000000001f3d1000 nid=0x7d39 waiting on condition
JNI global references: 579
这是发生问题的代码:
public class PageGetter {
private static final Log log = LogFactory.getLog("serviceLogger");
private static SyncBasicHttpParams httpParams = null;
private HttpClient httpClient = null;
private static PoolingClientConnectionManager connectionManager = null;
private static Integer errorSleepTime = new Integer(PropertyGetter.getInstance().getProperty("errorpagesleeptime"));
public String getFinalRedirectURL(String url) throws ClientProtocolException, IOException {
if (url == null) {
return null;
}
if (!url.trim().startsWith("http://")) {
url = "http://" + url;
}
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet(url);
if (connectionManager == null || httpParams == null || this.httpClient == null) {
this.initHttpManager();
}
this.httpClient.execute(httpget, localContext);
HttpHost target = (HttpHost) localContext.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
String finalURL = target.toString();
httpget.abort();
return finalURL;
}
private void initHttpManager() {
// Create and initialize HTTP parameters
if (httpParams == null) {
httpParams = new SyncBasicHttpParams();
httpParams.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 60000);
httpParams.setParameter(HTTP.CONTENT_ENCODING, "GBK");
HttpProtocolParams.setVersion(httpParams, HttpVersion.HTTP_1_1);
}
if (connectionManager == null) {
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
try {
//Security Socket
SSLContext ctx = SSLContext.getInstance("TLS");
X509TrustManager tm = new X509TrustManager() {
@Override
public X509Certificate[] getAcceptedIssuers() {
return null;
}
@Override
public void checkClientTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {
}
@Override
public void checkServerTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {
}
};
ctx.init(null, new TrustManager[] { tm }, null);
SSLSocketFactory ssf = new SSLSocketFactory(ctx, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
schemeRegistry.register(new Scheme("https", 443, ssf));
} catch (Exception e) {
log.error("Fail to create connection manager " + e);
}
connectionManager = new PoolingClientConnectionManager(schemeRegistry);
}
this.httpClient = new DefaultHttpClient(connectionManager, httpParams);
this.httpClient.getParams().setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true);
HttpClientParams.setCookiePolicy(this.httpClient.getParams(), CookiePolicy.BROWSER_COMPATIBILITY);
}
public String getPostPageContent(String url, List<NameValuePair> headers, List<NameValuePair> params) {
String strPage = null;
if (url == null) {
return null;
}
if (!url.trim().startsWith("http")) {
url = "http://" + url;
}
HttpPost httpPost = new HttpPost(url);
HttpEntity entity = null;
try {
if (params != null) {
httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
}
if (headers != null) {
Iterator<NameValuePair> itrHeaders = headers.iterator();
while (itrHeaders.hasNext()) {
NameValuePair header = itrHeaders.next();
httpPost.setHeader(header.getName(), header.getValue());
}
}
HttpContext localContext = new BasicHttpContext();
if (connectionManager == null || httpParams == null || this.httpClient == null) {
this.initHttpManager();
}
HttpResponse response = this.httpClient.execute(httpPost, localContext);
// get the response body as an array of bytes
entity = response.getEntity();
if (entity != null) {
strPage = EntityUtils.toString(entity);
}
} catch (Exception e) {
connectionManager = null;
httpParams = null;
this.httpClient = null;
this.errorHandler(url, e);
} finally {
httpPost.abort();
if (entity != null) {
try {
EntityUtils.consume(entity);
} catch (Exception e2) {
e2.printStackTrace();
log.error("Error when consume entity ", e2);
}
}
}
if (strPage != null) {
Pattern pattern = Pattern.compile("\\s");
Matcher matcher = pattern.matcher(strPage);
strPage = matcher.replaceAll(" ");
strPage = strPage.replaceAll("////", "");
}
return strPage;
}
private String getURL(String url, List<NameValuePair> headers) {
if (headers == null) {
return url;
}
Map<String, String> map = new HashMap<String, String>();
if (url.contains("?")) {
String str = url.substring(url.indexOf("?") + 1);
url = url.substring(0, url.indexOf("?"));
String[] strs = str.split("&");
for (int i = 0; strs != null && i < strs.length; i++) {
if (strs[i].contains("=")) {
String key = strs[i].substring(0, strs[i].indexOf("="));
String value = strs[i].substring(strs[i].indexOf("=") + 1);
map.put(key, value);
}
}
}
Iterator<NameValuePair> itrHeaders = headers.iterator();
while (itrHeaders.hasNext()) {
NameValuePair header = itrHeaders.next();
if (header != null && header.getName() != null && header.getValue() != null) {
try {
map.put(URLEncoder.encode(header.getName(), "UTF-8"), URLEncoder.encode(header.getValue(), "UTF-8"));
} catch (Exception e) {
log.error("Can not parse parameter " + e);
}
}
}
if (map.keySet() == null) {
return url;
}
Iterator<String> itr = map.keySet().iterator();
String parameters = "";
while (itr.hasNext()) {
String key = itr.next();
if (key != null && key.length() > 0) {
parameters = parameters + key + "=" + map.get(key) + "&";
}
}
if (parameters.length() > 0) {
url = url + "?" + parameters;
}
return url;
}
private void errorHandler(String url, Exception e) {
try {
Thread.sleep(errorSleepTime);
log.error("无法打开链接" + url + " sleep " + errorSleepTime + e);
} catch (Exception e1) {
log.error("Error sleeping " + errorSleepTime + e1);
}
}
/**
* Get page as string from URL
*
* @param url
* @return
*/
public String getPageContent(String url, List<NameValuePair> headers) {
url = StringFilter.replaceUnicode(url);
String strPage = null;
if (url == null) {
return null;
}
if (!url.trim().startsWith("http")) {
url = "http://" + url;
}
if (url.endsWith("/")) {
url = url.substring(0, url.lastIndexOf("/"));
}
HttpGet httpGet = null;
HttpEntity entity = null;
try {
if (connectionManager == null || httpParams == null || this.httpClient == null) {
this.initHttpManager();
}
url = this.getURL(url, headers);
httpGet = new HttpGet(url);
HttpContext localContext = new BasicHttpContext();
HttpResponse response = this.httpClient.execute(httpGet, localContext);
entity = response.getEntity();
if (entity != null) {
String charSet = ContentType.getOrDefault(entity).getCharset().toString();
if (charSet == null) {
charSet = "UTF-8";
}
strPage = EntityUtils.toString(entity, charSet);
strPage = strPage.replaceAll("\r", "");
strPage = strPage.replaceAll("\n", "");
//空格
strPage = strPage.replaceAll(" ", "");
}
} catch (Exception e) {
connectionManager = null;
httpParams = null;
this.httpClient = null;
this.errorHandler(url, e);
} finally {
try {
httpGet.abort();
} catch (Exception e) {
log.error("Error when httpclient aborting " + e);
}
if (entity != null) {
try {
EntityUtils.consume(entity);
} catch (Exception e2) {
e2.printStackTrace();
log.error("Error when consume entity ", e2);
}
}
}
return strPage;
}
public String getPageContentSms(String url, List<NameValuePair> headers) {
url = StringFilter.replaceUnicode(url);
String strPage = null;
if (url == null) {
return null;
}
if (!url.trim().startsWith("http")) {
url = "http://" + url;
}
if (url.endsWith("/")) {
url = url.substring(0, url.lastIndexOf("/"));
}
HttpGet httpGet = null;
HttpEntity entity = null;
try {
if (connectionManager == null || httpParams == null || this.httpClient == null) {
this.initHttpManager();
}
httpGet = new HttpGet(url);
HttpContext localContext = new BasicHttpContext();
HttpResponse response = this.httpClient.execute(httpGet, localContext);
entity = response.getEntity();
if (entity != null) {
String charSet = ContentType.getOrDefault(entity).getCharset().toString();
if (charSet == null) {
charSet = "UTF-8";
}
strPage = EntityUtils.toString(entity, charSet);
strPage = strPage.replaceAll("\r", "");
strPage = strPage.replaceAll("\n", "");
strPage = strPage.replaceAll(" ", "");
}
} catch (Exception e) {
connectionManager = null;
httpParams = null;
this.httpClient = null;
this.errorHandler(url, e);
} finally {
try {
httpGet.abort();
} catch (Exception e) {
log.error("Error when httpclient aborting " + e);
}
if (entity != null) {
try {
EntityUtils.consume(entity);
} catch (Exception e2) {
e2.printStackTrace();
log.error("Error when consume entity ", e2);
}
}
}
return strPage;
}
public List<NameValuePair> getDefaultHeaders() {
List<NameValuePair> headers = new ArrayList<NameValuePair>();
BasicNameValuePair nameValuePair = new BasicNameValuePair("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("Accept-Charset", "GBK,utf-8;q=0.7,*;q=0.3");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("Accept-Encoding", "GBK,utf-8;q=0.7,*;q=0.3");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("Accept-Language", "zh-CN,zh;q=0.8");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("Cache-Control", "max-age=0");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("Connection", "keep-alive");
headers.add(nameValuePair);
nameValuePair = new BasicNameValuePair("User-Agent",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.162 Safari/535.19");
headers.add(nameValuePair);
return headers;
}
public String getTaobaoContent(List<NameValuePair> params) {
return this.getPageContent(Constant.TAOBAO_API_URL, params);
}
最佳答案
我们在将 HttpClient 与 Jersey 一起使用时遇到了非常相似的问题。线程被阻塞等待连接。
在我们的案例中，当我们在 GET 请求中遇到 40 次错误时连接池耗尽。
原因是连接永远不会释放回池中，除非我们真正读取响应主体(在我们收到客户端错误代码的情况下我们没有这样做)。在您的源代码中，您似乎只是在没有阅读响应的情况下抛出 ClientProtocolException。它可能是相似的。
关于java - 使用 Java Apache PoolingClientConnectionManager 泄漏内存，如何解决？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/21082692/
27
4
0
此问题与 PoolingClientConnectionManager (HttpClient 4.2.5 API) 有关 我在池中最多有 5 个远程连接和一个包含 20 多个 url 的列表，可以从
我有以下使用 PoolingClientConnectionManager 的代码: public static void main(String[] args) { int NoOfSimu
在我的应用程序中，有多个 IntentService 根据用户与应用程序的交互以随机间隔与 Tomcat 服务器连接/通信。 我正在为 Http 使用以下 Singleton public class
我正在使用 Apache DefaultHttpClient与 PoolingClientConnectionManager和 BasicResponseHandler .这些在不同线程之间共享，每个
在 Apache httpclient 4.* 中，为什么所有的方法都被弃用了？如果全部弃用，我们如何有效地使用这个类。 当我执行 PoolingClientConnectionManager.get
PoolingHttpClientConnectionManager 和 PoolingClientConnectionManager 之间有什么区别。为什么 PoolingClientConnect
在我们使用 PoolingClientConnectionManager 4.2.1 的系统上(由于其他依赖项，我们目前无法更新它)。 当有超过一定数量的请求时，我们开始为单个请求获取 SSLPeer
我的网络应用程序在晚上运行作业!遇到问题!它占用了大量内存! 我用命令查找哪个函数占用了java资源! 结果是: [tomcat@uhzd006525 ~]$ jstack 2365
我正在使用 RestClient 连接到远程 JSON API。之前，在 Spring 3.0 中，我使用的是 Commons HTTPClient。我发现由于旧的多线程连接管理器中存在大量锁定，性能
当使用 Apache PoolingClientConnectionManager 时，例如: PoolingClientConnectionManager cm = new PoolingClien
尽管在我的 pom.xml 中没有将其明确引用为依赖项，但此依赖项出现问题。实际的 maven 依赖是: org.apache.httpcomponents ht
使用 Jboss AS7 和 Resteasy 运行默认的万能 key 示例，我完全按照此处的说明操作:https://github.com/resteasy/Resteasy/tree/master
这个问题已经有答案了: Why am I getting a NoClassDefFoundError in Java? (31 个回答) 已关闭 6 年前。 public static void m
我是一名优秀的程序员,十分优秀！