Java8 Apache Http 客户端 4.5.12 - SocketException

Question

下面是我的 ApacheHttpClient，它是一个 Spring bean

@Service
public class ApacheHttpClient implements IHttpClient {
    private static final Logger              LOGGER                                             = Logger
            .getInstance(ApacheHttpClient.class);
    private static final int                 DEFAULT_MAX_TOTAL_CONNECTIONS                      = 400;
    private static final int                 DEFAULT_IDLE_CONNECTION_EVICTION_FREQUENCY_SECONDS = 300;
    private static final int                 DEFAULT_MAX_CONNECTIONS_PER_ROUTE                  = DEFAULT_MAX_TOTAL_CONNECTIONS;

    private static final int                 DEFAULT_CONNECTION_TIMEOUT_MILLISECONDS            = (60 * 1000);
    private static final int                 DEFAULT_READ_TIMEOUT_MILLISECONDS                  = (4 * 60 * 1000);
    private static final int                 DEFAULT_WAIT_TIMEOUT_MILLISECONDS                  = (60 * 1000);
    private static final int                 DEFAULT_VALIDATE_AFTER_INACTIVITY_MILLISECONDS     = (5 * 60 * 1000);

    private static final int                 DEFAULT_KEEP_ALIVE_MILLISECONDS                    = (5 * 60 * 1000);
    private static final int                 DEFAULT_REQUEST_RETRY                              = 2;

    @Autowired
    private CPSSLContextHelper               cpSSLContext;

    @Autowired
    private CollaborationPortalConfiguration cpConfiguration;

    private int                              keepAlive                                          = DEFAULT_KEEP_ALIVE_MILLISECONDS;
    private int                              maxTotalConnections                                = DEFAULT_MAX_TOTAL_CONNECTIONS;

    private int                              maxConnectionsPerRoute                             = DEFAULT_MAX_CONNECTIONS_PER_ROUTE;
    private int                              connectTimeout                                     = DEFAULT_CONNECTION_TIMEOUT_MILLISECONDS;
    private int                              readTimeout                                        = DEFAULT_READ_TIMEOUT_MILLISECONDS;

    private int                              waitTimeout                                        = DEFAULT_WAIT_TIMEOUT_MILLISECONDS;
    private int                              requestRetry                                       = DEFAULT_REQUEST_RETRY;

    private CloseableHttpClient              httpClient;

    private ConnectionKeepAliveStrategy      keepAliveStrategy                                  = (response,
            context) -> {
                                                                                                    HeaderElementIterator it = new BasicHeaderElementIterator(
                                                                                                            response.headerIterator(
                                                                                                                    HTTP.CONN_KEEP_ALIVE));
                                                                                                    while (it
                                                                                                            .hasNext()) {
                                                                                                        HeaderElement he = it
                                                                                                                .nextElement();
                                                                                                        String param = he
                                                                                                                .getName();
                                                                                                        String value = he
                                                                                                                .getValue();
                                                                                                        if (value != null
                                                                                                                && param.equalsIgnoreCase(
                                                                                                                        "timeout")) {
                                                                                                            try {
                                                                                                                return Long
                                                                                                                        .parseLong(
                                                                                                                                value)
                                                                                                                        * 1000;
                                                                                                            } catch (NumberFormatException ignore) {}
                                                                                                        }
                                                                                                    }
                                                                                                    return keepAlive;
                                                                                                };


    @PostConstruct
    public void initializeApacheHttpClient() {

        // config timeout
        RequestConfig config = RequestConfig.custom()
                .setConnectTimeout(connectTimeout)
                .setConnectionRequestTimeout(waitTimeout)
                .setSocketTimeout(readTimeout).build();
        Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory> create()
                .register("http", PlainConnectionSocketFactory.INSTANCE)
                .register("https", new SSLConnectionSocketFactory(customSSLContext.getSSLContext())).build();

        PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);

        connManager.setMaxTotal(maxTotalConnections);

        // Increase default max connection per route
        connManager.setDefaultMaxPerRoute(maxConnectionsPerRoute);

        // Defines period of inactivity in milliseconds after which persistent connections must be re-validated prior to
        // being reused
        connManager.setValidateAfterInactivity(DEFAULT_VALIDATE_AFTER_INACTIVITY_MILLISECONDS);

        httpClient = HttpClients.custom().setKeepAliveStrategy(keepAliveStrategy).setConnectionManager(connManager)
                .setConnectionManagerShared(true).setSSLContext(customSSLContext.getSSLContext())
                .setDefaultRequestConfig(config)
                .setRetryHandler(new DefaultHttpRequestRetryHandler(requestRetry, true))
                .build();

        // detect idle and expired connections and close them
        IdleConnectionEvictor staleMonitor = new IdleConnectionEvictor(connManager, DEFAULT_IDLE_CONNECTION_EVICTION_FREQUENCY_SECONDS);
        staleMonitor.start();

        LOGGER.log(Level.INFO, "Initialize ApacheHttpClient is successful");
    }
}   

下面是我的IdleConnectionEvictor

public class IdleConnectionEvictor extends Thread {
    private static final Logger          LOGGER = Logger.getInstance(IdleConnectionEvictor.class);
    ReentrantLock                        lock   = new ReentrantLock();
    private NHttpClientConnectionManager nioConnMgr;

    private int                          httpClientIdleConnectionEvictionFrequency;

    private volatile boolean             shutdown;

    public IdleConnectionEvictor(HttpClientConnectionManager connMgr, int httpClientIdleConnectionEvictionFrequency) {
        super();
        this.connMgr = connMgr;
        this.httpClientIdleConnectionEvictionFrequency = httpClientIdleConnectionEvictionFrequency;
        LOGGER.log(Level.INFO, "Started IdleConnectionEvictor for Apache Http Client");
    }

    @Override
    public void run() {

        try {
            Thread.sleep(30 * 1000L);
            boolean isLockAcquired = lock.tryLock(1, TimeUnit.SECONDS);
            if (!isLockAcquired)
                LOGGER.log(Level.ERROR, "Couldnt acquire lock in 1 second to recycle Stale Http connections");
            while (!shutdown && isLockAcquired && !Thread.currentThread().interrupted()) {


                Optional.ofNullable(connMgr).ifPresent(HttpClientConnectionManager::closeExpiredConnections);
                Optional.ofNullable(connMgr).ifPresent(connManager -> connManager
                        .closeIdleConnections(httpClientIdleConnectionEvictionFrequency, TimeUnit.SECONDS));
                Optional.ofNullable(connMgr).ifPresent(connManager -> LOGGER.log(Level.DEBUG,
                        "Closed ExpiredConnections and IdleConnections for Apache Http Client"));

            }

        } catch (InterruptedException ex) {
            LOGGER.log(Level.ERROR, "InterruptedException while recycling Stale Http connections ", ex);
        } finally {
            lock.unlock();
        }

    }

    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }

}

RestService 调用 Http 请求

@Service
public class RestService{
    public <T> Response<T> call(HttpUriRequest request, ResponseHandler<T> responseHandler, long timeout) {
        Response<T> response;
        Optional<HttpResponse> optionalHttpResponse = null;
        CloseableHttpResponse httpResponse = null;
        try {
            try (CloseableHttpClient httpClient = getHttpClient()) {
                optionalHttpResponse = timeout == TIME_OUT_DISABLED ? execute(request, httpClient) : execute(request, httpClient, timeout);
                if (!optionalHttpResponse.isPresent())
                    throw new ClientMessageException("Empty/Null Response for " + request.getURI());
                httpResponse = (CloseableHttpResponse) optionalHttpResponse.get();
                HttpEntity entity = httpResponse.getEntity();
                try {
                    return new Response<>(httpResponse.getStatusLine().getStatusCode(), responseHandler.handleResponse(request, httpResponse, entity));
                } catch (Exception e) {
                    LOGGER.log(Level.ERROR, "Exception in Fetching Response from Server", e);
                    return new Response<>(httpResponse.getStatusLine().getStatusCode());
                } finally {
                    EntityUtils.consumeQuietly(entity);
                }

            }
        } catch (IOException e) {
            throw new ClientGeneralException(request, e);
        } finally {
            Optional.ofNullable(httpResponse).ifPresent(res -> {
                try {
                    res.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            });
            ((HttpRequestBase) request).releaseConnection();
        }
    }

    public Optional<HttpResponse> execute(HttpUriRequest request, Closeable httpClient) {

        if (!(httpClient instanceof CloseableHttpClient))
            throw new RuntimeException("UnSupported HttpClient Exception");
        CloseableHttpResponse httpResponse = null;
        try {
            CloseableHttpClient closeableHttpClient = (CloseableHttpClient) httpClient;
            httpResponse = closeableHttpClient.execute(request); //line 94
        } catch (ConnectionPoolTimeoutException e) {
            LOGGER.log(Level.ERROR,
                    "Connection pool is empty for request on uri: [" + request.getURI() + "]. Status code: ", e);
            throw new ResponseException("Connection pool is empty. " + e, request.getURI(), e);
        } catch (SocketTimeoutException | NoHttpResponseException e) {

            LOGGER.log(Level.ERROR, "Server on uri: [" + request.getURI() + "] is high loaded. Status code: ", e);
            throw new ResponseException("Remote server is high loaded. " + e, request.getURI(), e);
        } catch (ConnectTimeoutException e) {
            LOGGER.log(Level.ERROR, "HttpRequest is unable to establish a connection with the: [" + request.getURI()
                    + "] within the given period of time. Status code: ", e);
            throw new ResponseException(
                    "HttpRequest is unable to establish a connection within the given period of time. " + e,
                    request.getURI(), e);

        } catch (HttpHostConnectException e) {
            LOGGER.log(Level.ERROR, "Server on uri: [" + request.getURI() + "] is down. Status code: ", e);
            throw new ResponseException("Server is down. " + e, request.getURI(), e);
        } catch (ClientProtocolException e) {
            LOGGER.log(Level.ERROR, "URI: [" + request.getURI() + "]", e);
            throw new ResponseException(e.getMessage(), request.getURI(), e);
        } catch (IOException e) {
            LOGGER.log(Level.ERROR,
                    "Connection was aborted for request on uri: [" + request.getURI() + "]. Status code: ", e);
            throw new ResponseException("Connection was aborted. " + e, request.getURI(), e);
        }
        return Optional.ofNullable(httpResponse);

    }

    public Optional<HttpResponse> execute(HttpUriRequest request, Closeable httpClient, long timeOut) {
        Optional<HttpResponse> httpResponse;
        try {
            ExecutorService executorService = Executors.newCachedThreadPool();
            Future<Optional<HttpResponse>> future = executorService.submit(() -> execute(request, httpClient)); //line 129
            httpResponse = future.get(timeOut, TimeUnit.SECONDS);
            executorService.shutdown();
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
            LOGGER.log(Level.ERROR, "Request execution error occured ", e);
            throw new ResponseException(e.getMessage(), request.getURI(), e);

        }
        return httpResponse;
    }
}

下面是在调用 https://reports.abc.com:8443/show.json?screenName=invoiceReport 30 秒超时时随机出现的“Socket Closed”异常，可以吗？如果它是库的问题或者不能通过配置更改来修复，请帮助？

http client request: POST https://reports.abc.com:8443/show.json?screenName=invoiceReport
log debug: o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: default
log debug: o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: JSESSIONID][value: F3 ... [expiry: null] match [(secure)reports.abc.com:8443/show.json]
log debug: o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: isDocumentIndexingInP ... [expiry: null] match [(secure)reports.abc.com:8443/show.json]
log debug: o.a.h.c.protocol.RequestAuthCache - Auth cache not set in the context
log debug: o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {s}->http ... atacert.com:8443][total available: 56; route allocated: 66 of 400; total allocated: 67 of 400]
log debug: o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 134][route: { ... atacert.com:8443][total available: 56; route allocated: 67 of 400; total allocated: 68 of 400]
log debug: o.a.h.impl.execchain.MainClientExec - Opening connection {s}->https://reports.abc.com:8443
log debug: o.a.h.i.c.DefaultHttpClientConnectionOperator - Connecting to reports.abc.com/10.10.10.10:8443
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Connecting socket to reports.abc.com/10.10.10.10:8443 with timeout 60000
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_A ... TH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Starting handshake
log info: o.a.http.impl.execchain.RetryExec - I/O exception (java.net.SocketException) caught when processing request to {s}->https://reports.abc.com:8443: Socket Closed
**log debug: o.a.http.impl.execchain.RetryExec - java.net.SocketException: Socket Closed**
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead()
    at java.net.SocketInputStream.read()
    at java.net.SocketInputStream.read()
    at sun.security.ssl.InputRecord.readFully()
    at sun.security.ssl.InputRecord.read()
    at sun.security.ssl.SSLSocketImpl.readRecord()
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake()
    at sun.security.ssl.SSLSocketImpl.startHandshake()
    at sun.security.ssl.SSLSocketImpl.startHandshake()
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
    at com.apps.http.rest.impl.RestService.execute(RestService.java:94)
    at com.apps.http.rest.impl.RestService.lambda$execute$1(RestService.java:129)
    at java.util.concurrent.FutureTask.run()
    at java.util.concurrent.ThreadPoolExecutor.runWorker()
    at java.util.concurrent.ThreadPoolExecutor$Worker.run()
    at java.lang.Thread.run()

log info: o.a.http.impl.execchain.RetryExec - Retrying request to {s}->https://reports.abc.com:8443
log debug: o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: default
log debug: o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: JSESSIONID][value: A2 ... [expiry: null] match [(secure)reports.abc.com:8443/show.json]
log debug: o.a.h.c.protocol.RequestAddCookies - Cookie [version: 0][name: isDocumentIndexingInP ... [expiry: null] match [(secure)reports.abc.com:8443/show.json]
log debug: o.a.h.c.protocol.RequestAuthCache - Auth cache not set in the context
log debug: o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {s}->http ... atacert.com:8443][total available: 57; route allocated: 68 of 400; total allocated: 69 of 400]
log debug: o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 143][route: { ... atacert.com:8443][total available: 57; route allocated: 69 of 400; total allocated: 70 of 400]
log debug: o.a.h.impl.execchain.MainClientExec - Opening connection {s}->https://reports.abc.com:8443
log debug: o.a.h.i.c.DefaultHttpClientConnectionOperator - Connecting to reports.abc.com/10.10.10.10:8443
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Connecting socket to reports.abc.com/10.10.10.10:8443 with timeout 60000
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_A ... TH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
log debug: o.a.h.c.s.SSLConnectionSocketFactory - Starting handshake
this auxiliary thread was still running when the transaction ended
log error: c.d.a.c.http.rest.impl.RestService - Request execution error occured
java.util.concurrent.TimeoutException
exception
log debug: o.a.h.impl.execchain.MainClientExec - Cancelling request execution
log debug: o.a.h.i.c.DefaultManagedHttpClientConnection - http-outgoing-134: Shutdown connection
log debug: o.a.h.impl.execchain.MainClientExec - Connection discarded
log debug: o.a.h.i.c.PoolingHttpClientConnectionManager - Connection released: [id: 134][route: { ... .abc.com:8443][total available: 57; route allocated: 68 of 400; total allocated: 69 of 400]

Answer 1

我不知道您出现异常的原因，但我想我可能会建议一些可以帮助您更有效地诊断问题的工具。 Apache Http 客户端是一个伟大的、被广泛接受的工具。但是，为了涵盖 Http 协议(protocol)的所有部分，它也是一个相当复杂的工具。因此，有时尝试一些简单的工具很有用，这些工​​具可能无法提供如此广泛的功能和完整的功能，但使用起来非常简单，并且涵盖了在大多数情况下足够的基本功能。当我遇到类似问题时，我编写了自己的 Http 客户端，它是开源库的一部分。我建议尝试使用它而不是 Apache Http 客户端来查看问题是否重现。如果没有 - 很好，但如果有，它可能会使调试更简单。用法可以很简单:

HttpClient client = new HttpClient();
client.setConnectTimeout(timeOut, TimeUnit.MILISECONDS); //not required but may be useful
client.setReadTimeout(timeOut, TimeUnit.MILISECONDS);    //not required but may be useful
client.setContentType("...");
String content = client.sendHttpRequest(url, HttpClient.HttpMethod.GET);

BTW URL 可以设置为多次使用方法 public void setConnectionUrl(java.lang.String connectionUrl) 多次重复使用。然后使用方法 public java.lang.String sendHttpRequest(HttpClient.HttpMethod callMethod) 简单地发送请求。

此外，在同一个库中，还有一些您可能会觉得有用的其他实用程序。其中之一是不需要将其包装在 try-catch block 中的 TimeUtils.sleepFor() 方法。你可以只写:
TimeUtils.sleepFor(30, TimeUnit.SECONDS);
不用担心 InterruptedException。另一个是 Stacktrace 过滤，它使读取堆栈跟踪变得更加容易。不管怎样，这个库叫做 MgntUtils，你可以在 Maven Central 上找到它。在这里 Github包括源代码和 Javadoc。 Javadoc可以单独查看here并且可以找到关于图书馆的解释文章here .