使用 Java 6、HTMLUnit 和 Bouncy CaSTLe 的 TLSv1 SSL 握手失败

Question

我正在尝试连接 SSL 站点 https://link.bollore-logistics.com通过使用 Java 6 和 HTMLUnit，我使用 Bouncy CaSTLe 添加了更多受支持的密码。但是，我仍然收到握手失败。由于生产限制，我无法升级 Java 版本。请帮忙

配置代码:

HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
        public boolean verify(String hostname, SSLSession session) {
            return true;
        }
    });

    if (Security.getProvider("BC") == null){
        logger.info("Bouncy Castle provider is NOT available");
    }
    else{
        logger.info("Bouncy Castle provider is available");
    }

    Provider provider = new BouncyCastleProvider();
    Security.addProvider(provider);

    SSLContext context = SSLContext.getInstance("SSL");

    context.init(null, new X509TrustManager[]{new X509TrustManager() {

        @Override
        public void checkClientTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {
        }

        @Override
        public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
        }

        @Override
        public X509Certificate[] getAcceptedIssuers() {
            return null;
        }
    }}, new SecureRandom());



    String[] scs = context.getSocketFactory().getSupportedCipherSuites();
    Arrays.sort(scs);

    for(String s : scs) {
        System.out.println(s);
     }
    HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());


    // Create all-trusting host name verifier
    HostnameVerifier allHostsValid = new HostnameVerifier() {
        public boolean verify(String hostname, SSLSession session) {
            return true;
        }
    };
    HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

建立连接的代码(HTMLUnit):

WebClient webClient = new WebClient(BrowserVersion.CHROME);
webClient.getOptions().setUseInsecureSSL(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setCssEnabled(false);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getPage(url);

错误日志如下:

2017-12-06 11:27:40 DEBUG WebClient:325 - Get page for window named '', using WebRequest[<url="http://link.bollore-logistics.com/Contact/SmartSearch/77707575", GET, EncodingType[name=application/x-www-form-urlencoded], [], {Accept-Encoding=gzip, deflate, Accept=*/*}, null>]
2017-12-06 11:27:40 DEBUG WebClient:1259 - Load response for GET http://link.bollore-logistics.com/Contact/SmartSearch/77707575
trigger seeding of SecureRandom
done seeding SecureRandom
2017-12-06 11:27:40 DEBUG RequestAddCookies:122 - CookieSpec selected: mine
2017-12-06 11:27:40 DEBUG RequestAuthCache:76 - Auth cache not set in the context
2017-12-06 11:27:40 DEBUG PoolingHttpClientConnectionManager:249 - Connection request: [route: {}->http://link.bollore-logistics.com:80][total kept alive: 0; route allocated: 0 of 6; total allocated: 0 of 20]
2017-12-06 11:27:40 DEBUG PoolingHttpClientConnectionManager:282 - Connection leased: [id: 0][route: {}->http://link.bollore-logistics.com:80][total kept alive: 0; route allocated: 1 of 6; total allocated: 1 of 20]
2017-12-06 11:27:40 DEBUG MainClientExec:234 - Opening connection {}->http://link.bollore-logistics.com:80
2017-12-06 11:27:40 DEBUG DefaultHttpClientConnectionOperator:131 - Connecting to link.bollore-logistics.com/195.101.158.66:80
2017-12-06 11:27:40 DEBUG DefaultHttpClientConnectionOperator:138 - Connection established 192.168.171.168:52556<->195.101.158.66:80
2017-12-06 11:27:40 DEBUG MainClientExec:255 - Executing request GET /Contact/SmartSearch/77707575 HTTP/1.1
2017-12-06 11:27:40 DEBUG MainClientExec:260 - Target auth state: UNCHALLENGED
2017-12-06 11:27:40 DEBUG MainClientExec:266 - Proxy auth state: UNCHALLENGED
2017-12-06 11:27:40 DEBUG headers:124 - http-outgoing-0 >> GET /Contact/SmartSearch/77707575 HTTP/1.1
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> Host: link.bollore-logistics.com
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> Accept-Language: en-US
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> Accept-Encoding: gzip, deflate
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> Accept: */*
2017-12-06 11:27:40 DEBUG headers:127 - http-outgoing-0 >> Connection: Keep-Alive
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "GET /Contact/SmartSearch/77707575 HTTP/1.1[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "Host: link.bollore-logistics.com[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "Accept-Language: en-US[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "Accept-Encoding: gzip, deflate[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "Accept: */*[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
2017-12-06 11:27:40 DEBUG wire:72 - http-outgoing-0 >> "[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "HTTP/1.0 302 Found[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "Location: https://link.bollore-logistics.com/Contact/SmartSearch/77707575[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "Server: BigIP[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "Connection: Keep-Alive[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "Content-Length: 0[\r][\n]"
2017-12-06 11:27:41 DEBUG wire:72 - http-outgoing-0 << "[\r][\n]"
2017-12-06 11:27:41 DEBUG headers:113 - http-outgoing-0 << HTTP/1.0 302 Found
2017-12-06 11:27:41 DEBUG headers:116 - http-outgoing-0 << Location: https://link.bollore-logistics.com/Contact/SmartSearch/77707575
2017-12-06 11:27:41 DEBUG headers:116 - http-outgoing-0 << Server: BigIP
2017-12-06 11:27:41 DEBUG headers:116 - http-outgoing-0 << Connection: Keep-Alive
2017-12-06 11:27:41 DEBUG headers:116 - http-outgoing-0 << Content-Length: 0
2017-12-06 11:27:41 DEBUG MainClientExec:284 - Connection can be kept alive indefinitely
2017-12-06 11:27:41 DEBUG PoolingHttpClientConnectionManager:314 - Connection [id: 0][route: {}->http://link.bollore-logistics.com:80] can be kept alive indefinitely
2017-12-06 11:27:41 DEBUG PoolingHttpClientConnectionManager:320 - Connection released: [id: 0][route: {}->http://link.bollore-logistics.com:80][total kept alive: 1; route allocated: 1 of 6; total allocated: 1 of 20]
2017-12-06 11:27:41 DEBUG WebClient:1349 - Got a redirect status code [302] new location = [https://link.bollore-logistics.com/Contact/SmartSearch/77707575]
2017-12-06 11:27:41 DEBUG WebClient:1259 - Load response for GET https://link.bollore-logistics.com/Contact/SmartSearch/77707575
2017-12-06 11:27:41 DEBUG RequestAddCookies:122 - CookieSpec selected: mine
2017-12-06 11:27:41 DEBUG RequestAuthCache:76 - Auth cache not set in the context
2017-12-06 11:27:41 DEBUG PoolingHttpClientConnectionManager:249 - Connection request: [route: {s}->https://link.bollore-logistics.com:443][total kept alive: 1; route allocated: 0 of 6; total allocated: 1 of 20]
2017-12-06 11:27:41 DEBUG PoolingHttpClientConnectionManager:282 - Connection leased: [id: 1][route: {s}->https://link.bollore-logistics.com:443][total kept alive: 1; route allocated: 1 of 6; total allocated: 2 of 20]
2017-12-06 11:27:41 DEBUG MainClientExec:234 - Opening connection {s}->https://link.bollore-logistics.com:443
2017-12-06 11:27:41 DEBUG DefaultHttpClientConnectionOperator:131 - Connecting to link.bollore-logistics.com/195.101.158.66:443
2017-12-06 11:27:41 DEBUG HtmlUnitSSLConnectionSocketFactory:335 - Connecting socket to link.bollore-logistics.com/195.101.158.66:443 with timeout 90000
2017-12-06 11:27:41 DEBUG HtmlUnitSSLConnectionSocketFactory:388 - Enabled protocols: [TLSv1]
2017-12-06 11:27:41 DEBUG HtmlUnitSSLConnectionSocketFactory:389 - Enabled cipher suites:[SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA]
2017-12-06 11:27:41 DEBUG HtmlUnitSSLConnectionSocketFactory:393 - Starting handshake
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1512530605 bytes = { 147, 117, 177, 152, 172, 87, 109, 103, 219, 130, 24, 140, 216, 108, 154, 3, 162, 211, 205, 89, 29, 221, 91, 103, 85, 39, 206, 187 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
***
[write] MD5 and SHA1 hashes:  len = 161
0000: 01 00 00 9D 03 01 5A 27   63 AD 93 75 B1 98 AC 57  ......Z'c..u...W
0010: 6D 67 DB 82 18 8C D8 6C   9A 03 A2 D3 CD 59 1D DD  mg.....l.....Y..
0020: 5B 67 55 27 CE BB 00 00   36 00 04 00 05 00 2F C0  [gU'....6...../.
0030: 02 C0 04 C0 0C C0 0E C0   07 C0 09 C0 11 C0 13 00  ................
0040: 33 00 32 00 0A C0 03 C0   0D C0 08 C0 12 00 16 00  3.2.............
0050: 13 00 09 00 15 00 12 00   03 00 08 00 14 00 11 01  ................
0060: 00 00 3E 00 0A 00 34 00   32 00 17 00 01 00 03 00  ..>...4.2.......
0070: 13 00 15 00 06 00 07 00   09 00 0A 00 18 00 0B 00  ................
0080: 0C 00 19 00 0D 00 0E 00   0F 00 10 00 11 00 02 00  ................
0090: 12 00 04 00 05 00 14 00   08 00 16 00 0B 00 02 01  ................
00A0: 00                                                 .
http-bio-8080-exec-3, WRITE: TLSv1 Handshake, length = 161
[Raw write]: length = 166
0000: 16 03 01 00 A1 01 00 00   9D 03 01 5A 27 63 AD 93  ...........Z'c..
0010: 75 B1 98 AC 57 6D 67 DB   82 18 8C D8 6C 9A 03 A2  u...Wmg.....l...
0020: D3 CD 59 1D DD 5B 67 55   27 CE BB 00 00 36 00 04  ..Y..[gU'....6..
0030: 00 05 00 2F C0 02 C0 04   C0 0C C0 0E C0 07 C0 09  .../............
0040: C0 11 C0 13 00 33 00 32   00 0A C0 03 C0 0D C0 08  .....3.2........
0050: C0 12 00 16 00 13 00 09   00 15 00 12 00 03 00 08  ................
0060: 00 14 00 11 01 00 00 3E   00 0A 00 34 00 32 00 17  .......>...4.2..
0070: 00 01 00 03 00 13 00 15   00 06 00 07 00 09 00 0A  ................
0080: 00 18 00 0B 00 0C 00 19   00 0D 00 0E 00 0F 00 10  ................
0090: 00 11 00 02 00 12 00 04   00 05 00 14 00 08 00 16  ................
00A0: 00 0B 00 02 01 00                                  ......
[Raw read]: length = 5
0000: 15 03 01 00 02                                     .....
[Raw read]: length = 2
0000: 02 28                                              .(
http-bio-8080-exec-3, READ: TLSv1 Alert, length = 2
http-bio-8080-exec-3, RECV TLSv1 ALERT:  fatal, handshake_failure
http-bio-8080-exec-3, called closeSocket()
http-bio-8080-exec-3, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
2017-12-06 11:27:41 DEBUG DefaultManagedHttpClientConnection:87 - http-outgoing-1: Shutdown connection
2017-12-06 11:27:41 DEBUG MainClientExec:128 - Connection discarded
2017-12-06 11:27:41 DEBUG DefaultManagedHttpClientConnection:79 - http-outgoing-1: Close connection
2017-12-06 11:27:41 DEBUG PoolingHttpClientConnectionManager:320 - Connection released: [id: 1][route: {s}->https://link.bollore-logistics.com:443][total kept alive: 1; route allocated: 0 of 6; total allocated: 1 of 20]
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
    at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:136)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1694)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:939)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1120)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1147)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1131)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
    at com.gargoylesoftware.htmlunit.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:155)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:178)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1313)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1371)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1230)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:338)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:407)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:392)
    at com.grabber.Grabber.accessWebpage(Grabber.java:204)
    at com.grabber.Grabber.doGet(Grabber.java:53)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:619)

Answer 1

我有一个类似的问题(但在那种情况下没有使用 HtmlUnit。我直接使用了 org.apache.http.impl.client.CloseableHttpClient)。

直到 java 7 握手似乎有问题(在 java 8 中不再有问题)。如果您尝试访问提供“TLSv1.2”的服务器，您可能会收到您报告的错误。

如果您的情况下安全性不是什么大问题，您可以尝试强制使用 TLSv1.1(只要您尚未切换到 java 8)。这应该可以这样:

   WebClientOptions webClientOptions= new WebClientOptions();
   webClientOptions.setSSLClientProtocols(new String[]{"TLSv1.1"});
   SSLConnectionSocketFactory.buildSSLSocketFactory(webClientOptions);