java - 与客户端证书的 SSL 重新协商导致服务器缓冲区溢出

Question

我编写了一个 Java 客户端应用程序，该应用程序使用客户端证书通过 HTTPS 连接到 Apache Web 服务器，并执行到服务器的文件的 HTTP PUT。它适用于小文件，但处理大文件时会崩溃。

Apache 服务器日志显示如下:

...
OpenSSL: Handshake: done
...
Changed client verification type will force renegotiation
...
filling buffer, max size 131072 bytes
...
request body exceeds maximum size (131072) for SSL buffer
could not buffer message body to allow SSL renegotiation to proceed
...    
OpenSSL: I/O error, 5 bytes expected to read on BIO
(104)Connection reset by peer: SSL input filter read failed.
(32)Broken pipe: core_output_filter: writing data to the network
Connection closed to child 20 with standard shutdown

客户端的响应是:

java.io.IOException: Server returned HTTP response code: 401 for URL

我不熟悉这个过程，所以我不确定这里是否有必要重新协商，或者我是否可以做些什么来阻止它。或者我可以让客户等到重新协商完成后再发送申请数据？以下是客户端代码的摘录(删除了错误处理):

        URL url = new URL("my url goes here");
        con = (HttpsURLConnection) url.openConnection();
        con.setSSLSocketFactory(getMyCustomClientCertSocketFactory());
        con.setRequestMethod("PUT");
        con.setDoOutput(true);
        con.connect();
        writer = new OutputStreamWriter(con.getOutputStream());
        writer.write(xml);
        writer.close();

        parseServerResponse(con.getInputStream());

我在想也许我需要使用较低级别的 API(例如 SSLSocket)并利用 HandshakeCompletedListener？

我还想知道 Apache SSLVerifyDepth 指令是否与重新协商发生的原因有关。我在每个目录上下文(只有一个上传目录)中得到了值为 2 的指令，Apache 手册对此进行了说明:

In per-directory context it forces a SSL renegotation with the reconfigured client verification depth after the HTTP request was read but before the HTTP response is sent.

此处要求的是 Java 调试输出:

keyStore is : 
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
trustStore is: C:\Program Files\Java\jdk1.6.0_35\jre\lib\security\cacerts
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:
 ...
trigger seeding of SecureRandom
done seeding SecureRandom
***
found key for : key-alias
chain [0] = [
[
...
]
***
trigger seeding of SecureRandom
done seeding SecureRandom
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  ...
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: TLSv1 Handshake, length = 81
*** ServerHello, TLSv1
RandomCookie:  ...
Session ID:  ...
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
%% Created:  [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
** TLS_RSA_WITH_AES_128_CBC_SHA
main, READ: TLSv1 Handshake, length = 4392
*** Certificate chain
chain [0] = [
[
...
Certificate Extensions: 8
[1]: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: ...
   accessLocation: URIName: ...
   accessMethod: ...
   accessLocation: URIName: ...
]

[2]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
...
]
]
[3]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]
[4]: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [URIName: ...
]]
[5]: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: ...
[PolicyQualifierInfo: [
  qualifierID: ...
  qualifier: ...
]]  ]
]
[6]: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
  clientAuth
]
[7]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
]
[8]: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: ...
]
]
  Algorithm: [SHA1withRSA]
  Signature:
...
]
...
***
main, READ: TLSv1 Handshake, length = 4
*** ServerHelloDone
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1
main, WRITE: TLSv1 Handshake, length = 518
SESSION KEYGEN:
PreMaster Secret:
...
CONNECTION KEYGEN:
Client Nonce:
...
Server Nonce:
...
Master Secret:
...
Client MAC write Secret:
...
Server MAC write Secret:
...
Client write key:
...
Server write key:
...
Client write IV:
...
Server write IV:
...
main, WRITE: TLSv1 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 18, 162, 18, 251, 82, 111, 87, 133, 53, 240, 114, 155 }
***
main, WRITE: TLSv1 Handshake, length = 48
main, READ: TLSv1 Change Cipher Spec, length = 1
main, READ: TLSv1 Handshake, length = 48
*** Finished
verify_data:  { 46, 206, 8, 40, 63, 252, 99, 190, 251, 183, 110, 201 }
***
%% Cached client session: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
main, WRITE: TLSv1 Application Data, length = 256
main, WRITE: TLSv1 Application Data, length = 32
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
...
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 512
main, READ: TLSv1 Application Data, length = 304 

此处要求的是 getMyCustomClientCertSocketFactory 源(从 PEM 文件获取证书和 key ):

public static SSLSocketFactory getMyCustomClientCertSocketFactory(String pemPath,
        boolean verifyPeer)
        throws NoSuchAlgorithmException, FileNotFoundException, IOException,
        KeyStoreException, CertificateException, UnrecoverableKeyException,
        KeyManagementException, InvalidKeySpecException {
    SSLContext context = SSLContext.getInstance("TLS");

    byte[] certAndKey = IOUtil.fileToBytes(new File(pemPath));
    byte[] certBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN CERTIFICATE-----", "-----END CERTIFICATE-----");
    byte[] keyBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN PRIVATE KEY-----", "-----END PRIVATE KEY-----");

    X509Certificate cert = generateX509CertificateFromDER(certBytes);
    RSAPrivateKey key = generateRSAPrivateKeyFromDER(keyBytes);

    KeyStore keystore = KeyStore.getInstance("JKS");
    keystore.load(null);
    keystore.setCertificateEntry("cert-alias", cert);
    keystore.setKeyEntry("key-alias", key, "changeit".toCharArray(),
            new Certificate[]{cert});

    KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
    kmf.init(keystore, "changeit".toCharArray());

    KeyManager[] km = kmf.getKeyManagers();

    TrustManager[] tm = null;

    if (!verifyPeer) {
        tm = new TrustManager[]{new TrustyTrustManager()};
    }

    context.init(km, tm, null);

    return context.getSocketFactory();
}

Answer 1

似乎 Sun Java 中内置的 HttpsUrlConnection 工具无法以服务器友好的方式处理带有客户端证书的大型 HTTP PUT 方案(即不会溢出服务器 SSL 重新协商缓冲区)。

我检查了 curl 正在做什么以了解“服务器友好”的含义，结果发现有一个名为“Expect”的 HTTP 1.1 header ，curl 发送的值为“100-continue”(参见规范 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20) .这个 header 实质上是说“我有一个巨大的有效载荷，但在我发送之前请告诉我你是否可以处理它”。这使端点有时间在发送有效负载之前重新协商客户端证书。

在 Sun HttpUrlConnection 实现中，这个 header 似乎是不允许的，实际上在受限 header 列表中；这意味着即使您使用 HttpUrlConnection.setRequestProperty 方法设置它， header 实际上也不会发送到服务器。您可以使用系统属性 sun.net.http.allowRestrictedHeaders 覆盖受限 header ，但随后客户端会因套接字异常而崩溃，因为 Sun 实现不知道如何处理协议(protocol)的这一部分。

有趣的是，Java 的 OpenJDK 实现似乎确实支持此 header 。此外，Apache HTTP 客户端库支持此 header ( http://hc.apache.org/ )；我已经使用 Apache HTTP 客户端库实现了一个测试程序，它可以使用客户端证书和 Expect header 成功执行大文件的 HTTP PUT 请求。

总而言之，解决方案是:

1. 将 Apache SSLRenegBufferSize 指令设置为一个巨大的数字(如 64MB)。默认值为 128K。此解决方案可能会产生拒绝服务风险
2. 配置一台始终需要客户端证书的主机，而不是只有少数目录需要它的主机。这将避免重新协商。在我的场景中这不是一个好的选择，因为大多数用户都是匿名的或经过用户名/密码验证的。只有一个上传目录用于文件的编程上传。我们将不得不为这个目录创建一个具有自己的 SSL 证书的新虚拟主机。
3. 使用支持 HTTP 1.1 Expect header 的客户端。不幸的是，Sun Java 不支持开箱即用。必须使用 Apache HTTP 组件客户端库等第三方，或使用 Java 套接字 API 推出您自己的解决方案。
4. 通过最初发出负载不大但会导致重新协商的 HTTP 请求来利用 HTTP 1.1 持久连接(带保持 Activity 的流水线)，然后为 HTTP PUT 重用该连接。理论上，客户端应该能够在上传目录上发出 HTTP HEAD 或 OPTIONS，然后重用相同的连接来执行 PUT。为了使其正常工作，持久连接池可能只需要包含一个连接，以避免“启动”一个连接，然后为 PUT 发出另一个连接。但是，HttpUrlConnection 类似乎不会保留/重用涉及客户端证书或 SSL 的持久连接，因为我一直无法使此解决方案起作用。请参见(HttpsUrlConnection and keep-alive)。