java.nio.ScoketChannel 忽略 Content-Length 并使用 Transfer-Encoding : chunked based on User-Agent

Question

我想在 javax.servlet.Filter 中压缩响应体。这是我的代码

byte[] bytes =  // compressing response body
response.addHeader("Content-Encoding", "gzip");
response.addHeader("Content-Length", String.valueOf(bytes.length));
response.setContentLength(bytes.length);
response.setBufferSize(bytes.length * 2);
ServletOutputStream output = response.getOutputStream();
output.write(bytes);
output.flush();
output.close();

但我在 Chrome Dev Tool 中看到的实际响应是

Accept-Ranges: bytes
Cache-Control: max-age=2592000
Content-Type: application/javascript;charset=UTF-8
Date: Fri, 14 Dec 2018 15:34:25 GMT
Last-Modified: Tue, 09 Oct 2018 13:42:54 GMT
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked

我不期望 Transfer-Encoding: chunked，因为我声明了“Content-Length”。我在java上写了一个简单的测试

URLConnection connection = new URL("http://127.0.0.1:8081/js/ads.js").openConnection();
connection.addRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
connection.addRequestProperty("Accept-Encoding", "gzip, deflate");
connection.addRequestProperty("Accept-Language", "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7");
connection.addRequestProperty("Cache-Control", "no-cache");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Host", "127.0.0.1:8081");
connection.addRequestProperty("Pragma", "no-cache");
connection.addRequestProperty("Upgrade-Insecure-Requests", "1");
connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"); 
connection.connect();
connection.getHeaderFields().forEach((s, strings) ->
        System.out.println(s + ":" + String.join(",", strings)));

这是我发现的:

如果我评论设置“User-Agent”标题或将“User-Agent”更改为任何其他值，那么我会收到“Content-Length”的响应

如果“用户代理”指向 Chrome，那么我会得到 Transfer-Encoding: chunked。

我调试了 sun.nio.ch.SocketChannel#write 方法，它获得了带有 Content-Length header 值的正确 ByteBuffers。

我无法理解这种对分块的神奇转变发生在哪里？

更新

奇怪的是，我将 gziped 字节写入 Socket(我确定在调试到 SocketChannel 实现中的 native 方法 write 调用时)。但 URLConnection 返回我的 解压 如果我不指定 User-Agent header 或放置一些随机字符串，则使用 Chrome 的 User-Agent 的字节数组和正确的 gziped 字节数组。
SO 似乎在 Windows 套接字实现中的某个地方发生了魔法。

Answer 1

显示代码

我会假设您显示的代码有效并且问题出在其他地方。

设置

Windows 10

Tomcat 7.0.92

Chrome 71.0.3578.98

测试用例

我尝试创建一个小的过滤器示例，以便能够试用您的测试代码。

顺便说一下，更适合生产使用的压缩过滤器可以在
Tomcat 提供的示例(webapps\examples\WEB-INF\classes\compressionFilters)。

import java.io.*;
import java.util.zip.GZIPOutputStream;
import javax.servlet.*;
import javax.servlet.http.*;

public class CompressionFilter  implements Filter {

    public void init(FilterConfig filterConfig) { }
    public void destroy() { }

    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        final HttpServletRequest request = (HttpServletRequest) servletRequest;
        final HttpServletResponse response = (HttpServletResponse) servletResponse;

        ResponseWrapper wrapper = new ResponseWrapper(response);
        filterChain.doFilter(request, wrapper);
        byte[] uncompressed = wrapper.getBytes();

        byte[] bytes = compress(uncompressed);
        response.addHeader("Content-Encoding", "gzip");
        response.addHeader("Content-Length", String.valueOf(bytes.length));
        response.setContentLength(bytes.length);
        //response.setBufferSize(bytes.length * 2);
        ServletOutputStream output = response.getOutputStream();
        output.write(bytes);
        output.flush();
        output.close();

        System.out.println("request to:" +  request.getServletPath()
                + " size changed from: " + uncompressed.length
                + " to " + bytes.length);
    }

    private byte[] compress(byte[] bytes) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        GZIPOutputStream gzipOutputStream = new GZIPOutputStream(baos);
        gzipOutputStream.write(bytes);
        gzipOutputStream.close();
        return baos.toByteArray();
    }


    public class ResponseWrapper extends HttpServletResponseWrapper {
        private ByteArrayOutputStream output = new ByteArrayOutputStream();
        private PrintWriter printWriter = null;

        ResponseWrapper(HttpServletResponse response) {
            super(response);
        }

        byte[] getBytes() {
            if (printWriter != null)
                printWriter.flush();
            return output.toByteArray();
        }

        public PrintWriter getWriter() {
            if (printWriter == null)
                printWriter = new PrintWriter(output);
            return printWriter;
        }

        public ServletOutputStream getOutputStream() {
            return new ServletOutputStream() {
                private WriteListener writeListener;
                public boolean isReady() { return true; }
                public void setWriteListener(WriteListener writeListener) { this.writeListener  = writeListener; }
                public void write(int b) {
                    output.write(b);
                    if(writeListener != null)
                        writeListener.notify();
                }
            };
        }
    }

}

结果

Chrome 的开发人员工具中显示了三个带有静态 html 的测试用例、一个 JSP 生成的页面和带有一些虚拟内容的 Servlet 生成的页面:

a) 使用静态 html

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Accept-Ranges: bytes
ETag: W/"108-1545775482914"
Last-Modified: Tue, 25 Dec 2018 22:04:42 GMT
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 97
Date: Tue, 25 Dec 2018 22:34:41 GMT

b) JSP 生成

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 38
Date: Tue, 25 Dec 2018 22:49:17 GMT

c) Servlet 生成

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 65
Date: Tue, 25 Dec 2018 22:49:43 GMT

有了这个设置，就没有 Transfer-Encoding: chunked。那么也许可以在其他地方找到这个分块标题的原因？