Java 11 `HttpClient` 下载但没有？ (负内容长度)

Question

这对于像 https://speed.hetzner.de/10GB.bin 这样的一些 URL 非常有效，它会在我的订阅服务器中打印很多调试消息，所以我的控制台看起来像这样:

I got 16384 bytes (6396)
Got data!
I got 16384 bytes (6397)
Got data!
I got 16384 bytes (6398)
Got data!
I got 16384 bytes (6399)

taskmanager 说 java 正在以 ~50Mbit/s 的速度下载(我的完整下载速度)

但是，您可以获得文件的 url，您可以在这里下载 https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail这是非常不同的。控制台仅显示以下消息:

Starting download...
Version: HTTP_1_1 Status code 200 Got headers back java.net.http.HttpHeaders@d4ccf412 { {accept-ranges=[bytes], cache-control=[max-age=0, no-cache, no-store], cf-ray=[46ed5b5aeaf9721d-AMS], connection=[keep-alive], content-length=[2147483648], content-type=[application/octet-stream], date=[Wed, 24 Oct 2018 15:08:07 GMT], expect-ct=[max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"], server=[cloudflare], set-cookie=[__cfduid=dcba8bb98269e665c521493baf25d0b6d1540393686; expires=Thu, 24-Oct-19 15:08:06 GMT; path=/; domain=.usbcraft.net; HttpOnly], x-bz-content-sha1=[none], x-bz-file-id=[4_zd729cdbbb41cd50850150a12_f201d3478bea24efd_d20181024_m130349_c001_v0001109_t0059], x-bz-file-name=[java_http_client.fail], x-bz-info-src_last_modified_millis=[1540386142160], x-bz-upload-timestamp=[1540386229000]} }
Content-Length parsed to int: -2147483648

和 MySubscriber::onNext 永远不会被调用。但是 taskmanager 仍然显示我的文件正在全速下载

编辑:让它运行一段时间后，它会打印更多内容(这是另一个 URL，但对于我当前让它运行的更新后的 url 应该是相同的):

I got an error fixed content-length: -298366259, bytes received: 0
Exception in thread "main" java.io.IOException: fixed content-length: -298366259, bytes received: 0
    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:565)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:119)
    at TestKt.main(Test.kt:29)
Caused by: java.io.IOException: fixed content-length: -298366259, bytes received: 0
    at java.net.http/jdk.internal.net.http.common.Utils.wrapWithExtraDetail(Utils.java:296)
    at java.net.http/jdk.internal.net.http.Http1Response$BodyReader.onReadError(Http1Response.java:742)
    at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.checkForErrors(Http1AsyncReceiver.java:297)
    at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Http1AsyncReceiver.java:263)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.EOFException: EOF reached while reading
    at java.net.http/jdk.internal.net.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:591)
    at java.net.http/jdk.internal.net.http.common.SSLTube$DelegateWrapper.onComplete(SSLTube.java:268)
    at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.complete(SSLTube.java:411)
    at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onComplete(SSLTube.java:540)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.checkCompletion(SubscriberWrapper.java:443)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run1(SubscriberWrapper.java:322)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run(SubscriberWrapper.java:261)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.onComplete(SubscriberWrapper.java:419)
    at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:632)
    at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:833)
    at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowTask.run(SocketTube.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
    at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:763)
    at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:941)
    at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:245)
    at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:957)
    at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.lambda$run$3(HttpClientImpl.java:912)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
    at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:912)

因此，由于某种原因，java 将内容长度解析为负数，它将 header 解析为 Int 并导致整数溢出，但仅使用该 URL，因为另一个文件的大小为 10GB，但运行良好。

使用 wget 之类的工具或 firefox 之类的网络浏览器下载该文件也没有问题。 1fichier 上的所有文件都不是问题，只是一些。

有人能解释一下发生了什么吗？

我的代码:

import java.io.File
import java.net.CookieHandler
import java.net.URI
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse
import java.nio.ByteBuffer
import java.nio.file.Path
import java.time.Duration
import java.util.concurrent.CompletionStage
import java.util.concurrent.Flow

fun main(args: Array<String>) {
    val dir = File("C:\\Users\\kjh\\Downloads")
    val file = File(dir, "test.bin")

    file.delete()

    val client = HttpClient.newBuilder()
            .version(HttpClient.Version.HTTP_1_1)
            .connectTimeout(Duration.ofSeconds(10))
            .build()

    val request = HttpRequest.newBuilder()
            .url("https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail")
            //.url("https://speed.hetzner.de/10GB.bin")
            .build()

    println("Starting download...")
    client.send(request) {info ->
        println("Version: ${info.version()} Status code ${info.statusCode()} Got headers back ${info.headers()}")
        println("Content-Length parsed to int: ${info.headers().get("content-length")?.toLong()?.toInt()}")
        val responseHandler = HttpResponse.BodyHandlers.ofFile(file.toPath())
        MySubscriber(responseHandler.apply(info))
    }
    println("Done with download...")
}

class MySubscriber(val subscriber: HttpResponse.BodySubscriber<Path>) : HttpResponse.BodySubscriber<Path> {
    private var counter = 0
    private var total = 0L
    override fun onComplete() {
        println("Total bytes: $total")
        subscriber.onComplete()
    }

    override fun onSubscribe(subscription: Flow.Subscription?) {
        println("onSubscribe")
        subscriber.onSubscribe(subscription)
    }

    override fun onNext(item: MutableList<ByteBuffer>) {
        println("Got data!")
        item.forEach { size ->
            println("I got ${size.remaining()} bytes ($counter)")
            total += size.remaining()
        }
        counter++
        subscriber.onNext(item)
    }

    override fun onError(throwable: Throwable) {
        println("I got an error ${throwable.localizedMessage}")
        subscriber.onError(throwable)
    }

    override fun getBody(): CompletionStage<Path> {
        return subscriber.body
    }
}

fun HttpRequest.Builder.url(url: String) = this.uri(URI.create(url))

Answer 1

这是 Java 11 中新 HttpClient 的 HTTP/1.1 堆栈中的一个错误，可以在此处找到它的错误报告:https://bugs.openjdk.java.net/browse/JDK-8212926