http - panic : close of closed channel during persistent http call in GO(golang)

Question

我们将此函数用于 maxIdleConnsPerHost=100 的 HTTP 调用，其中我们获得高达 20000/分钟的请求以从第三方服务器获取结果。

功能:

func (hr *Client) Do(req *http.Request) (rsp *http.Response, err error) {
rspCh := make(chan *http.Response)
errCh := make(chan error)

var timeoutCh <-chan time.Time
if hr.Timeout > 0 {
    timeoutCh = time.After(hr.Timeout)
}

go func(req *http.Request) {
    rsp, err := hr.Client.Do(req)
    if err != nil {
        errCh <- err
        rspCh <- rsp
        logger.SetLog(" error in http_client_Do : ", logs.LevelError, err)
        if nil!=rsp{
            fmt.Println(rsp.Header)
        }else{
            fmt.Println("error with nil rsp")
        }       
    } else {
        rspCh <- rsp
        //errCh <- nil
    }
}(req)

var now time.Time

select {
case err = <-errCh:
    rsp = <-rspCh
case now = <-timeoutCh:
    go hr.Transport.CancelRequest(req)
    err = fmt.Errorf("error requesting %s: read timed out at %s after waiting %s",
        req.URL, now.Format(time.RFC3339), hr.Timeout)
case rsp = <-rspCh:
    err = nil
}
return
}

我不明白为什么这在某些情况下会中断。 (它运行很长时间很顺利)。

错误信息:

panic: close of closed channel

goroutine 3822098 [running]:
net/http.func·018()
        /usr/local/go/src/net/http/transport.go:517 +0x2a
net/http.(*Transport).CancelRequest(0xc2080b02d0, 0xc20880de10)
        /usr/local/go/src/net/http/transport.go:284 +0x97
created by Mediation/pkg/services/restApi.(*Client).Do
        /usr/local/gopath/src/ ***.go:179 +0x3d2

goroutine 1 [chan receive, 308 minutes]:
github.com/astaxie/beego.(*App).Run(0xc20800a4b0)
        /usr/local/gopath/src/github.com/astaxie/beego/app.go:171 +0x363
github.com/astaxie/beego.Run(0x0, 0x0, 0x0)
        /usr/local/gopath/src/github.com/astaxie/beego/beego.go:277 +0x17c
main.main()
        /usr/local/gopath/src/ ***/main.go:33 +0x423

goroutine 5 [syscall, 308 minutes]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
        /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 2163451 [select, 146 minutes]:
net/http.(*persistConn).readLoop(0xc208501ce0)
        /usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
        /usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 17 [chan receive, 8 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc20810a230)
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:340 +0xe3
github.com/Shopify/sarama.*Broker.(github.com/Shopify/sarama.responseReceiver)·fm()
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x27
github.com/Shopify/sarama.withRecover(0xc20800a6b0)
        /usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.func·008
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x610

goroutine 8 [select, 8 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2080b6100)
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:553 +0x2f3
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)·fm()
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x27
github.com/Shopify/sarama.withRecover(0xc2080ba730)
        /usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x8ce

Answer 1

transport.CancelRequest 以某种方式从您的代码中被多次调用。

您可能正在执行以下一项或两项操作:

• 您正在重新使用 *http.Request，导致查找可能从错误的往返行程中返回取消函数。

• 您设置了http.Client.Timeout，它也使用了Transport.CancelRequest机制，导致取消请求的竞争