继续获取扫描文件错误 : http: invalid Read on closed Body

Question

当我读取 1000 条记录文件时，我每隔 10~20 条记录就会收到错误消息:

scan file error: http: invalid Read on closed Body

这是我的代码

func parser(resp http.ResponseWriter, req *http.Request){

var count int

//....some of my code..

resp.Header().Set("Content-Type", "text/plain")
    scanner := bufio.NewScanner(req.Body)

    ctx := context.Background()
    for scanner.Scan() {
        itemID := scanner.Text()
        category := api.SearchAPI.FindCategory(itemID, lang, ctx)
        _, _ = fmt.Fprintf(resp, "%v,%v \n", itemID, category)
        count++
    }

    if err := scanner.Err(); err != nil {
        logger.Errorf("scan file error: %v", err)
        http.Error(resp, err.Error(), http.StatusBadRequest)
        return
    }

   //.....
}

Answer 1

看起来您的服务器正在关闭连接。检查您是否有任何超时以及请求花费这么长时间的原因。您可以异步处理 scanner.Text()，这样您的扫描就不会因 searchAPI 响应而被阻止，并且请求正文不会打开太久。

    resp.Header().Set("Content-Type", "text/plain")
    scanner := bufio.NewScanner(req.Body)

    ctx := context.Background()
    for scanner.Scan() {
        itemID := scanner.Text()
        go func(itemID string){
           category := api.SearchAPI.FindCategory(itemID, lang, ctx)
           _, _ = fmt.Fprintf(resp, "%v,%v \n", itemID, category)
           count++ //ENSURE YOU HAVE AN ATOMIC COUNTER INCREMENT, OR INCREMENT AFTER itemID IS READ
        }(itemID)
    }

    if err := scanner.Err(); err != nil {
        logger.Errorf("scan file error: %v", err)
        http.Error(resp, err.Error(), http.StatusBadRequest)
        return
    }

   //.....
}

或者，您可以简单地将所有 itemID 收集到一个 slice 中，关闭请求正文，然后一个一个地处理它们。

resp.Header().Set("Content-Type", "text/plain")
    scanner := bufio.NewScanner(req.Body)

    ctx := context.Background()
    itemIDs := make([]string, 0)
    for scanner.Scan() {
        itemID := scanner.Text()
        itemIDs = append(itemIDs, itemID)
    }

    if err := scanner.Err(); err != nil {
        logger.Errorf("scan file error: %v", err)
        http.Error(resp, err.Error(), http.StatusBadRequest)
        return
    }

    for _, itemID := range itemIDs {
        category := api.SearchAPI.FindCategory(itemID, lang, ctx)
        _, _ = fmt.Fprintf(resp, "%v,%v \n", itemID, category)
        count++
    }

   //.....
}