google-app-engine - Golang 卡在 WaitGroup

Question

我被困在自己的等待循环中，不确定为什么。该函数采用输入和输出 channel ，然后采用 channel 中的每个项目，对内容执行 http.GET 并从 html 中提取标签。

GET 和抓取的过程在一个 go 例程中，我设置了一个 WaitGroup (innerWait) 以确保在关闭输出 channel 之前我已经处理完所有内容。

   func (fp FeedProducer) getTitles(in <-chan feeds.Item,
    out chan<- feeds.Item,
    wg *sync.WaitGroup) {

    defer wg.Done()

    var innerWait sync.WaitGroup

    for item := range in {
        log.Infof(fp.c, "Incrementing inner WaitGroup.")
        innerWait.Add(1)
        go func(item feeds.Item) {
            defer innerWait.Done()
            defer log.Infof(fp.c, "Decriment inner wait group by defer.")
            client := urlfetch.Client(fp.c)
            resp, err := client.Get(item.Link.Href)
            log.Infof(fp.c, "Getting title for: %v", item.Link.Href)
            if err != nil {
                log.Errorf(fp.c, "Error retriving page. %v", err.Error())
                return
            }
            if strings.ToLower(resp.Header.Get("Content-Type")) == "text/html; charset=utf-8" {
                title := fp.scrapeTitle(resp)
                item.Title = title
            } else {
                log.Errorf(fp.c, "Wrong content type.  Received: %v from %v", resp.Header.Get("Content-Type"), item.Link.Href)
            }
            out <- item
        }(item)
    }
    log.Infof(fp.c, "Waiting for title pull wait group.")
    innerWait.Wait()
    log.Infof(fp.c, "Done waiting for title pull.")
    close(out)
}

func (fp FeedProducer) scrapeTitle(request *http.Response) string {
    defer request.Body.Close()
    tokenizer := html.NewTokenizer(request.Body)
    var titleIsNext bool
    for {
        token := tokenizer.Next()
        switch {
        case token == html.ErrorToken:
            log.Infof(fp.c, "Hit the end of the doc without finding title.")
            return ""
        case token == html.StartTagToken:
            tag := tokenizer.Token()
            isTitle := tag.Data == "title"

            if isTitle {
                titleIsNext = true
            }
        case titleIsNext && token == html.TextToken:
            title := tokenizer.Token().Data
            log.Infof(fp.c, "Pulled title: %v", title)
            return title
        }
    }
}

日志内容如下所示:

2015/08/09 22:02:10 INFO: Revived query parameter: golang
2015/08/09 22:02:10 INFO: Getting active tweets from the last 7 days.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Incrementing inner WaitGroup.
2015/08/09 22:02:10 INFO: Waiting for title pull wait group.
2015/08/09 22:02:10 INFO: Getting title for: http://devsisters.github.io/goquic/
2015/08/09 22:02:10 INFO: Pulled title: GoQuic by devsisters
2015/08/09 22:02:10 INFO: Getting title for: http://whizdumb.me/2015/03/03/matching-a-string-and-extracting-values-using-regex/
2015/08/09 22:02:10 INFO: Pulled title: Matching a string and extracting values using regex | Whizdumb's blog
2015/08/09 22:02:10 INFO: Getting title for: https://www.reddit.com/r/golang/comments/3g7tyv/dropboxs_infrastructure_is_go_at_a_huge_scale/
2015/08/09 22:02:10 INFO: Pulled title: Dropbox's infrastructure is Go at a huge scale : golang
2015/08/09 22:02:10 INFO: Getting title for: http://dave.cheney.net/2015/08/08/performance-without-the-event-loop
2015/08/09 22:02:10 INFO: Pulled title: Performance without the event loop | Dave Cheney
2015/08/09 22:02:11 INFO: Getting title for: https://github.com/ccirello/sublime-gosnippets
2015/08/09 22:02:11 INFO: Pulled title: ccirello/sublime-gosnippets · GitHub
2015/08/09 22:02:11 INFO: Getting title for: https://medium.com/iron-io-blog/an-easier-way-to-create-tiny-golang-docker-images-7ba2893b160?mkt_tok=3RkMMJWWfF9wsRonuqTMZKXonjHpfsX57ewoWaexlMI/0ER3fOvrPUfGjI4ATsNrI%2BSLDwEYGJlv6SgFQ7LMMaZq1rgMXBk%3D&utm_content=buffer45a1c&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
2015/08/09 22:02:11 INFO: Pulled title: An Easier Way to Create Tiny Golang Docker Images — Iron.io Blog — Medium

我可以看到我正在根据日志获取 innerWait.Wait() 命令，它还告诉我管道另一侧的入站 channel 已关闭。

匿名函数中的延迟语句似乎没有被调用，因为我看不到任何地方打印的延迟日志语句。但我终究无法说出原因，因为该 block 中的所有代码似乎都在执行。

感谢您的帮助。

Answer 1

goroutines 被困在这一行发送到 out:

        out <- item

修复方法是启动一个 goroutine 以在 out 上接收。

调试此类问题的一个好方法是通过向进程发送 SIGQUIT 来转储 goroutine 堆栈。