python - asyncore python 挂起

Question

我尝试使用 asyncore 做简单的异步 http 客户端:这段代码工作正常，输出是(足够快):

www.gmail.com  :  recv http code:  301
www.yandex.ru  :  recv http code:  200
www.python.org  :  recv http code:  200
www.google.ru  :  recv http code:  200
www.gravatar.com  :  recv http code:  302
www.com.com  :  recv http code:  302
www.yahoo.com  :  recv http code:  302
www.bom.com  :  recv http code:  301

但是我取消了主机不存在的行:

#c = AsyncHTTP('http://www.no-such-host.ru') #!this line breaks execution!

执行中断，代码挂起一段时间，输出部分数据并挂起，没有最后的数据输出:

connection error: [Errno -5] No address associated with hostname
www.gmail.com  :  recv http code:  301
www.yandex.ru  :  recv http code:  200
www.yahoo.com  :  recv http code:  302
www.com.com  :  recv http code:  302
www.bom.com  :  recv http code:  301
www.gravatar.com  :  recv http code:  302

...一些主机在这里丢失并在启动时延迟很长时间。

为什么会发生这种情况以及如何解决？

# coding=utf-8
import asyncore
import string, socket
import StringIO
import mimetools, urlparse

class AsyncHTTP(asyncore.dispatcher):
    # HTTP requestor

    def __init__(self, uri):
        asyncore.dispatcher.__init__(self)

        self.uri = uri


        # turn the uri into a valid request
        scheme, host, path, params, query, fragment = urlparse.urlparse(uri)
        assert scheme == "http", "only supports HTTP requests"
        try:
            host, port = string.split(host, ":", 1)
            port = int(port)
        except (TypeError, ValueError):
            port = 80 # default port
        if not path:
            path = "/"
        if params:
            path = path + ";" + params
        if query:
            path = path + "?" + query

        self.request = "GET %s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path, host)

        self.host = host
        self.port = port

        self.status = None
        self.header = None
        self.http_code = None
        self.data = ""

        # get things going!
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        #self.connect((host, port))
        #return
        try:
            self.connect((host, port))
        except Exception,e:
            self.close()
            self.handle_connect_expt(e)

    def handle_connect(self):
        self.send(self.request)

    def handle_expt(self):
        print "handle_expt error!"
        self.close()

    def handle_error(self):
        print "handle_error error!"
        self.close()

    def handle_connect_expt(self,expt):
        print "connection error:",expt

    def handle_code(self):        
        print self.host," : ","recv http code: ",self.http_code


    def handle_read(self):
        data = self.recv(2048)
        #print data
        if not self.header:
            self.data = self.data + data
            try:
                i = string.index(self.data, "\r\n\r\n")
            except ValueError:
                return # continue
            else:
                # parse header
                fp = StringIO.StringIO(self.data[:i+4])
                # status line is "HTTP/version status message"
                status = fp.readline()
                self.status = string.split(status, " ", 2)
                self.http_code = self.status[1]
                self.handle_code()      

                # followed by a rfc822-style message header
                self.header = mimetools.Message(fp)
                # followed by a newline, and the payload (if any)
                data = self.data[i+4:]
                self.data = ""
                #header recived
                #self.close()


    def handle_close(self):
        self.close()




c = AsyncHTTP('http://www.python.org')
c = AsyncHTTP('http://www.yandex.ru')
c = AsyncHTTP('http://www.google.ru')
c = AsyncHTTP('http://www.gmail.com')
c = AsyncHTTP('http://www.gravatar.com')
c = AsyncHTTP('http://www.yahoo.com')
c = AsyncHTTP('http://www.com.com')
c = AsyncHTTP('http://www.bom.com')
#c = AsyncHTTP('http://www.no-such-host.ru') #!this line breaks execution!
asyncore.loop()

ps:我的系统是ubuntu 11.10 + python 2.7.2

Answer 1

当您执行 self.connect((host, port)) 时，您会调用阻塞名称解析。结合您本地的DNS配置，这就是您的程序在启动时延迟很长的原因。

作为 asyncore 的替代品并弄清楚如何自己进行非阻塞名称解析，您可能会考虑使用 Twisted。 Twisted 的 TCP 连接设置 API(主要是 reactor.connectTCP 或构建在它之上的 API 之一)不会阻塞。因此，天真的使用它会保持适当的异步。