python - Twisted webapp 在生成 HTTP 响应时卡住

Question

我使用 Twisted 和 SQLAlchemy 创建了一个 Web 应用程序。由于 SQLAlchemy 与 Twisted 基于回调的设计 ( Twisted + SQLAlchemy and the best way to do it ) 不能很好地协同工作，因此我在根资源中使用 deferToThread() 以便在其自己的线程中运行每个请求。虽然这通常有效，但大约 10% 的请求会“卡住”。这意味着当我在浏览器中单击一个链接时，请求由 Twisted 处理，相应资源的代码运行并生成 HTML 输出。但无论出于何种原因，该输出永远不会发送回浏览器。相反，Twisted 发送 HTTP header (连同正确的 Content-Length)，但从不发送正文。连接会无限期地保持打开状态，浏览器会显示微调器图标。 Twisted 在日志文件中没有生成任何错误。

下面是一个最小的例子。如果您想运行它，请将其保存为 .tac 扩展名，然后运行 ​​twistd -noy example.tac。在我的服务器上，这个问题似乎很少出现在这段特定的代码中。使用类似 while true;做 wget -O- 'http://server.example.com:8080' >/dev/null;完成来测试它。

from twisted.web.server import Site
from twisted.application import service, internet
from twisted.web.resource import Resource
from twisted.internet import threads
from twisted.web.server import NOT_DONE_YET
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine, Column, Integer, String

Base = declarative_base()
class User(Base):
    '''A user account.'''
    __tablename__ = 'user'
    id = Column(Integer, primary_key=True)
    login = Column(String(64))


class WebInterface(Resource):

    def __init__(self):
        Resource.__init__(self)
        db_url = "mysql://user:password@mysql-server.example.com/myapp?charset=utf8"
        db_engine = create_engine(db_url, echo=False, pool_recycle=300) #discard connections after 300 seconds
        self.DBSession = sessionmaker(bind=db_engine)

    def on_request_done(self, _, request):
        '''All actions that need to be done after a request has been successfully handled.'''
        request.db_session.close()
        print('Session closed') #does get printed, so session should get closed properly


    def on_request_failed(self, err, call):
        '''What happens if the request failed on a network level, for example because the user aborted the request'''
        call.cancel()


    def on_error(self, err, request):
        '''What happens if an exception occurs during processing of the request'''
        request.setResponseCode(500)
        self.on_request_done(None, request)
        request.finish()
        return err 


    def getChild(self, name, request):
        '''We dispatch all requests to ourselves in order to be able to do the processing in separate threads'''
        return self


    def render(self, request):
        '''Dispatch the real work to a thread'''
        d = threads.deferToThread(self.do_work, request)
        d.addCallbacks(self.on_request_done, errback=self.on_error, callbackArgs=[request], errbackArgs=[request])
        #If the client aborts the request, we need to cancel it to avoid error messages from twisted
        request.notifyFinish().addErrback(self.on_request_failed, d)
        return NOT_DONE_YET


    def do_work(self, request):
        '''This method runs in thread context.'''
        db_session = self.DBSession()
        request.db_session = db_session
        user = db_session.query(User).first()

        body = 'Hello, {} '.format(user.login) * 1024 #generate some output data
        request.write(body)
        request.finish()


application = service.Application("My Testapp")
s = internet.TCPServer(8080, Site(WebInterface()), interface='0.0.0.0')
s.setServiceParent(application)

Answer 1

您可能没有使用 SQLAlchemy 关闭数据库连接或数据库中的某些死锁情况？我之前因未关闭连接/未结束交易而被 flask 锁定。