python - 无法关闭监听套接字以从单独的线程中止 accept()/select()

Question

我正在编写一个 Python (v3.7.3) 套接字服务器，我想为其使用阻塞 I/O。我使用 select() 没有超时来接受新客户以及从他们那里读取。我可以关闭监听套接字以中止 select()，并捕获 OSError 作为停止执行的指示。

但是，这在单独的线程中运行时似乎不起作用，我不明白为什么。

我知道还有其他方法可以完成此操作，例如使用超时、为 select() 使用虚拟套接字，或者与监听器建立虚拟连接以将其唤醒。但是这些都在某种程度上违背了使用 select() 的目的，并且在单线程中运行时不是必需的。

这是重现问题的基本示例，在我的实际代码中，它只代表许多线程中的一个(因此，我首先使用线程):

#!/usr/bin/env python3

import signal
import socket
import threading


class SocketCloseTest:
    """Simple test case for using socket.close() to abort select.select()"""

    def __init__(self, port, address=None):
        self.port = port
        self.address = address or ''

        self.socket = None

    def stop(self):
        """Close listening socket to stop select.select()"""

        if self.socket:
            print("Closing listener", self.socket)
            self.socket.close()
            print("Listener closed", self.socket)

    def threaded_run(self):
        """Run test in a separate thread"""

        thread = threading.Thread(target=self.run)
        print("Starting sub-thread")
        thread.start()
        thread.join()
        print("Sub-thread ended")

    def run(self):
        """Run test"""

        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        # Reuse port for quick re-launch of the application
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind((self.address, self.port))
        print("Starting listener")
        self.socket.listen()

        try:
            print("select() started")
            r, w, e = select.select([self.socket], [], [])
        except OSError:
            print("select() aborted")
        else:
            print("select() completed")


if __name__ == '__main__':
    tester = SocketCloseTest(5000, address='')

    # Set up signal handler for Ctrl-C
    def signal_handler(signum, frame):
        print("Received signal {}".format(signum))
        tester.stop()
    signal.signal(signal.SIGINT, signal_handler)

    # This works
    tester.run()

    # This doesn't work
    # tester.threaded_run()

    print("Main thread ended")

当使用 test.run() 时，它按预期运行并产生以下结果:

Starting listener
Select started
^CReceived signal 2
Closing listener <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 5000)>
Listener closed <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>
Select cancelled
Main thread ended

但是，当使用 tester.threaded_run() 运行时，它只是卡在对 select() 的调用应该中止的位置。奇怪的是，此时将作业置于后台会导致代码继续执行:

Starting sub-thread
Starting listener
Select started
^CReceived signal 2
Closing listener <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 5000)>
Listener closed <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>
--Ctrl-Z pressed here to suspend job in shell--

$ bg
--Shell reports job in background--
Select cancelled
Sub-thread ended
Main thread ended

谢谢...

• 编辑提到 accept() 有完全相同的症状。

Answer 1

close() 在多线程情况下不会执行您想要的操作。请改用您描述的其他机制之一。

在单线程情况下，控制权返回到 select()，它会重新启动并注意到现在已关闭的文件描述符上的 EBADF。 (当然，这是非常危险的，因为 fd #3 可能随时被任何其他线程甚至复杂的信号处理程序回收，尽管您的玩具程序看起来很安全。)在多线程情况下， close() 不会唤醒您的 select()ing 线程。

Python docs warn :

Note: close() releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call shutdown() before close().

其实这是一个比较棘手的和平台相关的问题。摘录 2008 article in the venerable Dr Dobb's :

On some operating systems, [ shutdown() instead of close() ] is also the only working solution: On FreeBSD, a close() without shutdown() does not wake up the processes waiting in a read() or select()....

Another issue to consider is that closing by shutdown() or by close() may not be considered a read event in the OS....

However, shutdown() works only on the sockets with established connections, not on the ones listening for new connections nor on the other kinds of file descriptors....

(就其值(value)而言，在我的系统上，shutdown() 确实唤醒了 select()ing 线程。)