python - pyzmq 上下文有时初始化很慢

Question

我最近开始使用 ZeroMQ 并观察到 ​​有时，ZeroMQ 上下文的初始化可能需要多达 40 秒 .通常它比这快得多。

例如:

import zmq

# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
    # connect / bind to socket, do stuff
    pass

这种行为似乎与我是否使用上下文管理器无关，例如以下行为与上述相同:

import zmq

# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()

# connect / bind to socket, do stuff

context.term()

这是在 Ubuntu 18.04.2 上的 Python 3.6.8 中运行的

我一直无法建立任何放缓的模式。

这种行为的可能原因是什么？

更新

根据@ user3666197 的建议，我进行了一些计时，并设法在第二次运行时表现出减速。在那之后，事情恢复了“正常”。

测试脚本的第二次运行开始放缓:

 = zmq.Context()] took +      704 [us] to execute
 1.~                          704  us
 2.~                  131 766 816  us
 3.~                          145  us
 4.~                          137  us

在 htop 上观看内存当时根本没有显示内存已满，但交换已经处于事件状态。

更新 2

使用@user3666197 建议脚本的精简版本:

import sys

import zmq
clock = zmq.Stopwatch()

clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()

并在循环中调用它，例如:

for x in $(seq 1 20);
do python3 test.zmq.py $x;
done

我可以在原服务器上重复这个问题 但还没有在另一台服务器上击中它 ，这表明存在环境问题(或与特定发行版和/或 python 版本有关，因为“好”服务器较旧)。

目前这对我来说已经足够了，但我稍后会再回来并收集足够的信息以向维护者提交一些内容。

Answer 1

如果您从未使用过 ZeroMQ，可以在这里先看看 "ZeroMQ Principles in less than Five Seconds"在深入了解更多细节之前

Q : " What are the possible causes of this _{( only "sometimes" present )} behaviour? "
_{I haven't been able to establish any pattern to the slowdowns.}

) 上面提到的那 40 秒可能是有意义的，但是要使用 RAM 来交换 I/O 操作。我无法想象这种时间块的任何其他原因(在我使用 ZeroMQ 的最后大约 10 年中，从来没有注意到 ZeroMQ 中接近 3/4 分钟的延迟，从来没有)

b ) 在提出任何 claim 之前，现实检查总是衡量各自的实际执行时间:
让我们开始对可重复和记录在案的测试用例进行系统和公平，好吗？像“有时”这样的类别不值得，数据才是。首先，我们可以使用以下模板化方法来区分任何实际延迟的根本原因:

M = "[{0:_>40s}] took +{1:_>9d} [us] to execute"
N = 0
from zmq import Stopwatch; aClk = Stopwatch()
#------------------------------
pass;                      aClk.start()
import zmq
pass;                  _ = aClk.stop(); print( M.format( "IMPORT", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context()
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
context = zmq.Context.instance()
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context.instance()", _ ) )
#------------------------------
pass;                      aClk.start()
context.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
with zmq.Context.instance() as aCtxInCtxMGR:
     N +=1
pass;                  _ = aClk.stop(); print( M.format( "with … as aCtxInCtxMGR:", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context(  0 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context(  0 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 10 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 10 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 20 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 20 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 50 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 50 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 99 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 99 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------

等等
有理有据

您的实际里程可能会有所不同(并且将会)

[__________________________________IMPORT] took +_____1198 [us] to execute
[_________________________= zmq.Context()] took +_____1523 [us] to execute
[________________= zmq.Context.instance()] took +_____1348 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____7909 [us] to execute
[_____________________= zmq.Context(  0 )] took +______966 [us] to execute
[_____________________= zmq.Context( 10 )] took +______944 [us] to execute
[_____________________= zmq.Context( 20 )] took +______936 [us] to execute
[_____________________= zmq.Context( 50 )] took +______962 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1015 [us] to execute

[__________________________________IMPORT] took +______879 [us] to execute
[_________________________= zmq.Context()] took +_____1076 [us] to execute
[_________________________________.term()] took +______881 [us] to execute
[________________= zmq.Context.instance()] took +_____1105 [us] to execute
[_________________________________.term()] took +______896 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____1474 [us] to execute
[_____________________= zmq.Context(  0 )] took +_____1163 [us] to execute
[_________________________________.term()] took +______982 [us] to execute
[_____________________= zmq.Context( 10 )] took +_____2646 [us] to execute
[_________________________________.term()] took +______873 [us] to execute
[_____________________= zmq.Context( 20 )] took +_____2689 [us] to execute
[_________________________________.term()] took +______981 [us] to execute
[_____________________= zmq.Context( 50 )] took +_____2772 [us] to execute
[_________________________________.term()] took +______910 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1086 [us] to execute
[_________________________________.term()] took +_____1010 [us] to execute