c++ - ZeroMQ REQ/REP 性能

Question

考虑以下两个过程:

发件人.cpp:

#include <zhelpers.h>
...
zmq::context_t ctx(1);
...
void foo(int i)
{
   zmq::socket_t sender(ctx, ZMQ_REQ);
   sender.connect("tcp://hostname:5000");

   std::stringstream ss;
   ss <<"bar_" <<i;
   std::string bar_i(std::move(ss.str());

   s_sendmore(sender, "foo ");
   (i != N) ? s_send(sender, bar, 0) : s_send(sender, "done", 0);
   s_recv(sender);
}

int main()
{
    for(int i=0; i<=100000; ++i)
        foo(i);
    return 0;
}

接收器.cpp

#include <zhelpers.h>
...
int main()
{
    zmq::context_t ctx(1);
    zmq::socket_t rcv(ctx, ZMQ_REP);
    rcv.bind("tcp://*:5000");

    std::string s1("");
    std::string s2("");

    while(s2 != "done")
    {
        s1 = std::move(s_recv(rcv));
        s2 = std::move(s_recv(rcv));
        std::cout <<"received: " <<s1 <<" " <<s2 <<"\n";
        s_send(rcv, "ACK");
    }

    return 0;
}

让我们开始这两个过程。我期望的是接收进程将接收发送方发送给它的所有消息并打印出来:

foo bar_1
foo bar_2
...

依此类推，直到:

...
foo bar_100000

而且我希望它会在没有任何阻塞的情况下执行此操作。

我的问题是接收器总是停留在第 28215 次迭代附近(总是在那个数字附近!!!)并阻塞到一分钟左右。然后它会更进一步直到 100000，但有时它会再次粘住。我的问题当然是:为什么会这样？我该如何解决？

我尝试将“发送者”放在全局范围内的 foo(.) 中，然后它起作用了:在那种情况下，所有打印输出都从 1 平滑且超快地进行，没有任何阻塞(当然在这种情况下，每次调用 foo(.) 时都不会创建套接字)。但不幸的是，在我的代码中我不能这样做。

我想了解为什么会出现此 block 。

Answer 1

首先，您的示例不太可靠，因为它们无法编译。所以这里有一些应该接近您的意图并实际编译的示例

发件人.cpp

#include <zmq.hpp>
#include <string>
#include <iostream>
#include <string>

void send(const std::string& msg)
{
    //  Prepare our context and socket
    zmq::context_t context (1);
    zmq::socket_t socket (context, ZMQ_REQ);

    std::cout << "Connecting to receiver ..." << std::endl;
    socket.connect ("tcp://localhost:5555");

    zmq::message_t request (100);
    memcpy (request.data (), msg.c_str(), 100);
    std::cout << "Sending message " << msg << "..." << std::endl;
    socket.send (request);
}

int main ()
{
    for(int i = 0; i < 100000; ++i)
    {
        send(std::to_string(i));
    }
    send("done");
}

使用链接

g++ -std=c++11 -I/home/dev/cppzmq -I/home/dev/libzmq/include sender.cpp -lzmq -o sender

接收器.cpp

#include <zmq.hpp>
#include <string>
#include <cstring>
#include <iostream>

int main () {
    //  Prepare our context and socket
    zmq::context_t context (1);
    zmq::socket_t socket (context, ZMQ_REP);
    socket.bind ("tcp://*:5555");

    char buf[100] = {0};
    while (std::string(buf).compare("done")) {
        zmq::message_t request;

        //  Wait for next request from client
        socket.recv (&request);
        std::memcpy(buf, request.data(), 100);
        std::cout << "Received message " << buf << std::endl;

        //  Send reply back to client
        zmq::message_t reply (5);
        memcpy (reply.data (), "Hello", 5);
        socket.send (reply);
    }
    return 0;
}

使用

g++ -std=c++11 -I/home/dev/cppzmq -I/home/dev/libzmq/include receiver.cpp -lzmq -o receiver

启动流程时，一切似乎都正常，接收器上的输出符合预期，没有中断:

Received message 99996
Received message 99997
Received message 99998
Received message 99999
Received message done

但如我所料:看看 netstat:

netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 localhost:38345         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:46228         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:60309         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:46916         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:47600         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:54454         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:46409         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:51142         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:40355         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:40005         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:45614         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:48974         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:41427         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:58740         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:58754         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:60044         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:57478         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:50419         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:44361         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:37284         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:38662         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:45968         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:57407         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:59200         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:41292         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:55243         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:51489         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:48865         localhost:5555          TIME_WAIT  
tcp        0      0 localhost:35491         localhost:5555          TIME_WAIT 
...

在一次运行后，我有超过 20k (!) 个这样的套接字处于 TIME_WAIT 状态。这是因为sender.cpp中void send(...)中socket的变量作用域。我不知道 zmq 在超出范围时销毁套接字时到底做了什么，但我很确定它会在某个地方调用套接字的 fd 上的 close() ，这将使你的套接字进入这个 TIME_WAIT 状态。即使我的发送器和接收器运行顺利，我也不知道您的系统如何处理这么多套接字。另外，我不知道您的 zhelpers.h 文件是做什么的。但我知道，如果您将套接字置于全局范围内，则在一个套接字的发送端只会发生一次 close() 调用。我从这里开始调查更多。也许，看看 how-to-forcibly-close-a-socket-in-time-wait ...