c - ZeroMQ - 模拟多个客户端到一台服务器的标准套接字

Question

我希望利用 ZeroMQ 来处理同时传入证明服务器的大量请求的排队问题。

我处于 HPC 环境中，因此有大量计算节点都运行与服务器进行验证的相同启动程序。该服务器代码在前端节点上运行，在成功验证客户端后，将释放 key 以便客户端解密作业数据。

当前使用标准套接字。当客户端最初向服务器发送某些内容时，会使用 sys/socket.h 中的 accept() 生成一个新套接字。这允许客户端和服务器在证明过程中相互发送多条消息(校验和等)，然后在成功返回 key 之前。

ZeroMQ 的问题是附加命令不是必需的，因此不会为该特定证明创建辅助套接字。所有来自所有客户端的消息都按其进入的顺序进行处理，导致多部分证明过程无法正常工作。我花了很长时间阅读指南并通过谷歌搜索来尝试找到类似的解决方案，但到目前为止还没有任何运气。

有没有一种方法可以利用 ZeroMQ 在此应用程序中提供与标准套接字相同的行为？

Answer 1

Q : Is there a way I can utilise ZeroMQ to give the same behaviour in this application as a standard socket?

ZeroMQ 是 very smart and rather behaviour-oriented distributed-systems 的信令/消息传递平台，不是套接字。

考虑到您的意图是对 HPC 生态系统有值(value)，解决方案假设/指导使用某种工具，并且必须尽可能地弯曲它，这样它就会变得接近于类似于 native 的行为，但是对于其他工具，似乎不是典型的 HPC 级方法。

HPC 代码通常经过精心设计，以便提高计算成本效率(并祝福所有那些老板、首席财务官和政府/军事资助人员，他们今天被允许不为以下目的设计 HPC 代码)最终的性能和硬件资源的使用效率 :o) ) - 在这里，如果有人在 ZeroMQ 实例化上支付费用，那么这些实例化的非零成本和“只是”一个套接字一样似乎没有任何好处从成本上看，这种行为具有负面的性能 yield ，并且对智能、集群范围的 ZeroMQ 服务(无论是 N+1 或 N+M 冗余、低延迟智能节点间集群信令、密码学、廉价的安全驱动的白名单，或任何可能代表任何额外的 HPC 级项目利益的东西，这可能证明初始 ZeroMQ 实例化的成本是合理的)。

<小时/>

ZMQ_STREAM 的定义原型(prototype)可能提供一些工具，但是，引用。上面

A socket of type ZMQ_STREAM is used to send and receive TCP data from a non-ØMQ peer, when using the tcp:// transport. A ZMQ_STREAM socket can act as client and/or server, sending and/or receiving TCP data asynchronously.

When receiving TCP data, a ZMQ_STREAM socket shall prepend a message part containing the identity of the originating peer to the message before passing it to the application. Messages received are fair-queued from among all connected peers.

When sending TCP data, a ZMQ_STREAM socket shall remove the first part of the message and use it to determine the identity of the peer the message shall be routed to, and unroutable messages shall cause an EHOSTUNREACH or EAGAIN error.

To open a connection to a server, use the zmq_connect call, and then fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.

To close a specific connection, send the identity frame followed by a zero-length message (see EXAMPLE section).

When a connection is made, a zero-length message will be received by the application. Similarly, when the peer disconnects (or the connection is lost), a zero-length message will be received by the application.

You must send one identity frame followed by one data frame. The ZMQ_SNDMORE flag is required for identity frames but is ignored on data frames.

ZMQ_STREAM 示例:

void *ctx = zmq_ctx_new ();                          assert (ctx     && "Context Instantiation Failed..." );
void *socket = zmq_socket (ctx, ZMQ_STREAM);         assert (socket  && "socket Instantiation Failed..." );
int rc = zmq_bind (socket, "tcp://*:8080");          assert (rc == 0 && "socket.bind() Failed..." );
uint8_t id [256];                                 /* Data structure to hold the ZMQ_STREAM ID */
size_t id_size = 256;
uint8_t raw [256];                                /* Data structure to hold the ZMQ_STREAM received data */
size_t raw_size = 256;
while (1) {
   id_size = zmq_recv (socket, id, 256, 0);          assert (id_size > 0 && "Get HTTP request; ID frame and then request; Failed..." )
   do {
        raw_size = zmq_recv (socket, raw, 256, 0);   assert (raw_size >= 0 && "socket.recv() Failed..." );
   } while (raw_size == 256);

   char http_response [] =                        /* Prepares the response */
            "HTTP/1.0 200 OK\r\n"
            "Content-Type: text/plain\r\n"
            "\r\n"
            "Hello, World!";
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Sends the ID frame followed by the response */
   zmq_send (socket, http_response, strlen (http_response), 0);
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Closes the connection by sending the ID frame followed by a zero response */
   zmq_send (socket, 0, 0, 0);
}
zmq_close (socket);
zmq_ctx_destroy (ctx);

<小时/>

ZeroMQ zmq_getsockopt() 可以提供 POSIX/SOCKET 描述符，用于低级技巧

The ZMQ_FD option shall retrieve the file descriptor associated with the specified socket. The returned file descriptor can be used to integrate the socket into an existing event loop; the ØMQ library shall signal any pending events on the socket in an edge-triggered fashion by making the file descriptor become ready for reading.

The ability to read from the returned file descriptor does not necessarily indicate that messages are available to be read from, or can be written to, the underlying socket; applications must retrieve the actual event state with a subsequent retrieval of the ZMQ_EVENTS option.

The returned file descriptor is also used internally by the zmq_send and zmq_recv functions. As the descriptor is edge triggered, applications must update the state of ZMQ_EVENTS after each invocation of zmq_send or zmq_recv.

To be more explicit: after calling zmq_send the socket may become readable (and vice versa) without triggering a read event on the file descriptor.

The returned file descriptor is intended for use with a poll or similar system call only. Applications must never attempt to read or write data to it directly, neither should they try to close it.

Option value type: int on POSIX systems, SOCKET on Windows

<小时/>

了解更多details on ZeroMQ tricks人们可能会喜欢阅读此处已经讨论过的解决方案、性能基准、延迟削减详细信息以及其他解决问题的技巧。