c++ - 使用 boost::beast 处理大型 http 响应

Question

以下代码用于获取http响应消息:

  boost::beast::tcp_stream stream_;

  boost::beast::flat_buffer buffer;
  boost::beast::http::response<boost::beast::http::dynamic_body> res;
  boost::beast::http::read(stream_, buffer, res);

但是，在某些情况下，根据前面的请求，我可以预期响应消息正文将包含大型二进制文件。

因此，我想直接将其读取到文件系统，而不是通过 buffer 变量，以避免过度使用进程内存。怎么办？

在 Objective-c 框架 NSUrlSession 中，有一种简单的方法可以使用 NSURLSessionDownloadTask 而不是 NSURLSessionDataTask 来完成此操作，所以我想知道它是否也存在正在插入。

谢谢!

Answer 1

一般情况下，您可以使用 http::buffer_body处理任意大的请求/响应消息。

如果您特别想从文件系统文件中读取/写入，则可以使用 http::file_body 来代替。

完整演示buffer_body

buffer_body 的文档示例位于此处 https://www.boost.org/doc/libs/1_77_0/libs/beast/doc/html/beast/using_http/parser_stream_operations/incremental_read.html .

用它写入 std::cout: Live On Coliru

#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>

namespace net       = boost::asio;
namespace beast     = boost::beast;
namespace http      = beast::http;
using tcp           = net::ip::tcp;
using socket_t      = tcp::socket;

/*  This function reads a message using a fixed size buffer to hold
    portions of the body, and prints the body contents to a `std::ostream`.
*/

template<
    bool isRequest,
    class SyncReadStream,
    class DynamicBuffer>
void
read_and_print_body(
    std::ostream& os,
    SyncReadStream& stream,
    DynamicBuffer& buffer,
    beast::error_code& ec)
{
    http::parser<isRequest, http::buffer_body> p;
    http::read_header(stream, buffer, p, ec);
    if(ec)
        return;
    while(! p.is_done())
    {
        char buf[512];
        p.get().body().data = buf;
        p.get().body().size = sizeof(buf);
        http::read(stream, buffer, p, ec);
        if(ec == http::error::need_buffer)
            ec = {};
        if(ec)
            return;
        os.write(buf, sizeof(buf) - p.get().body().size);
    }
}

int main() {
    std::string host = "173.203.57.63"; // COLIRU 20210901
    auto const port  = "80";

    net::io_context ioc;
    tcp::resolver   resolver{ioc};

    socket_t s{ioc};
    net::connect(s, resolver.resolve(host, port));

    write(s, http::request<http::empty_body>{http::verb::get, "/", 11});

    beast::error_code  ec;
    beast::flat_buffer buf;

    read_and_print_body<false>(std::cout, s, buf, ec);
}

完整的file_body示例

这要短得多，写入body.html:

<强> Live On Coliru

#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>

namespace net       = boost::asio;
namespace beast     = boost::beast;
namespace http      = beast::http;
using tcp           = net::ip::tcp;
using socket_t      = tcp::socket;

int main() {
    std::string host = "173.203.57.63"; // COLIRU 20210901
    auto const port  = "80";

    net::io_context ioc;
    tcp::resolver   resolver{ioc};

    socket_t s{ioc};
    net::connect(s, resolver.resolve(host, port));

    write(s, http::request<http::empty_body>{http::verb::get, "/", 11});

    beast::error_code  ec;
    beast::flat_buffer buf;
    http::response<http::file_body> res;
    res.body().open("body.html", beast::file_mode::write_new, ec);
    if (!ec.failed())
    {
        read(s, buf, res, ec);
    }

    std::cout << "Wrote 'body.html' (" << ec.message() << ")\n";
    std::cout << "Headers " << res.base() << "\n";
}

打印

Wrote 'body.html' (Success)
Headers HTTP/1.1 200 OK 
Content-Type: text/html;charset=utf-8
Content-Length: 8616
Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.0.2g
Date: Wed, 01 Sep 2021 19:52:20 GMT
Connection: Keep-Alive

与文件body.html； wc body.html 显示:

body.html: HTML document, ASCII text, with very long lines
 185  644 8616 body.html

超越:流式传输到子进程和流式处理

我在这里有一个高级示例:How to read data from Internet using muli-threading with connecting only once? .