rust - 东京 curl : capture output into a local `Vec` - may outlive borrowed value

Question

我对 Rust 的了解还不够深，无法理解生命周期和闭包...

尝试使用 tokio-curl 将下载的数据收集到向量中:

extern crate curl;
extern crate futures;
extern crate tokio_core;
extern crate tokio_curl;

use std::io::{self, Write};
use std::str;

use curl::easy::Easy;
use tokio_core::reactor::Core;
use tokio_curl::Session;

fn main() {
    // Create an event loop that we'll run on, as well as an HTTP `Session`
    // which we'll be routing all requests through.
    let mut lp = Core::new().unwrap();
    let mut out = Vec::new();
    let session = Session::new(lp.handle());

    // Prepare the HTTP request to be sent.
    let mut req = Easy::new();
    req.get(true).unwrap();
    req.url("https://www.rust-lang.org").unwrap();
    req.write_function(|data| {
            out.extend_from_slice(data);
            io::stdout().write_all(data).unwrap();
            Ok(data.len())
        })
        .unwrap();

    // Once we've got our session, issue an HTTP request to download the
    // rust-lang home page
    let request = session.perform(req);

    // Execute the request, and print the response code as well as the error
    // that happened (if any).
    let mut req = lp.run(request).unwrap();
    println!("{:?}", req.response_code());
    println!("out: {}", str::from_utf8(&out).unwrap());
} 

产生错误:

error[E0373]: closure may outlive the current function, but it borrows `out`, which is owned by the current function
  --> src/main.rs:25:24
   |
25 |     req.write_function(|data| {
   |                        ^^^^^^ may outlive borrowed value `out`
26 |             out.extend_from_slice(data);
   |             --- `out` is borrowed here
   |
help: to force the closure to take ownership of `out` (and any other referenced variables), use the `move` keyword, as shown:
   |     req.write_function(move |data| {

---

进一步调查，我发现 Easy::write_function 需要 'static 生命周期，但是如何从 curl-rust 文档收集输出的示例使用 Transfer::write_function 改为:

use curl::easy::Easy;

let mut data = Vec::new();
let mut handle = Easy::new();
handle.url("https://www.rust-lang.org/").unwrap();
{
    let mut transfer = handle.transfer();
    transfer.write_function(|new_data| {
        data.extend_from_slice(new_data);
        Ok(new_data.len())
    }).unwrap();
    transfer.perform().unwrap();
}
println!("{:?}", data);

Transfer::write_function does not require the 'static lifetime :

impl<'easy, 'data> Transfer<'easy, 'data> {
    /// Same as `Easy::write_function`, just takes a non `'static` lifetime
    /// corresponding to the lifetime of this transfer.
    pub fn write_function<F>(&mut self, f: F) -> Result<(), Error>
        where F: FnMut(&[u8]) -> Result<usize, WriteError> + 'data
    {
...

但我不能在 tokio-curl 的 Session::perform 上使用 Transfer 实例，因为它 requires the Easy type :

pub fn perform(&self, handle: Easy) -> Perform {

transfer.easy 是一个私有(private)字段，直接传递给 session.perform。

这是 tokio-curl 的问题吗？也许它应该将 transfer.easy 字段标记为 public 或实现像 perform_transfer 这样的新功能？有没有另一种方法可以在每次传输时使用 tokio-curl 收集输出？

Answer 1

使用 futures 库时，您必须了解的第一件事是您无法控制代码将在哪个线程上运行。

此外，curl 的文档 Easy::write_function说:

Note that the lifetime bound on this function is 'static, but that is often too restrictive. To use stack data consider calling the transfer method and then using write_function to configure a callback that can reference stack-local data.

最直接的解决方案是使用某种类型的锁定原语来确保一次只有一个线程可以访问向量。您还必须在主线程和闭包之间共享向量的所有权:

use std::sync::Mutex;
use std::sync::Arc;

let out = Arc::new(Mutex::new(Vec::new()));
let out_closure = out.clone();

// ...

req.write_function(move |data| {
    let mut out = out_closure.lock().expect("Unable to lock output");
    // ...
}).expect("Cannot set writing function");

// ...

let out = out.lock().expect("Unable to lock output");
println!("out: {}", str::from_utf8(&out).expect("Data was not UTF-8"));

不幸的是，tokio-curl 库目前不支持使用允许基于堆栈的数据的 Transfer 类型。