python - boost python : tie lifetime of argument to returned value using return_internal_reference

Question

我开始学习使用 boost python 并有一个菜鸟问题。

我想编写一个函数，可以将其参数的生命周期与其结果联系起来，这样当我调用 r = func(a) 时, 参数 a如果我还有 r 的引用，就永远不会被销毁.文档建议使用 return_internal_reference此类请求的调用策略。但这是否需要 r成为a的内部引用, 正如名字所暗示的那样？

在下面的(过度简化的)示例中，假设我想绑定(bind)输入数组 a 的生命周期使用生成的 lambda 函数，它不是输入的内部引用 a .

#include <functional>
#include <boost/python.hpp>
#include <boost/python/return_internal_reference.hpp>

using namespace std;
using namespace boost::python;

function<float(int)> func(const float* a) {
  return [=](int n) { return a[n]; };
}

BOOST_PYTHON_MODULE(test) {
  def("func", func, return_internal_reference<1>());
}

我希望能够在python中做到以下几点:

f = func(a)   # 'a' can be a temporary variable, say returned by another function
f(5)          # but 'a' should not be destroyed at this step, 
              # because its lifetime is tied to 'f'

当我尝试编译上面的代码时，我遇到了下面列出的一大堆错误，但是如果我删除 return_internal_reference<1>()调用策略，代码编译成功。

我很确定我错误地使用了这个调用策略，但我不确定如何正确使用它。任何指针将不胜感激。非常感谢!

$ g++ -std=c++11 -shared Test.cc -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -L/opt/local/lib -lboost_python-mt -lpython2.7 -o test.so
In file included from /opt/local/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:52:0,
                 from /opt/local/include/boost/python/detail/invoke.hpp:63,
                 from /opt/local/include/boost/python/detail/caller.hpp:16,
                 from /opt/local/include/boost/python/object/function_handle.hpp:8,
                 from /opt/local/include/boost/python/converter/arg_to_python.hpp:19,
                 from /opt/local/include/boost/python/call.hpp:15,
                 from /opt/local/include/boost/python/object_core.hpp:14,
                 from /opt/local/include/boost/python/args.hpp:25,
                 from /opt/local/include/boost/python.hpp:11,
                 from Test.cc:2:
/opt/local/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_<false, false>, const RC&, F&, AC0&) [with RC = boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >; F = std::function<float(int)> (*)(const float*); AC0 = boost::python::arg_from_python<const float*>; PyObject = _object]':
/opt/local/include/boost/python/detail/caller.hpp:223:13:   required from 'PyObject* boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::operator()(PyObject*, PyObject*) [with F = std::function<float(int)> (*)(const float*); Policies = boost::python::return_internal_reference<>; Sig = boost::mpl::vector2<std::function<float(int)>, const float*>; PyObject = _object]'
/opt/local/include/boost/python/object/py_function.hpp:38:33:   required from 'PyObject* boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller<std::function<float(int)> (*)(const float*), boost::python::return_internal_reference<>, boost::mpl::vector2<std::function<float(int)>, const float*> >; PyObject = _object]'
Test.cc:14:1:   required from here
/opt/local/include/boost/python/detail/invoke.hpp:75:82: error: no match for call to '(const boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >) (std::function<float(int)>)'
     return rc(f( BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT) ));
                                                                                  ^
In file included from /opt/local/include/boost/python/object/function_handle.hpp:8:0,
                 from /opt/local/include/boost/python/converter/arg_to_python.hpp:19,
                 from /opt/local/include/boost/python/call.hpp:15,
                 from /opt/local/include/boost/python/object_core.hpp:14,
                 from /opt/local/include/boost/python/args.hpp:25,
                 from /opt/local/include/boost/python.hpp:11,
                 from Test.cc:2:
/opt/local/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject* boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter = boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >; PyTypeObject = _typeobject]':
/opt/local/include/boost/python/detail/caller.hpp:240:19:   required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::signature() [with F = std::function<float(int)> (*)(const float*); Policies = boost::python::return_internal_reference<>; Sig = boost::mpl::vector2<std::function<float(int)>, const float*>]'
/opt/local/include/boost/python/object/py_function.hpp:48:35:   required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller<std::function<float(int)> (*)(const float*), boost::python::return_internal_reference<>, boost::mpl::vector2<std::function<float(int)>, const float*> >]'
Test.cc:14:1:   required from here
/opt/local/include/boost/python/detail/caller.hpp:102:109: error: 'struct boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >' has no member named 'get_pytype'
         return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype();
                                                                                                             ^

Answer 1

考虑使用 with_custodian_and_ward_postcall CallPolicy .此策略允许按值返回返回类型，同时仍将另一个对象的生命周期延长到至少与返回对象的生命周期一样长。

BOOST_PYTHON_MODULE(test) {
  def("func", func, with_custodian_and_ward_postcall<0, 1>());
}

---

如 return_internal_reference 中所述文档中，返回的对象引用了现有的内部对象:

return_internal_reference [...] allow pointers and references to objects held internally [...] to be returned safely without making a copy of the referent.

该文档还简要提到了它对 with_custodian_and_ward_postcall 的使用。总之，return_internal_reference 对公开的函数有两个值得注意的影响:

• 返回的 Python 对象既不具有引用的 C++ 对象的显式所有权，也不具有共享所有权。
• 返回的 Python 对象是一个保管人，并将延长由 owner_arg 指示的 ward 对象的生命周期，至少与保管人一样长。

由于返回的 Python 对象指的是既没有显式所有权也没有共享所有权的现有对象，因此 Boost.Python 执行类型检查以防止创建悬空引用。在示例代码中，func() 按值返回一个仿函数，导致编译器错误提示返回类型必须是指针或引用:

struct boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type

To explicitly control the lifetime of the returned object, one should consider using return_value_policy and models of ResultConverterGenerators. For example, if func() returned a functor by pointer that was created by new(), and wished to pass ownership of the object to Python while still maintaining the custodian and ward relationship, then one could chain policies with policy composition:

BOOST_PYTHON_MODULE(test) {
  def("func", func, 
    return_value_policy<manage_new_object,
      with_custodian_and_ward_postcall<0, 1> >());
}

---

这是一个完整的最小示例，它基于原始代码，详细输出到 demonstrate。 with_custodian_and_ward_postcall 行为:

#include <boost/python.hpp>
#include <iostream>

/// @brief Mockup class with verbose construction and destruction.
class foo
{
public:
  foo() { std::cout << "foo() " << this << std::endl; }
  foo(const foo&) { std::cout << "foo(const foo&) " << this << std::endl; }
  ~foo() { std::cout << "~foo() " << this << std::endl; }
};

/// @brief Mockup class with verbose construction and destruction.
class bar
{
public:
  bar() { std::cout << "bar() " << this << std::endl; }
  bar(const bar&) { std::cout << "bar(const bar&) " << this << std::endl; }
  ~bar() { std::cout << "~bar() " << this << std::endl; }
};

/// @brief Mockup factory function.
foo make_foo(bar& /* unused */)
{
  return foo();
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  // Do not allow Foo to be explicitly created from its type.
  python::class_<foo>("Foo", python::no_init);
  python::class_<bar>("Bar", python::init<>());

  // Expose make_foo, that returns a foo object when provided a
  // bar object.  The bar object's lifetime will be extended to
  // be at least as long as that of the returned foo object.
  python::def("make_foo", &make_foo,
    python::with_custodian_and_ward_postcall<
      0, // custodian = returned Foo object
      1  // ward = provided Bar object
    >());
}

交互使用:

>>> import example
>>> bar = example.Bar()
bar() 0x125ac30
>>> foo = example.make_foo(bar)
foo() 0x7fffa9b5efff
foo(const foo&) 0x7f1fcbe40090
~foo() 0x7fffa9b5efff
>>> bar = None
>>> foo = None
~foo() 0x7f1fcbe40090
~bar() 0x125ac30

请注意，尽管 bar 变量被设置为 None，实际的 ward 对象仍然存在，直到 foo 保管人被销毁。