c++ - 尽管链接到库，但未定义对函数的引用

Question

我正在尝试构建开源项目 he-transformer在 Docker 容器中。这是一个构建在 ngraph 之上的工具, 通过 ngraph-bridge 连接使用 tensorflow，可以使用基于图的抽象来编写深度学习程序。 he-transformer 是 ngraph 的扩展，允许使用同态加密对加密数据进行推理。

我在 master 分支 ( 6d5a5b39 ) 上使用最新的 he-transformer 提交。构建过程抛出许多 undefined reference 错误:

#29 282.5 /usr/bin/clang++-9   -O2 -Wformat -Wformat-security -D_FORTIFY_SOURCE=2 -fstack-protector-all -Wall -march=native -Wno-deprecated-declarations -fopenmp=libomp  -O2 -Wformat -Wformat-security -D_FORTIFY_SOURCE=2 -fstack-protector-all -Wall -march=native -Wno-deprecated-declarations -g   -pie -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now CMakeFiles/unit-test.dir/test_sum_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_subtract_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_slice_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_softmax_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_server_client_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_reverse_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_result_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_reshape_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_relu_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_read_write_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_power_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_pad_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_negate_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_max_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_multiply_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_minimum_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_maxpool_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_layers_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_rescale_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_dot_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_exp_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_divide_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_convolution_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_constant_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_concat_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_broadcast_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_basics_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_batch_norm_inference_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_avg_pool_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/test_add_HE_SEAL.cpp.o CMakeFiles/unit-test.dir/main.cpp.o CMakeFiles/unit-test.dir/test_seal.cpp.o CMakeFiles/unit-test.dir/test_encryption_parameters.cpp.o CMakeFiles/unit-test.dir/test_he_op_annotations.cpp.o CMakeFiles/unit-test.dir/test_he_plaintext.cpp.o CMakeFiles/unit-test.dir/test_he_tensor.cpp.o CMakeFiles/unit-test.dir/test_he_type.cpp.o CMakeFiles/unit-test.dir/test_he_util.cpp.o CMakeFiles/unit-test.dir/test_he_fusion.cpp.o CMakeFiles/unit-test.dir/test_he_supported_ops.cpp.o CMakeFiles/unit-test.dir/test_propagate_he_annotations.cpp.o CMakeFiles/unit-test.dir/test_he_seal_executable.cpp.o CMakeFiles/unit-test.dir/test_bounded_relu.cpp.o CMakeFiles/unit-test.dir/test_perf_micro.cpp.o CMakeFiles/unit-test.dir/test_protobuf.cpp.o CMakeFiles/unit-test.dir/test_seal_plaintext_wrapper.cpp.o CMakeFiles/unit-test.dir/test_seal_util.cpp.o CMakeFiles/unit-test.dir/test_tcp_message.cpp.o CMakeFiles/unit-test.dir/test_tcp_client.cpp.o CMakeFiles/unit-test.dir/test_ngraph_he_log.cpp.o  -o unit-test -Wl,-rpath,/home/he-transformer/build/src:/home/he-transformer/build/ext_ngraph/src/ext_ngraph-build/src/ngraph:/home/he-transformer/build/ext_zlib/src/ext_zlib-build ../ext_ngraph/src/ext_ngraph-build/test/util/libngraph_test_util.a -lpthread ../src/libhe_seal_backend.so ../protobuf/lib/libprotobuf.a ../gtest/build/googlemock/gtest/libgtest.a ../ext_ngraph/src/ext_ngraph-build/src/ngraph/libngraph.so ../external/lib/libseal-3.4.a ../ext_zlib/src/ext_zlib-build/libz.so 
#29 282.6 INFO: From Compiling tensorflow/lite/kernels/mfcc.cc:
#29 282.6 In file included from ./tensorflow/lite/kernels/internal/common.h:42:0,
#29 282.6                  from ./tensorflow/lite/kernels/cpu_backend_gemm_custom_gemv.h:41,
#29 282.6                  from ./tensorflow/lite/kernels/cpu_backend_gemm.h:22,
#29 282.6                  from ./tensorflow/lite/kernels/internal/optimized/optimized_ops.h:42,
#29 282.6                  from tensorflow/lite/kernels/mfcc.cc:21:
#29 282.6 external/arm_neon_2_x86_sse/NEON_2_SSE.h: In function 'void vst1q_lane_f32(float32_t*, float32x4_t, int)':
#29 282.6 external/arm_neon_2_x86_sse/NEON_2_SSE.h:9725:31: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
#29 282.6      *(ptr) =  *((float*)&ilane);
#29 282.6                                ^
#29 282.6 external/arm_neon_2_x86_sse/NEON_2_SSE.h: In function 'float32_t vgetq_lane_f32(float32x4_t, int)':
#29 282.6 external/arm_neon_2_x86_sse/NEON_2_SSE.h:11964:22: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
#29 282.6      return *(float*)&ilane;
#29 282.6                       ^
#29 283.3 /home/he-transformer/build/test/test_sum_HE_SEAL.cpp:35: undefined reference to `ngraph::runtime::Backend::create(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)'
#29 283.3 /home/he-transformer/build/test/test_sum_HE_SEAL.cpp:51: undefined reference to `ngraph::Node::get_name[abi:cxx11]() const'
#29 283.3 CMakeFiles/unit-test.dir/test_sum_HE_SEAL.cpp.o: In function `bool ngraph::runtime::he::test::all_close<float>(std::vector<float, std::allocator<float> > const&, std::vector<float, std::allocator<float> > const&, float)':
#29 283.3 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:264: undefined reference to `ngraph::default_logger_handler_func(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 283.3 CMakeFiles/unit-test.dir/test_sum_HE_SEAL.cpp.o: In function `bool ngraph::runtime::he::test::all_close<float>(std::vector<float, std::allocator<float> > const&, std::vector<float, std::allocator<float> > const&, float)':
#29 283.3 /home/he-transformer/test/./test_util.hpp:57: undefined reference to `ngraph::LogHelper::LogHelper(ngraph::LOG_TYPE, char const*, int, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)'
#29 283.3 CMakeFiles/unit-test.dir/test_sum_HE_SEAL.cpp.o: In function `void __gnu_cxx::new_allocator<ngraph::Function>::construct<ngraph::Function, std::shared_ptr<ngraph::op::v0::Sum>&, std::vector<std::shared_ptr<ngraph::op::v0::Parameter>, std::allocator<std::shared_ptr<ngraph::op::v0::Parameter> > > >(ngraph::Function*, std::shared_ptr<ngraph::op::v0::Sum>&, std::vector<std::shared_ptr<ngraph::op::v0::Parameter>, std::allocator<std::shared_ptr<ngraph::op::v0::Parameter> > >&&)':
#29 283.3 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/ext/new_allocator.h:136: undefined reference to `ngraph::Function::Function(std::shared_ptr<ngraph::Node> const&, std::vector<std::shared_ptr<ngraph::op::v0::Parameter>, std::allocator<std::shared_ptr<ngraph::op::v0::Parameter> > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 283.3 CMakeFiles/unit-test.dir/test_sum_HE_SEAL.cpp.o: In function `__cxx_global_var_init':
#29 283.3 /home/he-transformer/build/test/test_sum_HE_SEAL.cpp:65: undefined reference to `ngraph::combine_test_backend_and_case(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 283.3 /home/he-transformer/build/test/test_sum_HE_SEAL.cpp:65: undefined reference to `ngraph::prepend_disabled(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
...
#29 285.2 ../src/libhe_seal_backend.so: undefined reference to `ngraph::Node::get_provenance_tags[abi:cxx11]() const'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::file_util::read_file_to_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::to_lower(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::runtime::Backend::get_backend_op(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ...)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::runtime::BackendManager::register_backend(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<std::shared_ptr<ngraph::runtime::Backend> (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::file_util::exists(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::descriptor::Tensor::get_name[abi:cxx11]() const'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::descriptor::Tensor::Tensor(ngraph::element::Type const&, ngraph::PartialShape const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::split(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char, bool)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::Node::description[abi:cxx11]() const'
...
#29 285.3 clang: error: linker command failed with exit code 1 (use -v to see invocation)
#29 285.3 make[2]: *** [test/unit-test] Error 1
#29 285.3 test/CMakeFiles/unit-test.dir/build.make:825: recipe for target 'test/unit-test' failed
#29 285.3 make[2]: Leaving directory '/home/he-transformer/build'
#29 285.3 make[1]: *** [test/CMakeFiles/unit-test.dir/all] Error 2

请查看 this gist 的原始输出完整日志和this repository对于 Dockerfile我用来 build 他的变形金刚。(注意:我知道 he-transformer 本身也提供了一个 Dockerfile + Makefile，但它们对我不起作用。但是，我的是基于他们的 Ubuntu 18.04 Dockerfile。)

我对 cmake 不是很熟悉，项目的build设置非常复杂，有很多依赖项，但听起来 libhe_seal_backend.so 没有正确链接 ngraph这也是作为外部项目构建的，请参阅 ngraph.cmake .但是，如您在此 CMakeLists.txt 中所见，它已链接因为构建了 libhe_seal_backend 目标。

由于目标 unit-test 导致了问题，我认为可能需要将其链接到 ngraph。为此，我添加了相应的链接目标，并在通过 modified CMakeLists.txt 克隆 he-transformer 存储库后替换了文件。 .不幸的是，这会引发相同的错误。

注意如果您尝试重现此内容 - 我花了大约 45 分钟在 48 核 AWS 机器上构建它，因为它还将 tensorflow 构建为外部项目。

Answer 1

感谢他对 he-transformer 的存储库维护者 Fabian Boemer 的提示。他注意到日志表明 CXX ABI 存在问题.该问题是由使用旧版本的 g++ (< 5.1) 编译 ngraph 引起的。日志证实了这一点，请参阅编译器标识为 GNU 4.8.5。

从 Dockerfile 中删除 gcc 4.8 安装这样系统默认的gcc-7就解决了问题，可以成功编译he-transformer。