将 C 代码与 CUDA 链接时出现 CMake 错误

Question

我在使用 cmake 构建项目时遇到链接错误。该构建在使用 makefile(手动生成/无 cmake)的 Linux 上运行良好，但 Windows 构建给我带来了问题。这是一个演示我的方法的准系统示例:

我在同一目录中有 3 个文件(kernel.cu、kernel.h、main.c)

主.c:

extern void kernel_wrapper();
int main(){
    kernel_wrapper();
}

内核.h:

#ifndef KERNELH
#define KERNELH 
extern "C" void kernel_wrapper();
#endif

内核.cu:

#include <stdio.h>
#include "kernel.h"

__global__ void kernel (){
    printf("hello from GPU!");
}

void kernel_wrapper(){
    kernel<<<1,1>>>();
}

我想使用可分离编译，这样主机代码就可以使用 externs 与设备代码隔离。这是我正在使用的 CMakeLists.txt 文件:

cmake_minimum_required(VERSION 3.9 FATAL_ERROR)
project(cudatest LANGUAGES C CUDA)
add_executable(c-exec main.c)
add_library(cu-lib STATIC kernel.cu kernel.h)
set_target_properties(cu-lib PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
target_link_libraries(c-exec PUBLIC cu-lib)

库和可执行文件编译成功，但在链接步骤时，出现以下错误:

cu-lib.lib(kernel.obj) : error LNK2019: unresolved external symbol __cudaRegisterLinkedBi
nary_41_tmpxft_00003dd0_00000000_7_kernel_cpp1_ii_b81a68a1 referenced in function "void _
_cdecl __nv_cudaEntityRegisterCallback(void * *)" (?__nv_cudaEntityRegisterCallback@@YAXP
EAPEAX@Z) [C:\cudatest\build\c-exec.vcxproj]
C:\cudatest\build\Debug\c-exec.exe : fatal error LNK1120: 1 unresolved externals [C:\cuda
test\build\c-exec.vcxproj]

听起来链接器缺少一些库的包含(可能是 cudart？)但是，控制台上的链接命令对我来说看起来很正常，包含静态包含和所有内容:

Link:
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\link.exe /ERRORREP
  ORT:QUEUE /OUT:"C:\cudatest\build\Debug\c-exec.exe" /INCREMENTAL /NOLOGO /LIBPATH:"C:\P
  rogram Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\lib\x64" "Debug\cu-lib.lib" cudart_
  static.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut3
  2.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAc
  cess='false'" /manifest:embed /DEBUG /PDB:"C:/cudatest/build/Debug/c-exec.pdb" /SUBSYST
  EM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/cudatest/build/Debug/c-exec.lib"
   /MACHINE:X64  /machine:x64 "c-exec.dir\Debug\main.obj"

我现在一头雾水。如果我放弃单独编译(将 add_executable 参数更改为“c-exec main.c kernel.h kernel.cu”并跳过库步骤)问题消失，但我需要使用单独编译。

This问题看起来与我的相关，我尝试应用那里建议的解决方案。在 include 行和函数定义之前放置 externs 没有任何改变。

我将 Windows 10 与 Cmake v3.11.4、CUDA v9.2 一起使用，并使用“cmake -G "Visual Studio 15 2017 Win64"-T v140"选择 cmake 生成器，因为最新版本的 Visual Studio 不受支持由 CUDA 完成。

编辑:添加了完整的控制台输出

Build started 12.07.2018 03:03:42.
Project "C:\cudatest\build\ALL_BUILD.vcxproj" on node 1 (default targets).
Project "C:\cudatest\build\ALL_BUILD.vcxproj" (1) is building "C:\cudatest\build\ZERO_
CHECK.vcxproj" (2) on node 1 (default targets).
PrepareForBuild:
  Creating directory "x64\Debug\ZERO_CHECK\".
  Creating directory "x64\Debug\ZERO_CHECK\ZERO_CHECK.tlog\".
InitializeBuildStatus:
  Creating "x64\Debug\ZERO_CHECK\ZERO_CHECK.tlog\unsuccessfulbuild" because "AlwaysCre
  ate" was specified.
CustomBuild:
  Checking Build System
  CMake does not need to re-run because C:/cudatest/build/CMakeFiles/generate.stamp is
   up-to-date.
FinalizeBuildStatus:
  Deleting file "x64\Debug\ZERO_CHECK\ZERO_CHECK.tlog\unsuccessfulbuild".
  Touching "x64\Debug\ZERO_CHECK\ZERO_CHECK.tlog\ZERO_CHECK.lastbuildstate".
Done Building Project "C:\cudatest\build\ZERO_CHECK.vcxproj" (default targets).

Project "C:\cudatest\build\ALL_BUILD.vcxproj" (1) is building "C:\cudatest\build\c-exe
c.vcxproj" (3) on node 1 (default targets).
Project "C:\cudatest\build\c-exec.vcxproj" (3) is building "C:\cudatest\build\cu-lib.v
cxproj" (4) on node 1 (default targets).
PrepareForBuild:
  Creating directory "cu-lib.dir\Debug\".
  Creating directory "C:\cudatest\build\Debug\".
  Creating directory "cu-lib.dir\Debug\cu-lib.tlog\".
InitializeBuildStatus:
  Creating "cu-lib.dir\Debug\cu-lib.tlog\unsuccessfulbuild" because "AlwaysCreate" was
   specified.
CustomBuild:
  Building Custom Rule C:/cudatest/CMakeLists.txt
  CMake does not need to re-run because C:/cudatest/build/CMakeFiles/generate.stamp is
   up-to-date.
AddCudaCompileDeps:
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\cl.exe /E /nolo
  go /showIncludes /TP /D__CUDACC__ /D_WINDOWS /DCMAKE_INTDIR="Debug" /DCMAKE_INTDIR="
  Debug" /D_MBCS /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include" /
  I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin" /I"C:\Program Files\N
  VIDIA GPU Computing Toolkit\CUDA\v9.2\include" /I. /FIcuda_runtime.h /c C:\cudatest\
  kernel.cu
Project "C:\cudatest\build\cu-lib.vcxproj" (4) is building "C:\cudatest\build\cu-lib.v
cxproj" (4:2) on node 1 (CudaBuildCore target(s)).
CudaBuildCore:
  Compiling CUDA source file ..\kernel.cu...
  cmd.exe /C "C:\Users\dulls\AppData\Local\Temp\tmpd0a36f1624184bf982dc8082e9a727be.cm
  d"
  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin\nvcc.exe" -gencode=arch
  =compute_30,code=\"sm_30,compute_30\" --use-local-env -ccbin "C:\Program Files (x86)
  \Microsoft Visual Studio 14.0\VC\bin\x86_amd64" -x cu -rdc=true -I"C:\Program Files\
  NVIDIA GPU Computing Toolkit\CUDA\v9.2\include" -I"C:\Program Files\NVIDIA GPU Compu
  ting Toolkit\CUDA\v9.2\include"     --keep-dir x64\Debug -maxrregcount=0  --machine
  64 --compile -cudart static -Xcompiler="/EHsc -Zi -Ob0" -g   -D"_WINDOWS" -D"CMAKE_I
  NTDIR=\"Debug\"" -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -Xcompiler "/EHsc /W3 /nologo /O
  d /FS /Zi /RTC1 /MDd /GR" -o cu-lib.dir\Debug\kernel.obj "C:\cudatest\kernel.cu"

  C:\cudatest\build>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin\nvcc.
  exe" -gencode=arch=compute_30,code=\"sm_30,compute_30\" --use-local-env -ccbin "C:\P
  rogram Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64" -x cu -rdc=true -I
  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include" -I"C:\Program File
  s\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include"     --keep-dir x64\Debug -maxrregc
  ount=0  --machine 64 --compile -cudart static -Xcompiler="/EHsc -Zi -Ob0" -g   -D"_W
  INDOWS" -D"CMAKE_INTDIR=\"Debug\"" -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -Xcompiler "/E
  Hsc /W3 /nologo /Od /FS /Zi /RTC1 /MDd /GR" -o cu-lib.dir\Debug\kernel.obj "C:\cudat
  est\kernel.cu"
  kernel.cu
Done Building Project "C:\cudatest\build\cu-lib.vcxproj" (CudaBuildCore target(s)).

Lib:
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\Lib.exe /OUT:"C
  :\cudatest\build\Debug\cu-lib.lib" /NOLOGO /MACHINE:X64  /machine:x64 "cu-lib.dir\De
  bug\kernel.obj"
  cu-lib.vcxproj -> C:\cudatest\build\Debug\cu-lib.lib
FinalizeBuildStatus:
  Deleting file "cu-lib.dir\Debug\cu-lib.tlog\unsuccessfulbuild".
  Touching "cu-lib.dir\Debug\cu-lib.tlog\cu-lib.lastbuildstate".
Done Building Project "C:\cudatest\build\cu-lib.vcxproj" (default targets).

PrepareForBuild:
  Creating directory "c-exec.dir\Debug\".
  Creating directory "c-exec.dir\Debug\c-exec.tlog\".
InitializeBuildStatus:
  Creating "c-exec.dir\Debug\c-exec.tlog\unsuccessfulbuild" because "AlwaysCreate" was
   specified.
CustomBuild:
  Building Custom Rule C:/cudatest/CMakeLists.txt
  CMake does not need to re-run because C:/cudatest/build/CMakeFiles/generate.stamp is
   up-to-date.
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\CL.exe /c /I"C:
  \Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include" /Zi /nologo /W3 /WX-
  /Od /Ob0 /D "WIN32" /D "_WINDOWS" /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /RTC1 /M
  Dd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"c-exec.dir\Debug\\" /Fd"c
  -exec.dir\Debug\vc140.pdb" /Gd /TC /errorReport:queue C:\cudatest\main.c
  main.c
Link:
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\link.exe /ERROR
  REPORT:QUEUE /OUT:"C:\cudatest\build\Debug\c-exec.exe" /INCREMENTAL /NOLOGO /LIBPATH
  :"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\lib\x64" "Debug\cu-lib.lib
  " cudart_static.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32
  .lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='
  asInvoker' uiAccess='false'" /manifest:embed /DEBUG /PDB:"C:/cudatest/build/Debug/c-
  exec.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/cudatest/bu
  ild/Debug/c-exec.lib" /MACHINE:X64  /machine:x64 "c-exec.dir\Debug\main.obj"
     Creating library C:/cudatest/build/Debug/c-exec.lib and object C:/cudatest/build/
  Debug/c-exec.exp
cu-lib.lib(kernel.obj) : error LNK2019: unresolved external symbol __cudaRegisterLinke
dBinary_41_tmpxft_00013688_00000000_7_kernel_cpp1_ii_b81a68a1 referenced in function "
void __cdecl __nv_cudaEntityRegisterCallback(void * *)" (?__nv_cudaEntityRegisterCallb
ack@@YAXPEAPEAX@Z) [C:\cudatest\build\c-exec.vcxproj]
C:\cudatest\build\Debug\c-exec.exe : fatal error LNK1120: 1 unresolved externals [C:\c
udatest\build\c-exec.vcxproj]
Done Building Project "C:\cudatest\build\c-exec.vcxproj" (default targets) -- FAILED.

Done Building Project "C:\cudatest\build\ALL_BUILD.vcxproj" (default targets) -- FAILE
D.


Build FAILED.

"C:\cudatest\build\ALL_BUILD.vcxproj" (default target) (1) ->
"C:\cudatest\build\c-exec.vcxproj" (default target) (3) ->
(Link target) ->
  cu-lib.lib(kernel.obj) : error LNK2019: unresolved external symbol __cudaRegisterLin
kedBinary_41_tmpxft_00013688_00000000_7_kernel_cpp1_ii_b81a68a1 referenced in function
 "void __cdecl __nv_cudaEntityRegisterCallback(void * *)" (?__nv_cudaEntityRegisterCal
lback@@YAXPEAPEAX@Z) [C:\cudatest\build\c-exec.vcxproj]
  C:\cudatest\build\Debug\c-exec.exe : fatal error LNK1120: 1 unresolved externals [C:
\cudatest\build\c-exec.vcxproj]

    0 Warning(s)
    2 Error(s)

Answer 1

我想我找到了这个问题的解决方案。 (感谢@talonmies 的指导)

我不得不使用以下方法强制连接设备:

set_property(TARGET c-lib PROPERTY CUDA_RESOLVE_DEVICE_SYMBOLS ON)

而且有效!

来自 CUDA_RESOLVE_DEVICE_SYMBOLS 手册页:

CUDA only: Enables device linking for the specific static library target

If set this will enable device linking on this static library target. Normally device linking is deferred until a shared library or executable is generated, allowing for multiple static libraries to resolve device symbols at the same time.

我仍然不确定为什么我的可执行文件生成不会在主机将所有内容链接在一起之前触发设备链接。

编辑: 看起来是一个 known bug这会影响 Windows 构建，但在 Linux 上运行良好。