python - 当我尝试在 Windows 10 上安装 caffe 时，脚本找不到 python 库和 numpy

Question

我一直在使用this在我的 Windows 10 系统上安装 caffe 的资源。

我首先通过命令提示符为 Python 安装必要的依赖项:

C:\Users\MYNAME>conda config --add channels conda-forge
C:\Users\MYNAME>conda config --add channels willyd
C:\Users\MYNAME>conda install --yes cmake ninja numpy scipy protobuf==3.1.0 six scikit-image pyyaml pydotplus graphviz

然后我在我的 C:\ 目录中创建了一个名为 caffe 的空目录。然后我通过命令提示符进入该目录并键入以下命令:

C:\caffe\caffe>git clone https://github.com/BVLC/caffe.git
C:\caffe\caffe>cd caffe

然后我打开 /scripts 目录中的文件 build_win.cmd 并编辑了一些内容。编辑后的文件如下所示:

@echo off
@setlocal EnableDelayedExpansion

:: Default values
if DEFINED APPVEYOR (
    echo Setting Appveyor defaults
    if NOT DEFINED MSVC_VERSION set MSVC_VERSION=14
    if NOT DEFINED WITH_NINJA set WITH_NINJA=0
    if NOT DEFINED CPU_ONLY set CPU_ONLY=1
    if NOT DEFINED CUDA_ARCH_NAME set CUDA_ARCH_NAME=Auto
    if NOT DEFINED CMAKE_CONFIG set CMAKE_CONFIG=Release
    if NOT DEFINED USE_NCCL set USE_NCCL=0
    if NOT DEFINED CMAKE_BUILD_SHARED_LIBS set CMAKE_BUILD_SHARED_LIBS=0
    if NOT DEFINED PYTHON_VERSION set PYTHON_VERSION=3
    if NOT DEFINED BUILD_PYTHON set BUILD_PYTHON=1
    if NOT DEFINED BUILD_PYTHON_LAYER set BUILD_PYTHON_LAYER=1
    if NOT DEFINED BUILD_MATLAB set BUILD_MATLAB=0
    if NOT DEFINED PYTHON_EXE set PYTHON_EXE=python
    if NOT DEFINED RUN_TESTS set RUN_TESTS=1
    if NOT DEFINED RUN_LINT set RUN_LINT=1
    if NOT DEFINED RUN_INSTALL set RUN_INSTALL=1

:: Set python 2.7 with conda as the default python
if !PYTHON_VERSION! EQU 2 (
    **set CONDA_ROOT=C:\Users\MYNAME\Anaconda3**
)
:: Set python 3.5 with conda as the default python
if !PYTHON_VERSION! EQU 3 (
    **set CONDA_ROOT=C:\Users\MYNAME\Anaconda3**
)
**set PATH=C:\Users\MYNAME\Anaconda3;C:\Users\MYNAME\Anaconda3\Scripts;C:\Users\MYNAME\Anaconda3\Library\bin;!PATH!**

:: Check that we have the right python version
!PYTHON_EXE! --version
:: Add the required channels
conda config --add channels conda-forge
conda config --add channels willyd
:: Update conda
conda update conda -y
:: Download other required packages
conda install --yes cmake ninja numpy scipy protobuf==3.1.0 six scikit-image pyyaml pydotplus graphviz

if ERRORLEVEL 1  (
  echo ERROR: Conda update or install failed
  exit /b 1
)

:: Install cuda and disable tests if needed
if !WITH_CUDA! == 1 (
    call %~dp0\appveyor\appveyor_install_cuda.cmd
    set CPU_ONLY=0
    set RUN_TESTS=0
    set USE_NCCL=1
) else (
    set CPU_ONLY=1
)

:: Disable the tests in debug config
if "%CMAKE_CONFIG%" == "Debug" (
    echo Disabling tests on appveyor with config == %CMAKE_CONFIG%
    set RUN_TESTS=0
)

:: Disable linting with python 3 until we find why the script fails
if !PYTHON_VERSION! EQU 3 (
    set RUN_LINT=0
)

) else (
    :: Change the settings here to match your setup
    :: Change MSVC_VERSION to 12 to use VS 2013
    if NOT DEFINED MSVC_VERSION set MSVC_VERSION=14
    :: Change to 1 to use Ninja generator (builds much faster)
    if NOT DEFINED WITH_NINJA set WITH_NINJA=0
    :: Change to 1 to build caffe without CUDA support
    if NOT DEFINED CPU_ONLY set CPU_ONLY=1
    :: Change to generate CUDA code for one of the following GPU architectures
    :: [Fermi  Kepler  Maxwell  Pascal  All]
    if NOT DEFINED CUDA_ARCH_NAME set CUDA_ARCH_NAME=Auto
    :: Change to Debug to build Debug. This is only relevant for the Ninja generator the Visual Studio generator will generate both Debug and Release configs
    if NOT DEFINED CMAKE_CONFIG set CMAKE_CONFIG=Release
    :: Set to 1 to use NCCL
    if NOT DEFINED USE_NCCL set USE_NCCL=0
    :: Change to 1 to build a caffe.dll
    if NOT DEFINED CMAKE_BUILD_SHARED_LIBS set CMAKE_BUILD_SHARED_LIBS=0
    :: Change to 3 if using python 3.5 (only 2.7 and 3.5 are supported)
    if NOT DEFINED PYTHON_VERSION set PYTHON_VERSION=3
    :: Change these options for your needs.
    if NOT DEFINED BUILD_PYTHON set BUILD_PYTHON=1
    if NOT DEFINED BUILD_PYTHON_LAYER set BUILD_PYTHON_LAYER=1
    if NOT DEFINED BUILD_MATLAB set BUILD_MATLAB=0
    :: If python is on your path leave this alone
    if NOT DEFINED PYTHON_EXE set PYTHON_EXE=python
    :: Run the tests
    if NOT DEFINED RUN_TESTS set RUN_TESTS=0
    :: Run lint
    if NOT DEFINED RUN_LINT set RUN_LINT=0
    :: Build the install target
    if NOT DEFINED RUN_INSTALL set RUN_INSTALL=0
)

:: Set the appropriate CMake generator
:: Use the exclamation mark ! below to delay the
:: expansion of CMAKE_GENERATOR
if %WITH_NINJA% EQU 0 (
    if "%MSVC_VERSION%"=="14" (
        set CMAKE_GENERATOR=Visual Studio 14 2015 Win64
    )
    if "%MSVC_VERSION%"=="12" (
        set CMAKE_GENERATOR=Visual Studio 12 2013 Win64
    )
    if "!CMAKE_GENERATOR!"=="" (
        echo ERROR: Unsupported MSVC version
        exit /B 1
    )
) else (
    set CMAKE_GENERATOR=Ninja
)

echo INFO: ============================================================
echo INFO: Summary:
echo INFO: ============================================================
echo INFO: MSVC_VERSION               = !MSVC_VERSION!
echo INFO: WITH_NINJA                 = !WITH_NINJA!
echo INFO: CMAKE_GENERATOR            = "!CMAKE_GENERATOR!"
echo INFO: CPU_ONLY                   = !CPU_ONLY!
echo INFO: CUDA_ARCH_NAME             = !CUDA_ARCH_NAME!
echo INFO: CMAKE_CONFIG               = !CMAKE_CONFIG!
echo INFO: USE_NCCL                   = !USE_NCCL!
echo INFO: CMAKE_BUILD_SHARED_LIBS    = !CMAKE_BUILD_SHARED_LIBS!
echo INFO: PYTHON_VERSION             = !PYTHON_VERSION!
echo INFO: BUILD_PYTHON               = !BUILD_PYTHON!
echo INFO: BUILD_PYTHON_LAYER         = !BUILD_PYTHON_LAYER!
echo INFO: BUILD_MATLAB               = !BUILD_MATLAB!
echo INFO: PYTHON_EXE                 = "!PYTHON_EXE!"
echo INFO: RUN_TESTS                  = !RUN_TESTS!
echo INFO: RUN_LINT                   = !RUN_LINT!
echo INFO: RUN_INSTALL                = !RUN_INSTALL!
echo INFO: ============================================================

:: Build and exectute the tests
:: Do not run the tests with shared library
if !RUN_TESTS! EQU 1 (
    if %CMAKE_BUILD_SHARED_LIBS% EQU 1 (
        echo WARNING: Disabling tests with shared library build
        set RUN_TESTS=0
    )
)

if NOT EXIST build mkdir build
pushd build

:: Setup the environement for VS x64
set batch_file=!VS%MSVC_VERSION%0COMNTOOLS!..\..\VC\vcvarsall.bat
call "%batch_file%" amd64

:: Configure using cmake and using the caffe-builder dependencies
:: Add -DCUDNN_ROOT=C:/Projects/caffe/cudnn-8.0-windows10-x64-v5.1/cuda ^
:: below to use cuDNN
cmake -G"!CMAKE_GENERATOR!" ^
      -DBLAS=Open ^
      -DCMAKE_BUILD_TYPE:STRING=%CMAKE_CONFIG% ^
      -DBUILD_SHARED_LIBS:BOOL=%CMAKE_BUILD_SHARED_LIBS% ^
      -DBUILD_python:BOOL=%BUILD_PYTHON% ^
      -DBUILD_python_layer:BOOL=%BUILD_PYTHON_LAYER% ^
      -DBUILD_matlab:BOOL=%BUILD_MATLAB% ^
      -DCPU_ONLY:BOOL=%CPU_ONLY% ^
      -DCOPY_PREREQUISITES:BOOL=1 ^
      -DINSTALL_PREREQUISITES:BOOL=1 ^
      -DUSE_NCCL:BOOL=!USE_NCCL! ^
      -DCUDA_ARCH_NAME:STRING=%CUDA_ARCH_NAME% ^
      "%~dp0\.."

if ERRORLEVEL 1 (
  echo ERROR: Configure failed
  exit /b 1
)

:: Lint
if %RUN_LINT% EQU 1 (
    cmake --build . --target lint  --config %CMAKE_CONFIG%
)

if ERRORLEVEL 1 (
  echo ERROR: Lint failed
  exit /b 1
)

:: Build the library and tools
cmake --build . --config %CMAKE_CONFIG%

if ERRORLEVEL 1 (
  echo ERROR: Build failed
  exit /b 1
)

:: Build and exectute the tests
if !RUN_TESTS! EQU 1 (
    cmake --build . --target runtest --config %CMAKE_CONFIG%

    if ERRORLEVEL 1 (
        echo ERROR: Tests failed
        exit /b 1
    )

    if %BUILD_PYTHON% EQU 1 (
        if %BUILD_PYTHON_LAYER% EQU 1 (
            :: Run python tests only in Release build since
            :: the _caffe module is _caffe-d is debug
            if "%CMAKE_CONFIG%"=="Release" (
                :: Run the python tests
                cmake --build . --target pytest

                if ERRORLEVEL 1 (
                    echo ERROR: Python tests failed
                    exit /b 1
                )
            )
        )
    )
)

if %RUN_INSTALL% EQU 1 (
    cmake --build . --target install --config %CMAKE_CONFIG%
)

popd
@endlocal

在此之后，我运行了以下命令:

C:\caffe\caffe>scripts\build_win.cmd

这给了我以下输出:

The system cannot find the drive specified.
The system cannot find the drive specified.
INFO: ============================================================
INFO: Summary:
INFO: ============================================================
INFO: MSVC_VERSION               = 14
INFO: WITH_NINJA                 = 0
INFO: CMAKE_GENERATOR            = "Visual Studio 14 2015 Win64"
INFO: CPU_ONLY                   = 0
INFO: CUDA_ARCH_NAME             = Auto
INFO: CMAKE_CONFIG               = Release
INFO: USE_NCCL                   = 0
INFO: CMAKE_BUILD_SHARED_LIBS    = 0
INFO: PYTHON_VERSION             = 2
INFO: BUILD_PYTHON               = 1
INFO: BUILD_PYTHON_LAYER         = 1
INFO: BUILD_MATLAB               = 0
INFO: PYTHON_EXE                 = "python"
INFO: RUN_TESTS                  = 0
INFO: RUN_LINT                   = 0
INFO: RUN_INSTALL                = 0
INFO: ============================================================
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler using: Visual Studio 14 2015 Win64
-- Check for working C compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: C:/cygwin64/bin/python2.7.exe (found suitable version "2.7.14", minimum required is "2.7")
-- Downloading prebuilt dependencies to C:/Users/Abdullah Siddiqui/.caffe/dependencies/download/libraries_v140_x64_py27_1.1.0.tar.bz2
-- [download 0% complete]
...
-- Extracting dependencies
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic
-- Found GFlags: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found gflags  (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: gflags_shared)
-- Found Glog: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found glog    (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: glog)
-- Found Protobuf: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/bin/protoc.exe (found version "3.1.0")
-- Found PROTOBUF Compiler: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/bin/protoc.exe
-- Found LMDB: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found lmdb    (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: lmdb)
-- Found LevelDB: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found LevelDB (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: leveldb)
-- Found ZLIB: optimized;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlib.lib;debug;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlibd.lib (found version "1.2.8")
-- Found Snappy: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found Snappy  (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: snappy_static;optimized;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlib.lib;debug;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlibd.lib)
CMake Warning at cmake/Dependencies.cmake:97 (message):
  -- CUDA is not detected by cmake.  Building without it...
Call Stack (most recent call first):
  CMakeLists.txt:80 (include)


-- OpenCV found (C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries)
-- Found OpenBLAS libraries: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/libopenblas.dll.a
-- Found OpenBLAS include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
-- Could NOT find NumPy (missing:  NUMPY_INCLUDE_DIR NUMPY_VERSION) (Required is at least version "1.7.1")
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   python
-- Python interface is disabled or not all required dependencies found. Building without it...
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.10.1.windows.1")
--
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   1.0.0
--   Git               :   1.0-111-g509dae0-dirty
--   System            :   Windows
--   C++ compiler      :   C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
--   Release CXX flags :   /MD /O2 /Ob2 /D NDEBUG /DWIN32 /D_WINDOWS /W3 /GR /EHsc
--   Debug CXX flags   :   /D_DEBUG /MDd /Zi /Ob0 /Od /RTC1 /DWIN32 /D_WINDOWS /W3 /GR /EHsc
--   Build type        :   Release
--
--   BUILD_SHARED_LIBS :   0
--   BUILD_python      :   1
--   BUILD_matlab      :   0
--   BUILD_docs        :
--   CPU_ONLY          :   0
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   USE_NCCL          :   0
--   ALLOW_LMDB_NOLOCK :   OFF
--
-- Dependencies:
--   BLAS              :   Yes (Open)
--   Boost             :   Yes (ver. 1.61)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.1.0)
--   lmdb              :   Yes (ver. 0.9.70)
--   LevelDB           :   Yes (ver. 1.18)
--   Snappy            :   Yes (ver. 1.1.1)
--   OpenCV            :   Yes (ver. 3.1.0)
--   CUDA              :   No
--
-- Install:
--   Install path      :   C:/Users/Abdullah Siddiqui/caffe/build/install
--
-- Configuring done
CMake Error at CMakeLists.txt:143 (add_dependencies):
  The dependency target "pycaffe" of target "pytest" does not exist.


-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_python_layer
    CUDA_ARCH_NAME


-- Build files have been written to: C:/Users/Abdullah Siddiqui/caffe/build
ERROR: Configure failed

我对这些说法特别困惑:

-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
-- Could NOT find NumPy (missing:  NUMPY_INCLUDE_DIR NUMPY_VERSION) (Required is at least version "1.7.1")

当 build_win.cmd 明确包含以下行时，我的系统中如何检测不到这些文件:

set PATH=C:\Users\MYNAME\Anaconda3;C:\Users\ MYNAME\Anaconda3\Scripts;C:\Users\MYNAME\Anaconda3\Library\bin;!PATH!

相关的 python 文件也存在于我的系统中。.cmd 文件找不到的 PythonLibs 存在于 Anaconda3 目录中。

Numpy 也存在于该目录的一个子目录中。

我什至可以直接从命令行运行 python:

此时，我不知道为什么我会收到有关 numpy 和 Python 库的错误。如果您有任何建议，请告诉我。

编辑:

看完this video后，我在我的项目目录中运行了以下命令:

C:\caffe\caffe>python scripts\download_prebuilt_dependencies.py --msvc_version=v140

输出如下:

ERROR: Could not find url for MSVC version = v140 and Python version = 3.6.
Available combinations are: [('v120', '2.7'), ('v140', '2.7'), ('v140', '3.5')]

这是否意味着我必须切换到 Python 3.5 才能使用这个框架？

编辑:

我下载了python 3.5.5，执行了和上面一样的步骤，得到了同样的错误。

我更多地考虑了CristiFati的建议并执行了以下命令:

C:\caffe\caffe\build>cmake -DPYTHON_EXECUTABLE=C:\Users\Abdullah Siddiqui\Anaconda3\python.exe ..

这给了我以下输出:

-- Building for: Visual Studio 14 2015
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler using: Visual Studio 14 2015
-- Check for working C compiler using: Visual Studio 14 2015 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Visual Studio 14 2015
-- Check for working CXX compiler using: Visual Studio 14 2015 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find PythonInterp: Found unsuitable version "1.4", but required is at least "2.7" (found C:/Users/Abdullah)
CMake Error at cmake/WindowsDownloadPrebuiltDependencies.cmake:40 (message):
  Could not find url for MSVC version = 1900 and Python version = 1.4.
Call Stack (most recent call first):
  CMakeLists.txt:77 (include)


-- Configuring incomplete, errors occurred!
See also "C:/Users/.../projects/caffe/build/CMakeFiles/CMakeOutput.log".

我现在明白问题是由Python 2.7 的可执行文件用于编译引起的。 如何让构建脚本指向 python 3.5？

编辑:

我正确地运行了 cmake 命令，如下所示，但我现在看到了新的错误。

C:\Users\MYNAME\projects\caffe\build>cmake -DPYTHON_EXECUTABLE="C:\Users\MYNAME\Anaconda3\python.exe" ..

它给我这些错误:

-- Could NOT find Glog (missing:  GLOG_LIBRARY)
CMake Error at cmake/ProtoBuf.cmake:6 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version "".

Answer 1

cmake 有“标准”方式定位外部包安装(通过脚本)。这适用于 Python。当找到这样的包时，会设置一些变量，通常:

• ${PACKAGE}_FOUND=ON(包状态标志)
• ${PACKAGE}_INCLUDE_DIRS(包括目录 - 在编译时使用)
• ${PACKAGE}_LIBRARIES(库 - 在链接时使用)

当搜索 numpy ( [GitHub]: BVLC/caffe - (windows) caffe/cmake/Modules/FindNumPy.cmake ) 时，搜索 PythonInterp(标准 cmake 模块)(并使用)。如初始输出所示:

-- Found PythonInterp: C:/cygwin64/bin/python2.7.exe (found suitable version "2.7.14", minimum required is "2.7")

检测到

python 2.7(没有安装 numpy 模块)，导致 numpy发现失败。

查看 FindPythonInterp.cmake 脚本，我注意到它设置了(除其他外)${PYTHON_EXECUTABLE} (cmake) 变量。

将该变量“手动”设置为 cmake(通过传递 cmdline 参数)，将是下一个合乎逻辑的事情(因为我不知道这是否单独是够了，我只是添加了一条评论):

-DPYTHON_EXECUTABLE="c:\path\to\python3"

注意:不要混合使用 Cygwin 和 native Win 工具，因为您可能会遇到微妙的问题。