tensorflow - 无法在 Intel i7 930 CPU 上从源代码编译 TensorFlow； GTS-250 显卡

Question

我是 TF 新手，想从源代码编译，因为我的桌面没有支持 AVX 指令的 CPU 或 GPU。我的系统有一个 Intel i7 930 处理器(来自 nehalem 家族的 Bloomfield)和一个 Nvidea GTS-250 CPU。是的，我知道，两者都在变长。

以下是我在此过程中失败的完整堆栈。我正在按照以下网页的说明进行操作。

https://www.tensorflow.org/install/source_windows

我推进编译并最终看到以下错误......

H:\Python\TensorFlowCompile\tensorflow>python ./configure.py
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.29.1 installed.
Please specify the location of python. [Default is C:\Users\Zeek\AppData\Local\Programs\Python\Python37\python.exe]:


Found possible Python library paths:
  C:\Users\Zeek\AppData\Local\Programs\Python\Python37\lib\site-packages
Please input the desired Python library path to use.  Default is [C:\Users\Zeek\AppData\Local\Programs\Python\Python37\lib\site-packages]

Do you wish to build TensorFlow with XLA JIT support? [y/N]: N
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: N
No CUDA support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is /arch:AVX]: --config=v2 -march=nehalem


Would you like to override eigen strong inline for some C++ compilation to reduce the compilation time? [Y/n]: Y
Eigen strong inline overridden.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl            # Build with MKL support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=nonccl         # Disable NVIDIA NCCL support.

H:\Python\TensorFlowCompile\tensorflow>bazel build //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=269
INFO: Options provided by the client:
  'build' options: --python_path=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe
INFO: Reading rc options for 'build' from h:\python\tensorflowcompile\tensorflow\.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone --strategy=Genrule=standalone -c opt --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --config=v2
INFO: Reading rc options for 'build' from h:\python\tensorflowcompile\tensorflow\.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe --action_env PYTHON_LIB_PATH=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/lib/site-packages --python_path=C:/Users/Zeek/AppData/Local/Programs/Python/Python37/python.exe --config monolithic --copt=-w --host_copt=-w --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI --verbose_failures --distinct_host_configuration=false --define=override_eigen_strong_inline=true --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v2 in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define=tf_api_version=2
INFO: Found applicable config definition build:monolithic in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define framework_shared_object=false
INFO: Found applicable config definition build:monolithic in file h:\python\tensorflowcompile\tensorflow\.bazelrc: --define framework_shared_object=false
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (1 packages loaded, 0 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
INFO: Call stack for the definition of repository 'com_google_protobuf' which is a tf_http_archive (rule definition at H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl:121:19):
 - H:/python/tensorflowcompile/tensorflow/tensorflow/workspace.bzl:434:5
 - H:/python/tensorflowcompile/tensorflow/WORKSPACE:19:1
Analyzing: target //tensorflow/tools/pip_package:build_pip_package (6 packages loaded, 18 targets configured)
INFO: Repository 'com_google_protobuf' used the following cache hits instead of downloading the corresponding file.
 * Hash 'b9e92f9af8819bbbc514e2902aec860415b70209f31dfc8c4fa72515a5df9d59' for https://storage.googleapis.com/mirror.tensorflow.org/github.com/protocolbuffers/protobuf/archive/310ba5ee72661c081129eb878c1bbcec936b20f0.tar.gz
If the definition of 'com_google_protobuf' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'com_google_protobuf':
   Traceback (most recent call last):
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 101
                _apply_patch(ctx, ctx.attr.patch_file)
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 68, in _apply_patch
                _execute_and_check_ret_code(ctx, cmd)
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
                fail("Non-zero return code({1}) when ...))
Non-zero return code(127) when executing 'C:\msys64\usr\bin\bash.exe -l -c "patch" "-p1" "-d" "C:/users/Zeek/_bazel_Zeek/hfhzrtpt/external/com_google_protobuf" "-i" "H:/python/tensorflowcompile/tensorflow/third_party/protobuf/protobuf.patch"':
Stdout:
Stderr: /usr/bin/bash: patch: command not found
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: no such package '@com_google_protobuf//': Traceback (most recent call last):
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 101
                _apply_patch(ctx, ctx.attr.patch_file)
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 68, in _apply_patch
                _execute_and_check_ret_code(ctx, cmd)
        File "H:/python/tensorflowcompile/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
                fail("Non-zero return code({1}) when ...))
Non-zero return code(127) when executing 'C:\msys64\usr\bin\bash.exe -l -c "patch" "-p1" "-d" "C:/users/Zeek/_bazel_Zeek/hfhzrtpt/external/com_google_protobuf" "-i" "H:/python/tensorflowcompile/tensorflow/third_party/protobuf/protobuf.patch"':
Stdout:
Stderr: /usr/bin/bash: patch: command not found
INFO: Elapsed time: 19.298s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (6 packages loaded, 18 targets configured)
FAILED: Build did NOT complete successfully (6 packages loaded, 18 targets configured)

在此编译之前，我使用更新版本的 bazel 运行，但出现以下错误，这就是我恢复到 0.29.1 的原因

H:\Python\TensorFlowCompile\tensorflow>python ./configure.py
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 1.0.1 installed.
Please downgrade your bazel installation to version 0.29.1 or lower to build TensorFlow! To downgrade: download the installer for the old version (from https://github.com/bazelbuild/bazel/releases) then run the installer.

关于我哪里出错的任何想法，我对此还是很陌生，所以这可能是一些非常明显/容易的事情......
[编辑]这似乎通过设置未安装的补丁来解决:
pacman -Syuu 补丁

**我推得更远了，现在我在 zip 阶段失败了，如下所示.. **

INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...

[0 / 23] [Prepa] PythonZipper tensorflow/python/keras/api/create_tensorflow.python_api_1_keras_python_api_gen_compat_v1.zip ... (3 actions, 0 running)
ERROR: H:/python/tensorflowcompile/tensorflow/tensorflow/lite/python/BUILD:46:1: PythonZipper tensorflow/lite/python/tflite_convert.zip failed (Exit 255)
FATAL: MappedOutputFile(bazel-out/x64_windows-opt/bin/tensorflow/lite/python/tflite_convert.zip): CreateFileMapping failed
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 38.195s, Critical Path: 4.35s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

**然后我尝试返回 python 配置阶段以确保我选择了 v2 和正确的 CPU 系列，但是当我重做编译时，我看到另一个错误，它很快就会失败.. **

java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@local_config_cc' (requested by nodes 'REPOSITORY:@local_config_cc')
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.nio.file.InvalidPathException: Illegal char <> at index 60: C:/users/bill/_bazel_bill/hfhzrtpt/external/local_config_cc/*********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.0
** Copyright (c) 2017 Microsoft Corporation

C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Auxiliary/Build/VCVARSALL.BAT
at java.base/sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
at java.base/sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsPathParser.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsPath.parse(Unknown Source)
at java.base/sun.nio.fs.WindowsFileSystem.getPath(Unknown Source)
at java.base/java.nio.file.Path.of(Unknown Source)
at java.base/java.nio.file.Paths.get(Unknown Source)
at com.google.devtools.build.lib.vfs.JavaIoFileSystem.getNioPath(JavaIoFileSystem.java:84)
at com.google.devtools.build.lib.vfs.JavaIoFileSystem.exists(JavaIoFileSystem.java:119)
at com.google.devtools.build.lib.vfs.Path.exists(Path.java:356)
at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkPath.exists(SkylarkPath.java:79)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:135)
at com.google.devtools.build.lib.syntax.DotExpression.eval(DotExpression.java:126)
at com.google.devtools.build.lib.syntax.DotExpression.doEval(DotExpression.java:51)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.UnaryOperatorExpression.doEval(UnaryOperatorExpression.java:98)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:139)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.UnaryOperatorExpression.doEval(UnaryOperatorExpression.java:98)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:139)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execAssignment(Eval.java:72)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:192)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:144)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execAssignment(Eval.java:72)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:192)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
at com.google.devtools.build.lib.syntax.Eval.execIfBranch(Eval.java:83)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:198)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:140)
at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:474)
at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:436)
at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryFunction.fetch(SkylarkRepositoryFunction.java:173)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.fetchRepository(RepositoryDelegatorFunction.java:298)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:225)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)
... 7 more
FAILED: Build did NOT complete successfully (231 packages loaded, 3779 targets configured)
WARNING: Waiting for server process to terminate (waited 5 seconds, waiting at most 60)

H:\Python\TensorFlowCompile\tensorflow>

对不起，伙计们，我不确定此时我在哪里或如何展示这个方法......有没有办法重新开始编译，是否有我错过的东西未能通过 pythonzip 阶段，我的 bezal 有问题吗配置开始......

Answer 1

TensorFlow 尚不支持 Bazel 1.x(即将推出，但尚未完成)，所以在此之前您需要 0.29.1。

设置环境变量指向你的VC目录，见"Installing and Using Bazel" > "Installing on Windows" > "Troubleshooting" > "Problem: Bazel does not find Visual Studio or Visual C++" .

开始正常cmd.exe shell，无需运行 Visual Studio 命令行。

安装 TensorFlow 所需的 Python 模块:python -m pip install numpy keras_preprocessing

配置 TensorFlow:python configure.py

构建 pip 模块:bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package