python - pip install wheel 随机失败

Question

我有一个 python 项目，我在本地打包为一个 wheel，我想将这个 wheel 安装在基于 python:alpine3.7 的 docker 容器中的虚拟环境中。我的问题是有时我工作，有时它无法解决轮子中的依赖关系，似乎是随机的。

轮子的依赖是:

_install_requires = [
    'numpy>=1.15.0',
    'pandas>=0.21.0',
    'scipy>=1.0.0',
    'scikit-learn>=0.19.2'
]

而我的dockerfile如下:

FROM python:alpine3.7

# install dependencies
# the lapack package is only in the community repository
RUN echo "http://dl-4.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --update add --no-cache --virtual .build-deps \
    gcc \
    freetype-dev

RUN apk add --update lapack-dev python py-pip python-dev &&\
    apk add --update linux-headers

# Install dependencies
RUN apk add --no-cache --virtual .build-deps \
    gfortran \
    musl-dev \
    g++ \
    make &&\
    ln -s /usr/include/locale.h /usr/include/xlocale.h

RUN pip install --upgrade pip && pip install virtualenv

RUN virtualenv -p python3 ./venv

ADD dist/project-as-wheel-py3-none-any.whl .

RUN . ./venv/bin/activate && pip install ./project-as-wheel-py3-none-any.whl

在构建 docker 容器时，有时会失败，典型的日志是:

Processing /project-as-wheel-py3-none-any.whl
Collecting scikit-learn>=0.19.2 (from project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/25/b6/454cf208be93efa3db50ce06b732328c57ede005d1dcfa71d9a1548530b0/scikit-learn-0.19.2.tar.gz (9.7MB)
Collecting pandas>=0.21.0 (from project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/3e/56/82c4d4c049294f87ebd05b65fdcbc9ed68bd23fb0a7e4469caf9a75d199f/pandas-0.23.3.tar.gz (10.5MB)
Collecting numpy>=1.15.0 (from project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/3a/20/c81632328b1a4e1db65f45c0a1350a9c5341fd4bbb8ea66cdd98da56fe2e/numpy-1.15.0.zip (4.5MB)
Collecting scipy>=1.0.0 (from project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/07/76/7e844757b9f3bf5ab9f951ccd3e4a8eed91ab8720b0aac8c2adcc2fdae9f/scipy-1.1.0.tar.gz (15.6MB)
Collecting python-dateutil>=2.5.0 (from pandas>=0.21.0->project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)
Collecting pytz>=2011k (from pandas>=0.21.0->project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas>=0.21.0->project-as-wheel==0.2.6)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: scikit-learn, pandas, numpy, scipy
  Running setup.py bdist_wheel for scikit-learn: started
  Running setup.py bdist_wheel for scikit-learn: finished with status 'error'
  Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ogmdp3dv --python-tag cp37:
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 168, in get_numpy_status
      import numpy
  ModuleNotFoundError: No module named 'numpy'
  Traceback (most recent call last):
    File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 148, in get_scipy_status
      import scipy
  ModuleNotFoundError: No module named 'scipy'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 269, in <module>
      setup_package()
    File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 249, in setup_package
      .format(numpy_req_str, instructions))
  ImportError: Numerical Python (NumPy) is not installed.
  scikit-learn requires NumPy >= 1.8.2.
  Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html    


  ----------------------------------------
  Failed building wheel for scikit-learn
  Running setup.py clean for scikit-learn
  Running setup.py bdist_wheel for pandas: started
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: still running...
  Running setup.py bdist_wheel for pandas: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/8d/e7/58/7a882c73419e95e6709d925abb4b56c719c9be59ef826fe01c
  Running setup.py bdist_wheel for numpy: started
  Running setup.py bdist_wheel for numpy: still running...
  Running setup.py bdist_wheel for numpy: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/77/d2/00/03180c22fc40b6f17f6a5c6ea0fa1689734a9cf4cf54179a1c
  Running setup.py bdist_wheel for scipy: started
  Running setup.py bdist_wheel for scipy: finished with status 'error'
  Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-83ktidr8 --python-tag cp37:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-439jjmrz/scipy/setup.py", line 474, in <module>
      setup_package()
    File "/tmp/pip-install-439jjmrz/scipy/setup.py", line 450, in setup_package
      from numpy.distutils.core import setup
  ModuleNotFoundError: No module named 'numpy'    

  ----------------------------------------
  Failed building wheel for scipy
  Running setup.py clean for scipy
  Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:    

  `setup.py clean` is not supported, use one of the following instead:    

    - `git clean -xdf` (cleans all files)
    - `git clean -Xdf` (cleans all versioned files, doesn't touch
                        files that aren't checked into the git repo)    

  Add `--force` to your command to use it anyway if you must (unsupported).    


  ----------------------------------------
  Failed cleaning build dir for scipy
Successfully built pandas numpy
Failed to build scikit-learn scipy
Installing collected packages: scikit-learn, six, python-dateutil, pytz, numpy, pandas, scipy, project-as-wheel
  Running setup.py install for scikit-learn: started
    Running setup.py install for scikit-learn: finished with status 'error'
    Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-dwajrs1d/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.7/scikit-learn:
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 168, in get_numpy_status
        import numpy
    ModuleNotFoundError: No module named 'numpy'
    Traceback (most recent call last):
      File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 148, in get_scipy_status
        import scipy
    ModuleNotFoundError: No module named 'scipy'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 269, in <module>
        setup_package()
      File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 249, in setup_package
        .format(numpy_req_str, instructions))
    ImportError: Numerical Python (NumPy) is not installed.
    scikit-learn requires NumPy >= 1.8.2.
    Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html    


    ----------------------------------------
Command "/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-dwajrs1d/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.7/scikit-learn" failed with error code 1 in /tmp/pip-install-439jjmrz/scikit-learn/

让我感到困惑的是 pip 设法安装了 numpy 和 pandas，然后安装 scipy 失败并声称“没有名为‘numpy’的模块”。

编辑:日志中的 TL;DR 是:

1. pip 尝试安装 scikit-learn，但由于未安装 numpy 而失败。
2. pip 尝试安装 pandas，成功。
3. pip 尝试安装 numpy，成功。
4. pip 尝试安装 scipy，但由于未安装 numpy 而失败(但它已在上面的第 3 步中成功安装!)。
5. pip 声明它成功安装了 pandas 和 numpy，但没有安装 scikit-learn 和 scipy。
6. pip 再次尝试安装 scipy，再次失败，提示未安装 numpy。
7. pip 放弃。

Edit2:根据 Nuts 的建议，我将 dockerfile 中的相关行更改为

RUN . ./venv/bin/activate && pip install numpy && pip install scipy && pip install ./project-as-wheel-py3-none-any.whl

不太令人满意，因为我无法从 setup.py 中获取版本要求，但它可以工作。

Answer 1

对此更改要求

_install_requires = [
    'numpy>= 1.8.2',
    'pandas>=0.21.0',
    'scipy>=1.0.0',
    'scikit-learn>=0.19.2'
]