python - docker build 期间没有名为 'numpy' 的模块

Question

我正在按照说明( https://github.com/huggingface/transfer-learning-conv-ai )从 Huggingface 安装 conv-ai，但我卡在了 docker 构建步骤:docker build -t convai .我使用的是 Mac 10.15，python 3.8，将 Docker 内存增加到 4G。
我尝试了以下方法来解决问题:

添加 numpy在 requirements.txt

添加 RUN pip3 install --upgrade setuptools在 Dockerfile 中

添加 --upgrade至 RUN pip3 install -r /tmp/requirements.txt在 Dockerfile 中

添加 RUN pip3 install numpy之前 RUN pip3 install -r /tmp/requirements.txt在 Dockerfile 中

添加 RUN apt-get install python3-numpy之前 RUN pip3 install -r /tmp/requirements.txt在 Dockerfile 中

因此使用 python 3.6.13 post ，但它有完全相同的错误。

我目前正在通过在 RUN pip3 install requirements.txt 之前输入来在容器内进行调试。

谁可以帮我这个事？谢谢!!
错误:

 => [6/9] COPY . ./                                                                                                          0.0s
 => [7/9] COPY requirements.txt /tmp/requirements.txt                                                                        0.0s
 => ERROR [8/9] RUN pip3 install -r /tmp/requirements.txt                                                                   98.2s
------
 > [8/9] RUN pip3 install -r /tmp/requirements.txt:
#12 1.111 Collecting torch (from -r /tmp/requirements.txt (line 1))
#12 1.754   Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)
#12 81.11 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 2))
#12 81.76   Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#12 81.82 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 3))
#12 82.17   Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#12 82.29 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 4))
#12 82.50   Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#12 82.57 Collecting tensorflow (from -r /tmp/requirements.txt (line 5))
#12 83.12   Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#12 95.24 Collecting spacy (from -r /tmp/requirements.txt (line 6))
#12 95.81   Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#12 97.41     Complete output from command python setup.py egg_info:
#12 97.41     Traceback (most recent call last):
#12 97.41       File "<string>", line 1, in <module>
#12 97.41       File "/tmp/pip-build-cc3a804w/spacy/setup.py", line 5, in <module>
#12 97.41         import numpy
#12 97.41     ModuleNotFoundError: No module named 'numpy'
#12 97.41     
#12 97.41     ----------------------------------------
#12 98.11 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-cc3a804w/spacy/

@Håken Lid 如果我 RUN pip3 install numpy，我得到的错误就在之前 RUN pip3 install -r tmp/requirements :

 => [ 8/10] RUN pip3 install numpy                                                                                          10.1s
 => ERROR [ 9/10] RUN pip3 install -r /tmp/requirements.txt                                                                112.4s
------                                                                                                                            
 > [ 9/10] RUN pip3 install -r /tmp/requirements.txt:                                                                             
#13 1.067 Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from -r /tmp/requirements.txt (line 1)) 
#13 1.074 Collecting torch (from -r /tmp/requirements.txt (line 2))                                                               
#13 1.656   Downloading https://files.pythonhosted.org/packages/46/99/8b658e5095b9fb02e38ccb7ecc931eb1a03b5160d77148aecf68f8a7eeda/torch-1.8.0-cp36-cp36m-manylinux1_x86_64.whl (735.5MB)                                                                           
#13 96.46 Collecting pytorch-ignite (from -r /tmp/requirements.txt (line 3))
#13 97.02   Downloading https://files.pythonhosted.org/packages/f8/d3/640f70d69393b415e6a29b27c735047ad86267921ad62682d1d756556d48/pytorch_ignite-0.4.4-py3-none-any.whl (200kB)
#13 97.07 Collecting transformers==2.5.1 (from -r /tmp/requirements.txt (line 4))
#13 97.32   Downloading https://files.pythonhosted.org/packages/13/33/ffb67897a6985a7b7d8e5e7878c3628678f553634bd3836404fef06ef19b/transformers-2.5.1-py3-none-any.whl (499kB)
#13 97.43 Collecting tensorboardX==1.8 (from -r /tmp/requirements.txt (line 5))
#13 97.70   Downloading https://files.pythonhosted.org/packages/c3/12/dcaf67e1312475b26db9e45e7bb6f32b540671a9ee120b3a72d9e09bc517/tensorboardX-1.8-py2.py3-none-any.whl (216kB)
#13 97.76 Collecting tensorflow (from -r /tmp/requirements.txt (line 6))
#13 98.27   Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
#13 109.6 Collecting spacy (from -r /tmp/requirements.txt (line 7))
#13 110.0   Downloading https://files.pythonhosted.org/packages/65/01/fd65769520d4b146d92920170fd00e01e826cda39a366bde82a87ca249db/spacy-3.0.5.tar.gz (7.0MB)
#13 111.6     Complete output from command python setup.py egg_info:
#13 111.6     Traceback (most recent call last):
#13 111.6       File "<string>", line 1, in <module>
#13 111.6       File "/tmp/pip-build-t6n57csv/spacy/setup.py", line 10, in <module>
#13 111.6         from Cython.Build import cythonize
#13 111.6     ModuleNotFoundError: No module named 'Cython'
#13 111.6     
#13 111.6     ----------------------------------------
#13 112.3 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-t6n57csv/spacy/
------
executor failed running [/bin/sh -c pip3 install -r /tmp/requirements.txt]: exit code: 1

要求.txt:

torch
pytorch-ignite
transformers==2.5.1
tensorboardX==1.8
tensorflow  # for tensorboardX
spacy

Dockerfile:

FROM ubuntu:18.04

MAINTAINER Loreto Parisi loretoparisi@gmail.com

########################################  BASE SYSTEM
# set noninteractive installation
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y apt-utils
RUN apt-get install -y --no-install-recommends \
    build-essential \
    pkg-config \
    tzdata \
    curl

######################################## PYTHON3
RUN apt-get install -y \
    python3 \
    python3-pip

# set local timezone
RUN ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime && \
    dpkg-reconfigure --frontend noninteractive tzdata

# transfer-learning-conv-ai
ENV PYTHONPATH /usr/local/lib/python3.6 
COPY . ./
COPY requirements.txt /tmp/requirements.txt
RUN pip3 install -r /tmp/requirements.txt

# model zoo
RUN mkdir models && \
    curl https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/finetuned_chatbot_gpt.tar.gz > models/finetuned_chatbot_gpt.tar.gz && \
    cd models/ && \
    tar -xvzf finetuned_chatbot_gpt.tar.gz && \
    rm finetuned_chatbot_gpt.tar.gz
    
CMD ["bash"]

到目前为止我运行的步骤:

git clone https://github.com/huggingface/transfer-learning-conv-ai
cd transfer-learning-conv-ai
pip install -r requirements.txt
python -m spacy download en
docker build -t convai .

Answer 1

pip 似乎没有安装预先构建的轮子，而是尝试从源代码构建 spacy。这是一个脆弱的过程，需要额外的依赖。
为了避免这种情况，在继续安装之前，您应该确保 Python 包 pip、wheel 和 setuptools 是最新的。

# replace RUN pip3 install -r /tmp/requirements.txt

RUN python3 -m pip install --upgrade pip setuptools wheel                                                                                                                                                                                                
RUN python3 -m pip install -r /tmp/requirements.txt