python - Numpy 多阶段容器构建——Alpine

Question

我正在尝试多阶段容器构建以尝试使我的图像更小。违规包是numpy这显然不能很好地与 Alpine 配合使用。
我来自 numpy 的错误:

>>> import numpy 
Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/opt/venv/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/opt/venv/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/venv/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/venv/lib/python3.8/site-packages/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/opt/venv/lib/python3.8/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/opt/venv/bin/python"
  * The NumPy version is: "1.20.3"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/venv/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so)

这是我的 Dockerfile:

FROM python:3.8 AS builder

RUN apt-get update && \
    apt-get install -y --no-install-recommends build-essential gcc

COPY requirements.txt .

ENV LANG=C.UTF-8 \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    VIRTUAL_ENV=/opt/venv \
    PATH="/opt/venv/bin:$PATH" \
    PIP_DISABLE_PIP_VERSION_CHECK=1

RUN python3 -m venv $VIRTUAL_ENV
RUN pip3 install --requirement requirements.txt


FROM python:3.8-alpine AS Production

RUN apk update && \
    apk add --no-cache libc6-compat libexecinfo-dev musl-dev g++ gfortran linux-headers && \
    python3 -m venv /opt/venv && \
    adduser -D worker

USER worker
WORKDIR /home/worker

ENV LANG=C.UTF-8 \
  PYTHONDONTWRITEBYTECODE=1 \
  PYTHONUNBUFFERED=1 \
  VIRTUAL_ENV=/opt/venv \
  PATH="/opt/venv/bin:$PATH" 

COPY --chown=worker:worker --from=builder /opt/venv /opt/venv
COPY --chown=worker:worker ./src /home/worker

CMD ["sleep", "100000"]
#  ENTRYPOINT ["gunicorn"]
#  CMD ["--bind", "0.0.0.0:8080", "--workers", "2", "myapp.__main__:app"]

我尝试添加: apk add --no-cache libc6-compat libexecinfo-dev musl-dev g++ gfortran linux-headers我在 related SO 上看到的题。他们将 numpy 直接安装到他们的 alpine 镜像中，但我正在从构建容器中复制 numpy，因此它似乎没有帮助。
如果我使用 python:3.8-slim而不是 python:3.8-alpine它似乎有效，但图像并不小。 ld-linux-x86-64.so.2 Alpine 容器中缺少它，但我不知道如何获取它或为什么不从构建镜像中复制它。
要求.txt:

numpy==1.20.3
scipy==1.6.3

Answer 1

通过查看您的 Dockerfile，我建议不要使用 alpine 和多阶段构建，而是在基于 debian 的 python 镜像中安装预编译的轮子。下面的 Dockerfile 无需编译任何内容即可安装您的需求。构建时间快，体积也比较小。 -slim图像不包括构建工具，因此图像较小。

FROM python:3.8-slim
ENV VIRTUAL_ENV="/opt/venv"
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN python3 -m venv $VIRTUAL_ENV \
    && pip3 install --no-cache-dir \
        numpy==1.20.3 \
        scipy==1.6.3

如果您坚持使用 alpine，请继续阅读，我将在下一段中介绍构建多阶段 alpine 图像。注意尺寸优势不大... 上面基于 debian 的图像是 284 MB，下面基于 alpine 的图像是 211 MB。
问题是您在基于 debian 的镜像中安装 numpy，然后将其复制到 alpine 镜像中。 Alpine 使用 musl C，而 debian 和其他 linux 发行版使用 glibc。它们不兼容。 Numpy 没有为 musl C 提供预编译的轮子，所以如果你想使用 alpine，你必须编译 numpy。我已经包含了一个显示如何操作的最小 dockerfile。构建镜像可能需要 20 多分钟，因为 numpy 和 scipy 必须从源代码编译。

# Define these variables once and use throughout the dockerfile.
# This reduces chance of bugs...
ARG BASE_IMAGE="python:3.8-alpine"
ARG VIRTUAL_ENV="/opt/venv"

FROM $BASE_IMAGE AS builder
ARG VIRTUAL_ENV
ENV VIRTUAL_ENV=$VIRTUAL_ENV \
    PATH="$VIRTUAL_ENV/bin:$PATH"
RUN apk add --no-cache \
        build-base \
        gcc \
        gfortran \
        openblas-dev \
    && python3 -m venv $VIRTUAL_ENV \
    && pip3 install --no-cache-dir \
        numpy==1.20.3 \
        scipy==1.6.3

FROM $BASE_IMAGE AS production
ARG VIRTUAL_ENV
COPY --from=builder $VIRTUAL_ENV $VIRTUAL_ENV
ENV VIRTUAL_ENV=$VIRTUAL_ENV \
    PATH="$VIRTUAL_ENV/bin:$PATH"
RUN apk add --no-cache openblas