docker - Google Cloud Build for Python 应用程序触发的 docker 构建无法从私有(private) artefact 注册表中提取 pip 要求

Question

我有一个针对 Python Flask 应用程序的谷歌云构建，该应用程序是在对我的 git 存储库的拉取请求时触发的。我正在尝试添加存储在 google artefact 注册表中的私有(private) python 包依赖项。当我将服务帐户 json 复制到 docker 容器并将 GOOGLE_APPLICATION_CREDENTIALS 指向它时，这在本地工作得很好，但我不想将服务 key 提交到 github 并且希望避免服务 key 位于容器中。

这与此类似 question但这没有得到解答，仅建议使用短期访问 token ，但没有有关如何将其集成到自动云构建触发器中的详细信息/文档。

我的 Dockerfile 如下所示:

WORKDIR $APP_HOME
COPY . ./

# Let Hive know what env it is running in
ARG HIVE_BUILD_ENV=UNSET
ENV HIVE_ENV=$HIVE_BUILD_ENV

# setup timezone for Ireland and magics library for file type detection
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ="Europe/Dublin"
RUN apt-get update && apt-get install -y tzdata libmagic1


FROM base as staging

ENV DB_NAME=hive-staging

RUN pip install --upgrade pip
RUN pip install keyrings.google-artifactregistry-auth 
RUN pip install --no-cache-dir -r requirements.txt **# <<< build fails here**

CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 "hive:create_app()"

我的 cloudbuild-preview.yaml 开始看起来像这样:(我可以提供整个文件，但构建在这一步失败)

steps:
  - id: "build image"
    name: "gcr.io/cloud-builders/docker"
    args:
      [
        "build",
        "-t",
        "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:${_PR_NUMBER}-${SHORT_SHA}",
        ".",
        "--target",
        "staging",
        "--no-cache"
      ]

我的requirements.txt开始看起来像这样:

--index-url https://europe-west1-python.pkg.dev/hive-347910/hive-commons-art-repo/simple
--extra-index-url https://pypi.org/simple
hive-commons==0.0.6
Flask==2.2.2

当我尝试运行构建时，我收到以下错误输出:

Step #0 - "build image": Step 15/16 : RUN pip install --no-cache-dir -r requirements.txt
Step #0 - "build image":  ---> Running in 1d17c8df7022
Step #0 - "build image": Looking in indexes: https://europe-west1-python.pkg.dev/hive-347910/hive-commons-art-repo/simple, https://pypi.org/simple
Step #0 - "build image": WARNING: Compute Engine Metadata server unavailable on attempt 1 of 3. Reason: timed out
Step #0 - "build image": WARNING: Compute Engine Metadata server unavailable on attempt 2 of 3. Reason: timed out
Step #0 - "build image": WARNING: Compute Engine Metadata server unavailable on attempt 3 of 3. Reason: timed out
Step #0 - "build image": WARNING: Authentication failed using Compute Engine authentication due to unavailable metadata server.
Step #0 - "build image": WARNING: Failed to retrieve Application Default Credentials: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
Step #0 - "build image": WARNING: Trying to retrieve credentials from gcloud...
Step #0 - "build image": WARNING: Failed to retrieve credentials from gcloud: gcloud command exited with status: [Errno 2] No such file or directory: 'gcloud'
Step #0 - "build image": WARNING: Artifact Registry PyPI Keyring: No credentials could be found.
Step #0 - "build image": WARNING: Keyring is skipped due to an exception: Failed to find credentials, Please run: `gcloud auth application-default login or export GOOGLE_APPLICATION_CREDENTIALS=<path/to/service/account/key>`
Step #0 - "build image": User for europe-west1-python.pkg.dev: ERROR: Exception:
Step #0 - "build image": Traceback (most recent call last):
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
Step #0 - "build image":     status = run_func(*args)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
Step #0 - "build image":     return func(self, options, args)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 400, in run
Step #0 - "build image":     requirement_set = resolver.resolve(
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
Step #0 - "build image":     result = self._result = resolver.resolve(
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
Step #0 - "build image":     state = resolution.resolve(requirements, max_rounds=max_rounds)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
Step #0 - "build image":     self._add_to_criteria(self.state.criteria, r, parent=None)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
Step #0 - "build image":     if not criterion.candidates:
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
Step #0 - "build image":     return bool(self._sequence)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
Step #0 - "build image":     return any(self)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
Step #0 - "build image":     return (c for c in iterator if id(c) not in self._incompatible_ids)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 44, in _iter_built
Step #0 - "build image":     for version, func in infos:
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 279, in iter_index_candidate_infos
Step #0 - "build image":     result = self._finder.find_best_candidate(
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 889, in find_best_candidate
Step #0 - "build image":     candidates = self.find_all_candidates(project_name)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 830, in find_all_candidates
Step #0 - "build image":     page_candidates = list(page_candidates_it)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/sources.py", line 134, in page_candidates
Step #0 - "build image":     yield from self._candidates_from_page(self._link)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 790, in process_project_url
Step #0 - "build image":     index_response = self._link_collector.fetch_response(project_url)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 461, in fetch_response
Step #0 - "build image":     return _get_index_content(location, session=self.session)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 364, in _get_index_content
Step #0 - "build image":     resp = _get_simple_response(url, session=session)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 135, in _get_simple_response
Step #0 - "build image":     resp = session.get(
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 600, in get
Step #0 - "build image":     return self.request("GET", url, **kwargs)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 518, in request
Step #0 - "build image":     return super().request(method, url, *args, **kwargs)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 587, in request
Step #0 - "build image":     resp = self.send(prep, **send_kwargs)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 708, in send
Step #0 - "build image":     r = dispatch_hook("response", hooks, r, **kwargs)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/hooks.py", line 30, in dispatch_hook
Step #0 - "build image":     _hook_data = hook(hook_data, **kwargs)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 270, in handle_401
Step #0 - "build image":     username, password, save = self._prompt_for_password(parsed.netloc)
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 233, in _prompt_for_password
Step #0 - "build image":     username = ask_input(f"User for {netloc}: ")
Step #0 - "build image":   File "/usr/local/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 204, in ask_input
Step #0 - "build image":     return input(message)
Step #0 - "build image": EOFError: EOF when reading a line
Step #0 - "build image": The command '/bin/sh -c pip install --no-cache-dir -r requirements.txt' returned a non-zero code: 2
Finished Step #0 - "build image"
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 2
Step #0 - "build image": 

这两行是我的问题的根源:

WARNING: Authentication failed using Compute Engine authentication due to unavailable metadata server.WARNING: Failed to retrieve Application Default Credentials: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started

但是查看错误消息中链接的文档，我看不到一种方法来验证在没有服务帐户 json 的情况下在 docker 容器构建内部运行的 pip。我已经看到与此问题相关的其他问题，人们建议多阶段构建，如果我从我的开发机器运行构建，那么这将起作用，但我面临同样的问题，即多阶段构建中的第一阶段在哪里拉动服务帐户 key 仅作为自动化云构建的一部分从源中提取。

任何有关如何解决此问题的建议将不胜感激。

Answer 1

编辑:联系谷歌支持后，我看到了一种执行此步骤的方法，而无需使用 secret 管理器或服务帐户 json。

This page in the docs显示如果您将 "--network=cloudbuild" 添加到 yaml 文件中的 docker 构建步骤，它会将该步骤的容器附加到名为 cloudbuild 的本地 Docker 网络，该网络具有 pip 所需的 ADC 信用访问工件注册表。该解决方案非常简单，并且消除了对服务帐户信用文件的任何要求。更安全且更简单的 Dockerfile。所以我的 yaml 文件现在看起来像这样:

steps:
  - id: "build image"
    name: "gcr.io/cloud-builders/docker"
    args:
      [
        "build",
        "-t",
        "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:${_PR_NUMBER}-${SHORT_SHA}",
        ".",
        "--target",
        "staging",
        "--network=cloudbuild"
      ]

原始解决方案:找到了一个解决方案，不需要将服务帐户 json 提交到源代码管理。

云构建可以访问secrets stored in the secret manager .

因此，我将服务帐户 json 作为 --build-arg 传递到 docker build 命令中，然后将其保存到文件中并将 GOOGLE_APPLICATION_CREDENTIALS 指向该位置。之后我删除该文件并取消设置环境变量。

这是我的 yaml 文件的相关部分:

- id: "build image"
    name: "gcr.io/cloud-builders/docker"
    entrypoint: 'bash'
    args:
      [
        "-c",
        'docker build --tag gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:${_PR_NUMBER}-${SHORT_SHA} --target staging --build-arg SERVICE_ACCOUNT_JSON="$$CREDS_JSON" --no-cache .'
      ]
    secretEnv: ['CREDS_JSON']
availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/fake_secret_name/versions/latest
      env: 'CREDS_JSON'

以及 Dockerfile 中处理此问题的部分:

ARG CREDS_JSON

WORKDIR $APP_HOME
RUN touch creds.json
RUN bash -c 'echo -E "$CREDS_JSON" >> ./creds.json'
ARG GOOGLE_APPLICATION_CREDENTIALS="$APP_HOME/creds.json"

RUN pip install --upgrade pip
RUN pip install keyrings.google-artifactregistry-auth
RUN pip install --no-cache-dir -r requirements.txt

RUN rm $APP_HOME/creds.json

特别重要的是要注意 --build-arg 周围的双引号并使用 RUN bash -c 'echo -E "$CREDS_JSON">> ./creds.json' 代替使用内置 Docker 命令 RUN echo 来保留 json 文件中的空格和回车符，否则 key 环包无法将 cred 文件作为有效的 json 文件进行处理。

这解决了我从 artefact 存储库访问 pip 需求的问题，而无需在源代码管理中存储凭据，但服务帐户凭据 json 仍然暴露在 docker 镜像构建阶段的历史记录中，所以我不是 100%这个解决方案，所以我正在考虑使用多阶段构建来进一步限制这种暴露。我还将在短时间内轮换这些凭据。

理想情况下，我希望 docker 构建使用来自运行该构建的云构建服务帐户的凭据，并且我已联系 Google 云支持以查看是否可行。