docker - 构建时如何调试 docker-compose 缓存未命中

Question

我正在执行相同的 docker-compose build命令，我看到它错过了缓存

Building service1
Step 1/31 : FROM node:10 as build-stage
 ---> a7366e5a78b2
Step 2/31 : WORKDIR /app
 ---> Using cache
 ---> 8a744e522376
Step 3/31 : COPY package.json yarn.lock ./
 ---> Using cache
 ---> 66c9bb64a364
Step 4/31 : RUN yarn install --production
 ---> Running in 707365c332e7
yarn install v1.21.1
..
..
..

如您所见，缓存已丢失，但我不明白为什么
调试更改并尝试找出原因的最佳方法是什么

编辑:问题不是调试我的具体问题。但是我一般如何调试这种问题。我怎么知道为什么 docker-compose 认为事情发生了变化(尽管我很确定没有任何变化)，哪些文件/命令/结果不同？

Answer 1

how can I generally debug a problem of this kind. How can I know WHY docker-compose thinks things changed (although I'm pretty sure NOTHING changed), which files/commands/results are different

一般来说， as shown here :

I'm a bit bummed that I can't seem to find any way to make the Docker build more verbose

但是说到 docker-compose ，这取决于您使用的版本和选项。
moby/moby issue 30081解释 (by Sebastiaan van Stijn ( thaJeztah ) :

Current versions of docker-compose and docker build in many (or all) cases will not share the build cache, or at least not produce the same digest.

The reason for that is that when sending the build context with docker-compose, it will use a slightly different compression (docker-compose is written in Python, whereas the docker cli is written in Go).
There may be other differences due to them being a different implementation (and language).

(这也在 docker/compose issue 883 中讨论过)

The next release of docker compose will have an (currently opt-in) feature to make it use the actual docker cli to perform the build (by setting a COMPOSE_DOCKER_CLI_BUILD=1 environment variable). This was implemented in docker/compose#6865 (1.25.0-rc3+, Oct. 2019)

With that feature, docker compose can also use BuildKit for building images (setting the DOCKER_BUILDKIT=1 environment variable).
I would highly recommend using buildkit for your builds if possible.
When using BuildKit (requires Docker 18.09 or up, and at this point is not supported for building Windows containers), you will see a huge improvement in build-speed, and the duration taken to send the build-context to the daemon in repeated builds (buildkit uses an interactive session to send only those files that are needed during build, instead of uploading the entire build context).

因此，请首先仔细检查您的 docker-compose 是否使用 BuildKit，如果问题(缓存未重用)仍然存在，则:

COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build

塞巴斯蒂安加入 issue 4012 :

BuildKit is still opt-in (because there's no Windows support yet), but is production quality, and can be used as the default for building Linux images.

Finally, I realize that for the Azure pipelines, you (probably) don't have control over the versions of Docker and Docker Compose installed, but for your local machines, make sure to update to the latest 19.03 patch release; for example, Docker 19.03.3 and up have various improvements and fixes in the caching mechanisms of BuildKit (see, e.g., docker#373).

请注意，在您的特定情况下，即使这不是您问题的主要问题，了解以下内容是否有帮助也会很有趣:
yarnpkg/yarn/issue 749建议:

You wouldn't mount the Yarn cache directory. Instead, you should make sure you take advantage of Docker's image layer caching.

These are the commands I am using:

COPY package.json yarn.lock ./
RUN yarn --pure-lockfile

然后试试你的 yarn install命令，看看 docker 是否仍然不使用它的缓存。

RUN yarn install --frozen-lockfile --production && yarn cache clean 

别忘了 yarn cache clean 为了 prevent the yarn cache from winding up in docker layers .
如果问题仍然存在，请直接切换到 buildkit(用于测试)，使用 buildctl build --progress=plain查看更详细的输出，并调试缓存情况。

通常，一个 multi-stage approach, as shown here ，可能很有用:

FROM node:alpine
WORKDIR /usr/src/app
COPY . /usr/src/app/

# We don't need to do this cache clean, I guess it wastes time / saves space: https://github.com/yarnpkg/rfcs/pull/53
RUN set -ex; \
  yarn install --frozen-lockfile --production; \
  yarn cache clean; \
  yarn run build

FROM nginx:alpine
WORKDIR /usr/share/nginx/html
COPY --from=0 /usr/src/app/build/ /usr/share/nginx/html