gitlab-ci - gitlab-ci缓存如何在dockerRunner上工作？什么是/cache目录？什么是cache_dir？

Question

gitlab-ci缓存如何在dockerRunner上工作？
什么是/ cache目录？
什么是cache_dir？
与"paths" in "cache" gitlab-ci.yml匹配的文件存储在哪里以及如何存储？

Answer 1

安装到/cache目录的卷是在gitlab-runner安装时自动创建的，并由cache_dir设置管理

关于cache_dir的更多信息：


https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/executors/docker.md#the-builds-and-cache-storage

If you modify the /cache storage path, you also need to make sure to mark this
directory as persistent by defining it in volumes = ["/my/cache/"] under the
[runners.docker] section in config.toml.

TLDR

/cache目录与gitlab-ci.yml中的 cache配置不同


作业容器中的 /cache目录是存储缓存文件的位置
在作业结束时，将与gitlab-ci.yml中的 cache配置匹配的文件复制到 /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>
项目“管道”页面中的“清除流道缓存”按钮计划不将 /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.zip提取到gitlab-ci.yml的 cache配置中指定的目录中（而不是像我起初那样“删除/ cache文件夹的内容” ）


附言

在具有gitlab-runner（ https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382）的计算机上创建了名为 gitlab-runner-cache的容器

（看起来像）此容器用于创建存储/ cache数据的匿名卷。创建匿名卷后，该容器将停止。

作业容器（即通常在您的测试中运行的容器）挂载此匿名卷



证明

拥有gitlab-ci.yml

image: srghma/docker-nixos-with-git-crypt

cache:
  key: "test00000" # to reset cache - change this key OR clear cache in project settings page
  paths:
    - .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically)

testtest:
  script:
    - nix-env -i tree

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

    - echo "test" > /cache/test
    - mkdir -p ./.mycache
    - echo "test" > ./.mycache/test

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

输出：


第一次运行

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 675caa7 feat: cache update
From https://gitlab.com/srghma/myproject
   675caa7..3d1e223  nix        -> origin/nix
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache

0 directories, 0 files
$ ls -al ./.mycache || true
$ echo "test" > /cache/test
ls: ./.mycache: No such file or directory
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
`-- test

0 directories, 1 file
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded

在第二轮

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded

通过在项目“管道”页面中单击“清除运行器缓存”清除缓存后

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-12...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
ls: ./.mycache: No such file or directory
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:45 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:45 ..
-rw-r--r--    1 root     root             5 Feb 24 11:45 test
Creating cache test00000-12...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded