Docker在构建镜像时无法删除目录

Question

问题类似于Docker is unable to delete a file when building images

但是，路由原因不一样，因为在我的情况下启用了 d_type。

这是Dockerfile

FROM arm64v8/ubuntu:latest

RUN export DEBIAN_FRONTEND='noninteractive' && \
    rm -rf /var/lib/apt/lists/* /tmp/* /usr/share/doc

尝试构建图像时

docker build -t arm64v8-ubuntu-test .

出现以下错误

Sending build context to Docker daemon  3.584kB
Step 1/2 : FROM arm64v8/ubuntu:latest
 ---> 499b06aa549b
Step 2/2 : RUN export DEBIAN_FRONTEND='noninteractive' &&     rm -rf /var/lib/apt/lists/* /tmp/* /usr/share/doc
 ---> Running in 5ff328ef5a57
rm: cannot remove '/usr/share/doc/sed': Directory not empty
rm: cannot remove '/usr/share/doc/libpam-modules': Directory not empty
rm: cannot remove '/usr/share/doc/util-linux': Directory not empty
rm: cannot remove '/usr/share/doc/mawk': Directory not empty
rm: cannot remove '/usr/share/doc/gnupg': Directory not empty
rm: cannot remove '/usr/share/doc/apt': Directory not empty
rm: cannot remove '/usr/share/doc/libreadline6': Directory not empty
rm: cannot remove '/usr/share/doc/adduser/examples/adduser.local.conf.examples': Directory not empty
rm: cannot remove '/usr/share/doc/passwd': Directory not empty
rm: cannot remove '/usr/share/doc/mount': Directory not empty
rm: cannot remove '/usr/share/doc/procps': Directory not empty
The command '/bin/sh -c export DEBIAN_FRONTEND='noninteractive' &&     rm -rf /var/lib/apt/lists/* /tmp/* /usr/share/doc' returned a non-zero code: 1

我的 docker 信息是

$ docker info
Containers: 7
 Running: 1
 Paused: 0
 Stopped: 6
Images: 95
Server Version: 17.10.0-ce
Storage Driver: overlay
 Backing Filesystem: extfs
 Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: mjm8vu1ixmw1ls25dk7gomiva
 Is Manager: true
 ClusterID: lnspgxtrca8d4lm3kfv7zbz77
 Managers: 3
 Nodes: 3
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 10.0.4.71
 Manager Addresses:
  10.0.4.71:2377
  10.0.4.72:2377
  10.0.4.73:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 0351df1c5a66838d0c392b4ac4cf9450de844e2d
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.14.79-27-ARCH
Operating System: Arch Linux ARM
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 1.928GiB
Name: alarm-docker-manager-host
ID: Y5ZZ:5G3O:KIE2:MSQX:7MBB:XWOZ:CYLL:2SSY:GTMB:YCBG:4TTT:EJVN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

手动运行容器和尝试执行命令有相同的结果，例如

$ docker run --rm -it arm64v8/ubuntu:latest bash
root@87a3ea8fecdd:~# rm -rf /var/lib/apt/lists/* /tmp/* /usr/share/doc
rm: cannot remove '/usr/share/doc/sed': Directory not empty
rm: cannot remove '/usr/share/doc/libpam-modules': Directory not empty
rm: cannot remove '/usr/share/doc/util-linux': Directory not empty
rm: cannot remove '/usr/share/doc/mawk': Directory not empty
rm: cannot remove '/usr/share/doc/gnupg': Directory not empty
rm: cannot remove '/usr/share/doc/apt': Directory not empty
rm: cannot remove '/usr/share/doc/libreadline6': Directory not empty
rm: cannot remove '/usr/share/doc/adduser/examples/adduser.local.conf.examples': Directory not empty
rm: cannot remove '/usr/share/doc/passwd': Directory not empty
rm: cannot remove '/usr/share/doc/mount': Directory not empty
rm: cannot remove '/usr/share/doc/procps': Directory not empty

此外，权限似乎很好，目录在那里，并且没有被其他进程使用

root@87a3ea8fecdd:/# ls -lah /
total 68K
drwxr-xr-x   1 root root 4.0K Oct 19 06:43 .
drwxr-xr-x   1 root root 4.0K Oct 19 06:43 ..
-rwxr-xr-x   1 root root    0 Oct 19 06:43 .dockerenv
drwxr-xr-x   2 root root 4.0K Oct 19 05:39 bin
drwxr-xr-x   2 root root 4.0K Apr 12  2016 boot
drwxr-xr-x   5 root root  360 Oct 19 06:44 dev
drwxr-xr-x   1 root root 4.0K Oct 19 06:43 etc
drwxr-xr-x   2 root root 4.0K Apr 12  2016 home
drwxr-xr-x   8 root root 4.0K Oct 19 05:39 lib
drwxr-xr-x   2 root root 4.0K Oct  6 01:43 media
drwxr-xr-x   2 root root 4.0K Oct  6 01:43 mnt
drwxr-xr-x   2 root root 4.0K Oct  6 01:43 opt
dr-xr-xr-x 143 root root    0 Oct 19 06:43 proc
drwx------   2 root root 4.0K Oct 19 05:39 root
drwxr-xr-x   5 root root 4.0K Oct  6 01:44 run
drwxr-xr-x   2 root root 4.0K Oct 19 05:39 sbin
drwxr-xr-x   2 root root 4.0K Oct  6 01:43 srv
dr-xr-xr-x  12 root root    0 Oct 19 06:43 sys
drwxrwxrwt   2 root root 4.0K Oct  6 01:44 tmp
drwxr-xr-x  10 root root 4.0K Oct 19 05:39 usr
drwxr-xr-x  11 root root 4.0K Oct 19 05:39 var

root@87a3ea8fecdd:/# ls -lah /var/lib/apt
total 24K
drwxr-xr-x  6 root root 4.0K Oct 19 05:39 .
drwxr-xr-x 11 root root 4.0K Oct 19 05:39 ..
-rw-r--r--  5 root root    0 Oct  6 01:44 extended_states
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 keyrings
drwxr-xr-x  2 root root 4.0K Oct 11 14:01 lists
drwxr-xr-x  3 root root 4.0K Oct 19 05:39 mirrors
drwxr-xr-x  2 root root 4.0K Apr 14  2016 periodic

root@87a3ea8fecdd:/# ls -lah /usr/share
total 160K
drwxr-xr-x 40 root root 4.0K Oct 19 05:39 .
drwxr-xr-x 10 root root 4.0K Oct 19 05:39 ..
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 adduser
drwxr-xr-x  3 root root 4.0K Oct 19 05:39 apport
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 base-files
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 base-passwd
drwxr-xr-x  3 root root 4.0K Oct 19 05:39 bash-completion
drwxr-xr-x  6 root root 4.0K Oct 19 05:39 bug
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 common-licenses
drwxr-xr-x  4 root root 4.0K Oct 19 05:39 dbus-1
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 debconf
drwxr-xr-x  2 root root 4.0K Oct 19 05:39 debianutils
drwxr-xr-x  2 root root 4.0K Apr 12  2016 dict
drwxr-xr-x 94 root root 4.0K Oct 19 05:39 doc
...

root@df3ef4561fc3:/# lsof /usr/share/doc
root@df3ef4561fc3:/# rm -rf /usr/share/doc
rm: cannot remove '/usr/share/doc/sed': Directory not empty
rm: cannot remove '/usr/share/doc/libpam-modules': Directory not empty
rm: cannot remove '/usr/share/doc/util-linux': Directory not empty
rm: cannot remove '/usr/share/doc/mawk': Directory not empty
rm: cannot remove '/usr/share/doc/gnupg': Directory not empty
rm: cannot remove '/usr/share/doc/apt': Directory not empty
rm: cannot remove '/usr/share/doc/libreadline6': Directory not empty
rm: cannot remove '/usr/share/doc/adduser/examples/adduser.local.conf.examples': Directory not empty
rm: cannot remove '/usr/share/doc/passwd': Directory not empty
rm: cannot remove '/usr/share/doc/mount': Directory not empty
rm: cannot remove '/usr/share/doc/procps': Directory not empty

rmdir 也不行

root@df3ef4561fc3:/# rmdir /usr/share/doc
rmdir: failed to remove '/usr/share/doc': Directory not empty
root@df3ef4561fc3:/# lsof /usr/share/doc

直接运行rm命令也是如此

$ docker run --rm -it arm64v8/ubuntu:latest rm -rf /usr/share/doc
rm: cannot remove '/usr/share/doc/sed': Directory not empty
rm: cannot remove '/usr/share/doc/libpam-modules': Directory not empty
rm: cannot remove '/usr/share/doc/util-linux': Directory not empty
rm: cannot remove '/usr/share/doc/mawk': Directory not empty
rm: cannot remove '/usr/share/doc/gnupg': Directory not empty
rm: cannot remove '/usr/share/doc/apt': Directory not empty
rm: cannot remove '/usr/share/doc/libreadline6': Directory not empty
rm: cannot remove '/usr/share/doc/adduser/examples/adduser.local.conf.examples': Directory not empty
rm: cannot remove '/usr/share/doc/passwd': Directory not empty
rm: cannot remove '/usr/share/doc/mount': Directory not empty
rm: cannot remove '/usr/share/doc/procps': Directory not empty

还有 sudo

$ sudo docker run --rm -it arm64v8/ubuntu:latest rm -rf /usr/share/doc
rm: cannot remove '/usr/share/doc/sed': Directory not empty
rm: cannot remove '/usr/share/doc/libpam-modules': Directory not empty
rm: cannot remove '/usr/share/doc/util-linux': Directory not empty
rm: cannot remove '/usr/share/doc/mawk': Directory not empty
rm: cannot remove '/usr/share/doc/gnupg': Directory not empty
rm: cannot remove '/usr/share/doc/apt': Directory not empty
rm: cannot remove '/usr/share/doc/libreadline6': Directory not empty
rm: cannot remove '/usr/share/doc/adduser/examples/adduser.local.conf.examples': Directory not empty
rm: cannot remove '/usr/share/doc/passwd': Directory not empty
rm: cannot remove '/usr/share/doc/mount': Directory not empty
rm: cannot remove '/usr/share/doc/procps': Directory not empty

这未在 Docker Issues 中列为已知问题。如何进一步调试问题？

Answer 1

在 Docker overlay storage driver unable to delete directories from underlying image layer #35257 将其报告为问题后, 感谢 thaJeztah，我查到了真相。

记录在 https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/ ，覆盖存储驱动程序需要内核版本 3.18 或更高版本。

我的系统使用图像 ArchLinuxARM-odroid-c2-latest.tar.gz它具有较旧的内核版本 3.14。

出现此问题是因为 ALARM docker 包默认配置了覆盖驱动程序，该驱动程序仅适用于较新的内核版本(例如 ODROID-C2 的 RPi 镜像或主线内核)。

因此，解决方案是要么

• 升级内核版本。上次我检查时，ODROID-C2 的主线内核支持存在问题，因此下一个选项可能更可取。
• 或者将 docker 配置为使用 AUFS 存储驱动程序而不是覆盖驱动程序。