linux - 查找和删除重复文件 (md5sum)

Question

我有多个包含多个图像的目录，其中一些目录有重复的图像。我想在同一目录中找到所有重复的图像并将其删除。下面是我的代码。

我在删除重复图像时遇到问题。该代码可以识别重复的文件，但是当它尝试删除它时，它会显示此消息“rm:无法删除'FILENAME':没有这样的文件或目录”

for dir in *; do
count=1
for file in $dir/*.*; do
     md5sum * | sort | awk 'BEGIN{lasthash = ""} $1 == lasthash {print $2} {lasthash = $1}' | xargs rm
     let count=count+1
done
done

Answer 1

xargs 联机帮助页的以下摘录可能会解释您所看到的内容:

find /tmp -name core -type f -print | xargs /bin/rm -f

Find files named core in or below the directory /tmp and delete them. Note that this will work incorrectly if there are any filenames containing newlines or spaces.

 find /tmp -name core -type f -print0 | xargs -0 /bin/rm -f

Find files named core in or below the directory /tmp and delete them, processing filenames in such a way that file or directory names containing spaces or newlines are correctly handled.

如果一个文件有一个带空格的名字，比如my vacation in thai.jpg，xargs默认做的是在空格处分割它并调用多个rm:

rm my
rm vacation
rm in
rm thai.jpg

您需要让 awk 打印以 null 结尾的字符串并使用 xargs -0 来使用它们。在这个问题中:How can I output null-terminated strings in Awk?建议使用这一行:

  awk '{printf "%s\0", $0}'