linux - 远程检查 linux 中是否存在所有文件？

Question

我有一个下面的脚本，它迭代所有机器并远程检查每台机器:

• 是否有特定目录？
• 所有 2000 个文件是否存在于该目录中。

代码如下:

for machine in "${MACHINES[@]}"; do
   dircheck=($(ssh -o "StrictHostKeyChecking no" user@${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))
   if [[ $? != 0 ]] ;then
       echo "Folder $dir3 doesn't exist on $machine" >&2
       exit 1
   fi

    # this is for checking all the 2000 files
    # this check is very slow when it checks all 2000 files
    for el in ${FILES[@]}
    do
        printf -v cmd '%q ' test -e "$dir3/process_${el}.log"
        ssh user@${machine} "$cmd" \
        || { echo "File number $el missing on $machine." >&2;
             exit 1; }
    done
done

现在的问题是检查所有 2000 个文件需要很多时间，所以想看看是否有任何方法我们仍然可以做同样的事情但速度有点快？

更新:

总的来说，我的脚本是这样的:

readonly MACHINES=(machineA machineB machineC)
readonly dir3=/some_path
echo $dir3
FILES=({0..1999})

checkFunc() {
  test -d "$dir3" || echo "NODIR"

  local filename successCount=0
  while IFS= read -r filename; do
    test -e "$dir3/process_${filename}.log" && (( ++successCount ))
  done
  printf '%s\n' "$successCount"
}

for machine in "${MACHINES[@]}"; do
    actual=$(
      printf '%s\0' "${FILES[@]}" | \
        ssh "$machine" "$(declare -p dir3; declare -f checkFunc); checkFunc"
    ) || { echo "ERROR: Unable to retrieve remote file count" >&2; exit 1; }

    case $actual in
      (${#FILES[@]}) echo "SUCCESS: Expected, and found, $numberOfActuallyRemoteFiles files" ;;
      (NODIR)        echo "FAILURE: Directory $dir3 does not exist" ;;
      (*)            echo "FAILURE: Out of ${#FILES[@]} files, only $actual exist" ;;
    esac
done

Answer 1

与其创建 2,000 个单独的 SSH 连接，不如将整个循环推送到远程端:

# shell function which reads a NUL-delimited list of filenames on stdin
# and returns the number of them that actually exist on stdout, or "NODIR"
checkFunc() {
  test -d "$dir3" || { echo "NODIR"; return; }

  local filename successCount=0
  while IFS= read -r -d '' filename; do
    test -e "$dir3/process_${filename}.log" && (( ++successCount ))
  done
  printf '%s\n' "$successCount"
}

actual=$(
  printf '%s\0' "${FILES[@]}" | \
    ssh "$machine" "$(declare -p dir3; declare -f checkFunc); checkFunc"
) || { echo "ERROR: Unable to retrieve remote file count" >&2; exit 1; }

case $actual in
  (${#FILES[@]}) echo "SUCCESS: Expected, and found, $numberOfActuallyRemoteFiles files" ;;
  (NODIR)        echo "FAILURE: Directory $dir3 does not exist" ;;
  (*)            echo "FAILURE: Out of ${#FILES[@]} files, only $actual exist" ;;
esac

请注意 declare -p dir3 和 declare -f checkFunc 发出字符串，当由 bash 执行时，这些字符串将定义 dir3 变量或checkFunc 函数。