linux - 如何在 shell 脚本中异步运行迭代

Question

我有几个 .csv 文件，如下所示。

1. xyz0900@1#-1637746436.csv
2. xxx0900@1#-1637746436.csv
3. zzz0900@2#-1637746439.csv
4. yyy0900@1#-1637746436.csv
5. sss0900@2#-1637746439.csv

我编写了一个脚本来执行以下任务:

1. 根据我们作为参数传递给脚本的模式获取大文件。
2. 合并所有其他具有相同模式的文件并创建一个新文件
3. 从新文件中删除重复的标题。
4. 根据作为参数传递的参数将新文件移动到目的地。

Example: I am passing "1637746436@home/dest1,1637746436@home/dest2" as a second argument to the script. Below script will fetch the pattern(1637746436). Get the bigger file and merge all other files(having same pattern) with it. New file will be get created and same will be moved to the destination(home/dest1).

下面的脚本会依次进行模式匹配和执行。

如何使'for循环迭代'应该并行执行？我的意思是“1637746436@home/dest1,1637746436@home/dest2”的模式匹配应该同时执行(而不是一个接一个地执行).

请帮忙解决这个问题。

$merge.sh /home/dummy/17 "1637746436@home/dest1,1637746439@home/dest2" 


 #!/bin/bash
 current=`pwd`
 source=$1
 destination=$2
 echo "$destination" | tr "," "\n" > $current/out.txt
 cat out.txt | cut -d "@" -f1 > $current/pattern.txt

 for var in `cat pattern.txt`
 do
 getBiggerfile=$(ls -Sl $source/*$var.csv | head -1)
 cd $source
 getFileName=$(echo $getBiggerfile | cut -d " " -f9-)
 newFileName=$(echo $getFileName | cut -d "@" -f1)
 cat *$var.csv >> $getFileName
 header=$(head -n 1 $getFileName)
(printf "%s\n" "$header";
 grep -vFxe "$header" $getFileName
 ) > $newFileName.csv
 rm -rf *$var.csv
 cd $current

 for var1 in `cat out.txt`
 do
 target=`echo $var1 | cut -d "@" -f2`
 id=$(echo $var1 | cut -c-10)
 if [ $id = $var ]
 then
 mv $newFileName.csv $target
 fi
 done
done

Answer 1

最干净的方法是让循环内部成为一个函数，并在循环内部调用该函数，将其置于后台(子进程)，然后等待后台(子)进程完成:

function do_the_thing(){
    source="$1"
    current="$2"
    var="$3"
    getBiggerfile=$(ls -Sl $source/*$var.csv | head -1)
    cd $source
    getFileName=$(echo $getBiggerfile | cut -d " " -f9-)
    newFileName=$(echo $getFileName | cut -d "@" -f1)
    cat *$var.csv >> $getFileName
    header=$(head -n 1 $getFileName)
    (printf "%s\n" "$header";
       grep -vFxe "$header" $getFileName
    ) > $newFileName.csv
    rm -rf *$var.csv
    cd $current

    for var1 in `cat out.txt`
    do
        target=`echo $var1 | cut -d "@" -f2`
        id=$(echo $var1 | cut -c-10)
        if [ $id = $var ]
        then
            mv $newFileName.csv $target
        fi
    done
}

for var in `cat pattern.txt`
do
    do_the_thing "$source" "$current" "$var" &
done

wait