linux - 使用 awk linux 操作文件

Question

我有一个 1.txt 文件(字段分隔符为 ||o||):

aidagolf6@gmail.com||o||bb1e6b92d60454122037f302359d8a53||o||Aida||o||Aida||o||Muji?
aidagolf6@gmail.com||o||bcfddb5d06bd02b206ac7f9033f34677||o||Aida||o||Aida||o||Muji?
aidagolf6@gmail.com||o||bf6265003ae067b19b88fa4359d5c392||o||Aida||o||Aida||o||Garic Gara
aidagolf6@gmail.com||o||d3a6a8b1ed3640188e985f8a1efbfe22||o||Aida||o||Aida||o||Muji?
aidagolfa@hotmail.com||o||14f87ec1e760d16c0380c74ec7678b04||o||Aida||o||Aida||o||Rodriguez Puerto

2.txt(字段分隔符为:):

bf6265003ae067b19b88fa4359d5c392:hyworebu:@
14f87ec1e760d16c0380c74ec7678b04:sujycugu

我有一个 result.txt 文件(它将匹配 1.txt 的第二列和 2.txt 的第一列，如果结果匹配，它将用 2.txt 的第二列替换 1.txt 的第二列)

aidagolf6@gmail.com||o||hyworebu:@||o||Aida||o||Aida||o||Garic Gara
aidagolfa@hotmail.com||o||sujycugu||o||Aida||o||Aida||o||Rodriguez Puerto

还有一个 left.txt 文件(其中包含 1.txt 中的不匹配行，但在 2.txt 中没有匹配项):

aidagolf6@gmail.com||o||d3a6a8b1ed3640188e985f8a1efbfe22||o||Aida||o||Aida||o||Muji?
aidagolf6@gmail.com||o||bb1e6b92d60454122037f302359d8a53||o||Aida||o||Aida||o||Muji?
aidagolf6@gmail.com||o||bcfddb5d06bd02b206ac7f9033f34677||o||Aida||o||Aida||o||Muji?

我正在尝试的脚本是:

awk -F '[|][|]o[|][|]' -v s1="||o||"  '
NR==FNR {
a[$2] = $1; 
b[$2]= $3s1$4s1$5; 
next
} 
($1 in a){
$1 = "";
sub(/:/, "")
print a[$1]s1$2s1b[$1] > "result.txt";
next
}' 1.txt 2.txt

问题是脚本在 2.txt 中使用 ||o|| 也是因为我得到了错误的结果。

EDIT

修改后的脚本:

awk -v s1="||o||"  '
NR==FNR {
a[$2] = $1; 
b[$2]= $3s1$4s1$5; 
next
} 
($1 in a){
$1 = "";
sub(/:/, "")
print a[$1]s1$2s1b[$1] > "result.txt";
next
}' FS = "||o||" 1.txt FS = ":" 2.txt

现在，我收到以下错误:

awk: fatal: cannot open file `FS' for reading (No such file or directory)

Answer 1

我已经修改了你的原始脚本:

awk -F'[|][|]o[|][|]' -v s1="||o||" '

NR == FNR {
    a[$2] = $1; 
    b[$2] = $3 s1 $4 s1 $5;
    c[$2] = $0;                     # keep the line for left.txt
}

NR != FNR {
    split($0, d, ":");
    r = substr($0, index($0, ":") + 1);     # right side of the 1st ":"
    if (a[d[1]] != "") {
        print a[d[1]] s1 r s1 b[d[1]] > "result.txt";
            c[d[1]] = "";           # drop from the list of left.txt
    }
}

END {
    for (var in c) {
        if (c[var] != "") {
            print c[var] > "left.txt"
        }
    }
}' 1.txt 2.txt

下一版本改变文件读取顺序以减少内存消耗:

awk -F'[|][|]o[|][|]' -v s1="||o||" '

NR == FNR {
    split($0, a, ":");
        r = substr($0, index($0, ":") + 1); # right side of the 1st ":"
    map[a[1]] = r; 
}

NR != FNR {
    if (map[$2] != "") {
        print $1 s1 map[$2] s1 $3 s1 $4 s1 $5 > "result.txt";
    } else {
        print $0 > "left.txt"
    }
}' 2.txt 1.txt

最终版本使用基于文件的数据库，最大限度地减少了 DRAM 消耗，尽管我不确定 Perl 是否可以在您的系统中接受。

perl -e '
use DB_File;

$file1 = "1.txt";
$file2 = "2.txt";
$result = "result.txt";
$left = "left.txt";

my $dbfile = "tmp.db";
tie(%db, "DB_File", $dbfile, O_CREAT|O_RDWR, 0644) or die "$dbfile: $!";

open(FH, $file2) or die "$file2: $!";
while (<FH>) {
    chop;
    @_ = split(/:/, $_, 2);
    $db{$_[0]} = $_[1];
}
close FH;
open(FH, $file1) or die "$file1: $!";
open(RESULT, "> $result") or die "$result: $!";
open(LEFT, "> $left") or die "$left: $!";

while (<FH>) {
    @_ = split(/\|\|o\|\|/, $_);
    if (defined $db{$_[1]}) {
        $_[1] = $db{$_[1]};
        print RESULT join("||o||", @_);
    } else {
        print LEFT $_;
    }
}
close FH;
untie %db;
'
rm tmp.db