Perl Parallel::ForkManager wait_all_children() 耗时过长

Question

我有一个使用 Parallel::ForkManager 的脚本。然而，即使在所有子进程都完成之后，wait_all_children() 进程也会花费非常长的时间。我知道的方法是打印出一些时间戳(见下文)。有谁知道这可能是什么原因(我的机器上有 16 个 CPU 内核)？

my $pm = Parallel::ForkManager->new(16);
for my $i (1..16) {
    $pm->start($i) and next;

    ... do something within the child-process ...

    print (scalar localtime), " Process $i completed.\n";
    $pm->finish();
}
print (scalar localtime), " Waiting for some child process to finish.\n"; 
$pm->wait_all_children();
print (scalar localtime), " All processes finished.\n"; 

显然，我将首先收到 Waiting for some child process to finish 消息，其时间戳为 7:08:35。然后我会得到一个Process i completed 消息列表，最后一个在7:10:30。但是，直到 7:16:33(!) 我才收到消息 All Processes finished。为什么在 7:10:30 和 7:16:33 之间会有 6 分钟的延迟？谢谢!

Answer 1

我试过了:

#!/opt/perl/bin/perl

use strict; use warnings;

use Parallel::ForkManager;

my $pm = Parallel::ForkManager->new(16);

for my $i (1..16) {
    $pm->start($i) and next;
    sleep rand 20;
    printf "%s : Process %d completed\n", scalar localtime, $i;
    $pm->finish;
}

printf "%s: Waiting for some child to finish\n", scalar localtime;
$pm->wait_all_children;

printf "%s: All processes finished.\n", scalar localtime; 

我明白了:

[sinan@archardy Src]$ ./y.plThu Mar 11 17:14:16 2010 : Process 3 completedThu Mar 11 17:14:16 2010: Waiting for some child to finishThu Mar 11 17:14:18 2010 : Process 8 completedThu Mar 11 17:14:18 2010 : Process 14 completed<snip>...</snip>Thu Mar 11 17:14:34 2010 : Process 12 completedThu Mar 11 17:14:34 2010: All processes finished.

I have perl 5.10.1 on Linux with Parallel::ForkManager version 0.7.5.

Therefore, I conclude that whatever issue you are having is happening as a consequence of what happens when you

# ... do something within the child-process ...

更新:问题是，您在 finish 调用之前打印 Process finished 消息。试试下面的版本:

#!/opt/perl/bin/perl

use strict; use warnings;

use Parallel::ForkManager;

my $pm = Parallel::ForkManager->new(16);
$pm->run_on_finish( sub {
    printf "%s : Process completed: @_\n", scalar localtime
});

for my $i (1..16) {
    $pm->start($i) and next;
    sleep rand 20;
    $pm->finish;
}

printf "%s: Waiting for some child to finish\n", scalar localtime;
$pm->wait_all_children;

printf "%s: All processes finished.\n", scalar localtime;

见 Callbacks in Parallel::ForkManager documentation了解更多信息。如果延迟消失，那么您观察到的症状是由于您声称 fork 进程在完成之前已经完成。