perl - 为什么我的 'use my_module;'占用这么多堆内存？

Question

此示例脚本:

#!/usr/bin/perl -w

while (1)
{
  sleep(1);
}

大约需要 264 kB

grep -A1 heap  /proc/9216/smaps 
0817b000-081bd000 rw-p 0817b000 00:00 0          [heap]
Size:               264 kB

但是当我只添加我的模块时:

#!/usr/bin/perl -w

use my_module;

while (1)
{
  sleep(1);
}

需要 18092 kB!

grep -A1 heap  /proc/9219maps 
0817b000-09326000 rw-p 0817b000 00:00 0          [heap]
Size:             18092 kB

注意:'my_module'有很多'use module;'里面。

我怎样才能找到占用这么多内存的东西？

我怎样才能减少它？ (使用“使用模块(函数)”？)

感谢您的帮助。

Answer 1

插入 BEGIN {} block 以缩小问题的范围，如

#! /usr/bin/perl

sub grep_heap { print @_, "\n"; system "grep -A1 heap /proc/$$/smaps" }

BEGIN { grep_heap "<null>" }
use warnings;
BEGIN { grep_heap "+warnings" }
use strict;
BEGIN { grep_heap "+strict" }

use Data::Dumper;
BEGIN { grep_heap "+Data::Dumper" }
use POSIX;
BEGIN { grep_heap "+POSIX" }

print "Hi\n";

在我的 Linux 主机上，我看到了

$ ./prog.pl<null>0889b000-088de000 rw-p 0889b000 00:00 0                                  [heap]Size:               268 kB+warnings0889b000-08920000 rw-p 0889b000 00:00 0                                  [heap]Size:               532 kB+strict0889b000-08920000 rw-p 0889b000 00:00 0                                  [heap]Size:               532 kB+Data::Dumper0889b000-089a4000 rw-p 0889b000 00:00 0                                  [heap]Size:              1060 kB+POSIX0889b000-08ace000 rw-p 0889b000 00:00 0                                  [heap]Size:              2252 kBHi

As for what to do about it, you could implement replacement modules with scaled down functionality or ask yourself whether you really need a particular module at all. However, in general, perl's design prefers to throw memory at problems, and these days it's common for machines to have multiple gibibytes of main memory.

Is this resource issue causing performance problems?

Below is a program that reads through the list of modules in perlmodlib.pod and for each module forks a child to require and import it and check its own heap.

#! /usr/bin/perl

sub heap {
  my($heap) = @_;
  unless ($heap =~ /^([0-9a-f]+)-([0-9a-f]+)/m) {
    warn "$0: unexpected heap:\n$heap";
    return -1;
  }
  hex($2) - hex($1);
}

sub size {
  my($bytes) = @_;

  my @units = (
    [ MiB   => "%.1f", 1_024 * 1_024 ],
    [ KiB   => "%.1f", 1_024 ],
  );

  for (@units) {
    my($unit,$fmt,$n) = @$_;
    return sprintf "$fmt %s" => $bytes/$n, $unit
      if $bytes >= $n;
  }

  return "$bytes byte" . ($bytes == 1 ? "" : "s");
}

my %incr;

my $perlmodlib = `perldoc -l perlmodlib`;
die "$0: perldoc failed" unless defined $perlmodlib;

my $base = heap `grep heap /proc/$$/smaps`;
warn "$0: base=" . size($base) . "\n";

chomp $perlmodlib;
open my $fh, "<", $perlmodlib   or die "$0: open $perlmodlib: $!";

while (<$fh>) {
  next unless /^=head2 Pragmatic Modules/ ..
              /^=head2 Extension Modules/;

  if (/^=item (\w+(::\w+)*)/) {
    my $mod = $1;
    (my $path = "$mod.pm") =~ s!::!/!g;

    my $pid = open my $fh, "-|";
    die "$0: fork: $!" unless defined $pid;

    if ($pid == 0) {
      open STDERR, ">", "/dev/null" or warn "$0: open: $!";
      exec "perl", "-e", <<EOProgram;
BEGIN {
  require \"$path\";
  eval { $mod->import };
  system qq(grep heap /proc/\$\$/smaps);
}
EOProgram
      die "$0: exec: $!";
    }
    else {
      local $/;
      my $heap = <$fh>;
      unless (defined $heap && length $heap) {
        warn "$0: use $mod failed";
        next;
      }
      $heap = heap $heap;
      $incr{$mod} = $heap > 0 ? $heap-$base : $heap;
    }
  }
}

foreach my $mod (sort { $incr{$b} <=> $incr{$a} } keys %incr) {
  print "$mod - ", size($incr{$mod}), "\n";
}

perl-5.8.8 的前几名违规者:

CPAN::Nox - 9.7 MiBCPAN - 9.6 MiBExtUtils::MM_VMS - 5.3 MiBCPAN::FirstTime - 5.2 MiBExtUtils::Installed - 5.2 MiBB::CC - 5.2 MiBbigrat - 4.9 MiBMath::BigRat - 4.8 MiBExtUtils::MM_NW5 - 4.7 MiBExtUtils::MM_OS2 - 4.6 MiBExtUtils::MM_Win32 - 4.6 MiBExtUtils::MM_Win95 - 4.6 MiB