PHP 脚本在每次迭代中花费越来越长的时间来保存数据

Question

我编写了一个自动化脚本来解析本地的 XML 文件，从 XML 中提取一些 url，然后下载 url 指向的文件并存储数据(所有 jpg)。每个后续迭代都逐渐花费越来越长的时间。最终，脚本只是挂起，什么也没有发生。哦，内存使用似乎保持不变。 XML 解析工作正常，所以请不要过多关注它。如果我在脚本完全停止的游戏中启动脚本，则会在一秒钟内将文件保存到磁盘，因为这是脚本执行的第一次保存。是什么原因造成的？这是代码:

    <?php

$xml = simplexml_load_file('./playfin.xml', null, LIBXML_NOCDATA);

echo("Getting urls from the XML...\n");
$data = array();
foreach($xml->game as $game) {
  $smlBox = $game->small_boxshot_image->url;
  $lrgBox = $game->large_boxshot_image->url;

  foreach($game->screenshots->screenshot_image as $image) {
    $scrShots[] = array(
      'smlScr' => (string)$image->small_screenshot_image->url,
      'lrgScr' => (string)$image->large_screenshot_image->url  
    );
  }
  $data[] = array(
    'id' => (int)$game->game_id,
    'smlBox' => (string)$smlBox,
    'lrgBox' => (string)$lrgBox,
    'scrShots' => $scrShots
  );
} 
echo("Done pulling urls from XML...\n");

echo("Storing picture data to disk...\n");
$start = time();
$total = 0;
$now = 0;
for($i = 0; $i < count($data); $i++) {
  $total += $now;
  $now = time() - $start - $total;
  echo("Writing game id: ".$data[$i]['id']. 
    ." to disk at time=".(string)$now."          seconds.\n");
  echo("Memory usage: ".(string)memory_get_usage(true)."\n\n");
  $smlBoxStr = '%s_49_60.jpg';
  $lrgBoxStr = '%s_160_189.jpg';
  $smlScrStr = '%s_115_86_%d.jpg';
  $lrgScrStr = '%s_300_300_%d.jpg';

  writeData($data[$i]['smlBox'], sprintf($smlBoxStr, $data[$i]['id']), 'boxshot');
  writeData($data[$i]['lrgBox'], sprintf($lrgBoxStr, $data[$i]['id']), 'boxshot');
  for($j = 0; $j < count($data[$i]['scrShots']); $j++) {
    writeData(
      $data[$i]['scrShots'][$j]['smlScr'], 
      sprintf($smlScrStr, $data[$i]['id'], $j),
      'screenshot'
    );
    writeData(
      $data[$i]['scrShots'][$j]['lrgScr'], 
      sprintf($lrgScrStr, $data[$i]['id'], $j),
      'screenshot'
    );
  }  
}
echo("Done storing!\n");
echo("Done!\n");

function writeData($url, $file, $type) {
  $headers = get_headers($url, 1);
  $awayPic = fopen($url, 'rb');
  $localPic = fopen("./$type/$file", 'wb');
  while($picData = fread($awayPic, (int)$headers['Content-Length'])) {
    fwrite($localPic, $picData);
  }
  fclose($awayPic);
  fclose($localPic);
}  

?>

这是生成的日志文件:

Getting urls from the XML... Done pulling urls from XML... Storing picture data to disk... Writing game id: 671850 to disk at time=0 seconds. Memory usage: 77856768

Writing game id: 730950 to disk at time=0 seconds. Memory usage: 77856768

Writing game id: 621650 to disk at time=1 seconds. Memory usage: 77856768

Writing game id: 687250 to disk at time=1 seconds. Memory usage: 77856768

Writing game id: 633950 to disk at time=7 seconds. Memory usage: 77856768

Writing game id: 633850 to disk at time=2 seconds. Memory usage: 77856768

Writing game id: 720950 to disk at time=2 seconds. Memory usage: 77856768

Writing game id: 572250 to disk at time=3 seconds. Memory usage: 77856768

Writing game id: 674950 to disk at time=3 seconds. Memory usage: 77856768

Writing game id: 731450 to disk at time=3 seconds. Memory usage: 77856768

Writing game id: 656350 to disk at time=4 seconds. Memory usage: 77856768

Writing game id: 653550 to disk at time=4 seconds. Memory usage: 77856768

Writing game id: 585550 to disk at time=4 seconds. Memory usage: 77856768

Writing game id: 736750 to disk at time=5 seconds. Memory usage: 77856768

Writing game id: 671350 to disk at time=5 seconds. Memory usage: 77856768

Writing game id: 696250 to disk at time=5 seconds. Memory usage: 77856768

Writing game id: 645550 to disk at time=6 seconds. Memory usage: 77856768

Writing game id: 625650 to disk at time=6 seconds. Memory usage: 77856768

Writing game id: 696850 to disk at time=6 seconds. Memory usage: 77856768

Writing game id: 709550 to disk at time=7 seconds. Memory usage: 77856768

Writing game id: 575750 to disk at time=7 seconds. Memory usage: 77856768

Writing game id: 651950 to disk at time=7 seconds. Memory usage: 77856768

Writing game id: 685350 to disk at time=8 seconds. Memory usage: 77856768

Writing game id: 724150 to disk at time=9 seconds. Memory usage: 77856768

Writing game id: 522250 to disk at time=8 seconds. Memory usage: 77856768

Writing game id: 610350 to disk at time=10 seconds. Memory usage: 77856768

Writing game id: 645050 to disk at time=9 seconds. Memory usage: 77856768

Writing game id: 716950 to disk at time=9 seconds. Memory usage: 77856768

Writing game id: 672750 to disk at time=10 seconds. Memory usage: 77856768

Writing game id: 568650 to disk at time=11 seconds. Memory usage: 77856768

Writing game id: 668650 to disk at time=11 seconds. Memory usage: 77856768

Writing game id: 417950 to disk at time=11 seconds. Memory usage: 77856768

Writing game id: 497950 to disk at time=13 seconds. Memory usage: 77856768

Writing game id: 567950 to disk at time=12 seconds. Memory usage: 77856768

Writing game id: 692350 to disk at time=13 seconds. Memory usage: 77856768

Writing game id: 450950 to disk at time=13 seconds. Memory usage: 77856768

Writing game id: 452750 to disk at time=14 seconds. Memory usage: 77856768

Writing game id: 666450 to disk at time=14 seconds. Memory usage: 77856768

Writing game id: 754550 to disk at time=15 seconds. Memory usage: 77856768

Writing game id: 659050 to disk at time=15 seconds. Memory usage: 77856768

Writing game id: 712350 to disk at time=15 seconds. Memory usage: 77856768

Writing game id: 719250 to disk at time=16 seconds. Memory usage: 77856768

Writing game id: 529250 to disk at time=15 seconds. Memory usage: 77856768

Writing game id: 685150 to disk at time=17 seconds. Memory usage: 77856768

Writing game id: 736450 to disk at time=17 seconds. Memory usage: 77856768

Writing game id: 252750 to disk at time=17 seconds. Memory usage: 77856768

Writing game id: 719150 to disk at time=17 seconds. Memory usage: 77856768

Writing game id: 461150 to disk at time=18 seconds. Memory usage: 77856768

Writing game id: 699450 to disk at time=18 seconds. Memory usage: 77856768

Writing game id: 523550 to disk at time=18 seconds. Memory usage: 77856768

Writing game id: 451050 to disk at time=20 seconds. Memory usage: 77856768

Writing game id: 768350 to disk at time=19 seconds. Memory usage: 77856768

Writing game id: 724650 to disk at time=20 seconds. Memory usage: 77856768

Writing game id: 676550 to disk at time=21 seconds. Memory usage: 77856768

Writing game id: 730850 to disk at time=21 seconds. Memory usage: 77856768

Writing game id: 558250 to disk at time=22 seconds. Memory usage: 77856768

Writing game id: 674750 to disk at time=22 seconds. Memory usage: 77856768

Writing game id: 695450 to disk at time=22 seconds. Memory usage: 77856768

Writing game id: 682950 to disk at time=22 seconds. Memory usage: 77856768

Writing game id: 706450 to disk at time=24 seconds. Memory usage: 77856768

Writing game id: 546450 to disk at time=24 seconds. Memory usage: 77856768

Writing game id: 575350 to disk at time=24 seconds. Memory usage: 77856768

Writing game id: 616550 to disk at time=25 seconds. Memory usage: 77856768

Writing game id: 648250 to disk at time=26 seconds. Memory usage: 77856768

Writing game id: 763750 to disk at time=25 seconds. Memory usage: 77856768

Writing game id: 613850 to disk at time=26 seconds. Memory usage: 77856768

Writing game id: 645450 to disk at time=25 seconds. Memory usage: 77856768

Writing game id: 695950 to disk at time=33 seconds. Memory usage: 77856768

Writing game id: 661050 to disk at time=27 seconds. Memory usage: 77856768

Writing game id: 461050 to disk at time=27 seconds. Memory usage: 77856768

Writing game id: 693150 to disk at time=29 seconds. Memory usage: 77856768

谢谢!

Answer 1

问题是您在解析 XML 时没有在外循环中清除 $scrShots。因此，每次迭代不仅包含当前游戏的屏幕截图，还包含所有之前的游戏。 (这样，你下载第一个游戏的图像，然后是第一个游戏的图像加上第二个游戏的图像，然后是第一个加第二个加第三个，依此类推，这当然需要越来越长的时间。)

尝试在添加下一轮之前清空数组:

$scrShots = array();
foreach($game->screenshots->screenshot_image as $image) {
    $scrShots[] = array(
    ...