amazon-s3 - 从 Amazon S3 中提取文件和元数据的有效方法？

Question

是否有更有效的方法来列出 Amazon S3 中存储桶中的文件并提取每个文件的元数据？我正在使用 AWS PHP SDK。

if ($paths = $s3->get_object_list('my-bucket')) {
    foreach($paths AS $path) {
        $meta = $s3->get_object_metadata('my-bucket', $path);
        echo $path . ' was modified on ' . $meta['LastModified'] . '<br />';
    }
}

目前我需要运行 get_object_list()列出所有文件，然后 get_object_metadata()为每个文件获取其元数据。

如果我的存储桶中有 100 个文件，它会调用 101 次来获取这些数据。如果可以在 1 个电话中完成，那就太好了。

例如:

if ($paths = $s3->get_object_list('my-bucket')) {
    foreach($paths AS $path) {
        echo $path['FileName'] . ' was modified on ' . $path['LastModified'] . '<br />';
    }
}

Answer 1

我知道这有点旧，但我遇到了这个问题，为了解决它，我扩展了 Aws sdk 以使用批处理功能来解决此类问题。它可以更快地检索大量文件的自定义元数据。
这是我的代码:

    /**
     * Name: Steves_Amazon_S3
     * 
     * Extends the AmazonS3 class in order to create a function to 
     * more efficiently retrieve a list of
     * files and their custom metadata using the CFBatchRequest function.
     * 
     * 
     */
    class Steves_Amazon_S3 extends AmazonS3 {

        public function get_object_metadata_batch($bucket, $filenames, $opt = null) {
            $batch = new CFBatchRequest();

            foreach ($filenames as $filename) {

                $this->batch($batch)->get_object_headers($bucket, $filename); // Get content-type
            }

            $response = $this->batch($batch)->send();

            // Fail if any requests were unsuccessful
            if (!$response->areOK()) {
                return false;
            }
            foreach ($response as $file) {
                $temp = array();
                $temp['name'] = (string) basename($file->header['_info']['url']);
                $temp['etag'] = (string) basename($file->header['etag']);
                $temp['size'] = $this->util->size_readable((integer) basename($file->header['content-length']));
                $temp['size_raw'] = basename($file->header['content-length']);
                $temp['last_modified'] = (string) date("jS M Y H:i:s", strtotime($file->header['last-modified']));
                $temp['last_modified_raw'] = strtotime($file->header['last-modified']);
                @$temp['creator_id'] = (string) $file->header['x-amz-meta-creator'];
                @$temp['client_view'] = (string) $file->header['x-amz-meta-client-view'];
                @$temp['user_view'] = (string) $file->header['x-amz-meta-user-view'];

                $result[] = $temp;
            }

            return $result;
        }
    }