python - 使用 boto3 从 S3 存储桶中读取文件内容

Question

我通过执行读取 S3 存储桶中的文件名

objs = boto3.client.list_objects(Bucket='my_bucket')
    while 'Contents' in objs.keys():
        objs_contents = objs['Contents']
        for i in range(len(objs_contents)):
            filename = objs_contents[i]['Key']

现在，我需要获取文件的实际内容，类似于 open(filename).readlines()。最好的方法是什么？

Answer 1

boto3 提供了一种资源模型，使诸如遍历对象之类的任务变得更加容易。不幸的是，StreamingBody 不提供 readline 或 readlines。

s3 = boto3.resource('s3')
bucket = s3.Bucket('test-bucket')
# Iterates through all the objects, doing the pagination for you. Each obj
# is an ObjectSummary, so it doesn't contain the body. You'll need to call
# get to get the whole body.
for obj in bucket.objects.all():
    key = obj.key
    body = obj.get()['Body'].read()