amazon-web-services - AWS 服务通过校验和验证 S3 中文件的数据完整性？

Question

确保 S3 中的文件如其所声称的那样的一种方法是下载它、获取其校验和并将结果与​​您期望的校验和进行匹配。

AWS 是否提供任何服务，允许用户先下载文件？ (即理想情况下，提供 S3 文件校验和的简单请求/url，以便可以在下载文件之前对其进行验证)

到目前为止我尝试了什么

我可以按照以下思路想到一个 DIY 解决方案

• 创建一个接受带有 S3 文件 url 的 POST 请求的 API 端点
• 让 API 运行一个 lambda 来生成文件的校验和
• 返回校验和值

这可能可行，但已经有点复杂，需要进一步考虑，例如大文件可能需要很长时间才能生成校验和(例如 > 60 秒)

我希望 AWS 有一些简单的方法来验证 S3 文件？

Answer 1

针对每个对象创建了一个 ETag，它是对象内容的 MD5。

不过，似乎也有一些异常(exception)。

来自 Common Response Headers - Amazon Simple Storage Service :

ETag: The entity tag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. The ETag may or may not be an MD5 digest of the object data. Whether or not it is depends on how the object was created and how it is encrypted as described below:

• Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-S3 or plaintext, have ETags that are an MD5 digest of their object data.

• Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-C or SSE-KMS, have ETags that are not an MD5 digest of their object data.

• If an object is created by either the Multipart Upload or Part Copy operation, the ETag is not an MD5 digest, regardless of the method of encryption.

此外，分段上传的 ETag 计算可能很复杂。请参阅:s3cmd - What is the algorithm to compute the Amazon-S3 Etag for a file larger than 5GB? - Stack Overflow