amazon-web-services - 如何设置 InputStream 内容长度

Question

我正在将文件上传到 Amazon S3 存储桶。文件正在上传，但我收到以下警告。

WARNING: No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.

所以我将以下行添加到我的代码中

metaData.setContentLength(IOUtils.toByteArray(input).length);

但后来我收到了以下消息。我什至不知道这是警告还是什么。

Data read has a different length than the expected: dataLength=0; expectedLength=111992; includeSkipped=false; in.getClass()=class sun.net.httpserver.FixedLengthInputStream; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0

如何将 contentLength 设置为 InputSteam 的元数据？任何帮助将不胜感激。

Answer 1

当你用IOUtils.toByteArray读取数据时，这会消耗 InputStream。当 AWS API 尝试读取它时，它的长度为零。
将内容读入一个字节数组，并提供一个包装该数组的 InputStream 给 API:

byte[] bytes = IOUtils.toByteArray(input);
metaData.setContentLength(bytes.length);
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, byteArrayInputStream, metadata);
client.putObject(putObjectRequest);

您应该考虑使用分段上传 API 以避免将整个 InputStream 加载到内存中。例如:

byte[] bytes = new byte[BUFFER_SIZE];
String uploadId = client.initiateMultipartUpload(new InitiateMultipartUploadRequest(bucket, key)).getUploadId();

int bytesRead = 0;
int partNumber = 1;
List<UploadPartResult> results = new ArrayList<>();
bytesRead = input.read(bytes);
while (bytesRead >= 0) {
    UploadPartRequest part = new UploadPartRequest()
        .withBucketName(bucket)
        .withKey(key)
        .withUploadId(uploadId)
        .withPartNumber(partNumber)
        .withInputStream(new ByteArrayInputStream(bytes, 0, bytesRead))
        .withPartSize(bytesRead);
    results.add(client.uploadPart(part));
    bytesRead = input.read(bytes);
    partNumber++;
}
CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest()
    .withBucketName(bucket)
    .withKey(key)
    .withUploadId(uploadId)
    .withPartETags(results);
client.completeMultipartUpload(completeRequest);