java - 使用多线程上传图片到AmasonS3作为存储平台

Question

官方API中只有单次上传和部分上传，我写的线程池(20)个上传出现连接超时错误:

Exception in thread "pool-1-thread-12" com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to s3.amazonaws.com:80 [s3.amazonaws.com/54.231.66.16] failed: connect timed out

我看到了AmasonS3的API，但没有找到答案。

import com.amazonaws.ClientConfiguration;
import com.amazonaws.Protocol;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.gaodig.stream.common.Constants;
import com.gaodig.stream.config.VEEConfig;

public class CephS3Client {

    private static volatile AmazonS3 s3Client = null;

    public static AmazonS3 getClient() {
        if (s3Client == null) {
            synchronized (CephS3Client.class) {
                if (s3Client == null) {
                    AWSCredentials credentials = new BasicAWSCredentials(Constants.CephS3AccessKey,
                            Constants.CephS3SecretKey);
                    ClientConfiguration clientConfig = new ClientConfiguration();
                    clientConfig.setProtocol(Protocol.HTTP);
                    clientConfig.setSignerOverride("S3SignerType");

                    // AmazonS3 conn = new AmazonS3Client(credentials,
                    // clientConfig);
                    s3Client = new AmazonS3Client(credentials, clientConfig);

                    s3Client.setEndpoint(VEEConfig.CEPH_ENDPOINT());
                }
            }
        }
        return s3Client;
    }

}

Answer 1

如果我理解你的问题是正确的，这个功能没有在java sdk 1.0中实现。但在新的 amazon java sdk 中，他们为 dynamo db 和 s3 以及 http 客户端实现了许多异步功能。所以现在您可以使用 s3 的异步客户端。示例:

public static void main(String[] args) {
S3AsyncClient client = S3AsyncClient.create();
CompletableFuture<PutObjectResponse> future = client.putObject(
        PutObjectRequest.builder()
                        .bucket(BUCKET)
                        .key(KEY)
                        .build(),
        AsyncRequestProvider.fromFile(Paths.get("myfile.in"))
);
future.whenComplete((resp, err) -> {
    try {
        if (resp != null) {
            System.out.println(resp);
        } else {
            // Handle error
            err.printStackTrace();
        }
    } finally {
        // Lets the application shut down. Only close the client when you are completely done with it.
        FunctionalUtils.invokeSafely(client::close);
    }
});

}

您可以阅读此文档 new AWS SDK for Java也许这对你有帮助