java - Pig 用户定义函数中的 aws Amazon S3 客户端凭证

Question

我为我的 pig 脚本创建了一个用户定义的函数来删除文件夹。但我如何获得凭据？

REGISTER s3://emr-custom-jars/custom.jar;

如果我将寄存器放入我的 Pig 脚本中，它可以访问 aws 存储上的文件。所以我的凭证必须可供 pig 使用。我可以在不上传 jar 内的 accessKey 和 SecretKey 的情况下访问我的凭证或 AmazoneS3Client 吗？

    private AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
    private AmazonS3Client s3client = new AmazonS3Client(credentials);  

    @Override
    public void deleteObjectsInFolder(String bucketName, String folderPath) {
        for (S3ObjectSummary file : s3client.listObjects(bucketName, folderPath).getObjectSummaries()){           
            s3client.deleteObject(bucketName, file.getKey());
        }
    }

Answer 1

是的，有几种方法可以提供访问凭据，而无需对 jar 中的值进行硬编码。 AmazonS3Client

public AmazonS3Client() 

Constructs a new client to invoke service methods on Amazon S3. A credentials provider chain will be used that searches for credentials in this order:

• Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_KEY
• Java System Properties - aws.accessKeyId and aws.secretKey
• Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
• Instance Profile Credentials - delivered through the Amazon EC2 metadata service