java - 在java中为客户端生成azure存储SAS token

Question

为了上传媒体，我的客户端将向服务器发送请求:

message GetMediaUploadCredRequest {
  media.MediaType.Enum media_type = 1;
  media.Extension.Enum extension = 2;
  string file_name = 3;
  bool is_private = 4;
  string uploaded_by = 5;
}

服务器将生成一个 SAS token (例如来自 AWS 的 presigned_url)，并且 http_header 将发送回客户端。

message GetMediaUploadCredResponse {
  string upload_credential_url = 1;
  map<string, string> https_headers = 2;
  string media_id = 3;
}

然后，客户端将向带有 https_headers 的 URL 发出 PUT 请求，上传过程将完成。

这里是实现:

public class StorexServiceImpl extends StorexServiceGrpc.StorexServiceImplBase {
    private static final Logger log = LoggerFactory.getLogger(StorexServiceImpl.class);

    private static final String STORAGE_ACCOUNT_CONNECTION_STRING = "<connection-string>";
    
    private static final String CONTAINER_NAME = "<container-name>";



    @Override
    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        try{
            MediaType.Enum mediaType = request.getMediaType();
            Extension.Enum extension = request.getExtension();
            
            String fileName = String.format("%s.%s", UUID.randomUUID(), extension.toString());

            BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
            BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);
            BlobClient blobClient = containerClient.getBlobClient(fileName);
            BlobHttpHeaders headers = new BlobHttpHeaders().setContentEncoding("gzip");

            BlobSasPermission permission = new BlobSasPermission().setWritePermission(true); // Set the permission to allow uploading
            OffsetDateTime expiryTime = OffsetDateTime.now().plusHours(1); // Set the expiration time to 1 hour from now

            BlobServiceSasSignatureValues sasValues = new BlobServiceSasSignatureValues(expiryTime, permission)
                    .setProtocol(SasProtocol.HTTPS_HTTP);

            String sasUrl = blobClient.generateSas(sasValues);

            Map<String, String> httpHeaders = new TreeMap<>();
            httpHeaders.put("Content-Type", getContentType(mediaType, extension));
            if(!request.getIsPrivate()) {
                httpHeaders.put("x-amz-acl", "public-read");
            }

            String blobId = blobClient.getBlobName(); // get the ID of the blob

            GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                    .setMediaId(blobId) // set the blob ID in the response
                    .setUploadCredentialUrl(sasUrl)
                    .putAllHttpsHeaders(httpHeaders)
                    .build();
            response.onNext(res);
            response.onCompleted();
        } catch (Exception e){}
        super.getMediaUploadCred(request, response);

    }

    private String getContentType(MediaType.Enum mediaType, Extension.Enum extension) {
        if(mediaType == MediaType.Enum.IMAGE) {
            return "image/" + extension.toString().toLowerCase();
        } else if(mediaType == MediaType.Enum.AUDIO) {
            return "audio/mp3";
        } else if(mediaType == MediaType.Enum.VIDEO) {
            return "audio/mp4";
        } else if(mediaType == MediaType.Enum.FILE) {
            return "application/" + extension.toString().toLowerCase();
        }
        return "binary/octet-stream";
    }
}

但是对于结果我得到了这个:

{
    "uploadCredentialUrl": "sv=2021-10-04&spr=https%2Chttp&se=2023-03-24T13%3A36%3A38Z&sr=b&sp=w&sig=JUXXe1Qi13VWipgFWzx70mTOsVqadQCjmIF%2BxRl14cs%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "0ae4a0c5-167d-4a32-9752-0ad0d2b67e66.JPEG"
}

uploadCredentialUrl 看起来很奇怪，根本不像实际的 URL。

更新1:

找到这个post 。所以我稍微改变了我的代码:

    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        try{
            MediaType.Enum mediaType = request.getMediaType();
            Extension.Enum extension = request.getExtension();

            String fileName = String.format("%s.%s", UUID.randomUUID(), extension.toString());

            
            log.info("New Implementations Started");
            SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd");
            fmt.setTimeZone(TimeZone.getTimeZone("UTC"));
            Calendar cal = Calendar.getInstance();
            cal.setTime(new Date());
            cal.add(Calendar.DATE, -2);
            String start = fmt.format(cal.getTime());
            cal.add(Calendar.DATE, 4);
            String expiry =  fmt.format(cal.getTime());
            SecretKeySpec secretKey = new SecretKeySpec(Base64.getDecoder().decode(STORAGE_ACCOUNT_KEY), "HmacSHA256");
            Mac sha256HMAC = Mac.getInstance("HmacSHA256");
            sha256HMAC.init(secretKey);
            String  resource ="sc";
            String permissions ="rwdlac";
            String service = "b";
            String apiVersion="2019-07-07";
            String stringToSign = STORAGE_ACCOUNT_NAME + "\n" +
                    permissions +"\n" +  // signed permissions
                    service+"\n" + // signed service
                    resource+"\n" + // signed resource type
                    start + "\n" + // signed start
                    expiry + "\n" + // signed expiry
                    "\n" +  // signed IP
                    "https\n";
            log.info("string to sign: {}", stringToSign);

            String signature=Base64.getEncoder().encodeToString(sha256HMAC.doFinal(stringToSign.getBytes("UTF-8")));
            String sasToken = "sv=" + apiVersion +
                    "&ss=" + service+
                    "&srt=" + resource+
                    "&sp=" +permissions+
                    "&se=" + URLEncoder.encode(expiry, "UTF-8") +
                    "&st=" + URLEncoder.encode(start, "UTF-8") +
                    "&spr=https" +
                    "&sig=" + URLEncoder.encode(signature,"UTF-8");
            log.info("sas token: {}", sasToken);
            String resourceUrl = "https://" + STORAGE_ACCOUNT_NAME + ".blob.core.windows.net/" + CONTAINER_NAME + "?comp=block&" + sasToken;
            
            

            Map<String, String> httpHeaders = new TreeMap<>();
            httpHeaders.put("Content-Type", getContentType(mediaType, extension));
            if(!request.getIsPrivate()) {
                httpHeaders.put("x-amz-acl", "public-read");
            }

            GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                    .setMediaId("blob id") // set the blob ID in the response
                    .setUploadCredentialUrl(resourceUrl)
                    .putAllHttpsHeaders(httpHeaders)
                    .build();
            response.onNext(res);
            response.onCompleted();
        } catch (Exception e){}
    }

我得到了回复:

{
    "uploadCredentialUrl": "https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER_NAME>?comp=block&sv=2019-07-07&ss=b&srt=sc&sp=rwdlac&se=2023-03-28&st=2023-03-24&spr=https&sig=RpdV8prUgjzFApNo6bkuuiRAPHnw1mqww5l42cwVwyY%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "blob id"
}

但是当我尝试这个时:

curl -X PUT -T ~/Downloads/gfx100s_sample_04_thum-1.jpg -H "x-ms-date: $(date -u)" "https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER_NAME>/myimage.jpg?comp=block&sv=2019-07-07&ss=b&srt=sc&sp=rwdlac&se=2023-03-28&st=2023-03-24&spr=https&sig=RpdV8prUgjzFApNo6bkuuiRAPHnw1mqww5l42cwVwyY%3D"

我得到了这个:

<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:337c6e4d-301e-0019-7ebd-5f3baf000000
Time:2023-03-26T08:30:28.5705948Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was storageapollodevappstro
rwdlac
b
sc
2023-03-24
2023-03-28

https
2019-07-07
</AuthenticationErrorDetail></Error>%

更新2:

我已经按照 jccampanero 指南更新了我的代码:

    @Override
    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
        BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);
        BlobSasPermission permission = new BlobSasPermission().setReadPermission(true).setWritePermission(true);
        OffsetDateTime expiryTime = OffsetDateTime.now().plusMinutes(30);

        String blobName = containerClient.getBlobClient(request.getFileName()).getBlobName();
        String sasToken = generateSasToken(STORAGE_ACCOUNT_NAME, STORAGE_ACCOUNT_KEY, CONTAINER_NAME, blobName, permission, expiryTime);
        String url = String.format("https://%s.blob.core.windows.net/%s/%s?%s", STORAGE_ACCOUNT_NAME, CONTAINER_NAME, request.getFileName(), sasToken);
        GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                .setMediaId(blobName)
                .setUploadCredentialUrl(url)
                .build();
        response.onNext(res);
        response.onCompleted();
    }

    private static String generateSasToken(String accountName, String accountKey, String containerName, String blobName, BlobSasPermission permission, OffsetDateTime expiryTime) {
        String sasToken = null;
        try {
            String signedPermissions = permission.toString();
            String signedStart = OffsetDateTime.now().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME);
            String signedExpiry = expiryTime.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME);
            String canonicalizedResource = String.format("/blob/%s/%s/%s", accountName, containerName, blobName);
            String signedVersion = "2020-12-06";

            String stringToSign = signedPermissions + "\n" +
                    signedStart + "\n" +
                    signedExpiry + "\n" +
                    canonicalizedResource + "\n" +
                    "\n" + // signedKeyObjectId
                    "\n" + // signedKeyTenantId
                    "\n" + // signedKeyStart
                    "\n" + // signedKeyExpiry
                    "\n" + // signedKeyService
                    "\n" + // signedKeyVersion
                    "\n" + // signedAuthorizedUserObjectId
                    "\n" + // signedUnauthorizedUserObjectId
                    "\n" + // signedCorrelationId
                    "\n" + // signedIP
                    "https\n" + // signedProtocol
                    signedVersion + "\n" +
                    "\n" + // signedResource
                    "\n" + // signedSnapshotTime
                    "\n" + // signedEncryptionScope
                    "\n" + // rscc
                    "\n" + // rscd
                    "\n" + // rsce
                    "\n" + // rscl
                    "\n"; // rsct

            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(new SecretKeySpec(Base64.getDecoder().decode(accountKey), "HmacSHA256"));
            String signature = Base64.getEncoder().encodeToString(mac.doFinal(stringToSign.getBytes("UTF-8")));
            sasToken = String.format("sv=%s&st=%s&se=%s&sr=b&sp=%s&sig=%s",
                    signedVersion, signedStart, signedExpiry, permission.toString(), URLEncoder.encode(signature, "UTF-8"));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return sasToken;
    }

并运行 CURL 命令:

curl -X PUT \
  -T ~/Downloads/gfx100s_sample_04_thum-1.jpg \
  -H "x-ms-blob-type: BlockBlob" \
  -H "x-ms-meta-name: example" \
  "<URL-with sas token>"

给我:

<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:124d92c4-101e-001e-189f-6157cc000000
Time:2023-03-28T18:02:50.6977457Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>%

但我遵循了 docs 中的签名字段。那么为什么我仍然收到错误？

更新3:

        BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
        BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);

        OffsetDateTime expiryTime = OffsetDateTime.now().plusMinutes(30);
        BlobClient blobClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME)
                .getBlobClient(request.getFileName());
        BlobSasPermission permission = new BlobSasPermission().setReadPermission(true).setWritePermission(true);
        BlobServiceSasSignatureValues sasValues = new BlobServiceSasSignatureValues(expiryTime, permission);
        String sasToken = blobClient.generateSas(sasValues);

        String blobName = containerClient.getBlobClient(request.getFileName()).getBlobName();
        log.info(sasToken);
        String url = String.format("https://%s.blob.core.windows.net/%s/%s?%s", STORAGE_ACCOUNT_NAME, CONTAINER_NAME, request.getFileName(), sasToken);
        GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                .setMediaId(blobName)
                .setUploadCredentialUrl(url)
                .build();

错误:

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Answer 1

使用第一个代码片段，您实际上生成了一个有效的 SAS:

{
    "uploadCredentialUrl": "sv=2021-10-04&spr=https%2Chttp&se=2023-03-24T13%3A36%3A38Z&sr=b&sp=w&sig=JUXXe1Qi13VWipgFWzx70mTOsVqadQCjmIF%2BxRl14cs%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "0ae4a0c5-167d-4a32-9752-0ad0d2b67e66.JPEG"
}

您提到 uploadCredentialUrl 似乎不是有效的 URL，您是对的，因为实际上您正在接收 SAS token 。

SAS token 由一系列 query parameters 组成应将其附加到应应用它的资源的 URL 中。

您正在创建用于上传特定 blob 的 SAS(您可以为不同的 resources 创建 SAS)；这意味着您需要提供要创建的 Blob 资源的 URL 并将其附加到所获得的 SAS token 。

您的 curl 命令看起来确实不错，只需确保使用正确的 SAS token 即可:

curl -X PUT -T ~/Downloads/gfx100s_sample_04_thum-1.jpg -H "x-ms-date: $(date -u)" "https://<storage account name>.blob.core.windows.net/<container name>/myimage.jpg?<received SAS token>"

请考虑查看 Put Blob REST API documentation有关您可以指定的 header 的详细信息。

This related page还提供指导。

最后，您尝试自己生成签名，过程顺利 documented但通常会收到与签名不匹配相关的错误:如果可能，请更喜欢使用特定编程语言的 Azure SDK。