bash - 使用 Bash/Curl 从 Azure Blob 存储下载文件

Question

我尝试使用以下脚本从 Azure Blob 存储下载文件:

authorization="SharedKey"

HTTP_METHOD="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2009-09-19"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"
x_ms_blob_type_h="x-ms-blob-type:BlockBlob"


# Build the signature string
canonicalized_headers="$${x_ms_date_h}\n$${x_ms_version_h}"
canonicalized_resource="/${STORAGE_ACCOUNT}/${STORAGE_CONTAINER}"

string_to_sign="$${HTTP_METHOD}\n\n\n\n\n\n\n\n\n\n\n\n$${x_ms_blob_type_h}\n$${canonicalized_headers}\n$${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.

decoded_hex_key="$(echo -n ${STORAGE_KEY} | base64 -d -w0 | xxd -p -c256 | tr -d ' ')"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | base64 -w0)

authorization_header="Authorization: $authorization $STORAGE_ACCOUNT:$signature"
FILE_TYPE="application/x-yml"
DOWNLOAD_FILE="https://${STORAGE_ACCOUNT}.blob.core.windows.net/${STORAGE_CONTAINER}/${FILENAME}"

curl -H "$x_ms_date_h" \
     -H "$x_ms_version_h" \
     -H "$x_ms_blob_type_h" \
     -H "$authorization_header" \
     -H "Content-Type: $${FILE_TYPE}" \
     -f $${DOWNLOAD_FILE} -o ${FILENAME} 

我还使用 Terraform 的 template_file 提供程序来调用此脚本，并且我必须转义一些变量，因此出现了奇怪的插值。但我已经调试了脚本，所有变量似乎都放置正确。问题出在 SAS 一代上，我不断收到这样的消息:

+ curl -H 'x-ms-date:Fri, 13 Sep 2019 11:04:40 GMT' -H x-ms-version:2009-09-19 -H x-ms-blob-type:BlockBlob -H 'Authorization: SharedKey *masked*:vyD7pp7Rqu3JBuS5IkHW0GMS2L82BN9fNKbmDAjuEoQ=' -H 'Content-
Type: application/octet-stream' -f https://*masked*.blob.core.windows.net/*masked*/*masked* -o *masked*
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

你知道我在这里可能做错了什么吗？

Answer 1

这对我有用(based this other post):

#!/bin/bash
storage_account=$AZ_STG_ACCOUNT
container_name=$AZ_STG_CONTAINER
access_key=$AZ_STG_ACCESS_KEY
blob_name=PSCR_230113_000000Fri.xml

blob_store_url="blob.core.windows.net"
authorization="SharedKey"

request_method="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2015-02-21"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"

# Build the signature string
canonicalized_headers="${x_ms_date_h}\n${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}/${blob_name}"

string_to_sign="${request_method}\n\n\n\n\n\n\n\n\n\n\n\n${canonicalized_headers}\n${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(printf $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary |  base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"
# -v or --trace to enable tracing
curl -v \
  -H "$x_ms_date_h" \
  -H "$x_ms_version_h" \
  -H "$authorization_header" \
  "https://${storage_account}.${blob_store_url}/${container_name}/${blob_name}" -o ${blob_name}