azure - 通过 ARM 模板在 Azure 数据湖存储帐户中创建文件系统

Question

我已经使用 ARM 模板创建了第 2 代 Azure Data Lake。但现在我试图弄清楚如何在 ARM 中创建数据湖文件系统，但似乎找不到 API 来执行此操作。这不可用吗？是否可以通过其他方式实现？

尝试手动创建文件系统并在门户中导出模板，但似乎没有看到文件系统资源。只有这个警告。

因为这表明文件系统可能在 API 中作为“blobservices/containers”进行监听，所以我尝试将此资源添加到 ARM 模板中

 {
            "name": "[concat( parameters('DataLakeName'), '/default/input')]",
            "type": "Microsoft.Storage/storageAccounts/blobServices/containers",
            "dependsOn": [
                "[concat('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
            ],
            "apiVersion": "2018-07-01",
            "properties": {
                "publicAccess": "None",
                "metadata": {}
            },
            "resources": []
        }

但是不幸的是这不起作用并提供了以下错误消息:分层命名空间帐户尚不支持 Blob API。这让我想到这甚至可能无法通过 ARM 进行部署。有人已经尝试过这个吗？

我的数据湖存储帐户的 ARM 模板资源 block 作为上下文完成:

 {
            "name": "[parameters('DataLakeName')]",
            "type": "Microsoft.Storage/storageAccounts",
            "apiVersion": "2018-07-01",
            "location": "[parameters('location')]",
            "tags": {},
            "properties": {
                "accessTier": "[parameters('accessTier')]",
                "supportsHttpsTrafficOnly": "[parameters('supportsHttpsTrafficOnly')]",
                "isHnsEnabled": true
            },
            "resources": [
                {
                    "type": "providers/advancedThreatProtectionSettings",
                    "name": "Microsoft.Security/current",
                    "apiVersion": "2017-08-01-preview",
                    "dependsOn": [
                        "[resourceId('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
                    ],
                    "properties": {
                        "isEnabled": true
                    }
                }
            ],
            "dependsOn": [],
            "sku": {
                "name": "[parameters('accountType')]"
            },
            "kind": "StorageV2"
        }

Answer 1

我在这个github issue answer上找到了这个解决方案。来自一个发现还没有任何解决方案的人，所以这里要做的基本事情是手动调用 REST API 来执行此操作。说明和博客可以在此链接中找到: http://sql.pawlikowski.pro/2019/03/10/connecting-to-azure-data-lake-storage-gen2-from-powershell-using-rest-api-a-step-by-step-guide/

以下是用于创建文件系统的 Powershell 脚本，以防链接将被弃用:

所有功劳均归功于 Michał Pawlikowski，感谢您创建这个脚本，效果非常好。

[CmdletBinding()]
Param(
  [Parameter(Mandatory=$true,Position=1)] [string] $StorageAccountName,
  [Parameter(Mandatory=$True,Position=2)] [string] $FilesystemName,
  [Parameter(Mandatory=$True,Position=3)] [string] $AccessKey
)

# Rest documentation:
# https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/create

$date = [System.DateTime]::UtcNow.ToString("R") # ex: Sun, 10 Mar 2019 11:50:10 GMT

$n = "`n"
$method = "PUT"

$stringToSign = "$method$n" #VERB
$stringToSign += "$n" # Content-Encoding + "\n" +  
$stringToSign += "$n" # Content-Language + "\n" +  
$stringToSign += "$n" # Content-Length + "\n" +  
$stringToSign += "$n" # Content-MD5 + "\n" +  
$stringToSign += "$n" # Content-Type + "\n" +  
$stringToSign += "$n" # Date + "\n" +  
$stringToSign += "$n" # If-Modified-Since + "\n" +  
$stringToSign += "$n" # If-Match + "\n" +  
$stringToSign += "$n" # If-None-Match + "\n" +  
$stringToSign += "$n" # If-Unmodified-Since + "\n" +  
$stringToSign += "$n" # Range + "\n" + 
$stringToSign +=    
                    <# SECTION: CanonicalizedHeaders + "\n" #>
                    "x-ms-date:$date" + $n + 
                    "x-ms-version:2018-11-09" + $n # 
                    <# SECTION: CanonicalizedHeaders + "\n" #>

$stringToSign +=    
                    <# SECTION: CanonicalizedResource + "\n" #>
                    "/$StorageAccountName/$FilesystemName" + $n + 
                    "resource:filesystem"# 
                    <# SECTION: CanonicalizedResource + "\n" #>

$sharedKey = [System.Convert]::FromBase64String($AccessKey)
$hasher = New-Object System.Security.Cryptography.HMACSHA256
$hasher.Key = $sharedKey

$signedSignature = [System.Convert]::ToBase64String($hasher.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($stringToSign)))


$authHeader = "SharedKey ${StorageAccountName}:$signedSignature"

$headers = @{"x-ms-date"=$date} 
$headers.Add("x-ms-version","2018-11-09")
$headers.Add("Authorization",$authHeader)

$URI = "https://$StorageAccountName.dfs.core.windows.net/" + $FilesystemName + "?resource=filesystem"

Try {
    Invoke-RestMethod -method $method -Uri $URI -Headers $headers # returns empty response
}
catch {
    $ErrorMessage = $_.Exception.Message
    $StatusDescription = $_.Exception.Response.StatusDescription
    $false

    Throw $ErrorMessage + " " + $StatusDescription
}