Azure 二头肌将存储帐户连接字符串传递到 Secret Keyvault 循环问题

Question

我有一个以下二头肌脚本来执行以下步骤:

• 根据代码创建存储帐户
• 创建 key
• 使用正确的 key 加密存储帐户
• 生成 key 保管库中的 secret 连接字符串(存储帐户)

基础设施分为 2 个资源组:

rg-shared => where the 2 key vault are (1 keyvault for key and GUID, and 1 keyvault for secrets (connection string)

rg-storage-account => where all the storage account get created 

在 azure 的二头肌中，我有以下脚本:

存储.bicep

param ManagedIdentityid string
param uri string 
param kvname string
param keyvaultrg string = 'XXXX' //<== SHARED Resource Group
var keyVaultKeyPrefix = 'Key-Data-'
var storagePrefix = 'sthritesteur'
param tenantCodes array = [
  'aabb'
  'bbcc'
  'ccdd'
]


// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = [for tenantCode in tenantCodes: {
  name: '${storagePrefix}${tenantCode}'
  location: resourceGroup().location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_RAGRS'
  }
  // Assign the identity
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
        '${ManagedIdentityid}':{}
    }
  }
  properties: {
    allowCrossTenantReplication: true
    minimumTlsVersion: 'TLS1_2'
    allowBlobPublicAccess: false
    allowSharedKeyAccess: true
    networkAcls: {
      bypass: 'AzureServices'
      virtualNetworkRules: []
      ipRules: []
      defaultAction: 'Allow'
    }
    supportsHttpsTrafficOnly: true
    encryption: {
      identity: {
        // specify which identity to use
        userAssignedIdentity: ManagedIdentityid
      }
      keySource: 'Microsoft.Keyvault'
      keyvaultproperties: {
        keyname: '${kvname}-${keyVaultKeyPrefix}${toUpper(tenantCode)}'
        keyvaulturi:uri
      }
      services: {
        file: {
          keyType: 'Account'
          enabled: true
        }
        blob: {
          keyType: 'Account'
          enabled: true
        }
      }
    }
    accessTier: 'Hot'
  }

}]



resource storage_Accounts_name_default 'Microsoft.Storage/storageAccounts/blobServices@2021-04-01' = [ for (storageName, i) in tenantCodes :{
  parent: storageAccount[i]
  name: 'default'
  properties: {
    changeFeed: {
      enabled: false
    }
    restorePolicy: {
      enabled: false
    }
    containerDeleteRetentionPolicy: {
      enabled: true
      days: 7
    }
    cors: {
      corsRules: []
    }
    deleteRetentionPolicy: {
      enabled: true
      days: 30
    }
    isVersioningEnabled: true
  }
}]

module connectionString 'shared.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(keyvaultrg)
  name: storageName
  params: {
    storageAccountString: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
  }
}]

keyvaultclient.bicep

param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo

param storagerg string = 'XXXX' //<=== Storage Accounts Resource Groups
param sharedManagedIdentity string = 'mgn-identity-shared'
param keyvaultmain string = 'XXXX' //<=== KeyVault Name where to create GUID AND Keys
param tenantCodes array = [
  'aabb'
  'bbcc'
  'ccdd'
]
var clientDataKeyPrefix = 'Key-Data-'

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
  name: sharedManagedIdentity
  location: resourceGroup().location
}

resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultmain
}
resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
  parent:keyVaultClients
  name: '${keyVaultClients.name}-${clientDataKeyPrefix}${toUpper(code)}'
  properties: {
    keySize: 2048
    kty: 'RSA'
    // Assign the least permission
    keyOps: [
      'unwrapKey'
      'wrapKey'
    ]
  }
}]
resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2021-06-01-preview' = {
  parent:keyVaultClients
  name: 'add'
  properties: {
    accessPolicies: [
      {
        tenantId: subscription().tenantId
        objectId: managedIdentity.properties.principalId
        permissions: {
          // minimum required permission
          keys: [
            'get'
            'unwrapKey'
            'wrapKey'
          ]
        } 
      }
    ]
  }
}
resource clientLearnersGuid 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  parent:keyVaultClients
  name: '${keyVaultClients.name}${tenant}'
  properties: {
    contentType: 'GUID Key'
    value: '${deploymentIdOne}-${deploymentIdTwo}'
  }
  dependsOn:kvClientsKey
}]
module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(storagerg)
  name: storageName
  params: {
    ManagedIdentityid:managedIdentity.id
    kvname:keyVaultClients.name
    uri:keyVaultClients.properties.vaultUri
  }
  dependsOn:clientLearnersGuid
}]

共享.bicep

param keyvaultshared string = 'XXXX' //<=== Key Vault Where to Store the Storage Connection String Secret
param storageAccountString string
param tenantCodes array = [
  'aabb'
  'bbcc'
  'ccdd'
]
resource keyVaultShared 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultshared
}
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  parent:keyVaultShared
  name: '${keyVaultShared.name}-test${tenant}'
  properties:{
    contentType: '${tenant} Storage Account Connection String'
    value: storageAccountString
  }
}]

这些脚本根据tenantCode执行我需要的所有步骤。 。这是完美的。如果我只有 1 tenantCode声明后，一切都很顺利和完美，但是当我尝试声明超过 1 时，我面临的问题就会出现。这就是问题的细节。

当我声明超过 1 个代码时，脚本仍然创建了我需要的所有资源:存储帐户、 key 、加密、GUID 和 ConnectionStrings Secret。但无论如何它在 ConnectionStrings Secret 上失败了。

失败的原因是在这些文件中的代码块中:

共享.bicep

resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  parent:keyVaultShared
  name: '${keyVaultShared.name}-test${tenant}'
  properties:{
    contentType: '${tenant} Storage Account Connection String'
    value: storageAccountString
  }
}]

和

keuvaultsclient.bicep*

module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(storagerg)
  name: storageName
  params: {
    ManagedIdentityid:managedIdentity.id
    kvname:keyVaultClients.name
    uri:keyVaultClients.properties.vaultUri
  }
  dependsOn:clientLearnersGuid
}]

我有一个多重循环，我意识到在 secret 下的共享 keystore 中，我有正确数量的 secret (有 3 个租户代码，我有 3 个 secret )，并且在每个 secret 下我有 3 个版本(对于每个租户编码它为每个 secret 生成一个新版本)。此循环错误导致二头肌脚本 fail带有以下消息:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"Conflict\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n    \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n  }\\r\\n}\"\r\n          },\r\n          {\r\n            \"code\": \"Conflict\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n    \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

我在这个阶段完全受阻，因为我是二头肌初学者，所以我用尽了所有选项并尝试尝试解决这个问题。

如何重现

• 创建 2 个资源组(一个用于 key 保管库，一个用于存储帐户)
• 使用正确的 kv 名称更新参数
• 在所有文件中，每个文件至少添加 2 个租户代码
• 执行命令az deployment group create -f ./keyvaultsclient.bicep -g <rg-where-keyvaults-are>

我希望我已经足够清楚地解释了我面临的问题，如果您需要更多详细信息，请告诉我。

非常感谢您的时间和帮助

Answer 1

我通过做一些更改测试了相同的代码，请尝试在 .bicep 文件中进行相同的更改:

keyvaultclient.bicep:

删除了模块循环，因为它为同一事物创建了 2 个模块。

module StorageAccount './storage.bicep' = {
  scope: resourceGroup(storagerg)
  name: 'NestedStorage'
  params: {
    ManagedIdentityid:managedIdentity.id
    kvname:keyVaultClients.name
    uri:keyVaultClients.properties.vaultUri
  }
  dependsOn:clientLearnersGuid
}

storage.bicep:

删除了模块的循环，并添加了仅存储连接字符串的循环，该字符串将输出存储在数组中并将其传递到下一个模块。

module connectionString './shared.bicep'={
  scope: resourceGroup(keyvaultrg)
  name: 'KeyvaultNested'
  params: {
    storageAccountString: [for (tenant,i) in tenantCodes :{
    id:'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
  }]
}
}

shared.bicep:

将 StorageAccountString 的参数类型从字符串更改为数组，并在 secret 部分添加 [for (tenant,i) intenantCodes ，以便我可以将值指定为storageAccountString[i].id .

param storageAccountString array

resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for (tenant,i) in tenantCodes: {
  parent:keyVaultShared
  name: '${keyVaultShared.name}-test${tenant}'
  properties:{
    contentType: '${tenant} Storage Account Connection String'
    value: storageAccountString[i].id
  }
}]

输出: