azure - 使用 DevOps 持续集成和交付 Azure Synapse Analytics 工作区

Question

我正在使用 Azure DevOps 持续集成和交付 Azure Synapse Analytics 工作区，我已将工作区部署到开发、测试、预生产和生产环境。管道使用参数化的 LinkedServices、数据集和触发器。我正在关注这些 Microsoft 文档 document1 , document2 .

我有这些不同的工件，我需要使用 TemplateParametersForWorkspace.json 从 yaml 文件传递​​参数。

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "workspaceName": {
            "value": "devSynapse"
        },
        "LinkedService_connectionString": {
            "value": "Integrated Security=False;Encrypt=True;Connection Timeout=30;Data Source=;Initial Catalog="
        },
        "Trigger_Test_parameters_server": {
            "value": "devServer"
        }
    }
}

我需要您为我指明如何从 DevOps CI/CD 传递变量以根据 Synpase 工作区部署到的环境更改值的正确方向？

Answer 1

方法 1)

您可以在 ARM 模板中添加环境变量，并将其用于管道中的不同环境，如下所示:-

"connectionstring": {

"type": "string"

},

"environment": {

"type": "string"

}

},

"variables": {

"storageBlobDataContributorRoleID": "ba92f5b4-2d11-453d-a403-e96b0029c9fe",

"defaultDataLakeStorageAccountUrl": "[concat('https://', parameters('defaultDataLakeStorageAccountName'), '.dfs.core.windows.net')]",

"connectionString": "[if(equals(parameters('environment'), 'Production'), parameters('connectionStringProd'), parameters('connectionStringNonProd'))]"

输出:-

以上参数检查，如果环境变量设置为 Production，则使用 Production 连接字符串，否则使用其他环境。

方法 2)

您还可以在 Azure DevOps Pipeline 变量 yaml 脚本中提及每个产品，并在 Azure Pipelines 中使用它，如下所示:-

trigger:
- main

variables:
  - name: DevEnvironment
    value: "env-url"
  - name: TestEnvironment
    value: "env-url"
  - name: PreProdEnvironment
    value: "env-url"
  - name: ProdEnvironment
    value: "env-url"

stages:
- stage: Build
  displayName: 'Build stage'
  jobs:
  - job: Build
    displayName: 'Build job'
    steps:
    - script: echo 'Building the Synapse Analytics workspace...'
      displayName: 'Build step'

- stage: DeployToDev
  displayName: 'Deploy to Development'
  dependsOn: Build
  condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(DevEnvironment)'

- stage: DeployToTest
  displayName: 'Deploy to Test'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Schedule'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(TestEnvironment)'

- stage: DeployToPreProd
  displayName: 'Deploy to Pre-Production'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Manual'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(PreProdEnvironment)'

- stage: DeployToProd
  displayName: 'Deploy to Production'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Manual'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'

方法 3)

或者，您还可以配置不同阶段的开发、测试、生产、预生产，以在特定环境中部署 Synapse 工作区，如下所示:-