azure - 在模板参数中使用任务输出变量

Question

我的管道的第一阶段检查哪些服务实际发生了变化。这是为了在没有更改的情况下避免重建、重新测试、重新部署服务，从而加快管道速度。

这是该阶段的 changed.yaml:

parameters:
- name: comparedTo
  default: ''

stages:
- stage: Changed
  displayName: Check for changes in services and configs...
  jobs:
  - job: Changes
    displayName: Checking for changes in services and configs...
    steps:
    - bash: |
        mapfile -t changed < <(git diff  HEAD ${{ parameters.comparedTo }} --name-only | awk -F'/' 'NF!=1{print $1}' | sort -u)
        servicesChanged=()
        configChanged=()
        echo ""
        echo "Total Changed: ${#changed[@]}"
        for i in "${changed[@]}"
        do
          echo $i
          if [[ $i == 'admin' ]]; then
            echo "##vso[task.setvariable variable=adminChanged;isOutput=True]true"
            servicesChanged+=("admin")
          elif [[ $i == 'admin-v2' ]]; then
            echo "##vso[task.setvariable variable=adminV2Changed;isOutput=True]true"
            servicesChanged+=("admin-v2")
          elif [[ $i == 'api' ]]; then
            echo "##vso[task.setvariable variable=apiChanged;isOutput=True]true"
            servicesChanged+=("api")
          elif [[ $i == 'client' ]]; then
            echo "##vso[task.setvariable variable=clientChanged;isOutput=True]true"
            servicesChanged+=("client")
          elif [[ $i == 'k8s' ]]; then
            echo "##vso[task.setvariable variable=k8sChanged;isOutput=True]true"
            configsChanged+=("k8s")
          elif [[ $i == 'pipelines' ]]; then
            echo "##vso[task.setvariable variable=pipelineChanged;isOutput=True]true"
            configsChanged+=("pipelines")
          fi
        done
        echo ""
        echo "Services Changed: ${#servicesChanged[@]}"
        for i in "${servicesChanged[@]}"
        do
          echo $i
        done
        echo ""
        echo "Configs Changed: ${#configsChanged[@]}"
        for i in "${configsChanged[@]}"
        do
          echo $i
        done
        if [[ ${#servicesChanged[@]} > 0 ]]; then
          echo ""
          echo "Any services changed: True"
          echo "##vso[task.setvariable variable=anyServicesChanged;isOutput=true]true"
          echo "##vso[task.setvariable variable=servicesChanged;isOutput=true]${servicesChanged[@]}"
        fi
        if [[ ${#configsChanged[@]} > 0 ]]; then
          echo ""
          echo "Any configs changed: True"
          echo "##vso[task.setvariable variable=anyConfigsChanged;isOutput=true]true"
          echo "##vso[task.setvariable variable=configsChanged;isOutput=true]${configsChanged[@]}"
        fi
        echo ""
      name: detectChanges

如您所见，它创建了许多任务输出变量:

# This just indicates if the service has changed: true/false
echo "##vso[task.setvariable variable=<service-name>;isOutput=True]true"

# This should be creating a an output variable that is an array of the services that have changed
echo "##vso[task.setvariable variable=servicesChanged;isOutput=true]${servicesChanged[@]}"

因此，我给了自己两个选择:为每个服务仅提供一个 true/false 或(以某种方式)迭代已更改的服务数组。

每个阶段基本上有以下形式:

# pr.yaml
...
- template: templates/unitTests.yaml
  parameters: 
    services: 
    - api
    - admin
    - admin-v2
    - client
...

parameters:
- name: services
  type: object
  default: []

stages:
- stage: UnitTests
  displayName: Run unit tests on service...
  dependsOn: Changed
  condition: succeeded()
  jobs:
  - job: UnitTests
    condition: or(eq(stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'], true), eq(variables['Build.Reason'], 'Manual'))
    displayName: Running unit tests...
    steps:
    - ${{ each service in parameters.services }}:
      - bash: |
          echo "Now running ${{ service }} unit tests..."

这是我迄今为止尝试过的方法以及遇到的错误:

有条件地将每个服务添加到服务数组或添加已更改服务的数组:

- template: templates/changed.yaml
  parameters:
    comparedTo: origin/production
- template: templates/unitTests.yaml
  dependsOn: Changed
  parameters: 
    services: 
    - ${{ if eq(stageDependencies.Changed.Changes.outputs['detectChanges.apiChanged'], true) }}
      - api
    - ${{ if eq(stageDependencies.Changed.Changes.outputs['detectChanges.adminChanged'], true) }}
      - admin
    - ${{ if eq(stageDependencies.Changed.Changes.outputs['detectChanges.adminV2Changed'], true) }}
      - admin-v2
    - ${{ if eq(stageDependencies.Changed.Changes.outputs['detectChanges.clientChanged'], true) }}
      - client

或者...

- template: templates/changed.yaml
  parameters:
    comparedTo: origin/production
- template: templates/unitTests.yaml
  dependsOn: Changed
  parameters: 
    services: 
    - ${{ if eq(dependencies.Changed.outputs['Changes.detectChanges.apiChanged'], true) }}
      - api
    - ${{ if eq(dependencies.Changed.outputs['Changes.detectChanges.adminChanged'], true) }}
      - admin
    - ${{ if eq(dependencies.Changed.outputs['Changes.detectChanges.adminV2Changed'], true) }}
      - admin-v2
    - ${{ if eq(dependencies.Changed.outputs['Changes.detectChanges.clientChanged'], true) }}
      - client

或者...

- template: templates/changed.yaml
  parameters:
    comparedTo: origin/production
- template: templates/unitTests.yaml
  dependsOn: Changed
  parameters: 
    services: 
    - $[ stageDependencies.Changed.Changes.outputs['detectChanges.servicesChanged'] ] 

这会导致:

An error occurred while loading the YAML build pipeline. Object reference not set to an instance of an object.

我知道变量:只会接受字符串而不是数组。

一种解决方案是为每个 true/false 变量提供一个 variables:，然后根据 parameters.services 确定条件以及任务输出变量是否为true。

有什么建议吗？

<小时/>

引用:

• 任务输出变量:https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-variables-in-scripts
• 参数:https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#parameters
• 表达式:https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops

Answer 1

模板表达式 ${{}} 在编译时(作业运行之前)求值，这意味着它无法访问在运行时(作业运行之后)动态设置的变量开始)。因此，在上述场景中不能使用模板表达式 ${{}} 。请参阅以下描述 here .

Within a template expression, you have access to the parameters context that contains the values of parameters passed in. Additionally, you have access to the variables context that contains all the variables specified in the YAML file plus many of the predefined variables (noted on each variable in that topic). Importantly, it doesn't have runtime variables such as those stored on the pipeline or given when you start a run. Template expansion happens very early in the run, so those variables aren't available

您可以使用条件作为解决方法。您需要添加多个要根据条件执行的任务。请参阅下面的示例:

- template: templates/changed.yaml
  parameters:
    comparedTo: origin/production
- template: templates/unitTests.yaml
  dependsOn: Changed

<小时/>

#unitTests.yaml

stages:
- stage: UnitTests
  displayName: Run unit tests on service...
  dependsOn: Changed
  condition: succeeded()
  jobs:
  - job: UnitTests
    condition: or(eq(stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'], true), eq(variables['Build.Reason'], 'Manual'))
    displayName: Running unit tests...
    
    variables:
      changedServices: $[stageDependencies.Changed.Changes.outputs['detectChanges.servicesChanged']]
    
    steps:
    - bash: |
         echo "Now running api unit tests..."
      name: Api-unit-test
      conidtion: contains(variables['changedServices'], 'api')
   
    - bash: |
         echo "Now running admin unit tests..."
      name: admin-unit-test
      conidtion: contains(variables['changedServices'], 'admin')
    
    - bash: |
         echo "Now running client unit tests..."
      name: client-unit-test
      conidtion: contains(variables['changedServices'], 'client')

   

另一个解决方法是将管道分成两个管道。运行 Changed 阶段的第一个管道。然后调用rest api在脚本任务中触发第二个管道并传递请求正文中的变量。请参阅this类似的线程。