jenkins - 在节点上创建并行执行的多个阶段

Question

我们的跨平台项目要求 Jenkins 在多个不同的平台上执行，并为每个平台进行适当的测试和打包。我能够结合 parallel与 node但仅限于单个 stage走到这一步(见下文)

我希望能够将其分解为多个阶段。我想创建的阶段是:

构建:使用工件构建库/项目

UnitTest:构建和运行单元测试

TestApp:使用工件构建和运行测试应用程序

输入阶段:要上传？

上传:将工件上传到外部服务器

我会做以下事情吗:

在阶段之间复制一堆对象(使用 stash )

需要在 node 上保持一致性用法。 (在节点 label 上生成的对象应该相应地标记为 unstashed 在适当的节点 label 上)。我需要为每个节点唯一地标记每个存储。

这不是效率低得多吗？我会人为地复制大量数据，以便我可以创建阶段。

def checkoutAndBuild(Map args) {
    node("${args.nodeName}") {

        checkout([$class: 'GitSCM', 
                    branches: scm.branches,  
                    doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, 
                    extensions: scm.extensions + 
                                [[$class: 'SubmoduleOption', 
                                    disableSubmodules: false, 
                                    parentCredentials: false, 
                                    recursiveSubmodules: true, 
                                    reference: '', 
                                    trackingSubmodules: false]] +
                                [[$class: 'CleanCheckout']], 
                    userRemoteConfigs: scm.userRemoteConfigs
                ])

        step([$class: 'CopyArtifact', 
            filter: "AppCommon/*/**, cmake/**/*, core/**/*, thirdparty/prebuilt/${args.prebuiltDir}/**/*, tools/**/*", 
            fingerprintArtifacts: true, 
            projectName: "${args.engineDependency_Job}", 
            selector: [$class: 'SpecificBuildSelector', buildNumber: "${args.engineDependency_BuildNo}"], 
            target: 'engine'])

        dir("build/${args.buildDir}") {
            echo 'Building..'
            if (isUnix()) {
                sh './build.sh Release'
            } else {
                bat 'build.bat Release'
            }
        }

        def extras = args.additionalArtifacts ? ", ${args.additionalArtifacts}" : ""
        archiveArtifacts artifacts: "dist/**/*${extras}", fingerprint: true

        dir("test/build") {
            echo 'Building test App'
            sh "./full.sh ${args.buildDir} Release"
        }
    }
}

pipeline {
    agent none

    stages {
        stage('Info') {
            agent any
            steps {
                echo "Running ${env.JOB_NAME} / ${env.BUILD_ID} on ${env.JENKINS_URL}"
            }
        }

        stage('Build') {
            steps {
                parallel (
                    ios: {
                        checkoutAndBuild nodeName: 'iOS', prebuiltDir: 'ios', buildDir: 'ios', engineDependency_Job: 'engine_iOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.ios)
                    },
                    tvos: {
                        checkoutAndBuild nodeName: 'tvOS', prebuiltDir: 'tvos', buildDir: 'tvos', engineDependency_Job: 'engine_tvOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.tvos)
                    },
                    android: {
                        checkoutAndBuild nodeName: 'Android', prebuiltDir: 'android', buildDir: 'AndroidNative', engineDependency_Job: 'engine_Android_Release', engineDependency_BuildNo: String.valueOf(engineBuild.android), additionalArtifacts: 'src/java/*'
                    })
            }
        }
        stage('Test Build') {
            steps {
                echo 'Testing...'
            }
        }

        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }

    }

    post {
        success {
            slackSend channel: '#builds',
                        color: 'good',
                        message: "${currentBuild.fullDisplayName} succeeded. (<${env.BUILD_URL}|Open>)"

        }
        failure {
            slackSend channel: '#builds',
                        color: 'danger',
                        message: "${currentBuild.fullDisplayName} failed. (<${env.BUILD_URL}|Open>)"
        }
    }
}

Answer 1

不幸的是，声明式管道语法在您的情况下不是很灵活。
在阶段之间存储对象会非常繁重，并且会导致同步阶段，其中最快的构建目标在每个阶段之前等待较慢的构建目标。

如果您不需要同步运行阶段，我建议创建一个大方法，其中包含所有构建目标的所有阶段，即您想要并行运行的阶段。
为了区分阶段，您可以例如将节点名称附加到每个阶段标签。

def checkoutBuildTestDeploy(Map args) {
    node("${args.nodeName}") {

        stage("Build ${args.nodeName}") {
            checkout([$class: 'GitSCM', ... ])
            // And other build steps ...
        }
        stage("Unit test ${args.nodeName}") {
            // Unit test steps
        }
        stage("Test app ${args.nodeName}") {
            // Test steps
        }
        stage("Deploy ${args.nodeName}") {
            // Input answer and upload
        }
    }
}

pipeline {
    agent none
    stages {
        stage('Info') {
            agent any
            steps {
                echo "Running ${env.JOB_NAME} / ${env.BUILD_ID} on ${env.JENKINS_URL}"
            }
        }

        stage('Run builds parallel') {
            steps {
                parallel (
                    ios: {
                        checkoutBuildTestDeploy nodeName: 'iOS', prebuiltDir: 'ios', buildDir: 'ios', engineDependency_Job: 'engine_iOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.ios)
                    },
                    tvos: {
                        checkoutBuildTestDeploy nodeName: 'tvOS', prebuiltDir: 'tvos', buildDir: 'tvos', engineDependency_Job: 'engine_tvOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.tvos)
                    },
                    android: {
                        checkoutBuildTestDeploy nodeName: 'Android', prebuiltDir: 'android', buildDir: 'AndroidNative', engineDependency_Job: 'engine_Android_Release', engineDependency_BuildNo: String.valueOf(engineBuild.android), additionalArtifacts: 'src/java/*'
                    })
            }
        }
    }
    post {
        success {
            slackSend channel: '#builds',
                      color: 'good',
                      message: "${currentBuild.fullDisplayName} succeeded. (<${env.BUILD_URL}|Open>)"

        }
        failure {
            slackSend channel: '#builds',
                      color: 'danger',
                      message: "${currentBuild.fullDisplayName} failed. (<${env.BUILD_URL}|Open>)"
        }
    }
}