git - 如何仅使用 master git 分支使我的项目从开发自动部署到暂存并在 Jenkins 中手动部署到生产

Question

我有大约 30 个 Wordpress 网站，因此 Jenkins 的配置方式是我为每个网站都有一份工作。开发过程如下(我不知道它是否是最佳的，但我们是这样拥有的):

由于我们有外包开发人员，他们在自己的存储库托管提供商中托管了自己的存储库。每当代码准备好进行 QA 时，他们会将所有更改提交到我们的 master 分支中的存储库。

然后我使用 jenkinsfile 手动运行 Jenkins 作业，如下所示。

工作流程必须是:部署到开发环境，当且仅当之前的部署成功时，部署到阶段。在这里，我们必须停下来，让 QA 人员检查网站是否有任何损坏的链接、错误等。

如果一切都符合我们的预期并且没有发现错误，那么最后部署到生产环境。

注意 : 有些人建议我只有舞台和制作。我们没有那个配置的原因是因为dev环境不能在线访问，原因是我用这个环境来测试后端配置(例如apache conf等)。

此外，其他一些人建议为每个环境设置一个分支，这在理论上是有道理的，但我认为这将改变我们的外包开发人员将代码提交到存储库的方式，我的意思是，他们将始终必须提交代码到 dev 分支，然后 merge 到 stage 分支以部署到 stage，我认为这不太好。

现在，步骤 2-4 如下所示:
为了向您举例说明该流程的外观，我们将提供一个名为“Bearitos”的示例网站和工作:



在名为“Bearitos”的工作中，有一个名为“Bearitos to any”的项目



这基本上意味着在该项目内部，我有一个配置了三个阶段的管道:dev、staging 和 prod，它们使用以下参数进行参数化: DEPLOY_TO: Dev/staging/prod 和 DEPLOY_DB: Yes/No 。因此，根据用户的选择，Jenkins 将部署到该特定环境，我认为这甚至没有必要拥有这些选项，因为正确的部署流程应该是开发 -> 登台 -> 生产，不应该有场景dev 或 staging 将被跳过，然后在生产旁边部署，所以在我看来这应该更好地更新



在 Jenkinsfile 内部，我定义了三个阶段 Dev、Staging 或 Prod 以及是否选择构建数据库的选项，以下是我的 Jenkinsfile 的示例:

// Deployment template for CMS-based websites (Drupal or Wordpress)
// 
//
pipeline {
    agent any

    parameters {
        choice choices: ['Dev', 'Staging', 'Production'], description: "Choose which environment to push changes to.", name: "DEPLOY_TO"
        booleanParam defaultValue: true, "Choose whether to deploy the database.", name: "DEPLOY_DB"
    }

    environment {
         SITEID = "lb"
        NOFLAGS = "0"
        DBNAME = "wpress_myproject"
        DBSERVER = "dbserver"
        DBUSER = "WordpressUser"
        DBPASS = "hiddenpassword"
        EXCLUDE = "domain_commentmeta,domain_comments"  // separate multiple tables with commas
        DEPLOY_TO = "${params.DEPLOY_TO}"
        DEPLOY_DB = "${params.DEPLOY_DB}"
    }

    stages {
        stage("deploy-db-dev") {
            when {
                allOf { 
                    environment ignoreCase: true, name: "DEPLOY_TO", value: "dev"; 
                    environment ignoreCase: true, name: "DEPLOY_DB", value: "true"; 
                }
            }
            steps {
                // this stage only required until we make our dev the master DB
                // copy full dev database from bolwebdev1
                // import latest database dump to dev server
                script {
                    FILENM = sh(script: 'ls -t myproject-s-dump* | head -1', returnStdout: true)
                }
                //Fixing the problem with the collation existing in the sql dump file, refer to: https://stackoverflow.com/questions/42385099/1273-unknown-collation-utf8mb4-unicode-520-ci 
                //apparently, this is due to a version of mysql issue. Once the problem is fixed from the server side we can then remove the following lines. 

                sh """sed -i s/utf8mb4_unicode_520_ci/utf8mb4_unicode_ci/g ${FILENM}
                # The following line was added because the site is pointing to a staging server which we don't have control over, again, once this is fixed we can delete the following line of code. 
                sed -i s/myproject.staging.websites.3pth.com/myproject.example.net/g ${FILENM}
                mysql -h devserver2 -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_dev < ${WORKSPACE}/${FILENM}
                rm -f ${WORKSPACE}/${FILENM}"""
        }
        }
        stage("deploy-dev") {
            when {
                environment ignoreCase: true, name: "DEPLOY_TO", value: "dev"
            }
            steps {
                // copy files to devserver2
                // NOTE: if we move the repo to SVN, we should change httpdocs/ to ${env.SITEID}docs/
                sh """sudo chown jenkins:jenkins *

                #Replace the wp-config.php file with our domain file with our information. 
        /bin/cp httpdocs/wp-config-domain.php httpdocs/wp-config.php

                # prepare the dev server to receive files by changing the owner
                ssh webadmin@devserver2 'sudo chown -R webadmin:webadmin /var/opt/httpd/${env.SITEID}docs/'
                # copy files from control server to dev
                rsync --exclude=Jenkinsfile -rav -e ssh --delete ${WORKSPACE}/httpdocs/ webadmin@devserver2:/var/opt/httpd/${env.SITEID}docs/
                # fix the owner/permissions on the dev server
        ssh webadmin@devserver2 'sudo chown -R apache:${env.SITEID}-web /var/opt/httpd/${env.SITEID}docs/ && sudo chmod -R g+w /var/opt/httpd/${env.SITEID}docs/ && sudo find /var/opt/httpd/${env.SITEID}docs/ -type d -exec chmod g+s {} \\;'"""
            }
        }
        stage("deploy-db-staging") {
            when {
                allOf { 
                    environment ignoreCase: true, name: "DEPLOY_TO", value: "staging"; 
                    environment ignoreCase: true, name: "DEPLOY_DB", value: "true"; 
                }
            }
            steps {
                script {
                    def myexcludes = env.EXCLUDE.split(',').toList()
                    MYFLAGS = "-Q -K -c -e --default-character-set=utf8 "
                    if (env.NOFLAGS == "0") {
                        myexcludes.each {
                            MYFLAGS = "${MYFLAGS} --ignore-table=${env.DBNAME}_dev.${it}"
                        }
                    }
                }
                // pull a backup of the current dev database (may exclude some tables)
                sh """mysqldump -h devserver2 -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_dev ${MYFLAGS} > ${env.DBNAME}_dev.sql
        #Searching and replace for the URL to change from the dev sever to the staging server
                sed -i s/myproject.example.net/stage-myproject.example.net/g ${env.DBNAME}_dev.sql

        # create a backup copy of the current staging database (full backup)
                mysqldump -h ${env.DBSERVER} -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_stage > ${env.DBNAME}_stage_bak.sql
                # upload the dev database dump to the staging database
                mysql -h ${env.DBSERVER} -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_stage < ${WORKSPACE}/${env.DBNAME}_dev.sql
                rm -f ${WORKSPACE}/${env.DBNAME}_dev.sql"""
       }
        }
        stage("deploy-staging") {
            when {
                environment ignoreCase: true, name: "DEPLOY_TO", value: "staging"
            }
            steps {
                // copy files from dev to control server
                sh """rsync --exclude=.svn --exclude=.git -rav -e ssh webadmin@devserver2:/var/opt/httpd/${env.SITEID}docs/ /tmp/${env.SITEID}docs/

                #Replace the wp-config.php file with our domain file with our information. 
            /bin/cp httpdocs/wp-config-domain.php httpdocs/wp-config.php

                #prepare the staging server to receive files by changing the owner
                ssh webadmin@stageserver 'sudo chown -R webadmin:webadmin /var/opt/httpd/${env.SITEID}docs/'
                # copy files from control server to staging
                rsync --exclude=.svn --exclude=.git -rav -e ssh --delete /tmp/${env.SITEID}docs/ webadmin@stageserver:/var/opt/httpd/${env.SITEID}docs/
                # fix the owner/permissions on the staging server
                ssh webadmin@stageserver 'sudo chown -R apache:${env.SITEID}-web /var/opt/httpd/${env.SITEID}docs/ && sudo chmod -R g+w /var/opt/httpd/${env.SITEID}docs/ && sudo find /var/opt/httpd/${env.SITEID}docs/ -type d -exec chmod g+s {} \\;'

                #delete the temporary files on the control server
                rm -Rf /tmp/${env.SITEID}docs/
                # clear the Incapsula caches
                if [[ \$( curl -sS -X POST \"http://www.example.net/incapcache.php?api_key=asdaswwGR)feasdsdda&site_id=stage&resource_url=stage-myproject.example.net\" | jq -r .debug_info.id_info) != \"incapsula cache cleared successfuly\" ]]; then exit 255; fi"""
            }
        }
        stage("deploy-db-production") {
            when {
                allOf { 
                    environment ignoreCase: true, name: "DEPLOY_TO", value: "production"; 
                    environment ignoreCase: true, name: "DEPLOY_DB", value: "true"; 
                }
            }
            steps {
                script {
                    def myexcludes = env.EXCLUDE.split(',').toList()
                    MYFLAGS = "-Q -K -c -e --default-character-set=utf8 "
                    if (env.NOFLAGS == "0") {
                        myexcludes.each {
                            MYFLAGS = "${MYFLAGS} --ignore-table=${env.DBNAME}_stage.${it}"
                        }
                    }
                }
                sh """cd ${WORKSPACE}
                # pull a backup of the current staging database (may exclude some tables)
                mysqldump -h ${env.DBSERVER} -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_stage ${MYFLAGS} > ${env.DBNAME}_stage.sql
        #Searching and replace for the URL to change from the stage sever to the prod server
                sed -i s/stage-myproject.example.net/www.myproject.com/g ${env.DBNAME}_stage.sql

                # create a backup copy of the current production database (full backup)
                mysqldump -h ${env.DBSERVER} -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_prod > ${env.DBNAME}_prod_bak.sql
                # upload the staging database dump to the production database
                mysql -h ${env.DBSERVER} -u ${env.DBUSER} --password='${env.DBPASS}' ${env.DBNAME}_prod < ${WORKSPACE}/${env.DBNAME}_stage.sql
                rm -f ${WORKSPACE}/${env.DBNAME}_stage.sql"""
        }
        }
        stage("deploy-production") {
            when {
                environment ignoreCase: true, name: "DEPLOY_TO", value: "production"
            }
            steps {
                // copy files from staging to control server
                sh """rsync --exclude=.svn --exclude=.git -rav -e ssh webadmin@stageserver:/var/opt/httpd/${env.SITEID}docs/ /tmp/${env.SITEID}docs/

                # prepare the production server to receive files by changing the owner
                ssh webadmin@prodserver1 'sudo chown -R webadmin:webadmin /var/opt/httpd/${env.SITEID}docs'
                ssh webadmin@prodserver2 'sudo chown -R webadmin:webadmin /var/opt/httpd/${env.SITEID}docs'
                # copy files from control server to production
                rsync --exclude=.svn --exclude=.git -rav -e ssh --delete /tmp/${env.SITEID}docs/ webadmin@prodserver1:/var/opt/httpd/${env.SITEID}docs/
                rsync --exclude=.svn --exclude=.git -rav -e ssh --delete /tmp/${env.SITEID}docs/ webadmin@prodserver2:/var/opt/httpd/${env.SITEID}docs/
                # fix the owner/permissions on the production server
                ssh webadmin@prodserver1 'sudo chown -R apache:${env.SITEID}-web /var/opt/httpd/${env.SITEID}docs/'
                ssh webadmin@prodserver2 'sudo chown -R apache:${env.SITEID}-web /var/opt/httpd/${env.SITEID}docs/'
                ssh webadmin@prodserver1 'sudo chmod -R g+w /var/opt/httpd/${env.SITEID}docs/'
                ssh webadmin@prodserver2 'sudo chmod -R g+w /var/opt/httpd/${env.SITEID}docs/'
                ssh webadmin@prodserver1 'sudo find /var/opt/httpd/${env.SITEID}docs/ -type d -exec chmod g+s {} \\;'
                ssh webadmin@prodserver2 'sudo find /var/opt/httpd/${env.SITEID}docs/ -type d -exec chmod g+s {} \\;'

                # delete the temporary files on the control server
                rm -Rf /tmp/${env.SITEID}docs/
                # clear the Incapsula caches
                if [[ \$( curl -sS -X POST \"http://www.example.net/incapcache.php?api_key=asdaswwGR)feasdsdda&site_id=088&resource_url=www.myproject.com\" | jq -r .debug_info.id_info) != \"incapsula cache cleared successfuly\" ]]; then exit 255; fi"""
            }
        }
    }
}

我目前使用这种方法面临的问题是:

我不知道如何使部署自动化，因为它是一个
参数化管道，所以我不确定如何使其自动化。这
所需的过程是使部署自动化一次
Jenkins 在 git 存储库上每 X 分钟轮询一次，
自动部署到 Dev > Stage(仅当 Dev 部署成功时)然后停止
直到我们在 Staging 上进行 QA 之后手动部署到 Prod。

当前Git配置只配置了一个分支
(master) 这是开发人员在更改后推送更改的地方
想要部署到 Dev -> Stage -> Prod。但我认为
理想的情况是有一个用于开发部署的开发分支，然后
stage 分支用于部署到 Stage 环境，然后 master for
一旦我们将这些 dev 和 staging 分支 merge 到
主分支。我不确定这是否是最佳的，所以我会
感谢您对此的任何建议或想法。

所需的方法是解决所提到的问题，并且一旦开发 -> 分阶段部署成功，还有一种自动化的部署和通知方式。以及可以选择手动执行上述工作流程，就像我们现在正在做的一样(这不是那么重要，但拥有功能会很好)。

预先感谢您的帮助!

Answer 1

摆脱你的参数，它们是不需要的(会阻止你自动化)。

在 deploy to prod 中手动输入舞台

pipeline {
    agent any
    stages {
        stage('Deploy to prod') {
            input {
                message "Should we continue?"
                ok "Yes, we should."
            }
            steps {
                echo "Deploying."
            }
        }
    }
}

这应该是 Jenkins 中的多分支管道项目(因为您想在所有分支上使用它)

如果你想为不同的分支使用不同的阶段，请使用 when

pipeline {
    agent any
    stages {
        stage('Example Build') {
            steps {
                echo 'Hello World'
            }
        }
        stage('Example Deploy') {
            when {
                branch 'production'
            }
            steps {
                echo 'Deploying'
            }
        }
    }
}

至于建议 - 我会说您需要将您的 git 流程与您的 CI/CD 流程相匹配。给定 git 分支类型的生命周期是什么？给定阶段的结果是什么？是否要为所有分支和 deploy to prod 执行阶段？只为一个分支？