jenkins - Github webhook 不再触发 Jenkins 构建

Question

我们最近注意到我们的 Jenkins 构建停止自动触发。经过进一步调查，发现了很多问题。

1. Github webhook 不成功，Github 在 Github 的 webhook 配置 UI 中报告“无法连接到服务器”。我可以确认托管 Jenkins 的 ELB 和 EC2 实例正常运行且健康。此处未更改 DNS。

2. Jenkins 日志报告各种故障:

尽管具有有效的 Jenkins 用户名和密码凭据(密码是个人 API token )，但凭据无效:

没有具有管理 GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED] Hook 的管理员权限的凭据

无法删除提交后 Hook :

ALPN callback dropped: SPDY and HTTP/2 are disabled. Is alpn-boot on the boot class path?
Apr 12, 2019 6:15:43 PM WARNING org.jenkinsci.plugins.github.webhook.WebhookManager$2 applyNullSafe
Failed to add GitHub webhook for GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
java.io.FileNotFoundException: https://api.github.com/repos/REDACTED/REDACTED/hooks/101704125
    at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
    at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
    at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
    at org.kohsuke.github.Requester.parse(Requester.java:617)
    at org.kohsuke.github.Requester.parse(Requester.java:599)
    at org.kohsuke.github.Requester._to(Requester.java:277)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/hooks/#delete-a-hook"}
    at org.kohsuke.github.Requester.handleApiError(Requester.java:691)
    at org.kohsuke.github.Requester._to(Requester.java:298)
    at org.kohsuke.github.Requester.to(Requester.java:239)
    at org.kohsuke.github.Requester.to(Requester.java:227)
    at org.kohsuke.github.GHHook.delete(GHHook.java:56)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:344)
Caused: org.kohsuke.github.GHException: Failed to delete post-commit hook
    at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:347)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:341)
    at org.jenkinsci.plugins.github.util.misc.NullSafePredicate.apply(NullSafePredicate.java:19)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:649)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:119)
    at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:202)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:175)
    at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
    at com.google.common.collect.Iterators$8.next(Iterators.java:812)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:648)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:119)
    at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$1.run(WebhookManager.java:127)
    at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

创建钩子(Hook)失败:

Apr 12, 2019 6:15:44 PM WARNING org.jenkinsci.plugins.github.webhook.WebhookManager$2 applyNullSafe
Failed to add GitHub webhook for GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
java.io.FileNotFoundException: https://api.github.com/repos/REDACTED/REDACTED/hooks
    at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
    at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
    at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
    at org.kohsuke.github.Requester.parse(Requester.java:617)
    at org.kohsuke.github.Requester.parse(Requester.java:599)
    at org.kohsuke.github.Requester._to(Requester.java:277)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Validation Failed","errors":[{"resource":"Hook","code":"custom","message":"Hook already exists on this repository"}],"documentation_url":"https://developer.github.com/v3/repos/hooks/#create-a-hook"}
    at org.kohsuke.github.Requester.handleApiError(Requester.java:691)
    at org.kohsuke.github.Requester._to(Requester.java:298)
    at org.kohsuke.github.Requester.to(Requester.java:239)
    at org.kohsuke.github.GHHooks$Context.createHook(GHHooks.java:49)
    at org.kohsuke.github.GHRepository.createHook(GHRepository.java:1206)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:329)
Caused: org.kohsuke.github.GHException: Failed to create hook
    at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:331)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:316)
    at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:204)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:175)
    at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
    at com.google.common.collect.Iterators$8.next(Iterators.java:812)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:648)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:138)
    at com.google.common.collect.Lists.newArrayList(Lists.java:119)
    at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
    at org.jenkinsci.plugins.github.webhook.WebhookManager$1.run(WebhookManager.java:127)
    at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

这很奇怪，因为我也看到了积极的日志，例如:

GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO with [GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED_REPO]] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/develop with [GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED_REPO]] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/master with [] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/release%2F0.x with [] (events: [PULL_REQUEST, PUSH]

我们已经配置了以下 Jenkins 插件:

• Git Plugin
• Github 组织 Github Branch Source与 https://support.cloudbees.com/hc/en-us/articles/224543927-GitHub-webhook-configuration

我已按照 https://support.cloudbees.com/hc/en-us/articles/224621648-GitHub-webhook-troubleshooting 上的所有故障排除步骤进行操作但我不知道在哪里。失败的步骤:

• A.2。我尝试重新传送 webhook 负载但得到 Couldn't connect to server
• B.4 显示未能删除提交后 Hook 、无法创建 Hook 的列表
，并且 没有具有管理员权限的凭据来管理 GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED] 上的 Hook

什么起作用

• 能够扫描组织
• 能够使用 Build Now 手动触发构建

似乎有太多问题，我不知道下一步该看哪里。提前感谢您的帮助。

Answer 1

看起来我们的 AWS 安全组将 Github Hook IP 地址列入白名单 (https://api.github.com/meta)，而 Github 最近添加或更改了一个不在安全组中的 IP 地址。所以我们的 AWS 安全组拒绝了请求。我们添加并刷新了 IP 地址，现在可以使用了。