prometheus - 如何在 Alertmanager 中向一个接收者组发送多个警报？

Question

我在这里看过这个问题/答案:Prometheus AlertManager - Send Alerts to different clients based on routes

这对我来说是一个很好的开始，我希望我可以向那里的回答者提出一个快速的问题，但我没有代表。

无论如何，我有一个 alert.rules.yml 文件，其中包含两个组，如下所示:

groups:
- name: DevOpsAlerts
  rules:

  - alert: InstanceDown
    expr: up == 0
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.instance }} down"
      description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes. ({{ $value }} minutes)"

  - alert: InstanceHighCpu
    expr: 100 - (avg by (host) (irate(node_cpu{mode="idle"}[5m])) * 100) > 5
    for: 10m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.host }}: CPU High"
      description: "{{ $labels.host }} has high CPU activity"

- name: TestTeam2
  rules:

  - alert: - alert: InstanceLowMemory
    expr: node_memory_MemAvailable < 268435456
    for: 10m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.host }}: memory low"
      description: "{{ $labels.host }} has less than 256M memory available"

  - alert: InstanceLowDisk
    expr: node_filesystem_avail{mountpoint="/"} < 1073741824
    for: 10m
    labels:
      severity: critical
    annotations:
      summary: "Instance {{ $labels.host }}: low disk space"
      description: "{{ $labels.host }} has less than 1G FS space"

除此之外，我还有一个 alertmanager.yml 文件，看起来像

global:
  smtp_smarthost: 'smtpserver'
  smtp_from: 'alertsender@email.com'
  smtp_auth_username: 'alertsender@email.com'
  smtp_auth_password: 'verystrongpassword'
  smtp_require_tls: maybe

route:
  group_by: ['alertname', 'cluster', 'service']

  #default receiver
  receiver: DevOps
  routes:
    - match:
        alertname: InstanceDown
      receiver: DevOps

    - match:
        group: InstanceHighCpu
      receiver: test-team-1

inhibit_rules:
- source_match:
    severity: 'critical'
  target_match:
    severity: 'warning'
  equal: ['alertname', 'cluster', 'service']

receivers:
- name: DevOps
  email_configs:
  # - to: devops_dude@email.com

- name: test-team-1
  email_configs:
  - to: test-dude1@email.com #This can be any email specified from the team

- name: team-name-2
  email_configs:
  - to: test_email@test.com #This can be any email specified from the team

因此，根据我收集的信息，我可以通过从警报规则文件中指定警报名称并将其路由到特定接收器，将警报路由到特定接收器组。

我真正遇到的一个大问题是:有没有办法根据组名称而不是警报规则文件中的警报名称将警报路由到特定的接收者。

所以不是

routes:
  - match:
      alertname: InstanceDown
    receiver: DevOps

有没有什么方法可以实现:

routes:
  - match:
      group: DevOpsAlerts
    receiver: DevOps

我一直在互联网上搜索类似这样的示例，但我找不到任何东西。谢谢。

Answer 1

规则组名称不会暴露给 Alertmanager，它们主要用于在 Prometheus 端进行调试。

您可以为每个警报添加一个 group: DevOpsAlerts 标签。