prometheus - Helm/kube-prometheus-stack : Can I create rules for exporters in values. yaml？

Question

我希望能够指定我的所有规则，例如 prometheus-blackbox-exporter，因此已将其添加到 rules-mine.yaml 并使用

helm upgrade --install -n monitoring blackbox -f values.yaml -f rules-mine.yaml .

我看不到 http://localhost:9090/rules 中列出的任何规则，并且似乎没有任何内容被评估为没有警报......我需要以 IaC 的方式完成所有操作，并以自动化的方式通过 terraform 进行部署。

• 是否可以通过这种方式向导出商添加规则？
• 如果是这样，那么任何人都可以看到下面的文件有问题吗？
• 如果没有，我如何有效地向许多导出商添加规则？

rules-mine.yaml 文件包含:

prometheusRule:
  enabled:  true
  namespace: monitoring
  additionalLabels:
    team: foxtrot_blackbox
    environment: production
    cluster: cluster
    namespace: namespace_x
  namespace: "monitoring"

  rules:
  - alert: BlackboxProbeFailed
    expr: probe_success == 0
    for: 0m
    labels:
      severity: critical
    annotations:
      summary: Blackbox probe failed (instance {{`{{`}} $labels.instance {{`}}`}})
      description: "Probe failed\n  VALUE = {{`{{`}} $value {{`}}`}}"

  - alert: BlackboxSlowProbe
    expr: avg_over_time(probe_duration_seconds[1m]) > 1
    for: 1m
    labels:
      severity: warning
    annotations:
      summary: Blackbox slow probe (instance {{`{{`}} $labels.instance {{`}}`}})
      description: "Blackbox probe took more than 1s to complete\n  VALUE = {{`{{`}} $value {{`}}`}}"

感谢您的帮助......

Answer 1

我发现的最好方法似乎是将导出器规则添加到 kube-prometheus-stack values.yaml 文件中(我实际上创建了一个单独的 rules .yaml 文件)并将其提供给 helm:

• helm升级--install -n监控prometheus --create-namespace -f value-mine.yaml -f规则-mine.yaml prometheus-community/kube-prometheus-stack

然后按照我的意愿选择所有规则，这似乎是一个不错的解决方案。但我仍然希望它们与导出商分组 - 如果我找到解决方案，我会再次发布。

additionalPrometheusRulesMap:
  prometheus.rules:
    groups:
    - name: company.prometheus.rules
      rules:
      - alert: PrometheusNotificationsBacklog
        expr: min_over_time(prometheus_notifications_queue_length[10m]) > 0
        for: 0m
        labels:
          severity: warning
        annotations:
          summary: Prometheus notifications backlog (instance {{ $labels.instance }})
          description: The Prometheus notification queue has not been empty for 10 minutes\nVALUE = {{ $value }}
          dashboard_url: ${grafana_url}/d/blackbox/blackbox-exporter?var-instance={{ $labels.instance }}
          runbook_url: ${wiki_url}/{{ $labels.alertname }}

  company.blackbox.rules:
    groups:
    - name: company.blackbox.rules
      rules:
      - alert: BlackboxProbeFailed
        expr: probe_success == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: Blackbox probe failed (instance {{ $labels.instance }})
          description: Probe failed\nVALUE = {{ $value }}
          dashboard_url: ${grafana_url}/d/blackbox/blackbox-exporter?var-instance={{ $labels.instance }}
          runbook_url: ${wiki_url}/{{ $labels.alertname }}

      - alert: BlackboxSlowProbe
        expr: avg_over_time(probe_duration_seconds[1m]) > 1
        for: 3m
        labels:
          severity: warning
        annotations:
          summary: Blackbox slow probe (instance {{ $labels.instance }})
          description: "Blackbox probe took more than 1s to complete\nVALUE = {{ $value }}"
          dashboard_url: ${grafana_url}/d/blackbox/blackbox-exporter?var-instance={{ $labels.instance }}
          runbook_url: ${wiki_url}/{{ $labels.alertname }}

# etc....