kubernetes - 从external.api获取查询时的Prometheus Adapter Goroutine问题

Question

我已经实现并向k8s环境注册了外部api，但是在获取指标时遇到了一些问题。

这是外部api指标的规则:

  - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
    resources:
      overrides:
        kubernetes_namespace: {resource: "namespace"}
        kubernetes_pod_name: {resource: "pod"}
    name:
      matches: "^(.*)_total"
      as: "${1}"

当检查kubernetes外部api指标时，资源值不是null，看起来不错，但是，

{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "external.metrics.k8s.io/v1beta1",
  "resources": [
    {
      "name": "http_requests",
      "singularName": "",
      "namespaced": true,
      "kind": "ExternalMetricValueList",
      "verbs": [
        "get"
      ]
    }
  ]
}

完成所有这些操作后，当我想检查返回此错误的 http_requests度量kubernetes的值时；
Error from server (InternalError): Internal error occurred: unable to fetch metrics
我已经从 prometheus-adapter pod中检查了这个问题，这些是与此查询相关的错误日志；

E1005 01:21:26.387384       1 provider.go:47] unable to generate a query for the metric: empty query produced by metrics query template
I1005 01:21:26.387594       1 wrap.go:42] GET /apis/external.metrics.k8s.io/v1beta1/namespaces/default/http_requests: (504.956µs) 500
goroutine 1909 [running]:
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc420374930, 0x1f4)
   /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc420374930, 0x1f4)
   /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc42105b2a0, 0x1f4)
   /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics.(*ResponseWriterDelegator).WriteHeader(0xc42107f920, 0x1f4)
   /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go:307 +0x45
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.SerializeObject(0x1791e20, 0x10, 0x7f340078e6c8, 0xc4204ec580, 0x190fd20, 0xc420e8db88, 0xc421081600, 0x1f4, 0x18f9920, 0xc420039320)
   /go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:95 +0x8d

....
....
....

但是我已经解析了此错误消息: unable to generate a query for the metric: empty query produced by metrics query template
这个问题的原因是什么？

请帮忙

Answer 1

我认为您收到此错误:

unable to generate a query for the metric: empty query produced by metrics query template

因为您的API指标不包含 metricsQuery值。

第二件事是我不确定为什么您在 external.metrics中使用 http_request。在这种情况下，您可以使用 custom.metrics。
提及的指标是最受欢迎的指标之一，因此您可以在线逐步找到许多教程，以了解如何实现这一目标。
例如 this或此 Github tutorial。