Kubernetes HPA 使用来自另一个部署的指标

Question

我目前正在尝试使用 prometheus 和 prometheus 适配器运行一个自动缩放演示，我想知道是否有一种方法可以根据 prometheus 从另一个部署中收集的指标自动缩放我的一个部署。

我现在有 2 种不同的部署，kafka-consumer-application(我想扩展)和 kafka-exporter(它公开了我将用于扩展的 kafka 指标)。我知道，如果我将它们作为同一部署中的容器，则自动缩放会起作用，但问题是 kafka-exporter 也会自动缩放并且它不理想，所以我想将它们分开。我尝试使用以下 HPA，但无法正常工作:

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
  name: consumer-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kafka-consumer-application
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: object
    object:
      target: kafka-exporter
      metricName: "kafka_consumergroup_lag"
      targetValue: 5

我不确定我是不是做错了什么，或者这不是我能做的，所以欢迎任何建议。

谢谢!

注意:我正在使用此配置运行适配器:

rules:
  default: false
  resource: {}
  custom:
    - seriesQuery: 'kafka_consumergroup_lag'
      resources:
        overrides:
          kubernetes_namespace: {resource: "namespace"}
          kubernetes_pod_name: {resource: "pod"}
      name:
       matches: "kafka_consumergroup_lag"
       as: "kafka_consumergroup_lag"
      metricsQuery: 'avg_over_time(kafka_consumergroup_lag{topic="my-topic",consumergroup="we-consume"}[1m])'
``

Answer 1

在kubernetes documentation你可以阅读:

Autoscaling on metrics not related to Kubernetes objects Applications running on Kubernetes may need to autoscale based on metrics that don’t have an obvious relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case with external metrics

因此，使用外部指标，您的 HPA yaml 可能如下所示:

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta2
metadata:
  name: consumer-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kafka-consumer-application
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: kafka_consumergroup_lag
        #selector:
        #  matchLabels:
        #    topic: "my-topic"
      target:
        type: AverageValue
        averageValue: 5

如果你有多个 kafka-exporter，你可以使用 selector 来过滤它(source):

selector is the string-encoded form of a standard kubernetes label selector for the given metric When set, it is passed as an additional parameter to the metrics server for more specific metrics scoping. When unset, just the metricName will be used to gather metrics

另请查看 this Stack question .