kubernetes - 基于自定义指标的规模部署

Question

我正在尝试基于来自自定义指标服务器的自定义指标来扩展部署。我部署了服务器，当我这样做时
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/services/kubernetes/test-metric"
我回来了这个JSON

{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/services/kubernetes/test-metric"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Service",
        "namespace": "default",
        "name": "kubernetes",
        "apiVersion": "/v1"
      },
      "metricName": "test-metric",
      "timestamp": "2019-01-26T02:36:19Z",
      "value": "300m",
      "selector": null
    }
  ]
}

然后我使用这个创建了 hpa.yml

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: test-all-deployment
  namespace: default
spec:
  maxReplicas: 10
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: test-all-deployment
  metrics:
  - type: Object
    object:
      target:
        kind: Service
        name: kubernetes
        apiVersion: custom.metrics.k8s.io/v1beta1
      metricName: test-metric
      targetValue: 200m

但它无法扩展，我不确定哪里出了问题。运行 get hpa返回

NAME                        REFERENCE                              TARGETS          MINPODS   MAXPODS   REPLICAS   AGE
test-all-deployment   Deployment/test-all-deployment   <unknown>/200m   1         10        1          9m

我不确定的部分是hpa定义中 target集合中的 metrics对象。在这里查看文档 https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/

它有

  describedObject:
    apiVersion: extensions/v1beta1
    kind: Ingress
    name: main-route
  target:
    kind: Value
    value: 10k

但这给了我API v2beta1的验证错误。并在这里查看实际对象 https://github.com/kubernetes/api/blob/master/autoscaling/v2beta1/types.go#L296似乎不匹配。我不知道如何使用v2beta1 API进行指定。

Answer 1

好像documentation中有错误。在同一示例中，使用了两个不同的API版本。

autoscaling / v2beta1表示法:

  - type: Pods
    pods:
      metric:
        name: packets-per-second
      targetAverageValue: 1k

autoscaling / v2beta2表示法:

  - type: Resource
    resource:
      name: cpu
      target:
        type: AverageUtilization
        averageUtilization: 50

autoscaling / v2beta1和autoscaling / v2beta2 API之间存在区别:

kubectl get hpa.v2beta1.autoscaling -o yaml --export > hpa2b1-export.yaml
kubectl get hpa.v2beta2.autoscaling -o yaml --export > hpa2b2-export.yaml
diff -y hpa2b1-export.yaml hpa2b2-export.yaml 

#hpa.v2beta1.autoscaling                                        hpa.v2beta2.autoscaling
#-----------------------------------------------------------------------------------
apiVersion: v1                                                  apiVersion: v1
items:                                                          items:
- apiVersion: autoscaling/v2beta1                             | - apiVersion: autoscaling/v2beta2
  kind: HorizontalPodAutoscaler                                   kind: HorizontalPodAutoscaler
  metadata:                                                       metadata:
    creationTimestamp: "2019-03-21T13:17:47Z"                       creationTimestamp: "2019-03-21T13:17:47Z"
    name: php-apache                                                name: php-apache
    namespace: default                                              namespace: default
    resourceVersion: "8441304"                                      resourceVersion: "8441304"
    selfLink: /apis/autoscaling/v2beta1/namespaces/default/ho |     selfLink: /apis/autoscaling/v2beta2/namespaces/default/ho
    uid: b8490a0a-4bdb-11e9-9043-42010a9c0003                       uid: b8490a0a-4bdb-11e9-9043-42010a9c0003
  spec:                                                           spec:
    maxReplicas: 10                                                 maxReplicas: 10
    metrics:                                                        metrics:
    - resource:                                                     - resource:
        name: cpu                                                       name: cpu
        targetAverageUtilization: 50                          |         target:
                                                              >           averageUtilization: 50
                                                              >           type: Utilization
      type: Resource                                                  type: Resource
    minReplicas: 1                                                  minReplicas: 1
    scaleTargetRef:                                                 scaleTargetRef:
      apiVersion: extensions/v1beta1                                  apiVersion: extensions/v1beta1
      kind: Deployment                                                kind: Deployment
      name: php-apache                                                name: php-apache
  status:                                                         status:
    conditions:                                                     conditions:
    - lastTransitionTime: "2019-03-21T13:18:02Z"                    - lastTransitionTime: "2019-03-21T13:18:02Z"
      message: recommended size matches current size                  message: recommended size matches current size
      reason: ReadyForNewScale                                        reason: ReadyForNewScale
      status: "True"                                                  status: "True"
      type: AbleToScale                                               type: AbleToScale
    - lastTransitionTime: "2019-03-21T13:18:47Z"                    - lastTransitionTime: "2019-03-21T13:18:47Z"
      message: the HPA was able to successfully calculate a r         message: the HPA was able to successfully calculate a r
        resource utilization (percentage of request)                    resource utilization (percentage of request)
      reason: ValidMetricFound                                        reason: ValidMetricFound
      status: "True"                                                  status: "True"
      type: ScalingActive                                             type: ScalingActive
    - lastTransitionTime: "2019-03-21T13:23:13Z"                    - lastTransitionTime: "2019-03-21T13:23:13Z"
      message: the desired replica count is increasing faster         message: the desired replica count is increasing faster
        rate                                                            rate
      reason: TooFewReplicas                                          reason: TooFewReplicas
      status: "True"                                                  status: "True"
      type: ScalingLimited                                            type: ScalingLimited
    currentMetrics:                                                 currentMetrics:
    - resource:                                                     - resource:
        currentAverageUtilization: 0                          |         current:
        currentAverageValue: 1m                               |           averageUtilization: 0
                                                              >           averageValue: 1m
        name: cpu                                                       name: cpu
      type: Resource                                                  type: Resource
    currentReplicas: 1                                              currentReplicas: 1
    desiredReplicas: 1                                              desiredReplicas: 1
kind: List                                                      kind: List
metadata:                                                       metadata:
  resourceVersion: ""                                             resourceVersion: ""
  selfLink: ""                                                    selfLink: ""

这是对象定义的外观:

#hpa.v2beta1.autoscaling                                        hpa.v2beta2.autoscaling
#-----------------------------------------------------------------------------------

type: Object                                                    type: Object
object:                                                         object:
  metric:                                                         metric:
    name: requests-per-second                                       name: requests-per-second
  describedObject:                                                describedObject:
    apiVersion: extensions/v1beta1                                  apiVersion: extensions/v1beta1
    kind: Ingress                                                   kind: Ingress
    name: main-route                                                name: main-route
  targetValue: 2k                                                 target:
                                                                    type: Value
                                                                    value: 2k