kubernetes - 如何从 pod 内部公开 kubernetes 度量服务器 api 以 curl ？

Question

我正在使用 metric server获取我的 Kubernetes 集群的使用情况。但是为了从主机外部使用它，我需要使用“kubectl proxy”。但我不想这样做，因为它不打算在后台运行。我希望它作为服务持续运行

我怎样才能实现这些

预期输出
curl clusterip:8001/apis/metrics.k8s.io/v1beta1/nodes

{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
  },
  "items": [
    {
      "metadata": {
        "name": "manhattan-master",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/manhattan-master",
        "creationTimestamp": "2019-11-15T04:26:47Z"
      },
      "timestamp": "2019-11-15T04:26:33Z",
      "window": "30s",
      "usage": {
        "cpu": "222998424n",
        "memory": "3580660Ki"
      }
    }
  ]

我尝试使用 负载均衡服务
指标服务器服务.yaml

apiVersion: v1
kind: Service
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
    kubernetes.io/name: "Metrics-server"
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    k8s-app: metrics-server
  ports:
  - port: 443
    protocol: TCP
    targetPort: main-port
  externalTrafficPolicy: Local
  type: LoadBalancer

kubectl 描述服务指标-master -n kube-system

[root@manhattan-master 1.8+]# kubectl describe service metrics-server -n kube-system
Name:                     metrics-server
Namespace:                kube-system
Labels:                   kubernetes.io/cluster-service=true
                          kubernetes.io/name=Metrics-server
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"kubernetes.io/cluster-service":"true","kubernetes.io/name":"Me...
Selector:                 k8s-app=metrics-server
Type:                     LoadBalancer
IP:                       10.110.223.216
Port:                     <unset>  443/TCP
TargetPort:               main-port/TCP
NodePort:                 <unset>  31043/TCP
Endpoints:                10.32.0.7:4443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     32208
Events:                   <none>

Answer 1

这可以通过创建一个新服务来公开 Metrics Server 来实现。您的 Metrics Server 服务应如下所示:

apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/name: Metrics-server-ext
  name: metrics-server-ext
  namespace: kube-system
  selfLink: /api/v1/namespaces/kube-system/services/metrics-server
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
  sessionAffinity: None
  type: LoadBalancer

如果您尝试访问此服务，您将面临一些授权问题，您需要做一些事情来提供所有必要的授权。

创建服务后，您需要创建集群角色绑定(bind)，以便我们的服务可以访问数据:

$ kubectl create clusterrolebinding node-admin-default-svc --clusterrole=cluster-admin --serviceaccount=default:default

在运行 curl 命令之前，我们需要获取 token ，以便我们可以将其传递给 curl 命令:

$ TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='default')].data.token}"|base64 --decode)

获取您的服务外部 IP:

kubectl get svc/metrics-server-ext -n kube-system -o jsonpath='{..ip}'

您的 curl 命令应该通过 Token key 来获得授权:

curl -k https://34.89.228.98/apis/metrics.k8s.io/v1beta1/nodes --header "Authorization: Bearer $TOKEN" --insecure

样本输出:

{
 "kind": "NodeMetricsList",
 "apiVersion": "metrics.k8s.io/v1beta1",
 "metadata": {
   "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
 },
 "items": [
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-ntmc",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-ntmc",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:17Z",
     "window": "30s",
     "usage": {
       "cpu": "52046272n",
       "memory": "686768Ki"
     }
   },
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-tkj9",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-tkj9",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:21Z",
     "window": "30s",
     "usage": {
       "cpu": "52320505n",
       "memory": "687252Ki"
     }
   },
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-v7m3",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-v7m3",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:17Z",
     "window": "30s",
     "usage": {
       "cpu": "45602403n",
       "memory": "609968Ki"
     }
   }
 ]
}

编辑:

您还可以选择从您的 pod 访问它，因为您在默认服务帐户中使用 cluster-admin 角色创建了集群角色绑定(bind)。

例如，从包含 curl 命令的图像创建一个 pod:

$ kubectl run bb-$RANDOM --rm -i --image=ellerbrock/alpine-bash-curl-ssl --restart=Never --tty -- /bin/bash

比您需要执行到您的 pod 并运行:

$ curl -k -X GET https://kubernetes.default/apis/metrics.k8s.io/v1beta1/nodes --header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" --insecure

在这里，我们以完全不同的方式传递前面提到的相同 TOKEN。