azure - 如何在service-node-port-range默认范围之外的NodePort上启动kubernetes服务？

Question

我一直在尝试在默认端口范围之外的 NodePort 上启动 kubernetes-dashboard(以及最终的其他服务)，但收效甚微，这是我的设置:云提供商:Azure(不是Azure容器服务)操作系统:CentOS 7

这是我尝试过的:

更新主机

$ yum update

安装 kubeadm

$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
       https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
$ setenforce 0
$ yum install -y docker kubelet kubeadm kubectl kubernetes-cni
$ systemctl enable docker && systemctl start docker
$ systemctl enable kubelet && systemctl start kubelet

使用 kubeadm 启动集群

$ kubeadm init

允许在主节点上运行容器，因为我们有一个单节点集群

$ kubectl taint nodes --all dedicated-

安装 Pod 网络

$ kubectl apply -f https://git.io/weave-kube

我们的 kubernetes-dashboard 部署 (@ ~/kubernetes-dashboard.yaml

# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or     implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Configuration to deploy release version of the Dashboard UI.
#
# Example usage: kubectl create -f <this_file>

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubernetes-dashboard
  template:
    metadata:
      labels:
        app: kubernetes-dashboard
      # Comment the following annotation if Dashboard must not be deployed on master
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      containers:
      - name: kubernetes-dashboard
        image: gcr.io/google_containers/kubernetes-dashboard-amd64:v1.5.1
        imagePullPolicy: Always
        ports:
        - containerPort: 9090
          protocol: TCP
        args:
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          # - --apiserver-host=http://my-address:port
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 8880
    targetPort: 9090
    nodePort: 8880
  selector:
    app: kubernetes-dashboard

创建我们的部署

$ kubectl create -f ~/kubernetes-dashboard.yaml
deployment "kubernetes-dashboard" created
The Service "kubernetes-dashboard" is invalid: spec.ports[0].nodePort: Invalid value: 8880: provided port is not in the valid range. The range of valid ports is 30000-32767

我发现要更改有效端口的范围，我可以在 kube-apiserver 上设置 service-node-port-range 选项以允许不同的端口范围，所以我尝试了这个:

$ kubectl get po --namespace=kube-system
NAME                                    READY     STATUS    RESTARTS       AGE
dummy-2088944543-lr2zb                  1/1       Running   0              31m
etcd-test2-highr                        1/1       Running   0              31m
kube-apiserver-test2-highr              1/1       Running   0              31m
kube-controller-manager-test2-highr     1/1       Running   2              31m
kube-discovery-1769846148-wmbhb         1/1       Running   0              31m
kube-dns-2924299975-8vwjm               4/4       Running   0              31m
kube-proxy-0ls9c                        1/1       Running   0              31m
kube-scheduler-test2-highr              1/1       Running   2              31m
kubernetes-dashboard-3203831700-qrvdn   1/1       Running   0              22s
weave-net-m9rxh                         2/2       Running   0              31m

将“--service-node-port-range=8880-8880”添加到 kube-apiserver-test2-highr

$ kubectl edit po kube-apiserver-test2-highr --namespace=kube-system
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-apiserver",
    "namespace": "kube-system",
    "creationTimestamp": null,
    "labels": {
      "component": "kube-apiserver",
      "tier": "control-plane"
    }
  },
  "spec": {
    "volumes": [
      {
        "name": "k8s",
        "hostPath": {
          "path": "/etc/kubernetes"
        }
      },
      {
        "name": "certs",
        "hostPath": {
          "path": "/etc/ssl/certs"
        }
      },
      {
        "name": "pki",
        "hostPath": {
          "path": "/etc/pki"
        }
      }
    ],
    "containers": [
      {
        "name": "kube-apiserver",
        "image": "gcr.io/google_containers/kube-apiserver-amd64:v1.5.3",
        "command": [
          "kube-apiserver",
          "--insecure-bind-address=127.0.0.1",
          "--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota",
          "--service-cluster-ip-range=10.96.0.0/12",
          "--service-node-port-range=8880-8880",
          "--service-account-key-file=/etc/kubernetes/pki/apiserver-key.pem",
          "--client-ca-file=/etc/kubernetes/pki/ca.pem",
          "--tls-cert-file=/etc/kubernetes/pki/apiserver.pem",
          "--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem",
          "--token-auth-file=/etc/kubernetes/pki/tokens.csv",
          "--secure-port=6443",
          "--allow-privileged",
          "--advertise-address=100.112.226.5",
          "--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname",
          "--anonymous-auth=false",
          "--etcd-servers=http://127.0.0.1:2379"
        ],
        "resources": {
          "requests": {
            "cpu": "250m"
          }
        },
        "volumeMounts": [
          {
            "name": "k8s",
            "readOnly": true,
            "mountPath": "/etc/kubernetes/"
          },
          {
            "name": "certs",
            "mountPath": "/etc/ssl/certs"
          },
          {
            "name": "pki",
            "mountPath": "/etc/pki"
          }
        ],
        "livenessProbe": {
          "httpGet": {
            "path": "/healthz",
            "port": 8080,
            "host": "127.0.0.1"
          },
          "initialDelaySeconds": 15,
          "timeoutSeconds": 15,
          "failureThreshold": 8
        }
      }
    ],
    "hostNetwork": true
  },
  "status": {}

$ :wq

以下是截断的响应

# pods "kube-apiserver-test2-highr" was not valid:
# * spec: Forbidden: pod updates may not change fields other than `containers[*].image` or `spec.activeDeadlineSeconds`

所以我尝试了一种不同的方法，我编辑了 kube-apiserver 的部署文件，并进行了与上述相同的更改并运行以下命令:

$ kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.json --namespace=kube-system

得到了这样的回复:

The connection to the server localhost:8080 was refused - did you specify the right host or port?

现在我陷入困境，如何更改有效端口的范围？

Answer 1

您指定的 --service-node-port-range=8880-8880 错误。您仅将其设置为一个端口，将其设置为一个范围。

第二个问题:您将服务设置为使用 9090，但它不在范围内。

 ports:
  - port: 80
    targetPort: 9090
    nodePort: 9090

API 服务器也应该有一个部署，尝试在部署本身中编辑端口范围并删除 api 服务器 pod，以便通过新配置重新创建它。