portforwarding - kubectl 端口转发连接被拒绝 [ socat ]

Question

我在 kubernetes 的一个端口上运行 pyspark。我正在尝试转发到我的本地机器。执行 python 文件时出现此错误。

Forwarding from 127.0.0.1:7077 -> 7077
Forwarding from [::1]:7077 -> 7077
Handling connection for 7077
E0401 01:08:11.964798   20399 portforward.go:400] an error occurred forwarding 7077 -> 7077: error forwarding port 7077 to pod 68ced395bd081247d1ee6b431776ac2bd3fbfda4d516da156959b6271c2ad90c, uid : exit status 1: 2019/03/31 19:38:11 socat[1748104] E connect(5, AF=2 127.0.0.1:7077, 16): Connection refused

这是我的python文件的几行。我在定义 conf 的行中遇到错误。

from pyspark import SparkContext, SparkConf
from pyspark.sql import SQLContext

conf = SparkConf().setMaster("spark://localhost:7077").setAppName("Stand Alone Python Script")

我已经尝试安装 socat在 Kubernetes 上。我在本地使用 spark 版本 2.4.0。我什至尝试在 YAML 文件中公开端口 7077。没有成功。

这是用于部署的 YAML 文件。

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  creationTimestamp: 2018-10-07T15:23:35Z
  generation: 16
  labels:
    chart: spark-0.2.1
    component: m3-zeppelin
    heritage: Tiller
    release: m3
  name: m3-zeppelin
  namespace: default
  resourceVersion: "55461362"
  selfLink: /apis/apps/v1beta1/namespaces/default/statefulsets/m3-zeppelin
  uid: f56e86fa-ca44-11e8-af6c-42010a8a00f2
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      component: m3-zeppelin
  serviceName: m3-zeppelin
  template:
    metadata:
      creationTimestamp: null
      labels:
        chart: spark-0.2.1
        component: m3-zeppelin
        heritage: Tiller
        release: m3
    spec:
      containers:
      - args:
        - bash
        - -c
        - wget -qO- https://archive.apache.org/dist/spark/spark-2.2.2/spark-2.2.2-bin-hadoop2.7.tgz
          | tar xz; mv spark-2.2.2-bin-hadoop2.7 spark; curl -sSLO https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop2.jar;
          mv gcs-connector-latest-hadoop2.jar lib; ./bin/zeppelin.sh
        env:
        - name: SPARK_MASTER
          value: spark://m3-master:7077
        image: apache/zeppelin:0.8.0
        imagePullPolicy: IfNotPresent
        name: m3-zeppelin
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        resources:
          requests:
            cpu: 100m
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /zeppelin/conf
          name: m3-zeppelin-config
        - mountPath: /zeppelin/notebook
          name: m3-zeppelin-notebook
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      creationTimestamp: null
      name: m3-zeppelin-config
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10G
      storageClassName: standard
    status:
      phase: Pending
  - metadata:
      creationTimestamp: null
      name: m3-zeppelin-notebook
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10G
      storageClassName: standard
    status:
      phase: Pending
status:
  collisionCount: 0
  currentReplicas: 1
  currentRevision: m3-zeppelin-5779b84d99
  observedGeneration: 16
  readyReplicas: 1
  replicas: 1
  updateRevision: m3-zeppelin-5779b84d99
  updatedReplicas: 1

Answer 1

从 Kubernetes 的角度特别关注错误，它可能与以下内容有关:

发送请求的端口与接收端不匹配(例如向 NGINX 上的 port 实例发送请求:1234 )

Pod没有收听想要的 port .

我已经设法用 kubespray 创建的 Kubernetes 集群重现了这个错误。 .
假设您已运行以下步骤:

$ kubectl create deployment nginx --image=nginx

$ kubectl port-forward deployment/nginx 8080:80

一切都应该正确， NGINX运行时应出现欢迎页面: $ curl localhost:8080 .
如果我们对 port-forward 进行了更改如下命令(注意 1234 端口):

$ kubectl port-forward deployment/nginx 8080:1234

您将收到以下错误:

Forwarding from 127.0.0.1:8080 -> 1234
Forwarding from [::1]:8080 -> 1234
Handling connection for 8080
E0303 22:37:30.698827  625081 portforward.go:400] an error occurred forwarding 8080 -> 1234: error forwarding port 1234 to pod e535674b2c8fbf66252692b083f89e40f22e48b7a29dbb98495d8a15326cd4c4, uid : exit status 1: 2021/03/23 11:44:38 socat[674028] E connect(5, AF=2 127.0.0.1:1234, 16): Connection refused

这也适用于 Pod该应用程序没有绑定(bind)到端口和/或没有监听。

A side note!

You can simulate it by running an Ubuntu Pod where you try to curl its port 80. It will fail as nothing listens on its port. Try to exec into it and run $ apt update && apt install -y nginx and try to curl again (with kubectl port-forward configured). It will work and won't produce the error mentioned (socat).

解决问题的一部分:

I even tried exposing port 7077 in YAML file. Did not work out.

如果您的意思是您已包含 - containerPort: 8080 .该字段纯粹是信息性的，不进行任何配置。你可以在这里读更多关于它的内容:

Stackoverflow.com: Answer: Why do we need a port/containerPort in a Kuberntes deployment/container definition?

(除此之外，我认为您使用的端口不正确: 7077 )

至于 $ kubectl port-forward --address 0.0.0.0 .这是一种公开端口转发的方法，以便它可以监听所有接口(interface)(在主机上)。它可以允许访问您的 port-forward来自 LAN :

$ kubectl port-forward --help (部分):

  # Listen on port 8888 on localhost and selected IP, forwarding to 5000 in the pod
  kubectl port-forward --address localhost,10.19.21.23 pod/mypod 8888:5000

其他资源:

Kubernetes.io: Docs: Tasks: Access application cluster: Port forward access to application cluster