kubernetes - 在 minikube 上公开 cassandra 集群以供外部访问

Question

我正在尝试在 minikube 中部署一个 cassandra 多节点集群，我遵循了本教程 Example: Deploying Cassandra with Stateful Sets并进行了一些修改，集群启动并运行，使用 kubectl 我可以通过 cqlsh 连接，但我想从外部连接，我尝试通过 NodePort 公开服务并测试与 datastax studio (192.168.99.100:32554) 的连接但是没有成功，后来我想在spring boot中连接，我假设我必须使用svc名称或节点ip。

All host(s) tried for query failed (tried: /192.168.99.100:32554 (com.datastax.driver.core.exceptions.TransportException: [/192.168.99.100:32554] Cannot connect))

[cassandra-0]/etc/cassandra/cassandra.yaml

rpc_port: 9160
broadcast_rpc_address: 172.17.0.5
listen_address: 172.17.0.5
# listen_interface: eth0
start_rpc: true
rpc_address: 0.0.0.0
# rpc_interface: eth1
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "cassandra-0.cassandra.default.svc.cluster.local"

这是 svc 和 pods 的 minikube 输出

$ kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443
KubeDNS is running at https://192.168.99.100:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
cassandra    NodePort    10.102.236.158   <none>        9042:32554/TCP   20m
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP          22h

$ kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
cassandra-0   1/1     Running   0          20m   172.17.0.4   minikube   <none>           <none>
cassandra-1   1/1     Running   0          19m   172.17.0.5   minikube   <none>           <none>
cassandra-2   1/1     Running   1          19m   172.17.0.6   minikube   <none>           <none>

$ kubectl describe service cassandra
Name:                     cassandra
Namespace:                default
Labels:                   app=cassandra
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"cassandra"},"name":"cassandra","namespace":"default"},"s...
Selector:                 app=cassandra
Type:                     NodePort
IP:                       10.102.236.158
Port:                     <unset>  9042/TCP
TargetPort:               9042/TCP
NodePort:                 <unset>  32554/TCP
Endpoints:                172.17.0.4:9042,172.17.0.5:9042,172.17.0.6:9042
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

$ kubectl exec -it cassandra-0 -- nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
UN  172.17.0.5  104.72 KiB  256          68.1%             680bfcb9-b374-40a6-ba1d-4bf7ee80a57b  rack1
UN  172.17.0.4  69.9 KiB   256          66.5%             022009f8-112c-46c9-844b-ef062bac35aa  rack1
UN  172.17.0.6  125.31 KiB  256          65.4%             48ae76fe-b37c-45c7-84f9-3e6207da4818  rack1

$ kubectl exec -it cassandra-0 -- cqlsh 
Connected to K8Demo at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh> 

cassandra-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  type: NodePort
  ports:
  - port: 9042
  selector:
    app: cassandra

cassandra-statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  serviceName: cassandra
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      terminationGracePeriodSeconds: 1800
      containers:
      - name: cassandra
        image: cassandra:3.11
        ports:
        - containerPort: 7000
          name: intra-node
        - containerPort: 7001
          name: tls-intra-node
        - containerPort: 7199
          name: jmx
        - containerPort: 9042
          name: cql
        resources:
          limits:
            cpu: "500m"
            memory: 1Gi
          requests:
            cpu: "500m"
            memory: 1Gi
        securityContext:
          capabilities:
            add:
              - IPC_LOCK
        lifecycle:
          preStop:
            exec:
              command: 
              - /bin/sh
              - -c
              - nodetool drain
        env:
          - name: MAX_HEAP_SIZE
            value: 512M
          - name: HEAP_NEWSIZE
            value: 100M
          - name: CASSANDRA_SEEDS
            value: "cassandra-0.cassandra.default.svc.cluster.local"
          - name: CASSANDRA_CLUSTER_NAME
            value: "K8Demo"
          - name: CASSANDRA_DC
            value: "DC1-K8Demo"
          - name: CASSANDRA_RACK
            value: "Rack1-K8Demo"
          - name: CASSANDRA_START_RPC
            value: "true"
          - name: CASSANDRA_RPC_ADDRESS
            value: "0.0.0.0"
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
        # These volume mounts are persistent. They are like inline claims,
        # but not exactly because the names need to match exactly one of
        # the stateful pod volumes.
        volumeMounts:
        - name: cassandra-data
          mountPath: /var/lib/cassandra
  # These are converted to volume claims by the controller
  # and mounted at the paths mentioned above.
  # do not use these in production until ssd GCEPersistentDisk or other ssd pd
  volumeClaimTemplates:
  - metadata:
      name: cassandra-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: fast
      resources:
        requests:
          storage: 1Gi
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: fast
provisioner: k8s.io/minikube-hostpath
parameters:
  type: pd-standard

Answer 1

仅适用于遇到此问题的任何人:
阅读后docs在 datastax 上，我意识到 DataStax Studio 旨在与 DataStax Enterprise、本地开发和 cassanda 社区版一起使用，我正在使用 DataStax DevCenter 并且它可以工作。

对于 Spring 启动(在 minikube 上运行的 Cassandra 集群):

spring.data.cassandra.keyspacename=mykeyspacename
spring.data.cassandra.contactpoints=cassandra-0.cassandra.default.svc.cluster.local
spring.data.cassandra.port=9042
spring.data.cassandra.schemaaction=create_if_not_exists

对于 DataStax DevCenter(在 minikube 上运行的 Cassandra 集群):

ContactHost = 192.168.99.100
NativeProtocolPort: 300042

更新了 cassandra 服务

# ------------------- Cassandra Service ------------------- #
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  type: NodePort
  ports:
  - port: 9042
    nodePort: 30042
  selector:
    app: cassandra