kubernetes - 如何在 istio kubernetes 集群中拒绝默认但允许 HTTP 和 TCP 流量？

Question

我有一个启用了 istio 注入(inject)并定义了 cockroach db 状态集的集群:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cockroachdb-serviceaccount
---
apiVersion: v1
kind: Service
metadata:
  # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
  # automatically load balance connections to the different database pods.
  name: cockroachdb-public
  labels:
    app: cockroachdb
spec:
  ports:
  # The main port, served by gRPC, serves Postgres-flavor SQL, internode
  # traffic and the cli.
  - port: 26257
    targetPort: 26257
    name: tcp
  # The secondary port serves the UI as well as health and debug endpoints.
  - port: 8080
    targetPort: 8080
    name: http
  selector:
    app: cockroachdb
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cockroachdb-statefulset
  labels:
    version: v20.1.2
spec:
  serviceName: cockroachdb
  replicas: 3
  selector:
    matchLabels:
      app: cockroachdb
  template:
    metadata:
      labels:
        app: cockroachdb
        version: v20.1.2
    spec:
      serviceAccountName: cockroachdb-serviceaccount
      containers:
      - name: cockroachdb
        image: cockroachdb/cockroach:v20.1.2
        ports:
        - containerPort: 26257
          name: tcp
        - containerPort: 8080
          name: http
        volumeMounts:
        - name: datadir
          mountPath: /cockroach/cockroach-data
        env:
        - name: COCKROACH_CHANNEL
          value: kubernetes-insecure
        command:
          - "/bin/bash"
          - "-ecx"
          # The use of qualified `hostname -f` is crucial:
          # Other nodes aren't able to look up the unqualified hostname.
          - "exec /cockroach/cockroach start --logtostderr --insecure --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-statefulset-0.cockroachdb,cockroachdb-statefulset-1.cockroachdb,cockroachdb-statefulset-2.cockroachdb --cache 25% --max-sql-memory 25%"
      # No pre-stop hook is required, a SIGTERM plus some time is all that's
      # needed for graceful shutdown of a node.
      terminationGracePeriodSeconds: 5
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: 4Gi
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: cockroachdb-public
spec:
  host: cockroachdb-public
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cockroachdb-public
spec:
  hosts:
  - cockroachdb-public
  http:
  - match:
    - port: 8080
    route:
    - destination:
        host: cockroachdb-public
        port:
          number: 8080
  tcp:
  - match:
    - port: 26257
    route:
    - destination:
        host: cockroachdb-public
        port:
          number: 26257

以及访问它的服务:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: downstream-serviceaccount
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: downstream-deployment-v1
  labels:
    app: downstream
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: downstream
      version: v1
  template:
    metadata:
      labels:
        app: downstream
        version: v1
    spec:
      serviceAccountName: downstream-serviceaccount
      containers:
      - name: downstream
        image: downstream:0.1
        ports:
        - containerPort: 80
        env:
          - name: DATABASE_URL
            value: postgres://roach@cockroachdb-public:26257/roach?sslmode=disable
---
apiVersion: v1
kind: Service
metadata:
  name: downstream-service
  labels:
    app: downstream
spec:
  type: ClusterIP
  selector:
    app: downstream
  ports:
    - port: 80
      targetPort: 80
      name: http
      protocol: TCP
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: downstream-service
spec:
  host: downstream-service
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: downstream-service
spec:
  hosts:
  - downstream-service
  http:
  - name: "downstream-service-routes"
    match:
    - port: 80
    route:
    - destination:
        host: downstream-service
        port:
          number: 80

现在我想限制对 cockroach db 的访问权限为 downstream-service以及 cockroachdb 本身(因为节点之间需要相互通信)。
我正在尝试通过以下方式限制流量:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: default-deny-all
 namespace: default
spec:
  {}
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-downstream
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/downstream-serviceaccount"]
  - to:
    - operation:
       ports: ["26257"]
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-cockroachdb
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/cockroachdb-serviceaccount"]
  - to:
    - operation:
       ports: ["26257"]

但似乎什么也没做。我仍然可以例如访问 cockroachdb-public:8080来自下游服务的集群 HTTP UI。
现在，当我添加以下内容时:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: default-deny-all-to-cockroachdb
 namespace: default
spec:
  selector:
    matchLabels:
      app: cockroachdb
  action: DENY
  rules:
  - to:
    - operation:
        ports: ["26257"]

然后所有流量都被阻塞(包括cockroachdb节点之间的流量)。
我在这里做错了什么？

Answer 1

你和几天前的一个人有同样的问题。在您的授权策略中，您有两个策略:

服务帐号downstream-serviceaccount (以及 cockroachdb-serviceaccount 用于其他授权策略)来自默认命名空间可以访问带有标签的服务 app: cockroachdb在default 上的任何端口上命名空间。

来自任何命名空间的任何服务帐户都可以访问带有标签 app: cockroachdb 的服务，在端口 26257 上。

为了使其成为 AND，您将执行以下操作:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-cockroachdb
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/cockroachdb-serviceaccount"]
    to:                  <- remove the dash from here
    - operation:
       ports: ["26257"]

与其他 AuthorizationPolicy 相同目的。另请注意，您不需要显式创建 拒绝 政策。当您创建 允许 一，它会自动否认其他一切。