kubernetes - Istio 授权策略不适用于子网关

Question

我想要达到的目标:阻止所有流向服务的流量，包含在与服务相同的命名空间内处理此流量的代码。
为什么:这是将特定服务“锁定”到特定 IP/CIDR 的第一步
我有一个名为 istio-ingressgateway 的主要入口 GW这适用于服务。

$ kubectl describe gw istio-ingressgateway -n istio-system
Name:         istio-ingressgateway
Namespace:    istio-system
Labels:       operator.istio.io/component=IngressGateways
              operator.istio.io/managed=Reconcile
              operator.istio.io/version=1.5.5
              release=istio
Annotations:  API Version:  networking.istio.io/v1beta1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2020-08-28T15:45:10Z
  Generation:          1
  Resource Version:    95438963
  Self Link:           /apis/networking.istio.io/v1beta1/namespaces/istio-system/gateways/istio-ingressgateway
  UID:                 ae5dd2d0-44a3-4c2b-a7ba-4b29c26fa0b9
Spec:
  Selector:
    App:    istio-ingressgateway
    Istio:  ingressgateway
  Servers:
    Hosts:
      *
    Port:
      Name:      http
      Number:    80
      Protocol:  HTTP
Events:          <none>

我还有另一个“主要”GW，即 K8s 入口 GW 来支持 TLS(我想我会包括这个，尽可能明确)

k describe gw istio-autogenerated-k8s-ingress -n istio-system
Name:         istio-autogenerated-k8s-ingress
Namespace:    istio-system
Labels:       app=istio-ingressgateway
              istio=ingressgateway
              operator.istio.io/component=IngressGateways
              operator.istio.io/managed=Reconcile
              operator.istio.io/version=1.5.5
              release=istio
Annotations:  API Version:  networking.istio.io/v1beta1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2020-08-28T15:45:56Z
  Generation:          2
  Resource Version:    95439499
  Self Link:           /apis/networking.istio.io/v1beta1/namespaces/istio-system/gateways/istio-autogenerated-k8s-ingress
  UID:                 edd46c17-9975-4089-95ff-a2414d40954a
Spec:
  Selector:
    Istio:  ingressgateway
  Servers:
    Hosts:
      *
    Port:
      Name:      http
      Number:    80
      Protocol:  HTTP
    Hosts:
      *
    Port:
      Name:      https-default
      Number:    443
      Protocol:  HTTPS
    Tls:
      Credential Name:     ingress-cert
      Mode:                SIMPLE
      Private Key:         sds
      Server Certificate:  sds
Events:                    <none>

我希望能够在命名空间 x 中创建另一个 GW并具有附加到该 GW 的授权策略。
如果我在 istio-system 中创建授权策略命名空间，然后返回 RBAC: access denied这很棒 - 但这适用于使用主 GW 的所有服务。

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: block-all
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: DENY
  rules:
    - from:
        - source:
            ipBlocks: ["0.0.0.0/0"]

我目前拥有的不起作用。任何指针将不胜感激。以下都是在 x下创建的应用 kubectl apply -f files.yaml -n x 时的命名空间

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  annotations:
      app: x-ingress
  name: x-gw
  labels:
    app: x-ingress
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - x.y.com
      port:
        name: http
        number: 80
        protocol: HTTP
      tls:
        httpsRedirect: true
    - hosts:
        - x.y.com
      port:
        name: https
        number: 443
        protocol: HTTPS
      tls:
        mode: SIMPLE
        privateKey: sds
        serverCertificate: sds
        credentialName: ingress-cert
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: x
  labels:
    app: x
spec:
  hosts:
    - x.y.com
  gateways:
    - x-gw
  http:
    - route:
        - destination:
            host: x
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: x-ingress-policy
spec:
  selector:
    matchLabels:
      app: x-ingress
  action: DENY
  rules:
    - from:
        - source:
            ipBlocks: ["0.0.0.0/0"]
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: x
  labels:
    app: x
spec:
  hosts:
    - x.y.com
  gateways:
    - x-gw
  http:
    - route:
        - destination:
            host: x

以上应该阻止所有到 GW 的流量，因为它匹配 0.0.0.0/0 的 CIDR 范围。
我完全误解了 GW/AuthorizationPolicies 的概念，或者我错过了什么？
编辑
我最终创建了另一个具有 IP 限制块的 GW，因为 AWS 上的经典负载均衡器不支持 IP 转发。

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: istiocontrolplane
spec:
  profile: demo
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
      - name: admin-ingressgateway
        enabled: true
        label:
          istio: admin-ingressgateway
        k8s:
          serviceAnnotations:
            service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-all-admin
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: admin-ingressgateway
  action: ALLOW
  rules:
    - from:
        - source:
            ipBlocks: ["176.252.114.59/32"]

kubectl patch svc istio-ingressgateway -n istio-system -p '{"spec":{"externalTrafficPolicy":"Local"}}'

然后，我在我想要锁定的工作负载中使用了该网关。

Answer 1

据我所知你应该使用 AuthorizationPolicy 3种方式

在入口网关

在命名空间

关于具体服务

我试图让它像你一样在带有注释的特定网关上工作，但我无法让它为我工作。
例如
以下授权策略拒绝对命名空间 x 中的工作负载的所有请求。

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: deny-all
 namespace: x
spec:
  {}

以下授权策略拒绝入口网关上的所有请求。

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway

以下授权策略拒绝对 httpbin 的所有请求在 x 命名空间中。

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-service-x
  namespace: x
spec:
  selector:
    matchLabels:
      app: httpbin

假设您拒绝对 x 命名空间的所有请求，并且只允许对 httpbin 服务的请求。
然后您将使用此 AuthorizationPolicy 拒绝所有请求

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: x
spec:
  {}

而这个 AuthorizationPolicy 只允许获取请求。

apiVersion: "security.istio.io/v1beta1"
kind: "AuthorizationPolicy"
metadata:
  name: "x-viewer"
  namespace: x
spec:
  selector:
    matchLabels:
      app: httpbin
  rules:
  - to:
    - operation:
        methods: ["GET"]

还有一个主要问题，那就是 ipBlocks。有相关 github issue关于那个。
正如@incfly 在这里提到的

I guess the reason why it’s stop working when in non ingress pod is because the sourceIP attribute will not be the real client IP then.

According to https://github.com/istio/istio/issues/22341 7, (not done yet) this aims at providing better support without setting k8s externalTrafficPolicy to local, and supports CIDR range as well.

我已经尝试过来自 istio documentation 的这个例子让它工作，但它对我不起作用，即使我改变了 externalTrafficPolicy .然后是 envoyfilter 的解决方法来自上面的 istio 讨论线程。
@hleal18 here 提供的答案.

Got and example working successfully using EnvoyFilters, specifically with remote_ip condition applied on httbin.

Sharing the manifest for reference.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpbin
  namespace: foo
spec:
  workloadSelector:
    labels:
      app: httpbin
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: "envoy.http_connection_manager"
              subFilter:
                name: "envoy.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.rbac 
          config:
            rules:
              action: ALLOW
              policies:
                "ip-premissions":
                  permissions:
                    - any: true
                  principals:
                    - remote_ip:
                        address_prefix: xxx.xxx.xx.xx
                        prefix_len: 32

我已经在我的测试集群上尝试过上面的特使过滤器，据我所知它正在工作。
看看我做的以下步骤。
1.我已经改变了externalTrafficPolicy

kubectl patch svc istio-ingressgateway -n istio-system -p '{"spec":{"externalTrafficPolicy":"Local"}}'

2.我已经创建了命名空间 x 并启用了 istio-injection 并在此处部署了 httpbin。

kubectl create namespace x
kubectl label namespace x istio-injection=enabled
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.7/samples/httpbin/httpbin.yaml -n x
kubectl apply -f https://github.com/istio/istio/blob/master/samples/httpbin/httpbin-gateway.yaml -n x

3.我已经创建了envoyfilter

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpbin
  namespace: x
spec:
  workloadSelector:
    labels:
      app: httpbin
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: "envoy.http_connection_manager"
              subFilter:
                name: "envoy.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.rbac
          config:
            rules:
              action: ALLOW
              policies:
                "ip-premissions":
                  permissions:
                    - any: true
                  principals:
                    - remote_ip:
                        address_prefix: xx.xx.xx.xx
                        prefix_len: 32

address_prefix 是 CLIENT_IP ，有我用过的命令来获取它。

export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
curl "$INGRESS_HOST":"$INGRESS_PORT"/headers -s -o /dev/null -w "%{http_code}\n"
CLIENT_IP=$(curl "$INGRESS_HOST":"$INGRESS_PORT"/ip -s | grep "origin" | cut -d'"' -f 4) && echo "$CLIENT_IP"

4.我用curl和我的浏览器测试过。

curl "$INGRESS_HOST":"$INGRESS_PORT"/headers -s -o /dev/null -w "%{http_code}\n"
200

如果您还有其他问题，请告诉我，我可能会提供帮助。