elasticsearch - Filebeat Kubernetes 处理器和过滤

Question

我正在尝试使用 Filebeat 将我的 K8s pod 日志发送到 Elasticsearch。

我正在按照此处的在线指南进行操作:https://www.elastic.co/guide/en/beats/filebeat/6.0/running-on-kubernetes.html

一切都按预期工作，但我想从系统 pod 中过滤掉事件。我更新后的配置如下:

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-prospectors
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    - type: log
      paths:
        - /var/lib/docker/containers/*/*.log
  multiline.pattern: '^\s'
  multiline.match: after
  json.message_key: log
  json.keys_under_root: true
  processors:
    - add_kubernetes_metadata:
        in_cluster: true
        namespace: ${POD_NAMESPACE}
    - drop_event.when.regexp:
        or:
          kubernetes.pod.name: "weave-net.*"
          kubernetes.pod.name: "external-dns.*"
          kubernetes.pod.name: "nginx-ingress-controller.*"
          kubernetes.pod.name: "filebeat.*"

我试图忽略 weave-net、external-dns、ingress-controller 和 filebeat 事件通过:

- drop_event.when.regexp:
    or:
      kubernetes.pod.name: "weave-net.*"
      kubernetes.pod.name: "external-dns.*"
      kubernetes.pod.name: "nginx-ingress-controller.*"
      kubernetes.pod.name: "filebeat.*"

但是它们会继续到达 Elasticsearch。

Answer 1

条件需要是一个列表:

- drop_event.when.regexp:
    or:
      - kubernetes.pod.name: "weave-net.*"
      - kubernetes.pod.name: "external-dns.*"
      - kubernetes.pod.name: "nginx-ingress-controller.*"
      - kubernetes.pod.name: "filebeat.*"

我不确定您的参数顺序是否有效。我的一个工作示例如下所示:

- drop_event:
    when:
      or:
        # Exclude traces from Zipkin
        - contains.path: "/api/v"
        # Exclude Jolokia calls
        - contains.path: "/jolokia/?"
        # Exclude pinging metrics
        - equals.path: "/metrics"
        # Exclude pinging health
        - equals.path: "/health"