elasticsearch - filebeat忽略多个探矿者中的logiles

Question

我尝试使用多位探矿者配置文件拍。 Filebeat注册所有探矿者，但忽略来自appA的localhost日志文件和来自appB的日志文件

我的filebeat.yml:

    filebeat.prospectors:
    - type: log
      paths:
        - /vol1/appA_instance01/logs/wrapper_*.log
        - /vol1/appA_instance02/logs/wrapper_*.log
      fields: 
         log_type: "appAlogs"
         environment: "stage1"
      exclude_files: [".gz$"]
    - type: log
      paths:
        - /vol1/appA_instance01/logs/localhost.*.log
        - /vol1/appA_instance02/logs/localhost.*.log
      fields: 
         log_type: "localhostlogs"
         environment: "stage1"
      exclude_files: [".gz$"]
    - type: log
      paths:
        - /vol1/appB_instance01/logs/*.log
        - /vol1/appB_instance02/logs/*.log
      fields: 
         log_type: "appBlogs"
      environment: "stage1"
      exclude_files: [".gz$"]
   output.logstash:
   hosts: ["<HOST>:5044"]

filebeat日志文件:

2017-11-15T17:32:56+01:00 INFO Home path: [/usr/share/filebeat] Config   path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2017-11-15T17:32:56+01:00 INFO Setup Beat: filebeat; Version: 5.6.3
2017-11-15T17:32:56+01:00 INFO Max Retries set to: 3
2017-11-15T17:32:56+01:00 INFO Activated logstash as output plugin.
2017-11-15T17:32:56+01:00 INFO Publisher name: host
2017-11-15T17:32:56+01:00 INFO Flush Interval set to: 1s
2017-11-15T17:32:56+01:00 INFO Max Bulk Size set to: 2048
2017-11-15T17:32:56+01:00 INFO filebeat start running.
2017-11-15T17:32:56+01:00 INFO Registry file set to: /var/lib/filebeat/registry
2017-11-15T17:32:56+01:00 INFO Loading registrar data from /var/lib  /filebeat/registry
2017-11-15T17:32:56+01:00 INFO States Loaded from registrar: 222
2017-11-15T17:32:56+01:00 INFO Loading Prospectors: 3
2017-11-15T17:32:56+01:00 INFO Starting Registrar
2017-11-15T17:32:56+01:00 INFO Start sending events to output
2017-11-15T17:32:56+01:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 40
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 12115431240338587115 
2017-11-15T17:32:56+01:00 INFO Harvester started for file: /vol1/appA_instance01/logs/wrapper_20171115.log
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 182
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 18163435272915459714 
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 0
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 16959079668827945694 
2017-11-15T17:32:56+01:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 3
2017-11-15T17:33:06+01:00 INFO Harvester started for file: /vol1/appA_instance02/logs/wrapper_20171115.log

filebeat忽略logile的原因是什么？

/vol1/appA_instance01/logs/localhost.*.log

/vol1/appA_instance02/logs/localhost.*.log

/vol1/appB_instance01/logs/*.log

/vol1/appB_instance02/logs/*.log

问候niesel

Answer 1

随附的日志显示所有三个探矿者已启动，并且注册表文件似乎具有状态。您确定Filebeat之前没有读取过忽略的日志文件吗？它会从那些日志文件中读取新行吗？

Filebeat不会重新读取日志文件。因此，这些文件以前可能已被读取。