json - logstash_forwarder已连接到Lostash服务器IP，但从未收到事件

Question

我在同一服务器上将elasticsearch，logstash，kibana，ngix和logstash-forwarder安装到集中式日志。日志文件(allapp.json)是带有日志条目的json文件，如下所示:

"{\"timestamp\":\"2015-08-30 19:42:26.724\",\"MAC_Address\":\"A8:7C:01:CB:2D:09\",\"DeviceID\":\"96f389972de989d1\",\"RunningApp\":\"null{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}\",\"PhoneName\":\"samsung\",\"IP\":\"192.168.1.101\"}"

我的logstash.conf是:

 input {
lumberjack {
port => 5002
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}   
udp {
type   => "json"
port   => 5001 
}
  } 
 filter {
  json {
  "source" => "message"
  }
}
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}

我的logstash-forwarder.conf(与安装logstash的系统相同)是:

{
"network":{ 
"servers": [ "192.168.1.102:5002" ],
"timeout": 15,
"ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt" },
  "files": [
       {
           "paths":[ "/var/log/app-log/allapp.json" ],
           "fields": { "type": "json" }
       }
       ]
}

我的elasticsearch.yml是:

network.host: localhost

当我在终端中输入 tail -f /var/log/logstash-forwarder/logstash-forwarder.err时，我得到了:

2015/09/04 11:33:05.282495 Waiting for 1 prospectors to initialise
2015/09/04 11:33:05.282544 Launching harvester on new file: /var/log/app-log/allapp.json
2015/09/04 11:33:05.282591 harvest: "/var/log/app-log/allapp.json" (offset snapshot:0)
2015/09/04 11:33:05.283709 All prospectors initialised with 0 states to persist
2015/09/04 11:33:05.283806 Setting trusted CA from file: /etc/pki/tls/certs/logstash-forwarder.crt
2015/09/04 11:33:05.284254 Connecting to [192.168.1.102]:5002 (192.168.1.102) 
2015/09/04 11:33:05.417174 Connected to 192.168.1.102

allapp.json文件已被频繁更新，并在其中添加了新日志，但是在上面我从未看到如下日志:

注册服务商收到1个事件

注册服务商收到23个事件...

另外，我还有另一个具有logstash-forwarder的客户端，可以将其日志发送到kibana，该客户端上的logstash-forwarder可以正常工作，并且来自kibana中显示的日志，但在此客户端上则没有。
kibana中的所有结果如下所示:

    Time                            file    
September 4th 2015, 06:14:00.942    /var/log/suricata/eve.json
September 4th 2015, 06:14:00.942    /var/log/suricata/eve.json
September 4th 2015, 06:14:00.942    /var/log/suricata/eve.json
September 4th 2015, 06:14:00.942    /var/log/suricata/eve.json 

我也想在kibana中查看 /var/log/app-log/allapp.json 中的日志，这是什么问题？为什么没有在基巴纳语中显示它们？为什么一个客户端可以正常工作，但是在同一系统上使用logstash的logstash转发器却不起作用？

Answer 1

您可以在运行Logstash转发器时尝试使用以下选项:

-tail=false -verbose=false 

像这样的东西:

<logstash forwarder> -tail=false -verbose=false  -config=<logstash forwarder config> 

tail=false应该强制完全重新加载文件
verbose=false应该显示所有消息