logstash - Filebeat 到 Logstash -InvalidFrameProtocolException

Question

我正在尝试将数据从 filebeat 加载到 logstash。在加载时，在运行命令时->

bin/logstash -f first-pipeline.conf --config.reload.automatic 

，遇到以下错误:

[2018-06-05T11:30:43,987][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2018-06-05T11:30:44,047][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x969dfe run>"}
[2018-06-05T11:30:44,083][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2018-06-05T11:30:44,112][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-06-05T11:32:05,045][INFO ][org.logstash.beats.BeatsHandler] [local: 0:0:0:0:0:0:0:1:5044, remote: 0:0:0:0:0:0:0:1:31903] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 69

第一个 pipeline.conf 文件是:

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
beats {
        port => "5044"
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
stdout { codec => rubydebug }
}

Filebeat.yml 文件:

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - \C:\PATH-TO-DOC\elasticDoc\logstash-tutorial-dataset.log
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

logstash-tutorial-dataset.log 的示例数据集:

83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"

这个错误的原因是什么？之前已经有人问过这个问题，但没有回复。还请让我知道我可以在哪里进一步完善我在 logstash 和 filebeat 中的概念。我是初学者。

Answer 1

问题出在我在 filebeat.yml 中的文件名上。不需要扩展。同样在 first-pipeline.conf 文件中，我删除了编解码器并将我的日志直接发送到 Elasticsearch ，它开始为我工作。