elasticsearch - 当Elasticsearch死/无法写入磁盘时的Logstash可用性

Question

我有rsyslog通过TCP将日志转发到logstash。如果logstash不可用，则rsyslog将建立队列。

如果有logstash可用，但elasticsearch已死或由于某种原因而无法写入文件系统。

是否有logstash拒绝其他TCP消息的方法。

谢谢

Answer 1

根据life of an event description:

An output can fail or have problems because of some downstream cause, such as full disk, permissions problems, temporary network failures, or service outages. Most outputs should keep retrying to ship any events that were involved in the failure.

If an output is failing, the output thread will wait until this output is healthy again and able to successfully send the message. Therefore, the output queue will stop being read from by this output and will eventually fill up with events and block new events from being written to this queue.

A full output queue means filters will block trying to write to the output queue. Because filters will be stuck, blocked writing to the output queue, they will stop reading from the filter queue which will eventually cause the filter queue (input -> filter) to fill up.

A full filter queue will cause inputs to block when writing to the filters. This will cause each input to block, causing each input to stop processing new data from wherever that input is getting new events.

这意味着，如果elasticsearch输出开始失败，则整个管道将被阻塞，这正是您所需要的。您看到不同的地方了吗？