amazon-web-services - logsatsh 和 elasticsearch 的版本不兼容问题？

Question

我将 Logstash 1.4.1 与 Elasticsearch(安装为 EC2 集群)1.1.1 和 Elasticsearch AWS 插件 2.1.1 一起使用。

为了尝试 Logstash 是否正确地与 Elasticsearch 对话，我使用 -

bin/logstash -e 'input { stdin { } } output { elasticsearch { host => <ES_cluster_IP> } }'

我得到 -

log4j, [2014-06-10T18:30:17.622]  WARN: org.elasticsearch.discovery: [logstash-ip-xxxxxxxx-20308-2010] waited for 30s and no initial state was set by the discovery
Exception in thread ">output" org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
    at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
    at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(java/lang/Thread.java:744)

但是当我使用 -

bin/logstash -e 'input { stdin { } } output { elasticsearch_http { host => <ES_cluster_IP> } }'

它适用于以下警告 -

Using milestone 2 output plugin 'elasticsearch_http'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}

我不明白为什么我不能使用 elasticsearch而不是 elasticsearch_http即使版本兼容。

Answer 1

我会小心设置 protocol “http”、“transport”和“node”之一的选项。关于此的文档是矛盾的 - 一方面它声明它是可选的并且没有默认值，而最后它说默认值因代码集而异:

The ‘node’ protocol will connect to the cluster as a normal Elasticsearch node (but will not store data). This allows you to use things like multicast discovery. If you use the node protocol, you must permit bidirectional communication on the port 9300 (or whichever port you have configured).

The ‘transport’ protocol will connect to the host you specify and will not show up as a ‘node’ in the Elasticsearch cluster. This is useful in situations where you cannot permit connections outbound from the Elasticsearch cluster to this Logstash server.

The ‘http’ protocol will use the Elasticsearch REST/HTTP interface to talk to elasticsearch.

All protocols will use bulk requests when talking to Elasticsearch.

The default protocol setting under java/jruby is “node”. The default protocol on non-java rubies is “http”

这里的问题是协议(protocol)设置对您连接到 Elasticsearch 的方式以及它的运行方式有一些非常重要的影响，但是当您不设置协议(protocol)时，尚不清楚它会做什么。最好选择一个并设置它 -

http://logstash.net/docs/1.4.1/outputs/elasticsearch#protocol