elasticsearch - SearchPhaseExecutionException[无法执行阶段[查询]，所有分片都失败]

Question

最近我们的服务器在没有正确关闭 Elastic Search/Kibana 的情况下重新启动。重新启动后，两个应用程序都在运行，但不再创建索引。我在 Debug模式下检查了 logstash 设置，它正在向 Elastic Search 发送数据。

现在我创建的所有窗口都报告这个错误:

Oops! SearchPhaseExecutionException[Failed to execute phase [query], all shards failed]

我尝试重新启动 Elastic Search/Kibana，并清除了一些索引。我进行了很多搜索，但无法正确解决此问题。

如图所示，当前集群健康状态为红色。

关于如何排除故障的任何帮助都被点赞。谢谢

编辑:

[2015-05-06 00:00:01,561][WARN ][cluster.action.shard     ] [Indech] [logstash-2015.03.16][1] sending failed shard for [logstash-2015.03.16][1], node[fdSgUPDbQB2B3NQqX7MdMQ], [P], s[INITIALIZING], indexUUID [aBcfbqnNR4-AGEdIR8dVdg], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2015.03.16][1] failed to recover shard]; nested: ElasticsearchIllegalArgumentException[No version type match [101]]; ]]
[2015-05-06 00:00:01,561][WARN ][cluster.action.shard     ] [Indech] [logstash-2015.03.16][1] received shard failed for [logstash-2015.03.16][1], node[fdSgUPDbQB2B3NQqX7MdMQ], [P], s[INITIALIZING], indexUUID [aBcfbqnNR4-AGEdIR8dVdg], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2015.03.16][1] failed to recover shard]; nested: ElasticsearchIllegalArgumentException[No version type match [101]]; ]]
[2015-05-06 00:00:02,591][WARN ][indices.cluster          ] [Indech] [logstash-2015.04.21][4] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [logstash-2015.04.21][4] failed to recover shard
    at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:269)
    at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: No version type match [52]
    at org.elasticsearch.index.VersionType.fromValue(VersionType.java:307)
    at org.elasticsearch.index.translog.Translog$Create.readFrom(Translog.java:364)
    at org.elasticsearch.index.translog.TranslogStreams.readTranslogOperation(TranslogStreams.java:52)
    at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:241)

logs 中我关心的是:

[2015-05-06 15:13:48,059][DEBUG][action.search.type       ] All shards failed for phase: [query]



{
  "cluster_name" : "elasticsearch",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 8,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 120,
  "active_shards" : 120,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 310
}

Answer 1

您有许多损坏的 translog 文件，您需要将其删除。您可以在 data/{clustername}/nodes/0/indices/logstash-2015.04.21/4/translog 中找到它，另一个在 data/{clustername}/nodes/0/中索引/logstash-2015.03.16/1/translog。也许还有其他人，但这是我从您提供的代码片段中可以看出的。当然，会丢失 translog 文件中的内容。

如果索引不再有索引文件(只有 _state 文件夹存在于 data/{clustername}/nodes/0/indices/[index_name] 下)这意味着该索引中不再有数据，此时您可以删除该索引。如果仍然需要，则需要重新索引该数据。如果您决定删除索引，则需要关闭节点并删除 data/{clustername}/nodes/0/indices 下的索引文件夹，就像您提到的那个(空的，仅包含_state 文件夹)。