磁盘满后 Elasticsearch 不工作

Question

我腾出空间，重新启动服务(不好)然后重新启动，但我仍然在 elasticsearch.log 中得到这个:

[2015-02-16 13:35:19,625][WARN ][cluster.action.shard     ] [Server] [logstash-2015.02.16][1] sending failed shard for [logstash-2015.02.16][1], node[PFamB-ZJS7CwSdyyAcP_8A], [P], s[INITIALIZING], indexUUID [tZ3I9HZ6TDaZSicIuGWRWQ], 
    reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2015.02.16][1] 
    failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchIllegalArgumentException[No version type match [83]]; ]]
[2015-02-16 13:35:19,625][WARN ][cluster.action.shard     ] [Server] [logstash-2015.02.16][1] received shard failed for [logstash-2015.02.16][1], node[PFamB-ZJS7CwSdyyAcP_8A], [P], s[INITIALIZING], indexUUID [tZ3I9HZ6TDaZSicIuGWRWQ], 
    reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2015.02.16][1] 
    failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchIllegalArgumentException[No version type match [83]]; ]]
[2015-02-16 13:35:43,570][DEBUG][action.index             ] [Server] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m]
[2015-02-16 13:36:10,757][DEBUG][action.index             ] [Server] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m]

我该怎么办？

Answer 1

Bigfoot 的解决方案似乎是唯一可行的。

虽然观察到的堆栈跟踪似乎类似于:https://github.com/elastic/elasticsearch/issues/12055

此拉取请求应该可以解决问题:https://github.com/elastic/elasticsearch/pull/9797

但是升级到 v1.5.0 也不能达到目的。

因此唯一可行的是:

find  /var/lib/elasticsearch/elasticsearch/nodes/ -name "*.recovering"

并删除所有恢复文件。当然这有副作用。