elasticsearch - 即使有0个未分配的分片，Elasticsearch集群的状态仍为黄色

Question

我们有一种情况，未分配的分片为0，但Elasticsearch群集状态仍然为黄色。另一方面，某些索引数据正在被复制。我们看到相同的数据输入进行了3至4次以上。以下是集群的当前状态

{
  "cluster_name": "elasticsearch",
  "status": "yellow",
  "timed_out": false,
  "number_of_nodes": 5,
  "number_of_data_nodes": 2,
  "active_primary_shards": 3888,
  "active_shards": 7775,
  "relocating_shards": 0,
  "initializing_shards": 1,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 99.98713991769547
}

Elasticsearch日志

[2019-11-22T02:42:19,929][INFO ][o.e.m.j.JvmGcMonitorService] [elasticsearch-master-b94556bc6-q62jc] [gc][1605322] overhead, spent [347ms] collecting in the last [1.1s]
[2019-11-22T02:42:20,424][WARN ][o.e.x.m.MonitoringService] [elasticsearch-master-b94556bc6-q62jc] monitoring execution failed
org.elasticsearch.xpack.monitoring.exporter.ExportException: Exception when closing export bulk
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$1$1.<init>(ExportBulk.java:95) ~[x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$1.onFailure(ExportBulk.java:93) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound$1.onResponse(ExportBulk.java:206) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound$1.onResponse(ExportBulk.java:200) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:96) [x-pack-core-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$0(ExportBulk.java:164) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:68) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$doFlush$1(LocalBulk.java:115) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:68) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:50) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:91) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:87) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:76) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:484) [elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:74) [x-pack-core-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.doFlush(LocalBulk.java:108) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flush(ExportBulk.java:60) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$1(ExportBulk.java:154) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:81) [x-pack-core-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.doFlush(ExportBulk.java:170) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flushAndClose(ExportBulk.java:84) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.close(ExportBulk.java:74) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:196) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:258) [x-pack-monitoring-6.4.3.jar:6.4.3]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.4.3.jar:6.4.3]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) [?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.4.3.jar:6.4.3]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
    at java.lang.Thread.run(Thread.java:844) [?:?]
Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks
    at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$0(ExportBulk.java:156) [x-pack-monitoring-6.4.3.jar:6.4.3]
    ... 29 more
Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulk [default_local]
    ... 28 more
Caused by: java.lang.IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node.
    at org.elasticsearch.action.ingest.IngestActionForwarder.randomIngestNode(IngestActionForwarder.java:58) ~[elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.ingest.IngestActionForwarder.forwardIngestRequest(IngestActionForwarder.java:51) ~[elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:136) ~[elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:85) ~[elasticsearch-6.4.3.jar:6.4.3]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.4.3.jar:6.4.3]
    ... 23 more

这可能是什么原因？我们该如何解决？

Answer 1

您可以使用Cluster allocation explain API让Elasticsearch“解释”为什么它无法分配碎片:GET /_cluster/allocation/explain