ElasticSearch ILM 不删除索引

Question

我在 fluentd.* 上设置了一个简单的 ILM 策略在(用于测试 - )一小段时间后要删除的索引。

工业光魔:

PUT _ilm/policy/fluentd
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "1gb"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "4d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

索引模板:

PUT _template/fluentd
{
  "order": 0,
  "index_patterns": [
    "fluentd.*"
  ],
  "settings": {
    "index": {
      "lifecycle": {
        "name": "fluentd"
      }
    }
  },
  "aliases": {
    "fluent": {}
  }
}

通过这些设置，我预计 ES 会删除超过 5-6 天的索引，但 ES 中仍然存在 3 周前的索引。目前，它表示有 108 个与此 ILM 政策相关的指数。

它实际上在做什么，似乎它根本没有做任何事情......如何在 x 天后删除索引？

我首先尝试使用“索引模板”，但它没用，它不会将设置应用于每个索引(也许是，但仅限于创建？？？？？？)。

然后我手动将 ILM 放在索引上(另一个错误:您无法选择所有索引并点击“添加 ILM 策略”-您需要逐个添加策略)，这需要我单击大约 600 次。

现在的问题是，我定义了“热”阶段但它没有触发(它有问题？) - 因为热阶段没有触发(我设置为“索引创建后 1 天后翻转”) - 删除相也没有。当我删除热阶段并仅使用删除将 ILM 再次应用于索引时 - 它起作用了!但是添加和删除所有这些都是有问题的，我收到了糟糕的消息，这里和那里出了点问题。

我不明白为什么当我更改 ILM 策略中的某些内容时，我必须删除 ILM 并将其重新应用于每个索引。这 1000% 不方便。

ES 真的需要做一些工作，它仍然太测试版，我收到了很多状态代码 500，尽管我直接在 Elastic Cloud 上使用最新版本。

Answer 1

With these settings, I expected ES to delete indices older than 5-6 days, but there are still indices from 3 weeks ago in ES. Currently, it says there are 108 linked indices to this ILM policy.

根据您的设置，删除阶段从 开始的第 4 天开始。翻车 .如果您想在“索引创建”的第 4 天开始删除阶段，您需要从热阶段删除翻转操作:

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "4d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

I tried first to use the "index template" but it's useless, it does not apply settings to each index (maybe yes but only on creation????).

是的，它适用于索引创建。

Then I put the ILM on the index by hand (another bug: you can't select all index and hit "add ILM policy" - you need to add the policy one by one) which required me to click about 600 times.

Kibana 不允许您将 ILM 策略应用于所有索引，但是 elasticsearch API 允许!
只需打开 kibana 开发工具并运行以下请求:

PUT fluentd.*/_settings
{
  "index": {
    "lifecycle": {
      "name": "fluentd"
    }
  }
}

Now the problem was, I had "hot" phase defined but it didn't trigger (it's buggy?) - because the hot phase didn't trigger (i set to to "rollover after 1 day after index creation") - the delete phase didn't either. When I removed the hot phase and applied the ILM to index again with only delete - it worked! but adding and removing all this is buggy, I get Ooops, something went wrong errors here and there.

如果未触发翻转阶段，则 ILM 无法进行。

I don't understand why I have to remove the ILM and reapply it to each index when I change something in the ILM policy. It's 1000% inconvenient.

因为 ILM 定义缓存在每个索引上。
请参阅文档: https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-index-lifecycle.html#ilm-phase-execution