elasticsearch - 如何使用重新索引，摄取管道和处理器来构建反向的1:n Elasticsearch索引

Question

我已经开始尝试使用Elasticsearch接收管道和处理器，这是构建我可以描述为“反向索引”的一种可能更快的方法。

这是我要执行的操作:我有一个文档索引。每个文档都类似于以下内容:

{
  "id": "DOC1",
  "title": "Quiz no. 1",
  "questions": [
    {
      "question": "Who was the first person to walk on the Moon?",
      "choices": [
        { "answer": "Michael Jackson", "correct": false },
        { "answer": "Neil Armstrong", "correct": true }
      ]
    },
    {
      "question": "Who wrote the Macbeth?",
      "choices": [
        { "answer": "William Shakespeare", "correct": true },
        { "answer": "Dante Alighieri", "correct": false },
        { "answer": "Arthur Conan Doyle", "correct": false }
      ]
    }
  ]
}

我试图了解是否存在重新索引，管道和处理器的神奇组合，这些组合可以使我自动构建 问题索引。这是该索引的示例:

[
  {
    "question_id": "<randomly-generated-value-1>",
    "document_id": "DOC1",
    "question": "Who was the first person to walk on the Moon?",
    "choices": [
      { "answer": "Michael Jackson", "correct": false },
      { "answer": "Neil Armstrong", "correct": true }
    ]
  },
  {
    "question_id": "<randomly-generated-value-2>",
    "document_id": "DOC1",
    "question": "Who wrote the Macbeth?",
    "choices": [
      { "answer": "William Shakespeare", "correct": true },
      { "answer": "Dante Alighieri", "correct": false },
      { "answer": "Arthur Conan Doyle", "correct": false }
    ]
  }
]

在Elasticsearch文档中，提到可以使用特定管道执行 REINDEX。查找 simulate pipeline docs，我正在尝试一些处理器，包括 foreach 一个，但是我无法理解管道中生成的文档与原始索引是否仍为1:1或1个源文档是否可以生成多个目标文件(这是我需要的)。

这是我正在尝试的模拟管道:

{
  "pipeline": {
    "description": "Inverts the documents index into a questions index",
    "processors": [
      {
        "rename": {
          "field": "id",
          "target_field": "document_id",
          "ignore_missing": false
        }
      },
      {
        "foreach": {
          "field": "questions",
          "processor": {
            "rename": {
              "field": "_ingest._value.question",
              "target_field": "question"
            }
          }
        }
      },
      {
        "foreach": {
          "field": "questions",
          "processor": {
            "rename": {
              "field": "_ingest._value.choices",
              "target_field": "choices"
            }
          }
        }
      },
      {
        "remove": {
          "field": "questions"
        }
      }
    ]
  }
}

这是 几乎的工作方式。这种方法的问题在于，只有一个结果文档与第一个问题相对应。在模拟管道的输出中不存在第二个问题，
因此，我怀疑处理器的管道是否可以输出读取1个源文档的多个目标文档，还是我们被迫维持1:1的关系。

Answer 1

This answer似乎暗示我试图实现的目标是不可能的。