node.js - 使用单个 firehose 将数据索引到多个索引

Question

我试图了解是否可以使用 kinesis firehose 将数据索引到 elasticsearch 到多个索引，方法是将索引名称和类型传递给它(类似于 elasticsaerch bulk api)

AWS 文档 ( here) 说:

.... Also, the rest.action.multi.allow_explicit_index option for your Elasticsearch cluster must be set to true (default) in order to take bulk requests with an explicit index that is set per record. For more information, see Amazon ES Configure Advanced Options in the Amazon Elasticsearch Service Developer Guide.

但我找不到任何关于如何操作的文档，而且 firehose 配置需要索引名称。

这次尝试没有成功:

 const data = [
    {
        "value": "1",
        "_index": "index1",
        "_type": "type1"
    }];

const params = {
    DeliveryStreamName: 'XXX', /* required */
    Records: [/* required */
        {
            Data: JSON.stringify(data[0]) //new Buffer('...') || 'STRING_VALUE' /* Strings will be Base-64 encoded on your behalf */ /* required */
        }
    ]
};
firehose.putRecordBatch(params, (err, data) => {
    if (err) console.log(err, err.stack); // an error occurred
    else console.log(data);           // successful response
});

我们将不胜感激

Answer 1

问了很久，但仍然......根据 Kinesis 常见问题解答 (https://aws.amazon.com/kinesis/data-firehose/faqs/):

A single delivery stream can only deliver data to one Amazon Elasticsearch Service domain and one index currently. If you want to have data delivered to multiple Amazon Elasticsearch domains or indexes, you can create multiple delivery streams.