elasticsearch - 显示和更改参数threadpool.bulk.queue_size

Question

我正在使用Elasticsearch版本6.7.2，但出现错误429

{
    "error": {
        "root_cause": [
            {
                "type": "remote_transport_exception",
                "reason": "[localhost][localhost:9300][indices:data/write/bulk[s]]"
            }
        ],
        "type": "es_rejected_execution_exception",
        "reason": "rejected execution of processing of [682604930][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[hboauth2-2020.04.20-000113][0]] containing [index {[hboauth2][_doc][yRuEnHEBn6z33bhuCGjZ], source[{\"Action\":\"Service UpdateUserStatus Start\",\"ClientID\":\"\",\"Code\":0,\"Component\":\"UpdateUserStatus\",\"CustomMessage\":\"\",\"Data\":\"null\",\"LogDate\":1587468830939,\"Message\":\"{OK false}\",\"Type\":\"Info\",\"UserID\":\"4c220ab2-282d-466f-a9f5-da0e59b7b803\"}]}], target allocation id: wZFZUPi5TnSYFg-BNonHug, primary term: 1 on EsThreadPoolExecutor[name = localhost/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@744831b0[Running, pool size = 16, active threads = 16, queued tasks = 224, completed tasks = 264136589]]"
    },
    "status": 429
}

我找到了解决问题的方法，但是在我的版本中找不到参数threadpool.bulk.queue_size。
您能帮我找到和更改此参数吗

Answer 1

如official example所示，您可以使用elasticsearch.config中的以下配置更改批量线程池的队列大小，或更新集群设置。

注意，写入线程池用于批量请求，如同一链接中所述:

write For single-document index/delete/update and bulk requests. Thread pool type is fixed with a size of # of available processors, queue_size of 200. The maximum size for this pool is 1 + # of available processors.

thread_pool:
    write:
        size: 30. --> no if threads in your case, its 16 as shown in exception
        queue_size: 400 --> here you can add `queue_size`

有关更改这些设置的说明也提到了

Changing a specific thread pool can be done by setting its type-specific parameters; for example, changing the number of threads in the write thread pool:

编辑: 正如@val在注释部分所指出的那样，添加了有关更改这些默认设置的免责声明。

请了解这些设置的技术性及其影响，因为通常不建议更改这些设置，因此请参见 this official blog中的更多信息