perl - ElasticSearch (search_context_missing_exception) 与 Search::ElasticSearch::Scroll

Question

我正在使用 Search::Elasticsearch 和 Search::Elasticsearch::Scroll 进行搜索并滚动到我的 elasticsearch 服务器。

在滚动过程中，对于某些查询，我在滚动搜索结果时看到了下一个错误:

2016/03/22 11:03:38 - 265885 FATAL: [Daemon.pm][8221]: Something gone wrong, error $VAR1 = bless( {
 'msg' => '[Missing] ** [http://localhost:9200]-[404] Not Found, called from sub Search::Elasticsearch::Scroll::next at searcher.pl line 92. With vars: {\'body\' => {\'hits\' => {\'hits\' => [],\'max_score\' => \'0\',\'total\' => 5215},\'timed_out\' => bless( do{\\(my $o = 0)}, \'JSON::XS::Boolean\' ),\'_shards\' => {\'failures\' => [{\'index\' => undef,\'reason\' => {\'reason\' => \'No search context found for id [4920053]\',\'type\' => \'search_context_missing_exception\'},\'shard\' => -1},{\'index\' => undef,\'reason\' => {\'reason\' => \'No search context found for id [5051485]\',\'type\' => \'search_context_missing_exception\'},\'shard\' => -1},{\'index\' => undef,\'reason\' => {\'reason\' => \'No search context found for id [4920059]\',\'type\' => \'search_context_missing_exception\'},\'shard\' => -1},{\'index\' => undef,\'reason\' => {\'reason\' => \'No search context found for id [5051496]\',\'type\' => \'search_context_missing_exception\'},\'shard\' => -1},{\'index\' => undef,\'reason\' => {\'reason\' => \'No search context found for id [5051500]\',\'type\' => \'search_context_missing_exception\'},\'shard\' => -1}],\'failed\' => 5,\'successful\' => 0,\'total\' => 5},\'_scroll_id\' => \'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==\',\'took\' => 2},\'request\' => {\'serialize\' => \'std\',\'path\' => \'/_search/scroll\',\'ignore\' => [],\'mime_type\' => \'application/json\',\'body\' => \'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7\',\'qs\' => {\'scroll\' => \'1m\'},\'method\' => \'GET\'},\'status_code\' => 404}
',
 'stack' => [
              [
                'searcher.pl',
                92,
                'Search::Elasticsearch::Scroll::next'
              ]
            ],
 'text' => '[http://localhost:9200]-[404] Not Found',
 'vars' => {
             'body' => {
                         'hits' => {
                                     'hits' => [],
                                     'max_score' => '0',
                                     'total' => 5215
                                   },
                         'timed_out' => bless( do{\(my $o = 0)}, 'JSON::XS::Boolean' ),
                         '_shards' => {
                                        'failures' => [
                                                        {
                                                          'index' => undef,
                                                          'reason' => {
                                                                        'reason' => 'No search context found for id [4920053]',
                                                                        'type' => 'search_context_missing_exception'
                                                                      },
                                                          'shard' => -1
                                                        },
                                                        {
                                                          'index' => undef,
                                                          'reason' => {
                                                                        'reason' => 'No search context found for id [5051485]',
                                                                        'type' => 'search_context_missing_exception'
                                                                      },
                                                          'shard' => -1
                                                        },
                                                        {
                                                          'index' => undef,
                                                          'reason' => {
                                                                        'reason' => 'No search context found for id [4920059]',
                                                                        'type' => 'search_context_missing_exception'
                                                                      },
                                                          'shard' => -1
                                                        },
                                                        {
                                                          'index' => undef,
                                                          'reason' => {
                                                                        'reason' => 'No search context found for id [5051496]',
                                                                        'type' => 'search_context_missing_exception'
                                                                      },
                                                          'shard' => -1

                                                                                                        },
                                                        {
                                                          'index' => undef,
                                                          'reason' => {
                                                                        'reason' => 'No search context found for id [5051500]',
                                                                        'type' => 'search_context_missing_exception'
                                                                      },
                                                          'shard' => -1
                                                        }
                                                      ],
                                        'failed' => 5,
                                        'successful' => 0,
                                        'total' => 5
                                      },
                         '_scroll_id' => 'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==',
                         'took' => 2
                       },
             'request' => {
                            'serialize' => 'std',
                            'path' => '/_search/scroll',
                            'ignore' => [],
                            'mime_type' => 'application/json',
                            'body' => 'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7',
                            'qs' => {
                                      'scroll' => '1m'
                                    },
                            'method' => 'GET'
                          },
             'status_code' => 404
           },
 'type' => 'Missing'
}, 'Search::Elasticsearch::Error::Missing' );

我使用的代码是下一个(简化):

# Retrieve scroll 
my $scroll = $self->getScrollBySignature($item);

# Retrieve all affected documents ids 
while (my @docs = $scroll->next(500)) {
    # Do stuff with @docs
}

函数 getScrollBySignature 有下一个代码来调用 elasticSearch

my $scroll = $self->{ELASTIC}->scroll_helper(
    index              => $self->{INDEXES},
    search_type        => 'scan',
    ignore_unavailable => 1,
    body               => {
        size  => $self->{PAGINATION},
        query => {
            filtered => {
                filter => {
                    bool => {
                        must => [{term => {signature_id => $item->{profileId}}}, {terms => {channel_type_id => $type}}]
                    }
                }
            }
        }
    }
);

如您所见，我在不传递滚动参数的情况下进行滚动，然后如文档所述，滚动有效的时间为 1 分钟。

elasticSearch 是一个由 3 个服务器组成的集群，以该错误结束的查询检索了 5000 多个文档。

我的第一个解决方案是将滚动的生命周期更新为 5 分钟，但没有出现错误。

问题是，据我所知，每次我调用 $scroll->next() 时，受影响的滚动生命周期都会增加 1m，那么如何才能收到那些与上下文相关的错误？

我做事不礼貌？

谢谢大家

Answer 1

首先想到的是定时器没有更新。你查过这个吗？例如，您可以每 10 秒执行一次查询，看看在第 6 次查询时它是否给您错误 ...