elasticsearch - 范围查询中的 Java 高级 Rest 客户端内存使用情况

Question

在 Elasticsearch 上的函数运行范围查询之后，当我在多线程环境中重复使用此函数时，内存急剧增加，直到应用程序崩溃。
有没有人可以解决这个问题？

/**
  RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));
**/
public List<Map<String, Object>> getAllDocumentsInRange(long startTime, long endTime,
       RestHighLevelClient restHighLevelClient) {
    try {
        QueryBuilder queryBuilder = QueryBuilders//
                .rangeQuery("date")//
                .gte(startTime)//
                .lte(endTime);


        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        searchSourceBuilder.query(queryBuilder);

        searchSourceBuilder.sort(new FieldSortBuilder("date").order(SortOrder.ASC));

        searchSourceBuilder.size(10000);

        searchSourceBuilder.timeout(new TimeValue(20, TimeUnit.SECONDS));

        List<Map<String, Object>> docsMap = new ArrayList<>();

        SearchRequest searchRequest = new SearchRequest(new String[] { "mainIndex" }, searchSourceBuilder);

        Scroll scroll = new Scroll(TimeValue.timeValueSeconds(30));
        searchRequest.scroll(scroll);


        RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        String scrollId = searchResponse.getScrollId();

        SearchHit[] searchHits = searchResponse.getHits().getHits();

        while (searchHits != null && searchHits.length > 0) {

            for (SearchHit hit : searchHits) {
                Map<String, Object> elasticDocVersion = hit.getSourceAsMap();
                docsMap.add(elasticDocVersion);
            }

            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            scrollRequest.scroll(scroll);

            searchResponse = restHighLevelClient.scroll(scrollRequest, RequestOptions.DEFAULT);

            scrollId = searchResponse.getScrollId();

            searchHits = searchResponse.getHits().getHits();
        }

        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = //
                restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        boolean succeeded = clearScrollResponse.isSucceeded();
        logger.trace("search scroll clearation:{}", succeeded);
    } catch (Exception e) {
        logger.error("error in creaing QueryBuilder class: {}", e.getMessage());
    }
    return new ArrayList<>();
}

这是我运行应用程序后的内存使用情况图片

我尝试了不同的解决方案，例如同步上述功能，但都没有解决问题!

Answer 1

不确定您使用的是什么 elasticsearch 客户端版本，但您可能对 this 感兴趣elasticsearch 休息客户端内存泄漏。
另外，我不确定restHighLevelClient 是哪一个。传入或在方法中创建的那个。只有一个有效，但无论如何，我建议调用 restHighLevelClient.close()或在与客户端完成后在 try-with-resources 中使用它。

try (RestHighLevelClient restHighLevelClient = 
   new RestHighLevelClient(RestClient.builder(new HttpHost("localhost",9200,"http"))) {
...
}