java - Elasticsearch Rest 客户端仍然给出 IOException : Too Many Open Files

Question

这是对上一篇文章中提供给我的解决方案的跟进:

How to Properly Close Raw RestClient When Using Elastic Search 5.5.0 for Optimal Performance?

同样的错误信息又回来了!

2017-09-29 18:50:22.497 ERROR 11099 --- [8080-Acceptor-0] org.apache.tomcat.util.net.NioEndpoint   : Socket accept failed

java.io.IOException: Too many open files
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_141]
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) ~[na:1.8.0_141]
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) ~[na:1.8.0_141]
    at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:453) ~[tomcat-embed-core-8.5.15.jar!/:8.5.15]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]

2017-09-29 18:50:23.885  INFO 11099 --- [Thread-3] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@5387f9e0: startup date [Wed Sep 27 03:14:35 UTC 2017]; root of context hierarchy
2017-09-29 18:50:23.890  INFO 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483647
2017-09-29 18:50:23.891  WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor  : Failed to stop bean 'documentationPluginsBootstrapper'

    ... 7 common frames omitted

2017-09-29 18:50:53.891  WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor  : Failed to shut down 1 bean with phase value 2147483647 within timeout of 30000: [documentationPluginsBootstrapper]
2017-09-29 18:50:53.891  INFO 11099 --- [Thread-3] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
2017-09-29 18:50:53.894  INFO 11099 --- [Thread-3] com.app.controller.SearchController  : Closing the ES REST client

我尝试使用上一篇文章中的解决方案。

ElasticsearchConfig:

@Configuration
public class ElasticsearchConfig {

@Value("${elasticsearch.host}")
private String host;

@Value("${elasticsearch.port}")
private int port;

@Bean
public RestClient restClient() {
    return RestClient.builder(new HttpHost(host, port))
    .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
        @Override
        public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
            return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000);
        }
    }).setMaxRetryTimeoutMillis(60000).build();
}

搜索 Controller :

@RestController
@RequestMapping("/api/v1")
public class SearchController {

    @Autowired
    private RestClient restClient;

    @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
    public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException {

        // Setup HTTP Headers
        HttpHeaders headers = new HttpHeaders();
        headers.add("Content-Type", "application/json");

        // Setup query and send and return ResponseEntity...

        Response response = this.restClient.performRequest(...);
    }

    @PreDestroy
    public void cleanup() {
        try {
            logger.info("Closing the ES REST client");
            this.restClient.close();
        } 
        catch (IOException ioe) {
            logger.error("Problem occurred when closing the ES REST client", ioe);
        }
    }
}    

pom.xml:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.5.4.RELEASE</version>
</parent>

<dependencies>
    <!-- Spring -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>

    <!-- Elasticsearch -->
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>5.5.0</version>
    </dependency>

    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>5.5.0</version>
    </dependency>

    <!-- Apache Commons -->
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.6</version>
    </dependency>

    <!-- Log4j -->
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>
</dependencies>

这让我觉得 RestClient 从来没有明确关闭连接，首先......

这令人惊讶，因为我的基于 Elasticsearch Spring Boot 的微服务在两个不同的 AWS EC-2 服务器上实现了负载平衡。

日志文件报告该异常发生了 2000 次，直到最后 preDestroy() 才关闭客户端。请参阅 StackTrace 末尾记录的 @PreDestroy() 清理方法中的信息。

我是否需要在 SearchController 中显式放置一个 finally 子句并显式关闭 RestClient 连接？

确保此 IOException 不再发生非常重要，因为此搜索微服务依赖于许多不同的移动客户端(iOS 和 Android)。

需要它具有容错性和可扩展性……或者，至少，不会中断。

这是在日志文件底部的唯一原因:

2017-09-29 18:50:53.894  INFO 11099 --- [Thread-3] com.app.controller.SearchController : Closing the ES REST client

是因为我这样做了:

kill -3 jvm_pid

我是否应该保留 @PreDestory cleanup() 方法但更改我的 SearchController.getSearchResults() 方法的内容以反射(reflect)如下内容:

@RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException {
    // Setup HTTP Headers
    HttpHeaders headers = new HttpHeaders();
    headers.add("Content-Type", "application/json");

    // Setup query and send and return ResponseEntity...
    Response response = null;

    try {
        // Submit Query and Obtain Response
        response = this.restClient.performRequest("POST", endPoint, Collections.singletonMap("pretty", "true"), entity);
    } 
    catch(IOException ioe) {
        logger.error("Exception when performing POST request " + ioe);
    }
    finally {
        this.restClient.close();
    }
    // return response as EsResponse();
}

这样 RestClient 连接总是关闭...

如果有人能帮助我，我将不胜感激。

Answer 1

从我的角度来看，您做错的地方很少，但我会直接找到解决方案。

我不会写完整的解决方案(事实上，我没有执行或测试任何东西)，但重要的是理解它。此外，最好将所有与数据访问相关的内容移至另一层。 无论如何，这只是一个例子，所以设计并不完美。

第 1 步:导入正确的库。

实际上与您的示例相同。我更新了示例以使用版本 5.6.2 中推荐的最后一个客户端库

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.acervera</groupId>
  <artifactId>elastic-example</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>elastic-example</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <es.version>5.6.2</es.version>
  </properties>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.5.4.RELEASE</version>
  </parent>

  <dependencies>
    <!-- Spring -->
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
    </dependency>

    <!-- Elasticsearch -->
    <dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>${es.version}</version>
    </dependency>

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-high-level-client</artifactId>
  <version>${es.version}</version>
</dependency>

    <!-- Log4j -->
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.17</version>
    </dependency>
  </dependencies>
</project>

第 2 步:在 bean 工厂中创建和关闭客户端。

在bean factory中，创建和销毁它。您可以重复使用同一个客户端。

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import java.io.IOException;

@Configuration
public class ElasticsearchConfig {


    // Here all init stuff with @Value(....)


    RestClient lowLevelRestClient;
    RestHighLevelClient client;

    @PostConstruct
    public void init() {
        lowLevelRestClient = RestClient.builder(new HttpHost("host", 9200, "http")).build();
        client = new RestHighLevelClient(lowLevelRestClient);
    }

    @PreDestroy
    public void destroy() throws IOException {
        lowLevelRestClient.close();
    }

    @Bean
    public RestHighLevelClient getClient() {
        return client;
    }

}

第 3 步:使用 Java 传输客户端执行查询。

使用 Java 传输客户端执行查询。

import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.io.IOException;

@RestController
@RequestMapping("/api/v1")
public class SearchController {

    @Autowired
    private RestHighLevelClient client;

    @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
    public ResponseEntity<Tweet> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException {

        // This is only one example. Of course, this logic make non sense and you are going to put it in a DAO
        // layer with more logical stuff
        SearchRequest searchRequest = new SearchRequest();
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        SearchResponse searchResponse = client.search(searchRequest);

        if(searchResponse.getHits().totalHits > 0) {
            SearchHit searchHit = searchResponse.getHits().iterator().next();

            // Deserialize to Java. The best option is to use response.getSource() and Jackson
            // This is other option.
            Tweet tweet = new Tweet();
            tweet.setId(searchHit.getField("id").getValue().toString());
            tweet.setTittle(searchHit.getField("tittle").getValue().toString());
            return ResponseEntity.ok(tweet);
        } else {
            return ResponseEntity.notFound().build();
        }

    }

}

另外，使用一个 bean 来构建响应。

public class Tweet {

    private String id;

    private String tittle;

    public String getTittle() {
        return tittle;
    }

    public void setTittle(String tittle) {
        this.tittle = tittle;
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    // Here rest of bean stuff (equal, hash, etc) or Lombok
}

步骤:4

享受 Elasticsearch!

备注:Java REST Client [5.6] » Java High Level REST Client

附言。有必要重构示例。只是了解方式而已。