linux - Logstash 打开了太多文件？

Question

我正在运行 logstash 1.4.1 并且在运行它时我在控制台上得到以下消息:

    failed to open /home/Desktop/Input/2014-11-03/abc.log: Permission denied - /home/Desktop/Input/2014-11-03/abc.log {:level=>:warn}
    failed to open /home/Desktop/Input/2014-10-14/abc.log: Permission denied - /home/Desktop/Input/2014-10-14/abc.log {:level=>:warn}
    failed to open /home/Desktop/2014-10-16/abc.log: Permission denied - /home/Desktop/Input/2014-10-16/abc.log {:level=>:warn}

..............

    log4j, [2014-11-13T16:20:04.583]  WARN: org.elasticsearch.discovery.zen: [logstash-HP-Pro-3330-MT-7573-11818] failed to connect to master [[Kyle Gibney][eM0ZmUkAQpaXd6_n4JWjzA][-HP-Pro-3330-MT][inet[/xx.xx..xx..xx:9300]]], retrying...
    org.elasticsearch.transport.ConnectTransportException: [Kyle Gibney][inet[/xx.xx.xx.xx:9300]] General node connection failure
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:668)
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:615)
        at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
        at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:338)
        at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:79)
        at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
    Caused by: org.elasticsearch.common.netty.channel.ChannelException: Failed to open a socket.
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:43)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.<init>(NioClientSocketChannel.java:82)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:212)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:82)
        at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:212)
        at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
        at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:702)
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:647)
        ... 8 more
    Caused by: java.net.SocketException: Too many open files
        at sun.nio.ch.Net.socket0(Native Method)
        at sun.nio.ch.Net.socket(Net.java:423)
        at sun.nio.ch.Net.socket(Net.java:416)
        at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:104)
        at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
        at java.nio.channels.SocketChannel.open(SocketChannel.java:142)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:41)
        ... 15 more

所以 logstash 打开文件达到某个阈值，然后在该阈值之后拒绝对文件授予权限，并说打开的文件太多。

有什么想法吗？

Answer 1

您需要检查您的 Linux 用户的文件句柄限制是多少。您可以检查的一个地方是:

cat /etc/security/limits.conf

定义硬限制和软限制的位置。另一种方法是使用如下 ulimit 命令:

ulimit -a | grep nofiles ##Or ulimit -n