erlang - RabbitMQ 通用服务器rabbit_disk_monitor 终止/eheap_alloc : Cannot allocate 229520 bytes of memory (of type "old_heap")

Question

RabbitMQ 崩溃了。RabbitMQ 已正常工作很多天(10-15 天)。我不明白为什么它崩溃了。

<小时/>

我在 Erlang 17.0 上使用 RabbitMQ 3.4.0

<小时/>

erlang 已为崩溃创建转储文件。其中显示

eheap_alloc: Cannot allocate 229520 bytes of memory (of type "old_heap").

<小时/>

另请注意，rabbitmq 发布-订阅消息负载非常低。 (最大:1-2 条消息/秒)。并且 RabbitMQ 消息在到达时进行处理，因此 RabbitMQ 几乎一直是空的。磁盘空间和内存也足够。

More system info:
Limiting to approx 8092 file handles (7280 sockets)
Memory limit set to 6553MB of 16383MB total.
Disk free limit set to 50MB.

RabbitMQ 日志如下。

=ERROR REPORT==== 18-Jul-2015::04:29:31 ===
** Generic server rabbit_disk_monitor terminating 
** Last message in was update
** When Server state == {state,"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia",
                               50000000,28358258688,100,10000,
                               #Ref<0.0.106.70488>,false}
** Reason for termination == 
** {eacces,[{erlang,open_port,
                    [{spawn,"C:\\Windows\\system32\\cmd.exe /c dir /-C /W \"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia\""},
                     [stream,in,eof,hide]],
                    []},
            {os,cmd,1,[{file,"os.erl"},{line,204}]},
            {rabbit_disk_monitor,get_disk_free,2,[]},
            {rabbit_disk_monitor,internal_update,1,[]},
            {rabbit_disk_monitor,handle_info,2,[]},
            {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,599}]},
            {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}

=INFO REPORT==== 18-Jul-2015::04:29:31 ===
Disabling disk free space monitoring on unsupported platform:
{{'EXIT',{eacces,[{erlang,open_port,
                          [{spawn,"C:\\Windows\\system32\\cmd.exe /c dir /-C /W \"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia\""},
                           [stream,in,eof,hide]],
                          []},
                  {os,cmd,1,[{file,"os.erl"},{line,204}]},
                  {rabbit_disk_monitor,get_disk_free,2,[]},
                  {rabbit_disk_monitor,init,1,[]},
                  {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
                  {proc_lib,init_p_do_apply,3,
                            [{file,"proc_lib.erl"},{line,239}]}]}},
 17179336704}

=INFO REPORT==== 18-Jul-2015::04:29:31 ===
Disabling disk free space monitoring on unsupported platform:
{{'EXIT',{eacces,[{erlang,open_port,
                          [{spawn,"C:\\Windows\\system32\\cmd.exe /c dir /-C /W \"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia\""},
                           [stream,in,eof,hide]],
                          []},
                  {os,cmd,1,[{file,"os.erl"},{line,204}]},
                  {rabbit_disk_monitor,get_disk_free,2,[]},
                  {rabbit_disk_monitor,init,1,[]},
                  {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
                  {proc_lib,init_p_do_apply,3,
                            [{file,"proc_lib.erl"},{line,239}]}]}},
 17179336704}

<小时/>

=CRASH REPORT==== 18-Jul-2015::04:29:31 ===
  crasher:
    initial call: rabbit_disk_monitor:init/1
    pid: <0.167.0>
    registered_name: rabbit_disk_monitor
    exception exit: {eacces,
                        [{erlang,open_port,
                             [{spawn,
                                  "C:\\Windows\\system32\\cmd.exe /c dir /-C /W \"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia\""},
                              [stream,in,eof,hide]],
                             []},
                         {os,cmd,1,[{file,"os.erl"},{line,204}]},
                         {rabbit_disk_monitor,get_disk_free,2,[]},
                         {rabbit_disk_monitor,internal_update,1,[]},
                         {rabbit_disk_monitor,handle_info,2,[]},
                         {gen_server,handle_msg,5,
                             [{file,"gen_server.erl"},{line,599}]},
                         {proc_lib,init_p_do_apply,3,
                             [{file,"proc_lib.erl"},{line,239}]}]}
      in function  gen_server:terminate/6 (gen_server.erl, line 746)
    ancestors: [rabbit_disk_monitor_sup,rabbit_sup,<0.140.0>]
    messages: []
    links: [<0.166.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 4185
    stack_size: 27
    reductions: 481081978
  neighbours:

=SUPERVISOR REPORT==== 18-Jul-2015::04:29:31 ===
     Supervisor: {local,rabbit_disk_monitor_sup}
     Context:    child_terminated
     Reason:     {eacces,
                     [{erlang,open_port,
                          [{spawn,
                               "C:\\Windows\\system32\\cmd.exe /c dir /-C /W \"c:/Users/jasmin.joshi/AppData/Roaming/RabbitMQ/db/rabbit@localhost-mnesia\""},
                           [stream,in,eof,hide]],
                          []},
                      {os,cmd,1,[{file,"os.erl"},{line,204}]},
                      {rabbit_disk_monitor,get_disk_free,2,[]},
                      {rabbit_disk_monitor,internal_update,1,[]},
                      {rabbit_disk_monitor,handle_info,2,[]},
                      {gen_server,handle_msg,5,
                          [{file,"gen_server.erl"},{line,599}]},
                      {proc_lib,init_p_do_apply,3,
                          [{file,"proc_lib.erl"},{line,239}]}]}
     Offender:   [{pid,<0.167.0>},
                  {name,rabbit_disk_monitor},
                  {mfargs,{rabbit_disk_monitor,start_link,[50000000]}},
                  {restart_type,{transient,1}},
                  {shutdown,4294967295},
                  {child_type,worker}]


=CRASH REPORT==== 18-Jul-2015::04:29:31 ===
  crasher:
    initial call: rabbit_disk_monitor:init/1
    pid: <0.24989.51>
    registered_name: []
    exception exit: unsupported_platform
      in function  gen_server:init_it/6 (gen_server.erl, line 322)
    ancestors: [rabbit_disk_monitor_sup,rabbit_sup,<0.140.0>]
    messages: []
    links: [<0.166.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 650
  neighbours:

=SUPERVISOR REPORT==== 18-Jul-2015::04:29:31 ===
     Supervisor: {local,rabbit_disk_monitor_sup}
     Context:    start_error
     Reason:     unsupported_platform
     Offender:   [{pid,<0.167.0>},
                  {name,rabbit_disk_monitor},
                  {mfargs,{rabbit_disk_monitor,start_link,[50000000]}},
                  {restart_type,{transient,1}},
                  {shutdown,4294967295},
                  {child_type,worker}]


=CRASH REPORT==== 18-Jul-2015::04:29:31 ===
  crasher:
    initial call: rabbit_disk_monitor:init/1
    pid: <0.24991.51>
    registered_name: []
    exception exit: unsupported_platform
      in function  gen_server:init_it/6 (gen_server.erl, line 322)
    ancestors: [rabbit_disk_monitor_sup,rabbit_sup,<0.140.0>]
    messages: []
    links: [<0.166.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 650
  neighbours:

=SUPERVISOR REPORT==== 18-Jul-2015::04:29:31 ===
     Supervisor: {local,rabbit_disk_monitor_sup}
     Context:    start_error
     Reason:     unsupported_platform
     Offender:   [{pid,{restarting,<0.167.0>}},
                  {name,rabbit_disk_monitor},
                  {mfargs,{rabbit_disk_monitor,start_link,[50000000]}},
                  {restart_type,{transient,1}},
                  {shutdown,4294967295},
                  {child_type,worker}]

Answer 1

从错误信息来看，由于系统限制，rabbitmq无法打开更多文件。您可以将最大打开文件数设置为较高值以避免出现此问题。

https://serverfault.com/questions/249477/windows-server-2008-r2-max-open-files-limit