heroku - 捕获 TERM 并发送 QUIT 后，Heroku 上的 Unicorn 退出超时

Question

我收到运行 unicorn 和 sidekiq 的 Heroku 应用程序的 R12 退出超时错误。这些错误每天以及每次部署时都会发生 1-2 次。我知道我需要转换来自 Heroku 的关闭信号以使 unicorn 正确响应，但我认为我已经在下面的 unicorn 配置中这样做了:

worker_processes 3
timeout 30
preload_app true

before_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn master intercepting TERM and sending myself QUIT instead. My PID is #{Process.pid}"
    Process.kill 'QUIT', Process.pid
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection.disconnect!
    Rails.logger.info('Disconnected from ActiveRecord')
  end
end

after_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is #{Process.pid}"
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    Rails.logger.info('Connected to ActiveRecord')
  end

  Sidekiq.configure_client do |config|
    config.redis = { :size => 1 }
  end
end

我的有关错误的日志如下所示:

Stopping all processes with SIGTERM
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 7
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 11
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 15
Unicorn master intercepting TERM and sending myself QUIT instead. My PID is 2
Started GET "/manage"
reaped #<Process::Status: pid 11 exit 0> worker=1
reaped #<Process::Status: pid 7 exit 0> worker=0
reaped #<Process::Status: pid 15 exit 0> worker=2
master complete
Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
Stopping remaining processes with SIGKILL
Process exited with status 137

看来所有子进程在超时之前都已成功收获。难道师父还活着吗？另外，路由器在关闭期间是否仍应向测功机发送 Web 请求，如日志所示？

FWIW，我正在使用 Heroku 的零停机部署插件 ( https://devcenter.heroku.com/articles/labs-preboot/ )。

Answer 1

我认为您的自定义信号处理是导致此处超时的原因。

编辑:我因不同意 Heroku 的文档而被否决，我想解决这个问题。

将您的 Unicorn 应用程序配置为捕获并吞下 TERM 信号是导致应用程序挂起且无法正确关闭的最可能原因。

Heroku 似乎认为，捕获 TERM 信号并将其转换为 QUIT 信号是将硬关机转变为正常关机的正确行为。

但是，这样做似乎在某些情况下会带来根本无法关闭的风险 - 这是此错误的根源。体验运行 Unicorn 的悬挂测功机的用户应该考虑证据并根据首要原则做出自己的决定，而不仅仅是文档。