linux - 为什么 rsyslog 在 logrotate 之后启动失败？

Question

在午夜，cronjob 启动 logrotate 以轮换 4 个关键日志文件。这 4 个日志文件也被 rsyslog 发送到日志聚合服务器。尽管在每个日志轮转后运行 rsyslog logrotate 命令，但有时服务器会遇到“权限被拒绝”错误，从而阻止 rsyslog 发送属于新文件的日志。我不明白为什么会发生这种不可靠的情况。

这里是 rsyslog 报告的错误。请注意，它们仅在某些日期出现:

Jul 31 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 01 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 02 00:00:04 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/nginx.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 09 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 10 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/django.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 10 00:00:01 hostname rsyslogd[26343]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]
Aug 16 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 16 00:00:02 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/django.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 19 00:00:03 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]
Aug 20 00:00:06 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/app.log': Permission denied [v8.32.0]
Aug 30 00:00:01 hostname rsyslogd[856]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Sep 03 00:00:01 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/celery.log': Permission denied [v8.32.0]
Sep 03 00:00:04 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]

Logrotate 由 cron 作业触发，使用 root 用户。这是 logrotate 配置:

/home/ubuntu/remote_logs/*.log {
    rotate 365
    daily
    compress
    missingok
    notifempty
    dateext
    dateformat .%Y-%m-%d
    dateyesterday
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

/usr/lib/rsyslog/rsyslog-rotate

#!/bin/sh

if [ -d /run/systemd/system ]; then
    systemctl kill -s HUP rsyslog.service
else
    invoke-rc.d rsyslog rotate > /dev/null
fi

我知道 logrotate 本身是成功的，因为 logrotate 详细日志表明了这一点，而且我每天都看到旋转的日志文件。 rsyslog 似乎无法访问新创建的文件。

这是 logrotate 日志:

Handling 13 logs

rotating pattern: /home/ubuntu/remote_logs/*.log  after 1 days (365 rotations)
empty log files are not rotated, old logs are removed
switching euid to 0 and egid to 106
considering log /home/ubuntu/remote_logs/celery.log
  Now: 2020-10-14 00:00
  Last rotated at 2020-10-13 00:00
  log needs rotating
considering log /home/ubuntu/remote_logs/django.log
  Now: 2020-10-14 00:00
  Last rotated at 2020-10-13 00:00
  log needs rotating
considering log /home/ubuntu/remote_logs/app.log
  Now: 2020-10-14 00:00
  Last rotated at 2020-10-13 00:00
  log needs rotating
considering log /home/ubuntu/remote_logs/nginx.log
  Now: 2020-10-14 00:00
  Last rotated at 2020-10-13 00:00
  log needs rotating
rotating log /home/ubuntu/remote_logs/celery.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/celery.log to /home/ubuntu/remote_logs/celery.log.2020-10-13
creating new /home/ubuntu/remote_logs/celery.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/django.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/django.log to /home/ubuntu/remote_logs/django.log.2020-10-13
creating new /home/ubuntu/remote_logs/django.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/app.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/app.log to /home/ubuntu/remote_logs/app.log.2020-10-13
creating new /home/ubuntu/remote_logs/app.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/nginx.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/nginx.log to /home/ubuntu/remote_logs/nginx.log.2020-10-13
creating new /home/ubuntu/remote_logs/nginx.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
switching euid to 0 and egid to 0

rsyslog 发生了什么，阻止它偶尔访问日志文件？

Answer 1

看起来有些文件没有创建权限。您需要在配置文件中添加 create 0644 root root 以授予旋转(修改)文件内容的必要权限。
您还可以指定组名 create 0644 root grouname。但我会建议使用 root

/home/ubuntu/remote_logs/*.log {
    rotate 365
    daily
    compress
    missingok
    create 0644 root root
    notifempty
    dateext
    dateformat .%Y-%m-%d
    dateyesterday
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}