linux - 归档不同名称的每小时文件

Question

我有一个如下所示的日志文件:

2018/10/08 12:15:04 David access denied
2018/10/08 12:15:05 David access denied
2018/10/08 12:15:05 David access granted
2018/10/08 13:15:14 Karel Jan access granted
2018/10/08 13:15:19 Lydia access denied
2018/10/08 13:15:20 Lydia access denied
2018/10/08 13:15:21 Lydia access granted
2018/10/08 14:15:26 Henk access denied
2018/10/08 14:15:26 Henk access denied
2018/10/08 14:15:27 Henk access denied

脚本:

file="log.txt"
while read -r regel
do
        sort | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
                {print > ("logfile_" file ".txt")}'
        zip logfile_20181008.zip logfile_20181008{00..23}.txt   
done < "$file"

到目前为止，这是我在帮助下得到的结果，也出现了以下错误:

ArchiveerLog.sh: line 6: syntax error near unexpected token `('
ArchiveerLog.sh: line 6: `  {print > (prefix bestand".txt")}'

我有每小时的日志文件，并且想每天压缩它们，这样压缩文件将被称为 logfile_20181008.zip 如上所述，有没有办法不硬核这个？

Answer 1

如果您有一个未排序的日志文件和这种类型的日志键，您就会有点不安。可排序的日期时间格式的格式为 YYYY[c]MM[c]DD[c]hh[c]mm[c]ss[.sss] 并且始终在同一时区中表示.您提供的格式不能通过简单的 ascii 排序直接排序。作为一个简单的例子。键“01/01/2018.00:00:00”<“01/10/2018.00:00:00”<“10/10/1302.00:00:00”。

使用sort工具你可以建立一个复杂的排序结构:

sort -k1.7n,1.10 -k1.4n,1.5 -k1.1n,1.2 -k1.11 <logfile>

这将正确排序您的文件。现在您可以将其通过管道传输到 awk 中按小时进行拆分:

sort -k1.7n,1.10 -k1.4n,1.5 -k1.1n,1.2 -k1.11 <logfile> \
    | awk -v prefix="logfile_" '{file=substr($1,1,13); gsub(/[^0-9]/,"",file) }
                                {print > (prefix file".txt")}'

这将对文件进行排序并将所有行移动到文件 logfile_DDMMYYYYhh.txt

更新:问题更新后!

sort log.txt \
 | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
        {print > ("bestand_" file ".txt")}'

第二次更新:因此您的整个脚本现在可以写成:

#!/usr/bin/env bash

#######################################################
# THIS IS NOT TESTED BUT SHOULD BE UPDATED WHERE NEEDED
#######################################################

# This is your input
logfile="log.txt"

# create a temporary directory where to do all the work
tmpdir=$(mktemp -d)

# get the full path of logfile
logfile=$(readlink -f "$logfile")

cd "$tmpdir" || exit
sort "$logfile" | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
                       {print > ("logfile_" file ".txt")}'

# Now you have a $tmpdir with lots of files
# perform the zipping
oldstring=""
newstring=""
for files in ./*; do
   #remove last 6 characters ("hh.txt")
   newstring="${files%[0-9][0-9].txt}"
   [[ "${newstring}" == "${oldstring}" ]] && continue
   zip "$(dirname $logfile)/${newstring}.zip" ${newstring}[0-9][0-9].txt
   oldstring="${newstring}"
done

# uncomment this part only if you are sure it works
# # cleanup
# cd $(dirname $logfile)
# rm -rf "$tmpdir"