python - 解析并获取日志文件中两个时间对象之间的值

Question

我正在尝试编写自定义日志解析器。日志文件如下:

     09:57:25Host_Name  Trace                      00000                                                  
      <MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00"  abcd 
      some string ---
        SQ-> 
    09:57:25Host_Name  Trace                      00000                                                  
     <MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00" abcd 
      some string ---
        D--> 
        SQ-> 
    09:57:28Host_Name   Trace                      00000                                                  
     <MessageLogTraceRecord Time="2017-04-13T09:57:28.1393344+00:00" abcd 
      some string ---
        D--> 
        SQ-> 
    09:58:28Host_Name   Trace                      00000                                                  
     <MessageLogTraceRecord Time="2017-04-13T09:58:28.1393344+00:00" abcd 
      some string ---
        D--> 
        SQ-> 

The goal is to have json output in following format 
[{'host_name': host_name, 'time': '2017-04-13T09:58:28.1393344+00:00', 'msg
: '<MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00" abcd 
  some string ---
    D--> 
    SQ->'}, {'host_name': host_name, 'time': '2017-04-13T09:58:28.1393344+00:00', 'msg
: '<MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00" abcd 
  some string ---
    D--> 
    SQ->'}]

我面临的问题是获取两个时间对象和时间之间的值。

以下我尝试过:

jsonlist = []
jsonout = {}
li = [i.strip().split() for i in open(filepath).readlines()]
start_index, end_index=0,0
msg = ''
with open(filepath, 'r') as f:
  for index, line in enumerate(f):
    if start_index !=0 and end_index!=0:
      result =  list(itertools.chain.from_iterable(li[start_index: end_index]))
            msg =  ''.join(str(x) for x in result)
            jsonoutput['message'] =  msg.replace('"', '\\').strip()
            jsonoutput['time'] = msg.
            start_index, end_index = 0,0
    try:
      if start_index !=0:
        if parser(line.split()[0].split('Host_Name')[0]):
          end_index = index
        else:
          start_index = index 

我无法获取时间值和正确的消息。任何更好的建议都会非常有帮助

Answer 1

我自己写了代码:

import json
import re


def logs(file_path):
    """
    :param file_path: path to your log file, example: /home/user/my_file.log 
    """
    msg = ''
    final = []

    our_log = open(file_path, 'r')
    log_lines = our_log.readlines()

    for line in log_lines:
        time = re.search("^[\d]+:[\d]+:[\d]+", line)

        if time:
            if msg:
                final[-1].update(msg=msg)
                msg = ''

            time = time.group(0)
            host_name = re.search(time + '(.*)' + '  Trace', line).group(1)

            # If you need the time like "09:57:25", instead of "'2017-04-13T09:57:25.1393344+00:00"
            # then uncomment the line below
            # info = dict(time=time, host_name=host_name)

            # and comment the one below
            info = dict(host_name=host_name)

            final.append(info)

        else:
            # and also comment the next 3 lines
            if 'Time="' in line:
                time = re.search('Time="' + '(.*)' + '"', line).group(1)
                final[-1].update(time=time)
            msg += line.strip()

    final[-1].update(msg=msg)  # adds message for the last time-section

    json_out = json.dumps(final)

根据您提供的数据，var final 如下所示:

[{'msg': '<MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00"  abcdsome string ---SQ->', 'time': '2017-04-13T09:57:25.1393344+00:00', 'host_name': 'Host_Name'}, {'msg': '<MessageLogTraceRecord Time="2017-04-13T09:57:25.1393344+00:00" abcdsome string ---D-->SQ->', 'time': '2017-04-13T09:57:25.1393344+00:00', 'host_name': 'Host_Name'}, {'msg': '<MessageLogTraceRecord Time="2017-04-13T09:57:28.1393344+00:00" abcdsome string ---D-->SQ->', 'time': '2017-04-13T09:57:28.1393344+00:00', 'host_name': 'Host_Name '}, {'msg': '<MessageLogTraceRecord Time="2017-04-13T09:58:28.1393344+00:00" abcdsome string ---D-->SQ->', 'time': '2017-04-13T09:58:28.1393344+00:00', 'host_name': 'Host_Name '}]