python - 如何在 python 中优雅地记录多个非常相似的事件？

Question

与 python logging模块，有没有办法将多个事件收集到一个日志条目中？一个理想的解决方案是扩展 python 的 logging 模块或为其添加一个自定义格式化程序/过滤器，以便在后台收集相同类型的日志记录事件并且什么都不做需要在代码体中添加(例如，在每次调用日志功能时)。

这里有一个生成大量相同或非常相似的日志记录事件的示例:

import logging

for i in range(99999): 
    try:
        asdf[i]   # not defined!
    except NameError:
        logging.exception('foo') # generates large number of logging events
    else: pass

# ... more code with more logging ...

for i in range(88888): logging.info('more of the same %d' % i)

# ... and so on ...

所以我们有相同的异常 99999 次并记录它。如果日志只是说类似的话，那就太好了:

ERROR:root:foo (occured 99999 times)
Traceback (most recent call last):
  File "./exceptionlogging.py", line 10, in <module>
    asdf[i]   # not defined!
NameError: name 'asdf' is not defined

INFO:root:foo more of the same (occured 88888 times with various values)

Answer 1

您可能应该编写一个消息聚合/统计类，而不是尝试连接到日志系统的 singletons但我猜你可能有一个使用日志记录的现有代码库。

我还建议您实例化您的记录器，而不是总是使用默认的根目录。 Python Logging Cookbook有广泛的解释和例子。

下面的类应该做你所要求的。

import logging
import atexit
import pprint

class Aggregator(object):
    logs = {}

    @classmethod
    def _aggregate(cls, record):
        id = '{0[levelname]}:{0[name]}:{0[msg]}'.format(record.__dict__)
        if id not in cls.logs: # first occurrence
            cls.logs[id] = [1, record]
        else: # subsequent occurrence
            cls.logs[id][0] += 1

    @classmethod
    def _output(cls):
        for count, record in cls.logs.values():
            record.__dict__['msg'] += ' (occured {} times)'.format(count)
            logging.getLogger(record.__dict__['name']).handle(record)

    @staticmethod
    def filter(record):
        # pprint.pprint(record)
        Aggregator._aggregate(record)
        return False

    @staticmethod
    def exit():
        Aggregator._output()



logging.getLogger().addFilter(Aggregator)
atexit.register(Aggregator.exit)

for i in range(99999): 
    try:
        asdf[i]   # not defined!
    except NameError:
        logging.exception('foo') # generates large number of logging events
    else: pass

# ... more code with more logging ...
for i in range(88888): logging.error('more of the same')

# ... and so on ...    

请注意，在程序退出之前您不会收到任何日志。

运行结果如下:

    ERROR:root:foo (occured 99999 times)    Traceback (most recent call last):      File "C:\work\VEMS\python\logcount.py", line 38, in         asdf[i]   # not defined!    NameError: name 'asdf' is not defined    ERROR:root:more of the same (occured 88888 times)