gpt4 book ai didi

python - ProcessPoolExecutor 日志记录失败?

转载 作者:太空宇宙 更新时间:2023-11-03 10:54:27 37 4
gpt4 key购买 nike

我正在创建一个多处理程序来处理多个批处理,但我的日志记录无法将批处理记录到日志文件中,只会记录根 log.info,如何设置日志记录以正确打印到日志文件?

日志只会打印这样一行"INFO:root:this is root logging"

import logging
import concurrent.futures
def process_batchs():
batches = [i for i in range(100)]
logging.basicConfig(filename=r'doc\test_ProcessPoolExecutor.log', filemode='w+',level=logging.DEBUG)
logging.info('this is root logging')
with concurrent.futures.ProcessPoolExecutor(10) as e:
futures = []
for batch in batches:
future = e.submit(f, batch)
futures.append(future)
while True:
dones = [future.done() for future in futures]
if all(dones):
results = [future.result() for future in futures]
print results
break
def f(batch):
# do some thing
logging.info('this is sub logging' + str(batch))
return batch


if __name__ == '__main__':
process_batchs()

在windows/python2.7上运行

最佳答案

日志记录在每个子进程中使用不同的实例,并且无法写入同一个文件。应用 follow fix 将解决问题,但我认为更好的解决方案可能是使用 logging.getlogger('abc') 的单例模式?

import logging
import concurrent.futures
def process_batchs():
batches = [i for i in range(100)]
logging.basicConfig(filename=r'test_ProcessPoolExecutor.log', filemode='w+',level=logging.DEBUG)
logging.info('this is root logging')
with concurrent.futures.ProcessPoolExecutor(10) as e:
futures = []
for batch in batches:
future = e.submit(f, batch)
futures.append(future)
while True:
dones = [future.done() for future in futures]
if all(dones):
results = [future.result() for future in futures]
print results
break
def f(batch):
# do some thing
# Here is the trick, notice here!!!
########
logging.basicConfig(filename=r'test_ProcessPoolExecutor.log', filemode='w+',level=logging.DEBUG)
########
logging.info('this is sub logging' + str(batch))
return batch


if __name__ == '__main__':
process_batchs()

关于python - ProcessPoolExecutor 日志记录失败?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43949259/

37 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com