python - 使用装饰器来持久化 python 对象

Question

我从下面的链接获得的代码，可以将数据保存到磁盘。

http://tohyongcheng.github.io/python/2016/06/07/persisting-a-cache-in-python-to-disk.html

我试过了，但是没有生成文件。

import atexit
import pickle
# or import cPickle as pickle

def persist_cache_to_disk(filename):
    def decorator(original_func):
        try:
            cache = pickle.load(open(filename, 'r'))
        except (IOError, ValueError):
            cache = {}

        atexit.register(lambda: pickle.dump(cache, open(filename, "w")))

        def new_func(*args):
            if tuple(args) not in cache:
                cache[tuple(args)] = original_func(*args)
            return cache[args]

        return new_func

    return decorator

我尝试按照示例使用此代码...

@persist_cache_to_disk('users.p')
def get_all_users():
    x = 'some user'
    return x

更新:

这在 python 命令提示符下有效，但在 ipython notebook 中无效。

Answer 1

问题是该示例使用了 atexit，它仅在 python 退出时运行转储例程。每次更新缓存时都会转储此修改后的版本:

import atexit
import functools
import pickle
# or import cPickle as pickle

def persist_cache_to_disk(filename):
    def decorator(original_func):
        try:
            cache = pickle.load(open(filename, 'r'))
        except (IOError, ValueError):
            cache = {}

        # Your python script has to exit in order to run this line!
        # atexit.register(lambda: pickle.dump(cache, open(filename, "w")))
        #
        # Let's make a function and call it periodically:
        #
        def save_data():                                                        
            pickle.dump(cache, open(filename, "w"))  

        # You should wrap your func
        @functools.wraps(original_func)
        def new_func(*args):
            if tuple(args) not in cache:
                cache[tuple(args)] = original_func(*args)
                # Instead, dump your pickled data after
                # every call where the cache is changed.
                # This can be expensive!
                save_data()
            return cache[args]

        return new_func

    return decorator


@persist_cache_to_disk('users.p')
def get_all_users():
    x = 'some user'
    return x

get_all_users()

如果你想限制保存，你可以修改 save_data() 只保存，比如说，当 len(cache.keys()) 是一个倍数时的 100。

我还向您的装饰器添加了 functools.wraps。来自docs :

Without the use of this decorator factory, the name of the example function would have been 'wrapper', and the docstring of the original example() would have been lost.