python - 类型错误 : can't pickle generator objects

Question

我正在尝试将一些结果写入 pickle 文件，如下所示:

raw_X = (self.token_ques(text) for text in training_data)
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
    pickle.dump(raw_X, handle)

错误:

    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle generator objects

任何帮助将不胜感激。

Answer 1

当您想要 pickle 数据时，不要使用生成器表达式。请改用列表理解，或在生成器上调用 list() 以捕获所有生成的元素以进行 pickle 。

例如，以下工作正常:

raw_X = [self.token_ques(text) for text in training_data]
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
    pickle.dump(raw_X, handle)

同样:

raw_X = (self.token_ques(text) for text in training_data)
with open('/root/Desktop/classifier_result.pkl', 'wb') as handle:
    pickle.dump(list(raw_X), handle)