huggingface-transformers - 果馅饼-t5-xxl : ValueError: Need either a `state_dict` or a `save_folder` containing offloaded weights

Question

我尝试在 Mac M1 和 Google Colab 中运行 Hugging Face 中的 flan-t5-xxx 模型，但都出现相同的错误:

ValueError:需要包含卸载权重的 state_dict 或 save_folder。

模型卡中的代码:

from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-xxl")
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto")
input_text = "translate English to German: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

Answer 1

对于需要的人:

创建一个文件夹(例如名称 save_folder)。然后更新:

model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto")

至

model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto", offload_folder="save_folder")