transformer-model - OpenAI GPT-3 API : Fine tune a fine tuned model?

Question

微调 API 中关于 model 属性的 OpenAI 文档说明有点令人困惑:

model

The name of the base model to fine-tune. You can select one of "ada", "babbage", "curie", "davinci", or a fine-tuned model created after 2022-04-21.

我的问题:微调基础模型好还是微调模型好？

我使用文件 mydata1K.jsonl 从 ada 创建了一个微调模型:

ada + mydata1K.jsonl --> ada:ft-acme-inc-2022-06-25

现在我有一个更大的示例文件 mydata2K.jsonl，我想用它来改进微调模型。在这第二轮微调中，是再次微调 ada 还是微调我的微调模型 ada:ft-acme-inc-2022-06- 25？我假设这是可能的，因为我的微调模型是在 2022 年 4 月 21 日之后创建的。

ada + mydata2K.jsonl --> better-model

或

ada:ft-acme-inc-2022-06-25 + mydata2K.jsonl --> even-better-model?

Answer 1

如果您阅读 Fine-tuning documentation唯一谈论“微调微调模型”的部分是高级用法下的以下部分:

Continue fine-tuning from a fine-tuned model

If you have already fine-tuned a model for your task and now haveadditional training data that you would like to incorporate, you cancontinue fine-tuning from the model. This creates a model that haslearned from all of the training data without having to re-train fromscratch.

To do this, pass in the fine-tuned model name when creating a newfine-tuning job (e.g., -m curie:ft-<org>-<date>). Other trainingparameters do not have to be changed, however if your new trainingdata is much smaller than your previous training data, you may find ituseful to reduce learning_rate_multiplier by a factor of 2 to 4.

选择哪个选项？

你问的是两个选项:

• 选项 1:ada + bigger-training-dataset.jsonl
• 选项 2:ada:ft-acme-inc-2022-06-25 + additional-training-dataset.jsonl

文档没有说明哪个选项更好会产生更好的结果。

然而...

选择选项 2

为什么？

When training a fine-tuned model, the total tokens used will be billedaccording to our training rates.

如果您选择选项 1，您将为训练数据集中的某些代币支付两次费用。首先是对初始训练数据集进行微调，其次是对更大的训练数据集进行微调(即 bigger-training-dataset.jsonl = initial-training-dataset.jsonl + additional-training-dataset.jsonl )。

最好从经过微调的模型继续微调，因为您只需为附加训练数据集中的 token 付费。

阅读更多关于 fine-tuning pricing calculation 的信息.