openai-api - OpenAI GPT-4 API : Why does gpt-4-0613 hallucinate (make up) function parameters?

Question

我使用的是 gpt-4-0613 模型，具有单个函数，并在系统提示符中包含一些自定义数据。

如果该函数在聊天中很早就被触发，在前两个请求内，它的功能就很好，并且 API 会要求用户提供调用该函数所需的信息。

但是，如果稍后在对话中调用该函数，例如问题 5，则 API 将仅给出答案并发回函数调用。

如何阻止人工智能编造答案？ API 无法从对话上下文中获取这些值。它们都是 100% 化妆的。

completion = openai.ChatCompletion.create(
    model='gpt-4-0613',
    messages=prompts,
    functions=[
    {
        "name": "fill_form",
        "description": "Helps the user create an XYZ Report",
        "parameters": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string",
                    "description": "the full name of the person issuing this report"
                },
                "zip": {
                    "type": "string",
                    "description": "the 5 digit zip code of the address"
                },
                "address": {
                    "type": "string",
                    "description": "the street address, only the street and not the city, state or zip"
                },
                "year_end": {
                    "type": "string",
                    "description": "the full four digit year of the fiscal year"
                },
            },
            "required": ["name", "address", "year_end", "zip"]
        }
    }],
)

我尝试过使用和不使用

function_call='auto'

没有影响的选项。

感谢您的帮助。

API 应始终询问用户函数的值，而不是虚构它们。

Answer 1

OpenAI 承认模型可能会幻觉(即弥补)函数参数，如官方 OpenAI documentation 中所述。 :

The basic sequence of steps for function calling is as follows:

1. Call the model with the user query and a set of functions defined in the functions parameter.
2. The model can choose to call a function; if so, the content will be a stringified JSON object adhering to your custom schema (note: themodel may generate invalid JSON or hallucinate parameters).
3. Parse the string into JSON in your code, and call your function with the provided arguments if they exist.
4. Call the model again by appending the function response as a new message, and let the model summarize the results back to the user.

这可以通过系统消息来缓解，如官方 OpenAI documentation 中所述。 :

Hallucinated outputs in function calls can often be mitigated with asystem message. For example, if you find that a model is generatingfunction calls with functions that weren't provided to it, try using asystem message that says: "Only use the functions you have beenprovided with."

通常，对话首先使用系统消息进行格式化，然后是交替的用户和助理消息，如官方 OpenAI documentation 中所述。 。您可以添加系统消息，如下所示:

Python

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")

chatCompletion = openai.ChatCompletion.create(
  model = "gpt-3.5-turbo-0613",
  messages = [
        {"role": "system", "content": "Only use the functions you have been provided with."}
    ]
)

print(chatCompletion.choices[0].message.content)

NodeJS

注意:OpenAI NodeJS SDK v4是released于2023年8月16日发布，并且是对SDK的完全重写。下面的代码根据您当前拥有的版本而有所不同。请参阅v3至v4 migration guide .

• 如果您有 OpenAI NodeJS SDK v3 :

const { Configuration, OpenAIApi } = require("openai");

const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});

const openai = new OpenAIApi(configuration);

const chatCompletion = await openai.createChatCompletion({
  model: "gpt-3.5-turbo-0613",
  messages: [{"role": "system", "content": "Only use the functions you have been provided with."}],
});

console.log(chatCompletion.data.choices[0].message.content);

• 如果您有 OpenAI NodeJS SDK v4 :

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const chatCompletion = await openai.chat.completions.create({
  model: "gpt-3.5-turbo-0613",
  messages: [{"role": "user", "content": "Only use the functions you have been provided with."}],
});

console.log(chatCompletion.choices[0].message.content);

此外，尝试通过设置 function_call: {"name": "<insert-function-name>"} 强制模型调用特定函数。正如官方所说OpenAI documentation :

就您而言，它看起来像这样:

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model = "gpt-3.5-turbo-0613",
    messages = messages,
    functions = functions,
    function_call = "fill_form",  # Add this
)

print(completion.choices[0].message.content)