openai-api - 如何限制机器人仅回答与文档相关的问题

Question

我正在使用 langchain/openai/faiss 来创建聊天机器人，该机器人可以读取所有 PDF，并可以根据从其中学到的内容进行回答。

我想知道是否有一种方法可以限制仅从文档中获取知识的答案，如果答案不在文档中，机器人应该回答“我不知道”或类似的内容。

这是代码:

 llm = ChatOpenAI(temperature=0, max_tokens=1000,
                         model_name="gpt-3.5-turbo-16k")
    memory = ConversationBufferMemory(memory_key="chat_history")
    chat = ConversationalRetrievalChain.from_llm(
        llm=llm,retriever=vector_store.as_retriever(),memory=memory)
    
    if "messages" not in st.session_state:
        st.session_state.messages = []

    if not st.session_state.messages:
        welcome_message = {"role": "assistant",
                           "content": "Hello, how can i help?"}
        st.session_state.messages.append(welcome_message)

    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])


    if prompt := st.chat_input("State your question"):
        st.session_state.messages.append({"role": "user", "content": prompt})
        with st.chat_message("user"):
            st.markdown(prompt)
        result = chat({"question": prompt, "chat_history": [
                    (message["role"], message["content"]) for message in st.session_state.messages]})

        with st.chat_message("assistant"):
            full_response = result["answer"]
            st.markdown(full_response)

        st.session_state.messages.append(
            {"role": "assistant", "content": full_response})
        

Answer 1

是的，有。但首先，请记住，聊天机器人不会从您加载到矢量存储中的 PDF 文件文本中“学习”任何内容；它会比较用户的问题嵌入，以找到与矢量存储中的文本嵌入最相似的匹配项。

为了让您开始使用具有对话功能的文档 QA 聊天机器人，我建议您尝试此代理:

from langchain.chat_models import ChatOpenAI
from langchain.agents.openai_functions_agent.agent_token_buffer_memory import AgentTokenBufferMemory
from langchain.agents.openai_functions_agent.base import OpenAIFunctionsAgent
from langchain.schema.messages import SystemMessage
from langchain.prompts import MessagesPlaceholder
from langchain.agents import AgentExecutor

retriever = vector_store.as_retriever()

tool = create_retriever_tool(
    retriever, 
    "search_document_content",
    "Useful for searching and querying the relevant content."
    )

tools = [tool]

memory_key = "history"

memory = AgentTokenBufferMemory(memory_key = memory_key, llm = llm)

system_message = SystemMessage(
        content=("""Use only the tools provided to look for context to answer 
        the user's question. If you don't know the answers to the user 
        questions, truthfully say you don't know. Don't attempt to make up 
        answers or hallucinate.""")
        )

prompt = OpenAIFunctionsAgent.create_prompt(
        system_message=system_message,
        extra_prompt_messages = [MessagesPlaceholder(variable_name = memory_key)]
        )

agent = OpenAIFunctionsAgent(llm = llm, tools = tools, prompt = prompt)

agent_executor = AgentExecutor(
        agent = agent,
        tools = tools,
        memory = memory,
        verbose = True,
        return_intermediate_steps = True
        )

result = agent_executor({"input": "<This is your PDF-related question>"})

看here有关使用检索器进行 QA 的更简单方法的更多信息。