openai-api - 如何解决 "Models maximum content length exceeded"错误

Question

我正在使用 LangChain 和 OpenAI 与我的 postgres 数据库交互。它适用于小型数据库。但是当数据库较大时(比如超过 10K 行)，我会收到以下错误

此模型的最大上下文长度为 4097 个标记。然而，您的消息导致了 5100 个 token 。请缩短消息长度

显示此错误的公共(public)数据库示例是 MindsDB 房地产数据库:

postgresql+psycopg2://demo_user:demo_password@REDACTED:5432/demo

我的代码如下...此 python 代码可以连接到任何 postgres 数据库并允许您与其聊天，但如上面所述，在较大的数据库上它会失败

import sys
from langchain import OpenAI
from langchain import SQLDatabase
from langchain.chat_models import ChatOpenAI
from langchain_experimental.sql import SQLDatabaseChain

import environ
env = environ.Env()
environ.Env.read_env()

API_KEY = env('OPENAI_API_KEY')

if API_KEY == "":
    print("Missing OpenAPI key")
    exit()

if len(sys.argv) < 2:
    print("Missing db connection string.  Example 'postgresql+psycopg2://postgres:1234@localhost:6667/mydb'")
    exit()

dbstring = sys.argv[1]

print("Using OpenAPI with key ["+API_KEY+"] and Database ["+dbstring+"]")

# Setup database
db = SQLDatabase.from_uri(
    dbstring,
)

# setup llm
llm = ChatOpenAI(model_name="gpt-3.5-turbo",
    temperature=0,
    max_tokens=1000,
    openai_api_key=API_KEY)

# Create db chain
QUERY = """
Given an input question, first create a syntactically correct postgresql query to run, then look at the results of the query and return the answer.
Use the following format:

Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here

{question}
"""

# Setup the database chain
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)

def get_prompt():
    print("Type 'exit' to quit")

    while True:
        prompt = input("Enter a prompt: ")

        if prompt.lower() == 'exit':
            print('Exiting...')
            break
        else:
            try:
                question = QUERY.format(question=prompt)
                print(db_chain.run(question))
            except Exception as e:
                print(e)

get_prompt()

Answer 1

您可以使用此 split_query 方法对数据进行分块。您可以根据您的要求对其进行修改。此外，您还必须确保在对数据进行分块时不会丢失上下文详细信息。您可以尝试几个选项(拆分或使用最大 token )，以便它适合您的用例

        def split_query(query, max_tokens):

            tokens = query.split()
            chunks = []
            chunk = []
            for token in tokens:
                if len(chunk) + len(token) > max_tokens:
                    chunks.append(" ".join(chunk))
                    chunk = []
                chunk.append(token)
            if chunk:
                chunks.append(" ".join(chunk))
            return chunks

        def get_prompt():
            print("Type 'exit' to quit")

            while True:
                prompt = input("Enter a prompt: ")

                if prompt.lower() == 'exit':
                    print('Exiting...')
                    break
                else:
                    try:
                        query = prompt
                        chunks = split_query(query, 4097)
                        for chunk in chunks:
                            question = QUERY.format(question=chunk)
                            print(db_chain.run(question))
                    except Exception as e:
                        print(e)

        get_prompt()