python - 使用 Pickle 文件进行主题分类。 Python

Question

我正在尝试使用训练模型的 pickle 文件进行主题分类，但我遇到了错误“CountVectorizer - 词汇未安装”。有人可以指导我如何解决此错误吗？

训练数据集格式:

Topic   originalSentence 
Topic1  He has arrived with his sister's two young children.
Topic2  The Lowells have been living off the Colby fortune
Topic3  Fred and Janice Gage, who live off the Lowell  fortune, which would have gone to Alan Colby

我的训练代码:

import pandas as pd
from io import StringIO
from sklearn.feature_extraction.text import TfidfVectorizer,TfidfTransformer,CountVectorizer
from sklearn.model_selection import train_test_split
import numpy as np
import pickle

def train_model():
df = pd.read_csv('/Users/ra51646/Desktop/classification_training.csv')
df = df[pd.notnull(df['originalSentence'])]
df.columns = ['topic', 'originalSentence']
df['category_id'] = df['topic'].factorize()[0]
category_id_df = df[['topic', 'category_id']].drop_duplicates().sort_values('category_id')
category_to_id = dict(category_id_df.values)
id_to_category = dict(category_id_df[['category_id', 'topic']].values)
tfidf = TfidfVectorizer(sublinear_tf=True, min_df=5, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')
features = tfidf.fit_transform(df.originalSentence).toarray()
labels = df.category_id
X_train, X_test, y_train, y_test = train_test_split(df['originalSentence'], df['topic'], random_state = 0)
count_vect = CountVectorizer()
X_train_counts = count_vect.fit_transform(X_train)
tfidf_transformer = TfidfTransformer()
X_train_tfidf = tfidf_transformer.fit_transform(X_train_counts)
clf_SGD = SGDClassifier().fit(X_train_tfidf, y_train)
clf_inc = Incremental(clf_SGD)
final_model = clf_inc.fit(X_train_tfidf, y_train,classes=np.unique(y_train))
pickle.dump(final_model, open("/Users/ra51646/Desktop/Pickle/topic_classification.pkl","wb"))

(待解决的错误)我使用pickle文件进行主题分类的代码:

def find_topic1():
model = pickle.load(open("/Users/ra51646/Desktop/Pickle/topic_classification.pkl","rb"))
count_vect = CountVectorizer()
answer = model.predict(count_vect.transform(["Lindy and her family went camping in the Outback"]))
print(answer[0])
return answer

我收到错误NotFittedError:CountVectorizer - 词汇未安装。在 find_topic 方法中。请帮我解决这个错误。如何使用我的 pickle 文件(训练模型)进行主题分类。

Answer 1

您可能缺少 CountVectorizer 的参数，该参数使 count_vect 变量独立于 pickled 模型，从而导致错误。没有 MCVE无法确定。