python - tfidfvectorizer 根据所有单词打印结果

Question

虽然有六个不同的词。结果只打印了5个字。如何根据所有单词(6列向量)获得结果？

from sklearn.feature_extraction.text import TfidfVectorizer
sent=["This is a sample", "This is another example"]
tf = TfidfVectorizer(analyzer='word', ngram_range=(1,1), min_df = 0)
tfidf_matrix =  tf.fit_transform(sent)
print tfidf_matrix.toarray()

[[ 0. 0. 0.50154891 0.70490949 0.50154891] [ 0.57615236 0.57615236 0.40993715 0. 0.40993715]]

另外如何打印列详细信息(特征(单词))和行(文档)？

Answer 1

您正在使用默认的 token_pattern，它仅选择 2 个或更多字符的标记。

token_pattern :

“token”, only used if analyzer == 'word'. The default regexp selects tokens of 2 or more alphanumeric characters (punctuation is completely ignored and always treated as a token separator)

如果定义新的 token_pattern，您将获得“a”字符，例如:

from sklearn.feature_extraction.text import TfidfVectorizer
sent=["This is a sample", "This is another example"]
tf = TfidfVectorizer(token_pattern=u'(?u)\\b\\w+\\b')
tfidf_matrix =  tf.fit_transform(sent)
print tfidf_matrix.toarray()
tf.vocabulary_

[[ 0.57615236 0.0.0.40993715 0.57615236 0.40993715] [0.0.57615236 0.57615236 0.40993715 0.0.40993715]]

tf.vocabulary_

{u'a': 0、u'sample': 4、u'another': 1、u'this': 5、u'is': 3、u'example': 2}