python - 如何从 nltk 分类器中获取精度和召回率？

Question

import nltk
from nltk.corpus import movie_reviews
from nltk.tokenize import word_tokenize

documents = [(list(movie_reviews.words(fileid)), category)
             for category in movie_reviews.categories()
             for fileid in movie_reviews.fileids(category)]


all_words = []

for w in movie_reviews.words():
    all_words.append(w.lower())

all_words = nltk.FreqDist(all_words)

word_features = list(all_words.keys())[:3000]

def find_features(document):
    words = set(document)
    features = {}
    for w in word_features:
        features[w] = (w in words)

    return features

featuresets = [(find_features(rev), category) for (rev, category) in documents]

training_set = featuresets[500:1500]
testing_set = featuresets[:1500]

classifier = nltk.DecisionTreeClassifier.train(training_set)

print "Classifier accuracy percent:",(nltk.classify.accuracy(classifier, testing_set))*100 , "%"

string = raw_input("Enter the string: ")
print (classifier.classify(find_features(word_tokenize(string))))

此代码将显示分类器的准确性，然后获取用户的输入。并返回用户输入字符串的极性。

但这是我的问题:既然我可以通过使用 nltk.accuracy() 获得准确率，是否有可能同时获得它的准确率和召回率？

Answer 1

如果您使用的是 nltk 包，那么您似乎可以使用 nltk.metrics.scores 中的 recall 和 precision 函数(see the docs)。

调用后函数应该可用

from nltk.metrics.scores import (precision, recall)

然后您需要使用reference(已知标签)和test(您的分类器在测试集上的输出)集来调用它们。

类似于下面的代码应该将这些集合生成为 refsets 和 testsets

refsets = collections.defaultdict(set)
testsets = collections.defaultdict(set)

for i, (feats, label) in enumerate(testing_set):
    refsets[label].add(i)
    observed = classifier.classify(feats)
    testsets[observed].add(i)

然后，您可以使用类似

的方式查看积极预测的准确率和召回率

print( 'Precision:', nltk.metrics.precision(refsets['pos'], testsets['pos']) )
print( 'Recall:', nltk.metrics.recall(refsets['pos'], testsets['pos']) )
# `'pos'` is for the "positive" (as opposed to "negative") label