nlp - 词形还原与词干提取有什么区别？

Question

我什么时候使用每个？

另外...NLTK 词形还原是否依赖于词性？如果是这样不是更准确吗？

Answer 1

短而密:http://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html

The goal of both stemming and lemmatization is to reduce inflectional forms and sometimes derivationally related forms of a word to a common base form.

However, the two words differ in their flavor. Stemming usually refers to a crude heuristic process that chops off the ends of words in the hope of achieving this goal correctly most of the time, and often includes the removal of derivational affixes. Lemmatization usually refers to doing things properly with the use of a vocabulary and morphological analysis of words, normally aiming to remove inflectional endings only and to return the base or dictionary form of a word, which is known as the lemma .

来自 NLTK 文档:

Lemmatization and stemming are special cases of normalization. They identify a canonical representative for a set of related word forms.