R 文本挖掘 - 处理复数

Question

我正在学习 R 中的文本挖掘，并取得了相当大的成功。但我坚持如何处理复数。即我希望“nation”和“nations”被算作同一个词，理想情况下“dictionary”和“dictionaries”被算作同一个词。

x <- '"nation" and "nations" to be counted as the same word and ideally "dictionary" and "dictionaries" to be counted as the same word.'

Answer 1

一种可能的解决方案。这里我使用 pacman 包使解决方案自包含:

if (!require("pacman")) install.packages("pacman"); library(pacman)
p_load_gh('hrbrmstr/pluralize')
p_load(quanteda)

x <- '"nation" and "nations" to be counted as the same word and ideally "dictionary" and "dictionaries"'
singularize(unlist(tokenize(x)))

##  [1] "\""         "nation"     "\""         "and"        "\""         "nation"     "\""        
##  [8] "to"         "be"         "counted"    "a"          "the"        "same"       "word"      
## [15] "and"        "ideally"    "\""         "dictionary" "\""         "and"        "\""        
## [22] "dictionary" "\""