r - 如何使用 quanteda::tokens_select() 删除单字符和双字符标记

Question

我正在尝试删除单字符和双字符标记。

这是一个例子:

toks <- tokens(c("This is a sentence. This is a second sentence."), remove_punct = TRUE)

toks <- tokens_select(toks, min_nchar=1L, max_nchar=2L, selection = "remove")

toks

结果:

tokens from 1 document. text1 :

[1] "is" "a" "is" "a"

我希望得到不符合条件的代币，而不是符合条件的代币。

Answer 1

library(quanteda)

toks <- tokens(c("This is a sentence. This is a second sentence."), remove_punct = TRUE)
tokens_select(toks, min_nchar=3L)