r - quanteda:带有新文本和旧词汇的 dtm

Question

我使用 quanteda 来构建文档术语矩阵:

library(quanteda)
mytext = "This is my old text"
dtm <- dfm(mytext, tolower=T)
convert(dtm,to="data.frame")

其中产生:

  doc_id this is my old text
1  text1    1  1  1   1    1

我需要将"new"文本(一个新的语料库)与我现有的 dtm 匹配(使用相同的词汇表，以便出现相同的矩阵列)
假设我的"new"文本/语料库是:

newtext = "This is my new text"

如何将这个"new"文本/语料库与现有的 dtm 词汇表相匹配，从而获得如下矩阵:

  doc_id this is my old text
1  text1    1  1  1   0    1

Answer 1

你要dfm_match() , 在转换为 data.frame 之前。

library(quanteda)
## Package version: 2.1.2

mytext <- c(oldtext = "This is my old text")
dtm_old <- dfm(mytext)
dtm_old
## Document-feature matrix of: 1 document, 5 features (0.0% sparse).
##          features
## docs      this is my old text
##   oldtext    1  1  1   1    1

newtext <- c(newtext = "This is my new text")
dtm_new <- dfm(newtext)
dtm_new
## Document-feature matrix of: 1 document, 5 features (0.0% sparse).
##          features
## docs      this is my new text
##   newtext    1  1  1   1    1

要匹配它们，请使用 dfm_match()使新的 dfm 符合旧的功能集和顺序:

dtm_matched <- dfm_match(dtm_new, featnames(dtm_old))
dtm_matched
## Document-feature matrix of: 1 document, 5 features (20.0% sparse).
##          features
## docs      this is my old text
##   newtext    1  1  1   0    1

convert(dtm_matched, to = "data.frame")
##    doc_id this is my old text
## 1 newtext    1  1  1   0    1