r - 来自 C 堆栈溢出的段错误

Question

我正在研究数据帧转换，并在上一篇文章中与 Arun 和 Ricardo 合作

Previous Post

Arun，提出了一个绝妙的解决方案(矩阵乘法)来实现我正在尝试做的事情。

该解决方案适用于我在示例中提到的小型数据集，现在我在具有以下大小的数据框上运行相同的解决方案:

Total rows: 143345
Total Persons: 98461
Total Items :  30

现在，当我运行以下命令时

 A <- acast(Person~Item+BorS,data=df,fun.aggregate=length,drop=FALSE)

我收到这个错误..

Error: segfault from C stack overflow

这是因为，我没有足够的处理/内存能力。我的机器有 4 GB RAM、2.8 GHz i7 处理器(Macbook)？我们如何处理此类案例？

Answer 1

data.table 解决方案。这是通过首先聚合，然后创建新的 data.table 并通过引用填写

library(data.table)

# some sample data
DT <- data.table(Person = sample(98461, 144000, replace = TRUE), item = sample(c(letters,LETTERS[1:4]), 144000, replace = TRUE), BorS = sample(c('B','S'), 144000, replace = TRUE))
# aggregate to get the number of rows in each subgroup by list item and BorS 
# the `length` of each subgroup
DTl <- DT[,.N , by = list(Person, item, BorS)]
# the columns you want to create
newn <- sort(DT[, do.call(paste0,do.call(expand.grid,list(unique(item),unique(BorS) )))])
# create a column which has this id combination in DTl
DTl[, comnb := paste0(item, BorS)]
# set the key so we can join / subset easily
setkey(DTl, comnb)
# create a data.table that has 1 row for each person, and has  columns for all the combinations
# of item and BorS
DTb <- DTl[, list(Person)][, c(newn) := 0L]
# set the key so we can join / subset easily
setkey(DTb, Person)
# this bit could be far quicker, but I think
# would require a feature request to data.table
for(nn in newn){
  # for each of the cominations extract which persons have
  # this combination >0
  pp <- DTl[list(nn), list(Person,N)]
  # for the people who have N > 0
  # assign the correct numbers in the correct column in DTb
  DTb[list(pp[['Person']]), c(nn) := pp[['N']]]
}

要完成您的初始问题，您可以从 DTb 中提取适当的列作为矩阵

A <- DTb[,-1,with = FALSE]

results <- crossprod(A)