r - 我应该如何从 R data.table 中删除 NA block

Question

我有一个大 R data.table使用多列键，其中一些值列包含一些 NA。我想删除一个或多个值列中完全不适用的组，否则保留整个组。对键的每一列重复此操作。

举个简单的例子:

library(data.table)
DT = data.table(
    Series = rep(letters[1:12], each = 3), 
    Id = 1:12,
    Value1 = c(1:3, NA, 5:9, rep(NA,3), 1:3, NA, 5:9, rep(NA,3), 1:3, NA, 5:9, rep(NA,3)), 
    Value2 = c(rep(NA,3), 1:4, NA, 6:9, rep(NA,3), 1:9, 1:9, rep(NA,3)))
DT
    Series Id Value1 Value2
 1:      a  1      1     NA
 2:      a  2      2     NA
 3:      a  3      3     NA
 4:      b  4     NA      1
 5:      b  5      5      2
 6:      b  6      6      3
 7:      c  7      7      4
 8:      c  8      8     NA
 9:      c  9      9      6
10:      d 10     NA      7
11:      d 11     NA      8
12:      d 12     NA      9
13:      e  1      1     NA
14:      e  2      2     NA
15:      e  3      3     NA
16:      f  4     NA      1
17:      f  5      5      2
18:      f  6      6      3
19:      g  7      7      4
20:      g  8      8      5
21:      g  9      9      6
22:      h 10     NA      7
23:      h 11     NA      8
24:      h 12     NA      9
25:      i  1      1      1
26:      i  2      2      2
27:      i  3      3      3
28:      j  4     NA      4
29:      j  5      5      5
30:      j  6      6      6
31:      k  7      7      7
32:      k  8      8      8
33:      k  9      9      9
34:      l 10     NA     NA
35:      l 11     NA     NA
36:      l 12     NA     NA
    Series Id Value1 Value2

所以我想放弃:

系列:a、d、e、h 和 l

ID:4、10、11 和 12

正确的结果应该是这样的:

    Series Id Value1 Value2
 1:      b  5      5      2
 2:      b  6      6      3
 3:      c  7      7      4
 4:      c  8      8     NA
 5:      c  9      9      6
 6:      f  5      5      2
 7:      f  6      6      3
 8:      g  7      7      4
 9:      g  8      8      5
10:      g  9      9      6
11:      i  1      1      1
12:      i  2      2      2
13:      i  3      3      3
14:      j  5      5      5
15:      j  6      6      6
16:      k  7      7      7
17:      k  8      8      8
18:      k  9      9      9
    Series Id Value1 Value2

到目前为止我管理的内容:

我可以像这样找到 Value1 不适用的系列:

DT[, sum(1-is.na(Value1)) == 0, by = Series][V1 == TRUE]

我什至可以做

setkey(DT, Series)
DT = DT[DT[, sum(1-is.na(Value)) == 0, by = Series][V1 != TRUE]]

但现在我最终以 V1 出现在决赛 table 中。

Answer 1

您可以执行此操作以获取不是 ALL Value 的条目。是 NA :

setkey(DT, "Series")
DT[, .SD[(!all(is.na(Value)))], by=Series]

周围的括号 !all需要避免 Matthew 将研究的 not-join 语法(见评论)。与此相同:

DT[, .SD[as.logical(!all(is.na(Value)))], by=Series]

在此基础上回答新的澄清问题:

allNA = function(x) all(is.na(x))     # define helper function
for (i in c("Id","Series"))
    DT = DT[, if (!any(sapply(.SD,allNA))) .SD else NULL, by=i]
DT
    Series Id Value1 Value2
 1:      i  1      1      1
 2:      i  2      2      2
 3:      i  3      3      3
 4:      b  5      5      2
 5:      b  6      6      3
 6:      f  5      5      2
 7:      f  6      6      3
 8:      j  5      5      5
 9:      j  6      6      6
10:      c  7      7      4
11:      c  8      8     NA
12:      c  9      9      6
13:      g  7      7      4
14:      g  8      8      5
15:      g  9      9      6
16:      k  7      7      7
17:      k  8      8      8
18:      k  9      9      9

不过，这改变了顺序。所以不是所要求的结果。以下保持顺序，也应该更快。

# starting fresh from original DT in question again
DT[,drop:=FALSE]
for (i in c("Series","Id"))
    DT[,drop:=drop|any(sapply(.SD,allNA)),by=i]
DT[(!drop)][,drop:=NULL][]
    Series Id Value1 Value2
 1:      b  5      5      2
 2:      b  6      6      3
 3:      c  7      7      4
 4:      c  8      8     NA
 5:      c  9      9      6
 6:      f  5      5      2
 7:      f  6      6      3
 8:      g  7      7      4
 9:      g  8      8      5
10:      g  9      9      6
11:      i  1      1      1
12:      i  2      2      2
13:      i  3      3      3
14:      j  5      5      5
15:      j  6      6      6
16:      k  7      7      7
17:      k  8      8      8
18:      k  9      9      9