r - {: task 1 failed - “error returned from C call” using ncvar_get (ncdf4 package) within foreach loop中的错误

Question

我正在尝试从.nc文件提取数据。由于我的文件中有7个变量，因此我想使用foreach在所有7个变量之间循环ncvar_get函数。

这是我的代码:

 # EXTRACTING CLIMATE DATA FROM NETCDF4 FILE

library(dplyr)
library(data.table)
library(lubridate)
library(ncdf4)
library(parallel)
library(foreach)
library(doParallel)

# SET WORKING DIRECTORY
setwd('/storage/hpc/data/htnb4d/RIPS/UW_climate_data/')

# SETTING UP
cores <- detectCores()
cl <- makeCluster(cores) 
registerDoParallel(cl)

# READING INPUT FILE
infile <- nc_open("force_SERC_8th.1979_2016.nc")
vars <- attributes(infile$var)$names
climvars <- vars[1:7]

# EXTRACTING INFORMATION OF STUDY DOMAIN:
tab <- read.csv('SDGridArea.csv', header = T)
point <- sort(unique(tab$PointID)) #6013 points in the study area

# EXTRACTING DATA (P, TMAX, TMIN, LW, SW AND RH):
clusterEvalQ(cl, {
  library(ncdf4)
})
clusterExport(cl, c('infile','climvars','point'))
foreach(i = climvars) %dopar% {
   climvar <- ncvar_get(infile, varid = i) # all data points 13650 points
   dim <- dim(climvar)
   climMX <- aperm(climvar,c(3,2,1))
   dim(climMX) <- c(dim[3],dim[1]*dim[2])
   climdt <- data.frame(climMX[,point]) #getting 6013 points in the study area
   write.table(climdt,paste0('SD',i,'daily.csv'), sep = ',', row.names = F)
}
stopCluster(cl)

错误是:

Error in { : task 1 failed - "error returned from C call"
Calls: %dopar% -> <Anonymous>
Execution halted

您能解释一下这段代码有什么问题吗？我认为这与集群无法从文件中获取哪个变量有关，因为“从C调用返回的错误”通常来自ncvar_get varid参数。

Answer 1

我在MacBook Pro(OSX 10.12.5)上运行类似的R脚本时遇到相同的问题(错误消息相同)。问题似乎是来自foreach循环的不同工作程序尝试使用ncvar_get同时访问相同的.nc文件。这可以通过在foreach循环外使用ncvar_get(将所有数据存储在一个大数组中)并从foreach循环内访问该数组来解决。