r - 具有 613 列调查数据的稀疏 132MB 文件上的 fread segfault-6ren

r - 具有 613 列调查数据的稀疏 132MB 文件上的 fread segfault

转载作者：行者123 更新时间：2023-12-04 20:08:57

我一直在学习data.table最近。
但是，当我使用 fread 时要从“http://dl.dropbox.com/u/20498362/GSS.csv”读取数据，R 会因段错误而崩溃。我该如何进一步调查？要重现只需下载文件并键入:

fread("GSS.csv")

该文件有许多 NA 变量；第一列也缺少列名。但是，如果我添加“rownames=TRUE”，它仍然不起作用。

谢谢!

最佳答案

更新 :现在已在 CRAN 上的 v1.9.4 中修复。

上一个回答...

非常感谢可重现的示例!我也看到了崩溃。极好的!!

让我们打开 verbose=TRUE获取更多线索...

$ R
R version 3.0.2 (2013-09-25) -- "Frisbee Sailing"
Copyright (C) 2013 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

> require(data.table)
Loading required package: data.table
data.table 1.8.10  For help type: help("data.table")

> fread("GSS.csv", verbose=TRUE)
Detected eol as \n only (no \r afterwards), the UNIX and Mac standard.
Using line 30 to detect sep (the last non blank line in the first 'autostart') ... sep=','
Found 613 columns
First row with 613 fields occurs on line 1 (either column names or first row of data)
All the fields on line 1 are character fields. Treating as the column names.
Count of eol after first data row: 55088
Subtracted 1 for last eol and any trailing empty lines, leaving 55087 data rows
Type codes: 3002000030033030000033003000000033000300330000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003330003330000000000000000000000000000000000000000000000000003330000000000000003000303000000000000000000000000000000000033000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030000000000000000000000303 (first 5 rows)
Type codes: 3002000030033030000033003330000033032300333300000033000033330000000000000000000000000000000000000000000000000003300003333333330000000000000000000000300030000000000000000000000000000000000000000000000000000000000000000003333300003330000000033000000000000000000000000000000000000000000000000000000000000000000000000333000000000000300000003333333330000000000000000000000000000000000000000000000000003332000000000000003303333000000000000000003330000003000000333333333333333333333333300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030033333330300000000000333 (+middle 5 rows)
Type codes: 3002000033033033000033003333000033032300333300000033000033333330000000000000000000000000000000000000000000000003300003333333330000000000000000000000300030000000000000300000000000000000000000000000000000000000000000000003333300003330000000033000000000000000000000000000000000000000000000000000000000000000000000000333000000000000300000003333333330000000000000000000000000000000000000000000000000003332200000300033003303333000000000000000003330003333000000333333333333333333333333300000000000000000000000030030000000000000000000000000000000000000000000000000000000000000000000000000000000030033333330300000000000333 (+last 5 rows)
Bumping column 39 from INT to INT64 on data row 1614, field contains '"working class"'
Bumping column 39 from INT64 to REAL on data row 1614, field contains '"working class"'
Bumping column 39 from REAL to STR on data row 1614, field contains '"working class"'
Bumping column 225 from INT to INT64 on data row 1614, field contains '"disagree"'
Bumping column 225 from INT64 to REAL on data row 1614, field contains '"disagree"'
Bumping column 225 from REAL to STR on data row 1614, field contains '"disagree"'
Bumping column 226 from INT to INT64 on data row 1614, field contains '"disagree"'
Bumping column 226 from INT64 to REAL on data row 1614, field contains '"disagree"'
Bumping column 226 from REAL to STR on data row 1614, field contains '"disagree"'
Bumping column 227 from INT to INT64 on data row 1614, field contains '"disagree"'
Bumping column 227 from INT64 to REAL on data row 1614, field contains '"disagree"'
Bumping column 227 from REAL to STR on data row 1614, field contains '"disagree"'
Bumping column 228 from INT to INT64 on data row 1614, field contains '"disagree"'
Bumping column 228 from INT64 to REAL on data row 1614, field contains '"disagree"'
Bumping column 228 from REAL to STR on data row 1614, field contains '"disagree"'
Bumping column 232 from INT to INT64 on data row 1614, field contains '"agree"'
Bumping column 232 from INT64 to REAL on data row 1614, field contains '"agree"'
Bumping column 232 from REAL to STR on data row 1614, field contains '"agree"'
Bumping column 233 from INT to INT64 on data row 1614, field contains '"agree"'
Bumping column 233 from INT64 to REAL on data row 1614, field contains '"agree"'
Bumping column 233 from REAL to STR on data row 1614, field contains '"agree"'
Bumping column 307 from INT to INT64 on data row 1614, field contains '"no"'
Bumping column 307 from INT64 to REAL on data row 1614, field contains '"no"'
Bumping column 307 from REAL to STR on data row 1614, field contains '"no"'
Bumping column 308 from INT to INT64 on data row 1614, field contains '"no"'
Bumping column 308 from INT64 to REAL on data row 1614, field contains '"no"'
Bumping column 308 from REAL to STR on data row 1614, field contains '"no"'
Bumping column 309 from INT to INT64 on data row 1614, field contains '"no"'
Bumping column 309 from INT64 to REAL on data row 1614, field contains '"no"'
Bumping column 309 from REAL to STR on data row 1614, field contains '"no"'
Bumping column 310 from INT to INT64 on data row 1614, field contains '"no"'
Bumping column 310 from INT64 to REAL on data row 1614, field contains '"no"'
Bumping column 310 from REAL to STR on data row 1614, field contains '"no"'
Bumping column 311 from INT to INT64 on data row 1614, field contains '"no"'
Bumping column 311 from INT64 to REAL on data row 1614, field contains '"no"'
Bumping column 311 from REAL to STR on data row 1614, field contains '"no"'
Bumping column 3 from INT to INT64 on data row 9121, field contains '2.54999995231628'
Bumping column 3 from INT64 to REAL on data row 9121, field contains '2.54999995231628'
Bumping column 234 from INT to INT64 on data row 9121, field contains '"not feel"'
Bumping column 234 from INT64 to REAL on data row 9121, field contains '"not feel"'
Bumping column 234 from REAL to STR on data row 9121, field contains '"not feel"'
Bumping column 235 from INT to INT64 on data row 9121, field contains '"feel"'
Bumping column 235 from INT64 to REAL on data row 9121, field contains '"feel"'
Bumping column 235 from REAL to STR on data row 9121, field contains '"feel"'
Bumping column 236 from INT to INT64 on data row 9121, field contains '"feel"'
Bumping column 236 from INT64 to REAL on data row 9121, field contains '"feel"'
Bumping column 236 from REAL to STR on data row 9121, field contains '"feel"'
Bumping column 237 from INT to INT64 on data row 9121, field contains '"not feel"'
Bumping column 237 from INT64 to REAL on data row 9121, field contains '"not feel"'
Bumping column 237 from REAL to STR on data row 9121, field contains '"not feel"'
Bumping column 238 from INT to INT64 on data row 9121, field contains '"feel"'
Bumping column 238 from INT64 to REAL on data row 9121, field contains '"feel"'
Bumping column 238 from REAL to STR on data row 9121, field contains '"feel"'
Bumping column 239 from INT to INT64 on data row 9121, field contains '"feel"'
Bumping column 239 from INT64 to REAL on data row 9121, field contains '"feel"'
Bumping column 239 from REAL to STR on data row 9121, field contains '"feel"'
Bumping column 2 from INT to INT64 on data row 12121, field contains '1.23500001430511'
Bumping column 2 from INT64 to REAL on data row 12121, field contains '1.23500001430511'
Bumping column 49 from INT to INT64 on data row 12121, field contains '"now and then"'
Bumping column 49 from INT64 to REAL on data row 12121, field contains '"now and then"'
Bumping column 49 from REAL to STR on data row 12121, field contains '"now and then"'
Bumping column 330 from INT to INT64 on data row 12121, field contains '"worst kind"'
Bumping column 330 from INT64 to REAL on data row 12121, field contains '"worst kind"'
Bumping column 330 from REAL to STR on data row 12121, field contains '"worst kind"'
Bumping column 609 from INT to INT64 on data row 12121, field contains '"good purpose"'
Bumping column 609 from INT64 to REAL on data row 12121, field contains '"good purpose"'
Bumping column 609 from REAL to STR on data row 12121, field contains '"good purpose"'
Bumping column 610 from INT to INT64 on data row 12121, field contains '"most of the time"'
Bumping column 610 from INT64 to REAL on data row 12121, field contains '"most of the time"'
Bumping column 610 from REAL to STR on data row 12121, field contains '"most of the time"'
Bumping column 98 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 98 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 98 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 99 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 99 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 99 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 100 from INT to INT64 on data row 15580, field contains '"strongly agree"'
Bumping column 100 from INT64 to REAL on data row 15580, field contains '"strongly agree"'
Bumping column 100 from REAL to STR on data row 15580, field contains '"strongly agree"'
Bumping column 101 from INT to INT64 on data row 15580, field contains '"somewht disagree"'
Bumping column 101 from INT64 to REAL on data row 15580, field contains '"somewht disagree"'
Bumping column 101 from REAL to STR on data row 15580, field contains '"somewht disagree"'
Bumping column 102 from INT to INT64 on data row 15580, field contains '"strongly agree"'
Bumping column 102 from INT64 to REAL on data row 15580, field contains '"strongly agree"'
Bumping column 102 from REAL to STR on data row 15580, field contains '"strongly agree"'
Bumping column 103 from INT to INT64 on data row 15580, field contains '"strongly agree"'
Bumping column 103 from INT64 to REAL on data row 15580, field contains '"strongly agree"'
Bumping column 103 from REAL to STR on data row 15580, field contains '"strongly agree"'
Bumping column 104 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 104 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 104 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 250 from INT to INT64 on data row 15580, field contains '"somewht disagree"'
Bumping column 250 from INT64 to REAL on data row 15580, field contains '"somewht disagree"'
Bumping column 250 from REAL to STR on data row 15580, field contains '"somewht disagree"'
Bumping column 251 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 251 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 251 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 252 from INT to INT64 on data row 15580, field contains '"somewht disagree"'
Bumping column 252 from INT64 to REAL on data row 15580, field contains '"somewht disagree"'
Bumping column 252 from REAL to STR on data row 15580, field contains '"somewht disagree"'
Bumping column 254 from INT to INT64 on data row 15580, field contains '"somewht disagree"'
Bumping column 254 from INT64 to REAL on data row 15580, field contains '"somewht disagree"'
Bumping column 254 from REAL to STR on data row 15580, field contains '"somewht disagree"'
Bumping column 256 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 256 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 256 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 257 from INT to INT64 on data row 15580, field contains '"somewhat agree"'
Bumping column 257 from INT64 to REAL on data row 15580, field contains '"somewhat agree"'
Bumping column 257 from REAL to STR on data row 15580, field contains '"somewhat agree"'
Bumping column 105 from INT to INT64 on data row 15581, field contains '"somewhat agree"'
Bumping column 105 from INT64 to REAL on data row 15581, field contains '"somewhat agree"'
Bumping column 105 from REAL to STR on data row 15581, field contains '"somewhat agree"'
Bumping column 253 from INT to INT64 on data row 15581, field contains '"strngly disagree"'
Bumping column 253 from INT64 to REAL on data row 15581, field contains '"strngly disagree"'
Bumping column 253 from REAL to STR on data row 15581, field contains '"strngly disagree"'
Bumping column 255 from INT to INT64 on data row 15581, field contains '"strngly disagree"'
Bumping column 255 from INT64 to REAL on data row 15581, field contains '"strngly disagree"'
Bumping column 255 from REAL to STR on data row 15581, field contains '"strngly disagree"'
Bumping column 64 from INT to INT64 on data row 15584, field contains '"too little"'
Bumping column 64 from INT64 to REAL on data row 15584, field contains '"too little"'
Bumping column 64 from REAL to STR on data row 15584, field contains '"too little"'
Bumping column 65 from INT to INT64 on data row 15584, field contains '"too little"'
Bumping column 65 from INT64 to REAL on data row 15584, field contains '"too little"'
Bumping column 65 from REAL to STR on data row 15584, field contains '"too little"'
Bumping column 66 from INT to INT64 on data row 15584, field contains '"too little"'
Bumping column 66 from INT64 to REAL on data row 15584, field contains '"too little"'
Bumping column 66 from REAL to STR on data row 15584, field contains '"too little"'
Bumping column 67 from INT to INT64 on data row 15584, field contains '"too little"'
Bumping column 67 from INT64 to REAL on data row 15584, field contains '"too little"'
Bumping column 67 from REAL to STR on data row 15584, field contains '"too little"'
Bumping column 71 from INT to INT64 on data row 17053, field contains '"pay more"'
Bumping column 71 from INT64 to REAL on data row 17053, field contains '"pay more"'
Bumping column 71 from REAL to STR on data row 17053, field contains '"pay more"'
Bumping column 72 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 72 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 72 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 73 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 73 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 73 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 74 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 74 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 74 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 75 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 75 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 75 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 76 from INT to INT64 on data row 17053, field contains '"in favor"'
Bumping column 76 from INT64 to REAL on data row 17053, field contains '"in favor"'
Bumping column 76 from REAL to STR on data row 17053, field contains '"in favor"'
Bumping column 77 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 77 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 77 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 78 from INT to INT64 on data row 17053, field contains '"neither"'
Bumping column 78 from INT64 to REAL on data row 17053, field contains '"neither"'
Bumping column 78 from REAL to STR on data row 17053, field contains '"neither"'
Bumping column 79 from INT to INT64 on data row 17053, field contains '"spend same"'
Bumping column 79 from INT64 to REAL on data row 17053, field contains '"spend same"'
Bumping column 79 from REAL to STR on data row 17053, field contains '"spend same"'
Bumping column 80 from INT to INT64 on data row 17053, field contains '"spend more"'
Bumping column 80 from INT64 to REAL on data row 17053, field contains '"spend more"'
Bumping column 80 from REAL to STR on data row 17053, field contains '"spend more"'
Bumping column 81 from INT to INT64 on data row 17053, field contains '"spend same"'
Bumping column 81 from INT64 to REAL on data row 17053, field contains '"spend same"'
Bumping column 81 from REAL to STR on data row 17053, field contains '"spend same"'
Bumping column 82 from INT to INT64 on data row 17053, field contains '"spend more"'
Bumping column 82 from INT64 to REAL on data row 17053, field contains '"spend more"'
Bumping column 82 from REAL to STR on data row 17053, field contains '"spend more"'
Bumping column 83 from INT to INT64 on data row 17053, field contains '"spend less"'
Bumping column 83 from INT64 to REAL on data row 17053, field contains '"spend less"'
Bumping column 83 from REAL to STR on data row 17053, field contains '"spend less"'
Bumping column 84 from INT to INT64 on data row 17053, field contains '"spend same"'
Bumping column 84 from INT64 to REAL on data row 17053, field contains '"spend same"'
Bumping column 84 from REAL to STR on data row 17053, field contains '"spend same"'
Bumping column 85 from INT to INT64 on data row 17053, field contains '"spend same"'

 *** caught segfault ***
address 0x56a24, cause 'memory not mapped'

Traceback:
 1: fread("GSS.csv", verbose = TRUE)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

似乎正在发生的是 132MB 文件非常稀疏(许多空白字段)。有 613 列和 55087 行。由于稀疏，前 5 行、中间 5 行和后 5 行不足以检测这些列是 character .当它到达此类列的第一个填充字段时，它会正确提升许多列的列类型，这通常可以正常工作。然后它崩溃了。

非常感谢!我在这里提交了错误报告:

#493: Reproducible crash in fread

关于r - 具有 613 列调查数据的稀疏 132MB 文件上的 fread segfault，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/21138713/

文章推荐： d3.js - 多线图中的交错过渡

文章推荐： json - 如何使用 Spring Hateoas 和 HAL 向链接添加附加属性？

文章推荐： sql - 如何使用 SQL 计算一条路线行驶的次数？

释放后分配的自定义内存映射分配器 SEGFAULT
所以我需要一个简单的分配器来分配(有时使用清零)并随后从映射内存池中释放 4K block 。然而，在实现这个之后，在测试时我发现在释放一两个 block 之后，如果我尝试分配一个 block ，程序
C SegFault 通过打印修复？
我的任务是用 C 编写一个程序。该程序应该能够检查参数并创建与我提供的参数一样大的数组。我必须用随机数填充数组。到目前为止工作正常。稍后我的任务是使用指针对数组进行排序。第一件事是我不太明白指针是如何
c - 在未使用的符号上获取 SEGFAULT
我对 C 很陌生(仍然)所以如果我误解了一些基本的东西，请耐心等待我有一个简单的程序，它应该将文件作为字符串读取，然后将该字符串拆分成行 - 将结果存储到 n 个字符串数组中。但是，当我运行以下代码
c++ - 应用程序退出时出现 SegFault
我在工作中使用的应用程序之一遇到了一个奇怪且烦人的问题。该应用程序是用 C++ 编写的，当应用程序终止(主函数返回或调用 exit)时，它会因段错误而崩溃。段错误似乎是由 basic_string 类
Python共享库: RTLD_GLOBAL segfault
我使用 python swig 包装的 C++ 库。在它的 __init__.py 文件中，它 sets在导入包含实现代码的共享对象文件之前，使用 dlopen 标志 RTLD_GLOBAL。这会导
c - 请检查我的 segFault
我在这里遇到了段错误。我很困惑。请帮帮我。 f1 和 y 都是结构体节点的指针。我想把 y 的左转 f1 右转。 #include #include struct node{
c - 由于无效分配而导致 SegFault？
我有一个在公共(public)结构中声明的数组，如下所示: uint16_t *registers; 在一个函数中，我正在检索一个字符字符串(存储在缓冲区中，请参阅下面的代码)，其中包含以逗号分隔的数
c - 尝试从二叉搜索树中删除节点时获取 SEGFAULT
我正在用 C 实现二叉搜索树。下面的代码工作正常，只是当我尝试从树中删除子树时得到 SEGFAULT: 源代码: #include #include struct node { int dat
c - 将项目添加到链表的末尾 segfault
struct vehicle *add_vehicle(struct vehicle *v){ struct vehicle *newcar = (struct vehicle*)malloc
c - 内存空闲抛出 segFault
我正在使用链接列表实现符号表，代码工作正常，但代码中存在内存泄漏，我有以下结构 struct node { char* pcKey; void* pvValue; struct node
将字符串复制到数组 -- strcpy segfault
我正在尝试将字符串复制到数组并打印它。它适用于第一个 for 循环，但第二次出现 seg 错误。 main (int argc, char *argv[]){ int argcIndex; cha
c - 从不同函数访问分配的数组时出现 SegFault
自从我用 C 编写代码已经一年了，但我不明白为什么会出现段错误 // Assume all imports are made int printAgain(double** array, int si
c - 输入文件时出现 SegFault
这是我的代码。编辑:调用者包含在底部。该函数读取数据文件，确定有多少行和列，然后将数据存储到 data_array 中。 int getdata(double* *data_array, int*
c++ - 具有一个版本的等效代码的 SegFault？
我认为有两组代码是等效的，但一组会导致段错误，而另一组则不会。我真的很困惑为什么会这样...... 我想创建一个查找函数此代码确实有效: MyPair *> dummy(x, NULL);
C++ Segfault No 编译错误找不到原因
希望有人能提供帮助。我可以毫无错误地编译，我没有发现任何语法错误，但是当我运行它时，它崩溃了。在启动时调试段错误。全面披露，这是作业。我不是要找人来编写这个代码，只是看看我的问题和我现有的代码，也许会
c++ - OpenMP导致heisenbug segfault
我正在尝试在OpenMP中并行化相当大的for-loop。大约有20％的时间运行正常，但其余时间会因各种段错误而崩溃，例如： *** glibc detected *** ./execute: dou
c++ - 使用共享指针访问单例子类时出现 SEGFAULT
我有一个模板类 ISingleton class ISingleton { public: static T* getInstance() { lock_guard g
LibSVM 在大型特征向量上失败(SEGFAULT)
我正在为使用 LibSVM 的 Android 构建 NDK 应用程序。我在 XCode 中为我的 mac 构建了一个等价物(都是 C++) 我发现 Mac 可以高速准确地处理我给它的非常大的特征向量
c++ - 写入堆栈变量时出现 SEGFAULT
我在 ARM linux 平台上有一个由简单代码引起的非常奇怪的崩溃。问题是它很少重现(一天一次)，另一个问题是它在实际上无法重现的地方崩溃。让我们从 C++ 代码开始。线程函数执行此操作:
c++ - 在内存中的预定位置构造的对象 - SEGFAULT
我有这段代码 int main() { int *b = new int(8); cout<<" &b = "<

行者123

个人简介

我是一名优秀的程序员,十分优秀！

作者热门文章

滴滴打车优惠券免费领取

全站热门文章

首页

博学

6Ren·AI

商城

r - 具有 613 列调查数据的稀疏 132MB 文件上的 fread segfault