hadoop - ifile EBADF : Bad file descriptor while performing matrix addition 上的预读失败

Question

我正在尝试在 Apache Pig 中编写用于矩阵加法的代码。

matrixM = LOAD 'Mmatrix.txt' USING PigStorage (',') AS (i,j,v);
matrixN = LOAD 'Nmatrix.txt' USING PigStorage (',') AS (i,j,v);
unionres = UNION matrixM, matrixN;
DUMP unionres;
res = GROUP unionres BY (i,j);
DUMP res;
ILLUSTRATE res;
final_res = FOREACH res GENERATE group.$0 AS i, group.$1 AS j, SUM(unionres.v) AS v;
DUMP final_res;

在本地模式下运行代码时出现以下异常。

org.apache.hadoop.io.ReadaheadPool - Failed readahead on ifile
EBADF: Bad file descriptor
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
    at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:208)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

谁能解释一下我哪里出错了？

Answer 1

通常上面的消息是 WARN，之后会显示实际的错误消息。我只在磁盘损坏或已满时看到这种类型的 WARN 消息。您的本地/tmp 是否已满？