exception - MiniDFSCluster 给出 ioexception

Question

我正在尝试在 hadoop 中进行测试。有代码为: System.setProperty("test.build.data","/folder"); 配置=新配置()； cluster = new MiniDFSCluster(config,1,true,null);

但在 new MiniDFSCluster(config,1,true,null) 中，它抛出异常:

java.io.IOException: Cannot run program "du": CreateProcess error=2, The system cannot find the file specified.
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:470)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
    at org.apache.hadoop.util.Shell.run(Shell.java:134)
    at org.apache.hadoop.fs.DU.<init>(DU.java:53)
    at org.apache.hadoop.fs.DU.<init>(DU.java:63)
    at org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.<init>(FSDataset.java:333)
    at org.apache.hadoop.hdfs.server.datanode.FSDataset.<init>(FSDataset.java:689)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:302)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
    at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:417)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
    at ebay.Crawler.TestAll.testinit(TestAll.java:53)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
    at java.lang.reflect.Method.invoke(Method.java:599)
    at junit.framework.TestCase.runTest(TestCase.java:168)
    at junit.framework.TestCase.runBare(TestCase.java:134)
    at junit.framework.TestResult$1.protect(TestResult.java:110)
    at junit.framework.TestResult.runProtected(TestResult.java:128)
    at junit.framework.TestResult.run(TestResult.java:113)
    at junit.framework.TestCase.run(TestCase.java:124)
    at junit.framework.TestSuite.runTest(TestSuite.java:232)
    at junit.framework.TestSuite.run(TestSuite.java:227)
    at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified.
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:92)
    at java.lang.ProcessImpl.start(ProcessImpl.java:41)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:463)
    ... 33 more

有人可以给我一些提示如何解决这个问题吗？非常感谢。

Answer 1

看起来像 du命令不在系统上或不在 PATH 中。如果在 Windows 上使用 Hadoop，则 Cygwin必须安装。无论如何，which du 将给出 du 二进制文件的位置。