hadoop - 无法使用单节点 hadoop 服务器运行 pig

Question

我已经用 ubuntu 安装了一个虚拟机。它将 hadoop 作为单个节点运行。后来我在上面安装了apache pig。 apache pig 在本地模式下运行良好，但它总是提示 ERROR 2999: Unexpected internal error。无法创建 DataStorage

我遗漏了一些非常明显的东西。有人可以帮我运行吗？

更多详情:1. 我假设 hadoop 运行良好，因为我可以在 python 中运行 MapReduce 作业。2. pig -x local 如我所料运行。3. 当我输入 pig 时出现以下错误

Error before Pig is launched----------------------------ERROR 2999: Unexpected internal error. Failed to create DataStoragejava.lang.RuntimeException: Failed to create DataStorage    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.(HDataStorage.java:58)    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)    at org.apache.pig.impl.PigContext.connect(PigContext.java:183)    at org.apache.pig.PigServer.(PigServer.java:226)    at org.apache.pig.PigServer.(PigServer.java:215)    at org.apache.pig.tools.grunt.Grunt.(Grunt.java:55)    at org.apache.pig.Main.run(Main.java:452)    at org.apache.pig.Main.main(Main.java:107)Caused by: java.io.IOException: Call to localhost/127.0.0.1:54310 failed on local exception: java.io.EOFException    at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)    at org.apache.hadoop.ipc.Client.call(Client.java:743)    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)    at $Proxy0.getProtocolVersion(Unknown Source)    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)    at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)    at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:207)    at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:170)    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)    ... 9 moreCaused by: java.io.EOFException    at java.io.DataInputStream.readInt(DataInputStream.java:375)    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)================================================================================

Answer 1

Link帮助我了解失败的可能原因。

这是解决我的问题的方法。
1.重新编译没有hadoop的pig。
2. 更新 PIG_CLASSPATH 以获取 $HADOOP_HOME/lib 中的所有 jar
3.跑 pig 。

谢谢。