windows - Windows 7 上没有 Hadoop 的 Pig

Question

我试图在 Windows 7 机器上运行 PigUnit 测试，然后在 Ubuntu 集群上运行实际的 pig 脚本，我开始认为我对“withouthadoop”的理解是不正确的。 p>

Do I need to install Hadoop to locally run a PigUnit test on a Windows 7 machine?

我安装了:

1. eclipse 朱诺和 Ant
2. 西格温

我设置:

1. JAVA_HOME=C:\Program Files\Java\jdk1.6.0_39
2. PIG_HOME=C:\Users\john.doe\Java\eclipse\pig
3. PIG_CLASSPATH=%PIG_HOME%\bin

我使用 eclipse 的 Ant 构建器 jar-all 和 pigunit-jar 创建了:

1. pig.jar
2. pig-withouthadoop.jar
3. pigunit.jar

仍然当我在 cygwin 中键入 pig -x local 时，我得到:

$./pig -x local
cygpath: can't convert empty path
Exception in thread "main" java.io.IOException: Error opening job jar: /usr/lib/pig/pig-withouthadoop.jar
    at org.apache.hadoop.util.RunJar.main(RunJar.java:135)
Caused by: java.io.FileNotFoundException: \usr\lib\pig\pig-withouthadoop.jar (the systen cannot find the given path)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:127)
    at java.util.jar.JarFile.<init>(JarFile.java:136)
    at java.util.jar.JarFile.<init>(JarFile.java:73)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:133)

当我尝试从 http://pig.apache.org/docs/r0.10.0/test.html#pigunit 运行测试时在 eclipse 中使用选项“Run as JUnit”，我得到:

java.io.IOException
at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:62)
at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:171)
at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:267)
at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:262)
at da.utils.pigunit.PigUnitExample.testTop2Queries(PigUnitExample.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

我开始认为我错过了一些在 Windows 上运行 Pig 的关键基本信息，我还必须说我不是 Windows 7 和 cygwin 的有经验的用户，我来自 Unix 世界。

Answer 1

不要打它。从 Web 平台安装程序在 Windows 上安装 Hadoop HDInsight 服务器: http://www.microsoft.com/web/downloads/platform.aspx

它不会花费很长时间或占用那么多空间，并且整个 shebang 只是为您设置和运行。我无法让 Pig 脚本获取参数，也没有 HBase，但你有 HDFS、Pig、Hive。如果只遵循::http://social.msdn.microsoft.com/Forums/en-US/hdinsight/thread/885efc22-fb67-4df8-8648-4ff38098dac6/，你甚至可以让整个本地集群运行起来。