java - Storm 流 : Cannot run program "unzip" (in directory ".")

Question

我正在开发一个 Storm 项目，并且一直在本地模式下运行它。过去几天我一直在构建代码，今天早上刚刚运行它，并在控制台中收到此错误。

Working Directory = C:\dev\SE\SportsCloud\SEGOps\Storm
SLF4J: The requested version 1.5.8 by your slf4j binding is not compatible with [1.6, 1.7]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
0    [main] INFO  backtype.storm.zookeeper  - Starting inprocess zookeeper at port 2000 and dir C:\Users\t-erros\AppData\Local\Temp\/400384ae-630f-41de-ba92-31b63969d91e
229  [main] INFO  backtype.storm.daemon.nimbus  - Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
233  [main] INFO  backtype.storm.daemon.nimbus  - Using default scheduler
269  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
315  [main-EventThread] INFO  backtype.storm.zookeeper  - Zookeeper state update: :connected:none
338  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
378  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
384  [main-EventThread] INFO  backtype.storm.zookeeper  - Zookeeper state update: :connected:none
388  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
391  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
396  [main-EventThread] INFO  backtype.storm.zookeeper  - Zookeeper state update: :connected:none
399  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
420  [main] INFO  backtype.storm.daemon.supervisor  - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/0ba572e6-1955-4708-8b48-600dcd95511d", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
435  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
442  [main-EventThread] INFO  backtype.storm.zookeeper  - Zookeeper state update: :connected:none
444  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
472  [main] INFO  backtype.storm.daemon.supervisor  - Starting supervisor with id 415b310e-adb9-4f3e-addb-487b5dabd65d at host eross-pc.northamerica.corp.microsoft.com
477  [main] INFO  backtype.storm.daemon.supervisor  - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/111ba4fe-03cb-4d65-8f92-6531a0f8ece9", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (4 5 6), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
483  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
491  [main-EventThread] INFO  backtype.storm.zookeeper  - Zookeeper state update: :connected:none
495  [main] INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl  - Starting
509  [main] INFO  backtype.storm.daemon.supervisor  - Starting supervisor with id 2e0fd15c-80fc-4f31-a06e-481c55755164 at host eross-pc.northamerica.corp.microsoft.com
556  [main] INFO  backtype.storm.daemon.nimbus  - Received topology submission for xes-rtbi-stream-test with conf {"topology.max.task.parallelism" nil, "topology.acker.executors" 1, "topology.kryo.register" nil, "topology.kryo.decorators" (), "topology.name" "xes-rtbi-stream-test", "storm.id" "xes-rtbi-stream-test-1-1372358089", "topology.debug" false}
580  [main] INFO  backtype.storm.daemon.nimbus  - Activating xes-rtbi-stream-test: xes-rtbi-stream-test-1-1372358089
672  [main] INFO  backtype.storm.scheduler.EvenScheduler  - Available slots: (["2e0fd15c-80fc-4f31-a06e-481c55755164" 4] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 5] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 6] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 1] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 2] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 3])
698  [main] INFO  backtype.storm.daemon.nimbus  - Setting new assignment for topology id xes-rtbi-stream-test-1-1372358089: #backtype.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca/nimbus/stormdist/xes-rtbi-stream-test-1-1372358089", :node->host {"2e0fd15c-80fc-4f31-a06e-481c55755164" "eross-pc.northamerica.corp.microsoft.com"}, :executor->node+port {[1 1] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 4]}, :executor->start-time-secs {[1 1] 1372358089}}
1513 [Thread-8] INFO  backtype.storm.daemon.supervisor  - Downloading code for storm id xes-rtbi-stream-test-1-1372358089 from C:\Users\t-erros\AppData\Local\Temp\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca/nimbus/stormdist/xes-rtbi-stream-test-1-1372358089
1861 [Thread-8] INFO  backtype.storm.daemon.supervisor  - Extracting resources from jar at C:\Users\t-erros\Documents\Jars\commons-configuration-1.9-bin\commons-configuration-1.9\commons-configuration-1.9-javadoc.jar to C:\Users\t-erros\AppData\Local\Temp\/111ba4fe-03cb-4d65-8f92-6531a0f8ece9/supervisor/stormdist/xes-rtbi-stream-test-1-1372358089/resources
1870 [Thread-8] ERROR backtype.storm.event  - Error when processing event
java.io.IOException: Cannot run program "unzip" (in directory "."): CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessBuilder.start(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
    at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:254)
    at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:319)
    at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160)
    at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147)
    at backtype.storm.util$exec_command_BANG_.invoke(util.clj:324)
    at backtype.storm.util$extract_dir_from_jar.invoke(util.clj:329)
    at backtype.storm.daemon.supervisor$fn__4803.invoke(supervisor.clj:452)
    at clojure.lang.MultiFn.invoke(MultiFn.java:172)
    at backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__4716.invoke(supervisor.clj:295)
    at backtype.storm.event$event_manager$fn__2507.invoke(event.clj:24)
    at clojure.lang.AFn.run(AFn.java:24)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    ... 15 more
1878 [Thread-8] INFO  backtype.storm.util  - Halting process: ("Error when processing an event")

我注释掉了主函数中的所有内容，只剩下这个:

public class XESRealTimeOITopology 
{   
    public static void main(String[] args) throws Exception 
    {
        System.out.println("Working Directory = " + System.getProperty("user.dir"));

        TopologyBuilder builder = new TopologyBuilder();

        Config conf = new Config();
        conf.setDebug(false);

        LocalCluster cluster = new LocalCluster();
        cluster.submitTopology("xes-rtbi-stream-test", conf, builder.createTopology());

        boolean loop = true;
        while (loop)
        {
            Thread.sleep(60000);
        }


        cluster.shutdown();
    }  
}

在 cluster.submitToplogy 行抛出异常。我的同事在他的电脑上有完全相同的项目设置，也注释掉了这些行，它工作正常。我已经下载了解压缩程序，但我不确定把它放在哪里，或者为什么会抛出这个错误。有没有人有任何想法？

Answer 1

显然 Storm 不喜欢外部 jar 。我将它们放在引用库中，将它们全部移到用户定义的 Storm Lib 中，一切正常。感谢您的回答! :)