python - IRichBolt 在storm-1.0.0 和 pyleus-0.3.0 上运行拓扑时出错

Question

我正在使用 pyleus --verbose local xyz_topology.jar 运行 Storm 拓扑“storm-1.0.0, pyleus-0.3.0, centos-6.6 ”并得到错误

Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt

Running: java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-1.0.0
-Dstorm.log.dir=/usr/local/apache-storm-1.0.0/logs -Djava.library.path=/usr/local/    
lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-1.0.0/lib/asm-5.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/clojure-1.7.0.jar:/usr/local/apache-storm-1.0.0/lib/disruptor-3.3.2.jar:
/usr/local/apache-storm-1.0.0/lib/kryo-3.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-api-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-core-2.1.jar    
:/usr/local/apache-storm-1.0.0/lib/log4j-over-slf4j-1.6.6.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-slf4j-impl-2.1.jar:/usr/local/apache-storm-1.0.0/lib/minlog-1.3.0.jar:
/usr/local/apache-storm-1.0.0/lib/objenesis-2.1.jar:/usr/local/apache-storm-1.0.0/lib/reflectasm-1.10.1.jar:
/usr/local/apache-storm-1.0.0/lib/servlet-api-2.5.jar:/usr/local/apache-storm-1.0.0/lib/slf4j-api-1.7.7.jar:
/usr/local/apache-storm-1.0.0/lib/storm-core-1.0.0.jar:
/usr/local/apache-storm-1.0.0/lib/storm-rename-hack-1.0.0.jar:xyzTopology.jar:
/usr/local/storm/conf:/usr/local/apache-storm-1.0.0/bin
-Dstorm.jar=xyz_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichBolt
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 7 more
pyleus local: error: [StormError] Storm command failed. Run with --verbose for more info.

使用旧版本 storm-0.10.0 and pyleus-0.3.0 时，相同的拓扑运行良好且没有任何错误。 .

我查看了storm-0.10.0和storm-1.0.0文档，发现在storm-1.0.0中

(a) 包结构已更改并且

(b) 未提供 IRichBolt 类，取而代之的是两个新类 - BaseStatefulBoltExecutor 和 BasicBoltExecutor - 实现 IRichBolt 接口(interface)。

看起来是新的storm-1.0.0版本无法与 pyleus-0.3.0 一起使用就像旧的 storm-0.10.0 一样版本。

解决该问题并允许在 storm-1.0.0 上运行拓扑的最佳解决方案是什么？和pyleus-0.3.0 ？

Answer 1

Storm 1.0.0 对包名称进行了重大重构。所有包 backtype/* 都被 org/apache/storm 替换。因此，您需要使用 Storm 1.0.0 重新编译拓扑(包括一些代码更改；即，使用不同的 import 来与新的包结构保持一致)。

作为替代方案，您可以在 Storm 集群中启用向后兼容性(请参阅 https://storm.apache.org/releases/1.0.0/index.html )

NOTE

In the latest version, the class packages have been changed from "backtype.storm" to "org.apache.storm" so the topology code compiled with older version won't run on the Storm 1.0.0 just like that. Backward compatibility is available through following configuration

client.jartransformer.class: "org.apache.storm.hack.StormShadeTransformer"

You need to add the above config in storm installation if you want to run the code compiled with older versions of storm. The config should be added in the machine you use to submit your topologies.

Refer to https://issues.apache.org/jira/browse/STORM-1202 for more details.