- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
当我尝试运行 datastax spark-sql-thriftserver 时,出现以下错误:
dse spark-sql-thriftserver start \
--conf spark.cores.max=10 \
--conf spark.executor.memory=2g \
--hiveconf hive.server2.thrift.port=10001
Spark Command: /opt/jdk1.8.0_112/jre//bin/java -cp
/etc/dse/spark/:/usr/share/dse/spark/jars/*:/etc/dse/hadoop2-client/ -Djava.library.path=/usr/share/dse/hadoop2-client/lib/native:/usr/share/dse/cassandra/lib/sigar-bin: -Dcassandra.logdir=/var/log/cassandra -XX:MaxHeapFreeRatio=50 -XX:MinHeapFreeRatio=20 -Dguice_include_stack_traces=OFF -Ddse.system_memory_in_mb=32174 -Dcassandra.config.loader=com.datastax.bdp.config.DseConfigurationLoader -Dlogback.configurationFile=/etc/dse/spark/logback-spark.xml -Dcassandra.logdir=/var/log/cassandra -Ddse.client.configuration.impl=com.datastax.bdp.transport.client.HadoopBasedClientConfiguration -Dderby.stream.error.method=com.datastax.bdp.derby.LogbackBridge.getLogger -Xmx1024M org.apache.spark.deploy.SparkSubmit --conf spark.executor.memory=2g --conf spark.cores.max=10 --class
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2
spark-internal --hiveconf hive.server2.thrift.port=10001
======================================== WARN 2017-05-07 22:21:55 org.apache.spark.SparkContext: Use an existing SparkContext, some
configuration may not take effect. ERROR 2017-05-07 22:22:04
org.apache.spark.deploy.DseSparkSubmitBootstrapper: Failed to start or submit Spark application java.lang.NoSuchMethodError:
org.apache.hive.service.cli.operation.LogDivertAppender.setWriter(Ljava/io/Writer;)V at
org.apache.hive.service.cli.operation.LogDivertAppender.(LogDivertAppender.java:166) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.hive.service.cli.operation.OperationManager.initOperationLogCapture(OperationManager.java:85) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.hive.service.cli.operation.OperationManager.init(OperationManager.java:63) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
scala.collection.Iterator$class.foreach(Iterator.scala:893)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
~[scala-library-2.11.8.jar:na] at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterable.foreach(Iterable.scala:54)
~[scala-library-2.11.8.jar:na] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.initCompositeService(SparkSQLSessionManager.scala:36) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.init(SparkSQLSessionManager.scala:58) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
scala.collection.Iterator$class.foreach(Iterator.scala:893)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
~[scala-library-2.11.8.jar:na] at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterable.foreach(Iterable.scala:54)
~[scala-library-2.11.8.jar:na] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.initCompositeService(SparkSQLCLIService.scala:39) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:62) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
scala.collection.Iterator$class.foreach(Iterator.scala:893)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
~[scala-library-2.11.8.jar:na] at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
~[scala-library-2.11.8.jar:na] at
scala.collection.AbstractIterable.foreach(Iterable.scala:54)
~[scala-library-2.11.8.jar:na] at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.initCompositeService(HiveThriftServer2.scala:272) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.init(HiveThriftServer2.scala:292) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:94) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.8.0_112] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112] at
org.apache.spark.deploy.DseSparkSubmit$.org$apache$spark$deploy$DseSparkSubmit$$runMain(DseSparkSubmit.scala:730) ~[dse-spark-5.1.0.jar:5.1.0] at
org.apache.spark.deploy.DseSparkSubmit$.doRunMain$1(DseSparkSubmit.scala:175) ~[dse-spark-5.1.0.jar:5.1.0] at
org.apache.spark.deploy.DseSparkSubmit$.submit(DseSparkSubmit.scala:200) ~[dse-spark-5.1.0.jar:5.1.0] at
org.apache.spark.deploy.DseSparkSubmit$.main(DseSparkSubmit.scala:109) ~[dse-spark-5.1.0.jar:5.1.0] at
org.apache.spark.deploy.DseSparkSubmitBootstrapper$.main(DseSparkSubmitBootstrapper.scala:74) ~[dse-spark-5.1.0.jar:5.1.0] at
org.apache.spark.deploy.DseSparkSubmitBootstrapper.main(DseSparkSubmitBootstrapper.scala) [dse-spark-5.1.0.jar:5.1.0] ERROR 2017-05-07 22:22:15
org.apache.spark.util.Utils: Uncaught exception in thread Thread-0
java.lang.NullPointerException: null at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$$anonfun$main$1.apply$mcV$sp(HiveThriftServer2.scala:85) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:215) ~[spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:187) ~[spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:187) ~[spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:187) ~[spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1953) ~[spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:187) [spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:187) [spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:187) [spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
scala.util.Try$.apply(Try.scala:192) [scala-library-2.11.8.jar:na] at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:187) [spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:177) [spark-core_2.11-2.0.2.6.jar:2.0.2.6] at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) [hadoop-common-2.7.1.3.jar:na]
ERROR 2017-05-07 22:22:04
org.apache.spark.deploy.DseSparkSubmitBootstrapper: Failed to start or submit Spark application java.lang.NoSuchMethodError:
org.apache.hive.service.cli.operation.LogDivertAppender.setWriter(Ljava/io/Writer;)V at
org.apache.hive.service.cli.operation.LogDivertAppender.(LogDivertAppender.java:166) ~[spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6]
libhadoop.a libhadoop.so libhadoop.so.1.0.0 libhadooppipes.a libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0
empty
JavaEWAH-0.3.2.jar commons-dbcp-1.4.jar
httpclient-4.5.2.jar jersey-server-2.22.2.jar
netty-3.9.8.Final.jar reflectasm-1.10.1.jar
spark-sketch_2.11-2.0.2.6.jar RoaringBitmap-0.5.11.jar
commons-io-2.5.jar httpcore-4.4.4.jar
jline-2.14.2.jar noggit-0.6.jar
scala-compiler-2.11.8.jar
spark-sql_2.11-2.0.2.6.jar antlr4-runtime-4.5.3.jar
commons-lang3-3.4.jar httpmime-4.4.1.jar
joda-convert-1.2.jar objenesis-2.1.jar
scala-xml_2.11-1.0.4.jar
spark-streaming_2.11-2.0.2.6.jar aopalliance-repackaged-2.4.0-b34.jar commons-math3-3.4.1.jar ivy-2.4.0.jar
joda-time-2.9.3.jar opencsv-2.3.jar
scalap-2.11.8.jar
spark-tags_2.11-2.0.2.6.jar arpack_combined_all-0.1.jar
compress-lzf-1.0.3.jar jackson-annotations-2.5.3.jar
jodd-core-3.5.2.jar oro-2.0.8.jar
scalatest_2.11-2.2.6.jar
spark-unsafe_2.11-2.0.2.6.jar avro-1.7.7.jar
core-1.1.2.jar jackson-core-asl-1.9.13.jar
jpam-1.1.jar osgi-resource-locator-1.0.1.jar
snappy-0.2.jar
spire-macros_2.11-0.7.4.jar avro-ipc-1.7.7.jar
datanucleus-api-jdo-3.2.6.jar jackson-mapper-asl-1.9.13.jar
json4s-ast_2.11-3.2.11.jar paranamer-2.8.jar
solr-solrj-6.0.1.0.1596.jar
spire_2.11-0.7.4.jar avro-mapred-1.7.7-hadoop2.jar
datanucleus-core-3.2.10.jar jackson-module-scala_2.11-2.5.3.jar
json4s-core_2.11-3.2.11.jar parquet-column-1.7.0.jar
spark-cassandra-connector-unshaded_2.11-2.0.1.jar stax-api-1.0.1.jar bonecp-0.8.0.RELEASE.jar datanucleus-rdbms-3.2.9.jar
javax.annotation-api-1.2.jar
json4s-jackson_2.11-3.2.11.jar parquet-common-1.7.0.jar
spark-catalyst_2.11-2.0.2.6.jar stax2-api-3.1.4.jar breeze-macros_2.11-0.11.2.jar derby-10.10.2.0.jar
javax.inject-2.4.0-b34.jar jsr166e-1.1.0.jar
parquet-encoding-1.7.0.jar spark-core_2.11-2.0.2.6.jar
stream-2.7.0.jar breeze_2.11-0.11.2.jar
eigenbase-properties-1.1.5.jar javax.ws.rs-api-2.0.1.jar
jta-1.1.jar parquet-format-2.3.0-incubating.jar spark-graphx_2.11-2.0.2.6.jar super-csv-2.2.0.jar calcite-avatica-1.2.0-incubating.jar hive-beeline-1.2.1.spark2.jar
javolution-5.5.1.jar jtransforms-2.4.0.jar
parquet-generator-1.7.0.jar
spark-hive-thriftserver_2.11-2.0.2.6.jar
univocity-parsers-2.1.1.jar calcite-core-1.2.0-incubating.jar
hive-cli-1.2.1.spark2.jar jdo-api-3.0.1.jar
jul-to-slf4j-1.7.13.jar parquet-hadoop-1.7.0.jar
spark-hive_2.11-2.0.2.6.jar unused-1.0.0.jar calcite-linq4j-1.2.0-incubating.jar hive-exec-1.2.1.spark2.jar
jersey-client-2.22.2.jar kryo-3.0.3.jar
parquet-hadoop-bundle-1.6.0.jar spark-launcher_2.11-2.0.2.6.jar
velocity-1.7.jar cassandra-driver-mapping-3.1.4.jar
hive-jdbc-1.2.1.spark2.jar jersey-common-2.22.2.jar
libfb303-0.9.3.jar parquet-jackson-1.7.0.jar
spark-mllib-local_2.11-2.0.2.6.jar
woodstox-core-asl-4.4.1.jar chill-java-0.8.0.jar
hive-metastore-1.2.1.spark2.jar jersey-container-servlet-2.22.2.jar
mail-1.4.7.jar pmml-model-1.2.15.jar
spark-mllib_2.11-2.0.2.6.jar
xbean-asm5-shaded-4.4.jar chill_2.11-0.8.0.jar
hk2-api-2.4.0-b34.jar
jersey-container-servlet-core-2.22.2.jar mesos-0.21.1-shaded-protobuf.jar pmml-schema-1.2.15.jar
spark-network-common_2.11-2.0.2.6.jar commons-beanutils-1.9.3.jar
hk2-locator-2.4.0-b34.jar jersey-guava-2.22.2.jar
metrics-json-3.1.2.jar py4j-0.10.1.jar
spark-network-shuffle_2.11-2.0.2.6.jar commons-codec-1.10.jar
hk2-utils-2.4.0-b34.jar jersey-media-jaxb-2.22.2.jar
minlog-1.3.0.jar pyrolite-4.13.jar
spark-repl_2.11-2.0.2.6.jar
apache-log4j-extras-1.2.17.jar
在
/usr/share/dse/spark/lib
我收到错误消息(请参阅
https://pastebin.com/KjgsEhnw 了解整个日志):
Spark Command: /opt/jdk1.8.0_112/bin/java -cp /etc/dse/spark/:/usr/share/dse/spark/jars/*:/etc/dse/hadoop2-client/ -Djava.library.path=/usr/share/dse/hadoop2-client/lib/native:/usr/share/dse/cassandra/lib/sigar-bin: -Dcassandra.logdir=/var/log/cassandra -XX:MaxHeapFreeRatio=50 -XX:MinHeapFreeRatio=20 -Dguice_include_stack_traces=OFF -Ddse.system_memory_in_mb=32174 -Dcassandra.config.loader=com.datastax.bdp.config.DseConfigurationLoader -Dlogback.configurationFile=/etc/dse/spark/logback-spark.xml -Dcassandra.logdir=/var/log/cassandra -Ddse.client.configuration.impl=com.datastax.bdp.transport.client.HadoopBasedClientConfiguration -Dderby.stream.error.method=com.datastax.bdp.derby.LogbackBridge.getLogger -Xmx1024M org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal
========================================
WARN 2017-05-19 14:11:31 org.apache.spark.SparkContext: Use an existing SparkContext, some configuration may not take effect.
WARN 2017-05-19 14:11:36 org.apache.hadoop.hive.metastore.HiveMetaStore: Retrying creating default database after error: Unexpected exception caught.
javax.jdo.JDOFatalInternalException: Unexpected exception caught.
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1193) ~[jdo-api-3.0.1.jar:3.0.1]
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) ~[jdo-api-3.0.1.jar:3.0.1]
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) ~[jdo-api-3.0.1.jar:3.0.1]
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) ~[hadoop-common-2.7.1.3.jar:na]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) ~[hadoop-common-2.7.1.3.jar:na]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) ~[hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.8.0_112]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [na:1.8.0_112]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [na:1.8.0_112]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [na:1.8.0_112]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) [hive-metastore-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) [hive-exec-1.2.1.spark2.jar:1.2.1.spark2]
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:189) [spark-hive_2.11-2.0.2.6.jar:2.0.2.6]
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:247) [spark-hive_2.11-2.0.2.6.jar:2.0.2.6]
at org.apache.spark.sql.hive.HiveUtils$.newClientForExecution(HiveUtils.scala:250) [spark-hive_2.11-2.0.2.6.jar:2.0.2.6]
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:88) [spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6]
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala) [spark-hive-thriftserver_2.11-2.0.2.6.jar:2.0.2.6]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112]
at org.apache.spark.deploy.DseSparkSubmit$.org$apache$spark$deploy$DseSparkSubmit$$runMain(DseSparkSubmit.scala:730) [dse-spark-5.1.0.jar:5.1.0]
at org.apache.spark.deploy.DseSparkSubmit$.doRunMain$1(DseSparkSubmit.scala:175) [dse-spark-5.1.0.jar:5.1.0]
at org.apache.spark.deploy.DseSparkSubmit$.submit(DseSparkSubmit.scala:200) [dse-spark-5.1.0.jar:5.1.0]
at org.apache.spark.deploy.DseSparkSubmit$.main(DseSparkSubmit.scala:109) [dse-spark-5.1.0.jar:5.1.0]
at org.apache.spark.deploy.DseSparkSubmitBootstrapper$.main(DseSparkSubmitBootstrapper.scala:74) [dse-spark-5.1.0.jar:5.1.0]
at org.apache.spark.deploy.DseSparkSubmitBootstrapper.main(DseSparkSubmitBootstrapper.scala) [dse-spark-5.1.0.jar:5.1.0]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112]
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) ~[jdo-api-3.0.1.jar:3.0.1]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_112]
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) ~[jdo-api-3.0.1.jar:3.0.1]
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166) ~[jdo-api-3.0.1.jar:3.0.1]
... 48 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/apache/log4j/or/RendererMap
at org.apache.log4j.Hierarchy.<init>(Hierarchy.java:97) ~[apache-log4j-extras-1.2.17.jar:na]
at org.apache.log4j.LogManager.<clinit>(LogManager.java:82) ~[apache-log4j-extras-1.2.17.jar:na]
at org.apache.log4j.Logger.getLogger(Logger.java:104) ~[apache-log4j-extras-1.2.17.jar:na]
at org.datanucleus.util.Log4JLogger.<init>(Log4JLogger.java:49) ~[datanucleus-core-3.2.10.jar:na]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.8.0_112]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [na:1.8.0_112]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [na:1.8.0_112]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [na:1.8.0_112]
at org.datanucleus.util.NucleusLogger.getLoggerInstance(NucleusLogger.java:237) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.util.NucleusLogger.<clinit>(NucleusLogger.java:205) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.plugin.PluginRegistryFactory.newPluginRegistry(PluginRegistryFactory.java:74) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.plugin.PluginManager.<init>(PluginManager.java:61) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.plugin.PluginManager.createPluginManager(PluginManager.java:427) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.NucleusContext.<init>(NucleusContext.java:266) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.NucleusContext.<init>(NucleusContext.java:247) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.NucleusContext.<init>(NucleusContext.java:225) ~[datanucleus-core-3.2.10.jar:na]
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.<init>(JDOPersistenceManagerFactory.java:416) ~[datanucleus-api-jdo-3.2.6.jar:na]
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:301) ~[datanucleus-api-jdo-3.2.6.jar:na]
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202) ~[datanucleus-api-jdo-3.2.6.jar:na]
... 56 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.or.RendererMap
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_112]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_112]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_112]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_112]
... 75 common frames omitted
最佳答案
看着 org.apache.hive.service.cli.operation.LogDivertAppender.(LogDivertAppender.java:166)显示以下行:
setWriter(writer);
apache-log4j-extras-1.2.17.jar
我可以在我的 Spark 安装中看到
jars/apache-log4j-extras-1.2.17.jar
.
jars/apache-log4j-extras-1.2.17.jar
在似乎与受影响的缺失方法有某种关系的 jars 列表中。
apache-log4j-extras-1.2.17.jar
从其他一些 Spark 下载并将其放在
/usr/share/dse/spark/lib
下甚至
/usr/share/dse/spark/jars
因为它目前是空的。
关于java - 如何在 Datastax Enterprise 上启动 Spark Thrift Server(因 java.lang.NoSuchMethodError : . ..LogDivertAppender.setWriter 而失败)?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43837365/
我正在阅读 Scylla 的文档,因为我们正计划从 DSE 迁移到本地 Scylla。然而,in their documentation ,他们说不支持 DSE SSTable 格式。 Here他们提
我正在使用与 Datastax Enterprise 捆绑在一起的 gremlin-console (v3.2.7)。启动时它会自动连接到远程 gremlin 服务器。接下来,我创建一个别名来访问右侧
我正在阅读 Scylla 的文档,因为我们正计划从 DSE 迁移到本地 Scylla。然而,in their documentation ,他们说不支持 DSE SSTable 格式。 Here他们提
有谁知道如何在使用 amazon ec2 M3.Xlarge 机器的集群中使用 Datastax 企业(使用 opscenter)? 当我尝试使用这些类型的实例(使用 ssd)时,出现以下错误: 启动
我了解 DSE 5.1 运行 Solr 6.0 版本。我正在尝试使用 Solr 本地参数来使用 facet.pivot 功能,但它似乎不起作用。 我的数据如下 Simple 4 fields 我需要的
我在 Datastax 4.6 集群上有一个大型但简单的 Cassandra 数据库。对于这个非常简单的用例来说,许可证续订是禁止的,我正在尝试迁移到直接的 Apache 或 Datastax Com
我正在尝试在 Windows 7 机器上使用 Datastax C# 客户端驱动程序连接到在虚拟机器 (Ubuntu) 上运行的 DSE Cassandra。 这是我的代码: Cluster _clu
我从 CLI 中创建了这样的列族- create column family profile with key_validation_class = 'UTF8Type' and co
我正在使用 OpsCenter 5.1.1 运行 DSE 4.6.5 集群 (Cassandra 2.0.14.352) 一天一两次,其中一个节点(有时更多)停止报告指标,直到我手动重新启动 data
我需要在 cassandra 的表中选择不同的计数。 据我所知,cassandra 不支持直接不同计数,甚至像 rdbms 这样的嵌套查询也不支持。 select count(*) from (sel
我在尝试为创建的角色授予权限时遇到异常。 命令: GRANT ALL PERMISSIONS on KEYSPACE test_ks to ks_admin; 错误: ServerError: jav
我刚刚安装了 DataStax DevCenter。但是在加载时,会弹出自动更新和使用页面。但是,“确定”按钮被禁用且无法点击。所以我无法通过应用程序的启动加载。我怎样才能绕过这个? Click he
我试图在本地启动DSE 5.0.1 Cassandra(单节点)。 出现以下错误: CassandraDaemon.java:698 - Cannot start node if snitch's d
我刚刚安装了DataStax Community Edition,但是在尝试运行以下命令时; C:\> net start DataStax_DDC_Server 我得到以下回应: 谢谢 最佳答案 我
我使用 datastax java 驱动程序构建了一个选择查询。我使用限制选项设置限制。但是我看到另一个也可以设置的属性 setFetchSize(整数大小) 根据文档,DEFAULT_FETCH_S
刚刚在我的 Mac 上安装(从下载的 dmg 复制应用程序)DevCenter 1.6,运行安装了 jdk 8u152 的 macOs Sierra 10.12.6。应用程序加载了一个 UI(启动画面
我们可以像这样创建一个集群实例。 cluster = Cluster .builder() .addContactPoint("192.168.0.30") .withRetryPolicy(Def
我在127.0.1.1上运行自己的cassandra版本。我将rpc_address和address更改为127.0.1.1。 当我启动 Opscenter 时,系统会提示我安装agents,并通过推
我正在使用 Cassandra 1.2.5 和二级索引。当我运行准备好的语句时,没有返回数据。我有数据。另外,对于索引列,我确实有重复的值。我正在做的是根据 user_id 重新调整 video_id
我正在使用 Datastax 可爱的 cassandra java 驱动程序。我试图将所有查询字符串封装到内置访问器中以进行映射,但我需要能够设置查询的分页状态。 我发现这可以通过普通语句(Simpl
我是一名优秀的程序员,十分优秀!