gpt4 book ai didi

hadoop - cassandra Spark 连接器可从 Spark 壳工作,但在齐柏林飞艇中失败

转载 作者:行者123 更新时间:2023-12-02 21:17:54 26 4
gpt4 key购买 nike

我能够使用spark shell和spark-cassandra connect驱动程序运行查询。但齐柏林飞艇的同一命令失败。是否有Zeppelin的版本与cassandra连接器配合得很好。

环保
Hadoop 2.6.0
Spark 1.6.0
齐柏林飞艇0.5.6孵育。

命令

  import com.datastax.spark.connector._, org.apache.spark.SparkContext, org.apache.spark.SparkContext._, org.apache.spark.SparkConf
sc.stop
val conf = new SparkConf(true).set("spark.cassandra.connection.host", "localhost")
val sc = new SparkContext(conf)
val test_spark_rdd = sc.cassandraTable("dev", "emp")
test_spark_rdd.first

Error
java.lang.IllegalStateException: Detected Guava issue #1635 which indicates that a version of Guava less than 16.01 is in use. This introduces codec resolution issues and potentially other incompatibility issues in the driver. Please upgrade to Guava 16.01 or later.
at com.datastax.driver.core.SanityChecks.checkGuava(SanityChecks.java:62)
at com.datastax.driver.core.SanityChecks.check(SanityChecks.java:36)
at com.datastax.driver.core.Cluster.<clinit>(Cluster.java:67)
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.clusterBuilder(CassandraConnectionFactory.scala:35)
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:87)
at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:153)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$2.apply(CassandraConnector.scala:148)
at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56)
at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:81)
at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:109)
at com.datastax.spark.connector.cql.CassandraConnector.withClusterDo(CassandraConnector.scala:120)
at com.datastax.spark.connector.cql.Schema$.fromCassandra(Schema.scala:254)
at com.datastax.spark.connector.rdd.CassandraTableRowReaderProvider$class.tableDef(CassandraTableRowReaderProvider.scala:51)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef$lzycompute(CassandraTableScanRDD.scala:59)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef(CassandraTableScanRDD.scala:59)
at com.datastax.spark.connector.rdd.CassandraTableRowReaderProvider$class.verify(CassandraTableRowReaderProvider.scala:146)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.verify(CassandraTableScanRDD.scala:59)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.getPartitions(CassandraTableScanRDD.scala:143)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1293)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.take(RDD.scala:1288)
at com.datastax.spark.connector.rdd.CassandraRDD.take(CassandraRDD.scala:121)
at com.datastax.spark.connector.rdd.CassandraRDD.take(CassandraRDD.scala:122)
at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1328)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.first(RDD.scala:1327)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:47)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:49)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:51)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:53)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:55)
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:57)
at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:59)
at $iwC$$iwC$$iwC$$iwC.<init>(<console>:61)
at $iwC$$iwC$$iwC.<init>(<console>:63)
at $iwC$$iwC.<init>(<console>:65)
at $iwC.<init>(<console>:67)
at <init>(<console>:69)
at .<init>(<console>:73)
at .<clinit>(<console>)
at .<init>(<console>:7)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)
at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
at org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:709)
at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:674)
at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:667)
at org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
at org.apache.zeppelin.scheduler.Job.run(Job.java:169)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

以下 jar 装满了 Spark
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar  System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar System Classpath
http://192.168.99.1:51164/jars/com.twitter_jsr166e-1.1.0.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/guava-18.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar System Classpath
http://192.168.99.1:51164/jars/com.datastax.cassandra_cassandra-driver-core-3.0.0.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar System Classpath
http://192.168.99.1:51164/jars/joda-time_joda-time-2.3.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar System Classpath
http://192.168.99.1:51164/jars/io.netty_netty-common-4.0.33.Final.jar Added By User
http://192.168.99.1:51164/jars/io.netty_netty-codec-4.0.33.Final.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar System Classpath
http://192.168.99.1:51164/jars/io.netty_netty-transport-4.0.33.Final.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/etc/hadoop/ System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar System Classpath
http://192.168.99.1:51164/jars/io.dropwizard.metrics_metrics-core-3.1.2.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/ System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar System Classpath
http://192.168.99.1:51164/jars/io.netty_netty-buffer-4.0.33.Final.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar System Classpath
http://192.168.99.1:51164/jars/org.apache.cassandra_cassandra-clientutil-3.0.2.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar System Classpath
http://192.168.99.1:51164/jars/com.google.guava_guava-16.0.1.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar System Classpath
/Users/user/Downloads/zeppelin-0.5.6-incubating-bin-all/interpreter/spark/zeppelin-spark-0.5.6-incubating.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/contrib/capacity-scheduler/*.jar System Classpath
http://192.168.99.1:51164/jars/datastax_spark-cassandra-connector-1.5.1-s_2.10.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-18.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar System Classpath
/Users/user/jars/ System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar System Classpath
http://192.168.99.1:51164/jars/org.joda_joda-convert-1.2.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar System Classpath
/Users/user/Downloads/spark-1.6.0-bin-without-hadoop/lib/spark-assembly-1.6.0-hadoop2.2.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar System Classpath
http://192.168.99.1:51164/jars/org.apache.commons_commons-lang3-3.3.2.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar System Classpath
/Users/user/Downloads/spark-1.6.0-bin-without-hadoop/conf/ System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar System Classpath
http://192.168.99.1:51164/jars/org.scala-lang_scala-reflect-2.10.5.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar System Classpath
http://192.168.99.1:51164/jars/io.netty_netty-handler-4.0.33.Final.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar System Classpath
http://192.168.99.1:51164/jars/org.slf4j_slf4j-api-1.7.7.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar System Classpath
http://192.168.99.1:51164/jars/zeppelin-spark-0.5.6-incubating.jar Added By User
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/common/lib/guava-18.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar System Classpath
/Users/user/Downloads/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar System Classpath

最佳答案

齐柏林飞艇似乎是一个问题。我已经与他们https://issues.apache.org/jira/browse/ZEPPELIN-1205打开了JIRA。通过升级Zeppelin的Spark解释器中的 Guava 版本可以解决此问题。

将 Guava 版本从zeppelin / spark / pom.xml中的14.0.1更改为16.0.1,然后使用编译后,即可解决问题
mvn clean package -DskipTests -Pcassandra-spark-1.5 -Phadoop-2.6

关于hadoop - cassandra Spark 连接器可从 Spark 壳工作,但在齐柏林飞艇中失败,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/38282438/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com