json - 无法从 spark 作业写入 redis

Question

我正在创建一个 scala hashmap 并尝试将其作为 Java hashmap 从 spark 作业写入 redis 中，但它给出了这个错误:

java.lang.NoClassDefFoundError:
redisson/jackson/databind/jsontype/TypeResolverBuilder
2018-08-28 10:39:48:: at org.redisson.config.Config.<init> . 
(Config.java:101)
2018-08-28 10:39:48:: at org.redisson.Redisson.<init> . 
(Redisson.java:113)
2018-08-28 10:39:48:: at 
org.redisson.Redisson.create(Redisson.java:154)
2018-08-28 10:39:48:: at 
com.oto.sims.writers.RedisWriter$.apply(RedisWriter.scala:237)
2018-08-28 10:39:48:: at com.oto.sims.userfeatureaggregator.usersrecommendation.UserRecommendationAnalyzer$$anonfun$trackRecTypewiseRequests$1.apply(UserRecommendationAnalyzer.scala:158)
2018-08-28 10:39:48:: at com.oto.sims.userfeatureaggregator.usersrecommendation.UserRecommendationAnalyzer$$anonfun$trackRecTypewiseRequests$1.apply(UserRecommendationAnalyzer.scala:155)
2018-08-28 10:39:48:: at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
2018-08-28 10:39:48:: at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
2018-08-28 10:39:48::   at com.oto.sims.engine.UserRecommendationActor$$anonfun$receive$1.applyOrElse(UserRecommendationActor.scala:24)
2018-08-28 10:39:48::   at akka.actor.Actor$class.aroundReceive(Actor.scala:514)
2018-08-28 10:39:48::   at com.oto.sims.engine.UserRecommendationActor.aroundReceive(UserRecommendationActor.scala:16)
2018-08-28 10:39:48::   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
2018-08-28 10:39:48::   at akka.actor.ActorCell.invoke(ActorCell.scala:496)
2018-08-28 10:39:48::   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
2018-08-28 10:39:48::   at akka.dispatch.Mailbox.run(Mailbox.scala:224)
2018-08-28 10:39:48::   at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
2018-08-28 10:39:48::   at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
2018-08-28 10:39:48::   at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
2018-08-28 10:39:48::   at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2018-08-28 10:39:48::   at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2018-08-28 10:39:48::Caused by: java.lang.ClassNotFoundException: 
redisson.jackson.databind.jsontype.TypeResolverBuilder
2018-08-28 10:39:48::   at 
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
2018-08-28 10:39:48::   at 
java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2018-08-28 10:39:48::   at 
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2018-08-28 10:39:48::   ... 23 more

这是我的 Dependencies.scala 文件的内容:

val commonDependencies: Seq[ModuleID] = Seq(
"org.scala-lang" % "scala-library" % "2.11.11",
log4j,
"mysql" % "mysql-connector-java" % mysqlVersion,
"com.typesafe" % "config" % "1.2.1",
"org.redisson" % "redisson" % "3.4.2",
"joda-time" % "joda-time" % "2.8",
"com.fasterxml.jackson.core" % "jackson-core" % "2.7.9",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.7.9",
"com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.7.9"

)

我没有使用 Redis 的经验。我们将不胜感激。

Answer 1

您提供的详细信息不完整，您在 Redis 服务器中使用的端口号什么操作系统，你运行的是一台计算机和虚拟机，还是不同的计算机。

不过，您可以通过检查以下几点来解决此问题:

-确保你能ping通服务器ip

• 尝试检查端口是否打开，您可以在终端上使用 telnet，如果您使用的是 windows 操作系统，则使用 Nbtstat。还要检查db mysql端口

• 将 jackson lib 更新到最新版本或版本号 1.9.13，以防最近的版本不起作用。