gpt4 book ai didi

scala - 无法在 Spark Scala 中导入 org.apache.spark.streaming.twitter

转载 作者:行者123 更新时间:2023-12-02 00:03:54 24 4
gpt4 key购买 nike

以下导入无法在 SBT 中编译

import org.apache.spark.streaming.twitter._

[error] /home/hduser/workspace/TweetStream/src/main/scala/TweetStream.scala:8: object twitter is not a member of package org.apache.spark.streaming
[error] import org.apache.spark.streaming.twitter._
[error]

随后还有以下内容

val tweetStream = TwitterUtils.createStream(ssc, None, filters, StorageLevel.MEMORY_ONLY_SER_2).map(gson.toJson(_))


[error] /home/hduser/workspace/TweetStream/src/main/scala/TweetStream.scala:36: not found: value TwitterUtils
[error] val tweetStream = TwitterUtils.createStream(ssc, None, filters, StorageLevel.MEMORY_ONLY_SER_2).map(gson.toJson(_))
[error] ^
^

build.sbt 下面传递了所有依赖解析

name := "TweetStream"
version := "1.0"
scalaVersion := "2.11.7"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2"
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.5.2"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.11" % "1.5.2"
libraryDependencies += "com.google.code.gson" % "gson" % "2.7"
libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.4"

我是否添加了错误的依赖项?

最佳答案

您需要添加以下依赖项:

// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-twitter_2.11
libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.11" % "1.5.2"

PS:其他依赖项 Scala 版本可能会给您带来一些问题。您应该为其他 Spark 依赖项指定 _2.11。

关于scala - 无法在 Spark Scala 中导入 org.apache.spark.streaming.twitter,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/38145515/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com