gpt4 book ai didi

scala - 为什么使用案例类对 JSON 进行编码时会出现错误 "Unable to find encoder for type stored in a Dataset"?

转载 作者:行者123 更新时间:2023-12-03 15:13:12 25 4
gpt4 key购买 nike

我写过 Spark 工作:

object SimpleApp {
def main(args: Array[String]) {
val conf = new SparkConf().setAppName("Simple Application").setMaster("local")
val sc = new SparkContext(conf)
val ctx = new org.apache.spark.sql.SQLContext(sc)
import ctx.implicits._

case class Person(age: Long, city: String, id: String, lname: String, name: String, sex: String)
case class Person2(name: String, age: Long, city: String)

val persons = ctx.read.json("/tmp/persons.json").as[Person]
persons.printSchema()
}
}

在 IDE 中,当我运行 main 函数时,出现 2 个错误:
Error:(15, 67) Unable to find encoder for type stored in a Dataset.  Primitive types (Int, String, etc) and Product types (case classes) are supported by importing sqlContext.implicits._  Support for serializing other types will be added in future releases.
val persons = ctx.read.json("/tmp/persons.json").as[Person]
^

Error:(15, 67) not enough arguments for method as: (implicit evidence$1: org.apache.spark.sql.Encoder[Person])org.apache.spark.sql.Dataset[Person].
Unspecified value parameter evidence$1.
val persons = ctx.read.json("/tmp/persons.json").as[Person]
^

但在 Spark Shell 中,我可以运行此作业而不会出现任何错误。问题是什么?

最佳答案

错误消息说 Encoder无法采取Person案例类。

Error:(15, 67) Unable to find encoder for type stored in a Dataset.  Primitive types (Int, String, etc) and Product types (case classes) are supported by importing sqlContext.implicits._  Support for serializing other types will be added in future releases.

将案例类的声明移到 SimpleApp 范围之外.

关于scala - 为什么使用案例类对 JSON 进行编码时会出现错误 "Unable to find encoder for type stored in a Dataset"?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34715611/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com