gpt4 book ai didi

java - 如何在执行 'Unsupported class file major version 55' 时修复 'org.apache.spark.sql.DataSet.collectAsList()'

转载 作者:行者123 更新时间:2023-11-30 01:51:22 27 4
gpt4 key购买 nike

我正在创建一个 Java RESTAPI Spring Boot 应用程序,它使用 Spark 从服务器获取一些数据。当我尝试从数据集转换为列表时,它失败了。

我尝试过jdk8和jdk11来编译和执行代码,但我得到了相同的“java.lang.IllegalArgumentException:不支持的类文件主要版本55”,在过去,我通过更新Java版本解决了这个问题,但这不起作用。

我正在使用:

  • JDK 11.0.2

  • Spring Boot 2.1.4

  • Spark 2.4.2

这是我正在执行的代码:

Dataset<Row> dataFrame = sparkSession.read().json("/home/data/*.json");
dataFrame.createOrReplaceTempView("events");
Dataset<Row> resultDataFrame = sparkSession.sql("SELECT * FROM events WHERE " + predicate);
Dataset<Event> eventDataSet = resultDataFrame.as(Encoders.bean(Event.class));
return eventDataSet.collectAsList();

查询有效,实际上在调试时您可以在 resultDataFrame 和 eventDataSet 中看到信息。

我希望输出是正确的事件列表,但我遇到了异常:

[http-nio-8080-exec-2] ERROR org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalArgumentException: Unsupported class file major version 55] with root cause
java.lang.IllegalArgumentException: Unsupported class file major version 55
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:166)
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:148)
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:136)
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:237)
at org.apache.spark.util.ClosureCleaner$.getClassReader(ClosureCleaner.scala:49)
at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:517)
at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:500)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)
at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:134)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
at org.apache.spark.util.FieldAccessFinder$$anon$3.visitMethodInsn(ClosureCleaner.scala:500)
.....

按评论更新:对于 Java 8,我将 pom 更改为针对 java 8:

<java.version>1.8</java.version>

然后更新项目,maven clean,maven install然后运行。收到相同版本 55 错误

最佳答案

从spark-core依赖项中排除默认的XBean工件并添加最新版本的XBean工件,它对我有用。

<dependencies>
<dependency>
<groupId>org.apache.xbean</groupId>
<artifactId>xbean-asm6-shaded</artifactId>
<version>4.10</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.1</version>
<exclusions>
<exclusion>
<groupId>org.apache.xbean</groupId>
<artifactId>xbean-asm6-shaded</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>

关于java - 如何在执行 'Unsupported class file major version 55' 时修复 'org.apache.spark.sql.DataSet.collectAsList()',我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55923943/

27 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com