gpt4 book ai didi

java - Spark和java编译错误

转载 作者:行者123 更新时间:2023-11-30 06:32:17 25 4
gpt4 key购买 nike

当我使用maven编译spark java程序时,出现这样的编译错误

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/spark/java/src/main/java/SimpleApp.java:[9,36] cannot find symbol
symbol: variable read
location: variable spark of type org.apache.spark.sql.SparkSession

这是我的 JAVA 程序

import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.Dataset;


public class SimpleApp {
public static void main(String[] args) {
String logFile = "/home/spark/spark-2.2.0-bin-hadoop2.7/README.md"; // Should be some file on your system
SparkSession spark = SparkSession.builder().appName("Simple Application").getOrCreate();
Dataset<String> logData = spark.read.textFile(logFile).cache();
// Dataset<String> logData = SparkSession.builder().appName("Simple Application").getOrCreate().read.textFile(logFile).cache();
long numAs = logData.filter(s -> s.contains("a")).count();
long numBs = logData.filter(s -> s.contains("b")).count();

System.out.println("Lines with a: " + numAs + ", lines with b: " + numBs);

spark.stop();
}
}

我按照官方网站编写程序: http://spark.apache.org/docs/latest/quick-start.html这个错误是怎么来的..?

这是我的 pom.xml

<project>
<groupId>edu.berkeley</groupId>
<artifactId>simple-project</artifactId>
<modelVersion>4.0.0</modelVersion>
<name>Simple Project</name>
<packaging>jar</packaging>
<version>1.0</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>

<dependencies>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.2.0</version>
</dependency>
</dependencies>
</project>

最佳答案

好的...我找到了结果..
官方网站的例子是错误的..

spark.read.textFile(logFile).cache();   -->   spark.read().textFile(logFile).cache();

read 应该是 read()

关于java - Spark和java编译错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/45852522/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com