gpt4 book ai didi

java - 缺少应用程序资源

转载 作者:搜寻专家 更新时间:2023-11-01 03:47:25 28 4
gpt4 key购买 nike

使用java编写spark程序,代码如下:

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.Function;

public class SimpleApp {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setAppName("wordCount").setMaster("local");
JavaSparkContext sc = new JavaSparkContext(conf);
JavaRDD<String> input = sc.textFile("/bigdata/softwares/spark-2.1.0-bin-hadoop2.7/testdata/a.txt");
System.out.println();
Long bCount = input.filter(new Function<String,Boolean>(){
public Boolean call(String s){return s.contains("yes");}
}).count();
Long cCount = input.filter(new Function<String,Boolean>(){
public Boolean call(String s){return s.contains("ywq");}
}).count();
System.out.println("yes:"+bCount+" ywq:"+cCount+" all:");
// sc.stop();
}
}

pom如下:

<dependencies>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

出现以下错误ma​​ven所有资源都打包成jar文件,运行的时候报如下错误,刚开始学,谁知道教教,谢谢

enter image description here

最佳答案

您还必须使用 spark-submit 指定您的主类

spark-submit --class <your.package>.SimpleApp testjar/spark-0.0.1-SNAPSHOT.jar

关于java - 缺少应用程序资源,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/42834723/

28 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com