python - org.apache.spark.SparkException : Python worker exited unexpectedly (crashed)

Question

Spark 新手。只是想制作简单的字数统计程序，但在最后一步 counts.take(10) 时，出现以下错误。

input_file=sc.textFile("C:/Users/abhishek.vij/Desktop/abhi.txt")
map = input_file.flatMap(lambda line: line.split(" ")).map(lambda word: (word, 1))
counts = map.reduceByKey(lambda a, b: a + b)
counts.take(10)

19/10/26 21:14:42 ERROR Executor: Exception in task 1.0 in stage 0.0 (TID 1) org.apache.spark.SparkException: Python worker exited unexpectedly (crashed)

Answer 1

val counts = input_file.flatMap(line => line.split(" ")
     ).map(word => (word,1)).reduceByKey(_+_) counts.collect 

我认为这对你有用