gpt4 book ai didi

java - Pyspark:异常:Java 网关进程在向驱动程序发送其端口号之前退出

转载 作者:IT老高 更新时间:2023-10-28 20:53:36 29 4
gpt4 key购买 nike

我正在尝试在我的 macbook air 上运行 pyspark。当我尝试启动它时,我得到了错误:

Exception: Java gateway process exited before sending the driver its port number

当 sc = SparkContext() 在启动时被调用。我试过运行以下命令:

./bin/pyspark
./bin/spark-shell
export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

无济于事。我也看过这里:

Spark + Python - Java gateway process exited before sending the driver its port number?

但这个问题从未得到解答。请帮忙!谢谢。

最佳答案

一个可能的原因是没有设置 JAVA_HOME,因为没有安装 java。

我遇到了同样的问题。它说

Exception in thread "main" java.lang.UnsupportedClassVersionError: org/apache/spark/launcher/Main : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:643)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:296)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:406)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/spark/python/pyspark/conf.py", line 104, in __init__
SparkContext._ensure_initialized()
File "/opt/spark/python/pyspark/context.py", line 243, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway()
File "/opt/spark/python/pyspark/java_gateway.py", line 94, in launch_gateway
raise Exception("Java gateway process exited before sending the driver its port number")
Exception: Java gateway process exited before sending the driver its port number

sc = pyspark.SparkConf()。我通过运行解决了它

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

来自 https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

关于java - Pyspark:异常:Java 网关进程在向驱动程序发送其端口号之前退出,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31841509/

29 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com