apache-spark - 工作人员无法连接到同一台机器上的主机(无效关联)——即使 url 是正确的

Question

spark master日志包括以下内容:

15/05/19 21:05:19 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkMaster@mellyrn.local:7077]

但是 Worker 无法连接:

15/05/19 21:27:13 INFO Worker: Connecting to master akka.tcp://sparkMaster@mellyrn.local:7077/user/Master... 15/05/19 21:27:13 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@mellyrn.local:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: Connection refused: mellyrn.local/25.101.19.24:7077 15/05/19 21:27:25 INFO Worker: Retrying connection to master (attempt # 1) 15/05/19 21:27:25 INFO Worker: Connecting to master akka.tcp://sparkMaster@mellyrn.local:7077/user/Master... 15/05/19 21:27:25 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@mellyrn.local:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: Connection refused: mellyrn.local/25.101.19.24:7077

有什么提示可以在这里尝试吗？

Answer 1

检查你的文件 conf/spark-defaults.conf，将 spark.master 设置为 spark://mellyrn.local:7077

此外，请记住配置无密码的 ssh 访问。

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys