python - 在 AWS 上使用 starcluster 和 ipython 进行集群计算

Question

我正在尝试使用 AWS 上的集群计算进行一些试验。我对此完全陌生并且遇到了一些问题。我正在尝试按照此处找到的教程进行操作:http://star.mit.edu/cluster/docs/latest/plugins/ipython.html#using-the-ipython-cluster .我使用 starcluster 通过以下内容启动集群实例:

starcluster start mycluster

一切都按预期进行，它表明 ipython 插件已加载。然后，我尝试按照教程中所示执行以下命令:

starcluster sshmaster mycluster -u myuser

然而，连接失败，并告诉我

Permission denied (publickey).

我可以使用

登录

starcluster sshmaster mycluster

所以我尝试登录到主服务器继续教程，但是当我尝试创建客户端时收到错误消息:

AssertionError: Not a valid connection file or url: 
u'/root/.ipython/profile_default/security/ipcontroller-client.json'

我看到的唯一不寻常的事情是集群启动时出现:

>>> Running plugin ipcluster
>>> Writing IPython cluster config files
>>> Starting IPython cluster with 7 engines
>>> Waiting for JSON connector file... 
>>> Creating IPCluster cache directory: /Users/username/.starcluster/ipcluster
>>> Saving JSON connector file to '/Users/username/.starcluster/ipcluster/mycluster-us-east-1.json'
!!! ERROR - Error occurred while running plugin 'ipcluster':
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/cluster.py", line 1506, in run_plugin
    func(*args)
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/plugins/ipcluster.py", line 276, in run
    plug.run(nodes, master, user, user_shell, volumes)
  File "<string>", line 2, in run
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/utils.py", line 87, in wrap_f
    res = func(*arg, **kargs)
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/plugins/ipcluster.py", line 228, in run
    cfile = self._start_cluster(master, n, profile_dir)
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/plugins/ipcluster.py", line 173, in _start_cluster
    master.ssh.get(json, local_json)
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/sshutils/__init__.py", line 431, in get
    self.scp.get(remotepaths, localpath, recursive=recursive)
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/sshutils/scp.py", line 141, in get
    self._recv_all()
  File "/Library/Python/2.7/site-packages/StarCluster-0.93.3-py2.7.egg/starcluster/sshutils/scp.py", line 242, in _recv_all
    msg = self.channel.recv(1024)
  File "build/bdist.macosx-10.8-intel/egg/ssh/channel.py", line 611, in recv
    raise socket.timeout()
timeout

有什么想法吗？

Answer 1

本教程假定 ~/.starcluster/config 中的 CLUSTER_USER = myuser，即使默认情况下 CLUSTER_USER = sgeadmin