nvidia - 使用 Chapel 使用两个 Nvidia Jetson nano 开发工具包时遇到问题

Question

我在类的一个项目中使用 Chapel，我正在尝试使用我的两个 Nvidia Jetson nano 板进行多语言环境执行。按照本教程 https://chapel-lang.org/docs/usingchapel/multilocale.html#readme-multilocale .当我尝试运行 hello 可执行文件时，我不断收到错误消息。我通过执行“./hello -nl 2”来运行代码

这是我不断收到的错误信息

*** GASNET WARNING(Node 0): int sendPacket(ep_t, amudp_msg_t*, size_t, en_t, packet_type) returning an error code: AM_ERR_RESOURCE (Problem with requested resource)
  from function sendPacket
  at /home/chico/chapel-1.20.0/third-party/gasnet/gasnet-src/other/amudp/amudp_reqrep.cpp:112
  reason: Invalid argument

*** GASNET WARNING(Node 0): int AMUDP_RequestGeneric(amudp_category_t, ep_t, amudp_node_t, handler_t, void*, size_t, uintptr_t, int, va_list, uint8_t, uint8_t) returning an error code: AM_ERR_RESOURCE (Problem with requested resource)
  at /home/chico/chapel-1.20.0/third-party/gasnet/gasnet-src/other/amudp/amudp_reqrep.cpp:1045

GASNet gasnetc_AMRequestShort encountered an AM Error: AM_ERR_RESOURCE(3)
  at /home/chico/chapel-1.20.0/third-party/gasnet/gasnet-src/udp-conduit/gasnet_core.c:827
*** WARNING (proc 0): GASNet gasnetc_AMRequestShort returning an error code: GASNET_ERR_RESOURCE (Problem with requested resource)
  at /home/chico/chapel-1.20.0/third-party/gasnet/gasnet-src/udp-conduit/gasnet_core.c:829
*** FATAL ERROR(Node 1): An active message was returned to sender,
    and trapped by the default returned message handler (handler 0):
Error Code: ECONGESTION: Congestion at destination endpoint                
Message type: AM_REQUEST_M
Destination: (127.0.0.1:52171) (0)
Handler: 64
Tag: 0x7f0001010000629e
Arguments(5): 0x00000000  0x00000001  0x00000000  0x00000000  0x00000009  
Aborting...
*** Caught a fatal signal (proc 1): SIGABRT(6)
NOTICE: Before reporting bugs, run with GASNET_BACKTRACE=1 in the environment to generate a backtrace. 
NOTICE: We recommend linking the debug version of GASNet to assist you in resolving this application issue.
bash: line 1:  1658 Aborted                 (core dumped) env 'AMUDP_SLAVE_ARGS=1,JetsonNano:52561,' './hello_real' '-nl' '2' '-E' 'LD_LIBRARY_PATH=:/usr/local/cuda/lib64' '-E' 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' '-E' 'SSH_CONNECTION=132.241.216.227 8794 192.168.1.20 22' '-E' 'LESSCLOSE=/usr/bin/lesspipe %s %s' '-E' 'LANG=en_US.UTF-8' '-E' 'CHPL_REGEXP=none' '-E' 'OLDPWD=/home/chico/chapel-1.20.0' '-E' 'CHPL_GMP=none' '-E' 'LLVM_CONFIG=/usr/bin/llvm-config-7' '-E' 'XDG_SESSION_ID=132' '-E' 'USER=chico' '-E' 'PWD=/home/chico' '-E' 'HOME=/home/chico' '-E' 'SSH_CLIENT=132.241.216.227 8794 22' '-E' 'CHPL_COMM=gasnet' '-E' 'XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop' '-E' 'CHPL_MEM=cstdlib' '-E' 'GASNET_SSH_SERVERS=JetsonNano JetsonNano2' '-E' 'SPARK_HOME=/opt/spark' '-E' 'SSH_TTY=/dev/pts/4' '-E' 'MAIL=/var/mail/chico' '-E' 'TERM=xterm-256color' '-E' 'SHELL=/bin/bash' '-E' 'CHPL_TASKS=fifo' '-E' 'CHPL_LLVM=none' '-E' 'SHLVL=1' '-E' 'GASNET_SPAWNFN=S' '-E' 'MANPATH=/home/chico/chapel-1.20.0/man:' '-E' 'CHPL_HOME=/home/chico/chapel-1.20.0' '-E' 'LOGNAME=chico' '-E' 'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus' '-E' 'XDG_RUNTIME_DIR=/run/user/1001' '-E' 'PATH=/home/chico/.cargo/bin:/home/chico/.local/bin:/home/chico/chapel-1.20.0/bin/linux64-aarch64:/home/chico/chapel-1.20.0/util:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda-10.0/bin/:/opt/spark/bin:/opt/spark/sbin' '-E' 'LESSOPEN=| /usr/bin/lesspipe %s' '-E' '_=./hello'

如果我只使用本地主机，它就可以工作。

注意:我一直在我的两个板上使用 mpy4py 和 python 并且它有效。此外，我正在使用 ansible playbook 通过网络同步两个板。

Answer 1

此处报告的症状与 GASNet 初始化期间发生的故障有关，在该故障中，无法在实现 Chapel 程序的多个进程之间建立通信。不幸的是，有许多因素可能导致这种故障模式，例如不正确的 /etc/hosts 条目、防火墙阻止通信等。遇到此类问题的用户可以随时在 Chapel GitHub issues page 上提出一个问题尝试解决它。​​

具体案例中对问题的讨论和解决，读者不妨引用Chapel issue #14581特别是 this comment .