python-3.x - 如何使用任务来命中在 azure 批处理节点中运行的 flask 应用程序？

Question

我能够配置以下设置，

[ ]

我可以在作业中添加一个任务来运行 testapi.py 文件，该文件接受一个简单的字符串作为输入并将其发送到在 docker 容器内运行的 Flask 应用程序。

但是，

任务执行抛出错误

> Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 183, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "preload_testapi.py", line 37, in <module>
    r = requests.post(url, json=got_json)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))

• 我做过的事情

  1. 我确保启动任务(运行 Flask 应用程序)是由我创建的 sudo 用户(不是 _azbatch 用户)执行，任务内部的作业也由同一用户执行。

  2. 我通过 ssh 测试了任务执行的相同命令与我创建的用户登录名相同的节点并且命令运行
     很好。

  3. 当我从 _azbatch 运行命令时，它会抛出相同的错误。

  4. 这不是 flask 端口问题。

@fpark 我的启动任务执行已正确完成，在节点创建期间，我从 Azure 容器注册表下载一个镜像，从 azure 存储 blob 下载 2 个文件作为资源文件，启动任务是运行这些文件之一(shell 脚本)，其中创建一个容器，从该图像在其中运行一个 Flask 应用程序。添加的任务是使用第二个文件(testapi.py)来运行正在运行的 Flask 应用程序，每个任务具有不同的参数。

Answer 1

您可以使用任务来调用 Flask API。请按照以下方式操作:

向您的计算节点提供一个包含两个任务的作业，其中任务 1 运行 Flask API，任务 2 调用 API。在尝试此操作之前，还要为您的节点启用并行任务执行。

希望对你有帮助(～￣▽￣)～