python - PySFTP/Paramiko 异常泄漏到标准错误中

Question

我试图捕获 paramiko异常(exception)，但它们仍被写入标准错误。

有没有办法停止在那里写？

编辑:它甚至发生在 paramiko 参与之前:

import pysftp

try:
    pysftp.Connection(host="localhost")
except Exception as e:
    print(e)

结果是:



具有正确 SFTP 参数的示例:



更新:

$ pipenv graph
...
pysftp==0.2.9
  - paramiko [required: >=1.17, installed: 2.6.0]
...

$ pipenv run python
Python 3.7.3 (default, Jul 19 2019, 11:21:39)
[Clang 11.0.0 (clang-1100.0.28.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysftp
>>> try:
...     pysftp.Connection(host="localhost")
... except Exception as e:
...     print(e)
...
No hostkey for host localhost found.
Exception ignored in: <function Connection.__del__ at 0x10f7e8268>
Traceback (most recent call last):
  File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 1013, in __del__
    self.close()
  File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 784, in close
    if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
>>>

Answer 1

我想首先指出 PySFTP ( [PyPI]: PySFTP ) 没有维护 3+ 年(或已移动到不同的位置 - 到目前为止是保密的:))。

我重现了这个问题。以下是您的代码的更详细版本。

代码00.py:

#!/usr/bin/env python3

import sys
import pysftp
import traceback


def main(argv):
    hostname = argv[0] if argv else "localhost"
    print("Attempting to connect to {0:s} ...".format(hostname))
    try:
        print("----------Before conn----------")
        conn = pysftp.Connection(host=hostname)
        print("----------After conn----------")
    except:
        print("----------Before exc print----------")
        traceback.print_exc()
        print("----------After exc print----------")
    finally:
        print("----------Finally----------")
    print("----------After try / except / finally----------")

if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    print("pysftp version: {0:s}\n".format(pysftp.__version__))
    main(sys.argv[1:])
    print("\nDone.")

输出 :

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32
pysftp version: 0.2.9

Attempting to connect to localhost ...
----------Before conn----------
e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts.  You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None).
  warnings.warn(wmsg, UserWarning)
----------Before exc print----------
Traceback (most recent call last):
  File "code00.py", line 13, in main
    conn = pysftp.Connection(host=hostname)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 132, in __init__
    self._tconnect['hostkey'] = self._cnopts.get_hostkey(host)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey
    raise SSHException("No hostkey for host %s found." % host)
paramiko.ssh_exception.SSHException: No hostkey for host localhost found.
----------After exc print----------
Exception ignored in: <function Connection.__del__ at 0x000001CC720C80D0>
Traceback (most recent call last):
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 1013, in __del__
    self.close()
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 784, in close
    if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
----------Finally----------
----------After try / except / finally----------

Done.

这是一个 PySFTP 错误:

Connection 对象被构造(__new__)

初始化器(__init__)被调用

在初始化程序的某个地方发生异常

异常后的行没有执行

当对象(自动)被垃圾收集时(当它超出范围时，在 except block 的末尾)，在其关闭方法(由析构函数(__del__)调用)中，引用了一些属性

但由于这些属性初始化发生在 之后的 行(引发异常)，来自 #2.2。 ，它们从未初始化，因此它们的引用引发 AttributeError

修复很简单:在初始化程序的开头将属性初始化为一些默认值，因此如果发生上述情况，它们的引用并不代表问题。

我注意到您已经在 BitBucket 上提交了一个问题。

考虑到:

我不是 BitBucket 专家

我没有 [BitBucket]: dundeemt/pysftp - pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. 的权限(我既不能推送也不能提交拉取请求)

我创建了自己的存储库(在上述存储库中)，并将更改推送到: [BitBucket]: CristiFati0/pysftp - [Issue #144]: Exceptions leaking into stderr (到目前为止，一次提交)。

输出 (手动将修复应用到 pip 安装的文件后):

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32
pysftp version: 0.2.9

Attempting to connect to localhost ...
----------Before conn----------
e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts.  You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None).
  warnings.warn(wmsg, UserWarning)
----------Before exc print----------
Traceback (most recent call last):
  File "code00.py", line 13, in main
    conn = pysftp.Connection(host=hostname)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 135, in __init__
    self._tconnect['hostkey'] = self._cnopts.get_hostkey(host)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey
    raise SSHException("No hostkey for host %s found." % host)
paramiko.ssh_exception.SSHException: No hostkey for host localhost found.
----------After exc print----------
----------Finally----------
----------After try / except / finally----------

Done.

不用说，可能会针对不同的场景引发其他未处理的异常。

@EDIT0

显然，这不仅仅是表面上看到的。除了上面描述的 PySFTP 错误 之外，还有 2 件事会污染标准错误。

1. 警告

在我的情况下(因为我在 Win 上并且没有安装任何 native SSH 工具)它每次都会弹出(除非我在我的主目录中创建/复制一些有效的 known_hosts 文件)，但在 Nix 系统上它很可能不会吨。

无论如何，修复这个(如果需要)很容易，只需抑制 UserWarning 例如通过将 %PYTHONWARNINGS% env var 设置为 ignore::UserWarning (也可以通过代码实现 - 就像在下一个项目的情况下一样)。

2. Paramiko 异常

我能够通过手动修改 transport.py (并提高 socket.timeout )来重现这一点。

由 paramiko.Transport (由初始化程序调用)初始化的 pysftp.Connection._start_transport 在线程 中完成其工作 (通过子类化 threading.Thread )。在该线程中引发的任何异常 都无法被调用线程 (我们的)捕获。这是一个 Python 限制，计划在 v 3.8 ( [Python.Bugs]: threading.Thread should have way to catch an exception thrown within) 中解决。

对于这个，有一个(蹩脚的)解决方法(gainarie):重定向stderr。当然还有其他解决方法，但它们意味着修改 Paramiko，所以我建议不要使用它们。

下面是一个将 stderr 重定向到 stdout 的示例(但您可以选择任何其他文件 - 包括/dev/null(或 Win 上的 nul))。它是从代码中完成的(但也可以从解释器命令行中完成)，因此 它只会影响所需的(热)区域 。

代码01.py:

#!/usr/bin/env python3

import sys
import pysftp
import paramiko
import traceback
import threading


_sys_stderr = sys.stderr  # For restoring purposes


def main(argv):
    hostname = argv[0] if argv else "localhost"
    print("Attempting to connect to {0:s} ...".format(hostname))
    try:
        cnopts = pysftp.CnOpts()
        cnopts.hostkeys = None
        print("---------- STATS: {0:s} {1:d} ----------".format(__file__, threading.get_ident()))
        print("---------- Before conn ----------")
        sys.stderr.write("DUMMY TEXT before sent to stderr\n")
        sys.stderr = sys.stdout  # @TODO - cfati: decomment so that everything from stderr is redirected to stdout
        conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,)
        print("---------- After conn ----------")
    except:
        sys.stderr = _sys_stderr
        print("---------- Before exc tb ----------")
        traceback.print_exc(file=sys.stdout)
        print("---------- After exc tb ----------")
    finally:
        sys.stderr = _sys_stderr
        print("---------- Finally ----------")
        sys.stderr.write("DUMMY TEXT after sent to stderr\n")
    print("---------- After try / except / finally ----------")


if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    print("pysftp version: {0:s}\nparamiko version: {1:s}".format(pysftp.__version__, paramiko.__version__))
    main(sys.argv[1:])
    print("\nDone.")

输出 :

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]> dir /b
code00.py
code01.py

[prompt]> :: Suppress warning
[prompt]> set PYTHONWARNINGS=ignore::UserWarning

[prompt]> :: Redirect stdout and stderr to different files, so it is obvious which is which
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code01.py 1>1.out 2>1.err

[prompt]> type 1.err
DUMMY TEXT before sent to stderr
DUMMY TEXT after sent to stderr

[prompt]> type 1.out
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32
pysftp version: 0.2.9
paramiko version: 2.6.0
Attempting to connect to localhost ...
---------- STATS: code01.py 23016 ----------
---------- Before conn ----------
---------- STATS: e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py 45616 ----------
Exception: Error reading SSH protocol banner
Traceback (most recent call last):
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner
    raise socket.timeout()
socket.timeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run
    self._check_banner()
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner
    "Error reading SSH protocol banner" + str(e)
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

---------- Before exc tb ----------
Traceback (most recent call last):
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner
    raise socket.timeout()
socket.timeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "code01.py", line 23, in main
    conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 144, in __init__
    self._transport.connect(**self._tconnect)
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 1291, in connect
    self.start_client()
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 660, in start_client
    raise e
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run
    self._check_banner()
  File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner
    "Error reading SSH protocol banner" + str(e)
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
---------- After exc tb ----------
---------- Finally ----------
---------- After try / except / finally ----------

Done.

以及它从 PyCharm 中的外观(必须将其拉伸(stretch)到最大以适应整个事物):