python - Cassandra 未完成与 EventletConnection 的连接

Question

在一个稳定的环境中，当我调用cluster.connect()时，代码会无限期地挂起，没有错误或超时消息。该连接包含在 Nameko 服务内。我怀疑它与 Nameko 服务代码有关，特别是涉及的猴子修补，因为其他方面相同的代码在 Nameko 服务之外工作正常。我应该注意这似乎只是 Python 3.7 和 Cassandra 3.18 的问题。对于 Python 3.6 和 Cassandra 3.16，这可以正常工作。这是一个代码片段:

import logging
from cassandra import ConsistencyLevel, cluster

from nameko.extensions import DependencyProvider

from pipeline import config


class CassandraConnection(DependencyProvider):
    """
    Encapsulate a cassandra session with an open connection.
    """

    def __init__(self):

        self._logger = logging.getLogger('cassandra_writer')
        self._cluster = None
        self._session = None

    def setup(self):

        self._cluster = cluster.Cluster(config.get('cassandra', 'node').split(','))
        self._session = self._cluster.connect()
        self._session.set_keyspace(config.get('cassandra', 'keyspace_ts'))
        self._session.default_consistency_level = ConsistencyLevel.QUORUM

    def get_dependency(self, worker_ctx):

        return self._session

这是完整的启动日志:

INFO     starting services: cassandra_writer
DEBUG    starting <ServiceContainer [cassandra_writer] at 0x7fa5d1411b70>
DEBUG    starting <ExchangeEventHandler [cassandra_writer.write_objects] at 0x7fa5bc685ba8>
DEBUG    registering provider <ExchangeEventHandler [cassandra_writer.write_objects] at 0x7fa5bc685ba8> for <QueueConsumer at 0x7fa5bc685c18>
DEBUG    Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@USRMQQA03.alphanetcapital.com', 'copyright': 'Copyright (C) 2007-2019 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP 21.2.6', 'product': 'RabbitMQ', 'version': '3.7.13'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
WARNING  Cluster.__init__ called with contact_points specified, but no load_balancing_policy. In the next major version, this will raise an error; please specify a load-balancing policy. (contact_points = ['uscstardev01', 'uscstardev02', 'uscstardev03'], lbp = None)
INFO     <class 'cassandra.io.eventletreactor.EventletConnection'>
DEBUG    Connecting to cluster, contact points: ['uscstardev01', 'uscstardev02', 'uscstardev03']; protocol version: 4
DEBUG    Host 172.16.9.29:9042 is now marked up
DEBUG    Host 172.16.9.30:9042 is now marked up
DEBUG    Host 172.16.9.31:9042 is now marked up
DEBUG    [control connection] Opening new connection to 172.16.9.31:9042
DEBUG    Sending initial options message for new connection (140349806704400) to 172.16.9.31:9042
DEBUG    Received options response on new connection (140349806704400) from 172.16.9.31:9042
DEBUG    Sending StartupMessage on <EventletConnection(140349806704400) 172.16.9.31:9042>
DEBUG    Sent StartupMessage on <EventletConnection(140349806704400) 172.16.9.31:9042>
DEBUG    Got ReadyMessage on new connection (140349806704400) from 172.16.9.31:9042
DEBUG    [control connection] Established new connection <EventletConnection(140349806704400) 172.16.9.31:9042>, registering watchers and refreshing schema and topology
DEBUG    [control connection] Refreshing node list and token map using preloaded results
INFO     Using datacenter 'devus01' for DCAwareRoundRobinPolicy (via host '172.16.9.31:9042'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes
DEBUG    [control connection] Finished fetching ring info
DEBUG    [control connection] Rebuilding token map due to topology changes
DEBUG    Control connection created
DEBUG    Initializing connection for host 172.16.9.29:9042
DEBUG    Sending initial options message for new connection (140349806806968) to 172.16.9.29:9042
DEBUG    Initializing connection for host 172.16.9.30:9042
DEBUG    Sending initial options message for new connection (140349805903544) to 172.16.9.30:9042
DEBUG    Received options response on new connection (140349806806968) from 172.16.9.29:9042
DEBUG    Sending StartupMessage on <EventletConnection(140349806806968) 172.16.9.29:9042>
DEBUG    Sent StartupMessage on <EventletConnection(140349806806968) 172.16.9.29:9042>
DEBUG    Received options response on new connection (140349805903544) from 172.16.9.30:9042
DEBUG    Sending StartupMessage on <EventletConnection(140349805903544) 172.16.9.30:9042>
DEBUG    Sent StartupMessage on <EventletConnection(140349805903544) 172.16.9.30:9042>
DEBUG    Got ReadyMessage on new connection (140349806806968) from 172.16.9.29:9042
DEBUG    Finished initializing connection for host 172.16.9.29:9042
DEBUG    Added pool for host 172.16.9.29:9042 to session
DEBUG    Initializing connection for host 172.16.9.31:9042
DEBUG    Got ReadyMessage on new connection (140349805903544) from 172.16.9.30:9042
DEBUG    Finished initializing connection for host 172.16.9.30:9042
DEBUG    Added pool for host 172.16.9.30:9042 to session

最后一条消息是进程挂起之前出现的最后一条消息。

这是我的环境:

adyne_services_common  0.1  py37_0
alphasuite  1.79a1  py37_0
amqp  2.3.2  py_1
appdirs  1.4.3  py37h28b3542_0
arrow  0.12.1  py37_1
asn1crypto  0.24.0  py37_0
attrs  18.2.0  py37h28b3542_0
backcall  0.1.0  py37_0
backports  1  py37_1
backports.os  0.1.1  py37_0
bcrypt  3.1.5  py37h7b6447c_0
beautifulsoup4  4.6.3  py37_0
billiard  3.6.0.0  py37h516909a_0
blas  1  mkl
blpapi  3.12.2  py37_blpapicpp3.12.2.1_1
ca-certificates  2018.03.07  0
cached-property  1.5.1  py37_0
cachetools  3.0.0  py_0
cassandra-driver  3.18.0  py37_0
celery  4.3.0  py_1
certifi  2018.11.29  py37_0
cffi  1.11.5  py37he75722e_1
chardet  3.0.4  py37_1
conda  4.5.12  py37_0
conda-env  2.6.0  1
cryptography  2.4.2  py37h1ba5d50_0
cycler  0.10.0  py37_0
cython  0.28.5  py37hf484d3e_0
dbus  1.13.2  h714fa37_1
decorator  4.3.0  py37_0
defusedxml  0.5.0  py37_1
distro  1.4.0  py_0
elasticsearch  5.4.0  py37_0
et_xmlfile  1.0.1  py37_0
eventlet  0.23.0  py37_1000
expat  2.2.6  he6710b0_0
fontconfig  2.13.0  h9420a91_0
freetype  2.9.1  h8a8886c_1
glib  2.56.2  hd408876_0
greenlet  0.4.15  py37h7b6447c_0
gst-plugins-base  1.14.0  hbbd80ab_1
gstreamer  1.14.0  hb453b48_1
icu  58.2  h211956c_0
idna  2.8  py37_0
importlib_metadata  0.6  py37_0
intel-openmp  2019.1  144
ipython  7.2.0  py37h39e3cac_0
ipython_genutils  0.2.0  py37_0
isodate  0.6.0  py37_0
jdcal  1.4  py37_0
jedi  0.13.2  py37_0
jinja2  2.1  py37_0
jpeg  9b  habf39ab_1
jsonschema  2.6.0  py37_0
kiwisolver  1.0.1  py37hf484d3e_0
kombu  4.6.1  py_0
krb5  1.16.1  h173b8e3_7
libedit  3.1.20170329  h6b74fdf_2
libffi  3.2.1  h4deb6c0_3
libgcc-ng  8.2.0  hdf63c60_1
libgfortran-ng  7.3.0  hdf63c60_0
libpng  1.6.35  hbc83047_0
libsodium  1.0.16  h1bed415_0
libstdcxx-ng  8.2.0  hdf63c60_1
libuuid  1.0.3  h1bed415_2
libxcb  1.13  h1bed415_1
libxml2  2.9.8  h26e45fe_1
libxslt  1.1.32  h1312cb7_0
lxml  4.2.5  py37hefd8a0e_0
lz4  2.1.6  py37h7b6447c_0
lz4-c  1.8.1.2  h14c3975_0
markupsafe  1.1.0  py37h7b6447c_0
matplotlib  3.0.2  py37h5429711_0
mkl  2019.1  144
mkl_fft  1.0.6  py37hd81dba3_0
mkl_random  1.0.2  py37hd81dba3_0
mock  2.0.0  py37_0
mysqlclient  1.3.14  ubuntu1404
nameko  2.11.0  py37_3
ncurses  6.1  he6710b0_1
numpy  1.15.4  py37h7e9f1db_0
numpy-base  1.15.4  py37hde5b4d6_0
openjdk  8.0.121  1
openpyxl  2.5.12  py37_0
openssl  1.1.1  h7b6447c_0
pandas  0.23.4  py37h04863e7_0
paramiko  2.4.2  py37_0
parso  0.3.1  py37_0
path.py  11.5.0  py37_0
patsy  0.5.1  py37_0
pbr  5.1.1  py37_0
pcre  8.42  h439df22_0
pexpect  4.6.0  py37_0
pickleshare  0.7.5  py37_0
pika  0.12.0  py_1
pip  18.1  py37_0
pipeline  0.68a1  py37_1
prompt_toolkit  2.0.7  py37_0
ptyprocess  0.6.0  py37_0
pyasn1  0.4.5  py_0
pycosat  0.6.3  py37h14c3975_0
pycparser  2.19  py37_0
pygments  2.3.1  py37_0
pykerberos  1.2.1  py37h14c3975_0
pynacl  1.2.1  py37h14c3975_0
pyodbc  4.0.25  py37he6710b0_0
pyopenssl  18.0.0  py37_0
pyparsing  2.3.0  py37_0
pyqt  5.9.2  py37h05f1152_2
pysocks  1.6.8  py37_0
python  3.7.1  h0371630_7
python-dateutil  2.7.5  py37_0
pytz  2018.7  py37_0
pyvolar  3.10.8  py37_1
pyyaml  3.13  py37h14c3975_0
qt  5.9.7  h5867ecd_1
readline  7  h7b6447c_5
redis-py  3.2.0  py_0
requests  2.21.0  py37_0
requests-kerberos  0.12.0  py37_0
requests-toolbelt  0.8.0  py37_0
ruamel_yaml  0.15.46  py37h14c3975_0
scipy  1.1.0  py37h7c811a0_2
seaborn  0.9.0  py37_0
setuptools  40.6.3  py37_0
simplejson  3.16.0  py37h14c3975_0
sip  4.19.8  py37hf484d3e_0
six  1.12.0  py37_0
sqlalchemy  1.2.15  py37h7b6447c_0
sqlite  3.26.0  h7b6447c_0
statsmodels  0.9.0  py37h035aef0_0
tabula-py  1.1.1  py37_1000
tk  8.6.8  hbc83047_0
tornado  5.1.1  py37h7b6447c_0
traitlets  4.3.2  py37_0
ujson  1.35  py37h14c3975_0
unixodbc  2.3.7  h14c3975_0
urllib3  1.24.1  py37_0
vine  1.3.0  py_0
wcwidth  0.1.7  py37_0
werkzeug  0.14.1  py37_0
wheel  0.32.3  py37_0
wrapt  1.10.11  py37h14c3975_2
xlrd  1.2.0  py37_0
xlwt  1.3.0  py37_0
xz  5.2.4  h14c3975_4
yaml  0.1.7  h96e3832_1
zeep  3.2.0  py_1000
zlib  1.2.11  h7b6447c_3

Answer 1

这是我们在上一个 cassandra-driver 版本 (3.19.0) 中修复的问题。但是，您需要安装“futurist”包才能使 eventlet 与 Python 3.7 一起使用。更多详情请参见https://github.com/eventlet/eventlet/issues/508