python - 在 Raspberry Pi 3、python 3.4 上运行 Scrapy

Question

我在 Raspberry Pi 3 上运行 scrapy 时收到错误。

我已经成功安装了它，但是当我尝试使用之前创建的蜘蛛启动项目或爬网时，出现以下错误:

Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/usr/local/lib/python3.4/dist-packages/scrapy/cmdline.py", line 9, in <module>
    from scrapy.crawler import CrawlerProcess
  File "/usr/local/lib/python3.4/dist-packages/scrapy/crawler.py", line 7, in <module>
    from twisted.internet import reactor, defer
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/reactor.py", line 38, in <module>
    from twisted.internet import default
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 44, in _getInstallFunction
    from twisted.internet.epollreactor import install
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/epollreactor.py", line 24, in <module>
    from twisted.internet import posixbase
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "/usr/local/lib/python3.4/dist-packages/twisted/protocols/tls.py", line 65, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1865, in <module>
    "ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:"
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1845, in fromOpenSSLCipherString
    SSL.SSLv23_METHOD, SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1797, in _expandCipherString
    ctx.set_cipher_list(cipherString.encode('ascii'))
TypeError: must be str, not bytes

我不知道为什么会收到此消息或如何解决它，请帮忙？

解决方案:

感谢大家的帮助，我最终通过以下步骤让Scrapy在Raspberry Pi 3上工作:

首先安装virtualenv:

sudo pip install virtualenv

然后创建一个 virtualenv 并为 Scrapy 激活它:

virtualenv scrapyenv
source scrapyenv/bin/activate

然后我运行并更新了其中的所有内容:

apt-get update
apt-get upgrade

安装所有依赖:

apt-get install libffi-dev
apt-get install libxml2-dev
apt-get install libxslt1-dev
apt-get install libssl-dev
apt-get install python-dev

然后安装Scrapy

sudo pip install scrapy

然后我用这个更新了我的 pyOpenSSL:

pip -vvvv install --upgrade pyOpenSSL

这创建了很多日志文件并花费了一些时间，之后 scrapy 使用普通的 scrapy 命令运行良好并且我还运行了一个蜘蛛 - 一切正常。

Answer 1

您使用的是什么用户？我认为你需要使用 sudo。还更新您的 ssl。看起来 ssl 连接有问题..

你跑过吗:pip install --verbose twisted

同时更新你的 openssl

pip -vvvv 安装 --upgrade pyOpenSSL,请将输出复制到此处以检查是否已更新

问候