Scrapy Bench/Benchmark 命令错误

Question

我已经安装了 Scrapy 0.22.2 并且能够运行 DirBot 代码示例而没有任何问题。但是，当我运行 Bench 命令时，出现了一些错误和异常。除了端口 8998 不接受连接之外，下面还有什么问题吗？

    C:\>scrapy bench
Traceback (most recent call last):
  File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\lib\site-packages\scrapy-0.22.2-py2.7.egg\scrapy\tests\mocks
erver.py", line 198, in <module>
    os.path.join(os.path.dirname(__file__), 'keys/cert.pem'),
  File "C:\Python27\lib\site-packages\twisted\internet\ssl.py", line 70, in __in
it__
    self.cacheContext()
  File "C:\Python27\lib\site-packages\twisted\internet\ssl.py", line 79, in cach
eContext
    ctx.use_certificate_file(self.certificateFileName)
OpenSSL.SSL.Error: [('system library', 'fopen', 'No such process'), ('BIO routin
es', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_certificate_file'
, 'system lib')]
2014-04-07 14:30:39-0500 [scrapy] INFO: Scrapy 0.22.2 started (bot: scrapybot)
2014-04-07 14:30:39-0500 [scrapy] INFO: Optional features available: ssl, http11

2014-04-07 14:30:39-0500 [scrapy] INFO: Overridden settings: {'CLOSESPIDER_TIMEO
UT': 10, 'LOG_LEVEL': 'INFO', 'LOGSTATS_INTERVAL': 1}
2014-04-07 14:30:40-0500 [scrapy] INFO: Enabled extensions: LogStats, TelnetCons
ole, CloseSpider, WebService, CoreStats, SpiderState
2014-04-07 14:30:42-0500 [scrapy] INFO: Enabled downloader middlewares: HttpAuth
Middleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, Def
aultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, Redirec
tMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
2014-04-07 14:30:42-0500 [scrapy] INFO: Enabled spider middlewares: HttpErrorMid
dleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddlew
are
2014-04-07 14:30:42-0500 [scrapy] INFO: Enabled item pipelines:
2014-04-07 14:30:42-0500 [follow] INFO: Spider opened
2014-04-07 14:30:42-0500 [follow] INFO: Crawled 0 pages (at 0 pages/min), scrape
d 0 items (at 0 items/min)
2014-04-07 14:30:43-0500 [follow] INFO: Crawled 0 pages (at 0 pages/min), scrape
d 0 items (at 0 items/min)
2014-04-07 14:30:44-0500 [follow] INFO: Crawled 0 pages (at 0 pages/min), scrape
d 0 items (at 0 items/min)
2014-04-07 14:30:45-0500 [follow] INFO: Crawled 0 pages (at 0 pages/min), scrape
d 0 items (at 0 items/min)
2014-04-07 14:30:45-0500 [follow] ERROR: Error downloading <GET http://localhost
:8998/follow?total=100000&order=rand&maxlatency=0.0&show=20>: Connection was ref
used by other side: 10061: No connection could be made because the target machin
e actively refused it..
2014-04-07 14:30:45-0500 [follow] INFO: Closing spider (finished)
2014-04-07 14:30:45-0500 [follow] INFO: Dumping Scrapy stats:
        {'downloader/exception_count': 3,
         'downloader/exception_type_count/twisted.internet.error.ConnectionRefus
edError': 3,
         'downloader/request_bytes': 783,
         'downloader/request_count': 3,
         'downloader/request_method_count/GET': 3,
         'finish_reason': 'finished',
         'finish_time': datetime.datetime(2014, 4, 7, 19, 30, 45, 575000),
         'log_count/ERROR': 1,
         'log_count/INFO': 10,
         'scheduler/dequeued': 3,
         'scheduler/dequeued/memory': 3,
         'scheduler/enqueued': 3,
         'scheduler/enqueued/memory': 3,
         'start_time': datetime.datetime(2014, 4, 7, 19, 30, 42, 439000)}
2014-04-07 14:30:45-0500 [follow] INFO: Spider closed (finished)

Answer 1

您需要安装 cffi python 包，但在此之前你需要安装 ffi ，即 libffi-dev和 libffi在 Ubuntu 上:
sudo aptitude install libffi-dev libffisudo pip install cffi
您还需要安装 libssl-dev因为它用于 cryptography python 包。

之后，您应该使用以下命令重新安装scrapy:sudo pip install scrapy --upgrade
如果还没有解决问题，请从github安装最新版本的scrapy，tarball:
https://github.com/scrapy/scrapy/tarball/master
它对我有用..