python - 为什么 scrapy-redis 不起作用？

Question

我从 github 下载了 scrapy-redis 并按照说明运行它但是它失败并给出了这个错误:

2013-01-04 17:38:50+0800 [-] ERROR: Unhandled error in Deferred:
2013-01-04 17:38:50+0800 [-] Unhandled Error
        Traceback (most recent call last):
          File "/usr/local/lib/python2.7/dist-packages/Scrapy-0.16.3-py2.7.egg/scrapy/cmdline.py", line 138, in _run_command
            cmd.run(args, opts)
          File "/usr/local/lib/python2.7/dist-packages/Scrapy-0.16.3-py2.7.egg/scrapy/commands/crawl.py", line 44, in run
            self.crawler.crawl(spider)
          File "/usr/local/lib/python2.7/dist-packages/Scrapy-0.16.3-py2.7.egg/scrapy/crawler.py", line 47, in crawl
            return self.engine.open_spider(spider, requests)
          File "/usr/local/lib/python2.7/dist-packages/Twisted-12.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py", line 1187, in unwindGenerator
            return _inlineCallbacks(None, gen, Deferred())
        --- <exception caught here> ---
          File "/usr/local/lib/python2.7/dist-packages/Twisted-12.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py", line 1045, in _inlineCallbacks
            result = g.send(result)
          File "/usr/local/lib/python2.7/dist-packages/Scrapy-0.16.3-py2.7.egg/scrapy/core/engine.py", line 218, in open_spider
            scheduler = self.scheduler_cls.from_crawler(self.crawler)
        exceptions.AttributeError: type object 'Scheduler' has no attribute 'from_crawler'

如何处理？谢谢。

Answer 1

问题是 scrapy-redis 是一年前使用 scrapy 0.13 编写的。当前版本的 scrapy 是 0.17，如果你比较 current source code带有 version from last year 的 scrapy/core/engine.py ，您可以看到导致错误的行是在这两个版本之间的某个时间点添加的(实际上，它是在 August 21st 2012 上添加的)

简而言之，错误的发生是因为scrapy-redis不再兼容当前版本的scrapy。可能值得联系开发人员询问他/她项目是否仍然有效，否则将需要 fork 存储库并自行修复。