python - 如何根据父类中的 cli kwargs 重写 FEED_URI？

Question

我想在我的父爬虫类中设置以下内容，因为每个 child 的设置都应该相同，我该怎么做？

scrapy crawl spiderX -a full  >> FEED_URI = /xx/spiderX_full
scrapy crawl spiderX -a quick >> FEED_URI = /xx/spiderX_quick

这是我到目前为止所拥有的:

@classmethod
def update_settings(cls, settings):
    settings_dict = cls.custom_settings or {}
    feed_uri = path.join(settings.get('FEED_DIR'), '%s' % cls.name)
    settings_dict['FEED_URI'] = feed_uri
    settings.setdict(settings_dict, priority='spider')

如何从此函数访问快速/完整参数？我尝试这样做:

def __new__(cls, full=False, quick=False, *a, **kw):
    cls.full = full
    cls.quick = quick
    return super(MyCrawlSpider, cls).__new__(cls, *a, **kw)

但显然 update_settings 在它之前运行

Answer 1

尝试使用-s参数。

scrapy scraperX -s FEED_URI=s3://mybucket/path/to/export.csv