python - 尝试用Python制作一个递归爬行蜘蛛。语法错误: non-keyword arg after keyword arg

Question

我试图在 scrappy 中抓取多个页面，我的函数确实返回第一个起始网址，但我无法设法使蜘蛛的规则生效。

这是我到目前为止所拥有的:

import scrapy

from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from craigslist_sample.items import CraigslistSampleItem



class MySpider(CrawlSpider):
    name = "craigs"
    allowed_domains = ["craigslist.org"]
    start_urls = ["http://sfbay.craigslist.org/npo/"]



    rules = (
        Rule(SgmlLinkExtractor(allow=('.*?s=.*',), restrict_xpaths('a[@class="button next"]',)), callback='parse', follow=True),)

    def parse(self, response):
        for sel in response.xpath('//span[@class="pl"]'):
            item = CraigslistSampleItem()
            item['title'] = sel.xpath('a/text()').extract()
            item['link'] = sel.xpath('a/@href').extract()
            yield item`

我收到此错误

SyntaxError: non-keyword arg after keyword arg

更新:

感谢下面的回答。没有语法错误，但我的爬虫只是停留在同一页面，不爬行。

更新了代码

import scrapy

from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from craigslist_sample.items import CraigslistSampleItem
from scrapy.contrib.linkextractors import LinkExtractor


class MySpider(CrawlSpider):
    name = "craigs"
    allowed_domains = ["craigslist.org"]
    start_urls = ["http://sfbay.craigslist.org/npo/"]

    rules = (Rule(SgmlLinkExtractor(allow=['.*?s=.*'], restrict_xpaths=('a[@class="button next"]')), 
        callback='parse', follow=True, ),
)


    def parse(self, response):
        for sel in response.xpath('//span[@class="pl"]'):
            item = CraigslistSampleItem()
            item['title'] = sel.xpath('a/text()').extract()
            item['link'] = sel.xpath('a/@href').extract()
            yield item

Answer 1

您的问题与此类似(Python 3)

>>> print("hello")
hello
>>> print("hello", end=",,")
hello,,
>>> print(end=",,", "hello")
SyntaxError: non-keyword arg after keyword arg

行:

Rule(SgmlLinkExtractor(allow=('.*?s=.*',), restrict_xpaths('a[@class="button next"]',)), callback='parse', follow=True),)

必须称为:

Rule(SgmlLinkExtractor(restrict_xpaths('a[@class="button next"]'),allow=('.*?s=.*',)), callback='parse', follow=True),)