python - 在 scrapy 网络爬虫中获取错误

Question

您好，我尝试在我的代码中实现它。但是我收到以下错误:exceptions.NameError: global name 'Request' is not defined。

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector 
from bs4 import BeautifulSoup

class spider_aicte(BaseSpider):
    name = "Indian_Colleges"
    allowed_domains = ["http://www.domain.org"]
    start_urls = [
        "http://www.domain.org/appwebsite.html",
        ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        soup = BeautifulSoup(response.body)
        for link in soup.find_all('a'):
            download_link = link.get('href')
            if '.pdf' in download_link:
                pdf_link = "http://www.domain.org" + download_link
                print pdf_link
                class FileSpider(BaseSpider):
                    name = "fspider"
                    allowed_domains = ["www.domain.org"]
                    start_urls = [
                            pdf_link
                            ]
        for url in pdf_link:
            yield Request(url, callback=self.save_pdf)

    def save_pdf(self, response):
         path = self.get_path(response.url)
         with open(path, "wb") as f:
            f.write(response.body)

Answer 1

您应该导入 Request使用前:

from scrapy.http import Request

---

或者，还有一个“捷径”导入:

from scrapy import Request

或者，如果您有 import scrapy 行，请使用 scrapy.Request。