python - 如何在 django View 中使用参数运行 scrapy 蜘蛛

Question

用户可以在表单中输入关键字并提交，这样我就可以在 View 中获取关键字。然后我可以使用关键字创建 start_url 。如何将 start_url 传递给 scrapy 蜘蛛并启动它？

这是我的查看方法。

def results(request):
    """Return the search results"""
    key= request.GET['keyword'].strip()
    books = Book.objects.filter(title__contains=key)
    if books is None:
        # I want to call the scrapy spider here.
        pass
        books = Book.objects.filter(title__contains=key)
    context = {'books': books, 'key': title}
    return render(request, 'search/results.html', context)

这是我的蜘蛛类的 init() 方法。

def __init__(self, key):
    self.key = key
    url = "http://search.example.com/?key=" + key
    self.start_urls = [url]

Answer 1

这对我有用:

from scrapy.crawler import CrawlerRunner
from scrapy.utils.project import get_project_settings
if books is None:
    # I want to call the scrapy spider here.
    os.environ.setdefault("SCRAPY_SETTINGS_MODULE","whereyourscrapysettingsare")
crawler_settings = get_project_settings()
crawler = CrawlerRunner(crawler_settings)
crawler.crawl(yourspider, key=key)

来自 http://doc.scrapy.org/en/latest/topics/practices.html#run-scrapy-from-a-script