python - 为什么scrapy会重复抓取一个结果？

Question

请帮助我解决这个问题:以下蜘蛛代码预计会返回 start_url 的所有列出的作业。但是，它只返回第一个作业的许多副本。 Xpath 代码在“Xpath Checker”中被正确测试。怎么了？感谢您的投入!

from scrapy.spiders import Spider
from scrapy.selector import Selector
from Testjobs.items import TestjobsItem, TestjobsItemLoader

class TestjobSpider(Spider):
    name = "test"
    allowed_domains = ['http://careers.pathologyjobstoday.org/']
    start_urls = [
        'http://careers.pathologyjobstoday.org/jobseeker/search/results'
    ]

    def parse(self, response):
        hxs = Selector(response)  
        sites = hxs.xpath('//tr[contains(@id, "jt_jobrow_")]') 

        for site in sites:
            il = TestjobsItemLoader(response=response, selector=site)
            il.add_xpath('title', 'normalize-space(//div[@class="jt_jobs_title"]/text())') 
            yield il.load_item()

Answer 1

您需要通过在前面添加一个点来使您的“内部”XPath 上下文特定:

normalize-space(.//div[@class="jt_jobs_title"]/text())
            HERE^