python - 网页抓取工具不会转到正确的页面

Question

所以我一直尝试去http://ofcc.ohio.gov/Opportunities2#lt-126679-construction-bids获取投标 list 。然而，读取的数据始终来自主页，即: http://ofcc.ohio.gov/Opportunities2 。我已经用 Beautiful Soup 和 Scrapy 尝试过了，但没有效果。有什么建议吗？当我 curl 第一个网址时，它还会加载主页数据。代码:

    import scrapy

class QuotesSpider(scrapy.Spider):
name = "quotes"

def start_requests(self):
    urls = [
        'http://ofcc.ohio.gov/Opportunities2#lt-126679-construction-bids',
    ]
    for url in urls:
        yield scrapy.Request(url=url, callback=self.parse)

def parse(self, response):
    page = response.url.split("/")[-2]
    filename = 'construction%s.html' % page
    with open(filename, 'wb') as f:
        f.write(response.body)
    self.log('Saved file %s' % filename)

Answer 1

只需点击 http://ofcc.ohio.gov/Opportunities2#lt-126679-construction-bids我一直在等待出价加载...这是网站问题吗？

您收到任何错误消息吗？你的问题不是很详细