python - 组合两个 block 并循环两个 block

Question

您好，很抱歉主题模糊，但我正在使用 Selenium 练习网络抓取。我有一个链接列表“urls_to_scrape”，对于每个网址，我想访问链接并提取某些元素，我已经能够提取每个元素，但现在我很困惑如何为列表中的每个网址执行此操作。请参阅下面的代码。

urls_to_scrape # list containing urls I want to perform the code below for 
               # each url


results = []

articles = driver.find_elements_by_css_selector('#MainW article')

counter = 1

for article in articles:
  result = {}
  try:
     title = article.find_element_by_css_selector('a').text
  except: 
     continue

 counter = counter + 1

 excerpt = article.find_element_by_css_selector('div > div > p').text

 author = 
 article.find_element_by_css_selector('div > footer > address > a').text

 date = article.find_element_by_css_selector('div > footer > time').text

 link=
 article.find_element_by_css_selector('div>h2>a').get_attribute('href')

 result['title'] = title
 result['excerpt'] = excerpt
 result['author'] = author
 result['date'] = date
 result['link'] = link

 results.append(result)

Answer 1

创建一个执行抓取的函数(结果下面的所有内容 = [])，例如

def scrape(url):
    ...
    ...
     return result

那么，

for url in url_to_scrape:
    result = scrape(url)
    results.append(result)