python - 如何使用 beautifulsoup 解析来自特定站点的任何文章以及如何预览该链接的图像？

Question

我正在尝试从体育网站解析 html 提要中的新闻文章，我尝试使用以下代码，但出现“关键错误”

我试过的代码:

def get_cric_info_articles():

    cricinfo_article_link = "http://www.espncricinfo.com/ci/content/story/news.html"

    r = requests.get(cricinfo_article_link)
    cricinfo_article_html = r.text

    soup = BeautifulSoup(cricinfo_article_html, "html.parser")
    # print(soup.prettify())

    cric_info_items = soup.find_all("h2",
                                      {"class": "story-title"})

    cricinfo_article_dict = {}

    for div in cric_info_items:
      cricinfo_article_dict[div.find('a')['story-title']] = div.find('a')['href']

    return cricinfo_article_dict

错误信息:

KeyError: 'story-title'

Answer 1

您要查找的值在 a 标记内

import requests
from bs4 import BeautifulSoup


def get_cric_info_articles():

    cricinfo_article_link = "http://www.espncricinfo.com/ci/content/story/news.html"

    r = requests.get(cricinfo_article_link)
    cricinfo_article_html = r.text

    soup = BeautifulSoup(cricinfo_article_html, "html.parser")
    # print(soup.prettify())

    cric_info_items = soup.find_all("h2",
                                    {"class": "story-title"})
    cricinfo_article_dict = {}

    for div in cric_info_items:
        cricinfo_article_dict[div.find('a').string] = div.find('a')['href']

    return cricinfo_article_dict


print(get_cric_info_articles())

输出:

{'Bell-Drummond leads MCC in curtain-raiser': '/ci/content/story/1135157.html', 'Scotland pick Brad Wheal, Chris Sole for World Cup qualifiers': '/scotland/content/story/1135152.html', 'Newlands working to be water independent': '/southafrica/content/story/1135120.html', 'Scorchers bow out after Hurricanes pile up 210': '/australia/content/story/1135117.html', "'Strong evidence' of corruption in Ajman All Stars League - ICC ": '/ci/content/story/1135108.html', 'Du Plessis 120 powers South Africa to 269': '/south-africa-v-india-2018/content/story/1135099.html', "Plan is to expose India's middle, lower order - Harris": '/australia/content/story/1135091.html', 'Top order, King fire Scorchers into WBBL final': '/australia/content/story/1135084.html', 'Technical change brings prolific run for Mominul': '/bangladesh/content/story/1135077.html', 'Dhananjaya, Mendis lead strong Sri Lanka reply': '/bangladesh/content/story/1135075.html'}