python - 我想使用 python 中的 BeautifulSoup 库对谷歌新闻进行一些网络抓取

Question

from urllib.request import urlopen
from bs4 import BeautifulSoup
import re
'''this program is useful for retrieving useful article link from Google news'''

a = input("first name")
str(a)
p = input("last name")
str(p)
t =("https://news.google.co.in/news/section?cf=all&hl=en&pz=1&ned=in&q=" + a +
    "+" + p + "&topicsid=FRONTPAGE&ict=tnv3")
print(t)

html = urlopen(t)
bsObj = BeautifulSoup(html)
nameList = bsObj.findAll("span",{"class":"titletext"})

for name in nameList:
    print(name.get_text())

print(“在这段代码中，我只获取标题文本，但我想获取相关链接(所以我必须找到所有存在 href 和 titletext 的标签，那么如何获取 href 和标题文本同时 ”) }

Answer 1

span 标记 (a) 的父标记具有 href 属性。检索属性值:

...
html = urlopen(t)
bsObj = BeautifulSoup(html)
nameList = bsObj.findAll("span",{"class":"titletext"})

for name in nameList:
    print(name.get_text(), name.parent.get('href'))  # <----