Python 网络抓取 : problems with classes

Question

我正试图从这个 website 中抓取房地产经纪人的名字.

我的代码:

containers = page_soup.findAll("div",{"class":"team-details"})

for container in containers:
    agent_name = container.findAll("a", {"class":"team-name_link"})
    name = agent_name[0].text


    print("name: " + name)

但是，当我运行脚本时，我只收到前两个名称，然后是一条错误消息:

name: Michael Stavrianos
name: Kristalla Stavrianos
Traceback (most recent call last):
  File "C:\Users\Toby\Desktop\Webscrape\LjHooker - mark1.py", line 16, in <module>
    name = agent_name[0].text
IndexError: list index out of range

我发现前两个代理名称在“team-name_link”类下，而其余的在“team-name”类下。我不确定如何同时从两组类中抓取名称。

Answer 1

我认为你弄错了，所有名称都在所需标签内，但实际上你需要寻找 div:

from bs4 import BeautifulSoup
import requests

html = requests.get("https://woollahra.ljhooker.com.au/our-team").text
soup = BeautifulSoup(html, 'html.parser')
containers = soup.findAll("div",{"class":"team-details"})

for container in containers:
    agent_name = container.find("div", {"class":"team-name"})
    name = agent_name.text
    print(name)

以上代码输出:

Michael Stavrianos
              Licensee



Kristalla Stavrianos
              Principal



Jade Marshall
              Property Management Associate


Emma Phelan
              Property Management Associate


Isabella Marechal - Ross
              Property Management Associate


Victoria Empson
              Property Investment Manager