python-3.x - 如何使用 Selenium 按 class_name 从大学橄榄球数据中抓取图像 url 列表

Question

我正在尝试从大学橄榄球名册网站上抓取球员数据。我主要对获取玩家图像、体重和姓名感兴趣。我已经能够提取重量和名称，但正在努力使用 Selenium 提取图像。这是我到目前为止的代码。

driver = webdriver.Chrome("C:/Users/<my_user>/Downloads/chromedriver.exe")
driver.get(school["url"])
all_names = driver.find_elements(by=By.CLASS_NAME, value='sidearm-roster-player-name')
all_weights = driver.find_elements(by=By.CLASS_NAME, value='sidearm-roster-player-weight')
all_imgs = driver.find_elements(by=By.CLASS_NAME, value='sidearm-roster-player-image')

这是作为 school[url] 传入的示例。许多大学都使用这种格式。 https://rolltide.com/sports/football/roster该网站上的每个播放器都有以下 html 元素。

<div class="sidearm-roster-player-image column">

  <a data-bind="click: function() { return true; }, clickBubble: false" href="/sports/football/roster/jeremiah-alexander/8141" aria-label="Jeremiah Alexander - View Full Bio" title="View Full Bio">
    <img class=" lazyloaded" data-src="https://d1a8hwz3c6qyrc.cloudfront.net/images/2022/3/1/Alexander_Jeremiah.jpg?width=80" alt="Jeremiah Alexander" src="https://d1a8hwz3c6qyrc.cloudfront.net/images/2022/3/1/Alexander_Jeremiah.jpg?width=80">
  </a>

</div>

我遇到的问题是，all_imgs 中的每个 webElement 似乎没有一个名为“img”的属性，也没有我可以看到的代表图像链接位于元素内。如何获取该页面所有球员图片的链接？

Answer 1

实际上......您可以在没有selenium的情况下获取所有数据。一切都在源 HTML 中。

具体操作方法如下:

import json

import requests
from bs4 import BeautifulSoup
from tabulate import tabulate

url = "https://rolltide.com/sports/football/roster"

soup = BeautifulSoup(requests.get(url).text, "lxml")
height_weight = [
    " ".join(i.getText(strip=True, separator=" ").split()) for i in
    soup.select(".sidearm-roster-default-template .sidearm-roster-player-position")
]

images_and_names = [
    [
        player['name'],
        player['url'],
        player['image']['url']
    ] for player in
    json.loads(soup.find("script", {"type": "application/ld+json"}).string)["item"]
]

player_data = [[i, *j] for i, j in zip(height_weight, images_and_names)]
print(tabulate(player_data, headers=["Body Info", "Name", "Full Bio", "Image"]))

这应该输出:

Body Info           Name                  Full Bio                                    Image
------------------  --------------------  ------------------------------------------  -------------------------------------------------------------
LB 6'2" 258 lbs     Jeremiah Alexander    http://rolltide.com/roster.aspx?rp_id=8141  http://rolltide.com/images/2022/3/1/Alexander_Jeremiah.jpg
PK 6'2" 188 lbs     Chase Allen           http://rolltide.com/roster.aspx?rp_id=8048  http://rolltide.com/images/2020/7/8/Allen_Chase.jpg
WR 5'9" 184 lbs     Aaron Anderson        http://rolltide.com/roster.aspx?rp_id=8142  http://rolltide.com/images/2022/3/1/Anderson_Aaron.jpg
LB 6'4" 243 lbs     Will Anderson Jr.     http://rolltide.com/roster.aspx?rp_id=8049  http://rolltide.com/images/2020/3/19/Anderson_Jr_Will_71.jpg
DB 6'0" 188 lbs     Terrion Arnold        http://rolltide.com/roster.aspx?rp_id=8050  http://rolltide.com/images/2021/8/6/Aronld_Terrion.jpg
DL 6'5" 305 lbs     Anquin Barnes Jr.     http://rolltide.com/roster.aspx?rp_id=8051  http://rolltide.com/images/2021/8/6/Barnes_Anquin.jpg
DB 6'1" 206 lbs     Jordan Battle         http://rolltide.com/roster.aspx?rp_id=8052  http://rolltide.com/images/2020/3/19/Battle_Jordan_71.jpg
RB 5'8" 180 lbs     Jonathan Bennett      http://rolltide.com/roster.aspx?rp_id=8053  http://rolltide.com/images/2020/11/2/Bennett_Jonathan_71.jpg
LB 6'2" 233 lbs     Kendrick Blackshire   http://rolltide.com/roster.aspx?rp_id=8054  http://rolltide.com/images/2021/8/6/Blackshire_Kendrick.jpg
WR 6'0" 176 lbs     Bret Bolin            http://rolltide.com/roster.aspx?rp_id=8055  http://rolltide.com/images/2020/7/8/Bolin_Bret.jpg
OL 6'5" 332 lbs     Tyler Booker          http://rolltide.com/roster.aspx?rp_id=8143  http://rolltide.com/images/2022/3/1/Booker_Tyler.jpg
OL 6'5" 293 lbs     Tanner Bowles         http://rolltide.com/roster.aspx?rp_id=8056  http://rolltide.com/images/2020/3/19/Bowles_Tanner_71.jpg
WR 5'11" 182 lbs    Jacoby Boykins        http://rolltide.com/roster.aspx?rp_id=8057  http://rolltide.com/images/2021/8/6/Boykins_Jacoby.jpg
DB 6'0" 193 lbs     Brian Branch          http://rolltide.com/roster.aspx?rp_id=8058  http://rolltide.com/images/2021/6/17/Branch_Brian.jpg
LB 6'3" 240 lbs     Chris Braswell        http://rolltide.com/roster.aspx?rp_id=8059  http://rolltide.com/images/2020/3/19/Braswell_Chris_71.jpg
OL 6'3" 282 lbs     James Brockermeyer    http://rolltide.com/roster.aspx?rp_id=8060  http://rolltide.com/images/2021/2/9/Brockermeyer_James_71.jpg
OL 6'5" 304 lbs     Tommy Brockermeyer    http://rolltide.com/roster.aspx?rp_id=8061  http://rolltide.com/images/2021/2/9/Brockermeyer_Tommy.jpg
WR 6'2" 196 lbs     Ja'Corey Brooks       http://rolltide.com/roster.aspx?rp_id=8062  http://rolltide.com/images/2021/2/9/Brooks_Ja_Corey.jpg
TE 6'5" 238 lbs     Elijah Brown          http://rolltide.com/roster.aspx?rp_id=8144  http://rolltide.com/images/2022/3/1/Brown_Elijah.jpg
WR 6'3" 175 lbs     J'Kolbe Bulock        http://rolltide.com/roster.aspx?rp_id=8145  http://rolltide.com/images/2022/3/1/Bulock_J_Kolbe.jpg
P 6'6" 211 lbs      James Burnip          http://rolltide.com/roster.aspx?rp_id=8063  http://rolltide.com/images/2021/8/6/Burnip_James.jpg
DL 6'3" 309 lbs     Jamil Burroughs       http://rolltide.com/roster.aspx?rp_id=8064  http://rolltide.com/images/2020/7/8/Burroughs_Jamil.jpg
WR 6'0" 200 lbs     Jermaine Burton       http://rolltide.com/roster.aspx?rp_id=8146  http://rolltide.com/images/2022/3/1/Burton_Jermaine.jpg
LB 6'3" 225 lbs     Jihaad Campbell       http://rolltide.com/roster.aspx?rp_id=8147  http://rolltide.com/images/2022/3/1/Campbell_Jihaad.jpg
OL 6'4" 305 lbs     Javion Cohen          http://rolltide.com/roster.aspx?rp_id=8065  http://rolltide.com/images/2020/7/8/Cohen_Javion.jpg
RB 5'11" 210 lbs    Elijah Crockett       http://rolltide.com/roster.aspx?rp_id=8067  http://rolltide.com/images/2021/8/20/Crockett_Elijah.jpg
OL 6'3" 305 lbs     Darrian Dalcourt      http://rolltide.com/roster.aspx?rp_id=8068  http://rolltide.com/images/2020/3/19/Dalcourt_Darrian_53.jpg
DL 6'3" 300 lbs     DJ Dale               http://rolltide.com/roster.aspx?rp_id=8069  http://rolltide.com/images/2021/8/20/Dale_DJ.jpg
WR 5'10" 177 lbs    JoJo Earle            http://rolltide.com/roster.aspx?rp_id=8070  http://rolltide.com/images/2021/8/6/Earle_JoJo.jpg
DL 6'5" 292 lbs     Justin Eboigbe        http://rolltide.com/roster.aspx?rp_id=8071  http://rolltide.com/images/2020/8/17/Eboigbe_Justin.jpg
OL 6'3" 307 lbs     Emil Ekiyor Jr.       http://rolltide.com/roster.aspx?rp_id=8073  http://rolltide.com/images/2020/8/17/Ekiyor_Jr_Emil.jpg
TE 6'0" 220 lbs     Robert Ellis          http://rolltide.com/roster.aspx?rp_id=8074  http://rolltide.com/images/2020/11/2/Ellis_Robert.jpg
DB 6'2" 185 lbs     Tre'Quon Fegans       http://rolltide.com/roster.aspx?rp_id=8148  http://rolltide.com/images/2022/3/1/Fegans_Tre_Quon.jpg
OL 6'4" 300 lbs     Terrence Ferguson II  http://rolltide.com/roster.aspx?rp_id=8075  http://rolltide.com/images/2021/2/9/Ferguson_Terrence.jpg
LB 6'0" 212 lbs     Kyle Flood Jr.        http://rolltide.com/roster.aspx?rp_id=8076  http://rolltide.com/images/2022/3/1/Flood_Jr_Kyle.jpg
OL 6'6" 333 lbs     Damieon George Jr.    http://rolltide.com/roster.aspx?rp_id=8077  http://rolltide.com/images/2020/7/8/George_Jr_Damieon.jpg
RB 5'11" 200 lbs    Jahmyr Gibbs          http://rolltide.com/roster.aspx?rp_id=8149  http://rolltide.com/images/2022/3/1/Gibbs_Jahmyr.jpg
DL 6'4" 288 lbs     Monkell Goodwine      http://rolltide.com/roster.aspx?rp_id=8078  http://rolltide.com/images/2021/2/9/Goodwine_Monkell.jpg
WR 6'3" 202 lbs     Agiye Hall            http://rolltide.com/roster.aspx?rp_id=8079  http://rolltide.com/images/2021/2/9/Hall_Agiye.jpg
DB 6'1" 208 lbs     DeMarcco Hellams      http://rolltide.com/roster.aspx?rp_id=8080  http://rolltide.com/images/2020/3/19/Hellams_DeMarcco_58.jpg
SN 6'2" 245 lbs     Kneeland Hibbett      http://rolltide.com/roster.aspx?rp_id=8081  http://rolltide.com/images/2021/8/6/Hibbett_Kneeland.jpg
WR 6'3" 214 lbs     Traeshon Holden       http://rolltide.com/roster.aspx?rp_id=8082  http://rolltide.com/images/2020/3/19/Holden_Traeshon_53.jpg
DL 6'4" 295 lbs     Braylen Ingraham      http://rolltide.com/roster.aspx?rp_id=8084  http://rolltide.com/images/2020/3/19/Ingraham_Braylen_53.jpg
LB 6'1" 235 lbs     Ian Jackson           http://rolltide.com/roster.aspx?rp_id=8085  http://rolltide.com/images/2021/2/9/Jackson_Ian.jpg
DB 6'3" 198 lbs     Khyree Jackson        http://rolltide.com/roster.aspx?rp_id=8086  http://rolltide.com/images/2021/8/6/Jackson_Khyree.jpg
QB 6'2" 186 lbs     Blake Jarrett         http://rolltide.com/roster.aspx?rp_id=8150  http://rolltide.com/images/2022/3/1/Jarrett_Blake.jpg
LB 6'5" 230 lbs     Christian Johnson     http://rolltide.com/roster.aspx?rp_id=8087  http://rolltide.com/images/2021/8/16/Johnson_Christian.jpg
WR 6'0" 190 lbs     Thaiu Jones-Bell      http://rolltide.com/roster.aspx?rp_id=8089  http://rolltide.com/images/2020/3/19/Jones_Bell_Thaiu_71.jpg
DL 6'2" 343 lbs     Tim Keenan III        http://rolltide.com/roster.aspx?rp_id=8090  http://rolltide.com/images/2021/8/6/Keenan_III_Tim.jpg
LB 6'3" 220 lbs     Demouy Kennedy        http://rolltide.com/roster.aspx?rp_id=8091  http://rolltide.com/images/2020/8/20/Kennedy_Demouy.jpg
OL 6'7" 322 lbs     Amari Kight           http://rolltide.com/roster.aspx?rp_id=8092  http://rolltide.com/images/2020/3/19/Kight_Amari_81.jpg
LB 6'4" 231 lbs     Keanu Koht            http://rolltide.com/roster.aspx?rp_id=8093  http://rolltide.com/images/2021/2/9/Koht_Keanu.jpg
WR 6'2" 192 lbs     Grant Krieger         http://rolltide.com/roster.aspx?rp_id=8094  http://rolltide.com/images/2020/3/19/Krieger_Grant_30.jpg
DB 6'1" 170 lbs     Brylan Lanier         http://rolltide.com/roster.aspx?rp_id=8095  http://rolltide.com/images/2021/8/20/Lanier_Brylan.jpg
DL 6'3" 278 lbs     Jah-Marien Latham     http://rolltide.com/roster.aspx?rp_id=8096  http://rolltide.com/images/2020/7/8/Latham_Jah_Marien.jpg
OL 6'6" 326 lbs     JC Latham             http://rolltide.com/roster.aspx?rp_id=8097  http://rolltide.com/images/2021/2/9/Latham_JC.jpg
TE 6'5" 244 lbs     Cameron Latu          http://rolltide.com/roster.aspx?rp_id=8098  http://rolltide.com/images/2020/3/19/Latu_Cameron_53.jpg
WR 5'11" 193 lbs    Kendrick Law          http://rolltide.com/roster.aspx?rp_id=8151  http://rolltide.com/images/2022/3/1/Law_Kendrick.jpg
LB 6'2" 225 lbs     Deontae Lawson        http://rolltide.com/roster.aspx?rp_id=8099  http://rolltide.com/images/2021/2/9/Lawson_Deontae.jpg
WR 5'10" 175 lbs    Christian Leary       http://rolltide.com/roster.aspx?rp_id=8100  http://rolltide.com/images/2021/2/9/Leary_Christian.jpg
P 6'2" 207 lbs      Jack Martin           http://rolltide.com/roster.aspx?rp_id=8102  http://rolltide.com/images/2021/8/6/Martin_Jack.jpg
DB 6'1" 143 lbs     Jacobi McBride        http://rolltide.com/roster.aspx?rp_id=8179  http://rolltide.com/images/2021/3/3/McBride_Jacobi.jpg
RB 5'11" 212 lbs    Jase McClellan        http://rolltide.com/roster.aspx?rp_id=8103  http://rolltide.com/images/2020/3/19/McClellan_Jase_58.jpg
DB 6'1" 188 lbs     Kool-Aid McKinstry    http://rolltide.com/roster.aspx?rp_id=8104  http://rolltide.com/images/2021/2/9/McKinstry_Ga_Quincy.jpg
OL 6'4" 295 lbs     Seth McLaughlin       http://rolltide.com/roster.aspx?rp_id=8105  http://rolltide.com/images/2020/5/1/McLaughlin_Seth.jpg
RB 5'10" 201 lbs    Jamarion Miller       http://rolltide.com/roster.aspx?rp_id=8152  http://rolltide.com/images/2022/3/1/Miller_Jamarion.jpg
QB 6'2" 212 lbs     Jalen Milroe          http://rolltide.com/roster.aspx?rp_id=8106  http://rolltide.com/images/2021/2/9/Milroe_Jalen.jpg
LB 6'2" 225 lbs     Jaylen Moody          http://rolltide.com/roster.aspx?rp_id=8107  http://rolltide.com/images/2020/3/19/Moody_Jaylen_53.jpg
DB 6'0" 190 lbs     Malachi Moore         http://rolltide.com/roster.aspx?rp_id=8108  http://rolltide.com/images/2020/7/8/Moore_Malachi.jpg
LB 6'2" 224 lbs     Shawn Murphy          http://rolltide.com/roster.aspx?rp_id=8153  http://rolltide.com/images/2022/3/1/Murphy_Shawn.jpg
DL 6'5" 370 lbs     Jaheim Oatis          http://rolltide.com/roster.aspx?rp_id=8154  http://rolltide.com/images/2022/3/1/Oatis_Jaheim.jpg
TE 6'4" 258 lbs     Robbie Ouzts          http://rolltide.com/roster.aspx?rp_id=8109  http://rolltide.com/images/2021/2/9/Ouzts_Robbie.jpg
DL 6'4" 303 lbs     Damon Payne Jr.       http://rolltide.com/roster.aspx?rp_id=8111  http://rolltide.com/images/2021/8/6/Payne_Damon.jpg
DL 6'2" 264 lbs     Khurtiss Perry        http://rolltide.com/roster.aspx?rp_id=8155  http://rolltide.com/images/2022/3/1/Perry_Khurtiss.jpg
TE 6'6" 232 lbs     Jax Porter            http://rolltide.com/roster.aspx?rp_id=8156  http://rolltide.com/images/2022/3/1/Porter_Jax_71.jpg
DB 6'0" 175 lbs     Blake Pugh            http://rolltide.com/roster.aspx?rp_id=8112  http://rolltide.com/images/2021/8/20/Pugh_Blake.jpg
SN 6'5" 273 lbs     Gabe Pugh             http://rolltide.com/roster.aspx?rp_id=8113  http://rolltide.com/images/2021/8/16/Pugh_Gabe.jpg
DL 6'1" 236 lbs     Chase Quigley         http://rolltide.com/roster.aspx?rp_id=8114  http://rolltide.com/images/2021/8/6/Quigley_Chase.jpg
TE/OL 6'4" 298 lbs  Kendall Randolph      http://rolltide.com/roster.aspx?rp_id=8115  http://rolltide.com/images/2020/3/19/Randolph_Kendall_29.jpg
PK 6'1" 190 lbs     Will Reichard         http://rolltide.com/roster.aspx?rp_id=8116  http://rolltide.com/images/2020/8/17/Reichard_Will.jpg
DB 6'2" 190 lbs     Eli Ricks             http://rolltide.com/roster.aspx?rp_id=8157  http://rolltide.com/images/2022/3/1/Ricks_Eli.jpg
OL 6'5" 302 lbs     Jaeden Roberts        http://rolltide.com/roster.aspx?rp_id=8117  http://rolltide.com/images/2021/8/6/Roberts_Jaeden.jpg
DB 6'2" 197 lbs     Jahquez Robinson      http://rolltide.com/roster.aspx?rp_id=8118  http://rolltide.com/images/2020/3/19/Robinson_Jahquez_30.jpg
LB 6'5" 224 lbs     Quandarrius Robinson  http://rolltide.com/roster.aspx?rp_id=8119  http://rolltide.com/images/2020/7/8/Robinson_Quandarrius.jpg
OL 6'5" 285 lbs     Jackson Roby          http://rolltide.com/roster.aspx?rp_id=8120  http://rolltide.com/images/2020/3/19/Roby_Jackson_71.jpg
OL 6'3" 285 lbs     Graham Roten          http://rolltide.com/roster.aspx?rp_id=8121  http://rolltide.com/images/2021/8/6/Roten_Graham.jpg
RB 6'0" 214 lbs     Trey Sanders          http://rolltide.com/roster.aspx?rp_id=8122  http://rolltide.com/images/2020/8/17/Sanders_Trey.jpg
PK 6'0" 185 lbs     Reid Schuback         http://rolltide.com/roster.aspx?rp_id=8123  http://rolltide.com/images/2021/8/6/Schuback_Reid.jpg
OL 6'5" 308 lbs     Dayne Shor            http://rolltide.com/roster.aspx?rp_id=8158  http://rolltide.com/images/2022/3/1/Shor_Dayne.jpg
QB 6'2" 198 lbs     Ty Simpson            http://rolltide.com/roster.aspx?rp_id=8159  http://rolltide.com/images/2022/3/1/Simpson_Ty.jpg
TE 6'1" 232 lbs     Charlie Skehan        http://rolltide.com/roster.aspx?rp_id=8125  http://rolltide.com/images/2020/12/4/Skehan_Charlie.jpg
DB 6'0" 185 lbs     DeVonta Smith         http://rolltide.com/roster.aspx?rp_id=8126  http://rolltide.com/images/2021/8/6/Smith_DeVonta.jpg
LB 5'10" 210 lbs    Jordan Smith          http://rolltide.com/roster.aspx?rp_id=8127  http://rolltide.com/images/2021/8/16/Smith_Jordan.jpg
DL 6'4" 304 lbs     Tim Smith             http://rolltide.com/roster.aspx?rp_id=8128  http://rolltide.com/images/2020/7/8/Smith_Tim.jpg
DB 6'1" 211 lbs     Kristian Story        http://rolltide.com/roster.aspx?rp_id=8129  http://rolltide.com/images/2020/7/8/Story_Kristian.jpg
TE 6'5" 232 lbs     Adam Thorsland        http://rolltide.com/roster.aspx?rp_id=8130  http://rolltide.com/images/2021/8/6/Thorsland_Adam.jpg
LB 6'2" 228 lbs     Henry To'oTo'o        http://rolltide.com/roster.aspx?rp_id=8131  http://rolltide.com/images/2021/8/6/To_oto_o_Henry.jpg
LB 6'4" 240 lbs     Dallas Turner         http://rolltide.com/roster.aspx?rp_id=8132  http://rolltide.com/images/2021/8/6/Turner_Dallas.jpg
SN 5'9" 185 lbs     Kade Wehby            http://rolltide.com/roster.aspx?rp_id=8133  http://rolltide.com/images/2022/3/1/Wehby_Kade.jpg
LB 6'1" 222 lbs     Bennett  Whisenhunt   http://rolltide.com/roster.aspx?rp_id=8134  http://rolltide.com/images/2020/12/4/Whisenhunt_Bennett.jpg
DB 6'2" 203 lbs     Kaine Williams        http://rolltide.com/roster.aspx?rp_id=8135  http://rolltide.com/images/2021/8/6/Williams_Kaine.jpg
RB 5'10" 212 lbs    Roydell Williams      http://rolltide.com/roster.aspx?rp_id=8136  http://rolltide.com/images/2020/3/19/Williams_Roydell_53.jpg
WR 5'10" 165 lbs    Sam Willoughby        http://rolltide.com/roster.aspx?rp_id=8137  http://rolltide.com/images/2021/8/16/Willoughby_Sam.jpg
DL 6'4" 307 lbs     Stephon Wynn Jr.      http://rolltide.com/roster.aspx?rp_id=8138  http://rolltide.com/images/2020/3/19/Wynn_Jr_Stephon_71.jpg
QB 6'0" 194 lbs     Bryce Young           http://rolltide.com/roster.aspx?rp_id=8139  http://rolltide.com/images/2020/3/19/Young_Bryce_71.jpg
DL 6'3" 292 lbs     Byron Young           http://rolltide.com/roster.aspx?rp_id=8140  http://rolltide.com/images/2020/3/19/Young_Byron_76.jpg