python - 无法用尽加载更多按钮来揭开所有头条新闻

Question

我一直尝试点击位于网页底部(在其着陆页中)的 more 按钮以显示所有内容头条新闻。问题是当我执行我的脚本时，它只单击一次然后停止。我如何才能一直点击该按钮，直到没有更多选项可供点击？

Link to that website

到目前为止，这是我的脚本:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://www.newsnow.co.uk/h/Sport/Football/Championship/Transfer+News"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)

while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[class^='hl_more']")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except Exception: break
driver.quit()

Answer 1

尝试下面的代码来模拟所需的行为:

header = driver.find_element_by_id("phead")
driver.execute_script('arguments[0].style.position = "absolute";', header)
while True:
    try:    
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.hl_more.bg_tween:not(.nfloading)")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except: break