python - 脚本无法继续单击“加载更多”按钮

Question

我用 Python 编写了一个与 selenium 相关的脚本，用于不断单击 MORE 按钮来加载更多项目，直到没有任何新项目可供从网页加载。但是，我的下面的脚本可以单击该页面底部的 MORE 按钮。

Link to that site

这是我迄今为止的尝试:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
        print(elems.get_attribute("href"))

    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].scrollIntoView();", loadmore)
        loadmore.click()
    except Exception:break

driver.quit()

我怎样才能继续点击MORE按钮，直到没有这样的按钮可供点击和解析链接，因为我已经尝试使用for循环。

Answer 1

我已经成功地在现有脚本中解决了遵循安德森爵士逻辑的问题。这就是修改后的脚本的样子。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].click();", loadmore)
        wait.until(EC.staleness_of(loadmore))
    except Exception:break

for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
    print(elems.get_attribute("href"))

driver.quit()