python - 处理下一页链接时遇到问题

Question

我用 python 结合 selenium 编写了一个脚本，以从启用 JavaScript 的网页中获取一些数据。在单击下一页按钮之前需要执行三件事，因为只有当网页包含搜索结果时才会出现下一页链接。这三件事是:填写两个搜索框并单击搜索按钮。但是，我的脚本可以完美地完成这三件事，但在应该单击下一页链接时会中断(引发超时异常)。如您所见，我已经尝试使用三种不同的选项来单击下一页链接，但从未成功。我尝试过的其余两个已被注释掉。如何才能成功点击下一页按钮？

我尝试过的脚本:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("https://brokercheck.finra.org/")

wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Name or CRD#']"))).send_keys("Michael John")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "[placeholder='Firm Name or CRD# (optional)']"))).send_keys("Morgan Stanley")
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.md-button'))).click()
# wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,'.pagination-next a'))).click()
# wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a'))).click()
driver.quit()

下一页链接所在的元素是:

<ul class="pagination ng-pristine ng-untouched ng-valid ng-scope ng-isolate-scope" data-ng-if="listCtrl.getTotalResults()" total-items="listCtrl.getDisplayResults()" ng-model="listCtrl.currentPage" max-size="1" page-label="listCtrl.pageLabel($page)" items-per-page="listCtrl.itemsPerPage" ng-change="listCtrl.pageChanged()" boundary-links="true" previous-text="‹" next-text="›" first-text="«" last-text="»" aria-invalid="false">
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-first ng-scope disabled"><a href="" ng-click="selectPage(1, $event)" class="ng-binding">«</a></li><!-- end ngIf: ::boundaryLinks -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noPrevious()||ngDisabled}" class="pagination-prev ng-scope disabled"><a href="" ng-click="selectPage(page - 1, $event)" class="ng-binding">‹</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngRepeat: page in pages track by $index --><li ng-repeat="page in pages track by $index" ng-class="{active: page.active,disabled: ngDisabled&amp;&amp;!page.active}" class="pagination-page ng-scope active"><a href="" ng-click="selectPage(page.number, $event)" class="ng-binding">1 of 27 pages</a></li><!-- end ngRepeat: page in pages track by $index -->
  <!-- ngIf: ::directionLinks --><li ng-if="::directionLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-next ng-scope"><a href="" ng-click="selectPage(page + 1, $event)" class="ng-binding">›</a></li><!-- end ngIf: ::directionLinks -->
  <!-- ngIf: ::boundaryLinks --><li ng-if="::boundaryLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-last ng-scope"><a href="" ng-click="selectPage(totalPages, $event)" class="ng-binding">»</a></li><!-- end ngIf: ::boundaryLinks -->
</ul>

Answer 1

页面上有 2 个具有相同定位器的分页:顶部和底部。

要处理 top，您需要执行 driver.maximize_window() 使其可见，然后使用与您尝试的相同的代码:

link = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a')))
driver.execute_script('arguments[0].scrollIntoView();', link)
link.click()

处理底部分页:

wait.until(EC.visibility_of_element_located((By.XPATH,'(//*[contains(@class, "pagination-next")]//a)[2]'))).click()