python - 我怎样才能找到图片中突出显示的元素。我用 Selenium

Question

我不确定为什么找不到这个元素，我正在使用 selenium，因为页面是动态加载的。

这是我的代码。

driver.get(singleData['itemLink'])
WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,"section#description")))
srce = driver.page_source
sp = BeautifulSoup(srce, 'lxml')

我可以这样获取它的父元素

down = sp.find(id = "attachments-links")

但无法在其第一个 div 中找到它的 a 标签。我试过了:

down3 = sp.find("a", attrs={"class": "usa-button-small usa-button-gray ng-star-inserted"})
down = sp.select("#attachments-links>div.download-container-header>span>a")

它们都不能正常工作并返回我 []。

我可以转到它上面的 h2 标签。像这样

down = sp.find(id = "attachments-links").find('div') 

打印下来给我:

<div class="download-container-header"><h2 id="opp-view-attachments-section-title">Attachments/Links</h2><!-- --></div>

链接:https://beta.sam.gov/opp/8f1efc97df214010b46631c74e6a8aa0/view?keywords=&sort=-modifiedDate&index=opp&is_active=true&page=1

非常感谢您的帮助。

Answer 1

首先需要点击/滚动页面以提取正确的信息。

from selenium import webdriver
from bs4 import BeautifulSoup

URL = "https://beta.sam.gov/opp/8f1efc97df214010b46631c74e6a8aa0/view?keywords=&sort=-modifiedDate&index=opp&is_active=true&page=1"

driver = webdriver.Chrome()
driver.get(URL)
# Wait for the page to fully render
driver.implicitly_wait(5)

# Click on an element of the page
driver.find_element_by_css_selector("h2#opp-view-attachments-section-title").click()

soup = BeautifulSoup(driver.page_source, "lxml")

button = soup.find("a", attrs={"class": "usa-button-small usa-button-gray ng-star-inserted"})["href"]
print(button)

输出:

https://beta.sam.gov/api/prod/opps/v3/opportunities/8f1efc97df214010b46631c74e6a8aa0/resources/download/zip?api_key=null&token=