Python:Selenium，通用 XPATH 上的 NoSuchElementException

Question

我有代码允许我在给定关键字的情况下返回特定网站的所有搜索部分。

当使用搜索词“HL4RPV-50”时，我可以按预期取回所有返回值。

当我使用搜索词“FSJ4-50B”时，我得到了一个 NoSuchElementException 行:

    ---> 53     price = product.find_element_by_xpath(".//div[@class='price']").text.split('\n')[1]

直接的 XPATH 是:

    //*[@id="search"]/div[3]/div[2]/div[2]/div[2]/div[6]/div[2]/div[1]/div[1]/div/div[4]/div/add-product-to-cart/div[1]

对于两个部件 ID，这不是相同的直接 XPATH。此外，根据给定结果的部分位置，每个部分 ID 的 XPATH 略有不同。

我的印象是我可以引用一个相对的 XPATH 来解决这个问题。

我试图从中抓取的网站是 Tessco.com并且在下面的代码中指定了通用的 UN/PW。

识别 XPATH ID:

为了制作一个通用的 XPATH，我的印象是使用 . 来选择当前节点，并使用 // 从当前节点中选择文档中的节点匹配选择的节点，无论它们在哪里。

然后我指定了它的类型，这里是 div 然后是 @class='price'

对于“HL4RPV-50”，这给了我想要的，对于“FSJ4-50B”，它没有。

我相信我有错误的 XPATH，但不确定如何概括它。

有什么建议吗？

代码:

    import time
    #Need Selenium for interacting with web elements
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    #Need numpy/pandas to interact with large datasets
    import numpy as np
    import pandas as pd

    chrome_path = r"C:\Users\James\Documents\Python Scripts\jupyterNoteBooks\ScrapingData\chromedriver_win32\chromedriver.exe"
    driver = webdriver.Chrome(chrome_path)
    driver.get("https://www.tessco.com/login")

    userName = "FirstName.SurName321123@gmail.com"
    password = "PasswordForThis123"

    #Set a wait, for elements to load into the DOM
    wait10 = WebDriverWait(driver, 10)
    wait20 = WebDriverWait(driver, 20)
    wait30 = WebDriverWait(driver, 30)

    elem = wait10.until(EC.element_to_be_clickable((By.ID, "userID"))) 
    elem.send_keys(userName)

    elem = wait10.until(EC.element_to_be_clickable((By.ID, "password"))) 
    elem.send_keys(password)

    #Press the login button
    driver.find_element_by_xpath("/html/body/account-login/div/div[1]/form/div[6]/div/button").click()

    #Expand the search bar
    searchIcon = wait10.until(EC.element_to_be_clickable((By.XPATH, "/html/body/header/div[2]/div/div/ul/li[2]/i"))) 
    searchIcon.click()

    searchBar = wait10.until(EC.element_to_be_clickable((By.XPATH, '/html/body/header/div[3]/input'))) 
    searchBar.click()

    #load in manufacture part number from a collection of components, via an Excel file

    #Enter information into the search bar
    searchBar.send_keys("FSJ4-50B" + '\n')

    # wait for the products information to be loaded
    products = wait30.until(EC.presence_of_all_elements_located((By.XPATH,"//div[@class='CoveoResult']")))
    # create a dictionary to store product and price
    productInfo = {}
    # iterate through all products in the search result and add details to dictionary
    for product in products:
        # get product name
        productName = product.find_element_by_xpath(".//a[@class='productName CoveoResultLink hidden-xs']").text
        # get price
        price = product.find_element_by_xpath(".//div[@class='price']").text.split('\n')[1]
        # add details to dictionary
        productInfo[productName] = price
    # print products information   
    print(productInfo)

    #time.sleep(5)
    driver.close()

Answer 1

这是工作代码我禁用了图像，因为我的互联网连接速度很慢，而且网站加载页面需要时间。我使用 css 选择器代替 xPath 来获取价格和它的完全工作>

import time
#Need Selenium for interacting with web elements
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
#Need numpy/pandas to interact with large datasets
import numpy as np
import pandas as pd

chrome_path = r".\web_driver\chromedriver.exe"
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.managed_default_content_settings.images": 2}
chrome_options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_path, chrome_options=chrome_options)
driver.maximize_window()
driver.get("https://www.tessco.com/login")

userName = "FirstName.SurName321123@gmail.com"
password = "PasswordForThis123"

#Set a wait, for elements to load into the DOM
wait10 = WebDriverWait(driver, 10)
wait20 = WebDriverWait(driver, 20)
wait30 = WebDriverWait(driver, 30)

elem = wait10.until(EC.element_to_be_clickable((By.ID, "userID")))
elem.send_keys(userName)

elem = wait10.until(EC.element_to_be_clickable((By.ID, "password")))
elem.send_keys(password)

#Press the login button
driver.find_element_by_xpath("/html/body/account-login/div/div[1]/form/div[6]/div/button").click()

#Expand the search bar
# searchIcon = wait10.until(EC.element_to_be_clickable((By.XPATH, "")))
# searchIcon.click()

searchBar = wait10.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#searchBar input")))

#Enter information into the search bar
searchBar.send_keys("FSJ4-50B")
driver.find_element_by_css_selector('a.inputButton').click()
time.sleep(5)

# wait for the products information to be loaded
products = driver.find_elements_by_xpath( "//div[@class='CoveoResult']")
# create a dictionary to store product and price
productInfo = {}
# iterate through all products in the search result and add details to dictionary
for product in products:
    # get product name
    productName = product.find_element_by_xpath("//a[@class='productName CoveoResultLink hidden-xs']").text
    # get price
    price = product.find_element_by_css_selector("div.price").text.split('\n')[1]
    # add details to dictionary
    productInfo[productName] = price
# print products information
print(productInfo)
#time.sleep(5)
driver.close()

输出:

{"8' Jumper-FSJ4-50B NM/NM": '$147.55'}

已编辑:

如何选择选择器

如您在上面的屏幕截图中所见，我将鼠标悬停在 searchBar 上，发现它有一个 ID，我们知道 ID> 始终是网页上的唯一元素，因此我们还可以使用:

driver.find_element_by_id("searchBar")

但要到达输入字段，我更喜欢css_selector，然后发送 key 。

寻找a.inputButton CSS 选择器:

对于 a.button css 选择器请参阅选择搜索按钮，您将在 dom 中看到以下 html:

<a class="CoveoSearchButton inputButton button"><span class="coveo-icon">Search</span><i class="fa fa-search" aria-hidden="true"></i></a>

我们知道<a>是 anchor 标签，从上面的html，我们可以推导出其中一个css_selector可以是:

a.inputButton

注意

但是在这种情况下这是唯一的，有时同一个类名可以在同一页面的不同元素中多次使用，所以你必须使用上层节点到达子 CSS 元素 节点。例如 a.inputButton也可以遍历为:

searchButton 的另一个 css_selector

div.divCoveoSearchbox > a.inputButton

作为div是我们 inputButton 的 anchor 标记的父元素。

我希望我已经阐明了你的观点？