python - 如何在一个站点表格内容中定位元素

Question

我尝试在下面的网站上抓取产品项目详细信息，但脚本总是失败，并显示错误没有这样的元素，尽管该元素在那里。任何人都可以帮助解决这个问题吗？我的代码:

from time import sleep

from scrapy import Spider
from selenium import webdriver
from scrapy.selector import Selector
from scrapy.http import Request
from selenium.common.exceptions import NoSuchElementException
driver = webdriver.Chrome('D:\chromedriver_win32\chromedriver.exe')
driver.get('http://www.tesensors.com/global/en/product/inductive-capacitive/xs-xt-ref')
sleep(5)
#soemtime the site ask you select language and country so need click button as below
sign_in_button = driver.find_element_by_id('edit-submit--4')
sign_in_button.click()
sleep(5)
# scrapy content.total 1168 items, here there is no result.
product_model_name=driver.find_elements_by_xpath('span[@itemprop="name"]')
product_desc=driver.find_elements_by_xpath('span[@itemprop="description"]')

Answer 1

iframe 内的产品数据

您可以使用 XPath 来定位:

iframe = driver.find_element_by_xpath("//iframe[@id='ecat']")

然后切换到:

driver.switch_to.frame(iframe)

以下是如何切换回默认内容(超出默认内容):

driver.switch_to.default_content()

不要使用time-sleep模块，尝试explicit-waits .

see差异。

例如:

from scrapy import Spider
from selenium import webdriver
from scrapy.selector import Selector
from scrapy.http import Request
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome('D:\chromedriver_win32\chromedriver.exe')
driver.get('http://www.tesensors.com/global/en/product/inductive-capacitive/xs-xt-ref')

#soemtime the site ask you select language and country so need click button as below
sign_in_button = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "edit-submit--4")))
sign_in_button.click()

#switch iframe
iframe = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//iframe[@id='ecat']")))
driver.switch_to.frame(iframe)

# scrapy content.total 1168 items, here there is no result.
product_model_name = driver.find_elements_by_xpath('//span[@itemprop="name"]')
print(product_model_name[0].text)

product_desc=driver.find_elements_by_xpath('//span[@itemprop="description"]')

print(product_model_name[0].text)