python - 在页面上使用 selenium 查找电子邮件地址

Question

我正在尝试从网站获取电子邮件地址列表并且非常接近。我的代码可以在下面看到。我收到以下错误。

发生的事情是点击一个链接页面，然后在下一页中有一个电子邮件地址。

我试图在点击链接后打印出每个页面内的电子邮件地址。

Here is an example of a page that the link clicks through to .

xTraceback (most recent call last): File "scrape.py", line 34, in lookup(driver) File "scrape.py", line 26, in lookup emailAdress = driver.find_element_by_xpath('//div[@id="widget-contact"]//a‌​').get_attribute('hr‌​ef') File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 293, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 752, in find_element 'value': value})['value'] File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidSelectorException:

我正在使用 python 2.7.13。

# -*- coding: utf-8 -*-

from lxml import html
import requests
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

def init_driver():
    driver = webdriver.Firefox()
    driver.wait = WebDriverWait(driver, 5)
    return driver


def lookup(driver):
    driver.get("http://www.sportbirmingham.org/directory?sport=&radius=15&postcode=B16+8QG&submit=Search")
    try:
        for link in driver.find_elements_by_xpath('//h2[@class="heading"]/a'):
            link.click()
            emailAdress = driver.find_element_by_xpath('//div[@id="widget-contact"]//a‌​').get_attribute('hr‌​ef')
            print emailAdress
    except TimeoutException:
        print "not found"


if __name__ == "__main__":
    driver = init_driver()
    lookup(driver)
    time.sleep(5)
    driver.quit()

当我尝试继续访问下一页链接时，出现以下错误

File "scrape.py", line 43, in lookup(driver) File "scrape.py", line 26, in lookup links.extend([link.get_attribute('href') for link in driver.find_elements_by_xpath('//h2[@class="heading"]/a')]) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 139, in get_attribute self, name) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 465, in execute_script 'args': converted_args})['value'] File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: The element reference is stale. Either the element is no longer attached to the DOM or the page has been refreshed.

Answer 1

您只需要更精确的 X-PATH(以及调用 text 方法):

emailAdress = driver.find_element_by_xpath('//div[@class="body"]/dl/dd[2]').text

但是这个例子适用于 Python3。请让我知道这对你有没有用。我还建议为 Chrome 使用“XPath Helper”扩展。