python - 使用 Xpath 按文本查找元素并在 python 中打印下一个元素文本

Question

对于方法中的搜索买家，我需要获取该买家的价格

def getprice(search_buyer):
    try:
        # webscrape url
        url = 'http://econpy.pythonanywhere.com/ex/001.html'
        response = requests.get(url)
        print(response)

        tree = html.fromstring(response.content)

        buyers =     tree.xpath('//div[contains(text(),"'+search_buyer+'")]/following-sibling:://span[@class="item-price"]')
   for div in buyers:
            print(";;;;;",div)
   except Exception:
        print("No buyer found")
getprice("Ben D. Rules")

(myvir) Administrators-Mac-mini:RESTAPI_ python JavaDeveloper$ python3 test.py 
<Response [200]>
No buyer found

Answer 1

考虑使用 bs4 4.7.1 和 css 伪类 :contains。我发现 css 语法不那么脆弱。可能对 future 的读者有用。

import requests
from bs4 import BeautifulSoup as bs

def get_price(buyer_name, soup):    
    price = soup.select_one('div:contains("' + buyer_name + '") + .item-price')
    if price is None:
        price = 'Not found'
    else:
        price = price.text
    return price

buyers = ['Patty Cakes', 'Derri Anne Connecticut', 'Moe Dess']
r = requests.get('http://econpy.pythonanywhere.com/ex/001.html')
soup = bs(r.content, 'html.parser')

for buyer_name in buyers:
    print(get_price(buyer_name, soup))

---

更高效，传递买家名单:

def get_prices(buyers):   

    import requests
    from bs4 import BeautifulSoup as bs

    r = requests.get('http://econpy.pythonanywhere.com/ex/001.html')
    soup = bs(r.content, 'html.parser')
    results = [None] * len(buyers)
    for index, buyer in enumerate(buyers):
        price = soup.select_one('div:contains("' + buyer + '") + .item-price')
        if price is None:
            price = 'Not found'
        else:
            price = price.text
        results[index] = price
    return list(zip(buyers,results))

buyers = ['Patty Cakes', 'Derri Anne Connecticut', 'Moe Dess']

print(get_prices(buyers))