python - 如何定义 BeautifulSoup 的 "source.find"部分

Question

我需要从外卖网站上抓取一份餐厅链接列表，然后再抓取他们的菜单。这是我想抓取的网站:https://www.pedidosya.com.ar/restaurantes/buenos-aires/monserrat/empanadas-delivery?bt=RESTAURANT&page=1

检查 HTML:

<a href="https://www.pedidosya.com.ar/restaurantes/buenos-aires/el- 
noble-galerias-pacifico-menu" title="El Noble Galerías Pacífico" 
class="arrivalName">El Noble Galerías Pacífico</a>

我感兴趣的部分是:

网址 = https://www.pedidosya.com.ar/restaurantes/buenos-aires/el-noble-galerias-pacifico-menu

名称 = El Noble Galerías Pacífico

保存在不同的列表中，然后创建一个 .CSV

网站有几个页面，但我稍后会解决这个问题哈哈。这是我的代码:

urls = []

source = BeautifulSoup(page_soup, "html.parser")
table = source.find('h3', attrs={'a': 'arrivalName'})
for a in table.find_all("a", href=True):
    print(a)
    urls.append(a)

我收到这个错误:TypeError: 'NoneType' 对象不可调用

Answer 1

您可以使用类名来定位餐厅链接，然后提取链接的 href 和名称的 .text。我生成了一个更整洁的 df，但您可以切出名称列和链接。

from bs4 import BeautifulSoup as bs
import requests
import pandas as pd
import math

url = 'https://www.pedidosya.com.ar/restaurantes/buenos-aires/monserrat/empanadas-delivery?bt=RESTAURANT&page=1'
results = []
with requests.Session() as s:
    r = requests.get(url)
    soup = bs(r.content, 'lxml')
    restuarants = [(item.text, item['href']) for item in soup.select('.arrivalName')]
    results.append(restuarants)
    num_per_page = 50
    total = int(soup.select_one('.addressTitle i').text)
    num_pages = math.ceil(total/num_per_page)
    if num_pages > 1:
        for page in range(2, num_pages + 1):
            r = s.get('https://www.pedidosya.com.ar/restaurantes/buenos-aires/monserrat/empanadas-delivery?bt=RESTAURANT&page=1'.format(page))     
            soup = bs(r.content, 'lxml')
            restuarants = [(item.text, item['href']) for item in soup.select('.arrivalName')]
            results.append(restuarants)

final =  [item for sublist in results for item in sublist]
df = pd.DataFrame(final, columns = ['name', 'link'])
df.to_csv(r"C:\Users\User\Desktop\Data.csv", sep=',', encoding='utf-8-sig',index = False )