python - Beautiful Soup - 列表中所有项目的结果转为 CSV

Question

下面的代码片段“有效”，但仅将第一条记录输出到 CSV。我试图让它输出相同的输出，但对于 all_links 列表中枪支网址列表中的每把枪。

我使用输出打印对其进行的任何修改(只是为了查看它的工作原理)都会打印相同的结果

或者如果我制作一个gun_details列表并尝试打印它，得到相同的一项输出。

我如何将所有的gun_details标签和跨度打印到CSV中？

import csv
import urllib.request

import requests
from bs4 import BeautifulSoup

all_links = []


url = "https://www.guntrader.uk/dealers/minsterley/minsterley-ranges/guns?page={}"
for page in range(1, 3):
    res = requests.get(url).text
    soup = BeautifulSoup(res, "html.parser")
    for link in soup.select(
        'a[href*="dealers/minsterley/minsterley-ranges/guns/shotguns/"]'
    ):
        all_links.append("https://www.guntrader.uk" + link["href"])

for a_link in all_links:

    gun_label = []
    gun_span = []

    res = urllib.request.urlopen(a_link)
    # res = requests.get(a_link)
    soup = BeautifulSoup(res, "html.parser")
    for gun_details in soup.select("div.gunDetails"):
        for l in gun_details.select("label"):
            gun_label.append(l.text.replace(":", ""))
        for s in gun_details.select("span"):
            gun_span.append(s.text)

my_dict = dict(zip(gun_label, gun_span))
with open("mycsvfile.csv", "w") as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames=None)
    for key in my_dict.keys():
        csvfile.write(f"{key},{my_dict[key]}\n")

Answer 1

尝试以这种方式运行中间部分:

for a_link in all_links:
    gun_label = []
    gun_span = []

    res = requests.get(a_link)
    soup = bs(res.content, 'html.parser') #note it's 'res.content', not just 'res'
    for gun_details in soup.select('div.gunDetails'):
        for l in gun_details.select('label'):
            gun_label.append(l.text.replace(':',''))
    for s in gun_details.select('span'):
        gun_span.append(s.text)

    #this block is now indented differently - it's INSIDE the 'for' loop
    my_dict = dict(zip(gun_label, gun_span))
    with open('mycsvfile.csv', 'a') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=None)
        for key in my_dict.keys():
            csvfile.write(f"{key},{my_dict[key]}\n")