python - scrapy 如何在 div 类中获取第二个 ？

Question

我只是想得到 url来自 div class .但是这个div有两个<a href> .所以，那个可以得到第一个，但那个应该得到第二个。我该怎么做？

这是网站的html代码:

 <div class="active"> 
          <a href="javascript:;" class=" add-favorite  j-lightbox-popup-button " data-target="howGoingCheckoutNotRedirect"> <i data-feather="heart"></i> </a> 
          <a href="/urun/yohegoalhareketli-kale-112432"> 

这是我的 Python 代码:

product_link = container.a["href"]

它得到 javascript:; a["href"] 内部

我所有的代码:

from bs4 import BeautifulSoup as soup
import pandas as pd
from tabulate import tabulate
import os
from joker import joker
import requests


product_joker_array = pd.DataFrame(columns = ['Shop Name','Product Name','Product Price','Product Image Url','Product Url']) 


for categories in range(0,len(joker.joker_category_names)):
    url = joker.joker_url  + joker.joker_category_names[categories]
    print(url)


    for pages in range(0,250):
        url = (joker.joker_url  + joker.joker_category_names[categories] + "#/page={}".format(pages))
        print(url)
        response = requests.get(url)
        data = response.text

        page_soup = soup(data, 'lxml')

        #time.sleep(10)
        containers = page_soup.findAll("div",{"class": "active"})
        #print(containers)
        len(containers)
        try:
            container = containers[0]
        except IndexError:
            print("Index Error")
            break

        #print(container)

        for container in containers:

            product_image = container.findAll("div", {"class": "image-link"})
            product_final_image = product_image[0].img["src"].strip()
            print("Product Image Url : " + product_final_image )

            product_desc = container.findAll("div", {"class": "image-link"})
            #print("asdasdas", product_desc)
            product_final_desc = product_desc[0].img["alt"].strip()
            print("product_final_desc : ",product_final_desc)

            #product_link = container.a["href"]
            product_link = container.xpath('.//a[2]/@href').extract_first()
            print("link : ",product_link)

            product_final_link = joker.joker_url + product_link
            print("final link : ",product_final_link)     



            product_price = container.findAll("span",{"class": "discount-price"})

            product_final_price = product_price[0].text.replace(" ", "")
            product_final_price = product_final_price.strip()
            print("product_final : ",product_final_price)
            #if "," not in product_final_price:
                                         #product_final_price = product_final_price + ",00"
                                         #print("product ,00",product_final_price)
            product_market_name = 'Joker'

            #print("Product Price:" + product_final_price )
            product_joker_array.loc[len(product_joker_array)] = [product_market_name,product_final_desc,product_final_price,product_final_image,product_final_link]

Answer 1

如果您使用 scrapy 选择器方法获取 url，请使用以下语法:

response.css('base::attr(href)').getall()