python - 使用 requests.post 时 标签不显示 Href 属性

Question

我正在尝试下载此 page 上显示的 .csv 文件当我将数据提交为

数据用于:安全方面的价格量和可交割头寸数据
符号:3INFOTECH
选择系列:全部
期限:24 个月

我的代码是

symbol = "3IINFOTECH"
url = "https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp"
data = {

    "dataType":"priceVolumeDeliverable",
    "symbol":symbol,
    "segmentLink":"3",
    "symbolCount":"2",
    "series":"ALL",
    "rdPeriod":"groupPeriod",
    "dateRange":"24month"
}

headers = {
   'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11'
}
print("fetching for " + symbol)
session = requests.session()
response = requests.post(url, data, headers = headers)
html_content = response.text

soup = BeautifulSoup(html_content, "html.parser")

download_link = soup.findAll("span", attrs = {"class":"download-data-link"})[0]
print(download_link.a["href"])

现在在检查元素时我看到了这个

如何下​​载 csv 文件？来 self 的代码的发布请求没有显示 href 属性。

Answer 1

要获得链接，您必须单击按钮，这样您可以使用 selenium 或类似的东西，但是您自己解析数据非常容易，因为您从 post 请求返回的只是数据:

symbol = "3IINFOTECH"
url = "https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp"
data = {

    "dataType": "priceVolumeDeliverable",
    "symbol": symbol,
    "segmentLink": "3",
    "symbolCount": "2",
    "series": "ALL",
    "rdPeriod": "groupPeriod",
    "dateRange": "24month"
}

headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11'
}
print("fetching for " + symbol)
import csv
response = requests.post(url, data, headers=headers)
html_content = response.text
soup = BeautifulSoup(html_content, "html.parser")
cols = [th.text for th in soup.select("th")]

rows = ([td.text for td in row.select("td")] for row in soup.select("tr  + tr"))

with open("data.csv", "w") as f:
    wr = csv.writer(f)
    wr.writerow(cols)
    wr.writerows(rows)

data.csv 片段:

Symbol,Series,Date,Prev Close,Open Price,High Price,Low Price,Last Price,Close Price,VWAP,Total Traded Quantity,Turnover in Lacs,No. of Trades,DeliverableQty,% Dly Qt toTraded Qty
3IINFOTECH,EQ,23-May-2014,9.90,10.25,10.70,9.70,10.10,10.10,10.23,"84,99,408",869.20,"16,539","40,35,648",47.48
3IINFOTECH,EQ,26-May-2014,10.10,10.40,10.60,9.10,9.30,9.20,9.97,"59,15,990",589.88,"9,894","27,10,021",45.81
3IINFOTECH,EQ,27-May-2014,9.20,9.20,9.30,8.30,8.60,8.55,8.53,"34,95,072",298.18,"3,600","14,71,141",42.09
3IINFOTECH,EQ,28-May-2014,8.55,8.60,9.40,8.45,9.30,9.15,9.07,"36,09,261",327.27,"3,955","13,92,733",38.59
3IINFOTECH,EQ,29-May-2014,9.15,9.25,9.50,8.80,9.40,9.35,9.28,"30,13,036",279.69,"3,090","15,20,654",50.47
3IINFOTECH,EQ,30-May-2014,9.35,9.35,9.55,8.90,9.00,9.00,9.13,"13,97,140",127.53,"1,992","7,43,964",53.25