python 以 : urllib2 ClientForm 形式下载文件

Question

我正在疯狂尝试从以下网址下载 csv 文件:

网站上有 4 个表单，我设法在正确的表单上设置日期，然后我发布那个表单，我得到了带有正确 html 的 http 响应。但我实际上想下载 csv 而不是响应的 html。我以为我必须提交 2 份表格，首先是日期，然后是 csv 选择，但在第一个回复中我没有得到任何可以与之对话的表格。

这是我的代码:

#!/usr/bin/env python
import csv
from urllib2 import urlopen
from ClientForm import ParseResponse
import urllib2

proxy = urllib2.ProxyHandler({'http': '172.26.10.100:8080'})
# proxy = urllib2.ProxyHandler({})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)

headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/2010010' \
    '1 Firefox/4.0.1',
    'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language':'en-us,en;q=0.5',
    'Accept-Charset':'ISO-8859-1,utf-8;q=0.7,*;q=0.7'}

# set the request
url = "http://www.opcom.ro/rapoarte/raportPIPsiVolumTranzactionat.php?lang=en"
request = urllib2.Request(url, None, headers)

try:
    response = urllib2.urlopen(request)

except urllib2.HTTPError, response:
    pass

print response.geturl()
print response.info() # headers
# print response.read() # body

# get forms from response
forms = ParseResponse(response, backwards_compat=False)
response.close()

# print "###FORMS: " ,len(forms)    
# for i in range(len(forms)):
    # print "@@@@@"
    # print forms[i]

form1 = forms[1]

#  setting a specific date in the form
form1.set_value("7", kind="text", nr=0)
form1.set_value("10", kind="text", nr=2)
form1.set_value("2011", kind="text", nr=4)
print form1

# # # SEND THE FORM
request2 = forms[1].click()  # urllib2.Request object      BIEN

try:
    response2 = urllib2.urlopen(request2)

except urllib2.HTTPError, response2:
    pass

print response2.geturl()
print response2.info() # headers
# print response2.read() # body
with open('salida.txt', 'w') as f:
    f.write(response2.read())

forms2 = ParseResponse(response2, backwards_compat=False)
response2.close()

print "###FORMS 2: " ,len(forms2)

请注意，第一个表单(日期选择器)是表单数组中的表单[1]。而 forms[2] 是 CSV 文件或 XML 的选择框。选择 CSV 的代码是:

# form2 = forms2[2]


# # Select CSV file in selection control
# form2.find_control("menu_sari").items[1].selected = True  # check

但我评论了它，因为在收到回复后我没有收到此表格。

非常欢迎任何帮助/反馈。

Answer 1

在查看源代码时，您可以看到当您填写表单时，会调用一个 php 脚本来生成 csv 文件，结果是一个用于保存文件的对话框。

例如开启时:

http://www.opcom.ro/rapoarte/raportPIPsiVolumTranzactionat.php?lang=en

如果您选择 13/1/2012 和 csv 填写表格，将调用以下 url:

http://www.opcom.ro/rapoarte/export_csv_raportPIPsiVolumTranzactionat.php?zi=13&luna=1&an=2012&limba=ro

以下对我有用:

>>> import urllib2
>>> url = "http://www.opcom.ro/rapoarte/export_csv_raportPIPsiVolumTranzactionat.php?zi=13&luna=1&an=2012&limba=ro"
>>> request = urllib2.Request(url)
>>> response = urllib2.urlopen(request)
>>> print response.read()
"PIP si volum tranzactionat pentru ziua de livrare: 13/1/2012"

"","Pret mediu [lei/MWh]","Volum [MWh]"
"ROPEX_DAM_Base (1-24)","185.23","31226.488"
"ROPEX_DAM_Peak (7-22)","239.68","22773.036"
"ROPEX_DAM_Off_Peak (1-6) & (23-24)","76.32","8453.452"

"Interval","Pret de Inchidere a Pietei [lei/MWh]","Volum Tranzactionat [MWh]"
"1","29.99","876.148"
"2","70.00","1057.729"
"3","50.00","1058.868"
"4","50.00","1044.700"
"5","50.00","1061.574"
"6","61.71","1015.513"
"7","86.08","1070.586"
"8","181.00","1187.392"
"9","222.00","1434.829"
"10","230.00","1515.633"
"11","262.60","1539.495"
"12","226.00","1538.931"
"13","225.00","1559.273"
"14","271.00","1515.113"
"15","266.42","1513.220"
"16","250.00","1534.506"
"17","263.02","1481.099"
"18","298.00","1351.114"
"19","283.42","1336.266"
"20","280.00","1398.646"
"21","271.00","1450.183"
"22","219.34","1346.750"
"23","170.00","1219.772"
"24","128.88","1119.148"

>>> 

看看这是否有帮助。如果您发现无法使表单正常工作，您可以随时按照其模式构建 url。