python - 用 mechanize 下载文件

Question

我有一个已打开页面的浏览器实例。我想下载并保存所有链接(它们是 PDF)。有人知道怎么做吗？

谢谢

Answer 1

import urllib, urllib2,cookielib, re
#http://www.crummy.com/software/BeautifulSoup/ - required
from BeautifulSoup import BeautifulSoup

HOST = 'https://www.adobe.com/'

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))

req = opener.open( HOST + 'pdf' )
responce = req.read()

soup = BeautifulSoup( responce )
pdfs = soup.findAll(name = 'a', attrs = { 'href': re.compile('\.pdf') })
for pdf in pdfs:
    if 'https://' not in pdf['href']:
        url = HOST + pdf['href']
    else:
        url = pdf['href']
    try:
        #http://docs.python.org/library/urllib.html#urllib.urlretrieve
        urllib.urlretrieve(url)
    except Exception, e:
        print 'cannot obtain url %s' % ( url, )
        print 'from href %s' % ( pdf['href'], )
        print e
    else:
        print 'downloaded file'
        print url