python - 使用代码中的 Mechanize 错误进行网页抓取

Question

import mechanize
br=mechanize.Browser()
r=br.open("http://www.drugs.com/search-wildcard-phonetic.html")
br.select_form(nr=0)
br.form['searchterm']='panadol'
br.submit()
print br.response().read()

error in above code:
Traceback (most recent call last):
  File "mech2.py", line 6, in <module>
    br.submit()
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 541, in submit
    return self.open(self.click(*args, **kwds))
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 203, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 255, in _mech_open
    raise response
mechanize._response.httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt

请帮助更正上述代码

Answer 1

看起来您的代码没有任何问题。你的错误信息

mechanize._response.httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt

看起来您违反了 robots.txt文件。如果您不想看到此错误消息，请停止滥用此网站，并联系他们以获取可接受的数据使用方式。