- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
我知道这很顽皮,但我这样做只是为了学习目的。我知道如何通过 xpath 获取一个链接以及如何使用标签获取所有链接,但我正在尝试通过标签获取一个图像链接,这是我目前所拥有的
它不会返回错误或链接
import urllib
import mechanize
from bs4 import BeautifulSoup
from urlparse import urlparse
def get_pic(search):
try:
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('user-agent','Mozilla')]
htmltext = browser.open("https://www.google.com/search?site=imghp&tbm=isch&source=hp&biw=1366&bih=648&q=" + search)
img_urls = []
soup = BeautifulSoup(htmltext)
iti = 0
for link in soup.find_all("a"):
img = link.get('href')
img_urls.append(img)
iti += 1
if iti == 25:
break
print img_urls[24]
except:
print "error"
get_pic("ccd")
最佳答案
我会去requests模块 - 尽管最好尝试使用 selenium( 仅用于测试目的 )
import requests
from bs4 import BeautifulSoup
data = {"Host": "www.google.com","User-Agent": "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0","Accept": "image/png,image/*;q=0.8,*/*;q=0.5",
"Accept-Language": "en-US,am;q=0.7,zh-HK;q=0.3","Accept-Encoding": "gzip, deflate","Referer": "https://www.google.com","Cookie": "PREF=ID=1111111111111111:FF=0:LD=en:CR=2:TM=1439993585:LM=1445565646:V=1:S=cGak3Dk6YKLPadm7; NID=72=Kd8i3KUAz64m6HZ9YDGnSAXDTGzj5YOqDuEIq52mLqcOwyyp4LXeUfoK_S76eoOys8GQu0k26e7DCMj2N48l75-mdQDKXzLghZPQMzPGiYH7wt4yVAVDjJ4WrGba5VhogYWnEoDVb3IJbcRJgAxkS29vEYFQGxOkZ_PGtvrFWg_5oR9rbc1XNysRIS0rZGTgBkI0L-FwD-tJSqvHS7R4zKMbxfCZv9u0pIeFbA; OGP=-5061451:","Connection": "keep-alive"}
def get_pic(search):
img_urls = []
try:
res = requests.get("https://www.google.com/search?site=imghp&tbm=isch&source=hp&biw=1366&bih=648&q=" + search,params=data)
soup = BeautifulSoup(res.content,'html.parser')
for link in soup.find_all("a"):
img = link.get('href')
img_urls.append(img)
for i in img_urls:
print i
except:
print "error"
get_pic("ccd")
https://www.google.com/search?biw=1366&bih=648&q=ccd&Accept-Language=en-US,am%3Bq%3D0.7,zh-HK%3Bq%3D0.3&Connection=keep-alive&Accept=image/png,image/*%3Bq%3D0.8,*/*%3Bq%3D0.5&Host=www.google.com&Referer=https://www.google.com&um=1&ie=UTF-8&sa=N&tab=iw
https://maps.google.com/maps?biw=1366&bih=648&q=ccd&Accept-Language=en-US,am%3Bq%3D0.7,zh-HK%3Bq%3D0.3&Connection=keep-alive&Accept=image/png,image/*%3Bq%3D0.8,*/*%3Bq%3D0.5&Host=www.google.com&Referer=https://www.google.com&um=1&ie=UTF-8&hl=en&sa=N&tab=il
https://play.google.com/?biw=1366&bih=648&q=ccd&Accept-Language=en-US,am%3Bq%3D0.7,zh-HK%3Bq%3D0.3&Connection=keep-alive&Accept=image/png,image/*%3Bq%3D0.8,*/*%3Bq%3D0.5&Host=www.google.com&Referer=https://www.google.com&um=1&ie=UTF-8&hl=en&sa=N&tab=i8
https://www.youtube.com/results?biw=1366&bih=648&q=ccd&Accept-Language=en-US,am%3Bq%3D0.7,zh-HK%3Bq%3D0.3&Connection=keep-alive&Accept=image/png,image/*%3Bq%3D0.8,*/*%3Bq%3D0.5&Host=www.google.com&Referer=https://www.google.com&um=1&ie=UTF-8&sa=N&tab=i1
https://news.google.com/nwshp?hl=en&tab=in
https://mail.google.com/mail/?tab=im
https://drive.google.com/?tab=io
https://www.google.com/intl/en/options/
http://www.google.com/history/optout?hl=en
/preferences?hl=en
https://accounts.google.com/ServiceLogin?hl=en&continue=https://www.google.com/search%3Fsite%3Dimghp%26tbm%3Disch%26source%3Dhp%26biw%3D1366%26bih%3D648%26q%3Dccd%26Accept-Language%3Den-US,am%253Bq%253D0.7,zh-HK%253Bq%253D0.3%26Connection%3Dkeep-alive%26Accept%3Dimage/png,image/*%253Bq%253D0.8,*/*%253Bq%253D0.5%26Host%3Dwww.google.com%26Referer%3Dhttps://www.google.com
/webhp?hl=en
/preferences?q=ccd&biw=1366&bih=648&ie=UTF-8&tbm=isch&sa=F
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&source=lnms&sa=X&ved=0CAQQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=vid&source=lnms&sa=X&ved=0CAYQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=nws&source=lnms&sa=X&ved=0CAcQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=shop&source=lnms&sa=X&ved=0CAgQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
https://maps.google.com/maps?biw=1366&bih=648&q=ccd&um=1&ie=UTF-8&sa=X&ved=0CAkQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=bks&source=lnms&sa=X&ved=0CAoQ_AVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=isz:l&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=isz:m&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=isz:i&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=ic:color&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=ic:gray&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=ic:trans&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=itp:face&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=itp:photo&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=itp:clipart&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=itp:lineart&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=itp:animated&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=qdr:d&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=qdr:w&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=sur:fmc&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=sur:fc&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=sur:fm&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&source=lnt&tbs=sur:f&sa=X&ved=0CA8QpwVqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&tbas=0&sa=X&ved=0CBAQuAtqFQoTCKSnl6P76sgCFYETpgodBT0O-g
/url?q=https://en.wikipedia.org/wiki/Charge-coupled_device&sa=U&ved=0CBYQwW4wAGoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNFxLr62u--uN3Gn8LXaisGss7tjpA
/url?q=https://en.wikipedia.org/wiki/Charge-coupled_device&sa=U&ved=0CBgQwW4wAWoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNFxLr62u--uN3Gn8LXaisGss7tjpA
/url?q=http://www.olympusmicro.com/primer/digitalimaging/concepts/fullframe.html&sa=U&ved=0CBoQwW4wAmoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNGRbaM6tPF3q0Lu0KbaQ_3lVdS_NQ
/url?q=http://www.globalspec.com/learnmore/video_imaging_equipment/video_cameras_accessories/ccd_cameras&sa=U&ved=0CBwQwW4wA2oVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNETFZVB3HYh4YhkQf7nLoN7k-TRNQ
/url?q=http://www.gxccd.com/art%3Fid%3D374&sa=U&ved=0CB4QwW4wBGoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNG520BsoU3bRh3HwMvtFRUCSAx9SQ
/url?q=http://hamamatsu.magnet.fsu.edu/articles/frametransfer.html&sa=U&ved=0CCAQwW4wBWoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNGEsiVVqmZA109KsbZDTp1gDGc7AA
/url?q=http://www.directindustry.com/prod/dalsa/product-25439-1173223.html&sa=U&ved=0CCIQwW4wBmoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNEt8IenufbdqPkp2EaBOmt-ET4-gA
/url?q=https://www.microscopyu.com/articles/digitalimaging/digitalintro.html&sa=U&ved=0CCQQwW4wB2oVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHxJf1kM4zJrMojSCn977uhHS3uig
/url?q=https://commons.wikimedia.org/wiki/File:IR.Lowpass.Filter.CCD.jpg&sa=U&ved=0CCYQwW4wCGoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNFWfyl-cnHiTg094w5GJBE6VpQp7g
/url?q=http://oneslidephotography.com/ccd-vs-cmos-dslr-camera-wich-one-is-better/&sa=U&ved=0CCgQwW4wCWoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNGyPh6c7w-CHCIiSUQDpqcO9ZQsyg
/url?q=https://www.microscopyu.com/articles/digitalimaging/ccdintro.html&sa=U&ved=0CCoQwW4wCmoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNEBGoOu8i-EJpb_S04_s67hKww88Q
/url?q=http://www.2mcctv.com/blog/2012_07_25-ccd-vs-cmos-image-sensor-technology/&sa=U&ved=0CCwQwW4wC2oVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHCRYMh0bSBgdrUlM0WN_RV48x_ZQ
/url?q=http://www.globalspec.com/learnmore/video_imaging_equipment/video_cameras_accessories/ccd_cameras&sa=U&ved=0CC4QwW4wDGoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNETFZVB3HYh4YhkQf7nLoN7k-TRNQ
/url?q=http://www.astrosurf.com/cavadore/technical/detectors/chungara/&sa=U&ved=0CDAQwW4wDWoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNGzkrvZ13Dmp4239t-puzALRnJpEg
/url?q=https://commons.wikimedia.org/wiki/File:CCD_sensor.JPG&sa=U&ved=0CDIQwW4wDmoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHh50a1Hi0xQiBOQtOb8BKOTQEqFw
/url?q=http://thegadgetsquare.com/1539/difference-between-cmos-and-ccd-image-sensors/&sa=U&ved=0CDQQwW4wD2oVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHKbhLuw1N3MH1uS_nqHU9QX7Sx9g
/url?q=http://www.digitalbolex.com/global-shutter/&sa=U&ved=0CDYQwW4wEGoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNErEeyDj4kT5z8PAVFDQpB4xuTeZA
/url?q=http://www.olympusmicro.com/primer/digitalimaging/concepts/ccdanatomy.html&sa=U&ved=0CDgQwW4wEWoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHF8Vz-uXRmyIv9egrSpoYYB3NWvA
/url?q=http://kcs.kcjh.ptc.edu.tw/~spt/computer/digital-image/CCD-CMOS.htm&sa=U&ved=0CDoQwW4wEmoVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHFRugkrrJ_MNdqfmGhbKdTzAc9_Q
/url?q=https://commons.wikimedia.org/wiki/File:CCD_Image_sensor.jpg&sa=U&ved=0CDwQwW4wE2oVChMIpKeXo_vqyAIVgROmCh0FPQ76&usg=AFQjCNHkZhfp_OdU2HyR0RzHp6hkSL6baA
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=colony+collapse+disorder&revid=483561372&sa=X&ved=0CD8Q1QIoAGoVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=ccd+catholic&revid=483561372&sa=X&ved=0CEAQ1QIoAWoVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=ccd+school&revid=483561372&sa=X&ved=0CEEQ1QIoAmoVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=colony+collapse+disorder+empty+hive&revid=483561372&sa=X&ved=0CEIQ1QIoA2oVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=ccd+bees&revid=483561372&sa=X&ved=0CEMQ1QIoBGoVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&q=carbonate+compensation+depth&revid=483561372&sa=X&ved=0CEQQ1QIoBWoVChMIpKeXo_vqyAIVgROmCh0FPQ76
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=20&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=40&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=60&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=80&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=100&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=120&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=140&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=160&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=180&sa=N
/search?q=ccd&biw=1366&bih=648&site=imghp&ie=UTF-8&tbm=isch&ei=b8kzVuSzD4GnmAWF-rjQDw&start=20&sa=N
https://www.google.com/advanced_image_search?biw=1366&bih=648&q=ccd&tbm=isch
https://www.google.com/imghp?hl=en
http://images.google.com/support/?hl=en
/tools/feedback/survey/html?productId=196&hl=en&query=ccd
/
/intl/en/ads
/services
/intl/en/policies/privacy/
/intl/en/policies/terms/
/intl/en/about.html
关于python - 如何用漂亮的汤从谷歌抓取一张图片?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/33442923/
这两个包看起来非常相似: http://www.passportjs.org/packages/passport-google-oauth2/ http://www.passportjs.org/pa
我想在我的网站上添加通过 Google 和 Twitter 登录的按钮。我需要只使用应用程序的客户端而不是服务器端来完成此操作。但我没有找到任何 API。对于我发现的所有内容,我需要使用带有 key
我使用此链接通过 google plus 共享我的页面。 https://plus.google.com/share?url=http%3A%2F%2Fexample.com%2Fcompany%2
我正在尝试学习 google API,并且我的经验是使用 Python,因此我尝试使用 google api python 客户端来访问一些 google 服务,但在构建服务对象时遇到错误。 从 ap
在其实际的实时托管平台上构建实时站点的努力中,有没有办法告诉谷歌不要索引该网站?我发现了以下内容: http://support.google.com/webmasters/bin/answer.py
我正在开发一个 iOS 应用程序。当我运行用于 google+ 登录的程序时,在我点击允许访问按钮后,会显示此消息。 You've reached this page because we have
我有一个非常复杂的网站,每个页面包含 11 个 js 文件。 我最近添加了 google +1 按钮,代码如下: 这会正确显示 +1 按钮,直到我单击它。当我单击它时,出现此错误:https://
我正在尝试使用 google API 创建一个 html 文件,以便在 google MAPS 上显示 KML 文件。 这是 HTML 代码: function initMap() {
我是使用 Google Benchmark 的新手,在本地运行代码与在 Quick-Bench.com 上运行代码时,我收到了运行相同基准测试(下方)的不同结果,该基准测试使用 C++ 检索本地时间.
我已按照 Google 网站上的说明通过添加以下元标记在我的 AngularJS 网站上启用 Ajax 抓取: 呈现的内容有一些链接,如: User 1 User 2 User 3 还有一些呈现动态
通过 Google 手册实现 Google AppInvite - link . 启动 Invite Activity 并在 LogCat 中获取下一步: E/AppInviteAgent: Get
那么有人用过 Google 的 Go 吗?我想知道数学性能(例如触发器)与其他具有垃圾收集器的语言(如 Java 或 .NET)相比如何? 有人调查过吗? 最佳答案 理论性能:纯 Go 程序的理论性能
Stackdriver 测试我的网站启动速度慢 我们使用 cloudflare 作为我们的站点 CDN 提供商。我们使用 stackdriver 从外部测试站点可用性,我们将时间检查间隔设置为 1 分
我正在尝试使用 stax.GeneralConv() ( https://jax.readthedocs.io/en/latest/_modules/jax/experimental/stax.htm
我有一个从谷歌金融中提取日内数据的软件。但是,由于昨天 Google 更新了 API,所以软件报错了 Conversion from string HTML HEAD meta http-equiv=
我们在尝试从 Google 获取 oAuth token 时遇到“redirect_uri_mismatch”错误: [client 127.0.0.1:49892] {\n "error" : "
我的网站正在使用 Google reCAPTCHA 控件,但我听说它被阻止了 中国,反正我看到有人报告说将 API 更改为 https://www.recaptcha.net在中国工作? Anyone
背景 WordPress Google Adsense 谷歌自动插入 anchor 定广告 https://pptmon.com 问题 如下图所示,主播广告的容器高度太大了! 如何调整高度? 这是谷歌
我在使用 Google Colab 时遇到问题。当我想制作一个新的 Python3 Notebook 时,由于我登录了我的 Google 帐户,因此无法加载刚刚打开的新页面。 我该怎么办? 感谢您的帮
我正在使用 facebook和 google oauth2使用 passport js 登录, 有了这个流 用户点击登录按钮 重定向到 facebook/google auth 页面(取决于用户选择的
我是一名优秀的程序员,十分优秀!