python BeautifulSoup soup.findAll()，如何使搜索结果匹配

Question

我最近学习了 BeautifulSoup，作为练习，我想使用 BeautifulSoup 从职位发布中读取和提取公司和位置信息。这是我的代码:

import urllib
from BeautifulSoup import *

url="http://www.indeed.com/jobs?q=hadoop&start=50"
html=urllib.urlopen(url).read()
soup=BeautifulSoup(html)
company=soup.findAll("span",{"class":"company"})
location=soup.findAll("span",{"class":"location"})

# for c in company:
#   print c.text
# print 
# for l in location:
#   print l.text

print len(company)
print len(location)

我发现公司的长度和地点不一样。所以我不知道哪对(公司、地点)不完整。我怎样才能使它们匹配？

Answer 1

您需要迭代搜索结果 block 并获取每个 block 的公司位置对:

for result in soup.find_all("div", {"class": "result"}):  # or soup.select("div.result")
    company = result.find("span", {"class": "company"}).get_text(strip=True)
    location = result.find("span", {"class": "location"}).get_text(strip=True)

    print(company, location)

您还应该切换到 BeautifulSoup4 ，您使用的版本相当旧:

pip install beautifulsoup4

并替换:

from BeautifulSoup import *

与:

from bs4 import BeautifulSoup

<小时/>

上面的代码打印:

(u'PsiNapse', u'San Mateo, CA')
(u'Videology', u'Baltimore, MD')
(u'Charles Schwab', u'Lone Tree, CO')
(u'Cognizant', u'Dover, NH')
...
(u'Concur', u'Bellevue, WA')