python - 使用 BeautifulSoup 提取标签内的文本

Question

我正在尝试使用 BeautifulSoup 抓取网站源代码中的文本。 部分源代码如下所示:

        <hr />
        <div class="see-more inline canwrap" itemprop="genre">
            <h4 class="inline">Genres:</h4>
<a href="/genre/Horror?ref_=tt_stry_gnr"
> Horror</a>&nbsp;<span>|</span>
<a href="/genre/Mystery?ref_=tt_stry_gnr"
> Mystery</a>&nbsp;<span>|</span>
<a href="/genre/Thriller?ref_=tt_stry_gnr"
> Thriller</a>
        </div>      

所以我一直在尝试用这些代码提取文本“恐怖”“神秘”和“惊悚”:

import requests
from bs4 import BeautifulSoup
url1='http://www.imdb.com/title/tt5308322/?ref_=inth_ov_tt'
r1=requests.get(url1)
soup1= BeautifulSoup(r1.text, 'lxml')
genre1=soup1.find('div',attrs={'itemprop':'genre'}).contents
print(genre1)

但返回结果为:

['\n', <h4 class="inline">Genres:</h4>, '\n', <a href="/genre/Horror?
ref_=tt_stry_gnr"> Horror</a>, '\xa0', <span>|</span>, '\n', <a 
href="/genre/Mystery?ref_=tt_stry_gnr"> Mystery</a>, '\xa0', <span>|</span>, 
'\n', <a href="/genre/Thriller?ref_=tt_stry_gnr"> Thriller</a>, '\n']

我在 python 和 webscraping 方面还很陌生，所以我会很感激我能得到的所有帮助。谢谢!

Answer 1

使用简单的 BeautifulSoup.select() 函数将需要的元素提取到 CSS 选择器:

import requests
from bs4 import BeautifulSoup

url1 = 'http://www.imdb.com/title/tt5308322/?ref_=inth_ov_tt'
soup = BeautifulSoup(requests.get(url1).text, 'lxml')
genres = [a.text.strip() for a in soup.select("div[itemprop='genre'] > a")]

print(genres)

输出:

['Horror', 'Mystery', 'Thriller']

https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors