python - 使用 BeautifulSoup 分析和编辑 html 代码中的链接

Question

我有一部分html页面。我必须从中找出所有链接并将其替换为标记 <can_be_link> .

下一个代码几乎可以完成我想要的所有操作，但它在位于某些行(而不是一行)上的链接上失败，并且该行以制表符开头(在我的示例中，这是与 http://bad.com 的链接)。

如何正确解决这个问题？

# -*- coding: utf-8 -*-
import BeautifulSoup
import re

if __name__=="__main__":
    body = """                  
    <a href="http://good.com" target="_blank">good link</a>
    <ul>
                        <li class="FOLLOW">
                            <a href="http://bad.com" target="_blank">
                                <em></em>
                                <span>
                                    <strong class="FOLLOW-text">Follow On</strong>
                                    <strong class="FOLLOW-logo"></strong>
                                </span>
                            </a>
                        </li>
    </ul>

"""
    metka_link = '<can_be_link>'
    soup = BeautifulSoup.BeautifulSoup(body)
    hrefs = soup.findAll(name = 'a', attrs = { 'href': re.compile('\.*') })
    repl = {}
    for t in hrefs:
        line = str(t)
            # print '\n'*2, line
        if not t.has_key('href'):
            continue
        href = t['href'].lower()
        if href.find('http') == 0 or href.find('//') == 0:
            body = body.replace(line, metka_link)

    print body

结果是

<can_be_link>
<ul>
                                        <li class="FOLLOW">
                                                <a href="http://bad.com" target="_blank">
                                                        <em></em>
                                                        <span>
                                                                <strong class="FOLLOW-text">Follow On</strong>
                                                                <strong class="FOLLOW-logo"></strong>
                                                        </span>
                                                </a>
                                        </li>
</ul>

但是想要的结果一定是

<can_be_link>
<ul>
                                        <li class="FOLLOW">
                                                <can_be_link>
                                        </li>
</ul>

Answer 1

使用replace_with()方法:

PageElement.replace_with() removes a tag or string from the tree, and replaces it with the tag or string of your choice

# -*- coding: utf-8 -*-
from bs4 import BeautifulSoup

body = """
<a href="http://good.com" target="_blank">good link</a>
<ul>
                    <li class="FOLLOW">
                        <a href="http://bad.com" target="_blank">
                            <em></em>
                            <span>
                                <strong class="FOLLOW-text">Follow On</strong>
                                <strong class="FOLLOW-logo"></strong>
                            </span>
                        </a>
                    </li>
</ul>

"""

soup = BeautifulSoup(body, 'html.parser')

links = soup.find_all('a')
for link in links:
    link = link.replace_with('<can_be_link>')

print soup.prettify(formatter=None)

打印:

<can_be_link>
<ul>
 <li class="FOLLOW">
  <can_be_link>
 </li>
</ul>

请注意导入语句 - 使用第 4 个 BeautifulSoup 版本，因为 Beautiful Soup 3 不再开发，建议所有新项目使用 Beautiful Soup 4。