python - 从篮球引用网站解析 NBA 赛季统计数据，如何删除 html 评论标签

Question

我正在尝试解析篮球引用.com ( https://www.basketball-reference.com/leagues/NBA_1980.html ) 中的杂项统计表。但是，我想解析的表位于 html 注释内。

使用以下代码

html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
cleaned_soup = BeautifulSoup(re.sub("<!--|-->","", html))

结果如下

TypeError                                 Traceback (most recent call last)
<ipython-input-35-93508687bbc6> in <module>()
----> 1 cleaned_soup = BeautifulSoup(re.sub("<!--|-->","", html))

~/.pyenv/versions/3.7.0/lib/python3.7/re.py in sub(pattern, repl, string, count, flags)
    190     a callable, it's passed the Match object and must return
    191     a replacement string to be used."""
--> 192     return _compile(pattern, flags).sub(repl, string, count)
    193 
    194 def subn(pattern, repl, string, count=0, flags=0):

TypeError: cannot use a string pattern on a bytes-like object

我使用的是python3.7。

Answer 1

您可以使用 BeautifulSoup 仅返回 HTML 中的注释，而不是尝试使用 re 将注释中的所有 HTML 放入您的 HTML 中。然后也可以使用 BeautifulSoup 解析这些内容以根据需要提取任何表格元素，例如:

import requests
from bs4 import BeautifulSoup, Comment


html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
soup = BeautifulSoup(html, "html.parser")

for comment in soup.find_all(text=lambda t : isinstance(t, Comment)):
    comment_html = BeautifulSoup(comment, "html.parser")

    for table in comment_html.find_all("table"):
        for tr in table.find_all("tr"):
            row = [td.text for td in tr.find_all("td")]
            print(row)
        print()

这将为您提供表格中的行，开头为:

['Finals', 'Cleveland Cavaliers \nover \nGolden State Warriors\n\xa0(4-3)\n', 'Series Stats']
['\n\n\nGame 1\nThu, June 2\nCleveland Cavaliers\n89@ Golden State Warriors\n104\n\nGame 2\nSun, June 5\nCleveland Cavaliers\n77@ Golden State Warriors\n110\n\nGame 3\nWed, June 8\nGolden State Warriors\n90@ Cleveland Cavaliers\n120\n\nGame 4\nFri, June 10\nGolden State Warriors\n108@ Cleveland Cavaliers\n97\n\nGame 5\nMon, June 13\nCleveland Cavaliers\n112@ Golden State Warriors\n97\n\nGame 6\nThu, June 16\nGolden State Warriors\n101@ Cleveland Cavaliers\n115\n\nGame 7\nSun, June 19\nCleveland Cavaliers\n93@ Golden State Warriors\n89\n\n\n', 'Game 1', 'Thu, June 2', 'Cleveland Cavaliers', '89', '@ Golden State Warriors', '104', 'Game 2', 'Sun, June 5', 'Cleveland Cavaliers', '77', '@ Golden State Warriors', '110', 'Game 3', 'Wed, June 8', 'Golden State Warriors', '90', '@ Cleveland Cavaliers', '120', 'Game 4', 'Fri, June 10', 'Golden State Warriors', '108', '@ Cleveland Cavaliers', '97', 'Game 5', 'Mon, June 13', 'Cleveland Cavaliers', '112', '@ Golden State Warriors', '97', 'Game 6', 'Thu, June 16', 'Golden State Warriors', '101', '@ Cleveland Cavaliers', '115', 'Game 7', 'Sun, June 19', 'Cleveland Cavaliers', '93', '@ Golden State Warriors', '89']
['Game 1', 'Thu, June 2', 'Cleveland Cavaliers', '89', '@ Golden State Warriors', '104']
['Game 2', 'Sun, June 5', 'Cleveland Cavaliers', '77', '@ Golden State Warriors', '110']
['Game 3', 'Wed, June 8', 'Golden State Warriors', '90', '@ Cleveland Cavaliers', '120']
['Game 4', 'Fri, June 10', 'Golden State Warriors', '108', '@ Cleveland Cavaliers', '97']

注意:为了避免出现无法在类似字节的对象上使用字符串模式，您可以使用.text而不是.content 将字符串传递给正则表达式。