python - Beautiful Soup 和 Pandas 的 UTF-8 错误

Question

下面是一个漂亮的 Python 汤刮刀，它曾经成功地从 MLB.com 上刮取球队名单。现在，当我尝试运行代码时，出现以下错误。

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x97 in position 0: invalid start byte

在阅读多个 Stackoverflow 线程后，我认为我必须更改“with open”行，但我对如何更改当前的代码格式而不必牺牲 CSV writer 和 df 到 MYSQL 格式感到困惑。有谁知道如何调整代码来解决这个 utf-8 问题？提前致谢!

import requests
import csv
import pandas as pd
from bs4 import BeautifulSoup
from sqlalchemy import create_engine

team_list={'orioles','yankees','redsox','rays','indians','twins','tigers','whitesox','royals','astros','mariners','athletics',
           'angels','rangers','phillies','braves','nationals','marlins','mets','cubs','brewers','cardinals','pirates','reds',
           'dodgers','dbacks','rockies','giants','padres','bluejays'}

header_added = False

for team in team_list:
    page = requests.get('http://m.{}.mlb.com/roster/'.format(team))
    soup = BeautifulSoup(page.text, 'html.parser')

    soup.find(class_='nav-tabset-container').decompose()
    soup.find(class_='column secondary span-5 right').decompose()

    roster = soup.find(class_='page page-index')
    names = [n.contents[0] for n in roster.find_all('a')]
    ids = [n['href'].split('/')[2] for n in roster.find_all('a')]
    number = [n.contents[0] for n in roster.find_all('td', index='0')]
    handedness = [n.contents[0] for n in roster.find_all('td', index='3')]
    height = [n.contents[0] for n in roster.find_all('td', index='4')]
    weight = [n.contents[0] for n in roster.find_all('td', index='5')]
    DOB = [n.contents[0] for n in roster.find_all('td', index='6')]
    team = [soup.find('meta',property='og:site_name')['content']] * len(names)

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        if not header_added:
            f.writerow(['Name', 'ID', 'Number', 'Hand', 'Height', 'Weight', 'DOB', 'Team'])
            header_added=True
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

    df = pd.read_csv('MLB_Active_Roster.csv')

    engine = create_engine("mysql+pymysql://{user}:{pw}@localhost/{db}"
                           .format(user="user",
                                   pw="password",
                                   db="mlb"))
    conn = engine.connect()
    df.to_sql(con=engine, name='mlbactiveroster', if_exists='replace')

Answer 1

更改此行

df = pd.read_csv('MLB_Active_Roster.csv')

到

df = pd.read_csv('MLB_Active_Roster.csv', encoding='ISO-8859-1')

为了处理不同格式的文件。