python - 多项式回归度数增加误差

Question

我正在尝试预测波士顿房价。当我选择多项式回归次数为1或2时，R2分数还可以。但 3 度会降低 R2 分数。

# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Importing the dataset
from sklearn.datasets import load_boston
boston_dataset = load_boston()
dataset = pd.DataFrame(boston_dataset.data, columns = boston_dataset.feature_names)
dataset['MEDV'] = boston_dataset.target

X = dataset.iloc[:, 0:13].values
y = dataset.iloc[:, 13].values.reshape(-1,1)

# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

# Fitting Linear Regression to the dataset
from sklearn.linear_model import LinearRegression

# Fitting Polynomial Regression to the dataset
from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree = 2)   # <-- Tuning to 3
X_poly = poly_reg.fit_transform(X_train)
poly_reg.fit(X_poly, y_train)
lin_reg_2 = LinearRegression()
lin_reg_2.fit(X_poly, y_train)

y_pred = lin_reg_2.predict(poly_reg.fit_transform(X_test))

from sklearn.metrics import r2_score
print('Prediction Score is: ', r2_score(y_test, y_pred))

输出(度=2):

Prediction Score is:  0.6903318065831567

输出(度=3):

Prediction Score is:  -12898.308114085281

Answer 1

这称为模型过度拟合。您所做的是将模型完美地拟合到训练集上，这将导致高方差。当您在训练集上很好地拟合您的假设时，它将在测试集上失败。您可以使用 r2_score(X_train,y_train) 检查训练集的 r2_score。会很高。您需要平衡偏差和方差之间的权衡。

您可以尝试其他回归模型，例如 lasso 和 ridge，并且可以使用它们的 alpha 值，以防您正在寻找高 r2_score。为了更好地理解，我放置了一张图像，该图像将显示假设线如何随着多项式次数的增加而受到影响。