machine-learning - scikit学习: elastic net approaching ridge

Question

因此，弹性网络应该是岭回归(L2 正则化)和 lasso(L1 正则化)之间的混合体。然而，似乎即使 l1_ratio 为 0，我也没有得到与 ridge 相同的结果。我知道岭使用梯度下降，弹性网使用坐标下降，但最优值应该是相同的，不是吗？此外，我发现 elasticnet 经常无明显原因地抛出 ConvergenceWarnings，而 lasso 和 ridge 则不会。这是一个片段:

from sklearn.datasets import load_boston
from sklearn.utils import shuffle
from sklearn.linear_model import ElasticNet, Ridge, Lasso
from sklearn.model_selection import train_test_split

data = load_boston()
X, y = shuffle(data.data, data.target, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=43)
alpha = 1

en = ElasticNet(alpha=alpha, l1_ratio=0)
en.fit(X_train, y_train)
print('en train score: ', en.score(X_train, y_train))

rr = Ridge(alpha=alpha)
rr.fit(X_train, y_train)
print('rr train score: ', rr.score(X_train, y_train))

lr = Lasso(alpha=alpha)
lr.fit(X_train, y_train)
print('lr train score: ', lr.score(X_train, y_train))
print('---')
print('en test score: ', en.score(X_test, y_test))
print('rr test score: ', rr.score(X_test, y_test))
print('lr test score: ', lr.score(X_test, y_test))
print('---')
print('en coef: ', en.coef_)
print('rr coef: ', rr.coef_)
print('lr coef: ', lr.coef_)

即使 l1_ratio 为 0，弹性网络的训练和测试分数也接近 lasso 分数(而不是您期望的岭)。此外，即使我增加 max_iter (甚至达到 1000000 似乎没有效果)和 tol (0.1 仍然会抛出错误，但 0.2 不会)，弹性网似乎会抛出 ConvergenceWarning。增加 alpha(如警告所示)也没有效果。

Answer 1

根据@sascha的回答，可以匹配两个模型之间的结果:

import sklearn
print(sklearn.__version__)

from sklearn.linear_model import Ridge, ElasticNet
from sklearn.datasets import load_boston

dataset = load_boston()
X = dataset.data
y = dataset.target

f = Ridge(alpha=1, 
          fit_intercept=True, normalize=False, 
          copy_X=True, max_iter=1000, tol=1e-4, random_state=42, 
          solver='auto')
g = ElasticNet(alpha=1/X.shape[0], l1_ratio=1e-16, 
               fit_intercept=True, normalize=False, 
               copy_X=True, max_iter=1000, tol=1e-4, random_state=42, 
               precompute=False, warm_start=False, 
               positive=False, selection='cyclic')

f.fit(X, y)
g.fit(X, y)

print(abs(f.coef_ - g.coef_) / abs(f.coef_))

输出:

0.19.2
[1.19195623e-14 1.17076625e-15 3.25973465e-13 1.61694280e-14
 4.77274767e-15 4.15332538e-15 6.15640568e-14 1.61772832e-15
 4.56125088e-14 5.44320605e-14 8.99189018e-15 2.31213025e-15
 3.74181954e-15]