python - SK学习: Gaussian Process Regression not changed during learning

Question

我正在尝试使用 GaussianProcessRegressor 拟合 GP，我注意到我的超参数仍处于初始值。我在 gpr.py 中做了一些步骤，但无法查明具体原因。使用初始值进行预测会产生一条零线。

我的数据包含 5400 个样本，每个样本具有 12 个特征，映射到一个输出变量。尽管设计可能不是那么好，但我仍然希望能学到一些东西。

所需文件:

features.txt

output.txt

import pandas as pd
import numpy as np
import time
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel,WhiteKernel

designmatrix = pd.read_csv('features.txt', index_col = 0)
y = pd.read_csv('output.txt', header=None, index_col = 0)

# The RBF kernel is a stationary kernel. It is also known as the “squared exponential” kernel. 
# It is parameterized by a length-scale parameter length_scale>0, which can either be a scalar (isotropic variant of the kernel) 
# or a vector with the same number of dimensions as the inputs X (anisotropic variant of the kernel). 
# 
# The ConstantKernel can be used as part of a product-kernel where it scales the magnitude of the other factor (kernel) or as 
# part of a sum-kernel, where it modifies the mean of the Gaussian process.
#
# The main use-case of the White kernel is as part of a sum-kernel where it explains the noise-component of the signal. 
# Tuning its parameter corresponds to estimating the noise-level: k(x_1, x_2) = noise_level if x_1 == x_2 else 0

kernel = ConstantKernel(0.1, (1e-23, 1e5)) * 
RBF(0.1*np.ones(designmatrix.shape[1]), (1e-23, 1e10) ) + WhiteKernel(0.1, (1e-23, 1e5))

gp = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=0)

print('Training')
t = time.time()
gp = gp.fit(designmatrix, y)
elapsed = time.time() - t
print(elapsed)

score = gp.score(designmatrix, y)
print(score)

print("initial params")
params = gp.get_params()
print(params)
print("learned kernel params")
print(gp.kernel_.get_params())

结果如下:

initial params

{'alpha': 1e-10, 'copy_X_train': True, 'kernel__k1': 1**2, 'kernel__k2': RBF(len
gth_scale=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]), 'kernel__k1__constant_value': 1
.0, 'kernel__k1__constant_value_bounds': (1e-05, 100000.0), 'kernel__k2__length_
scale': array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.]), 'ke
rnel__k2__length_scale_bounds': (1e-05, 100000.0), 'kernel': 1**2 * RBF(length_s
cale=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]), 'n_restarts_optimizer': 0, 'normaliz
e_y': False, 'optimizer': 'fmin_l_bfgs_b', 'random_state': None}

learned kernel params

{'k1': 1**2, 'k2': RBF(length_scale=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]), 'k1__
constant_value': 1.0, 'k1__constant_value_bounds': (1e-05, 100000.0), 'k2__lengt
h_scale': array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.]), '
k2__length_scale_bounds': (1e-05, 100000.0)}

所以，内核参数不变...

• 有没有办法检查警告？

• 我做错了什么，或者有什么我可以检查的吗？

任何帮助将不胜感激......

本

Answer 1

还没有答案(还)

开始记笔记

数据对于一个 SO 问题来说太大了，我们测试你的问题的时间也太长了。我已将您的代码更改为仅包含每个文件的前 600 行。您粘贴在此处的代码也无法运行，我已修复该问题。

尾注

使用 python 3.6.4、scikit-learn==0.19.1 和 numpy==1.14.2。

正如您在 n_restarts_optimizer 的文档中看到的，如果您想要优化内核超参数，您需要将其设置为大于 0。\

n_restarts_optimizer : int, optional (default: 0)
    The number of restarts of the optimizer for finding the kernel's
    parameters which maximize the log-marginal likelihood. The first run
    of the optimizer is performed from the kernel's initial parameters,
    the remaining ones (if any) from thetas sampled log-uniform randomly
    from the space of allowed theta-values. If greater than 0, all bounds
    must be finite. Note that n_restarts_optimizer == 0 implies that one
    run is performed.

因此，将代码中的值从 0 更改为 2，会产生以下输出:

import pandas as pd
import numpy as np
import time
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel,WhiteKernel

designmatrix = pd.read_csv('features.txt', index_col = 0).iloc[0:600,]
y = pd.read_csv('output.txt', header=None, index_col = 0).iloc[0:600,]

# The RBF kernel is a stationary kernel. It is also known as the “squared exponential” kernel. 
# It is parameterized by a length-scale parameter length_scale>0, which can either be a scalar (isotropic variant of the kernel) 
# or a vector with the same number of dimensions as the inputs X (anisotropic variant of the kernel). 
# 
# The ConstantKernel can be used as part of a product-kernel where it scales the magnitude of the other factor (kernel) or as 
# part of a sum-kernel, where it modifies the mean of the Gaussian process.
#
# The main use-case of the White kernel is as part of a sum-kernel where it explains the noise-component of the signal. 
# Tuning its parameter corresponds to estimating the noise-level: k(x_1, x_2) = noise_level if x_1 == x_2 else 0

kernel = ConstantKernel(0.1, (1e-23, 1e5)) * \
         RBF(0.1*np.ones(designmatrix.shape[1]), (1e-23, 1e10) ) + \
         WhiteKernel(0.1, (1e-23, 1e5))

gp = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=2)

print("initial params")
params = gp.get_params()
print(params)

print('Training')
t = time.time()
gp = gp.fit(designmatrix, y)
elapsed = time.time() - t
print(elapsed)

score = gp.score(designmatrix, y)
print(score)

print("learned kernel params")
print(gp.kernel_.get_params())

输出:

initial params
{'alpha': 1e-10, 'copy_X_train': True, 'kernel__k1': 0.316**2 * RBF(length_scale=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]), 'kernel__k2': WhiteKernel(noise_level=0.1), 'kernel__k1__k1': 0.316**2, 'kernel__k1__k2': RBF(length_scale=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]), 'kernel__k1__k1__constant_value': 0.1, 'kernel__k1__k1__constant_value_bounds': (1e-23, 100000.0), 'kernel__k1__k2__length_scale': array([0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]), 'kernel__k1__k2__length_scale_bounds': (1e-23, 10000000000.0), 'kernel__k2__noise_level': 0.1, 'kernel__k2__noise_level_bounds': (1e-23, 100000.0), 'kernel': 0.316**2 * RBF(length_scale=[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]) + WhiteKernel(noise_level=0.1), 'n_restarts_optimizer': 2, 'normalize_y': False, 'optimizer': 'fmin_l_bfgs_b', 'random_state': None}
Training
3.9108407497406006
1.0
learned kernel params
{'k1': 20.3**2 * RBF(length_scale=[0.00289, 9.29e-15, 8.81e-20, 0.00165, 2.7e+08, 3.2e+06, 0.233, 5.62e+07, 8.78e+07, 0.0169, 4.88e-21, 3.23e-20]), 'k2': WhiteKernel(noise_level=2.17e-13), 'k1__k1': 20.3**2, 'k1__k2': RBF(length_scale=[0.00289, 9.29e-15, 8.81e-20, 0.00165, 2.7e+08, 3.2e+06, 0.233, 5.62e+07, 8.78e+07, 0.0169, 4.88e-21, 3.23e-20]), 'k1__k1__constant_value': 411.28699807005, 'k1__k1__constant_value_bounds': (1e-23, 100000.0), 'k1__k2__length_scale': array([2.88935323e-03, 9.29401433e-15, 8.81112330e-20, 1.64832813e-03,
       2.70454686e+08, 3.20194179e+06, 2.32646715e-01, 5.62487948e+07,
       8.77636837e+07, 1.68642019e-02, 4.88384874e-21, 3.22536538e-20]), 'k1__k2__length_scale_bounds': (1e-23, 10000000000.0), 'k2__noise_level': 2.171274720012903e-13, 'k2__noise_level_bounds': (1e-23, 100000.0)}

能否请您编辑您的问题，以便可以重现您的观察结果？