python - 如何使用 optuna trial 在 sklearn MLPRegressor 中设置 hidden_​​layer_sizes

Question

我想将 [OPTUNA][1] 与 sklearn [MLPRegressor][1] 模型一起使用。

对于几乎所有超参数，如何为它们设置 OPTUNA 都非常简单。例如，要设置学习率:learning_rate_init = trial.suggest_float('learning_rate_init ',0.0001, 0.1001, step=0.005)

我的问题是如何为 hidden_​​layer_sizes 设置它，因为它是一个元组。假设我想要两个隐藏层，第一个有 100 个神经元，第二个有 50 个神经元。如果没有 OPTUNA，我会这样做:

MLPRegressor( hidden_​​layer_sizes =(100,50))

但是如果我想让 OPTUNA 在每一层尝试不同的神经元呢？例如，从 100 到 500，如何设置？ MLPRegressor 需要一个元组

Answer 1

您可以按如下方式设置目标函数:

import optuna
import warnings
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
warnings.filterwarnings('ignore')

X, y = make_regression(random_state=1)

X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=1)

def objective(trial):

    params = {
        'learning_rate_init': trial.suggest_float('learning_rate_init ', 0.0001, 0.1, step=0.005),
        'first_layer_neurons': trial.suggest_int('first_layer_neurons', 10, 100, step=10),
        'second_layer_neurons': trial.suggest_int('second_layer_neurons', 10, 100, step=10),
        'activation': trial.suggest_categorical('activation', ['identity', 'tanh', 'relu']),
    }

    model = MLPRegressor(
        hidden_layer_sizes=(params['first_layer_neurons'], params['second_layer_neurons']),
        learning_rate_init=params['learning_rate_init'],
        activation=params['activation'],
        random_state=1,
        max_iter=100
    )

    model.fit(X_train, y_train)

    return mean_squared_error(y_valid, model.predict(X_valid), squared=False)

study = optuna.create_study(direction='minimize')
study.optimize(objective, n_trials=3)
# [I 2021-11-11 18:04:02,216] A new study created in memory with name: no-name-14c92e38-b8cd-4b8d-8a95-77158d996f20
# [I 2021-11-11 18:04:02,283] Trial 0 finished with value: 161.8347337123744 and parameters: {'learning_rate_init ': 0.0651, 'first_layer_neurons': 20, 'second_layer_neurons': 40, 'activation': 'tanh'}. Best is trial 0 with value: 161.8347337123744.
# [I 2021-11-11 18:04:02,368] Trial 1 finished with value: 159.55535852658082 and parameters: {'learning_rate_init ': 0.0551, 'first_layer_neurons': 90, 'second_layer_neurons': 70, 'activation': 'relu'}. Best is trial 1 with value: 159.55535852658082.
# [I 2021-11-11 18:04:02,440] Trial 2 finished with value: 161.73980822730888 and parameters: {'learning_rate_init ': 0.0051, 'first_layer_neurons': 100, 'second_layer_neurons': 30, 'activation': 'identity'}. Best is trial 1 with value: 159.55535852658082.