python - Optuna 从 "outside"传递参数字典

Question

我正在使用 Optuna 优化一些目标函数。我想创建“包装”标准 Optuna 代码的自定义类。

例如，这是我的类(class)(它仍在进行中!):

class Optimizer(object):
    
    def __init__(self, param_dict, model, train_x, valid_x, train_y, valid_y):
        self.model = model
        self.param_dict = param_dict
        self.train_x, self.valid_x, self.train_y, self.valid_y = train_x, valid_x, train_y, valid_y
        
    def optimization_function(self, trial):
        self.dtrain = lgb.Dataset(self.train_x, label=self.train_y)
        gbm = lgb.train(param, dtrain)
        
        preds = gbm.predict(self.valid_x)
        pred_labels = np.rint(preds)
        accuracy = sklearn.metrics.accuracy_score(self.valid_y, pred_labels)
        return accuracy
    
    
    def optimize(self, direction, n_trials):
        study = optuna.create_study(direction = direction)
        study.optimize(self.optimization_function, n_trials = n_trials)    
        return study.best_trial

我试图在这个类中包含 optuna 优化的所有“逻辑”，而不是每次都编写一些代码如下(来自文档):

import optuna


class Objective(object):
    def __init__(self, min_x, max_x):
        # Hold this implementation specific arguments as the fields of the class.
        self.min_x = min_x
        self.max_x = max_x

    def __call__(self, trial):
        # Calculate an objective value by using the extra arguments.
        x = trial.suggest_float("x", self.min_x, self.max_x)
        return (x - 2) ** 2


# Execute an optimization by using an `Objective` instance.
study = optuna.create_study()
study.optimize(Objective(-100, 100), n_trials=100)

我想让我的代码“模块化”并将所有内容合并到一个类中。我的最终目标是根据 __init__ 函数中给定的输入模型设置优化函数的不同"template"。

所以，回到主要问题，我想从外部传递 param 字典。基本上我希望能够从我的类外部声明它并在 __init__ 函数中传递我的字典。

然而，Optuna 代码中常用的范围和分布取决于 trial 对象，因此我无法执行以下操作:

my_dict = {
    'objective': 'binary',
    'metric': 'binary_logloss',
    'verbosity': -1,
    'boosting_type': 'gbdt',
     # HERE I HAVE A DEPENDENCY FROM trial.suggest_loguniform, I can't declare the dictionary outside the objective function
    'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),
    'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),
    'num_leaves': trial.suggest_int('num_leaves', 2, 256),
    'feature_fraction': trial.suggest_uniform('feature_fraction', 0.4, 1.0),
    'bagging_fraction': trial.suggest_uniform('bagging_fraction', 0.4, 1.0),
    'bagging_freq': trial.suggest_int('bagging_freq', 1, 7),
    'min_child_samples': trial.suggest_int('min_child_samples', 5, 100),
} 
my_optimizer = Optimizer(my_dict, ..., ..., ..., ......)
best_result = my_optimizer.optimize('maximize', 100)

是否有任何变通或解决方案来通过此字典？

Answer 1

我不确定我是否理解你的问题；但你的意思是你想将字典传递给目标函数吗？

如果是，这对我有用，使用 lambda，来自 optuna 的常见问题解答:

import optuna

# Objective function that takes three arguments.
def objective(trial, min_x, max_x):
    x = trial.suggest_float("x", min_x, max_x)
    return (x - 2) ** 2


# Extra arguments.
min_x = -100
max_x = 100

# Execute an optimization by using the above objective function wrapped by `lambda`.
study = optuna.create_study()
study.optimize(lambda trial: objective(trial, min_x, max_x), n_trials=100)