python - sklearn异常错误: fitfailed warning from GridSearchCV in tensorflow

Question

我想针对图像分类任务(多类)优化CNN模型的超参数。为此，我使用了gridSearchCV中的sklearn，但始终出现如下警告和错误值:

/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py", line 531, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "/usr/local/lib/python3.6/dist-packages/keras/wrappers/scikit_learn.py", line 209, in fit
    return super(KerasClassifier, self).fit(x, y, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/keras/wrappers/scikit_learn.py", line 151, in fit
    history = self.model.fit(x, y, **fit_args)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 235, in fit
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 593, in _process_training_inputs
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 646, in _process_inputs
    x, y, sample_weight=sample_weights)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2383, in _standardize_user_data
    batch_size=batch_size)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2489, in _standardize_tensors
    y, self._feed_loss_fns, feed_output_shapes)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_utils.py", line 810, in check_loss_and_target_compatibility
    ' while using as loss `' + loss_name + '`. '
**ValueError:** A target array with shape (500, 99) was passed for an output of shape (None, 100) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.

  FitFailedWarning)

对我来说，这个错误可能是由sklearn引起的，我不确定如何摆脱它以获得最佳的参数发现。有什么办法可以解决这些问题？任何想法？
最小示例:
这只是最小的示例:

X = np.random.randn(1000, 2048)
y = np.array([i for i in range(100)]*10) # <- 1D array with target labels

def myCNN():
  model = keras.models.Sequential()
  model.add(keras.layers.Dense(100, input_dim=2048, activation='softmax'))
  model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  return model
  

model = KerasClassifier(build_fn=myCNN)
parameters = { 'epochs': [10, 20, 30],
               'batch_size':[1, 2, 3, 4, 5, 6, 7,8] }              

grid_search = GridSearchCV(estimator=model,
                           param_grid=parameters,
                           scoring='accuracy',
                           cv=2)

grid_search = grid_search.fit(X, y)
print (grid_search.best_params_)

从上面粘贴的错误消息中，我可以假定错误可能是从sklearn追溯的。我有什么要解决的方法吗？有什么想法吗？

Answer 1

我建议您可以使用来自bayes_opt的贝叶斯优化，它比GridSearchCV效率高得多，并且执行速度快。这是如何使用贝叶斯优化的简单示例:

import tensorflow as tf
from bayes_opt import BayesianOptimization

def myCNN(fully_conn_size, len_layer_conv, kernel_size, learning_rate, pooling_size, multiply,dropout_rate, beta, activation, batch_normalization, batch_size = 10):
  model = keras.models.Sequential()
  model.add(keras.layers.Dense(100, input_dim=2048, activation='softmax'))
  model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  return model


accbest = 0.0
NN_BAYESIAN = BayesianOptimization(myCNN, 
                              {'fully_conn_size': (16, 128),
                               'len_layer_conv': (3, 5),
                               'kernel_size': (2, 7),
                               'learning_rate': (0.0001, 1),
                               'pooling_size': (2, 4),
                               'multiply': (1, 3),
                               'dropout_rate': (0.1, 0.99),
                               'beta': (0.000001, 0.49),
                               'activation': (0, 2),
                               'batch_normalization': (0, 1)
                              })
NN_BAYESIAN.maximize(init_points = 20, n_iter = 40, acq = 'ei', xi = 0.0)

我想您应该为模型自定义参数，然后像上面显示的那样运行贝叶斯优化。祝您好运，如果您有任何疑问，请告诉我。