Keras:使用 flow_from_directory 的 fit_generator 的多个输入和多个输出

Question

多任务学习模型接受三个输入。我正在使用 keras 数据生成器。是否可以将三个数据生成器传递给 model.fit_generator 函数？

问题定义

我正在解决分类问题。我使用的数据集是 Painters by number，这是由 kaggle 主办的竞赛。任务是识别给定绘画的画家、风格和流派。

我开发了单独的模型来执行每项任务。现在，我想结合多任务学习，看看它是否优于单个模型。

 Model                       No of classes (Softmax)
------                     ------------------------                                   
 Model predicting painter         8
 given paintings

 Model predicting style           10
 given paintings      


 Model predicting genre           23
 given paintings    

上表详细介绍了各个模型以及每个模型的输出类数量。

现在，我想做多任务学习，所以我想出了下面的简单架构 Multi Task Learning Architecture

 style   = Input(shape=(64,64,3))
 genre   = Input(shape=(64,64,3))
 painter = Input(shape=(64,64,3))


 shared_conv = Convolution2D(
            filters = 5,# 5 feature maps
            kernel_size = (5,5),
            strides = 1) 

 shared_conv_layer_A = shared_conv(style)
 shared_conv_layer_B = shared_conv(genre)
 shared_conv_layer_C = shared_conv(painter)

 merged_layer = keras.layers.concatenate([shared_conv_layer_A,shared_conv_layer_B,shared_conv_layer_C],axis=-1)

 pooling = MaxPooling2D(
        pool_size = (2,2),
        strides = 2
      )(merged_layer)

 dense = Flatten()(pooling)

 out_style = Dense(
        no_classes_style, 
        kernel_initializer=glorot_normal(seed=seed_val), 
        bias_initializer = 'zero', 
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)

 out_genre = Dense(
        no_classes_genre, 
        kernel_initializer=glorot_normal(seed=seed_val), 
        bias_initializer = 'zero', 
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)

 out_painter = Dense(
        no_classes_painter, 
        kernel_initializer=glorot_normal(seed=seed_val), 
        bias_initializer = 'zero', 
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)


 multi_tasking_model = Model(inputs=[style,genre,painter],outputs=[out_style,out_genre,out_painter])
 multi_tasking_model.summary()


 multi_tasking_model.compile(
   loss = 'categorical_crossentropy', 
   optimizer=Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=0.00000001 ),
   metrics=['accuracy'] 
 )

Now i want to pass three keras image data generators. So, i came up with a custom data generator

 def create_data_generator(style_generator,genre_generator,painter_generator):
    # Input
    _style_generator = style_generator[0]
    _genre_generator = genre_generator[0]
    _painter_generator = painter_generator[0]

   # Label
   _lstyle_generator = style_generator[1]
   _lgenre_generator = genre_generator[1]
   _lpainter_generator = painter_generator[1]

return [_style_generator,_genre_generator,_painter_generator],[_lstyle_generator,_genre_generator,_painter_generator]

train_mulitle_data_generator = create_data_generator(trainStyleDataGenerator,trainGenreDataGenerator,trainPainterDataGenerator) 
valid_mulitle_data_generator = create_data_generator(validationStyleDataGenerator,validationGenreDataGenerator,validationPainterDataGenerator)




history = multi_tasking_model.fit_generator(
     generator = train_mulitle_data_generator,
     steps_per_epoch= len(train_mulitle_data_generator),
     epochs = no_epoch,
     validation_data = valid_mulitle_data_generator,
 )

The error i encountered

   'tuple' object has no attribute 'ndim'

Is there any alternative way to pass multiple inputs and multiple outputs. Any suggestions or tips would be greatly helpful please ?.

Answer 1

目前create_data_generator没有定义生成器。试试这个:

def create_data_generator(style_generator,genre_generator,painter_generator):

    while(True):
        _style_generator, _lstyle_generator = next(style_generator)
        _genre_generator, _lgenre_generator = next(genre_generator)
        _painter_generator, _lpainter_generator = next(painter_generator)

        yield [_style_generator,_genre_generator,_painter_generator], [_lstyle_generator,_genre_generator,_painter_generator]