keras - 自定义 ImageDataGenerator keras

Question

我一直在尝试实现 Keras 自定义图像数据生成器，以便我可以进行头发和显微镜图像增强。

这是数据生成器类:

class DataGenerator( Sequence ):

    def __init__(self,image_paths,labels, augmentations, batch_size=32, image_dimension=(224,224,3), shuffle=False):
        self.image_paths = image_paths
        self.labels = labels
        self.batch_size = batch_size
        self.image_dimension = image_dimension
        self.shuffle = shuffle
        self.augment = augmentations

    def __len__(self):
        return int(np.ceil(len(self.image_paths) / self.batch_size ))

    def _getitem__(self,index):
        indexes = self.indexes[index*self.batch_size : (index+1)*self.batch_size]
        batch_y = np.array([self.labels[k] for k in indexes])
        batch_x = [cv2.cvtColor(cv2.imread(self.image_paths[k]), cv2.COLOR_RGB2BGR) for k in indexes]

        return np.stack([
            self.augment(image=x)["image"] for x in batch_x
        ], axis=0), np.array(batch_y)

下面的代码用于 albumentations 增强(只是尝试 albualbumentations 增强来测试数据生成器是否工作):

AUGMENTATIONS_TRAIN = Compose([
    HorizontalFlip(p=0.5),
    RandomContrast(limit=0.2, p=0.5),
    RandomGamma(gamma_limit=(80, 120), p=0.5),
    RandomBrightness(limit=0.2, p=0.5),
    HueSaturationValue(hue_shift_limit=5, sat_shift_limit=20,
                       val_shift_limit=10, p=.9),
    # CLAHE(p=1.0, clip_limit=2.0),
    ShiftScaleRotate(
        shift_limit=0.0625, scale_limit=0.1, 
        rotate_limit=15, border_mode=cv2.BORDER_REFLECT_101, p=0.8), 
    ToFloat(max_value=255)
])

    AUGMENTATIONS_TEST = Compose([
    # CLAHE(p=1.0, clip_limit=2.0),
    ToFloat(max_value=255)
])

现在创建 DataGenerator 对象:

train_datagen = DataGenerator(  train['images'],
                                train['target'],
                                augmentations=AUGMENTATIONS_TRAIN,
                                batch_size=32,
                                image_dimension=(224,224,3) )
val_datagen = DataGenerator(   validation['images'],
                               validation['target'],
                               augmentations=AUGMENTATIONS_TEST,
                               batch_size=16,
                               image_dimension=(224,224,3) )`

当我出现 NonImplementedError运行 model.fit_generator(generator=train_datagen,steps_per_epoch=30,epochs = 30,validation_data=val_datagen,validation_steps=15)

我已经分享了我的内核 here和我正在从 here 寻求帮助.我也一直在寻找其他增强方式，它们都是一样的。

如果有人能说出问题的原因和位置，我将不胜感激？还有在 keras 中进行自定义图像增强的任何其他好方法。

Answer 1

你可以看看 imgaug 库。 albumentations 和 imgaug 几乎相同。编写操作序列，然后将其放入 Imagedatagenerator preprocessing_function 中。我尝试使用 albumentations 库，但遇到了一些错误。

from imgaug import augmenters as iaa

seq = iaa.Sequential([
    iaa.Fliplr(0.5), # horizontally flip
    # sometimes(iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05), per_channel=0.5)),
    iaa.OneOf([
        iaa.Sharpen(alpha=(0, 1.0), lightness=(0.75, 1.5)),
        iaa.Emboss(alpha=(0, 1.0), strength=(0, 2.0)),
        # iaa.Noop(),
        iaa.GaussianBlur(sigma=(0.0, 1.0)),
        # iaa.Noop(),
        iaa.Affine(rotate=(-10, 10), translate_percent={"x": (-0.25, 0.25)}, mode='symmetric', cval=(0)),
        # iaa.Noop(),
        # iaa.PerspectiveTransform(scale=(0.04, 0.08)),
        # # iaa.Noop(),
        # iaa.PiecewiseAffine(scale=(0.05, 0.1), mode='edge', cval=(0)),
        
    ]),
    sometimes(iaa.ElasticTransformation(alpha=(0.5, 3.5), sigma=0.25)),
    # More as you want ...
], random_order=True)

datagen = ImageDataGenerator(preprocessing_function=seq.augment_image)

有一些高级数据增强实践，例如剪切、随机删除和混合。它们很容易在 Keras 中实现。对于 mixup，示例如下:

training_generator = MixupGenerator(trainX, trainY, batch_size=8, alpha=0.2, datagen=datagen)()
x, y = next(training_generator)

# To visualize the batch images
for i in range(9):
    plt.subplot(330+1+i)
    # batch = it.next()
    img = x[i]
    plt.imshow(img.reshape(224, 224, 3))
plt.savefig("mixup_batch.png")

H = model.fit_generator(
    # datagen.flow(trainX, trainY, batch_size=args.batch_size),
    training_generator,
    steps_per_epoch=len(trainX) // args.batch_size,
    validation_data=(valX, valY),
    validation_steps=len(valX) // args.batch_size,
    epochs=args.epochs,
    # workers=4,
    callbacks=[model_checkpoint, lr_reducer, stopping, lr_schedule],
)

不过，我在这方面遇到的问题是，对于随机删除，我们需要将其放入 ImageDataGenerator preprocessing_function 中，并且我们已经将 imgaug 增强放入其中。可能的替代方案是使用两个数据生成器。