python - 是否可以在 transform.compose 中使用非 pytorch 增强

Question

我正在研究将图像作为 Pytorch 中的输入的数据分类问题。我想使用 imgaug 库，但不幸的是我不断出错。这是我的代码。

#import necessary libraries
from torch import nn
from torchvision import models
import imgaug as ia
import imgaug.augmenters as iaa
from torchvision import datasets
from torch.utils.data.dataloader import DataLoader
from torchvision import transforms
from torch import optim
import numpy as np
from PIL import Image
import glob
from matplotlib import image

#preprocess images
#create data transformers
seq = iaa.Sequential([iaa.Sometimes(0.5,iaa.GaussianBlur(sigma=(0,3.0))),
                      iaa.Sometimes(0.5,iaa.LinearContrast((0.75,1.5))),
                      iaa.AdditiveGaussianNoise(loc=0,scale=(0.0,0.05*255),per_channel=0.5),
                      iaa.Sometimes(0.5,iaa.Affine(
        scale={"x": (0.8, 1.2), "y": (0.8, 1.2)},
        translate_percent={"x": (-0.2, 0.2), "y": (-0.2, 0.2)},
        rotate=(-25, 25),
        shear=(-8, 8)))],random_order=True)
        

 

train_transformation = transforms.Compose([transforms.RandomResizedCrop(300),
                                           seq,
                                           transforms.ToTensor()])

train_data = datasets.ImageFolder(root = 'train')
train_loader = DataLoader(train_data,shuffle = True,batch_size = 32,num_workers = 0)
train_iter = iter(train_loader)
train_iter.next()

Jupyter Server: local
Python 3.8.4 64-bit: Idle
CNN Cancer Detector
Melanoma
Intro
Skin cancer is the most common form of cancer, with 1 in 5 Americans developping it by the time they reach 70 years old. Over 2 people die of skin cancer in the US every hour.[1] Early detection is key in saving peoples lives with skin cancer, with the early detection 5 year survival rate being 99%[1]. Dermatologist have to look at patients one by one, and must assess by eye whether or not a blemish is malignant or benign. Dermatologist's have around a 66% accuracy rate in assessing 752 different skin diseases, while CNN's, such as the one detailed in *Dermatologist-level classification of skin cancer with deep neural networks* published in Nature have achieved greater accuracy levels then dermatologist's, around 72.1%[2].
By converting cancer detection to easily deployable software, you could allow people to get accurate cancer testing at home, saving resources and time. By making cancer detection more accesible, people would be more likely to get tested, saving lives in the process. Below I will detail my process and results from a melanoma (the most deadly form of skin cancer) detector model using CNN's.

[2]



from PIL import Image
import glob
from matplotlib import image



[3]



#preprocess images
#create data transformers
seq = iaa.Sequential([iaa.Sometimes(0.5,iaa.GaussianBlur(sigma=(0,3.0))),
                      iaa.Sometimes(0.5,iaa.LinearContrast((0.75,1.5))),
                      iaa.AdditiveGaussianNoise(loc=0,scale=(0.0,0.05*255),per_channel=0.5),
                      iaa.Sometimes(0.5,iaa.Affine(
        scale={"x": (0.8, 1.2), "y": (0.8, 1.2)},
        translate_percent={"x": (-0.2, 0.2), "y": (-0.2, 0.2)},
        rotate=(-25, 25),
        shear=(-8, 8)))],random_order=True)
…train_iter = iter(train_loader)
train_iter.next()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in 
     20 train_loader = DataLoader(train_data,shuffle = True,batch_size = 32,num_workers = 0)
     21 train_iter = iter(train_loader)
---> 22 train_iter.next()

D:\Python\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
    343 
    344     def __next__(self):
--> 345         data = self._next_data()
    346         self._num_yielded += 1
    347         if self._dataset_kind == _DatasetKind.Iterable and \

D:\Python\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
    383     def _next_data(self):
    384         index = self._next_index()  # may raise StopIteration
--> 385         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    386         if self._pin_memory:
    387             data = _utils.pin_memory.pin_memory(data)

D:\Python\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
     45         else:
     46             data = self.dataset[possibly_batched_index]
---> 47         return self.collate_fn(data)

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in default_collate(batch)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80 
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in (.0)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80 
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in default_collate(batch)
     79         return [default_collate(samples) for samples in transposed]
     80 
---> 81     raise TypeError(default_collate_err_msg_format.format(elem_type))

TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found 

我知道 imgaug 转换器的输入必须是一个 numpy 数组，但我不确定如何将其合并到我的 transform.compose 中(如果我可以的话)。当 imgaug seq 不在 transform.compose 中时，它可以正常工作。

谢谢你的帮助!

Answer 1

查看 pytorch 中的转换文档可以提示我们如何操作:https://pytorch.org/docs/stable/torchvision/transforms.html#generic-transforms

我会尝试这样的事情:

train_transformation = transforms.Compose([transforms.RandomResizedCrop(300),
                                           transforms.Lambda(lambda x: seq(x)),
                                           transforms.ToTensor()])