gpt4 book ai didi

python-3.x - 如何解决错误 : RuntimeError: received 0 items of ancdata

转载 作者:行者123 更新时间:2023-12-02 01:40:32 62 4
gpt4 key购买 nike

我有一个 torch.utils.data.DataLoader。我使用以下代码创建了它们。

transform_train = transforms.Compose([
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
])

trainset = CIFAR100WithIdx(root='.',
train=True,
download=True,
transform=transform_train,
rand_fraction=args.rand_fraction)

train_loader = torch.utils.data.DataLoader(trainset,
batch_size=args.batch_size,
shuffle=True,
num_workers=args.workers)

但是当我运行以下代码时出现错误。

train_loader_2 = []
for i, (inputs, target, index_dataset) in enumerate(train_loader):
train_loader_2.append((inputs, target, index_dataset))

错误是

Traceback (most recent call last):
File "main_superloss.py", line 460, in <module>
main()
File "main_superloss.py", line 456, in main
main_worker(args)
File "main_superloss.py", line 374, in main_worker
train_loader, val_loader = get_train_and_val_loader(args)
File "main_superloss.py", line 120, in get_train_and_val_loader
for i, (inputs, target, index_dataset) in enumerate(train_loader):
File "/home/C00423766/.conda/envs/dp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 804, in __next__
idx, data = self._get_data()
File "/home/C00423766/.conda/envs/dp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 771, in _get_data
success, data = self._try_get_data()
File "/home/C00423766/.conda/envs/dp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 724, in _try_get_data
data = self.data_queue.get(timeout=timeout)
File "/home/C00423766/.conda/envs/dp/lib/python3.7/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/home/C00423766/.conda/envs/dp/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 284, in rebuild_storage_fd
fd = df.detach()
File "/home/C00423766/.conda/envs/dp/lib/python3.7/multiprocessing/resource_sharer.py", line 58, in detach
return reduction.recv_handle(conn)
File "/home/C00423766/.conda/envs/dp/lib/python3.7/multiprocessing/reduction.py", line 185, in recv_handle
return recvfds(s, 1)[0]
File "/home/C00423766/.conda/envs/dp/lib/python3.7/multiprocessing/reduction.py", line 161, in recvfds
len(ancdata))
RuntimeError: received 0 items of ancdata

我想在列表中获取数据的原因是因为我想重新排序样本。而且不是以随机的方式,而是以特定的方式。我该怎么做?

最佳答案

基于一些讨论,我的代码也遇到了类似的问题(检查 #1#2#3)。我使用 ulimit -n 2048 来增加进程可以拥有的最大文件描述符数。您可以阅读有关 ulimit here 的更多信息.

关于这个问题——讨论表明它必须对 pytorch 的 fork 多处理代码做一些事情。

关于问题的第二部分,如何重新排序数据加载器 - 您可以引用此 answer

关于python-3.x - 如何解决错误 : RuntimeError: received 0 items of ancdata,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/71642653/

62 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com