machine-learning - ResNet 如何达到文档中的精度？

Question

我按照这个文档https://arxiv.org/pdf/1512.03385.pdf实现了cifar 10的ResNet但我的准确度与文档中得到的准确度有明显差异我的 - 86%个人电脑女儿 - 94%我的错误是什么？

https://github.com/slavaglaps/ResNet_cifar10

Answer 1

你的问题有点太笼统了，我的观点是网络过度拟合训练数据集，你可以看到训练损失相当低，但在第 50 纪元之后验证损失不再改善.

我没有深入阅读这篇论文，所以我不知道他们是如何解决这个问题的，但增加正则化可能会有所帮助。以下链接将为您指明正确的方向 http://cs231n.github.io/neural-networks-3/

下面我复制了文本摘要:

Summary

To train a Neural Network:

• Gradient check your implementation with a small batch of data and be aware of the pitfalls.
• As a sanity check, make sure your initial loss is reasonable, and that you can achieve 100% training accuracy on a very small portion of the data
• During training, monitor the loss, the training/validation accuracy, and if you’re feeling fancier, the magnitude of updates in relation to parameter values (it should be ~1e-3), and when dealing with ConvNets, the first-layer weights.
• The two recommended updates to use are either SGD+Nesterov Momentum or Adam.
• Decay your learning rate over the period of the training. For example, halve the learning rate after a fixed number of epochs, or whenever the validation accuracy tops off.
• Search for good hyperparameters with random search (not grid search). Stage your search from coarse (wide hyperparameter ranges, training only for 1-5 epochs), to fine (narrower rangers, training for many more epochs)
• Form model ensembles for extra performance