Empirical Study of Extreme Overfitting Points of Neural Networks
- 作者: Merkulov D.1,2, Oseledets I.1,3
-
隶属关系:
- Skolkovo Institute of Science and Technology, Center for Computational and Data-Intensive Science and Engineering
- Moscow Institute of Physics and Technology
- Institute of Numerical Mathematics, Russian Academy of Sciences
- 期: 卷 64, 编号 12 (2019)
- 页面: 1527-1534
- 栏目: Artificial Intellect
- URL: https://journals.rcsi.science/1064-2269/article/view/201762
- DOI: https://doi.org/10.1134/S1064226919120118
- ID: 201762
如何引用文章
详细
In this paper we propose a method of obtaining points of extreme overfitting—parameters of modern neural networks, at which they demonstrate close to 100% training accuracy, simultaneously with almost zero accuracy on the test sample. Despite the widespread opinion that the overwhelming majority of critical points of the loss function of a neural network have equally good generalizing ability, such points have a huge generalization error. The paper studies the properties of such points and their location on the surface of the loss function of modern neural networks.
作者简介
D. Merkulov
Skolkovo Institute of Science and Technology, Center for Computational and Data-Intensive Science and Engineering; Moscow Institute of Physics and Technology
编辑信件的主要联系方式.
Email: daniil.merkulov@skolkovotech.ru
俄罗斯联邦, Moscow; Moscow
I. Oseledets
Skolkovo Institute of Science and Technology, Center for Computational and Data-Intensive Science and Engineering; Institute of Numerical Mathematics, Russian Academy of Sciences
编辑信件的主要联系方式.
Email: i.oseledets@skoltech.ru
俄罗斯联邦, Moscow; Moscow