It is commonly accepted that the Vision Transformer model requires sophisticated regularization techniques to excel at ImageNet-1k scale data. Surprisingly, we ﬁnd this is not the case and standard data augmentation is sufﬁcient. This note presents a few minor modiﬁcations to the original Vision Transformer (ViT) vanilla training setting that dramatically improve the performance of plain ViT models. Notably, 90 epochs of training surpass 76% top-1 accuracy in under seven hours on a TPUv3-8, similar to the classic ResNet50 baseline, and 300 epochs of training reach 80% in less than one day. 2022: L. Beyer, Xiaohua Zhai, Alexander Kolesnikov https://arxiv.org/pdf/2205.01580v1.pdf
To leave or reply to comments, please download free Podbean or
To leave or reply to comments,
please download free Podbean App.