Thursday Jul 14, 2022
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Transformers have quickly shined in the computer vision world since the emergence of Vision Transformers (ViTs). The dominant role of convolutional neural networks (CNNs) seems to be challenged by increasingly effective transformer-based models. Very recently, a couple of advanced convolutional models strike back with large kernels motivated by the local but large attention mechanism, showing appealing performance and efficiency. We propose Sparse Large Kernel Network ( SLaK ), a pure CNN architecture equipped with 51 × 51 kernels that can perform on par with or better than state-of-the-art hierarchical Transformers and modern ConvNet architectures like ConvNeXt and RepLKNet, on ImageNet classification as well as typical downstream tasks. 2022: S. Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Q. Xiao, Boqian Wu, Mykola Pechenizkiy, D. Mocanu, Zhangyang Wang https://arxiv.org/pdf/2207.03620v1.pdf
Comments (0)
To leave or reply to comments, please download free Podbean or
No Comments
To leave or reply to comments,
please download free Podbean App.