A survey on image classification of lightweight convolutional neural network
- Authors: Liu, Ying , Xiao, Peng , Fang, Jie , Zhang, Dengsheng
- Date: 2023
- Type: Text , Conference paper
- Relation: 19th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2023, Harbin, China, 29-31 July 2023, 2023 19th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)
- Full Text: false
- Reviewed:
- Description: In recent years, deep neural networks have achieved tremendous success in image classification in both academic and industrial settings. However, the high hardware requirements imposed by their intensive and complex computations pose a challenge for deployment on low-storage devices. To address this challenge, lightweight networks provide a viable solution. This paper provides a detailed review of recent lightweight image classification algorithms, which can be categorized into low-redundancy network model design and neural network compression algorithms. The former reduces network computations by replacing traditional convolution with efficient lightweight convolution, while the latter reduces redundancy in the network by employing methods such as network pruning, knowledge distillation, and parameter quantization. We summarize the experimental results of some classical models and algorithms on ImageNet2012 and CIFAR-10 datasets, and analyze the characteristics, advantages and disadvantages of these models respectively. Finally, future research directions for lightweight algorithms in the field of image classification are identified. © 2023 IEEE.
Fine-grained image classification based on knowledge distillation
- Authors: Liu, Ying , Feng, Hao , Zhang, Weidong , Fang, Jie , Xiao, Peng , Zhang, Dengsheng
- Date: 2023
- Type: Text , Conference paper
- Relation: 19th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2023, Harbin, China, 29-31 July 2023, 2023 19th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)
- Full Text: false
- Reviewed:
- Description: Despite the outstanding performance of deep learning-based fine-grained image classification methods, the commonly used models still suffer from high cost of computation and memory Therefore, this paper proposes a mobile-based CNN network that focuses on discriminative features of fine-grained images by embedding a hybrid-domain attention module to achieve higher accuracy in recognition. Specifically, under the premise of reducing network parameters, this paper presents a classification method that combines transfer learning and knowledge distillation to enhance the model's generalization performance and resistance to overfitting. Different knowledge transfer strategies are validated through the experiments in the knowledge distillation process. Mobile models such as SqueezeNet, MobileNetV2, and CBAM MobileNetV2 all demonstrate enhanced performance the knowledge distillation optimization. The proposed method in this paper can be used to develop a lightweight mobile-based CNN model with comparable performance to complex models making it more advantageous in real-life scenarios with limited storage resources and low hardware computation levels. Additionally, the model compression process utilizes only the intermediate features of the original dataset, meeting the confidentiality requirements of the original data in the field of public security. © 2023 IEEE.