Tsukuba, Japan—Remarkably advancements are being made in the realm of convolutional neural networks (CNNs), which are pivotal in applications such as facial recognition at airport immigration and object detection in autonomous vehicles. CNNs are composed of convolutional and fully connected layers; the former simulates human vision, while the latter enables the brain to deduce the type of image from visual data. By reducing the number of data bits used in computations, CNNs can maintain recognition accuracy while substantially reducing computational demands. This efficiency allows the supporting hardware to be more compact.
Three reduction methods have been identified so far: network slimming (NS) to minimize the visual components, deep compression (DC) to reduce the neuronal components, and integer quantization (IQ) to decrease the number of bits used. Previously, there was no definitive guideline on the order of implementation or allocation of these methods. The current study establishes that the optimal sequence of these methods for minimizing the data amount is IQ, followed by NS and DC. Moreover, the researchers have created an algorithm that determines the application ratio of each method autonomously, removing the necessity for trial and error. This algorithm enables a CNN to be compressed to 28 times smaller and 76 times faster than previous models.
The implications of this research are poised to transform AI image recognition technology by dramatically reducing computational complexity, power consumption, and the size of AI semiconductor devices. This breakthrough will likely enhance the widespread feasibility of deploying advanced AI systems.
###
This work was supported in part by JST SPRING under Grant JPMJSP2124, in part by JST PRESTO under Grant JPMJPR203A, and in part by JST AIP Acceleration Research under Grant JPMJCR24U4.
Title of original paper:
Heuristic Compression Method for CNN Model applying Quantization to a Combination of Structured and Unstructured Pruning Techniques
Journal:
IEEE Access
DOI:
10.1109/ACCESS.2024.3399541
Associate Professor YAMAGIWA, Shinichi
Institute of Systems and Information Engineering, University of Tsukuba
Institute of Systems and Information Engineering
IEEE Access
Heuristic Compression Method for CNN Model Applying Quantization to a Combination of Structured and Unstructured Pruning Techniques
9-May-2024