A Review of FPGA-Based Custom Computing Architecture for Convolutional Neural Network Inference

Convolutional neural network (CNN) has been widely adopted in many tasks. Its inference process is usually applied on edge devices where the computing resources and power consumption are limited. At present, the performance of general processors cannot meet the requirement for CNN models with high computation complexity and large number of parameters. Field-programmable gate array (FPGA)-based custom computing architecture is a promising solution to further enhance the CNN inference performance. The software/hardware co-design can effectively reduce the computing overhead, and improve the inference performance while ensuring accuracy. In this paper, the mainstream methods of CNN structure design, hardware-oriented model compression and FPGA-based custom architecture design are summarized, and the improvement of CNN inference performance is demonstrated through an example. Challenges and possible research directions in the future are concluded to foster research efforts in this domain.