Effective Index-Mapping of Quantized Values for Low-Precision Neural Networks on Power-Efficient Embedded Devices

Abstract

Neural networks are sets of algorithms that together can approximate general functions. To approximate a function, the network must first be trained by a framework that can give informed feedback to reinforce correct predictions.

As these function approximations can be trained ahead of time, neural networks are often used for work that will have previously unseen inputs such as those seen in the field of Computer Vision.

The Intel® Movidius Myriad VPU is an embedded processor that is integrated into many hand-held and battery powered devices. In Intel® Movidius s latest processor, a hardware component was included to accelerate neural network software. In this thesis, we explore a particular feature of this hardware component An index mapping of the neural network s intermediary and trained values.

We propose several new approaches to configuring this component and how they could be used to improve classification rates for very low precision networks. Of particular note, is the LeNet network where our 4 bit results match those of a 32 bit equivalent. However, we find that our proposed algorithms are suitable for different scenarios and would be best used as a suite.

Finally, we demonstrate the performance of the VPU using the hardware component to achieve 4 times lower data transfer sizes and consequentially, 4x faster processing of a layer.