Deep Neural Networks (DNNs) are increasingly being used in a variety of applications. However, DNNs have huge computational and memory requirements. One way to reduce these requirements is to sparsify DNNs by using smoothed LASSO (Least Absolute Shrinkage and Selection Operator) functions. In this paper, we show that for the same maximum error with respect to the LASSO function, the sparsity values obtained using various smoothed LASSO functions are similar. We also propose a layer-wise DNN pruning algorithm, where the layers are pruned based on their individual allocated accuracy loss budget determined by estimates of the reduction in number of multiply-accumulate operations (in convolutional layers) and weights (in fully connected layers). Further, the structured LASSO variants in both convolutional and fully connected layers are explored within the smoothed LASSO framework and the tradeoffs involved are discussed. The efficacy of proposed algorithm in enhancing the sparsity within the allowed degradation in DNN accuracy and results obtained on structured LASSO variants are shown on MNIST, SVHN, CIFAR-10, and Imagenette datasets.

The results presented in the paper are based on simulations on benchmark spiking neural networks. The methodology is described in the paper.

Approximate circuit design has gained significance in recent years targeting error tolerant applications. In this paper, we consider the problem of minimizing the power for a given accuracy, in a signal processing application with accurate adders replaced by low-power approximate adders. We first demonstrate that the commonly used assumption that the inputs to the adder are uniformly distributed results in an inaccurate prediction of error statistics for multi-level circuits. To overcome this problem, we propose the use of parameterized error models for adders, with input static probabilities as parameters. The static probability computation in our work considers not just the functionality of the adder but also its position in the circuit, functionality of its parents and the number of approximate bits in the parent blocks. This parameterized error model can be incorporated in any optimization framework. We demonstrate up to 6.5 dB improvement in the accuracy of noise power prediction when the proposed model is used to optimize an 8x8 DCT.