Large-scale classification poses great challenges to  storage and computation. There are two major solutions to address the problem:  data compression and  quantization. In the paper, we study the method of first reducing data dimension by random projection and then quantizing the projections to ternary and binary codes, which has been widely applied in practice. Often, the extreme quantization would  degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that  the quantization could result in   performance improvement, rather than degradation, if the data for quantization are preprocessed by sparse transform.  Also,  the quantization gain could be obtained with the random projections of the data, if both the data and random projection matrices are  sparse enough, such that the resulting projections remain sparse.  The intriguing  performance is verified and analyzed  with extensive experiments.

Recently, it has been observed that $\{0,\pm1\}$-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform $\{-1, 1\}$-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method, and then generate ternary codes. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the proposed joint learning indeed could produce better ternary codes.

Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The target propagation is an ill-posed problem, due to involving the inversion of nonlinear activations from low-dimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it leads to higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network’s each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.