Terahertz (THz) spectrum is identified as a potential enabler for advanced sensing and positioning, where THz-Time domain spectroscopy (THz-TDS) is specified for investigating the unique material properties. The transmission THz-TDS measures the light absorption of materials. This paper proposes a novel low-complex deep neural network (DNN)-based multi-class classification architecture to sense a wide variety of materials from the transmission spectroscopy. Based on the spectroscopic measurements made across a chosen THz region of interest, DNN extracts and learns the distinctive crystal structure of materials as features. With sufficient quantities of noisy spectroscopic data and labels, we train and validate the model. In low SNR regions, the proposed DNN classification architecture achieves about 92%success rate, which is greater than those of the state-of-the-art methods.