This paper introduces the Subspace Rotation Algorithm (SRA) for training the Restricted Hopfield Network (RHN) as an auto-associative memory. SRA is a gradient-free subspace tracking method based on Singular Value Decomposition (SVD) to update the weight matrix. Despite having slightly worse time complexity than Back-propagation (BP) theoretically, in practice, SRA completes training faster since it requires fewer iterations to converge. Comparative analysis with BP for training RHN reveals that SRA consistently reaches the optimal solution, whereas BP fails to achieve comparable performance if the weight initialization is not within the appropriate basin of attraction. Experiments involving the memorization of 10, 50, and 100 patterns from the MNIST dataset show that RHN trained with SRA exhibits better robustness to noisy and corrupted patterns compared to RHN trained with BP. These findings suggest that SRA offers a more reliable and effective method for training RHNs in applications needing high tolerance to input distortions.

Traditionally, time series data augmentation has primarily focused on improving the architecture of Generative Adversarial Network (GAN), with the aim of closely matching the original data distribution while also preserving the dynamic behavior of the original data. However, even state-of-the-art GAN models like TimeGAN fall short in preserving the temporal dynamics present in the original time series due to the absence of first-order difference information. To address this limitation, this study proposes a novel process for generating multivariate time series data. The proposed process comprises four essential modules: a) the GAN module for generating multivariate time series data, b) the sampling module for preserving the first-order difference distribution, c) the smoothing module for refining the generated data, and d) an evaluation module using the Kolmogorov-Smirnov Test (KS-test) and Hilbert-Schmidt Independence Criterion (HSIC), along with other metrics to test the synthetic time series data. This comprehensive approach ensures that the synthetic time series data maintains both the distribution and the dynamic behavior of the original data. We extensively discuss the role of the β factor in the modified Metropolis-Hastings algorithm (in the sampling module), which controls the level of information preservation from the original time series. Our experiments reveal that with small β values, periodic information can be retained effectively. The joint distribution of the first-order difference of the synthetic time series data remains consistent when the same β value is applied in the modified Metropolis-Hastings algorithm. However, we observe that β has no impact on the partial autocorrelation functions. Nevertheless, the generated data from the sampling module maintains the memoryless property of the Markov Chain. Therefore, in the smoothing module, we apply the exponential moving average (EMA) method to simulate the long-term relationships within the original time series, and find that an optimal α value is approximately 0.4 or 0.5. Lastly, we employ the synthetic time series data to train a neural network model developed in another work. Our findings indicate that the neural network model trained on synthetic time series data exhibits performance comparable to that of a model trained on the original data.

This paper introduces the QLSTM-DLEM model, a quantum model combining Quantum Long Short Term Memory (QLSTM) and Dynamic Land Ecosystem Model (DLEM), for predicting nitrous oxide (N2O) emissions in agricultural fields. Given the time series data collected from different bifurcation branches, integrating all the data into a single model becomes challenging. To address this, time series data, accompanied by signatures, is utilized to distinguish data from different branches. The auto-correlation function (ACF), partial auto-correlation function (PACF), data distribution, and joint distribution of the first-order difference illustrate that time series data collected under different initial conditions and parameters exhibit slight differences but share several similar characteristics. Performance evaluation metrics, including mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination (R 2), are employed to assess the model. The results indicate that the QLSTM-DLEM model outperforms the classical LSTM-DLEM model in terms of generalization and stability, possibly attributed to the parallelism and superposition of quantum computing. Overall, our simulation results demonstrate that the proposed QLSTM-DLEM hybrid model can effectively predict N2O emissions in agricultural fields, with potential applications in environmental monitoring and management.

Both the Restricted Hopfield Network (RHN) and Dense Associative Memory (DAM) are derived from the classical Hopfield Neural Network (HNN), but they improve upon HNN using different mechanisms. This paper presents a comparative analysis of RHN and DAM with respect to storage capacity, time complexity, training efficiency, and retrieval from incomplete or noisy patterns. The analysis reveals that RHN and DAM use different mechanisms for storing patterns. RHN, trained using the Subspace Rotation Algorithm (SRA), exhibits better time complexity than DAM, which is trained using the Energy-based Algorithm (EBA). Furthermore, in practice, RHN requires significantly less time than DAM to memorize the same number of patterns, as DAM needs more epochs to converge. The experimental results demonstrate that when storing 1,000 character patterns (825 bits), RHN with two hidden layers outperforms DAM in terms of retrieval from noisy and incomplete patterns. When memorizing 10, 50, and 100 patterns from both the MNIST and Fashion MNIST datasets, RHN shows a slight advantage over DAM in retrieval from incomplete and noisy patterns. However, the performance of both models degrades as the number of hidden nodes or stored patterns increases.  The link to the source code: https://github.com/Developer2046/rhn_dam.

This study introduces the Subspace Rotation Algorithm (SRA), an innovative gradientfree method designed to discover the global optimal weight matrix. The SRA consists of two fundamental algorithms: the Left Subspace Rotation Algorithm (LSRA) and the Right Subspace Rotation Algorithm (RSRA). The combination of LSRA and RSRA, in two formats, LSRA-RSRA and RSRA-LSRA, can harness the advantages of both individual algorithms, resulting in enhanced performance. Our observations reveal that shallow and wide Multilayer Perceptrons (MLP), trained using RSRA-LSRA, achieve higher training accuracy compared to the Backpropagation (BP) algorithm. Moreover, when combining SRA with the BP algorithm, a remarkable impact on training MLP models is observed. Our experiments demonstrate that the BP algorithm may become trapped in local optima, while RSRA-LSRA, though capable of escaping local optima, may not fully realize its potential when the number of hidden nodes is limited. The synergy of RSRA-LSRA and the BP algorithm allows for the utilization of the advantages from both approaches, achieving optimal MLP performance with fewer hidden nodes.

This paper introduces Conjugate Neural Networks (CoNNs), a novel architecture designed to enhance robustness against adversarial attacks. Adversarial attacks pose significant security challenges to neural networks by manipulating input data to cause incorrect model predictions. The proposed CoNNs architecture leverages a dual-model framework where two complementary neural networks are trained in tandem, each using adversarial samples generated by the other, to cancel out adversarial effects and improve overall resilience. We evaluate CoNNs using various deep learning models, including MLP, ResNet, LSTM, and Agricultural-Informed Neural Networks (AINN), across multiple datasets such as Fashion-MNIST, CIFAR10, the data generated by Lorenz system, and agricultural N 2 O emissions data. The results demonstrate that CoNNs significantly outperform traditional single neural network in resisting both Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM) attacks. CoNNs maintain higher accuracy and exhibit greater stability under adversarial conditions, making them a promising approach for improving the robustness of neural network-based systems.