Artificial Neural Networks and Convolutional Neural Networks have become common tools for classification and object detection, owing to their ability to learn features without prior knowledge. During training, these networks learn the parameters, weights, and biases. This paper proposes a simple Neural Network and Convolutional Neural Network (CNN) for a classification task. Furthermore, the Bayesian neural network work is reproduced as a baseline for comparing my proposed networks. All experiments were conducted using the MNIST dataset. While the simple neural networks and the convolutional networks adjust their parameters based on the cost function during training, the Bayesian convolutional neural network updates its parameters based on the backdrop that drives a variational approximation to the true posterior. Hyperparameters such as optimizer, learning rate, regularizers, dropout, epochs, etc., were varied to train the two proposed networks. The proposed networks achieved better classification accuracy, approximately 99\%, than the previously implemented Bayesian convolutional neural network. However, it is difficult to predict the certainty of the predictions made by my proposed networks, unlike Bayesian learning, which makes it easy to do so. \href{https://github.com/Simeon340703/Classification_Networks}{You can find the code for this work at}.