Principal Component Analysis (PCA) is a widely adopted technique for dimensionality reduction and data exploration. In this paper we have implemented from scratch and applied to both a synthetically generated random dataset and the Abalone dataset to evaluate its effectiveness. PCA, a vital dimensionality reduction technique, simplifies complex datasets, enhances interpretability, and improves computational efficiency while preserving key patterns. The study involves constructing a random dataset to validate the robustness of the custom PCA algorithm via eigenvalue decomposition of the covariance matrix. This approach is then extended to the Abalone dataset to extract principal components, analyze variance contributions, and visualize the data in reduced dimensions. Results demonstrate that the handcrafted PCA algorithm effectively captures the underlying structure of the data, providing critical insights into feature importance and enabling efficient data visualization. This dual application highlights the versatility and efficacy of PCA in handling diverse datasets, confirming its utility in various data analysis scenarios.

The k-Nearest Neighbors (kNN) algorithm is a widely-utilized nonparametric and lazy learning technique commonly applied in classification tasks. Its simplicity and effectiveness make it an attractive option for various applications, including remote sensing. This project focuses on using kNN to classify the central pixel in a 3x3 neighborhood within a multi-spectral Landsat satellite image. Each pixel in the dataset is represented by four spectral bands, and the goal is to predict the class of the central pixel based on its neighbors' spectral values. This study provides an overview of the kNN algorithm, detailing its core concepts, operational principles, and applications specific to satellite imagery classification. We explore the influence of the parameter k, the choice of distance metric, and the impact of different weighting schemes on the algorithm's performance. By comparing kNN's effectiveness with traditional statistical methods, this project aims to demonstrate the potential of kNN in handling diverse remote sensing data types and enhancing pixel classification accuracy.

Decision trees are popular and highly interpretable machine learning algorithms used for both classification and regression tasks. These trees are structured with internal nodes representing features and leaf nodes indicating class labels or predictions. They are constructed through recursive partitioning based on the most informative features, employing criteria such as the Gini Index or Information Gain. Decision trees are appreciated for their interpretability, ease of use, and ability to handle both categorical and numerical features. They are robust against outliers and can effectively capture non-linear relationships. In this study, we explore the capabilities of decision trees by applying them to the MAGIC Gamma Telescope dataset to classify particles as gamma or hadron. By constructing a decision tree model, we aim to uncover the key factors and predictors for particle classification. The process includes data preprocessing, model construction, and evaluation to extract valuable insights and validate the model's predictive accuracy.

This lab report presents our work on building artificial neural networks (ANNs) from scratch and evaluating their performance on the MNIST and Kuzushiji-MNIST datasets. We experimented with different configurations to assess their impact on model accuracy, including random initialization, Kaiming/Xavier initialization, dropout regularization, and combinations of these methods. The highest accuracy of 90.611% for the MNIST dataset was achieved with a randomly initialized 5-layered network, while the Kuzushiji-MNIST dataset attained the best performance with Kaiming/Xavier initialization at 90.386%. Incorporating regularization techniques generally resulted in lower performance, underscoring the complexity of optimizing ANNs for specific datasets. These results offer valuable insights into the selection and tuning of neural network configurations for enhanced accuracy.

We explored Naïve Bayes classification methods on the Page Blocks Classification Dataset, which categorizes page blocks into text, horizontal line, graphic, vertical line, and picture. Utilizing both Gaussian Naïve Bayes and Categorical Naïve Bayes classifiers, we evaluated their performance on this multi-class classification problem. The dataset, with 5473 instances and 10 attributes, presents a practical challenge for distinguishing various page block types in document images. We compared the accuracy, precision, recall, and F1-score of the classifiers across various scenarios. Results show that the Gaussian Naïve Bayes classifier slightly outperforms the Categorical Naïve Bayes in accuracy and F1-score, likely due to the continuous nature of the dataset's attributes. Additionally, the Gaussian classifier demonstrates faster processing times. Our findings suggest that while both models are viable, the Gaussian Naïve Bayes classifier may be more suitable for similar datasets. Future research directions include integrating feature selection techniques and exploring hybrid models.