Cataracts are a major cause of blindness worldwide. Most cases occur in rural areas of underprivileged communities, often due to a lack of access to eye specialist services. The researchers in this study proposed an accessible and cost-effective system for early screening using computer vision based on images captured using consumer-grade cameras. The research focuses on a hybrid system that combines classical techniques in image preprocessing, such as Gaussian blur, contour recognition, and Hough Circle Transform, with state-of-the-art machine learning classifiers to detect cataracts. The study employed the VGG16 model for feature extraction, feeding the outputs to Support Vector Machines (SVM) and/or XGBoost classifiers. Alternatively, we used convolutional neural networks (CNNs) for both feature extraction and classification, eliminating the need for preprocessing. The model results were evaluated using approximately 500 optical images. The models achieved accuracies of 90.91% for XGBoost, 89.90% for SVM, and 84.85% for CNN for 10 training epochs. A small change in accuracy was observed in both the SVM (89.95%) and XGBoost (90.97%) classifiers after hyperparameter tuning using GridSearch. At 50 epochs, the CNN achieved an accuracy of 91.92%. In the first experiment, XGBoost achieved the highest accuracy rate of 90.91%, whereas in the second experiment, the CNN achieved the highest accuracy rate of 91.92%. The obtained SSIM varied with different thresholds of 0.3, 0.4, and 0.5, demonstrating the ability of the system to accurately distinguish cataract-affected eyes from normal eyes. The findings show that using XGBoost will suit our proposed cataract detection system, which has the potential to screen for cataracts in rural, semi-urban, and low-resource communities, where eye specialist facilities are not accessible. We hope that this research will pave the way for the use of computer vision in ophthalmology, potentially alleviating the agony of cataract-induced preventable blindness by promptly detecting it in remote areas.