Drivers’ drowsiness has been considered one of the prime reasons for accidents and road fatalities, causing a detrimental effect on human lives. This article aims to propose attention deep learning framework employing a “you only look once” version 3 (YOLOv3) network for drivers’ drowsiness detection based on a convolutional neural network module. An attention YOLOv3 framework has been modeled for detecting drivers’ eye regions followed by eye state classification and interpretation. A low-power consumer imaging system based on OV5647, 1080P Infrared-Cut camera embedded in a Raspberry Pi has been employed for capturing both day and night mode images. Feature extraction has been carried out via Darknet-53 module followed by a detection module comprising upsampling layers and multiple convolutional operations. Finally, multiscale fusion along with the non-maximum suppression method has been applied to detect and classify the eye region. Moreover, the eye region has been interpreted via a classification activation map using the proposed attention module. Experimental evaluations reveal the efficacy of the proposed framework on our acquired dataset and benchmark datasets. The proposed object detection and classification module is a generic one that can possess good potential by increasing safety in an intelligent transportation system.