An accurate estimation of muscle fatigue is critical for adaptive control of existing assistive devices, such as an exoskeleton, prosthesis, and functional electrical stimulation (FES)-based neuroprostheses. However, the estimation of muscle fatigue using surface electromyography (sEMG) for a long duration of time becomes challenging due to loosening of sEMG sensors, sweating, and other accidental failures. These problems can be potentially solved by forecasting future sEMG signals using initially recorded high-quality data points. For the first time, we attempt to forecast the fatigue-induced electromyography signal using the initial sEMG recorded for a shorter interval of time, during biceps curl with weights of 1 kg, 2 kg, 3 kg, and 4 kg. An attention-based deep CNN-BiLSTM neural network model that captures input sEMG dynamics to forecast future sEMG signals corresponding to fatigue state was trained and tested. An average mean absolute percentage error (MAPE) of 26.7% between forecasted and recorded sEMG was observed across eight subjects, five muscles, and four weights. In addition, the time domain features like integrated EMG (IEMG), root-mean-square (RMS) value, and variance of EMG (VEMG) were compared between forecasted and recorded sEMG (fatigue state), which yielded an average MAPE of 8%, 19.2%, and 31.7%, across eight subjects, five muscles, and four weights, for (IEMG and MAV), RMS, and (VEMG and SSI) respectively. The results encourage combining the proposed approach with wearable technology for forecasting fatigue-induced sEMG to drive stimulation devices like FES and robotic devices.