In this paper, we consider strategies to label mental fatigue states during a prolonged visuospatial working memory exercise. Specifically, we look to address the need to accurately forecast these states with unobtrusive measures derived from cardiac electrical activity. We formulate this challenge as a multi-step-ahead, multivariate time-series forecasting problem, where we consider the effect of annotation variations for fatigue labels on model accuracy and reliability. We found that a sequence-to-sequence (Seq2Seq) LSTM architecture was effective in identifying fatigue representations using heart rate and its variability (HR/V). Using this architecture we analyzed a heuristic-based annotation approach for fatigue that relies on changing patterns of HR/V as a signal for state changes in the human. We contrast perception-, performance- and heuristic-based labels across the forecast horizon, the input vector horizon, and sample generalizability. Our findings indicate that, under controlled conditions, HR/V can serve as a viable neurocognitive index of fatigue, however, the impact of activity, time scale, and other perturbations demand further investigation to evaluate the diagnosticity and robustness of the proposed indices.