2.6 Specification curve analysis
To further explore whether the predictability between language control and executive control influenced behavioral performance, we conducted the specification curve analysis (SCA, Simonsohn et al., 2020). The SCA is a framework of modeling all possible specifications (consisting of independent, dependent and control variables) to provide an unbiased brain-behavior relationship to test all reasonable specifications. Recent studies demonstrated its benefits lie in reducing the overall false-positive rate and strengthening the robustness of a given finding (Cosme & Lopez 2020; Flournoy et al. 2020; Yuan et al., 2023). In the current study, we aimed to reveal the relationship between brain activity and corresponding behavioral performance in the flanker task. Furthermore, we tentatively examined the probability of the predicted values derived from the RR model would interact with behavioral performance.
This analysis was performed with the specr package (Masur & Scharkow, 2020) in R. First, the mean RTs of congruent and incongruent trials across three switching contexts were separately specified in the flanker task as the outcome variables. Then, the mean EEG amplitudes of each flanker condition in three switching contexts were extracted within a 0-800ms time window (the real values). Additionally, using the RR prediction model, the weights of the prediction model were multiplied with the independent variable X (i.e., amplitudes during the picture-naming trials). This calculation yielded the predicted values (the predicted amplitude of the flanker task by the model) in each switching context. Both the real and predicted values from three contexts (each model for congruent and incongruent conditions contained 6 values.) were then input into the SCA as the neural predictors to examine the corresponding brain-behavior relationships and how brain-to-brain predictions interact with behavioral performance (i.e., significant interactions between the predicted values and RTs). In addition, participants’ demographics (i.e., age and gender), language proficiency (L1 self-rating scores, L2 self-rating scores, OPT scores and CET-4 scores), and the L2 age of acquisition were added as control variables. In each specification, a single indicator (i.e., each value) was set as the predict of interest, and its association with the outcome variable was examined with the changes of specifications and covariables included in the model. The standardized regression coefficients for each predictor were subsequently calculated and ordered by the effect size to plot the specification curve. Finally, the median standardized regression coefficient was tested with the proportion of the statistically significant positive and negative coefficients. The regression coefficients were statistically inferred through a bootstrapping process (1000 times), which generated confidence intervals around the median of the curve and assessed the discrepancy between the observed curve parameters and the null distribution which assumed no statistically significant relationship between each predictor-outcome pair.