Discussion

Todeschini and Baccini (31) recommended that the ideal author-level indicator of citation performance should (i) have an unequivocal mathematical definition, (ii) be easily computed from available data (for a detailed breakdown of implementation steps and the R code function, see github.com/cjabradshaw/EpsilonIndex), (iii) balance rankings between more experienced and novice researchers (iv) while preserving sensitivity to the performance of top researchers, and (iv) be sensitive to the number and distribution of citations and articles. Our new ε-index not only meets these criteria, it also adds the ability to compare across disciplines by using a simple scaling approach, and can easily be adjusted for career gaps by subtracting research-inactive periods from the total number of years publishing (t). In this way, the ε-index could prove invaluable as we move toward greater interdisciplinarity, where tenure committees have had difficulty assessing the performance of candidates straddling disciplines (32, 33). The ε-index does not ignore high-citation papers, but neither does it overemphasise them, and it includes an element of publication frequency (i10) while simultaneously incorporating an element of ‘quality’ by including the h-index.
Like all other existing metrics, the ε-index does have some disadvantages in terms of not correcting for author contribution — such as the hm-index (9) or gm-index (10) — even though these types of metrics can be cumbersome to calculate. Early career researchers who have published but have yet to be cited will not yet be able to calculate their ε-index, as they will not have an h-index score, so would require different types of assessment. Another potential limitation is that the ε-index alone does not correct for any systemic gender biases associated with the many reasons why women tend to be cited less than men (24-30, 34), but it does easily allow an assessor to benchmark any subset of researchers (e.g., women-only or men-only) to adjust the threshold accordingly. Thus, women can be compared to other women and ranked accordingly such that the ranks are more comparable between these two genders. Alternatively, dividing the genders and benchmarking them separately followed by a combined re-ranking (Fig. 6) effectively removes the gender bias in the ε-index, which is difficult or impossible to do with other ranking metrics. We certainly advocate this approach when assessing mixed-gender samples (the same approach could be applied to other subsets of researchers deemed a priori to be at a disadvantage).
The ε-index also potentially suffers from the requirement of the constituent citation data upon which it is based being accurate and up-to-date (35, 36). Regardless, should an assessor have access to potentially more rigorous citation databases (e.g., Scopus), the ε-index can still be readily calculated, although within-sample consistency must be maintained for the ranks to be meaningful. We also show that the distribution of the ε-index is relatively more Gaussian in behaviour than the time-corrected m-quotient, with the added advantage of identifying a threshold above and below which individuals are deemed to be performing better or worse than expected relative to their sample peers. While there are potentially subjective rules of thumb for thresholds to be applied to the m -quotient, the residual nature of the ε -index makes it a more objective metric for assessing relative rank, and the ε -index is less-sensitive than the m -quotient regarding the innate rise of ranking as a researcher progresses through her career (Fig. 4).
We reiterate that while the ε-index is an advance on existing approaches to rank researchers according to their citation history, a single metric should never be the sole measure of a researcher’s productivity or potential. Nonetheless, the objectivity, ease of calculation, and flexibility of its application argue that theε -index is a needed tool in the quest to provide fair initial appraisals of a researcher’s publication performance.