Discussion
Todeschini and Baccini (31) recommended that the ideal author-level
indicator of citation performance should (
i) have an unequivocal
mathematical definition, (
ii) be easily computed from available
data (for a detailed breakdown of implementation steps and the R code
function, see
github.com/cjabradshaw/EpsilonIndex), (
iii) balance
rankings between more experienced and novice researchers (
iv)
while preserving sensitivity to the performance of top researchers, and
(
iv) be sensitive to the number and distribution of citations and
articles. Our new
ε-index not only meets these criteria, it also
adds the ability to compare across disciplines by using a simple scaling
approach, and can easily be adjusted for career gaps by subtracting
research-inactive periods from the total number of years publishing
(
t). In this way, the
ε-index could prove invaluable as we
move toward greater interdisciplinarity, where tenure committees have
had difficulty assessing the performance of candidates straddling
disciplines (32, 33). The
ε-index does not ignore high-citation
papers, but neither does it overemphasise them, and it includes an
element of publication frequency (
i10) while
simultaneously incorporating an element of ‘quality’ by including the
h-index.
Like all other existing metrics, the ε-index does have some
disadvantages in terms of not correcting for author contribution —
such as the hm-index (9) or gm-index (10) — even though these types of
metrics can be cumbersome to calculate. Early career researchers who
have published but have yet to be cited will not yet be able to
calculate their ε-index, as they will not have an h-index
score, so would require different types of assessment. Another potential
limitation is that the ε-index alone does not correct for any
systemic gender biases associated with the many reasons why women tend
to be cited less than men (24-30, 34), but it does easily allow an
assessor to benchmark any subset of researchers (e.g., women-only or
men-only) to adjust the threshold accordingly. Thus, women can be
compared to other women and ranked accordingly such that the ranks are
more comparable between these two genders. Alternatively, dividing the
genders and benchmarking them separately followed by a combined
re-ranking (Fig. 6) effectively removes the gender bias in the ε-index, which is difficult or impossible to do with other
ranking metrics. We certainly advocate this approach when assessing
mixed-gender samples (the same approach could be applied to other
subsets of researchers deemed a priori to be at a disadvantage).
The ε-index also potentially suffers from the requirement of the
constituent citation data upon which it is based being accurate and
up-to-date (35, 36). Regardless, should an assessor have access to
potentially more rigorous citation databases (e.g., Scopus), the ε-index can still be readily calculated, although within-sample
consistency must be maintained for the ranks to be meaningful. We also
show that the distribution of the ε-index is relatively more
Gaussian in behaviour than the time-corrected m-quotient, with the
added advantage of identifying a threshold above and below which
individuals are deemed to be performing better or worse than expected
relative to their sample peers. While there are potentially subjective
rules of thumb for thresholds to be applied to the m -quotient,
the residual nature of the ε -index makes it a more objective
metric for assessing relative rank, and the ε -index is
less-sensitive than the m -quotient regarding the innate rise of
ranking as a researcher progresses through her career (Fig. 4).
We reiterate that while the ε-index is an advance on existing
approaches to rank researchers according to their citation history, a
single metric should never be the sole measure of a researcher’s
productivity or potential. Nonetheless, the objectivity, ease of
calculation, and flexibility of its application argue that theε -index is a needed tool in the quest to provide fair initial
appraisals of a researcher’s publication performance.