Introduction

Deriving a fair, unbiased, and easily generated quantitative index serving as a reasonable first-pass metric for comparing the relative performance of academic researchers is — by the very complexity, diversity, and intangibility of research output across academic disciplines — impossible (1). However, that unachievable aim has not discouraged bibliometricians and non-bibliometricians alike from developing scores of citation-based variants (2, 3) in an attempt to do exactly that, from the better-known h-index (4, 5) (papers with at least h citations), m-quotient (4, 5) (h-index ÷ number of years publishing), and g-index (6) (unique largest number such that the top g papers decreasingly ordered by citations have least g2 citations), to the scores of variants of these and other indices — e.g., h2-index, e-index (7), χ-index (8), hm-index (9), gm-index (10), etc. (3). Each metric has its own biases and strengths (11-13), suggesting that several should be used simultaneously to assess citation performance. For example, the arguably most-popular h-index down-weights quality relative to quantity (14), ignores the majority of accumulated citations in the most highly cited papers (15), has markedly different distributions among disciplines (16), and tends to increase with experience (17). The h-index can even rise following the death of the researcher, because the h-index can never decline (2) and citations can continue to accumulate posthumously.
Despite their broad use in everything from assessing candidates applying for academic positions, comparing the track records of researchers applying for grants, to applications for promotion (3, 18) inter alia, single-value citation metrics are rarely meant to (nor should they) be definitive assessment tools (3). Instead, their most valuable (and fair) application is to provide a quick ‘first pass’ to rank a sample of researchers, followed by more detailed assessment of publication quality, experience, grant successes, mentorship, collegiality and all the other characteristics that make a researcher more or less competitive for rare positions and grant monies. But despite the many different metrics available and arguable improvements that have been proposed since 2005 when the h-index was first developed (4, 5), few are used regularly in these regards. This is because they are difficult to calculate without detailed data of a candidate’s publication history, they are not readily available on open-access websites, and/or they tend to be highly correlated with the h-index anyway (19). It is for these reasons that the admittedly flawed (20, 21) h-index and its experienced-corrected variant, the m-quotient, are still the dominant (h-index much more so than the m-quotient) (2) metrics employed given that they are easily calculated (2, 22) and found for most researchers on open-access websites such as Google Scholar (23) (scholar.google.com). The lack of access and detailed understanding of the many other citation-based metrics means that most of them go unused (3), and are essentially valueless for everyday applications of researcher assessment.
The specific weaknesses of the h-index or m-quotient make the comparison of researchers in different career stages, genders, and disciplines unfair because they are not normalised in any way. Furthermore, there is no quantitatively supported threshold above or below which assessors can easily ascertain minimum citation performance for particular applications — while assessors certainly use subjective ‘rules of thumb’, a more objective approach is preferable. For this reason, an ideal citation-based metric should only be considered as a relative index of performance, but relative to what, and to whom?
To address these issues and to provide assessors with an easy, rapid, yet objective relative index of citation performance for any group of researchers, we designed a new index we call the ‘ε-index’ (the ‘ε ’ signifies the use of residuals, or deviance from a trend) that is simple to construct, can be standardised across disciplines, is meaningful only as a relative index for a particular sample of researchers, can be corrected for career breaks (see Methods), and provides a sample-specific threshold above and below which assessors can determine whether individual performance is greater or less than that expected relative to the other researchers in the specific sample.
With the R code we provide, an assessor need only acquire four separate items of information from Google Scholar (or if they have access, from other databases such as Scopus — scopus.com) to calculate a researcher’s ε-index: (i) the number of citations acquired for the researcher’s top-cited paper (i.e., the first entry in the Google Scholar profile), (ii) the i10-index (number of articles with at least 10 citations), (iii) the h-index, and (iv) the year in which the researcher’s first peer-reviewed paper was published. While the last item requires sorting a researcher’s outputs by year and scrolling to the earliest paper, this is not a time-consuming process. We demonstrate the performance of the ε-index using Google Scholar citation data we collected for 480 researchers in eight separate disciplines spread equally across genders and career stages to show how the index performs relative to the m-quotient (the only other readily available, opportunity-corrected citation index available on Google Scholar) across disciplines, career stages, and genders. We also provide a simple method to scale the index across disciplines (ε′-index) to make researchers in different areas comparable despite variable citation trends within their respective areas.