So you work extremely hard on your research, write it up for publication, submit to a peer-reviewed journal, meet the demands of the reviewers and editor, and seek your work appear in print.  Then, no one cites it.  What are the odds?

It turns out that the odds are pretty high. A recent paper in the journal BMC Medical Research Methodology looked at more than 30,000 original article and reviews in 235 journals published in current and surgical fields during 2001.  The authors found through October 2003 that “16.7% of articles in a journal accrue half the total number of citations to that journal… 23.7% of articles had not yet been cited.”

But these numbers look pretty high when compared to a more comprehensive study reported by Science magazine in 1991 (available here via JSTOR, subscription required).  Science reported that “55% of the papers published between 1981 and 1985 in journals indexed by [ISI] received no citations at all after they were published.”  And self-citation “accounts for between 5% and 20% of all citations.”  

In a follow-up article, Science reported uncitedness data (over four years) for specific disciplines for papers published in 1984.  It’s not a pretty picture:

Low Disciplinary Uncitedness Rates (papers published 1984, citations through 1988)

Physics 36.7%
Chemistry 38.8%
Biology 41.3%
Geosciences 43.6%
Average uncitedness rate for “physical sciences” = 47.4%

High Disciplinary Uncitedness Rates (papers published 1984, citations through 1988)

History 95.5%
Philosophy 92.1%
Political science 90.1%
Average uncitedness rate for “social sciences” = 74.7%
Average uncitedness for “arts and humanities” = 96%

Now these numbers are very interesting (imagine my surprise to learn of them in 1991 as a doctoral student in political science!), but they raise a broader question, “So what?”

Predictably, the two Science 1991 articles motivated a heated debate about the significance of being uncited.  Some argued that it shows that the vast majority of research is not even valued by fellow researchers, and thus indicates a waste of money on research.  Others argue that the time period being looked at for citations doesn’t square with how research I used by fellow researchers, particularly in the social sciences and humanities where research has a long shelf life (cf., Plato).  Still others argue that citations are a poor metric of value anyway.

This is a debate worth revisiting.  Uncitedness rates of any kind are not routinely reported.  They should be.  As the BMC Medical Research Methodology paper cited about concludes, “Non-citation levels should therefore be made available for all journals.”

References:

The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor
Andy R Weale , Mick Bailey  and Paul A Lear

BMC Medical Research Methodology 2004, 4:14
Published   28 May 2004

http://www.biomedcentral.com/1471-2288/4/14/abstract

Publishing by-and for?-the Numbers
David P. Hamilton
Science, New Series, Vol. 250, No. 4986. (Dec. 7, 1990), pp. 1331-1332.
Stable URL: http://links.jstor.org/sici?sici=0036-8075%2819901207%293%3A250%3A4986%3C1331%3APBFN%3E2.0.CO%3B2-Y

Research Papers: Who’s Uncited Now?
David P. Hamilton
Science, New Series, Vol. 251, No. 4989. (Jan. 4, 1991), p. 25.
Stable URL: http://links.jstor.org/sici?sici=0036-8075%2819910104%293%3A251%3A4989%3C25%3ARPWUN%3E2.0.CO%3B2-N