In the following guest post, Eric Schwitzgebel (UC Riverside) recounts what he found when, prompted by claims about how infrequently academic philosophy articles are cited, he looked at the citation rates of articles published in a few journals a decade ago.
(A version of this post first appeared at his blog, The Splintered Mind.)
How Often Are Philosophy Articles Actually Cited? Encouraging News
by Eric Schwitzgebel
You hear terrible things. You hear, for instance, that over 50% of philosophy articles are entirely uncited. You hear that the average philosophy article is cited less than 5 times. People will sometimes say things like the “median number of readers for a philosophy article is 1” or “there’s not much difference between publishing a paper and throwing it away“. Both of the last two comments appear in a recent Twitter thread launched by Helen De Cruz. It was reading this thread that inspired me to do the analyses I’ll share here.
Generally my reaction to analyses and comments of this sort is to think that they are considerably underestimating how often philosophy articles are actually cited. The “big data” interdisciplinary analyses often use methods that are a terrible fit for philosophy, such as looking at citations in the past two years. (Often in philosophy it takes two years or more to write and publish an article.) Also, I wonder what counts as an “article”. If we’re including two-page book reviews, it’s little wonder that they’d be little cited, and similarly if we’re including publications in predatory or obscure journals. What would be more interesting to know—and what I think most people in this discussion really care about—is how frequently cited are full-length research articles in “respectable” mainstream philosophy journals?
Before you look at my analyses, any guesses?
Method
I selected six representative general philosophy journals for analysis: two “top ranked” journals (Philosophical Review and Noûs), two mid-ranked journals (Canadian Journal of Philosophy and Pacific Philosophical Quarterly), and two unranked but reputable journals (Philosophia and Southern Journal of Philosophy). I then downloaded the entire table of contents of these journals from the year 2012—giving a ten-year citation window—and excluded anything that wasn’t an ordinary full-length research article (e.g., book reviews, editors’ introductions, symposium proceedings). From each journal, I randomly selected 15 articles and noted their total number of citations in Google Scholar.
Results
Although this is not a large database, the results from this carefully designed search are, I think, striking.
Overall, the mean article was cited 31 times, and the median number of citations was 20. For the elite journals the mean was 50 and the median was 36; for the mid-ranked journals the mean was 25 and the median was 8; for the unranked journals, the mean was 17 and the median was 12. Only three of the 90 articles (3%) were cited zero times.
Here’s a more specific breakdown, journal by journal. Of course with only 15 articles per journal, there will be lots of noise in the numbers when considered at this fine a grain.
Histogram of total citations by article for articles published in 2012 from six representative general philosophy journals.
Conclusions
I draw the following general conclusions:
(1) Very few articles published in mainstream general philosophy journals remain entirely uncited. My estimate of 3% might be off, due to the smallish sample size and the particular journals selected; but it’s highly unlikely that non-citation rates of mainstream general philosophy articles are anything near 50%.
(2) The majority of articles are cited often enough to be “in the conversation”. While an outsider to academia might not think that ten citations is much impact, I submit that a more appropriate perspective is this: If your article is cited at least ten times, then it is having an impact on other specialists in your subfield. Your article isn’t falling into the void. It is part of the conversation, and other scholars are reacting to it. A journal article needn’t have a huge influence outside its subarea to be successful. The beauty of academic journals is that they host technical pieces that often can only be appreciated by a few dozen specialists. If academia is worthwhile, then continuing those specialists’ conversations is worthwhile, and high citation rates for technical pieces should not be expected. (I intend this ten citation criterion as an approximately sufficient condition for impact, not a necessary condition.)
(3) Articles in elite journals are cited only about three times as often as articles in unranked but reputable journals. To me, this was the most surprising result, and part of me wonders whether we’d see different results if we looked at different years or different unranked but reputable journals. Philosophical Review is much more prestigious in the eyes of the typical mainstream academic philosopher than is the typical unranked but reputable journal. I find this result encouraging, especially given the tiny acceptance rates at the elite journals (Philosophical Review reports accepting about twelve articles per year out of 600 submitted). Publishing in unranked journals is not shouting into the void. It is not defeat. The dynamic is less jackpot-or-nothing than I would have expected. You don’t have to get into a top-ten journal to have an impact. Philosophers do regularly read and cite articles from the less prestigious journals.
(4) Citation skew is much less extreme than it could be. Another respect in which philosophy citation practices are not jackpot-or-nothing is revealed by the smallish differences between the means and medians. Of course the means are higher: there’s right skew in the data, a tendency for high-end outliers to pull up the mean. But it’s not like most articles are cited 0-5 times and a few are a cited hundreds of times. Authors can reasonably expect that a decent article in a decent journal will have at least a moderate impact on their subfield.
Related: “Philosophy Has High Rate of Uncited Publications“, “Citation Patterns Across Journals“, “Philosophers Don’t Read and Cite Enough“, “Citation Problems in Philosophy—and Some Fixes“, “Citation Rankings of Philosophers Based on Scopus Data“. See also, “Where Are They Now? The Healy 2100” by Jonathan Weisberg at his site.
Originally appeared on Daily Nous Read More
