Over Time, Citations Get More Uneven

In an earlier post, I compared the distribution of citations within Journal of Philosophy, 1976 and Philosophical Studies, 2009. And I noted that although the two ended up in similar places, they got there in different ways. Journal of Philosophy was largely relying on some hugely cited papers; while the distribution of citations in Philosophical Studies was more even. And I suggested that this was part of a trend in the discipline towards more egalitarian citation practices.

In Facebook comments on this post, Ben Blumson and Peter Michael Gerdes suggested an alternative explanation. Perhaps the citations are more unbalanced in Journal of Philosophy, 1976 because the longer a journal is out, the more a consensus builds up around what the great papers are in it, and the more those papers (and those papers only) get cited. In effect, I was seeing an age effect, and viewing it as a cohort effect.

So I looked at the data, and it looks like they were very largely correct. All the points below come from investigating the citations to papers in Journal of Philosophy, 1976.

I broke the citations up into those appearing before and after 1995. In fact more of the citations come after 1995, which was surprising. (That might be due to Web of Science’s coverage getting more comprehensive in recent years.) We can then calculate a Gini coefficient for the citations before and after 1995. (We usually do Gini coefficients for things like income or wealth, but the same idea can be used for citations; think of each cite to an article as a bit of wealth that article gets.)

The numbers ended up being:

  • For citations before 1995, Gini is 0.83.
  • For citations after 1995, Gini is 0.92.

Now both of these numbers are really high, but that’s largely because I included the book reviews, which basically get 0 citations, in the mix. But the second number is much higher. Note that Gini goes on a 0-1 scale, so going from 0.83 to 0.92 is going half way to a situation where only one article gets all the citations; it’s a big jump.

Here’s another way to look at the data. I broke down the articles into those that had (across all 40 years) the 10 highest citation counts, and the rest of the articles. (There are about 130 of them, though many are discussion notes or book reviews.)

  • Among the 10 highest cited articles, 72% of citations came after 1995.
  • Among the rest of the articles, 47% of the citations came after 1995.

So there’s a pattern here. And while it’s only one year, it isn’t surprising.

So I made a mistake here, though I think it’s an interesting mistake. I noticed that within any given year, the citations were getting more evenly spread the closer we got to the present. (That is, the Gini coefficient was going down over time.) I thought that was a cohort effect; it was something about how people interact with post-2000 journals as opposed to how they interact with pre-2000 journals. But it is more plausible that it is an age effect; it is something about how people interact with 20+ year-old journals as opposed to how they interact with younger journals.

It’s easy to confuse cohort effects and age effects. Lots of people look at voting data and conclude that people get more conservative as they get older. This isn’t, on the whole, true. It’s just that, in recent years in the English speaking world, each generation has been less conservative than the one that came before it. This hasn’t always been true; the boomers are much much more conservative than people who could remember the Great Depression. And historically the graph of voting pattern vs age had an inflection point around the time it got to people born in 1930. (That’s too small a group now to show up in the graphs.) And it isn’t true in modern day France, for example. (In the recent election, old folks did largely vote for Fillon, but Le Pen’s support was relatively constant across age groups.)

I’ve made the opposite confusion here; seeing something that really is an age effect as a cohort effect. Thanks to Ben and Peter for pointing this out.

Citations are Getting More Evenly Distributed

I claimed yesterday that citations within journals are getting more egalitarian. To verify this, I pulled the citation data for two prominent years: The Journal of Philosophy, 1976 and Philosophical Studies, 2009.

I couldn’t find an easy way to get just the citations in Arts & Humanities, so this table includes all the citations in Web of Science. (I checked a few articles, and the non-humanities citations average around 20% of the total citations, though the articles in political philosophy typically run a fair bit higher than that.)

The citations here are just through 2015; that makes a big difference to the most cited articles from 1976, and to many articles from 2009.

Both years I’m looking at have a number of discussion notes includes. The Journal of Philosophy used to publish papers presented at the APA, and the commentaries on them. So I’ve included a column showing how many pages are in each article, to give you a sense of when an article is really just a discussion note.

First, the citations of articles in The Journal of Philosophy, 1976. I’ve deleted the book reviews, and the discussion notes that were 3 pages or less.

Title Author Cites Pages
Discrimination And Perceptual Knowledge Goldman, AI 370 21
Schizophrenia Of Modern Ethical Theories Stocker, M 172 14
Motive Utilitarianism Adams, RM 55 15
Explanation, Conjunction, And Unification Kitcher, P 52 6
Putnam’s Theory On Reference Of Substance Terms Zemach, EM 40 12
Two Types Of Foundationalism Alston, WP 36 21
Grades Of Discriminability Quine, WV 33 4
Humes Cognitive Theory Of Pride Davidson, D 32 14
Worlds Away Quine, WV 30 5
Truth And Assertibility Brandom, R 28 13
Knowledge, Causality, And Defeasibility Klein, PD 26 21
What Is A Logical Constant Peacocke, C 22 20
Social Choice And Derivation Of Rawls’s Difference Principle Strasnick, S 21 15
Necessity Of Origin Mcginn, C 21 9
Counterfactuals With Disjunctive Antecedents Loewer, B 17 7
Direction Of Causation And Direction Of Conditionship Sanford, DH 16 15
Save Phenomena Van Fraassen, BC 15 10
History And Hermeneutics Ricoeur, P 13 13
Mentality And Neutrality Rosenthal, DM 12 30
Causation – Matter Of Life And Death Earman, J 11 21
Psychology Of Benevolence And Its Implications For Philosophy Brandt, RB 7 25
Comments On Nozicks Entitlement Theory Davis, L 7 9
Theory Of Language Harris, Z 6 24
Identity Statements And Microreductions Enc, B 6 22
Mechanism, Functionalism, And Identity Theory Nelson, RJ 6 21
Entitlement Theory Of Distributive Justice Goldman, AH 6 13
Ontological Reduction Gottlieb, D 5 20
Method For Ontology, With Applications To Numbers And Events Gottlieb, D 3 15
Inferential Justification And Empiricism Fumerton, RA 3 13
Discernibility Of Identicals Moravcsik, JME 3 12
Labor Theory Of Property Acquisition Becker, LC 3 12
Rawls’s Original Position And Difference Principle Goldman, AH 3 5
Space And Objects Oneill, O 2 17
Meaning And Perception Pastin, M 2 15
Strasnicks Derivation Of Rawlss Difference Principle Wolff, RP 2 10
Projectability Unscathed Ullian, J; Goodman, N 2 5
Carrots, Noses, Snow, Rose, Roses Gass, WH 1 15
Strawson On Predication Moravcsik, JME 0 20
Practical Reason And Concept Of A Human Being Scott, S 0 14
Charles Taylor’s Hegel Soll, I 0 14
Identity Of Indiscernibles Nagel, G 0 6

Next, the citations of articles in Philosophical Studies, 2009. I’ve deleted the articles that were 3 pages or less, and the introductions to the book symposia. I’ve left the rest of the symposia in, though many of these have very few citations. (The symposia contributions mostly are around 10 pages or so.)

Title Author Cites Pages
Knowledge and credit Lackey, Jennifer 35 16
Spacetime the one substance Schaffer, Jonathan 31 18
Means-end coherence, stringency, and subjective reasons Schroeder, Mark 30 26
Absence of evidence and evidence of absence Sober, Elliott 28 28
Shared intention and personal intentions Gilbert, Margaret 28 21
Models and fictions in science Godfrey-Smith, Peter 27 16
Knowledge and success from ability Greco, John 25 10
Oughts and ends Finlay, Stephen 25 26
Modest sociality and the distinctiveness of intention Bratman, Michael E. 20 17
Weighing the aim of belief Steglich-Petersen, Asbjorn 20 11
Thought-experiment intuitions and truth in fiction Ichikawa, Jonathan; Jarvis, Benjamin 19 26
A better best system account of lawhood Cohen, Jonathan; Callender, Craig 19 34
A Virtue Epistemology Pritchard, Duncan 17 10
In defense of adaptive preferences Bruckner, Donald W. 17 18
Models, measurement and computer simulation Morrison, Margaret 16 25
Epistemology without metaphysics Field, Hartry 16 42
The possibility of pragmatic reasons for belief Reisner, Andrew 15 16
Determination, realization and mental causation Wilson, Jessica 15 21
Motivated contextualism Henderson, David 15 13
Individuals Dasgupta, Shamik 14 33
The folk on knowing how Bengson, John; Moffett, Marc A.; Wright, Jennifer C. 14 15
The neural evidence for simulation is weaker than I think you think it is Saxe, Rebecca 13 10
Empathy, social psychology, and global helping traits Miller, Christian B. 13 29
Self-representationalism and phenomenology Kriegel, Uriah 13 25
Intuitions are inclinations to believe Earlenbaugh, Joshua; Molyneux, Bernard 12 21
The logic, intentionality, and phenomenology of emotion Montague, Michelle 11 22
The open future Barnes, Elizabeth; Cameron, Ross 11 19
Revisionism about free will Vargas, Manuel 11 18
The perils of Perrin, in the hands of philosophers van Fraassen, Bas C. 11 20
Inter-species variation in colour perception Allen, Keith 11 24
Assertion, Moore, and Bayes Douven, Igor 11 15
What good is a diachronic will? Ferrero, Luca 10 28
Knowing full well Sosa, Ernest 9 11
Moral judgment purposivism Bedke, M. S. 9 21
Replies Williamson, Timothy 9 12
A new argument for skepticism Reed, Baron 9 14
Imagination and other scripts Funkhouser, Eric; Spaulding, Shannon 9 24
Dubious assertions Sosa, David 9 4
Evidence-based policy Cartwright, Nancy 8 10
Indeterminacy and variability in meta-ethics Gill, Michael B. 8 20
Reduction and emergence: a critique of Kim Needham, Paul 8 24
The irrationality of recalcitrant emotions Brady, Michael S. 8 18
Experience and self-consciousness Schear, Joseph K. 8 11
Semantic intuitions, conceptual analysis, and cross-cultural variation Jackman, Henry 7 19
Compatibilism & desert McKenna, Michael 7 11
Drawing the boundary between low-level and high-level mindreading de Vignemont, Frederique 7 10
A consistent way with paradox Goldstein, Laurence 7 13
Utterance at a distance Stevens, Graham 7 9
Concept Cartesianism, Concept Pragmatism, and Frege Cases Rives, Bradley 6 28
Triviality arguments against functionalism Godfrey-Smith, Peter 6 23
Hard incompatibilism and its rivals Pereboom, Derk 6 13
Imaginability, morality, and fictional truth Todd, Cain Samuel 5 25
Is computer simulation changing the face of experimentation? Giere, Ronald N. 5 4
Contextualism, relativism and ordinary speakers’ judgments Montminy, Martin 5 16
Structural equations and causation Hitchcock, Christopher 5 11
Moral responsibility and agents’ histories Mele, Alfred 5 21
Summation relations and portions of stuff Donnelly, Maureen; Bittner, Thomas 5 19
Luminous enough for a cognitive home Fumerton, Richard 5 10
Virtuous intuitions Boghossian, Paul 4 9
Randomized controlled trials and the flow of information Roush, Sherrilyn 4 9
Replies Goldman, Alvin I. 4 15
Neither here nor there Debes, Remy 4 27
Knowing the intuition and knowing the counterfactual Ichikawa, Jonathan 4 9
Against Cognitivism about Practical Rationality Brunero, John 4 15
Advice for fallibilists Fantl, Jeremy; McGrath, Matthew 4 12
Towards a semantics for biscuit conditionals Predelli, Stefano 4 13
Reference, perception, and attention Raftopoulos, Athanasios 4 22
On doing better, experimental-style Weinberg, Jonathan M. 4 10
Simulation and the first-person Carruthers, Peter 4 9
The modal status of materialism Levine, Joseph; Trogdon, Kelly 4 12
Moral advice and moral theory Leibowitz, Uri D. 4 11
Promises beyond assurance Southwood, Nicholas; Friedrich, Daniel 4 20
Intentional psychologism Pitt, David 4 22
Restricting factiveness Stjernberg, Fredrik 4 20
Replies to commentators Sosa, Ernest 3 11
I won’t do it Louise, Jennie 3 22
Truth-conditions, truth-bearers and the new B-theory of time Torre, Stephan 3 20
Defending a possibilist insight in consequentialist thought Vessel, Jean-Paul 3 13
Non-identity, self-defeat, and attitudes to future children Kahane, Guy 3 22
Liberalism and the general justifiability of punishment Hanna, Nathan 3 25
Virtue theory, ideal observers, and the supererogatory Kawall, Jason 3 18
The Loop Case and Kamm’s Doctrine of Triple Effect Liao, S. Matthew 3 9
Objective evidence and absence Strevens, Michael 3 10
The myth of the categorical counterfactual Barnett, David 3 16
Naturalism, fallibilism, and the a priori Warenski, Lisa 3 24
Difficult times for Humean identity? Garrett, Don 3 9
Normativity without artifice Bauer, Mark 3 21
Bennett and proxy actualism Nelson, Michael; Zalta, Edward N. 3 16
Reliabilism in philosophy Goldberg, Sanford C. 3 13
Sosa in perspective Kornblith, Hilary 3 10
Parity, incomparability and rationally justified choice Boot, Martijn 2 18
Reupholstering a discipline Martin, M. G. F. 2 9
Contextualism, safety and epistemic relevance Blome-Tillmann, Michael 2 12
Simulation a la Goldman Perner, Josef; Brandl, Johannes L. 2 12
Bootstrapping and knowledge of reliability Brueckner, Anthony; Buford, Christopher T. 2 6
Why is a truth-predicate like a pronoun? Bave, Arvid 2 14
Replies Sosa, Ernest 2 14
Physicalism and sparse ontology Trogdon, Kelly 2 19
Analyzing a priori knowledge Casullo, Albert 2 14
Fodor’s riddle of abduction Rellihan, Matthew J. 2 26
Knowledge as aptness Cohen, Stewart 2 5
Libertarianism Kane, Robert 2 10
Defense Draper, Kai 2 20
Aggregation, Partiality, and the Strong Beneficence Principle Dorsey, Dale 1 19
Comments Dreyfus, Hubert L. 1 8
Indirect perceptual realism and demonstratives Brown, Derek Henry 1 18
A problem for Russellian theories of belief Ostertag, Gary 1 19
Replies Baxter, Donald L. M. 1 11
Science fictions Fine, Arthur 1 9
Plural signification and the Liar paradox Read, Stephen 1 13
Against structured referring expressions Sullivan, Arthur 1 26
The fate of a warrior culture Sherman, Nancy 1 10
Is knowledge a natural kind? Pernu, Tuomas K. 1 16
The fixity of reasons Gallois, Andre Norman 1 16
The event of color Pasnau, Robert 1 17
Ineliminable tension Lenard, Patti Tamara; Moore, Margaret R. 0 7
What is wrong with the indeterminacy of language-attribution? Khatchirian, Arpy 0 25
The a priori defended Thurow, Joshua C. 0 17
The Chrysippus intuition and contextual theories of truth Newhard, Jay 0 8
Two-dimensionalism and the epistemology of recognition Valaris, Markos 0 19
Trumping the causal influence account of causation Stone, Jim 0 8
Who they are and what de se Giberman, Daniel 0 15
Evidentialism and the problem of stored beliefs Piazza, Tommaso 0 14
Response Lear, Jonathan 0 13
Perilous thoughts Longino, Helen 0 8
Ultimacy and alternative possibilities Fischer, John Martin 0 6
Hume and Baxter on identity over time Falkenstein, Lorne 0 9
Noncomparabilism in epistemology Wunderlich, Mark Emerson 0 19
The inessential quasi-indexical Alward, Peter 0 21
A Virtue Epistemology, vol 1 Conee, Earl 0 10
Justification and awareness Markie, Peter J. 0 17
Hume and Frege on identity Perry, John 0 11
Fictions within fictions Hayaki, Reina 0 20
Reliability as a virtue Audi, Robert 0 12
The ethics of morphing Hare, Caspar 0 20
Sosa on scepticism Brown, Jessica 0 9

Now on the one hand there are a lot of articles here that didn’t get a lot of citations by 2015. On the other hand, there are 48 articles that had at least 7 citations (so at least 1/year), by 2015. That’s a huge number, and that’s why Philosophical Studies is getting so many citations – it publishes so many things that get at least some uptake. And that, I think, is very impressive.

Two More Graphs

Just as a follow up to the previous post, here are the graphs for six more journals. First their citations in the other 31.

Then their citations in the whole of Web of Science Arts and Humanities. (Note that this sometimes severely understates how many citations they have elsewhere in the academy, especially for philosophy of science and political philosophy papers.)

The big story of these, I think, is the overall upwards trends of the graphs. This is most dramatic in the case of PPR, but it is there across the board.

AJP has suffered a little bit since 2000 without Lewis articles to boost its citation counts. Its recent years compare well to its earlier years that didn’t contain a Lewis paper, but not so well to (for example) 1983, 1984 and 1996. It’s hard to find papers like “New Work”, “Putnam’s Paradox” and “Elusive Knowledge” to publish every year.

Changes in Who is Cited

The dataset of citations I’m looking at starts in 1976. And I can break down not just which journals are cited, but which years of those journals are cited. (And which years they are cited in, but I’m setting aside that data for now.) So as well as asking which journal is the most cited – it’s The Journal of Philosophy by a reasonable amount – we can ask which journals papers in year X ended up being the most cited.

And from 1976 through to the early 1990s, it is The Journal of Philosophy by a lot. Fourteen of the 25 most cited journal-years are from the 1976-1991 run of The Journal of Philosophy.

Through the 1990s The Journal is still the most cited, but it is much closer. (That is, for most years in the 1990s, the articles in The Journal from that year ended up with more citations than articles from any other journal.)

But from around 2002 onwards, the most cited journal is Philosophical Studies. Here are a couple of graphs that show the change.

First, I’ve graphed the number of citations each journal has in the other 31 journals in the data set. Each dot represents a single journal-year. So the top right at the top left represents that the single most cited journal-year in the data set is The Journal of Philosophy‘s 1976 volume. It featured really influential pieces by Alvin Goldman and Michael Stocker, and those articles alone would have been enough to make it a standout year.

The lines are loess curves through the data points; the dots are the actual measurements. The lines mostly slope down towards the end, because articles published in recent years have fewer chances to build up a citation count.

Note that almost all the dots towards the top of the graph are red, they are for The Journal of Philosophy. There is a dot for Nous 1979 (which included famous articles by Perry, Lewis and Cartwright), but otherwise the high points on the left are all red dots. But as we go into the 2000s, the purple line moves ahead, and the purple dots become more visible at the top edge of the graph.

I stopped the graph in 2010, because after that the number of citations starts to get very small. There just hasn’t been enough time for some of those articles to be cited. But the same pattern holds; indeed, Philosophical Studies takes a more pronounced lead.

The pattern is the same if we look not just at citations in the other 31 philosophy journals I focussed on, but at all journals in the Web of Science Arts & Humanities Index.

Now part of this is that Philosophical Studies publishes so many articles that it can do very well on graphs of raw citation count. But this isn’t the whole story. After all, Philosophical Studies has published a lot of articles per year for a long time, but the jump in citations is relatively recent.

There is one other striking thing about this recent surge in citations to Philosophical Studies: it largely happens without any individual articles getting huge numbers of citations. Some years there are articles in Philosophical Studies with huge citations. In 2006 the articles by Knobe and by Street are a big reason why Philosophical Studies does so well. But not all years are like that. A big part of the story is that Philosophical Studies went from having its not-so-huge articles get 5 citations a piece, to getting 10 citations a piece, and that adds up.

The underlying story is that citations are getting much more egalitarian, both across and within journals.

And it is striking just how big a role The Journal of Philosophy played in the philosophy world up to 1990, and how big a role Philosophical Studies plays nowadays.

Citation Graphs, One Journal at a Time

The big graphs I posted earlier have so much information, that much of it gets lost in the mess of lines. So I’ve broken down the data to create a graph just involving each journal’s interactions with the other 31. Here are the graphs. (As always, click on each for a larger image.)

British Journal of Aesthetics Journal of Aesthetics and Art Criticism
Journal of the History of Philosophy Phronesis
Kant Studien Journal of Political Philosophy
Philosophy and Public Affairs Ethics
Biology and Philosophy Synthese
British Journal for the Philosophy of Science Philosophy of Science
Economics and Philosophy Review of Metaphysics
Journal of Philosophical Logic Review of Symbolic Logic
Mind and Language Linguistics and Philosophy
Episteme Philosophical Studies
Mind Philosophical Quarterly
Analysis Australasian Journal of Philosophy
Philosophy and Phenomenological Research Noûs
Philosophical Review Journal of Philosophy
Pacific Philosophical Quarterly Southern Journal of Philosophy
American Philosophical Quarterly Canadian Journal of Philosophy

I don’t have a lot of commentary on these beyond what I’ve said already, but here are a few quick thoughts about them.

  • When comparing different graphs, look at the scale before making judgments. The numbers are really different across journals.
  • CJP is more balanced than other journals, especially on moral/political and history. The red bar near the start of the outer ring is a sign of how much it interacts with Ethics.
  • Philosophical Studies is a huge part of the ecosystem. Partially that’s because of its size, but not entirely.
  • By the citation numbers, Review of Metaphysics is basically a history journal at this stage, and Economics and Philosophy is, insofar as it is a philosophy journal, a moral/political journal.
  • Mind has slightly more interactions with the logic journals than the other generalist journals do, but it isn’t dramatic.
  • Looking at the history journals makes Philosophical Review’s interactions with them more prominent than it is on the larger graph.
  • The difference between how much Episteme cites the American generalist journals and how much it cites the Commonwealth journals is fascinating. I don’t know how much this is the (quite normal) bias journals have towards journals that are geographically near them, and how much it reflects a different attitude towards epistemology in the US vs Commonwealth journals.
  • The citations to Analysis are much more balanced across the generalist journals than I expected; I thought journals would differ in how much they wanted to engage with Analysis articles.

Citation Graphs and Methodology

Last week I posted some graphs showing how often various journals cite other journals. Here is one of those graphs, just to remind you what they looked like.

I didn’t say particularly clearly how I got the data, or even exactly what the graphs meant. So here’s the long methodology post showing how I ended up with graphs like this one.

The data all come from Web of Science, and in particular its “Cited Reference Search”. For each of the 32 journals I was looking at, and each of the 40 years in 1976–2015, I searched for all citations to that journal in that year. So, for instance, I did a search under “Cited Reference Search” where the source was Australasian Journal of Philosophy, and the year was 1996. I set the “Timespan” to be 1976–2015, and the Indexes to search to be just the Arts & Humanities Citation Index.

When you do that, you get a list of possible articles that you might want to look at the citations of. (These are, roughly, all the things published in that journal in that year.) I hit select all, because I wanted to see everything that cited at least one of those journals, and then downloaded some information about everything that turned up.

Note three things about doing the search this way:

  1. I didn’t even download any information about which individual articles were being cited. The method got me more information than I needed (or even wanted) about the citing articles, but none at all about what was cited. Obviously there is a lot of interest in which articles are being cited, but that will be left for others to do.
  2. This method will only turn up citations to the article in the journal. So if someone just cited, say, “Elusive Knowledge” just in its reprint in Papers in Metaphysics and Epistemology, and not in the original AJP publication, it won’t show up. I don’t think this made a huge difference, but it affected some things. For instance, Stephen Barker’s 2011 Noûs paper “Can Counterfactuals Really Be about Possible Worlds?” doesn’t show up in the list of papers citing AJP 1996, because the reference to “Elusive Knowledge” is just to the reprint. So this is one way some noise creeps into the system.
  3. At each of the 1280 stages, I’m doing a disjunctive search: Find all things that cite that journal in that year. The search doesn’t discriminate between things that cite five articles from that year and things that cite one article from that year. I think this is probably a good thing; citations to multiple articles in one year are usually citations to a single thread, and I’d rather treat them as a single citation. But it is a complication.

There was one more annoying wrinkle. Web of Science doesn’t separate out Noûs from Philosophical Issues and (usually) Philosophical Perspectives. As far as I could tell, every issue of Philosophical Issues is coded as a special issue of Noûs, and about half the issues of Philosophical Perspectives are. I wanted to get this noise out of the data. So when I was searching for Noûs I had to go through by hand and search just for those things that cited the real Noûs articles, not the ones citing the special issues. This wasn’t too hard, because the special issues had page numbers and/or issue numbers starting with ‘S’, but it was a bit of a pain. We’ll come back to that.

The result, after these 32 by 40 searches, was a file with roughly 240,000 citations in it. But a lot of these were citations in journals I wasn’t looking at. So I made a restricted file where the citations were just to the 32 journals I was looking at. This was mostly just a matter of filtering the large file by the citing journal, though again there was a bit of a pain filtering out the different things coded as ‘Noûs’. (It wasn’t too hard this time, to be honest, because the downloaded data about citing papers included issue and page number, so searching for things like ‘S’ got rid of most of them.) This was less automated than most of the process, so there was a higher chance of errors creeping in.

The result was a file with about 106,000 citations in it. The graph you saw above comes from a slightly smaller file, one that deletes all of the citations to articles in the same journal. That covers a lot of citations, so we’re now down to about 82,000 citations. Journals, it turns out, love publishing papers that cite other things in that very journal. For 24 journals, the journal they most frequently cite is themselves. For 7 others, it is Journal of Philosophy, and for 1, the Journal of Political Philosophy, it is Philosophy & Public Affairs. So we cut out a fair bit here.

I used that 82,000 strong list to build a 32 by 32 table, with the cited journal on the rows, and the citing journal on the columns. Each cell had a count of how often that pair showed up in the data set, from 0 (any number of times), to 1534 (citations by Sythese of Philosophy of Science). These are raw counts; so journals that publish a lot will naturally have bigger numbers (in both the rows and columns) than smaller journals. I’ll come back to this point in later posts; I’ve been working a fair bit this week on ways to address this.

Then I arranged the journals so that similar journals were nearby. I was using a fairly rough and ready version of similarity, and there were probably better ways to do this. There ended up being a big jump between the philosophy of science journals and the ethics journals, and a jump (though actually a bit smaller) between CJP in the generalist journals and the history journals.

It’s striking that it is possible to go by relatively small steps from the generalist journals to the philosophy of science journals, but not to the ethics journals. A large part of the explanation here is that Synthese exists as a bridge between the two, but no similar journal exists for bridging the ethics journals to the generalist journals. Economics and Philosophy sort of functions as such a bridge, since it connects to the political philosophy journals and the philosophy of science/formal epistemology journals, but it’s too small. Given the important of ethics-and-epistemology to young philosophers these days, I suspect that situation will change in the next few years.

There ends up being something like a category of ethics, aesthetics and history journals in the data set I have. This is not because these journals are all intrinsically similar. It is rather that they are all linked to Kant Studien.

Because Mind is linked to the other UK/Australian journals, and to Mind and Language, the UK/Australian journals ended up nearer to the specialist journals than the North American journals did. If I get a chance, I’d like to write more about the geographic patterns in the journals, because these are fairly interesting to me.

Then I had to colour code the journals. I went through a lot of options here before settling on what you see. I wanted nearby journals to get similar colours, while different categories to get very different colours, and the whole thing to not look terrible. And I would have liked to have very different colours for each journal, but I ended up having to seriously compromise on that. I landed on green for generalist journals, going through blue-ish greens for specialist journals in areas the generalists cover a lot, into darker blues and purples for philosophy of science, then jumping to reds for ethics and political, and oranges for aesthetics and for history. I alternated light/dark colours around the circle, but the light/dark doesn’t mean anything; it was just to make it easier to detect edges.

And I fed that table into Circos. The result is what you saw. Here’s how to read it.

Between each pair of (distinct) journals, there are a pair of lines. Each line starts on an outer ring and ends on an inner ring. The outer ring is the journal being cited, the inner ring is the journal doing the citing. The colour of the line is the colour of the journal being cited.

Around the edges there are three arcs, each with an array of colours. These represent the outbound citations, the inbound citations, and (on the outside) the sum of these. They are ordered by size. If the colours were more distinct, you could easily see which journal the particular journal interacts with the most. As it stands, looking for the purples, reds and browns on those arcs gives you a bit of a sense with how much the journal interacts with philosophy of science, with ethics, and with aesthetics/history. (Those colours usually come way towards the end of the arc, though Synthese obviously has more purple towards the top end.)

That’s about enough, I think, to show what’s going on. I have four big projects going forward.

  1. Building graphs that just highlight specific journals. It’s impossible to make out anything about the history journals, for instance, at the scale shown here. So I’ll in effect do some magnification.
  2. Building graphs (and perhaps gifs) that show the evolution over time of citation patterns. I might go back to some old fashioned line graphs to show the change in citation to various journals, and how much more egalitarian it has become.
  3. Looking at ways to highlight the geographical features of the citation patterns. This is something I’m really fascinated by.
  4. Figuring out the best way to normalise the data to account for the fact that some journals are bigger than others. I have some ideas here, but it’s a non-trivial challenge.

Citations of History Journals

Yesterday I posted some data about how often journals cite other journals. And one of the things that jumped out was how rarely generalist journals cite various specialist journals. I suggested an inference from that: generalist journals are not as generalist as they are often thought to be. And I think that’s largely correct, and I’ll be presenting more data on it over upcoming days and weeks.

But today I want to issue a caveat. I don’t think we can draw many inferences about a journal’s coverage of history from who it cites.

I started out looking through recent editions of the Philosophical Review to see how many history articles it had published. I’ve been an external editor for the Review for several years, so I’m somewhat biased here, but I thought the coverage of many areas of history of philosophy was decent. And indeed, it seems there have been several articles in the 2010s that are naturally classified as history articles.

The list includes (and I may have missed some):

  1. Deliberation as Inquiry: Aristotle’s Alternative to the Presumption of Open Alternatives, Karen Margrethe Nielsen, 2011
  2. Kant’s Conception of Number, Daniel Sutherland, 2017.
  3. Kant’s First Paralogism, Ian Proops, 2010.
  4. Leibniz and the Foundations of Physics: The Later Years, Jeffrey McDonough, 2016.
  5. Leibniz and the Ground of Possibility, Samuel Newlands, 2013.
  6. Leibniz and the Puzzle of Incompossibility: The Packing Strategy, Jeffrey McDonough, 2010.
  7. Locke’s Simple Account of Sensitive Knowledge, Jennifer Smalligan Marušić, 2016.
  8. Minds, Composition, and Hume’s Skepticism in the Appendix, Jonathan Cottrell, 2015.
  9. Russell on Substitutivity and the Abandonment of Propositions, Ian Proops, 2011.
  10. Sidgwick’s Axioms and Consequentialism, Robert Shaver, 2014.
  11. Space as Form of Intuition and as Formal Intuition: On the Note to B160 in Kant’s Critique of Pure Reason, Christian Onof and Dennis Schulting, 2015.
  12. Substance and Independence in Descartes, Anat Schechtman, 2016,
  13. Theories about Consciousness in Spinoza’s Ethics, Michael LeBuffe, 2010.

Given how few articles the Review publishes, roughly 12 per year, that seems like a reasonable number of history papers. If one wanted one could quibble over the distribution – it’s very concentrated on Big Names from Modern Philosophy. But now we’re getting into deep questions about what a generalist journal with few articles per year could even look like. The main thing to note is that 13 history articles out of the roughly 100 we’ve published since 2010 is a reasonable number.

So how do these publications show up in the citations? Well, I went back and looked at the articles Web of Science lists as being cited by one of these 13 publications. And it’s a bit of a mess, because of the weaknesses of the Web of Science database. But as far as I can tell, here are the journals cited 3 times or more across those 13 articles.

Journal Cites
Philosophical Review 18
Journal of the History of Philosophy 12
Archiv Fur Geschichte Der Philosophie 8
Pacific Philosophical Quarterly 6
Nous 4
Philosophical Studies 4
Synthese 4
Hume Studies 3
Kant-Studien 3
Kantian Review 3
Mind 3
Studies in History and Philosophy of Science 3

And those 86 citations really wouldn’t make a big dent in the graphs I’ve been using. The 18 citations to the Review itself would get thrown out, because I’m largely ignoring within journal citations. Of the other 68, only 29 are to dedicated history journals, and several of those I wasn’t even including in the research I did.

So this is just a limitation of the kind of study I was doing. I don’t think it tells us a lot about about history journals. It tells us something. CJP and PPR do cite JHP much more than other journals, and that’s roughly what you’d expect. But it’s possible for a journal to have very reasonable history coverage, just as much as you’d expect a generalist journal to have, and that not show up a lot in how much it cites specialist history journals.

Could it also be true that a journal to have very reasonable ethics coverage, just as much as you’d expect a generalist journal to have, and that not show up a lot in how much it cites specialist ethics journals? That is also possible, though I think it’s a bit less likely than in history. But since it is possible, that should put a limit on how strong a conclusion one draws from the data I posted yesterday.

Citation Patterns Across Journals

I’ve been interested for a while in the different things that get attention in different philosophy journals. Part of what got me interested in this was looking at the ways that different well known articles get cited, or not, in different journals. Below is a table showing how many times (according to Web of Science), six prominent turn of the century articles were cited. The articles are:

  1. Elizabeth Anderson, What is the Point of Equality, Ethics, 1999
  2. Peter K. Machamer, Lindley Darden & Carl F. Craver, Thinking about Mechanisms, Philosophy of Science, 2000.
  3. David Lewis, Causation as Influence, Journal of Philosophy, 2000.
  4. Michael G. F. Martin, The Transparency of Experience, Mind and Language, 2002.
  5. James Pryor, The Skeptic and the Dogmatist, Noûs, 2000.
  6. Jason Stanley & Zoltán Gendler Szabó, On Quantifier Domain Restriction, Mind and Language, 2000.

Here are the 21 journals these six papers have (collectively) been most cited in, along with a count of how often they were cited in each.

Journal 1 2 3 4 5 6
Phil Studies 3 7 15 19 51 19
Synthese 0 50 8 4 23 9
Phil of Science 0 69 4 0 0 0
PPR 1 4 5 10 21 5
Erkenntnis 0 17 13 2 7 4
Mind & Lang 0 1 1 5 1 28
Biology & Phil 0 30 4 0 0 0
AJP 0 1 8 4 11 6
BJPS 0 17 9 0 1 1
Nous 1 1 7 3 6 7
Ethics 21 0 0 0 2 0
Analysis 1 0 3 2 5 10
Phil Psych 1 16 1 2 1 0
J Phil 4 1 10 0 3 1
Mind 2 0 0 7 5 5
J of Pol Phil 16 0 0 0 0 0
Phil Review 1 0 6 1 6 2
CJP 5 0 3 0 3 4
Econ and Phil 12 3 0 0 0 0
EJP 1 0 0 8 5 1

A few quick notes about this data.

  • It all comes from Web of Science, and as I’ve discussed previously, there are flaws with that data. I think the flaws don’t make a huge difference to the points I’m making, but they exist.
  • I’ve tried to sort the citations to Nous from the citations to Philosophical Issues and Philosophical Perspectives, which Web of Science sometimes (but only sometimes) collapses.
  • These are the journals that most cited the six articles in Web of Science. If a journal that Web of Science doesn’t index cited the six of them 15 or more times, I wouldn’t know about it. And Web of Science added a lot of journals in 2008, so even some journals it now indexes won’t appear there.

The last point affects one thing that might jump out at you from that list, or at least jumped out at me. These are almost all pretty high prestige journals. And, conversely, almost all the high prestige journals are on the list. (Philosophical Quarterly is probably the most notable omission, relative to my personal ranking of journals.) You can imagine a world where the really significant articles of 15-20 years ago are discussed much more in lower ranking journals, perhaps because they are mimicking what was going on in elite journals some time before. A priori, I wouldn’t have been surprised to find we were in such a world. But this is some (weak) evidence against it.

But what I really want to stress is how uneven the pattern is here. The two most cited articles of those six, by a lot, are the Anderson, and the Machamer et al. Here are the citations each has on Google Scholar, and on Web of Science as of today. (These numbers are sure to change in the future.)

Article Google Cites Web of Science
Anderson 2104 552
Machamer et al 1844 668
Lewis 703 183
Martin 490 163
Pryor 768 268
Stanley and Szabó 777 239

So in terms of their impact on the journals collectively, those two articles have each been something like three times more influential than the other four articles. And those ‘other four articles’ rank somewhere between essential and field-defining in their importance to their subfields. And both the Anderson, and the Machamer et al, articles are in fields that I thought were pretty central to contemporary philosophy: political philosophy and philosophy of science. So they should be cited a lot in the ‘generalist’ journals, right? Right? Well, here’s the table above, this time with articles 1-2 collapses and articles 3-6 collapsed, and just the ‘generalist’ journals listed, and with the sort by the number of times articles 1 + 2 were cited.

Journal Cites to 1 + 2 Cites to 3-6
Phil Studies 10 104
PPR 5 41
J Phil 5 14
CJP 5 10
Nous 2 23
Mind 2 17
AJP 1 29
Analysis 1 20
Phil Review 1 15
EJP 1 14

It’s only six articles, and I guess anything can happen in a small sample, but it was enough to suggest to me a hypothesis.

There is no such thing as a generalist philosophy journal.

How could we test this? Here was one way that I thought of. Philosophy has journals that are recognised as specialist journals. Some of them, like Ethics and Philosophy of Science are roughly as high status as the generalist journals. We could measure how often articles in each journal cite other journals. If there was a generalist journal, it should routinely cite the high status specialist journals, and it should be routinely cited in both generalist and specialist journals.

So to that end I decided, again using Web of Science, to make a giant database of which journals cited which other journals. To avoid the problem that Web of Science can be a bit erratic in when it updates, I decided to focus on citations where both the cited article and the citing article were published between 1976 and 2015. And I focussed just on journals that Web of Science had indexed throughout that range, or (if they started after 1976) which Web of Science indexed from their foundation.

I chose 30 journals that were a mix of generalist journals and specialist journals. In part the choice was influenced by what Web of Science had available. And in part it was based on a deliberate over-sampling of specialist journals, because what I really cared about is the interaction between generalist and specialist journals. Here is the list of 30 I ended up with.

Abbreviation Journal
BJA British Journal of Aesthetics
JAAC Journal of Aesthetics and Art Criticism
JHP Journal of the History of Philosophy
PHR Phronesis
KS Kant Studien
JPP Journal of Political Philosophy
PPA Philosophy and Public Affairs
ETH Ethics
BP Biology and Philosophy
PSCI Philosophy of Science
BJPS British Journal for the Philosophy of Science
SYN Synthese
EP Economics and Philosophy
RSL Review of Symbolic Logic
JPL Journal of Philosophical Logic
RM Review of Metaphysics
EPI Episteme
LP Linguistics and Philosophy
ML Mind and Language
Mind Mind
AN Analysis
PQ Philosophical Quarterly
AJP Australasian Journal of Philosophy
PS Philosophical Studies
PPR Philosophy and Phenomenological Research
Nous Nous
PR Philosophical Review
JP The Journal of Philosophy
PPQ Pacific Philosophical Quarterly
SJP Southern Journal of Philosophy
APQ American Philosophical Quarterly
CJP Canadian Journal of Philosophy

In part because it was easier to collect the data this way, and in part because I think it is a more meaningful measure, I didn’t count cites of more than one article from a journal in a given year, in a particular citing article. So if an article cited both my Intrinsic Properties and Combinatorial Principles, and David Lewis’s Redefining ‘Intrinsic’, both of which are PPR 2001, that would count as citing PPR once. I think that’s the right way to measure things; otherwise we’ll end up treating engagement with a symposium as a bigger deal than it really is.

And, as mentioned above, I separated out the Nous citations from Philosophical Issues and Philosophical Perspectives.

I fed this data into the (amazing) Circos Table Viewer, and here’s the first of the outputs. (Click on any image to get a higher res version.)

Between each (distinct) pair of journals, there are two lines, showing how often they cite each other. The lines go from the outer circle to the inner circle, and represent how many times the journal on the outer circle is cited by the journal on the inner circle. So you can see at a glance, for instance, that among this group, the Journal of Philosophy is cited many more times than it cites other journals, and the Canadian Journal of Philosophy cites other journals (in this group) much more often than it is cited by them.

The graph is a bit of a mess, so I tidied it up in two ways. First, I removed 8 journals that weren’t adding a lot to the graph, because they had very similar patterns to other journals that I included. (The 8 are BJA, PHR, KS, JPP, EP, RSL, RM and EPI.) We’ll come back to those 8 in a bit, but for now I want to focus on the 22 remaining. Second, I used Circos’s feature that lets you cut out the thinnest 50% of the lines, so we can focus a bit more on what’s remaining. Here’s the result:

And we see a large scale version of what the six articles above suggested. With perhaps the exception of Philosophical Studies, there are very few journals that have strong connections to both the leading political philosophy journal, Philosophy and Public Affairs, and the leading Philosophy of Science journals, Philosophy of Science and BJPS. What really surprised me was how weak the links to Ethics are. I’m not surprised that some generalist journals don’t publish much aesthetics, feminist philosophy or history of philosophy: I knew those were gaps in coverage. I was surprised at how little philosophy of science, political philosophy, and even ethics, there seems to be in some leading journals.

I’ll write more about this in future posts, including deeper dives into which journals do and don’t interact with which specialist journals, and some important caveats to the no generalist journals conclusion. (Spoiler alert: Past performance is not necessarily a guide to future results.)

And I’ll do a long look at how the journals have changed over time. The four takeaways will be:

  1. The Journal of Philosophy was unbelievably influential in the 1980s, but is now merely one leading journal among many.
  2. Stewart Cohen has done an incredible job with Philosophical Studies. (This is a bit of a recurring theme actually.)
  3. David Lewis gets cited a lot. It ends up being important to remember when the Lewis articles appear when getting a sense of the impact of various journals across time.
  4. The interactions between generalist and specialist journals don’t change a lot over time, but they do change a bit, and some of the differences are worth pausing over.

FOLLOW UP POSTS (this will be edited as I add more)

Come to Michigan!

It’s the time of year when students thinking of applying to PhD programs have to decide where to apply, and faculty need to advise these students on applications. So I thought I’d add a few words about the ways in which my department (Michigan philosophy) has changed over the last few years in ways that are relevant to these students. A bunch of things have happened that make us a much more attractive destination than we were a little while back for a large class of students. (I won’t say much about the ways in which this place continues to be great in ethics, and in epistemology, and in philosophy of language, and in philosophy of physics, and in many other fields. The focus here is on new strengths.)

One is that it is now a really great place to do work on race and gender. Obviously that’s always been possible, with Liz Anderson here. But having hired Ishani Maitra, Derrick Darby and Meena Krishnamurthy in the last few years, it’s now as good a place as any to do work at the intersection of analytic philosophy with work on race and gender. The department has (and has had for a few years now) an active Minorities and Philosophy chapter, and a (graduate student run) feminist philosophy and philosophy of race reading group. There is a good critical mass of students interested in philosophical issues about race and gender, which makes it (I think) a good environment to work in. And you see flow on effects from that; lots of students, and faculty, who don’t primarily work on race and gender nevertheless have active research programs in those fields. So there are lots of opportunities for feedback on one’s work.

We currently have 23 full-time faculty. We may expand this year, either due to hiring through the Collegiate Postdoctoral Fellowship Program, or through our search (joint with Asian Languages and Culture) for a new faculty member in Chinese Philosophy. But there are a lot of supports for philosophy here beyond the department. Michigan is in many ways a very theory-friendly university, so there are many departments that have faculty with philosophical interests. Some of those faculty have cross-listed appointments in philosophy. Chandra Sripada, whose tenure home is in psychiatry, is quarter-time here. And Scott Hershovitz and Gabe Mendlow in the law school both do a lot of philosophical work. A number of our students are working with Scott on various projects, and Gabe is teaching a graduate seminar in our department next term. When you add in the faculty in our department with interests in philosophy of law, we’re a very strong place for students with legal and philosophical interests. (Not coincidentally, we have a lot of graduate students who have either completed law degrees, or are completing law degrees while doing the PhD.)

The department is more diverse demographically than it used to be. Four of our last five hires have been non-white philosophers. (I’m the fifth.) Three of the last five hires have been women, so now the full-time faculty is over 1/3 women. And the graduate student body is a lot more diverse, along many dimensions, than was typical for an analytic philosophy department not too long ago. This is not to say that we are as diverse as, say, the psychology department here. But the trends are promising.

The university has a fellowship, the Rackham Merit Fellowship, that aims to promote diversity by “encouraging the admission and funding of students who represent a broad array of life experiences and perspectives”. This program has been around for a while, but some recent changes to its administration have meant that it has become much more accessible to philosophy students. Indeed, in the last two years we have recruited three students who were awarded these fellowships. The fellowships only go to students who had been already admitted to the program, but they result in extra funding, fellowship time and resources. So they make UM a more attractive destination for students from a range of non-traditional backgrounds.

Note that by state law, the fellowships can’t be awarded on the basis of race or gender. And they are not just intended for promoting racial diversity. They are also for first-generation immigrants, first-generation college students, community college graduates, students from financially disadvantaged backgrounds, students from geographic areas that are under-represented in universities, and so on. So there are a lot of different students who benefit from the program.

And one thing I didn’t realise about UM before I got here is that there is a lot of encouragement of inter-disciplinary work at the graduate level. Graduate students across the university are required to do some of their coursework outside their home department, and to have a faculty member from outside their home department on their thesis committee. This means we have more non-philosophy students in our seminars. And it means that our students get to take classes with faculty like Susan Gelman and Catherine Mackinnon. There is now a graduate certificate in Cognitive Science, and our students teach in Cognitive Science and in Psychology. So it’s a great place to do inter-disciplinary work.

So there has been quite a bit happening here. For what it’s worth, I’m sure we’re not the only department that has been rapidly changing. Especially if you are advising a prospective graduate student on where to apply, it’s worth taking some time to get up to date information on what different departments are like right now, and how they might or might not be suitable to your student’s interests. Hopefully many of these students will apply to, and end up working at, Michigan.

A Correction

In a previous post I said that the study of Shakespeare was well outside the bounds of philosophy as it is practiced, though it easily have been inside. This was a mistake. Klaas Kraay pointed out to me that there is even an upcoming conference on Shakespeare: The Philosopher.

I’m very happy to have been proven wrong about this. Shakespeare’s connection to philosophy seems like a rich and interesting field of study, and I’m thrilled to see people working on it.

It is a little interesting that the conference doesn’t look like it is growing out of work in history of modern philosophy, or even history of Renaissance philosophy, but out of aesthetics. That wasn’t what I expected either, though perhaps I should have. I suspect in general there are interesting connections to be drawn between the work of the leading poets, playwrights and, eventually, novelists. I wonder if we’ll think of work looking at those connections as being part of aesthetics, or part of history of philosophy? Either way, it’s wonderful to see this kind of work being done.