Skip to main content.
12 July, 2013

Update on Journal-to-Journal Citation

In the previous post, I mentioned some problems with Web of Knowledge. It might be best to say what these are more explicitly. (I’m grateful here to work Neil Levy did.)

That said, Web of Knowledge is not bad for citations in top journals of articles in top journals, provided you allow for some lag time. And that’s why I’ve focussed on journal-to-journal citation.

I’ve had some requests for looking at other relatively general journals. I actually wasn’t originally that interested in them; I was interested in the specialist journals, and wanted the more general journals as baseline. But for interest, here are the numbers for PPR.

Journal of Philosophy: 31
Mind: 41
Philosophical Review: 34

Brian Leiter summarised the data as saying these three journals cite other journals not a lot. I think that’s half-true; these three journals cite other journals with similar missions (e.g., Philosophical Studies, or Philosophy and Phenomenological Research) plenty. What they don’t do is cite the specialist journals, even ones on issues that I would have thought were pretty important parts of philosophy, such as feminism and legal theory.

One last data point about self-citation, because I really want to drive home the point that this isn’t something distinctive to the top journals. The number of articles in Philosophical Quarterly that cite other articles from 2000-2013 in Philosophical Quarterly is 106. Over the same period, it cites the Journal of Philosophy 14 times, and Philosophical Review 11 times. This is in part because, like Mind, Philosophical Quarterly has an active, and very good, discussion section. (One that I’ve been happy to publish in.) It is also in part because it has established itself as a home for certain discussions (especially about anthropic principle type reasoning), and these articles cite the leading articles in this discussion, which of course are in the Quarterly. There’s nothing particularly nefarious going on here, it’s part of running a good journal.

Posted by Brian Weatherson at 11:37 am

No Comments »

11 July, 2013

How often do journals cite other journals?

I’ve been playing around with the data about citation patterns in top journals of recent articles, and I started to think that a lot of it was very noisy. So I was looking for ways to aggregate the data that would be revealing. I decided to look at how many times recent articles, that is articles from 2000 onwards, from different journals were cited in each of the ‘top 3’ philosophy journals – Journal of Philosophy, Mind and Philosophical Review. A few disclaimers before I start.

That said, here are the data. I’ll leave commentary on the data for another time, because they are interesting enough on their own. I would be very interested in hearing in the comments for suggestions for other journals that might suitably be analysed this way.

Citing Journal
Source Journal of Philosophy      Mind      Philosophical Review
Journal of Philosophy 74 33 26
Mind 19 177 24
Philosophical Review 24 32 48
Philosophical Studies 57 56 35
Philosophical Quarterly 14 32 11
Australasian Journal of Philosophy 16 20 15
Ethics 15 16 12
Philosophy and Public Affairs 10 3 3
Philosophy of Science 21 8 10
BJPS 14 14 8
Linguistics and Philosophy 6 13 13
Mind and Language 12 15 5
Journal of Philosophical Logic 3 16 2
Journal of Symbolic Logic 1 3 1
Hypatia 0 0 1
Journal of Aesthetics and Art Criticism 1 2 2
Law and Philosophy 1 0 0
Legal Theory 1 1 0

UPDATE 7/12: I’ve added some more thoughts in a new post.

Posted by Brian Weatherson at 11:35 am

5 Comments »

10 July, 2013

Three Links

Posted by Brian Weatherson at 4:09 pm

No Comments »

3 July, 2013

Self-Citation within Journals

One of the things to look for when doing citation searches is self-citations. If Joe Bloggs’s 1999 masterpiece has lots of citations, but it turns out that’s only because Bloggs has been citing Bloggs 1999 in everything he writes, then it isn’t clear that citation counts are measuring impact correctly. For that reason, many measures of citation exclude self-citation by authors. There are complications with this, especially with co-authored pieces. (If a paper by Stanley and Williamson cites Stanley and Szabo 2000, is that a self-cite? Should it be, given the rationale for excluding self-cites?) But overall I think it gives a better measure of citation impact.

We can ask the same question about journals. How much is the work in journal X being cited overall is a different question from how much it is being cited elsewhere. To get a sense of how big an issue this is, we can look at how frequently the journals publish articles citing other articles in that very journal. And that’s what I did. The table below counts how many articles in each of the big 3 journals cite articles published in the big 3 journals from 2000 to the present.

A few caveats before the table. I’ve taken this from Web of Knowledge, because I couldn’t figure out how to do the same thing with Google Scholar. I haven’t included Nous, partially because I’m lazy, and partially because of technical challenges. Perhaps most importantly, Web of Knowledge includes book reviews in Mind as articles, but not book reviews in Philosophical Review. That means the raw number of articles in Mind is much greater than in the other 2. (Roughly 600 articles in Mind, versus under 200 in Philosophical Review, and 350 in Journal of Philosophy.) That said, here’s the table.


Source\Citing Journal J Phil Mind Phil Review
J Phil 74 33 26
Mind 19 177 24
Phil Review 24 32 48

As you can see, journals cite themselves a lot. The huge number in the middle in part reflects Mind’s love of discussion notes, replies to discussion notes, and so on. (And in part reflects the odd fact that book reviews in Mind are counted in Web of Knowledge.) But don’t let that distract you too much; every journal is like this to some degree.

Of course, this is what you’d expect if the journals specialised. If article authors think that discussions about conceivability and possibility go to Philosophical Review, about conditionals go to Mind, and about the nature of function go to Journal of Philosophy, and they think that on the excellent ground that that’s where such articles in the past have gone, then of course those articles will end up citing largely other articles from the same journal. (This fact, that people send articles on X to places that have recently published on X, I think explains the bulk of the data.)

It’s also what you’d expect if either (a) editors demanded that authors recognise their perceptiveness in what they had previously chosen to publish by citing it well, or (b) authors believed that editors demanded this, whether or not they do.

And it’s also what you’d expect if editors of journal X were aware that a paper previously published in X was relevant to some new article, and properly insisted on it being cited, while they were not as aware of (or not as insistent about) relevant literature in journals X and Y.

Figuring out which of these three explanations – or others – is correct is for another time. I just wanted to note the data.

One last point. I started to do this because I’d been asked whether the odd data in the previous post (about the Journal not citing some classic papers in recent philosophical semantics) was part of a broader trend of the Journal not citing recent literature as much as the other top journals do. I don’t think this data alone either confirms or disconfirms that hypothesis. It’s true that the Journal cites the other top 2 journals at a lower rate than either Mind or Phil Review cite the other top 2 journals. But it’s not by much, and this could be a consequence of the Journal having (either because of editorial policy or submission patterns by authors) a different range of topics than the other top 2.

Posted by Brian Weatherson at 11:35 am

No Comments »

1 July, 2013

How Journal of Philosophy is Different

Three of the most important papers in the philosophy of language/semantics overlap in recent years are:

Between them they have nearly 1500 Google Scholar cites, and nearly 500 Web of Knowledge cites. They appear in Kieran Healy’s dataset 57 times between them. You might suspect that would mean that they would have to appear prominently in all four of the journals Kieran looks at. You’d be wrong.

According to Web of Knowledge, those papers have been mentioned, between them, exactly once in the Journal of Philosophy. In Jason Turner’s Ontological Pluralism there is a mention, in passing, of the Stanley and Szabo paper. The other 56 citations in the Healy dataset come from the other 3 journals he looks at.

According to Google Scholar, things have recently gotten better. There has been one more citation of one of these papers. In Andreas Stokke’s Lying and Asserting, published in the next-to-most-recent issue of the Journal, there is a mention, among a list of dissenters from radical contextualism, of Stanley’s Context and Logical Form. I assume that’s too recent to make the latest update to Web of Knowledge. (Neil Levy pointed out in comments to the previous post that Google Scholar often gives more accurate citation counts than Web of Knowledge. He’s right, especially about recent publications, and that prompted the search which led to this paragraph. Thanks Neil!)

But that’s it. There are no papers, at all, from the tradition of formal philosophy of language that Stanley and Szabo are writing in who cite their work in the Journal. There is no one at all who cites Edgington. By contrast, her paper has been cited in Mind at least 15 times that I found, and I possibly didn’t find all the cases.

Obviously any journal will have areas it thinks are less interesting, or less important, than other journals do. That’s a good thing. But I think this particular gap in the Journal‘s coverage of philosophy is unfortunate.

Posted by Brian Weatherson at 11:07 pm

1 Comment »

How the Top 4 journals are different

If you look at which philosophers have made the most impact by publishing in refereed philosophy journals over the last 10 years, it is easy to conclude that the one with the most impact, across academia, is Joshua Knobe. He not only has some very highly cited articles, he has a lot of articles that are cited many times. Indeed, his Web of Knowledge h-score for the last decade is, I believe, 11, which is nearly as good as some journals.

The only place Knobe’s impact doesn’t show up is in the Top 4 philosophy journals. He’s been cited 5 times in Journal of Philosophy, twice in Philosophical Review (both in the last year), 4 times in Nous (twice in his own articles), and never in Mind.

That’s not because he is only getting cited outside philosophy. He has 20 citations in Philosophical Studies, 19 in Mind & Language, 9 in Analysis, 8 in PPR, and so on. It is something very distinctive about those four journals that means work like Knobe’s isn’t getting cited there.

Posted by Brian Weatherson at 3:45 pm

7 Comments »

26 June, 2013

h-scores of Philosophy Journals

A quick and dirty (and probably inaccurate) count of the h-scores of various philosophy journals over the last 10 years (i.e., for papers published 2004 and later). This is the largest n such that the journal has published at least n articles cited at least n times since 2004. (I thought this might be relevant to this poll.)

Mind & Language – 24
P&PA – 19
Philosophy of Science – 18
BJPS – 18
Philosophical Studies – 18
Ethics – 18
Journal of Political Philosophy – 17
Nous – 16
PPR – 16
Linguistics & Philosophy – 16
Phil Review – 15
JPhil – 14
Mind – 14
PQ – 13
AJP – 12
Economics & Philosophy – 11
Erkenntnis – 10
Monist – 7
APQ – 8
CJP – 7
Journal of the History of Philosophy – 7
Review of Metaphysics – 3

Posted by Brian Weatherson at 3:42 pm

2 Comments »

Most Cited Articles from Philosophy Journals

In my last post, I expressed some surprise at which articles were turning up a lot in the citations. So I got to wondering what other surprises would be in store if I looked at the data some more. It turns out my view of what’s been making a splash was a bit of a way from reality.

I used the Web of Science database, restricted to the Arts & Humanities to find the most cited journal articles over various time periods from philosophy journals. I don’t think this list is complete; it doesn’t seem to include Philosophical Topics, for example, which might explain why “Shifting Sands” isn’t any list here. And I searched manually for philosophy journals from the lists of most cited articles over various periods.

This last step involved a lot of contentious decisions. I used a very narrow definition of a philosophy journal. Excluded were Behavior and Brain Science, Journal of Pragmatics, Language, Linguistic Inquiry, Journal of Consciousness Studies, Critical Inquiry, Cognition, Signs, Feminist Economics (which has a philosophically important and widely cited Nussbaum article), Trends in Cognitive Science, and several others. I included Mind and Language, and Linguistics and Philosophy, though I could see a case for excluding them too.

The citation counts Web of Science gives are much lower than you might be used to if you use Google Scholar. In general they are about 1/3 to 1/4 the size. But I think they are more reliable; they don’t include references to drafts, lecture notes, etc.

Although I’ve been fussy about what articles were in philosophy journals, I included all citations. So some of these articles pick up a lot of their citations from outside philosophy. That said, here are the lists. I’m going to list the categories 1993-1999, 2000-2002, 2003-2005, 2006-2007, 2008-2009 and 2010-present. You might like to guess which you think the most cited articles from each of those periods will be before peeking at the lists.

Read the rest of this entry »

Posted by Brian Weatherson at 12:59 pm

1 Comment »

24 June, 2013

Thoughts on Citation Data

As you probably have seen, Kieran Healy has done some amazing work extracting and presenting data about the citation patterns in Philosophical Review, Mind, Nous and Journal of Philosophy over the past 20 years. The point of this post is to pull out the data about work published in the last 11 years, and make a few brief comments on that work.

Here are the journal articles since 2002 that have 10 or more citations in some combination of Philosophical Review, Mind, Nous and Journal of Philosophy. They are presented in reverse chronological order:

“The Limits of Self-Awareness”, M. G. F. Martin, Philosophical Studies, 2004 (14 cites)
“The Silence of the Senses”, Charles Travis, Mind, 2004 (12 cites)
“What Conditional Probabilities Could Not Be”, Alan Hajek, Synthese, 2003 (12 cites)
“Some problems for conditionalization and reflection”, Frank Arntzenius, Journal of Philosophy, 2003 (10 cites)
“Color Realism and Color Science”, Alex Byrne and David Hilbert, Behavior and Brain Sciences, 2003 (13 cites)
“Assertion, Knowledge and Context”, Keith DeRose, Philosophical Review, 2002 (15 cites)
“Evidence, Pragmatics and Justification”, Jeremy Fantl and Matthew McGrath, Philosophical Review, 2002 (15 cites)
“The Transparency of Experience”, M. G. F. Martin, Mind, 2002 (14 cites)
“Basic Knowledge and the Problem of Easy Knowledge”, Stewart Cohen, PPR, 2002 (10 cites)
“(Anti-)Sceptics Simple and Subtle: G.E. Moore and John McDowell”, Crispin Wright, PPR, 2002 (10 cites)

And then there’s several more articles once we start going back to 12 or more years ago.

Here are the recent-ish books that have appeared more than 10 times in these four journals.

A Virtue Epistemology: Apt Belief and Reflective Knowledge, by Ernest Sosa, 2007 (12 cites).
The Philosophy of Philosophy, by Timothy Williamson, 2007 (11 cites)
Knowledge and Practical Interests, by Jason Stanley, 2005 (26 cites)
Insensitive Semantics, by Herman Cappelen and Ernest LePore, 2005 (11 cites)
Knowledge and Lotteries, by John Hawthorne, 2004 (36 cites)
Action and Perception, by Alva Noe, 2004 (15 cites)
The Realm of Reason, by Christopher Peacocke, 2004 (13 cites)
Evidentialism: Essays in Epistemology, by Earl Conee and Richard Feldman, 2004 (10 cites)
The Things We Mean, by Stephen Schiffer, 2003 (17 cites)
Making Things Happen, by James Woodward, 2003 (16 cites)
A Philosophical Guide to Conditionals, by Jonathan Bennett, 2003 (15 cites)
Thinking How to Live, by Allan Gibbard, 2003 (14 cites)
Reference and Consciousness, by John Campbell, 2002 (18 cites)
Beyond Rigidity: The Unfinished Semantic Agenda of Naming and Necessity, by Scott Soames (10 cites)

A few quick reflections on these lists.

All the entries are by men. I don’t think the best, or even the most influential, books and articles over this time period have all been by men. So this is more evidence in favour of the thesis that there is something wrong with our citation practices as a profession.

I believe that at least three of the papers (Martin 2004, Cohen and Wright) are from conference proceedings, not the usual blind submission to a journal. That’s a little bit fluky – none of the 11 papers on the list from 2000-2001 are conference proceedings. (Though 2 of them were from the special J Phil issue on causation.) Still, it’s striking that in the last 11 years, only 7 papers that were sent to blind review have made it onto this list of most cited entries. It takes a long time to get this kind of recognition.

There are no metaphysics papers. The only metaphysics book is James Woodward’s 2003 Making Things Happen, and you could just as easily classify that as philosophy of science as metaphysics. There is nothing from the material that you see most discussed at recent metaphysics conferences: existence, grounding, meta-ontology, indeterminacy, etc. From 2001 and further back, there are plenty of metaphysics books and papers on the list. The Healy data makes it look like metaphysics is much less central to philosophy than it was in, say, 2000. (This is a reason why I don’t like talking about ‘core’ parts of philosophy; the core changes rapidly.)

There are also no papers in ethics, political philosophy or history of philosopy. If you look at books, you get just one, Gibbard’s Thinking How to Live. If you go further back than 2002, you do start getting more ethics books – including books in normative ethics, not meta-ethics. But the most recent ethics paper I see on the list is David Widerker’s 1995 piece on Frankfurt cases (assuming free will is in ethics, broadly construed, not metaphysics). And I’m not sure I see any others after 1990. In part I think this reflects different subdisciplinary norms; value theorists write books more than, say, epistemologists do. And in part it reflects that the four journals we’re looking at don’t really represent philosophy as a whole. There is so much more ethics and history in the Philosophical Review than in the others, for instance, and even it has less ethics and history than many might like.

The 10 papers also don’t really match up with my internal feel for what’s being central to philosophy over that time, but that’s probably because my ‘feel’ isn’t perfectly reliable. (Although I have seen a few open searches recently, so I have some feel for what grad students are working on.)

The four epistemology papers towards the end of the list have been important, and widely discussed. The Arntzenius paper I believe gets there because one of its key examples, the Shangri La example, is so nice, and so widely discussed.

But the Hajek paper is I think a good example of why citation counts don’t necessarily correlate with what’s being talked about. Al’s paper is great. It is completely convincing, and I often cite it, both in papers and in conversation. But it’s not because I’m talking about that subject. It’s because it points out a common mistake, and it’s a mistake people keep making, and rather than explain why it’s a mistake I simply say “Don’t make that mistake, and read Hajek (2003) on why it’s a mistake.” But it’s not like there’s a rolling conversation about the nature of conditional probability that this represents.

The four philosophy of mind papers seem even less connected to what the bulk of people I see are talking about – though that’s perhaps less true of the Byrne and Hilbert paper, which is obviously central to active ongoing debates about color.

I saw several hundred job applications last year. Not that many of them were in philosophy of mind, despite UM specifically encouraging people with cog sci interests to apply. Those that were in mind typically were doing very different, much more empirical, work than Martin or Travis. That’s in part a selection effect due to our ad. But I didn’t see many of the people in the Leiter hiring thread who were doing this kind of philosophy of mind either. So I don’t really think the fact that the Martin and Travis papers are that often cited is that good a guide to what people are talking about; it might simply reflect how good those papers are, or something about the journals that cite them.

As well as the omissions mentioned above (women, metaphysics, ethics), there are two other surprising omissions from this list.

One is that there aren’t any book chapters represented. I don’t know whether that’s in part because the citations of those are a mess, and they don’t always turn up cleanly in databased, or simply that people aren’t citing them as much as I would have guessed. But given the explosion in edited volumes there seems to have been in recent years, I’m a little surprised that journal articles and books exhaust the list.

The other surprise is that there’s nothing about relativism, or at least nothing pro-relativism. (Relativism is a target, though not the primary target in two of the books shown.) John MacFarlane’s 1580 Google cites aren’t enough to get him into this company. That will change soon enough, especially when he publishes a book, but it’s one striking omission. And a quick search by hand through the Google Scholar citations of the widely cited relativism papers suggests that none of them are near 10 references in these four journals. Relatedly, there are no philosophy of language papers on the list, though there are four books.

The point is that it is a little simple to say that these four journals over emphasize language and epistemology relative to the rest of the profession. They do, but not all language and epistemology is created equal. To a first approximation, what’s widely cited in these four journals is something of a lagging indicator of what people are talking about. And that’s true even if we look at recent work that’s widely cited.

One final omission that perhaps isn’t that surprising: young people. None of these philosophers got their undergraduate degrees in the last 20 years. Only a handful (Cappelen, Fantl, McGrath and Stanley if I’m counting correctly) got theirs in the last 25. (Though there are several others who got degrees between 1988 and 1992 on the list if we include papers published before 2002: Pryor, Paul, Rysiew, Huemer and perhaps some others.) It takes a long time to get this kind of recognition.

Links

Updates: June 25, 1:07 PM

Posted by Brian Weatherson at 3:27 pm

5 Comments »

20 June, 2013

Gordon Belot, “Bayesian Orgulity”

I’ve been thinking over the last few days about my colleague Gordon Belot’s forthcoming paper Bayesian Orgulity. In it he poses a series of very difficult challenges to Bayesianism. I’ve been trying to think about how the imprecise Bayesian can respond to these challenges. (I’m thinking of what response an imprecise Bayesian who thinks all updating goes by conditionalisation could make to Gordon’s arguments. This isn’t my view about updating.)

Here’s one example that Gordon uses. The agent, call her A, is going to get data coming in in the form of a series of 0s and 1s. She is investigating the hypothesis that the data is periodic. Say that she succeeds iff one of the following two conditions hold.

Call the data sequence for which a prior succeeds its success set, and its complement its failure set.

Gordon suggests the following two constraints on a prior:

  1. For any initial data sequence x, there are further data sequences y and z such that (a) the agent will have credence greater than 0.5 that the sequence is periodic after getting x + y, and (b) the agent will have credence less than 0.5 that the sequence is periodic after getting x + z. Call any prior with this property open-minded.
  2. The probability that the agent using this prior will succeed (in the sense described above) is not 1.

Much of the paper is an argument for the second condition. The argument, if I’ve understood it correctly, is that for any open-minded prior, the data sequences for which it succeeds are highly atypical. Its success set is measure 0 and meagre, while its failre set is dense (and obviously the complement of a meagre measure 0 set.)

And, as you might have guessed by now, it is impossible to meet these two conditions as a Bayesian agent. Any open-minded prior gives probability 0 to its own failure set. Gordon argues this is a very bad result for Bayesians, and I’m inclined to agree.

This post has gone on long enough, so I’ll leave how the imprecise Bayesian could respond to another post. I think this is a real problem, and indicative of deeper problems that Bayesians (especially precise Bayesians) have with countable infinities.

Posted by Brian Weatherson at 11:19 am

No Comments »

« Previous Page« Previous Entries  Next Entries »Next Page »