I’m generally sceptical of the value of surveys, as currently conducted by practitioners of ‘experimental philosophy’ as a way of getting clear about what’s going on in philosophically interesting thought experiments. The most systematic reason for this scepticism comes from thinking about what exactly is going on in thought experiments.
Following Jonathan Ichikawa and Ben Jarvis, I think examples in philosophical works should be thought of as small fictions. In particular, they’re a type of genre fiction. The genre is the same one, broadly, that fables and parables fall into. (This way of thinking about thought experiments makes Aesop an important figure in the Western philosophical canon, which isn’t a bad result I think.) Like any kind of genre fiction, there are important interpretative constraints on these fictions. If you present the story in a different ‘mode’, it should be, and will be, interpreted differently.
The way in which fables/parables/thought experiments should be interpreted has some particularly quirky features.
At least some of the time, realism isn’t important. We don’t object to Aesop’s stories because they feature talking animals. We don’t object to parables if the story doesn’t really make sense from the perspective of non-central characters. (The striking effect we get when retelling a familiar biblical or mythical story from the perspective of a non-central character is something Nick Cave has made good use of over the years.) And we shouldn’t object to thought experiments because they require a curious series of coincidences. Indeed, the best fables/parables/thought experiments are often quite unrealistic because everything about the back story is so ‘neat’. They don’t have characters like Ulysses’ M’Intosh who simply don’t fit into the story, and people simply know, in a way that doesn’t leave the possibility of doubt even open, the things that are stipulated as true.
These stories are meant to have a point, or a moral. The intended point guides interpretation of the story. We’re meant to interpret the story in a way that makes it a fitting illustration of the moral. It would be wrong to interpret the story of the fox and the grapes as one in which the fox gets evidence that the grapes are sour and therefore leaves. And that would be wrong in part because the point is about our attitude towards what we cannot have.
The same thing is true in philosophical experiments I think. It would be wrong to interpret the Gettier example as one in which the subject has independent evidence for the justified true belief that isn’t known, or in which they aren’t justified in inferring the target proposition because the evidence for it is from a source they have independent reason to doubt. It isn’t even really necessary to state this in the example, because once we know it’s an attempt to show that justified true belief without knowledge is possible, general principles of interpretation will fill in the details.
But that means we have to know what principle the example is meant to show. And that’s why I suspect there’s a deep problem here for experimentation on the examples. We suspect that telling people the point that an example is meant to show will seriously interfere with how they evaluate the example. (I assume this is why subjects in existing surveys are not normally told what hypothesis is being tested.) But not telling people the intended point of the example will interfere with how they interpret the example.
I suspect the best way out of this problem is to investigate people who do know the intended point of the example, and hence who know how it is meant to be interpreted. But I don’t think they are suitable subjects for a controlled experiment.
This point is related to the worry that it takes a bit of training to be able to distinguish between different hypotheses that the thought experiment might be intended to show. But I suspect it goes a little deeper. In principle we could explain the distinctions without telling people the intended outcome of the experiment, and hence ‘contaminating’ it. Not so if the intended outcome is an essential part of interpreting the story being told.
Having said all that, I want to strongly agree with something that Alan White said in the previous thread. Experimental work on subjects who aren’t familiar with the debate can tell us a lot about how people interpret these thought experiments. And that can be incredibly useful for communicating the results and arguments, either to colleagues in other disciplines, or to students. I know that’s not what experimental philosophers are aiming to show, but I think it’s a valuable side-effect of their work. Indeed, if I’m right about thought experiments being genre fictions, and the genre being one philosophical training makes you much more familiar with, we’re probably all making lots of mistakes about how people interpret our examples. Experimental work is an incredibly good way of correcting these misimpressions.
To be sure, I know that experimental philosophers are aiming much higher than merely clarifying philosophical examples. But I think those of us who are sceptical of some experimental work should not overlook its values that aren’t affected by extant criticism.