Experimental philosophy, as outlined in this volume,1 is a new approach to philosophical inquiry that seeks to investigate the nature of knowledge, morally responsible action, and other such philosophically interesting phenomena – not, as traditional analytic philosophers would have it, by consulting one's own ‘intuitions’ (i.e. what we’d be inclined to say or think) about whether Gettier cases or trolley scenarios represent instances of these phenomena, but by doing systematic experimentation on what people of different backgrounds, cultures, or educational levels would spontaneously say about such cases, and on whether subtle variations in the descriptions of the cases can significantly influence those verdicts.

In their introductory ‘manifesto’ that begins this volume, Knobe and Nichols note that traditional philosophers’ reactions to experimental philosophy have been ‘polarized’, but argue that all philosophers should embrace this approach, since it can reveal information about the sources of our intuitions and the factors that may distort them, and thus provide more reliable data for philosophical theory. Indeed, they argue that experimental philosophy represents not a break with tradition, but a return to the ‘questions … that animated Plato, Aristotle, Spinoza, Hume, Nietzsche, and so many others … ’ (14). Following this ‘manifesto’ are nine chapters that display experimental philosophy in action (2–9, 11), and two chapters (10, 12) that provide an overview and critical evaluation of this new (or perhaps not so new) approach to philosophical investigation.

There is much to admire in this volume. First, it includes investigations of a variety of topics and populations, some of which should interest even those not sympathetic to experimental philosophy's methods. Second, it includes papers that provide direct commentary on one another, follow-up experiments on the questions raised by earlier studies, and suggestions for further research, which (though sometimes reading like a limited conversation among a small group of friends) helps to give readers a sense of the range of these experimental methods. Finally, though the editors are themselves advocates (and practitioners) of experimental philosophy, they have included papers that, in different ways, are sceptical about the approach. In the end, however – or so I’ll argue – it seems that the results of the most methodologically sound and philosophically relevant studies discussed in this volume could have been obtained from the armchair, and thus that experimental philosophy may not present a serious challenge to the traditional methods of analytic philosophy.

The earliest papers (Chapters 2 and 3, first published in 2001 and 2004) are best read as manifestos themselves, as they seem primarily concerned to throw down the gauntlet, and challenge the traditional methods of philosophical inquiry. This is understandable, perhaps bracing, but can raise questions about the significance of both the data and their interpretation. For example, in Chapter 2 (by J. Weinberg, S. Nichols and S. Stich), the authors present a survey of Western and East Asian college students that shows that, while a majority of the Westerners judge that the protagonist in a typical Gettier case ‘only believes’, rather than ‘really knows’ that his friend drives an American car, the verdicts are reversed for the East Asians.2 They explain this divergence as reflecting the tendency of East Asians to ‘make categorical judgements on the basis of similarity’ and Westerners to ‘focus on causation in describing the world and classifying things’ (28) – a tendency, they note, also reported in earlier studies of East Asians’ and Westerners’ judgements in other domains.

Now this diagnosis may be correct, but it's hard to tell without, first, a clearer explanation of why these results reflect a difference in categorization by similarity rather than cause, and – more important – an assurance that the subjects are fully aware of what is being asked, and are not making various assumptions about such things as the car buying habits of the protagonists. Without follow-up studies that provide such assurances, these results may seem more like ethnic stereotyping than a probe of deep cultural differences in intuitions about the norms of knowledge and justification.3 This is true as well of the authors’ discussion of differences in the epistemic judgements of exclusively Western populations with high vs. low socio-economic status. For example, they found that significantly more individuals of low than high SES attribute knowledge to a person who, on the basis of substantial evidence, truly believes that non-smoking uses of nicotine do not cause cancer – even when the story adds that it is possible that the tobacco companies made up and publicized this evidence (31). But they suggest that this discrepancy can be explained by the hypothesis that ‘low SES subjects have lower standards for knowledge’ (34), rather than, say, that high SES subjects are gullible consumers of conspiracy theories and thus have lower standards for what counts as a relevant counterpossibility.4

Similar questions arise for the findings of differences, reported in Chapter 3 (by E. Machery, R. Mallon, S. Nichols and S. Stich), in the ‘semantic intuitions’ of Westerners and East Asians. When queried about the referents of certain proper names, such as ‘Gödel’ in Kripke's well-known scenario involving Gödel and Schmidt, East Asians tend to judge that they are the individuals who fit a certain description, while Westerners tend to judge that they are the individuals causally connected to our use of those names. Here too the authors take these findings to suggest that Westerners have a ‘greater tendency to make causation-based judgements’ than East Asians (53), and, in addition, to support a finding by Nisbett that the differences between East Asians and Westerners ‘can be loosely grouped together under the heading of holistic vs. analytic thought’ (50) – though here the authors are more circumspect about their claims and suggest alternative hypotheses that could also explain their results. Still, they express confidence that the variation they find in individuals’ answers to such questions shows, at least, that the attempt to construct a theory of reference by gathering ‘semantic intuitions’ from the armchair ‘smacks of narcissism in the extreme’ (54), without explaining why these answers should be taken to reveal the ‘intuitions’ that philosophers consider relevant to constructing a theory of reference.

In many of the studies described in later chapters, however – including those performed by some of the authors of Chapters 1 and 2 – there is greater attention to experimental method, and more serious consideration of alternative hypotheses, much of it encouraged by questions raised by experimental philosophers themselves. But even in these more sophisticated studies, the data collection and interpretation often raise similar questions.

Most of this later work focuses on judgements about intentional action and moral responsibility. Chapters 4 and 5 report investigations of the claim, made by various philosophers who hold that determinism is incompatible with moral responsibility, that people are ‘intuitive incompatibilists’ – and each concludes, at least tentatively, that this is not so. In experiments detailed in Chapter 4 (by R. Woolfolk, J. Doris and J. Darley), subjects were queried to determine whether they attribute responsibility to agents who identify with the actions they perform, even when unable to do otherwise – and found, indeed, that they tended to do so, suggesting that ‘the folk’ have at least somewhat compatibilist intuitions about moral responsibility. Chapter 5 (by E. Nahmias, S. Morris, T. Nadelhoffer and J.Turner) discusses studies in which subjects are given a number of differently worded descriptions of a deterministic universe, and asked whether agents in those universes could be held morally responsible for various (intuitively) good and bad actions. Most subjects judged that agents were morally responsible for both the good and bad actions in all these circumstances.5 Here too the authors conclude, tentatively, that incompatibilism is not intuitive, and argue that this undermines the contention (of philosophers such as Kane, G. Strawson, Ekstrom and Pink) that compatibilist theories violate the ‘natural’ or ‘pretheoretical’ intuitions of the folk (82).

Chapter 6 (by Nichols and Knobe) is in part a response to the arguments of Chapters 4 and 5. The authors contend that, contrary to the conclusions of those arguments, people are indeed ‘intuitive incompatibilists’. They support this view by citing studies that find that, while subjects deny moral responsibility to agents when given abstract descriptions of universes in which their actions are completely causally determined, they are more likely to assign moral responsibility to such agents when their actions are described concretely enough to trigger their emotions, particularly when those emotions are negative (though Nadelhoffer, in Chapter 8, reports similar results with positive emotions as well). Thus, they argue, the compatibilist intuitions elicited in the studies described in Chapters 4 and 5 do not reveal subjects’ implicit theory about whether determinism is compatible with moral responsibility, but reflect a kind of ‘performance error’ prompted by the emotional resonance of the immoral act (113).6

The authors consider some alternative explanations for this phenomenon: first, that the effect may be due merely to the concreteness of the description, rather than its emotional resonance, and, second, that the underlying theory may really be compatibilist, while the incompatibilist judgements prompted by abstract descriptions are themselves performance errors. To test their hypothesis further, they performed a follow-up experiment (116–8), in which subjects are told the story of Bill, who ‘as he has done many times in the past … stalks and rapes a stranger’, and of Mark, who ‘as he has done many times in the past … arranges to cheat on his taxes’, as examples of actions described in equally concrete, but differentially affective, terms. The subjects were subsequently divided into two groups – one which is told that Bill's and Mark's universe is indeterministic, the other that the universe is deterministic – and were then asked whether it is ‘possible that Bill/Mark is fully responsible for raping the stranger/cheating on his taxes’. The result is that in the indeterministic scenario, 95% of the subjects judge Bill to be responsible, and 89% judge Mark to be responsible, but in the deterministic scenario, the verdicts are, respectively, 63% and 23% (116), which suggests, according to Nichols and Knobe, that it is the emotional resonance of the description, and not its concreteness, that accounts for the subjects’ attribution of responsibility in the deterministic case.

This conclusion is disputed by W. Sinnott-Armstrong (Chapter 11), who objects that the ‘concreteness’ hypothesis cannot be eliminated, since the description of ‘stalking and raping’ is more concrete than the description of ‘arranging to cheat on his taxes’ (215). Moreover, he contends, the ‘concreteness’ hypothesis accounts for a similar divergence of intuitions about whether the inability to rule out sceptical scenarios undermines knowledge, and whether deterrence should be considered in determining the appropriate punishment for a crime (220–2). Here too, however – as in the studies described in the first two chapters – the description of the experiments undertaken to test these hypotheses leaves it unclear whether subjects understood just what they were being asked – or why their answers should be taken to support the authors’ hypotheses.7

Nichols and Knobe also argue that their subjects’ verdicts about Bill and Mark reflect performance errors, rather than reveal their competence with responsibility attributions (118), since a large percentage of subjects judge both Bill and Mark to be responsible in the indeterministic scenario. But there may be confounding factors here as well, since the descriptions of Mark's and Bill's actions display a further asymmetry. To describe Mark as ‘arranging to cheat on his taxes’ suggests that Mark, like many people, uses an accountant to prepare his tax forms. And thus when subjects are asked whether Mark is ‘fully responsible’ for cheating, they may understand this as a question about whether the responsibility is solely his. This in itself won’t answer the question of why so many subjects judge Mark to be ‘fully responsible’ in the indeterministic case – but compatibilists could suggest that when subjects are asked whether agents bear responsibility for their actions in a deterministic universe, they pay more attention to the description of the action so that they can determine whether coercive or pathological conditions obtain, which could lead Nichol's and Knobe's subjects to focus on the possibility (ignored in the indeterministic condition) that Mark did not act alone. In any case, whatever the explanation for the divergent attributions of responsibility in these scenarios, there remains the further question that has arisen before, namely, why any set of the subjects’ initial responses to such questions should be taken to reflect their underlying ‘competence’ rather than performance error, and thus provide data for a theory of responsibility.

Chapters 7–9 concern judgements about intentional action. In Chapter 7, Knobe cites data, gathered from subjects from a wide variety of cultures and backgrounds, that suggest that if a ‘side-effect’ of an action (i.e. a consequence of the action that was foreseen but not intended) is regarded as bad, people will judge that the agent did it intentionally (130–1). He also cites data from other studies that suggest that when people intend to do something, but accomplish it only via a ‘deviant causal chain’, they are regarded as having done it intentionally if the consequences are bad or good – but not if the action is merely an attempt to demonstrate a competence or skill (133–4). These results, he argues, show that considerations of the blame- or praiseworthiness of an action play a role in peoples’ judgements about whether the action was intentional – and thus the distinction between intentional and unintentional action cannot be, as folk psychology demands, merely a matter of differences in their causal roles. Knobe considers various alternative explanations of these data, and constructs further experiments to decide between them (135–40). But it's not clear that these experiments are the right ones for homing in on one's concept of intentional action. It would be interesting, for example, to see an experiment that attempts to determine if there are cases in which subjects judge – correctly or not – various foreseen consequences of an action to be constitutive of that action, rather than mere ‘side-effects’ of it, since, if thinking of the bad effects of an action skews one's judgement about that distinction, then these judgements would not conflict with folk psychology. It would also be interesting to see an experiment that attempts to determine whether, when subjects say that an agent's action was intentional, they instead mean that the agent did something intentionally that led to the consequences in question. In short, though I agree that ‘[i]nstead of starting out with certain preconceptions about the nature of folk psychology and then trying to square the data with those preconceptions, [it is better to] start out with the data and try to figure out what the data might be telling us about the nature of folk psychology’ (141), it seems that analogous considerations should guide experimental design and the collection of data. Finally – and I’ll return to this later – it would be interesting to see some information about what people say when explicitly shown that they give a different verdict about whether actions are intentional when the actions are good than when they are bad.

Chapter 8 (by T. Nadelhoffer) presents a number of further studies that show that peoples’ judgements of the moral badness and goodness of an act (and also their unconscious appraisals of the general moral attributes of the agent) influences both their judgements of whether the act was done intentionally, and whether it was done knowingly (156–7). His major concern, however, is their implications not for our concept of intentional action, or the methodology of analytic philosophy, but for the efficacy of our jury system: if jurors’ judgements about whether an act was intentional (or done knowingly) are affected by their evaluations of the morality of the act, then it will be hard to reach impartial verdicts and fair sentences; and if, as seems likely, these evaluations arise spontaneously and unconsciously, it will be difficult for jurors to discount them, even if they are explicitly instructed to do so by a judge, and want to comply (159–61). Unlike Knobe, Nadelhoffer contends that the influence of moral considerations on subjects’ judgements about whether ‘structurally similar’ acts are intentional is due to performance error rather than conceptual competence (162), but argues that, even if Knobe is right, this influence produces judgements that violate the norms of our jury system, in ways that are hard to counteract. This does indeed seem to be a problem that philosophers – and others – should take seriously.

Chapter 9 (by F. Cushman and A. Mele) is also devoted to the discussion of our concept of intentional action. Their main concern is to develop and test further hypotheses about what contributes to judgements about whether an action is intentional (174–82), and they conclude that the data suggest that there may be more than one concept of intentional action that people routinely deploy. And, though their experimental procedure, too, raises questions about whether the subjects really understand what is being asked of them (I myself was baffled by many of the ‘vignettes’ they use to prompt intuitions, 183–6), along the way they cite some studies in which ‘[t]he evidence suggests that people sometimes override or reject their intuitive responses when they fail to align with their consciously held views, especially when the contradiction is made apparent … ’ (177). This finding helps to answer my earlier question about what subjects do when made explicitly aware of discrepancies in their judgements, and may also point the way to a reasonable account of when judgements reflect conceptual competence rather than ‘performance error’, namely, when subjects affirm these judgements on reflection, and regard them as their considered judgements. Perhaps experimental philosophers should attempt to devise methods that elicit judgements of this sort.

If so, they would be helped by examining the methods used, or at least aspired to, by analytic philosophers engaged in a serious effort to determine whether their initial intuitions about responsibility and determinism, or knowledge and justification, or intentional and blameworthy action are consistent, and cohere with their intuitions about other phenomena and the principles they endorse; in short, the process of reflective equilibrium that embodies the methodology of the field (particularly when dissenting colleagues are continually challenging one's views). But the results of these experiments, it seems, would most likely match the results of ‘experiments’ conducted from the armchair.

It may well be, of course, that even in this state of ‘reflective equilibrium’, different subjects will arrive at different judgements – and will not be able to be moved – or a single individual will be pulled in two directions. In this case, we would have to conclude, as do Cushman and Mele about intentional action, that there are two concepts at hand. And we would have to take notice of a finding of Knobe and Nichols, mentioned in their (Chapter 6) discussion of subjects’ conflicting judgements about moral responsibility, that when the subjects are made aware of these judgements, and ‘asked to adjudicate the conflict between the compatibilist and incompatibilist intuitions … approximately half the subjects chose to hold onto the judgement that the particular agent was morally responsible, while the other half chose to hold onto the judgement that no one can be held responsible in a deterministic universe’ (121). It's not clear whether they take this to show that there are indeed two distinct concepts of responsibility (rather than pervasive performance error in one population or the other), but their conclusion is that ‘there is no more consensus about these issues among the folk than there is among philosophers’ (121). However, one can equally well conclude – against those who argue that only surveys of those from all backgrounds can counter the theory-tainted judgements of philosophers – that there may be no more consensus about these issues among philosophers than among the folk. And thus it seems that traditional analytic methods may well suffice for identifying the judgements that provide the right sorts of data for philosophical theories, contentious though the theories may be.

Moreover, such methods may have practical benefits as well. For example, in Nadelhoffer's chapter about the problems emotional responses pose for juror impartiality, he cites studies (by Wilson and Brekke) that suggest that for individuals to ‘avoid cognitive biases … they must be made aware of the unwanted [usually unconscious] processes in question … [and] be motivated to correct the error … [and] be aware of the direction and the magnitude of the bias’. In addition, they must have ‘sufficient control over their mental processes to be able to correct for the biases in question’, and this, Nadelhoffer worries, may be prohibitively difficult to achieve. Indeed. But awareness of inconsistencies among one's initial ‘intuitions’ and one's other cognitive commitments may, at least, alert jurors (and the rest of us) that in certain situations one should proceed with caution before making a categorical judgement. So, although the importance of determining the contours of our concepts may pale when compared to the importance of encouraging fair and impartial judicial verdicts, it seems that the methodology of analytic philosophy may in the end provide the best tool for both jobs.

These reflections echo the view expressed by E. Sosa (Chapter 12), who argues that, though experimental philosophy may indeed tell us something about how the mind works, it does not threaten the more traditional methodology of contemporary analytic philosophy. The reason, he argues (234–5), is that even if survey results show that two populations diverge in their judgements, it may be that these disagreements are merely verbal, as can occur when subjects do not understand the questions in the way the experimenters intend. On the other hand, it may well reflect a true conceptual distinction that may take some work to recognize (as with the difference between the ‘attributivity’ and ‘accountability’ senses of moral responsibility, or Sosa's own distinction between ‘animal’ and ‘reflective’ knowledge – if not the difference between concepts of intentional action). But there's nothing that would prevent armchair methods, at least in principle, from achieving this recognition on their own.

Yet a different view of the relation between empirical methods and philosophical inquiry emerges in Chapter 10, in which J. Prinz distinguishes ‘Empirical Philosophy’ from ‘Experimental Philosophy’. Instead of conducting surveys of the folk aimed to get untainted evidence about the contours of our concepts, empirical philosophers look to results established by cognitive scientists and neuroscientists to confirm, refute, or suggest modifications of philosophical theories of what those concepts (considered as natural kind concepts) denote, especially concepts of mental phenomena such as consciousness, perception, and imagery (197–8). And, while Prinz points to studies done by experimental philosophers that have direct relevance to the concerns of empirical philosophers (203, 204), and has even contributed to the discipline himself, he seems to think it best for philosophers to leave the experimentation to the professionals, and merely avail ourselves of the spoils.

This seems reasonable, as far as it goes. But what about philosophers who work on topics outside the philosophy of mind (or those who dispute the view that mental state concepts are natural kind concepts): can they stay in their armchairs? As I’ve already noted, if the best evidence for philosophical theories is our ‘considered’ intuitions – that is, those that remain stable, given extensive reflection on how our intuitively compelling judgements about particular cases cohere with our other intuitions and theoretical commitments – then it seems that we can get most of this evidence using armchair methodology.

However, it's important to recognize that what counts as armchair methodology can be broader than suggested by classic mid-twentieth century conceptual analysis – even independent of the experimentalist turn – thanks to what seems to be the general acceptance of a post-Quinean looseness about what counts as a conceptual truth, and thus of the permeability of the border between philosophy and science. This suggests that (unlike perceptual illusions that remain fixed despite countervailing evidence) even our first-off, untutored intuitions can change with the growth of scientific knowledge, as ways of categorizing the world that once seemed counterintuitive become part of the common coin. So, not only may the reflective equilibrium of armchair methodology give us much of what the experimentalists aim for, but it may also – as long as one keeps a steady eye on the Discovery Channel – result in intuitively compelling judgements that are likely to be true.

Given the range of studies described in this volume, the intrinsic interest of many of their findings, and the inclusion of both advocates and critics of this approach to philosophical inquiry, I recommend Experimental Philosophy, the book, to philosophers interested in the evidential status of intuition in philosophical inquiry (as well as to anyone – especially potential jurors – interested in learning more about the contingencies that can affect one's untutored intuitive judgements about a range of important things). As for Experimental Philosophy, the discipline, I’d say the jury is still out!8

1 Experimental Philosophy, eds. J. Knobe and S. Nichols (Oxford University Press, 2008. xii + 244 pp. £54.00 cloth, £13.99 paper).
2 This is the probe: ‘Bob has a friend, Jill, who has driven a Buick for many years. Bob therefore thinks that Jill drives an American car. He is not aware, however, that her Buick has recently been stolen, and he is not aware that Jill has replaced it with a Pontiac, which is a different kind of American car. Does Bob really know that Jill drives an American car, or does he just believe it?’ (29)
3 This is also true of the authors’ discussion of the finding that a larger minority of Westerners than Asians judge that certain ‘Truetemp cases’ – that is, cases in which a subject uses a method that reliably produces true belief about what the temperature is, but is unaware that the method is reliable – count as knowledge. (In addition, one wonders why this finding is emphasized, rather than the fact that a significant majority of both populations, in all cases, contend that the subject doesn’t really know.)
4 Similar worries about these cases, and also the cross-cultural studies detailed in these papers, have been expressed by many philosophers. See, for example, Sosa 2005.
5 The descriptions are: (a) all events are predictable, given knowledge of laws and initial conditions, (b) everything would happen the same way no matter how many times the universe is recreated, and (c) all persons’ actions are ‘caused completely’ by their genes and upbringing (87–9). Most subjects also judged that the agents acted freely, though – interestingly – while more subjects judged the agents to be responsible than to be free in the first two scenarios, the verdicts were reversed in the third scenario.
6 One wonders, however, what would happen if you describe a seemingly good/bad act, then point out that even if the person chose to do it, the choice was caused by psychological states that were the product of genetics, upbringing, etc. This would be a concrete case with affective elements – but cases like these (at least for my introductory students) seem to prompt incompatibilist intuitions. And – famously – it worked for Clarence Darrow in the trial of Leopold and Loeb!
7 For example, in an experiment designed to test whether abstract descriptions prompt more sympathy with scepticism, all subjects are told that ‘people sometimes believe things for no good reason. For example, people sometimes believe what a politician says about the economy when they have no good reason to trust what the politician says. Our question is about knowledge’. But this general claim was followed, for one group of subjects, by (abstract condition) the question ‘if a person cannot give any good reason to believe a claim, is it possible that the person knows the claim is correct?’, and, for a second group of subjects, by (concrete condition) ‘if you cannot give any good reason to believe that the person whom you believe to be your mother really is your mother, is it possible that you know that she is your mother?’ (221). The results were that 52% of the subjects in the ‘abstract’ case, and 88% in the ‘concrete’ case answered ‘yes’. But the wording of these questions seems to introduce a number of confounding factors. First, when answering the question, subjects in the first group may also be thinking of a concrete case – but one in which there is no good reason for belief, in particular, the previously specified scenario in which people are stipulated to have no good reason to trust a politician about the economy. Second, it is odd to think of a situation in which one must give (rather than merely have) good reasons to believe that someone is one's mother. (In addition, it is unclear why a question about one's mother should count as a description that is concrete, rather than emotionally resonant.) And once again, I have found that the best way to turn undergraduates into (temporary) sceptics is to ask then whether they have any good reason to think that they are not brains in a vat.
8 I’m grateful to Kadri Vihvelin, Terrance Tomkow, and Eric Schwitzgebel for comments and discussion of these issues.


A defense of the use of intuitions in philosophy
Stich and His Critics