BEHAVIORAL AND BRAIN SCIENCES (2010) 33, 315–365 doi:10.1017/S0140525X10000907 Person as scientist, person as moralist Joshua Knobe Program in Cognitive Science and Department of Philosophy, Yale University, New Haven, CT 06520-8306 Knobe: Person as scientist, person as moralist the like, can be understood on the model of a scientific example, in work on causal cognition, researchers some- investigation. times proceed by looking to the statistical methods that This approach has a strong intuitive appeal, and recent appear in systematic scientific research and then suggesting theoretical work has led to the development of specific that those same methods are at work in people’s ordinary hypotheses that spell it out with impressive clarity and pre- causal judgments (Gopnik et al. 2004; Kelley 1967; Wood- cision. There is just one problem. The actual experimental ward 2004). Different theories of this type appeal to quite results never seem to support these hypotheses. Indeed, different statistical methods, but these differences will not the results point toward a far more radical view. They be relevant here. The thing to focus on is just the general suggest that moral considerations actually figure in the idea that people’s ordinary causal cognition is in some way competencies people use to make sense of human beings analogous to a scientific inquiry. and their actions. And it is not only the study of causal cognition that pro- ceeds in this way. A similar viewpoint can be found in the theory-of-mind literature (Gopnik & Meltzoff 1997), 2. Introducing the person-as-scientist theory where it sometimes goes under the slogan “Child as Scien- tist.” There, a central claim is that children refine their In the existing literature on causal cognition and theory-of- understanding of the mind in much the same way that mind, it has often been suggested that people’s ordinary scientists refine their theories. Hence, it is suggested way of making sense of the world is in certain respects ana- that we can look at the way Kepler developed his theory logous to a scientific theory (Churchland 1981; Gopnik & of the orbits of the planets and then suggest that children Meltzoff 1997; Sloman 2005). This is an important and use the same basic approach as they are acquiring the provocative suggestion, but if we are to grapple with it concept of belief (Gopnik & Wellman 1992). Once properly, we need to get a better understanding of pre- again, the idea is that the cognitive processes people use cisely what it means and how experimental evidence in ordinary life show a deep similarity to the ones at might bear on it. work in systematic science. It is this idea that we will be taking up here. Genuinely scientific inquiry seems to be sensitive to a quite specific 2.1. Ordinary understanding and scientific theory range of considerations and seems to take those consider- To begin with, we will need to distinguish two different ations into account in a highly distinctive manner. What aspects of the claim that people’s ordinary understanding we want to know is whether certain aspects of ordinary is analogous to a scientific theory. First, there is the claim cognition work in more or less this same way. that human thought might sometimes take the form of a theory. To assess this first claim, one would have to pick 2.2. Refining the question out the characteristics that distinguish theories from other sorts of knowledge structures and then ask whether But now it might seem that the answer is obvious. For it these characteristics can be found in ordinary cognition. has been known for decades that people’s ordinary intui- This is certainly a worthwhile endeavor, but it has already tions show certain patterns that one would never expect been pursued in a considerable body of recent research to find in a systematic scientific investigation. People (e.g., Carey & Spelke 1996; Goldman 2006; Murphy & make wildly inappropriate inferences from contingency Medin 1985), and I will have nothing further to say about tables, show shocking failures to properly detect corre- it here. Instead, the focus of this target article will be on lations, display a tendency to attribute causation to which- a second claim, namely, the claim that certain facets of ever factor is most perceptually salient (Chapman & human cognition are properly understood as scientific. Chapman 1967; McArthur & Post 1977; Smedslund To begin with, it should be emphasized that this second 1963). How could one possibly reconcile these facts claim is distinct from the first. If one looks to the usual about people’s ordinary intuitions with a theory according sorts of criteria for characterizing a particular knowledge to which people’s ordinary cognition is based on some- structure as a “theory” (e.g., Premack & Woodruff 1978), thing like a scientific methodology? one sees immediately that these criteria could easily be sat- The answer, I think, is that we need to interpret that isfied by, for example, a religious doctrine. A religious doc- theory in a somewhat more nuanced fashion. The theory trine could offer systematic principles; it could posit is not plausibly understood as an attempt to describe all unobservable entities and processes; it could yield definite of the factors that can influence people’s intuitions. predictions. For all these reasons, it seems perfectly Instead, it is best understood as an attempt to capture reasonable to say that a religious doctrine could give us a the “fundamental” or “underlying” nature of certain cogni- certain kind of “theory” about how the world works. Yet, tive capacities. There might then be various factors that although the doctrine might offer us a theory, it does not interfere with our ability to apply those capacities cor- appear to offer us a specifically scientific theory. In par- rectly, but the existence of these additional factors would ticular, it seems that religious thinking often involves in no way impugn the theory itself. attending to different sorts of considerations from the To get a rough sense for the strategy here, it might be ones we would expect to find in a properly scientific inves- helpful to return to the comparison with religion. Faced tigation. Our task here, then, is to figure out whether with a discussion over religious doctrine, we might say: certain aspects of human cognition qualify as “scientific” “This discussion isn’t best understood as a kind of scientific in this distinctive sense. inquiry; it is something else entirely. So if we find that the One common view is that certain aspects of human participants in this discussion are diverging from proper cognition do indeed make use of the very same sorts of con- scientific methods, the best interpretation is that they siderations we find in the systematic sciences. So, for simply weren’t trying to use those methods in the first 316 BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 Knobe: Person as scientist, person as moralist place.” This would certainly be a reasonable approach 3.1. Intentional action to the study of religious discourse, but the key claim of Perhaps the most highly studied of these effects is the the person-as-scientist theory is that it would not be the impact of people’s moral judgments on their use of the right approach to understanding certain aspects of our concept of intentional action. This is the concept people ordinary cognition. Looking at these aspects of ordinary use to distinguish between behaviors that are performed cognition, a defender of the person-as-scientist view intentionally (e.g., hammering in a nail) and those that would adopt a very different stance. For example, she are performed unintentionally (e.g., accidentally bringing might say: “Yes, it’s true that people sometimes diverge the hammer down on one’s own thumb). It might at first from proper scientific methods, but that is not because appear that people’s use of this distinction depends they are engaging in some fundamentally different sort of entirely on certain facts about the role of the agent’s activity. Rather, their underlying capacities for causal cogni- mental states in his or her behavior, but experimental tion and theory-of-mind really are governed by scientific studies consistently indicate that something more methods; it’s just that there are also various additional complex is actually at work here. It seems that people’s factors that get in the way and sometimes lead people into moral judgments can somehow influence their intuitions errors.” about whether a behavior is intentional or unintentional. Of course, it can be difficult to make sense of this talk of To demonstrate the existence of this effect, we can con- certain capacities being “underlying” or “fundamental,” struct pairs of cases that are exactly the same in almost and different researchers might unpack these notions in 1 every respect but differ in their moral status. For a different ways: simple example, consider the following vignette: 1. One view would be that people have a domain- The vice-president of a company went to the chairman of the specific capacity for making certain kinds of judgments board and said, “We are thinking of starting a new program. It but then various other factors intrude and allow these will help us increase profits, but it will also harm the environ- judgments to be affected by irrelevant considerations. ment.” 2. Another would be that people have a representation The chairman of the board answered, “I don’t care at all of the criteria governing certain concepts but that they are about harming the environment. I just want to make as not always able to apply these representations correctly. much profit as I can. Let’s start the new program.” 3. A third would be that the claim is best understood They started the new program. Sure enough, the environ- counterfactually, as a hypothesis about how people ment was harmed. would respond if they only had sufficient cognitive Faced with this vignette, most subjects say that the chair- resources and freedom from certain kinds of biases. man intentionally harmed the environment. One might I will not be concerned here with the particular differ- initially suppose that this intuition relies only on certain ences between these different views. Instead, let us intro- facts about the chairman’s own mental states (e.g., that duce a vocabulary that allows us to abstract away from he specifically knew his behavior would result in environ- these details and talk about this approach more generally. mental harm). But the data suggest that something more is Regardless of the specifics, I will say that the approach is to going on here. For people’s intuitions change radically posit an underlying competence and then to posit various when one alters the moral status of the chairman’s behav- additional factors that get in the way of people’s ability ior by simply replacing the word “harm” with “help”: to apply that competence correctly. The vice-president of a company went to the chairman of the With this framework in place, we can now return to our board and said, “We are thinking of starting a new program. It investigation of the impact of moral considerations on will help us increase profits, and it will also help the environ- people’s intuitions. How is this impact to be explained? ment.” One strategy would be to start out by finding some way The chairman of the board answered, “I don’t care at all to distinguish people’s underlying competencies from about helping the environment. I just want to make as much the various interfering factors. Then one could say that profit as I can. Let’s start the new program.” the competencies themselves are entirely scientific in They started the new program. Sure enough, the environ- ment was helped. nature, but that the interfering factors then prevent people from applying these competencies correctly and Faced with this second version of the story, most subjects allow moral considerations to affect their intuitions. This actually say that the chairman unintentionally helped the strategy is certainly a promising one, and I shall discuss environment. Yet it seems that the only major difference it in further detail later. But it is important to keep in between the two vignettes lies in the moral status of the mind that we also have open another, very different chairman’s behavior. So it appears that people’s moral option. It could always turn out that there simply is no judgments are somehow impacting their intuitions about underlying level at which the relevant cognitive capacities intentional action. are purely scientific, that the whole process is suffused Of course, it would be unwise to draw any strong con- through and through with moral considerations. clusions from the results of just one experiment, but this basic effect has been replicated and extended in numerous further studies. To begin with, subsequent experiments have further explored the harm and help cases to see 3. Intuitions and moral judgments what exactly about them leads to the difference in Before we think any further about these two types of people’s intuitions. These experiments suggest that explanations, we will need to get a better grasp of the moral judgments truly are playing a key role, since partici- phenomena to be explained. Let us begin, then, just by pants who start out with different moral judgments about considering a few cases in which moral considerations the act of harming the environment end up arriving at appear to be impacting people’s intuitions. different intuitions about whether the chairman acted BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 317 Knobe: Person as scientist, person as moralist intentionally (Tannenbaum et al. 2009). But the effect is the chairman harmed intentionally and moderate dis- not limited to vignettes involving environmental harm; it agreement with the claim that he helped intentionally emerges when researchers use different cases (Cushman (Knobe 2004a). The difference between the ratings in & Mele 2008; Knobe 2003a) and even when they turn to these two conditions provides evidence that people’s cases with quite different structures that do not involve moral intuitions are affecting their intuitions about inten- side-effects in any way (Knobe 2003b; Nadelhoffer tional action. 2005). Nor does the effect appear to be limited to any It appears, however, that this effect is not limited to the one particular population: It emerges when the whole concept of intentional action specifically. For example, study is translated into Hindi and conducted on Hindi- suppose we eliminate the word “intentionally” and speakers (Knobe & Burra 2006) and even when it is sim- instead use the word “decided.” The two sentences then plified and given to 4-year-old children (Leslie et al. become: 2006a). At this point, there is really a great deal of evidence (2) a. The chairman decided to harm the environment. for the claim that people’s moral judgments are somehow b. The chairman decided to help the environment. impacting their intuitions about intentional action. Faced with these revised sentences, subjects show more or Still, as long as all of the studies are concerned only with less the same pattern of intuitions. They tend to agree with intuitions about intentional action specifically, it seems the claim that the agent decided to harm, and they tend to that our argument will suffer from a fatal weakness. For disagree with the claim that the agent decided to help someone might say: “Surely, we have very strong reason (Pettit & Knobe 2009). to suppose that the concept of intentional action works Now suppose we make the case a little bit more in more or less the same way as the other concepts complex. Suppose we do not use the adverb “intentionally” people normally use to understand human action. But but instead use the verb “intend.” So the sentences come we have good theories of many of these other concepts – out as: the concepts of deciding, wanting, causing, and so forth – and these other theories do not assign any role to moral (3) a. The chairman intended to harm the environment. considerations. So the best bet is that moral considerations b. The chairman intended to help the environment. do not play any role in the concept of intentional action One then finds a rather surprising result. People’s either.” responses in both conditions are shifted over quite far In my view, this is actually quite a powerful argument. toward the “disagree” side. In fact, people’s intuitions Even if we have strong evidence for a certain view about end up being shifted over so far that they do not, on the the concept of intentional action specifically, it might whole, agree in either of the two conditions (Shepard well make sense to abandon this view in light of theories 2009; cf. Cushman 2010; Knobe 2004b; McCann 2005). we hold about various other, seemingly similar concepts. Nonetheless, the basic pattern of the responses remains the same. Even though people’s responses don’t go all 3.2. Further mental states the way over to the “agree” side of the scale in either con- dition, they are still more inclined to agree in the harm As it happens, though, the impact of moral considerations case than they are in the help case. does not appear to be limited to people’s use of the word Once one conceptualizes the issue in this way, it “intentionally.” The very same effect also arises for numer- becomes possible to find an impact of moral consider- ous other expressions: “intention,” “deciding,” “desire,” “in ations in numerous other domains. Take people’s appli- favor of,” “advocating,” and so forth. cation of the concept in favor. Now consider a case in To get a grip on this phenomenon, it may be helpful to which an agent says: look in more detail at the actual procedure involved in conducting these studies. In one common experimental I know that this new procedure will [bring about some design, subjects are randomly assigned to receive either outcome]. But that is not what we should be concerned about. The new procedure will increase profits, and that the story about harming the environment or the story should be our goal. about helping the environment and then, depending on the case, are asked about the degree to which they agree Will people say in such a case that the agent is “in favor” of or disagree with one of the following sentences: bringing about the outcome? (1) Here again, it seems that moral judgments play a role. a. The chairman of the board harmed the environment People disagree with the claim that the agent is “in intentionally. favor” when the outcome is morally good, whereas they W W W W W W W stand at just about the midpoint between agreement and disagreement when the outcome is morally bad (Pettit & definitely unsure definitely Knobe 2009). And similar effects have been observed for disagree agree people’s use of many other concepts: desiring, intending, choosing, and so forth (Pettit & Knobe 2009; Tannenbaum b. The chairman of the board helped the environment et al. 2009). intentionally. W W W W W W W Overall, these results suggest that the effect obtained for intuitions about intentional action is just one example of a definitely unsure definitely far broader phenomenon. The effect does not appear to be disagree agree limited to the concept intentionally, nor even to closely related concepts such as intention and intending. Rather, When the study is conducted in this way, one finds that it seems that we are tapping into a much more general ten- subjects show moderate agreement with the claim that dency, whereby moral judgments impact the application of 318 BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 Knobe: Person as scientist, person as moralist a whole range of different concepts used to pick out harming the environment is morally wrong, they thereby mental states and processes. come to represent the corresponding node on the action tree as “collapsing” into a lower node (see Fig. 2). The asymmetries we find for “in order to” and “by” 3.3. Action trees would then follow immediately, without the need for any But the scope of the effect does not stop there. It seems controversial assumptions about the semantics of these also to apply to intuitions about the relations that obtain specific expressions. Although the issue here is a among the various actions an agent performs. Philoso- complex one, recent research does seem to be supporting phers and cognitive scientists have often suggested that the claim that moral judgments are affecting action tree such relations could be represented in terms of an action representations in this way (Knobe, forthcoming; tree (Goldman 1970; Mikhail 2007). Hence, the various Ulatowski 2009). actions performed by our chairman in the help case might be represented with the tree in Figure 1. Needless to say, ordinary folks do not actually commu- 3.4. Causation nicate with each other by writing out little diagrams like this one. Still, it seems that we can get a sense of how All of the phenomena we have been discussing thus far people are representing the action tree by looking at may appear to be quite tightly related, and one might their use of various ordinary English expressions, for therefore suspect that the effect of morality would disap- example, by looking at the way they use the expressions pear as soon as one turns to other, rather different cases. “in order to” and “by.” That, however, seems not to be the case. Indeed, the very A number of complex issues arise here, but simplifying same effect arises in people’s intuitions about causation slightly, the key thing to keep in mind is that people only (Alicke 2000; Cushman 2010; Hitchcock & Knobe 2009; use “in order to” for relations that go upward in the tree, Knobe, forthcoming; Knobe & Fraser 2008; Solan & and they only use “by” for relations that go downward. Darley 2001). Thus, people are willing to say that the chairman For a simple example here, consider the following “implemented the program in order to increase profits” vignette: but not that he “increased profits in order to implement The receptionist in the philosophy department keeps her desk the program.” And, conversely, they are willing to say stocked with pens. The administrative assistants are allowed to that he “increased profits by implementing the program” take pens, but faculty members are supposed to buy their own. but not that he “implemented the program by increasing The administrative assistants typically do take the pens. profits.” Looking at people’s intuitions about simple Unfortunately, so do the faculty members. The receptionist expressions like these, we can get a good sense of how repeatedly e-mailed them reminders that only administrators are allowed to take the pens. they are representing the geometry of the action tree itself. On Monday morning, one of the administrative assistants But now comes the tricky part. Experimental results encounters Professor Smith walking past the receptionist’s indicate that people’s intuitions about the proper use of desk. Both take pens. Later that day, the receptionist needs these expressions can actually be influenced by their to take an important message . . . but she has a problem. moral judgments (Knobe 2004b; forthcoming). Hence, There are no pens left on her desk. people are willing to say: Faced with this vignette, most subjects say that the pro- The chairman harmed the environment in order to increase fessor did cause the problem but that the administrative profits. assistant did not cause the problem (Knobe & Fraser but not: 2008). Yet, when we examine the case from a purely scien- The chairman helped the environment in order to increase tific standpoint, it seems that the professor’s action and the profits. administrative assistant’s action bear precisely the same And, similarly, they are willing to say: relation to the problem that eventually arose. The main The chairman increased profits by harming the environment. difference between these two causal factors is just that the professor is doing something wrong (violating the but not: departmental rule) while the administrative assistant is The chairman increased profits by helping the environment. doing exactly what she is supposed to (acting in accordance One natural way of explaining these asymmetries would with the rules of the department). So it appears that be to suggest that people’s moral judgments are having an people’s judgment that the professor is doing something effect on their representations of the action tree itself. For wrong is somehow affecting their intuitions about example, suppose that when people make a judgment that Figure 1. Action tree for the help case. Figure 2. Action tree for the harm case. BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 319 Knobe: Person as scientist, person as moralist whether or not the professor caused the events that Notice that the doctor performs exactly the same behav- followed. ior in these two vignettes, and in both vignettes, he per- Now, looking just at this one case, one might be tempted forms this behavior in the hopes that it will bring about to suppose that the effect is not at all a matter of moral the man’s death. The only difference between the cases judgment but simply reflects people’s intuitive sense that lies in the moral character of the doctor’s reasons for the professor’s action is more “unusual” or “strange” hoping that the man will die. Yet this moral difference than the administrative assistant’s. But subsequent led to a striking difference in people’s intuitions about studies strongly suggest that there is something more doing versus allowing. Subjects who received the first afoot here. People continue to show the same basic vignette tended to say that the doctor “allowed” the effect even when they are informed that the administrative man’s life to end, whereas subjects who received the assistants never take pens whereas the professors always second vignette tended to say that the doctor “ended” do (Roxborough & Cumby 2009), and there is a statisti- the man’s life. (Moreover, even within the first vignette, cally significant effect whereby pro-life subjects are more there was a correlation whereby subjects who thought inclined than pro-choice subjects to regard the act of that euthanasia was generally morally wrong were less seeking an abortion as a cause of subsequent outcomes inclined to classify the act as an “allowing.”) Overall, (Cushman et al. 2008). All in all, the evidence seems then, the results of the study suggest that people’s moral strongly to suggest that people’s moral judgments are actu- judgments are influencing their intuitions here as well. ally impacting their causal intuitions. It would, of course, be foolhardy to draw any very general conclusions from this one study, but the very same effect has also been observed in other studies using quite different methodologies (Cushman et al. 2008), 3.5. Doing and allowing and there is now at least some good provisional evidence People ordinarily distinguish between actually breaking in support of the view that people’s intuitions about something and merely allowing it to break, between actu- doing and allowing can actually be influenced by their ally raising something and merely allowing it to rise, moral judgments. between actually killing someone and merely allowing someone to die. This distinction has come to be known as the distinction between doing and allowing. 3.6. Additional effects To explore the relationship between people’s intuitions Here we have discussed just a smattering of different ways about doing and allowing and their moral judgments, we in which people’s moral judgments can impact their intui- used more or less the same methodology employed in tions about apparently non-moral questions. But our these earlier studies (Cushman et al. 2008). Subjects review has been far from exhaustive: there are also were randomly assigned to receive different vignettes. studies showing that moral judgments can affect intuitions Subjects in one condition received a vignette in which about knowledge (Beebe & Buckwalter, forthcoming), the agent performs an action that appears to be morally happiness (Nyholm 2009), valuing (Knobe & Roedder permissible: 2009), act individuation (Ulatowski 2009), freedom (Phil- Dr. Bennett is an emergency-room physician. An uncon- lips & Knobe 2009), and naturalness (Martin 2009). scious homeless man is brought in, and his identity is Given that all of these studies were conducted just in the unknown. His organ systems have shut down and a nurse past few years, it seems highly probable that a number has hooked him up to a respirator. Without the respirator of additional effects along the same basic lines will he would die. With the respirator and some attention from emerge in the years to come. Dr. Bennett he would live for a week or two, but he would never regain consciousness and could not live longer than two weeks. Dr. Bennett thinks to himself, “This poor man deserves to 4. Alternative explanations die with dignity. He shouldn’t spend his last days hooked up to such a horrible machine. The best thing to do would be to Thus far, we have seen that people’s ordinary application disconnect him from the machine.” of a variety of different concepts can be influenced by For just that reason, Dr. Bennett disconnects the home- moral considerations. The key question now is how to less man from the respirator, and the man quickly dies. explain this effect. Here we face a choice between two These subjects were then asked whether it would be more basic approaches. One approach would be to suggest appropriate to say that the doctor ended the homeless that moral considerations actually figure in the competen- man’s life or that he allowed the homeless man’s life to cies people use to understand the world. The other would end. be to adopt what I will call an alternative explanation. That Meanwhile, subjects in the other condition were given a is, one could suggest that moral considerations play no role vignette that was almost exactly the same, except that the at all in the relevant competencies, but that certain doctor’s internal monologue takes a somewhat different additional factors are somehow “biasing” or “distorting” turn: people’s cognitive processes and thereby allowing their . . . Dr. Bennett thinks to himself, “This bum deserves to die. intuitions to be affected by moral judgments. He shouldn’t sit here soaking up my valuable time and The first thing to notice about the debate between these resources. The best thing to do would be to disconnect two approaches is that we are unlikely to make much pro- him from the machine.” gress on it as long as the two positions are described only in These subjects were asked the same question: whether it these abstract, programmatic terms. Thus, suppose that would be more appropriate to say that the doctor ended we are discussing a new experimental result and the man’s life or allowed it to end. someone says: “Well, it could always turn out that this 320 BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 Knobe: Person as scientist, person as moralist effect is due to some kind of interfering factor.” How would we even begin to test such a conjecture? As long as the claim is just about the possibility of “some kind of interfering factor,” it is hard to know where one could go to look for confirming or disconfirming evidence. Fortunately, however, the defenders of alternative Figure 3. Traditional account of the process underlying blame hypotheses have not simply put forward these sorts of ascription. abstract, programmatic conjectures. Instead, they have developed sophisticated models that make it possible to offer detailed explanations of the available experimental it be that their moral judgments affect their intuitions data. Such models start out with the idea that people’s about causation? actual competence includes no role for moral consider- To resolve this question, one might develop a model ations, but they then posit various additional psychological that goes more like the one shown in Figure 4 factors that explain how people’s moral judgments might In this revised model, there is a reciprocal relationship nonetheless influence their intuitions in specific cases. between people’s blame judgments and their intuitions Each such alternative explanation then generates further about intention, causation, et cetera. As soon as people predictions, which can in turn be subjected to experimen- observe behavior of a certain type, they become motivated tal test. There has been a great deal of research in recent to find some way of blaming the agent. They then look to years devoted to testing these models, including some the evidence and try to find a plausible argument in favor ingenious new experiments that enable one to get a of the view that the agent fulfills all of the usual criteria better handle on the complex cognitive processes under- for responsibility. If they can construct a plausible argument lying people’s intuitions. At this point, then, the best there, they immediately blame the agent. Otherwise, they approach is probably just to look in detail at some of the reluctantly determine that the agent was not actually blame- most prominent explanations that have actually been pro- worthy after all. In short, the hypothesis says that people’s posed and the various experiments that have been devised intuitions about intention and causation affect their blame to test them. judgments but that the causal arrow can also go in the other direction, with people’s drive to blame the agent dis- torting their intuitions about intention and causation. 4.1. The motivational bias hypothesis One of the main sources of support for such a hypoth- Think of the way a District Attorney’s office might conduct esis is the well-established body of theoretical and exper- its business. The DA decides to prosecute a suspect and imental work within social psychology exploring similar hands the task over to a team of lawyers. These lawyers effects in other domains. There is now overwhelming evi- then begin looking at the case. Presumably, though, they dence that motivational biases can indeed lead people to do not examine the evidence with perfectly unbiased interpret evidence in a biased manner (for a review, see eyes. They have been hired to secure a conviction, and Kunda 1990), and, within moral psychology specifically, they are looking at the evidence with a view to achieving there is a growing body of evidence suggesting that this goal (cf. Tetlock 2002). One might say that they are people often adopt certain views as part of a post hoc under the influence of a motivational bias. attempt to justify prior moral intuitions (Ditto et al. A number of researchers have suggested that a similar 2009; Haidt 2001). So the motivational bias hypothesis is mechanism might be at the root of the effects we have perhaps best understood as the application to a new been discussing here (Alicke 2008; Nadelhoffer 2006a). domain of a theoretical perspective that is already quite Perhaps people just read through the story and rapidly well supported elsewhere. and automatically conclude that the agent is to blame. More importantly, the hypothesis makes it possible to Then, after they have already reached this conclusion, explain all of the existing results without supposing that they begin casting about for ways to justify it. They try to moral considerations actually play any role at all in any attribute anything they can – intention, causation, et of the relevant competencies. The thought is that cetera – that will help to justify the blame they have people’s competencies are entirely non-moral but that a already assigned. In essence, the suggestion is that the motivational bias then interferes with our ability to apply phenomena under discussion here can be understood as these concepts correctly. (An analogous case: If John the results of a motivational bias. sleeps with Bill’s girlfriend, Bill may end up concluding This suggestion would involve a reversal of the usual that John’s poetry was never really any good – but that view about the relationship between people’s blame judg- does not mean that Bill’s criteria for poetry actually ments and their intuitions about intention, causation, and involve any reference to sexual behavior.) so forth. The usual view of this relationship looks some- All in all, then, what we have here is an excellent thing like what’s shown in Figure 3 hypothesis. It draws on well-established psychological Here, the idea is that people first determine that the theory, provides a clear explanation of existing results, agent fulfilled the usual criteria for moral responsibility (intention, cause, etc.) and then, on the basis of this initial judgment, go on to determine that the agent deserves blame. This sort of model has a strong intuitive appeal, but it does not seem capable of explaining the experimental data reviewed above. After all, if people determine whether or not the agent caused the outcome before they make any sort of moral judgment, how could Figure 4. Motivational bias account of blame ascription. BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 321 Knobe: Person as scientist, person as moralist and offers a wealth of new empirically testable predictions. judgment of blame. This is the type of judgment we have The one problem is that when researchers actually went been discussing thus far, and it certainly does play an out and tested those new predictions, none of them were important role in people’s psychology. But it is not the empirically confirmed. Instead, the experimental results only type of moral judgment people make. They also again and again seemed to go against what would have make judgments about whether an agent did something been predicted on the motivational bias view. At this morally wrong, about whether a behavior violated point, the vast majority of researchers working on these people’s moral rights, about whether its consequences questions have therefore concluded that the motivational were bad. A complete theory of moral cognition would bias hypothesis cannot explain the full range of experimen- have to distinguish carefully between these various types tal findings and that some other sort of psychological of moral judgments and explain how each relates to process must be at work here (Hindriks 2008; Machery people’s intuitions about intention, causation, and the like. 2008; McCann 2005; Nichols & Ulatowski 2007; Turner In any case, as soon as we distinguish these various types 2004; Wright & Bengson 2009; Young et al. 2006). of moral judgment, we see that it would be possible for people’s intuitions to be influenced by their moral judg- 4.1.1. Neuropsychological studies. The usual way of ments even if these intuitions are not influenced by understanding the motivational bias hypothesis is that blame in particular. In fact, a growing body of experimen- reading through certain kinds of vignettes triggers an tal evidence suggests that the process actually proceeds in immediate affective reaction, which then distorts a quite different way (see Fig. 5). people’s subsequent reasoning (Nadelhoffer 2006a). An This model involves a quite radical rejection of the view obvious methodology for testing the hypothesis is there- that people’s intuitions about intention, causation, et fore to find people who don’t have these immediate affec- cetera, are distorted by judgments of blame. Not only tive reactions and then check to see whether these people are these intuitions not distorted by blame, they are not still show the usual effect. even influenced by blame at all. Rather, people start out Young et al. (2006) did just that. They took the cases of by making some other type of moral judgment, which the corporate executive who harms or helps the environ- then influences their intuitions about intention and causa- ment and gave these cases to subjects who had lesions in tion, which in turn serves as input to the process of asses- the ventromedial prefrontal cortex (VMPFC). Previous sing blame. experiments had shown that such subjects have massive Though this model may at first seem counterintuitive, it deficits in the ordinary capacity for affective response. has received support from experimental studies using a They show little or no affective response in situations wide variety of methodologies. To take one example, where normal subjects would respond strongly (Damasio Guglielmo and Malle (in press) gave subjects the vignette et al. 1990), and when they are presented with moral about the chairman and the environment and then used dilemmas in which most people’s answers seem to be structural equation modeling to test various hypotheses shaped by affective responses, they end up giving about the relations among the observed variables. The answers that are radically different from those given by results did not support a model in which blame judgments normal subjects (e.g., Koenigs et al. 2007). The big ques- affected intuitions about intentional action. In fact, the tion was whether they would also give unusual answers analysis supported a causal model that went in precisely on the types of questions we have been examining here. the opposite direction: it seems that people are first arriv- The results showed that they did not (Young et al. 2006). ing at an intuition about intentional action, and that this Just like normal subjects, the VMPFC patients said that intuition is then impacting their blame judgments. In the chairman harmed the environment intentionally but short, whatever judgment it is that affects people’s inten- helped the environment unintentionally. In fact, one tional action intuitions, the statistical results suggest that hundred percent of patients in this study said that the it is not a judgment of blame per se. environmental harm was intentional. On the basis of this In a separate experiment, Guglielmo and Malle (2009) experimental result, Young and colleagues concluded used reaction time measures to determine how long it that the asymmetry observed in normal subjects was not, took subjects to make a variety of different types of judg- in fact, due to an affective reaction. ments. The results showed that people generally made But, of course, even if it turns out that affective reac- judgments of intentional action before they made judg- tions play no role in these effects, the motivational bias ments of blame. (There was even a significant effect in hypothesis would not necessarily be refuted (Alicke this direction for some, though not all, of the specific 2008). After all, it is important to distinguish carefully cases we have been considering here.) But if the blame between affect and motivation, and we need to acknowl- judgment does not even take place until after the inten- edge the possibility that people are experiencing a motiva- tional action judgment has been completed, it seems that tional bias that does not involve any kind of affect at all. people’s intentional action judgments cannot be distorted Perhaps people just calmly observe certain behaviors, by feedback from blame. rapidly arrive at certain moral appraisals, and then find Finally, Keys and Pizarro (unpublished data) developed themselves trying to justify a judgment of blame. a method that allowed them to manipulate blame and then This proposal is, I believe, an interesting and suggestive one. To address it properly, we will need to develop a more complex theoretical framework. 4.1.2. Types of moral judgment. To begin with, we need to distinguish between a variety of different types of moral judgment. One type of moral judgment is a Figure 5. Distinct processes of moral judgment. 322 BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 Knobe: Person as scientist, person as moralist look for an effect on intuitions about intentional action. The key claim now is that it is the second of these judg- Subjects were given the vignettes about the agent who ments, rather than the first, that is influencing people’s either helps or harms the environment, but they were intuition that the professor caused the outcome. also randomly assigned to receive different kinds of infor- To test this claim empirically, we need to come up with a mation about the character of this agent. Some were given case in which the agent is judged to have performed a bad information that made agent look like a generally nice action but in which the agent is nonetheless not judged to person; others were given information that made the be blameworthy for the outcome that results. One way to agent look like a generally nasty person. The researchers construct such a case would be to modify our original could then examine the impact of this manipulation on story by switching the outcome over to something good. intuitions about blame and about intentional action. (For example: the receptionist was planning to stab the Unsurprisingly, people’s intuitions about blame were department chair’s eye out with a pen, but now that all of affected by the information they received about the the pens have been taken, her plan is thwarted, and the agent’s character, but – and this is the key result of the department chair’s eyes are saved.) In such a case, the pro- experiment – this information had no significant impact fessor would still be performing a bad action, but there on people’s intuitions about intentional action. Instead, would not even be a question as to whether he was “to intuitions about intentional action were affected only by blame” for the outcome that resulted, since there would information about the actual behavior (helping vs. be no bad outcome for which anyone could deserve blame. 2 harming) the agent was said to have performed. Experiments using this basic structure have arrived at a In the face of these new results, friends of the motiva- surprising pattern of results (Hitchcock & Knobe 2009). tional bias view might simply retreat to a weaker position. Even when the outcome has been switched to something They might say: “Okay, so we initially suggested that good, people continue to have the same causal intuitions. people’s intuitions were distorted by an affective reaction They still conclude that the agent who performed the associated with an impulse to blame, but we now see bad action is more of a cause than the agent who per- that the effect is not driven by affect and is not caused formed the good action. Yet when the outcome is some- specifically by blame. Still, the basic idea behind the thing good, it seems impossible to explain this pattern in theory could nonetheless be on track. That is to say, it terms of a motivational bias. After all, friends of the moti- could still be that people’s intuitions are being distorted vational bias hypothesis would then have to say that people by an effort to justify some kind of moral judgment.” are displeased with the agent who performs the bad action, that their intuitions thereby become distorted by moral judgment, and that they end up being motivated to con- 4.1.3. Cause and blame. This approach certainly sounds clude: “This bad guy must have been the sole cause of good in the abstract, but as one proceeds to look carefully the wonderful outcome that resulted.” It seems quite dif- at the patterns of intuition observed in specific cases, it ficult, however, to see how such a conclusion could poss- starts to seem less and less plausible. The difficulty is ibly serve as a post hoc justification for some kind of that the actual patterns observed in these cases just don’t negative moral judgment. make any sense as an attempt to justify prior moral judgments. 4.1.4. Conclusion. Of course, it might ultimately prove For a simple example, consider the case in which the possible to wriggle out of all of these difficulties and receptionist runs out of pens and people conclude that show that the data reviewed here do not refute the motiva- the professor is the sole cause of the problem that tional bias hypothesis. But even then, a larger problem results. In this case, it seems that some kind of moral judg- would still remain. This problem is that no one ever ment is influencing people’s intuitions about causation, seems to be able to produce any positive evidence in but which moral judgment is doing the work here? One favor of the hypothesis. That is, no one seems to be able obvious hypothesis would be that people’s intuitions to provide evidence that motivational biases are at the about causations are being influenced by a judgment root of the particular effects under discussion here. that the agent deserves blame for the outcome. If this There is, of course, plenty of evidence that motivational hypothesis were correct, it would make a lot of sense to biases do in general exist (e.g., Kunda 1990), and there are suggest that people’s intuitions were being distorted by a beautiful experimental results showing the influence of motivational bias. The idea would be that people want to motivational biases in other aspects of moral cognition conclude that the professor is to blame for a particular (Alicke 2000; Ditto et al. 2009; Haidt 2001), but when it outcome and, to justify this conclusion, they say that he comes to the specific effects under discussion here, there is the sole cause of this outcome. are no such experiments. Instead, the argument always The one problem is that the data don’t actually suggest proceeds by drawing on experimental studies in one that people’s causal intuitions are being influenced by a domain to provide evidence about the psychological pro- judgment that the agent is to blame for the outcome. cesses at work in another (see, e.g., Nadelhoffer 2006a). Instead, the data appear to suggest that these intuitions That is, the argument has roughly the form: “This expla- are being influenced by a judgment that the agent’s nation turned out to be true for so many other effects, so action itself is bad. So, for example, in the case at hand, it is probably true for these ones, as well.” we can distinguish two different moral judgments that It now appears that this strategy may have been leading people might make: us astray. The basic concepts at work in the motivational bias explanation – affective reactions, post hoc rationaliz- (a) The professor is to blame for the outcome (the reception- ist’s lack of pens). ation, motivated reasoning – have proved extraordinarily (b) There is something bad about the professor’s action (taking helpful in understanding other aspects of moral cognition. a pen from the desk). But moral cognition is a heterogeneous phenomenon. BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4 323 Knobe: Person as scientist, person as moralist What proves helpful in thinking about certain aspects of it It might be thought that people’s concept of causation may prove utterly irrelevant in thinking about others. does apply in cases like this one, but it also seems that sub- jects might quite reasonably infer that the real point of the question is to figure out whether the administrative assist- ant deserves blame for this outcome and that they might 4.2. The conversational pragmatics hypothesis therefore check the circle marked “definitely disagree.” Let us turn, then, to a second possible alternative hypoth- Before going on any further, it might be helpful to take a esis. When people are engaged in ordinary discussions, moment to emphasize just how different this pragmatic their use of words does not simply serve as a straightfor- hypothesis is from the motivational bias hypothesis we dis- ward reflection of the way they apply the corresponding cussed above. The motivational bias hypothesis posits an concepts. Instead, people strive to act as helpful conversa- error that affects people’s understanding of certain tion partners, following certain complex principles that morally relevant events. By contrast, the pragmatic enable them to provide useful information to their audi- hypothesis does not involve any error or even any effect ence. The study of these principles falls under the on people’s understanding of events. It simply suggests heading of “conversational pragmatics,” and researchers that people are applying certain kinds of conversational engaged in this study have illuminated many puzzling rules. The basic idea is that moral considerations aren’t aspects of the way people ordinarily use language in com- actually affecting people’s understanding of the situation; munication. A number of researchers have suggested that it’s just that moral considerations do sometimes affect this approach might also serve to explain the phenomena people’s view about which particular words would be we are trying to understand here (Adams & Steadman best used to describe it. 2004a; 2004b; Driver 2008a; 2008b). In any case, although the two hypotheses are very differ- To get a sense for this hypothesis, it might be helpful to ent in their theoretical approaches, they have proved start out by looking at a potentially analogous case in remarkably similar in their ultimate fate. Like the motiva- another domain. Imagine that you have a bathroom in tional bias hypothesis, the pragmatic hypothesis initially your building but that this bathroom is completely non- looked very promising – a clear and plausible explanation, functional and has been boarded up for the past three backed by a well-supported theoretical framework – but, years. And now imagine that someone hands you a ques- as it happened, the actual empirical data just never came tionnaire that asks: out the way the pragmatic hypothesis would predict. Do you have a bathroom in your building? Indeed, the pragmatic hypothesis suffers from many of __Yes __No the same problems that plagued the motivational bias It does seem that your actual concept bathroom might cor- hypothesis, along with a few additional ones that are all rectly apply to the room in your building, but when you its own. receive this question, you immediately have an under- standing of what the questioner really wants to know – 4.2.1. Patient studies. One way to test the hypothesis namely, whether or not you have a bathroom that actually would be to identify subjects who show an inability to works – and you might therefore choose to check the box use conversational pragmatics in the normal way, and marked “No.” then to check to see whether these subjects still show With these thoughts in mind, consider what might the usual effect. Zalla, Machery, and Leboyer did exactly happen when subjects receive a questionnaire that asks that in a recent study (Zalla et al. 2010). They took the whether they agree or disagree with the sentence: story about the chairman who harms or helps the environ- ment and presented it to subjects with Asperger’s syn- The chairman of the board harmed the environment intentionally. drome, a developmental disorder characterized by difficulties in certain forms of communication and a strik- W W W W W W W ing inability to interact normally with others. Previous studies had shown that subjects with Asperger’s display definitely unsure definitely remarkable deficits in the capacity to understand conver- disagree agree sational pragmatics, tending instead to answer questions in the most literal possible way (e.g., De Villiers et al. It might be thought that people’s concept of intentional 2006; Surian et al. 1996). If the original effect had been action does not, in fact, apply to cases like this one; but due entirely to pragmatic processes, one might therefore that, as soon as they receive the questionnaire, they form have expected subjects with Asperger’s to respond quite an understanding of what the questioner really wants to differently from neurotypical subjects. know. The real question here, they might think, is But that is not what Zalla and colleagues found. Instead, whether the chairman deserves to be blamed for his be- they found that subjects with Asperger’s showed exactly havior, and they might therefore check the circle marked the same pattern of responses observed in previous “definitely agree.” studies. Just like neurotypical subjects, people with Asper- Similar remarks might be applied to many of the other ger’s tended to say that the chairman harmed the environ- effects described above. Thus, suppose that subjects are ment intentionally but helped it unintentionally. This asked whether they agree or disagree with the sentence: result suggests that the pattern displayed by subjects in The administrative assistant caused the problem. earlier studies is not, in fact, a product of their mastery of complex pragmatic principles. W W W W W W W definitely unsure definitely 4.2.2. Cancelation. Of course, the study of linguistic def- disagree agree icits in people with Asperger’s brings up a host of complex 324 BEHAVIORAL AND BRAIN SCIENCES (2010) 33:4