Journal of Empirical Legal Studies Volume 5, Issue 4, 751-789, December 2008 Nothing But the Truth? Experiments on Adversarial Competition, Expert Testimony, and Decision Making Cheryl Boudreau and Mathew D. McCubbins* Many scholars debate whether a competition between experts in legal, political, or economic contexts elicits truthful information and, in turn, enables people to make informed decisions. Thus, we analyze experimen- tally the conditions under which competition between experts induces the experts to make truthful statements and enables jurors listening to these statements to improve their decisions. Our results demonstrate that, con- trary to game theoretic predictions and contrary to critics of our adversarial legal system, competition induces enough truth telling to allow jurors to improve their decisions. Then, when we impose additional institutions (such as penalties for lying or the threat of verification) on the competing experts, we observe even larger improvements in the experts' propensity to tell the truth and in jurors' decisions. We find similar improvements when the competing experts are permitted to exchange reasons for why their statements may be correct. *Address correspondence to Cheryl Boudreau, University of California, Davis, Department of Political Science, One Shields Ave., Davis, CA 95616; email: [email protected]. Boudreau is Assistant Professor of Political Science, University of California, Davis. McCubbins is Chancel- lor's Associates Chair of Political Science, University of California, San Diego; Visiting Professor of Law, University of Southern California; Adjunct Professor of Law, University of San Diego. This article was presented at the Conference on Empirical Legal Studies at New York Uni- versity, 2007. We thank Jennifer Arlen, Geoffrey Miller, and Ted Eisenberg for their generous invitation to present our work at that conference. We also thank the National Science Founda- tion, Grant SES-0616904, the Kavli Institute for Brain and Mind, and the Chancellor's Associates Chair VIII at UC San Diego for providing financial support for these experiments. We are also gratefil to William Heller, Scott MacKenzie, Rebecca Morton, Jeff Rachlinski, Dan Rodriguez, Joel Sobel, Matthew Spitzer, Jeff Staton, Lydia Tiede, and members of the University of San Diego Law School for helpful comments on an earlier draft of this article. © 2008, Copyright the Authors Journal compilation © 2008, Cornell Law School and Wiley Periodicals, Inc. 752 Boudreau and McCubbins "LJ]urors sitting in a civil case frequently encounter not only one but several- usually conflicting-presentations of expert testimony." Kutnjak Ivkovic and Hans (2003:478) "[The] central precept of [the] adversary process is that out of the sharp clash of proofs presented by adversaries in a highly structured ... setting is most likely to come the information from which a neutral ... decision maker can resolve a litigated dispute." Landsman (1983:714) In many legal, political, and economic contexts, people must make decisions about which they are not fully informed. Because people typically lack the opportunity and/or inclination to gather detailed information about these decisions for themselves, they must often base their decisions on the state- ments of others (Sniderman et al., 1991; Lupia, 1992, 1994; Mondak, 1993; Lupia & McCubbins, 1998; Boudreau, 2006). For example, when deciding a question at trial, jurors must rely on the statements of competing attorneys and their witnesses. Or, when choosing among different candidates for office, uninformed voters may rely on the statements of politicians compet- ing in a debate and on the endorsements of trusted allies. Similarly, when choosing among products, consumers often rely on information provided by competing sellers and on the opinions of trusted endorsers, such as Con- sumer Reports, the Better Business Bureau, Good Housekeeping, and Morn- ingstar. A key feature of the above contexts is competition between experts (who may be attorneys, witnesses, politicians, or sellers). Although much research in economics and political science suggests that competition does not necessarily produce beneficial outcomes (Crawford & Sobel, 1982; Austen-Smith, 1990a, 1990b, 1993; Lupia & McCubbins, 1998), there is a long-standing debate in legal scholarship over the use of expert testimony in courtroom settings. Specifically, some scholars fear that our adversarial legal system does not necessarily reveal truthful infor- mation and allow jurors to make informed decisions (Pound, 1906; Frank, 1945; Tullock, 1975, 1980; Bundy & Elhauge, 1991; Kaplow & Shavell, 1989). On the other hand, several scholars suggest that competition between experts in a courtroom will lead to the revelation of truthful infor- mation (Milgrom & Roberts, 1986; Lipman & Seppi, 1995; Froeb & Koba- yashi, 1996). Given the disagreement among scholars about the effects of competi- tion, we use laboratory experiments to assess whether and when competition induces experts to make truthful statements and enablesjurors to learn from Adversarial Competition, Expert Testimony, and Decision Making these statements and improve their decisions.' Contrary to expectations from cheap-talk games (Crawford & Sobel, 1982; Lupia & McCubbins, 1998) and contrary to critics of our adversarial legal system, our experiments demonstrate that competition between experts does in fact help subjects to improve their decisions. This result is surprising because, in our experi- ments, subjects receive statements from two competing experts, but they do not know which expert has an incentive to tell them the truth and which expert does not. Thus, as in cheap-talk models, subjects cannot readily distinguish true statements from lies. In such situations, subjects should ignore the experts' statements and make their decisions on their own (see, e.g., Lupia & McCubbins, 1998), but this is not what we observe. Specifically, we find that competition by itself (even when it is cheap talk) induces enough truth telling from the experts to allow subjects to make better decisions than subjects in a control group who make decisions without the benefit of experts' statements. Importantly, our results show that the positive effect of competition is driven by those instances in which both competing experts make truthful statements to subjects. Rather than viewing the experts' statements as cheap talk and ignoring them (as expected), subjects appear to rely on a simple (and effective) rule of thumb. Namely, assume that when both experts make the same statements, they must be telling the truth. This rule of thumb turns out to be quite helpful, as it enables subjects to improve their decisions even when they are exposed to competing experts that do not both have an incentive to be truthful. When we impose additional institutions (such as a penalty for lying or a threat of verification) on the competing experts, we find that these addi- tional institutions frequently increase the experts' propensity to make truth- ful statements and enable subjects to achieve even larger improvements in their decisions. Interestingly (and somewhat surprisingly), we find that, for the range of penalties we examine, smaller penalties for lying can be just as effective as large ones at increasing the experts' propensity to tell the truth and helping subjects to improve their decisions. Further, our results show that adding a small penalty for lying to the threat of verification dramatically increases the extent to which verification induces the experts to make truth- ful statements and helps subjects to improve their decisions. 1We refer to the people receiving the competing experts' statements asjurors throughout this article. That said, the lessons about the effects of competition also apply to voters and consumers who receive the statements of competing experts in political and economic contexts. 754 Boudreau and McCubbins When the competing experts have an opportunity to exchange reasons with one another, subjects are also able to improve their decisions. This improvement occurs despite the absence of additional institutions in this treatment condition and the fact that the exchange of reasons does not increase the experts' propensity to tell the truth. Indeed, although subjects in this experimental condition always receive conflicting statements from the experts, the exchange of reasons helps them improve their decisions, over and above the improvements subjects achieve when the competing experts do not exchange reasons with one another. This result is quite interesting, given that the competing experts' exchange of reasons is still cheap talk and should not (and indeed does not) increase the experts' propensity to tell the truth. Thus, it appears that the improvements in subjects' decisions are driven by the experts' reasons alone, and not by an increase in truth telling. This article proceeds as follows. We begin by describing our experi- mental design, which exposes subjects to the statements of two competing experts before they must make their decisions. Then, we make predictions regarding whether and under what conditions (1) the experts will be more likely to make truthful statements and (2) subjects receiving the experts' statements will improve their decisions. Next, we present our experimental results. We conclude with a discussion of the implications that our results have for debates about the effects of competition in legal, political, and economic contexts. Specifically, we emphasize that because competition in our adversarial legal system (and in many political and economic settings) incorporates penalties for lying, verification, and the exchange of reasons, our results suggest that testimony from competing experts can in fact lead to large improvements in decision making. I. THE DEBATE: DOES COMPETITION BETWEEN EXPERTS HELP JURORS LEARN? For decades, scholars have extolled the virtues of competition, arguing that we should incorporate more competitive practices into our legal, political, and economic systems. This idea that competition will produce beneficial outcomes is supported by a long line of theoretical and empirical research suggesting that competition between actors leads to the exposure or verifi- cation of false statements and, therefore, reveals truthful information and improves decision making (Milgrom & Roberts, 1986; Lipman & Seppi, 1995; Froeb & Kobayashi, 1996; Freedman, 1975; Kim, 2001; Walpin, 2003). Adversarial Competition, Expert Testimony, and Decision Making For example, Milgrom and Roberts (1986) demonstrate that an unsophisti- cated decisionmaker will make a fully informed decision as long as the interests of the competing litigants are sufficiently opposed. On the other hand, many scholars caution that competition is benefi- cial only under certain conditions and that it may even have negative effects (Pound, 1906; Frank, 1945; Tullock, 1975, 1980; Frankel, 1980; Langbein, 1985; Bundy & Elhauge, 1991; Kaplow & Shavell, 1989; Shin, 1998; Posner, 1999; Daughety & Reinganum, 2000; Lupia & McCubbins, 1998; Weigend, 2003; Sniderman & Theriault, 2004; Chong & Druckman, 2007). Further, research in cognitive science, political science, and economics questions the assumption (implicit in many arguments in favor of competition) thatjurors listen to competing experts, as well as trust and learn from the information that they offer. Indeed, the cognitive science literature emphasizes that listening and learning are costly behaviors, at least insofar as people must forego the opportunity to do other things while they listen and learn (Jack- endoff, 1980; for a survey, see Lupia & McCubbins, 1998; Lupia, 2002; McCubbins & Rodriguez, 2006). Further, because humans have limited time and energy, they are able to pay attention to and remember only a small fraction of the information available to them (Schacter, 2001; Lupia, 2004; Lupia & McCubbins, 1998). And, as the political science and economics literatures demonstrate, "talk is cheap"; that is, without some cost associated with making false statements, there is no way to ensure that competing experts tell the truth (Crawford & Sobel, 1982; Austen-Smith, 1990a, 1990b, 1993; Lupia & McCubbins, 1998). It is this research on the effects of competition that we build on in this study. Specifically, we draw on the insights of Lupia and McCubbins (1998) and the experimental design of Boudreau (2006) to analyze the conditions under which competition between two experts consistently induces both experts to make truthful statements and enables jurors to trust those state- ments and, thereby, improve their decisions. Although there are many theo- retical, experimental, and empirical studies of competition, our experimental 'McCubbins and Rodriguez (2006) conduct experiments in which subjects can receive infor- mation from others, either for free or after paying a cost. Their results demonstrate that when listening is free, 97 percent of subjects choose to receive information from others. When there is a small cost to receive information, the percentage of subjects that choose to receive infor- mation drops to 50 percent. When a larger cost is imposed, only 4 percent of subjects choose to receive information. Taken together, these results demonstrate that people are less willing to listen to information from others when there is a cost associated with doing so. 756 Boudreau and McCubbins design makes a number of new contributions to the literature and has several important advantages. The first advantage of our design stems from the nature of the choices that subjects make in our experiments. Specifically, instead of asking subjects to make decisions about fictional court cases or to vote for fictional candi- dates (as psychologists, legal scholars, and political scientists often do when running experiments), we ask subjects to make choices about math problems after they hear two expert subjects make statements about whether answer "a" or answer "b" is the correct answer. (Note that the two subjects who are chosen to be the experts are shown the correct answer to a particular math problem before they make their statements; thus, they are made expert by the experimenter. This is common knowledge to all participants in the experiment.) One reason that this type of decision is advantageous is that solving math problems provides a straightforward way to identify correct decisions and to assess whether and when the experts' statements induce an improvement in decision making. Stated differently, although it is often difficult to identify when jurors have chosen the "correct" legal decision or candidate,' it is very easy to tell when they have chosen the correct answer to a math problem. Similarly, because we know the correct answer to each math problem and show that answer to the competing experts, we can easily identify whether and when the experts tell the truth. The second advantage of our design is that math problems, even though they do not look like legal or political decisions on the surface, capture many key characteristics of the information that jurors receive and the decisions that jurors make in real-world competitive contexts. Thus, they can tell us a great deal about howjurors in the real world make choices. For example, in real-world competitive contexts,jurors are not blank slates when they listen to the statements of experts. That is, they often have preexisting knowledge or beliefs about the topics that the experts discuss. Similarly, subjects in our experiments are not blank slates when they hear the experts' statements about whether answer "a" or answer "b" is the correct choice because they have preexisting knowledge about how to solve math problems. 3See Seidman Diamond (2003) for a discussion of the difficulties associated with identifying whether and when jurors make "correct" decisions. Indeed, she states: "To assess how the jury operates as a decision-maker, we cannot compare the jury's verdict with some gold standard of truth because no such dependable standard exists... [I] n the end we cannot be certain that the correct conclusions have been drawn" (2003:150-51). Adversarial Competition, Expert Testimony, and Decision Making That said, jurors in the real world might be uncertain about their decisions; that is, they may not know which choice (e.g., guilty vs. not guilty) is the correct one. Similarly, subjects in our experiments may be uncertain about whether "a" or "b" is the correct choice. As in the real world, the uncertainty that subjects experience depends, in part, on their levels of sophistication. Indeed,just as unsophisticatedjurors in the real world may be more uncertain about which choice is correct, so, too, may unsophisticated subjects in our experiments be more uncertain about whether "a" or "b" is the correct choice. And,just as jurors in the real world vary in their levels of sophistication, so, too, do our subjects, as their SAT math scores range from 450 (the 27th percentile) to 800 (a perfect score). Further, in real-world competitive contexts, there is something at stake for jurors when they make their decisions, but the stakes may not be very large. This is especially true in low-profile, run-of-the-mill court cases. Simi- larly, there is something at stake for subjects in our experiments because they earn money if they make a correct choice and lose money if they make an incorrect choice. As is the case in many real-world settings, the stakes in our experiments are not very large, as subjects either earn or lose 50 cents for each decision they make. Further, jurors in real-world competitive contexts often receive factual information, and they must then make decisions that are objectively correct or incorrect. For example, in courtroom settings, jurors listen to factual information that competing witnesses provide and then make a decision about whether the accused is guilty or innocent or whether a party to a civil suit is liable or not. Interestingly, in both criminal and civil trials, jurors are often exposed to mathematical information that is not unlike the informa- tion that subjects in our experiments receive (Lilly, 2001; Fisher, 2000- 2001).' Like jurors in the real world, subjects in our experiments receive factual, mathematical information and then make decisions that are either correct or incorrect. Given the many similarities between real-world legal and political decisions and decisions about math problems, there is a close mapping between the psychological processes that occur in our experiments and the psychological processes used by jurors in real-world competitive 4Indeed, Lilly (2001:71) states: "[W]ith increasing frequency, contemporary juries are faced with sophisticated, highly technical evidence drawn from such diverse fields as economics, mathematics, statistics, psychiatry, engineering, epidemiology, toxicology, serology, and genetics." 758 Boudreau and McCubbins contexts (stated differently, our experiments have a great deal of psychologi- cal realism; see Aronson et al., 1998). II. EXPERIMENTAL DESIGN To shed light on debates about the effects of competition, we conduct laboratory experiments. To this end, we randomly assign subjects to either a control group or to one of several different treatment groups. We then ask subjects in all groups to answer binary-choice math problems that are drawn from an SAT math test and consist of several different types of problems and various levels of difficulty. We tell subjects in both our treatment and control groups that they have 60 seconds to answer each math problem and that they will earn 50 cents for each problem they answer correctly, lose 50 cents for each problem they answer incor- rectly, and neither earn nor lose 50 cents if they leave a problem blank. The main difference between the treatment and control groups has to do with the conditions under which subjects answer the math problems. In the control group, subjects answer the math problems on their own, one at a time. The purpose of the control group is to establish a baseline for how well subjects perform on the math problems when they must make their choices on their own (i.e., without an opportunity to learn from two com- peting experts' statements). In our various treatment groups, subjects answer subsets of the math problems that subjects in the control group answer;5 however, subjects in the treatment groups receive the statements of two competing experts before they make their decisions. Specifically, before each math problem, the experimenter selects two subjects at random to act as the experts for that particular math problem. The experts are then shown the correct answer to a particular math problem (i.e., the experts are given knowledge about the correct choice), and they then choose what statement they 5Subjects in the control group answer 24 math problems. Because it takes a great deal of time to explain particular treatment conditions to subjects and select two subjects to act as the experts for each math problem, we were unable to have subjects answer 24 math problems in each treatment condition. Thus, subjects in each treatment condition answer subsets of the math problems that subjects in the control group answer. We control for the difficulty of the math problems in our statistical analyses to ensure that our results are not driven by differ- ences in the difficulty of the math problems used in each treatment condition and in the control group. Adversarial Competition, Expert Testimony, and Decision Making would like to make to subjects (statements take the form of answer "a" or answer "b"). Once the experts choose their statements, the "testimony" of the two experts is passed on to the remaining subjects (either on a single sheet of paper or read aloud).6 In either case, subjects receive a joint state- ment from the experts (e.g., "a, a"; "b, b"; "a, b"). Then, subjects must decide within 60 seconds whether to answer the problem, and if they choose to answer, whether to pick "a" or "b." Once subjects make their decisions, we move on to the next math problem. At this point, we randomly select two new subjects to act as the experts.7 We select two new subjects for each math problem to ensure that the competing experts do not develop reputations from one problem to the next.' Indeed, avoiding repeat-play effects is important because such effects could confound the treatments we impose. Further, because jurors' interac- tions with competing experts typically do not repeat over time (e.g., they are typically limited to the course of a trial), it makes sense to have subjects make "one-shot" decisions in our experiments.' That said, the key to our experimental design is threefold. First, one of the experts knows that he or she shares common interests with subjects (i.e., he or she benefits when the remaining subjects answer the math problem correctly), and the other expert knows that he or she has conflicting interests 6Note that the experimenter reads both experts' statements aloud to the subjects in order to prevent subjects from learning anything about the experts from the sound of their voices. Also, all subjects (including the experts) sit behind partitions so that their identities are anonymous. Note also that we do not actually refer to the experts as "experts" in the experiments. We simply refer to them as the reporters. 7In some experiments, we randomly select two new subjects to act as the experts by pulling two subject numbers out of a hat before each math problem. In other experiments, we ran- domly select four or more subjects at the beginning of the experiment to act as a panel from which we draw two competing experts on any particular math problem. Then, before distrib- uting each math problem, we draw two numbers out of a hat to determine which two subjects on the panel will act as the experts on that problem. We repeat this procedure for each math problem. "The total number of math problems that any one subject (acting as an expert) makes state- ments about depends on which procedure is used to select the experts, as well as the number of math problems used in a particular experiment. Across all experiments, the total number of math problems that an expert makes statements about ranges from 14 problems to 1 problem. Also, subjects answer between 5 and 24 math problems in each experiment. We control for this in our statistical analyses, and it does not affect our results. 'For an example of a repeated communication game, see Sobel (1985). 760 Boudreau and McCubbins with subjects (i.e., he or she benefits when the remaining subjects answer the math problem incorrectly). We refer to these as the common-interest and conflicting-interest experts, respectively. Second, it is common knowledge to all subjects that one expert shares common interests with them and that one expert has conflicting interests with them, but they are not told which expert shares common interests with them on any particular problem. Stated dif- ferently, subjects know that the experts are adversaries, but they do not know which expert's interests are aligned with their own."° Third, both experts and the subjects know that the experts can lie about the correct answer to the math problem or they can tell the truth; it is entirely up to them. The experts' ability to make whatever statement they wish is constant throughout this experiment and is designed to be an analogy to Crawford and Sobel's (1982) and Lupia and McCubbins's (1998) models, as well as to many real-world competitive settings. So how do we induce competition between the two experts within the context of our experiments? In short, we induce competition by manipulat- ing the ways that the experts and the subjects earn money. Specifically, the common-interest expert is paid 50 cents for each subject who answers a particular math problem correctly. The conflicting-interest expert is paid 50 cents for each subject who answers a particular math problem incorrectly. So, for example, if 11 subjects (the typical number used in our experiments) answer the math problem correctly, they earn 50 cents each, the common- "'We design our experiments in this way because we are interested in analyzing the conditions under which jurors can learn from competing experts who they do not know. This aspect of our experiments corresponds to many legal and political settings. Indeed, when learning from competing experts in a courtroom, jurors may not know which expert's interests are aligned with their own. Similarly, in primary elections, voters may not know which candidate (of the many candidates who share their party label) is more likely to have interests that are aligned with their own. That said, we could design our experiments differently. Indeed, if we were interested in analyzing the conditions under which jurors can learn from competing experts who they know something about, we could tell subjects that there is a 70 percent chance that the second expert has conflicting interests with them. Because the two experts in our experiments are adversaries, this means that subjects would also know that there is a 70 percent chance that the first expert has common interests with them. Knowing this, subjects should ignore the second expert's statement, pay attention to the first expert's statement, and base their choices on it. However, because this design simply turns our experiments into a test of how perceived common interests between an expert and ajuror affect trust, persuasion, and learning, and because the effects of common interests are well understood (see Lupia & McCubbins, 1998 for a game theoretic model and experiments demonstrating the effects of common interests, which work as expected), we focus instead on situations in which subjects do not know which expert is more likely to share common interests with them on any particular problem.