Arbitrage-Free Correlated Equilibria Robert F. Nau The Fuqua School of Business Duke University Durham, North Carolina 27708-0120 E-mail: [email protected] Fax: 919-684-2818 December 7, 1995 ABSTRACT A refinement of subjective correlated equilibrium—“arbitrage-free correlated equilibrium”— is derived from the requirement that the outcome of a noncooperative game should not present arbitrage opportunities to an outside observer when the players publicly accept small side-gambles consistent with their beliefs and preferences, regardless of whether they are risk neutral. An arbitrage-free correlated equilibrium is a correlated equilibrium in which the common prior assumption applies to the players’ risk neutral probabilities (products of probabilities and relative marginal utilities for money) rather than their true probabilities. This reformulation of the common prior assumption guarantees that equilib- rium expected payoffs for risk averse players are Pareto efficient, a condition not satisfied by completely mixed Nash or objective correlated equilibria. Key words: correlated equilibrium, joint coherence, arbitrage, common knowledge, com- mon prior assumption, risk neutral probabilities, separation of probability and utility The author is grateful to Jim Anton, Bob Aumann, Doug Foster, Maarten Janssen, Kevin McCardle, Herv´e Moulin, and participants in the Stony Brook Workshop on Knowledge and Game Theory and the Duke-UNC Microtheory Workshop for comments on earlier drafts. This research was supported by the Business Associates Fund and the Hanes Cor- poration Foundation Fund at the Fuqua School of Business. “If the players in this game are so smart, why are they playing this silly game? Why don’t they change the rules and play a game where they can do better?” (Remark of Abba Schwartz, cited by Kreps 1990a) 1. Introduction Suppose that the players in a noncooperative game may make monetary side- gambles with respect to actions of nature and their opponents. What kind of strategic be- havior is rational under such conditions, and what equilibrium concepts are appropriate? These questions are important for several reasons. First, side- gambles provide a medium through which players can credibly reveal information about their beliefs and preferences to each other and to outside observers. A game accompanied by side- gambles therefore meets a high operational standard of numerically precise common knowledge of utilities and probabilities, namely “putting one’s money where one’s mouth is.” The concepts of rationality that emerge here provide a normative standard against which rational play un- der murkier conditions can be judged. Second, game theoretic analysis is increasingly ap- plied directly to the study of strategic behavior in financial markets—environments where many forms of contingent claims are routinely traded and where new financial instruments can be created in response to incentives for efficiency and information. A theory of how ideal games should be played in contingent claim markets is a natural starting point for asking what insights game theory can bring to finance, and vice versa. To illustrate how side-gambles can be used to reveal the payoff structure and even determine the equilibrium solution of a game, consider the familiar matching-pennies game whose standard payoff matrix is shown in Table 1. Table 1. Matching pennies L R T (1, -1) (-1, 1) B (-1, 1) (1, -1) – 1 – Assume that payoffs are in units of money and that the players are risk neutral, so that money payoffs may also be interpreted as utilities. (The risk-averse case and its implica- tions for efficiency will be treated in the next section.) Player 1 (row) faces a choice be- tween two lotteries u and u , whose payoffs as a function of player 2’s strategy are 1T 1B u = (+1,−1) and u = (−1,+1), where the first (second) number in parentheses is 1T 1B the payoff when player 2 plays L (R). Let E (u) denote player 1’s expected value for an 1 arbitrary lottery u over player 2’s strategy. Then player 1 will rationally choose T if and only if E (u ) ≥ E (u ), or equivalently E (u − u ) ≥ 0. Notice that in this case 1 1T 1 1B 1 1T 1B (i.e., the case in which player 1 chooses T), a small side-gamble whose payoffs are propor- tional to u − u = (+2,−2) would also be acceptable to player 1, because such a side- 1T 1B gamble would yield a non-negative increase in her total expected payoff. That is, for any small α > 0: E (u ) ≥ E (u ) ⇐⇒ E (u +α(u −u )) ≥ E (u ). 1 1T 1 1B 1 1T 1T 1B 1 1T Now consider an outside observer who finds player 1 offering to accept a gamble proportional to u − u in the event she plays T. From the observer’s perspective, 1T 1B the strategies of both players are uncertain events, so a gamble between an observer and a player must be represented as a 4-tuple whose elements are the payoffs to be made to the player in the four possible outcomes of the game. The acceptability of the gamble u −u is conditioned on the event that player 1 plays T: its payoff is therefore zero (i.e., 1T 1B the gamble is called off) if player 1 does not choose T, and otherwise its payoff is +2 or -2 according to whether player 2 chooses L or R. Since the gamble is subject to an arbitary multiplier, it can be renormalized without loss of generality to a maximum absolute value of 1. The unconditional vector of payoffs from the observer to the player for the renormal- ized gamble is then: u = (+1,−1,0,0), (1a) 1TB – 2 – where the order of the outcomes inside the parentheses is TL, TR, BL, BR. (The nota- tion u should be read as “player 1’s relative utility differences between T and B, condi- 1TB tional on the former strategy being played.”) Similarly, in the event that player 1 chooses B, she presumably finds that E (u − u ) ≥ 0, in which case she would also accept a 1 1B 1T small gamble proportional to u − u = (−2,+2). The corresponding (renormalized) 1B 1T unconditional payoff vector is: u = (0,0,−1,+1), (1b) 1BT where the payoffs are now zeroed-out in the case that B is not played. Player 2, in turn, faces a choice between two lotteries u = (−1,+1) and u = (+1,−1), where the first 2L 2R (second) number in parentheses is the payoff if player 1 plays T (B). Letting E (u) denote 2 player 2’s expected value for an arbitrary such lottery u over player 1’s strategy, player 2 will choose L precisely in the case that E (u − u ) ≥ 0, in which case a small gamble 2 2L 2R proportional to u −u would also be acceptable to her. Conditioning the latter gamble 2L 2R on the event that player 2 chooses L yields the following renormalized payoff vector for a gamble between player 2 and an observer: u = (−1,0,+1,0) (1c) 2LR Finally, the payoff vector for the gamble that would be acceptable to player 2 in the event that she chooses R is: u = (0,+1,0,−1) (1d) 2RL The side-gambles described above do not change the information structure of the game nor the players’ strategic incentives. If the payoff matrix of the game is com- mon knowledge, as is normally assumed, then the acceptability of the four gambles {u ,u ,u ,u } is also naturally common knowledge among risk neutral play- 1TB 1BT 2LR 2RL ers. Moreover, the acceptance of any positive multiples of these gambles does not affect – 3 – the players’ preferences among their own strategies given their beliefs: the gambles merely amplify existing differences in expected payoffs between chosen and unchosen strategies. Side-gambles constructed in this manner, whose parameters depend only the players’ rela- tive preferences for outcomes and not on their beliefs, are generically known as “preference gambles” (Nau 1992, 1995a). There is no loss of generality, then, in assuming that the gambles (1a–d) are accept- able to the players—at least when they are risk neutral. Now abstract away the prior as- sumption of a commonly known game matrix and players who rationally choose strate- gies so as to maximize their expected payoffs, and never mind the troublesome assumption that rationality itself is common knowledge. Instead, consider only the perspective of an outside observer who finds available to him a set of gambles whose payoffs are pegged to four mutually exclusive and collectively exhaustive events which may (or may not) be un- der the control of various “players” to whom payments are made. What can the observer infer about the players’ beliefs concerning the outcome, and under what conditions will the observer conclude that they have behaved rationally? Since the gambles are acceptable in arbitrary non-negative multiples and are in a common currency, the observer may take any non-negative linear combination of them. A minimal ex ante standard of rationality is that it should be impossible for the observer to so construct a strictly negative aggregate payoff for the players—a.k.a. an arbitrage op- portunity or “Dutch book.” By the well-known theorem of de Finetti (1937, 1974), a col- lection of acceptable gambles does not admit a Dutch book if and only if there exists at least one probability distribution on the outcome space which assigns non-negative expec- tation to every gamble.1 If there is a unique such distribution, and if all the gambles have 1 Actually, an outright Dutch book—i.e., a uniformly negative payoff to the players—can never be constructed from preferencegamblesderivedfromthepayoffmatrixofagameamongriskneutralplayers. Thisresult,whichwasestablishedby NauandMcCardle(1990)usingaMarkovchainargument,yieldsanelementaryproofoftheexistenceofcorrelatedequilibria. TheMarkovchainargumentsubsequentlyhasbeenadaptedbyMyerson(1995)todefinetheconceptof“dualreduction.” – 4 – been offered by the same risk neutral individual, we interpret the distribution to represent the subjective beliefs of that individual. If different gambles have been offered by different individuals, the supporting probability distribution can be interpreted to represent their collective or commonly held beliefs. In the case of the matching pennies game, there is in fact a unique distribution assigning non-negative expectation to all four preference gam- bles, namely the distribution placing probability 1/4 on each outcome. This is also the unique Nash equilibrium distribution of the game, so it turns out that merely by accept- ing the gambles whose acceptability follows naturally from the payoff structure of the game, the players reveal an apparent collective belief that they are employing the Nash equilibrium concept. Nau and McCardle (1990) show that, when acceptable preference gambles are de- rived from a general multi-player noncooperative game in the manner described above, the probability distributions assigning them all non-negative expectation are in general objective correlated equilibria2 (Aumann 1974, 1987) rather than Nash equilibria. Further- more, the observer can construct partial Dutch books in which the aggregate payoff to the players is non-positive in all outcomes of the game and strictly negative in any outcome which does not occur with positive probability in some objective correlated equilibrium: the latter outcomes are “jointly incoherent” and should never occur among rational play- ers. Hence, when common knowledge of the rules of the game is defined by the acceptance of preference gambles and common knowledge of Bayesian rationality is defined by the avoidance of jointly incoherent outcomes, the solution concept that emerges is objective correlated equilibrium. This operational approach to defining knowledge and rationality in games deals seamlessly with the transition from individual rationality to strategic ra- tionality, and the results support Aumann’s (1987) contention that objective correlated 2 AnobjectivecorrelatedequilibriumisageneralizationofNashequilibriuminwhichrandomizedstrategiesofdifferent playersmaybecorrelated. Suchcorrelationcanbeinducedbytheuseofcommunicationdevicesand/orcorrelatedrandomization mechanisms. – 5 – equilibrium is the natural expression of Bayesian rationality in noncooperative games—at least among risk neutral players. The same approach has been applied by Nau (1992) to games with incomplete information, where it leads to the concepts of correlated Bayesian equilibrium (Forges 1995) and communication equilibrium (Myerson 1985, Forges 1986). The purpose of the present paper is to generalize the Nau- McCardle results to the case of players who are not risk neutral—i.e., who may have nonconstant marginal utility for money. The equilibrium concept which emerges here is no longer objective correlated equilibrium, but rather a refinement of subjective correlated equilibrium which will be called “arbitrage-free correlated equilibrium.” In a subjective correlated equilibrium, the players’ true conditional probabilities need not satisfy the Harsanyi doctrine of the com- mon prior distribution.3 However, in an arbitrage-free correlated equilibrium, the common prior assumption is satisfied by the players’ risk neutral probabilities, which are products of their true probabilities and relative marginal utilities for money. This reformulation of the Harsanyi doctrine reconciles the properties of noncooperative solution concepts (namely, that the players’ reciprocal beliefs should form an appropriate equilbrium) with the prop- erties of efficient allocation in financial markets and syndicates (namely, that the product of probability and relative marginal utility should be equalized across agents; Wilson 1968, Dr`eze 1970). It also deflects the criticism that that is often leveled at the Harsanyi doc- trine, namely that mutually consistent subjective probabilities are unlikely to arise spon- taneously in practice (Binmore and Brandenburger 1990, Kreps 1990b, Gul 1992) But mu- tually consistent risk neutral probabilities are merely the natural result of arbitrage trad- ing (Nau 1995b). 3 Inhisformulationofgamesofincompleteinformation, Harsanyi(1967)introducedtheassumptionthattheplayers’ beliefs about each others’ types, conditional on their own types, are consistent with a common prior distribution over types, thus obtaining an internally consistent model of an infinite regress of reciprocal beliefs. In a game of complete information, a Nash equilibrium or objective correlated equilibrium also has the property that the players’ conditional beliefs about their opponents strategies, given their own choices of strategies, are consistent with a common prior distribution over outcomes of thegame. – 6 – In addition to requiring a reformulation of the Harsanyi doctrine, non-risk-neutrality introduces two other novel elements into the analysis of games with side-gambles. One is that gambling by non-risk-neutral players partially endogenizes the rules of the game, admitting the possibility that a superior solution might be achieved by tampering with the rules. In extreme cases—e.g. strictly competitive games with completely mixed equilibria—the players may even entirely decouple their actions in this manner. This phe- nomenon illustrates that the natural decoupling effect of monetary transactions in public markets can sometimes mitigate the need for strategic behavior when the game is accom- panied by side-gambles.4 The second novel element is that the “true” rules and solution of the game—i.e., the players’ true payoff functions and conditional probabilities—need not be assumed to be directly observable. Instead, the “revealed” payoff functions and probabilities of the players may be distorted by their marginal utilities for money, reflecting the fact that pref- erence measurements which are tied to material rewards generally do not permit a com- plete separation of probability from utility (Kadane and Winkler 1988; Schervish, Seiden- feld, and Kadane 1990). An analysis of the same phenomenon in the context of single- agent decision analysis is given by Nau (1995a). Despite the fact that common knowl- edge of “true” utilities is normally considered to be the starting point for game-theoretic analysis—indeed, this was the motivation for von Neumann and Morgenstern’s (1944) ax- iomatization of cardinal utility—the distortions of probability and utility encountered here merely reshape some familiar solution concepts rather than rendering analysis impossible. The remainder of the paper is organized as follows. Section 2 presents examples (matching-pennies and battle-of-the-sexes with risk averse players) which illustrate the principles and techniques to be developed later. Section 3 formalizes the assumptions 4 Strategic behavior does not disappear altogether when players decouple their actions in the game through an accu- mulationofgambles. Rather,strategicinteractionsoccuratamoreprimitivelevelinwhichthesurplusgeneratedbygambling isdivided—ifitisnotfirstskimmedoffbyanarbitrageur. – 7 – about the “true” game that is being played. Section 4 derives the properties of the “re- vealed” game that is determined by small monetary gambles acceptable to the players. Section 5 introduces the concept of joint coherence (no arbitrage) as a standard of strate- gic rationality and proves the main results concerning the properties of arbitrage-free cor- related equilibria. Section 6 proves that, under conditions of risk aversion (decreasing util- ity for money), the revealed game typically looks like an imprecisely specified version of the true game. Section 7 provides a concluding discussion. 2. Games with side-gambles among risk averse players The effects of risk aversion on information and efficiency will now be illustrated for several well-known games. First, return to the matching-pennies game of Table 1, and suppose that the game is still zero-sum in terms of the original money payoffs but that the players are now risk averse in the sense that their marginal utilities for money are strictly decreasing functions of their utility levels. The acceptance of side-gambles now has the po- tential to affect the players’ utilities, so henceforth it will be assumed that the game pay- offs are “large” (±$1000) while the side-gambles are “small” (on the order of ±$1). For purposes of illustration, suppose that the players have identical negative-exponential util- ity functions u(x) = 1 − exp(−x/r) with risk tolerances r = $1000/ln(2) ≈ $1443, so that u($1000) = 2.0 and u(−$1000) = 0.5. The game is still constant-sum5 in utilities, and the unique Nash/objective correlated equilibrium still assigns equal probability to all outcomes. The corresponding marginal utility functions u0(x) have the property that u0(x) ∝ 1−u(x), so that each player’s relative marginal utility for money is 4 times higher when she ends up with a payoff of −$1000 than when she ends up with a payoff of +$1000. This dependence of marginal utilities for money on the outcome of the game distorts the 5 The important feature of this game is not that it is zero-sum in money or constant-sum in utilities, but rather only thatitisstrictlycompetitive—i.e.,theplayers’payoffsvaryoppositelywiththeoutcome. – 8 –