IIASA (cid:2) International Institute for Applied Systems Analysis (cid:15) A-2361 Laxenburg (cid:15) Austria Tel: +43 2236 807 (cid:15) Fax: +43 2236 71313 (cid:15) E-mail: [email protected] (cid:15) Web: www.iiasa.ac.at INTERIMREPORT IR-98-076/September (cid:78)(cid:111)(cid:114)(cid:109)(cid:97)(cid:108) (cid:66)(cid:101)(cid:104)(cid:97)(cid:118)(cid:105)(cid:111)(cid:114)(cid:44) (cid:65)(cid:108)(cid:116)(cid:114)(cid:117)(cid:105)(cid:115)(cid:109) (cid:97)(cid:110)(cid:100) (cid:65)(cid:103)(cid:103)(cid:114)(cid:101)(cid:115)(cid:115)(cid:105)(cid:111)(cid:110) (cid:105)(cid:110) (cid:67)(cid:111)(cid:111)(cid:112)(cid:101)(cid:114)(cid:97)(cid:116)(cid:105)(cid:118)(cid:101) (cid:71)(cid:97)(cid:109)(cid:101) (cid:68)(cid:121)(cid:110)(cid:97)(cid:109)(cid:105)(cid:99)(cid:115) (cid:65)(cid:46)(cid:70)(cid:46)(cid:75)(cid:108)(cid:101)(cid:105)(cid:109)(cid:101)(cid:110)(cid:111)(cid:118)(cid:40)(cid:107)(cid:108)(cid:101)(cid:105)(cid:109)(cid:101)(cid:110)(cid:111)(cid:118)(cid:64)(cid:105)(cid:109)(cid:109)(cid:46)(cid:117)(cid:114)(cid:97)(cid:110)(cid:46)(cid:114)(cid:117)(cid:41) (cid:65)(cid:46)(cid:86)(cid:46)(cid:75)(cid:114)(cid:121)(cid:97)(cid:122)(cid:104)(cid:105)(cid:109)(cid:115)(cid:107)(cid:105)(cid:105)(cid:40)(cid:107)(cid:114)(cid:121)(cid:97)(cid:122)(cid:104)(cid:105)(cid:109)(cid:64)(cid:105)(cid:105)(cid:97)(cid:115)(cid:97)(cid:46)(cid:97)(cid:99)(cid:46)(cid:97)(cid:116)(cid:44)(cid:107)(cid:114)(cid:121)(cid:97)(cid:122)(cid:104)(cid:105)(cid:109)(cid:64)(cid:109)(cid:105)(cid:46)(cid:114)(cid:97)(cid:115)(cid:46)(cid:114)(cid:117)(cid:41) (cid:65)(cid:112)(cid:112)(cid:114)(cid:111)(cid:118)(cid:101)(cid:100) (cid:98)(cid:121) (cid:71)(cid:111)(cid:114)(cid:100)(cid:111)(cid:110) (cid:77)(cid:97)(cid:99)(cid:68)(cid:111)(cid:110)(cid:97)(cid:108)(cid:100) (cid:40)(cid:109)(cid:97)(cid:99)(cid:100)(cid:111)(cid:110)(cid:64)(cid:105)(cid:105)(cid:97)(cid:115)(cid:97)(cid:46)(cid:97)(cid:99)(cid:46)(cid:97)(cid:116)(cid:41) (cid:68)(cid:105)(cid:114)(cid:101)(cid:99)(cid:116)(cid:111)(cid:114)(cid:44)(cid:73)(cid:73)(cid:65)(cid:83)(cid:65) InterimReportsonworkoftheInternationalInstituteforAppliedSystemsAnalysisreceiveonlylimited review. Views or opinions expressed herein do not necessarily represent those of the Institute, its NationalMemberOrganizations,orotherorganizationssupportingthework. {ii{ Abstract The paperintroducesalocalcooperationpatternforrepeatedbimatrixgames: theplayers choose a mutually acceptable strategy pair in every next round. A mutually acceptable strategypairprovides each playerwitha payo(cid:11)no smallerthan thatexpected, in average, at a historical distribution of players’ actions recorded up to the latest round. It may happen that at some points mutually acceptable strategy pairs do not exist. A game round at such \still" points indicates that at least one player revises his/her payo(cid:11)s and switchesfromanormalbehaviortoabnormal. Weconsiderpayo(cid:11)switchesassociatedwith altruistic and aggressive behaviors, and de(cid:12)ne measures of all combinations of normal, altruistic and aggressive behaviors on every game trajectory. These behavior measures serve as criteria for the global analysis of game trajectories. Given a class of trajectories, onecanidentifythemeasuresofdesirableandundesirablebehaviorsoneachtrajectoryand select optimal trajectories, which carry the minimum measure of undesirable behaviors. In the paper, the behavior analysis of particular classes of trajectories in the repeated Prisoner’s Dilemma is carried out. {iii{ Contents 1 Cooperative game dynamics 3 1.1 Cooperative repeated game . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Normal behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Basic behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4 Universality of basic trajectories . . . . . . . . . . . . . . . . . . . . . . . . 7 1.5 Measures of basic behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.6 Behavior assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.7 Behavior optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 Cooperative dynamics in repeated Prisoner’s Dilemma 9 2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Characterization of basic trajectories . . . . . . . . . . . . . . . . . . . . . 11 3 Behavior assessment of (cid:12)ctitious play trajectories 16 3.1 Fictitious play . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2 Assessment of normal and aggressive behaviors . . . . . . . . . . . . . . . 16 4 Optimal paths to cooperation 19 4.1 Problem of optimal behavior . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.2 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.3 Optimal trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.4 Optimal behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5 Appendix 1. Cooperative game dynamics 25 5.1 Proof of Proposition 1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 5.2 Proof of Proposition 1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 5.3 Proof of Proposition 1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 6 Appendix 2. Cooperative dynamics in repeated Prisoner’s Dilemma 26 6.1 Proof of Lemma 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 6.2 Characterization of normal trajectories . . . . . . . . . . . . . . . . . . . . 27 6.3 Characterization of 2-altruistic and 1-altruistic trajectories . . . . . . . . . 31 6.4 Characterization of 1-aggressive-2-altruisticand 1-altruistic- 2-aggressive trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 6.5 Characterization of aggressive trajectories. . . . . . . . . . . . . . . . . . . 34 7 Appendix 3. Behavior assessment of (cid:12)ctitious play trajectories 35 7.1 Analysis of (cid:12)ctitious play trajectories . . . . . . . . . . . . . . . . . . . . . 35 8 Appendix 4. Optimal paths to cooperation 39 8.1 Proof of Lemma 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 8.2 Proof of Proposition 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 8.3 Proof of Lemma 4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 8.4 Proof of Lemma 4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 {iv{ 8.5 Proof of Proposition 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 8.6 Proof of Corollary 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 8.7 Proof of Proposition 4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 8.8 Proof of Lemma 4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 9 Conclusion 46 10 References 46 {v{ About the Authors Arkadii V. Kryazhimskii Mathematical Steklov Institute Russian Academy of Sciences Moscow, Russia and International Institute for Applied Systems Analysis Laxenburg, Austria Anatoli F. Kleimenov Department of Dynamical Systems Institute of Mathematics and Mechanics Ekaterinburg, Russia Acknowledgment This work was partially supported by the Russian Foundation for Basic Research under grant 97-00161. {1{ Normal Behavior, Altruism and Aggression in Cooperative Game Dynamics A. F. Kleimenov A. V. Kryazhimskii Introduction Altruism and aggression are extreme modes of interaction. When twoplayers (cid:12)nd actions pro(cid:12)table for both, one may view their behavior as desirable or normal. When they do not (cid:12)nd such actions (and are still forced to interact), at least one of them loses. If the player 1 loses, the player 2 either wins, or loses, too. Player 2 wins if player 1 goes for a compromise, i.e., adopts (temporarily) the interest of player 2 and acts so as to help this player. Player 2 loses if player 1 acts against his/her interest (which may in particular be driven by a desire to move to a \better" state where normal behavior is feasible again). In the (cid:12)rst case player 1 acts as an altruist with respect to player 2. In the second case player 1 acts as an aggressor with respect to player 2. Certainly, player 2 may also adopt altruism or aggression with respect to player 1. Accordingly, di(cid:11)erent combinations of players’ behaviors may occur. This informal classi(cid:12)cation of behaviors lies in the base of our study. We do not pretend to give an explanation of players’ motives when they act normally, altruistically, or aggressively (we slightly touch this issue when we consider a problem of designing optimal behaviors in section 4). Our goal is to describe a game-theoretical method for identifying players’behaviors in one-round interactionsand show how this method can be used in the analysis of multi-round interactions. Ourmodeloperatesunder theinformationalconditionsof(cid:12)ctitiousplay. The(cid:12)ctitious playdynamicsproposedbyBrown(1951)andRobinson(1951)isaround-by-roundprocess ofupdatingstrategiesinanonzerosumbimatrixgame. Ineveryround,eachplayerchooses a strategy, which gives him/her the largest expected payo(cid:11) on the historical distribution of thestrategiesof theother player. In oursetting,the playersupdate theiractionsbasing on the historical distributions of the strategies of both players. The (cid:12)ctitious play dynamics was analyzed and generalized in di(cid:11)erent aspects. Fu- denberg and Kreps (1993) viewed a (modi(cid:12)ed) Brown-Robinson procedure as a model of rational behavior and proved its convergence for 2(cid:2)2 bimatrix games with a unique mixed Nash equilibrium. Kaniovski and Young (1995) gave an economic interpretationof a stochastically perturbed (cid:12)ctitious play dynamics and showed its convergence to Nash equilibria for general 2(cid:2)2 bimatrix games; a further step in this direction was made in Kaniovski, et. al. (1997). Gaunersdorfer and Hofbauer (1995) analyzed the asymptotics of the (cid:12)ctitious play trajectories for a class of three-strategy bimatrix games and found connections with the replicatorgame dynamics (see Hofbauer and Sigmund, 1988). Smale (1980) considerably extended the frames of (cid:12)ctitious play by introducing (in the context of the repeated Prisoner’s Dilemma game) a class of general strategy updating rules fed back withthe historicaldistributionsofpayo(cid:11)s. Thisapproach wasgeneralizedin Benaim and Hirsch (1994). {2{ We de(cid:12)ne strategy updating rules through the comparison of the current payo(cid:11)s with those expected on the historical distributions of players’ strategies. Di(cid:11)erent preferences in comparison are associated with di(cid:11)erent behaviors. The strategy updating rules are to a certain extend close to that used in (cid:12)ctitious play. There are two essential di(cid:11)erences, however. First,allstrategies,forwhich the payo(cid:11)sareno smallerthan the averagepayo(cid:11)s on the historicalstrategydistributions are viewed as acceptable (recallthat (cid:12)ctitious play admits strategies maximizing the average payo(cid:11)s). Second, the proposed decisionmaking pattern is cooperative: every new strategy pair must be acceptable for each player, in other words, whenever an acceptable strategy pair is chosen, no one of the players loses (in the (cid:12)ctitious play dynamics the players update their strategies independently). If in some round the players (cid:12)nd an acceptable strategy pair and act so that no one of them loses, their behavior in this round is quali(cid:12)ed as normal. Situations where at least one player loses arise when normal behavior is changed due to a change of the acceptable strategypairs,or,equivalently,thepayo(cid:11)matricies. Inthispaper, weassumethatplayer’s payo(cid:11)matrixcan be changed toeither the payo(cid:11)matrixof the other player,orthattaken with the opposite sign. In the (cid:12)rst case the player identi(cid:12)es himself/herself with his/her rival and adopts altruism. In the second case the player identi(cid:12)es himself/herself with his/her rival’s opponent and adopts aggression. It is important that every one-round transition, which is not normal, can be identi(cid:12)ed as a combination of altruistic and/or aggressivebehaviors. Inthiscontext,ourapproachdevelopsKleimenov(1997,1998)where the ideaofidentifyingbehaviorsthroughswitchesinpayo(cid:11)swasproposedfornonzerosum di(cid:11)erential games and population evolutionary games. Our basic analytic tool is a measure of a given behavior on arbitrary game trajectory. The measure is de(cid:12)ned as, roughly, the number of rounds, in which the given behavior is registered (as long as a one-round behavior is, generally, identi(cid:12)ed not uniquely, the minimum and maximum measures are introduced). We use the behavior measures for the estimation of the proportions of desirable and not desirable behaviors on the game trajectories. Namely, we consider a problem of behavior assessment and a problem of optimal behavior. Dealing with the problem of behavior assessment, we estimate the measures of desirable and not desirable behaviors on the trajectoriesgenerated by a given strategy updating rule. We focus, in particular, on the assessment of normal (desirable) and aggressive (not desirable) behaviors on the trajectories driven by the (cid:12)ctitious play dynamics. Dealing withthe problem of optimalbehavior, we minimize the measure of not desirable behaviors over a given class of game trajectories. In particular, we focus on the problem of minimizing the measure of abnormal behavior. The paper is organized as follows. Our general method is presented in section 1. In the rest of the paper we apply the method to the analysis of the repeated Prisoner’s Dilemma, in which the players choose between cooperation and defection. This game is often used for modeling socially desirable behaviors (see, e.g., Smale, 1980; Axelrod, 1984; Nowak and Sigmund, 1994). In section 2 we characterize the trajectories driven by di(cid:11)erentcombinationsofplayers’basicbehaviors(normal,altruisticandaggressive)inthe repeated Prisoner’s Dilemma. In section 3 this characterizationis used for the estimation of the measures of normal and aggressive behaviors on the (cid:12)ctitious play trajectories (on which the players never cooperate). We state that (cid:12)ctitious play may exhibit normal behavior and exclude aggression if mutual defection has a relatively high payo(cid:11), namely, tworounds ofmutualdefection providea higher payo(cid:11)than around of cooperationversus defection and a round of defection versus cooperation. In the opposite situation normal behavior is eliminated and aggressive behavior dominates on many trajectories. In section 4 we solve the problem of minimizing the measure of abnormal behavior on the trajectories convergent to the point of mutual cooperation. All optimal trajectories {3{ (moving in the space of empirical frequencies of cooperation and defection) embark on a \cooperationroad"in a(cid:12)nite round and then develop cooperatively. In aneighborhood of the \road" all other behaviors except altruism of a \more cooperative" player are admis- sible. Beyond the neighborhood normal behavior is eliminated. Moreover, in this domain mutual defection is (under some circumstances) admissible, whereas mutual cooperation isnot. An intuitiveexplanationis thatitis\tooearly"toadoptmutualcooperationwhen one of the players is much \less cooperative" in the past. The technicalmaterialforsections 1,2, 3and 4is presented in Appendix 1 (section5), Appendix 2 (section 6), Appendix 3 (section 7) and Appendix 4 (section 8), respectively. 1 Cooperative game dynamics 1.1 Cooperative repeated game We consider arepeated two-playergame. The player1 hasn strategiesnumbered 1;:::;n, and player 2 has m strategies numbered 1;:::;m. The players choose their strategies sequentiallyin rounds 1,2,... . The empiricalfrequency of a strategyi ofplayer 1 in round k is the ratio xi = ni=k where ni is the number of rounds r (cid:20) k, in which player 1 k k k chooses i. Similarly, the empirical frequency of a strategy j of player 2 in round k is the ratio yj = mj=k where mj is the number of rounds r (cid:20) k, in which player 2 chooses j. k k k The empirical frequency vectors x = (x1;:::;xn) and y = (y1;:::;ym) belong to the k k k k k k n− 1-dimensional simplex Sn−1 and the m−1-dimensional simplex Sm−1, respectively; recall that the p − 1-dimensional simplex Sp−1 is the set of all p-dimensional vectors x = (x1:::;xp) with nonnegative coordinates whose sum is equal to 1. We shall call S = Sn−1 (cid:2) Sm−1 the state space of the repeated game. Elements of S will be called states. Note that all states (x ;y ) admissible in round k cover a (cid:12)nite subset of S. k k Following the pattern of (cid:12)ctitious play, we assume that in each round k the players observethecurrentstate(x ;y )andchooseastrategypair(i ;j )forthenextround. k k k+1 k+1 The number of rounds r (cid:20) k+1, in which player 1 chooses strategy i changes as follows: ni = ni +1 if i = i and ni = ni if i 6= i . Hence, for the empirical frequency k+1 k+1 k+1 k+1 vector of player 1 we have: i i i nk+1 +1 nk+1 nk+1 1 xik+1 = k = k − k + ; k+1 k+1 k k(k+1) k+1 ni ni ni xi = k = k − k (i6= i ); k+1 k+1 k k(k+1) k+1 or i xk+1 +1 xik+1 = xik+1 − k ; (1.1) k+1 k k+1 xi xi = xi − k (i6= i ): (1.2) k+1 k k+1 k+1 Similarly, j y k+1 +1 yjk+1 = yjk+1 − k ; (1.3) k+1 k k+1 yj yj = yj − k (j 6= j ): (1.4) k+1 k k+1 k+1 A (cid:12)niteorin(cid:12)nitesequence t = ((x ;y ))in S (k = k ;k +1:::)willbe calledatrajectory k k 0 0 if for all indecies k = k ;k +1;:::(except the (cid:12)nal one provided t is (cid:12)nite) the equalities 0 0 {4{ (1.1) { (1.4) hold with some strategy pairs (i ;j ); the indecies k are identi(cid:12)ed with k+1 k+1 game rounds; the state (x ;y ) will be called the initial state of t; we shall also say that k0 k0 (x ;y ) gives rise to t in round k , and t originates from (x ;y ) in round k . We k0 k0 0 k0 k0 0 de(cid:12)ne the length of a trajectory t to be the di(cid:11)erence between its (cid:12)nal and initial rounds if t is (cid:12)nite and 1 if t is in(cid:12)nite. A trajectory t = ((x ;y )) (k = k ;:::) will be called k k 0 stationary if (x ;y )= (x ;y ) for all k (cid:21) k . k k k0 k0 0 Weconsiderthefollowingruleforupdatingstrategies. Inroundk,eachplayeridenti(cid:12)es a set of strategypairs acceptable forhim/her in round k+1. If the players (cid:12)nd a strategy pair acceptable for both, they choose it for (i ;j ). If the players’ acceptable sets do k+1 k+1 not intersect, (x ;y ) is the (cid:12)nal state on the trajectory. k k Let us specify the structure of the acceptable sets and introduce the associated trajec- tories. Let f , and g be payo(cid:11)s to player 1 and player 2, respectively, for a strategy pair ij ij (i;j). The expected payo(cid:11)s (briefly, the payo(cid:11)s) to players 1 and 2 at a state (x ;y ) are k k de(cid:12)ned by Xn Xm f(x ;y ) = xiyjf ; (1.5) k k k k ij i=1j=1 Xn Xm g(x ;y ) = xiyjg ; (1.6) k k k k ij i=1j=1 respectively. In roundk, aplayerviewsastrategypair(i ;j )asacceptable forround k+1 k+1 k +1 if his/her payo(cid:11) at this strategy pair (i.e., f for player 1 and g for i ;j i ;j k+1 k+1 k+1 k+1 player 2) is no smaller than his/her payo(cid:11) at the state (x ;y ). k k Later,weshalladmitchangesinthepayo(cid:11)functions(andassociatethemwithswitches in players’ behavior). Therefore, we formally de(cid:12)ne the acceptability of strategy pairs not only with respect to the original payo(cid:11) functions f and g but also with respect to arbitrary \surrogate" payo(cid:11) functions. We understand a surrogate payo(cid:11) function as a scalar function ’ on S, which has the same structure as f and g: Xn Xm ’(x ;y ) = xiyj’ : k k k k ij i=1j=1 Given a surrogate payo(cid:11) function ’, we call a strategy pair (i ;j ) ’-acceptable if k+1 k+1 ’ (cid:21) ’(x ;j ). The set of all strategy pairs ’-acceptable at the state (x ;y ) will i ;j k k k k k+1 k+1 be denoted by A (x ;y ). ’ k k Given a pair of surrogate payo(cid:11) functions, (’; ),a trajectoryt = ((x ;y )) described k k by (1.1) { (1.4) will be called a (’; )-trajectory if in every round k (except the (cid:12)nal one provided t is (cid:12)nite) the newly chosen strategy pair (i ;j ) is ’-acceptable and k+1 k+1 -acceptable at (x ;y ), i.e., (i ;j )2 A (x ;y )\A (x ;y ). k k k+1 k+1 ’ k k k k The set of all states (x ;y ) such that the intersection A (x ;y ) \ A (x ;y ) is k k ’ k k k k nonempty will be called the (’; )-active domain. Every state from the (’; )-active domain will be called (’; )-active. By de(cid:12)nition every (’; )-activestate in every round gives rise to a (’; )-trajectory whose length is no smaller than 1. Every (’; )-active state, which gives rise to an in(cid:12)nite (’; )-trajectoryin every round will be called (’; )- kernel-active. The set of all (’; )-kernel-active states will be called the (’; )-kernel- active domain. A state will be called stationary (’; )-kernel-active if in every round it gives rise to a single in(cid:12)nite (’; )-trajectory, and the latter is stationary. The set of all (’; )-kernel-active states, which are not stationary, will be called the nonstationary (’; )-kernel-active domain. In every round treated as initial, every state beyond the (’; )-active domain gives rise to a single (’; )-trajectory whose length is 0; we shall {5{ call such states (’; )-still. The set of all (’; )-still states will be called the (’; )-still domain. A (’; )-trajectory will be called nonextendible if it is either in(cid:12)nite, or (cid:12)nite and its (cid:12)nal state is (’; )-still. The next proposition describes a simple class of stationary (’; )-kernel-activestates. Weshallcallastrategypair(i(cid:3);j(cid:3))(’; )-Paretomaximaliftheredoesnotexistastrategy pair (i(cid:3);j(cid:3)) such that ’i(cid:3)j(cid:3) (cid:21) ’i(cid:3)j(cid:3), i(cid:3)j(cid:3) (cid:21) i(cid:3)j(cid:3), and at least one of these inequalities is strict. Proposition 1.1 Let a strategy pair (i(cid:3);j(cid:3)) be (’; )-Pareto maximal and there do not exist a strategy pair (i(cid:3);j(cid:3)) 6= (i(cid:3);j(cid:3)) such that ’i(cid:3);j(cid:3) = ’i(cid:3);j(cid:3) and i(cid:3);j(cid:3) = i(cid:3);j(cid:3). Then a state (x(cid:3);y(cid:3)) de(cid:12)ned by xi(cid:3)(cid:3) = 1; xi(cid:3) = 0 (i6= i(cid:3)); y(cid:3)j(cid:3) = 1; y(cid:3)j = 0 (j 6= j(cid:3)) (1.7) is stationary (’; )-kernel-active. A proof is given in Appendix 1. Let us provide a characterization of the nonstationary (’; )-kernel-active states in a special case where there is a strategy pair (’; )-acceptable at every (’; )-active state. Proposition 1.2 Let there be a strategy pair (i(cid:3);j(cid:3)) (’; )-acceptable at every nonsta- tionary (’; )-active state, and a state (x(cid:3);y(cid:3)) be de(cid:12)ned by (1.7). Then a state (x;y)6= (x(cid:3);y(cid:3)) is nonstationary (’; )-kernelactive if and only if the closed segment with the end points (x;y) and (x(cid:3);y(cid:3)) is contained in the (’; )-active domain. A proof is given in Appendix 1. 1.2 Normal behavior By de(cid:12)nition the surrogate payo(cid:11)s ’ and do not decrease along the (’; )-trajectories. In particular, the actual payo(cid:11)s f and g do not decrease along the (f;g)-trajectories. In this sense, the (f;g)-trajectoriesrepresent normal behavior bene(cid:12)cial forboth players. We identify every (f;g)-trajectory as normal. We also identify the active domain of normal behavior, G00,thekernel-activedomain of normal behavior, G00,thenonstationary kernel- 1 active domain of normal behavior, G00, and the still domain of normal behavior, G00, with 1 ; the (f;g)-active domain, the (f;g)-kernel-active domain, the nonstationary (f;g)-kernel- active domain, and the (f;g)-still domain, respectively. Stationary (f;g)-kernel-active states will be called stationary for normal behavior. 1.3 Basic behaviors When a state of the game is in the still domain of normal behavior, G00, the players are ; unable to make a new round via normal behavior. In order to make a new round, at least one player must change the behavior. We shall understand a change in behavior as a switch from the originalpayo(cid:11) function to a surrogate one. Player’s switch to a surrogate payo(cid:11) function means that this player replaces the strategy pairs acceptable with respect to his/her original payo(cid:11) function by those acceptable with respect to the surrogate one. We shall consider altruistic and aggressive behaviors. When switching to altruistic behavior a player identi(cid:12)es his/her interest with his/her partner’s. In this situation, the player replaces his/her original payo(cid:11) function by his/her partner’s. When switching to aggressive behavior, the player views himself/herself as partner’s opponent and changes his/her payo(cid:11) function for his/her partner’s taken with the opposite sign. Combina- tions of individual behaviors generate joint behaviors; we willqualify them as 1-altruistic,