The Price of Anarchy Measuring the inefficiency of selfish networking Master’s thesis Floris Olsthoorn Thesis advisor: dr. F.M. Spieksma Mathematisch Instituut, Universiteit Leiden April 3, 2012 Contents Introduction v 1 Selfish routing 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 The model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Equivalence of optimal and Nash flows . . . . . . . . . . . . . . . 6 1.5 Existence and uniqueness of flows . . . . . . . . . . . . . . . . . . 9 1.6 The Price of Anarchy of Routing Games . . . . . . . . . . . . . . 10 1.7 Atomic Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.7.2 The model . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.7.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2 Network formation 17 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 Local Connection Game . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.2 The model . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.3 Price of Stability of the Local Connection Game . . . . . 19 2.2.4 Price of Anarchy of the Local Connection Game . . . . . 21 2.3 Global Connection Game . . . . . . . . . . . . . . . . . . . . . . 29 2.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3.2 The model . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3.3 Price of Anarchy and Stability of the Global Connection Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.4 Facility Location Game . . . . . . . . . . . . . . . . . . . . . . . 32 2.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.4.2 The model . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.4.3 Properties of the FLG . . . . . . . . . . . . . . . . . . . . 33 2.5 Utility Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.5.3 Facility Location Game as a Utility Game . . . . . . . . . 34 2.5.4 Price of Anarchy and Stability of Utility Games . . . . . 35 iii iv CONTENTS 3 Potential Games 37 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.2 Definition and a characterization . . . . . . . . . . . . . . . . . . 37 3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.4 Properties of potential games . . . . . . . . . . . . . . . . . . . . 43 3.5 Applications to specific games . . . . . . . . . . . . . . . . . . . . 44 A Biconnected components 45 B Finite strategic games 49 Introduction The internet is operated by a huge number of independent institutions called transit ISPs (Internet Service Providers). They operate in their own interest, which often leads to great inefficiencies and instabilities in the global network. For example, ISPs tend to pass packets on to a neighboring ISP as quickly as possible,likeahotpotato. Thispolicy,calledearly-exitrouting,isbeneficialto theISP,sinceitminimizestheloadonitsownnetwork. However,itcangreatly increase the total length of the path that a packet has to traverse to reach its goal [17]. So the selfish actions of the ISPs are detrimental to the welfare of the network as a whole. In general, selfish users cause some measure of deviation from the theoret- ically optimal solution to networking problems. This thesis asks the question: how bad can this deviation get? Or more colourfully: what is the price of an- archy? In particular, we review some of the answers that have been proposed by mathematicians and computer scientist over the last decade or so. The an- swers provide a mixed message about selfish networking. In some cases selfish networking is guaranteed to deviate at most a small factor from the optimal solution. In some other cases, unfortunately, the deviation from the optimum caused by selfish users could be unbounded. Game theory Before we rush to these answers, however, we must first make mathematical sense of the vaguely put question above. This thesis considers a game theoretic approach of research. A game is a purely mathematical object that models a situation where a group of players is confronted with strategic choices. Ifallplayershavechosenastrategy, theneachplayerreceivesapay-off dependent on the choices of himself and his competitors (and/or allies). The pay-off represents some value the players wish to maximize, such as profit, or minimize, such as latency experienced due to congestion. A game doesn’t tell us what strategies the players will choose. But this is exactly what we need to know, since we want to compare the solution found by selfish players to the optimal solution. Fortunately, in game theory there is a widely used predictor for what course of action the players will take. It is called the Nash equilibrium, which was first suggested by the mathematician John Forbes Nash in 1950 [9]. Suppose all players have chosen their strategy. If some player could profit by unilaterally changing his strategy and thereby getting a better pay-off, then obviously the player has the incentive to do just that. We are, therefore, not in a stable situation. If, on the other hand, no player can profit by unilaterally v vi INTRODUCTION changing his strategy, the players are said to be in a Nash equilibrium1. The Price of Anarchy and Stability Once the players have settled on a choice of strategies, we would like to quantify the impact on the system as a whole. For this, we introduce a social utility function, which returns for each choice of strategies some number. This number could respresent, for example, the total profit gained by the players or the average latency each player expe- riences. Once we have defined the social utility function, we can compare the social utility of a Nash equilibrium with the value of optimal solutions. In this context an optimal solution is a choice of strategies that yields the best value for the social utility function. The most popular metric for the impact of selfishness is called the Price of Anarchy. It is the proportion between the worst possible social utility from a Nash equilibrium and the optimal social utility, not necessarily from a Nash equilibrium. Notice that for this definition to make sense, a game should allow at least one Nash equilibrium. Not all games do, so for each game we need to check what, if any, the Nash equilibria are. Sometimes we’re also interested in the best-case scenario. We define the Price of Stability as the proportion between the best possible social utility of a Nash equilibrium and the optimal social utility. In other words, the Price of Stability measures how far we are from truly optimal when we reach the best possible solution that everyone can agree on. Results In this thesis we find low constant bounds on the Price of Anarchy for several games. For example, any instance of the routing game discussed in Chapter 1 has a Price of Anarchy of at most 4/3, provided the latency functions of the edges in the network of the instance are affine. We prove a more general bound using elementary methods from continuous optimization. Specifically, readers familiar with Karush-Kuhn-Tucker theory should find the reasoning easy to follow. Chapter 2 is dedicated to the analysis of network formation games. These are games where players build some type of network together, but they only want to maximize their own gain (or minimize their own cost). For example, in the Local Connection Game, players want to be closely connected to all other players, but want to build as few connections as possible, since each connection costs resources to build. The Price of Stability of any Local Connection Game is at most 4/3. We prove that all games in Chapter 2, except the Local Connection Game, are examples of a special class of games called potential games. These games arestudiedinChapter3. Potentialgamesallowapotentialfunction, whichisa single function that tracks the changes in utility as players change their strate- gies. The mere existence of such a function guarantees some powerful results. For example, potential games always have a (deterministic) Nash equilibrium. Also,best-responsedynamics,whereeachturnoneplayerchangestoastrategy that maximizes his utility given the current strategies of the other players, will converge to a Nash equilibrium. 1NotallgameshaveaNashequilibrium. Ifplayersareallowedtohavea‘mixed’strategy, i.e. choose each strategy with a certain probability, then a Nash equilibrium is guaranteed to exist, provided the amount of players and strategies are finite [9]. However, we will not considermixedstrategiesinthisthesis. vii Algorithmic Game Theory and further research The study of the inef- ficiency of Nash equilibria is a topic in the broader field of Algorithmic Game Theory. This relatively young field aims to find constructive answers to ques- tions that arise when one studies the internet. Some examples of topics studied in Algorithmic Game theory are online auctions, peer-to-peer systems and net- workdesign. Foranoverviewofthefieldsee[15]. Thestructureofthisthesisis basedlargelyonChapters17,18and19from[10],thefirstbookonAlgorithmic Game Theory. The book is freely available online. This thesis only covers the fundamentals of the Price of Anarchy. We don’t discuss such related topics as applications to network design, other equilibrium conceptsand thecomputionalaspects offindingNashequilibria. Thebook[10] covers some of these topics. For a variation on the Price of Anarchy, see for instance [1]. This article defines the strong equilibrium in the context of job scheduling and network formation. The strong equilibrium is introduced to ac- countforsituationswhereplayersmayformcoalitions. ItisaNashequilibrium that is also resilient to deviations by coalitions. Even though the lack of coor- dination is resolved in a strong equilibrium, the (Strong) Price of Anarchy may still be larger than 1. viii INTRODUCTION Chapter 1 Selfish routing 1.1 Introduction This chapter focuses on Tim Roughgarden’s work on routing games [14,16], the ‘paradigmatic’ study in the area of Price of Anarchy [11]. The games are basedonoldermodelsfromtransportationtheory. Aroutinggameconsistsofa networkwhereplayerswanttoroutetrafficfromasourcetoadestination. Each playerchoosesapaththroughthenetworkconnectinghissourceanddestination. On each edge in his path, a player experiences latency dependent on the total amount of players who route their traffic along the same edge. The precise amount is determined by the cost function of the edge. The socially optimal solution to the routing problem is attained when the total latency experienced is minimal. Even in very simple networks the social optimum is not a ‘selfish solution’, i.e. a Nash equilibrium, as we will see in Example 1.3.1. Most of this section focuses on the situation that the traffic is formed by a very large set of players, each of whom controls a negligible fraction of the traffic. We call this nonatomic routing. In this situation, the problem of deter- miningthePriceofAnarchyinanyroutinggamehasessentiallybeensolvedby Roughgarden. The Price of Anarchy of any given routing game depends only on the type of cost functions used. So other factors, such as the topology of the network or the distribution of sources and destinations, are irrelevant. For any set of cost functions, there is a strict upper bound for the Price of Anarchy (Definition 1.6.1). For certain sets of functions this upper bound can be calculated explicitly. For example, if the cost functions are affine, then the price of anarchy is at most 4/3. This means that the total latency in any Nash equilibrium is at most 33% worse than in the optimal solution; a positive result indeed. On the other hand, if the cost functions are polynomial, then the Price of Anarchy becomes potentially unbounded. In the closing section of this chapter, we will consider routing games with only a finite amount of players, each controlling a non-negligible amount of traffic, so called atomic routing. The analysis in this case is less ‘clean’ than in the nonatomic case. For instance, not all atomic routing games have a Nash equilibrium,incontrastwithnonatomicrouting. Still,wecandeduceapositive result. Just like in nonatomic routing, if the cost functions are affine, then the Price of Anarchy is bounded by a small constant (∼2.618, see Theorem 1.7.3). 1 2 CHAPTER 1. SELFISH ROUTING 1.2 The model We will use a generalized version of the Wardrop model of transportation net- works from [19]. In the model a flow is routed across a directed graph. This flow represents the traffic of a continuum of players, where each player controls an infinitessimal amount of traffic. This model is called nonatomic routing1. A network N is a directed finite graph G=(V,E) together with k ∈Z N ≥1 commodities{s ,t }wheres ∈V iscalledthesourcenodeofthecommodity i i i and t ∈ V is called thesink node of the commodity. Different commodities i can share the same source or destination. A (nonatomic) instance is a triple (N,r,c), where N is a network, r is a k-dimensional vector of traffic rates r ∈ R for each commodity {s ,t } in i >0 i i N, and c is an #E-dimensional vector of cost functions c : R → R for e ≥0 ≥0 each edge e in N. A cost function is sometimes also called a latency function and measures the amount of latency per unit of traffic. Let (N,r,c) be an instance. The set of {s ,t }-paths, where {s ,t } is a i i i i commodity, is denoted by P . The set P of commodity paths in N is defined i by P = ∪ P . A flow f on N is a vector in R#P, where f denotes the flow i i ≥0 P (cid:80) over path P ∈ P. A flow f on N is feasible if f = r for each each P∈Pi P i commodity {s ,t }. For each e the flow on e is given by i i (cid:88) f = f . e P P∈P:e∈P We interpret the cost functions of an instance as measuring the cost or latencyofanedgeexperiencedbyaunitoftraffic. Thus,thelatencyexperienced by the traffic f on edge e is c (f )f . The cost of a flow f on N is given by e e e e (cid:88) C(f)= c (f )f . e e e e∈E A flow f on N is considered optimal for the instance if f is feasible and opt opt C(f ) is minimal, i.e. opt (1.2.1) C(f )=min{C(f(cid:48)):f(cid:48) on N is feasible}. opt 1.2.1 Remark. An objective function such as the total cost function defined above,wherewesumtheplayers’costs,iscalledautilitarian objective func- tion. Another type of objective function is the egalitarian objective func- tion, which is often used in scheduling problems. This function is determined by the maximum of the players’ costs. We define the cost of a path P ∈P by (cid:88) c (f)= c (f ). P e e e∈P Using this definition, we can rewrite (1.2.1) to (cid:88) C(f)= c (f)f . P P P∈P 1AtomicroutingisdiscussedinSection1.7