Lecture 4 TOPICS IN GAME THEORY

J orgen Weibull

September 18, 2012

 An extensive-form game: captures the dynamics of the interaction in question - who moves when, who observes what, and who cares about what (Bernoulli functions over plays)

A normal-form game: represents the game structure in a mathematically convenient way - everybodys strategy sets and payo functions

What is an extensive-form game?

Kuhn (1950, 1953), Selten (1965, 1975), Kreps and Wilson (1982) = hN, A, , P , I , C , p, r, v i where: 1. N = {1, ..., n} the (nite) set of personal players 2. A directed (and connected) tree: (a) A (nite) set A of nodes, with a root, a0 A (b) A predecessor function:, : A\ {a0} A (c) From each node a 6= a0, the root can be reached by a nite number of iterations of

(d) a0 precedes a if a0 = ( (... (a))). Write a0 < a (e) A A the terminal nodes, nodes that precedes no node (a / (A))

3. T the set of plays , ways to go from a0 to A

4. P = {P0, P1, ..., Pn} the player partitioning of non-terminal nodes 5. I = iN Ii, where each Ii is the information partitioning of Pi A into information sets I Ii. Regularity conditions: (a) Each play intersects every information set at most once (b) All nodes in an info set have the same number of outgoing branches

6. C = {CI : I I}, each CI being the choice partitioning at I (a) notation c < a

7. p : P0 [0, 1] the probabilities of natures random moves 8. r : T D the result (or outcome ) mapping, assigning material outcomes (money, food,...) to plays 9. v : T Rn the combined Bernoulli function (6= payo function) = hN, A, , P , I , C , pi the game form = hN, A, , P , I , C , p, ri the game protocol

Example: A Seller-Buyer Game

Player 1: A used-car seller. Player 2: a representative buyer Quality of the car is either H (high) or L (low), observed only by the seller

The seller chooses either a low price, pL, or a high price, pH (and can condition on the quality)

Seeing the price, the buyer chooses either B (buy) or N .(not buy) What is = hN, A, , P , I , C , p, r, vi?

What is = (N, A, , P , I , C , p, r, v )?

Example: Is the following an extensive form?

B A A B

3 A

2 A

L 1

1.1

Subgames
n o

The follower set F (a) =

a0 A : a a0

Subroots are nodes a for which: F (a) I 6= I F (a) . A subgame of is the tree starting at a subroot a, endowed with the same partitionings etc. and denoted a (in particular, a0 = is a subgame)

Strategies, realization probabilities and payo functions

2.1

Pure strategies

Pure strategies si Si. Each pure strategy species exactly one choice at each information set belonging to the player in question

Pure-strategy proles s = (s1, ..., sn) S = Si Realization probabilities for plays T : ( , s)

 Payo functions i : S R i (s) =

( , s) vi ( )

Recall that a pure strategy thus is more than what people usually think...

2.2

Mixed strategies

Mixed strategies xi Xi = (Si). As if each player randomizes before starting to play.

Mixed-strategy proles x = (x1, ..., xn) X = (S ) = i (Si)

 Realization probabilities: ( , x) =

sS

Xh

j N xj sj

( , s)

Payo functions i (x) =

( , x) vi ( )

Example:
(1,3) (2,2) (0,0)

Game 4
(

1 (x) = x11 + 2x12x21 2 (x) = 3x11 + 2x12x21

2.3

Behavior strategies

Local strategies: statistically independent randomizations over choice sets, yiI YiI = (CI ) Behavior strategies: functions yi that assign local strategies to information sets I Ii, yi Yi = I Ii YiI - as if players randomize as the play proceeds

Behavior strategy proles: y Y = iN Yi

 Realization probabilities: ( , y ) = the product of all choice probabilities along

Payo functions i (y ) =

( , y ) vi ( )

In the preceding example: Y = X because each player has only one info set Denition 2.1 Outcome of strategy prole = induced probability distribution over plays (or end-nodes) Denition 2.2 Path of strategy prole = the set of plays assigned positive probabilities

Perfect recall and Kuhns theorem

Mixed strategies: global randomizations performed at the beginning of the play of the game Behavior strategies: local randomizations performed during the course of play of the game Equivalence in terms of realization probabilities?

Denition 3.1 (Kuhn 1950,1953) An extensive form has perfect recall if c < a c < a0 for each player i N , pair of information sets I, J Ii, choice c CI and nodes a, a0 J .

 Note: - All perfect-information games have perfect recall - If each player has only one information set, then perfect recall - Bernoulli values and payos are irrelevant for the denition of perfect recall

Informally:

Theorem 3.1 (Kuhns Theorem) If has perfect recall, then, for each mixed strategy, a realization-equivalent behavior strategy.

To state this more exactly, and prove it: Consider a player i in a nite extensive form . Denition 3.2 A (behavior-strategy) mixture, wi, is a nite-support randomization over the players behavior Wi, where Wi is the strategies: wi 1 , ..., w y k set of probability vectors wi = wi yi for some k N and i i
1, ..., y k Y . yi i i

Every behavior strategy yi Yi can be viewed as a (degenerate) behavior-strategy mixture, the mixture wi that assigns unit probability to yi. Every mixed strategy xi Xi can be viewed as the mixture wi that h assigns probability xih [0, 1] to the (degenerate) behavior strategy yi that assigns unit probability to the choices made under pure strategy h Si.

0 W are realization equivalent if the realDenition 3.3 Mixtures wi, wi i 0 00 00 ization probabilities under the proles wi, wi and wi, wi are identical for all mixture proles w00 W = n j =1Wj .

Theorem 3.2 (Kuhn 1950, Selten 1975) Consider a player i in a nite extensive form with perfect recall. For each behavior-strategy mixture 0 W that assigns wi Wi there exists a realization-equivalent mixture wi i unit probability to a behavior strategy yi Yi.

Proof sketch:

1. Consider those of is information sets I that are possible under wi in 00 00 the sense that I is on the path of wi, w i for some w W 2. Note that conditional probabilities over the nodes in any of i0s information sets I do not depend on is own strategy

The behavior induced at information sets by a mixed-strategy prole Equivalence in terms of realization probabilities We henceforth will assume perfect recall.

The ve NF-games associated with an EF-form game

Five normal forms associated with any given extensive form: 1. The pure-strategy normal form 2. The mixed-strategy normal form 3. The behavior-strategy normal form 4. The quasi-reduced normal form 5. The reduced normal form

 The pure-strategy normal-form: G = hN, S, i where The strategy set of player i is Si = I Ii CI S = iN Si is the set of pure-strategy strategy proles s = (si)iN : S Rn is the combined payo function, i (s) R the payo to player i under s = hN, X, The mixed-strategy normal-form: G i where X = iN Xi is the set of mixed-strategy proles x = (xi)iN where Xi = 4 (Si) : X Rn is the combined payo function, i (x) R the payo to player i under x

 The behavior-strategy normal-form representation of is a triplet = hN, Y, G i where Y = iN Yi is the set of behavior-strategy proles y = (yi)iN where Yi = I Ii (CI ) : Y Rn is the combined payo function, i (y ) R the payo to player i under y

00 Denition 4.1 A players two pure strategies, s0 i, si Si are equivalent if

0 00 si, si = si , si si Si.

Note that one requires that all players should be indierent The quasi-reduced normal-form game is the triplet G = hN, S, i where all equivalent pure-strategy pairs have been replaced by a single pure strategy until no further such replacement is possible.

Denition 4.2 A pure strategy, si Si is redundant in a pure-strategy normal form game G = hN, S, i if there exists a mixed strategy xi 4 (Si) such that (si, si) = (xi, si) si Si.

 The reduced normal-form game is the pure-strategy normal-form game G = hN, S, i where all equivalent and redundant pure strategies have been removed.

Example:
(3,2) (0,0)

E (1,2) 1

C (2,1) 2

Pure-strategy normal form: AE AF BE BF C 2, 1 2, 1 1, 2 1, 2 D 2, 1 2, 1 3, 2 0, 0

Quasi-reduced (and reduced) normal-form representation: C D A 2, 1 2, 1 BE 1, 2 3, 2 BF 1, 2 0, 0

Solution concepts
Now we are in a position to dene and analyze dierent solution concepts for extensive-form games.

Focus on behavior strategies in nite EF games with perfect recall (recall Kuhns Theorem)

Solution concepts: Nash equilibrium, subgame perfect equilibrium, sequential equilibrium, perfect Bayesian equilibrium, and extensive-form perfect equilibrium

5.1

Nash equilibrium

= (N, Y, Let G ) be the behavior-strategy NF representation of always exist? Q1: Do NE in G need not Mathematical diculty: the best-reply correspondence in G be convex-valued

 = (N, Y, Proposition 5.1 NE of G ). = (N, X, Proof : Consider mixed-strategy NF G ) and use Kuhns Theorem!

Q2: How nd NE directly in the extensive form?

1. For any node a, information set I , and behavior-strategy prole y , let (a, y ) be the probability that node a is reached when y is played, and let (I, y ) =
aI

(a, y )

2. Denition: I I is on the path of y Y if (I, y ) > 0 3. Consider I on the path of y . By Bayes rule, the conditional probabilities over the node in I are Pr (a | y ) = (a, y ) (I, y ) a I

Denition 5.1 Suppose that I Ii is on the path of y Y . A behavior Y is a best reply to y at I if: strategy yi i
aI

, y ) Pr (a | y ) ia (yi i

where ia (, yi) is the conditional payo-function for player i when play starts at node a I . Denition 5.2 A behavior-strategy prole y Y is sequentially rational on its own path in if, for all players i N , yi Yi is a best reply to y at all information sets I Ii on its path.

aI

0, y Pr (a | y ) ia yi i

0 Y yi i

Proposition 5.2 (van Damme, 1984) A behavior-strategy prole y is a NE i it is sequentially rational on its own path in . of G

Proof sketch: . Then it prescribes a suboptimal 1. Suppose that y is not a NE of G move somewhere on its own path

2. Suppose that y does not prescribe a best reply to itself at some info set on its path

5.2

Subgame perfection

Denition 5.3 (Selten, 1965) A behavior strategy prole y is a subgameperfect equilibrium (SPE) of if its restriction y a to each subgame a is a. a NE of G

In simultaneous-move games: SPE=NE In games of perfect information: Proposition 5.3 Every nite EF games of perfect information has at least one SPE in pure strategies. For generic payos, this SPE is unique.

Proof sketch in class: use Kuhns algorithm.

 By a slight generalization of Kuhns algorithm: Proposition 5.4 Every nite EF game with perfect recall has at least one SPE.

Proof sketch in class: use a generalized version of Kuhns algorithm.

Example 5.1 Consider a battle-of-the-sexes game in which player 1 has an outside option (go to a caf e with a friend):

(3,1) a

(0,0) b 1

(0,0) a

(1,3) b

A (2,v) 2 L R

Find all subgame perfect equilibria! (Does the value v R matter?)

 However, SPE is sensitive to details of the EF form. Example 5.2 Add a payo-equivalent move to the entry-deterrence game. In this extensive-form game, s = (A, F ) becomes subgame perfect (although still implausible):
0 0 F Y 2 2 2 0 0 F Y 2 2

1 3

E1

E2

 Subgame perfection ; sequential rationality at all singleton-information sets Example 5.3 Seltens horse
0 0 0 L R 3 3 2 2 0 0 1 L R 4 4 0 1 1 1

y = (L, R, R) is a NE and hence a SPE.

 But 2s move is not a best reply to y In all solution concepts: a strategy is not only a contingent plan for the player in question, but also others expectation about the players moves.

5.3

Sequential equilibrium

Rene SPE by generalizing the stochastic dynamic programming approach from 1 decision-maker to n decision-makers

Kreps and Wilson (1982) Denition 5.4 A belief system is a function : AA [0, 1] such that
aI

(a) = 1

I I

Denition 5.5 An belief system is consistent with a behavior-strategy t y such prole y if sequence of interior behavior-strategy pro les y t that Pr | y ().

 (I, y ) > 0 agrees with Pr ( | y ) on I Denition 5.6 A behavior-strategy prole y is sequentially rational under a belief system if for every player i and each information set I Ii:
aI

(a) ia (y )

aI

0, y (a) ia yi i

0 Y yi i

Denition 5.7 A behavior-strategy prole y is a sequential equilibrium (SE) if y is sequentially rational under some belief system that is consistent with y .

Proposition 5.5 Every sequential equilibrium is subgame perfect.

 Reconsider: (1) The above battle-of-sexes with an outside option (2) The above entry-deterrence game with added move (3) Seltens horse (see next slide)

Example 5.4 Reconsider Seltens horse

0 0 0 L R

3 2 2

0 0 1 L 3 R

4 4 0

1 1 1

We rst note that the arguably unsatisfactory subgame-perfect equilibrium s = (L, R, R) is not a sequential equilibrium. The reason is that player 2s (pure) behavior strategy, s2 = R, is not sequentially rational, since s0 2=L

is a better reply than R, at 2s node, against s (yielding a payo of 4 instead of 1) Next, we note that all sequential equilibria are of the form y = (R, R, y3), where y3 assigns probability 3/4 to L. Under such a strategy prole, 3s information set is not on the path, and any conditional probabilities (a) and (b) (both non-negative and summing to 1) that player 3 may attach to her 2 nodes are part of a consistent belief system. Her strategy L is sequentially rational i (a) 1/3, and she is indierent between her strategies L and R i (a) = 1/3, and in that case, any strategy for her is sequentially rational. However, if she would play R with a probability > 1/4, then player 2s strategy in y , R, would not be sequentially rational. Hence, sequential equilibrium requires that y3L 3/4.

5.4

Perfect Bayesian equilibrium

Relaxing the consistency requirement in sequential equilibrium: Denition 5.8 A belief system is weakly consistent with a behaviorstrategy prole y if agrees with the conditional probability distribution Pr ( | y ), induced by y , on each information set on y s path. Denition 5.9 A behavior-strategy prole y is a ( weak) perfect Bayesian equilibrium (PBE) if there exists a weakly consistent belief system under which y is sequentially rational. Clearly SE PBE NE There are more restrictive denitions of PBE, but no consensus among game theorists

Properness implies sequentiality

The normal-form renement properness has a beautiful implication for extensive-form analysis [van Damme (1983), Kohlberg and Mertens (1986)]:

. Proposition 6.1 Let G be a nite game with mixed strategy extension G and every EF game with G as its For every proper equilibrium x in G NF, there exists a realization-equivalent SE y in . , and x be as stated Proof sketch: Let G, G 1. a sequence of t-proper proles xt int [ (S )] with t 0 and xt x

2. For each xt a realization-equivalent behavior-strategy y t in 3. Since xt int [ (S )], each info set in is on the path of y t 4. Since xt x, y t y Y . Sucient to verify that y is a SE!
t t 5. For each t N, let = | y . Then = limt t is a belief system consistent with y

6. Suppose that y is not sequentially rational under : player i, 0 Y such that y 0 6= y and information set I Ii and yi i iI iI
aI

(a)

X 0 (a) ia (y ) ia yi, yi > aI

7. But this leads to a contradiction.

Forward induction
Discussion in class of the outside-option game in the light of forward induction reasoning (see Kohlberg and Mertens, 1986):

(3,1) a

(0,0) b 1

(0,0) a

(1,3) b

A (2,v) 2 L R

THE END