Quantal Response Equilibrium for Sponsored Search Auctions

Parameter Estimation with Unknown

CTRs

Then the parameters can be estimated by maximizing the

likelihood as shown in the following optimization problem:

fixed, it is easy to know that the likelihood has the following

discontinuous points:
j bj 
j bj
{
0<
< 1, j I \ {i}, bi Bi , bj Bj }.
bi
bi
Then we can partition the feasible domain of i into several
intervals where the likelihood function L is continuous with
respect to i , and then by solving the optimization problem
defined in Eq. (A.6) in each interval we can find a better i
given all the other parameters:

max L(v, , , |)

(A.2)

max L(v, , , |) s.t. i in the continuous interval.

(A.3)

(A.6)
Similarly, the above optimization problem is not convex. To
avoid being tracked into a bad local maximum, we can also
find a set of local maxima with different starting points and
choose the best one.

Given a QRE strategy , the logarithmic likelihood of the

unknown parameters v, , and is
Y Y
L(v, , , |) = log(
ij (i )ij )
(A.1)
iI oj Bi

v,,,

v , 1, s s+1 0,
s.t. i i
0 < i , s < 1.

Since in SSAs, the allocation rule is not a continuous function of ad CTR profile , the utility of a bidder is not continuous with respect to ; as a result, the likelihood defined in
Eq. (A.1) is not continuous with respect to .
To address the discontinuity of the likelihood function, we
split the unknown parameters into two groups and sequentially optimize them in each iteration: we treat v, , as a group
and as the other group; in each iteration, we first optimize
v, , and then .
The function L(v, , , |) in Eq. (A.4) is continuous
with respect to the parameters in the first group. We can learn
a better set of v, , by solving the following sub optimization problem:
max L(v, , , |)
(A.4)

Algorithm A.1: Parameter estimation with unknown CTRs

1
2
3
4
5
6

v,,


s.t.

vi , i 1, s s+1 0,
0 < s < 1.

8
9

L ;
Randomly generate an ad CTR profile ;
while True do
Fix and update v, , by solving the problem shown in
Eqs. (A.4) and (A.5);
for i 1, 2, , N do
Fix j , j 6= i, v, , and update i by solving the
sub problem as shown in Eq. (A.6) in each continuous
interval;
with the updated parameters
Calculate the likelihood L
v, , , ;
L > then L L;

if L
else return the learned parameters v, , , ;

(A.5)

Since the above optimization problem is non-convex, it is difficult to find the global maximum. We turn to find a set of
local maxima with different starting points and then choose
the best one to improve the possibility of reaching the global
maximum of the sub problem.
As aforementioned, the likelihood function is not continuous with respect to . Here we do not optimize bidders
ad CTRs simultaneously. Instead, we deal with them one by
one. Let us take the ad CTR i of bidder i as an example and
keep j , j 6= i fixed. Given that all the other parameters are

The complete procedure is presented in Algorithm ??. In

line 1 we initialize the likelihood of the original optimization
problem with negative infinity. Line 2 sets an initial . Lines
3-9 iteratively optimize the two groups of parameters. Line
4 fixes and updates (v, , ). Lines 5-7 fix (v, , ) and
update . Line 8 checks the performance of the updated parameters. Line 9 control the optimization process: if we make
progress in this iteration, we continue the optimization; otherwise, we return the latest parameters. Again, to avoid a bad
local maximum, we run the algorithm multiple times with d-

ifferent initial s in Line 2 and choose the best parameters as

the final output in our experiments.

Additional Experimental Results

B.1

Experimental Results for Queries 2 and 3

query
2
3

N
1
2
3
1
2
3

v
10.92
15.05
9.60
6.00
1.65
5.67

4711
2945
3060
6251
222
4700

.1770
.1683
.3685
.1348
.6466
.1827

.2137
.1898
.0742
.1515
.1011
.0504

O
1
1
error
u1
2
2
error
u2
3
3
error
u3

4.0

.6348
.6359
.0011
.4250

6.5
.6417
.6422
.0005
.0910

8.0

.7660
.7623
.0037
.2179

-1.8535

1.0

.6897
.6881
.0016
.0633

2.0
.2927
.2682
.0245
.0339

2.5

.1466
.1040
.0426
.0548
.0522
.0524
.0002
.0581

3.0
.2439
.2682
.0243
.0339
.1207
.1040
.0167
.0548

3.5

.1304
.1304
.0000
.0583

8.5

.3652
.3641
.0011
.4249

4.0

.8174
.8172
.0002
.0587

10.0
.3583
.3578
.0005
.0909
.2340
.2377
.0037
.2175

4.5
.4634
.4635
.0001
.0340

5.0

.0431
.1040
.0609
.0548

Table B.3: Real and predicted mixed strategies for query 3

Evaluation of Mixed Strategy Nash

Equilibrium for Real Data

We fit the mixed strategy Nash equilibrium (MSNE) into the

real data and make a comparison with the QRE model. In
the MSNE, each bidders expected utilities for different pure
strategies are the same. Then the logarithmic likelihood function of MSNE with unknown parameters v, , and , given
, is
Y Y uij (i ) 1
L(v, , , |) = log(
(P
) |Bi | ). (A.7)
iI oj Bi

N
1
2
3
1
2
3
1
2
3

v
18.16
12.80
112.28
1.68106
1.29106
0.80106
8.37106
4.75106
3.08106

.5066
.9091
.1641
.3614
.6129
.3459
.0220
.0474
.0910

.8292
.0186
.0153
.0750
.0377
.0376
.0700
.0692
.0691

L
-3.5835

-2.0794

-4.7594

Table B.4: Estimated parameters for MSNE

-2.6098

Table B.2: Real and predicted mixed strategies for query 2

O
1
1
error
u1
2
2
error
u2
3
3
error
u2

Table B.1: Estimated parameters

B.2

query

ok Bi

We apply Eq. (A.7) into Algorithm A.1 to estimate the parameters of the three queries presented in the experiment section.
We use 10 iterations and the results are depicted in Table B.4.

We see from Table B.4 that the likelihoods of MSNE are

smaller than the corresponding likelihoods of QRE model,
which indicates that QRE is more accurate for fitting the
real data. Besides, the estimated parameters of MSNE are
very strange. Specifically, the values of 1 1 and 2 1
seems overly large since the CTRs of SSAs are usually less
than 10%. Note that the maximal bids of queries 2 and 3
are not greater than 10, while the estimated values are almost at the magnitude of 106 , which are very abnormal. This
phenomenon may be explained as follows: in order to make
bidders expected utilities indifferent for various pure strategies, Algorithm A.1 needs to assign large numbers to bidders values to ensure that pij (bi )  vi and thus uij (bi ) =
(vi pij (bi ))i rij (bi ) vi i rij (bi ) , oj Bi , which
reduce the effects of different pure strategies.
Overall, the MSNE can not fit the real data as well as the
QRE model did.