Chapter 1

LI NEAR ALGEBRA AND MATRI X METHODS I N

ECONOMETRI CS
HENRI THEIL*
University of Florida
Contents
1. Introduction 5
2. Why are matrix methods useful in econometrics? 5
2.1. Linear systems and quadratic forms 5
2.2. Vectors and matrices in statistical theory 7
2.3. Least squares in the standard linear model 8
2.4. Vectors and matrices in consumption theory 10
3. Partitioned matrices 12
3.1. The algebra of partitioned matrices 12
3.2. Block-recursive systems 14
3.3. Income and price derivatives revisited 15
4. Kronecker products and the vectorization of matrices 16
4.1. The algebra of Kronecker products 16
4.2. Joint generalized least-squares estimation of several equations 17
4.3. Vectorization of matrices 19
5. Differential demand and supply systems 20
5.1. A differential consumer demand system 20
52. A comparison with simultaneous equation systems 22
5.3. An extension to the inputs of a firm: A singularity problem 23
5.4. A differential input demand system 23
5.5. Allocation systems 25
5.6. Extensions 25
6. Definite and semidefinite square matrices 27
6.1. Covariance matrices and Gauss-Markov further considered 2?
6.2. Maxima and minima 29
6.3. Block-diagonal definite matrices 30
7. Diagonalizations 30
7.1. The standard diagonalization of a square matrix 30
*Research supported in part by NSF Grant SOC76-82718. The author is indebted to Kenneth
Clements (Reserve Bank of Australia, Sydney) and Michael Intriligator (University of California, Los
Angeles) for comments on an earlier draft of this chapter.
Handbook of Econometrics, Volume I, Edited by Z. Griliches and M.D. Intriligator
North-Holland Publishing Company, 1983
11. Theil
7.2. Special cases
7.3. Aitken's theorem
7.4. The Cholesky decomposition
7.5. Vectors written as diagonal matrices
7.6. A simultaneous diagonalization of two square matrices
7.7. Latent roots of an asymmetric matrix
8. Pri nci pal component s and ext ensi ons
8. I. Principal components
8.2. Derivations
8.3. Further discussion of principal components
8.4. The independence transformation in microeconomic theory
8.5. An example
8.6. A principal component interpretation
9. The model i ng of a di st ur bance covari ance mat ri x
9.1. Rational random behavior
9.2. The asymptotics of rational random behavior
9.3. Applications to demand and supply
10. The Moor e- Penr os e i nverse
10.1. Proof of the existence and uniqueness
10.2. Special cases
10.3. A generalization of Aitken's theorem
10.4. Deleting an equation from an allocation model
Appendi x A: Li near i ndependence and rel at ed topics
Appendi x B: The i ndependence t r ansf or mat i on
Appendi x C: Rat i onal r andom behavi or
References
32
33
34
34
35
36
37
37
38
40
40
43
44
45
46
47
49
51
51
52
53
56
57
58
61
64
Ch. 1: Linear Algebra and Matrix Methods 5
1. I nt r oduc t i on
Vect ors and mat ri ces pl ayed a mi nor role in the economet r i c l i t erat ure publ i shed
bef or e Wor l d War II, but t hey have become an i ndi spensabl e tool in t he last
several decades. Part of this devel opment results f r om t he i mpor t ance of mat ri x
tools for t he statistical component of economet ri cs; anot her r eason is t he in-
creased use of mat ri x al gebra in t he economi c t heor y underl yi ng economet ri c
relations. The objective of this chapt er is t o pr ovi de a selective survey of bot h
areas. El ement ar y propert i es of mat ri ces and det er mi nant s are assumed to be
known, i ncl udi ng summat i on, mul t i pl i cat i on, inversion, and transposition, but the
concept s of l i near dependence and ort hogonal i t y of vectors and the r ank of a
mat r i x are bri efl y reviewed in Appendi x A. Ref er ence is made t o Dhr ymes (1978),
Graybi l l (1969), or Hadl ey (1961) for el ement ary propert i es not covered in this
chapter.
Matrices are i ndi cat ed by bol df ace italic upper case letters (such as A), col umn
vect ors by bol df ace italic lower case letters ( a) , and r ow vectors by bol dface italic
l ower case letters with a pri me added ( a' ) to i ndi cat e t hat t hey are obt ai ned f r om
t he correspondi ng col umn vect or by transposition. The following abbrevi at i ons
are used:
LS -~ least squares,
GLS --- generalized least squares,
ML = maxi mum likelihood,
8ij = Kr onecker del t a ( = 1 if i = j , 0 if i ~ j ) .
2. Why are mat ri x me t h o d s us e f ul i n e c o no me t r i c s ?
2.1. Linear systems and quadratic forms
A maj or reason why mat r i x met hods are useful is t hat many topics in economet -
rics have a mul t i vari at e character. For example, consi der a syst em of L simulta-
neous linear equat i ons in L endogenous and K exogenous variables. We write Y~/
and x~k for the ~t h observat i on on the / t h endogenous and t he kt h exogenous
variable. Then t h e j t h equat i on for observat i on c~ takes t he f or m
L K
v, j yo, + 2 = ( 2 . 1 )
/ =1 k=l
6 H. Theil
wher e e, j is a r a n d o m di s t ur bance a nd t he 7 ' s a n d / 3 ' s ar e coef f i ci ent s. We can
wr i t e (2. 1) f o r j = 1 . . . . . L i n t he f o r m
y ' F + x' ~B = e' ,, ( 2. 2)
wher e y" = [Y,1--.Y,~L] and x" = [x, 1 . . . x~K ] ar e obs e r va t i on vect or s on t he e ndog-
enous and t he exogenous vari abl es, r espect i vel y, e'~ = [ e ~v. . e e l ] is a di s t ur ba nc e
vect or , a nd I" and B ar e coef f i ci ent mat r i ces of or der L x L a nd K L, r espec-
t i vel y:
- =
711 ]/12""-YIL
]/21, T22" Y2 L
[YL1 7L2.-' YLL
B =
/ 3 1 1 / 3 1 2 " " " / 3 1 L
/ 3 2 1 / 3 2 2 . . . / 3 2 L
/ 3 < 1 / 3 K 2 . - - / 3 I < L
Wh e n t her e are n obs er vat i ons (c~ = 1 . . . . . n), t her e ar e Ln equat i ons of t he f o r m
(2. 1) a nd n equat i ons of t he f o r m (2.2). We can c o mb i n e t hese e qua t i ons
c o mp a c t l y i nt o
v r + x B = E, ( 2. 3)
wher e Y and X are obs er vat i on mat r i ces of t he t wo sets of var i abl es of or der
n L a nd n K, r espect i vel y:
y =
I 1
/ 1 x .
Y21 Y22. . "Y2L I X = x21 x22
: : '
L Y : I 2 "Y:,L Xm X, 2. . . X, I <
a nd E is an n L di s t ur bance mat r i x:
E =
1
El l E l 2 " ' ' e l L /
1
/~21 E22" "' E2L
Enl En2--- EnL
No t e t h a t / " is squar e ( L L) . I f / " is al so non- s i ngul ar , we can pos t mul t i py
(2. 3) b y / - - 1 :
Y = - XBI ' - l + E F - ~ ( 2 . 4 )
Ch . 1: L i n e a r A l g e b r a a n d M a t r i x M e t h o d s
Thi s is the reduced form f or all n observat i ons on all L endogenous variables, each
of which is descri bed l i nearl y i n t erms of exogenous values and disturbances. By
cont rast , t he equat i ons (2.1) or (2.2) or (2.3) f r om whi ch (2.4) is deri ved const i t ut e
t he structural form of t he equat i on system.
The previ ous paragraphs illustrate t he conveni ence of mat ri ces for linear
systems. However, t he expressi on "l i near al gebra" shoul d not be i nt erpret ed in
t he sense t hat mat ri ces are useful for linear systems only. The t r eat ment of
quadrat i c funct i ons can also be simplified by means of matrices. Let g( z 1 . . . . . Zk)
be a t hree times di fferent i abl e funct i on. A Tayl or expansi on yields
k Og
g ( Z l . . . . . Z k ) = g ( i f l . . . . . i f ' k ) + E ( Z i - - i f i ) OZ i
i = 1
1 k k
+~ E E(~i-2, )~ ~g
i =, s - , , ~%-
i f j ) -.I- O 3 ,
(2. 5)
where 03 is a t hi r d- or der r emai nder term, while t he derivatives Og/Oz~ and
OZg/Ozi Ozj are all eval uat ed at z 1 = if~ . . . . . Zk = ifk- We i nt r oduce z and ~ as
vect ors wi t h i t h el ement s z i and if i, respectively. Then (2.5) can be wri t t en in t he
mor e compact f or m
_~, Og 1 _. , 02g
g(~) = g( ~) + (z - z ) ~ + ~( z - z ) o- ~- ~(z - ~) +o 3,
(2. 6)
where the col umn vect or Og/ Sz = [Og/Ozi] is the gradient of g( - ) at ~ (the vect or
of fi rst -order derivatives) and the mat ri x 02g/ OzOz' =[O2g/ Szi Szj ] is the
Hessian matrix of g( - ) at ~ (the mat ri x of second-order derivatives). A Hessi an
mat r i x is always symmet ri c when t he funct i on is t hree times differentiable.
2. 2. Vectors and matrices in stat&tical theory
Vect ors and mat ri ces are also i mpor t ant in t he statistical component of economet -
rics. Let r be a col umn vect or consisting of t he r andom variables r 1 . . . . . r n. The
expect at i on Er is def i ned as t he col umn vect or of expect at i ons Er~ . . . . . Er n. Next
consi der
I r I - ~r 1 ]
( r - E r ) ( r - E r ) ' = r2 Er2 [ r I - - Er~
r . E r .
r ~ - ~ r ~ . . . r . - ~ r . ]
8 H. Theil
and take t he expect at i on of each el ement of this pr oduct matrix. When defi ni ng
t he expect at i on of a r andom mat ri x as t he mat r i x of t he expect at i ons of the
const i t uent elements, we obt ai n:
var r, coy(r1, r 2) . . . coy(r1, r n) ]
E [ ( r - E v ) ( r - E r ) ' ] = c v ( r 2 ' r ' ) varr2 "'" cv(r2'rn) . . ]
cov(l n, r , ) cov( r , , r 2) . . - var r, j
Thi s is t he var i ance- covar i ance mat r i x (covari ance matrix, for short ) of t he vect or
r, to be wri t t en ~ ( r ) . The covari ance mat r i x is always symmet ri c and cont ai ns the
variances along the diagonal. I f the el ement s of r are pairwise uncorrel at ed, ~ ( r )
is a di agonal matrix. I f these elements also have equal variances (equal to o 2, say),
<( ( r ) is a scalar matrix, o21; t hat is, a scalar mul t i pl e 0 2 of t he uni t or i dent i t y
matrix.
The mul t i vari at e nat ur e of economet ri cs was emphasi zed at t he begi nni ng of
this section. Thi s will usually i mpl y t hat t here are several unknown paramet ers;
we arrange these in a vect or 0. The pr obl em is t hen t o obt ai n a " good" est i mat or
/~ of 0 as well as a sat i sfact ory measure of how good t he est i mat or is; t he most
popul ar measure is t he covari ance mat ri x ~7(~). Somet i mes this pr obl em is
simple, but t hat is not always the case, in part i cul ar when the model is non-l i near
in t he paramet ers. A general met hod of est i mat i on is maxi mum l i kel i hood ( ML)
whi ch can be shown to have cert ai n opt i mal propert i es for large samples under
relatively weak condi t i ons. The deri vat i on of t he ML estimates and their large-
sample covari ance mat ri x involves t he information matrix, whi ch is (apart f r om
sign) t he expect at i on of the mat ri x of second-order derivatives of the log-likeli-
hood funct i on with respect to t he paramet ers. The pr omi nence of ML est i mat i on
in recent years has greatly cont r i but ed t o t he i ncreased use of mat r i x met hods in
economet ri cs.
2.3. Least squares in the standard linear model
We consi der the model
y = X/3 + e, (2.7)
where y is an n- el ement col umn vect or of observat i ons on t he dependent (or
endogenous) variable, X is an n x K observat i on mat r i x of r ank K on t he K
i ndependent (or exogenous) variables, fl is a par amet er vector, and e is a
Ch. 1: Linear Algebra and Matrix Methods
di st ur bance vect or. The standard linear model post ul at es t hat e has zero expect a-
t i on and covar i ance mat r i x o21, wher e 0 2 is an unknown posi t i ve par amet er , and
t hat the el ement s of X are all non-st ochast i c. Not e t hat this model can be vi ewed
as a special case of (2.3) f o r / " = I and L = 1.
The pr obl em is to est i mat e/ 3 and 0 2. The least-squares (LS) est i mat or of / 3 is
b = ( x' x) - l x' y ( 2 . 8 )
whi ch owes its name to t he fact t hat it mi ni mi zes t he resi dual s um of squares. To
veri fy this pr oposi t i on we write e = y - Xb for t he resi dual vect or; t hen the
resi dual sum of squares equals
e' e = y' y - 2 y ' Xb + b' X' Xb , (2. 9)
whi ch is to be mi ni mi zed by var yi ng b. Thi s is achi eved by equat i ng t he gradi ent
of (2.9) to zero. A compar i s on of (2.9) wi t h (2.5) and (2.6), wi t h z i nt er pr et ed as b,
shows t hat t he gradi ent of (2.9) equal s - 2X' y + 2X' Xb, f r om whi ch the sol ut i on
(2.8) follows directly.
Subst i t ut i on of (2.7) i nt o (2.8) yields b - 13 = ( X' X) IX'e. Hence, given Ee - - 0
and the non- r andomnes s of X, b is an unbi ased est i mat or of / 3. It s covari ance
mat r i x is
= ( x' x) o 2 ( x ' x ) =' (2. 10)
because X' ~( e ) X = o2Xt X follows f r om ~ ( e ) = o21. The Ga u s s - Ma r k o v t heo-
r em states t hat b is a best l i near unbi ased est i mat or of / 3, whi ch amount s to an
opt i mum LS pr oper t y wi t hi n t he class of 13 est i mat or s t hat are l i near in y and
unbi ased. Thi s pr oper t y i mpl i es t hat each el ement of b has the smal l est possi bl e
vari ance; t hat is, t here exists no ot her l i near unbi ased est i mat or of 13 whose
el ement s have smal l er vari ances t han t hose of the cor r espondi ng el ement s of b. A
mor e general f or mul at i on of the Ga u s s - Ma r k o v t heor em will be given and
pr oved in Sect i on 6.
Subst i t ut i on of (2.8) i nt o e = y - Xb yields e = My, where M is the symmet r i c
mat r i x
M: I - X( X' X) - I X ' (2.11)
whi ch satisfies MX = 0; therefore, e = My = M( X/ 3 + e) = Me. Also, M is
idempotent, i.e. M 2 = M. The LS resi dual sum of squares equal s e' e = gM' Me =
e' MZe and hence
e' e = gMe . (2.12)
10 H. T h e i l
I t is shown in t he next par agr aph t hat E ( e ' Me ) = a2( n - K ) so t hat (2.12) i mpl i es
t hat 0 2 is est i mat ed unbi asedl y by e ' e / ( n - K) : the LS resi dual s um of squares
di vi ded by the excess of the numbe r of obser vat i ons ( n) over t he numbe r of
coeffi ci ent s adj ust ed ( K) .
To pr ove E ( g Me ) = o2( n - K ) we defi ne t he trace of a square mat r i x as the
sum of its di agonal el ement s: t r A= a~l + + ann. We use t r A B = t r BA ( i f AB
and BA exist) to write g a l e as t r Me e ' . Next we use t r(A + B) = t r A + t r B (if A
and B are square of t he same order) to wri t e t r Me g as t r e e ' - t r X( X' X) - ~ X' e e '
[see (2.11)]. Thus, since X is non- st ochast i c and t he t r ace is a l i near oper at or ,
E ( e ' Me ) : t r E (ee' ) - t r X ( X ' X ) - 1 X ' E ( ee' )
= o Z t r l - a 2 t r X ( X , X ) - 1 X,
= o2n - o Z t r ( X ' X ) 1XtX,
whi ch conf i r ms E( e ' Me ) = o 2 ( n - K ) because ( X ' X ) 1 X' X = I of order K X K.
If, in addi t i on to t he condi t i ons listed in t he di scussi on fol l owi ng eq. (2.7), t he
el ement s of e are nor mal l y di st ri but ed, the LS est i mat or b of 13 is i dent i cal t o the
ML est i mat or; also, ( n - K ) s 2 / o 2 is t hen di st r i but ed as X 2 wi t h n - K degrees of
f r eedom and b and s 2 are i ndependent l y di st ri but ed. For a pr oof of this resul t see,
f or exampl e, Thei l (1971, sec. 3.5).
I f the covar i ance mat r i x of e is o 2 V r at her t han o21, where V is a non- si ngul ar
mat ri x, we can ext end t he Ga u s s - Ma r k o v t heor em to Ai t ken' s (1935) t heorem.
The best l i near unbi ased est i mat or of fl is now
1~ = ( X ' V - I X ) - J X ' V - l y, (2. 13)
and its covar i ance mat r i x is
~(1~ ) = oZ( X ' V - 1X ) - 1. ( 2 . 1 4 )
The est i mat or / ~ is the general i zed l east -squares ( GLS) est i mat or of r ; we shall see
in Sect i on 7 how it can be deri ved f r om t he LS est i mat or b.
2.4. Vectors and mat ri ces in consumpt i on theory
I t woul d be i nappr opr i at e t o leave t he i mpr essi on t hat vect ors and mat r i ces are
i mpor t ant in economet r i cs pr i mar i l y because of pr obl ems of st at i st i cal i nference.
They are al so i mpor t ant for t he pr obl em of how t o speci fy economi c relations. We
shall i l l ust rat e this here f or t he anal ysi s of cons umer demand, whi ch is one of t he
ol dest t opi cs in appl i ed economet ri cs. Ref er ences for t he account whi ch follows
Ch. 1: Linear Algebra and Matrix Methods 11
i ncl ude Bart en (1977), Brown and Deat on (1972), Phlips (1974), Thei l (1975-76),
and Deat on' s chapt er on demand analysis in this Ha ndbook ( Chapt er 30).
Let t here be N goods in t he market pl ace. We write p = [p; ] and q = [qg] for the
pri ce and quant i t y vectors. The consumer' s preferences are measured by a ut i l i t y
funct i on u(q) whi ch is assumed t o be t hree times di fferent i abl e. Hi s pr obl em is to
maxi mi ze u(q) by varyi ng q subject t o the budget const rai nt s p'q = M, where M is
t he given posi t i ve amount of t ot al expendi t ur e (t o be called i ncome for brevi t y' s
sake). Prices are also assumed t o be positive and given f r om t he consumer' s poi nt
of view. Once he has solved this probl em, t he demand f or each good becomes a
f unct i on of i ncome and prices. What can be said about t he derivatives of demand,
Oqi / OM and Oqi/Opj?
Neoclassical consumpt i on t heory answers this quest i on by const ruct i ng the
Lagrangi an funct i on u ( q ) - h ( p ' q - M) and di fferent i at i ng this funct i on with
respect to t he qi' s. When these derivatives are equat ed t o zero, we obt ai n the
familiar pr opor t i onal i t y of margi nal utilities and prices:
Ou
iOq-- = Xp, , i = 1 . . . . . N, (2. 15)
or, in vect or not at i on, OU/&l = Xp: t he gradi ent of the utility funct i on at the
opt i mal poi nt is pr opor t i onal t o t he pri ce vector. The pr opor t i onal i t y coeffi ci ent
has t he i nt erpret at i on as t he margi nal utility of income.1
The pr opor t i onal i t y (2.15) and t he budget const rai nt p'q = M provi de N+ 1
equat i ons in N + 1 unknowns: q and )t. Since these equat i ons hol d identically in
M and p, we can di fferent i at e t hem wi t h respect to t hese variables. Di fferent i at i on
of p' q = M wi t h respect t o M yields ~ p~(Oqi / OM ) = 1 or
P' O- ~ =1 , (2.16)
where Oq/ OM=[ Oqi / OM ] is t he vect or of i ncome derivatives of demand.
Di fferent i at i on of P ! / = M with respect to pj yields ~i Pi( Oqi/OPj)+ qj = 0 ( j =
1 . . . . . N ) or
p, 0q (2.17)
Op' = - q "
where Oq/ Op' = [Oqi/Opj ] is t he N N mat ri x of pri ce derivatives of demand.
Di fferent i at i on of (2.15) with respect to M and appl i cat i on of the chai n rule
IDividing both sides of (2.15) by Pi yields Ou/O(piqi)= h, which shows that an extra dollar of
income spent on any of the N goods raises utility by X. This provides an intuitive justification for the
interpretation. A more rigorous justification would require the introduction of the indirect utility
function, which is beyond the scope of this chapter.
12 H. Theil
yields:
N 02U Oqk 07,
~ OqiOqk OM Pi - OM'
k = l
i = 1 . . . . . N.
Similarly, di fferent i at i on of (2.15) wi t h respect to pj yields:
N 02U Oqk 07,
Y. Pi ~p j + ?,~iS, i, j = 1 . . . . . N,
k = 1 Oqi Oqk Opj
where 8ij is t he Kr onecker del t a ( = 1 if i = j , 0 if i ~ j ) . We can wri t e the last two
equat i ons in mat ri x f or m as
aq
0?`
a a
0?,
U U-~7_ ~, = ?, I + p (2. 18)
OM = OM p ' Op ~ p "
where U = 02u/ OqOq t is t he Hessi an mat r i x of the consumer' s ut i l i t y funct i on.
We show at the end of Sect i on 3 how t he four equat i ons di spl ayed in (2. 16)-(2. 18)
can be combi ned in part i t i oned mat r i x f or m and how t hey can be used to pr ovi de
solutions for t he i ncome and pri ce derivatives of demand under appr opr i at e
condi t i ons.
3. Partitioned matrices
Part i t i oni ng a mat ri x i nt o submat ri ces is one device f or the expl oi t at i on of the
mat hemat i cal st ruct ure of this matrix. Thi s can be of consi derabl e i mpor t ance in
mul t i vari at e situations.
3.1. The algebra of part i t i oned matrices
We write t he l eft -most mat ri x in (2.3) as Y= [Y~ Y2], where
Y I
lY l1 1 2 ] I3 Y I4 1 L 1 Yzl Y22 723 Y24 "Y2L
~ Y 2 ~ . .
The part i t i oni ng Y= [Y~ Y2] is by sets of columns, t he observat i ons on t he first
t wo endogenous variables bei ng separat ed f r om those c,n the others. Part i t i oni ng
Ch. 1: Linear Algebra and Matrix Methods 13
may take place by row sets and column sets. The addition rule for matrices can be
applied in partitioned form,
A21
A31
A12
A22 +
A32
BI, BI2] j a i l + BI1 AI2 + BI2]
B21 B221 = /A21+B21 A22+B221 '
B31 B 3 2 1 LA31+ B3, A32 + B32J
provided A ij and B,j have the same order for each (i, j ) . A similar result holds for
multiplication,
PI, P12][Ql , Q , 2 ] = [ P l l Q I , + P 1 2 0 2 , P,1Q,z+P12Q22]
P2~ P22] [ Q2, Q22] P 2 i Q~ l +P 2 2 Q2 , P2 1 Q, 2 +P2 2 Q2 2 1 '
provided that the number of columns of Pl l and P2~ is equal to the number of
rows of Qll and Qi2 (similarly for Pl2, P22, Qz l , Q22)-
The inverse of a symmetric partitioned matrix is frequently needed. Two
alternative expressions are available:
, l l i o o , c , I
' C - C - 1 B ' D C 1 + C l B ' D B C - 1 '
where O = ( A - B C 1B' ) l and E = ( C - B ' A - I B ) - i. The use of (3.1) requires
that C be non-singular; for (3.2) we must assume that A is non-singular. The
verification of these results is a matter of straightforward partitioned multiplica-
tion; for a constructive proof see Theil (1971, sec. 1.2).
The density function of the L-variate normal distribution with mean vector/x
and non-singular covariance matrix ~ is
1
f ( x ) ( 2 ~ r ) L / 2 1 X i l / 2 e x p ( - (x - / x ) ' X- l ( x - / ~) ) , (3.3)
where 12~ I is the determinant value of ~. Suppose that each of the first L' variates
is uncorrelated with all L - L' other variates. Then/~ and ~ may be partitioned,
where (/~l, ~l) contains the first- and second-order moments of the first L'
14 H. Theil
variates and (~2, ~2) those of the last L - L' . The density funct i on (3.3) can now
be written as the product of
/ I ( X l ) =
1 e xp( - ( x l - / ~ , ) ' Z ( ~ ( x I - / ~l ) )
( 2~r) L' / 2l ~l l 1/2
and analogous funct i on f2(x2). Clearly, the L-el ement normal vector consists of
two subvectors which are i ndependent l y distributed.
3.2. Bl ock- recursive syst ems
We ret urn to the equat i on system (2.3) and assume t hat the rows of E are
i ndependent L-variate normal vectors wi t h zero mean and covariance mat ri x Z, as
shown in (2.4), Zl being of order L' X L' . We also assume t hat /" can be
part i t i oned as
r 3 ] , ( 3 . 5 )
/ ' 2
wi t h/ ' l of order L' L' . Then we can write (2.3) as
0 ~- X[ BI B2] = [El E2]
or
(3.6)
(3.7)
E2] with Y1 and E 1 of order
Y~F 1 + X B t = El ,
Y2Fz+[ X Y1][ Bz]F3 = E 2 '
where Y=[ Yl Y2], B= [ B l B2], a n d E = [ E 1
n L' and B l of order K X L' .
There is not hi ng special about (3,6), which is an equat i on system comparabl e to
(2.3) but of smaller size. However, (3.7) is an equat i on system in which the L'
variables whose observations are arranged i n Y1 can be viewed as exogenous
rat her t han endogenous. This is i ndi cat ed by combi ni ng Yl with X in part i t i oned
mat ri x form. There are two reasons why Yi can be viewed as exogenous in (3.7).
First, Yl is obt ai ned from the system (3.6) which does not involve Y2. Secondly,
the r andom component E 1 in (3.6) is i ndependent of E 2 in (3.7) because of the
assumed normal i t y with a block-diagonal Z. The case discussed here is t hat of a
Ch. 1: Linear Algebra and Matrix Methods 15
block-recursive system, wi t h a bl ock-t ri angul ar/ " [see (3.5)] and a block-diagonal Z
[see (3.4)]. Under appropri at e i dent i fi cat i on conditions, ML est i mat i on of the
unknown elements of F and B can be applied to the two subsystems (3.6) and
(3.7) separately.
3.3. Income and price derivatives revisited
It is readily verified t hat eqs. (2.16)-(2.18) can be written in part i t i oned mat ri x
form as
- O) ~ / OM - O ) ~ / O p ' ] = [ ~ )~I _ q t
which is Barten' s (1964) f undament al mat ri x equat i on in consumpt i on theory. All
three part i t i oned matrices in (3.8) are of order ( N + 1) ( N + 1), and the left-most
mat ri x is the Hessian mat ri x of utility funct i on bordered by prices. If U is
non-singular, we can use (3.2) for the inverse of this bordered matrix:
Premultiplication of (3.8) by this inverse yields solutions for the i ncome and price
derivatives:
O q _ 1 _ 0 ~ _ 1
OM p ' U- l p U lp, OM p ' U- lp
Oq ?~u_ , ~ U _ l p ( U _ , p ) , 1
op' p' v '1, p' v- ' p
- - u- ' pq' .
(3.9)
(3.10)
It follows from (3.9) t hat we can write the i ncome derivatives of demand as
Oq 07~ U_l p ' (3.11)
OM OM
and from (3.9)~ and (3.10) t hat we can simplify the price derivatives to
_ ~ u _ ~ ~ ~ 0q'
Op' O?~/OM OM OM q'" (3.12)
The last matrix, - ( Oq / OM) q ' , represents the income effect of the price changes
16 H. Theil
on demand. Not e t hat this mat ri x has uni t rank and is not symmetric. The two
other matrices on the right in (3.12) are symmet ri c and j oi nt l y represent the
subst i t ut i on effect of the price changes. The first matrix, XU-~, gives the specific
substitution effect and the second (which has uni t rank) gives the general substitu-
tion effect. The latter effect describes the general compet i t i on of all goods for an
extra dollar of income. The distinction between the two component s of the
subst i t ut i on effect is from Hout hakker (1960). We can combi ne these component s
by writing (3.12) in the form
= p a q ' 1 a q , Oq X U - ' ( I - (3.13)
Op' ff--M ] - - ~ q '
which is obt ai ned by using (3.1 l) for the first Oq/ OM t hat occurs in (3.12).
4. Kronecker products and the vectorization of matrices
A special form of part i t i oni ng is t hat in which all submatrices are scalar multiples
of the same matrix B of order p x q. We write this as
AB =
a l l B a l z B . . . a l n B ]
a2:B a z 2 B. . . a 2 n B ,
aml B am2B. . . amnB
and refer to A B as the Kronecker product of A = [aij ] and B. The order of this
product is mp nq. Kronecker product s are part i cul arl y conveni ent when several
equations are analyzed simultaneously.
4. 1. The algebra o f Kronecker products
It is a mat t er of straightforward part i t i oned mul t i pl i cat i on to verify t hat
( A B ) ( C D ) = A C B D , (4.1)
provi ded A C and BD exist. Also, i f A and B are square and non-singular, t hen
( A e B ) - ' = A- I B - 1 (4.2)
because (4.1) implies ( A B ) ( A - ~ B - l ) = A A - ~ B B -1 = 1 1 = I , where the
three uni t matrices will in general be of di fferent order. We can obviously extend
Ch. 1: Linear Algebra and Matrix Methods 17
(4.1) to
( A, Bi ) ( AzBz) ( A3B3) = AI Az A3BI Bz B 3
provided AI A2A 3 and BIBzB3 exist.
Other useful properties of Kronecker product s are:
( A B ) ' =A ' B ' , (4.3)
A ( B + C) = AB + aC, (4.4)
( B + c) a = B eA + C eA , ( 4 . 5 )
A( BC) = ( AB )C. (4.6)
Not e the implication of (4.3) t hat AB is symmet ri c when A and B are
symmetric. Ot her properties of Kronecker product s are considered in Section 7.
4.2. Joint generalized least-squares estimation of several equations
In (2.1) and (2.3) we considered a system of L linear equations in L endogenous
variables. Here we consider the special case in which each equat i on describes one
endogenous variable in terms of exogenous variables only. If the observations on
all variables are a = 1 . . . . . n, we can write the L equations in a form similar to
(2.7):
y/ = Xj~j + e/, j = 1 . . . . . L, (4.7)
where yj = [ y: j ] is the observation vector on the j t h endogenous variable, ej =
[e:j] is the associated di st urbance vector with zero expectation, Xj is the observa-
t i on mat ri x on the Kj exogenous variables in the j t h equation, and fij is the
K: el ement paramet er vector.
We can write (4.7) for a l l j in part i t i oned mat ri x form:
I i ]
X 1 0. . . 0
o x2...o
0 O...XL
~2 e2
+
L eiL
(4.8)
or, more briefly, as
y = Z/3 + e, (4.9)
18 H. Theil
where y and e are Ln- el ement vect ors and Z cont ai ns Ln rows, whi l e the numbe r
of col umns of Z and t hat of the el ement s of /3 are bot h K 1 + . . . + K L. The
covar i ance mat r i x of e is t hus of or der Ln Ln and can be par t i t i oned i nt o L 2
submat r i ces of t he f or m E(eje~). For j = l this s ubmat r i x equal s t he covar i ance
mat r i x ~V(ej). We assume t hat the n di st ur bances of each of t he L equat i ons have
equal vari ance and are uncor r el at ed so t hat ~V(e/) = ojiI, where og = var e~/ ( each
a) . For j ~ l t he submat r i x E(e/e~) cont ai ns t he " c ont e mpor a ne ous " covar i ances
E (e, j e~t ) for a = 1 . . . . . n in the di agonal . We assume t hat these covar i ances ar e all
equal to ~l and t hat all non- cont empor aneous covar i ances vani sh: E( e~/ %l ) = 0
f or a :~ ~/. Therefore, E(eje~) = ojfl, whi ch cont ai ns cV(e/) = o//I as a speci al case.
The full covar i ance mat r i x of t he Ln- el ement vect or e is thus:
o111 012I...O1LI
~ ( e ) = 2 1 1 02~I ' "2LI
OLII OL2I . . . OLLI
= Z I , (4. 10)
where N= [o/1 ] is t he cont empor aneous covar i ance mat ri x, i.e. t he covar i ance
mat r i x of [e~l...e~/.] for ot = 1, . . . , n.
Suppose t hat N is non-si ngul ar so t hat N- 1 I is the i nverse of t he mat r i x (4.10)
in vi ew of (4.2). Also, suppose t hat X1, . . . ,Xt. and hence Z have full col umn rank.
Appl i cat i on of t he GLS results (2.13) and (2.14) t o (4.9) and (4.10) t hen yi el ds
(4. 11)
as t he best l i near unbi ased est i mat or of / 3 wi t h t he fol l owi ng covar i ance mat r i x:
= ' / ) z ] ' (4. 12)
I n general , / ~ is superi or t o LS appl i ed to each of the L equat i ons separat el y, but
t here are t wo special cases in whi ch t hese est i mat i on pr ocedur es are identical.
The first case is t hat in whi ch X 1 . . . . . X L are all identical. We can t hen wri t e X
f or each of these mat ri ces so t hat t he obser vat i on mat r i x on t he exogenous
vari abl es in (4.8) and (4.9) t akes t he f or m
Z =
X 0 . . . 0
0 X. . . O
0 O. . . X
= l X. ( 4 . 1 3 )
Ch. 1: Li near Al gebra and Mat r i x Met hods
Thi s implies
Z ' ( Z - ' I ) Z = ( I X' ) ( Z - ' I ) ( I X) = Z ' X ' X
and
19
[ z' ( z- ' I ) z] - ' z' ( z- ' 1) =
= 10 ( x' x) - ' x' .
It is now readi l y verified f r om (4.11) t hat fi consists of L subvectors of t he LS
f or m ( X ' X ) - ~X'yj. The si t uat i on of i dent i cal mat ri ces Xl , . . . ,Xz, occurs relatively
frequent l y in appl i ed economet ri cs; an exampl e is t he reduced f or m (2.4) f or each
of t he L endogenous variables.
The second case in whi ch (4.11) degenerat es i nt o subvect ors equal to LS vect ors
is t hat of uncor r el at ed cont empor aneous di st urbances. Then 2~ is diagonal and it
is easily verified t hat / ~ consists of subvect ors of t he f or m ' - ~ ' (X/iXi) Xjyj. See Thei l
(1971, pp. 311- 312) for t he case in which ~ is bl ock-di agonal .
Not e t hat t he comput at i on of t he j oi nt GLS est i mat or (4.11) requires ,~ to be
known. Thi s is usually not t rue and the unknown ~ is t hen repl aced by the
sampl e moment mat r i x of t he LS residuals [see Zel l ner (1962)]. Thi s approxi ma-
t i on is asympt ot i cal l y (for large n) accept abl e under cert ai n condi t i ons; we shall
come back t o this mat t er in t he openi ng par agr aph of Section 9.
4.3. Vectorization of matrices
I n eq. (2.3) we wrot e Ln equat i ons in mat ri x f or m wi t h par amet er matrices F and
B, each consisting of several columns, whereas in (4.8) and (4.9) we wr ot e Ln
equat i ons in mat r i x f or m with a " l ong" par amet er vect or 13. If Z takes the f or m
(4.13), we can write (4.8) in the equivalent f or m Y= XB + E, where Y, B, and E
are mat ri ces consisting of L col umns of t he f or m yj, flj, and ej. Thus, the el ement s
of t he par amet er vect or 13 are t hen rearranged i nt o t he mat ri x B. On the ot her
hand, there are si t uat i ons in whi ch it is mor e at t ract i ve to work with vect ors
r at her t han mat ri ces t hat consist of several columns. For example, if /~ is an
unbi ased est i mat or of t he par amet er vect or fl wi t h finite second moment s, we
obt ai n the covari ance mat r i x of / ~ by post mul t i pl yi ng/ ~ - 13 by its t ranspose and
t aki ng the expect at i on, but this pr ocedur e does not work when t he paramet ers are
arranged in a mat ri x B which consists of several columns. It is t hen appr opr i at e to
rearrange t he par amet er s in vect or form. This is a mat t er of designing an
appr opr i at e not at i on and evaluating the associated algebra.
Let A = [ a l . . . a q ] be a p x q matrix, a,. being t he i t h col umn of A. We defi ne
vecA = [ a' 1 a'2...aq]', whi ch is a pq-el ement col umn vect or consisting of q
20 H. Theil
subvectors, the first cont ai ni ng the p elements of a ~, the second the p elements of
a2, and so on. It is readily verified t hat vec(A + B) = vecA + vecB, provi ded t hat
A and B are of the same order. Also, if the mat ri x product s A B and B C exist,
vecAB = ( I A ) v e c B = ( B' I ) ve c A,
vec A B C = ( I A B ) v e c C = ( C' A )vecB = ( C ' B ' I ) v e c a .
For proofs and extensions of these results see Dhrymes (1978, ch. 4).
5. Differential demand and supply systems
The differential approach to microeconomic t heory provides interesting compari-
sons with equat i on systems such as (2.3) and (4.9). Let g ( z ) be a vect or of
functions of a vector z; the approach uses the t ot al differential of g(-),
Og
d g = Oz-- ~ d z , (5.1)
and it exploits what is known about O g / a z ' . For example, the t ot al di fferent i al of
Consumer demand is dq = ( 0 q / a M) d M + ( O q / O p ' ) d p . Subst i t ut i on from (3.13)
yields:
which shows t hat the i ncome effect of the price changes is used to defl at e the
change in money i ncome and, similarly, the general subst i t ut i on effect to deflate
the specific effect. Our first objective is to write the system (5.2) in a more
attractive form.
5.1. A di f f er ent i al c ons ume r d e ma n d s y s t e m
We i nt roduce the budget share wi and the margi nal share O~ of good i:
P' q' O, O( p i q i ) (5.3)
Wi - - M ' OM '
and also the Divisia (1925) volume i ndex d( l ogQ) and the Fri sch (1932) price
index d(log P' ):
N N
d ( l o g Q) = 2 w i d ( l o g q i ) , d ( l o g e ' ) = ~ O,d(logp~), (5.4)
i =1 i ~l
Ch. 1: Linear Algebra and Mat ri x Methods 21
whe r e l og ( her e a nd el sewher e) s t ands f or na t ur a l l ogar i t hm. We p r o v e i n t he next
pa r a gr a ph t ha t (5.2) c a n b e wr i t t en i n scal ar f or m as
w~ d( l og q~) = 0 r d( l og Q) + q, ~. 0ij d l og ~ 7 ,
j = l
whe r e d[l og(pj / P' )] is an a bbr e vi a t i on of d ( l o g p / ) - d ( l o g P ' ) , whi l e q~ is t he
r eci pr ocal of t he i nc ome el ast i ci t y of t he mar gi nal ut i l i t y of i ncome:
Ol ogX )--~
* = Ol o g M ' ( 5. 6)
and Oij is an el ement of t he s ymme t r i c N N mat r i x
O) = 7 MP U - 1e, ( 5 . 7 )
wi t h P de f i ne d as t he di agonal ma t r i x wi t h t he pr i ces p ~,... ,PN on t he di agonal .
To ver i f y (5.5) we a p p l y (5.1) tO M = P! / , yi el di ng d M = q ' d p + p ' d q so t hat
d M - q ' d p = Md ( l o g Q ) f ol l ows f r om (5.3) a nd (5.4). Ther ef or e, pr emul t i pl i ca-
t i on of (5.2) b y ( 1 / M ) P gives:
_( Oq' , ] ,
1 p d q = P - ~ d ( l o g Q ) + ~ P U - ' P [ P - ' d p \-0---~ d p ) ( 5 . 8 )
whe r e ~ = P - ~p is a ve c t or of N uni t d e me n t s . The i t h el ement of ( 1 / M ) P d q
equal s ( p ~ / M) d q ~ = wi d( l og q,), whi ch conf i r ms t he l ef t si de of (5.5). The ve c t or
P( Oq/ OM) equal s t he mar gi nal shar e ve c t or 0 = [Oi], t hus conf i r mi ng t he real -
i nc ome t e r m of (5.5). Th e j t h el ement of t he ve c t or i n br a c ke t s in (5.8) equal s
d( l og p / ) - d ( l o g P' ) , whi c h agrees wi t h t he s ubs t i t ut i on t e r m of (5.5). The veri fi ca-
t i o n of (5.5) is c o mp l e t e d b y ( X/ M) PU- 1 P=e o @ [see (5.7)]. No t e t hat @e=
( X / O M) P U - lp = P( Oq / Om) [see (3. 11) a n d (5.6)]. Ther ef or e,
O ~ = 0 , e ' OL = L' 0 = 1 , ( 5 . 9 )
wher e t'O = ~ i O i = 1 f ol l ows f r om (2.16). We c onc l ude f r om Ot = 0 t hat t he 0 i / s
of t he i t h e qua t i on s um t o t he i t h mar gi nal share, a nd f r o m gOL = 1 t hat t he Otj's
of t he ent i r e s ys t e m s um t o 1. The l at t er p r o p e r t y is expr es s ed b y r ef er r i ng t o t he
Oij's as t he normalized price coefficients.
22 H. Theil
5. 2. A c o mp a r i s o n wi t h s i mu l t a n e o u s e q u a t i o n s y s t e ms
The N-equat i on system (5.5) describes t he change in t he demand f or each good,
measured by its cont r i but i on to the Divisia i ndex [see (5.4)], 2 as t he sum of a
real -i ncome component and a subst i t ut i on component . This system ma y be
compar ed with t he L- equat i on system (2.1). Ther e is a di fference i n t hat t he l at t er
system cont ai ns in pri nci pl e mor e t han one endogenous vari abl e in each equat i on,
whereas (5.5) has onl y one such vari abl e if we assume t he d( l ogQ) and all pri ce
changes are exogenous. 3 Yet, the di fferent i al demand system is t r ul y a syst em
because of t he cross-equat i on const rai nt s i mpl i ed by t he symmet r y of t he nor mal -
ized price coefficient mat ri x O.
A more i mpor t ant di fference results f r om the ut i l i t y-maxi mi zi ng t heor y behi nd
(5.5), which implies t hat the coefficients are mor e di rect l y i nt erpret abl e t han the
-{'s and f l ' s of (2.1). Wri t i ng [0 U] = 0 -1 and i nvert i ng (5.7), we obt ai n:
Oi j = epM 02U
X O ( p , qg) 3 ( p j q j ) ' (5. 10)
whi ch shows t hat 0 ij measures (apart f r om ~ M / X which does not involve i and j )
t he change in t he margi nal utility of a dol l ar spent on i caused by an ext ra dol l ar
spent on j . Equivalently, t he normal i zed pri ce coeffi ci ent mat ri x O is i nversel y
pr opor t i onal to t he Hessi an mat ri x of the ut i l i t y f unct i on in expendi t ur e terms.
The rel at i on (5.7) bet ween O and U allows us to anal yze special preference
structures. Suppose t hat the consumer' s tastes can be represent ed by a ut i l i t y
f unct i on whi ch is the sum of N funct i ons, one f or each good. Then t he margi nal
utility of each good is i ndependent of the consumpt i on of all ot her goods, which
we express by referri ng to this case as p r e f e r e n c e i nde pe nde nc e . The Hessi an U is
t hen di agonal and so is O [see (5.7)], while O, = 0 in (5.9) is simplified t o Oi i = 0 i.
Thus, we can write (5.5) under preference i ndependence as
w i d ( l o g qi ) = O l d ( l o g O ) + dp0id(log~-~l ),
( 5 . 1 1 )
whi ch cont ai ns onl y one Fri sch-defl at ed price. The syst em (5.11) f or i = 1 , . . . , N
cont ai ns onl y N unconst r ai ned coefficients, namel y ~ and N- 1 unconst r ai ned
margi nal shares.
The appl i cat i on of di fferent i al demand systems to dat a requires a paramet eri za-
t i on which post ul at es t hat cert ai n coefficients are const ant . Several solutions have
2Note that this way of measuring the change in demand permits the exploitation of the symmetry of
O. When we have d(log q i ) on the left, the coefficient of the Ffisch-deflated price becomes 8i j / wi ,
which is an element of an asymmetric matrix.
3This assumption may be relaxed; see Theil (1975-76, ch. 9-10) for an analysis of endogenous price
changes.
Ch. 1." Linear Algebra and Mat ri x Methods 23
been proposed, but these are beyond t he scope of this chapt er; see the references
quot ed in Sect i on 2.4 above and also, for a furt her compar i son wi t h models of the
t ype (2.1), Thei l and Clements (1980).
5.3. An ext ensi on to t he i nput s of a firm." A singularity probl em
Let t hepi ' s and qi' s be t he prices and quantities of N i nput s whi ch a f i r m' buys to
make a pr oduct , the out put of whi ch is z. Let z = g ( q ) be the fi rm' s pr oduct i on
funct i on, g( - ) bei ng t hree times differentiable. Let t he firm' s objective be to
minimize i nput expendi t ure p~/ s ubj ect to z = g ( q ) f or given out put z and i nput
prices p. Our objective will be t o anal yze whet her this mi ni mum pr obl em yields a
di fferent i al i nput demand system similar t o (5.5).
As in the consumer' s case we const ruct a Lagrangi an funct i on, which now takes
t he f or m p' q - O[ g ( q ) - z]. By equat i ng the derivative of this funct i on with respect
to q to zero we obt ai n a pr opor t i onal i t y of Og/ Oq to p [compare (2.15)]. Thi s
pr opor t i onal i t y and t he pr oduct i on funct i on pr ovi de N + 1 equat i ons in N + 1
unknowns: q and O. Next we di fferent i at e these equat i ons with respect t o z and p ,
and we collect the derivatives in par t i t i oned mat ri x form. The result is similar to
t he mat ri x equat i on (3.8) of consumpt i on t heory, and t he Hessi an U now becomes
t he Hessi an 02g/OqOq ' of the pr oduct i on funct i on. We can t hen proceed as in
(3.9) and following t ext i f O2g/OqOq ' is non-singular, but this is unf or t unat el y not
t r ue when t he fi rm operat es under const ant ret urns t o scale. It is clearly
unat t r act i ve t o make an assumpt i on which excludes this i mpor t ant case. In the
account whi ch follows 4 we solve this pr obl em by formul at i ng the pr oduct i on
f unct i on in l ogari t hmi c form.
log z = h ( q) , (5. 12)
and using t he following N N Hessi an mat ri x:
2h ] ( 5 . 1 3 )
H = 0 ( l o g qi) O ( l o g q j ) "
5.4. A di f f erent i al i nput de mand syst em
The mi ni mum o f p ' q subject t o (5.12) for given z a n d p will be a funct i on of z and
p. We write C( z , p ) for this mi ni mum: the cost of pr oduci ng out put z at t he i nput
4Derivations are omitted; the procedure is identical to t hat which is outlined above except t hat it
systematically uses logarithms of output, inputs, and i nput prices. See Laitinen (1980), Lai t i nen and
Theil (1978), and Theil (1977, 1980).
24 H. Theil
pr i ces p. We def i ne
01ogC 1 1 021ogC
- - = 1 + ( 5 . 1 4 )
"{ 01ogz ' ~ ~2 0 ( l o g z)2 '
so t hat "/ is t he o u t p u t el ast i ci t y of cos t a nd ~ < 1 ( > 1) whe n t hi s el as t i ci t y
i ncr eases ( decr eas es ) wi t h i ncr easi ng out put ; t hus, ~p is a c ur va t ur e me a s ur e o f t he
l ogar i t hmi c cos t f unct i on. I t c a n be s hown t hat t he i n p u t d e ma n d e qua t i ons ma y
b e wr i t t en as
N
f i d ( l o g q i ) = ~ ' O i d ( l o g z ) - q ~ ~ O i j d ( l o g ~ 7 , ) , ( 5. 15)
j = l
whi ch s houl d be c o mp a r e d wi t h (5.5). I n (5.15), f i is t he f a c t or shar e of i nput i (its
s har e i n t ot al cost ) a nd 0 i is i t s mar gi nal shar e ( t he shar e i n mar gi nal cost ),
f~ = P ' q' O, - O( p, q, ) / Oz ( 5 . 1 6 )
C ' a C / O z '
whi ch is t he i nput ver si on of (5.3). The Fr i s ch pr i ce i ndex on t he f ar ri ght i n (5. 15)
is as s hown i n (5.4) b u t wi t h 0 i de f i ne d i n (5.16). The coef f i ci ent 0ij i n (5. 15) is t he
( i , j ) t h el ement of t he s ymme t r i c ma t r i x
O= + F ( F - y H) - I F ,
( 5. 17)
whe r e H is gi ven i n (5. 13) a nd F is t he di agonal ma t r i x wi t h t he f a c t or shar es
f l . . . . . f N o n t he di agonal . Thi s 0 sat i sf i es (5.9) wi t h 0 = [Oi] de f i ne d i n (5.16).
A f i r m is cal l ed i n p u t i n d e p e n d e n t whe n t he el ast i ci t y of its o u t p u t wi t h r e s pe c t
t o each i nput is i n d e p e n d e n t of all ot he r i nput s . I t f ol l ows f r om (5. 12) a nd (5. 13)
t ha t H is t hen di agonal ; hence, O is al s o di a gona l [see (5.17)] a nd Ot = 0 b e c o me s
O. = 0 i so t hat we c a n s i mpl i f y (5. 15) t o
f i d ( l o g q i ) = " [ O i d ( l g z ) - ~Oi d( l og ~ ) , ( 5. 18)
whi ch is t o be c o mp a r e d wi t h t he c o n s u me r ' s e qua t i on (5. 11) u n d e r pr e f e r e nc e
i nde pe nde nc e . The C o b b - Do u g l a s t e c hnol ogy is a speci al case of i nput i nde pe n-
de nc e wi t h H = 0, i mpl yi ng t hat F ( F - y H ) - I F i n (5. 17) equal s t he di agonal
ma t r i x F. Si nce C o b b - D o u g l a s ma y ha ve c ons t a nt r e t ur ns t o scale, t hi s i l l ust r at es
t ha t t he l ogar i t hmi c f or mul a t i on s ucces s f ul l y avoi ds t he s i ngul ar i t y p r o b l e m
me n t i o n e d i n t he pr evi ous s ubs ect i on.
Ch. 1: Linear Algebra and Matrix Methods 25
5.5. Al l ocat i on s y s t e ms
Summat i on of (5.5) over i yields the i dent i t y d ( l o g Q) = d(l ogQ), which means
t hat (5.5) is an allocation s y s t e m in the sense t hat it describes how the change in
t ot al expenditure is allocated to the N goods, given the changes in real i ncome
and relative prices. To verify this identity, we write (5.5) for i = 1 . . . . , N in mat ri x
form as
WI = ( ( W ~ ) O + ~, O( I - ,O')~r, (5.19)
where W is the di agonal mat ri x with wl , . . . , w~ on the diagonal and ~r = [d(log Pi)]
and ~ = [d(log qi)] are the vectors logarithmic price and quant i t y changes so t hat
d(log Q) = ( Wg, d(log P ' ) = 0'~r. The proof is compl et ed by premultiplying (5.19)
by ( , which yields ( W~ = ( Wx in view of (5.9). Not e t hat the subst i t ut i on terms
of the N demand equations have zero sum.
The i nput demand system (5.15) is not an allocation system because the fi rm
does not t ake total i nput expenditure as given; rather, it minimizes this expendi-
t ure for given out put z and given i nput prices p. Summat i on of (5.15) over i
yields:
d(log Q) = v d(log z) , (5.20)
where d(log Q) = ~ , i f i d ( l o g qi) = t'FK is the Divisia i nput volume index. Substitu-
t i on of (5.20) i nt o (5.15) yields:
f i d ( l o g q i ) = O i d ( l o g Q ) - + ~ Oi j d ( l o g ~ 7 , ) . (5.21)
j =l
We can interpret (5.20) as specifying the aggregate i nput change which is required
to produce the given change in out put , and (5.21) as an allocation system for the
individual i nput s given the aggregate i nput change and the changes in the relative
i nput prices. It follows from (5.9) t hat we can write (5.19) and (5.21) for each i as
w, = + , o( t- ( 5 . 2 2 )
= + o ( x - ( 5 . 2 3 )
which shows t hat the normalized price coefficient mat ri x O and the scalars q~ and
are the onl y coefficients in the t wo allocation systems.
5. 6. Ex t e n s i o n s
Let the firm adjust out put z by maximizing its profi t under competitive condi-
tions, the price y of the product being exogenous from the firm' s point of view.
26 H. Theil
Then mar gi nal cost OC/ Oz equals y, while 0 i of (5.16) equal s O( p i q i ) / O( y z ) : the
addi t i onal expendi t ur e on i nput i resul t i ng f r om an ext ra dol l ar of out put
revenue. Not e t hat this is much closer to t he cons umer ' s 0~ defi ni t i on (5.3) t han is
(5.16).
I f the f i r m sells m pr oduct s wi t h out put s z 1, . . . , z m at exogenous pri ces y~ . . . . . Ym,
t ot al revenue equals R = Y'.rYrZr and gr = Y r Z J R is t he revenue share of pr oduct
r, while
m
d ( l o g Z ) = ~ g r d ( l o g z ~) (5. 24)
r = l
is t he Di vi si a out put vol ume i ndex of the mul t i pr oduct firm. Ther e are now m
mar gi nal costs, OC/ Oz r f or r = 1 . . . . . m, and each i nput has m mar gi nal shares: 0, r
defi ned as O( pi qi ) / OZr di vi ded by OC/Oz, . , whi ch becomes 0f = O( pi qi ) / O( Y~Zr )
under pr of i t maxi mi zat i on. Mul t i pr oduct i nput demand equat i ons can be for-
mul at ed so t hat t he subst i t ut i on t er m in (5.15) is unchanged, but t he out put t er m
becomes
m
y ~. Or gr d( l ogz r ) , (5. 25)
r ~ l
whi ch shows t hat i nput i can be of ei t her mor e or less i mpor t ance for pr oduct r
t han for pr oduct s dependi ng on t he val ues of 0i ~ and 07.
Maxi mi zi ng pr of i t by adj ust i ng out put s yields an out put suppl y syst em whi ch
will now be bri efl y descri bed. The r t h suppl y equat i on is
m , ( Y s ) ( 5 . 2 6 )
gr d( l og Zr) = ~* F. 0L d log - ~ ,
s =l
whi ch descri bes t he change 5 in the suppl y of pr oduct r in t er ms of all out put pri ce
changes, each defl at ed by t he cor r espondi ng Fri sch i nput pri ce i ndex:
N
d ( l o g P ' r ) = Z O[ d ( l o g p i ) . (5. 27)
i = l
Ast eri sks are added to the coeffi ci ent s of (5.26) in or der t o di st i ngui sh out put
suppl y f r om i nput demand. The coeffi ci ent +* is positive, while 0~* is a nor mal -
i zed pri ce coeffi ci ent defi ned as
1
* = c r* , ( 5 . 2 8 )
0;.~ + , R y r y~
5This change is measured by the contribution of product r to the Divisia output volume index
(5.24). Note that this is similar to the left variables in (5.5) and (5.15).
Ch. 1: Linear Algebra and Matrix Methods 27
where c rs is an element of the inverse of the symmetric m m matrix
[ 02C/ OZr Oz s]. The similarity between (5.28) and (5.7) should be noted; we shall
consider this matter further in Section 6. A multiproduct firm is called out put
i ndependent when its cost function is the sum of m functions, one for each
product. 6 Then [ 02C/ Oz r Oz,] and [0;*] are diagonal [see (5.28)] so that the change
in the supply of each product depends only on the change in its own deflated
price [see (5.26)]. Not e the similarity to preference and input independence [see
(5.11) and (5.18)].
6. De f i n i t e and s e mi de f i ni t e s quare mat r i c e s
The expression x ' A x is a quadratic form in the vector x. We met several examples
in earlier sections: the second-order term in the Taylor expansion (2.6), e ' Me in
the residual sum of squares (2.12), the expression in the exponent in the normal
density function (3.3), the d e n o mi n a t o r p ' U- l p in (3.9), and t' Ot in (5.9). A more
systematic analysis of quadratic forms is in order.
6.1. Covari ance mat ri ces and Gaus s - Mar k ov f ur t her consi dered
Let r be a random vector with expectation Er and covariance matrix Z. Let w' r be
a linear function of r with non-stochastic weight vector w so that E (w' r) = w' Er .
The variance of w' r is the expectation of
[ w ' ( r - 2 = w ' ( r -
SO that var w' r = w ' ~ ( r ) w = w' , ~w. Thus, the variance of any linear function of r
equals a quadratic form with the covariance matrix of r as matrix.
If the quadratic form x ' A x is positive for any x ~ 0, A is said to be posi t i ve
def i ni t e. An example is a diagonal matrix A with positive diagonal elements. If
x ' A x >~ 0 for any x, A is called posi t i ve semi def i ni t e. The covariance matrix ~ of
any random vector is always positive semidefinite because we just proved that
w ' Z w is the variance of a linear function and variances are non-negative. This
covariance matrix is positive semidefinite but not positive definite if w ' Z w = 0
holds for some w ~ 0, i.e. if there exists a non-stochastic linear function of the
random vector. For example, consider the input allocation system (5.23) with a
6Hall (1973) has shown that the additivity of the cost function in the m outputs is a necessary and
sufficient condition in order that the multiproduct firm can be broken up into m single-product firms
in the following way: when the latter firms independently maximize profit by adjusting output, they
use the same aggregate level of each input and produce the same level of output as the multiproduct
firm.
28
di st urbance vect or e added:
H. Theil
e ~ = ( , ' r ~ ) o , - +o(t - , , ' o ) , ~ + ~ . (6. 1)
Premul t i pl i cat i on by L' and use of (5.9) yields t'FK = t'FK + t'e, or t'e = 0, whi ch
means t hat t he di st urbances of the N equat i ons sum to zero with uni t probabi l i t y.
Thi s pr oper t y results f r om t he al l ocat i on char act er of t he system (6.1).
We r et ur n to t he st andard linear model descri bed in t he discussion fol l owi ng
eq. (2.7). The Ga us s - Ma r kov t heor em states t hat the LS est i mat or b in (2.8) is
best linear unbi ased in the following sense: any ot her est i mat or / ~ of fl whi ch is
also linear in y and unbi ased has t he pr oper t y t hat ~( / ~) - CV( b) is a posi t i ve
semidefinite matrix. That is,
w' [ C~- ( f l ) - ~V( b) ] w>~O for any w,
or w' C~(fl)w >1 w'CV(b)w. Since bot h sides of this i nequal i t y are t he vari ance of an
est i mat or of w' fl, the i mpl i cat i on is t hat wi t hi n t he class of l i near unbi ased
est i mat ors LS provi des t he est i mat or of any l i near f unct i on of fl wi t h the smallest
possible variance. This is a st ronger result t han t he st at ement in t he discussion
following eq. (2.10); t hat st at ement is conf i ned to t he est i mat i on of dement s
r a t he r t han general linear funct i ons of ft.
To pr ove the Ga us s - Ma r kov t heor em we use the l i neari t y of / ~ in y t o wri t e
= By, where B is a K n mat ri x consisting of non-st ochast i c elements. We
defi ne C = B - ( X' X) - i x ' so t hat / ~ = By can be wri t t en as
[ c +( x ' x ) - ' x ' ] y = [ c + ( x ' x ) - ' x ' ] ( x a + ~)
= ( C X + I ) f l + [ C + ( X ' X ) l X ' ]e.
The expectatio~a of / ~ is thus ( C X + I ) f l , which must be i dent i cal l y equal t o fl in
or der t hat the est i mat or be unbi ased. Therefore, C X = O and /~ =f l + [ C+
( X ' X ) - l x ' ] e so t hat ~V(/~) equals
[c +( x,x) - ' x' ] v( ~) [ c +(x' x)- ' x' ]'
: o~cc'+ o~(x' x)-' + ,,~cx(x'x)='+ o~(x' x)-' x' c' .
It t hus follows f r om (2.10) and C X = 0 t hat ~ ( / ~ ) - C~ ( b ) = o 2 C C ,, which is a
positive semi defi ni t e mat ri x because o 2 w' CC' w = ( o C' w) ' ( o C' w) is t he non-nega-
tive squared length of t he vect or oC' w.
Ch. 1: Linear Algebra and Matrix Methods
6.2. M a x i m a a n d mi n i ma
29
The mat ri x A is called n e g a t i v e s e mi d e f i n i t e if x ' A x <~ 0 hol ds for any x and
n e g a t i v e de f i ni t e if x ' A x < 0 hol ds for any x ~ 0. I f A is positive definite, - A is
negative defi ni t e (similarly for semidefiniteness). I f A is positive (negative)
definite, all di agonal elements of A are positive (negative). This may be verified
by consi deri ng x ' A x wi t h x specified as a col umn of t he uni t mat ri x of ap-
pr opr i at e order. If A is positive (negative) definite, A is non-si ngul ar because
singularity woul d i mpl y the exi st ence of an x ~ 0 so t hat A x = 0, whi ch is
cont r adi ct ed by x ' A x > 0 ( < 0). I f A is symmet ri c posi t i ve (negative) definite, so
is A - 1, which is verified by consi deri ng x ' A x with x = A - ly for y = 0.
For the f unct i on g( - ) of (2.6) to have a st at i onary val ue at z = ~ it is necessary
and sufficient t hat the gradi ent Og / Oz at this poi nt be zero. For this st at i onary
val ue to be a local maxi mum (mi ni mum) it is sufficient t hat t he Hessi an mat ri x
OZg/ Oz Oz ' at this poi nt be negative (positive) definite. We can appl y this t o the
suppl y equat i on (5.26) whi ch is obt ai ned by adjusting the out put vect or z so as to
maxi mi ze profi t . We write pr of i t as y ' z - C, whe r e y is t he out put pri ce vect or and
C = cost. The gradi ent of pr of i t as a f unct i on of z is y - OC/ Oz ( y is i ndepen-
dent of z because y is exogenous by assumpt i on) and t he Hessi an mat ri x is
- - 02C/ Oz OZ t so t hat a positive defi ni t e mat r i x 3 2 C/ Oz Oz ' is a sufficient condi-
t i on f or maxi mum profi t . Since qJ* and R in (5.28) are positive, t he mat ri x [0"] of
t he suppl y syst em (5.26) is positive definite. The di agonal el ement s of this mat r i x
are t herefore positive so t hat an increase in the pri ce of a pr oduct raises its
supply.
Similarly, a sufficient condi t i ons for maxi mum utility is t hat t he Hessi an U be
negative definite, i mpl yi ng q~ < 0 [see (3.9) and (5.6)], and a sufficient condi t i on
f or mi ni mum cost is t hat F - y H in (5.17) be positive definite. The result is t hat
[0ij ] in bot h (5.5) and (5.15) is also positive definite. Since q~ and - q, in these
equat i ons are negative, an increase i n t he Fri sch-defl at ed pri ce of any good
(consumer good or i nput ) reduces the demand for this good. For t wo goods, i and
j , a positive (negative) 0ij = 0ji implies t han an i ncrease in t he Fri sch-defl at ed
pri ce of either good reduces (raises) t he demand for t he other; t he two goods are
t hen said to be s peci f i c c o mp l e me n t s ( s u b s t i t u t e s ) . Under preference or i nput
i ndependence no good is a specific subst i t ut e or compl ement of any ot her good
[see (5.11) and (5.18)]. The di st i nct i on bet ween specific substitutes and compl e-
ment s is f r om Hout hakker (1960); he pr oposed it f or consumer goods, but it can
be equally appl i ed to a firm' s inputs and also to out put s based on the sign of
0* = 0* in (5.26).
The assumpt i on of a defi ni t e U or F - H is mor e t han strictly necessary. In
the consumer' s case, when utility is maxi mi zed subject to the budget const rai nt
p ' q = M, it is sufficient to assume c o n s t r a i n e d negative definiteness, i.e. x ' U x < 0
for all x ~ 0 which satisfy p ' x = 0. It is easy to const ruct examples of an i ndefi ni t e
30 H. Theil
or singular semi defi ni t e mat r i x U whi ch sat i sfy this condi t i on. Defi ni t eness
obvi ousl y i mpl i es const r ai ned definiteness; we shall assume t hat U and F - 7 / /
sat i sfy t he st ronger condi t i ons so t hat the above anal ysi s hol ds true.
6.3. Block-diagonal definite matrices
I f a mat r i x is bot h defi ni t e and bl ock-di agonal , t he rel evant pr i nci pal submat r i ces
are also definite. For exampl e, if N of (3.4) is posi t i ve definite, t hen x ~ , l x 1 +
x'2~,2x 2 > 0 if ei t her x I ~ 0 or x 2 :~ 0, whi ch woul d be vi ol at ed if ei t her N1 or N2
were not definite.
Anot her exampl e is t hat of a l ogar i t hmi c pr oduct i on funct i on (5.12) when the
i nput s can b e gr ouped i nt o i nput gr oups s o t hat t he elasticity of out put wi t h
respect to each i nput is i ndependent of all i nput s bel ongi ng to di fferent groups.
Then H of (5.13) is bl ock- di agonal and so is @ [see (5.17)]. Thus, if i bel ongs to
i nput gr oup Sg ( g = 1,2 . . . . ), the s ummat i on over j in t he subst i t ut i on t er m of
(5.15) can be confi ned t o j ~ Sg; equi val ent l y, no i nput is t hen a specific subst i t ut e
or compl ement of any i nput bel ongi ng to a di fferent group. Also, s ummat i on of
t he i nput de ma nd equat i ons over all i nput s of a gr oup yields a composi t e de ma nd
equat i on for the i nput gr oup whi ch t akes a si mi l ar f or m, while an appr opr i at e
combi nat i on of this composi t e equat i on wi t h a de ma nd equat i on f or an i ndi vi dual
i nput yields a condi t i onal demand equat i on f or the i nput wi t hi n t hei r group.
These devel opment s can also be appl i ed to out put s and consumer goods, but t hey
are beyond the scope of this chapt er.
7. Diagonalizations
7.1. The standard diagonalization of a square matrix
For some n n mat r i x A we seek a vect or x so t hat A x equals a scal ar mul t i pl e 2t
of x. Thi s is trivially satisfied by x = 0, so we i mpos e x ' x = 1 i mpl yi ng x ~ 0. Since
A x = Xx is equi val ent to ( A - X I ) x = 0, we t hus have
( A - X l ) x =O, x ' x =l , (7. 1)
so t hat A - XI is singular. Thi s i mpl i es a zero det er mi nant value,
IA - XI[ = 0, (7. 2)
whi ch is known as the characteristic equation of A. For exampl e, if A is di agonal
Ch. 1: Linear Algebra and Mat ri x Methods 31
wi t h d I . . . . . d n on the di agonal , (7.2) st at es t hat t he pr oduct of d i - ~ o v e r i
vani shes so t hat each d~ is a sol ut i on of the charact eri st i c equat i on. Mor e
generally, t he charact eri st i c equat i on of an n n mat r i x A is a pol ynomi al of
degree n and t hus yields n sol ut i ons ~ . . . . ,~n- These ?~i's are the l at e nt root s of A;
t he pr oduct of t he ?~' s equal s the det er mi nant of A and the sum of the ?~' s equals
t he t race of A. A vect or xi whi ch satisfies Ax ~ = ?~ix~ and x~x~ = 1 is called a
char act er i s t i c v e c t o r of A cor r espondi ng to r oot 2~.
Even if A consi st s of real el ement s, its r oot s need not be real, but these root s
are all real when A is a real symmet r i c mat ri x. For suppose, t o the cont r ar y, t hat ?~
is a compl ex r oot and x + i y is a charact eri st i c vect or cor r espondi ng t o this ?~,
wher e i = v rZ- 1. Then A ( x + i y ) = 2~ ( x + i y) , whi ch we pr emul t i pl y by ( x - i y) ' :
x ' A x + y ' A y + i ( x ' A y - y ' A x ) = X ( x ' x + y ' y ) . (7. 3)
But x ' A y = y ' A x if A is symmet r i c, so t hat (7.3) shows t hat ?, is t he rat i o of t wo
real number s, x ' A x + y ' A y and x ' x + y ' y . Root s of asymmet r i c mat ri ces are
consi dered at t he end of this section.
Let X i and ?~j be t wo di fferent root s (2,, ~ ?~j) of a symmet r i c mat r i x A and let
x i and x j be cor r espondi ng charact eri st i c vectors. We pr emul t i pl y A x i = Xi xg by xj
and A x j = ?~jxj by x; and subt ract :
x ' j a x , - x ; A x j = ( X , - X j ) x ; x j .
Since the l eft side vani shes for a symmet r i c mat r i x A, we mus t have x ~x j = 0
because ~,~ * Xj. Thi s pr oves t hat charact eri st i c vect ors of a symmet r i c mat r i x are
or t hogonal when t hey cor r espond to di fferent root s. When all r oot s of a symmet -
ric n n mat r i x A are distinct, we t hus have x ~x j = 6ij for all (i, j ) . Thi s is
equi val ent to
x ' x =l , wh e r e X= I x, x 2 . . . x , ] . (7. 4)
Also,
A X = [ A x I . . . A X n ] : [ ~ l X l . . . ~ k n X n ] ,
o r
AX = X a , ( 7 . 5 )
where A is t he di agonal mat r i x wi t h X 1 . . . . . X, on the di agonal . Pr emul t i pl i cat i on
of (7.5) by X' yi el ds X ' A X = X ' X A , or
X ' A X = A (7. 6)
32 H. Theil
in view of (7.4). Therefore, when we post mul t i pl y a symmet r i c mat r i x A by a
mat r i x X consi st i ng of charact eri st i c vect ors of A and pr emul t i pl y by X' , we
obt ai n the di agonal mat r i x cont ai ni ng the l at ent r oot s of A. Thi s doubl e mul t i pl i -
cat i on amount s to a di agonal i z at i on of A. Also, post mul t i pl i cat i on of (7.5) by X'
yields A X X ' = X A X ' and hence, since (7.4) i mpl i es X' = X -1 or X X ' = I ,
?/
A = X A X ' = ~, ) ~x ~x ; . (7. 7)
i =1
I n the previ ous par agr aph we assumed t hat t he ~ ' s are distinct, but it ma y be
shown t hat for any symmet r i c A t here exists an X whi ch satisfies (7. 4)-(7. 7), t he
col umns of X bei ng charact eri st i c vect ors of A and A bei ng di agonal wi t h t he
l at ent root s of A on the di agonal . The onl y di fference is t hat in t he case of
mul t i pl e root s ()~ = )~j for i :~ j ) the associ at ed charact eri st i c vect ors (x~ and x j )
are not unique. Not e t hat even when all )~'s are distinct, each xi ma y be ar bi t r ar i l y
mul t i pl i ed by - 1 because this affect s nei t her Ax ~ = ) ~x~ nor x ~ x j = 0 f or any
( i , j ) ; however, this sign i ndet er mi nacy will be i rrel evant for our purposes.
Z2. S p e c i a l cas es
Let A be square and pr emul t i pl y A x = ~ x by A t o obt ai n A 2 x = ~ A x = ~2x. Thi s
shows t hat A 2 has the same charact eri st i c vect ors as A and l at ent root s equal to
t he squares of t hose of A. I n part i cul ar, if a mat r i x is symmet r i c i dempot ent , such
as M of (2.11), all l at ent root s are 0 or 1 because these are the onl y real number s
t hat do not change when squared. For a symmet r i c non- si ngul ar A, pr emul t i pl y
A x = ~ x by (?, A) t to obt ai n A - ix = ( 1 / ~ ) x . Thus, A -~ has t he s ame char act er -
istic vect ors as t hose of A and l at ent r oot s equal to the reci procal s of t hose of A.
I f t he symmet r i c n X n mat r i x A is si ngul ar and has r ank r, (7.2) is sat i sfi ed by
= 0 and this zero root has mul t i pl i ci t y n - r. I t thus follows f r om (7.7) t hat A
can t hen be wri t t en as the sum of r mat ri ces of uni t rank, each of t he f or m ~.ix~x~,
wi t h ?~i :~ 0.
Premul t i pl i cat i on of (7.7) by y ' and post mul t i pl i cat i on by y yields y ' A y =
~ ?~ic~, wi t h c~ = y ' x i. Since y ' A y is posi t i ve (negat i ve) for any y ~ 0 if A is
posi t i ve (negat i ve) definite, this shows t hat all l at ent r oot s of a symmet r i c posi t i ve
(negat i ve) defi ni t e mat r i x are posi t i ve (negative). Similarly, all l at ent root s of a
symmet r i c posi t i ve (negative) semi defi ni t e mat r i x are non- negat i ve (non-posi t i ve).
Let A m be a symmet r i c m x m mat r i x wi t h r oot s ?~ . . . . . ?~,, and charact eri st i c
vect ors x~ . . . . . xm; let B n be a symmet r i c n x n mat r i x wi t h root s #~ . . . . ,/~, and
charact eri st i c vect ors y~ . . . . . Yn. Hence, A m B n is of or der mn x mn and has mn
l at ent root s and charact eri st i c vect ors. We use A m x ~ = ?~x~ and Bn y j = l ~j yj in
( AmB, , ) ( x , ~) = ( Amx, ) ( n o y j ) = ( ~ , x ~ ) ( ~ j y j ) = ~ j ( x ~ ~ j ) ,
Ch. 1: Linear AIgebra and Matrix Methods 33
whi ch shows t hat x i y j is a characteristic vect or of A m n n correspondi ng to r oot
Xi/~ j. It is easi l y verified t hat these charact eri st i c vect ors f or m an ort hogonal
mat r i x of or der mn ran:
( x l y l ) ' ( x l y l ) = ( X ' l y ~ ) ( x , y l ) = ( X ' l X , ) ( y ~ y , ) = 1,
( x , e y l ) ' ( x 2 Y l ) = ( x ~ x 2 ) ( y~ Y l ) = 0 .
Since the det er mi nant of AmB ~ equals the pr oduct of the roots, we have
[h,.g,I = [ I l q ~ , , t ~ j = l q = I A m l I g . I .
. =
i =l j =l t 1 j
It may similarly be verified t hat t he r ank (trace) of A m B n equals t he pr oduct of
t he ranks (traces) of A m and B, .
7.3. Aitken' s theorem
Any symmet ri c positive defi ni t e mat r i x A can be wri t t en as A = QQ' , where Q is
some non-si ngul ar matrix. For example, we can use (7.7) and specify Q = XA 1/2,
where A ~/2 is t he di agonal mat ri x which cont ai ns the positive square root s of t he
l at ent root s of A on t he diagonal. Since t he root s of A are all positive, A 1/2 is
non-singular; X is non-si ngul ar in view of (7.4); t herefore, Q = XA 1/2 is also
non-singular.
Consi der in part i cul ar t he di st urbance covari ance mat r i x oZv in the discussion
precedi ng eq. (2.13). Since o 2 > 0 and V is non-si ngul ar by assumpt i on, this
covari ance mat ri x is symmet ri c positive definite. Therefore, V- 1 is also symmet ri c
positive defi ni t e and we can write V - l = QQ' for some non-si ngul ar Q. We
premul t i pl y (2.7) by Q' :
Q' y = ( Q' X) f l + Q'e. (7. 8)
The di st urbance vect or Q' e has zero expect at i on and a covari ance mat ri x equal to
0 2 Q , V Q = 0 2 Q, ( Q Q , ) - I Q = 0 2 Q, ( Q , ) - I Q_ 1Q = 021,
so t hat the st andar d l i near model and t he Ga us s - Ma r kov t heor em are now
applicable. Thus, we est i mat e/ 3 by appl yi ng LS to (7.8):
[ ( Q' X ) ' Q' X ] - ' ( Q' X ) ' Q' y = ( X' QQ' X) ' X' QQ' y = ( x ' v I x ) - l x ' v - l y ,
whi ch is t he GLS est i mat or (2.13). The covari ance mat ri x (2.14) is also easily
verified.
34 H. Theil
7.4. The Cholesky decomposition
The di agonal i zat i on (7.6) uses an or t hogonal mat ri x X [see (7.4)], but it is also
possible t o use a t ri angul ar matrix. For exampl e, consi der a di agonal mat r i x D
and an upper t ri angul ar mat ri x C with units in t he diagonal,
1 c12 c13 ]
C= 0 1 c a , D=
0 0
d 1 0 0 [
0 d 2 0
J
0 0 d 3 '
yielding
C' DC =
dl dlC l2 dlCl3 ]
d~cl2 dlC~2 + d 2 dl C12C13 + d2c23 .
d l C l 3 dlC12C13 + d 2 c 2 3 dl C23 + d2c23 + d 3
It is readi l y verified t hat any 3 3 symmet ri c positive defi ni t e mat r i x A = [ai j ]
can be uni quel y wri t t en as C' DC (d~ = a l l , C~2 =al 2/ a~l , etc.). Thi s is t he
so-called Cholesky decomposition of a mat ri x; f or appl i cat i ons to demand analysis
see Bart en and Geyskens (1975) and Thei l and Lai t i nen (1979). Also, not e t hat
O = ( C' ) - 1AC- 1 and t hat C- 1 is upper t ri angul ar with uni t s in t he diagonal.
7.5. Vectors written as diagonal matrices
On many occasions we want to write a vect or in t he f or m of a di agonal mat ri x.
An exampl e is t he price vect or p whi ch occurs as a di agonal mat r i x P in (5.8). An
al t ernat i ve not at i on i s/ $, with the hat i ndi cat i ng t hat t he vect or has become a
di agonal matrix. However, such not at i ons are awkward when t he vect or whi ch we
want to write as a diagonal mat ri x is itself t he pr oduct of one or several mat ri ces
and a vector. For example, in Section 8 we shall meet a nonsi ngul ar N N mat r i x
X and the vect or X- 1 ~ . We write this vect or in di agonal mat ri x f or m as
( X - lt) z a =
E j x is 0 . . . o
o E j x 2: . . . 0
0 0 . . . E j x Nj
(7.9)
where x ij is an el ement of X- 1 and all summat i ons are ove r j = 1 . . . . . N. It is easily
Ch. 1: Linear Algebra and Matrix Methods
veri fi ed t hat
( X - ' , ) ~ , =X - ' , , , ' ( X- 1,)~ = , ' ( X' ) - '
35
(7.10)
7.6. A si mul t aneous di agonaf i zat i on o f two square mat ri ces
We ext end (7.1) t o
(A-Xn)x=O, x'nx=l, (711)
where A and B are symmet r i c n n mat ri ces, B bei ng posi t i ve definite. Thus, B-
is symmet r i c posi t i ve defi ni t e so t hat B- ~ = QQ' f or some non-si ngul ar Q. I t is
easi l y seen t hat (7.11) is equi val ent to
( Q' A Q- X I ) y =O, y ' y =l , y =Q- l x . (7. 12)
Thi s shows t hat (7.11) can be reduced t o (7.1) wi t h A i nt er pr et ed as Q' A Q. I f A is
symmet r i c, so is Q' AQ. Therefore, all results for symmet r i c mat ri ces descri bed
earl i er in this sect i on are appl i cabl e. I n part i cul ar, (7.11) has n solutions, )~1 . . . . . 7~,
and x I . . . . . x, , t he x~' s bei ng uni que when t he )~i's are distinct. Fr om y ' y / = 6+j
and y~ = Q- lx~ we have x~Bxj = 8;/ and hence X ' B X = I , where X is t he n n
mat r i x wi t h xi as t he i t h col umn. We wri t e ( A - ) ~B) x = 0 as A x i = )~+Bx~ f or t he
i t h sol ut i on and as A X = B X A f or all sol ut i ons j oi nt l y, where A is di agonal wi t h
)~ . . . . . )~, on t he di agonal . Pr emul t i pl i cat i on of A X =B X A by X' t hen yields
X ' A X = X ' B X A = A. Therefore,
X ' A X = A , X ' B X = I , (7.13)
whi ch shows t hat bot h mat ri ces are si mul t aneousl y di agonal i zed, A bei ng t rans-
f or med i nt o t he l at ent r oot mat r i x A, and B i nt o t he uni t mat ri x.
I t is not ewor t hy t hat (7.11) can be i nt er pr et ed i n t er ms of a const r ai ned
ext r emum pr obl em. Let us seek t he ma xi mum of t he quadr at i c f or m x ' A x for
var i at i ons in x subj ect t o x ' B x = 1. So we const r uct t he Lagr angi an f unct i on
x ' A x - I ~ ( x ' B x - 1), whi ch we di fferent i at e wi t h respect t o x, yi el di ng 2 A x -
2 ~ Bx . By equat i ng this deri vat i ve t o zero we obt ai n A x = I~Bx, whi ch shows t hat
/~ mus t be a r oot X i of (7.11). Next , we pr emul t i pl y A x = i. tBx by x' , whi ch gives
x ' A x = I ~x' Bx =/ ~ and shows t hat t he l argest r oot )~t is the ma xi mum of x ' A x
subj ect to x ' B x = 1. Similarly, the smal l est r oot is t he mi ni mum of x ' A x subj ect t o
x ' B x = 1, and all n root s are st at i onar y val ues of x ' A x subj ect to x ' B x = 1.
36
7. 7. Lat ent roots of an asymmet ri c mat r i x
H. Theil
Some or all l at ent r oot s of an a s ymme t r i c s quar e mat r i x A ma y be compl ex. I f
(7. 2) yi el ds c ompl e x r oot s, t hey oc c ur i n c onj uga t e c ompl e x pai r s o f t he f o r m
a _+ bi. Th e abs ol ut e val ue of such a r oot is def i ned as ~ + b 2 , whi c h equal s [a[
if b = 0, i.e. if t he r oot is real. I f A is as ymmet r i c, t he l at ent r oot s of A and A' are
still t he s ame but a char act er i st i c vect or of A ne e d not be a char act er i s t i c ve c t or of
A' . I f A is a s ymme t r i c and has mul t i pl e r oot s, it ma y have f ewer char act er i s t i c
vect or s t ha n t he n u mb e r of its r ows and col umns . Fo r exampl e,
is an a s ymme t r i c 2 x 2 mat r i x wi t h a doubl e uni t r oot , but i t has onl y one
char act er i s t i c vect or , [1 0]' . A f ur t her anal ysi s of this subj ect i nvol ves t he J o r d a n
c a noni c a l f or m, whi ch is b e y o n d t he s cope of t hi s chapt er ; see Bel l man (1960, ch.
11).
La t e nt r oot s of a s ymme t r i c mat r i ces pl a y a r ol e i n t he st abi l i t y anal ysi s of
d y n a mi c equat i ons syst ems. Cons i de r t he r e duc e d f o r m
y t =a+Ay t l +A*x t +ut ,
( 7. 14)
wher e Yt is an L- el ement obs er vat i on vect or on e ndoge nous var i abl es at t i me t, a is
a vect or of c ons t a nt t er ms, A i s a s quar e coef f i ci ent mat r i x, A* is an L K
coef f i ci ent mat r i x, x t is a K- el ement obs e r va t i on vect or o n e xoge nous var i abl es at
t, and u~ is a di s t ur bance vect or . Al t h o u g h A is square, t her e is n o r eas on wh y it
s houl d be s ymmet r i c so t ha t its l at ent r oot s ma y i ncl ude conj ugat e c ompl e x pai rs.
I n t he next p a r a g r a p h we shal l be i nt er est ed i n t he l i mi t of A' as s - , ~ . Recal l
t h a t .4 2 has r oot s equal t o t he s quar es of t hos e of A; this al so hol ds f or t he
c ompl e x r oot s of an a s ymme t r i c A. Ther ef or e, A' has l at ent r oot s equal t o t he s t h
p o we r of t hos e of A. I f t he r oot s of A ar e all less t ha n 1 i n abs ol ut e val ue, t hos e of
A s will all conver ge t o zer o as s ~ ~ , whi ch me a ns t hat t he l i mi t of A s f or s ~
is a zer o mat r i x. Al so, l et S = I + A + + AS; t hen, b y s ubt r a c t i ng AS = A +
A 2 + + A ' +1 we obt a i n ( I - A) S = I - A ~+1 so t ha t we have t he f ol l owi ng
r esul t f or t he l i mi t of S whe n all r oot s of A ar e less t ha n 1 i n abs ol ut e val ue:
l i m ( I +A+ . . . +A~)=(I - A) -1
S ~O0
( 7 . 1 5 )
We pr oc e e d t o appl y t hese resul t s t o (7. 14) by l aggi ng it b y one per i od,
Yt l = a + Ay t 2 - { - A*x t - 1 q- igt-1, a nd s ubs t i t ut i ng t hi s i nt o (7. 14):
y, = ( I + A ) a + h2yt _ 2 q- A*x t + AA*x t_ 1 - ~ - Ut -~- Aut - 1
Ch. 1: Linear Algebra and Matrix Methods 37
When we do this s times, we obt ai n:
y t =( l + A + . . . + A S ) a + A S +l y t _ s _ 1
+ A * x t + A A * x t - l + " " " + A S A * x t - s
+ u t + A u t - I + " "" + A S u t ~" (7.16)
I f all root s of A are less t han 1 in absol ut e value, so t hat A s converges to zero as
s ---> ~ and (7.15) holds, t he limit of (7.16) for s ---> c~ becomes
y, = ( I -- A ) l a -4.- E A S A * x t - s + A~'u, ~, ( 7 . 1 7 )
s = 0 s = 0
whi ch is the f i n a l f o r m of t he equat i on syst em. Thi s f or m expresses each current
endogenous var i abl e in t er ms of cur r ent and l agged exogenous vari abl es as well as
cur r ent and l agged di st urbances; all l agged endogenous vari abl es are el i mi nat ed
f r om the r educed f or m (7.14) by successive l agged appl i cat i on of (7.14). The
coeffi ci ent mat ri ces ASA * of x t _ s for s = 0, 1,2 . . . . in (7.17) ma y be vi ewed as
mat r i x mul t i pl i ers; see Gol dber ger (1959). The behavi or of t he el ement s of A~A *
as s ---, ~ is domi nat ed by t he r oot of A wi t h the l argest absol ut e value. I f this
r oot is a conj ugat e compl ex pair, t he behavi or of t hese el ement s f or i ncreasi ng s is
of t he da mpe d osci l l at ory type.
Endogenous var i abl es occur in (7.14) onl y wi t h a one- per i od lag. Suppose t hat
Ay e _ 1 in (7.14) is ext ended t o A 1 Y t - I q- " ' " + A . c y t - ~ - , where ~" is the l argest lag
whi ch occurs in the equat i on syst em. I t ma y be shown t hat t he rel evant de-
t er mi nant al equat i on is now
I x , ( - 1 ) + x 1A, + . . . + A , I = 0 . (7.18)
When there are L endogenous vari abl es, (7.18) yields L~- sol ut i ons whi ch may
i ncl ude conj ugat e compl ex pairs. All these sol ut i ons shoul d be less t han 1 in
absol ut e val ue in or der t hat the syst em be stable, i.e. i n or der t hat t he coeffi ci ent
mat r i x of x t , in the final f or m converges to zero as s ~ oc. I t is readi l y veri fi ed
t hat for ~- = 1 this condi t i on refers to t he l at ent root s of A l, in agr eement wi t h the
condi t i on under l yi ng (7.17).
8. Principal components and ext ensi ons
8. 1. Pr i n c i p a l c o mp o n e n t s
Consi der an n K obser vat i on mat r i x Z on K vari abl es. Our obj ect i ve is to
appr oxi mat e Z by a mat r i x of uni t rank, v c ' , where v is an n- el ement vect or of
38 1t. Theil
val ues t aken by some var i abl e (to be const r uct ed bel ow) and c is a K- el ement
coeffi ci ent vect or. Thus, the appr oxi mat i on descri bes each col umn of Z as
pr opor t i onal to v. I t is obvi ous t hat if t he r ank of Z exceeds 1, t here will be a
non- zer o n K di scr epancy mat r i x Z - v c ' no mat t er how we speci fy v and c; we
select v and c by mi ni mi zi ng t he sum of t he squares of all Kn di screpanci es. Also,
since vc' r emai ns unchanged when v is mul t i pl i ed by k ~= 0 and c by 1 / k , we shall
i mpos e v ' v = 1. I t is shown in the next subsect i on t hat t he sol ut i on is v = v~ and
c = c I, defi ned by
( Z Z ' - ~ , I ) v, = 0, (8. 1)
cl = Z' vl , ( 8 . 2 )
where )~1 is t he l argest l at ent r oot of t he symmet r i c posi t i ve se~nidefinite mat r i x
Z Z ' . Thus, (8.1) st at es t hat v~ is a charact eri st i c vect or of Z Z ' cor r es pondi ng t o
t he l argest l at ent root . (We assume t hat t he non- zer o r oot s of Z Z ' are distinct. )
Not e t hat v I ma y be ar bi t r ar i l y mul t i pl i ed by - 1 in (8.1) but t hat this changes c~
i nt o - c I i n (8.2) so t hat the pr oduct VlC ~ r emai ns unchanged.
Our next obj ect i ve is to appr oxi mat e t he di scr epancy mat r i x Z - vLc ~ by a
mat r i x of uni t r ank v2e~, so t hat Z is appr oxi mat ed by v l c ~ + v2c ~. The cri t eri on is
agai n the resi dual sum of squares, whi ch we mi ni mi ze by var yi ng v 2 and c 2 subj ect
t o t he const r ai nt s v~v 2 = 1 and v~v~ = 0. I t is shown in t he next subsect i on t hat the
sol ut i on is i dent i cal to (8.1) and (8.2) except t hat the subscri pt 1 becomes 2 wi t h
X2 i nt er pr et ed as the second l argest l at ent r oot of Z Z ' . The vect ors v~ and v 2 are
known as t he first and second pr i nc i pal component s of the K vari abl es whose
obser vat i ons are ar r anged in t he n x K mat r i x Z.
Mor e generally, t he i t h pr i nci pal c ompone nt v i and t he associ at ed coeffi ci ent
vect or c, are obt ai ned f r om
( Z Z ' - - Xi I ) v ~ = 0, (8. 3)
ei = Z' vi , " ( 8 . 4 )
where Xi is the i t h l argest r oot of Z Z ' . Thi s sol ut i on is obt ai ned by appr oxi mat i ng
Z - v i e ~ . . . . . v i - l c ; - 1 by a mat r i x vice, t he cri t eri on bei ng t he sum of the
squar ed di screpanci es subj ect to t he uni t l engt h condi t i on v~v i = 1 and the
or t hogonal i t y condi t i ons v; ~ = 0 for j = 1 . . . . . i - 1.
8. 2. Der i vat i ons
I t is easily veri fi ed t hat t he sum of t he squares of all el ement s of any mat r i x A
(square or rect angul ar) is equal to t r A ' A . Thus, the s um of t he squares of the
Ch. 1: Linear Algebra and Matrix Methods 39
el ement s of t he di s c r e pa nc y mat r i x Z - v c ' equal s
t r ( Z - v c ' ) ' ( Z - v c ' ) = t r Z ' Z - t r c v ' Z - t r Z ' v e ' + t r c v ' v c '
= t r Z ' Z - 2 v ' Z c + ( v ' v ) ( c ' c ) ,
whi ch can be si mpl i f i ed t o
t r Z ' Z - 2 v ' Z c + c ' c ( 8. 5)
i n vi ew of v ' v = 1. The der i vat i ve of (8. 5) wi t h r espect t o c is - 2 Z ' v + 2c so t hat
mi ni mi zi ng (8. 5) b y var yi ng c f or gi ven v yi el ds c = Z ' v , i n a gr e e me nt wi t h (8.2).
By s ubs t i t ut i ng c = Z ' v i nt o (8.5) we obt a i n t r Z ' Z - v ' Z Z ' v ; hence, our next st ep
is t o maxi mi ze v ' Z Z ' v f or var i at i ons i n v subj ect t o v ' v = 1. So we cons t r uct t he
La gr a ngi a n f unc t i on v ' Z Z ' v - / * ( v ' v - 1) and di f f er ent i at e it wi t h r espect t o v and
e qua t e t he der i vat i ve t o zero. Thi s yi el ds Z Z ' v = / * v so t ha t v mus t be a
char act er i s t i c vect or of Z Z ' c or r e s pondi ng t o r oot / *. Thi s c onf i r ms (8.1) i f we can
pr ove t ha t / * equal s t he l ar gest r oot X~ of Z Z ' . Fo r t hi s pur pos e we pr e mul t i pl y
Z Z ' v = / * v b y v' , whi ch gi ves v ' Z Z ' v = / * v ' v =/ *. Si nce we seek t he ma x i mu m of
v ' Z Z ' v , t hi s s hows t hat / * mus t be t he l ar gest r oot of Z Z ' .
To ver i f y (8. 3) a nd (8. 4) f or i = 2, we cons i der
t r ( Z - v , c ~ - v 2 c ' 2 ) ' ( Z - v , c ~ - v 2 c ~ )
= t r ( Z - v , c ~ ) ' ( Z - v , c ' l ) - 2 t r ( Z - v , c ' l ) ' v 2 c ~ + t r c 2 v ~ v 2 c ~
= t r ( Z - v , c ' , ) ' ( Z - v , c ~ ) - 2 c ~ Z ' v 2 + c ~c 2 , ( 8. 6)
wher e t he l ast st ep is ba s e d on v ' l v 2 = 0 and v'2v 2 = 1. Mi ni mi z a t i on of (8.6) wi t h
r es pect t o c 2 f or gi ven v 2 t hus yi el ds c2 = Z ' v 2 , i n a gr e e me nt wi t h (8.4). Subst i t u-
t i on of c 2 = Z ' v 2 i nt o (8. 6) shows t hat t he f unc t i on t o be mi ni mi zed wi t h r es pect
t o v 2 t akes t he f o r m of a c ons t a nt [equal t o t he t r ace i n t he l ast l i ne of (8.6)] mi nus
v ~ Z Z ' v 2. So we ma xi mi z e v ~ Z Z ' v 2 b y var yi ng v 2 subj ect t o v ~ v 2 = 0 and v ~v 2 = 1,
us i ng t he La gr a ngi a n f unc t i on v ~ Z Z ' v 2 - / * l v ~ v 2 - / * 2 ( v ~ v 2 - 1). We t ake t he de-
r i vat i ve of t hi s f unc t i on wi t h r espect t o v 2 a nd equat e i t t o zer o:
2 Z Z ' v 2 - / * l V l - 2 / . 2 v 2 = 0.
( 8. 7)
We pr emul t i pl y t hi s b y v], whi ch yi el ds 2 v ' l Z Z ' v 2 = / * l l ) ] V l = ILl becaus e v ' l v 2 = O.
But v ' l Z Z ' v 2 = 0 a nd hence/ *1 = 0 becaus e (8. 1) i mpl i es v ' 2 Z Z ' v 1 = )~lv;v 1 = 0. We
can t hus si mpl i f y (8.7) t o Z Z ' v 2 =/*2Vz so t ha t v 2 is a char act er i s t i c vect or of Z Z '
c or r e s pondi ng t o r oot / *2. Thi s vect or mus t be o r t h o g o n a l t o t he char act er i s t i c
vect or v~ c or r e s pondi ng t o t he l ar gest r oot X1, whi l e t he r oot / *2 mus t be as l ar ge
as possi bl e becaus e t he obj ect i ve is t o maxi mi ze v ~ Z Z ' v 2 =/ * 2 v ; v 2 =/ *2- Ther ef or e,
4 0 H. Theil
/~2 mus t be the second l argest r oot ~2 of Z Z ' , whi ch compl et es t he pr oof of (8.3)
and (8.4) f or i = 2. The ext ensi on to l arger val ues of i is l eft as an exercise f or the
reader.
8. 3. Further discussion of principal components
I f t he r ank of Z is r, (8.3) yields r pr i nci pal component s cor r espondi ng to posi t i ve
r oot s ~ . . . . . ?~r- I n what follows we assume t hat Z has full col umn r ank so t hat
t here are K pr i nci pal component s cor r espondi ng t o K posi t i ve root s, X~ . . . . . ~K"
By pr emul t i pl yi ng (8.3) by Z ' and usi ng (8.4) we obt ai n:
(z' z - x , t ) e i = o , ( 8 . 8 )
so t hat t he coeffi ci ent vect or c i is a charact eri st i c vect or of Z ' Z cor r es pondi ng to
r oot Xi. The vect ors c~ . . . . . c K are or t hogonal , but t hey do not have uni t length. To
pr ove this we i nt r oduce t he mat r i x V of all pr i nci pal component s and t he
associ at ed coeffi ci ent mat r i x C:
V = [ I ) 1 . . - V K ] , C = [ c 1 . . . C K ] ,
so t hat (8.3) and (8.4) for i = 1 . . . . . K can be wri t t en as
(8. 9)
zz' v= va, ( 8 . 1 0 )
c =z ' v , ( 8 . 1 1 )
where A is t he di agonal mat r i x wi t h ~' t , ---, 2tK on t he di agonal . By pr emul t i pl yi ng
(8.10) by V' and usi ng (8.11) and V' V= I we obt ai n:
C' C=A , (8. 12)
whi ch shows t hat Cl , . . . , e K are or t hogonal vect ors and t hat t he squar ed l engt h of
e i equals X i.
I f the obser ved vari abl es are measur ed as devi at i ons f r om t hei r means, Z ' Z in
(8.8) equal s t hei r sampl e covari ance mat r i x mul t i pl i ed b y n. Since Z ' Z need not
be di agonal , the obser ved vari abl es ma y be correl at ed. But t he pr i nci pal compo-
nent s are all uncor r el at ed because v~vj = 0 for i * j . Therefore, t hese component s
can be vi ewed as uncorrelated linear combinations of correlated variables.
8.4. The independence transformation in microeconomic theory
The pr i nci pal component t echni que can be ext ended so t hat t wo square mat r i ces
are si mul t aneousl y di agonal i zed. An at t r act i ve way of discussing this ext ensi on is
Ch. 1: Li near Al gebra and Mat r i x Met hods
41
i n t er ms of t he di f f er ent i al d e ma n d a nd s uppl y e qua t i ons of Sect i on 5. Recal l t hat
u n d e r pr e f e r e nc e i n d e p e n d e n c e t he d e ma n d e qua t i on (5.5) t akes t he f or m (5. 11)
wi t h onl y one r el at i ve pri ce. Pr e f e r e nc e i n d e p e n d e n c e a mo u n t s t o addi t i ve ut i l i t y
a nd is t hus qui t e rest ri ct i ve. But i f t he c ons ume r is n o t pr e f e r e nc e i n d e p e n d e n t
wi t h r es pect t o t he N obs e r ve d goods , we ma y a s k whe t he r i t is pos s i bl e t o
t r a ns f or m t hese goods so t hat t he c o n s u me r is pr e f e r e nc e i n d e p e n d e n t wi t h
r e s pe c t t o t he t r a n s f o r me d goods. Si mi l arl y, i f a f i r m is n o t i nput i nde pe nde nt ,
c a n we der i ve t r a n s f o r me d i nput s so t hat t he f i r m is i nput i n d e p e n d e n t wi t h
r e s pe c t t o t hese? An a na l ogous que s t i on c a n b e a s ke d f or t he o u t p u t s of a
mu l t i p r o d u c t f i r m; b e l o w we cons i der t he i nput s of a s i ngl e - pr oduc t f i r m i n or de r
t o fi x t he at t ent i on.
Cons i de r t he i nput al l ocat i on e q u a t i o n (5. 21) a nd di vi de b y f i :
q ,
d( l og q, ) = ~ d( l og Q ) - ~
N
y ' 0i / d( l og pP--~J, ) . ( 8. 13)
j =l
Thi s shows t ha t O i / f i is t he el ast i ci t y o f t he d e ma n d f or i nput i wi t h r es pect t o t he
Di vi s i a i nput vol ume i ndex; we shal l expr ess t hi s b y r ef er r i ng t o O ~ / f ~ as t he
D i v i s i a e l a s t i c i t y of i nput i, whi ch is t he f i r m' s i n p u t ver s i on of t he c ons ume r ' s
i nc ome el ast i ci t y of t he d e ma n d f or a g o o d ] Al so, (8. 13) s hows t hat - ~ / O i j / f ~ is
t he el ast i ci t y of i nput i wi t h r es pect t o t he Fr i s c h- de f l a t e d pr i ce o f i n p u t j . Un d e r
i nput i n d e p e n d e n c e t he s ubs t i t ut i on t e r m is s i mpl i f i ed [see (5. 15) and (5.18)] so
t ha t (8.13) b e c o me s
d ( l g q i ) = ~ i i d ( l O g Q ) - ~ b O i ~ i d(lgPP---~")"
( 8. 14)
He nc e , all pr i ce el ast i ci t i es vani s h except t he own- pr i c e el ast i ci t i es; t he l at t er t ake
t he f or m - + 0 i / f i a nd ar e t hus p r o p o r t i o n a l t o t he Di vi s i a el ast i ci t i es wi t h - ~b as
t he (negat i ve) p r o p o r t i o n a l i t y coef f i ci ent .
Ne x t cons i der t he i n p u t al l ocat i on s ys t e m i n t he f o r m (5.23):
rK = (, ' r K )o, - q,o( I -
( 8 . 1 5 )
Our obj ect i ve is t o def i ne t r a ns f or me d i nput s whi ch di agonal i ze O. We p e r f o r m
t hi s unde r t he c ondi t i on t hat t ot al i nput e xpe ndi t ur e a nd i t s Di vi s i a de c ompos i -
t i on ar e i nvar i ant unde r t he t r a ns f or ma t i on. The de r i va t i on is gi ven i n Ap p e n d i x
B a nd t he r esul t ma y b e de s c r i be d b y me a ns of a s i mul t a ne ous di agonal i zat i on
VThe consumer's version of O i / f i is O i / w , ; it is easily verified [see (5.3)] that O i / w ~ equals the
elasticity of q~ with respect to M.
42 H. Theil
similar to(7. 13):
X ' O X = A , X ' F X = I , (8.16)
where A is the diagonal mat ri x with the roots XI, . . . , XN on the diagonal. These
roots are the Divisia elasticities of the t ransformed inputs. The allocation equa-
t i on for t ransformed i nput i takes the form
d ( l o g q , r i ) =X i d ( l o g Q) - +X i d ( l o g - ~ , i ) ,
(8.17)
where the subscript T st ands for "t r ansf or med". A compari son of (8.17) and
(8.14) shows t hat the Divisia volume and Frisch price indexes and q~ are all
i nvari ant under the t ransformat i on.
Recall from (7.11) and (7.13) t hat any col umn x~ of the mat ri x X in (8.16)
satisfies
( 0 - X i F ) x i = 0. (8.18)
We premul t i pl y this by - ~pF-I:
[ - ~ p F- l O- ( - +Xi ) I ]x~ = O. (8.19)
Since - q~F-IO = [ - ~Oq/ f i ] is the price elasticity mat ri x of the observed i nput s
[see (8.13)] and -q' ?~i is the own-price elasticity of t ransformed i nput i [see
(8.17)], (8.19) shows t hat the latter elasticity is a l at ent root of the price elasticity
mat ri x - q~F-~O of the observed inputs. This is an asymmet ri c matrix, but the
X i' s are nevertheless real. To prove this we premul t i pl y (8.18) by F- t / 2 and write
the result as
( F - 1 / 2 0 F - 1 / 2 - ~ i I ) F 1 / 2 x i = O . (8.20)
Since F- l / 20F -1/ 2 is symmetric positive definite, the Xi's are all real and
positive. Hence, all t ransformed i nput s have positive Divisia elasticities. The
diagonalization (8.20) is uni que when the Xi's are distinct. This means t hat the
t ransformed inputs are identified by their Divisia elasticities.
These elasticities can be used as a tool for the i nt erpret at i on of the t ransformed
inputs. Anot her tool is the so-called composition matrix
r = - ' , ( 8 . 2 1 )
where (X-I/,)A is defined in (7.9). The col umn sums of T are the factor shares
f l . . . . . fN of the observed inputs and the row sums are the factor shares of the
Ch. 1." Li near Al gebra and Mat ri x Met hods 43
t r a n s f o r me d i nput s . Ea c h r ow of T gi ves t he c o mp o s i t i o n of t he f a c t or shar e o f a
t r a n s f o r me d i n p u t i n t e r ms of obs e r ve d i nput s ; e a c h c ol umn of T s hows t he
c ompos i t i on of t he f a c t or shar e of an obs e r ve d i nput i n t er ms of t r a ns f or me d
i nput s . Fo r p r o o f s of t hese r esul t s we r ef er t o Ap p e n d i x B; b e l o w we cons i der an
e xa mpl e i l l ust r at i ng t hese resul t s, af t er whi ch a c o mp a r i s o n wi t h pr i nc i pa l c o mp o -
ne nt s will f ol l ow at t he e nd of t hi s sect i on.
8.5. An example
We cons i der a t wo- i nput t r ans l og s peci f i cat i on of (5.12):
l og z = c ons t a nt + a l og K + f l l og L + ~ g c ~ l og K l og L, ( 8. 22)
whe r e K is capi t al , L is l abor , a nd a, fl, a nd ~ ar e c ons t a nt s s at i s f yi ng a > 0, 13 > 0,
a nd - 1 < ~ < 1 ; uni t s ar e chos en so t hat K = L = I hol ds at t he poi nt of
mi ni mum i nput expendi t ur e. The n i t ma y be s hown t ha t t he 2 2 pr i ce coef f i ci ent
mat r i x - + O = [ - +0i j ] of (8. 15) equal s
~ O = - 1 [ fK ~ l capi t al
- - - ( 8 . 2 3 )
2 A ] l a b o r '
whe r e fK is t he f a c t or shar e of capi t al a nd f L t hat of l a b o r ( f K + f L = 1). Recal l
f r om Sect i on 6 t hat i nput s i and j ar e cal l ed speci fi c c o mp l e me n t s ( s ubs t i t ut es )
whe n 0ij = 0ji is pos i t i ve (negat i ve). Thus, (8. 23) c o mb i n e d wi t h ~b > 0 s hows t hat
capi t al and l a b o r ar e speci f i c c o mp l e me n t s ( s ubs t i t ut es ) whe n ~ is pos i t i ve
(negat i ve), i.e. whe n t he el ast i ci t y of o u t p u t wi t h r e s pe c t t o ei t her i nput is an
i ncr easi ng ( decr easi ng) f unc t i on of t he ot her i nput [see (8.22)].
The i nput i n d e p e n d e n c e t r a ns f or ma t i on el i mi nat es al l speci f i c s ubs t i t ut a bi l i t y
a nd c ompl e me nt a r i t y r el at i ons. The ma t he ma t i c a l t ool is t he s i mul t a ne ous di-
agonal i zat i on (8.16). I t ma y be ver i f i ed t hat , f or - ~kO gi ven i n (8.23); t he ma t r i x
1
X = ~ [ / f f ~ K ~ j ( 8 . 2 4 )
sat i sfi es X' FX= I a nd t hat X ' ( ~ O) X is a di agonal mat r i x whos e di agonal
el ement s ar e 1/ ( 1 - ~) a nd 1/ ( 1 + ~). A c ompa r i s on wi t h (8. 16) a nd (8.17) s hows
t hat t he own- pr i c e el ast i ci t i es of t he t r a ns f or me d i nput s ar e
- 1 - 1
- ~ba, = 1 - ~ ' - ~bx2 = 1 + ~" ( 8. 25)
44 H. Thei l
Mul t i pl e roots occur when ~ = 0, but this is the uninteresting case in which (8.22)
becomes Cobb- Dougl as, which is in i nput i ndependent form and t hus requires no
t ransformat i on.
Substitution of (8.24) i nt o (8.21) yields the composi t i on mat ri x
1 [ fK + fK~f~ fL + ~f~/ ~-] (T1)
= JL- 1 (T2)
(capital) (labor)
(8.26)
The col umn sums are the fact or shares of the observed inputs: fK for capital and
f L for labor. The row sums are the fact or shares of the t ransformed inputs:
+ fK~f~ for the i nput T L corresponding to root Xl and - ~ for T 2
corresponding to ?t 2. The following is a numerical specification of (8.26), bordered
by row and col umn sums, for fK = 0.2 and f L = 0.8:
0.3 0.6 0.9 (Tl )
- 0 . 1 0.2 0.1 (T2)
0.2 0.8 1
(capital) (labor)
Bot h observed inputs are positively represented in TI, whereas T 2 is a contrast
bet ween l abor and capital. When the firm buys more T2, its operat i on becomes
more labor-intensive, each dollar spent on T 2 being equivalent to two dollars
wort h of l abor compensat ed by one dollar wort h of capital services which is given
up.
8.6. A principal component interpretation
We return to (8.8) with Z ' Z interpreted as n times the mat ri x of mean squares and
product s of the values taken by the observed variables. In many applications of
the principal component technique, the observed variables have di fferent dimen-
sions (dollars, dollars per year, gallons, etc.). This causes a problem, because
principal component s change in value when we change the units in which the
observed variables are measured. To solve this problem, statisticians frequent l y
st andardi ze these variables by using their st andard deviations as units. This
amount s to replacing Z ' Z in (8.8) by D- ~ / 2 Z' ZD- i / 2 , where D is the di agonal
mat ri x whose diagonal is identical to t hat of Z ' Z . Thus, h i of (8.8) is now
obt ai ned from the characteristic equat i on
11 ) - I / 2 Z t Z D - 1/ 2 _ Xii[ = 0. (8.27)
Ch . 1: L i n e a r A l g e b r a a n d M a t r i x Me t h o d s
It is of interest to compare this with
IF 1/2@F- 1/2 _ h i l l = O,
45
(8.28)
which is the characteristic equat i on associated with (8.20). In bot h cases, (8.27)
and (8.28), we determine a l at ent root of a symmetric positive definite matrix
( Z' Z or @) pre- and postmultiplied by a diagonal matrix. However, the diagonal
elements of F are not identical to those of O, which is in contrast to D and Z' Z in
(8.27). The di agonal elements of F describe the expenditure levels of the i nput s
(measured as fractions of t ot al expenditure), whereas each di agonal element of O
describes the change in the demand for an input caused by a change in its
Frisch-deflated price.
Thus, while D in (8.27) is directly obt ai ned from Z' Z, the analogous mat ri x F
in (8.28) is unrel at ed to @. Why do we have this unrel at ed F, which describes
expenditure levels, in (8.28) and in the simultaneous diagonalization (8.16)? The
answer is t hat the i nput i ndependence t ransformat i on is subject to the const rai nt
t hat t hat total i nput expenditure and its Divisia decomposi t i on remai n invariant.
We may view this t ransformat i on as a cost-constrained principal component
t ransformat i on. Similarly, when the t ransformat i on is applied to the consumer
demand system (5.5) or t o the out put supply system (5.26), it is budget-con-
strained in the first case and revenue-constrained in the second. Such constraints
are more meani ngful from an economic poi nt of view t han the st andardi zat i on
procedure in (8.27).
9. The modeling of a disturbance covariance matrix
We ment i oned in Section 4 t hat the di st urbance covariance mat ri x ~ which occurs
in the GLS estimator (4.11) is normal l y unknown and t hat it is then usually
replaced by the sample moment mat ri x of the LS residuals. Al t hough this
approxi mat i on is acceptable under certain condi t i ons when the sample is suffi-
ciently large, it is less satisfactory when the number of equations, L in (4.7) and
(4.8), is also large. The reason is t hat ~ cont ai ns many unknowns when L is large
or even moderat el y large. In fact, the sample moment mat ri x S of the residuals
may be singular so t hat ~ - I in (4.11) cannot be approxi mat ed by S ~. This
situation oft en occurs in applied econometrics, e.g. in the est i mat i on of a fairly
large system of demand equations. One way of solving this probl em is by
model i ng the mat ri x Z. Below we describe how this can be performed when the
equations are behavioral equations of some decision-maker.
46 H. Theil
9.1. Rational random behavior
Let x = [x~. . . xk]' be the vector of variables cont rol l ed by this decision-maker. We
write J for the feasible region of x; 2 for the opt i mal value of x (.~ ~ J ) ; and
l (x, 2) for the loss incurred when the decision is x rat her t han 2:
l ( x , . ~ ) = 0 i f x = ~ , ' (9.1)
> 0 if x :* ~.
We assume t hat the opt i mal decision .~ depends on numerous factors, some of
which are unknown, so t hat 2 is onl y theoretically optimal in the sense t hat it is
opt i mal under perfect knowledge. The decision-maker can i mprove on his
ignorance by acquiring i nformat i on. If he does not do this, we describe the
decision made as r andom with a differentiable density funct i on po(X), to be
called the prior density function. (The assumpt i on of randomness is j ust i fi ed
by the decision-maker' s uncert ai nt y as to the factors det ermi ni ng 2.) If he does
acquire i nformat i on, P0( ' ) i s t ransformed i nt o some other density funct i on p ( . )
and the amount of i nformat i on received is defi ned as
I = f j P( x ) l og ~ d x , . . . d x k , (9.2)
which is a concept from statistical i nf or mat i on t heory [see Theil (1967)]. We write
c ( I ) for the cost of i nformat i on and
i = f j ( x , ) p ( x ) d x , . . . d x k ( 9 . 3 )
for the expected loss. If c ( I ) and [ ar e measured in the same uni t (dollars or any
ot her unit), the nat ural solution is the decision di st ri but i on with densi t y funct i on
p ( . ) which minimizes c ( I ) + L This p( - ) was derived by Barbosa (1975) and the
result (see Appendi x C) is
p(x) P 0 ( x ) e x p ( l(x, 2)
cx c' ) i f x ~ J , (9.4)
where ~x means "i s proport i onal t o", the proport i onal i t y coefficient being inde-
pendent of x, and e' is the marginal cost of i nformat i on d c / d I at the solution
(9.4).
Behavior generated by the di st ri but i on which densi t y funct i on (9.4) is called
rational random behavior. This di st ri but i on is det ermi ned by three factors: the
prior density funct i on Po(X), the loss funct i on l (x, ~), and the margi nal cost of
Ch. 1: Linear Algebra and Matrix Methods 47
i nformat i on c'. For example, let x be a scalar and the loss funct i on quadratic:
l ( x, 2) = l ( x - ~)2. Then, if po(X) equals a const ant i ndependent of x for each
x ~ J, (9.4) becomes
( 1 ( x - x ) 2 } i f x ~ J , (9.5)
p ( x ) cx exp 2 c'
which is the density funct i on of a t runcat ed normal decision di st ri but i on over the
interval J.
9.2. The asymptotics of rational random behavior
The case of a small margi nal cost of i nformat i on is of part i cul ar interest. Imagine
t hat the prices of the goods and services which the decision-maker buys in order
to acquire i nformat i on decline so t hat e' converges to zero; it is shown in
Appendi x C t hat the r andom decision with density funct i on (9.4) t hen converges
in probability to the theoretically opt i mal decision 2. Also, if the loss funct i on has
a zero gradient and a symmet ri c positive definite Hessian mat ri x A at x = 2,
~ x 02
l ( x , ~ ) =O and Oxox, l ( x , ~ ) =A a t x = . ~ , (9.6)
t hen as c'---, 0 the density funct i on p ( x ) of (9.4) converges to
1 1 A 1/2 f
(9.7)
which is the densi t y funct i on of the mul t i normal decision di st ri but i on [see (3.3)]
with mean vector and covariance mat ri x e ' A- 1. Not e t hat (9.7) is completely
det ermi ned by c' and two characteristics of the loss function: the theoretically
opt i mal decision 2 at which the loss vanishes [see (9.1)] and the Hessian mat ri x A
of this funct i on at this point. The relationship between the covariance mat ri x
c ' A- 1 and the Hessian mat ri x A of the loss funct i on enables us to model the
disturbance covariance mat ri x of the decision-maker' s behavioral equations;
examples will follow in the next subsection.
The prior density funct i on P0(' ) does not occur in the asympt ot i c result (9.7).
This reflects the fact t hat when i nformat i on is cheap in the sense t hat its marginal
cost ' is small, the decision-maker acquires i nformat i on to such an extent t hat his
behavior becomes i ndependent of his prior notions. Thus, whereas we obt ai ned
(9.5) under the assumpt i on t hat po(X) is a const ant i ndependent of x, this
assumpt i on is unnecessary in the asympt ot i c case e' ~ 0. Also, (9.5) is the densi t y
48 H. Theil
funct i on of a t runcat ed normal distribution, but the t runcat i on becomes irrelevant
as c'---, 0. The asympt ot i c version of (9.5) is the univariate normal densi t y
funct i on wi t h mean equal to the theoretically opt i mal decision ~ and variance
equal to the marginal cost of i nformat i on c'. The declining variance as c ' ~ 0
reflects the attractiveness of a more extensive search for the theoretical opt i mum
when i nformat i on becomes cheaper.
It is of interest to compare the density funct i on (9.7) of the asympt ot i c normal
decision di st ri but i on with the asympt ot i c normal densi t y of a statistical estimator.
In fact, it is not difficult to prove t hat rat i onal r andom behavior for small c' is
equivalent to large-sample ML est i mat i on of the theoretically opt i mal decision 2,
wi t h a large sample interpreted as a small margi nal cost of i nformat i on. The clue
for this equivalence is the similarity of the roles pl ayed by the Hessian mat ri x of
the loss funct i on and the i nformat i on mat ri x in ML theory.
A second statistical compari son is t hat with Kadane' s (1971) small-o asymp-
totics, which consists of an asympt ot i c series of a multiple o of the variance of the
di st urbance of a structural equation. If this equat i on is a behavioral equat i on of a
decision-maker, Kadane' s approach is equivalent to the asympt ot i c version of
rat i onal r andom behavior when we i dent i fy o with c'.
Anot her statistical compari son of interest is t hat with the t heorem which states
t hat out of all distributions with range ( - ~ , ~ ) and a given mean vector and a
given covariance matrix, the mul t i normal di st ri but i on has the largest entropy. The
link between this theorem and the normal density funct i on (9.7) is the i nforma-
t i on definition (9.2); bot h (9.2) and the ent ropy are measures from i nformat i on
theory. However, not e t hat the normal densi t y (9.7) is not obt ai ned by i mposi ng a
given mean vector and covariance mat ri x a pr i or i . The mean vector and covari-
ance matrix (9.7) are det ermi ned by the loss function, apart from the scalar c' .
Yet anot her statistical compari son is with Bayesian inference. There is consid-
erable similarity between the exact (i.e., non-asympt ot i c) result (9.4) and the
Bayesian derivation of the posterior densi t y funct i on of a paramet er vector. The
occurrence of the prior density funct i on on the fight in (9.4) provides one
similarity. Anot her is the presence of c' (which depends on the i nf or mat i on
acquired) in the exponent of (9.4); this should be compared wi t h the role of the
likelihood funct i on (representing the i nformat i on obt ai ned from the sample) in
the Bayesian formula. A third similarity is the disappearance of the prior densi t y
funct i on from the asympt ot i c result (9.7). In Bayesian analysis, too, the posterior
densi t y funct i on is domi nat ed by the likelihood funct i on and is no longer affected
by the prior density funct i on when the sample is large. All these similarities reflect
the fact t hat rat i onal r andom behavior and Bayesian inference bot h describe
learning processes based on acquiring ~nformation. Nevertheless, the two theories
are not equivalent because of the occurrence of the unknown const ant ~ in (9.4).
The likelihood funct i on in Bayesian analysis involves no unknown const ant s; this
funct i on is det ermi ned by the paramet er vector, which is viewed as random, and
Ch. 1: Linear Algebra and Matrix Methods 49
the sample, which is viewed as a set of known const ant s for the derivation of the
posterior densi t y function.
9. 3. A p p l i c a t i o n s t o d e m a n d a n d s u p p l y
When we appl y the t heory of rat i onal r andom behavior to the utility-maximizing
consumer or the cost-minimizing firm, we must t ake i nt o consideration t hat the
criterion funct i on is subject to a const rai nt (a budget or t echnol ogy constraint).
This can be solved by using the const rai nt to eliminate one of the goods. The
consumer' s loss funct i on in (9.4) t hen involves N- 1 quantities and its derivation
from an algebraically specified ut i l i t y funct i on is straightforward. However, the
differential approach provides no such specification so t hat (9.4) cannot be used;
the same holds for the fi rm because the approach provides no algebraic specifica-
t i on of the product i on function. But it is possible to use the asympt ot i c result
(9.7) which requires onl y the theoretically optimal decision and the Hessian
mat ri x of the loss function. The account which follows is therefore based on the
asympt ot i c decision di st ri but i on (9.7) rather t han (9.4); this also has the ad-
vantage of not requiring a specification of the prior densi t y funct i on P0(' )-
Consider the i nput allocation system in the form (6.1),
r ~ = ( , ' r ~ ) o , - , o ( i - , , ' o ) , ~ + ~, ( 9 . 8 )
or in scalar form, using O t = 0 = [ 0 i],
< 9 . 9 ) f i d ( l o g q i ) = O,d(logQ)- ~ ~ 0,jd log ~ + ~,,
j =l
where [e~] = e. The left variable in (9.9) is the i t h decision variable of the firm. The
right side, excluding ei, is the theoretical opt i mum of this variable, while e~ is the
r andom deviation from this opt i mum which is predicted by the t heory of rat i onal
r andom behavior. Since (9.7) implies normal i t y wi t h a mean equal to the
theoretically opt i mal decision, the ei' s are mul t i normal with zero mean. Their
covariance mat ri x (see Appendi x C) is
~ r ( ~ ) = 2 ( 0 - o , , ' o ) , (9.10)
or in scalar form,
cov(~,, + ) = o: ( 0, + - o i o j ) , (9.11)
whe r e 0 2 is a coefficient which is proport i onal to the margi nal cost of i nformat i on
50 H. Theil
c' . The covariance matrix (9.10) equals a scalar mul t i pl e of the coefficient mat ri x
of ~r =[ d( l ogpi ) ] in (9.8) so t hat the covariance (9.11) is proport i onal to the
subst i t ut i on effect (specific plus general) of a change in the price of j on the
demand for i.
The above result also holds when (9.7) is applied to the consumer; the onl y
modi fi cat i on required is t hat fi and ~ in (9.9) become w i and - q~, respectively [see
(5.5)]. Not e in particular t hat the di st urbance covariance mat ri x (9.10) involves
onl y unknown (a 2) in addi t i on to O which al ready occurs in the systematic part
of the equations. Thus, the implications of rat i onal r andom behavi or are quite
strong. We obt ai n even stronger implications for the demand and supply equa-
tions of the mul t i product firm; a brief account follows below.
Recall t hat when the firm makes m products, the out put t erm of the i nput
demand system (5.15) takes the form (5.25). So, by addi ng a di st urbance e, we
obtain:
f i d ( l o g q i ) = 7 Oirgrd(logzr)-~b Y'~ 0ijd log~-7 +e, .
r =l j =l
(9.12)
Appl i cat i on of (9.7) yields the result t hat the e i' s of this i nput demand system are
mul t i normal with zero means and t hat their variances and covariances take the
same form (9.11) which also holds for the consumer and the single-product firm.
Next, by assuming t hat the fi rm adjusts its out put s so as to maximize profit, we
obt ai n the out put supply system (5.26) which we reproduce with a di st urbance e*
added:
gr d( l ogzr ) =qJ * ~O * d( l og~,~) +~ * . (9.13)
By applying (9.7) we fi nd t hat the e*'s are mul t i normal wi t h zero means and the
following vari ance-covari ance structure:
a 2 + *
cov(e*, e*) = , - ~- - 0; *.
(9.14)
Since a 2 al ready occurs in (9.11) for the i nput demand disturbances, (9.14)
provides no further unknowns. In addition, (9.7) implies t hat the input demand
disturbances (the ei' s ) are stochastically independent o f the output supply dis-
turbances (the e*'s). This i ndependence has i mpor t ant implications for statistical
inference in demand and supply models; it implies t hat grd(log zr) can be viewed
as predet ermi ned in the input demand system (9.12). It is also i mpor t ant for the
probl em of how to organize the firm in terms of its i nput and out put manage-
ment , but such mat t ers are beyond the scope of this chapter.
Ch. 1: Linear Algebra and Matrix Methods 51
10. The Mo o r e - P e n r o s e i nve r s e
A ma t r i x ha s a n i nve r s e o n l y i f i t i s s qua r e a n d nons i ngul a r , b u t a n y m x n ma t r i x
A o f r a n k r ha s a u n i q u e Mo o r e - P e n r o s e i nver s e, wr i t t e n A +, whi c h i s d e t e r mi n e d
b y t he f ol l owi ng f our c o n d i t i o n s :
A A +A =A , ( 10. 1)
A +AA + = A +, ( 10. 2)
A A + a n d A +A a r e s y mme t r i c . ( 10. 3)
I t ma y be ve r i f i e d t ha t t he s e c o n d i t i o n s ar e s a t i s f i e d b y A + = A - 1 i n t he s pe c i a l
c a s e m = n = r . Ou r f i r st obj e c t i ve i s t o p r o v e t h a t A + exi s t s a n d i s uni que . 8
10.1. Pr oof o f t he exi s t ence and uni queness
Th e uni que ne s s of A + i s e s t a b l i s h e d b y a s s u mi n g t h a t c o n d i t i o n s ( 10. 1) - ( 10. 3)
ha ve t wo s ol ut i ons , A + = B a n d A + = C, a n d ve r i f yi ng t he f ol l owi ng 16 s t e ps
b a s e d on ( 10. 1) - ( 10. 3) :
B =BAB =B(AB)' = BB' A' = BB' (ACA) ' = BB' A' C' A'
= B ( n n ) ' ( A C ) ' = nAnAC = nAC = BACAC = ( BA) ' ( CA) ' C
= A' n' A' C' C = ( a na) ' C' C = A' C' C = ( CA) ' C = CA C = C.
The r e f or e , B = C, whi c h p r o v e s t ha t A + i s u n i q u e wh e n i t exi st s.
To pr ove t he e xi s t e nc e of A + we c ons i de r f i r s t a z e r o ma t r i x A of o r d e r m X n;
t h e n A + e qua l s t he n X m zer o ma t r i x , whi c h ma y b e ve r i f i e d b y c he c ki ng
( 10. 1) - ( 10. 3) . Ne x t c o n s i d e r a n o n - z e r o ma t r i x A so t h a t i t s r a n k r i s pos i t i ve.
Th e n A' A i s a s y mme t r i c pos i t i ve s e mi de f i ni t e ma t r i x of o r d e r n x n a n d r a n k r ,
a n d i t i s p o s s i b l e t o e xpr e s s A + i n t e r ms of t he p o s i t i v e l a t e n t r o o t s of A' A a n d
t he c h a r a c t e r i s t i c ve c t or s a s s o c i a t e d wi t h t he s e r oot s . Wr i t e D f or t he d i a g o n a l
r x r ma t r i x wh i c h c o n t a i n s t he pos i t i ve r oot s of A ' A o n t he d i a g o n a l a n d / / f o r
a n n x r ma t r i x wh o s e c o l u mn s a r e c h a r a c t e r i s t i c ve c t or s c o r r e s p o n d i n g t o t he s e
r oot s . Th e n (7. 7) a p p l i e d t o A' A yi e l ds
A' A = HDt t ' , ( 10. 4)
8There are other generalized inverses besides the Moore-Penrose inverse, most of which are
obtained by deleting one or more of the four conditions. For example, using (10.1) and (10.2) but
deleting (10.3) yields the reflexive generalized inverse, which in not unique; see Laitinen and Theil
(1979) for an application of this inverse to consumption theory. Monographs on applications of
generalized inverses to statistics include Albert (1972), Ben-Israel and Greville (1974), Boullion and
Odell (1971), Pringle and Rayner (1970, and Rao and Mitra (!971).
52 H. Theil
and t he resul t for A + is
A + = HD- ~H' A' , (10. 5)
whi ch is an n m mat r i x of r ank r.
To veri fy (10.5) we i nt r oduce an n x ( n - r ) mat r i x K whose col umns are
charact eri st i c vect ors of A' A cor r espondi ng t o t he zero root s:
A' AK = 0. (10. 6)
The n n mat r i x [ H K] consists of charact eri st i c vect ors of A' A cor r espondi ng
t o all root s and is t herefore an or t hogonal mat ri x, whi ch can be expressed in t wo
ways. Premul t i pl yi ng [ H K] by its t r anspose and equat i ng t he pr oduct to t he
uni t mat r i x yields
H' H = I , K' K = I , H' K = 0, (10. 7)
while post mul t i pl yi ng [ H K] by its t r anspose and equat i ng the pr oduct t o t he
uni t mat r i x gives
nn' + xx' = t . ( 1 0 . 8 )
The veri fi cat i on of (10.5) is now a mat t er of checki ng condi t i ons (10. 1)-(10. 3).
Pr emul t i pl yi ng (10.5) by A yields AA = AHD- 1H' A' , whi ch is symmet r i c. Ne xt
we post mul t i pl y (10.5) by A, A+A =HD- 1H' A' A, and hence in vi ew of (10.4)
and (10.7), A A = HD- 1H' HDH' = HH' , whi ch is also symmet ri c. We post mul t i -
pl y this by (10.5):
A +AA + = HH' HD- IH' A' = l i D- 1H' A' = A +,
whi ch conf i r ms (10.2). Fi nal l y, we post mul t i pl y AA + = AHD- 1H' A' by A :
AA +A = AH D- 1H ' A' A = AH D- 1H ' H DH ' = AH H ' = A .
To veri fy t he l ast step, AHH' = A' , we pr emul t i pl y (10.6) by K' , whi ch gives
( AK) ' AK = 0 or AK = 0. Therefore, AKK' = 0 so t hat pr emul t i pl i cat i on of (10.8)
by A yields AHH' = A.
10.2. Special cases
I f A has full col umn r ank so t hat ( A' A) -1 exists, A + = ( A' A) 1A' , whi ch ma y
ei t her be veri fi ed f r om (10.4) and (10.5) for r = n or by checki ng (10. 1)-(10. 3). We
ma y thus wri t e t he LS coeffi ci ent vect or in (2.8) as b = X+y, whi ch ma y be
Ch. 1: Linear Algebra and Matrix Methods
53
vi ewed as a n ext ens i on of b = X- l y i n t he speci al case of a squar e non- s i ngul a r X
(as ma n y r egr essor s as obser vat i ons) .
I f A is a s ymme t r i c n n mat r i x of r a nk r, t hen
r f
A = 2 X i x i x ; , A + = 2 ~ X i X ; , ( 10. 9)
i =l i =1
wher e X~, . . . , Xr ar e t he non- z e r o l at ent r oot s of A a nd x ~, . . . , x r ar e char act er i st i c
vect or s associ at ed wi t h t hese r oot s. Al so, A x i = 0 and A +x i = 0 f or i = r + 1, . . . , n ,
wher e Xr+ 1 . . . . . X,, ar e char act er i s t i c vect or s of A c or r e s pondi ng t o t he zer o r oot s.
Thus , i f A is s ymmet r i c, A has char act er i s t i c vect or s i dent i cal t o t hose of A, t he
s ame n u mb e r of zer o r oot s, and non- z e r o r oot s equal t o t he r eci pr ocal s of t he
non- z e r o r oot s of A. The ver i f i cat i on of t hese resul t s is agai n a ma t t e r of checki ng
( 10. 1) - ( 10. 3) a nd usi ng x ; x j = 8~j. Si nce a s ymme t r i c i de mpot e nt mat r i x such as
M i n (2.11) has onl y zer o a nd uni t r oot s, it t hus f ol l ows as a cor ol l ar y t hat such a
mat r i x is equal t o its own Mo o r e - P e n r o s e i nverse.
10.3. A gener al i z at i on o f A i t k e n ' s t heor em
We r et ur n t o t he l i near mode l (2.7), r e pr oduc e d her e:
y = Xf l + e. ( 10. 10)
As bef or e, we as s ume t ha t X is an n K mat r i x of r a nk K consi st i ng of
non- s t oc ha s t i c el ement s a n d t hat e has zer o expect at i on, b u t we n o w as s ume t hat
t he covar i ance mat r i x of e t akes t he si ngul ar f o r m o 2 V, t he n x n mat r i x V ha vi ng
r a nk r < n. Hence, t he Ai t ke n es t i mat or (2. 13) does not exist, but is seems
r eas onabl e t o ask whet her
[3 = ( X ' V + X ) - I X ' V + y ( 10. 11)
exi st s and is a bes t l i near unbi a s e d es t i mat or of ft. I t wi l l a ppe a r t ha t each of t hese
pr oper t i es ( t he exi st ence and t he best l i near unbi as ednes s ) r equi r es a speci al
c ondi t i on i nvol vi ng b o t h V a nd X.
Th e mat r i x V is c ompa r a bl e t o A ' A i n (10.4) a n d (10.6) i n t hat b o t h ar e
s ymme t r i c posi t i ve semi def i ni t e n x n mat r i ces of r a nk r. Ther ef or e, we can a ppl y
(10. 4) and (10.6) t o V r at her t han A ' A :
V = H D H ' , ( 10. 12)
V K = O , ( 10. 13)
54 H. Theil
where D is now t he r X r di agonal mat r i x with t he positive l at ent r oot s of V on t he
diagonal, H is an n r mat ri x whose col umns are charact eri st i c vect ors of V
cor r espondi ng to these roots, and K is an n ( n - r ) mat ri x consisting of
charact eri st i c vect ors of V t hat cor r espond t o t he zero roots. The results (10.7)
and (10.8) are also valid in t he present i nt erpret at i on. In addi t i on, (10.9) and
(10.12) i mpl y
V + = HD- 1 H' . (10. 14)
Our strategy, similar t o t hat of the pr oof of Ai t ken' s t heor em in Sect i on 7, will
be to premul t i pl y (10.10) by an appr opr i at e mat ri x so t hat t he t r ansf or med
di st urbance vect or has a scalar covari ance matrix. We select D- ~/ 2H' , where
D- 1/ 2 is t he diagonal mat ri x with the reci procal s of the positive square root s of
the di agonal elements of D in the diagonal:
D I / 2 H' y = ( D I / 2 H' X ) f l +D 1/2H'e.
(lO.15)
The covari ance mat ri x of D- I/ 2H' e is
E ( D 1/ 2H' ee' HD t / 2 ) =o Z D 1/ 2H' VHD 1/2:021,
where the last step is based on H' V H = D, whi ch is obt ai ned by premul t i pl yi ng
(10.12) by H' and post mul t i pl yi ng b y / / a n d using H' H = ! [see (10.7)]. Since
D- 1/ 2H' e thus has a scalar covari ance mat ri x, let us appl y LS t o (10.15).
Assumi ng t hat H' X and hence D- ~ / 2 H' X have full col umn rank, we fi nd the
following est i mat or of fl:
( D- t / 2 H' X) + D- 1 / 2 H ' y = ( X ' HD- 1H' X) - 1 X' HD- l H' y.
(10. 16)
Thi s is i ndeed i dent i cal to (10.11) in view of (10.14).
Two consi derat i ons are i mpor t ant f or the apprai sal of this procedure. First, we
assumed t hat H" X has full col umn rank; if t he r ank is smaller, t he mat r i x pr oduct
in parent heses on the right in (10.16) is singular so t hat (10.11) does not exist.
Therefore, a necessary and sufficient condi t i on f or t he existence of t he est i mat or
(10.11) is t hat H' X have maxi mum rank, where H consists of r charact eri st i c
vect ors of V correspondi ng t o t he posi t i ve roots. Secondl y, we obt ai ned (10.15) by
premul t i pl yi ng (10.10) by D I/ 2H' , whi ch reduces t he number of observat i ons
f r om n to r. We can recover t he "mi ssi ng" n - r observat i ons by pr emul t i pl i cat i on
by K' , yielding K' y = K' Xf l + K' e. The covari ance mat r i x of K' e is o z K' VK = 0
[see (10.13)] so t hat K' e vanishes with uni t probabi l i t y. Therefore,
K' y = K' Xf l ,
whi ch amount s to a linear const rai nt on fl unless K' X = O.
(10. 17)
Ch. 1: Linear Algebra and Matrix Methods 55
To clarify this situation, consi der t he following exampl e for K = 1, n = 3, and
r = 2 :
I 1 1
X= , V= 0 1 ,
1 0 0 0
Her e X has full col umn
H = 1 , K = .
0
( 1 0 . 1 8 )
rank but H ' X = 0 so t hat t he mat r i x pr oduct in
parent heses on t he right in (10.16) is singular; in fact, the underl yi ng equat i on
(10.15) does not cont ai n fl at all when H ' X = O. Thus, the est i mat or (10.11) does
not exist, but in t he case of (10.18) it is nevertheless possible t o det er mi ne/ 3 (a
scalar in this case) exactly! The reason is t hat (10.18) implies K ' y = y 3 and
K' X = 1 so t hat (10.17) states t hat Y3 equals t he paramet er. Ul t i mat el y, this results
f r om the zero val ue of t he t hi rd di agonal el ement of V in (10.18) and the non- zer o
t hi rd el ement of X.
Under t he assumpt i ons st at ed in t he discussion following eq. (10.10), the
est i mat or (10.11) exists when H ' X has full col umn r ank and it is a best linear
unbi ased est i mat or of fl when K ' X = 0 [so t hat (10.17) is not a real const rai nt on
fl]. A pr oof of t he l at t er st at ement follows in t he next paragraph. If K ' X is a
non- zer o matrix, (10.17) is a linear const rai nt on/ 3 whi ch shoul d be i ncor por at ed
in t he est i mat i on pr ocedur e; see Thei l (1971, sec. 6.8).
We can write any linear est i mat or of / 3 as
[3= [ A + ( X ' V + X ) - I X ' V + ] y , (10.19)
where A is some K n mat r i x consisting of non-st ochast i c elements. By substitut-
ing X/3 + e for y in (10.19) and taking t he expect at i on we fi nd t hat the unbi ased-
ness of / ~ requires
A X = 0, (10.20)
so t hat / ~ - / 3 = [A + ( X ' V + X ) - 1X' V+ ]e and the covari ance mat r i x of/ ~ equals
For A = O we have / ~=/ ~ in view of (10.19). Thus, using V + V V + = V + and
(10.21), we obt ai n:
~ ( ~ ) = o2( X, V+ X ) - l, (10.22)
whi ch is a general i zat i on of (2.14). The excess of (10.21) over (10.22) equals a
mul t i pl e o 2 of A VA' + A V V + X ( X ' V + X ) - 1 + ( X, V +X) - I X' V +VA' . B u t A VV +X
= 0 so t hat ~( / ~) - c V( / ~) =o 2 A VA' , which is positive semidefinite and thus
56 H. Theil
est abl i shes t hat / ~ is best l i near unbi ased. To veri fy t hat A VV+X is a zero mat r i x
we use (10.12) and (10.14) in
VV + = H DH ' H D- 1H' = HH' = I - KK' ,
wher e the l ast t wo steps are based on (10.7) and (10.8). So, usi ng (10.20) and
K' X = 0 also, we have
A VV+X = AX- AKK' X = 0 - 0 = O.
The mat r i x ~( ( ~) - ~( ~) = o2A VA' is obvi ousl y zero when we select A = 0, but
it ma y also be zero for A ~ 0 when V is singular, whi ch suggests t hat t here is no
uni que best l i near unbi ased est i mat or of ft. Thi s is not true, however; i f t he
est i mat or (10.11) exists, i.e. if H' X has full col umn rank, it is t he uni que best
l i near unbi ased est i mat or of fl when K' X = 0. The r eason is t hat A VA' = 0 is
equi val ent to E[Ae( Ae) ' ] = 0 so t hat Ae is a zero vect or wi t h uni t pr obabi l i t y.
Usi ng (10.20) also, we obt ai n Ay = A( Xf l + e ) = 0, whi ch in conj unct i on wi t h
(10.19) shows t hat the best l i near unbi ased est i mat or of fl must be of the f or m
(10.11), even t hough A ma y be a non- zer o mat ri x.
10.4. Deleting an equation f rom an allocation model
The Moor e - Pe nr os e inverse can also be conveni ent l y used to pr ove t hat when we
est i mat e an N- equat i on al l ocat i on syst em such as (6.1), we can si mpl y del et e one
of t he N equat i ons (it does not mat t er which). The clue is t he fact t hat each
equat i on can be obt ai ned by addi ng t he N- 1 others. We pr ove t hi s bel ow f or an
al l ocat i on syst em which is l i near in t he par amet er s. The st rat egy of t he pr oof will
be t o st art wi t h GLS est i mat i on of N- 1 equat i ons wi t h a non- si ngul ar dis-
t ur bance covar i ance mat ri x, fol l owed b y addi ng t he del et ed equat i on (so t hat t he
di st ur bance covari ance mat r i x becomes singular), and t hen pr ovi ng t hat the
resul t i ng est i mat or (10.11) is i dent i cal to the ori gi nal GLS est i mat or.
We can f or mul at e the pr obl em in t he fol l owi ng mor e gener al way. Let y = Xf l + e
have a non-si ngul ar covari ance mat r i x ~ ( e ) = o 2V of or der n X n. We pr emul t i pl y
by a mat r i x B of order (n + n' ) X n and r ank n:
By = BXf l + Be. (10. 23)
For exampl e, t ake B' = [ I C], whi ch means t hat we add to t he ori gi nal n
obser vat i ons n' l i near combi nat i ons of these observat i ons. The covar i ance mat r i x
of Be t akes the singular f or m a2BVB '. Thus, the mat r i x V of the previ ous
subsect i on becomes B VB' here, while X becomes BX. We concl ude t hat condi t i on
Ch. 1: Linear Algebra and Matrix Methods 57
K! X = 0 i s n o w K' ( B X ) = 0, whe r e K i s a ma t r i x whos e n' c o l u mn s ar e c h a r a c t e r -
i s t i c ve c t or s of B V B ' c o r r e s p o n d i n g t o t he z e r o r oot s : ( B V B ' ) K = 0 a n d K' K = I.
Evi de nt l y, a s uf f i c i e nt c o n d i t i o n f or K i s B ' K = 0 a n d K' K = I. Suc h a K c a n be
o b t a i n e d as a ma t r i x whos e c o l u mn s a r e c h a r a c t e r i s t i c ve c t or s of t he i d e mp o t e n t
ma t r i x I - B ( B ' B ) - I B ' c o r r e s p o n d i n g t o t he uni t r o o t s :
[ 1 - n ( e ' n ) - ' e ' ] x = x .
Th e GLS e s t i ma t o r (10. 11) of B i n (10. 23) i s t hen
[ X' B' ( BVB' ) + B X ] - ' X ' B ' ( n v n ' ) + ny . ( 10. 24)
Thi s i s i d e n t i c a l t o ( X ' V - I X ) - ~ X ' V - ~ y , a n d he nc e t o t he GL S e s t i ma t o r ob-
t a i n e d f r om t he or i gi na l n o b s e r v a t i o n s , b e c a u s e
B ' ( B V B ' ) + B = V 1,
whi c h f ol l ows f r o m BVB' ( BVB' ) +BVB ' = BVB' [see (10. 1)] p r e mu l t i p l i e d b y
V- 1 ( B' B) - 1 B' a n d p o s t mu l t i p l i e d b y B( B ' B ) - 1V- 1. I t i s u n n e c e s s a r y t o c he c k
t he c o n d i t i o n t ha t H' ( B X ) ha s f ul l c o l u mn r a nk, H b e i n g a ma t r i x whos e n
c o l u mn s a r e c h a r a c t e r i s t i c ve c t or s of BVB' c o r r e s p o n d i n g t o t he pos i t i ve r oot s .
Th e r e a s o n i s t ha t t he e s t i ma t o r (10. 24) wo u l d not exi s t i f t he c o n d i t i o n wer e n o t
s at i s f i ed, wh e r e a s we k n o w t ha t (10. 24) e qua l s ( X ' V - 1X) - I X' V- ly.
Appendix A: Linear independence and related topics
Co n s i d e r a ma t r i x V= [ vL. . . vn] a n d a l i ne a r c o mb i n a t i o n Vc of i t s n c ol umns .
Th e vect or s v 1 . . . . , v n a r e s a i d t o b e linearly independent i f Vc = 0 i mpl i e s c = 0, i . e.
i f t he r e exi st s no n o n - t r i v i a l l i ne a r c o mb i n a t i o n of t he ~ ' s whi c h i s a zer o vect or .
F o r e xa mpl e , t he c o l u mn s of t he 2 2 uni t ma t r i x a r e l i n e a r l y i n d e p e n d e n t
b e c a u s e
imp.e
b u t v 1 = [1 0]' a n d v 2 = [2 0]' a r e n o t l i n e a r l y i n d e p e n d e n t b e c a u s e cLv 1 + c2v 2
= 0 i f ( f or e x a mp l e ) c I = 2 a n d c 2 = - 1.
F o r a ny m n ma t r i x A t he c o l u mn r a n k i s d e f i n e d as t he l a r ge s t n u mb e r of
l i n e a r l y i n d e p e n d e n t c ol umns , a n d t he r ow r a n k as t he l a r ge s t n u mb e r of l i ne a r l y
i n d e p e n d e n t r ows. I t c a n b e s hown t ha t t hes e t wo r a n k s a r e a l wa ys equal ; we c a n
t hus s pe a k a b o u t t he rank r of A, whi c h o b v i o u s l y sat i sf i es r ~ m, n. I f al l
58 11, Thei l
col umns (rows) of A are l i nearl y i ndependent , A is said t o have full col umn (row)
rank. For any A, t he r anks of A , A' , A ' A , and A A ' are all equal. Also, the r ank of
A B is at mos t equal to t he r ank of A and t hat of B. For exampl e,
whi ch illustrates t hat the r ank of A B ma y be smal l er t han bot h t hat of A and t hat
of B. (A zero mat r i x has zero rank. ) I f A is squar e (n x n) and has full r ank
( r = n), it is called non- s i ngul ar and its i nverse A 1 exists.
For any vect or v = [v~], its l engt h is defi ned as the posi t i ve squar e r oot of
v ' v = ~ v 2. I f v ' v = 1, v is sai d to have uni t l engt h. The i nner pr oduct of t wo
vect ors v = [v,] and w = [w~] consi st i ng of t he same numbe r of el ement s is def i ned
as v ' w = ~ i viwi- I f v' w = O, v and w are cal l ed ort hogonal vect ors.
A square mat r i x X whi ch satisfies X' = X- ~ is called an ort hogonal ma t r i x .
Pr emul t i pl i cat i on of X' = X- l by X gives X X ' = I , whi ch shows t hat each r ow of
X has uni t l engt h and t hat any two rows of X are or t hogonal vect ors. Post mul t i -
pl i cat i on of X' = X- ~ by X gives X ' X = ! so t hat each col umn of X (each r ow of
X' ) also has uni t l engt h and any t wo col umns of X are also or t hogonal vect ors.
Appendix B: The independence transformation
The i ndependence t r ansf or mat i on is based on t hree axi oms, the first bei ng t he
i nvar i ance of t ot al expendi t ure. Let a dol l ar spent on obser ved i n p u t j resul t in ris
dol l ars spent on t r ans f or med i nput i, so t hat t he expendi t ur e on i equal s ~ i r~ i Pi qi
and the t ot al expendi t ur e on all t r ans f or med i nput s equal s E j ( E i r i j ) p j q j wl f l c h
mus t be i dent i cal to ~ j p j q j because of t he i nvar i ance post ul at ed. Ther ef or e,
~ i r i j = 1 f or each j , or
,'R = , ' , (B. 1)
wher e R = [rij ]. By dividing the expendi t ur e ~ _ a j F i j p j q j on t r ans f or med i nput i by
t ot al expendi t ur e C (which is i nvari ant ) we obt ai n the f act or share f x i of this
i nput . Therefore, fTi = Y~i rijfJ ' or
eT, = Re, , (B.2)
where F T is the di agonal f act or share mat r i x of t he t r ansf or med i nput s.
The second axi om st at es t hat the l ogar i t hmi c pri ce and quant i t y changes of t he
t r ans f or med i nput s are l i near combi nat i ons of t hei r obser ved count er par t s, ~r T =
Slur and IT = S2K, so t hat t he associ at ed Di vi si a i ndexes are i nvari ant . The Di vi si a
Ch. 1: Linear Algebra and Matrix Methods 59
vol ume i ndex is d ( l o g Q) = , ' Fx and its t r ansf or med count er par t is dFTgT =
d F T S 2 ~ =d F ( R ' S 2 ) x [see (B.2)]. Thus, the i nvari ance of this i ndex requires
R ' S 2 = I or S 2 = ( R' ) - 1. We can pr oceed similarly for t he pri ce i ndex dF~r, whi ch
yields the same result for S~, so t hat t he pri ce and quant i t y t ransformat i ons use
t he same matrix, ,r T = Srr and K T = Sg, where S = ( R' ) - 1. See r emar k (3) bel ow
f or t he case of a singular R.
The third axi om diagonalizes O. We premul t i pl y (8.15) by R, which yields
R F x = R F R ' S ~ = RFR' K T on t he left because R ' S = I and SK = K x. When we
pr oceed similarly on the ri ght and use (B.1) also, we obt ai n:
R F R ' g m = ( dFK ) ( R~ g R' ) , - 6 R~g R' [ I - t d( R@R' ) ] efT, ( B . 3 )
whi ch is an al l ocat i on syst em of t he same f or m as (8.15), with l ogari t hmi c pri ce
and quant i t y changes Ir T and XT, pr ovi ded R F R ' on t he left equals t he di agonal
f act or share mat r i x F T. The new normal i zed price coeffi ci ent mat r i x is R OR ' ,
whi ch occurs in t he same t hree places in (B.3) as O does in (8.15). [The mat r i x
R OR ' is i ndeed normal i zed because d ROR' L = t'OL = 1 follows f r om (B.1).] There-
fore, R F R ' = F T and R OR ' = di agonal are t he condi t i ons under whi ch (B.3) is an
i nput i ndependent al l ocat i on system. These are two condi t i ons on R, which must
satisfy (B.1) also.
We proceed t o pr ove t hat
R : ( X- l , ) a X ' (B. 4)
satisfies these t hree condi t i ons, with X defi ned in (8.16) and ( X lt)a in (7.9).
First, dR = d is t rue for (B.4) i n view of (7.10). Secondly, R F R ' =
( X - I Oa X ' F X ( X - l t ) a = ( X- 1 L ) ~ [see (8.16)] so that
R F R ' = F T = ( X - t , ) ~ = diagonal. ( B . 5 )
Thi rdl y, using @ = ( X ' ) - l A X i [see (8.16)], we have R ~ k R ' = ( X 1L)a2A, which is
diagonal. So, using (B.5) also and premul t i pl yi ng (B.3) by ( R F R ' ) - 1 = ( X - I L ) S 2 ,
we obt ai n:
x T = ( d F x ) A t - +A ( I - tdR~gR' ) ~rT, (B. 6)
whi ch is t he mat r i x version of (8.17). The expressi on which is subt ract ed in
parent heses in t he subst i t ut i on t er m of (B.6) represent s t he defl at i on by the Fri sch
pri ce index, whi ch is invariant. To pr ove this we not e t hat the margi nal share
vect or of t he t r ansf or med i nput s equals ROt = RO in view of t he real -i ncome t er m
in (B.3) and R' t = L; t he invariance of t he Frisch i ndex t hen follows f r om
( R O ) ' ~ r T = O' R ' ST r = O' er.
60 H~ Theil
The expendi t ur e on t r ansf or med i nput i equals r ~j pj qj dollars i nsofar as it
originates with observed i nput j . By dividing this amount by t ot al expendi t ur e C
we obt ai n %f j , whi ch is thus the fact or share of t r ansf or med i nput i i nsofar as it
originates wi t h observed i nput j . Thi s r~jfj is an el ement of t he mat r i x R F , t o be
wri t t en T:
T=RF= ( X- 11) AX -1 , (B. 7)
where the last step is based on (B.4) and F = ( X' ) - IX 1 [see (8.16)]. Post mul t i -
pl i cat i on of (B.7) by t gives TL = R F t = FTt [see (B.2)]; hence t he r ow sums of T
are t he f act or shares of the t r ansf or med inputs. Also, ~'T = t ' R F = ~'F, so t hat t he
col umn sums of T are the f act or shares of t he observed inputs. Not e t hat (B.7)
and its row and col umn sums conf i r m t he results on t he composi t i on matrix. Not e
f ur t her t hat F = ( X ' ) - I X - 1 a n d @ = ( X ' ) - I A X - l [see (8.16)] i mpl y t hat t he pri ce
elasticity mat r i x - ~F- 1@ in (8.19) equals - ~ p X A X 1. So, using (B.7) also, we
have
r ( - r-1o) = - + ( x- ' , ) a a x = - = - +a t .
Combi ni ng the first and last member yields t ~( - ~pF 10) = - ~p?~it~,.where t~ is
t he i t h row of T, or
t ; [ - ~ p e - ' O - ( - ~ p ) ~ i ) I ] =O.
Therefore, each row of t he composi t i on mat r i x is a charact eri st i c row vect or of t he
(asymmet ri c) pri ce elasticity mat ri x of t he observed inputs.
We concl ude wi t h the following remarks.
(1) Al t hough the solution (B.4) satisfies all t hree condi t i ons, it is not uni que.
However, it may be shown t hat this sol ut i on is uni que up to premul t i pl i cat i on by
an ar bi t r ar y per mut at i on matrix; such a mul t i pl i cat i on affects onl y t he or der in
whi ch the t r ansf or med i nput s are listed.
(2) We pr oved in the second par agr aph t hat t he pri ce and quant i t y t r ansf or ma-
t i ons take the f or m ~r T = S~r and gT = S, where S = ( R' ) - 1. It t hus follows f r om
(B.1) t hat S- I ~ = ~ or St = L. Therefore, when t he prices of the observed i nput s
change proport i onat el y, ~r bei ng a scalar mul t i pl e k of L, t he pri ce of each
t r ansf or med i nput changes in the same pr opor t i on: ~r a- = S ( k t ) = k SL = k t . The
quant i t i es have t he same desirable pr oper t y.
(3) It follows f r om (B.4) t hat R is singular when ( X- l~)a cont ai ns a zero di agonal
element, and f r om (B.5) t hat this implies a zero f act or share of one of t he
t r ansf or med inputs. In t hat case S = ( R' ) - 1 does not exist. The simplest way to
i nt erpret this si t uat i on is by means of a per t ur bat i on of t he fi rm' s t echnol ogy so
Ch. 1: Linear Algebra and Matrix Methods 61
t hat t he i t h el ement of X- 1~ converges f r om a smal l non- zer o val ue to zero. I t ma y
be shown t hat d(l og Pa-i) t hen i ncreases beyond bounds. I f the i ncrease is t owar d
oo, t r ansf or med i nput i is pri ced out of t he mar ket ; if it is t owar d - oo, i becomes
a free good; in bot h cases no money is spent on i i n t he limit. I n part i cul ar, if
(5.12) represent s a homot het i c t echnol ogy, N- 1 el ement s of X-1L are zero and all
obser ved i nput s collectively behave as one t r ans f or med i nput wi t h uni t ar y Di vi si a
elasticity; no money is spent on any t r ans f or med i nput whose Di vi si a elasticity
di ffers f r om 1. For pr oof s of these results see Theil (1977).
(4) The i ndependence t r ans f or mat i on was first f or mul at ed by Br ooks (1970) and
axi omat i cal l y j ust i fi ed by Thei l (1975-76, ch. 12) f or a fi ni t e-change versi on of
t he consumer de ma nd syst em (5.22). The )t i ' s are t hen i ncome elasticities of
t r ans f or med consumer goods. Rossi (1979a) pr oved t hat when all obser ved goods
are specific subst i t ut es, t he t r ans f or med good wi t h t he smal l est i ncome elasticity
represent s all obser ved goods posi t i vel y and t hat all ot her t r ans f or med goods are
cont r ast s bet ween obser ved goods si mi l ar to T 2 in (8.26). The f or mer t r ansf or med
good serves t o sat i sfy t he consumer ' s want s associ at ed wi t h the obser ved goods in
t he l east l uxuri ous manner ; this resul t is of par t i cul ar i nt erest when the t r ansf or -
mat i on is appl i ed to a gr oup of goods whi ch sat i sfy si mi l ar want s such as di fferent
br ands of the same t ype of commodi t y. 9 For an i nt egr at ed exposi t i on of t he
i ndependence t r ans f or mat i on in cons umpt i on and pr oduct i on t heor y see Thei l
(1980, ch. 10-11).
Appendix C: Rational random behavior
To veri fy (9.4) we wri t e p * ( x ) = p ( x ) + 6 f ( x ) for some densi t y funct i on ot her
t hat the p ( x ) of (9.4), where 8 is i ndependent of x so t hat f ( . ) mus t satisfy
f J ( x ) d x , . . . d x k = O .
The i nf or mat i on I * and t he expect ed loss l * associ at ed wi t h p*( - ) are
(C. 1)
1" = f j [ p ( x ) + 6 f ( x ) ] l o g p ( x ) + 6 f ( x )
p o ( x ) dXl . . . d x ~ ,
[* = [ + 6 f j l ( x , Yc) f ( x ) d x , . . . d X k ,
( c . 2 )
( c . 3 )
9When O is block-diagonal, so is X in (8.16), which means that the independence transformation
can be applied to each block separately. We have a block-diagonal O under block independence. See
the end of Section 6 for block independent inputs; the extension to block independent consumer
goods is straightforward.
62 H. Theil
where [ is the expected loss (9.3) associated with the p ( . ) of (9.4). We appl y a
Tayl or expansion t o (C.2) as a funct i on of &
I* = I + k, 3 + k232 + O( 33) , (C. 4)
where I is the i nformat i on (9.2) associated wi t h (9.4) and
p ( x )
k, = f f ( x) l og p o - ~ dx 1 . . . d x k , (C. 5)
k2 = f j [ f ( x ) ] ~ dx 1 ...dxz~. (C. 6)
Next we appl y a Tayl or expansion to c(I*), writing c ' = d c / d I and c" = d 2 c / d I 2
for the derivatives of c( . ) at the I of (9.4):
c( I*) = c( I ) + 3k l C' + 1 3 2 ( k2ct + k 2 c t ' ) -~- 0 ( 3 3 )
and we add this to (C.3):
c ( I * ) + [ * = c ( I ) + [ + 3 [ k l c ' + f J ( x , ~ ) f ( x ) d x l . . . d x k ]
+ 3 2 ( k 2 c ' + + o ( 3 3 ) ( c . 7 )
For c ( I ) + [ to be minimal we require the coefficient of 3 in (C.7) to vanish for
any f ( . ) satisfying (C.1) and t hat of 32 to be positive. The latter condi t i on is
satisfied when c ' > 0 and c">~ 0 (a positive nondecreasing margi nal cost of
i nformat i on) because (C.6) implies k 2 > 0 when f ( x ) * 0 for some x. It follows
from (C.5) t hat the former condi t i on amount s to a zero value of
This integral vanishes, given (C. 1), when the expression in brackets is a const ant
i ndependent of x, which yields (9.4) directly.
To prove the asympt ot i c results for small c' we take the logarithm of (9.4):
l o g p ( x ) = cons t ant +l ogpo( X ) l ( x , Yc)
c ' ' ( c . 8 )
and substitute ~ for x, using (9.1):
log p (X:) = const ant + log P0 ( ~ ).
Ch. 1: Linear Algebra and Matrix Methods
Since the const ant s in these equations are equal, subt ract i on yields
p ( 2 ) , po(YC) l ( x , 2 )
og P ( x) = log ~ + c'
63
(C. 9)
It follows from (9.1) t hat as c ' ~ 0 the last term increases beyond bounds for any
x * ~, so t hat the same holds for p ( ~ ) / p ( x ) on the left. Hence, as c ' ~ 0 the
densi t y p ( x ) becomes zero for each x * 2 and the r andom decision with density
funct i on (9.4) t hus converges in probabi l i t y to ~.
To verify the asympt ot i c di st ri but i on (9.7), we define
1
v = ~ c , ( x - ~) , (C.10)
so t hat l ( x , ~ ) = l ( Yc + vrc' v , ~ ). We appl y a Tayl or expansi on to l ( x , Yc ) / c' , using
(9.6):
l ( x , ~ ) 1 r i , rr7
, = t tw
= v ' A v + O( x/ ~) . (C.11)
We assume t hat p o ( x ) is positive and differentiable around 2. Hence, we can
appl y a Tayl or expansion to l ogp0( x ) and write it as logp0(.~ ) plus a linear
remainder term i n x - i . Therefore, in view of (C.10),
log p0( x ) -- l ogp0( ~) + O(vCeT),
which in conj unct i on wi t h (C.8) and (C. 11) shows t hat log p ( x ) equals a const ant
mi nus v ' Av plus two remai nder terms which bot h converge to zero as c' ~ 0. The
result (9.7) is t hen obt ai ned by substitution from (C.10) for v in l v ' A v .
We obt ai n (9.11) from (9.7) by using the budget or t echnol ogy constraint to
eliminate one of the decision variables from the criterion function. Let these
variables be the quantities bought by the consumer; it was shown by Theil
(1975-76, sec. 2.6-2.7) t hat (9.7) t hen yields variances and covariances of the
form
_ k ( k u , j X Oqi Oqj t
cov(qi, q j ) =
t
OX/OM OM OM]'
(C.12)
where k > 0 is proport i onal to the marginal cost of i nformat i on c' . A compari son
of (C.12) with (3.12) shows t hat cov(qi, q j ) is proport i onal to the substitution
64 H. Theil
component (specific pl us general) of Oqi /Opj . We obtai n (9.11) from (C.12) by
rearrangements required by the left variable in (5.5).
The results (9.11) and (9.14) for the mul ti product firm and the stochasti c
i ndependence of the i nput demand disturbances and the out put suppl y dis-
turbances were derived by Laitinen and Theil (1978). Reference shoul d al so be
made to Bowman et al. (1979) and to Rossi (1979b, 1979c) for a compari son of
rational random behavior and search theory.
References
Altken, A. C. (1935) " On Least Squares and Li near Combi nat i on of Observations", Proceedings of the
Royal Society of Edinburgh, 55, 42-48.
Albert, A. (1972) Regression and the Moore-Penrose Pseudoinverse. New York: Academic Press.
Barbosa, F. de H. (1975) "Rat i onal Random Behavior: Extensions and Applications", Doct oral
dissertation, The University of Chicago.
Barten, A. P. (1964) "Consumer Demand Funct i ons under Condi t i ons of Almost Additive Prefer-
ences", Econometrica, 32, 1-38.
Barren, A. P. (1977) "The Systems of Consumer Demand Funct i ons Approach: A Review",
Econometrica, 45, 23-51.
Barten, A. P. and E. Geyskens (1975) "The Negativity Condi t i on in Consumer Demand", European
Economic Review, 6, 227-260.
Bdl man, R. (1960) Introduction to Matrix Analysis. New York: McGraw-Hill Book Company.
Ben-Israel, A. and T. N. E. Greville (1974) Generalized Inverses: Theory and Applications. New York:
John Wiley and Sons.
Boullion, T. L. and P. L. Odell (1971) Generalized Inverse Matrices. New York: John Wiley and Sons.
Bowman, J. P., K. Laitinen and H. Theil (1979) " New Results on Rational Random Behavior",
Economics Letters, 2, 201-204.
Brooks, R. B. (1970) "Di agonal i zi ng the Hessian Matrix of the Consumer' s Ut i l i t y Funct i on",
Doctoral dissertation, The University of Chicago.
Brown, A. and A. Deat on (1972) "Surveys i n Applied Economics: Models of Consumer Behaviour",
Economic Journal, 82, 1145-1236.
Dhrymes, P. J. (1978) Mathematics for Econometrics. New York: Springer-Verlag.
Divisia, F. (1925) "L' i ndi ce mon~taire et la th~orie de la monnai e", Revue d'Economie Politique, 39,
980-1008.
Friseh, R. (1932) New Methods of Measuring Marginal Utility TiJbingen: J. C. B. Mohr.
Goldberger, A. S. (1959) Impact Multipliers and Dynamic Properties of the Klein-Goldberger Model.
Amsterdam: Nort h-Hol l and Publishing Company.
Graybill, F. A. (1969) Introduction to Matrices with Applications in Statistics. Belmont, Cal.: Wads-
wort h Publishing Company.
Hadley, G. (1961) Linear Algebra. Reading, Mass.: Addison-Wesley Publishing Company.
Hall, R. E. (1973) " The Specification of Technology with Several Ki nds of Out put ", Journal of
Political Economy, 81, 878-892.
Houthakker, H. S. (1960) "Addi t i ve Preferences", Econometrica, 28, 244-257; Errata, 30 (1962) 633.
Kadane, J. B. (1971) "Compar i son of k-Class Estimators when the Di st urbances are Small",
Econometrica, 39, 723-737.
Laitinen, K. (1980) A Theory of the Multiproduct Firm. Amsterdam: Nort h-Hol l and Publishing
Company.
Laitinen, K. and H. Theil (1978) "Suppl y and Demand of the Mul t i produet Fi rm", European
Economic Review, 11, 107-154.
Laitinen, K. and H. Theil (1979) " The Antonelli Mat ri x and the Reciprocal Slutsky Matrix",
Economics Letters, 3, 153-157.
Ch. 1: Linear Algebra and Matrix Methods 65
Phlips, L. (1974) Applied Consumption Analysis. Amsterdam: North-Holland Publishing Company.
Pringle, R. M. and A. A. Rayner (1971) Generalized Inverse Matrices with Applications to Statistics.
London: Charles Griffin and Co.
Rao, C. R. and S. K. Mitra (1971) Generalized Inverse of Matrices and Its Applications. New York:
John Wiley and Sons.
Rossi, P. E. (1979a) "The Independence Transformation of Specific Substitutes and Specific Comple-
ments", Economics Letters, 2,299-301.
Rossi, P. E. (1979b) "The Cost of Search and Rational Random Behavior", Economics Letters, 3, 5-8.
Rossi, P. E. (1979c) "Asymptotic Search Behavior Based on the Weibull Distribution", Economics
Letters, 3, 211-213.
Theil, H. (1967) Economics and Information Theory. Amsterdam: North-Holland Publishing Company.
Theil, H. (1971) Principles of Econometrics. New York: John Wiley and Sons.
Theil, H. (1975-76) Theory and Measurement of Consumer Demand, 2 vols. Amsterdam: North-
Holland Publishing Company.
Theil, H. (1977) "The Independent Inputs of Production", Econometrica, 45, 1303-1327.
Theil, H. (1980) The Syst em-Wi de Approach to Microeeonomics. Chicago: University of Chicago Press.
Theil, H. and K. W. Clements (1980) "Recent Methodological Advances in Economic Equation
Systems", American Behavioral Scientist, 23, 789-809.
Theil, H. and K. Laitinen (1979) "Maxi mum Likelihood Estimation of the Rotterdam Model under
Two Different Conditions", Economics Letters, 2, 239-244.
Zellner, A. (1962) " An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for
Aggregation Bias", Journal of the American Statistical Association, 57, 348-368.