Notations:

x, d, b, etc, that is characters in boldface represent (column) vectors.

ai , the ith column of A.
Given a linear programming problem
Max cT x
(1)
subject to Amn x bm1 , x 0.
Let us denote the set {x Rn : Ax b, x 0} by F ea(P ).
The Dual of this problem is given by
Min bT x
(2)
subject to ATnn y cn1 , y 0.
Let us denote the set {y Rm : AT y c, y 0} by F ea(D).
The linear programming problem (1) is called the Primal problem.
Theorem 1: If x F ea(P ) and y F ea(D), then cT x bT y.
Proof: Since Ax b and y 0,
yi (Ax)i yi bi for allPi = 1, 2, ..., m. P
m
T
This implies, yT Ax = m
i=1 yi bi = b y
i=1 yi (Ax)i
T
Also since A y c and x 0, we can similarly get
xT AT y xT c = cT x
(ii)
(i) and (ii) together bT y yT Ax = xT AT y cT x.

(i)

Corollary 1: Let x0 F ea(P ) and y0 F ea(D), be such that cT x0 = bT y0 , then x0 and y0

are optimal for the Primal and the Dual, respectively.
Proof: If x is any feasible solution of the Primal, then since y0 F ea(D), from Theorem 1 we
get
cT x bT y0 = cT x0 .
Hence the result.
Let us again consider the problem:
Example 1: Max 5x + 2y
subject to
3x + 2y 6
x + 2y 4
x 0, y 0.
If we can get a feasible solution w of the dual of this problem such that cT z = bT w, where
c = [5, 2]T , b = [6, 4]T and z = [2, 0]T (already obtained before), then using the above corollary,
we will again be able to conclude that z is indeed optimal for this LPP (we had already seen this
before). Note that cT z = 5 2 + 2 0 = 10.
The dual of this problem is given by,

Min 6y1 + 4y2

subject to
3y1 + y2 5
2y1 + 2y2 2,
y1 0, y2 0.
Check that w = [1, 1]T is a feasible solution of the dual and bT w = 6 1 + 4 1 = 10. Hence
z = [2, 0]T , is optimal for the primal and w = [1, 1]T is optimal for the dual.
Theorem 2: ( Fundamental Theorem of Duality ) If both the Primal and the Dual have
feasible solutions then both have optimal solutions and the optimal value is equal ( that is Min
bT y = Max cT x). If one of them (the Primal or the Dual) does not have a feasible solution then
the other does not have an optimal solution.
Proof: Note that the first part of the theorem is easy to see, that is if both the primal and the
dual have feasible solutions, then both have optimal solutions.
Let y0 be a feasible solution of the dual, then for all x F ea(P )
cT x bT y0 ,
but this implies that there cannot exist a direction d of (P) such that cT d > 0 (we have understood
this before). Since (P) is a maximization problem, this implies that (P) has an optimal solution.
Similarly if x0 is a feasible solution of the primal (P), then for all y F ea(D),
bT y cT x0 ,
hence there cannot exist a direction d0 of F ea(D) such that bT d0 < 0, which implies that (D) has
an optimal solution.
We need to show that the optimal value is same, when both have optimal solutions.
Also we need to show that if one of (P) or (D) does not have a feasible solution then the other does
not have an optimal solution.
We will do these remaining parts of the proof later.
Let us first check the validity of the result of the above theorem through some examples.
Example 2:
Min x + 2y
subject to
x + 2y 1
x + y 1,
x 0, y 0.
We had already seen that this problem does not have an optimal solution although it has a feasible
solution.
The dual of this problem is given by
Max u v
subject to
u + v 1
2u v 2,
u 0, v 0.
Clearly this problem does not have any feasible solution, since the first constraint cannot be satisfied by any nonnegative u and v.
Example 3:
Max x + 2y
2

subject to
x + 2y 1
x + y 1,
x, y 0.
We have seen before that this problem does not have a feasible solution, hence does not have an
optimal solution.
The dual of the above problem is given by
Min u v
subject to
u + v 1
2u v 2
u 0, v 0.
Check that the above problem has a feasible solution but not an optimal solution.
In order to however prove the fundamental theorem of duality we need to look at solutions of
certain systems of equations and inequalities.
Theorem 3: Exactly one of the following systems has a solution.
Amn x = b
yT A = 01n ,

(1)

yT b = 1

(2)

Note that Ax = b has a solution means that there exists real numbers, x1 , . . . , xn , such that
1 x1 + a
2 x2 + ... + a
n xn = b, that is b lies in the plane spanned by the columns of A.
a
T
T
1 = yT a
2 = .... = yT a
n = 0, that is there exists
y A = 0, y b = 1 has a solution means that yT a
a vector y which is orthogonal to every column of A but not orthogonal to b( it actually makes an
acute angle with b).
Proof:
()
It is obvious that both (1) and (2) cannot have solutions.
This is because, if we multiply equation (1) with yT the solution of (2), then we get
yT Ax = yT b = 0, but then this contradicts that yT b = 1.
()
Suppose if (1) does not have a solution then to show that (2) has a solution.
If (1) does not have a solution then the columns of A do not span the whole of Rm . Hence
rank(A) = r < m.
WLOG let {a1 , a2 , ...., ar } be a collection of r LI columns of A then (1) does not have a solution
implies that the set {a1 , a2 , ...., ar , b} is LI.
Let Cm(r+1) be the matrix defined as Cm(r+1) = [a1 , a2 , ...., ar , b], then rank(C) = r + 1.
Since rank(C T ) = r + 1 and since the columns of C T are elements of Rr+1 , hence the m columns
of C T span the whole of Rr+1 .
So if we consider the vector z, with (r + 1) components, such that z = [0, ..., 0, 1]T ,
T
then there exists a y Rm such that C(r+1)m
y = z, that is yT C = zT = [0, ..., 0, 1]
or yT ai = 0 for i = 1, . . . , r and yT b = 1.
Note that this implies that yT ai = 0 for all i = 1, 2, . . . , n, since for all i = 1, 2, . . . , n
ai = ui1 a1 + . . . + uir ar for some real uij s, i = 1, . . . , n, j = 1, 2, . . . , r.
Hence yT ai = 0 for i = 1, . . . , r, implies yT ai = 0 for all i = 1, 2, . . . , n, which together with
yT b = 1 shows that (2) has a solution.

()
Note that if (2) does not have a solution then (1) must have a solution, since
if (1) does not have a solution then we have just proved in () that (2) must have a solution.
Hence (*), (**), (***) together proves that exactly one of the two systems has a solution.
Theorem 4: (Farkas Lemma) Exactly one of the following two systems has a solution.
Amn x = b, x 0

(1)

yT A 01n , yT b < 0

(2)

System 1 has a solution means that b belongs to the cone generated by the columns of A.
Whereas system 2 has a solution means there exists a hyperplane (with normal y) which separates
the cone generated by the columns of A and the vector b, in the sense that the cone lies in one of the
closed half spaces associated with this hyperplane and the vector b lies in the other open half space.
Proof: It is easy to check that both the two systems cannot have a solution.
If we multiply equation (1) with yT , the solution of (2) then we get
0 yT Ax = yT b, but this contradicts that yT b < 0.
The proof of the part that if (1) does not have a solution then (2) has a solution, and vice versa,
we will postpone it for later consideration.
Corollary 4: Exactly one of the following two systems has a solution.
Amn x b, x 0
yT A 0,

yT b < 0,

(1)
y0

(2)

0
Proof: Note that system

(1) has a solution if and only if system (1) given below, has a solution.
x
Amn x + u = [A : I]
= b, x 0, u 0
(1)0
u

But from the previous theorem, exactly one of the following two systems has a solution.

x
x
[A : I]
= b,
0
(1)0
u
u
yT [A : I] [01n , 01m ],

yT b < 0

(2)0

But note that system (2)0 is same as system (2).

Hence the result.
Fundamental Theorem of Duality (revisited):
Note that since F ea(D) = {y Rm : AT y c, y 0,
the set of all directions of F ea(D) is given by
DD = {d Rm : AT d 0, d 0}, and the dual (D) has an optimal solution if and only if either
F ea(D) is bounded or if not then bT d 0 for all d DD .
So from corollary 4 it is obvious that if the primal does not have a feasible solution, that is system
(1) does not have a solution, then since (2) must have a solution so there exists a y Rm such
that AT y 0, y 0 and yT b < 0, that is there exists a direction d(= y) of F ea(D) such that
dT b < 0, which implies that the dual does not have an optimal solution (the dual may or may not
have a feasible solution).
Similarly (since the dual can be converted to a problem of the same form as the primal (P)) we
4

can show that if the dual does not have a feasible solution then the primal (dual of the dual) does
not have an optimal solution.
So the proof of the Fundamental theorem of duality will be complete if we can show that if
F ea(P ) and F ea(D) are both nonempty, then their optimal values will be same ( we have already
seen that in this case both will have optimal solutions ).
So we need to show that if F ea(P ) 6= and F ea(D) 6= , then the following system (1) has a
solution:

Amn x b, x 0

(1)

AT y c, y 0
cT x = bT y
The first two inequalities correspond to F ea(P ) and F ea(D), respectively. We have already
seen that if x F ea(P ) and y F ea(D) then cT x bT y, hence system (1) given above has a
solution if and only if, system (1)0 given below has a solution:
(1)0

Amn x b, x 0
AT y c, y 0
cT x bT y
System (1)0 can be written as:

Amn 0mm
b
x
0nn ATnm
c ,
y
T
T
0
c1n b1m

x
y

0n1
0m1

(1)0 .

If system (1)0 does not have a solution then from corollary 4, system (2)0 given below has a
solution.

A
0
b
mn
mm
T

T
T
z1m w1n
a11 0nn ATnm 01n 01m ,
z1m w1n
a11 c < 0 (2)0
0
cT1n bT1m

T
zT1m w1n
a11

0m+n+1

If we simplify the inequalities in system (2)0 , then we get the following:

zT A acT 0, zT 0, a 0
AT z acT , z 0, a 0

or
(i)

wT AT + abT 0, wT 0, a 0
or
Aw ab, w 0, a 0
(ii)
zT b < w T c

z
a

(iii)

Case1: If a > 0, then from (i) and (ii) it follows that

F ea(D) and w
a F ea(P ), but this contradicts (iii).

Case 2: If a = 0, then from (i) and (ii) it follows that

zT A 0, zT 0 and wT AT 0, wT 0
(iv)
Since F ea(P ) 6= , F ea(D) 6= , let x F ea(P ) and y F ea(D),
hence Ax b, x 0 and AT y c, y 0.
(v)
Then from (iv) and (v) after appropriate multiplication we get
0 zT Ax zT b and 0 wT AT y wT c which contradicts (iii).
Hence in both the two cases system (2)0 does not have a solution, hence system (1)0 has a solution.
Hence the proof of the Fundamental Theorem of Duality is complete.
Interpretation of the Dual and the Complementary Slackness Theorem (or Equilibrium Theorem).
Now P
consider again the diet problem which is of the form
Min ni=1 ci xi
subject
to
Pn
a
xj bi , for all i = 1, 2, ..., m,
ij
j=1
xj 0 for all j = 1, 2, . . . , n.
The
P dual of the above problem is given by
Max m
i=1 bi yi
subject
to
Pm
T
a
i=1 ij yi cj , j = 1, 2, ..., n, or Anm ym1 cn1 ,
yi 0 for all i = 1, 2, . . . , m, or y 0.
Note that here the yi s should correspond to some type of cost or value associated with the ith
nutrient, since the right hand side of the constraints of the dual give costs and aij s give quantity
of nutrients in unit amount of Fj . For example if there were tablets or pills of the corresponding
nutrients each tablet corresponding to unit amount of the nutrient, then yi could be thought of as
price of one pill of the ith nutrient. So if the jth food product, Fj was to be substituted by pills
of nutrients Ni , then in order to get the same value of nutrients as one unit of Fj , one will have to
take
(a1j amount of N1 )+( a2j amount of N2P
)+....+(amj amount of Nm ),
the total cost of which comes out to be m
i=1 aij yi .
So a seller of these nutrient
price of these pills so as to maximize his return or
P pills can fix theP
m
profit which is given by m
b
y
as
long
as
i
i
i=1
i=1 aij yi does not exceed the market cost of a unit
amount of Fj given by cj , for all j = 1, 2, . . . , n.
If the Primal (P) is given by
Max cT x
(1) subject to Ax b, x 0.
We have already seen that the Dual (D) of this problem is given by
Min bT x
(2) subject to AT y c, y 0.
Theorem 5 ( Complementary Slackness Theorem ): Let x F ea(P ) and y F ea(D),
then x and y are optimal for the primal and the dual respectively if and only if
xj = 0 whenever (AT y)j > cj , j = 1, 2, .., n (1)
and
yi = 0 whenever (Ax)i < bi , i = 1, 2, ..., m. (1 )
Proof: If part.
6

If (1) and (1) holds, then from (1)

xj (AT y)
= 1, 2, ..., n.
Pj = xj cj for all j P
Hence nj=1 xj (AT y)j = nj=1 xj cj .
P
P
Hence xT AT y = nj=1 xj (AT y)j = nj=1 xj cj = cT x.
(2)
Similarly from (1) we get
yi (Ax)i = yi bi for
m.
P all i = 1, 2, ...,P
m
T
Hence yT Ax = m
y
(Ax)
=
(2*)
i
i=1 i
i=1 yi bi = b y.
T
T
T
Since x A y = y Ax,
(2) and (2*) together implies cT x = bT y, hence from corollary 1 we can conclude that x and y
are optimal for (P) and (D), respectively.
Only If part.
On the other hand if x and y are optimal then by the fundamental theorem of duality the
optimal values should be equal, hence yT b = xT c.
Also since x F ea(P ) and y F ea(D), we get the following after appropriate multiplication,
xT c xT AT y = yT Ax yT b = xT c.
Hence xT c = xT AT y or xT (c AT y) = 0,
and yT Ax = yT b or yT (Ax b) = 0.
Since x 0 and c AT y 0,
xT (c AT y) = 0 xj ((AT y)j cj ) = 0 for all j = 1, 2, ..., n.
Similarly since y 0 and b Ax 0,
yT (Ax b) = 0 yi ((Ax)i bi ) = 0 for all i = 1, 2, ..., m.
Hence we get (1) and (1)*.
If we look at the application of this theorem to the diet problem then condition (1*) tells us
that if the cost of the nutrients in a unit amount of Fj taken together, obtained by taking pills of
the nutrients, is strictly less than the unit cost of Fj , that is cj , then xj = 0 or Fj need not be used
in the diet. Similarly (1) says that if the amount of Ni obtained from all the food products taken
together exceeds the required amount bi , then no one will buy the pill of nutrient Ni , so its cost
given by yi would eventually drop to 0.
If x F ea(P ) and y F ea(D) satisfy condition (1) and (1)*, then x F ea(P ) and y F ea(D)
is said to satisfy the complementary slackness property.
The following example illustrates how complementary slackness theorem can be used to solve
certain (not so complicated problems).
Example 1: Consider the following primal problem (P):
Max 4x1 + 4x2 + 2x3
subject to
2x1 + 3x2 + 4x3 10
(i)
2x1 + x2 + 3x3 4
(ii)
x 1 , x2 , x3 0
The dual (D) of the above problem is given by:
Min 10y1 + 4y2
subject to
2y1 + 2y2 4
(i)0
3y1 + y2 4
(ii)0

3y1 + 4y2 2
y1 , y2 0

(iii)0

Let us try to obtain an optimal solution of the primal by starting with simple feasible solutions
of the primal (P) and trying to find feasible solutions of the dual satisfying complementary slackness
property with the feasible solutions of the primal.
Note that since the primal (P) is a maximization problem, and the increase in the value of the objective function for unit increase in x1 and x2 is twice as compared to x3 . Also from the inequalities
it can be checked that the maximum value of x3 is 43 , whereas the maximum value of x1 is 2 and
the maximum value of x2 is 10
3 . Hence if only one of the variables is to remain positive, then it is
more sensible to take either x1 or x2 to be positive.
So let us start by taking an x F ea(P ) such that x1 > 0 and x2 = x3 = 0, then note
that inequality (i) of the primal cannot be satisfied as an equality by x, hence if x F ea(P )
and y F ea(D) satisfy the complementary slackness property, then y1 = 0. Also since
x1 > 0, if x and y satisfy the complementary slackness property, then 2y1 + 2y2 = 4, which
gives y2 = 2, but then inequality (ii)0 of the dual cannot be satisfied by y. Hence there exists
no y = [y1 , y2 ]T F ea(D) which satisfies complementary slackness property with the above
chosen x F ea(P ), hence the optimal solution of (P) cannot be of the form x1 > 0 and x2 = x3 = 0.
Suppose now if we take an x F ea(P ) such that x2 > 0 and x1 = x3 = 0, then note
that inequality (ii) of the primal cannot be satisfied as an equality by x, hence if x F ea(P )
and y F ea(D) satisfy the complementary slackness property, then, y2 = 0. Also since
x2 > 0, if x and y satisfy the complementary slackness property, then 3y1 + y2 = 4, which
gives y1 = 43 , but then such a y cannot satisfy inequality (i)0 of the dual. Hence there exists
no y = [y1 , y2 ]T F ea(D) which satisfies complementary slackness property with the above
chosen x F ea(P ), hence the optimal solution of (P) cannot be of the form x2 > 0 and x1 = x3 = 0.
Now suppose if we take both x1 , x2 > 0 and x3 = 0, then again if x F ea(P ) and y F ea(D)
satisfy the complementary slackness property, then both inequality (i)0 and (ii)0 of the dual
has to be satisfied as an equality by y. Solving (i)0 and (ii)0 we get y1 = 1, y2 = 1.
The value of the objective function of the dual (D) for this y is equal to 14.
If x and y satisfy the complementary slackness property, then x has to satisfy inequality (i)
and (ii) of (P) as an equality, hence on solving we get x2 = 3 and x1 = 12 .
The value of the objective function of the primal (P) for this x is also 14.
Hence either by complementary slackness theorem or by corollary 1 we can conclude that x
and y are optimal for the primal and the dual respectively.