Mixed-Integer Linear Programming

Integer Programs (IP)

An optimization model is an Integer Program if any of its decision
Mixed-Integer Linear Programming (MILP): Model variables is discrete

Formulation If all variables are discrete, the model is a pure integer program
Otherwise, the model is a mixed-integer program

Benoı̂t Chachuat <benoit@mcmaster.ca> Integer variables appear in many problems:

McMaster University Trays in a distillation column Can be solved continuous, then

Department of Chemical Engineering
ChE 4G03: Optimization in Chemical Engineering
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03
Number of employees (1000’s) rounded to nearest integer
Number of parallel chemical reactors
Whether or not to operate boiler#2 Not appropriate to solve con-
on Monday tinuous and round after
Scheduling people and equipment to
tasks over time
1 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 2 / 26

Mixed-Integer Linear Programming Mixed-Integer Linear Programming

Class Exercise: Give more examples of integer decisions in the field of Linear vs. Nonlinear Integer Programs
Chemical Engineering:
An IP model is an integer linear program (ILP) if its (single)
1 Fluid flow: objective function and all its constraints are linear
◮ Otherwise, it is an integer nonlinear program (INLP)
◮

Standard Mixed-Integer Linear Programming (MILP) Formulation:

2 Heat transfer:
◮ ∆
◮
min z = cT x + dT y
x,y
 
3 Mass transfer:  ≤ 
◮
s.t. Ax + Ey = b
≥
 
◮

xmin ≤ x ≤ xmax , y ∈ {0, 1}ny

4 Reactor design:
◮ Mixed-Integer Nonlinear Programs (MINLPs) are very difficult to solve
◮
This is a great motivation for the continued use of LP methods!
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 3 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 4 / 26
Outline “Lumpy” Linear Programs
One broad class of “lumpy” LPs is when either/or decisions are added to
Modeling MILP requires knowledge and ingenuity — We will learn some what is otherwise an LP model
typical MILP formulations here:
Expressing All-or-Nothing Requirements
1 Native MILP Formulations All-or-nothing variable requirements of the form
“Lumpy” Linear Programs x j = 0 ∨ x j = uj
Knapsack Models
Assignment and Matching Models can be modeled by substituting
Scheduling Models x j ← uj y j , s.t. yj ∈ {0, 1}

2 Approximations of Nonlinear Models as MILPs Example: In a blending model, use none or all of a given ingredient
Separable Programming min 18x1 + 3x2 + 9x3
x1 ,x2 ,x3

s.t. 2x1 + x2 + 7x3 ≤ 150 ⇐⇒

For additional examples, see Rardin (1998), Chapter 11 0 ≤ x1 ≤ 60
0 ≤ x2 ≤ 30
x3 = 0 ∨ x3 = 20
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 5 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 6 / 26

“Lumpy” Linear Programs (cont’d) Formulating Models for Fixed-Charged Objectives

Θ(xj ) Class Exercise: Consider a fixed-charge objective function
min Θ1 (x1 ) + Θ2 (x2 )
x1 ,x2
Another source of “lumpy” LP models is cj
when the objective involves fixed charges where
(i.e., start-up cost)

fj ∆ 150 + 7x1 , if x1 > 0
Θ1 (x1 ) =
0, otherwise
xj
uj

∆ 110 + 9x2 , if x2 > 0
Θ2 (x2 ) =
0, otherwise
Expressing Fixed-Charge Requirements
and x1 , x2 satisfy

fj + cj xj , if xj > 0
∆
Fixed-cost requirements of the form: Θ(xj ) =
0, otherwise x1 + x2 ≥ 8
with nonnegative fixed charge fj > 0 and variable xj ≤ uj , can be modeled 0 ≤ x1 ≤ 3
by substituting
0 ≤ x2 ≤ 8.
Θ(xj ) ← fj yj + cj xj , s.t. xj ≤ uj yj and yj ∈ {0, 1}
Formulate a corresponding MILP model.

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 7 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 8 / 26
“Lumpy” Linear Programs (cont’d) Formulating Models for Semi-Continuous Variables
cj xj Class Exercise: Consider the following blending problem, where the
ingredient x3 is a semi-continuous variable:
Yet another source of “lumpy” LP min 18x1 + 3x2 + 9x3
cj x1 ,x2 ,x3
models is when a decision variable is
semi-continuous s.t. 2x1 + x2 + 7x3 ≤ 150
0 ≤ x1 ≤ 60
xj 0 ≤ x2 ≤ 30
lj uj
x3 = 0 ∨ 10 ≤ x3 ≤ 20
Expressing Semi-Continuous Requirements
Semi-continuous variables of the form: xj = 0 ∨ lj ≤ xj ≤ uj , with lj > 0 Formulate a corresponding MILP model.
can be modeled by introducing 2 new continuous variables xjl , xju ≥ 0, 1
new binary variable yj ∈ {0, 1}, and 2 additional constraints:
xj = lj xjl + uj xju
1 = yj + xjl + xju

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 9 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 10 / 26

Knapsack Models Knapsack Models (cont’d)

Value
1111
0000000
111
I4 111
000
The problem is to select a maximum
value collection of items subject to
0000
1111
0000
1111
1 Dependence of choice j on choice i can be enforced on
000
111
000
111 corresponding binary variables by adding constraints of the form:
limitations on resources consumed 000
111
I3
000
111
1111
0000 000
111
Knapsack models are the simplest 0000
1111
0000
1111
I2
0000
1111
00
11
0000
1111
of all (pure) integer linear programs 00
11
00
11
00
11
I1 2 Mutually exclusiveness conditions allowing at most one of a set of
00
11
(ILPs) 00
11 Capacity choices J can be enforced by adding constraints of the form:
max.
Each element is either all in or all out of the selection:

∆ 1, if item j selected
yj = Similarly, at most p of a set of choices J can be enforced as:
0, otherwise

Example: Selection of projects subject to limitations on budgets

Class Exercise: Readily available Canadian coins are 1¢, 5¢, 10¢ and 25¢. 3 Constraints can also enforce selection of exactly p or at least p of a
Formulate an ILP model to minimize the number of coins needed to set of choices J
provide change amount for q¢.
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 11 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 12 / 26
Formulating ILP Models Assignment Models
Class Exercise: The Department of Chemical Engineering at McMaster is Set I Set J
acquiring optimization software for use in ChE 4G03. The issue here is optimal matching
The 4 codes available and the types of algorithms they provide are or pairing of objects of two distinct
indicated by ×’s in the following table: types
It is standard to model all
Algorithm Code, j assignment forms with the decision
Type 1 2 3 4 variables:
LP × × × ×
IP — × — × 
NLP — — × × ∆ 1, if i of the 1st set is matched with j of the 2nd
yi ,j =
Cost 3 4 6 14 0, otherwise

Examples: Assign sales personnel to customers, jobs to machine, etc.

1 Formulate a MILP model to acquire a minimum cost collection of
Assignment constraints forcing every i and/or every j to be assigned
codes providing LP, IP and NLP capability
are of the form:
2 Formulate a MILP model to acquire a minimum cost collection of
codes with exactly one providing LP, one providing IP, and one
providing NLP capability
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 13 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 14 / 26

Assignment Models (cont’d) Matching Models

Set I
Generalized assignment constraints encompass cases where allocation Unlike assignment models, matching
of i to j requires fixed space si ,j within j capacity bj : models eliminate the distinction
between the sets
Decision variables of matching
Linear assignment models minimize/maximize a linear objective models typically are:
function of the form: 
∆ 1, if i is paired with i ′
XX yi ,i =
′
ci ,j yi ,j 0, otherwise
i ∈I j∈J
where i ′ > i , by convention, to avoid double counting
where ci ,j is the cost of assigning i to j
Matching constraints forcing some i ∈ I to pair with and only with
Class Exercise: Items i = 1, . . . , 100 of volume ci are being stored in an
some other i ′ ∈ I are of the form:
automated warehouse. Storage location j = 1, . . . , 20 are at a distance dj
from the input/output station, and all have capacity b. Formulate a MILP
model to store all items at minimum total travel distance.

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 15 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 16 / 26
Matching Models (cont’d) Process Scheduling Models
Class Exercise: The instructor of ChE 4G03 wants to assign his students Scheduling is the allocation of resources over time
to 2-person teams for a graded tutorial. each student s has scored his/her Single-process scheduling problems seek an optimal sequence in
preference ps,s ′ for working with each other student s ′ . which to complete a given collection of tasks on a single process that
1 Formulate a MILP model to form teams in a way that maximizes can accommodate only one job at a time
total preference.
Feed 1 Product 1 111111
000000
p1
000000
111111
000000
111111 Time
r1 d1
Feed 2 Product 2 11111
00000
p2
00000
11111 Time
r2 d2
Feed 3 Product 3 11111
00000
p3
00000
11111
00000
11111 Time
r3 d3
Typical Data:
∆
pj = estimated process time for task j
∆
rj = release time at which task j becomes available for processing
∆
dj = due date by which task j should be completed
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 17 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 18 / 26

Process Scheduling Models (cont’d) Process Scheduling Models (cont’d)

x1 x2 x3 x1 x2 x3
111111
000000
Feed 1 Product 1 000000
111111
p1 111111
000000
000000
111111
000000
111111 Time Feed 1 Product 1 p1
000000
111111
000000
111111
r1 Time
d1 r1 d1
11111
00000
00000
11111
Feed 2 Product 2 p2
00000
11111 Time Feed 2 Product 2 11111
00000
p2
00000
11111 Time
r2 d2 r2 d2
11111
00000
Feed 3 Product 3 00000
11111
p3 11111
00000
00000
11111
00000
11111 Time Feed 3 Product 3 p3
00000
11111
00000
11111
r3 Time
d3 r3 d3
Time Decision Variables and Related Constraints: Objective Functions — Often minimizes one of the following:
∆
xj = start time for task j
Max. completion time (makespan) max{xj + pj : j = 1, . . . , n}
resource availability: xj ≥ rj , ∀j n
1X
Mean completion time (xj + pj )
Handling of Due Dates: n j=1

Due dates are usually handled as goals to be reflected in the objective Max. lateness max{xj + pj − dj : j = 1, . . . , n}
n
1X
function rather than explicit constraints Mean lateness (xj + pj − dj )
n j=1
This avoids the situation where there is no feasible schedule that
Max. tardiness max{0, max{xj + pj − dj : j = 1, . . . , n}}
meets all due dates n
1X
Dates that must be met are termed deadlines to distinguish and can Mean tardiness max{0, xj + pj − dj }
n j=1
be easily enforced as: xj + pj ≤ dj , ∀j
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 19 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 20 / 26
Process Scheduling Models (cont’d) Process Scheduling Models (cont’d)
Class Exercise: The following table shows the process time, release times, x1 x2 x3
due dates, and scheduled starts for three jobs. Compute the corresponding Feed 1 Product 1 000000
111111
p1
111111
000000
000000
111111 Time
value of each of the six objective functions listed previously. r1 d1
Feed 2 Product 2 11111
00000
p2
Job 1 Job 2 Job 3 r2 00000
11111 Time
d2
Process time, pi 15 6 9 Feed 3 Product 3 11111
00000
p3
00000
11111
00000
11111 Time
Release time, ri 5 10 0 r3 d3
Due Date, di 20 25 36
Scheduled start, xi 9 24 0
Conflict Constraints:
1 Completion times (xi + pi ) are: xj + pj ≤ xj ′ ∨ xj ′ + pj ′ ≤ xj , ∀j, j ′ = 1, . . . , n, j 6= j ′
9 + 15 = 24, 24 + 6 = 30, 0+9= 9
Maximum completion time is max{24, 30, 9} = 30 Reformulation Using Disjunctive Variables:
Mean completion time is 13 (24 + 30 + 9) = 21 
∆ 1, if j binding comes before j ′
2 Lateness of the three jobs (xj + pj − dj ) is: yj,j ′ =
0, if j ′ binding comes before j
9 + 15 − 20 = 4, 24 + 6 − 25 = 5, 0 + 9 − 36 = − 27
Maximum lateness is max{4, 5, −27} = 5 Illustrative Example: y1,2 = y1,3 = 1, y2,3 = 0
Mean completion time is 31 (4 + 5 − 27) = −6
Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 21 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 22 / 26

Process Scheduling Models (cont’d) Separable Programming

x1 x2 x3
Feed 1 Product 1 111111
000000
p1
000000
111111
Consider the following mathematical program:
000000
111111 Time
r1 d1 n
Feed 2 Product 2 11111
00000
p2 ∆
X
r2 00000
11111 Time min
x
z= fj (xj ) = f1 (x1 ) + · · · + fn (xn )
d2
11111
00000 j=1
Feed 3 Product 3 p3
00000
11111
00000
11111 Time n
r3 d3 X
s.t. ai ,j xj ≤ bi , i = 1, . . . , m
Big-M Method j=1
Conflicts between tasks j and j ′ can be prevented with disjunctive xj ≥ 0, j = 1, . . . , n
constraint pairs:
xj + pj ≤ xj ′ + M(1 − yj,j ′ ) ◮ The objective consists of n nonlinear, separable terms fj (xj ), each a
function of a single variable only
xj ′ + pj ′ ≤ xj + Myj,j ′ ◮ The m constraints are linear
where M is a large positive constant (e.g., the max. makespan) When each fj is convex or concave on the feasible region, the
separable program can be approximated with an LP
Class Exercise: Formulate integer linear constraints for feasible schedules
on a single process with 3 tasks having process times 14, 3, and 7. Otherwise, the separable program can be approximated with an MILP

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 23 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 24 / 26
Approximating General Separable Programs as MILPs Approximating General Separable Programs as MILPs
Piecewise affine approximation: Piecewise affine approximation:
Define: Disjunctive Variables:
(Nj intervals) (Nj intervals)
(k) (k−1) 
(2) (1) (k) fj − fj (2) (1) ∆ 0, if ωj,k = 0
fj (xj )
c2 =
fj −fj
cj = fj (xj )
c2 =
fj −fj yj,k =
(2) (1) (k) (k−1) (2) (1) 1, if ωj,k > 0
xj −xj
xj − xj xj −xj

(k) (k−1) Disjunctive Constraints:

fj
(1) 0≤ ωj,k ≤ xj − xj fj
(1)
h i
(3) (3) (k) (k−1)
fj Substitute each variable xj fj ωj,k ≤ xj − xj yj,k ,
and function fj by:
(2) (2)
∀k = 1, . . . , Nj
fj fj h i
Nj (k) (k−1)
fj
(0)
(0)
X fj
(0) ωj,k ≥ xj − xj yj,k+1 ,
0 ≤ ωj,2 ≤
(2)
xj −
(1)
xj xj ← xj + ωj,k 0 ≤ ωj,2 ≤
(2)
xj −
(1)
xj
k=1 ∀k = 1, . . . , Nj − 1
xj xj
(0) (1) (2) (3) Nj (0) (1) (2) (3)
xj xj xj xj xj xj xj xj
(0)
X (k) Do you see why this works?
fj (xj ) ← fj + cj ωj,k
k=1

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 25 / 26 Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 25 / 26

Approximating Separable Programs as MILPs (cont’d)

Approximate Mixed-Integer Linear Program (on Nj Intervals):
n Nj
n X
(0) (k)
X X
min fj + cj ωjk
ω,y
j=1 j=1 k=1
Nj
n X n
(0)
X X
s.t. ai ,j ωj,k ≤ bi − ai ,j xj , i = 1, . . . , m
j=1 k=1 j=1
h i
(k) (k−1)
ωj,k ≤ xj − xj yj,k , k = 1, . . . , Nj
h i
(k) (k−1)
ωj,k ≥ xj − xj yj,k+1 , k = 1, . . . , Nj − 1
yj,k ∈ {0, 1}, j = 1, . . . , n, k = 1, . . . , Nj

The solution can be made as accurate as desired by using enough

intervals — One pays the price in terms of increased problem size!

Benoı̂t Chachuat (McMaster University) MILP: Model Formulation 4G03 26 / 26