SOLVING GENERALIZED FRACTIONAL PROGRAMMING PROBLEMS WITH APPROXIMATION
S¸. I. Birbil ^{†} , J. B. G. Frenk ^{‡} and S. Zhang ^{} ^{∗}
˙
^{†} Faculty of Engineering and Natural Sciences, Sabancı University, OrhanliTuzla, 34956, Istanbul, Turkey.
^{‡} Econometric Institute, Erasmus University Rotterdam Postbus 1738, 3000 DR Rotterdam, The Netherlands.
^{} Systems Engineering and Engineering Management The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.
ABSTRACT. The present paper proposes a new approach to solve generalized fractional program ming problems with approximation. Capitalizing on two alternatives, we review the Dinkelbachtype methods and set forth the main difﬁculty in applying these methods. In order to cope with this dif ﬁculty, we propose an approximation approach that can be controlled by a predetermined parameter. The proposed approach is promising particularly when it is acceptable to ﬁnd an effective, but near optimal value in an efﬁcient manner. Once the approximation parameter is ﬁxed, our analysis shows how good is the value found by the approximation. In addition, we present several observations that may be suitable for improving the performance of the proposed approach. Finally, we support our discussion through numerical experiments.
Keywords. Generalized fractional programming, approximation approach, nearoptimal solution, per formance improvement observations
1
Introduction
Decision makers in different application areas demand effective and easytouse solution methods for their optimization problems. In many circumstances, they are ready to welcome a new approach even though this approach may yield a value closeenough to the actual (or theoretical) optimal value. One way of satisfying this demand is incorporating approximation approaches into wellknown solution methods. To our advantage, the precision of an approximation approach depends on predeﬁned parameters. Therefore through an approximation parameter, the tradeoff between the quality of the results and the simpliﬁcation of the problem can be controlled.
In this paper, we focus on solving a generalized fractional programming (GFP) problem by an approx imation approach. GFP problems arise in many reallife applications. One early example dates back to the time of von Neumann when he used GFP in modeling an expanding economy [28]. Another im portant application is within the ﬁeld of multiobjective programming. In this case, the main goal is to minimize the maximum deviation of the fractional objectives, like inputoutput ratios, from their target values [20, 26, 25, 24]. Also in numerical analysis, GFP methods are used for solving rational Chebyshev
^{∗} Research supported by Hong Kong RGC Earmarked Grants CUHK4233/01E and CUHK4174/03E.
1
April 13, 2005
2
GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD
approximations [2, 11, 21, 19]. Some other applications of fractional programming have been given in the literature; see [17].
To study the theoretical properties of GFP problems, researchers made use of the tools from duality theory, nonlinear programming and numerical analysis. Naturally, the study of the theoretical properties led to the development of various solution methods [13, 14, 12, 6, 5, 3, 20, 22, 27]. The majority of these methods are the variations of the approach suggested in the pioneering work of Crouzeix` et. al. [13]. In the literature, these methods are also known as Dinkelbachtype methods [15]. Although not immediately clear, it can be shown that Dinkelbachtype methods are actually cutting plane methods [4]. These methods are based on the idea of solving a sequence of auxiliary problems so that the solutions of the auxiliary problems converge to the solution of the GFP problem. These methods are very powerful. However, their performances heavily depend on the effective solutions of the auxiliary problems. In general the auxiliary problems are nonsmooth and nonlinear. Therefore, they might pose a major difﬁculty in solving the GFP problems effectively and easily. In a recent review by Frenk and Schaible, the theoretical properties of fractional programming problems are extensively studied [18].
The main idea of this paper is replacing the difﬁcult auxiliary problems in Dinkelbachtype methods with relatively simple approximations. Approximations of these sort yield εoptimal (closeenough) values to the auxiliary problems. Consequently, we will analyze the error committed by replacing the auxiliary problems with their approximated counterparts. Especially for large size problems, the task of speeding up the algorithms plays an important role. Therefore, we will also propose several approaches that may lead to a performance improvement. During our analysis, we will work with a generic approximation function since the choice of the approximation function does not alter our results. In order to illustrate our idea, we will then give two particular approximation approaches that will be also used in the numerical results section. Overall, this paper aims at providing a useful reference for a decision maker who agrees to solve a GFP problem effectively at the cost of ﬁnding inexact but controlled solutions.
2 Generalized Fractional Programming and the Dinkelbach Method
In this section we give an overview on generalized fractional programming and the most common solution approach, namely, the Dinkelbach method. Before proceeding to our subsequent analysis, let us ﬁrst introduce the generalized fractional programming problem
λ ^{∗} :=
inf
x∈X ^{m}^{a}^{x}
i∈I
f _{i} (x) g _{i} (x)
(P )
where X ⊆ R ^{n} is the feasible region, I := {1, 2, · · · , m} is a ﬁnite index set and for all i ∈ I, f _{i} : R ^{n} → R and g _{i} : R ^{n} → R are the problem functions. In addition, if we deﬁne the function p : X → (−∞, ∞] by
(2.1)
p(x) := max
i∈I
f _{i} (x) g _{i} (x)
,
then the optimization problem (P ) can be written as
λ ^{∗} :=
_{x}_{∈}_{X} p(x).
inf
(2.2)
In the sequel we denote the optimal solution of problem (P ) by x ^{∗} .
Introducing now the functions g _{m}_{i}_{n} : R ^{n} → R by
g _{m}_{i}_{n} (x) := min
i∈I
g _{i} (x),
(2.3)
it is necessary to impose the following assumption to avoid pathological instances of problem (P ).
2
April 13, 2005
2
GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD
Assumption 2.1 g _{m}_{i}_{n} (x) > 0 for every x ∈ X.
The main idea of the Dinkelbach method is to provide a solution to the difﬁcult optimization problem (P ) by solving a sequence of relatively simple parameterized problems. In this overview section, we focus on two main Dinkelbachtype approaches. The ﬁrst approach is based on solving constrained subprob lems, whereas the second approach aims at reformulating the parameterized subproblems as unconstrained nonlinear optimization problems.
We start with the ﬁrst approach by deﬁning for every i ∈ I, the functions φ _{i} : R ^{n}^{+}^{1} → R given by
φ _{i} (λ, x) := f _{i} (x) − λg _{i} (x) 
(2.4) 

and for every (λ, x) ∈ R ^{n}^{+}^{1} , the function 

φ(λ, x) := max i∈I 
φ _{i} (λ, x). 
(2.5) 
This leads to the associated parametric constrained optimization problem 

φ(λ) := inf _{x}_{∈}_{X} φ(λ, x). 
(P _{λ} ) 
After minor modiﬁcations of the proofs in [13], one can show the following important results (see also
[18]).
Lemma 2.1 φ(λ) ≥ 0 if and only if −∞ < λ ≤ λ ^{∗} .
Lemma 2.2 The following conditions are equivalent:
1. The optimization problem (P ) has an optimal solution.
2. The set V deﬁned by
V := {λ ∈ R :
Problem (P _{λ} ) has an optimal solution and φ(λ) = 0}
is nonempty.
3. The set V is a singleton with V = {λ ^{∗} }.
We now discuss the unconstrained approach. In this case the feasible region X is given by
X := {x ∈ R ^{n} : h _{j} (x) ≤ 0, j ∈ J},
(2.6)
where J
with h(x) := (h _{1} (x), · · · , h _{p} (x)) ^{T} . Clearly, this form of the feasible set is a commonly used form for numerical purposes.
: R ^{n} → R are realvalued functions
:= {1, 2, · · · , p} is a ﬁnite index set and for all j ∈ J, h _{j}
In addition to Assumption 2.1, we also assume for the unconstrained approach that the following condi tion holds.
Assumption 2.2 The function x → h _{j} (x), j ∈ J are convex and the set X has a nonempty interior, i.e., there exist some x _{0} ∈ R ^{n} such that max _{j}_{∈}_{J} h _{j} (x _{0} ) < 0.
3
April 13, 2005
2
GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD
As before, we start by introducing the parametric function ψ _{i} : R ^{n}^{+}^{1} → R given by
_{ψ} i _{(}_{λ}_{,} _{x}_{)} _{:}_{=} ^{}
f _{i} (x) − λg _{i} (x), h _{i} (x),
for i ∈ I
for i ∈
_{J}
and consider for (λ, x) ∈ R ^{n}^{+}^{1} , the function
ψ(λ, x) := max
_{i}_{∈}_{I}_{∪}_{J} ψ _{i} (λ, x)
with the parametric unconstrained optimization problem
ψ(λ) :=
inf _{n} ψ(λ, x).
x∈R
(2.7)
(2.8)
¯
( P _{λ} )
It is shown in [27, 1] that the following versions of the previous two lemmas hold.
Lemma 2.3 ψ(λ) ≥ 0 if and only if −∞ < λ ≤ λ ^{∗} .
Lemma 2.4 The following conditions are equivalent:
1. The optimization problem (P ) has an optimal solution.
2. The set W deﬁned by
¯
W := {λ ∈ R : Problem ( P _{λ} ) has an optimal solution and ψ(λ) = 0}
is nonempty.
3. The set W is a singleton with W = {λ ^{∗} }.
Using the above results, we can discuss the computational procedures for both constrained and uncon strained cases. In order to apply the Dinkelbachtype approach corresponding to the constrained case, we always assume that problem (P ) and problems (P _{λ} ) for λ ≥ λ ^{∗} are solvable. The general scheme of the constrained case has now the form given in Algorithm 2.1.
Algorithm 2.1 Constrained Case
1. Select x _{0} ∈ X
and set k := 1.
2. Set λ _{k} := p(x _{k}_{−}_{1} ).
3. Determine an element x _{k} of the set {x ∈ X : φ(λ _{k} , x) < 0}. If such an element does not exist, then return (λ _{k} , x _{k}_{−}_{1} ), else set k := k + 1 and return to Step 2.
¯
As in the constrained case, we assume that problem (P ) and problems ( P _{λ} ) for λ ≥ λ ^{∗} are solvable.
Observe, if ψ(λ _{k} , x _{k} ) < 0, then it follows automatically that h _{j} (x _{k} ) < 0 for every j ∈ J, and hence,
The algorithm for the unconstrained Dinkelbachtype approach has now the form given in
Algorithm 2.2.
x _{k}
∈
X.
4
April 13, 2005
2
GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD
Figure 1: Two successive iterations of Algorithm 2.1.
Algorithm 2.2 Unconstrained Case
1. Select x _{0} ∈ X and set k := 1.
2. Set λ _{k} := p(x _{k}_{−}_{1} ).
3. Determine an element x _{k} of the set {x ∈ R ^{n} : ψ(λ _{k} , x) < 0}. If such an element does not exist, then return (λ _{k} , x _{k}_{−}_{1} ), else set k := k + 1 and return to Step 2.
The geometrical interpretation of Algorithm 2.1 is illustrated in Figure 1. Clearly, for any x _{k} ∈ {x ∈ X : φ(λ, x) < 0} we have φ(λ) ≤ φ(λ, x _{k} ). Hence, at the k ^{t}^{h} iteration of Algorithm 2.1, the nonconvex function φ is approximated from above by the polyhedral convex function λ → φ(λ, x _{k} ). As a direct consequence of Lemma 2.2, our main goal is to compute λ that satisﬁes φ(λ) = 0. Instead of this ambitious goal, we solve the approximating equation φ(λ, x _{k} ) = 0, which leads to
f _{i} (x _{k} ) g _{i} (x _{k} )
.
λ = max
i∈I
(2.9)
Notice that this solution is indeed the next iteration in Algorithm 2.1 (Step 2). A similar interpretation can be given for Algorithm 2.2, where the function ψ is approximated by the polyhedral convex function λ → ψ(λ, x _{k} ).
In Step 3 of both algorithms we suggest to solve membership problems instead of solving the opti
¯
mization problems (P _{λ} ) and ( P _{λ} ). The reason for this suggestion is given by the following observation. Consider the constrained approach and suppose at iteration k we compute the optimal solution x _{k} of prob lem (P _{λ} _{k} ) and at the same time we choose a different vector x _{k} satisfying φ(λ _{k} , x _{k} ) < 0. It might now happen that the value λ _{k}_{+}_{1} := p(x _{k} ) is closer to λ ^{∗} than the value λ _{k}_{+}_{1} derived from x _{k} . An instance showing this behavior is given in Example 2.1.
∗
∗
Example 2.1 Suppose we want to solve
λ ^{∗} =
x∈[0,2] ^{m}^{a}^{x}^{{} −7x + 1
inf
3x − 2 

2x + 2 
_{,} −18x + 2 4x + 1 
^{,} 
16x + 
_{3} }.
5
April 13, 2005
3
THE APPROXIMATION APPROACH
Let x _{0} = 1 then λ _{1} = p(x _{0} ) = 0.5263. Notice that problem (P _{λ} _{1} ) can be solved by a standard linear
= 0.3295 with the
objective function value φ(λ _{1} , x ^{∗} ) = −1.4468. At the next iteration, we need to compute λ _{2} := p(x ^{∗} ) =
programming solver.
After solving problem (P _{λ} _{1} ), we ﬁnd the optimal solution x ^{∗}
1
1
∗
1
−0.1228. On the other hand, instead of the optimal solution, if we select x _{1} = 0.30 yielding φ(λ _{1} , x _{1} ) = −1.2368, then the corresponding value of λ at the next iteration becomes λ _{2} := p(x _{1} ) = −0.1410, which
is less than λ
∗
2 ^{.}
This suggests that investing time in solving problem (P _{λ} ) to optimality might not always be the best strategy. On the other hand, to determine whether there exists an element of the set {x ∈ X : φ(λ _{k} , x) < 0} or decide whether this set is empty, it is most natural to solve problem (P _{λ} _{k} ), and so x _{k} becomes a natural candidate. This observation may also become useful whenever the optimization routine for solving the auxiliary problems requires a considerable effort. In this case, one may stop as soon as the optimization routine ﬁnds a feasible, but not necessarily optimal, point x that solves the membership problem. As Lemma 2.1 and Lemma 2.3 show, from a practical point of view, we only need a decreasing sequence of λ _{k} values.
∗
3 The Approximation Approach
Notice that in both Algorithm 2.1 and Algorithm 2.2, the membership problems at Step 3 can be solved by reformulating the problems as nonlinear optimization problems, i.e., one needs to solve (approximately, see Example 2.1), in the constrained case, the auxiliary optimization problem
min max{f _{1} (x) − λg _{1} (x), · · · , f _{m} (x) − λg _{m} (x)}
x∈X
and in the unconstrained case, the problem
min _{n} max{f _{1} (x) − λg _{1} (x), · · · , f _{m} (x) − λg _{m} (x), h _{1} (x), · · · , h _{p} (x)}.
x∈R
(3.1)
(3.2)
Although the max function is a convex increasing function, unfortunately it is nonsmooth. Therefore, we still encounter a nonsmooth objective function in the auxiliary optimization problems. One promising solution to this obstacle is to approximate the max function by a smooth alternative. Consider now a smooth, convex and increasing function Π _{s} : R ^{s} → R that satisﬁes
lim
ε↓0
εΠ _{s} ( ^{y} ) = max{y _{1} , · · · , y _{s} }
ε
(3.3)
and
0 ≤ εΠ _{s} ( ^{y} ) − max{y _{1} , · · ·
ε
,
y _{s} } ≤ β _{s} (ε)
(3.4)
for every y ∈ R ^{s} . As a direct consequence of the above two relations, the function y → εΠ _{s} ( ^{y} ) becomes a good candidate for approximating the nonsmooth objective functions of the auxiliary problems. Let us now give two examples that will also be used in the numerical results section.
ε
Example 3.1 It is easy to check that the entropytype function
Π _{s} (y) = log
s
i=1
e ^{y} ^{i}
is convex, differentiable and increasing [16]. Moreover, it satisﬁes relation (3.3) and relation (3.4) with β _{s} (ε) = ε log s [7].
6
April 13, 2005
3.1
Constrained Case with Approximation
3
THE APPROXIMATION APPROACH
Example 3.2 In some problems, entropytype function in Example 3.1 may create overﬂow problems be cause of the exponential function. Recently Birbil et. al. proposed a more stable approximation method that aims at extending two dimensional approximation functions to higher dimensions by recursion [9]. The main idea of the method is based on the observation that the higher dimensional max function can be written recursively as follows
max{y _{1} , · · ·
, y _{s} } = max{max{y _{1} , · · · , y _{q} }, max{y _{q}_{+}_{1} , · · · , y _{s} }},
where 1 < q < s. Following this observation, the method proposed by Birbil et. al. provides a twice differentiable convex increasing function Π _{s} that satisﬁes relation (3.3). In this case, the error term in (3.4) is given by β _{s} (ε) = _{2} ^{ε} (log _{2} (s − 1) + 1)Π _{2} (0), where Π _{2} : R ^{2} → R is the base function that is used for approximating the two dimensional max function, i.e., lim _{ε}_{↓}_{0} εΠ _{2} (y/ε) = max{y _{1} , y _{2} }.
3.1 Constrained Case with Approximation
We ﬁrst start with the constrained case of the Dinkelbachtype method. Given ε > 0, deﬁne the function φ _{ε} : R × X → R by
φ _{ε} (λ, x) := εΠ _{m} ^{f} ^{1} ^{(}^{x}^{)} ^{−} ^{λ}^{g} ^{1} ^{(}^{x}^{)}
ε
_{,} _{·} _{·} _{·} _{,} f _{m} (x) − λg _{m} (x)
ε
.
(3.5)
Hence, instead of problem (3.1), we can consider the smooth optimization problem
φ _{ε} (λ) := min
x∈X
φ _{ε} (λ, x).
ε
λ
(P
)
To derive the difference in the optimal objective function values between optimization problems (P _{λ} ) and
(P
(3.6)
By the same relation, we also obtain that φ _{ε} (λ, x) ≥ φ(λ, x) for every x ∈ X and λ ∈ R and this shows
ε ) we observe by relation (3.4) and x(λ) belonging to arg min _{x}_{∈}_{X} φ _{ε} (λ, x) that
λ
φ _{ε} (λ) − φ(λ) ≤ φ _{ε} (λ, x(λ)) −
φ(λ, x(λ)) ≤ β _{m} (ε).
φ _{ε} (λ) ≥ φ(λ)
for every λ ∈ R. Hence it follows by relations (3.6) and (3.7) that
(3.8)
for every λ ∈ R. Therefore at the expense of some limited error, we consider the following approximated Dinkelbachtype method.
(3.7)
0 ≤ φ _{ε} (λ) − φ(λ) ≤ β _{m} (ε)
Algorithm 3.1 Constrained Case with Approximation
1. Select x _{0} ∈ X and set k := 1.
2. Set λ _{k} := p(x _{k}_{−}_{1} ).
3. Determine an element x _{k} of the set {x ∈ X : φ _{ε} (λ _{k} , x) < 0}. If such an element does not exist, then return (λ _{k} , x _{k}_{−}_{1} ), else set k := k + 1 and return to Step 2.
Before discussing the properties of Algorithm 3.1, we ﬁrst introduce the nonempty setvalued mappings S : R × X ⇒ I and L : X ⇒ I given by
S(λ, x) := {i ∈ I : φ _{i} (λ, x) = φ(λ, x)}
7
April 13, 2005
3.1 
Constrained Case with Approximation 
3 
THE APPROXIMATION APPROACH 
and 
L(x) := {i ∈ I :
f _{i} (x) g _{i} (x)
= p(x)}.
It is now easy to derive the following upper and lower bounds on the difference λ _{k}_{+}_{1} − λ _{k} for every k < k _{0} , with k _{0} ≤ ∞ being the total number of iterations spent in Algorithm 3.1 before meeting the stopping condition.
Lemma 3.1 The sequence λ _{k} , k < k _{0} generated by Algorithm 3.1 is strictly decreasing and satisﬁes
φ(λ _{k} , x _{k} )
^{m}^{a}^{x} i∈S(λ _{k} ,x _{k} )
_{g} i _{(}_{x} k _{)} ≤ λ _{k}_{+}_{1} − λ _{k} ≤
φ(λ _{k} , x _{k} )
min _{i}_{∈}_{L}_{(}_{x} _{k} _{)} g _{i} (x _{k} ) ^{.}
Proof. For every k < k _{0} we know by relation (3.4) and (3.5) that φ(λ _{k} , x _{k} ) ≤ φ _{ε} (λ _{k} , x _{k} ) < 0, and since φ(λ _{k}_{+}_{1} , x _{k} ) = 0 and λ → φ(λ, x _{k} ) is strictly decreasing, this yields λ _{k}_{+}_{1} < λ _{k} . We now verify the above inequalities. By the deﬁnition of λ _{k}_{+}_{1} , it follows for every i ^{∗} belonging to arg max _{i}_{∈}_{S}_{(}_{λ} _{k} _{,}_{x} _{k} _{)} g _{i} (x _{k} ) satisfying g _{i} ∗ (x _{k} ) = max _{i}_{∈}_{S}_{(}_{λ} _{k} _{,}_{x} _{k} _{)} g _{i} (x _{k} ) that
_{λ} k+1 _{−} _{λ} k _{≥} f _{i} ∗ (x _{k} ) − λ _{k} g _{i} ∗ (x _{k} ) _{=} g _{i} ∗ (x _{k} )
φ(λ _{k} , x _{k} )
^{m}^{a}^{x} i∈S(λ _{k} ,x _{k} )
^{g} i ^{(}^{x} k ^{)} .
Also for every i _{∗} belonging to arg min _{i}_{∈}_{L}_{(}_{x} _{k} _{)} g _{i} (x _{k} ) satisfying g _{i} _{∗} (x _{k} ) = min _{i}_{∈}_{L}_{(}_{x} _{k} _{)} g _{i} (x _{k} ), we obtain by using again the deﬁnition of λ _{k}_{+}_{1} that
_{λ} k+1 _{−} _{λ} k _{=} f _{i} _{∗} (x _{k} ) − λ _{k} g _{i} _{∗} (x _{k} ) g _{i} _{∗} (x _{k} )
and this completes the proof.
≤
φ(λ _{k} , x _{k} ) min _{i}_{∈}_{L}_{(}_{x} _{k} _{)} g _{i} (x _{k} ) ^{,}
If at the kth step of Algorithm 3.1, we observe that φ _{ε} (λ _{k} , x _{k} ) ≤ δ for some δ < 0, and it holds that g ^{∗} := max _{x}_{∈}_{X} max _{i}_{∈}_{I} g _{i} (x) < ∞, then it follows by Lemma 3.1 and relation (3.4) that
λ k+1 − λ k ≤
φ _{ε} (λ _{k} , x _{k} )
^{m}^{i}^{n} _{i}_{∈}_{L}_{(}_{x} _{k} _{)} ^{g} _{i} (x _{k} )
^{≤} −δ
^{∗}
.
g
(3.9)
Hence in this case, the improvement in the objective function value is at least −δ/g ^{∗} , and in most cases it can be expected that this is a conservative bound. In addition to the number of iterations, we are also interested in the quality of the solution generated by Algorithm 3.1. This means that one likes to derive an upper bound on the difference λ ^{∗} − λ _{k} _{0} with k _{0} being the iteration at which the stopping condition applies. First we need the following lemma.
Lemma 3.2 If g _{∗} := inf _{x}_{∈}_{X} g _{m}_{i}_{n} (x) > 0, then it follows for every k < k _{0} that
_{0} _{≥}
_{λ} _{∗} _{−} _{λ} k+1 _{≥} φ _{ε} (λ _{k}_{+}_{1} ) − β _{m} (ε)
g ∗
.
Proof. It is obvious that 0 ≥ λ ^{∗} − λ _{k}_{+}_{1} and to prove the reverse inequality, we observe for every k < k _{0} , x ^{∗} an optimal solution of optimization problem (P ) and i ^{∗} belonging to arg max _{i}_{∈}_{S}_{(}_{λ} _{k}_{+}_{1} _{,}_{x} ∗ _{)} g _{i} (x ^{∗} ) that
_{0} _{≥} _{λ} _{∗} _{−} _{λ} k+1 _{≥} f _{i} ∗ (x ^{∗} ) − λ _{k}_{+}_{1} g _{i} ∗ (x ^{∗} ) _{=} g _{i} ∗ (x ^{∗} )
φ(λ _{k}_{+}_{1} , x ^{∗} )
^{m}^{a}^{x} i∈S(λ _{k}_{+}_{1} ,x ^{∗} )
_{g} i _{(}_{x} _{∗} _{)} .
(3.10)
8
April 13, 2005
3.2
Unconstrained Case with Approximation
3
THE APPROXIMATION APPROACH
This shows that φ(λ _{k}_{+}_{1} , x ^{∗} ) ≤ 0, and since max _{i}_{∈}_{S}_{(}_{λ} _{k}_{+}_{1} _{,}_{x} ∗ _{)} g _{i} (x ^{∗} ) ≥ g _{∗} , by using relation (3.10) we obtain that
(3.11)
φ(λ _{k}_{+}_{1} , x ^{∗} )
g i (x ∗ ) _{≥} φ(λ _{k}_{+}_{1} , x ^{∗} )
g
∗
.
λ ^{∗} − λ _{k}_{+}_{1} ≥
^{m}^{a}^{x} i∈S(λ _{k}_{+}_{1} ,x ^{∗} )
Applying relation (3.4) yields
φ(λ _{k}_{+}_{1} , x ^{∗} ) ≥ φ _{ε} (λ _{k}_{+}_{1} , x ^{∗} ) − β _{m} (ε) ≥ φ _{ε} (λ _{k}_{+}_{1} ) − β _{m} (ε),
and by relation (3.11) the desired result follows.
Notice that the stopping condition in Algorithm 3.1 is satisﬁed if φ _{ε} (λ _{k} _{0} ) ≥ 0 for ﬁnite k _{0} . Therefore, in combination with Lemma 3.2, we have
_{λ} _{∗} _{−} _{λ} k 0 _{≥} −β _{m} (ε)
g
∗
.
(3.12)
Finally we observe in this subsection that the optimization problem (P
λ
ε ) is a smooth convex optimization
problem if λ > 0, X is a convex set deﬁned by differentiable convex inequalities and the functions f _{i} , −g _{i} for all i ∈ I are differentiable and convex. In this case, the function x → g _{m}_{i}_{n} (x) is also a concave function and hence, the value g _{∗} can be computed by solving a concave minimization problem. Such a problem is in general NPhard [23].
3.2 Unconstrained Case with Approximation
Similar to the previous section, we start with introducing an approximation function ψ _{ε} : R ^{n}^{+}^{1} → R deﬁned by
ψ _{ε} (λ, x) := εΠ _{m}_{+}_{p} ^{f} ^{1} ^{(}^{x}^{)} ^{−} ^{λ}^{g} ^{1} ^{(}^{x}^{)}
ε
_{,} _{·} _{·} _{·} _{,} f _{m} (x) − λg _{m} (x) _{,} h _{1} (x)
ε
ε
, · · · , ^{h} ^{p} ^{(}^{x}^{)} .
ε
We can now consider the smooth optimization problem
ψ _{ε} (λ) := min _{n} ψ _{ε} (λ, x).
x∈R
(3.13)
(
¯
P
ε
λ
)
In this case the counterpart of relation (3.8) becomes
(3.14)
for every λ ∈ R. The adapted version of Algorithm 2.2 with the approximation is now given in Algorithm
3.2.
0 ≤ ψ _{ε} (λ) − ψ(λ) ≤ β _{m}_{+}_{p} (ε)
Algorithm 3.2 Unconstrained Case with Approximation
1. Select x _{0} ∈ X and set k := 1.
2. Set λ _{k} := p(x _{k}_{−}_{1} ).
3. Determine an element x _{k} of the set {x ∈ R ^{n} : ψ _{ε} (λ _{k} , x) < 0}. If such an element does not exist, then return (λ _{k} , x _{k}_{−}_{1} ), else set k := k + 1 and return to Step 2.
Observe, if ψ _{ε} (λ _{k} , x) < 0, then ψ(λ _{k} , x) ≤ ψ _{ε} (λ _{k} , x) < 0, and hence x belongs to X. As before we are now interested in the difference λ _{k}_{+}_{1} − λ _{k} with the sequence of λ _{k} values generated by Algorithm 3.2. Again, k _{0} denotes the total number of iterations spent in Algorithm 3.2 before meeting the stopping condition.
9
April 13, 2005
3.2
Unconstrained Case with Approximation
3
THE APPROXIMATION APPROACH
Lemma 3.3 The sequence λ _{k} , k < k _{0} generated by Algorithm 3.2 is strictly decreasing and satisﬁes
λ _{k}_{+}_{1} − λ _{k} ≤ (ψ(λ _{k} , x _{k} ))( min k _{)} g _{i} (x _{k} )) ^{−}^{1} .
i∈L(x
Proof. Redoing the last part of the proof of Lemma 3.1 leads to
λ _{k}_{+}_{1} − λ _{k} ≤ (φ(λ _{k} , x _{k} ))( min k _{)} g _{i} (x _{k} )) ^{−}^{1}
i∈L(x
and using now φ(λ _{k} , x _{k} ) ≤ ψ(λ _{k} , x _{k} ) yields the desired result.
In order to give an upper bound on the error λ _{k} −λ ^{∗} we will use the strong duality theorem for Lagrangean duality. To apply this result, we need to impose some well known additional properties on the functions f _{i} and g _{i} .
Assumption 3.1 The following conditions hold:
1. The functions f _{i} : R ^{n} → R, i ∈ I are convex on R ^{n} and nonnegative on X. Moreover, the functions g _{i} : R ^{n} → R, i ∈ I are concave on R ^{n} and nonnegative on X.
2. The functions h _{j} max _{j}_{∈}_{J} h _{j} (x _{0} ) <
: R ^{n}
0.
→ R, j
∈ J are convex and there exists some x _{0} ∈ R ^{n} such that α _{0} :=
If the functions f _{i} , i ∈ I are convex and g _{i} , i ∈ I are concave, this does not necessarily imply that the ratio of these two functions is convex. Consequently, the generalized fractional programming problem (P ) is not necessarily a convex optimization problem.
Introducing now for every α _{0} < α < 0 the nonempty perturbed feasible region
X(α) := {x ∈ R ^{n} : h _{j} (x) ≤ α,
j ∈ J}
we deﬁne for α _{0} < α ≤ 0 the following perturbed optimization problem
v(α) := min _{x}_{∈}_{X}_{(}_{α}_{)} p(x).
By the deﬁnition of v(α), it is clear that λ ^{∗} = v(0) and 0 ≤ v(α) ≤ λ ^{∗} for α _{0} < α < 0. In the next lemma we give a lower bound on λ ^{∗} − v(α).
Lemma 3.4 If g _{∗} := inf _{x}_{∈}_{X} g _{m}_{i}_{n} (x) > 0 and Assumption 3.1 holds, then it follows that
λ ^{∗} − v(α) ≥
^{µ} ^{} (α)α
∗
g ∗
with µ ^{} (α) the optimal Lagrangean multiplier associated with the convex optimization problem
∗
min _{x}_{∈}_{X}_{(}_{α}_{)} φ(v(α), x).
Proof. By the strong duality theorem (cf.[8]) we obtain for every α _{0} < α < 0 that
φ(v(α)) = min _{x}_{∈}_{X} φ(v(α), x)
= max _{µ}_{≥}_{0} min _{x}_{∈}_{R} n φ(v(α), x) + µ ^{} (h(x) − α) + µ ^{} α.
10
April 13, 2005
4
OBSERVATIONS FOR PERFORMANCE IMPROVEMENT
This shows applying Lemma 2.2 and using again the strong duality theorem that
φ(v(α)) ≥ min _{x}_{∈}_{R} n φ(v(α), x) + µ ^{} (α)(h(x) − α) + µ ^{} (α)α
∗
∗
= max _{µ}_{≥}_{0} min _{x}_{∈}_{R} n φ(v(α), x) + µ ^{} (h(x) − α) + µ ^{} (α)α
= min _{x}_{∈}_{X}_{(}_{α}_{)} φ(v(α), x) + µ ^{} (α)α = µ ^{} (α)α.
∗
∗
∗
(3.15)
By relation (3.15) it follows for every x ∈ X that there exist some i ∈ I satisfying
and this shows the desired result.
f _{i} (x) − v(α)g _{i} (x) ≥ µ ^{} (α)α.
∗
Using Lemma 3.4 one can show the following result.
Lemma 3.5 Let Assumption 3.1 hold and assume g _{∗} > 0 and 2κ > α _{0} with κ := −β _{m}_{+}_{p} ( ). If the algorithm stops at iteration k _{0} , then the vector x _{k} _{0} _{−}_{1} belongs to X and
0 ≥ λ ^{∗} − λ _{k} _{0} ≥ ^{2}^{κ}^{µ} ^{}
∗
(2κe)e + κ
g ∗
with µ ^{} (2κe) the optimal Lagrangean multiplier associated with the convex optimization problem
∗
min _{x}_{∈}_{X}_{(}_{2}_{κ}_{e}_{)} φ(v(2κe), x).
Proof. If the algorithm stops at iteration k _{0} , then clearly x _{k} _{0} _{−}_{1} belongs to X and so we obtain 0 ≥ λ ^{∗} −λ _{k} _{0} . Moreover, by the stopping rule we know that ψ _{} (λ _{k} _{0} ) ≥ 0 and this implies for every x ∈ R ^{n} that
ψ _{} (λ _{k} _{0} , x) ≥ 0. Applying now relation (3.4) and (3.13) yields for every x ∈ R ^{n} that
ψ(λ _{k} _{0} , x) ≥ κ
(3.16)
By relation (3.16) and using the deﬁnition of ψ(λ _{k} _{0} , x) we obtain for every x ∈ X(2κe) that
φ(λ _{k} _{0} ,
x) = ψ(λ _{k} _{0} , x) ≥ κ.
This implies for every x ∈ X(2κ) that p(x) − λ _{k} _{0} ≥ κg
−1
∗
, and so, it follows that
v(2κe) − λ _{k} _{0} ≥ κg
−1
∗ .
Applying now Lemma 3.4 and relation (3.17) yields
λ ^{∗} − λ _{k} _{0} = λ ^{∗} − v(2κe) + v(2κe) − λ _{k} _{0} ≥ (2κµ ^{} (2κe)e + κ)g
∗
and this shows the desired inequality.
−1
∗
(3.17)
Finally we note for the unconstrained case that if additionally the functions f _{i} and −g _{i} are convex for all
ε ) is a smooth unconstrained convex optimization problem as a
¯
i ∈ I, then the optimization problem ( P
λ
direct consequence of Assumption 2.2 and Assumption 3.1.
4 Observations for Performance Improvement
Decision makers often face large size problems and in many cases, the performance of an algorithm worsens when the size of the problem increases. So, improving the performance of an algorithm plays an important role for solving real problems. In this section, we present several observations that may lead to a performance improvement for the approximation approach discussed in the previous section.
11
April 13, 2005
4.1 Weaker Stopping Condition
4
OBSERVATIONS FOR PERFORMANCE IMPROVEMENT
4.1 Weaker Stopping Condition
Notice that in the constrained case of the proposed approximation method, the stopping rule, φ _{ε} (λ _{k} ) ≥ 0 in Step 3 of Algorithm 3.1 can be replaced by a weaker condition φ _{ε} (λ _{k} ) ≥ −γ for some γ > 0. Implementing this observation may reduce the number of iterations. However, in this case the accuracy of the results may also decrease. The following result provides an upper bound on the number of subproblems to be solved and gives an upper bound on the difference between the optimal objective function value, λ ^{∗} and the objective function value computed after applying the weaker stopping condition.
Lemma 4.1 Suppose k _{0} := min{k : φ _{ε} (λ _{k} ) ≥ −γ}, then it follows that
_{k} 0 _{≤} (λ _{1} − λ ^{∗} )g ^{∗}
γ
and λ _{k} _{0} − λ ^{∗} ≤ ^{γ} ^{+} ^{β} ^{m} ^{(}^{ε}^{)}
g
∗
Proof. By using λ _{k} ≥ λ ^{∗} for all k < k _{0} and relation (3.9), it follows that
λ ^{∗} − λ _{1} ≤
k _{0} −1
k=1
(λ _{k}_{+}_{1} − λ _{k} ) ≤ ^{−}^{γ}^{k} g ^{∗} ^{0}
^{.}
Multiplying both sides by −1 yields the ﬁrst part. Using now φ _{ε} (λ _{k} ) ≥ −γ and Lemma 3.2 shows that
λ ^{∗} − λ _{k} _{0} ≥ (−γ − β _{m} (ε))(g _{∗} ) ^{−}^{1} . Again multiplying both sides by −1 shows the second part.
A similar but less exciting discussion can be given for the unconstrained case after replacing the stopping rule, ψ _{ε} (λ _{k} ) ≥ 0 in Step 3 of Algorithm 3.2, by the weaker condition ψ _{ε} (λ _{k} ) ≥ −γ for some γ > 0. Here we state the result without the proof.
Lemma 4.2 Suppose k _{0} := min{k : ψ _{ε} (λ _{k} ) ≥ −γ}, then it follows that
_{k} 0 _{≤} (λ _{1} − λ ^{∗} )g ^{∗}
γ
and λ _{k} _{0} − λ ^{∗} ≤ ^{−}^{(}^{2}^{κ}^{µ} ^{}
∗
(2κe)e + κ)
g ∗
.
4.2 Normalization
In order to improve the rate of convergence, Crouzeix`
k by
et. al. proposed to replace problem (P _{λ} ) at iteration
min
x∈X
max
i∈I
^{} f _{i} (x) − λg _{i} (x) g _{i} (x _{k}_{−}_{1} )
.
(4.1)
Crouzeix` et. al. have shown that the rate of convergence becomes at least linear with the above modiﬁca tion and they have proved that whenever the sequence {x _{k} } is convergent, the convergence is superlinear [14]. Following our previous results, one way to explain the intuition behind the normalization is as
¯
follows. Introduce for every i ∈ I the functions φ _{i} : R ^{2}^{n}^{+}^{1} → R given by
φ _{i} (λ, x, y) := ^{f} ^{i} ^{(}^{x}^{)} ^{−} ^{λ}^{g} ^{i} ^{(}^{x}^{)}
¯
g _{i} (y)
¯
and the function φ : R ^{2}^{n}^{+}^{1} → R deﬁned by
¯
¯
φ(λ, x, y) := max φ _{i} (λ, x, y).
i∈I
The parametric optimization problem associated with the above function is now given by
¯
¯
φ(λ, y) := min φ(λ, x, y).
x∈X
12
April 13, 2005
4.3 Additional Remarks
4
OBSERVATIONS FOR PERFORMANCE IMPROVEMENT
Moreover, to deﬁne the approximation let
¯
φ _{ε} (λ, x, y) := εΠ _{m} (
¯
φ _{1} (λ, x, y)
_{ε}
,
·
·
·
,
¯
φ _{m} (λ, x, y)
ε
)
and consider the approximated parametric optimization problem
¯ ¯
φ _{ε} (λ, y) = min φ _{ε} (λ, x, y).
x∈X
The constrained algorithm with normalization as well as approximation is now given in Algorithm 4.1.
Algorithm 4.1 Constrained Case with Normalization and Approximation
1. Select x _{0} ∈ X
and set k := 1.
2. Set λ _{k} := p(x _{k}_{−}_{1} ).
3. Determine an element x _{k} of the set {x ∈ X :
¯
φ _{ε} (λ _{k} , x, x _{k}_{−}_{1} ) < 0}. If such an element does not
exist, then return (λ _{k} , x _{k}_{−}_{1} ), else set k := k + 1 and return to Step 2.
Before discussing the bounds on the improvement λ _{k}_{+}_{1} − λ _{k} for this algorithm, we introduce the fol
¯
lowing set valued mapping S : R × X × X ⇒ R given by
¯ 
¯ 
¯ 
S(λ, x, y) := {i ∈ I : 
φ _{i} (λ, x, y) = 
φ(λ, x, y)}. 
Lemma 4.3 Suppose Algorithm 4.1 stops at iteration k _{0} . The sequence λ _{k} , k < k _{0} generated by Algo rithm 4.1 is strictly decreasing and satisﬁes
g
_{i} (x _{k}_{−}_{1} )
¯
¯
φ(λ _{k} , x _{k} , x _{k}_{−}_{1} ) min _{g} _{i} _{(}_{x} _{k} _{)} ≤ λ _{k}_{+}_{1} − λ _{k} ≤ φ(λ _{k} , x _{k} , x _{k}_{−}_{1} ) max
¯
i∈ S
i∈L(x _{k} )
¯ ¯
where S := S(λ _{k} , x _{k} , x _{k}_{−}_{1} ).
g
_{i} (x _{k}_{−}_{1} )
g _{i} (x _{k} )
Proof. A straightforward modiﬁcation of the proof of Lemma 3.1.
Using similar discussions as in Lemma 3.2 and Lemma 4.1, we can also give results regarding the differ
¯
ence λ ^{∗} − λ _{k} _{0} and the number of iterations required to satisfy φ(λ _{k} , x _{k} , x _{k}_{−}_{1} ) > −γ. Notice that if x _{k} is close to x _{k}_{−}_{1} , then the value of max _{i}_{∈}_{L}_{(}_{x} _{k} _{)} (g _{i} (x _{k}_{−}_{1} )/g _{i} (x _{k} )) is approximately equal to 1. Therefore, in certain instances we expect that the convergence of the proposed method will improve considerably. Our
numerical experiments in Section 5 supports this observation.
4.3 Additional Remarks
The membership problems in Step 3 of Algorithm 3.1 and Algorithm 3.2 are usually solved by refor mulating them as optimization problems. As we have discussed before, the optimal solutions of these optimization problems do not necessarily yield a λ _{k} value that is closer to λ ^{∗} than a λ _{k} value computed by a point that satisﬁes the condition in Step 3 (see Example 2.1). Therefore, at any iteration if the solution method for the optimization problems ﬁnds a point that satisﬁes the condition in Step 3, then the solution method can be terminated and the algorithm can move to the next iteration. Solving the optimization prob lems in Step 3 constitutes the major computational burden of Algorithm 3.1 and Algorithm 3.2. In this
13
April 13, 2005
5
NUMERICAL RESULTS
regard, terminating the solution methods before ﬁnding the optimal solution may save some computational effort.
An important advantage of Algorithm 3.2 over Algorithm 3.1 is that at each iteration one needs to solve an unconstrained membership problem rather than a constrained membership problem. Usually, solving unconstrained problems is easier than solving the constrained counterparts. However, in Algorithm 3.2 the constraints are also incorporated into the problem and only an approximated function is used. Thus, the solution of the unconstrained problem may be an infeasible point of the original problem. On the other hand, the solution of the constrained membership problem in Algorithm 3.1 is always feasible. In order to balance the tradeoff between the speed and feasibility, at early iterations one can start with solving the unconstrained problems and then switch to solving the constrained problems.
Recall that in Lemma 3.1 and Lemma 3.3, the differences between λ _{k}_{+}_{1} and λ _{k} are given by
and
λ k − λ k+1 ≥
λ k − λ k+1 ≥
−φ(λ _{k} , x)
^{m}^{i}^{n} i∈L(x _{k} ) ^{g} i
_{(}_{x} k _{)}
−ψ(λ _{k} , x _{k} )
^{m}^{i}^{n} i∈L(x _{k} ) ^{g} i
_{(}_{x} k _{)} ,
(4.2)
(4.3)
respectively. Therefore, for faster convergence, one may consider to increase these differences as much as possible. In this case it seems reasonable to solve, in the constrained case, the optimization problem,
max
x∈X
−φ(λ _{k} , x)
^{m}^{i}^{n} _{i}_{∈}_{L}_{(}_{x}_{)} ^{g} _{i} _{(}_{x}_{)}
(4.4)
and in the unconstrained case, the following problem should be solved
max
Гораздо больше, чем просто документы.
Откройте для себя все, что может предложить Scribd, включая книги и аудиокниги от крупных издательств.
Отменить можно в любой момент.