Вы находитесь на странице: 1из 20

SOLVING GENERALIZED FRACTIONAL PROGRAMMING PROBLEMS WITH APPROXIMATION

S¸. I. Birbil , J. B. G. Frenk and S. Zhang

˙

Faculty of Engineering and Natural Sciences, Sabancı University, Orhanli-Tuzla, 34956, Istanbul, Turkey.

Econometric Institute, Erasmus University Rotterdam Postbus 1738, 3000 DR Rotterdam, The Netherlands.

Systems Engineering and Engineering Management The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.

ABSTRACT. The present paper proposes a new approach to solve generalized fractional program- ming problems with approximation. Capitalizing on two alternatives, we review the Dinkelbach-type methods and set forth the main difficulty in applying these methods. In order to cope with this dif- ficulty, we propose an approximation approach that can be controlled by a predetermined parameter. The proposed approach is promising particularly when it is acceptable to find an effective, but near- optimal value in an efficient manner. Once the approximation parameter is fixed, our analysis shows how good is the value found by the approximation. In addition, we present several observations that may be suitable for improving the performance of the proposed approach. Finally, we support our discussion through numerical experiments.

Keywords. Generalized fractional programming, approximation approach, near-optimal solution, per- formance improvement observations

1

Introduction

Decision makers in different application areas demand effective and easy-to-use solution methods for their optimization problems. In many circumstances, they are ready to welcome a new approach even though this approach may yield a value close-enough to the actual (or theoretical) optimal value. One way of satisfying this demand is incorporating approximation approaches into well-known solution methods. To our advantage, the precision of an approximation approach depends on predefined parameters. Therefore through an approximation parameter, the trade-off between the quality of the results and the simplification of the problem can be controlled.

In this paper, we focus on solving a generalized fractional programming (GFP) problem by an approx- imation approach. GFP problems arise in many real-life applications. One early example dates back to the time of von Neumann when he used GFP in modeling an expanding economy [28]. Another im- portant application is within the field of multi-objective programming. In this case, the main goal is to minimize the maximum deviation of the fractional objectives, like input-output ratios, from their target values [20, 26, 25, 24]. Also in numerical analysis, GFP methods are used for solving rational Chebyshev

Research supported by Hong Kong RGC Earmarked Grants CUHK4233/01E and CUHK4174/03E.

2

GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD

approximations [2, 11, 21, 19]. Some other applications of fractional programming have been given in the literature; see [17].

To study the theoretical properties of GFP problems, researchers made use of the tools from duality theory, nonlinear programming and numerical analysis. Naturally, the study of the theoretical properties led to the development of various solution methods [13, 14, 12, 6, 5, 3, 20, 22, 27]. The majority of these methods are the variations of the approach suggested in the pioneering work of Crouzeix` et. al. [13]. In the literature, these methods are also known as Dinkelbach-type methods [15]. Although not immediately clear, it can be shown that Dinkelbach-type methods are actually cutting plane methods [4]. These methods are based on the idea of solving a sequence of auxiliary problems so that the solutions of the auxiliary problems converge to the solution of the GFP problem. These methods are very powerful. However, their performances heavily depend on the effective solutions of the auxiliary problems. In general the auxiliary problems are nonsmooth and nonlinear. Therefore, they might pose a major difficulty in solving the GFP problems effectively and easily. In a recent review by Frenk and Schaible, the theoretical properties of fractional programming problems are extensively studied [18].

The main idea of this paper is replacing the difficult auxiliary problems in Dinkelbach-type methods with relatively simple approximations. Approximations of these sort yield ε-optimal (close-enough) values to the auxiliary problems. Consequently, we will analyze the error committed by replacing the auxiliary problems with their approximated counterparts. Especially for large size problems, the task of speeding up the algorithms plays an important role. Therefore, we will also propose several approaches that may lead to a performance improvement. During our analysis, we will work with a generic approximation function since the choice of the approximation function does not alter our results. In order to illustrate our idea, we will then give two particular approximation approaches that will be also used in the numerical results section. Overall, this paper aims at providing a useful reference for a decision maker who agrees to solve a GFP problem effectively at the cost of finding inexact but controlled solutions.

2 Generalized Fractional Programming and the Dinkelbach Method

In this section we give an overview on generalized fractional programming and the most common solution approach, namely, the Dinkelbach method. Before proceeding to our subsequent analysis, let us first introduce the generalized fractional programming problem

λ :=

inf

xX max

iI

f i (x) g i (x)

(P )

where X R n is the feasible region, I := {1, 2, · · · , m} is a finite index set and for all i I, f i : R n R and g i : R n R are the problem functions. In addition, if we define the function p : X (−∞, ] by

(2.1)

p(x) := max

iI

f i (x) g i (x)

,

then the optimization problem (P ) can be written as

λ :=

xX p(x).

inf

(2.2)

In the sequel we denote the optimal solution of problem (P ) by x .

Introducing now the functions g min : R n R by

g min (x) := min

iI

g i (x),

(2.3)

it is necessary to impose the following assumption to avoid pathological instances of problem (P ).

2

GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD

Assumption 2.1 g min (x) > 0 for every x X.

The main idea of the Dinkelbach method is to provide a solution to the difficult optimization problem (P ) by solving a sequence of relatively simple parameterized problems. In this overview section, we focus on two main Dinkelbach-type approaches. The first approach is based on solving constrained subprob- lems, whereas the second approach aims at reformulating the parameterized subproblems as unconstrained nonlinear optimization problems.

We start with the first approach by defining for every i I, the functions φ i : R n+1 R given by

φ i (λ, x) := f i (x) λg i (x)

(2.4)

and for every (λ, x) R n+1 , the function

φ(λ, x) := max

iI

φ i (λ, x).

(2.5)

This leads to the associated parametric constrained optimization problem

φ(λ) := inf

xX φ(λ, x).

(P λ )

After minor modifications of the proofs in [13], one can show the following important results (see also

[18]).

Lemma 2.1 φ(λ) 0 if and only if −∞ < λ λ .

Lemma 2.2 The following conditions are equivalent:

1. The optimization problem (P ) has an optimal solution.

2. The set V defined by

V := {λ R :

Problem (P λ ) has an optimal solution and φ(λ) = 0}

is nonempty.

3. The set V is a singleton with V = {λ }.

We now discuss the unconstrained approach. In this case the feasible region X is given by

X := {x R n : h j (x) 0, j J},

(2.6)

where J

with h(x) := (h 1 (x), · · · , h p (x)) T . Clearly, this form of the feasible set is a commonly used form for numerical purposes.

: R n R are real-valued functions

:= {1, 2, · · · , p} is a finite index set and for all j J, h j

In addition to Assumption 2.1, we also assume for the unconstrained approach that the following condi- tion holds.

Assumption 2.2 The function x h j (x), j J are convex and the set X has a nonempty interior, i.e., there exist some x 0 R n such that max jJ h j (x 0 ) < 0.

2

GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD

As before, we start by introducing the parametric function ψ i : R n+1 R given by

ψ i (λ, x) :=

f i (x) λg i (x), h i (x),

for i I

for i

J

and consider for (λ, x) R n+1 , the function

ψ(λ, x) := max

iIJ ψ i (λ, x)

with the parametric unconstrained optimization problem

ψ(λ) :=

inf n ψ(λ, x).

xR

(2.7)

(2.8)

¯

( P λ )

It is shown in [27, 1] that the following versions of the previous two lemmas hold.

Lemma 2.3 ψ(λ) 0 if and only if −∞ < λ λ .

Lemma 2.4 The following conditions are equivalent:

1. The optimization problem (P ) has an optimal solution.

2. The set W defined by

¯

W := {λ R : Problem ( P λ ) has an optimal solution and ψ(λ) = 0}

is nonempty.

3. The set W is a singleton with W = {λ }.

Using the above results, we can discuss the computational procedures for both constrained and uncon- strained cases. In order to apply the Dinkelbach-type approach corresponding to the constrained case, we always assume that problem (P ) and problems (P λ ) for λ λ are solvable. The general scheme of the constrained case has now the form given in Algorithm 2.1.

Algorithm 2.1 Constrained Case

1. Select x 0 X

and set k := 1.

2. Set λ k := p(x k1 ).

3. Determine an element x k of the set {x X : φ(λ k , x) < 0}. If such an element does not exist, then return (λ k , x k1 ), else set k := k + 1 and return to Step 2.

¯

As in the constrained case, we assume that problem (P ) and problems ( P λ ) for λ λ are solvable.

Observe, if ψ(λ k , x k ) < 0, then it follows automatically that h j (x k ) < 0 for every j J, and hence,

The algorithm for the unconstrained Dinkelbach-type approach has now the form given in

Algorithm 2.2.

x k

X.

2

GENERALIZED FRACTIONAL PROGRAMMING AND THE DINKELBACH METHOD

( ) ( ,x k+1 ) ( ,x ) k k+1 k k+2
( )
( ,x
k+1
)
( ,x )
k
k+1
k
k+2

Figure 1: Two successive iterations of Algorithm 2.1.

Algorithm 2.2 Unconstrained Case

1. Select x 0 X and set k := 1.

2. Set λ k := p(x k1 ).

3. Determine an element x k of the set {x R n : ψ(λ k , x) < 0}. If such an element does not exist, then return (λ k , x k1 ), else set k := k + 1 and return to Step 2.

The geometrical interpretation of Algorithm 2.1 is illustrated in Figure 1. Clearly, for any x k ∈ {x X : φ(λ, x) < 0} we have φ(λ) φ(λ, x k ). Hence, at the k th iteration of Algorithm 2.1, the nonconvex function φ is approximated from above by the polyhedral convex function λ φ(λ, x k ). As a direct consequence of Lemma 2.2, our main goal is to compute λ that satisfies φ(λ) = 0. Instead of this ambitious goal, we solve the approximating equation φ(λ, x k ) = 0, which leads to

f i (x k ) g i (x k )

.

λ = max

iI

(2.9)

Notice that this solution is indeed the next iteration in Algorithm 2.1 (Step 2). A similar interpretation can be given for Algorithm 2.2, where the function ψ is approximated by the polyhedral convex function λ ψ(λ, x k ).

In Step 3 of both algorithms we suggest to solve membership problems instead of solving the opti-

¯

mization problems (P λ ) and ( P λ ). The reason for this suggestion is given by the following observation. Consider the constrained approach and suppose at iteration k we compute the optimal solution x k of prob- lem (P λ k ) and at the same time we choose a different vector x k satisfying φ(λ k , x k ) < 0. It might now happen that the value λ k+1 := p(x k ) is closer to λ than the value λ k+1 derived from x k . An instance showing this behavior is given in Example 2.1.

Example 2.1 Suppose we want to solve

λ =

x[0,2] max{ 7x + 1

inf

 

3x 2

2x + 2

, 18x + 2 4x + 1

,

16x +

3 }.

3

THE APPROXIMATION APPROACH

Let x 0 = 1 then λ 1 = p(x 0 ) = 0.5263. Notice that problem (P λ 1 ) can be solved by a standard linear

= 0.3295 with the

objective function value φ(λ 1 , x ) = 1.4468. At the next iteration, we need to compute λ 2 := p(x ) =

programming solver.

After solving problem (P λ 1 ), we find the optimal solution x

1

1

1

0.1228. On the other hand, instead of the optimal solution, if we select x 1 = 0.30 yielding φ(λ 1 , x 1 ) = 1.2368, then the corresponding value of λ at the next iteration becomes λ 2 := p(x 1 ) = 0.1410, which

is less than λ

2 .

This suggests that investing time in solving problem (P λ ) to optimality might not always be the best strategy. On the other hand, to determine whether there exists an element of the set {x X : φ(λ k , x) < 0} or decide whether this set is empty, it is most natural to solve problem (P λ k ), and so x k becomes a natural candidate. This observation may also become useful whenever the optimization routine for solving the auxiliary problems requires a considerable effort. In this case, one may stop as soon as the optimization routine finds a feasible, but not necessarily optimal, point x that solves the membership problem. As Lemma 2.1 and Lemma 2.3 show, from a practical point of view, we only need a decreasing sequence of λ k values.

3 The Approximation Approach

Notice that in both Algorithm 2.1 and Algorithm 2.2, the membership problems at Step 3 can be solved by reformulating the problems as nonlinear optimization problems, i.e., one needs to solve (approximately, see Example 2.1), in the constrained case, the auxiliary optimization problem

min max{f 1 (x) λg 1 (x), · · · , f m (x) λg m (x)}

xX

and in the unconstrained case, the problem

min n max{f 1 (x) λg 1 (x), · · · , f m (x) λg m (x), h 1 (x), · · · , h p (x)}.

xR

(3.1)

(3.2)

Although the max function is a convex increasing function, unfortunately it is nonsmooth. Therefore, we still encounter a nonsmooth objective function in the auxiliary optimization problems. One promising solution to this obstacle is to approximate the max function by a smooth alternative. Consider now a smooth, convex and increasing function Π s : R s R that satisfies

lim

ε0

εΠ s ( y ) = max{y 1 , · · · , y s }

ε

(3.3)

and

0 εΠ s ( y ) max{y 1 , · · ·

ε

,

y s } ≤ β s (ε)

(3.4)

for every y R s . As a direct consequence of the above two relations, the function y εΠ s ( y ) becomes a good candidate for approximating the nonsmooth objective functions of the auxiliary problems. Let us now give two examples that will also be used in the numerical results section.

ε

Example 3.1 It is easy to check that the entropy-type function

Π s (y) = log

s

i=1

e y i

is convex, differentiable and increasing [16]. Moreover, it satisfies relation (3.3) and relation (3.4) with β s (ε) = ε log s [7].

3.1

Constrained Case with Approximation

3

THE APPROXIMATION APPROACH

Example 3.2 In some problems, entropy-type function in Example 3.1 may create overflow problems be- cause of the exponential function. Recently Birbil et. al. proposed a more stable approximation method that aims at extending two dimensional approximation functions to higher dimensions by recursion [9]. The main idea of the method is based on the observation that the higher dimensional max function can be written recursively as follows

max{y 1 , · · ·

, y s } = max{max{y 1 , · · · , y q }, max{y q+1 , · · · , y s }},

where 1 < q < s. Following this observation, the method proposed by Birbil et. al. provides a twice differentiable convex increasing function Π s that satisfies relation (3.3). In this case, the error term in (3.4) is given by β s (ε) = 2 ε (log 2 (s 1) + 1)Π 2 (0), where Π 2 : R 2 R is the base function that is used for approximating the two dimensional max function, i.e., lim ε0 εΠ 2 (y/ε) = max{y 1 , y 2 }.

3.1 Constrained Case with Approximation

We first start with the constrained case of the Dinkelbach-type method. Given ε > 0, define the function φ ε : R × X R by

φ ε (λ, x) := εΠ m f 1 (x) λg 1 (x)

ε

, · · · , f m (x) λg m (x)

ε

.

(3.5)

Hence, instead of problem (3.1), we can consider the smooth optimization problem

φ ε (λ) := min

xX

φ ε (λ, x).

ε

λ

(P

)

To derive the difference in the optimal objective function values between optimization problems (P λ ) and

(P

(3.6)

By the same relation, we also obtain that φ ε (λ, x) φ(λ, x) for every x X and λ R and this shows

ε ) we observe by relation (3.4) and x(λ) belonging to arg min xX φ ε (λ, x) that

λ

φ ε (λ) φ(λ) φ ε (λ, x(λ))

φ(λ, x(λ)) β m (ε).

φ ε (λ) φ(λ)

for every λ R. Hence it follows by relations (3.6) and (3.7) that

(3.8)

for every λ R. Therefore at the expense of some limited error, we consider the following approximated Dinkelbach-type method.

(3.7)

0 φ ε (λ) φ(λ) β m (ε)

Algorithm 3.1 Constrained Case with Approximation

1. Select x 0 X and set k := 1.

2. Set λ k := p(x k1 ).

3. Determine an element x k of the set {x X : φ ε (λ k , x) < 0}. If such an element does not exist, then return (λ k , x k1 ), else set k := k + 1 and return to Step 2.

Before discussing the properties of Algorithm 3.1, we first introduce the nonempty set-valued mappings S : R × X I and L : X I given by

S(λ, x) := {i I : φ i (λ, x) = φ(λ, x)}

3.1

Constrained Case with Approximation

3

THE APPROXIMATION APPROACH

and

L(x) := {i I :

f i (x) g i (x)

= p(x)}.

It is now easy to derive the following upper and lower bounds on the difference λ k+1 λ k for every k < k 0 , with k 0 ≤ ∞ being the total number of iterations spent in Algorithm 3.1 before meeting the stopping condition.

Lemma 3.1 The sequence λ k , k < k 0 generated by Algorithm 3.1 is strictly decreasing and satisfies

φ(λ k , x k )

max iS(λ k ,x k )

g i (x k ) λ k+1 λ k

φ(λ k , x k )

min iL(x k ) g i (x k ) .

Proof. For every k < k 0 we know by relation (3.4) and (3.5) that φ(λ k , x k ) φ ε (λ k , x k ) < 0, and since φ(λ k+1 , x k ) = 0 and λ φ(λ, x k ) is strictly decreasing, this yields λ k+1 < λ k . We now verify the above inequalities. By the definition of λ k+1 , it follows for every i belonging to arg max iS(λ k ,x k ) g i (x k ) satisfying g i (x k ) = max iS(λ k ,x k ) g i (x k ) that

λ k+1 λ k f i (x k ) λ k g i (x k ) = g i (x k )

φ(λ k , x k )

max iS(λ k ,x k )

g i (x k ) .

Also for every i belonging to arg min iL(x k ) g i (x k ) satisfying g i (x k ) = min iL(x k ) g i (x k ), we obtain by using again the definition of λ k+1 that

λ k+1 λ k = f i (x k ) λ k g i (x k ) g i (x k )

and this completes the proof.

φ(λ k , x k ) min iL(x k ) g i (x k ) ,

If at the kth step of Algorithm 3.1, we observe that φ ε (λ k , x k ) δ for some δ < 0, and it holds that g := max xX max iI g i (x) < , then it follows by Lemma 3.1 and relation (3.4) that

λ k+1 λ k

φ ε (λ k , x k )

min iL(x k ) g i (x k )

δ

.

g

(3.9)

Hence in this case, the improvement in the objective function value is at least δ/g , and in most cases it can be expected that this is a conservative bound. In addition to the number of iterations, we are also interested in the quality of the solution generated by Algorithm 3.1. This means that one likes to derive an upper bound on the difference λ λ k 0 with k 0 being the iteration at which the stopping condition applies. First we need the following lemma.

Lemma 3.2 If g := inf xX g min (x) > 0, then it follows for every k < k 0 that

0

λ λ k+1 φ ε (λ k+1 ) β m (ε)

g

.

Proof. It is obvious that 0 λ λ k+1 and to prove the reverse inequality, we observe for every k < k 0 , x an optimal solution of optimization problem (P ) and i belonging to arg max iS(λ k+1 ,x ) g i (x ) that

0 λ λ k+1 f i (x ) λ k+1 g i (x ) = g i (x )

φ(λ k+1 , x )

max iS(λ k+1 ,x )

g i (x ) .

(3.10)

3.2

Unconstrained Case with Approximation

3

THE APPROXIMATION APPROACH

This shows that φ(λ k+1 , x ) 0, and since max iS(λ k+1 ,x ) g i (x ) g , by using relation (3.10) we obtain that

(3.11)

φ(λ k+1 , x )

g i (x ) φ(λ k+1 , x )

g

.

λ λ k+1

max iS(λ k+1 ,x )

Applying relation (3.4) yields

φ(λ k+1 , x ) φ ε (λ k+1 , x ) β m (ε) φ ε (λ k+1 ) β m (ε),

and by relation (3.11) the desired result follows.

Notice that the stopping condition in Algorithm 3.1 is satisfied if φ ε (λ k 0 ) 0 for finite k 0 . Therefore, in combination with Lemma 3.2, we have

λ λ k 0 β m (ε)

g

.

(3.12)

Finally we observe in this subsection that the optimization problem (P

λ

ε ) is a smooth convex optimization

problem if λ > 0, X is a convex set defined by differentiable convex inequalities and the functions f i , g i for all i I are differentiable and convex. In this case, the function x g min (x) is also a concave function and hence, the value g can be computed by solving a concave minimization problem. Such a problem is in general NP-hard [23].

3.2 Unconstrained Case with Approximation

Similar to the previous section, we start with introducing an approximation function ψ ε : R n+1 R defined by

ψ ε (λ, x) := εΠ m+p f 1 (x) λg 1 (x)

ε

, · · · , f m (x) λg m (x) , h 1 (x)

ε

ε

, · · · , h p (x) .

ε

We can now consider the smooth optimization problem

ψ ε (λ) := min n ψ ε (λ, x).

xR

(3.13)

(

¯

P

ε

λ

)

In this case the counterpart of relation (3.8) becomes

(3.14)

for every λ R. The adapted version of Algorithm 2.2 with the approximation is now given in Algorithm

3.2.

0 ψ ε (λ) ψ(λ) β m+p (ε)

Algorithm 3.2 Unconstrained Case with Approximation

1. Select x 0 X and set k := 1.

2. Set λ k := p(x k1 ).

3. Determine an element x k of the set {x R n : ψ ε (λ k , x) < 0}. If such an element does not exist, then return (λ k , x k1 ), else set k := k + 1 and return to Step 2.

Observe, if ψ ε (λ k , x) < 0, then ψ(λ k , x) ψ ε (λ k , x) < 0, and hence x belongs to X. As before we are now interested in the difference λ k+1 λ k with the sequence of λ k values generated by Algorithm 3.2. Again, k 0 denotes the total number of iterations spent in Algorithm 3.2 before meeting the stopping condition.

3.2

Unconstrained Case with Approximation

3

THE APPROXIMATION APPROACH

Lemma 3.3 The sequence λ k , k < k 0 generated by Algorithm 3.2 is strictly decreasing and satisfies

λ k+1 λ k (ψ(λ k , x k ))( min k ) g i (x k )) 1 .

iL(x

Proof. Redoing the last part of the proof of Lemma 3.1 leads to

λ k+1 λ k (φ(λ k , x k ))( min k ) g i (x k )) 1

iL(x

and using now φ(λ k , x k ) ψ(λ k , x k ) yields the desired result.

In order to give an upper bound on the error λ k λ we will use the strong duality theorem for Lagrangean duality. To apply this result, we need to impose some well known additional properties on the functions f i and g i .

Assumption 3.1 The following conditions hold:

1. The functions f i : R n R, i I are convex on R n and nonnegative on X. Moreover, the functions g i : R n R, i I are concave on R n and nonnegative on X.

2. The functions h j max jJ h j (x 0 ) <

: R n

0.

R, j

J are convex and there exists some x 0 R n such that α 0 :=

If the functions f i , i I are convex and g i , i I are concave, this does not necessarily imply that the ratio of these two functions is convex. Consequently, the generalized fractional programming problem (P ) is not necessarily a convex optimization problem.

Introducing now for every α 0 < α < 0 the nonempty perturbed feasible region

X(α) := {x R n : h j (x) α,

j J}

we define for α 0 < α 0 the following perturbed optimization problem

v(α) := min xX(α) p(x).

By the definition of v(α), it is clear that λ = v(0) and 0 v(α) λ for α 0 < α < 0. In the next lemma we give a lower bound on λ v(α).

Lemma 3.4 If g := inf xX g min (x) > 0 and Assumption 3.1 holds, then it follows that

λ v(α)

µ (α)α

g

with µ (α) the optimal Lagrangean multiplier associated with the convex optimization problem

min xX(α) φ(v(α), x).

Proof. By the strong duality theorem (cf.[8]) we obtain for every α 0 < α < 0 that

φ(v(α)) = min xX φ(v(α), x)

= max µ0 min xR n φ(v(α), x) + µ (h(x) α) + µ α.

4

OBSERVATIONS FOR PERFORMANCE IMPROVEMENT

This shows applying Lemma 2.2 and using again the strong duality theorem that

φ(v(α)) min xR n φ(v(α), x) + µ (α)(h(x) α) + µ (α)α

= max µ0 min xR n φ(v(α), x) + µ (h(x) α) + µ (α)α

= min xX(α) φ(v(α), x) + µ (α)α = µ (α)α.

(3.15)

By relation (3.15) it follows for every x X that there exist some i I satisfying

and this shows the desired result.

f i (x) v(α)g i (x) µ (α)α.

Using Lemma 3.4 one can show the following result.

Lemma 3.5 Let Assumption 3.1 hold and assume g > 0 and 2κ > α 0 with κ := β m+p ( ). If the algorithm stops at iteration k 0 , then the vector x k 0 1 belongs to X and

0 λ λ k 0 2κµ

(2κe)e + κ

g

with µ (2κe) the optimal Lagrangean multiplier associated with the convex optimization problem

min xX(2κe) φ(v(2κe), x).

Proof. If the algorithm stops at iteration k 0 , then clearly x k 0 1 belongs to X and so we obtain 0 λ λ k 0 . Moreover, by the stopping rule we know that ψ (λ k 0 ) 0 and this implies for every x R n that

ψ (λ k 0 , x) 0. Applying now relation (3.4) and (3.13) yields for every x R n that

ψ(λ k 0 , x) κ

(3.16)

By relation (3.16) and using the definition of ψ(λ k 0 , x) we obtain for every x X(2κe) that

φ(λ k 0 ,

x) = ψ(λ k 0 , x) κ.

This implies for every x X(2κ) that p(x) λ k 0 κg

1

, and so, it follows that

v(2κe) λ k 0 κg

1

.

Applying now Lemma 3.4 and relation (3.17) yields

λ λ k 0 = λ v(2κe) + v(2κe) λ k 0 (2κµ (2κe)e + κ)g

and this shows the desired inequality.

1

(3.17)

Finally we note for the unconstrained case that if additionally the functions f i and g i are convex for all

ε ) is a smooth unconstrained convex optimization problem as a

¯

i I, then the optimization problem ( P

λ

direct consequence of Assumption 2.2 and Assumption 3.1.

4 Observations for Performance Improvement

Decision makers often face large size problems and in many cases, the performance of an algorithm worsens when the size of the problem increases. So, improving the performance of an algorithm plays an important role for solving real problems. In this section, we present several observations that may lead to a performance improvement for the approximation approach discussed in the previous section.

4.1 Weaker Stopping Condition

4

OBSERVATIONS FOR PERFORMANCE IMPROVEMENT

4.1 Weaker Stopping Condition

Notice that in the constrained case of the proposed approximation method, the stopping rule, φ ε (λ k ) 0 in Step 3 of Algorithm 3.1 can be replaced by a weaker condition φ ε (λ k ) ≥ −γ for some γ > 0. Implementing this observation may reduce the number of iterations. However, in this case the accuracy of the results may also decrease. The following result provides an upper bound on the number of subproblems to be solved and gives an upper bound on the difference between the optimal objective function value, λ and the objective function value computed after applying the weaker stopping condition.

Lemma 4.1 Suppose k 0 := min{k : φ ε (λ k ) ≥ −γ}, then it follows that

k 0 (λ 1 λ )g

γ

and λ k 0 λ γ + β m (ε)

g

Proof. By using λ k λ for all k < k 0 and relation (3.9), it follows that

λ λ 1

k 0 1

k=1

(λ k+1 λ k ) γk g 0

.

Multiplying both sides by 1 yields the first part. Using now φ ε (λ k ) ≥ −γ and Lemma 3.2 shows that

λ λ k 0 (γ β m (ε))(g ) 1 . Again multiplying both sides by 1 shows the second part.

A similar but less exciting discussion can be given for the unconstrained case after replacing the stopping rule, ψ ε (λ k ) 0 in Step 3 of Algorithm 3.2, by the weaker condition ψ ε (λ k ) ≥ −γ for some γ > 0. Here we state the result without the proof.

Lemma 4.2 Suppose k 0 := min{k : ψ ε (λ k ) ≥ −γ}, then it follows that

k 0 (λ 1 λ )g

γ

and λ k 0 λ (2κµ

(2κe)e + κ)

g

.

4.2 Normalization

In order to improve the rate of convergence, Crouzeix`

k by

et. al. proposed to replace problem (P λ ) at iteration

min

xX

max

iI

f i (x) λg i (x) g i (x k1 )

.

(4.1)

Crouzeix` et. al. have shown that the rate of convergence becomes at least linear with the above modifica- tion and they have proved that whenever the sequence {x k } is convergent, the convergence is superlinear [14]. Following our previous results, one way to explain the intuition behind the normalization is as

¯

follows. Introduce for every i I the functions φ i : R 2n+1 R given by

φ i (λ, x, y) := f i (x) λg i (x)

¯

g i (y)

¯

and the function φ : R 2n+1 R defined by

¯

¯

φ(λ, x, y) := max φ i (λ, x, y).

iI

The parametric optimization problem associated with the above function is now given by

¯

¯

φ(λ, y) := min φ(λ, x, y).

xX

4.3 Additional Remarks

4

OBSERVATIONS FOR PERFORMANCE IMPROVEMENT

Moreover, to define the approximation let

¯

φ ε (λ, x, y) := εΠ m (

¯

φ 1 (λ, x, y)

ε

,

·

·

·

,

¯

φ m (λ, x, y)

ε

)

and consider the approximated parametric optimization problem

¯ ¯

φ ε (λ, y) = min φ ε (λ, x, y).

xX

The constrained algorithm with normalization as well as approximation is now given in Algorithm 4.1.

Algorithm 4.1 Constrained Case with Normalization and Approximation

1. Select x 0 X

and set k := 1.

2. Set λ k := p(x k1 ).

3. Determine an element x k of the set {x X :

¯

φ ε (λ k , x, x k1 ) < 0}. If such an element does not

exist, then return (λ k , x k1 ), else set k := k + 1 and return to Step 2.

Before discussing the bounds on the improvement λ k+1 λ k for this algorithm, we introduce the fol-

¯

lowing set valued mapping S : R × X × X R given by

¯

¯

¯

S(λ, x, y) := {i I :

φ i (λ, x, y) =

φ(λ, x, y)}.

Lemma 4.3 Suppose Algorithm 4.1 stops at iteration k 0 . The sequence λ k , k < k 0 generated by Algo- rithm 4.1 is strictly decreasing and satisfies

g

i (x k1 )

¯

¯

φ(λ k , x k , x k1 ) min g i (x k ) λ k+1 λ k φ(λ k , x k , x k1 ) max

¯

iS

iL(x k )

¯ ¯

where S := S(λ k , x k , x k1 ).

g

i (x k1 )

g i (x k )

Proof. A straightforward modification of the proof of Lemma 3.1.

Using similar discussions as in Lemma 3.2 and Lemma 4.1, we can also give results regarding the differ-

¯

ence λ λ k 0 and the number of iterations required to satisfy φ(λ k , x k , x k1 ) > γ. Notice that if x k is close to x k1 , then the value of max iL(x k ) (g i (x k1 )/g i (x k )) is approximately equal to 1. Therefore, in certain instances we expect that the convergence of the proposed method will improve considerably. Our

numerical experiments in Section 5 supports this observation.

4.3 Additional Remarks

The membership problems in Step 3 of Algorithm 3.1 and Algorithm 3.2 are usually solved by refor- mulating them as optimization problems. As we have discussed before, the optimal solutions of these optimization problems do not necessarily yield a λ k value that is closer to λ than a λ k value computed by a point that satisfies the condition in Step 3 (see Example 2.1). Therefore, at any iteration if the solution method for the optimization problems finds a point that satisfies the condition in Step 3, then the solution method can be terminated and the algorithm can move to the next iteration. Solving the optimization prob- lems in Step 3 constitutes the major computational burden of Algorithm 3.1 and Algorithm 3.2. In this

5

NUMERICAL RESULTS

regard, terminating the solution methods before finding the optimal solution may save some computational effort.

An important advantage of Algorithm 3.2 over Algorithm 3.1 is that at each iteration one needs to solve an unconstrained membership problem rather than a constrained membership problem. Usually, solving unconstrained problems is easier than solving the constrained counterparts. However, in Algorithm 3.2 the constraints are also incorporated into the problem and only an approximated function is used. Thus, the solution of the unconstrained problem may be an infeasible point of the original problem. On the other hand, the solution of the constrained membership problem in Algorithm 3.1 is always feasible. In order to balance the trade-off between the speed and feasibility, at early iterations one can start with solving the unconstrained problems and then switch to solving the constrained problems.

Recall that in Lemma 3.1 and Lemma 3.3, the differences between λ k+1 and λ k are given by

and

λ k λ k+1

λ k λ k+1

φ(λ k , x)

min iL(x k ) g i

(x k )

ψ(λ k , x k )

min iL(x k ) g i

(x k ) ,

(4.2)

(4.3)

respectively. Therefore, for faster convergence, one may consider to increase these differences as much as possible. In this case it seems reasonable to solve, in the constrained case, the optimization problem,

max

xX

φ(λ k , x)

min iL(x) g i (x)

(4.4)

and in the unconstrained case, the following problem should be solved

max