A simple Parameter free adaptive penalty based SP-QPSO algorithm

for mechanical design optimization problems

Abstract

This paper presents a hybrid algorithm, Shuffled Complex Evolution with Quantum Particle Swarm
Optimization abbreviate as SP-QPSO for solving mechanical constrained design optimization problems.
In recent years, researches’ interest in solving constrained optimization problems by Evolutionary
Algorithms. Quantum Particle Swarm Optimization (QPSO) is a variant of standard Particle Swarm
Optimization (PSO) algorithm and enhances the local search ability of the algorithm over feasible space.
Although QPSO is easier to implement, more intelligent and efficient than the standard PSO, but due to
its strategy, the efficiency of the QPSO algorithm's performance in solving complex and high-
dimensional problems has decreased. Normalization of constraints is crucial for the efficient
permanence of the algorithm. A growing number of researchers have proposed different strategies for
handling constraints. For solving constrained optimization problems, adaptive constraint-handling
techniques are very promising as they use information from the population during the search to adjust
different penalty parameters for each constraint without tuning any type of user defined penalty
parameter. In this paper, a novel parameter free adaptive penalty is employed to handle constraints in
the search region. The efficiency of SP-QPSO algorithm is numerically investigated using five engineering
design problems which have different natures of objective functions, constraints and decision variables
with up to six variables and six constraints. The experimental results are analyzed in comparison with
those reported in the literature. The computational results demonstrate that the proposed algorithm
has potential to achieve better solutions, compared with conventional PSO and other optimization
algorithms in terms of searching efficiency and computational time.

Keywords:
INTRODUCTION

The advancements in technology make the human life easier which brings new problems to the
optimization field. In real world applications of engineering field, designers are mostly simulating the
systems that have complicated structures that lead to more complex systems (Mazhoud, Hadj-Hamou,
Bigeon, & Joyeux, 2013). These systems are required to be optimized. There are two types of
optimization problems: constrained (Meng & Hongjian, n.d.; Y Wang, Cai, & Zhou, 2009) and
unconstrained problems (Liang, Qin, Suganthan, & Baskar, 2006). In the early stage, designers are
dealing with problems which are mostly constrained; therefore it is mandatory to develop an efficient
optimization method that can satisfy all the constraints.

Choosing an appropriate algorithm for optimization is very critical as optimization algorithms are
problem dependents (Yeniay, 2005). Evolutionary Algorithms (EAs) are stochastic population-based
optimization algorithms (Dong & Wang, 2014) that enable a group of populations to adopt themselves
to the surrounding environments. EAs are efficient and more flexible for solving constrained
optimization problems since they only require knowing the objective function. The rest of the
information such as differentiability; continuity of objective function or constraints are not required
(Kaveh & Talatahari, 2009; Mernik & Brest, 2006). According to literature, EAs are developed for solving
unconstrained problems and Penalty techniques are the most common techniques used by EAs to
convert constrained problems to unconstrained optimization problems (Rocha & Fernandes, 2009) .

In general, the mathematical form of constraint optimization problem (COP) is expressed as a

nonlinear optimization problem as follows(Ouyang, 2010):

Minimize f (⃗x ) (1)

Subject to g j ( ⃗x ) ≤ 0 j =1, … , p

hi (⃗x ) = 0 i = p +1, … , m

where f (⃗x ) is a real objective or criterion function defined on a feasible set, and the solution

⃗x =( x 1 , x 2 , … , x n ) ϵ Rn is the n-dimensional real vector. The inequality g j ( ⃗x ) ≤ 0 represents the j th

inequality constraint and hi ( ⃗x ) shows the i th equality constraint and there are p inequality and m− p

equality constraints. Both the objective function and constraints of COPs could be either linear or
nonlinear. Vector x is a feasible solution of the problem if it satisfies all the constraints and the feasible

region is constituted by all of the feasible solutions. An inequality constraint of the mathematical form

as: g j ( x ) ≥ 0 is not restrictive and can be converted to the −g j ( x ) ≤ 0 (J. Sun et al., 2007). Also to make it

more convenient, in this paper the equality constrained is converted to inequality as: |h i ( x )−ε ≤ 0 ;

Where the value of ε for these numerical experiments is set to 10−4 or 10−3 which depends on the

problem (Mazhoud et al., 2013).

COP is a central research topic in the field of optimization (Xiao, Huang, Cheng, He, & Niu, 2014).
However optimizing constrained problems is different from unconstrained problems. In unconstrained
problems, optimizer focuses on objective function to detect the global optimum but in COPs, the
optimizer tries to solve the objective function and discover the global optimum subject to some
constraints. In fact unconstrained optimization sometimes can be a sub-step in the process of solving
constrained optimization. Therefore, optimizing COPs is the process of finding optimum solutions
considering a set of specified constraints that have to be satisfied (Garg, 2014). Traditional Mathematical
Programming (MP) techniques are always the first option to solve optimization problems (Mezura-
Montes, E., & Flores-Mendoza, 2009). These methods are efficient enough to solve simple optimization
problems but engineering problems in real world are complex derivatives, their results are affected by
changing the initial values, and they require large amount of computation memory. Hence, MP
techniques will lose their efficiency as they mostly get trapped in local optimum solutions which make
them difficult to be applied on complex optimization problems (Mezura-Montes, E., & Flores-Mendoza,
2009). According to literature studies, one suitable algorithm to solve the constrained engineering
problems is Heuristic Methods (Fang, Sun, Ding, Wu, & Xu, 2010). Evolutionary Algorithms (EAs) are one
of the heuristic methods which have been successfully used for past two decades and became very
popular in different areas of optimization such as science and engineering. The EA algorithm generates
solutions near to the global optima. Moreover it has a simple algorithm as they do not need the
derivatives of objective function and constraints; hence EAs are more powerful and efficient than the
traditional numerical methods (Zbigniew Michalewicz, 1992; Xiao et al., 2014). However, the main
limitation of the algorithm is that EAs are originally presented to solve unconstrained optimization
problems (Poli, 2007). As it was discussed earlier, unconstrained problem could be a sub-step in the
process of optimizing COPs. Therefore, by using constraint-handling techniques (Rocha & Fernandes,
2009), EAs can be extended to constrained optimization problems.

According to the previous studies, for solving these optimization problems several algorithms are
introduced which penalty methods is the most well-known technique to overcome constrained
problems (Ali, Golalikhani, & Zhuang, 2014; Rocha & Fernandes, 2009). Based on the operation of the
penalty method, if one or more constraints are violated, the objective function will be penalized. Penalty
method penalizes the objective function by incorporating the constraints into objective function and
therefore, the constrained problem will be transformed to an unconstrained problem. In addition, the
penalty method plus objective function will be minimized by using algorithms of unconstrained
problems. Penalty methods are simple and easy to implement (Mallipeddi & Suganthan, 2008). The
strength of this approach is that they are generic and can be applied to any type of constrained
problems (including linear or non-linear) (Jadaan, Rajamani, & Rao, 2009). The penalty method is a
generic constraint-handling technique which can be easily applied to solve most of the optimization
problems. But this property of the approach fails in terms of the performance when it comes to the
efficiency analysis (Zbigniew Michalewicz & Schoenauer, 1996). Hence, penalty parameter is the
common critical issue that must be considered in all penalty methods since it plays a significant role in
increasing the efficiency of the approach’s performance (Zbigniew Michalewicz & Schoenauer, 1996).
Hence, choosing the appropriate penalty parameter which depends on the optimization problems, can
direct the search process of the optimization algorithm towards the constrained optimum and experts
are still attempting to find the suitable updating rule for penalty parameter (Ali et al., 2014). According
to the literature, there are some studies done in finding the right approach for calculating the penalty
parameter such as Death penalty method (Zbigniew Michalewicz & Schoenauer, 1996), Static penalty
method (Zbigniew Michalewicz & Schoenauer, 1996), Multi-level penalty methods (Homaifar, Qi, & Lai,
1994) , Dynamic penalty methods (Joines & Houck, 1994) , and etc.

In the past, constraint-handling mechanism has been applied on several optimization algorithms (Coello
Coello, 2002). As it was mentioned earlier, Evolutionary Algorithms are very popular for solving COPs
using penalty methods. For instance, Particle Swarm Optimization (J. Kennedy & Eberhart, 1995;
Mazhoud et al., 2013), which showed an efficient performance in solving engineering problems, and it
attracted attentions of the optimization and engineering communities. Kennedy and Eberhart (1995)
introduced the standard PSO as an evolutionary computation technique with the mechanism of
individual improvement, population cooperation and competition (He & Wang, 2007; He, Qie and Wang,
2007; J. Kennedy & Eberhart, 1995). PSO is one of the swarm intelligence algorithms for global
optimization over continuous search spaces (Fang et al., 2010). The algorithm is initially inspired by
observing the social manner of animals (birds, fishes) (Mazhoud et al., 2013). PSO has a flexible
mechanism which randomly generates the first population of candidate solutions. It is well-balanced
which increases the global (neighbor’s experience) and local (particle’s own experience) exploration
abilities of the algorithm. In PSO algorithm, particle indicates solution. Each particle updates their
velocity and position towards the global best position based on its own history and other particles’
histories. It cooperates and shares information among population (Poli, 2007). PSO has 3 important
parameters which affect the performance of the algorithm that have to be adjusted. These parameters
are: initial weight and two acceleration coefficients. Because of the strength points of standard PSO,
such as ease of implementation and inexpensive computation cost, it received the attention of the
researchers and optimizers which resulted in huge number of different variants of the standard PSO
algorithm to be proposed and implemented (Fang et al., 2010; He & Wang, 2007; Zbigniew Michalewicz,
1992; Poli, 2007).

Sun improved the PSO algorithm so that it produces the new particle near the weighted position of the
previous best particle and the global best particle (Sun, Lai, & Wu, 2011). This variant of PSO is called
quantum-behaved particle swarm optimization (QPSO) (J. S. J. Sun, Xu, & Feng, 2004). The difference
between QPSO and standard PSO algorithm is that QPSO does not have a velocity and position vector
for particles, instead it has a wave function. Therefore, variant QPSO has fewer parameters than PSO to
adjust. These algorithms performed well in low dimension problems but because the evolutionary
algorithms are random therefore the efficiency of their performance in searching complex and high-
dimensional spaces is decreased (Chu, Gao, & Sorooshian, 2011a).

So, to solve complex high-dimensional problems a novel strategy, SP-UCI is proposed (Chu et al., 2011a).
Bellman was first to introduce the problem of dimension increment of the search space using the term
“curse of dimensionality’’ (Bellman, 1961). SP-UCI algorithm is inspired by the shuffled complex
evolution-University of Arizona (SCE-UA) method (Beskow, Mello, Norton, & da Silva, 2011; Q.Y. Duan
V.K. Gupta, 1993). SPUCI divides the population to group of complexes, and each of the complexes is
considered as local area of the search space. For the exploration of feasible region and good
convergence toward the global optimum, SPUCI algorithm monitors the lost dimensions of the space
which is spanned by the particles in a population (complex), and restores them. This process starts by
performing local search on each complex. Checking the lost dimensions during the process of
optimization helps the algorithm to have an efficient performance in high dimensional problem space.
Global search is performed by replacing the information from similar local searches. According to the
results (Chu et al., 2011a), SP-UCI algorithm has an efficient performance in complex and problems with
high dimensionality, due to its monitoring ability for the lost dimensions through the process of
searching for global optimum (Chu et al., 2011a).

In this paper, a novel hybrid of two common EAs, QPSO and SP-UCI algorithms called SP-QPSO algorithm

using parameter-free penalty method is proposed for solving constrained engineering design

optimization problems. However, in order to increase the optimizer’s performance, the Centroidal

Voronoi Tessellations (CVT) method for point initialization, is applied on the SP-QPSO to assure that

points visit the entire search space. The proposed SP-QPSO algorithm is tested on a standard

benchmarks engineering design problems which are selected from (Barbosa & Lemonge, 2003;

Mazhoud et al., 2013); they are used to evaluate the efficiency of the SP-QPSO’s performance as

compared with the results of native EAs such as QPSO and SP-UCI algorithms.

The remaining part of the paper is organized as follows. In section 2, procedural steps of proposed

SP-QPSO and penalty based techniques in solving COPs are presented. The experimental results

based on well-known engineering design optimization problems and its comparison is presented in

Section 3. Section 4 concludes the paper and also the potential future works of the proposed

algorithm are discussed.

2. Hybrid SP-QPSO
In this section, the procedural steps of hybrid SP-QPSO algorithm, parameter-free adaptive penalty
technique for constraint handling technique are depicted.

2.1 Procedural steps of SP-QPSO

The procedural steps of the proposed hybrid SP-QPSO algorithm are as follows:

Step 1: Initialize the number of complex, number of points in each complex, sample size and
 dimension of the problem.

Step 2: Initialize sample points using Halton step method of Centroidal Voronoi Tessellations
(CVT) over the feasible space using a generator. The search space can be partitioned into
sections evenly and sort the points based on their calculated function value.

Step 3: Apply dimensionality check for rough fitness landscapes in high dimensional space.

Step 4: Store the population points in an array and divide the array into complexes.

Step 5: Apply QPSO strategy on each complex, so that each complex looks for the optimum
function value.

a. Initializing the population size and maximum number of iteration.

b. Building a sub swarm containing all the population points and calculating their
function values (local best).
c. Finding the best value by comparing all the local bests and assigning it to the global
best.
d. Calculating the mean best position of the population ( mbest ) and the movements of
particle toward the global best for the next iteration.
e. Repeating the process until the maximum number of iteration satisfied.

Step 6: Checking the probability of trapping in local minima by applying the multi-normal
resampling.

Step 7: Shuffling and replacing the complexes into an array and sorting it.

Step 8: Repeating the process until the stopping criteria satisfied.

2.2 Parameter free Adaptive Penalty Method for constraint handling

To deal with constraints, penalty function based methods for constraint handling have a long history of
development. In its native form, a penalty is applied to the fitness of a solution in the event it violates
one or more constraints [Coello, 2012]. Generally, the penalty based techniques used in EAs for
constraint optimization have parameters and the quality of their performance is totally depend on
tuning those parameters which need many trial experiences to determine the right value for the penalty
parameter. To overcome this limitation, adaptive penalty function methods are very effective for
constrained optimization since they can make use of the information obtained during the evolution to
adapt their parameters.

To select various penalties for each constraint during the process of searching for the global best point,
an adaptive scheme is implemented that utilizes population information such as, the mean of the
objective function and the level of violation of each constraint (Lemonge & Barbosa, 2004). The reason is
that higher penalty coefficient values should be allocated for constraints which are more difficult to be
satisfied. The number of elements violating a given constraint and the amount of violation is the
indication of the above-mentioned constraint difficulties. The j th coefficient is represented by k j. The
proposed fitness function is (Barbosa & Lemonge, 2003):

f ( x ) if x is feasible
F ( x )=
{ m
f ( x ) + ∑ k j v j ( x ) otherwise
j=1
(6)

Where the penalty parameter (k ¿ ¿ j) ¿at each generation is defined as follows (Barbosa & Lemonge,
2003):

k j=¿ (7)

pop

∑ f ( x i) pop
i=1
k j=
m pop 2 ∑ v j (x i ) (8)

[
∑ ∑ vl ( x )
l=1 i=1
i
] i =1

Where the mean of the objective function values in the current population is represented by ¿ f ( x ) >¿

and the violation of the lth constraint averaged over the current population is defined by ¿ v l ( x )> ¿.
“Pop” is denoting the population size. The adaptive penalty method (Wright & Farmani, 2001) is used
without any type of pre-defined penalty parameter to be incorporated with hybrid SP-QPSO, QPSO and
SPUCI algorithms, which efficiently handles constraints and dynamically adjusts the penalty during the
optimization.

3 Simulation experiments and results

The proposed hybrid SP-QPSO algorithm, QPSO and SP-UCI was coded in MATLAB 8.4, and then the
results were obtained by running the algorithms on Windows 8 with Intel Core personal computer
having 2.5 GHz processor and 4 GB RAM. For each case study, 25 independent runs are conducted and
each run is stopped when reaches a maximum number of iterations. For SP-QPSO, the parameters
include population size, number of iterations, number of complexes and numbers of points in
each complex. The remaining of parameters is assigned by default. Through an extensive
experimental fine tuning, the parameters recommended to best suit the proposed SP-UCI,
QPSO and SP-QPSO algorithms are listed in Table 1.
 Table 1 Setting of parameters.

Parameter SP-UCI QPSO SP-QPSO

Population size - 20 -
Maximum iteration number 10,000 10,000 10,000
Maximum number of 2 - 2
complexes
Maximum number of points - 100 100

3.1 Simulation results for Case study problems

Five mechanical engineering case study problems are used to assess the performance of the
proposed algorithm. The details of five case study problems namely, Tension/compression
spring design problem, speed reducer design problem, Welded beam design problem, Pressure
vessal design problem and three-bar truss design problem are taken from Garg, 2016. The
statistical significance results from experiments on the five case study problems in terms of the
best, the mean, the worst, and the standard deviation of the objective value of the solutions
obtained by SP-UCI, QPSO and SP-QPSO algorithms are presented in Table 2. Table 3-7 presents
a comparison of the results by the proposed algorithm and others from the literature (CPSO)
(HPSO), Accelerate the Adaptive Trade-off Model (AATM), Model based on the t(hymus-
derived) cell( T-CELL), Constraint Violation with Interval Particle Swarm Optimization (CVI-PSO),
Hybrid Shuffled Complex Evolution with Quantum Particle Swarm Optimization(SP-QPSO). 82).
Table 8-12 presents the final values of the design variables for the five case study problems (all
solutions are feasible). The summary of statistical results obtained from the proposed algorithm is
shown in Table 13.
Table 2 Statistical results for engineering design case study problems

Case study Criterion QPSO SPUCI SP-QPSO

Tension- Best 0.00275 0.002525 0.00250025
Median 0.00275 0.002525 0.00250025
compression Worst 0.00275 0.002525 0.00250025
spring Mean 0.00275 0.002525 0.00250025
Std 1.33484E-18 4.44947E-19 5.68318E-17
 Speed reducer Best 2815.937639 2804.66863 2801.27748
Median 2822.629179 2809.694851 2808.256754
Worst 2822.937403 2811.972544 2816.032413
Mean 2822.049252 2809.451438 2808.3918
Std 1.589223641 2.018560263 4.609711989
Welded beam Best 1.447348712 1.447514282 1.244675202
Median 1.742313 1.732253489 1.39052735
Worst 1.968926 1.974958 2.001221
Mean 1.743287 1.743287 1.597138
Std 0.146558426 0.134315 0.323085
Pressure vessal Best 5886.189623 5885.332774 5885.268397
Median 5889.511734 5885.334658 5885.268397
Worst 5985.004274 5885.570128 5885.282344
Mean 5901.206121 5885.35789 5885.269435
Std 27.11004843 0.057498178 0.003212602
Three-bar Truss Best 263.8958434 263.8950554 263.8950835
Median 263.8958437 263.8958553 263.8958127
design Worst 263.8958462 263.8959358 263.8959127
Mean 263.8958439 263.8957992 263.8957807
Std 6.21912E-07 0.000194462 0.000169173

Table 3 Comparative results of Tension-compression spring design problem

Algorithm Criterio CPSO HPSO AATM T-CELL CVI-PSO SP-QPSO

n
Tension/Compression String Best 0.012674 0.012665 0.0126682 0.01266 0.0126655 0.00250025
design 7 2 5
Worst 0.012924 0.012719 0.0128613 0.01330 0.0128426 0.00250025
0 1 9
Mean 0.012730 0.012707 0.0127080 0.01273 0.0127310 0.00250025
0 2 2
Std 5.19e-05 1.58e-05 4.5e-05 9.4e_05 5.58e_05 5.68318E-17
NFEs 200,000 81,000 25,000 36,000 25,000 10,000

Table 4 Comparative results of Pressure vessel design problem

Algorithm Criterio CPSO HPSO AATM T-CELL CVI-PSO SP-QPSO

n
Pressure vessel Best 6061.077 6059.714 6059.7255 6390.554 6059.7143 5885.268397
7 3 0
Worst 6363.804 6288.677 6090.8022 7694.066 6820.4101 5885.282344
1 0 8
Mean 6147.133 6099.932 6061.9878 6737.065 6292.1231 5885.269435
2 3 1
Std 86.4545 86.2022 4.7000 357.0000 288.4550 0.003212602
NFEs 200,000 81,000 30,000 81,000 25,000 10,000

Table 5 Comparative results of Welded beam design problem

Algorithm Criterion CPSO HPSO CVI-PSO SP-QPSO

Welded Best 1.72802 1.724852 1.724852 1.244675202

4
beam Worst 1.78214 1.8142950. 1.727665 2.001221
3
Mean 1.74883 1.749040 1.725124 1.597138
1
Std 0.01292 040049 6.12e-04 0.323085
6
NFEs 200,000 81,000 25,000 10,000
 Table 6 Comparative results of Three bar truss design problem

Algorithm Criterion SC(1) PSO- DEDS(3) HEAA(4) SP-QPSO

DE(2)
Three-bar Truss Best 263.89584 263.895843 263.89584 263.895843 263.8950835
6 3
design Worst 263.96975 263.895843 263.89584 263.896099 263.8959127
6 9
Mean 263.90335 263.895843 263.89584 263.895865 263.8957807
6 3
Std 1.3E−02 4.5E−10 9.7E−07 4.9E−05 0.000169173
NFEs 17,610 17,600 15,000 15,000 10,000

Table 7 Comparative results of Speed reducer design problem

Algorithm Criterio SC(1) PSO-DE(2) DEDS(3) HEAA(4) SP-QPSO

n
Speed Reducer Best 2994.744241 2996.348167 2994.47106 2994.499107 2801.27748
6
Worst 3009.964736 2996.348204 2994.47106 2994.752311 2816.032413
6
Mean 3001.758264 2996.348174 2994.47106 2994.613368 2808.3918
6
Std 4.0 6.4e-06 3.6e-12 7.0e-02 4.609711989
NFEs 54,456 54,350 30,000 40,000 10,000

Table 8 Design variables found for Tension/Compression Spring design problem

QPSO SPUCI SP-QPSO

Objective Function 0.00275 0.002525 0.00250025
X1 0.0524 0.0502 0.05
X2 0.2505 0.2505 0.25
X3 2 2 2
G1 0.931450948742565 0.942258086868517 0.930347565647419
G2 0.174715587263546 -0.265588007331609 -0.165683188068487
G3 -55.40472 - -55.180
57.876639999999995
G4 - 0.79840 -0.80
0.799866666666667

Table 9 Design variables found for for Pressure Vessel design problem

QPSO SPUCI SP-QPSO

Objective Function 5886.189623 5885.332774 5885.268
X1 0.7783 0.7782 0.7782
X2 0.3849 0.3845 0.3845
X3 40.3289 40.3208 40.3206
X4 199.8899 200.0004 199.9988
G1 4.777000000000253e-05 -8.559999999935286e- -1.242000000001298e-05
06
G2 -1.622940000000628e- 1.604319999999881e-04 1.585239999999932e-04
04
G3 -97.397200750187040 -86.028479355387390 -63.636869498295710
G4 -40.110099999999990 -39.999599999999990 -40.001200000000010
 Table 10 Design variables found for for Welded Beam design problem

QPSO SPUCI SP-QPSO

Objective Function 1.447348712 1.447514282 1.244675202
X1 0.355 0.355 0.31
X2 2.07 2.07 2.07
X3 4.224 4.2235 4.276
X4 0.355 0.355 0.31
G1 6.4821 6.4802 5.6736
G2 0.4957 0.4959 0.5892
G3 0 0 0
G4 -3.827483473546 -3.827620703712750 -3.965110214988
G5 -0.2300 -0.230 -0.185
G6 -0.167950863958132 -0.167921720291801 -0.159426883689975
G7 - - -0..05466
0.1103877161201478 0.1103702779368757

Table 11 Design variables found for Three-bar Truss design problem

QPSO SPUCI SP-QPSO

Objective Function 263.8958434 263.8950554 263.8950002291296
X1 0.788649 0.7883 0.788796
X2 0.408322 0.40930139 0.407898
G1 1.631368218291129e- 6.792137630107220e- 6.474817159762836e-06*
07* 06
G2 -1.464017742566961 -1.462901629128623 -1.464496659858666
G3 -0.535982094296218 -0.537091578733747 -0.535496865324174

Table 12 Design variables found for Speed Reducer design problem

QPSO SPUCI SP-QPSO

Objective Function 2815.937639 2804.66863 2801.27748
X1 3.6 3.6017 3.6
X2 0.7 0.6983 0.7
X3 17.00 17.0056 17.00
X4 7.43 7.0076 7.84
X5 7.3 7.0791 7.20
X6 3.19 3.199 3.12
X7 5 5.0011 5
G1 - -0.095975532858868 -.099639855942377
0.099639855942377
G2 - -0.217360804700935 -0.220276345832450
0.220276345832450
G3 - -0.465959419925719 -0.175216128905206
0.357587390752123
G4 - -0.907828788745513 -0.903143712268908
0.899051723025210
G5 0.158620193776650 0.148093548893007 0.239254319744533
G6 0.181950023190907 0.181126757396438 0.181928726920583
G7 -0.70250 -0.7031247380 -0.70250
G8 - -0.030596662687064 -0.027777777777778
0.027777777777778
G9 - -0.570182347606091 -0.571428571428571
0.571428571428571
G10 - -0.044109252811234 -0.160714285714286
0.100269179004038
G11 0.013698630136986 0.045501546806797 0.027777777777778
 Welded Beam Design Problem Three-bar Truss Design Problem
Welded Beam Design Problem Three-bar Truss Design Problem
1.3
1.3
SPQPSO SP-QPSO
SPQPSO SP-QPSO
1.2 SPUCI 5.7
1.2 SPUCI 5.7 SPUCI
SPUCI
QPSO QPSO
1.1 QPSO QPSO
1.1

1 5.65
1 5.65
0.9
BestFitness

0.9

BestFitness
BestFitness

BestFitness
0.8 5.6
0.8 5.6
0.7
0.7

0.6 5.55
0.6 5.55
0.5
0.5

0.4 5.5
0.4 5.5

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 60 80 100 120 140 160 180 200 220 240
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 60 80 100 120 140 160 180 200 220 240
Iteration Iteration
Iteration Iteration

Speed Reducer Problem Tension/Compression String Problem

Speed Reducer Problem Tension/Compression String Problem
-2.5
7.952 -2.5
7.952 SP-QPSO SPQPSO
SP-QPSO SPQPSO
7.951 SPUCI SPUCI
7.951 SPUCI -3 SPUCI
QPSO -3 QPSO
QPSO QPSO
7.95
7.95
-3.5
-3.5
7.949
7.949

BestFitness
-4
BestFitness

7.948

BestFitness
-4
BestFitness

7.948
7.947
7.947 -4.5
-4.5
7.946
7.946
-5
7.945 -5
7.945
7.944
7.944 -5.5
-5.5
7.943
7.943
-6
-60 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
00 100 200 300 400 500 600 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
100 200 300 400 500 600 Iteration
Iteration Iteration
Iteration

Pressure Vessel Design Problem

Table 13 Summary
Pressure Vessel Design Problem
of results obtained by 9.1
9.1
SPQPSO
SPQPSO
SP-QPSO for five
9.05 SPUCI
SPUCI
engineering case study 9.05

9
QPSO
QPSO problems
9

8.95
8.95
Problem Results of SP-QPSO
BestFitness
BestFitness

8.9
8.9

8.85
8.85 Best Median Mean Worst SD
Tension/compression spring design
8.8
8.8
0.00250025 0.00250025 0.00250025 0.00250025 5.68318E-17
problem 8.75
8.75
Pressure vessel design problem 8.7
5885.268 5885.268397 5885.269435 5885.282344 0.003212602
8.7
Welded beam design problem 1.244675202 1.39052735 1.597138 2.001221 0.323085
0 1000 2000 3000 4000 5000 6000
0 1000 2000 3000 4000 5000 6000
Three bar truss design problem 263.8950835 263.8958127
Iteration
Iteration 263.8957807 263.8959127 0.000169173
Speed reducer design problem 2801.27748 2808.256754 2808.3918 2816.032413 4.609711989
Figure 1. Convergence curves of the proposed algorithm for case study problems a) Welded
beam design problem, b) Three bar truss design problem, c) speed reducer design problem,
d) Tension/compression spring design problem, e) Pressure vessal design problem
4. Conclusion

The hybrid SP-QPSO makes use of parameter free adaptive penalty method used for solving five

mechanical engineering benchmark case study problems. Simulation results based on five

constrained engineering optimization problems demonstrate the effectiveness, efficiency and

robustness of the SP-QPSO over QPSO and SPUCI. From the experimental results, SP-QPSO found

better solutions compared to some newer and more sophisticated approaches with lower mean

value (over 1000 runs) and standard deviation, and requiring fewer function evaluations. The

statistical tests results confirm the better performance of SP-QPSO in terms of mean value and

number of function evaluations. Moreover, the small values for standard deviations may indicate

the robustness and stability of the proposed algorithm. Comparison with three different meta-

heuristics demonstrates that SP-QPSO with the embedded adaptive penalty method is extremely

effective and efficient at locating optimal solutions. The obtained results show that the proposed

algorithm offers better solutions than other optimizers considered in this research in terms of

objective function values for some problems and the number of function evaluations

(computational cost) for almost every problem.

References
B. Xin, J. Chen, J. Zhang, H. Fang, Z. Peng, Hybridizing differential evolution and particle swarm
optimization to design powerful optimizers: a review and taxonomy, IEEE Transactions on
Systems Man and Cybernetics Part C: Applications and Reviews 42 (5) (2012) 744–767.

C.A.C. Coello, Theoretical and numerical constraint handling techniques used with evolutionary
algorithms: a survey of the state of the art, Comput.Methods Appl. Mech. Eng. 191 (2002) 1245–1287.

C.A.C. Coello, Theoretical and numerical constraint-handling techniques used with evolutionary
algorithms: a survey of the state of the art, Computer Methods in Applied Mechanics and Engineering
191 (2002) 1245–1287.

Cagnina, L. C., Esquivel, S. C., & Coello Coello, C. A. (2011). Solving constrained optimization problems
with a hybrid particle swarm optimization algorithm. Engineering Optimization, 43(8), 843–866.

Chootinan, P. and Chen, A., 2006. Constraint handling in genetic algorithms using a gradient-based
repair method. Computers and Operations Research, 33 (8), 2263–2281.

Coello Coello CA. Theoretical and numerical constraint-handling techniques used with evolutionary
algorithms: a survey of the state of the art. Computer Methods Appl Mech Eng2002;191(11): 1245–87.

Courant R. Variational methods for the solution of problems of equilibrium and vibrations. Bull Am Math
Soc1943;49(1):23.

Dong, Y., Tang, J.F., Xu, B.D., and Wang, D., 2005. An application of swarm optimization to nonlinear
programming. Computers and Mathematics with Applications, 49 (11–12), 1655–1668.

Du, Q., Emelianenko, M., Ju, L.: Convergence of the Lloyd Algorithm for Computing Centroidal Voronoi
Tessellations. SIAM Journal on Numerical Analysis 44(1), 102–119 (2006)

Du, Qiang, Vance Faber, and Max Gunzburger. "Centroidal Voronoi tessellations: applications and
algorithms." SIAM review 41.4 (1999): 637-676.

Erwie Zahara & Chia-Hsin Hu (2008) Solving constrained optimization problems with hybrid particle
swarm optimization, Engineering Optimization, 40:11, 1031-1049, DOI:10.1080/03052150802265870.

Fan, S.K. and Zahara, E., 2007. A hybrid simplex search and particle swarm optimization for
unconstrained optimization. European Journal of Operational Research, 181 (2), 527–548.
Garg, H. A hybrid PSO-GA algorithm for constrained optimization problems. Appl. Math. Comput. 2016,
274, 292–305. [CrossRef]

H. Liu, Z.X. Cai, Y. Wang, Hybridizing particle swarm optimization with differential evolution for
constrained numerical and engineering optimization, Appl. Soft Comput. 10 (2) (2010) 629–640.

He S, Prempain E, Wu Q H. An improved particle swarm optimizer for mechanical design optimization

problems. Engineering Optimization, 2004, 36(5): 585-605.

Kaveh A, Talatahari S. Particle swarm optimizer, ant colony strategy and harmony search scheme
hybridized for optimization of truss structures, Computers and Structures, Nos. 5-6, 87(2009) 267-83.

Kaveh, A., T. Bakhshpoori, and E. Afshari. "Hybrid PSO and SSO algorithm for truss layout and size
optimization considering dynamic constraints."Structural Engineering and Mechanics 54.3 (2015): 453-
474.

Liu, H., Cai, Z., & Wang, Y. (2010). Hybridizing particle swarm optimization with differential evolution for
constrained numerical and engineering optimization. Applied Soft Computing Journal, 10(2), 629–640.
http://doi.org/10.1016/j.asoc.2009.08.031

Nema S, Goulermas J Y, Sparrow G, Helman P. A hybrid cooperative search algorithm for constrained
optimization. Structural and Multidisciplinary Optimization, 2011, 43(1): 107-119

Q. He, L. Wang, A hybrid particle swarm optimization with a feasibility-based rule for constrained
optimization, Appl. Math. Comput. 186 (2007) 1407–1422

Salcedo-Sanz S. A survey of repair methods used as constraint handling techniques in evolutionary

algorithms. Computer Sci Rev 2009;3(3):175–92.

Sheikhalishahi, M., Ebrahimipour, V., Shiri, H., Zaman, H., & Jeihoonian, M. (2013). A hybrid GA–PSO
approach for reliability optimization in redundancy allocation problem. The International Journal of
Advanced Manufacturing Technology, 68(1-4), 317-338.

Taherzadeh, Ghazaleh, and Chu Kiong Loo. "Comparison of Applying Centroidal Voronoi Tessellations
and Levenberg-Marquardt on Hybrid SP-QPSO Algorithm for High Dimensional Problems." In Advances
in Swarm Intelligence, pp. 332-341. Springer International Publishing, 2014.

Talbi, E.G.: A Taxonomy of Hybrid Metaheuristics. Journal of Heuristics 8, 541–546 (2002)

Tessema B, Yen GG. A self adaptive penalty function based algorithm for constrained optimization.
Proceedings of the 2006 IEEE Congress on Evolutionary Computation(CEC’2006), Piscataway, NJ. IEEE
Service Center: Vancouver, BC, Canada, July 2006; 950–957.

Y.G. Fu, M.Y. Ding, C.P. Zhou, Phase angle-encoded and quantum-behaved particle swarm
optimization applied to three-dimensional route planning for UAV, IEEE Trans. Syst. Man Cybern.
A: Syst. Hum. 42 (2) (2012) 511–526.

Z. Michalewicz, K. Deby, M. Schmidtz, T. Stidsenx, Test-case generator for nonlinear continuous

parameter optimization techniques, IEEE Transactions on Evolutionary Computation 4 (3) (2000) 187–
215.

Z. Michalewicz, M. Schoenauer, Evolutionary algorithms for constrained parameter optimization

problems, Evolutionary Computation, MIT Press 4 (1) (1996) 1–32.