A performance appraisal and promotion ranking system based on fuzzy logic:

An implementation case in military organizations

Chiung Moon
a,
*, Joosung Lee
b,
*, Siyeong Lim
c
a
Gueulri Advanced Technology, Ansan, Republic of Korea
b
Department of Information and Industrial Engineering, Yonsei University, Seoul, Republic of Korea
c
National Infrastructure & GIS Research Division, KRIHS, Gyeonggi-do, Republic of Korea
1. Introduction
The shift to knowledge-based capitalism makes it critical for all
organizations to maintain talented knowledge workers. It is
important for them to nd and promote the most qualied
candidates because superior human talent becomes the prime
source of an organizations competitive advantage [1,2].
When managing the human resources of an organization,
appraising the performance of applicants for a particular position
isacentral task[3]. However, it isoftendifcult toassignanaggregate
scoreforacandidatesperformancewhenpreviousassessmentswere
qualitative and originated from other organizations that have
different performance evaluation criteria [4,5]. For example, college
admissions ofces review applications that come from diverse
schools, while corporate headquarters review applicants that come
fromdifferent work environments. Although, the applicants records
may include quantitative measures such as school grades, standard
entrance exam scores, and foreign language prociency levels, they
also often contain very qualitative descriptions of applicants such as
social, hard-working or creative. The difculty is to objectively
combine quantitative and qualitative evaluations of applicants to
determine their acceptability to the organization.
In military organizations, transparent and fair appraisal of
personnel is essential for decisions pertaining to promotions and
operations. For an appraisal systemto be effective, organizational
members must believe that their opinions are reected in the
appraisal process [6]. Such appraisal involves a number of
evaluators (or decision makers) with equal authority to assess
each candidate based on both qualitative and quantitative multi-
performance criteria. The impacts and the relationships among
the characteristics used to assign a score can sometimes be
described by linguistic terms, e.g. very high, poor, medium,
etc. The appraisal results are then aggregated to rank order the
performance of the candidates and select the nalists to be
promoted. Note that the candidates come from various military
organizations and have expertise in their specialized elds. Their
performance scores have been recorded at their respective past
organizations.
This paper introduces a methodology that uses fuzzy set
theory and electronic nominal group technique for multi-criteria
evaluation in the group decision making of military promotion
screening. The methodology makes it possible to rank order the
performance of candidates evaluated by multiple criteria. The
nominal group technique is a structured group decision-making
process for generating ideas, identifying problems, and providing a
prioritized list of ideas through voting by group members. This
ensures equal participation from all group members and allows
managers to use analytical procedures in the nal decision making.
Applied Soft Computing 10 (2010) 512519
A R T I C L E I N F O
Article history:
Received 13 February 2007
Received in revised form 25 July 2009
Accepted 23 August 2009
Available online 3 September 2009
Keywords:
Performance appraisal
Fuzzy theory
Military promotion screening
Ranking system
Group decision making
A B S T R A C T
Systematic performance appraisal and ranking of candidates applying for promotion is important in
strategic human resource management. This paper discusses an approach for the promotion screening of
candidates applying for a particular commission in a military organization. The approach uses a fuzzy set
theory and electronic nominal group technique for ranking decisions fairly through the multi-criteria
performance appraisal process. A new ranking procedure considering the metric distance and fuzzy
mean value is proposed, which makes it possible to rank order the performance of the candidates by
aggregating the scores from each evaluator. A new system for performance appraisal and promotion
ranking is also developed. The system has a monitoring function which utilizes performance evaluation
data without abnormal evaluation data, which could occur when a particular evaluator produces an
incorrect result. The systemwas applied to a military organization in Korea. The results of example show
that the systematic approach of the fuzzy procedure is an effective method for transparent and impartial
multi-criteria performance evaluation.
Crown Copyright 2009 Published by Elsevier B.V. All rights reserved.
* Corresponding authors.
E-mail addresses: cumoon@yonsei.ac.kr (C. Moon), JSL@yonsei.ac.kr (J. Lee).
Contents lists available at ScienceDirect
Applied Soft Computing
j our nal homepage: www. el sevi er . com/ l ocat e/ asoc
1568-4946/$ see front matter. Crown Copyright 2009 Published by Elsevier B.V. All rights reserved.
doi:10.1016/j.asoc.2009.08.035
The steps in nominal group technique are composed of (1)
generating ideas regarding the appraisal problem, (2) recording
ideas fromgroup members, (3) discussing each idea for evaluation,
(4) rating and ranking the ideas, and (5) priority ordering of the
alternatives based on voting and analytical methods. This
technique has been successfully applied to a number of facilitating
group or group decision-making problems [79] since it was
suggested by Delbecq and Van de Ven [10].
To assign ranks, we also propose a newprocedure that uses the
metric distance and the fuzzy mean value concurrently. There are
various ranking methods available from a number of previous
research articles [1119].
Lee and Li [20] introduced a ranking method that utilizes the
mean and variance of fuzzy numbers. Candidates, who attain
scores with both higher mean values and lower spread, rank higher
using their procedure. In decision-making process, human intui-
tion favors fuzzy numbers with a higher mean value and lower
spread. However, when scores with higher mean values and higher
spreads, or with lower mean value and lower spreads exist, it is
difcult to compare the orders of the scores directly. In order to
resolve this difculty, Cheng [21] proposed a ranking method that
uses coefcient of variation (CV). However, this method has a
limitation: the difference between CVs could be negligible when
evaluation-scores are normalized.
In this paper, we borrow the concept of TOPSIS (the Technique
for Order Preference by Similarity to Ideal Solution [22]) for
assignment the rank. The point is that the best solution should be
closest to the positive-ideal solution and farthest from the
negative-ideal solution. The positive (negative)-ideal solution is
the collection of the best (worst) scores among all candidates
scores from each criterion in the original meaning. However, in
promotion screening systems, it is not important to achieve higher
scores than other candidates. It is important only to garner enough
points for promotion. Thus, a collection of the maximum
(minimum) scores, the dened limits for the candidate in each
criterion, are chosen as the positive (negative)-ideal solution.
The remainder of this paper is organized as follows. In Section 2,
we describe the process of promotion screening. Anewfuzzy based
approach is proposed in Section 3, followed by the performance
appraisal and ranking systemin Section 4. Numerical examples are
given in Section 5, and conclusions are presented in Section 6.
2. Promotion screening process
Performance appraisal for promotion in the military is typically
conducted for number of ofcers (candidates) from independent
organizations. These organizations are characterized by the need
for specialized personnel in policy making, planning, acquisition
and the like. There are differences among the evaluation systems of
the army, air force and navy. Since the metrics and processes of
measuring performance differ among the three organizations,
military headquarters typically combine quantitative and quali-
tative performance scores from the past and present records of the
personnel.
In order to combine such mixed performance scores, human
resource department rst develop evaluation criteria and establish
relative weighting among them. For example, performance indices
such as past position/education, awards, and organization con-
tribution are assigned to experience, job expertise, and miscella-
neous categories in Table 1. Based on the metrics and relative
weighting, the headquarters selects a group of evaluators to
conduct promotion appraisal.
When the evaluation indices are determined, brainstorming,
nominal group technique or Delphi methods are typically
implemented. Nominal group technique provides a useful way
to generate ideas, prioritize them, and come to a group consensus
among organizational members. In order to use these techniques
for sensitive decision-making processes, as in performance
appraisals of military personnel, it is necessary to collect various
performance evaluation criteria at different organizational levels,
and analyze the relative importance of each. This is to prevent any
particular organizations self-interests from dominating the
selection and weighting of the performance evaluation criteria.
Finally, the nalists to be promoted are selected.
The selection of evaluators and aggregation of individual
evaluators appraisal results inuence the nal scores of perfor-
mance evaluation. Problems could occur when there is an overly
inuential member in the evaluators group, or when a particular
evaluator assigns evaluation results that are too high or low
compared to the average scores. To avoid such problems, U.S.
military organizations often exclude the maximum and minimum
scores from the nal performance evaluation results [23].
However, this method requires a large number of evaluators.
Also, it is not clear how to remove data points when multiple
evaluators ascribe identical maximum or minimum scores.
Therefore, an improved approach is necessary, one that uses all
performance evaluation data without removal.
3. Fuzzy methods
In this section, denitions of fuzzy set theory and linguistic
variables as described by Zimmermann are reviewed (2001). Then,
a ranking method that simultaneously considers the metric
distance and fuzzy mean value is proposed. The distance from
the ideal solution and the fuzzy mean value are usual criteria for
ranking fuzzy numbers. Importantly, however, a shorter distance
does not always mean a larger mean value. If the score of one
candidate has a shorter distance from the positive-ideal solution,
but a lower mean value than the other candidate, which candidate
is better? This illustrates the importance off a ranking method that
can consider both criteria simultaneously.
3.1. Fuzzy number and linguistic variable
Denition 1. If X is a collection of objects denoted generically by x,
then a fuzzy set

A in X is a set of ordered pairs:

A = (x; m

A
(x))[x X
m

A
(x) is the membership function or grade of membership of x in

A
that maps X to the membership space M(when Mcontains only the
two points 0 and 1,

A is nonfuzzy and m

A
(x) is identical to the
characteristic function of a nonfuzzy set). The range of the
Table 1
Key performance criteria for promotion screening.
Key performance criteria Performance indices
Service rating 1. Completeness of job objectives
2. Service rating on job assignment
Multi-area aptitude 1. Creativity
2. Organizational contribution
3. Management capability
4. Achievement
5. Job expertise
6. Teamwork
Growth potential 1. Planning
2. Communication
3. Group discussion
4. Foreign language
5. Information systems usage
Innovativeness 1. Innovation score
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 513
membership function is a subset of the nonnegative real numbers
whose supremum is nite. Elements with a membership of zero
degrees are normally not listed.
Denition 2. The (crisp) set of elements that belong to the fuzzy
set

A at least to the degree a is called the a-level set:
A
a
= x X[m

A
(x) _a
A
/
a
= x X[m

A
(x) >a is called strong a-level set or strong a-
cut.
Denition 3. A fuzzy number

Mis a convex normalized fuzzy set

M
of the real line R such that
1. It exists exactly one x
o
R with m

M
(x
0
) = 1.
2. m

M
(x
0
) is piecewise continuous.
In this paper, trapezoidal fuzzy numbers are used. A fuzzy
number

Mcan be dened as (a, b, c, d) as shown in Fig. 1. In addition
its membership function is dened as in Eq. (1).
m

M
=
0; x <a
x a
b a
; a _ x _ b
1; b _ x _ c
x d
c d
; c _ x _ d
0; x >d
8
>
>
>
>
>
>
<
>
>
>
>
>
>
:
(1)
For example, a fuzzy number, which could be approximately
5, would normally be dened as the quadruple (3, 4, 6, 7). If b = C
in a fuzzy number

M = (a; b; c; d), then

Mis called a triangular fuzzy
number. Additionally, a nonfuzzy number k can be expressed as (k,
k, k).
The basic operations of the fuzzy numbers used in this paper are
dened as follows:

M
1

M
2
= (m
1l
m
2l
; m
1lm
m
2lm
; m
1um
m
2um
; m
1u
m
2u
) (2)

M
1

M
2
= (m
1l
m
2l
; m
1lm
m
2lm
; m
1um
m
2um
m
1u
m
2u
) (3)

M
1
= (m
1u
; m
1um
; m
1lm
; m
1l
) (4)
1

M
1
=
1
m
1u
;
1
m
1um
;
1
m
1lm
;
1
m
1l

(5)
where

M
1
= (m
1l
; m
1lm
; m
1um
; m
1u
) and

M
2
= (m
2l
; m
2lm
; m
2um
; m
2u
)
represent two trapezoidal fuzzy numbers with lower, lower modal,
upper modal and upper values.
Denition 4. A linguistic variable is characterized by a quintuple
(x; T(x); U; G;

S) in which x is the name of the variable; T(x) (or

simple T) denotes the termset of x, i.e., the set of names of linguistic
values of x, with each value being a fuzzy variable denoted gen-
erically by X and ranging over a universe of discourse U that is
associated with the base variable u; G is a syntactic rule (which
usually has the formof a grammar) for generation of the name, X, of
values of x; and

S is a semantic rule for associating with each X its
meaning,

S(x), which is a fuzzy subset of U.
Typical values of cardinality used in the linguistic models are
odd ones, such as 7 or 9, with an upper limit of granularity of 11 or
no more than 13, where the midterm represents an assessment of
middle value, with the rest of the terms being placed
symmetrically around it [24].
In this paper, the candidates are scored by the linguistic
variables. Accordingly, we propose that a set of seven terms, T,
could be given as follows:
T(scored) = VG; G; MG; M; MB; B; VB
The following semantics are proposed for the set of seven terms
like those in Fig. 2.
VG = Very Good = (8; 9; 10; 10)
G = Good = (7; 8; 8; 9)
MG = MediumGood = (5; 6; 7; 8)
M = Medium = (4; 5; 5; 6)
MB = MediumBad = (2; 3; 4; 5)
B = Bad = (1; 2; 3)
VB = Very Bad = (0; 0; 1; 2)
The membership function of VG is
m
verygood
=
0; x <8
x 8; 8 _ x _ 9
1; 9 _ x _ 10
0; x >10
8
>
>
<
>
>
:
(6)
Each linguistic variable has its membership function like that in
Eq. (6).
3.2. Ranking method
In this situation, candidates are scored by linguistic variables. It
is assumed that there are m candidates, n evaluators and l criteria.
Each of n evaluators classies into grade m candidates for each of l
criteria, respectively. The scores range between 0 and 10 in fuzzy
concept.
We dene the following sets:
mcandidates; A
i
[1 _ i _ m
nevaluators; D
i
[1 _ i _ n
l criteria; C
i
[1 _ i _ l
Fig. 1. Trapezoidal fuzzy number

M. Fig. 2. The membership functions for fuzzy numbers.
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 514
Let x
ijk
(1 _ i _ m, 1 _ j _ n, 1 _ k _ l) be the ith candidates
grade classied by jth evaluator for k criterion. For example, if the
rst candidate receives a grade of G fromthe second evaluator for
the third criterion, then x
123
should be (7, 8, 8, 9). There are three
procedures to rank the candidates. The rst is to get the weights for
criteria, and the second is to aggregate the evaluators scores for
each candidate. The last is to rank the candidates by our proposed
ranking method.
(1) Weighting for criteria
When decision-making problems arise, there exist two
kinds of weighting, one for the criteria and the other for the
evaluators. In general decision-making situations, opinions of
some evaluators such as chief are more important than those of
the others. But in our systems, the weighting of evaluators is
equal because the effects of evaluators should be fair. Thus,
only weighting for performance criteria is taken into con-
sideration.
Let w
k
(1 _ k _ l) be the weights for each criterion. These are
predetermined by the experts using the intranet/internet. The
weights are fuzzy numbers such as the scores for candidates.
High(Low) is used in place of Good/Bad. For example, a fuzzy
number, which could be assigned VH(Very High) to a
criterion, would normally be dened as the quadruple (8, 9,
10, 10). We can derive a normalized weight such that
w
k
=
w
k
P
l
i=1
w
i
=
w
k
l
P
l
i=1
w
i
u
;
w
k
lm
P
l
i=1
w
i
um
;
w
k
um
P
l
i=1
w
i
lm
;
w
k
u
P
l
i=1
w
i
l
!
(7)
where, w
i
= (w
i
l
; w
i
lm
; w
i
um
; w
i
u
) represents a trapezoidal fuzzy
number with lower, lower modal, upper modal and upper
values.
(2) Aggregating the scores
The fuzzy number y
t
(1 _ i _ m) is dened as the aggregated
score for the ith candidate. Then, y
t
is obtained as follows.
where x
ijk
= (x
ijk,l
, x
ijk,lm
, x
ijk,um
, x
ijk,u
) represents a trapezoidal
fuzzy number with lower, lower modal, upper modal and upper
values. y
i
= (y
i,l
, y
i,lm
, y
i,um
, y
i,u
) denotes the unied score for the
ith candidate from all evaluators by all criteria.
According to Boender et al. [25], the normalized set of fuzzy
numbers has to satisfy the conditions that the sum of the
middle values is 1 and the sum of the products of the low and
high values is 1. The rst term in summation satises these
conditions. The second term in the summation signies the
average of ratings from all evaluators, and corresponds to the
objective, which is a fair screening.
(3) Ranking the candidates
From now on, the candidates will be ranked by the aggregated
scores, y
i
s. Thus, one who scores Very Good (Very Bad) in all
criteria is chosen as the positive (negative)-ideal solution, even if
none of the real candidates do so in all criteria. The value of the
positive (negative)-ideal solution is dependent on the weights for
criteria.
In Goetschel and Voxman [26], the method of ranking the fuzzy
numbers is found using their means. The ordering method is
dened as such that if
m =
Z
1
0
a[a(a) b(a)[da; v =
Z
1
0
a[c(a) d(a)[da; (9)
and m _ v, then

M
1
_

M
2
, where (a(a), b(a)) and (c(a), d(a))
denote the a-cut of the fuzzy number

M
1
,

M
2
.
Let y
*
= (a
*
, b
*
, c
*
, d
*
), y

= (a

, b

, c

, d

), and y
i
= (a
i
, b
i
, c
i
, d
i
) be
the positive-ideal solution, negative-ideal solution, and ith
candidates score, respectively. Also, let m
*
, m

, and m
i
be the
mean values of the positive-ideal solution, negative-ideal solution,
and ith candidate.
We dene the following distance measurements:
d
+
=

(a
+
)
2
(b
+
)
2
(c
+
)
2
(d
+
)
2
q
(10)
d

(a

)
2
(b

)
2
(c

)
2
(d

)
2
q
(11)
d
i
=

a
2
i
b
2
i
c
2
i
d
2
i
q
(12)
Using Eqs. (9)(12), we suggest the following ranking measures.
M
+
i
=
1
2
d
i
d

d
+
d

m
i
m

m
+
m

(13)
where d

_ d
i
_ d
*
and m

_ m
i
_ m
*
.
The values of M
+
i
are between 0 and 1. If the value of M
+
i
is larger,
the candidate is closer to the ideal solution and farther from the
negative solution. In (13), each candidates scores are simulta-
neously compared with the positive (negative)-ideal solution in
terms of the distance and fuzzy mean. The greater the distance
from the negative-ideal solution and the larger the fuzzy mean
value compared to the negative-ideal solution, the higher the
candidate ranks.
Eq. (13) is the mixture of the ranking methods in Hwang and
Yoon [22] and Goetshel and Voxman [26]. The rst termof Eq. (13)
shows the distance from the ideal solution and the second term
shows the difference fromthe ideal solutions fuzzy mean. The fact
that one candidates score is closer to the ideal solution does not
necessarily mean that it has a greater fuzzy mean than the others.
We consider the distance fromthe ideal solution and fuzzy mean at
the same time. We derive Eq. (13) from the concept that the ideal
solution has to be the collection of the best solutions in each
criterion. This is the difference between our approach and TOPSIS
introduced by [22]. In the promotion screening process, it is not
y
i
=
X
l
k=1
w
k

1
n
X
n
j=1
x
i jk
0
@
1
A
8
<
:
9
=
;
=
X
l
k=1
w
k
l
P
l
h=1
w
h
u
;
w
k
l
P
l
h=1
w
h
um
;
w
k
l
P
l
h=1
w
h
lm
;
w
k
l
P
l
h=1
w
h
l
!

P
n
j=1
x
i jk;l
n
;
P
n
j=1
x
i jk;lm
n
;
P
n
j=1
x
i jk;um
n
;
P
n
j=1
x
i jk;u
n
! ( )
=
X
l
k=1
w
k
l
P
l
h=1
w
h
u

P
n
j=1
x
i jk;l
n
;
w
k
l
P
l
h=1
w
h
um

P
n
j=1
x
i jk;um
n
;
w
k
l
P
l
h=1
w
h
lm

P
n
j=1
x
i jk;lm
n
;
w
k
l
P
l
h=1
w
h
l

P
n
j=1
x
i jk;u
n
! ( )
=
X
l
k=1
w
k
l
P
l
h=1
w
h
u

P
n
j=1
x
i jk;l
n
( )
;
X
l
k=1
w
k
l
P
l
h=1
w
h
um

P
n
j=1
x
i jk;um
n
( )
;
X
l
k=1
w
k
l
P
l
h=1
w
h
lm

P
n
j=1
x
i jk;lm
n
( )
;
X
l
k=1
w
k
l
P
l
i=h
w
h
l

P
n
j=1
x
i jk;u
n
( ) !
(8)
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 515
important that one candidate has better grades than the others. It
is more important that the grades of candidates could exceed the
level necessary for promotion.
4. Performance appraisal and promotion ranking system
Performance appraisal and promotion ranking (PAPR) is the
main primary role of human resource management in military
organization. Therefore, to support the PAPR function, we develop
a system called the PAPR system. The system is effective in
performance appraisal and ranking, and can help organizations
transform the employee evaluation process into a well-dened,
fair, and transparent process. This developed system is composed
of three main functions: (1) determining the weights of evaluation
criteria, (2) rating and data monitoring, and (3) data aggregation
and ranking.
4.1. Determining the weights of evaluation criteria
In order to aggregate decision data, the nominal group
technique plays an important role in generating ideas, prioritizing
them, and coming to a group consensus among organizational
members. An effective electronic nominal group technique is
developed for generating evaluation criteria and providing a
prioritized list of criteria through voting by group members. The
weight of evaluation criteria can be determined by aggregating the
decisions of external and internal experts in the eld of human
resource management. The process for determining weights using
the electronic technique is shown in Fig. 3.
In Fig. 3, the process for determining the weights of evaluation
criteria consists of three parts as follows:
(1) The department of human resource management develops a
list of key criteria. In this appraisal and ranking system for
promotion screening, four criteria, such as, service rating,
multi-area aptitude, growth potential and innovativeness, are
considered.
(2) Decision makers evaluate the criteria through the electronic
nominal group technique. The technique provides an advanta-
geous decision-making environment to equalize participation,
encouraging the free ow of ideas in a nonthreatening setting
and enabling participaints to reach nal decisions.
(3) The weights of key criteria are determined by aggregating the
decision makers results.
This technology can be used to collect and assess the relative
importance of evaluation criteria at various organizational levels.
The linguistic weighting variables, seen in Fig. 2, are used to assess
the weight of each criterion.
4.2. Rating and data monitoring
The second functionconsists of rating, data monitoring and data
processing. The committee for decision making involves a number
of evaluators with equal authority who assess each candidate and
implement individual analyses. The candidates performance
scores recorded at their respective past organizations are provided
to the human resource management department for rating.
Fig. 3. Process of determining the weights of criteria.
Fig. 4. Example of data monitoring for abnormality and incorrectness. Fig. 5. Monitoring procedure.
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 516
The monitoring process is employed to examine abnormal
or incorrect data received from the evaluators. The degree
of scattering is used to control abnormal data as shown in Fig. 4.
The procedure for data monitoring is shown in Fig. 5.
This procedure utilizes all performance evaluation data without
excluding the maximum and minimum scores from the nal
performance evaluation results. If all input is perceived as normal,
the linguistic assessments received from evaluators are converted
intotrapezoidal fuzzynumbers toconstruct a fuzzy-decisionmatrix,
andfromthis, the fuzzyweight of eachcriterionis determined. Then,
a weighted-normalized fuzzy-decision matrix is constructed.
4.3. Data aggregation and ranking
The appraisal results of individual evaluators should be
aggregated to compute the nal appraisal scores to determine
the ranking of each candidate. The data aggregation and ranking
Fig. 6. Overall schematic ow diagram of the PAPR system.
Fig. 7. Assigning the weights of evaluation criteria.
Table 2
The weights of evaluation criteria.
Criteria Fuzzy number
C
1
0.2056 0.2551 0.2766 0.3415
C
2
0.1963 0.2449 0.2553 0.3293
C
3
0.2056 0.2551 0.2766 0.3415
C
4
0.1589 0.2041 0.2340 0.2927
Table 3
The aggregated scores for each candidate.
Candidate Fuzzy number
y
1
5.5140 7.8571 8.9574 12.0813
y
2
5.3330 7.6293 8.7305 11.8780
y
3
4.9003 7.6905 8.6028 11.8577
y
4
5.5296 7.8741 8.9858 12.0976
y
5
5.5885 8.0782 9.2199 12.2927
y
6
5.6667 8.0544 9.4397 12.3821
y
7
4.8162 6.9830 8.0993 11.3252
y
8
5.6760 8.0646 9.2057 12.2805
y
9
5.5140 7.8571 8.9574 12.0813
y
10
5.2617 7.5408 8.5319 11.7602
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 517
function transforms the several individual multi-criteria scored
lists of candidates into one aggregated rank-ordered list. Then,
using the ranking measures to determine the ranking of each
candidate, nalists can be selected for promotion.
The overall schematic ow diagram of the PAPR system is
shown in Fig. 6.
5. Numerical example
An example of ranking promotion candidates is introduced and
demonstrates the effectiveness of the PAPR system. In this
example, there are three evaluators, four criteria, and 10
candidates. A detailed procedure for ranking is shown in Section
3. Eqs. (2)(5) are used for the fuzzy operations. First, the weights
of evaluation criteria are determined at by aggregating the
decisions of external and internal experts in the eld of human
resource management, as shown in Fig. 7. Then, the aggregated
weights of the criteria are summarized in Table 2.
Then, the candidates scores are evaluated for each criterion.
Appendix A shows the scores that each candidate received from
each evaluator. The rst candidates score from the second
evaluator for the third criterion is G, so x
123
= (7, 8, 8, 9). From
Eq. (8), the aggregated scores of each candidate can be expressed
by a fuzzy number, y
i
. Table 3 displays the fuzzy numbers
representing the aggregated scores of each candidate.
In this case the positive and negative-ideal solutions are
y
*
= (6.1308, 8.6327, 10.4255, 13.0488), and y

= (0, 0, 1.0426,
2.6098) which are derived from Table 2, the weights for criteria,
and Eq. (6), fuzzy membership functions.
Lastly, the candidates are ordered by the proposed ranking
method. The distance and fuzzy means are derived using Eqs. (9)
(12), and the candidates are ranked using Eq. (13). In Table 4, the
value represents the degree of separation fromthe ideal solution. A
larger value means that the candidate is closer to the ideal solution,
in this case, the sixth candidate attains the highest ranking. Fig. 8
shows the visual ranking results from the PAPR system.
From the experimental result, we know that the developed
PAPR system can handle applicant records that contain both
qualitative and quantitative information. Finally, the system
provided performance evaluations of all candidates.
6. Conclusions
This paper develops an efcient performance appraisal and
ranking systemfor the promotion screening of candidates applying
for a particular commission in a military organization. The system
uses fuzzy theory and electronic nominal group technique to
produce fair ranking decisions through a multi-criteria perfor-
mance appraisal process. The electronic nominal group technique
is adopted to collect and assess the relative importance of various
performance evaluation criteria collected at different organiza-
tional levels. This technology can prevent any particular organiza-
tions self-interests from dominating the selection and weighting
of performance evaluation criteria.
A new ranking procedure considering the metric distance and
fuzzy mean value is also proposed, which makes it possible to rank
order the performance of candidates by aggregating the scores of
multiple evaluators. The systemalso has a monitoring function that
uses all performance evaluation data without any removal. This
function is to prevent abnormal evaluation data which could occur
when there is an overly inuential member in the evaluators group,
or when a particular evaluator gives an incorrect evaluation result.
The system developed was applied to a military organization in
Korea. The results of this example have shown that this systematic
approach with a fuzzy procedure is a suitable method to produce
transparent and fair multi-criteria performance evaluations in
military organizations.
Appendix A
We assess 10 candidates scores for four criteria by three
evaluators. In the actual system, the evaluators score the candidates
using a computer program and the rawdata are not shown. However,
for the purpose of demonstrating the procedure used in the example,
it is necessary to show the raw scores here.
Criteria Candidates Evaluators Criteria Candidates Evaluators
D
1
D
2
D
3
D
1
D
2
D
3
C
1
A
1
VG G G C
3
A
1
VG G G
A
2
VG G VG A
2
G VG G
A
3
G VG G A
3
G G G
A
4
G G VG A
4
G VG VG
A
5
G G VG A
5
VG G VG
A
6
VG VG G A
6
G MG G
A
7
VG G MG A
7
MG G G
A
8
G G G A
8
VG VG VG
A
9
G VG G A
9
VG G G
A
10
G G G A
10
G VG G
C
2
A
1
G VG G C
4
A
1
G VG MG
A
2
G G MG A
2
MG G G
A
3
G G G A
3
MG G VG
A
4
VG G G A
4
MG G G
A
5
G G VG A
5
G VG G
A
6
VG VG VG A
6
G VG VG
A
7
G MG G A
7
MG G MG
A
8
G G G A
8
G VG VG
A
9
G G VG A
9
MG G VG
A
10
G MG VG A
10
MG G G
Table 4
The nal ranking.
Candidate M
+
1
M
+
2
M
+
3
M
+
4
M
+
5
M
+
6
M
+
7
M
+
8
M
+
9
M
+
10
Value 0.8853 0.8610 0.8502 0.8876 0.9107 0.9197 0.7932 0.9092 0.8853 0.8467
Ranking 5 7 8 4 2 1 10 3 6 9
Fig. 8. . Visual ranking results from PAPR system.
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 518
References
[1] P. Allan, Avoiding common pitfalls in performance appraisal, Industrial Manage-
ment (1992) 3032.
[2] H.G. Chambers, M. Foulon, H. Haneld-Jones, S.M. Hankin, E.G. Michaels III, The
war for talent, McKinsey Quarterly (1998).
[3] G. Lowenberg, K.A. Conrad, Current Perspectives in Industrial/Organizational
Psychology, Allyn & Bacon, Boston, MA, 1998.
[4] A.G. Momeyer, Why no one likes your performance appraisal system? Training 23
(1986) 9598.
[5] G. Dressler, Human Resource Management, 6th ed., Prentice-Hall, Englewood
Cliffs, NJ, 1994.
[6] S.W. Gilliland, J.C. Langdon, Creating performance management systems that
promote perceptions of fairness, in: J.W. Smither (Ed.), Performance Appraisal:
State of the Art in Practice, Jossey-Bass, San Francisco, CA, 1998.
[7] E.W. Duggan, C.S. Thachenkary, Integrating nominal group technique and joint
application development for improved systems requirement determination,
Information & Management 41 (2004) 399411.
[8] J. Perry, S. Linsley, The use of the nominal group technique as an evaluative tool in
the teaching and summative assessment of the inter-personal skills of student
mental health nurses, Nurse Education Today 26 (2006) 346353.
[9] P.L. Williams, N. White, R. Klem, S.E. Wilson, P. Bartholomew, Clinical education
and training: using the nominal group technique in research with radiographers
to identify factors affecting quality and capacity, Radiography 12 (2006) 215224.
[10] A.L. Delbecq, A.H. Van de Ven, Nominal versus interacting group processes for
committee decision making effectiveness, Journal of Academy Management 14
(1971) 203212.
[11] G. Bortolan, R. Degani, A reviewof some methods for ranking fuzzy subsets, Fuzzy
Sets and Systems 15 (1985) 119.
[12] J.J. Buckley, Ranking alternatives using fuzzy numbers, Fuzzy Sets and Systems 15
(1985) 2131.
[13] J.J. Buckley, Fuzzy hierarchical analysis, Fuzzy Sets and Systems 17 (1985) 233
247.
[14] S.H. Chen, Ranking fuzzy numbers with maximizing set and minimizing set, Fuzzy
Sets and Systems 17 (1985) 113129.
[15] L.S. Chen, C.H. Cheng, Selection IS personnel use fuzzy GDSS based on metric
distance method, European Journal of Operational Research 160 (2005) 803820.
[16] C.T. Chen, C.T. Lin, S.F. Huang, A fuzzy approach for supplier evaluation and
selection in supply chain management, International Journal of Production
Economics 102 (2006) 289901.
[17] D. Dubois, H. Prade, The mean value of a fuzzy number, Fuzzy Sets and Systems 24
(1987) 279300.
[18] M. Modarres, S. Sadi-Nezhad, Ranking fuzzy numbers by preference ration, Fuzzy
Sets and Systems 118 (2001) 429436.
[19] E. Triantaphyllou, C.T. Lin, Development and evaluation of ve fuzzy multi-
attribute decision-making methods, International Journal of Approximate Rea-
soning 15 (1996) 281300.
[20] E.S Lee, R.L. Li, Comparison of fuzzy numbers based on the probability measure of
fuzzy events, Computers and Mathematics with Applications 15 (1988) 887896.
[21] C.H. Cheng, A newapproach of ranking fuzzy numbers by distance method, Fuzzy
Sets and Systems 95 (1998) 307317.
[22] C.L. Hwang, K. Yoon, Multiple attribute decision making, Lecture Notes in Eco-
nomics and Mathematical Systems, vol. 198, Springer, Berlin, 1981.
[23] Korean Military, Guide Book for the Human Resource Management in Korea
Military, Korea Military, Seoul, 2005.
[24] F. Herrera, E. Herrera-Viedma, Linguistic decision analysis: steps for solving
decision problems under linguistic information, Fuzzy Sets and Systems 115
(2000) 6782.
[25] C.G.E. Boender, J.G. de Graan, F.A. Lootsma, Multi-criteria decision analysis with
fuzzy pairwise comparisons, Fuzzy Sets and Systems 29 (1989) 133143.
[26] R. Goetschel Jr., W. Voxman, Elementary fuzzy calculus, Fuzzy Sets and Systems
18 (1986) 3143.
C. Moon et al. / Applied Soft Computing 10 (2010) 512519 519