Multi-Criteria

Decision Analysis
Tools

Developed by Dr. Martyn Jones, The University of

Manchester, January 2016.
 About this module

Content and learning objectives:

This module is an introduction to MCDA. It gives a brief overview of MCDA and two
frequently used methods.
 What is multi-criteria decision
analysis
Multi-criteria decision analysis (MCDA) is a formal, structured and transparent decision
making methodology. Its ism is to assist groups or individual decision makers to explore
their decisions in the case of complex situations with multiple criteria.

MCDA does not provide the ‘right’ answer.

MCDA does not provide an objective analysis.
MCDA does not relieve decision makers of the responsibility of making difficult
judgments.

MCDA assists the decision maker in confidently reaching a decision by:

• enabling decision makers to gain a better understanding of the problem faced;

• organising and synthesising the entire range of information;
• integrating objective measurements with value judgements;
• making explicit and managing the decision maker’s subjectivity; and
• ensuring that all criteria and decision factors have been taken properly into account.
 MCDA vs unstructured decision
making process
All decisions are subjective.
All decision makers are biased.

Avoiding bias and subjectivity may be feasible only if a problem is sufficiently simplified,
such that some numeric objective criteria could be designed:

We may simplify evaluation of a chemical process to an individual

criteria, such as RISK, where risk is a product of HAZARD and
EXPOSURE. If HAZARD is then quantified numerically by using e.g., a
toxicity scale, and EXPOSURE is known from materials flows, than
RISK could be evaluated numerically and the process option with the
minimum RISK value would be considered the best option.

But the process with minimum RISK value is not automatically the
best one, if we consider more criteria: energy, cost, resource
efficiency, etc. Which of these criteria are more important?
 MCDA vs unstructured decision
making process
Through the use of MCDA a larger number, of perhaps individually less important,
indicators do not get ignored in the final decision during a simplification that relies too
heavily on a small number of key criteria.

Unstructured decision making often fails to make use of or consider the uncertainty
surrounding different criteria.

MCDA can be used in conjunction with Monte Carlo simulation to take account of
modelled uncertainty of the criteria values, uncertainty of the subjective criteria
preferences and provide a known level of certainty in the proposed decision,
unachievable in unstructured decision making.
 MCDA methodology

MCDA is an umbrella term for a range of tools and methodologies. The level of
complexity, interaction with the decision maker and level of detail utilised in the decision
making process can vary substantially.

In general the decision maker follows the same process:

1. Identify multiple criteria on which to base their decision;

2. Identify multiple alternative solutions to their decision;
3. Provide (subjective) ranking or weighting of criteria; and
4. Provide values, rankings or weighting of alternatives for each criteria.
 MCDA in sustainability
assessment
Here we show a number of published studies where MCDA tools have been incorporated
into sustainability assessment:

Azapagic, A. and S. Perdan (2005 a) An integrated sustainability decision-support

framework Part II: Problem analysis. International Journal of Sustainable Development &
World Ecology, 12(2), 112-131.

Azapagic, A. and S. Perdan (2005 b) An integrated sustainability decision-support

framework Part I: Problem structuring. International Journal of Sustainable Development
& World Ecology, 12(2), 98-111.

Benetto, E. and C. Dujet (2003) Uncertainty analysis and MCDA: a case study from the life
cycle assessment (LCA) practice. 57th Meeting of the European working group on
multicriteria decision aiding, Viterbo, 27–29 March.

Cinelli, M., S. Coles and K. Kirwan (2014) Analysis of the potentials of multi-criteria
decision analysis methods to conduct sustainability assessment. Ecological Indicators, 46,
138-148.
 MCDA methodology

The Analytical Hierarchy Process (AHP)

AHP was developed in the late 1970s. Today it is the most widely used MCDA method.

AHP generates all criteria weighting and alternative preference within each criteria by
eliciting these values from the decision maker through a series of pairwise comparisons,
as opposed to utilising numerical values directly.

Thus, a complex decision is reduced to a series of simpler ones, between pairs of

alternative values within criteria or between pairs of criteria. The decision maker’s
preference is always explicit. However, the decision maker may be asked to make very
many small decisions. Hence, it becomes important to generate an optimised hierarchy of
criteria and alternatives, to reduce the number of pairwise decisions.
 AHP

Step 1: Construct the problem hierarchy

Model, usually visually, the problem decision identifying relationships between criteria
and alternatives.

Step 2: Pairwise comparison of criteria

Undertake pairwise comparison between criteria, identifying decision maker preference
for criteria on which alternatives are evaluated.

Step 3: Pairwise comparison of alternatives within each criterion

Undertake pairwise comparison between alternatives based on their performance within
each criterion.

Step 4: Compute the vector of criteria weights

From a matrix of pairwise comparison results AHP utilises a variety of matrix
transformations to calculate criteria weight vectors representing normalised criteria
weightings.
 AHP

Step 5: Compute the matrix of alternative scores

From the results of the pairwise comparisons on alternatives within each criterion a nxm
(where n is the number of criteria and m is the number of alternatives) matrix is
constructed representing the normalised performance (score) of each alternative for each
criteria.

Step 6: Ranking the alternatives

Utilising the vectors of criteria weights and the matrix of alternative scores a global score
and hence ranking for each alternative is calculated using:

where: a is the alternative, c is the criteria, g is the global score of the alternative, w is the
criteria weight and s is the alternative score. A function of the ranking equation,
aggregating across each criteria means that trade-offs between criteria in fundamental to
the final ranking.
 AHP

Problem Hierarchy
The problem hierarchy provides a structured, usually visual, means of modelling the
decision being processed. As the first step in the analytical hierarchy process the creation
of a hierarchy that models the decision problem enables decision makers to increase their
understanding of the problem, its context and, in the case of group decision making, see
alternative approaches to the problem across different stakeholders.

The AHP problem hierarchy consists of a goal (the decision), a number of alternatives for
reaching that goal, and a number of criteria on which the alternatives can be judged that
relate to the goal.

Here as an example a simple

AHP hierarchy was generated
in DECERNS tool.
 AHP

For most realistic analyses criteria are multi-tiered. Criteria weightings are calculated as
pairwise comparisons.

Consider a problem hierarchy

with 12 criteria on which the
three alternatives are to be
judged. For pairwise
comparison the first criteria
will be compared against the
remaining 11, the second
criteria against the remaining
10, the third criteria against
the remaining 9 etc. In total 66
(11+10+9+8+7+6+5+4+3+2+1)
pairwise comparisons will
need to be undertaken in
order to determine the
weighting of the 12 criteria.
 AHP

Pairwise Comparisons

Within AHP pairwise comparison is the process of comparing entities in pairs so as to judge
which is preferred and by how much. Comparisons are undertaken to determine criteria
weighting and also assess the value or score of different alternatives within each criteria.

A 9-point scale to elicit the scale of preference from a decision maker:

the more preferable entity within the pair scores

1 when it is showing no preference

3 when it is showing moderate preference
5 when it is showing strong preference
7 when it is showing very strong preference
9 when it is showing extreme preference.

The less preferable entity within the pair scores the inverse, for example the less preferable
entity where the more preferable entity shows very strong preference would score 1/7.
 AHP

Groups of pairwise comparisons are undertaken between every alternative value within a
single criteria, and every criteria within the goal (or for multi-tier hierarchies within their
parent criteria). For each group a matrix is completed with the results of the pairwise
comparison, such as that shown in the table below, following the example from figure on
Slide 12.

The results of the matrix would provide the normalised criteria weights for criteria A1 to A4.
Similar matrices would be completed for criteria B1 to B4, for C1 to C4 and also one
comparing criteria A, B and C. Finally, pairwise comparisons would be undertaken to fill
matrices for each criteria comparing the performance of each alternative within that
criteria.
 AHP

Consistency across pairwise comparisons

The consistency of the decision maker across a number of pairwise comparisons is a
significant complexity. Consider the very simple comparison of three criteria: A, B and C. If
the decision maker judges A to be more preferable than B, and A to be less preferable
than C then the decision maker must not judge B to be more preferable than C.

In a group that contains a large number of pairwise comparisons or where the difference
is between moderate and very strong preference it can be seen that lack of consistency is
a largely inevitable consequence of complex decision processes within AHP.

The AHP method attempts to address the issue of consistency by implementing a

consistency index that is a function of opposing comparisons. Above a threshold a lack of
consistency is highlighted and no analysis results are presented. An unfortunate
consequence is that decision makers begin to fulfil pairwise comparisons not on their
actual judgements but rather in order to maintain acceptable consistency.

An effective approach to limit the issue of consistency is to utilise a multi-tier hierarchy

thereby reducing the number of pairwise comparisons undertaken within each group.
 AHP

Rank reversal
If the inclusion or exclusion of a non-outperforming alternative, or duplicate alternative
alters the ranking of the remaining alternatives a rank reversal occurs.

AHP method and other MCDA methods are susceptible to rank reversal and experienced
users must be aware of this.
 AHP

Recording the decision making process and decision makers subjectivity

It is useful to have a record of decision making process. This gives some idea of how the
decision was reached. The problem hierarchy gives insight into how the decision was
structured.

Most AHP tools allow to view the pairwise comparison matrices showing the preference
values applied to each pair. However, this does not make explicit the subjectivity inherent
in the judgements made by the decision maker; the reasoning and understanding behind
those simple judgements is lost.

For complex problems where a large number of pairwise comparisons have been
undertaken it is unlikely that enough subjective detail could be recorded at the time of
making the decision to suggest the information available could be considered transparent
or suitable for any form of audit. Similarly it is unlikely that returning to this information in
the future would enable the reasoning behind any mistakes to be observed.
 PROMETHEE

Preference Ranking Organization Method for Enrichment Evaluations

This is a family of methods developed since 1980s. PROMETHEE is classified as an

outranking method allowing for a finite number of alternatives to be ranked based on a
finite number of criteria, which are often conflicting.

The PROMETHEE family includes a number of methods (PROMETHEE I, II, III, IV, V and VI)
although PROMETHEE I for partial ranking of alternatives and particularly PROMETHEE II
for complete ranking of alternatives are the most commonly used.

Arguably, PROMETHEE II is the most relevant to decision making in process development

and innovation.

PROMETHEE II has been developed in order to provide a complete ranking of a finite set
of alternatives from the best to the worst. The ranking is calculated using a pair-wise
comparison of alternatives for each criteria utilising preference functions which are then
aggregated using criteria weighting to provide a net outranking flow and hence a
complete ranking of alternatives.
PROMETHEE
 PROMETHEE

Each criterion used within a PROMETHEE model to help rank alternatives is assigned a
preference function by the decision maker. The preference function translates the
difference (either positive or negative) in the value of a criterion between two alternatives
in a pairwise comparison into a preference degree ranging from zero to one.

Typically, 6 preference functions are used:

Type 1: Usual criterion

Indifference only exists between alternatives a and b when they are equal and in this case
they will be assigned a preference degree of 0. However, the smallest difference in value
represents a strict preference on behalf of the decision maker and is assigned a preference
degree of 1.

Type 2: Quasi-criterion
In this case alternatives a and b are seen to be indifferent within a range defined by the
decision maker and will be assigned a preference degree of 0. Beyond that area of
indifference the preference becomes strict, with a preference degree of 1 being assigned.
 PROMETHEE

Type 3: Criterion with linear preference

Extending the usual criterion, in this case the decision maker’s preference increases
progressively for progressively larger deviations between alternatives a and b. Where a
equals b the alternatives are indifferent and assigned a preference degree of 0. From that
point the intensity of preference increases linearly to a threshold value defined by the
decision maker at which point and beyond which the preference is strict and assigned a
preference degree of 1.

Type 4: Level criterion

An extension to the quasi-criterion, in this case alternatives a and b are seen to be
indifferent within a range defined by the decision maker and will be assigned a preference
degree of 0. Beyond that a second range defined by the decision maker provides a weak
preference where a preference degree of 0.5 will be assigned. Beyond that range a strict
preference with a preference degree of 1 is given.
 PROMETHEE

Type 5: Criterion with linear preference and indifference area

A combination of types 2 and 3, in this case alternatives a and b are seen to be indifferent
within a range defined by the decision maker and will be assigned a preference degree of 0.
Beyond this range the decision maker’s preference increases progressively for progressively
larger deviations between alternatives a and b. The intensity of preference increases
linearly to a threshold value defined by the decision maker at which point and beyond
which the preference is strict and assigned a preference degree of 1.

Type 6: Gaussian criteria

Similar to type 3, if a particular criterion is of the Gaussian type, the preference of the
decision maker still grows with increasing deviation between alternatives a and b but the
relationship is not linear. Through the decision maker providing a value of σ, representing
the distance between the origin and point of inflection, the relationship between difference
in alternatives a and b and the preference degree is given. The preference degree will vary
from 0 where a equals b to approaching 1 where the difference is very large.
 PROMETHEE
Preference functions
 PROMETHEE
Criteria Weighting
As with most MCDA methods it is possible, indeed preferred, to define the relative importance
of each criteria within the decision making process. This is done using criteria weighting.

Within the PROMETHEE model normalised weights, aggregating to 1 are used. There are many
techniques to elicit the weightings from the decision maker with the choice often being one of
the software being used or personal preference of the decision maker. As an example, the
MCDA tool DECERNS provides 4 common methods of eliciting criteria weights within its
PROMETHEE model:

1. Direct weighting
The decision maker directly provides numerical weights for each criterion representing its
perceived importance in the decision process. These weights will be normalised before being
used in the PROMETHEE model.

2. Ranking
The decision maker is asked to rank in order of preference the criteria. The rankings are
converted to equally spaced numerical values, normalised to aggregate to 1.
 PROMETHEE
Criteria Weighting

3. Rating
The decision maker attributes a score of 100 to the criteria perceived as most important.
Subsequently scores of less than 100 are applied to each of the remaining criteria. The scores
are normalised to aggregate to 1.

4. Pairwise comparison
Using the same process and scale as AHP, the decision maker considers each possible pair of
criteria and states a preference on a 9 point scale from equal preference through moderate,
strong, very strong and extreme preference. Once all pairwise comparisons are complete
scores for each criterion are aggregated and normalised.
 PROMETHEE
Multi-level criteria, multi-level weighting

In this example the higher level criteria

have been defined for economic,
environmental and social impacts with
the relevant criteria flowing into each.

Criteria weighting occurs within

the economic higher level criteria
for criteria X and Y and positive,
negative and net flows for the
alternatives are calculated at that
level. At the task level weighting
of the three higher levels occurs
and is applied to the flows
calculated for the economic,
environmental and social criteria
to provide flows, and hence a
ranking, for the overall task.
 PROMETHEE
Barriers to wide adoption

In principle, PROMETHEE is a relatively simple ranking method in its idea and application.

Key methodological difficulty is the definition of preference functions suitable for each
criterion. This may require a specific guidance following statistical analysis of decision
outcomes depending on preference functions used, as well as a guidance for preference
functions suitable for specific situations.

PROMETHEE tools

Decision Lab
DECERNS
D-Sight
Smart Picker Pro
Visual PROMETHEE
 PROMETHEE
Recording the decision making process and decision makers subjectivity

Due to the relative simplicity of the PROMETHEE approach it is possible to provide a simple and
yet complete overview of the decision making process and decision maker’s subjective input.

This information provides transparency and audit of the decision process and allows the
decision to be recreated at a later date.

The following data is readily available from all PROMETHEE software tools:

Criteria/Alternative value matrix

Criteria hierarchy map (if using multi-tier weighting)
Preference functions for each criteria
Criteria Weighting (for each tier if using multi-tier weighting)
 Literature and further reading

• Belton, V. and Stewart T. (2002) Multiple Criteria Decision Analysis: An Integrated

Approach. New York, Kluwer Academic.
• Azapagic, A. and S. Perdan (2005 a) An integrated sustainability decision-support
framework Part II: Problem analysis. International Journal of Sustainable
Development & World Ecology, 12(2), 112-131
• Azapagic, A. and S. Perdan (2005 b) An integrated sustainability decision-support
framework Part I: Problem structuring. International Journal of Sustainable
Development & World Ecology, 12(2), 98-111
• Benetto, E. and C. Dujet (2003) Uncertainty analysis and MCDA: a case study from
the life cycle assessment (LCA) practice. 57th Meeting of the European working
group on multicriteria decision aiding, Viterbo, 27–29 March.
• Cinelli, M., S. Coles and K. Kirwan (2014) Analysis of the potentials of multi-criteria
decision analysis methods to conduct sustainability assessment. Ecological
Indicators, 46, 138-148.