Annals of Management Science

Vol. 3, No. 2, December 2014, 65- 86

A Multicriteria Model to Evaluate the Performance of Transport

Companies: the Case of Tunisia

Inaam Zaouali* and Hachmi Khouaja

lHEC Institute of the high commercial Studies Carthage Presidence- Tunis 2016 Tunisia

Abstract

It is not enough to base the evaluation of the performance of transport companies on

only economic criteria. Criteria for evaluating the performance of transport companies
should include both quantitative and qualitative criteria that address the desires,
expectations and the needs of all stakeholders (passengers, investors, the community,
the government and the owners/authorities of transport companies). This paper
develops a desegregated multi-criteria approach for evaluating performance of
transport companies. The aim of the paper is to present an effective method or
procedure that can be used by top managements of transport companies to evaluate the
performance of their companies. The set of criteria developed in the paper covers
different aspects of transportation service performance metrics such as quality, safety,
operational performance - in terms of efficiency and effectiveness. A case study on 12
bus companies of a public transport system in Tunisia is conducted to illustrate the
effectiveness of the approach .We presented and applied the ELECTRE TRI method to
assign the different alternatives(transport companies) in predefined categories. In
order to rank these alternatives the ELECTRE TRI method is adopted. Sensitivity
analysis is conducted to test the validity of the results.

Keywords: multi-criteria decision aid, performance assessment, and public transport

1. Introduction

Performance evaluation is an important means of promoting the operational

efficiency and service quality of public transport system (Hsing et al., 2000).
However, the criteria for evaluating the performance of transport companies are
generally multiple and involve subjective assessments resulting in qualitative data.
Many studies focus on economic criteria in terms of efficiency and
effectiveness (Colson et al 1998; De Borger et al., 2002; Sabri et al., 2008). Some
investigations focus on the quality of transport service (Toni 2013). Others focus on
safety, using indicators related to the number of accidents and deaths (Tingvall et al.,
2010).
Despite the increasing amount of attention being paid to the evaluation of
transport performance, many researchers and practitioners still fail to consider or use
quantitative and qualitative criteria while doing the evaluation. In fact major
studies already published in this area are focused mainly on the use of quantitative
criteria. Thus, most of these studies are not able to adequately evaluate the different
*
Corresponding author. Email: zaoualiinaam@gmail.com
66 I. Zaouali and H. Khouaja

aspects of transport performance (Sabri et al., 2008;Colson et al., 1998; Tingvall et

al., 2010; Gianfranco et al., 2014a). What this means is that current methodologies
for evaluating performance are not sufficiently comprehensive, particularly when it
comes to taking account different aspects of performance.
Indeed the use of frontier methods for evaluating the performance of public
transport (De Borger et al., 2002; Gianfranco et al., 2014a) companies can lead
operators to missing results. Generally these models have their shortcomings. The
shortcomings include: (a) failure to sufficiently take the goals of public transport
companies - in terms of safety (number of accidents) and quality of service (comfort,
cleanliness, etc.) – into account and (b) the inadequacy in using only the operational
criteria of performance (efficiency and effectiveness).
Our study is of paramount importance to academicians and practitioners as the
proposed framework is expected to uncover many new or previously unused
qualitative and quantitative criteria that address the desires, expectations and the
needs of all stakeholders. The study can also be seen as an answer to the call for
research seeking more investigation into transport performance evaluation problem
(Hsing et al., 2000; Gianfranco et al., 2014b) by identifying appropriate performance
measures.
The technique adopted in the study is the multi-criteria analysis. This approach
is used in decision analysis to make selections among different alternatives when a
decision maker is faced with complex decision problems (De Brucker et al., 2004).
Out of many different available multi-criteria decision analysis approaches, we will
use desegregated multi-criteria approach in the study. The proposed methodology
will also address the existing literature gap with respect to the use of performance
measures or of more than one performance measure.
The methodology/model and performance measure or criteria developed here
will be used to assess the performance of 12 public bus companies in Tunisia, which
is a practical or an important real world case.
In the study, we will show that the use of multi-criteria decision analysis
method can be more interesting and effective in assessing the performance of public
transport. Our goal is to build a set of criteria which covers all different aspects of
transport or public transport performance.
This work is divided into two parts: In the first part, we propose a set of main
criteria that covers different aspects of transport performance and which can be a
good basis for the evaluation of transport companies’ performance. In the second,
we present a case study concerning twelve transport companies in Tunisia.

2. Evaluation of the performance of transportation companies using Multi-

criteria Decision Analysis

Of recent, researchers have shown interest in multi-criteria methods for evaluating

the performance of public transport companies (Colson et al., 1998; Hsing Yeh et
al., 2000; Ihsan Kaya et al., 2012; Sabri et al., 2008; Gianfranco et al., 2014b).
Usually, the performance evaluation process involves the following steps: the
identification of stakeholders and their objectives, the construction of a criteria
family covering the different aspects of transport performance (mode of evaluation,
Annals of Management Science 67

scale, and criteria discriminating power), the determination of relative weight of

importance for the criteria and the choice of a method or technique for aggregating
preferences.

2.1. Identification of Stakeholders and their Objectives

2.1.1. Identification of Stakeholders in the Transport Sector

Incorporation of stakeholder expectations is a necessary step for evaluating the

performance of a service (Toni 2013). Stakeholders constitute a core constituency in
any transport company or organization (Steg et al., 2005).
Hsing et al (2000) and Sohail et al (2004) identify three main stakeholder
groups that can be responsible for the evaluation of transport performance: transport
authorities, operators and users. These actors are strongly interdependent. Each one
may well have different perceptions, objectives, constraints, and options which all
need to be balanced.

The main stakeholders in the transport sector are as follows (see Figure 1)

Transport authorities

and

and have
regarding

paysrepresentatives

responsability for
Elects their representatives
servicdelivered

transportation
that is delivered
Accounts regarding the

pays taxes

Organize and have

Delegate public transport

Organizefor
taxes

transportation
service Accounts

responsibility
ts their
and
D

Co-product and finance

the company

Operators Users

Carry the user

Figure 1. Stakeholders in public transport sector

2.1.1.1. The transport authorities

The transport authority, defined as a public authority, is a group of government or

public institution, responsible, directly or by delegation, for the establishment and
organization of a regular land-based public transport service that is people-oriented
and is not for tourism. Transport authorities are primarily the transport ministry and
other government agencies and also the municipal traffic departments who regulate
the traffic and licensing authorities (Sohail et al., 2004).
68 I. Zaouali and H. Khouaja

2.1.1.2. The operators

The providers and operators of the services are transport companies, drivers
associations, and owners associations (Sohail et al., 2004). The operating companies
of transport networks will manage the public transport service when the transport
authority gives them the charge to do so under an agreement that specifies the
responsibilities and sets targets for proposals and revenues. These are regularly
evaluated and revised.

2.1.1.3. The users

The traveler, the passenger or more accurately the user of public transport, is the one
who co-produces and finances the transport company. All users who use public
transportation are required to submit their transport subscription whatsoever
(subscription to lines network, subscription to school transport, etc...). The users are
particularly low-income households including men, women, children, elderly, and
disabled members of the community (Sohail et al., 2004).

2.1.2. The Main Expectations of Stakeholders

Toni et al (2013) take into primary consideration the users' expectations. The users '
point of view is linked to their perceived quality of the delivered service and can be
considered the main driver of the investment choices to improve the service quality.
Several quality dimensions have been proposed to evaluate the quality of service
(Leurent 2009). These include punctuality, comfort, cleanliness, passenger
information etc.
The transport authorities can add other criteria that relate to their concerns
such as social responsibility, safety and security (Hsing et al., 2000). According to
Toni et al (2013), the transport authorities’ point of view is affected by matters
relating to equipment, in terms of quantity, quality and safety, and the environmental
impact of the service.
The operators’ point of view (particularly Transport Company) essentially
tends to focus on costs efficiency / effectiveness and service-effectiveness (Bertini et
al., 2003).
The transport performance evaluation should necessarily reflect the
expectations of different stakeholders like the operators and the transport authorities
that are directly or indirectly involved in the transit service (Toni et al, 2013).
Consequently the key performance metrics on which the performances of transport
companies are evaluated with respect to how much they are meeting stakeholder
expectations and improving the attractiveness of public transport are:

2.1.2.1. Service quality

For public transport users, service quality is fundamental. Improving the quality of
service be at the heart of transport policy makers (AFNOR, 1997).
Annals of Management Science 69

Several criteria can be used to measure service quality in a transport company

(Leurent 2009; Garrido2005;Eboli and Mazzula 2007).
The main criteria that we have identified in this research are: (a) Punctuality,
(b) Comfort (c) Cleanliness (d) Passenger information and (e) Environmental
impact (See table 1)

Table 1. Dimensions of service quality (Source adapted from Eboli and Mazzula .,
2007)
Dimensions Description
Punctuality Reliability of the scheduled runs, Punctuality of the runs;
Comfort Bus crowding Comfort of bus seats ,
Availability of shelter and beaches at bus stops
Cleanliness Cleanliness of bus interior, seats and windows,
Cleanliness of bus exterior
Passenger Information Availability of schedule/maps on bus ,Availability of
schedule/maps at bus stops, availability of information by
phone internet ,Personnel appearance ,Personnel
helpfulness
Environmental impact Air condition on bus, Level of vibration on bus , air
pollution, the greenhouse effect, the smells, the
congestion

2.1.2.2. Road safety

Several attributes of safety and security are identified by Eboli & Mazzula (2007)
such bus reliability, competence of drivers, security against crime on bus, security
against crime at bus stops , number of accidents etc .
Indeed transport companies must give great importance to the safety of
passengers and staff on public transport. Security represents a crucial issue for
public transport (Labathe, 2005).

2.1.2.3. Operational performance (efficiency and effectiveness)

Cost-efficiency, cost-effectiveness and service-effectiveness are commonly used as

performance indicators to evaluate the operational performance of organizations
transit (Charte des services locaux, 2008;Hsing et al., 2000; Cruz et al., 2012;
Bertini et al., 2003) .
These indicators can be used to represent the relationship between the service
inputs, the service outputs and the consumption of transit operations. Efficiency
measures reflect the degree of the proper use of resources and effectiveness
measures reflect the extent to which the transport service meets the needs of
passengers. (See Figure 2).
70 I. Zaouali and H. Khouaja

Cost-
efficiency

Service Outputs:
Service Inputs:
Vehicle Hours,
Labor, Capital, Fuel Vehicle Kilometers,
Capacity Kilometers
Operationnel
performance

Service
Consumptions:
Cost Passengers, Passenger Service
effectiveness effectiveness
Kilometers, Operating
Revenue
Figure 2. Relationship between efficiency and effectiveness indicators (source
adapted from Bertini & El Geneidy, 2003)

Service inputs is the amount of resources used to produce the transport

service in monetary or non-monetary terms (the number of employees, number of
vehicles, the operating cost (amount spent on operations vehicle, administration,
maintenance, etc.), hours of work capital investment and energy consumption (fuel
cost or volume). Service outputs is the amount of service produced by the service
system .
Service consumption is the amount of service consumed by the public
expressed in monetary or non-monetary terms (the number of trips, number of
passengers, operating revenue etc...).
According to Bertini & El Geneidy (2003) and Hsing (2000), these indicators
have been widely used as performance indicators for evaluating the operational
performance of public transport. They may be used to represent the service inputs,
service outputs and consumption of public transport operations respectively. In the
case of bus companies, cost efficiency, cost effectiveness, and service effectiveness
are measured by the total vehicle-kms run per employee, the total number of
passenger trips served per employee, and the total number of passenger trips carried
per vehicle-km, respectively ((Hsing et al., 2000).

2.2. Criteria for Evaluating the Performance

In decision analysis, the choice of consistent family of criteria presents the most
important step required by multi-criteria approach (Roy, 1985).
Annals of Management Science 71

Depending on the expectations of different stakeholders, various criteria can

be used to evaluate public transport systems. Three important dimensions are
considered: (a) service quality, (b) safety and (c) economic efficiency (see Table 2).

Table 2. Criteria evaluating the performance of transport companies

Dimensions Code Nom Evaluation direction
mode

C1 Timeliness of the service rate Max

C2 Respect of Time rate Max

schedule
Service quality

C3 Cleanliness Qualitative Max

C4 Comfort Qualitative Max

C5 Passenger Information Qualitative Max

C6 Environment nuisance Qualitative Min

C7 Accidents rate rate Min

Safety

C8 Mean age of vehicles quantitative Min

C9 Cost effectiveness earnings/agent Max

Economic
efficiency

C10 Cost efficiency Miles/agent Max

C11 Service effectiveness Customers/mile Max

2.2.1. Criterion 1(C1): Timeliness of the Service

It is an important and objective factor in assessing the quality of service. Scheduled

vehicle departure times and punctuality of vehicles at departure points have a great
impact on user or passenger satisfaction. The more vehicles are punctual at the
departure point the more satisfied travelers feel. This criterion is to be maximized. It
is evaluated in terms of a percentage calculated from a survey of customer
satisfaction.

2.2.2. Criterion 2 (C2):C2 Respect for Time Schedule

This is also considered an important and objective element in assessing the quality
of service. Indeed the respect for time schedule improves transport service quality.
The more respect that transport operators have for time schedule the more punctual
that vehicles arrive at arrival point and the more satisfied users feel. This criterion is
72 I. Zaouali and H. Khouaja

also to be maximized. It is measured in terms of a percentage calculated from a

survey of customer satisfaction.

2.2.3. Criterion 3(C3): Cleanliness:

This criterion measures the degree of cleanliness of stations and the inside of the
vehicles provided by the transport company. This criterion is to be maximized. It is
measured with qualitative scale.

Table 3. Scale of Cleanliness criterion

Qualitative scale Codification
Clean 5
Fairly clean 3
Not clean at all 0

2.2.4. Criterion 4(C4): Comfort:

This assesses psychological comfort (feeling of safety (stations and on vehicles (not
flying) brightness) and physical comfort (sitting, standing). We use a qualitative
scale to measure this criterion.

Table 4. Scale of comfort criterion

Scale Coding
Very comfortable 10
Comfortable 7
Moderately comfortable 5
Uncomfortable 2
Very uncomfortable 0

2.2.5. Criterion5 (C5): Passenger Information

A high quality and accessible user information and a warm welcome by the staff is
a prerequisite for a better and greater use of public transport. The lack of information
and respect for users then constitute a great impediment to the use of public
transport. We use a satisfaction scale to measure this criterion.

Table 5. Scale of passenger information criterion

Scale Coding
Highly satisfied 5
Satisfied 4
Partially satisfied 3
Not satisfied 1
Not at all satisfied 0
Annals of Management Science 73

2.2.6. Criterion6 (C6): Environment Nuisance

The pollution caused by passenger transport is noise and vibration, air pollution, the
greenhouse effect, the smells, the congestion (the concentration of automobile traffic
in a limited space, etc. This criterion evaluates the impact generated by vehicles on
users following the issuance of such nuisances.

Table 6. Scale of environnent nuisance

Scale Coding
Very high impact 5
High impact 4
Average impact 3
Low impact 1
Very low impact 0

2.2.7-Criterion 7 (C7): Accidents Rate

This is an indicator of the level of road safety for the transport company. The
accident rate corresponds to the number of accidents observed during a given period
(year in this case) divided by the number of miles travelled by all vehicles in a year:

2.2.8. Criterion 8( C8): Mean Age of Vehicles:

This measures the average age of the fleet of the company. We use as measuring
unit the number of years.

2.2.9. Criterion 9:C9: Cost Effectiveness

This indicator represents the relationship between the service consumption and the
service –inputs. The cost effectiveness is measured by the following rate:

2.2.10. Criterion 10:C10: Cost Efficiency

This indicator represents the relationship between service –inputs and service-
outputs. The cost efficiency is measured by the following rate:
74 I. Zaouali and H. Khouaja

2.2.11. Criterion 11:C11: Service Effectiveness

This indicator represents the relationship between the service -outputs and service-
consumption. The cost effectiveness is measured by the following rate:

3. Application to Evaluate Tunisian Transport Companies

The performance evaluation, in this case, covers multiple performance indicators,

namely: the importance of investment, demand satisfaction, local and urban
development. In this research, we have carried out two types of evaluations. These
are absolute evaluations in terms of good, so good ..., bad and a relative evaluation
in terms of ranking.

3.1. Absolute Evaluation

Companies are compared to predetermined levels of performance. The objective is

to assign each company to a given category taking into account their evaluation on
the criteria family.
The operational aggregation approach retained is "ELECTRE TRI ". It uses
two allocation procedures, pessimistic and optimistic. This enables the method to
take into account factors related to the imprecision, uncertainty and indeterminacy of
data and to avoid making arbitrary judgments. Indeed ELECTRE TRI presents a
multi-criteria decision aid tool specifically designed to address the sorting problems.
Given a finite set of alternatives, the sorting problem assigns each alternative
or action (Transport Company in our case) to one of the pre-defined categories. It
formulates the decision problem in terms of a classification so as to assign each
action from A to one of the predefined categories. The assignment of an action “a”
to the appropriate category should rely on the intrinsic value of “a” (and not on the
comparison of “a” to other actions from A) (Mousseau et al., 1998). (see Figure 3)
Annals of Management Science 75

Actions A

x xx Category 1
x

xx
Category 2
xx
xx

xx
Category 3
xx

Figure 3. Sorting problem

The sorting problem is related to absolute judgments. It consists of assigning each

alternative to one of the categories which are pre-defined by some standards
corresponding to vectors of scores on some particular criteria, called profiles
defining the limits of the categories, either separating the categories or playing the
role of central reference points in the categories.
The assignment of an alternative a results from the intrinsic evaluation of a’s
role on all criteria with respect to the profiles defining the categories (the assignment
of a to a specific category does not influence the category to which another
alternative should be assigned) (Koen 2008).
Let F presents the set of the indices of the criteria g1, g2,….gm (F =
{1,2,….,m }and B the set of indices of the profiles defining p+1 categories (B=
{1,2……,p}), bh being the upper limit of category Ch and the lower limit of category
Ch+1 h=1,2,…..,p (see Figure 4, where the profiles bp+1 and b0 concern the ideal and
the anti-ideal alternatives respectively).
ELECTRE TRI assigns alternatives to categories through two consecutive
steps (Mousseau and al, 2000). Building an outranking relation S that characterizes
how alternatives compare to the limits of categories. Indeed, ELECTRE TRI builds
an outranking relation S and validates or invalidates the assertion aSbh (and bhSa)
where aSbh means a is at least as good as bh. Preferences restricted to the
significance axis of each criterion are presented through pseudo-criteria. The
indifference and preference thresholds (qj (bh) and pj (bh )) present the intra-
criterion preferential information. They account for the imprecise nature of the
evaluations gj (a).
76 I. Zaouali and H. Khouaja

 qj (bh ) represents the largest difference gj (a) - gj (bh ) that preserves

indifference between a and bh on criterion gj.
 pj (bh ) defines the smallest difference gj (a)-gj (bh ) compatible with a
preference in favor of a on criterion gj .

Figure 4. Definition of categories using limit profiles

ELECTRE TRI builds an outranking relation S using an index ∂(a, bh) (∂ (bh ,
a) ϵ [0,1], resp.) that defined the degree of credibility of the assertion aSbh (bhSa,
resp.), "a ϵA," h ϵB.
Exploitation of the relation S conducted to assign each alternative to a specific
category. Since the assignment of alternatives to categories does not result directly
from the relation S, an exploitation procedure is necessary. It requires the relation S
to be “defuzzy fied “ using a so called λ-cut the assertion aSbh (bhS a, resp.) is
valided if ∂(a, bh) ≥λ (∂(bh , a) ≥λ, resp.),λ being a `cutting level or the threshold
cut): a such that λϵ[0.5,1]. This λ-cut represents the preference situation between a
and bh:
 ∂(a, bh ) ≥ λ and ∂(bh, a) ≥ λ aSbh and bhSa aIbh , i.e., a is
indifferent to bh , (1)
 ∂(a, bh ) ≥ λ and (bh ,a) ˂ λ aSbh and not bhSa a›bh , i.e., a is preferred
to bh (weakly orstrongly),
(2)
 ∂(a, bh) ˂λ and ∂(bh , a) ≥ λ not aSbh and bhSa bh› a, i.e., bh is
preferred to a
Weakly or strongly
(3)
 ∂(a, bh )˂λ and ∂(bh, a) ˂ λ not aSbh and not bhSa aRbh, i.e., a is
incomparable to bh (4).
Annals of Management Science 77

The role of the exploitation procedure is to analyze the way in which an

alternative a compares to the profiles so as to determine the category to which a
should be assigned. Two assignment procedures are available.
The pessimistic procedure assigns each alternative to the highest category for
which that alternative outranks the category inferior reference profile. The procedure
runs thus:
 Compare successively alternative a with reference profile bi, with i varying
from p to 0
(from higher to lower categories);
 Let bh be the first reference profile for which aSbh; then a is assigned to
category Ch+1.
The optimistic procedure compares a successively to bi for i =1, 2…p+1,-
bh being the "First profile such that if aSbh a ,then a is assigned to category Ch .
If bh+1 and bh represent the lower and upper profile of the category Ch, the
pessimistic procedure assigns alternative a to the highest category Ch such that a
outperforms bh-1, (aSbh-1). When using this procedure with λ=1, an alternative a can
be assigned to category Ch only if gj (a) equals (up to a threshold) or exceeds gj (bh)
for each criterion (conjunctive rule).
The optimistic procedure assigns a to the lowest category Ch for which the
lower profile bh is preferred to a ( bh Sa.) When using this procedure with λ=1, an
alternative a can be assigned to category Ch when gj (bh) exceeds gj (a) (by some
threshold) for at least one criterion (disjunctive rule). When λ decreases, the
conjunctive and disjunctive characters of these rules are weakened.
We can summarize Electre tri using the following figure.

Definition of Definition of Definition of Decision maker

criteria alternatives categories preference

Reference Preferential
Evaluation matrix parameters
Profiles

Outranking
relation

Assignment
procedure

Assignement of
alternatives

Figure 5. General scheme of the use of Electre Tri

78 I. Zaouali and H. Khouaja

3.1.1. The Data

Our case decision-problem relates to regional transport companies. The input data
and parameter settings are presented in Table 7 below. Rows present the twelve
transport companies
(A1 to A12) and (wi, pi, qi) are respectively, the weight, the thresholds of
indifference and preference. Colons present a set of criteria (C1 to C2).The
intersection present he evaluation of each company on each criterion:

Table 7. Performance matrix

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11

A1 73 70 3 2 1,53 3 0,4 6,7 13,4 23,5 4,7

A2 66, 55 3 5 2,5 4 0,5 6,9 17,6 23,0 3,1

A3 64 54 0 0 3,64 1 0,6 8 30,3 36,3 2,0

A4 50 46 0 2 1,75 3 0,35 7 21,4 20,5 5,7

A5 55 42. 0 2 1.54 4 0,47 8 34,6 25,0 2,1

A6 45 43 0 7 3,75 5 0,67 6.5 11,3 35,4 4,0

A7 55 47 3 0 2.25 3 0,43 6,7 12,5 18,5 3,7

A8 48 45 3 2 2,72 5 0,59 8,9 17,6 22,1 2,5

A9 33 34 5 7 2.77 3 0,56 9 32,3 27,3 3,1

A10 58 55 0 0 2,34 5 0,54 6,7 23,4 21,5 5,7

A11 73 65 3 2 2,57 1 0,75 7.9 17,7 19,9 2,5

A12 56. 54 0 5 2,62 5 0,56 8 14,3 32,3 4,1

Sens Max Max Max Max Max Min Min Min Max Max Max

wi 0,10 0.10 0,07 0,07 0,08 0,05 0,14 0,07 0,10 0,10 0,12

qi 5 5 1 1 0 1 0 0 1 1 0,5

pi 10 10 2 2 1 2 0,1 0,5 3 3 1
Annals of Management Science 79

The use of the ELECTRE TRI method requires the arbitrary specification of a
number of parameters. First, a weighting system must be associated to the chosen
criteria in the first part (wi). Other parameters called thresholds are also defined.
Indeed , ELECTRE TRI allows us to take into account pseudo- criteria instead of
real - criteria by using thresholds of indifference (qi)and preference (pi) non- nulls.
(wi, pi, qi are respectively, the weight, the thresholds of indifference and
preference).
Four categories of companies are defined. These are:
1. The category of companies classified as”very successful” (VS). These are
high-performing companies in terms of quality of service, road safety and
economic efficiency and performance.
2. The category of companies classified as”successful” (S). These companies
have satisfactory performance in terms of service quality, road safety and
economic efficiency.
3. The category of companies classified as "less efficient" (LS). This category
of companies is characterized by poor performance in terms of service
quality, road safety and economic efficiency.
4. The category of "non-classified" companies (NC). These companies are
characterized by very poor performance in terms of service quality, road
safety and economic efficiency.
Each category is characterized by a reference profile (R3, R2, and R1). R3 (the
high profile) is greater than R2 (the satisfactory profile) and R2 is greater than R1
(low profile). Each profile represents the boundary between two adjacent categories.
They define the four categories already identified above (VS, S, LS, and NC). R3 is
the boundary between VS and S, R2 is the boundary between S and LS and R1 is the
boundary between LS and NC. Each profile depends on the performances of the
companies on each of the eleven criteria and preferences of different stakeholders
(importance of the criteria, etc.). R3 profile presents the vector of optimal values of
the criteria. R2 profile is the vector of satisfactory values. R1 profile is the vector of
lower values (see Table 7):

Table 8. The profiles

Profils
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11

R3 90 90 5 10 5 0 0.3 6.5 32 36 5
R2 60 51 3 5 3 3 0.5 7,5 22 25 3
R1 30 30 0 0 0 5 0.7 9 12 18 2

From table 8 the reference profiles values are calculated taking into account
the stakeholders’ expectations and the judgments of some experts in the transport
field.
The weights are determined by a pair wise comparison of the criteria using
the approach developed by Saaty (1990). The results are then validated by the
relevant stakeholders intervening in the decision aid process (see table 8).
80 I. Zaouali and H. Khouaja

Table 9. The weights of criteria with AHP method (CI = 0.022)

Criteria C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11

Weights 0.1029 0.1029 0.0443 0.0787 0.0848 0.0523 0.1386 0.067 0.103 0.103 0.1226

The weights are calculated using the analytic hierarchy process software. This
software also allows you to evaluate a posteriori the consistency of judgments using
the CI (Consistency Index). The purpose for calculating the consistency index is to
make sure that the original preference ratings were consistent. In practice, a CI of
0.1 or below is considered acceptable. Any higher value at any level indicate that
the judgments warrant re-examination (If the CI is greater than 0.1, the matrix
evaluations should be reassessed).
This method has several advantages. On the one hand it takes into account all
the tangible and intangible criteria, quantitative and qualitative. On the other hand it
is characterized by its simplicity in calculating the weights. It does not also
necessarily seek consensus but develops a representative compromise of various
judgments. Additionally, it allows us to evaluate the logical consistency of
judgments using the CI (Consistency Index).

3.1.2. The Results of the Electre Tri Method

As a reminder, we have three reference profiles such as R3> R2> R1 and therefore
four ordered categories VS > S > LS > NC. The pessimistic assignment procedure
does rankings as follows. The company Ei is compared successively with R3, R1
and R2 so that if:
 Ei Outperforms R3, Ei is assigned to the VS category: The Company is
considered very successful
 Ei outperforms R2 but does not outperform R3, Ei is assigned to the S
category: Ei will then be considered as successful.
 Ei outperforms R1 but does not outperforms R2, Ei is assigned to the LS
category: The company is then considered less successful.
 R1 outperforms Ei; Ei is then assigned to the NC category and the company
is assigned to the not- classified category.
The results of the pessimistic assignment procedure for Cut threshold λ=0.76
(the default value of the software) are:

Table 10. Results of pessimistic procedure

Categories
VS : High performance
S : Satisfactory performance
LS : Poor performance
NC : non classified
Cut threshold λ=0,76
A1, A2, A4
,A3,A5,A6,A7,A8,A9,A10,A11,A12

From this table (table 10) we can conclude that only three alternatives are classified
to satisfactory performance. Others companies are assigned to poor performance.
Annals of Management Science 81

The results of optimistic assignment procedure for a cut threshold λ = 0.76 are:

Table 11. Results of optimistic procedure

Categories Cut threshold λ=0,76

VS : High performance

S :Satisfactory performance A1, A2, A4, A7, A11,

LS :Poor performance A3, A5, A6, A8, A9,

A10, A12

NC : non classified

From this table (table 11) we can conclude that more companies are classified
to satisfactory performance category.
It can be seen from table 10 and 11 that the sorting results of the ELECTRE
TRI method using two procedures are different. The pessimistic procedure tends to
classify the major companies to the poor performance category (LS). Only three
companies are classified to the satisfactory performance category (S). This
procedure is applied when a policy of prudence is necessary because it tends to
classify the companies to the lower possible category.
The optimistic procedure tends to classify more companies to the satisfactory
performance category (S). In fact this procedure tends to classify the companies to
the higher possible category in contrast to the pessimistic procedure.
We retain the results of the pessimistic procedure of assignment for sensitivity
analysis. The reason for this is that the optimistic procedure tends to classify the
companies to the higher possible category. This is in contrast to the pessimistic
procedure that tends to classify the companies to the lower possible category. This
procedure is applied when a policy of prudence is necessary. In ELECTRE TRI
method, the assignment depends primarily on the value of the cutting level and the
allocation procedure.
A sensitivity analysis with respect to thresholds cutting yields results that
assess the significance of changes in the allocation obtained. We distinguish three
types of changes, namely:
a) Change type 1: occurs when a company is moved from category C1 to C2, or
C2 to C 3 or C3 to C4 and vice versa.
b) Change type 2: occurs when a company is moved from category C1 to C3 or
C2 to C4 and vice versa.
c) Change type 3: occurs when a company is moved from category C1 to C4
and vice versa.
82 I. Zaouali and H. Khouaja

The results of the sensitivity analysis with respect to the variation of the
threshold cuts are summarized in Table 12.

Table 12. The sensitivity analysis

λ=1 λ =0,9 λ =0,8 λ = 0,6
VS
S A1, A2, A4 A1, A2, A4 A1, A2, A4 A1, A2, A4, A7,
A11
LS ,A3,A5,A6,A7, ,A3,A5,A6,A7,A8 ,A3,A5,A6,A7, A3, A5, A6, A8,
A8,A9,A10,A1 ,A9A10,A11,A12 A8,A9,A10,A11 A9, A10, A12
1,A12 ,A12
NC

The importance of changes for different thresholds cutting reveals that when
the cutting threshold varies between 0 .76 and 1, assignments remain the same.
There is no change type 2 or 3. There are two changes type 1 for λ = 0.6. This
indicates stability of the results.

3.2. Relative Evaluation

Here, we base the analysis, on a classification procedure which aims to rank

companies in a decreasing order (Electre III method). Instead of focusing his
investigation on the identification of defined classes from the intrinsic value of
companies, an analyst may like to compare companies in order to group them into
classes and rank the classes. The classes do not belong to an a priori definition: the
meaning of each is only relative since it depends on its position in the rank. .
The ranking algorithm allows the development of both partial and
contradictory pre-orders. Downward distillation allows extracting all possible
actions, a subset of best actions with the application of a certain selection rule,
relatively severe. In this subset, we try again to select the best alternatives (the best
of the best), but this time with the application of a little less demanding rule.
The new subset logically smaller than the preceding subset, is extracted from
the preceding subset. With a less and less demanding rule, we extract each time a
smaller subset from the proceeding subset until one does not have any more in the
last subset but only one action or actions that we cannot peel any more. This action
or actions are the first class, the best. This process is a distillation process.
The second distillation takes place in the same way: we get back to all
potential actions. We remove only the actions of the first class (the best) and applies
the same procedure. In double distillation, one finishes by obtaining final distillate
which forms the second class of the future ranking: The best of what remains after
extraction of the best. With each distillation, one thus goes down from a notch in the
row in which the actions of the class thus extracted will occupy distillation stops
when the unit remaining becomes empty.
Annals of Management Science 83

In contrast to descending distillation (downward distillation) which selects the

best rated alternatives initially and finish with the worst, the ascending distillation
selects the worst rated alternatives first and terminates with the assignment of the
best alternatives (Roy, 1978).

3.2.1. Presentation of the Results of ELECTRE III Method

The results are presented in the following graph (see Figure7).The company A1
outranks all the other companies. The second best position is occupied by A2 and
A4. Companies A7 and A11 are third. The fourth position is occupied by companies
A3 and A12 while A5 and A10 are ranked fifth. The company A8 occupies the 6th
place while the last class is occupied by A6 and A9.
We can conclude that company A1 is the most efficient. This ranking is stable
for different sets of weights considered. Company A4 and A1 rank first when road
safety criterion has a larger importance than those of service quality and operational
performance criteria. In the case where the quality of service criteria are considered
more important than the two other types of criteria, A1 is ranked higher than the
other companies in terms of overall performance. This result is consistent with those
obtained in absolute evaluation.

Figure 7. The ranking of companies using ELECTRE III

84 I. Zaouali and H. Khouaja

4. Summary

This study shows the applicability and usefulness of multi-criteria decision analysis
technique in evaluating the performance of transport companies. Absolute
evaluation poses problem in terms of sorting companies by categories when each
category has an intrinsic definition. That is, when each category has no reference to
other categories. Relative evaluation poses problems in terms of ranking of these
companies.
This paper was focused on the application of a multi- criteria method for the
evaluation of the performance of transport companies in Tunisia and for ranking
them according to their performance. To model this problem we began by building a
consistent family of criteria: 11 criteria grouped into three categories were used.
These categories are service quality, safety and operational performance in terms of
efficiency and effectiveness. We presented and applied firstly the ELECTRE TRI
method to assign the different alternatives in predefined categories. To test the
validity of the results, we did a sensitivity analyzes .These analyzes show that the
results obtained with the threshold method are stable. Then the ELECTRE III
method is applied to classify the companies.
The application of partial aggregation methods (ELECTRE TRI and
ELECTRE III), in evaluating the performance of public transport companies allows
us to have a global view. Its other advantages are:
a) The use of quantitative and qualitative data.
b) Possibility of using cardinal and ordinal criteria.
c) Limit the phenomena of compensation between the criteria.
d) Avoid the aggregation of multiple criteria in a single synthesis.
e) The possibility of using fuzzy or uncertain data by using the concept of
pseudo-criteria and thresholds.

References

AFNOR(1997), Norme expérimentale. Identification des critères de qualité pour les

transports de voyageurs. XP X 50-805
Bertini RL,ElGeneidy A (2003) .Using archived data to generate transit performance
measures. In 82 TRB Annual Meeting Washington D 12-16 Jannuary
Charte des services publics locaux. (2008). Indicateurs de performance des services
de transportpublic
[Online]Available :http://www.lesechosevents.fr/data/classes/produit_partenai
re/fichier_2827_433.pdf (Mars,2008)
Colson G. & Mbangala M (1998).Évaluation multicritère d’entreprises publiques
du rail. La revue canadienne FINÉCO, 8(1), 1er semestre 1998
De Borger B., Kerstens K. & Costa A. (2002) .Public transit performance: what does
one learn from frontier studies? Transport Reviews, 22(1),1-38
De Brucker, K., Verbeke, A., & Macharis, C. (2004). The applicability of multi-
criteria-analysis to the evaluation of intelligent transport systems (ITS).
Research in Transportation Economics, 8(15), 1-179.
Annals of Management Science 85

Eboli L,Mazzula,G (2007). Service quality attributes affecting customer satisfaction

for bus transit. Journal of Public Transport, 10 (3) 21-34
Hsing Yeh C., Deng H. & Chang Y. (2000). Fuzzy multi-criteria analysis for
performance evaluation of bus companies. European Journal of Operational
Research, 126(2000) 459-473
Ihsan kaya, Başar öztayşi, & Cengiz kahraman (2012). A two-phased fuzzy
multicriteria selection among public transportation investments for policy-
making and risk governance. International Journal of Uncertainty, Fuzziness
and Knowledge-Based Systems, 20, Issue supp01, 31-48
Koen Renée (2008). Aspects of MCDA classification and sorting methods. [Online]
Available
http://uir.unisa.ac.za/bitstream/handle/10500/2678/dissertation_koen_%20r.pd
f?sequence=1 (November, 2008)
Gianfranco F, Barbara U, Paolo F (2014a). Data Envelopment Analysis (D.E.A.)
for urban road system performance assessment. Procedia - Social and
Behavioral Sciences, 111, 780 – 789.
Gianfranco F , Carta M.a and Fadda P (2014b). A modeling tool for measuring the
performance of urban road networks. Procedia - Social and Behavioral
Sciences, 111, 559 – 566.
Garrido M. (2005). Le système de transports collectifs à Saint-Quentin-en-Yvelines
La qualité de service : évolutions, problèmes et perspectives. [Online]
Available :
http://www.transponts.com/fichiers/travaux/travaux_2005/MGQualite_service
_SQY.pdf(Janvier, 2005).
Leurent F. (2009). Qualité de service en Transport Collectif, et aide à la décision.
Présentation au pôle ADVANCITY. Université Paris-Est, LVMT France
Luis Cruz, Eduardo Barata & João-Pedro Ferreira.(2012).Performance in urban
public transport systems: a critical analysis of the Portuguese case.
International Journal of Productivity and Performance Management, 61(7),
30 – 751.
Mousseau, Slowinski & Zielniewicz .(2000) .A user-oriented implementation of the
ELECTRE TRI method integrating preference elicitation support. Computers
& Operations Research, 27, 757-777
Labathe MC.(2005). Sécurité des transports publics occasionnels et collectifs en
autocar.
Available :http://www.ladocumentationfrancaise.fr/var/storage/rapportspublics
/054000220/0000.pdf (février ,2005)
OFT Office fédéral des transports (2013). Conception de la sécurité à l’OFT.
(janvier,2013).
Roy, (1978) ELECTRE III : Un algorithme de classement fondé sur une
représentation floue des préférences en présences de critères multiples. Cahier
du CERO, 20, 3-24.
Roy B. , (1985) Méthodologie multicritère d’aide à la décision. Economica Paris
(ouvrage) 423 pages
Sabri k., Colson G.E. & Mbangala A.M. (2008) .Multiple Criteria and Multiple
86 I. Zaouali and H. Khouaja

Periods Performance Analysis: The Comparison of North African Railways.

AIP Conference Proceeding, 351-365
Saaty, (1990). How to make a decision: The Analytic Hierarchy Process. European
Journal of Operational Research, 48, 9-26,
Steg, L., & Gifford, R. (2005). Sustainable transportation and quality of life. Journal
Of Transport Geography, 13, 59–69.
Tingvall,C., Stigson, H., Eriksson, L., Johansson, R., Krafft, M. & Lie, A. (2010).
The properties of Safety Performance Indicators in target setting, projections
and safety design of the road transport system., Accident Analysis &
Prevention, 42(2), 372–376.
Toni Lupo.(2013). Handling stakeholder uncertain judgments in strategic transport
service analyses. Transport Policy, 29 (2013) 54–63.
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.