Fuzzy risk metrics for the evaluation of future ATM scenarios

a
Dušan Crnogorac
a
University of Belgrade - Faculty of Transport and Traffic Engineering, Vojvode Stepe 305, 11000 Belgrade, Serbia

ARTICLE INFO ABSTRACT

Keywords: The paper presents fuzzy model developed in order to evaluate safety performance indicators of future air traffic
Fuzzy logic management (ATM) system. This model uses results from APACHE project (a SESAR Exploratory Research Project)
Air traffic management Separation Violation Detection Module (SVDM). Instead of processing data with computer time consuming TCAS
Safety parameters activation module and Risk of conflict assessment module in order to calculate safety performance indicators, this
model approximates safety performance indicators values. The fuzzy model is tested on 24 hour planned traffic for 10
different scenarios through French airspace. The model shows the ability to evaluate safety performance indicators with
high precision, giving airspace and traffic planners valuable data.

1. Introduction

The backbone of economic prosperity of any country and one of the fastest growing branches of world economy is air transport
industry. The expectations of the ATM community and the whole society are much bigger and primarily related to an increase in safety,
environmental protection, reduction in delays and ticket prices, etc. In such circumstances, the existing ATM system has to undergo certain
changes that will allow it to meet these often-contradictory requirements in the future. [1]
"Performance Review Commission" (PRC) was founded in 1998 by EUROCONTROL in order to establish transparent and
independent performance management system within the European ATM system. Directly involved in collecting and analyzing performance
data is "Performance Review Unit" (PRU). Each year, PRC issues "Performance Review Reports" (PRR) with information on the air traffic
demand and performance of the European ATM system in the four main Key Performance Areas (KPAs): safety, capacity, environment and
cost-efficiency. [2]
Safety has the highest priority in aviation and the ATM system plays an important role in ensuring overall safety of air traffic.
Uniform safety standards and practices in the field of safety management should be systematically applied within the ATM system. During the
implementation of the future global aviation system, safety should be assessed against appropriate criteria and in accordance with appropriate
and globally standardized safety management processes and practices. [1]
The APACHE project proposes a new framework to assess European ATM (air traffic management) performance based on
simulation, optimization and performance assessment tools that will be able to capture the complex interdependencies between KPAs at
different modelling scales. [3] Overall concept of APACHE framework is shown on Figure 1. Several scenarios to be studied are defined,
setting up different options regarding the demand of traffic, airspace capacities and eventual restrictions. [4]
The APACHE TAP (trajectory and airspace planner), which could be seen as a small prototype of an ATM simulator (Figure 1), has
a double functionality in the project [4]: a) to synthesize traffic and airspace scenarios representative enough of current operations; or
emulating future operational concepts in line with the SESAR 2020 ConOps, and b) to support the implementation of novel ATM PIs, which
require from some advanced functionalities (such as optimal fuel trajectories considering real weather conditions, optimal airspace opening
schemes, large-scale conflict detection, etc.). Then, the ATM Performance Analyzer (PA) module (Figure 1) implements all the PIs of the
APACHE performance framework, including as well some indicators from the current performance scheme for benchmarking purposes. [4]

Figure 1: The APACHE framework [4]

A part of APACHE platform - Risk Assessment (RA) belonging to the ATM performance Analyzer related to assessment of Safety
PIs of future ATM system is presented in this paper. Section 2. presents Safety PIs, Section 3. presents RA modeling approach. In Section 4.
fuzzy model is presented, Section 5. consists of numerical results, while Section 6. concludes paper.

2. Safety Performance Indicators

A part of the wider APACHE framework are Safety Performance Indicators (SPIs). Related to the scope of APACHE project, the
PRU is currently assessing a range of PIs in the field of safety, e.g. number of accidents and serious incidents, number of reported unauthorized
penetrations of airspace, number of reported separation minima infringements, etc., among which two are used as KPIs: total commercial air
transport accidents; and the number of accidents with air navigation service contribution. [1]
APACHE proposes two categories of SPIs: absolute and relative one. [1]
Absolute values of indicators are given in Table 1: Traffic Alert (TA) warnings (SAF-1), Resolution Advisories (RA) issued (SAF-
2), Near Mid Air Collisions (NMAC) (SAF-3). Similarly, number of potential Separation Violations (SV) could be used to indicate safety
(SAF-4). All indicators could be given as rates of specific occurrences as counts normalized by the number of flights or total flight hours
through the given airspace. [2] A complexity level in a given airspace is better described this way.

Table 1.
New Safety PI proposed, indicators with absolute values [1]

Indicators Unit Description

SAF-1: Number of Traffic Alerts warnings # TAs Count of TAs

SAF-1.1: Traffic Alerts warnings TAs/flight (hour) (Number of TAs) / (Number of flights or flight hours)
SAF-2: Number of Resolution Advisors issued RAs/flight (hour) Count of RAs
SAF-2.1: Resolution Advisors issued RAs/flight (hour) (Number of RAs) / (Number of flights or flight hours)
SAF-3: Number of Near Mid Air Collisions – NMACs # NMACs Count of NMACs
SAF-3.1: Near Mid Air Collisions – NMACs NMACs/flight (hour) (Number of NMACs) / (Number of flights or flight hours)
SAF-4: Number of separation violations # SVs Count of separation violations
SAF-4.1: Separation violations SVs/flight (hour) (Number of separation violations) / (Number of flights or flight hours)

Duration of separation violation situation is measured as a time period in which actual separation is lower than separation minima,
while severity presents a measure of how close the difference between actual separation and separation minima is to zero (Figure 3). Risk of
conflict represents a combination of duration and severity of separation violation. [5] The potential conflict starts at moment t1 and ending at
moment t2 (Figure 2). The change of aircraft separation is then given by the parabolic function u(t). This approach is applicable both in the
horizontal and in the vertical plane. [5]

Figure 2: The potential conflict duration and severity in the horizontal plane (one aircraft pair) [5]

3. RA modeling approach

RA is intended for simulation of air traffic consisting of optimal flights trajectories (output of Trajectory Planner (TP) and Traffic
and Capacity Planner component (TCP), Figure 1) through a given airspace sectorisation (output from Airspace Planner (ASP) component,
Figure 1) with aim to assess safety performances and to provide outputs in form of SPIs as well as safety feedback (which could be considered
by TCP and ASP components in case that proposed flight trajectories and sector boundaries are not suitable from the safety point of view) [3].
RA is a network based simulation model. [2]
RA model consists of three modules (Figure 3) [6]:
 Separation Violation Detection Module (SVDM);
 TCAS activation module (TAM); and
 Risk of conflict assessment module (RCAM).

Figure 3: RA model architecture [6]

The RA component is based on the assumption that conflict between pair of aircraft exists when either horizontal and/or vertical
separation minima are violated. The Separation violation detection module compares actual separation of aircraft (both in horizontal and
vertical plane) with given separation minima in order to detect potential conflict. Once conflict is detected this module counts them (SAF-4)
[3].
If the situation worsens the TCAS activation module is activated. It counts Traffic Alerts (TA, SAF-1) and Resolution
Advisories (RA, SAF-2) warnings and based on them number of NMACs (SAF-3) [3].
4. Fuzzy model

Instead of processing data with computer time consuming TCAS activation module and Risk of conflict assessment module in order
to calculate safety performance indicators, fuzzy model for approximation of safety indicators has been developed.

Every Fuzzy Logic System (FLS) consists of several elements (Figure 4):
 rules;
 fuzzifier;
 inference engine;
 defuzzifier. [7]

input variables output variables

Fuzzifier Defuzzifier

Rules Inference engine

Figure 4: FLS [7]

Input variables of every FLS can be numerical or linguistic. Fuzzyfier converts the crisp input to a linguistic variable using the
membership functions stored in the fuzzy knowledge base. Fuzzy rules are knowledge base containing a number of fuzzy IF–THEN rules.
Inference engine uses IF-THEN type fuzzy rules and converts the fuzzy input to the fuzzy output. Defuzzyfier converts the fuzzy output of the
inference engine to crisp using membership functions analogous to the ones used by the fuzzifier. [7] This FLS uses Mamdani inference
method introduced in 1975.
For development of our fuzzy model, MatLab software package was used. Fuzzy Logic Designer was used to generate membership
functions for both, input and output. FLS was stored as 'TCAS_Approximator.fis'.

Boundaries used for approximation of safety indicators are shown on Figure 5.

Figure 5: Boundaries for approximation of safety indicators

Thus, in our model we use two input variables. First one is Horizontal separation (HS) and second is Vertical separation (VS).
Values for HS and VS are gathered from Separation Violation Detection Module and they represent values at closest point of approach (CPA)
for each pair of airplanes with violated separation.

Figure 6. Membership functions for Horizontal separation and Vertical separation

Figure 7. Output of FLS

 a) b)

Figure 8. a) Rules of FLS; b) Rules surface view

On Figure 8.a) are presented rules of FLS used and 8.b) presents rules surface view. When horizontal and vertical separation are
close to zero value, total separation is close to zero. So, the closer output value is to zero, higher is the risk. In example, rule number 1 says that
if HS is a member of NMAC-h function and VS is a member of NMAC-v function then the output belongs to NMAC function and its value is
close to 0.

After generating membership functions, in the Command Window of MatLab, output is created within next steps:
 Step 1: Defining simple name to FLS (ie: t=readfis(' TCAS_Approximator.fis');
 Step 2: Defining name and data for scenario we want to evaluate (ie. filename105 = fullfile('FIS_S105.csv', where file 'FIS_S105.csv'
contains two columns with values for horizontal and vertical separation);
 Step 3: Reading input data (ie: S105Data= readtable(filename105,'Delimiter',';','ReadVariableNames',false));
 Step 4: Transforming table data to dataset (ie: ds105 = table2dataset(S105Data));
 Step 5: Generating results from input given (ie: res105=evalfis([ds105.Var1 ds105.Var2],t))

By this sequence of commands, results have been generated, for all pairs of airplanes that have violated separation, into one table.
This table contains 1 column with output values.

Those values were further processed for normalization. Four clusters were formed on the normalization interval [0,1] which related
output data and Safety Indicators:
 interval [0 - 0.001] - SAF3
 interval [0 - 0.125] - SAF2
 interval [0.125 - 0.555] - SAF1
 interval [0 - 1] - SAF4

5. Numerical tests and results

5.1. Simulation scenarios

In order to capture interdependencies among KPAs when assessing ATM performance, a wide set of simulation scenarios will be
studied in the APACHE Project. These scenarios have been identified and summarised in Table 2.

Table 2.
Part of preliminary set of scenarios used by FLS [3]

Trajectory planner Traffic and capacity planner Airspace Planner Uncertainties

Traffic demand: historical trajectory records

S0 Structured Route Current demand and capacity balancing Static sectors No

Flight Levels

Traffic demand: historical flight plans

S1 Structured Route Current demand and capacity balancing Static sectors No

Flight Levels
S2 Structured Route Current demand and capacity balancing Dynamic sectors No
Flight Levels
S3 Free Route Current demand and capacity balancing Static sectors No
Flight Levels

Scenarios are designed for three different dates representing three traffic demand. Date 01 is 2016/07/28 (summer, middle demand),
date 03 is 2017/02/20 (winter, low demand) and date 05 is 2023/07/21 (summer, high demand). Comparison of RA module and FLS is
conducted on 10 different inputs (ie. S001 means scenario S0 for the date 01, S205 means scenario S2 for the date 05). Each scenario is tested
on French airspace for one 24-hour period, which includes approximately 22.500 flights.
 5.2. SVDM Data

Table 3.
Input data for FLS - obtained from Separation Violation Detection Module

Scenario S001 S101 S103 S105 S201 S203 S205 S301 S303 S305
Separation
302 4726 7268 7268 2755 1973 3894 1752 1400 2582
Violations

5.3. FLS results

For the given input, values for safety indicators are presented in Table 4.

Table 4.
Results for FLS compared to RA module results

Scenario Results type NMAC (SAF-3) RA (SAF-2) TA (SAF-1)

FLS 26 34 94
S001
RA module 24 35 91
FLS 182 277 1084
S101
RA module 173 269 1023
FLS 328 466 1704
S103
RA module 322 455 1603
FLS 328 466 1704
S105
RA module 322 455 1603
FLS 27 40 487
S201
RA module 27 42 454
FLS 39 48 275
S203
RA module 39 48 262
FLS 73 104 631
S205
RA module 68 102 597
FLS 29 49 346
S301
RA module 29 47 317
FLS 20 31 275
S303
RA module 18 26 291
FLS 53 89 549
S305
RA module 47 87 510

Figure 9 illustrates results comparison of FLS and RA module for four test cases.

Scenario 001: total separation violation = 602 Scenario 101: total separation violation = 4726

TA TA

RA RA module RA RA module

FLS FLS
NMAC NMAC

0 50 100 0 500 1000 1500

Scenario 201: total separation violation = 2755 Scenario 301: total separation violation = 1752

TA TA

RA RA module RA RA module
FLS FLS
NMAC NMAC

0 200 400 600 0 100 200 300 400

Figure 9. Results comparison of FLS and RA module

6. Conclusion

It is obvious from output data that FLS gives results are close to RA module results. FLS is formed in a way that large amount of
input data can be processed in a short time (considering data preparation and processing). On the other hand, TCAS module needs much more
time. Preprocessing of data for TCAS is more complicated. In example, if TCAS is to be run on a particular pair of airplanes, first position (as
far as possible) back in time has to be noticed with constrain that heading and rate of descent/climb for both flights are not changed before
reaching CPA. Then, it is needed to manually enter parameters for both flights (position, horizontal and vertical speed, heading, ...) into TCAS
module and run it. After finished TCAS simulation, results are processed and belonging to Safety Indicator is defined. And this is to be done
for each pair of airplanes included into analysis.
Difference of around 5% navigates PA to use less time consuming FLS (instead of RA module) to obtain SAF parameters indicators
values good enough for evaluation of specific scenario.
On the other hand, FLS is based on Separation Violation Detection Module and cannot be used independently.
Nevertheless, this FLS is able to provide valuable results (verified by RA module results) and thus become fuzzy risk metrics for
evaluation of ATM scenarios.

References:

[1] APACHE (2017a). Deliverable 3.1: Review of current KPIs and proposal for new ones
[2] F. Netjasov, D. Crnogorac. (2018). Potential Safety Occurences as Indicators of Air Traffic Management Safety Performances - A
Network Based Simulation Model. In proceedings of the International Conference for Research in Air Transportation (ICRAT).
Barcelona, Spain
[3] APACHE (2017b). Deliverable 3.2: Functional requirements and specifications for the ATM performance assessment framework
[4] X. Prats, C. Barrado, A. Vidosavljevic, D. Delahaye, F. Netjasov and D.Crnogorac. 2017 (Nov). Assessing ATM Performance with
Simulation and Optimisation Tools: The APACHE Project. In proceedings of the 7th SESAR Innovation Days. Belgrade, Serbia
[5] F. Netjasov (2012). Framework for airspace planning and design based on conflict risk assessment, Part 2: Conflict risk assessment
model for airspace tactical planning. Transportation Research Part C, 24, 213–226.
[6] F. Netjasov, D. Crnogorac. (2017). Assessment of safety performance indicators of future air traffic management system. In
proceedings of the XLIV Symposium on operational research (SYM-OP-IS). Zlatibor, Serbia.
[7] Dušan Teodorović, Milica Šelmić (2012). Računarska inteligencija u saobraćaju, Beograd: Saobraćajni fakultet.