Sensor Performance Evaluation

Site Location

Edition: 1
Date: 16-Dec-09
Status: Final

IE ASIA-PACIFIC Pty Ltd

Canberra Technology Park
Suite A101/49 Phillip Avenue
Watson ACT Australia 2602
Tel +61 2 6241 9237
Fax +61 2 6108 3556
support@ie-asia-pacific.com

ACN 139 467 744

ABN 21 139 467 744
Sensor Performance Evaluation Edition Date: 16-Dec-09

DOCUMENT CHARACTERISTICS

General

Sensor Performance Evaluation

Site Location
Edition: 1
Edition Date: 16-December-2009
Status: Final
Keywords: Level 1 and Level 2 Radar Sensor Performance Evaluation
Abstract: This document describes a baseline radar sensor performance
evaluation

Contact Information

Author: Len Harrison

Editor: Darren Gillam
Contact Person: Darren Gillam
Tel: +61 437 377 474
E-mail Address: darren.gillam@ie-asia-pacific.com

Document Control Information

Document Name: PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt

Path: C:\Documents and Settings\elke\Desktop\Website\Services - OK\
Host System: MAC OS X 10.6.1
Software: OpenOffice.org 3.1.1
Size: Bytes

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 2/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

DOCUMENT CHANGE RECORD

Pages Approved
Revision Date Reasons for change
Affected by
001 23/10/09 New Document All

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 3/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

TABLE OF CONTENTS

1. INTRODUCTION.............................................................................................................7
1.1. Background.........................................................................................................................................7
1.1.1. RASS Level 1 (RASS-R Data Analysis)........................................................................................7
1.1.2. RASS Level 2 (RASS-S Sensor Baseline Evaluation)..................................................................7
1.1.3. RASS Level 3 (Special Evaluation)...............................................................................................8
1.2. Purpose of This Document................................................................................................................8

1.3. Out of Scope.......................................................................................................................................8

2. MEASUREMENTS AND ANALYSIS.......................................................................................9

2.1. RASS Level 1 Data Analysis..............................................................................................................9

2.2. RASS Level 2 Sensor Baseline Evaluation (on-site)......................................................................10

3. REPORT CONTENT.....................................................................................................13

4. DELIVERABLES..........................................................................................................13

5. TIMEFRAME..............................................................................................................14

6. RESOURCES.............................................................................................................14

7. ORDERING AN EVALUATION.........................................................................................14

8. ADDITIONAL SCOPE....................................................................................................14

9. ACCESS TO SITES......................................................................................................14

10. INTELLECTUAL PROPERTY..........................................................................................14

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 4/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

GLOSSARY OF TERMS
ANSP Air Navigation Service Provider
FAA Federal Aviation Authority
FAT Factory Acceptance Test
ICAO International Civil Aviation Organisation
IE Intersoft Electronics NV
IE-AP IE Asia Pacific Pty Ltd
NATO North Atlantic Treaty Organisation
PDD Product Description Document
Product IE defines a RASS Product as an output from a Project
RASS Radar Analysis and Support System
RASS Engineer Suitably qualified Engineer or Technician trained and experienced
in the use of RASS
RES Radar Environment Simulator
RF Radio Frequency
RTG Radar Target Generator
SAT Site Acceptance Test
USAF United States Air Force

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 5/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

REFERENCED DOCUMENTS
SUR.ET1.ST03.1000.STD-01-01 Eurocontrol – Radar Sensor Performance Analysis – June 1997
ICAO Doc 8071, First Edition - Testing of Surveillance Radar Systems
1998

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 6/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

1. INTRODUCTION
Comprehensive and continuous radar coverage of high quality and reliability is essential for the uninterrupted
provision of radar services and the application of specific radar separation standards1.

The Radar Analysis and Support System (RASS) is a suite of state-of-the-art analysis tools that assists the
Air Navigation Service Provider (ANSP), Defence Agency or Maintenance Repair Organisation to determine
the performance state of a surveillance system against manufacturer’s specifications, contracted
performance requirements or international regulatory requirements for safe air traffic management (e.g.
ICAO). RASS is a recognised evaluation tool1 used to analyse radar performance and is currently in use
worldwide by organisations to verify radar performance against mandated standards (i.e. Eurocontrol, NATO,
USAF and FAA).

Intersoft Electronics (IE) has extensive experience in the provision of RASS related services worldwide. IE
field resources are highly trained and experienced in the operation of RASS. IE field resources have
significant knowledge and experience with the use of RASS on radar systems produced by many different
manufacturers.

RASS can perform quantitative testing of RF signals in space to verify certain specifications that cannot be
performed by the use of conventional test equipment. In addition, RASS can expedite acceptance testing by
reducing SATs and Flight Tests by 40-50% in duration and produce significant savings in aircraft operating
costs. RASS will not totally eliminate the need for aircraft flight testing, but can reduce the number of hours
by a proven factor of 2.

1.1. Background
The performance evaluation of a radar sensor is structured to provide investigation and analysis to three
distinct levels dependent on client requirements.

1.1.1. RASS Level 1 (RASS-R Data Analysis)

A RASS Level 1 evaluation provides a near real time monitoring of key radar quality parameters on a 24/7
basis. Key performance parameters are monitored through trend analysis to ensure correct operation of a
sensor. By monitoring performance trends it is possible to provide early detection of system degradation
before system failure occurs. These analyses are completed through the use of the RASS-R family of COTS
products.

Real time monitoring is commenced on a radar sensor only after the sensor's baseline performance has
been established.

1.1.2. RASS Level 2 (RASS-S Sensor Baseline Evaluation)

A RASS Level 2 evaluation provides a detailed technical analysis of the performance of the individual sub-
systems that comprise the radar system. This analysis will result in definition of the sensors coverage
capabilities and operational performance capabilities and/or limitations against established standards or
requirements. The level 2 evaluation is conducted using the RASS-S family of COTS hardware and software
components.

A baseline evaluation can identify the reasons for possible performance degradation of a radar sensor. The
baseline evaluation also provides the ability to test the efficacy of the radar sensor's optimisation.

1 Sur.ET1.ST03.1000-STD-01-01, Radar Sensor Performance Analysis, Edition 0.1, June 1997

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 7/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

1.1.3. RASS Level 3 (Special Evaluation)

A RASS Level 3 evaluation can be a single or multiple series of special tests designed to assess a radar
system at a deeper level than the tests conducted in a sensor baseline evaluation. These tests are typically
conducted during Factory Acceptance Testing (FAT) or Site Acceptance Testing (SAT). A special evaluation
can also be used to assess the readiness of a radar sensor for flight inspection by injecting MSSR or PSR
targets and comparing the results while the radar is working in its full operational state. Level 3 evaluations
are made using the RASS-S family of equipment as well as specialist COTS equipment such as the Radar
Environment Simulator (RES) and Radar Target Generator (RTG).

1.2. Purpose of This Document

From time to time a client may seek the unique and niche services that IE-AP has to offer in terms of
delivering one or more levels of RASS performance evaluations at any of their radar locations.

This document describes the definition and composition of a RASS-S Level 1 and Level 2 evaluation, which
will provide an analysis of a radar sensor’s performance against established standards and specifications.
This evaluation provides an invaluable insight into the operational state and health of the radar sensor.

Specific measurements have been selected in accordance with a Statement of Work, Contract or other
relevant forms of correspondence is order to ensure the correct output is achieved for the client.

1.3. Out of Scope

This document does not describe measurements to reduce any specified SAT testing. Should a client
request the addition of SAT measurements to take advantage of additional time savings, IE-AP will review
the requirements and re-scope accordingly.

The scope described within this document does not represent the limit of the capability of the RASS suite of
test tools.

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 8/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

2. MEASUREMENTS AND ANALYSIS

2.1. RASS Level 1 Data Analysis
Large amounts of Data (Plot & Track), Video and radar timing signals/triggers are recorded simultaneously
on-site for further analysis. The data and video is processed to analyse the performance of the Interrogators,
Plot Extractors and Tracker systems, and to compare against the results extracted from the video output of
the interrogator under test. The test will result in Data (Plot & Track) output from the Interrogator and Tracker
System as well as the Video over the full coverage of the radar being captured, recorded and displayed. This
is analysed to identify any anomalies in system performance or reconstruct ADS-B trajectories independent
of radar performance. Table 2 identifies the tests which will be performed on-site as part of the RASS Level 2
Baseline Evaluation of the sensor.

Specifications for pass/fail criteria (i.e. Eurocontrol, Contract, etc) of the system are to be provided by the
client in advance of the measurements being undertaken.

Table 1: Level 1 Analysis Description

Ref # Description YES/NO/NA

PSR

L1-001 Pd Analysis
The Probability of Detection (Pd) will be calculated for the coverage
volume of the PSR radar according to the specifications in order to
characterise the ability of the radar to detect targets.

L1-002 Positional Accuracy Analysis

The position of the reported targets will be checked for it accuracy in both
range and azimuth within the defined coverage volume of the PSR radar
according to the specifications. The reported targets’ timestamps will also
be checked for delays. This check is done in order to characterise the
ability of the radar to report an accurate position, both in space and time,
of the targets detected.

L1-003 PSR Random Errors Analysis

The radar under test data will be examined for random, non-systematic
errors, both in range and azimuth. This analysis will reveal problems that
may compromise the integrity of the radar data.

L1-004 False Target Analysis

The data from the radar will be examined for false target reports
according to the specification. These targets could indicate multi path,
reflection or antenna issues.

MSSR

L1-005 Pd Analysis
The Probability of Detection will be calculated for the coverage volume of
the MSSR radar according to the specifications in order to characterise
the ability of the radar to detect targets.

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 9/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

L1-006 Positional Accuracy Analysis

The position of the reported targets will be checked for it accuracy in both
range and azimuth within the defined coverage volume of the MSSR
radar according to the specifications. The time-stamp of the targets will
also be checked for delays. This check is done in order to evaluate the
ability of the radar to report an accurate target position in both space and
time.

L1-007 Data Validity Analysis

The data from the radar under test will be examined and assessed for
invalid aircraft responses against the specifications. This analysis will
include invalid Mode A and C responses and replies to side lobes or
reflected signals.

L1-008 MSSR Random Error Analysis

The radar data will be examined for random, non-systematic errors, in
both range and azimuth. This analysis will reveal problems that may
compromise the integrity of the radar data.

L1-009 False Target Analysis

The data from the radar will be examined for false target reports
according to the specifications. These targets could indicate multi path,
reflection or antenna issues.

Combined Radar

L1-010 False Association Analysis

The False Association Ratio and Combination Rate of PSR and SSR
targets will be examined.

2.2. RASS Level 2 Sensor Baseline Evaluation (on-site)

Table 2 identifies the tests which will be performed on-site as part of the RASS Level 2 Baseline Evaluation
of the sensor.

Table 2: Level 2 Analysis Description

Ref # Description YES/NO/NA

PSR

L2-001 Receiver Sensitivity

This recording is conducted to evaluate the performance of each PSR
receiver (up to 2 receivers). Measurements will be conducted on 2 spot
Frequencies as detailed by the client. This test will indicate if the receiver
is functioning within the required parameters. The calibration curve is
further used by other RASS processes to analyse system performance.

L2-002 Receiver Bandwidth

This recording is conducted to evaluate the performance of each PSR
receiver (up to 2 receivers) across its operating bandwidth.
Measurements will be conducted on 2 spot Frequencies as detailed by

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 10/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

the client. The test will indicate if the receiver is functioning to

specification.

L2-003 Sectorial STC

This recording is performed to evaluate the STC performance of each
PSR receiver (up to 2 receivers) within the operational STC parameters.
STC performance is measured with the radar operating as specified in
the site specific configuration documentation. This test will verify that the
applied STC maps are relevant for operating environment.

L2-004 Windowed Video

This recording is performed to evaluate the clutter levels as detected by
the PSR receivers. The Widowed Video Recording is used to visualise
received PSR video in great detail, to observe abnormal or unwanted
performance issues such as reflections, noise and clutter. The windowed
video recordings are analysed at a later time to identify anomalies in
system performance.

L2-005 Solar – Vertical Polar

This recording is performed to determine the relative shape, gain and
peak elevation angle of the low/main beam and high/auxiliary beam
vertical radiation patterns of the PSR antenna system. This test will
record the level of solar noise over a 2-3 hour period, either immediately
leading up to sunset or immediately following sunrise, as the sun transits
through the vertical beam pattern of the antenna.

L2-006 Uplink
The uplink recording is performed to determine the shape of the
horizontal pattern radiated from the PSR antenna. Recordings are made
from up to three separate geographic locations to assess the effects of
the RF environment on the radiated pattern and the performance of the
radar antenna itself.

L2-007 PSR Data Recording

Large amounts of Data (Plot & Track) will be recorded from the output of
the tracker systems. The data can be displayed in a inventory tool to
examine many different parameters contained in the data streams such
as probability of detection and tracking system performance.

MSSR

L2-008 Receiver Sensitivity

This recording is performed to evaluate the performance of each
Interrogator’s receiver (Channel A and B). This Test will indicate if the
receiver is functioning within required parameters. The calibration curve
is further used by other RASS processes to analyse system performance.
Receiver sensitivity calibration curves are taken on Sum, Delta and
Omega channels.

L2-009 Receiver Bandwidth

This recording is performed to evaluate the performance of each
Interrogator’s receiver (Channel A and B) across its operating bandwidth.
The test will indicate if the receiver is functioning to specification.
Receiver Bandwidth will be measured on the Sum, Delta and Omega
channels of each interrogator.

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 11/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

L2-010 Windowed Video

This recording is conducted to evaluate the performance of the radar
interrogator, remote field monitor (RFM or PARROT) and RF
environment. Careful analysis of the windowed video recording can help
determine the monopulse performance of the MSSR LVA and
SUM/DELTA receiver channels. The Widowed Video Recording is used to
visualise received transponder video in great detail, to observe abnormal
or unwanted performance parameters such as reflections, noise or
FRUIT.

L2-011 Pulse Video, including Vertical Polar

This recording will collect data from a pulse video recording to generate a
vertical polar diagram of the MSSR LVA. The Video Pulse Recording
collects a large amount of received video pulses from opportunity traffic
as well as interrogation triggers and azimuth timing signals. The results
are extracted to show these events in great detail, to observe any
abnormal or unwanted performance parameters such as reflections,
noise and FRUIT. The results are compared against the data output of
the interrogator under test. The comparison will evaluate the performance
of the extractor and radar tracking sub-systems.

L2-012 Downlink
This recording is performed to evaluate the performance of the MSSR
antenna as recorded at the radar site. This test provides an indication of
the SUM, DELTA and OMEGA channel performance. Measurement
results will provide a horizontal pattern for all three channels of the LVA.
The symmetry of the SUM/DELTA and SUM/OMEGA crossovers is an
important factor for correct monopulse operation.

L2-013 Mode A/C Decode Functional Test

This test is performed to evaluate the radar’s ability to decode and
display the emergency codes and altitude sent from a transponder. This
test is performed by firstly injecting emergency A codes 7700, 7600 and
7500 with the RFA set up as a Remote Field Monitor, recording the
decoded signals with RASS and visually confirming the correct code
responses on the operator display; and then changing the C code
50200ft, 83800ft and 99900ft to confirm the correct slant range correction
is being applied.

L2-014 Uplink
This recording is conducted to evaluate the performance of the MSSR
antenna as recorded in the field. This test will determine if the
interrogation pattern being transmitted from the MSSR antenna exhibits
normal performance and/or is affected by influences external to the radar
system. This measurement results in a horizontal polar diagram of the
transmitted interrogation pattern. By performing this measurement from
three separate geographic locations, it can be determined whether
anomalies in the interrogation pattern are contributed to by problems in
the antenna and/or the environment.

L2-015 MSSR Data Recording

Large amounts of data (Plot & Track) will be recorded from the output of
the Interrogator and tracker systems. The data can be displayed in a
inventory tool to examine many different parameters contained in the

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 12/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

data streams. Examination of the opportunity traffic can reveal details

about probability of detection, interrogator performance and tracking
system performance.

Combined Radar

L2-016 Radar Processor Data Recording

Recordings of large amounts of operational data (Plots & Tracks) will be
made at the radar site. These recordings will be made on SSR, PSR and
Combined outputs of the radar where available. The data can be
displayed in a inventory tool to examine many different parameters
contained in the data streams.

L2-017 Multi-Level Data Analysis

Data from the operational output of the sensor is recorded and used for
later analysis to evaluate the overall performance of the radar sensor
system. Aircraft tracks can be filtered to separate SSR, PSR and
combined tracks. These tracks can then be overlaid to give a visual
indication of performance of the radar sensor. This task will require
accurate survey coordinates of the radar site. (Surveying of coordinates
is out of scope for a baseline evaluation).

L2-018 Antenna Gyro/Inclinometer

This recording is performed to evaluate the mechanical performance of
the antenna turning equipment, the azimuth encoders and the inclination
of the rotating platform in the horizontal plane. This test will determine the
accuracy of the Antenna Reference Pulses (ARPs) and Azimuth Change
Pulses (ACPs) being generated by the azimuth encoder/s by comparing
the recorded azimuth information against actual antenna movement
recorded by an independent three axis gyroscope that has been
temporarily mounted on the rotating platform.

3. REPORT CONTENT
At the conclusion of the measurement phase of the Project, IE-AP will undertake analysis of the above
measurements to determine the overall performance of the radar under test.

A detailed report will be delivered to the client. The report will include an Executive Summary on the radar’s
performance, detailed analysis on the measurements taken and a Recommendations section for the client’s
consideration in restoring the radar to its previous baseline condition (if applicable).

4. DELIVERABLES
The following deliverable will be provided to the client at the conclusion of the evaluation;

• Test Schedule as agreed by all parties,

• PDF copy of report by e-mail,

• DVD disk/s containing:

◦ PDF copy of report

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 13/14

Sensor Performance Evaluation Edition Date: 16-Dec-09

◦ Complete evaluation directory, including data collected, photographs taken and documents
generated.

5. TIMEFRAME
Unless otherwise stated or agreed at the time of quote approval, all Project Deliverables will be dispatched to
the client no later than two (2) weeks (10 working days) following the completion of the on-site work.

For the avoidance of doubt, completion of the on-site work will be the final site completed if consecutive site
measurements are undertaken.

6. RESOURCES
Two RASS Engineers are required for any on-site measurement evaluations.

7. ORDERING AN EVALUATION
To order an evaluation, the client is requested to contact IE Asia Pacific Pty Ltd via email at support@ie-asia-
pacific.com or fax on +61 2 6108 3556.

8. ADDITIONAL SCOPE
A RASS evaluation may result in the need to take additional measurements and/or undertake further
analysis. IE-AP will quote on any scope outside of the baseline product on the basis of a Survey and Quote.

9. ACCESS TO SITES
Once the quote is approved, the client will liaise with the local maintenance manager of the radar to be
tested. It is the clients responsibility to ensure that all site access requirements are coordinated and
managed.

10. INTELLECTUAL PROPERTY

All data, photographs and reports collected, analysed and generated as a result of a RASS evaluation will
remain the property of the client. IE-AP may keep copies of this property for internal use; however, it shall not
be divulged to a third party without the written consent of the client.

PDD-IEAP-L1&L2 Sensor Performance Evaluation V1.0-161209.odt 14/14