21, rue d’Artois, F-75008 PARIS B5-209 CIGRE 2012

http : //www.cigre.org

Digital Fault Recordings: practices and perspectives for automated data

retrieval, information management and analysis in a large Brazilian utility

M.A.M. Rodrigues (*) M.A.F. Ramos

D.A.M. Villela R. Farizele
J.E.Silva Fo.
ELETROBRAS - CEPEL ELETROBRAS - FURNAS
BRASIL

SUMMARY

Digital fault recording (DFR) has been established as an essential post-event analysis source of
information, particularly after new regulatory requirements imposed strict economic and legal issues
regarding system availability. In Furnas Centrais Elétricas, one of the largest generation and
transmission utilities in Brazil, this topic has always been taken very seriously: since the first
(analogue) fault recording devices have been installed, all results have been stored in a very orderly
way, so that they could be retrieved at any time. With the introduction of digital fault recorders, the
company developed a number of new approaches for retrieval and storage of files from its numerous
recorders. Automated fault analysis applications help at a great extent specialized personnel
responsible for detailed fault analysis in gathering information relevant for the complete post-event
understanding of the disturbances. With the progressive introduction of the new multi-function
Integrated Electronic Devices (IEDs), IEC 61850 standard based, the management of oscillographic
data needs to be revisited. It is no longer be reasonable to separate distinct data sources such as
SCADA remote terminal units, protection relays and fault recorders in distinct IEDs. SCADA/EMS
systems are expected to act as natural gateways also for data not directly related to real-time operation
functions, mainly because of the already well established and reliable communication infrastructure.
Another important matter of concern for the mentioned review is the recent deployment of Phasor
Measurement Units (PMUs) and the potential use of this new data for fault analysis. The idea of
extending the fault investigation tools through phasor data becomes possible with the increasing
number of devices and software prepared to deal with this information. The paper will present the
current status of DFR use in FURNAS transmission network, including the associated system
architecture, devices, software and statistics emerging from daily activities. Then, the evolutional
perspectives of these processes will be discussed, in view of the recent technological and technical
tendencies and developments in the area.

KEYWORDS

Transmission – Fault – Recorder – Brazil – IEC 61850 - Phasor

(*) mamr@cepel.br
1. DIGITAL FAULT RECORDING ROLE IN POWER SYSTEM UTILITIES

Digital fault recordings have been used in many power systems operation and planning activities.
They consist of oscillographic recordings, stored in digital form, of the power system quantities,
mainly phase voltages and currents, i.e., quantities values registered against time. These values can
represent the original sinusoidal and transitory waveforms or the phasor magnitude and angle
calculations obtained using a sliding time window. The former will be referred herein as long-term
oscillography, while the first as short-term oscillography.

Digital fault recordings find use in many applications, such as:

 Fault understanding.
 Protection systems performance evaluation.
 Equipment maintenance and statistical data gathering.
 Operation planning (data source for playback type simulations).

For protection systems in transmission lines, fault recordings can be useful in providing the real fault
behavior of the line impedance, serving as input to digital and real-time simulations and also
indicating the need for protection devices and settings maintenance.

For transmission line maintenance, fault recordings play a key role as a source of data for fault
location calculation. Combined with fast data transmission available today and automated analysis,
regional operation centers can be updated in few minutes in order to take timely actions, like
dispatching and orienting maintenance teams in the field. Figure 2 shows an example of such use.

Post-fault analysis can bring understanding of the fault and of the equipments behavior and damage.
Fault characterization includes determining faulty phases, fault duration, protection operation and
system state after the event (line openings, reclosure, breaker response times etc.). It is also possible to
reveal malfunctions in the measuring chain (e.g. current and voltage transformers).

Most of these tasks can be performed manually by a skilled engineer, but as the number of events and
recordings increase, such work becomes prohibitive. For this reason, automated fault analysis is
receiving increasing attention. It enables not only automating much of the previously mentioned tasks,
but also to enlarge the scope of the analysis, by using data from other sources, like SCADA systems,
power quality and phasor measurements. Also, static data such as power system topology, protection
schemes and protection settings can be used to improve the usefulness of the analysis results. Cigré
WG B5.20 (“New Trends for Automated Fault & Disturbance Recording & Analysis”) has produced
an excellent report about this topic [1].

Automated fault analysis systems found their way into utilities mainly due to regulatory enforcement,
which determines issues like availability of the transmission assets up to number of points needed to
be monitored with fault recorders. These rules imposed a fast response to faults in an ever growing
complex power system, resulting, equivalently, in a huge amount of data. It must be remembered that
files from fault recordings are generated not only for fault events in a given power system equipment,
but also for other reasons such as voltage and current variations due to a fault in adjacent circuits or
due to line and equipment switching, in the case of lightning strokes and, sometimes, for testing the
overall availability of the acquisition chain and communication network (in this case, commands are
sent to trigger DFRs periodically).

2. FURNAS FAULT RECORDING SYSTEM

Furnas Centrais Elétricas is one of the largest generation and transmission utilities in Brazil (over 11
GW generating capacity, transmission network including 600 kV HVDC and 765 kV AC). Due to its

1
huge size, the number of fault recording devices is quite large and the company has developed a
number of approaches for retrieval and storage of fault recording files, Table I.

Table I: Furnas assets, values from 2011.

Assets Values
Substations 51
Transformers 659
Transforming installed capacity 103,304 MVA
Switch breakers 1044
Static and synchronous compensators 13
Transmission lines extension 19,398 km
Regional operation centers 4
Hydroelectric power plants 15
Thermoelectric power plants 2
Installed power generation capacity 11,200 MW
Yearly power generation 35,362 GWh
Dedicated DFR devices in transmission at 345 kV and over 82
Dedicated DFR devices in transmission below 345 kV 46
Dedicated DFR devices in other uses 68
Total number of dedicated DFR devices in FURNAS 196
Relays with DFR function in RARP 390
Average number of digital fault recordings per month 9700

Furnas organizes disturbance files in two sets: files generated by dedicated DFR devices and by relays
with DFR function. This separation is still justifiable, nowadays, due to technical aspects as:
 Flexibility in adjusting DFR trigger independently of relay setting: the number of digital
relays from past generations that have no such flexibility justifies the use of dedicated devices to
capture events that did not lead to a relay trip.
 Extended capability of dedicated DFR devices: in many cases, dedicated devices will enable
recordings with extra time length or higher sample rates, making the recording of situations like
breaker reclosure and disturbances in the presence of higher harmonics possible.
 Different ways to collect and deliver DFR files: the connection to relays requires, in general,
proprietary software while a more common and suitable for automation approach is usually
available for DFR dedicated devices, as shown in the following.
 Faster access to DFR data: as DFR dedicated devices are not involved in control, security
policies associated to remote access are less restrictive than those of protection relays.

2.1 FURNAS Digital Fault Recording System for dedicated devices

Furnas has adopted a hierarchical architecture for files from dedicated DFR devices. It consists of data
concentrators at substations and a centralized repository in the central office. The system, developed
from 1997 to 2002 and still used until today, is called REDEOSC, an acronym for fault recording
network. Its most important part is the data concentrators whose main features are:
 They are industrial type computers with Intel architecture running Windows operational system.
Most machines have a watchdog circuit to ensure continuous operations.
 Proprietary software from manufactures to fetch files from recorders. In the fault recorder
specification it is required that the software can be executed in a batch routine to update the
concentrator. The proprietary software is expected to store the fault recording in the computer
using the COMTRADE [2] format.
 Corporative software to organize the files in a remote repository and transmit them to a central
repository.
 TCP/IP Ethernet network implemented over a number of physical layers (microwave radio,
dedicated wire links, optical fibers) to transmit the files to the central repository.
2
At the concentrator, proprietary automated fault analysis software was used to identify line switching
and fault location. It was replaced in 2011 by SINAPE.NET, as explained later.

2.2 FURNAS Digital Fault Recording System for multifunction IEDs

For fault recordings from relays, FURNAS developed a system called RARP (acronym for Protection
Relays Access Network, in Portuguese) [3] [4]. RARP aims at establishing standardized methods for
communication with protection relays and IEDs produced by distinct manufacturers, providing a
means for executing routine tasks without compromising security, flexibility, reliability and
availability issues. These routine tasks are, for example, automated periodical retrieving of fault
recordings and IED parameter setting. Development of RARP focuses on solving problems such as:
 It is inappropriate for maintenance teams to use laptops on site to directly access relays and IEDs
for a number of reasons (impossible to access DRF files with a small periodicity; reduces system
reliability; cables are not standardized; devices use distinct proprietary software).
 By accessing protection relays using manufacturer provided software, one can inadvertently
change critical relay settings, leading to undesired protection behavior. For this reason, access to
relays in this way needs special permissions from system operation area and can not be
implemented in a regular base.
 Periodical collecting of fault recordings is a difficult task which needs either to involve
maintenance personnel acting on site or a remote connection. The second option requires many
manual operations being very time consuming and error prone.

RARP deals with all these questions in many steps. The first one is done by including a number of
minimal requirements in the IEDs specification during the acquisition process, such as: a
communication interface that allows for retrieving fault recordings automatically and saving them with
an unique name in COMTRADE format [8]; and a configuration interface to change protection relays
settings. One condition is that each software must not lock any resource permanently (ports, keyboard,
screen) as application for many IEDs will be running concurrently

The next step is the system structure. RARP is constructed using an industrial type computer, called
“Concentrator Microcomputer” (CM). It is installed on the substation relay panels, being directly
connected to IEDs and digital relays, using physical layer protocols such as EIA-232/485 serial
protocol and ethernet, being the first one the most usual option due to the number of legacy devices.
However, FURNAS is currently revising its specifications in order to use the IEC 61850 substation
LAN as the common physical media for disturbance file transfers.. The system is depicted in Figure 1.

FURNAS has two segregated networks: operational (or real-time) network and intranet. Access to the
first one involves some cyber-security policies that deserve to be highlighted:
 Connection to relays are granted only locally (physical connection to IED ports) or logging to a
CM using remote terminal emulators. Direct connection from a computer in the intranet to the IED
is not allowed.
 A firewall and authentication servers implement a Virtual Private Network (VPN) to grant access
only to authorized users coming from the companies’ intranet
 Operational (real-time) network uses Virtual Local Area Networks (VLAN) to optimize
performance for critical communication tasks.
 The CM has an integrated security system that blocks unauthorized access to critical relay
functions.
 Read-only access to fault recording files in the CM is granted using file transfer protocol (FTP).

The third step involves configuration and setup of all the software from manufacturers and third
parties in the CM, together with development of dedicated software. This software consists of:
 Manufacturer interfaces for file retrieving and relay parameterization: software for all the relays in
the substation need to be installed and commissioned to work concurrently.

3
 Third party software: specialized functions such as FTP to send DRF files for further, centralized
processing.
 Dedicated software: used to fill in same needs such as renaming according to COMNAME
recommendation [8], calculate an MD5 (Message-Digest algorithm 5) id to authenticate the file,
fixing known errors in files (date-time incorrect format, wrong value of total number of channels,
wrong COMTRADE year of revision, wrong channel characteristics etc.) and organizing the files
into a directory structure [4].

Figure 1: RARP system diagram.

2.3 Automated Fault Analysis System

Fault recordings are periodically sent to a web server in the company’ intranet for automated fault
analysis, which, at Furnas, is performed by two distinct systems: for files coming from DFR dedicated
devices a home-made analysis system, developed from 1997 to 2002, was adopted until 2011. For files
coming form RARP, SINAPE.NET, developed by CEPEL (Electric Energy Research Center, at Rio de
Janeiro, Brazil) was used. As both systems meet Furnas basic requirements and as the last one is
evolving with periodically new developed features, FURNAS has moved towards adopting
SINAPE.NET for both file sets.

SINAPE.NET has a large number of features and some of them are shown below [5][6]:
 Managing DFR files (registering the file status, backup, etc).
 Automated fault analysis.
 File and event filtering according to date, substation, power system equipment and fault type.
 Automatic fault location using one or two line terminals
 Keeps and allows for editing a power system topology information database, including the
configuration of all fault recorders.
 Displaying, in WEB pages, preliminary diagnoses about the recorded event.

4
 Displaying, in WEB pages, information and statistics from the DFR system.

Figure 2: SINAPE.NET windows: synoptic fault report (left) and fault location (right).

3. INTEGRATION TO NEW PROTECTION AND AUTOMATION TECHNOLOGIES

The above mentioned infrastructure is performing well, but, with the progressive introduction of new
multi-function IEDs, of the IEC 61850 standard and of high bandwidth computer networks, the
acquisition and management of oscillographic data needs to be revisited.

It will no longer be possible to separate distinct functions such as remote terminal units, protection
relays, fault recorders and PMU in distinct IEDs. In fact, the design strategy for automation systems
shall, in the near future, converge to functions integration and to minimization of the total number of
IEDs in a bay, according to constraints such as performance and redundancy [7]. The IEC 61850 will,
then, act as an enabling technology.

Consider the IEDs in Figure 1, which are expected to implement real-time protection and automation
functions. Two distinct teams in the company access the same IED using different approaches: one
team, devoted to protection and DFR devices maintenance, accesses the IED using the CM; and the
other team, devoted to real-time system operation devices, accesses the IED using a SCADA system.
The network security scheme can be enhanced if the IEDs are to be accessed only from the latter.

SCADA systems are expected to act as natural gateways also for data not directly related to real-time
operation functions, mainly because of the already well established, fast and reliable communication
infrastructure in most installations. CEPEL also develops SAGE (Energy Management Open System),
a SCADA/EMS system, which holds a large share of the Brazilian SCADA market, with over 400
installations covering most of the HV Brazilian network. Among other features, SAGE supports the
IEC 61850 standard for substation automation, including file transfers, which can be used to securely
retrieve DFR files from IEDs.

By using this feature, files coming from IEC 61850 capable IEDs can be stored in a directory in
SAGE’s computer. The CM approach will still be necessary while legacy devices are in use. The
advantages of using SAGE for this application are many: SAGE will have an economical impact
acting as an enabler for the introduction of IEC 61850 capable devices; a more secure management
strategy, performed by a unique team will be possible; a highly tested and robust software and
hardware platform will be made available; an easier implementation process (no additional cables,
ports or protocols, just a software configuration).

In order to implement the function of DFR retrieving, the SCADA system has to implement or to
allow for the implementation of some special features:
 Unique file name: it may happen that distinct IEDs in the same network may give similar names
to DRF files. It is more prone to happen with similar devices of the same manufacturer. The

5
 SCADA will have to care in order to not overwrite older files.
 Source IED identification: the SCADA system will have to add the identification of the IED that
produced the file in a way that it will be accessible after the file is transferred out of the system.
This can be easily done by renaming the files with a prefix identifying the IED (in the same
manner COMNAME [8] does) or by storing the files in a directory structure with IED names in
subdirectories.
 Consider COMTRADE file structure: COMTRADE version 1999 defined 4 distinct files
related to an event (with extensions “CFG”, “DAT”, “HDR” and “INF”). The new version of this
standard, to be published, is expected to consider additional file extensions. The SCADA
implementation must guarantee that all these files are retrieved.
 Determine a way to transfer DFR files to a computer not in the operational network: this can
be easily and securely accomplished using synchronization software for file mirroring or the FTP
protocol in a read-only access mode.

There are two potential issues to be addressed about this idea. First, as real-time operation and fault
recording analysis tasks are performed by distinct personnel, an agreement between both teams will
have to be achieved in order to implement DFR file delivering as a SCADA system added service.
However, this issue is not new and modern automation systems architecture indicates that such
integration is more beneficial than the traditional approach.

A second issue could be the effect of augmented time latency on priority messages, like GOOSE,
caused by the traffic of large DFR files. However, as the Ethernet is a packet based communications
technology, i.e., all data to be transmitted is segmented into packets, the problem is the same for any
data set whose size is larger than the maximum packet size, which is 1500 bytes. In addition, modern
switches implement a priority scheme, meaning that the latency for a GOOSE message will depend on
the maximum Ethernet packet size, not on the original file size. Furthermore, switches can implement
other schemes to improve the traffic of critical packets, such as tagging of priority messages, what is
the case with GOOSE messages, and traffic segregation in the switch (Virtual LANs, or
simplyVLANs) to avoid flooding of messages to devices unrelated to specific functions. Although a
precise calculation depends on many factors, this latency can be shown to be within the 4 ms range for
Ethernet bit rates including 100 Mbps and above [9].

Concerning the latency time for the transmission of DFR files from IEDs to the SCADA computer, we
consider, for instance, the example of Adrianópolis substation, a typical large sized substation of
Furnas located near Rio de Janeiro city. It has about twelve 138 kV bays, eleven 345 bays and seven
500 kV bays. Considering one DFR per bay, there should be, during a large disturbance where most
fault recordings are triggered, an amount of 30 oscillograms. If we estimate the size of the files related
to an oscillogram to be 5 Mbytes, a quite exaggerated figure, a total of 150 Mbytes is generated. This
amount of data can be transferred to the SCADA computer and simultaneously to a server in the
intranet in few minutes. This time frame is sufficiently short to meet any post-event analysis needs,
including fault location in transmission lines, used to help the work of maintenance teams. In fact, this
function is used today in many substations where communication links are somewhat more limited.

4. USE OF PMU AND PDC DATA FOR FAULT ANALYSIS

The Brazilian electric power industry is making an effort, lead by independent system operator ONS,
to implement a large-scale Wide-Area Measurement System (WAMS) utilizing the synchronized
phasor measurement technology [10]. This project is being conduced taking into account a number of
important and carefully chosen requirements aiming to attend all operation challenges of the National
Interconnected Power System (NIPS).

Since mid 2000s, FURNAS realized that it could implement a low cost experimental Synchronized
Phasor Measurement System (SPMS), so that some background on this new area could be developed.
The strategy, started in 2006, was to use some of its digital relays and DFRs which were capable of

6
delivering phasor data. As these devices were connected to the operational network, see Figure 1, most
of the effort relay on implementing a Phasor Data Concentrator (PDC). Initially, manufacturer
software was used. Restricted interoperability among these computer programs and IEDs of other
manufacturers were verified. As the project evolved, it was decided to specify, purchase and install an
open source based PDC platform, eventually put into operation in early 2010. By this time FURNAS’
SPMS had about 13 measuring points across its power system [11], see Figure 3. Synchrophasor data
has been used in helping post-event analysis and in calculating the transmission line positive sequence
impedance (Z1).

Figure 3 : FURNAS SPMS measurement points, from [11].

FURNAS experimental SPMS has the advantage of not needing any special hardware or connection
besides those used to install the IEDs. Furthermore, the deployment of an open source based PDC
platform makes it easier to add new IEEE C37.118 compliant IEDs into the SPMS as well as to
upgrade PDC software versions.

With the ongoing harmonization process between IEEE C37.118 and IEC 61850 standards, provided
by the IEC/TR 61850-90-5, further levels of integration are expected to be achieved, such as [12]:
 PMU and PDC functions integrated into IEC 61850 compliant IEDs, with issues like LAN-Based
Time Synchronization (using C37.238-2011 profile for use of IEEE Std. 1588 Precision Time
Protocol in power system applications) and cyber security (using IEC 62351) resolved.
 PMU and PDC can be introduced in the overall IEC 61850 life-cycle (design, implementation,
commissioning, maintenance etc.).
 Many use cases are previewed for synchrophasors, including post-event analysis applications such
as continuous or event data archiving, using COMTRADE file format.
 SCADA systems capable of using synchrophasor data, like SAGE [13], will immediately benefit
from this standardization.
 IEC/TR 61850-90-5 includes the use of advanced IP multicast and IP subscription to provide a
reliable communication infrastructure that minimizes the number of data concentrators required to
distribute synchrophasor information over wide area networks.

In a medium/long term, FURNAS foresees a number of synchrophasor applications to improve system

operation:
 Complementing DFR based automated fault analysis with synchrophasor data.
 Continuous transmission line positive sequence parameter validation.
 Enhancement of its state estimator mathematical models.
 Power system positive sequence voltage angle real time monitoring for bulk transmission stability
and synchronism check.

7
5. CONCLUSIONS

Digital Fault Recordings practices and perspectives for automated data retrieval, information
management and analysis have been discussed from the theoretical and practical perspectives,
including some experience from a real implementation. The nature and role of automated power
system fault analysis was presented in the first section. FURNAS fault recording systems, one for
DFR dedicated devices and other for relays with DFR function, were described. The main features of
SINAPE.NET, an automated fault analysis system, used in FURNAS, were highlighted. Then, the
impact of integrating new protection and automation technologies, such as IEC 61850 and SPMS to
the already existing systems was considered. The authors understand that, although these new
technologies, supported by improvements in related standards, will add a large number of facilities,
not only in the execution of daily tasks, but also in the area of device life-cycle management
(specification, setting and maintenance), companies will have to review many internal processes to
benefit adequately. This challenge will require time and resources to be accomplished.

BIBLIOGRAPHY

[1] Working Group SC B5-20 CIGRE. “New Trends for Automated Fault & Disturbance Recording
& Analysis”, Cigré Electra Magazine, number 251, August 2010, pages 60-69. In English and
French.
[2] IEEE Std C37.111-1999, “IEEE standard common format for transient data exchange
(COMTRADE) for power systems,” Mar. 1999.
[3] M.A.F. RAMOS, R.L.F. PINTO, E.M. BRANDI, “Operational and security aspects of a
segregated protection relays access network.” (“Rede de Acesso a Relés de Proteção com
Segurança e Controle pela Operação.”), XI Symposium of Specialists in Electric Operational
and Expansion Planning, 2009, Belém, Pará, Brazil. In Portuguese.
[4] M.A.F. RAMOS, R.L.F. PINTO, E.M. BRANDI, “Mitigation of problems in a protection relay
network.”(“Mitigação de Problemas de uma Rede de Relés”), IX Protection and Control
Technical Seminar (STPC), 2008, Belo Horizonte, Minas Gerais, Brazil. In Portuguese.
[5] M.A.M. RODRIGUES et alia, “Evaluation of Protection System Performance using DFR and
Relay Comtrade Files”, Cigré General Section, article B5.02.10, Paris, France, 2008
[6] J.C.C. OLIVEIRA et alia, “Features of a Fault Analysis System for Transmission Lines”
(“Características de um Sistema de Análise de Oscilografias para Linhas de Transmissão”), XI
Symposium of Specialists in Electric Operational and Expansion Planning, 2009, Belém, Pará,
Brazil. In Portuguese.
[7] W. BAASS, K. BRAND, A. MENON “Acceptable Function Integration of Protection and
Control at Bay Level”, Study Committee B5 Colloquium, article 217, Madrid, SPAIN, October
2007.
[8] IEEE Standard PC37.232, “Recommended Practice for naming time sequence data files –
COMNAMES”, August 2007.
[9] R. MOORE, M. GORAJ, “Ethernet for IEC 61850”, PacWorld Magazine, September 2010.
[10] R. MORAES, H. VOLSKIS, YI HU, “Deploying a large-scale PMU system for the Brazilian
interconnected power system”, Third International Conference on Electric Utility Deregulation
and Restructuring and Power Technologies, Nanjuing, China, April 2008, pp. 143 – 149, DOI
10.1109/DRPT.2008.4523392
[11] J.E. SILVA, G.M.D. BORBA, M.A.F. RAMOS, “New applications using a PDC and a low cost
PMU network” (“Novas aplicações utilizando um PDC e rede de PMUs de baixo custo”) X
Protection and Control Technical Seminar (STPC), October 2010, Recife, Pernambuco, Brazil.
In Portuguese.
[12] G. FITZPATRICK et alia, “PAP13: Harmonizing IEEE 37.118 with IEC 61850 and Precision
Time Synchronization”, Smart Grid Interoperability Panel, December 2011. Available at
http://www.pointview.com/data/2011/12/53/pdf/Ron-Farquharson-SKKKEMVL-15465.pdf.
[13] M.A.M. RODRIGUES et alia, “Synchrophasor data applications for power system control
centers in Brazil, a proof-of-concept initiative for control center candidate functions”, Cigré
Study Committee B5 Colloquium, article 314, Paris, Jeju Island, South Korea, October 2009.