Tutorial on Networking for Digital

Substations

Harsh Vardhan – GE Grid Solutions

2019 Texas A&MProtective Relaying Conference

Agenda
• Performa nce requirements
• Good Pra ctices
• Conclusion
Digital Substation
Network (SCADA / Station Bus) PTP GOOSE MMS

Relay BCU DFR Relay BCU Relay Relay Relay

Bay 1 Bay 2 Bay 3 Bay 4 Bay 5 Bay 6 Bay 7 Bay 8

MU MU MU MU MU MU MU MU
MU MU MU MU MU MU MU MU
RIO RIO RIO RIO RIO RIO RIO RIO
Network (Process Bus) GOOSE SV PTP
 – Bandwidth Requirements
Digital Substation messages
• GOOSEmessa ging – sta tus a nd control fla gs between devices

• Sampled Values (SV) messa ging – insta nta neous sa mples of current a nd volta ge
published by merging units (MUs) to other IEDs

• IEEE 1588 (PTP) messa ging – highly a ccura te time synchroniza tion over pa cket-
ba sed networks
Message Steady State Burst

GOOSE ~1 Kb/s ~1 Mb/s

~5 Mb/s (50 Hz),
SV (Protection) ~6 Mb/s (60 Hz) -NA-

~10 Mb/s (50 Hz),

SV (Metering) -NA-
~12 Mb/s (60 Hz)
PTP ~1 Kb/s -NA-
 – Latency Requirements
Digital Substation messages

Message Max. Delay m

( s) Bandwidth Application

GOOSE (Trip) 3 Low Protection

GOOSE (other) 10 to 100 Low Protection

Sa mpled Va lues 4 High Process Bus

PTP - Low Genera l Pha sors, SVs

MMS (low speed) < 100 Low Control

MMS (med speed) < 500 Low Control

As per IEC 61850-90-4
Selection of Network Devices
 Substa tion Ha rdened, Dua l Power Supply, Wa tchdog conta ct
 Number a nd Type of Ethernet ports : Don’t forget to include spa re ports
 Port Speed, Ba ndwidth : Trunk Ports, Edge ports Ba ndwidth req might be different
 La yer 2 Functiona lities: MAC, VLAN, ACL for tra ffic segrega tion
 PTP : Tra nspa rent clock (1-step or 2-step)
 Redunda ncy : PRP, HSR, RSTP, RedBox etc.
 Other fea tures :
• SNMP for monitoring
• IGMP for PMU/ R-GOOSE a pplica tion
• L3 fea tures
• Cybersecurity etc.
Good Practices
 -Receiver as close as possible
Keep Sender
• Minimize La tency
• Minimize Ba ndwidth requirements on Trunk Ports
• Avoid Loss of pa ckets during congestion
 - RSTP
Redundancy of Network
 - PRP
Redundancy of Network
SAN DAN

RedBox
DAN DAN
SAN
 - HSR
Redundancy of Network

Issues with Large

Network:
• Bigger the ring, bigger
a re the cha nces of
ha ving more tha n
single fa ilure
HSR HSR
• All the IEDs need to
support a ll the da ta
flow
 – MAC Filtering
Network Traffic Segregation

A MAC a ddress is given to a network

a da pter when it is ma nufactured. It is
ha rdwired or ha rd-coded onto your
computer's network interfa ce ca rd (NIC)
a nd is unique to it

A MAC Address Table is used to define

the aging time of learned MAC
addresses, their mode of operation,
whitelisting and blacklisting devices
according to their right to access
subnetworks.
 – MAC Filtering
Network Traffic Segregation
• Static MAC Filtering

• Dynamic MAC Filteringusing GMRP etc.

IED A: GOOSE Publishing Pa ra meters
 – VLAN
Network Traffic Segregation
Virtua l LAN technology a llows logica l sepa ra tion of the physica l network.
 - VLAN
Network Traffic Segregation

IED A: GOOSE Publishing Pa ra meters

Access Control Lists
• Block or a llow whole types of tra ffic ba sed on EtherType
• Ca n explicitly block or a llow GOOSE, SV, PTP on a ny ports
• Deny a ll other tra ffic by defa ult
 – Prioritizing Traffic
Quality of Service (QoS)
incoming data outgoingdata

data packet switching data packet

functions
data packet
data packet
data packet

high priority high priority

data packet data packet

Message Priority in network QoS

PTP Medium 4
Sa mpled Va lues Medium 4
GOOSE (control) High 5
GOOSE (trip) Highest 6
MMS a nd TCP/IP comms Low 0
Network FAT
• Wha t a FAT is:
• Off-site commissioning of the substa tion network
• Performed with softwa re simula tors / test equipment
• Tested in a controlled scena rio

• Wha t a FAT does:

• Reduce commissioning time
• Reduces insta lla tion time
• On-site work is verifying communica tions ca ble connections
Summary
• Our substa tion networks don’t cha nge frequently
• Tra ffic flows a re consistent a nd predicta ble
• Network requirements only cha nge with ca pita l projects

• Simple, sta nda rd networking tools cover our needs

• Link speed to provide ba ndwidth
• Tra ffic sha ping through VLANs, MAC a ddress filtering, Access Control Lists
• QoS, tra ffic sha ping to a ddress la tency
• Network topology (RSTP, PRP, HSR) provides relia bility

• First step is good project engineering

Thank You

Questions?