Industrial IP/MPLS

for Power Utilities

Dominique Verhulst
March 1st 2016

1 © Nokia 2016
UTILITY COMMUNICATIONS REQUIREMENTS

LOW LATENCY

LOW JITTER

LOW & STABLE ASYMMETRY

SOLID SYNCHRONISATION

SECURE
2 © Nokia 2016
THE RISE OF ETHERNET & IP
there is no way to avoid it
Ethernet & IP ubiquity
WiFi
3G Mobile, LTE, 5G
VoIP
CCTV
IEC 104 & DNP3 over TCP/IP
IEC61850
Synchrophasors
WAMS
PAGA
3 © Nokia 2016
 INDUSTRIAL
IP/MPLS
4 © Nokia 2016
 SDH

1x SDN
VSR
PCE

PDH INDUSTRIAL NFV

Segment Routing
IP/MPLS
IP

… BACKWARDS
… FUTURE
COMPATIBLE
PROOF
7750 SR 7705 SAR 7210 SAS 5620 SAM

5 © Nokia 2016
 INDUSTRIAL IP/MPLS

deterministic resilience support for low latency sync Ethernet environmental cyber operational
behavior legacy & jitter hardening security simplicity
& ip services
services

6 © Nokia 2016
TELEPROTECTION VALIDATION ON INDUSTRIAL IP/MPLS

SEL

7 © Nokia 2016
UNDERSTAND THE IMPACT OF PACKETISATION

JITTER BUFFER PAYLOAD

8 © Nokia 2016
 1 DIP5000
DIFFERENTIAL PROTECTION 2 7SD52

DISTANCE PROTECTION 3 TPU1

8 P545
4
SV over 61850
4 5 7XV

6 7XV

7 6
7 7SD52
2 10
9 P541
5
7 7XV
NSD70 NSD570
1 8 TPS64
3

9 DIP5000

10 7SD52
9 © Nokia 2016
TELEPROTECTION VALIDATION RESULTS
Vendor Model Type Latency (ms) Jitter Interface
(µs)

Siemens 7XV Distance 3.12 5 X.21

ABB NSD570 Distance 3.6 5 Ethernet
Areva DIP5000 Distance 3 5 X.21
ZIV/Dimat TPU1 Distance 5.92 E1
ZIV/Dimat TPU1 Distance 8 E&M
Nokia TPS64 Distance 8.3 G.703
Siemens 7SD52 Differential 3.12 5 X.21
Siemens 7SD52 Differential 1.30 C37.94
Siemens 7SD52 Differential 1.15 E1
Alstom P541/P591 Differential 3.25 5 X.21
Alstom P545 Differential 3.48 C37.94
Toshiba GRL100 Differential 2.9 3 X.21
Toshiba GRL100 Differential 0.1 3 Ethernet
Strathclyde Uni SV over GOOSE Differential 0,1 Ethernet
10 © Nokia 2016
 ADVANCING INDUSTRIAL MPLS
ASYMMETRICAL DELAY COMPENSATION

Measure residence time Compare with Adjust to less than 1ms

Use precision timestamps Configured JB difference

Repeat process & adjust if difference is above 1/4ms

TDM TDM
data
Packetised data
data

Packetisation Jitter Buffer

11 © Nokia 2016
Validating Asymmetrical Delay Compensation
with network impairment
Automated script:
1. Toggle c-pipes status (enable and disable)
2. Listening to RTDS GOOSE for trip or comms
2 loss
2
1 1
RTDS LSP Tunnel
RTDS
7705 7705
C37.94 SAR-8 SAR-8
C37.94
Alstom
P545
 Alstom
P545
Calnex Paragon-X

ADC IP/MPLS ADC

on|off on|off

0.3-5ms
12 © Nokia 2016
UNDERSTAND THE IMPORTANCE OF SYNCHRONISATION

Precision Time Protocols are subject to latency and jitter

Synchronous Ethernet may not be available everywhere

You may not have or want GPS everywhere

Loss of synchronisation WILL cause network issues

13 © Nokia 2016
Validating Asymmetrical Delay Compensation
with degraded synchronization
Industrial IP/MPLS

RTDS LSP Tunnel

RTDS
7705 7705
C37.94 SAR-8 SAR-8
C37.94
Alstom
P545
 Alstom
P545
Calnex Paragon-X

Clock wandering at both 7705 SARs causing

asymmetrical latency

Elapsed time
ADC disabled ADC enabled
(min)

40 False trip No false trip

14 © Nokia 2016
 CYBERSECURITY FIREWALL
control in | out data flows

ENCRYPTION
AUTHENTICATION
ensure privacy & integrity of data
NAT
conceal & simplify network topology

SECURITY MANAGEMENT
provision, monitor & report
IDS | IPS
IP|MPLS detect suspicious behaviour

MANAGEMENT | CONTROL | SR
DATA | PHYSICAL PLANE (D)DOS
rich security feature set
OS
MITIGATION
withstand (D)DoS attacks

15 © Nokia 2016
ENCRYPT WHERE IT MAKES SENSE
Optical Layer
- Leased fiber bundles
IP layer
- Carrier services LATENCY

- Public network backup

Service layer
- VLL’s CONSIDERATIONS &
TRADE OFF’s PERFORMANCE
- VPLS
- VPRN
Application layer
KEY MANAGEMENT
- Teleprotection
- SCADA data
- CCTV video
- Voice
16 © Nokia 2016
ENCRYPTION OF DIFFERENTIAL PROTECTION
Industrial IP/MPLS

RTDS NGE Encrypted tunnel RTDS

7705
SAR-8
C37.94 C37.94
Alstom Alstom
P545 GOOSE/SV GOOSE/SV P545

Test Result Propagation delay (ms)

IEEE C37.94 No encryption 1.68
With encryption 1.70
IEC61850 No encryption < 1*
(GOOSE/SV) With encryption <1*

Network Group Encryption-incurred end-to-end delay is

only 20µs
* The microcontroller didn’t provide propagation delay data, latency is typically below 1ms

17 © Nokia 2016
OPERATIONAL SIMPLICITY

- Show a service and

its supporting
topology and drill
down to other
network layers
- Link Utilization
- Alarm statuses, root
cause analysis,
object states, etc.
showed on nodes
and links

18 © Nokia 2016
CONCLUSIONS
Industrial IP/MPLS meets the Power Utility Communications
needs
Latency, Jitter and Asymmetry requirements are met
1 Takes packetisation impact into consideration
Puts the right Traffic Engineering and QoS models in place
Combines SyncE, 1588v2 and GPS for synchronisation
Is secure
Is simple to configure and operate

19
Provides a path to the future of SDN and IoT
© Nokia 2016
 PDH/SDH Pure Ethernet MPLS-TP IP/MPLS

future proof? No limited limited yes

Innovations in communications are in packet technologies, one more than the other…

which transport legacy limited legacy, extensive legacy, extensive legacy,

services can it ethernet only (L2) ethernet only (L2) ethernet (L2),
support?
IP (L3),
multicast
The communication network needs to be capable of transporting data from old, current and future
applications. The application decides/defines what transport service to use (legacy/L2/L3/multicast).

resiliency okay limited limited full

Application and therefore network availability being of utmost importance, a wide range of resiliency
and recovery mechanisms needs to be implemented.

20 © Nokia 2016
 PDH/SDH Pure Ethernet MPLS-TP IP/MPLS

traffic Yes limited yes yes

separation

Communication from concurring applications should not merge … SCADA data should not handled
the same way as VoIP.

scalability no no limited yes

(capability to
create very
large networks) Networks can grow to 1.000, 10.000 and in the future even beyond 100.000 endpoints , the network
technology needs to assist that effortlessly.

topology very limited very limited limited fully flexible

flexibility

The application defines/decides its interconnection topology: fully-meshed, hub & spoke, ring …

21 © Nokia 2016
 Pure
Industrial
PDH/SDH Ethernet MPLS-TP IP/MPLS
no future, not scalable, inflexible all transport services
(IP/MPLS subset), (legacy,L2,L3,multicast),
inefficient packet inflexible (topology),
transport L2 services only, full resiliency & recovery
limited recovery mechanisms,
mechanisms, limited recovery
mechanisms highly scalable,
L2 services only
secure,
SDN | NFV ready

22 © Nokia 2016
Industry Accolades
highlight>
2015 winner

Product of the year 7705 SAR-

Wx

2014 winner 2014 winner 2014 winner 2014 winner

Best LTE APCO hot product Best network Product of the year 7705
BH solution 7705 SAR/9500MPR technology SAR-Hc

2009/2011 winner 2011 winner 2011/2012 winner 2012 winner 2013 winner

Best Telecom equipment Best Carrier Eth. Product of the year 7705 Best Carrier Eth. Best LTE
7705 SAR-8 MBH product SAR-8/7705 SAR-H MBH product BH solution

23 © Nokia 2016
INDUSTRIAL IP/MPLS IN UTILITIES
Nokia deployments 2009-2015

122
UTILITIES HAVE ALREADY
MADE THE TRANSITION TO
Americas
APAC
EMEA
NOKIA INDUSTRIAL IP/MPLS

24 © Nokia 2016 <Change information classification in footer>

Find out more
Validating Secure and Reliable IP/MPLS Communications
for Current Differential Protection

25 © Nokia 2016 <Change information classification in footer>

 IP/MPLS HAS BECOME
THE LEADING TECHNOLOGY IN
ENERGY, TRANSPORTATION & PUBLIC
SECTOR

26 © Nokia 2016
27 © Nokia 2016 <Change information classification in footer>