SEBA:

SDN Enabled Broadband Access

Aseem Parikh & Andrea Campanella
 CORD – Central Office Rearchitected as a Datacenter

Residential Mobile Enterprise

vOLT, vSG, vRouter, vBBU. vMME, vSGW, vPGW, vCDN vCarrierEthernet, vOAM, vWanEx, vIDS
vCDN

Access Service Orchestration & Control Software Stack

White-box Switches with
Merchant Silicon

Cloud

BBUs PON COTS (Off-the-Shelf) x86 ROADM

OLTs Compute (Core)
SDN NFV Economies of a datacenter, Agility of a cloud provider
2
 SEBA: Built on CORD
Mobile Enterprise Residential
Services Services Services
Mobile Enterprise Residential
NEM
Network Edge Mediator

SEBA
OSS mediation & FCAPS support for operationalization
Leverages XOS project & toolchain

Trellis

ONOS

ORAN VOLTHA Stratum ODTN

Radio PON PON ROADM

Units OLTs OLTs Shared Cloud Infrastructure (Core)

3
 Traditional Residential Access

Residence Central Office

ETH Backbone
RG ONU OLT AGG BNG Network

RG – Residential Gateway
OLT – Optical Line Termination
BNG – Broadband Network Gateway
 Disaggregated Residential Access
vBNG
Virtual OLT • VLAN termination,
manages and • Hierarchical QoS
abstracts PON • DHCP

Residence Central Office

vBNG
AAA
VOLTHA

ETH Backbone
RG ONU OLT AGG BNG Network
OLT Leaf-Spine
MAC Fabric

Fabric provides
• Eth
aggregation
• Routing
+ the ability to introduce other edge-compute services per subscriber
 ONF Reference Designs
Complementing Open Source with Operator Led Specifications

https://www.opennetworking.org/refe
rence-designs-dashboard/
 SEBA – SDN Enabled Broadband Access
Operator Driven: AT&T, DT, Google Fiber Telefonica, & Turk Telekom

• RD developed over 2018

• Exemplar platform delivered in Jan ‘19
• Supply-chain of like-minded vendors
forming to build common infrastructure
• Production deployments start in 2H2019

Technology Highlights
• Containers run in Kubernetes as cloud
underlayer
• Edge cloud orchestration option for
convergence to Akraino
• Enable any Northbound CAP - Legacy OSS
and new systems like ONAP

7
Virtualized Broadband:
ONF SDN Enabled Broadband Access (SEBA)
2019 Reference Design (RD) 2.0 Planning

Tom Moore
Principal Member of Technical Staff
AT&T Converged Access and Device Technology
April 2019
SEBA RD 2.0 Steps

Published - SEBA RD 1.0

Targeted 2019 Steps & Dates

• April 2019
• Solicitation of Priorities for RD 2.0
• Prioritization Completed for RD 2.0

• July 2019
• Draft RD 2.0 for ONF Membership review
• September 10-13, 2019
• Review opportunity at ONF Connect 2019
• October 2019
• Public Release of SEBA RD 2.0

9
Candidate RD Priorities
SEBA RD 2.0 Priorities: Proposals & Solicitation

Candidate Priorities Identified by RD Team (no order)

1 Analytics Function in NEM

1 2
2 Akraino SEBA Blueprint for Network Cloud

3 SDN Controller (ONOS) Evolution

4 P4 Runtime (in addition to OpenFlow)

3
Additional Access Technologies – including but not 4
5 limited to Gfast, xDSL, NGPON2, EPON, 25GPON,
DOCSIS, Fixed Wireless, Ethernet

6 BNG in SEBA – Lessons learned and expanded 6 7

definitions

7 Merging workloads for SEBA (fixed access), COMAC 5

(wireless) and/or OMEC (wireless core)
8
8 Third-party apps via Multi Access Edge Cloud (MAEC);
Virtual CPE (vCPE)
? SEBA RD Team Solicits Additional Priorities
10 ?
SEBA Exemplar Platform – Key Components

12
 Outline
• VOLTHA – Disaggregated OLTs

• Trellis – Multi-purpose Leaf-Spine Fabric

• CORD platform – Service Delivery at the Edge

• SEBA Exemplar Implementation

• SEBA Development & Roadmap

VOLTHA: Disaggregated OLTs

14
 OLT Disaggregation → VOLTHA
VLANs AAA Mcast …

ONOS
VLANs, IGMP, Backplane
802.1x, Mcast … OpenFlow NetConf

PON
MAC OpenFlow Agent NetConf
blades

Compute OLT API OMCI Stack

blades VOLTHA

GPON OLT

VOLTHA: Virtual OLT Hardware Abstraction GPON OLT MAC

15
 VOLTHA Operation
Bridging/
VLANs dhcp mcast
Routing

ONOS

Logical
switch
VOLTHA

ONU
UNI
Switch NNI OLT

UNI
 VOLTHA Architecture

TP
Tech
TP
Profiles
 Industry’s First White-Box XGS-PON OLT
VOLTHA
White-Box = Open-Hardware Specs (OCP)
core + Open-source software
OpenOLT Adaptor
(ONF+OCP)
Py GRPC client

Edgecore
GRPC OpenOLT ASFvOLT16
generic
protobufs
Software Whitebox OLT

C++ GRPC srv

Chip Driver

SDK(s)

Whitebox OLT 18
(including EdgeCore)
 March 2018 Why OpenOLT Adaptor?
VOLTHA Generic OLT adaptor - ease of onboarding for new vendors (including whitebox
core vendors)
OpenOLT Adaptor
Py GRPC client

VOLTHA CORE
OpenOLT OpenOMCI ONU
GRPC
(generic Alpha ONU

protobufs) Iskratel ONU

Arcadyan ONU
CIG ONU
CIG EdgeCore Iskratel
OLT OLT OLT Tellabs ONU
C++ GRPC srv
Chip Driver Tellabs
OLT
SDK(s)

Whitebox OLT
(including EdgeCore)
Trellis: Multi-purpose leaf-spine fabric

20
 Trellis Overview
Multi-purpose leaf-spine Bare-metal hardware
Spine
Open-source software
fabric designed for NFV
SDN-based (built on ONOS)
Quagga

Spine

Paired
Leaf
Leaves

Access
Devices Upstream
Router
Dual homed hosts

Single homed hosts

Field Office (2nd stage) Central Office (1st stage) DHCP Server
 Trellis Features
● Bridging with Access & Trunk VLANs (within a rack)
● Routing (inter-rack)
○ IPv4 & IPv6 Unicast routing with MPLS Segment-Routing
○ IPv4 & IPv6 Multicast routing
● Dual-homing for compute-nodes and external routers
● Multi-stage fabrics (2 layers of spines)
● vRouter - entire fabric behaves as a single router
○ BGP (v4/v6) support for external (upstream) connectivity
○ Static routes, route blackholing
○ DHCP L3 relay (IPv4/v6)
● MPLS Pseudowires
● QinQ termination
● T3 - Trellis Troubleshooting Tool
● ASIC Support
○ Broadcom Qumran, Tomahawk, Trident2 switches from EdgeCore & QCT
○ Preliminary support for Cavium Xpliant switches and P4-based Tofino switches
 White-Box = Bare-metal hw + Open-Source sw
Spine Switch Leaf/Spine Switch Software Stack
White Box Switch GE mgmt. to controller
EdgeCore 6712,7712 OpenFlow 1.3

Indigo OF Agent
32 x 40G/100G ports downlink to leaf switches OCP
Software OF-DPA
Trident2, Tomahawk, Qumran
(ONL,ONIE)
Leaf Switch BRCM ASIC
OCP Bare Metal Hardware
White Box Switch
OCP: Open Compute Project
EdgeCore 5712,5912
GE mgmt. ONL: Open Network Linux
QCT LY8
ONIE: Open Network Install Environment
BRCM: Broadcom Merchant Silicon ASICs
48 x 10G, 6 x 40G/100G OF-DPA: OpenFlow Datapath Abstraction
 Fabric ASIC Pipeline* (BRCM’s OF-DPA)
* Simplified view
Why OF-DPA? Multi-
cast
Abstracts underlying ASIC Routing
Table MPLS Phy
Enables programming of all L3
Label
Group
Port

flow-tables & port-groups ECMP

Group MPLS
Label
Phy
Unicast Port
Group
Routing
Termin- Table L2
Phy Ingress ACL
Interface Phy
Vlan ation L3
Por Port
Table MAC
Policy Group Port
Mcast
t Table
Table
Table
Group L2
Interface Phy
Group Port
MPLS
MPLS
Table
Vlan 1 L2 L2 Phy
Table Port L2 Interface
Group Port
Table Flood
Group L2
Interface Phy
Group Port
Bridging
Table

24
 Trellis & P4 Enhanced with P4
Same set of Trellis program deployment and
applications on ONOS pipeline configuration
Segment Routing DHCP L3 Relay vRouter Multicast SPGW-app

ONOS Cluster P4
OF-DPA driver fabric.p4 driver

Allowing new
functionality on
OpenFlow NetConf P4Runtime gNMI hardware (demo
at MWC ‘18)

Brcm Qumran Barefoot Tofino

P4
Brcm Tomahawk Cavium Xpliant
capable
Brcm Trident2 Mellanox hardware

25
CORD Platform: service delivery @ the edge

26
Issues with the previous CORD platform & build
• No distiniction between development vs deployment
• Most of the software built from source at “deploy time”
• Big scripts, scripts calling scripts, long process
• Monolithic: If something breaks, wipe and start from scratch
• Brittle: Small variations of hardware/connectivity could break build

• Not flexible or modular

• Required specific versions of server OS
• Tightly coupled components – cannot exclude or replace components
• Required internet access for install

• Hard to use
• Choose a profile at start; to change profiles reinstall
• No means for adding services at runtime
 New CORD 6.0 platform (July-2018)
CORD control
software → set of ONOS Grafana ELK Kafka Prometheus XOS profile
Docker containers
deployed on Kubernetes
Kubernetes using
OS OS OS
standard tools like
Helm charts
Bare metal server Bare metal server Bare metal server

Management network

Access
Leaf Leaf Upstream
Node OS OS
OS
Dataplane network

Spine Spine 28
OS OS
#OpenCORD
 SEBA – A profile loaded on the CORD platform
CORD control onos apps workflow VOLTHA Runtime config & service instantiation
software → set of ONOS Grafana ELK Kafka Prometheus XOS profile
Docker containers
deployed on Kubernetes
Kubernetes using
OS OS OS
standard tools like
Helm charts
Bare metal server Bare metal server Bare metal server

Management network

Access
Leaf Leaf Upstream
Node OS OS
OS
Dataplane network

Spine Spine 29
OS OS
#OpenCORD
 Issues with the previous R-CORD
CORD Head Node
ONOS XOS VOLTHA … Radius Subscriber Traffic through
vSG (virt. Service
CORD Compute Node CDN Gateway)
OvS vSG

Internet
Whitebox ToR Fabric Switch
EdgeCore 7712 (Tomahawk 32 x 100G)
Residential
Subscribers

EdgeCore Whitebox OLT BCM R

(ASFvOLT16) ONU G

Central
Office
SEBA Exemplar Implementation
- putting it all together

31
 Legacy
SEBA Exemplar ONAP
OSAM-Central
OSS
Built on the new CORD platform
OSAM-Local EMS/NMS-adaptor

SEBA POD Network Edge Mediator (NEM)

Providing FCAPS
& Inventory

vOLT dhcp mcast SR FPM T3

All control
software deployed ONOS Cluster
as containers on REST
compute nodes Kafka OF OF
using Kubernetes Redfish Redfish

VOLTHA TP
TP
TP

ONU
External BNG
UNI AGG Switch
Subscriber traffic OLT
‘fast-path’ to
Internet NNI AGG Switch

Compute
 NEM Orchestration & SEBA Workflows
Different operators == Different workflows

SEBA POD NEM

XOS GUI
XOS Core
Chameleon
Synchronizer
Tosca
Databas
s e

vOLT dhcp mcast SR FPM T3

Authentication is a
workflow option ONOS Cluster BNG is a workflow option
• 802.1x based • External physical BNG
• PPPOE based OF OF • vBNG in compute
Redfish Redfish
• DHCP based • BNG in Agg switches
• BNG in OLT boxes

VOLTHA TP
TP
TP

ONU
Subscriber services
are workflow options UNI AGG Switch
• HSIA OLT
• Voice
• IPTV NNI AGG Switch
• Business services
• Technology
profiles
• Speed profiles
Compute
NEM Monitoring GRAFANA KIBANA

Infrastructure PROMETHEUS ELK STACK XOS

NEM
Operator OSS KAFKA

Alarms, events & stats ONOS

Logs

VOLTHA

ONU BNG
UNI
RG OLT AGG

NNI

Compute
 NEM Dashboards

NEM User Interface: runtime

service instantiation, inventory,
workflow status

NEM Monitoring Dashboard:

stats, events, logs (FCAPS)
 Demo setup BBWF 2018
SEBA Peripheral/PNF/Pod Radius
Server
Network Edge Elasticsearch Grafana/Kibana Kafka XOS GUI Chameleon Tosca
Synchronizer Databas
Mediator (NEM) Logstash Prometheus Exporter XOS Core
s e
Configuration
Software Stack

sadi
vOLT dhcp aaa kafka-i SR
s

ONOS

VOLTHA

BBSim
ONU
RG Alpha DHCP Serve
(subscriber RG
RG Iskratel EdgeCore OLT addressing)
BNG
Hardware

RG Arcadyan
Adtran OLT
RG Adtran Public
EdgeCore Interne
CIG CIG OLT AGG Switch t
RG

RG
Iskratel Iskratel OLT (X)
RG
Iskratel Compute
Iskratel OLT (G)
Nodes
Demo setup BBWF 2018

ONUs: Arcadyan,
Alpha, Adtran, CIG,
Iskratel

OLTs: Adtran, CIG,

EdgeCore & Iskratel

AGG switch: EdgeCore

Servers: VOLTHA, ONOS, XOS,
K8s, ELK, Docker, Prometheus,
Grafana, Kibana
SEBA Development & Roadmap

38
 Distributed DevOps - SEBA
Simulator

SEBA System
OLT, Switch End-to-End OLT & ONU
Hardware
Redfish APIs

VOLTHA 2.0 AT&T

Test automation
Certification
 SEBA Distributed DevOps – ONF Responsibilities

Joint Support Deliver SEBA/VOLTHA Tooling QA

Development VOLTHA/SEBA New Stability &
with AT&T Community Features Scale
Foundry • Weekly SEBA public • ONF team resposible • Focus on stability of • BBSim – Work • Developing
• Daily scrum meetings for all ONOS and VOLTHA – with NTT in the automated feature
• Bugs - issue NEM feature robustness and development of tests
• JIRA issues – kanban
analysis; root-cause; development repeatability PON simulator • Extending
methodology for scale testing framework for
• Design discussions • FCAPS features – • Investigate
for new features • Mailing lists, Slack kpis, alarms & logs in scalability and • SEBA-in-a-BOX: hardware based
• Documentation community support SEBA monitoring performance of full Complete end to end-to-end tests
• Recommend best- • VOLTHA meetings, infrastructure SEBA system end emulation of • Work with
practices/training TST, architectural • AT&T workflow SEBA system community to
guidance development with ponsim & develop test
• Integrate with new mininet automation and
ONU and OLT certification
vendors procedures
• AGG switch features
 SEBA Deployment Goal @ AT&T

SEBA POD
Up to 64 ONUs
per PON port
Internal GE Management Switch DHCP Server
ONU+RG (subscriber
Up to 16 PON RG
ports / OLT addressing)
device
ONU+RG EdgeCore OLT
DHCP
OLT Server (POD
addressing)
ONU+RG
OLT AGG Switch AT&T
(Tomahawk External Radius
32 x 100G) BNG Server
Up to 16
OLT
devices External OSS
VOLTHA, ONOS,
Compute Node NEM, Kubernetes,
OLT Compute Node Docker registry, Public
Gateway VM Interne
Compute Node Abstract OLT t

41
 SEBA Modularity
ONAP/OSAM

Abstract OLT

NEM Monitoring
NEM Orchestration (XOS)
(Prom,Grafana, ELK)

Access Fabric
apps apps

ONOS

VOLTHA Trellis
Kafka
K8s
OS OS OS

ONU
AGG Compute
OLT
ONU switch Nodes

42
 Roadmap
• BNG Disaggregation • Redundancy

• Using P4 in Aggregation • Integrating VOLTHA 2.0 &

switch Technology profiles

• Implementing more • ISSU

operator workflows
• Integrating M-CORD profile
• Performance & scale to use SEBA as mobile
improvements for Trials backhaul
 Summary
• ONF: Operator driven curated open source
• CORD is the flagship umbrella project
• SEBA exemplar implementation is built on the CORD platform

• Modular Components:
• VOLTHA abstracts the PON as a quasi-Ethernet switch to the SDN controller
• Trellis manages a multi-purpose leaf-spine fabric
• ONOS – SDN controller for both VOLTHA & Trellis
• XOS: service delivery & orchestration of workflows
• CORD platform: set of Docker containers managed by K8s

• SEBA: SDN Enabled Broadband Access

• SEBA – a profile instantiated on CORD, jointly developed by ONF, AT&T & community
• NEM – provides northbound interfaces for integration with operator backends
• Significant focus on FCAPS infrastructure, multiple operator workflows
• Headed to trials at AT&T, significant interest from operators worldwide