Optical Packet-Switched WDM Networks: a Cost and Energy Perspective

Rodney S. Tucker ARC Special Research Centre for Ultra-Broadband Information Networks (CUBIN) University of Melbourne r.tucker@ee.unimelb.edu.au

Acknowledgements:

Jayant Baliga, Kerry Hinton, Rob Ayre, Gangxiang Shen, Wayne Sorin Australian Research Council, Cisco

Summary
Optical packet-switched networks Network architectures Point-to-point WDM IP networks IP networks with optical grooming Optical burst switching Optical packet switching CAPEX and network architectures Scaling Energy consumption and network architectures Core and access networks Disclaimers: Numbers given here are approximate - YMMV OPEX not included E-mail me for copies of these slides

Impact on network growth

IP Packets

Core Router

Edge Routers

OXC

Optical Packet-Switched WDM Network

1h

op

Is this simple model realistic?

The Reality: Map of the Internet

Patent(s) Pending and Copyright Lumeta Corporation 2007

Not Included: Enterprise Networks Content Distribution Networks Data Centres

Number of Hops in the Internet

2006 Data

0.1 0.08

Pr [ H = k ]

0.06 0.04 0.02 0

15 10 Number of Hops, k

20

25

Source: P. Van Mieghem, Performance Analysis of Computer Systems and Networks, Cambridge (2006)

Evolution of Optical Packet-Switched Networks

Conventional Wisdom
Capacity CAPEX? Energy Consumption? Optical Packet Switching (OPS)

Stage 1

Optical Burst Switching (OBS) Optical Circuit Switching (OCS)

Point-to-Point WDM (P2P) Time

1. Point-to-Point WDM Network

Lightpath Fiber O/E/O Converters

Core Router

O/E/O Converters Access Routers

Router provides sub-wavelength grooming by statistical multiplexing

2. Optical IP Network
Core Router

Edge Routers
1h

OXC

op

2. Optical IP Network
Lightpath Fiber Core Router

O/E/O Converters

OXC/ROADM Optical bypass Sub-wavelength grooming

O/E/O Converters Access Routers

3. Waveband IP Network
Lightpath Waveband Fiber Core Router

O/E/O Converters

MUX OXC/ROADM Waveband grooming Optical bypass Sub-wavelength grooming Access Routers

Cost and Scalability of Optical Networks

Consider three network architectures:
P2P WDM Optical IP Waveband IP

Number of users: 10 million

Compare CAPEX

Parthiban et al., 2003

Input Parameters
Average Access rate per user: 1 - 100 Mb/s 10 million users 5 90 Tb/s 1000 ports 40 Gb/s 250 lightpaths

Capacities: Router: OXC: Lightpath: Fiber:

1000

1000 1 = 40 Gb/s

250
Component costs [Ferreira 02, Sengupta 03] OXC, router chassis, port Fiber, amplifiers, lightpath terminations Ignore Access network, search engines, data centers, etc. Optical impairment cost e.g. regenerators Protection & restoration Multiple domains Cost reductions as technology matures OPEX

Results Network Cost

104 P2P WDM
= pe o l S 1

Cost per User ($)

Waveband IP 102 Optical IP

10
Todays Internet (~ 100 kb/s)
Parthiban et al., 2003

10 Average Access Rate (Mb/s)

100

(Access rate per user in a busy period)

Sanity Check
90

Cumulative Capex $ Billions

80 70 60 50 40 30 20 10 0

Cumulative Internet Capex (North America)

150 million users

~$400 / user

Total Core and Metro Optical transport Access

2000 2005

Year

2010

Nemertes, November 19, 2007: User demand could outpace network capacity by 2010 $137 billion global infrastructure investment needed
Based on data from Nemertes Research, 2007

Observations
Optical IP network Can save costs in todays network Optical bypass reduces router ports Waveband IP network Eliminates the bottleneck in number of ports and lightpaths Least costly for high access rates There is no such thing as a free lunch If you want more bandwidth, you will have to pay for it

Energy Consumption of the Network

Hot spot

Why worry about energy consumption? OPEX Greenhouse Impact Managing Hot Spots - Getting the energy in - Getting the heat out Energy-limited capacity bottlenecks

Power In

Energy Model of Simple IP Network

CRS-1 ~ 10 kW / rack Packet over Sonet Fibre Amps WDM

Core

Core

Core

Core

12816 Edge ~ 4 kW

Metro Edge

Edge

Passive Optical Network

OLT - 100W

Curb

Curb Curb

Curb

Access

ONU ~ 5-10W

0.1 - 1000 Mb/s to the user

Baliga et al., 2007

Power Consumption of IP Network

25 20 hops 20 Power (W/user) 2008 Technology 1.0 % of Electricity Supply

Total
15 10 5
Todays Internet (~ 100 kb/s)

Core Access +Metro SDH/WDM Links

0.5

10

Average Access Rate (Mb/s)

Baliga et al., 2007

Power Consumption of IP Network

200 20 hops 150 Power (W/user) 2008 Technology 10

Total
5.0

100

Core
50 2.5

Access +Metro
0 0 20 40 60

SDH/WDM
0

80

100

Average Access Rate (Mb/s)

Baliga et al., 2007

% of Electricity Supply

7.5

Observations / Questions
Access network dominates energy consumption at low rates Standby mode? Core network dominates at higher rates Reduce hop count? What is the bottleneck in the core? Speed or energy? Optical packet switching Optical transport (WDM) consumes relatively little energy < 5% of energy > 25% of CAPEX Annual CAPEX / Annual energy OPEX > 2 1 3 2

Electronic Router
Forwarding Engine

Fibers 1
1

1 1 J
F

Switch Fabric K

F Demutiplexers O/E Converters Switch Fabrics Buffers

F Multiplexers

Reduced bit rate Electronics Optics

Power Consumption in Routers

1,000,000 100,000 10,000 10 nJ/bit 1,000

P = C2/3
where P is in Watts where C is in Mb/s

Power consumption (W)

P ~ 10
100 100 nJ/bit 10 1

1 Mb/s

1 Gb/s

1 Tb/s

1 Pb/s

Router Throughput

Source: METI, 2006, Nordman, 2007

Energy per Bit in Routers

100,000 10,000

Energy per Bit (nJ)

1,000 100 nJ/bit 100 10 nJ/bit 10 ? 1 0.1

1 nJ/bit

1 Mb/s

1 Gb/s

1 Tb/s

1 Pb/s

Router Throughput

Heat Load
Air Cooling Limit 1 nJ/bit

Year
Source: K. Brill, The Uptime Institute

Energy in High-End Electronic Router

Line Card Buffer O/E Forwarding Engine Buffer O/E Forwarding Engine
Buffer

I/O

Switch Fabric

Routing Tables Power supply inefficiency Fans and blowers

Switch Control

Routing Engine Control Plane

Data Plane

Energy/bit Fraction of Total

0.7 nJ 7%

3.2 nJ 32%

0.5 nJ 5%

1.0 nJ 10%

1.1 nJ 11%

3.5 nJ 35%

Source: G. Epps, Cisco, 2007

Network Energy Consumption per Bit

10-3 20 hops 10-4 Energy per bit (J) ~1 J/b 10-5 10-6 10-7 10-8 0.1 1 10 100 Average Access Rate (Mb/s) Total

Core Metro + Access WDM Links

Energy in Electronic Integrated Circuits

Cwire
CMOS Gates CMOS IC

Energy =

Egate +

1 2

Cwire V 2

Power = Energy x Bit Rate

Diversion: Capacitance of the Core Network

Source Destination
C1 C2

Energy per bit per user:

V = 2, E = 1 J 2

E =1 2

C V 2
Capacitance per user

C 200 nF

50% efficiency
~ 150 million users CNetwork = 200 nF x1.5 108

C Network 30 F (North America)

C Network 300 F (Global)

Energy Consumption in Access Networks

NEC CM7710T

Access N/W Edge Node

Cisco 12816 NEC CM7700S

Cabinet Splitter Cabinet PON

NEC VF200F6

FTTN
Cisco 4503 Zyxel VES-1616F-34 NEC GM100

PtP
Axxcelera ExcelMax CPE Axxcelera ExcelMax BTS

WiMAX

Baliga et al., OThT6

Energy Consumption in Access Networks

WiMAX
10-5

Energy per bit (J)

10-6

FTTN PtP

10-7

PON

10-8

10 100 Average Access Rate (Mb/s)

1000

Wireless access consumes more energy than optical access PON FTTH is greener than FTTN Standby mode shows significant potential (OThT6)

Baliga et al., OThT6

Evolution of Optical Packet-Switched Networks

Capacity Optical Packet Switching (OPS) Optical Burst Switching (OBS) Optical Circuit Switching (OCS)

Stage 2
CAPEX and Energy Barrier

All-optical sub-wavelength grooming

Stage 1
Point-toPoint WDM (P2P)

Time

Stage 2 Evolution OCS to OBS

Core Router Edge Node

Optical Burst Switches Electronics WDM Link

OXC

Edge Router

Access Routers Burst Assembly Router

Optical Burst Switched Network

Optical Burst Switch No Buffering

IP Packets (duration < s)

Data Bursts (duration < ms) IP Packets t Burst Assembly Router Headers
Offset

C. Qiao and M. Yoo, JHSN, 1999

Optical Burst Switching

Source Header Destination

Offset

Time

Burst length < ms

D A T A

Switch

4 Distance

Normalized Cost: OBS vs. IP

20 1 Gbit/s Access Rate Normalized Cost 15 Without wavelength conversion
OBS

10 With wavelength conversion 5

Optical IP

Waveband IP

10-6

10-5

10-4

10-3

10-2

10-1

Average Path Blocking Probability

Parthiban et al., OFC 2005

Why is OBS more Costly?

OBS MUX
Waveband Bursts Wavelengths Wavelengths

DEMX
Waveband

Bursts

Requires increased number of lightpaths for a given blocking probability Switch technology requires fast reconfiguration time: More costly per port than slow OXCs (MEMS etc.)

Solution: Waveband Burst Switching (WBS)

Waveband Route Burst

WBS Waveband Waveband

Pros: Requires fewer OXC ports Cons: OXC ports must be wideband Requires waveband (i.e. multi-channel) wavelength conversion Dispersion issues
Y. Huang et al., OFC 2004, Pathiban et al., OFC 2006

Waveband Burst Switching (WBS)

1 Gbit/s Access Rate 7 Optical IP Normalized Cost 5 WBS with wavelength conversion

Waveband IP

WBS (WC) + Deflection Routing + Burst Segmentation

10-5

-2 -1 -2 -1 10 10 10 10 Average Path Blocking Probability

10-4

10-3

Parthiban et al., OFC 2006

Evolution of Optical Packet-Switched Networks

Capacity

Stage 3
Optical Packet Switching (OPS)

Stage 2
Optical Circuit Switching (OCS)

Optical Burst Switching (OBS)

Stage 1
Point-toPoint WDM (P2P)

Time

Optical Packet-Switched Network

The Holy Grail of Optical Networking

Optical Packet Switches With Buffering

OXC

Access Router

Will a viable optical buffering technology emerge? Can OPS reduce energy consumption?

Optical Packet Switch (OPS)

Electronics

Forwarding Engine

Optics

1
WavelengthInterchanging Cross Connect + Header Replacement Output Buffers

F
Demutiplexers

F
Mutiplexers

K
Input Synchronizers

Optical Buffer Structures

Control

Variable Delay Line

Delay Line

Delay

Delay

Recirculating Loop

Cross Point

Staggered Delay Line

Cross Point

Delay

Delay

Cross Point

Optical Fiber Buffers

Total buffer capacity of 104 Gigabits ~ 103 RAM chips Cost < US$ 50k Buffer power dissipation < 1 kW Cisco CRS-1 with 1000 ports, 250 ms buffering per port

Total fibre length = 40 Gm 150 times distance from Earth to Moon!

.

OPS with 1000 ports, 250 ms buffering per port optical fiber delay lines

Reduced Buffer Size

Minimum feasible buffer size? Optical fiber delay lines

~5 s (~20 packets) buffering per port

Total fibre length = 1000 km Loss per port = 0.2 dB

Planar waveguide or slow light delay lines (0.1 dB/cm)

Loss per port = 33,000 dB Loss

Tucker, PS 2007

Energy

Holographic Buffers
Holographic Medium: Photorefractive material (e.g LiNbO3, polymers) Output image Write speeds up to 1 Gb/s Read speeds up to 10 Gb/s Reference beam Input image Storage density 1/3 Retention time Access Time ~ 50 s Write Time > 500 s

Orlov et al., Proc IEEE, 2004 Psaltis, CLEO 2002 Ashley et al. IBM J. Res. Dev., May 2000

Comparison of Optical Buffer Technologies

Technology Access Time Retention Time Capacity (Packets) Energy/bit Physical Size Chirp Sensitivity Fiber Structuredependent > 500 s > 2,000 ~ 1 fJ Very Large No Planar WG, Slow Light Structuredependent < 5 s < 20 ~ 1 pJ Medium Small Optical Resonator Small 1-100 ns << 1 ~ 1 pJ Medium Large Holographic ~ 50 s CMOS 200 ps 64 ms

~ 1 pJ Small Large

~ 1 fJ Very Small No

Show stopper Challenge

Tucker, PS 2007

Packet Switch Cross-Connect Technologies

Projected Energy Consumption AWG-Based
~5 pJ/bit

CMOS

~10 pJ/bit

3-stage Clos
AWGs Wavelength (1.5 pJ/bit) Converters Optical Interconnects (1 pJ/bit) >100 pJ/bit (10 pJ/bit per gate)

Chips (2 pJ/bit)

SOA-Based
Benes Array

Tucker, JLT 2005, OSN 2008

Benes

Observations on Optical Packet Switching

No viable optical buffering technology in sight Optical switch fabrics may become competitive with CMOS Not clear whether optical packet switching will solve the energy bottleneck problem

Summary
Optical bypass reduces CAPEX and energy consumption - Promising future for optical cross-connects and ROADMs Energy bottleneck in routers is looming - More significant than the so-called electronic speed bottleneck Can optical packet switching overcome the energy bottleneck? - Optical buffering is currently a show-stopper Think Energy per Bit