OpenStack Networking

Paul Sim
Cloud Consultant
paul.sim@canonical.com
Index

Network as a Service : Neutron

Nova-network

Neutron - OpenvSwitch plugin VLAN

Neutron - OpenvSwitch plugin GRE

Neutron - Software Defined Networking

Neutron - Modular Layer 2

Network as a Service - Neutron
Nova-network

Flat DHCP Network Manager VLAN Network Manager

VM VM VM VM VM VM

G/W G/W

G/W dnsmasq Bridge 1 Bridge 2 dnsmasq

Bridge dnsmasq

vlan 100 vlan 101

eth0 eth0
 * Network NameSpace

without Network NameSpace with Network NameSpace

Process Process Process

Process Process Process

Process
Process
Share

BMW Ford Benz

Network Resources NameSpace NameSpace NameSpace

Routing table Network Network Network

Resources Resources Resources

Address

Netfilter rules Network Resources

eth0 eth1 eth2 eth0 eth1 eth2

Network NameSpace provides isolation of the system resources associated with networking. Thus, each network
namespace has its own network devices, IP addresses, IP routing tables, /proc/net directory, port numbers, and so on.
- http://lwn.net/Articles/531114/
 Installation - OpenvSwitch plugin VLAN, GRE

External network 192.168.122.0/24

eth0 eth0 eth0 eth0

Controller node Network node Compute node - 1 Compute node - 2

Neutron server
Neutron Neutron
Nova Keystone
Neutron openvswitch-plugin openvswitch-plugin
openvswitch-plugin
Glance Horizon Neutron metadata- Nova compute Nova compute
agent
Neutron L3/dhcp-
agent

eth1 eth2 eth1 eth2 eth1 eth2 eth1 eth2

Management 192.168.20.0/24

Data 192.168.10.0/24
Network Topology

ext_net : external network - 192.168.122.0/24

net_proj_one : user_one tenant - 50.50.1.0/24
net_proj_two : user_one tenant - 50.50.2.0/24
net_proj_new : user_new tenant - 60.60.1.0/24
Big picture - Neutron OVS plugin VLAN
OpenStack Havana OpenvSwitch plug-in VLAN mode
- LibvirtGenericVIFDriver

Network node Compute node - 1

net_proj_one net_proj_two net_proj_new

VM VM VM
tap~ tap~ tap~

tap~ tap~ tap~

qr~ qr~ qr~
tag: 1 tag:2 tag:2

br-int int-br-eth1
int-br-eth1 br-int

qg~ qg~ qg~ phy-br-eth1 Data phy-br-eth1

192.168.10.0/24
br-ex br-eth1 eth1 eth1 br-eth1

eth0

OVS port
qg~~~ : external gateway interface
OVS Bridge
qr~~~ : virtual router interface
Neutron OVS plugin VLAN - Compute node
OpenStack Havana OpenvSwitch plug-in VLAN mode
- LibvirtGenericVIFDriver

Compute node - 1

VM VM VM VM
br-eth1

tap~ tap~ tap~ tap~

eth1

tag: 1 tag:2 tag:2 tag:3

veth pair
phy-br-eth1 int-br-eth1 br-int

Packet conversion
mod_vlan_vid

mod_vlan_vid Security Group[1]

 Neutron OVS plugin VLAN - Compute node

Packet conversion
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-eth1
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=90455.716s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=2 actions=drop
cookie=0x0, duration=89606.096s, table=0, n_packets=9484, n_bytes=2312018, priority=4,in_port=2,dl_vlan=1
actions=mod_vlan_vid:1024,NORMAL
cookie=0x0, duration=90456.248s, table=0, n_packets=6813, n_bytes=1325511, priority=1 actions=NORMAL

janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-int

NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=90458.482s, table=0, n_packets=64, n_bytes=4644, priority=2,in_port=1 actions=drop
cookie=0x0, duration=89608.755s, table=0, n_packets=6499, n_bytes=1283680, priority=3,in_port=1,dl_vlan=1024
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=90459.075s, table=0, n_packets=9820, n_bytes=2323195, priority=1 actions=NORMAL

openvswitch-agent.log
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,
idle_timeout=0,priority=3,in_port=1,dl_vlan=1024,actions=mod_vl
an_vid:1,normal']
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-eth1', 'hard_timeout=0,
idle_timeout=0,priority=4,in_port=2,dl_vlan=1,actions=mod_vlan
_vid:1024,normal']
Neutron OVS plugin VLAN - Network node
OpenStack Havana OpenvSwitch plug-in VLAN mode
- LibvirtGenericVIFDriver
Network node
tap~ tap~ tap~

Namespcae Namespcae Namespcae

qr~ qr~ qr~

br-eth1

eth1
qg~ qg~ qg~

veth pair

br-int int-br-eth1 phy-br-eth1

br-ex

eth0
net_proj_one Packet conversion
mod_vlan_id
net_proj_two
Floating-IP(NAT) mod_vlan_id
net_proj_new
 Neutron OVS plugin VLAN - Network node

Packet conversion
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=7370.307s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=6 actions=drop
cookie=0x0, duration=7368.424s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=2048 actions=mod_vlan_vid:2,NORMAL
cookie=0x0, duration=7367.991s, table=0, n_packets=764, n_bytes=191460, priority=3,in_port=6,dl_vlan=1024 actions=mod_vlan_vid:3,
NORMAL
cookie=0x0, duration=7369.073s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=500 actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=7370.924s, table=0, n_packets=549, n_bytes=104066, priority=1 actions=NORMAL

janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-eth1

NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=7373.826s, table=0, n_packets=14, n_bytes=1104, priority=2,in_port=2 actions=drop
cookie=0x0, duration=7372.725s, table=0, n_packets=13, n_bytes=922, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:500,NORMAL
cookie=0x0, duration=7371.663s, table=0, n_packets=519, n_bytes=103966, priority=4,in_port=2,dl_vlan=3 actions=mod_vlan_vid:1024,
NORMAL
cookie=0x0, duration=7372.09s, table=0, n_packets=9, n_bytes=634, priority=4,in_port=2,dl_vlan=2 actions=mod_vlan_vid:2048,NORMAL
cookie=0x0, duration=7374.384s, table=0, n_packets=764, n_bytes=191460, priority=1 actions=NORMAL
* LibvirtHybridOVSBridgeDriver

libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
Big picture - Neutron OVS plugin GRE
OpenStack Havana OpenvSwitch plug-in GRE tunneling
- LibvirtGenericVIFDriver

Network node Compute node - 1

net_proj_one net_proj_two net_proj_new

tap~ tap~ tap~ Data

VM VM
192.168.10.0/24

qr~ qr~ qr~ Tunnel

br-tun
br-tun tap~ tap~

gre~

gre~
tag: 1 tag:2
patch

br-int patch

patch
patch br-int
qg~ qg~ qg~

br-ex
eth0

OVS port
qg~~~ : external gateway interface
OVS Bridge
qr~~~ : virtual router interface
Neutron OVS plugin GRE - Compute node
OpenStack Havana OpenvSwitch plug-in GRE tunneling
- LibvirtGenericVIFDriver

Compute node - 1

VM VM VM VM
Tunnel
br-tun
gre~

tap~ tap~ tap~ tap~

tag: 1 tag:2 tag:2 tag:3

patch patch br-int

Packet conversion
mod_vlan_vid
Security Group[1]
set_tunnel id
 Neutron OVS plugin GRE - Compute node

Packet conversion

janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-tun

NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=87770.027s, table=0, n_packets=0, n_bytes=0, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:
00 actions=mod_vlan_vid:1,output:1
cookie=0x0, duration=87770.09s, table=0, n_packets=8786, n_bytes=1893724, priority=4,in_port=1,dl_vlan=1
actions=set_tunnel:0x1,NORMAL
cookie=0x0, duration=87769.693s, table=0, n_packets=3031, n_bytes=617650, priority=3,tun_id=0x1,dl_dst=fa:16:3e:db:08:63
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=87769.966s, table=0, n_packets=6320, n_bytes=4432680, priority=3,tun_id=0x1,dl_dst=fa:16:3e:e0:73:95
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=87771.753s, table=0, n_packets=2921, n_bytes=951454, priority=1 actions=drop
Neutron OVS plugin GRE - Network node
OpenStack Havana OpenvSwitch plug-in GRE tunneling
- LibvirtGenericVIFDriver
Network node
tap~ tap~ tap~

Namespcae Namespcae Namespcae

qr~ qr~ qr~

Tunnel

br-tun

gre~
qg~ qg~ qg~

patch
br-int patch

br-ex

eth0
net_proj_one Packet conversion
set_tunnel id
net_proj_two
Floating-IP(NAT) mod_vlan_id
net_proj_new
 Neutron OVS plugin GRE - Network node

Packet conversion
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=474674.446s, table=0, n_packets=7899, n_bytes=2572502, priority=3,tun_id=0x3,dl_dst=01:00:00:00:00:00/01:00:00:00:00:
00 actions=mod_vlan_vid:2,output:1
cookie=0x0, duration=473163.123s, table=0, n_packets=7876, n_bytes=2565284, priority=3,tun_id=0x4,dl_dst=01:00:00:00:00:00/01:00:00:00:00:
00 actions=mod_vlan_vid:3,output:1
cookie=0x0, duration=633937.826s, table=0, n_packets=10543, n_bytes=3426814, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:
00 actions=mod_vlan_vid:1,output:1
cookie=0x0, duration=473163.329s, table=0, n_packets=16484, n_bytes=3348666, priority=4,in_port=1,dl_vlan=3 actions=set_tunnel:0x4,
NORMAL
cookie=0x0, duration=474674.541s, table=0, n_packets=16864, n_bytes=3389132, priority=4,in_port=1,dl_vlan=2 actions=set_tunnel:0x3,
NORMAL
cookie=0x0, duration=633937.905s, table=0, n_packets=62044, n_bytes=37320316, priority=4,in_port=1,dl_vlan=1 actions=set_tunnel:0x1,
NORMAL
cookie=0x0, duration=472911.069s, table=0, n_packets=16335, n_bytes=3551350, priority=3,tun_id=0x4,dl_dst=fa:16:3e:89:fd:ce
actions=mod_vlan_vid:3,NORMAL
cookie=0x0, duration=474336.184s, table=0, n_packets=16360, n_bytes=3560332, priority=3,tun_id=0x3,dl_dst=fa:16:3e:d8:d5:29
actions=mod_vlan_vid:2,NORMAL
cookie=0x0, duration=474674.351s, table=0, n_packets=525, n_bytes=52427, priority=3,tun_id=0x3,dl_dst=fa:16:3e:69:ca:97
actions=mod_vlan_vid:2,NORMAL
cookie=0x0, duration=473162.912s, table=0, n_packets=197, n_bytes=19365, priority=3,tun_id=0x4,dl_dst=fa:16:3e:d6:b8:07
actions=mod_vlan_vid:3,NORMAL
cookie=0x0, duration=633937.746s, table=0, n_packets=6207, n_bytes=630043, priority=3,tun_id=0x1,dl_dst=fa:16:3e:c7:ec:bd
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=474794.912s, table=0, n_packets=36912, n_bytes=7440964, priority=3,tun_id=0x1,dl_dst=fa:16:3e:8b:a6:d7
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=636252.069s, table=0, n_packets=163, n_bytes=36046, priority=1 actions=drop
Neutron OVS plugin Security Group - VLAN, GRE

FORWARD

neutron-filter-top

neutron-openvswi-local
Security group is applied here
neutron-openvswi-FORWARD

neutron-openvswi-sg-chain

neutron-openvswi-iTAP_NUMBER

neutron-openvswi-sg-fallback

neutron-openvswi-oTAP_NUMBER

neutron-openvswi-sg-fallback
 Neutron OVS plugin Security Group - VLAN, GRE
Chain neutron-openvswi-sg-chain (4 references)
target prot opt source destination
neutron-openvswi-i21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap21767f1f-45 --physdev-is-bridged
neutron-openvswi-o21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap21767f1f-45 --physdev-is-bridged
neutron-openvswi-i7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap7903fd30-74 --physdev-is-bridged
neutron-openvswi-o7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap7903fd30-74 --physdev-is-bridged
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0

Chain neutron-openvswi-i7903fd30-7 (1 references)

target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN icmp -- 0.0.0.0/0 0.0.0.0/0
RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
RETURN udp -- 50.50.1.3 0.0.0.0/0 udp spt:67 dpt:68
neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

Chain neutron-openvswi-o7903fd30-7 (2 references)

target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:DB:08:63
RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
DROP all -- !50.50.1.2 0.0.0.0/0
DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN all -- 0.0.0.0/0 0.0.0.0/0
neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

[1] Note, OpenStack uses iptables rules on the TAP devices such as tap~~ to implement security groups.
However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an
Open vSwitch port.
Neutron OVS plugin NameSpace - VLAN, GRE

janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1

qg-fa243f49-d6 Link encap:Ethernet HWaddr fa:16:3e:9f:4b:63

inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe9f:4b63/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

qr-bc654dc2-f1 Link encap:Ethernet HWaddr fa:16:3e:c7:ec:bd

inet addr:50.50.1.1 Bcast:50.50.1.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fec7:ecbd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 route

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.122.1 0.0.0.0 UG 0 0 0 qg-fa243f49-d6
50.50.1.0 * 255.255.255.0 U 0 0 0 qr-bc654dc2-f1
192.168.122.0 * 255.255.255.0 U 0 0 0 qg-fa243f49-d6
 Neutron OVS plugin Floating-IP(NAT) - VLAN, GRE

NameSpace
janghoon@Network-node:~$ sudo ip netns show
qdhcp-4c2f2346-ffaa-41a0-ab76-34cadf0163f5
qrouter-e1b88ce4-51e9-4744-be80-d70d04c6a59b
qdhcp-c19e22a0-1700-4b3b-91e5-2c961ef0a353
qrouter-244fff3f-f935-4bdd-949d-739f1ce81dd0
qdhcp-f37b681a-4be8-47b8-8063-3d17d24ee1ae
qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0

Floating-IP(NAT)
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 iptables -L -n -t nat
Chain neutron-l3-agent-PREROUTING (1 references)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697
DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.1.2

Chain neutron-l3-agent-float-snat (1 references)

target prot opt source destination
SNAT all -- 50.50.1.2 0.0.0.0/0 to:192.168.122.51

Chain neutron-l3-agent-snat (1 references)

target prot opt source destination
neutron-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0
SNAT all -- 50.50.1.0/24 0.0.0.0/0 to:192.168.122.50
 Installation - SDN

External network 192.168.122.0/24

eth0 eth0 eth0 eth0

Controller node Network node Compute node - 1 Compute node - 2

Quantum plugin
ryu-agent Quantum plugin Quantum plugin
Nova Keystone
ryu-agent ryu-agent
Ryu-manager

Glance Horizon Quantum Nova compute Nova compute

metadata-agent
Quantum L3/dhcp-
Quantum - Server
agent

eth1 eth2 eth1 eth2 eth1 eth2 eth1 eth2

Management 192.168.20.0/24

Data 192.168.10.0/24
Overview

Controller node Network node

Quantum - Server Ryu-manager

AMQP

REST API

Compute node Compute node OpenFlow

OVSDB protocol
ryu-agent ryu-agent

ovs-vswitchd ovs-vswitchd
Big picture - Neutron Ryu plugin
OpenStack Grizzly Ryu plugin GRE tunneling

Network node Compute node - 1

net_proj_one net_proj_two net_proj_new

VM VM
ns~ ns~ ns~
Data
192.168.10.0
/24 tap~ tap~
qr~ qr~ qr~
tag: 1 tag:2
Tunnel

gre~

gre~
br-int br-int

qg~ qg~ qg~

br-ex
eth0

OVS port
qg~~~ : external gateway interface
OVS Bridge
qr~~~ : virtual router interface
Neutron Ryu plugin - Compute node
OpenStack Grizzly Ryu plugin GRE tunneling

Compute node - 1

VM VM VM VM

tap~ tap~ tap~ tap~

Tunnel
gre~

br-int

Packet conversion

set_tunnel id
Security Group[1]
 Neutron Ryu plugin - Compute node

Flow table
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=90146.068s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=3 actions=drop
cookie=0x0, duration=90146.989s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=4 actions=drop
cookie=0x0, duration=90146.068s, table=0, n_packets=3273, n_bytes=643066, tun_id=0x2,in_port=4
actions=resubmit(,2)
cookie=0x0, duration=90146.068s, table=0, n_packets=4720, n_bytes=1164172, in_port=3,dl_src=fa:16:3e:cf:dc:42
actions=set_tunnel:0x2,resubmit(,1)
cookie=0x0, duration=90146.068s, table=1, n_packets=6, n_bytes=468, priority=8192,tun_id=0x2 actions=resubmit(,2)
cookie=0x0, duration=90146.068s, table=1, n_packets=1504, n_bytes=483460, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:
ff:ff:ff actions=output:4,resubmit(,2)
cookie=0x0, duration=90146.068s, table=1, n_packets=3000, n_bytes=659756, tun_id=0x2,dl_dst=fa:16:3e:a2:0e:f1
actions=output:4,resubmit(,2)
cookie=0x0, duration=90146.068s, table=1, n_packets=210, n_bytes=20488, tun_id=0x2,dl_dst=fa:16:3e:ee:aa:8c
actions=output:4,resubmit(,2)
cookie=0x0, duration=90146.068s, table=2, n_packets=3216, n_bytes=680712, priority=8192,tun_id=0x2 actions=drop
cookie=0x0, duration=90146.068s, table=2, n_packets=1610, n_bytes=487912, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:
ff:ff:ff actions=output:3
cookie=0x0, duration=90146.068s, table=2, n_packets=3167, n_bytes=638614, tun_id=0x2,dl_dst=fa:16:3e:cf:dc:42
actions=output:3
Neutron Ryu plugin - Network node
OpenStack Grizzly Ryu plugin GRE tunneling
Network node

Namespace Namespace
Namespace

Namespace Namespace Namespace

ns~ ns~
ns~
qr~ qr~ qr~
veth pair

qg~ qg~ qg~

tap~ tap~ tap~ tap~ tap~ tap~

gre~
br-int

tap~ tap~ tap~

br-ex
eth0
net_proj_one Packet conversion
set_tunnel id
net_proj_two
Floating-IP(NAT)
net_proj_new
 Neutron Ryu plugin - Network node

Flow table
janghoon@network:~$ sudo ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=144003.213s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=3 actions=drop
cookie=0x0, duration=142257.013s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=4 actions=drop
cookie=0x0, duration=144003.261s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=2 actions=drop
cookie=0x0, duration=142256.093s, table=0, n_packets=7335, n_bytes=1825414, tun_id=0x2,in_port=4
actions=resubmit(,2)
cookie=0x0, duration=144003.261s, table=0, n_packets=4748, n_bytes=977976, in_port=2,dl_src=fa:16:3e:a2:0e:f1
actions=set_tunnel:0x2,resubmit(,1)
cookie=0x0, duration=144003.213s, table=0, n_packets=544, n_bytes=58344, in_port=3,dl_src=fa:16:3e:ee:aa:8c
actions=set_tunnel:0x2,resubmit(,1)
cookie=0x0, duration=144003.261s, table=1, n_packets=27, n_bytes=5010, priority=8192,tun_id=0x2 actions=resubmit(,2)
cookie=0x0, duration=142256.093s, table=1, n_packets=113, n_bytes=4746, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:ff:
ff:ff actions=output:4,resubmit(,2)
cookie=0x0, duration=142256.093s, table=1, n_packets=4914, n_bytes=998000, tun_id=0x2,dl_dst=fa:16:3e:cf:dc:42
actions=output:4,resubmit(,2)
cookie=0x0, duration=144003.261s, table=2, n_packets=5177, n_bytes=1031490, priority=8192,tun_id=0x2 actions=drop
cookie=0x0, duration=144003.253s, table=2, n_packets=504, n_bytes=49439, tun_id=0x2,dl_dst=fa:16:3e:ee:aa:8c
actions=output:3
cookie=0x0, duration=144003.261s, table=2, n_packets=4733, n_bytes=1041550, tun_id=0x2,dl_dst=fa:16:3e:a2:0e:f1
actions=output:2
cookie=0x0, duration=144003.261s, table=2, n_packets=2495, n_bytes=769266, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:
ff:ff:ff actions=output:2,output:3
Neutron Ryu plugin Security Group

FORWARD

quantum-filter-top

quantum-ryu-agen-local
Security group is applied here
quantum-ryu-agen-FORWARD

quantum-ryu-agen-sg-chain

quantum-ryu-agen-iTAP_NUMBER

quantum-ryu-agen-sg-fallback

quantum-ryu-agen-oTAP_NUMBER

quantum-ryu-agen-sg-fallback
 Neutron Ryu plugin Security Group

Chain quantum-ryu-agen-sg-chain (2 references)

target prot opt source destination
quantum-ryu-agen-ib7fa734b-e all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapb7fa734b-e0 --physdev-is-bridged
quantum-ryu-agen-ob7fa734b-e all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapb7fa734b-e0 --physdev-is-bridged
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0

Chain quantum-ryu-agen-ib7fa734b-e (1 references)

target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN tcp -- 192.168.228.122 0.0.0.0/0 tcp dpt:80
RETURN udp -- 50.50.2.2 0.0.0.0/0 udp spt:67 dpt:68
quantum-ryu-agen-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

Chain quantum-ryu-agen-ob7fa734b-e (2 references)

target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:CF:DC:42
RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
DROP all -- !50.50.2.4 0.0.0.0/0
DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN all -- 0.0.0.0/0 0.0.0.0/0
quantum-ryu-agen-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

[1] Note, OpenStack uses iptables rules on the TAP devices such as tap~~ to implement security groups,.
However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an
Open vSwitch port.
 Neutron Ryu plugin NameSpace

janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1

qg-afcc5de0-46 Link encap:Ethernet HWaddr fa:16:3e:62:e4:4b

inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe62:e44b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

qr-33616671-f3 Link encap:Ethernet HWaddr fa:16:3e:ee:aa:8c

inet addr:50.50.2.1 Bcast:50.50.2.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:feee:aa8c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 route

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.122.1 0.0.0.0 UG 0 0 0 qg-afcc5de0-46
50.50.2.0 * 255.255.255.0 U 0 0 0 qr-33616671-f3
192.168.122.0 * 255.255.255.0 U 0 0 0 qg-afcc5de0-46
 Neutron Ryu plugin Floating-IP(NAT)

Floating-IP(NAT)
janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 iptables -L -n -t nat
Chain quantum-l3-agent-PREROUTING (1 references)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697
DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.2.4

Chain quantum-l3-agent-float-snat (1 references)

target prot opt source destination
SNAT all -- 50.50.2.4 0.0.0.0/0 to:192.168.122.51

Chain quantum-l3-agent-snat (1 references)

target prot opt source destination
quantum-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0
SNAT all -- 50.50.2.0/24 0.0.0.0/0 to:192.168.122.50
 Ryu-Controller

Configuration - ryu.conf
[DEFAULT]
app_lists = ryu.app.gre_tunnel,ryu.app.quantum_adapter,ryu.app.rest,ryu.app.rest_conf_switch,ryu.app.rest_quantum,ryu.app.
rest_tunnel,ryu.app.tunnel_port_updater

wsapi_host = 0.0.0.0
wsapi_port = 8080
ofp_listen_host = 0.0.0.0
ofp_tcp_listen_port = 6633

quantum_url=http://192.168.20.10:9696
quantum_admin_username=quantum
quantum_admin_password=*********
quantum_admin_tenant_name=service
quantum_admin_auth_url=http://192.168.20.10:35357/v2.0
quantum_auth_strategy=keystone
quantum_controller_addr = tcp:192.168.20.11:6633
Neutron ML2

The Modular Layer 2 (ML2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety
of layer 2 networking technologies found in complex real-world data centers. It currently works with the existing
openvswitch, linuxbridge, and hyperv L2 agents, and is intended to replace and deprecate the monolithic plugins
associated with those L2 agents.

Neutron

ML2 Plugin

TypeDriver MechanismDriver

Arista

Cisco Nexus
OpenDaylight
OpenvSwitch

Hyper-V
VLAN GRE VxLAN Flat

pSwitch

TypeDriver : TypeDrivers maintain any needed type-specific network state, and perform provider network validation
and tenant network allocation.
MechanismDriver : The MechanismDriver is responsible for taking the information established by the TypeDriver and
ensuring that it is properly applied given the specific networking mechanisms that have been enabled.

https://wiki.openstack.org/wiki/Neutron/ML2
Neutron ML2

eth0 eth0 eth0

Network node Compute node - 1 Compute node - 2

Neutron ML2-agent
Neutron Neutron
ML2-agent ML2-agent
Neutron server

Neutron metadata- Nova compute Nova compute

agent
Neutron L3/dhcp-
agent

eth1 eth2 eth1 eth2 eth1 eth2

* Another option

Cisco and Canonical are collaborating

to offer customers the Nexus 1000V
virtual networking solution on Ubuntu
Linux & Ubuntu OpenStack cloud
orchestration for the first time.
The solution will enable Nexus 1000V
customers to embrace Ubuntu
OpenStack, the largest commercial
distribution of the open source cloud
platform.

http://www.cisco.
com/c/en/us/products/collateral/switches/nexu
s-1000v-kvm/solution-overview-c22-730808.
html