Deep-Dive on Container

Networking Architectures

Frans Van Rooyen

Infrastructure Architect
Adobe
@jfvanrooyen
Agenda
Container Networking

• Power of Containers

• Single Host

• Multiple Hosts

• What else I should know?

2 © Copyright 2017 Dell Inc.

What’s the problem - Deploying an App

Apache

My App

Repo
Glibc

Quality Engineering Beta Production

3 © Copyright 2017 Dell Inc.

Deploying a Docker App

App Docker
Repo
Docker Server

Quality Engineering Beta Production

App App App App App App App App

Docker Server Docker Server Docker Server

4 © Copyright 2017 Dell Inc.

Foundations – Single Host

When you install Docker, it creates three networks automatically.

Bridge: The bridge network represents the docker0 network present in all Docker installations. Unless you
specify otherwise with the docker run --network=<NETWORK> option, the Docker daemon connects
containers to this network by default.

None: The none network adds a container to a container-specific network stack. That container lacks a
network interface.

Host: The host network adds a container on the hosts network stack. You’ll find the network configuration
inside the container is identical to the host.

5 © Copyright 2017 Dell Inc.

The default Bridge network in detail

6 © Copyright 2017 Dell Inc.

User-defined networks

• You can create a new bridge network, overlay

network or MACVLAN network.

• You can also create a network plugin or remote

network written to your own specifications.

• You can create multiple networks.

• You can add containers to more than one

network. Containers can only communicate within
networks but not across networks.

• A container attached to two networks can

communicate with member containers in either
network.

7 © Copyright 2017 Dell Inc.

What happens when we do this on thousands of
hosts…

Not Simple: Lots of manual configuration per host

Not Scalable: Hard to keep track off and maintain

Not Secure: No ability to define policies

8 © Copyright 2017 Dell Inc.

Overlay

• Overlays use networking tunnels to deliver

communication across hosts

• This allows containers to behave as if they are on the

same machine by tunneling network subnets from one
host to the next; in essence, spanning one network
across multiple hosts

• Many tunneling technologies exist, such as virtual

extensible local area network (VXLAN)

9 © Copyright 2017 Dell Inc.

Issues with Overlays

Not Simple: Complex to deploy and operate

Limited Scalable: Limitations on controllers

Performance: Extra hot because of encapsulation

10 © Copyright 2017 Dell Inc.

Underlay
Underlay network drivers expose host interfaces (i.e., the
physical network interface at eth0) directly to containers or
VMs running on the host.

MACvlan
Ipvlan
Direct Routing
Fan Networking
Point-to-Point

11 © Copyright 2017 Dell Inc.

Direct Routing

Simple:
L3 + BGP FTW

Scalable:
BGP?

Performance:
NO L2!

Secure: Policy based routing

12 © Copyright 2017 Dell Inc.

Public Services? (Service Discovery)

New service comes up at random port

How do we get to it programmatically?
Tools used to do this:
Etcd
Consul
Zookeeper

13 © Copyright 2017 Dell Inc.

Load Balancing in a Micro-service World
Internal
HAProxy
Ngnix
Marathon-LB

External
AVI

14 © Copyright 2017 Dell Inc.

Security for Containers

• Cilium is open source software for providing

and transparently securing the network
connectivity between application services
deployed using Linux container management
platforms like Docker and Kubernetes.

• At the foundation of Cilium is a new Linux

kernel technology called eBPF, which
enables the dynamic insertion of BPF
bytecode into the Linux kernel. Cilium
generates individual BPF programs for each
container to provide networking, security and
visibility.

15 © Copyright 2017 Dell Inc.

A word about Orchestrators
DC/OS, K8S, Swarm

16 © Copyright 2017 Dell Inc.

Adobe Use Case – Project Ethos
Consumer
Developer

Service Discovery
LB

Build and Deploy

Dev Cluster QE Cluster Prod Cluster

Platform

CoreOS CoreOS CoreOS

Infrastructure
AWS Azure Private

17 © Copyright 2017 Dell Inc.