WTF is a microservice?

Rafael Schloming
Co-Founder & Chief Architect
History
Datawire

Founded in 2014
Focused on microservices

Me

Lots of distributed systems experience

Starting from zero with microservices

datawire.io 2
What is a microservice?
Wikipedia: ...no industry consensus
...implementation approach for SoA
...processes that communicate with each other to fulfill a goal
...Naturally enforces a modular structure...

Everything else:
Volumes of essays good, bad, and ugly...

datawire.io 3
 Three aspects of Microservices

Process

People

Technology

datawire.io 4
 From Three Sources

Experts Bootstrapping

Migrating

datawire.io 5
Starting Point
Technical:

An application composed of a network of small services

Building your application from microservices forces you to create clear
boundaries, better abstractions, ...

Process:

???

People:

???

datawire.io 6
The Expert Source
Read just about every firsthand story out there

Went to conferences

Talked to everyone we could

Started the practitioner summit

And armed with a little bit of knowledge, we started filling in our picture

datawire.io 7
People Picture
Developer Happiness/Tooling/Platform Team

Builds the infrastructure

Service teams

Builds the features

datawire.io 8
Technical Picture
Control Plane Reference Architecture

Service Discovery
Logging + Metrics
Configuration
Smart Endpoints

Traffic Layer

HTTP
RPC
Messaging

datawire.io 9
First Picture
Technical:

A network of small services

Connected via a control plane and traffic layer

Process:

???

People:

Platform team and service teams

datawire.io 10
The Bootstrap Perspective
Five engineers building an out of the box control plane...

Ingest interesting application level events:

start, stop, heartbeat events
log messages

Store them in an appropriate piece of infrastructure:

Service registry
Log store

Transform and Present:

Realtime view of: routing table, service health
Historic view of: request traces, ...

datawire.io 11
 Ubiquitous Data Processing Pipeline

Ingest Source of Truth Transform Present

Template for many data driven businesses

datawire.io 12
V1: Started with Discovery
Requirements:
highly available
low throughput
low latency
low operational complexity
able to survive a complete restart
capable of handling spikes

Initial Choices:
vert.x + hazelcast
websockets
smart clients
auth0 + python shim

Total Services: 2
datawire.io 13
V2: Added Tracing (PoC)
Requirements:
high throughput
highish latency ok
cannot impact application

Initial choices:
vert.x, hazelcast (only retained transient buffer of last 1000 log messages)
websockets
smart circular buffer minimized impact on application

Total Services: 3

datawire.io 14
V3: Added Persistence for Tracing
Requirements:
keep extended history
provide full text search
filtering, sorting, etc

Initial Choices:
elasticsearch for storage/search
query service

Total Services: 4

datawire.io 15
First hint of pain...
Rerouting data pathways:
touched multiple services
coupled changes

Poor local dev experience:

manually fire up and wire the whole fabric

Slow deployment pipeline:

bunched up changes

All this resulted in a big scary cutover

datawire.io 16
V4: Adding Persistence for Discovery
Requirements:
track errors associated with particular service nodes
store routing strategies

Initial Choices:
postgres (RDS) for persistence

Yet another big cutover enough is enough!

Lets fix our tooling once and for all...

datawire.io 17
Deployment Requirements
Stuff we had tried:

Deliver everything as a docker image

Still too much wiring to bootstrap the system
Use kubernetes for everything
Nice dev experience with minikube, but we use amazon services

Need to meet both dev & operational requirements

Fast dev cycle

Good visibility
Fast rollback
Ability to leverage commodity services

datawire.io 18
Deployment Redesign
Complete system definition in git
Contains all the information necessary to bootstrap the system from scratch in all of its operating
environments
System definition is well factored with respect to its environments
Abstract definition: my service needs postgres and redis
Development: service -> docker image, postgres -> docker image, redis -> docker image
Use minikube to run the whole system
Test: <same as dev for now>
Production: service -> docker image, postgres -> RDS, redis -> elasticache
Kubernetes cluster for stateless services
Tooling caters to the needs of each environment
Development: fast feedback cycle
Test: repeatable environments
Production: quick and safe updates/rollbacks
Tooling helps maintain environment parity

datawire.io 19
DevOps?
DevOps is presented as a solution to an organizational problem, but we all sat in the
same room

We were thinking about operational factors from day one:

throughput, latency, availability,

building a service, not a server

This forced us to follow an incremental process:

tooling for this process was inadequate

when we thought about the process it helped us figure out the tooling

datawire.io 20
Process: Architecture vs Development (SoA vs SoD)
Systems (their shape in particular) are traditionally architected

Architecture

lots of up front thinking

slow feedback cycle

Development

frequent small changes

quick feedback cycle
measure the impact at every step

Microservices are about enabling a developmental methodology for systems

datawire.io 21
Methodology for Developing Systems
Principles
small frequent changes
rapid feedback and good visibility

Applied to codebases:
Tooling for rapid feedback: compilers, incremental builds, test suites
Tooling for good visibility: printf, logging, debuggers, profilers

Applied to systems:
Key characteristics go beyond just logic and correctness
Performance within specified tolerance of the running system is a critical feature

Tests dont cut it anymore...

datawire.io 22
Update the Dev Cycle
Tests assess impact on correctness...

Build -> Test -> Deploy

We need a way to assess impact on the system

Build -> Test -> Assess Impact -> Deploy

How do you measure system level impact?

Measure impact against defined Service Level Objectives (SLOs):

throughput, latency, and availability (error rate)

datawire.io 23
Back to the Experts...
Canary Testing
Circuit Breakers
Dark Launching
Tracing
Metrics
Deployment

All ways to enable the dev cycle for running systems:

make small frequent changes

measure the impact on the running system
provide good visibility

datawire.io 24
Second Picture
Technical:

A network of small services

Scaffolding to safely enable small frequent changes

Process:

Service oriented Development

Small frequent changes with good visibility and feedback

People:

Platform team and service teams

datawire.io 25
The Migration Perspective
Variety of stages...

Monolith: django, rails, ...

Monolith++: mothership + several little ducklings
SoA-ish: small flock of services (maybe 5-10)
Inbetweeners

Some moving really slowly...

Months to create just one microservice

Some moving much faster

Whats the difference?

datawire.io 26
Migration is about people
Starting point: team vs tech

Picking a tech stack for the entire eng org to adopt is slow
lots of organizational friction
Replatforming/refactoring an entire existing monolith is slow
lots of organizational and orchestrational friction
Creating a relatively autonomous team to tackle a particular problem in the form
of a service

Growing pains: stability vs progress

some orgs hit a sticking point, some didnt

datawire.io 27
The People Picture: Dividing up the Work
The work has two aspects:
build the features (dev)
keeping the system running (ops)

You cant usefully divide up the work along these lines:

new features are the biggest source of instability (bugs)
separate roles creates misaligned incentives (devops)
yet a big part of the work is keeping things running

Microservices is about how to go about dividing up work:

break the big app into smaller ones
divide operational responsibility in a way that aligns incentives

datawire.io 28
Third Picture
Technical:

A network of small services

Scaffolding to quickly and safely enable small frequent changes

Process:

Service oriented Development

Small frequent changes with good visibility and feedback

People:

Dividing up the work

Service teams deliver features to users
Platform team supports service teams

datawire.io 29
 The Hard Way

1. Start with Tech

2. Reverse Engineer The Process + People
3. Make lots of mistakes along the way
4. Learn from them

datawire.io 30
 The Easy Way

1. Understand the principles of People and

Process
2. Use this as a framework to
a. pick tech that fits
b. learn from other people's mistakes

datawire.io 31
 Microservices Cheat Sheet (What, Why & How)
People
Microservices are a way to divide the work of
building a cloud application
This work falls into two categories:
Keep the system running (ops)
Build new features (dev).
Dividing work along these categories creates
conflicting incentives between progress and
stability. New features from dev eventually
become the biggest source of instability for
ops.
Unifying these roles (devops) allows you to
minimize the tradeoff between progress and
stability, but you now need to divide up the
work by dividing up the app. This results in a
network of services.
Give your dev teams operational responsibility!
Define service level objectives & agreements
for each service:
SLOs: throughput, latency, availability
SLAs: what happens when these arent met
Commoditize common operational overhead.
datawire.io
Process
Microservices are built from a process of
frequent small changes with rapid feedback
and good visibility
This is the application of the traditional dev
cycle to systems rather than codebases, and
for it to work, key system properties must
become a first class features for developers.
Extend the dev cycle to include a stage to
assess the impact on key system properties
(SLOs)
Build -> Test -> Deploy
Build -> Test -> Assess Impact -> Deploy
Technology
Microservices are an application that is made up
of a network of small services
This requires dev tooling to support quickly and
safely assessing system impact.
This requires fast deployment tooling and good
visibility into key system level properties:
Throughput
Latency
Availability (error-rate)
Depending on your system, this may require
tooling for:
Fancy request routing (for canary
testing, dark launching)
Start with a fast deployment pipeline that
incorporates basic system level metrics and
monitoring for each service.
32
 Questions?

datawire.io 33
 Microservices Cheat Sheet (What, Why & How)
People

Microservices are a way to divide the work of building a cloud application

Two aspects of work: keep it running (ops), build new features (dev)

Dividing by aspect creates conflicting incentives between progress and stability.

Unifying roles (devops) to minimize tradeoff... divide work by dividing the app

Give your dev teams operational responsibility!

Define service level objectives & agreements for each service:

SLOs: throughput, latency, availability
SLAs: what happens when these arent met

Commoditize common operational overhead.

datawire.io 34
 Microservices Cheat Sheet (What, Why & How)
Process

Microservices are built from a process of frequent small changes with rapid
feedback and good visibility

This is the application of the traditional dev cycle to systems rather than
codebases, and for it to work, key system properties must become a first class
features for developers.

Extend the dev cycle to include a stage to assess the impact on key system
properties (SLOs)

Build -> Test -> Deploy

Build -> Test -> Assess Impact -> Deploy

datawire.io 35
 Microservices Cheat Sheet (What, Why & How)
Technology

Microservices are an application that is made up of a network of small services

This requires dev tooling to support quickly and safely assessing system impact.

This requires fast deployment tooling and good visibility into key system level
properties:
Throughput
Latency
Availability (error-rate)

Depending on your system, this may require tooling for:

Fancy request routing (for canary testing, dark launching)

Start with a fast deployment pipeline that incorporates basic system level metrics
and monitoring for each service.

datawire.io 36
 DevOps: you cant split the work (along these lines)

Dev
DevOps

Ops

User User

datawire.io 37
 Features are the largest source of bugs

Dev

Ops

User Dev
Dev Dev

Ops

User

datawire.io 38
 Microservices: Divide the work by dividing the app

Dev Infra

Ops Dev Dev

Dev

User
User

datawire.io 39
 Dividing up Work

Dev Infra

Ops Dev Dev

Dev

User Dev
Dev Dev User

User

User

datawire.io 40
datawire.io 41