Chapter 1. Section 1.

3, "Understanding Improvement"
This is "Introduction to DevOps", I'm the instructor, John Willis, @botchagalupe on
Twitter.
So, one of the things that we talk about also in DevOps is this concept of
"unicorns and horses".
So earlier, in the last section, or the first section, we actually talked about the
Google and Amazon and how they deploy...
Well, here's some real numbers: in 2015, Google has stated that they do somewhere
about 5,000 code commits a day, 75 million test cases per day;
Amazon, 136 thousand code deploys per day, 15 million per year;
Netflix, 500 code deploys per day; Etsy, hundreds plus.
So, a lot of these companies ... but we do consider these kind of the unicorns.
So, again, we want to focus that there is no single prescriptive or definition
model in DevOps, but
and I would argue that, arguing and debating for a definition, is a waste of time.
But what we really need to do is look for patterns, anti-patterns and patterns.
So ,if we can understand anti-patterns that make, that kind of statistically, or
more often, define low-performing organizations,
then, how do we go ahead and look at those for understanding how we can improve.
And then, we also want to look at best practice patterns
as learning opportunities as well. So, the key here is less about looking for a
definition or specific canonical model
and more about looking at, you know, kind of maybe any patterns in your
organization, best practices from other organizations or literature or this
course ...
and, ultimately, understanding how to improve.
You know, one of the things about improvement we talked about and, we'll talk more
about this, is the Lean influence, or Japan's influence on DevOps and it's
significant.
But, a couple of things ...and if I had to just kind of drive a lot of this
influence from the manufacturing model
of, you know, of kind of an automobile in a line to software in a pipeline,
there's a couple of Japanese terms that I think are very helpful, that are kind of
cornerstones of
the Toyota Production System and, ultimately, Lean: Muda, Muri,
Muda, Mura, Muri, Kaizen and Kata. So, let's go through these.
So, Muda is about waste; so, a lot of what we learned from Lean is to eliminate
waste wherever possible.
We do this in DevOps all the time. We try to automate. We try to remove manual
steps.
You know, if we made the assumption that Knight Capital
possibly, was manually installing servers and missed this eighth one, right?
That would be ... we would automate that. That would be waste . The waste is the
fact that a person might have been using a checklist actually, to install software
and create systems.
Mura is about flow. It's about evenness. A lot of what we do in our pipelines is
about getting that pipeline flow consistent.
We'll talk a little bit about some management scientists, particularly one called
Eliyahu Goldratt, who wrote "The Goal".
He ultimately defined the concept of Theory of Constraints and Bottlenecks.
And so, a lot about flow is understanding bottlenecks
in the pipeline, understanding how global optimization always trumps over local
optimization.
And then, you have Muri, which is stress.
And, if you kind of tie all these together, you're kind of .... your waste, flow,
and stress ... so, waste and flow work brilliantly together, right? Because you're
reducing waste,
but you're doing it in an evenness fashion.
So, you're not just fixing some local waste. You're looking at the global
performance
of the flow, to determine, you know, where do you redress ... what do you do. So
it's kind of a systems thinking.
And what this does, this reduces stress on the system, allows everything to flow
quicker.
And then, we have these too, last two: Kaizen, which is basically Japanese for
continuous improvement.
We're always trying to create like a Kaizen event.
We're always thinking about, you know, everything is like improvement.
Where he talks about this concept of really turning all the kind of Muda, Mura,
Muri, and Kaizen into memory muscle, very much like karate
or Kata is also described for Kabuki theatre, which is this repetitive actions that
become memory muscle.
So, the Toyota Production System, in their greatest form,
were able to optimize even this flow and eliminate waste and reduce stress on the
system
through continuous improvement, to a point where it was second nature,
it was Kata, very much like the way a dancer in Kabuki or a karate expert can block
...
block a punch, without even thinking about it.
So, a couple of other things about understanding improvement: high-performance
organizations, in general, make work visible.
You'll often see there a storyboard or a Kanban board with stickies or an
electronic version.
Making work visible allows you to understand and see the bottlenecks, allows you to
see the flow.
It can see the waste. A Kanban is a fantastic tool, but there are things like
Scrumban, there are all sorts of variants.
And then, when you see the work, the ability to manage the flow, because you can
actually understand the work-in-process.
WIP limits. You can't manually see WIP limits, so the workflow you're creating is
you want to basically create the evenness by creating buffers.
So, a lot of what Theory of Constraints is about is making sure you have buffers.
DevOps examples of this are companies that put a lot of buffer time.
For example, Google has a great example: for the longest time, they called it the
20% time. Everybody was able to work on 20%. You'll see,
a lot of companies will actually go ahead and have hackathons one day a week
and again, these are forms of buffers. And also, high-performing organizations,
great high-trust environments.
They're very collaborative and we'll look at the Ron Westrum model, degenerative,
right?
We encourage collaboration.
And then, we have high-performing organizations that have a healthy attitude
towards failure,
we'll spend more time on failure, understanding failure and how it makes you more
resilient, as opposed to the fear of failure,
We'll spend a fair amount of time discussing those kind of models and how you can
improve
by embracing failure. And it goes back to that scientific method
or the Deming cycle: if everything is kind of a scientific method or in the form of
a Plan-Do-Check-Act,
there really is no such thing as a failure.
There is basically a plan you do ... you implement ... you check the results and
you decide what to do based on the results.
So, it's not just the unicorns though, that have these kind of examples of
improvement and
organizations that actually continuously try to improve.
Ticketmaster, for example, there's a presentation I've given, I got a link to a lot
of the DevOps Enterprise, in fact, all of DevOps Enterprise Summit videos.
Ticketmaster, 98% reduction in MTTR and this was based on them moving fast.
So, not only the unicorns are moving fast and getting better resilience.
These companies that have been around ...
I mean, Nordstrom has been around for a hundred years. They get ... there is a
presentation, where 20% across the board ... shorter lead time
Target: full stack deploys from three months to minutes.
USAA releases from 28 days to7 days.
And all these stories include examples of fast and reliability.
Ing: 500 application teams doing DevOps.
At Target, they've created this thing called the DevOps Dojo.
It's 15,000 square feet and their headquarters is in Minnesota
and they actually bring groups in from all over the organization, to immerse them
into this kind of DevOps mindset.
CSG: they have a presentation where they show they went from 200 incidents per
release to 18.
Faster and more reliable.
These two books I think they're pretty important for this course:
one is a novel called "The Phoenix Project" by Gene Kim.
It is actually a rewrite of the original Eliyahu Goldratt, "The Goal", whom I've
referenced as the father of the Theory of Constraints.
Eliyahu Goldratt wrote a novel about constraints in a manufacturing plant,
automated robots and enclaves.
Gene wanted to rewrite that story with a Java stack and a Java developer.
It's an excellent book and it's setting the foundation for a whole generation of
organizations, particularly enterprises, understanding DevOps.
Myself, Gene Kim, Jez Humble, and Patrick Dubois have been working for quite a few
years on this book called
"The DevOps Handbook".
The intent was to be a prescriptive guide for "The Phoenix Project".
We finally have released it. A lot of the material in this course
basically comes from the ideas in "The DevOps Handbook".
I would highly suggest reading "The Phoenix Project" at the end of this chapter.
I know that sounds like a big ask, but if you were taking a college course, that
wouldn't be.
It's optional, whether you want to buy "The DevOps Handbook". There is basically
a couple ways you could do this: one, you can buy "The DevOps Handbook" after you
take the course.
So you can buy "The DevOps Handbook and then look for the kind of associated
sections as we go through different items.
Or, three, if you really want a deep dive, then read "The DevOps Handbook" right
now,
before you go into any other section.