Release Management

Ship It!
Phyllis Kaiden
Release Infrastructure Manager
The Cobalt Group, Inc.

December 6, 2004

Outline
What

is Release Management?
Why does it matter?
How does it work?
Would you ship it?
What did we learn?

December 6, 2004

What is Release
Management?
Release Management Definition
Release Management Responsibilities
What is Configuration Management?
Product and Project Management

December 6, 2004

Release Management Definition

The process by which a product is made ready
for distribution to customers. Release
management is the coordination of all the
activities leading to and including product
release to customers.
The purpose of release management is to ensure
products are ready when promised.
Focus on software release management but
basics apply to all product releases.
December 6, 2004

Its Like Baking

Recipe the set of activities in order
Ingredients quality components
Variations Icing? Nuts?
Mixing executing the recipe steps
Baking time varies based on desired outcome
Is it done? knowing when its just right
Final product presentation, consumption

December 6, 2004

Baking & Releasing

Recipe the set of activities in order (the plan)
Ingredients quality components (content, schedule,
resources)
Variations Icing? Nuts? (special considerations)
Mixing executing the recipe steps (do the plan)
Baking time varies based on desired outcome
(priority of time to market, features, quality)
Is it done? knowing when its just right (assess
readiness against measurable criteria)
Final product presentation, consumption (packaging
and distribution)
December 6, 2004

What is a release?
Delivery of a system from a supplier to a
customer.
The system consists of a set of
authorized and integrated components.
The supplier is usually the development
organization.
The customer is the internal or external
recipient of the system.
December 6, 2004

Key Practices

Release Planning: Manage and publish release

schedule
Define deliverables included in a release
Build and Version the integrated release package
Manage dependencies across components
Coordinate activities across company
Control changes that would impact the release
schedule
Establish release criteria
Assess release readiness

December 6, 2004

Key Principle: Define Done

Deliverables
Dependencies
Avoid

scope creep but allow change

Release criteria

Objective
Quantifiable
Cross-functional
December 6, 2004

Components of a release
Golden

Build (software): CD-ROM, files

Documentation and Help
Training materials
Marketing collateral
Internal documents
Customer Notification
Release Notes
December 6, 2004

10

Types of Products

Consumer: shrink-wrapped, commercial off-the-shelf

(COTS), e.g. MS Word
Open-source: download as needed, e.g. MySQL
Hosted: Application Service Provider (ASP), e.g. Yahoo
mail
Corporate: Enterprise Resource Planning (ERP), e.g.
Oracle Financials
Corporate desktop: enterprise distribution of desktop
applications, e.g. MS Office
Operating system: core tools for IT, e.g. Sun Solaris

December 6, 2004

11

Attributes by Product Type

Consumer

OpenSource

Hosted

Installed by

Self

Self or
IT

Vendor

Vendor

IT

IT

Distribution

Pull

Pull

Push

Pull

Push

Push

$$

$$$

Cost

Corporate Corporate
Desktop

Operating
System

Training

No

No

Maybe

Yes

Maybe

Maybe

Support

Vendor

No

Vendor

Vendor

IT

IT

Yes

Maybe

No

Yes

Yes

Yes

Multiple
supported
versions

December 6, 2004

12

What is Configuration Management?

The process of building the product,
tracking its components, and managing
change.
The purpose of configuration management
is to guarantee reproducibility of the
product and traceability of all changes.
Release Management and Configuration
Management are often the same group.
December 6, 2004

13

Common Responsibilities of CM

Source Code and Version Control

Reproducible components of the release
Traceable what changed in this release
Definitive Software Library: Baseline + delta
baseline

Version 1.0

delta

1.1

baseline

delta

1.2

branch

merge

Version 2.0

December 6, 2004

14

Product and Project Management

ABCs of Release Management by Mario Moreira. CM Crossroads, Aug 17 2004. http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat

=&Number=31323&Main=31323#Post31323

December 6, 2004

15

Why does Release

Management matter?
Common Release Problems
Benefits of Release Management
Who Cares?

December 6, 2004

16

A Story Timeliness, Quality

Three companies decide independently to
develop appointment scheduling software.
They all have the knowledge and funding to
create a good product.
Company A releases on time with 100 known
issues and is a success.
Company B releases on time with 50 known
issues and fails.
Company C releases late with 10 known issues
but fails because Company A had already
captured the market.
December 6, 2004

17

Common Release Problems

Take too long impacts revenue

Unpredictable impacts dependent plans
Poor quality impacts support
Not useful impacts revenue
Contents unplanned impacts integrity
Contents uncontrolled impacts reliability
Overlooked distribution/installation impacts
support

Cowham, Robert. Release Management Making it Lean and Agile CM Crossroads, Aug 16, 2004 http://
www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=31243

December 6, 2004

18

The Challenge
Time to Market vs Features vs Quality
Only one top priority
Balance dont ignore the other two
Release criteria reflect priorities
Product development methodology supports priorities:

concurrent engineering, test-first development, prototyping

Time to Market

Features

December 6, 2004

Quality

19

Benefits of Release Management

More

successful releases
Consistent release process
Predictability
Integration
Completeness
Quality
Communication
December 6, 2004

20

Ready to sell?
Demo updated
Trained sales force
Pricing
Order Fulfillment
Contracts ready

Commitments to
Customers,
Stockholders,
Employees
Legal Considerations
Return on Investment

Documentation ready
Training materials
Trained support staff
Release Notes
Known Problems
FAQ, Knowledge Base
Implementation impact

Executives
Sales &
Marketing

Support

IT
Dependable Installation
Resources to support
Predictable schedule
Configuration Options
Technical Documentation
Stability, Scalability, Reliability
December 6, 2004

Worth installing?
Installable
Usable
Supported
Budget
Timing

Customers

Who
Cares?

Developers

Quality
Assurance

Pro*
Management

Dependencies managed
Meets requirements
Minimize merging
No death march
Next project

Well-defined scope
Resources to support
Predictable schedule
Sufficient test time
Meets requirements

Product Management
Market position
Product Roadmap
Project Management
Project delivery on-time
Program Management
Program meets expectations

21

How does it work?

Types of Releases
Software Development Life Cycle
Release Process

December 6, 2004

22

Types of Releases

Major = significant new features, product

launch, planned, pull, annual
Minor = small new features, backwards
compatible, planned, pull, semi-annual
Patch = collection of bug fixes, planned,
pull, quarterly
Emergency = Urgent bug fix, harmful if
not fixed, unplanned, push, as needed

December 6, 2004

23

Attributes by Release Type

Features/ Bugs
Functions

Major

Cost

Urgent Scheduled Pull Backwards

Compatible

(new)

Minor

Y
(upgrade)

Patch

Fix

December 6, 2004

24

Release Management in the

Software Development Life Cycle

December 6, 2004

25

Activity Distribution
Release activities
account for
10% of effort and
15% of schedule.

McConnell, Steve. Software Project Survival Guide. Microsoft Press, 1998.

December 6, 2004

26

RELEASE CYCLE

NEW CUSTOMER
RELEASE

Sources of Incidents
Customers
Systems Integrators

TRACK
INCIDENTS

RM

Sources of Changes &

Defects
Product Management
Marketing
Sales
Product Support
Development
QA
User Group
Staff
etc.

OWNERS
CCB Change Control Board
DEV Development
IS
Information Services
MRK Marketing
PM
Product Management
QA
Quality Assurance
RM
Release Management
SAL Sales
SUP Support

SUP

FIX LATER OR
NEW FEATURE

Emergency?
SUP

FIX NOW

DEV

TRACK
CHANGES &
DEFECTS
RM

3.2.0
3.1.0

BASELINE
BASELINE

CHANGE
CONTROL
BOARD (CCB)

DEFER
RM

APPROVE
PM

DEVELOPMENT
LIFE CYCLE
DEV

REJECT
Resolve
With
Initiator

MORE
INFORMATION

3.2.0

BECOMES NEW
BASELINE

UPGRADE
RELEASE

SCHEDULED
NEW FEATURES
RM

ROLL UP
3.1.4
3.1.1

MAINTENANCE
RELEASE

SCHEDULED
MAINTENANCE

RM

ROLL UP

3.1.3
3.1.2

December 6, 2004

PATCH
RELEASE

UNSCHEDULED
MAINTENANCE
RM

27

Release Process
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

Define Content
Assign Version
Plan Release
Build
Package
Deploy
Iterate
Test
Control Change
Assess Readiness
Release

December 6, 2004

28

1. Define Content
Golden

Build (software)
Documentation
Training materials
Marketing collateral
Internal documents
Customer Notification
Release Notes
December 6, 2004

29

1. Define Content: Must and Nice

Disp

Nice
Nice

ID

Severity

Pri

Description

41928

enhanceme
nt

P3

Performance - Publish: loadSiteLite() to replace loadSite()

P4

Misalignment of Editable site and Proof site sections on Manage site screen once the
actor checks Lock-site checkbox.

59439

trivial

Nice

59929

minor

P5

Error message:Get A Quote form:The listed items in error message may need some
order; by logical or by alphabetic

Nice

57954

major

P2

Only Ford Make is displayed in 'Quick Quote Form' : GM Design

Nice

59368

minor

P2

Masthead descenders are being chopped off in Klamath_B_Dst design

Nice

57019

minor

P3

Can't tell mouse over selection from current page selection in nav

Nice

59726

minor

P3

Design Celandine: Titles (Header) for static pages are not bold as those for dynamic
pages

Nice

60108

major

P3

Misalignment of footer in Hours and directions page-Celandine designs

64793

enhanceme
nt

P3

create new Saab luxury design and homepage

64796

enhanceme
nt

P3

create new Land Rover luxury design and homepage

Must
Must

December 6, 2004

30

2. Assign version
Version is a unique identifier for the product state as
of a certain point in time. Not all versions are
released. Versions are sequential.
Versioning Schemes relate to release types, product
management, and configuration management:
Major.Minor.Patch.Fix, e.g. 1.0, 1.1, 1.1.2, 1.1.2.1
Major.Feature.Patch, e.g. 1.0, 1.1, 1.1.2
Major.Feature.Patch.Build, e.g. 1.0.0.301
December 6, 2004

31

3. Plan Release
Dependencies
Customer

commitments
Revenue recognition
Resource availability: people, environments
Introducing too much change at once
Stabilizing between releases
Organizational capability
December 6, 2004

32

3. Plan Release: Milestones

Scope Freeze

9/7

COMPLETE

Functional Freeze

9/17

COMPLETE

Functional Test Complete

9/30

COMPLETE

Code Freeze

10/1

COMPLETE

Regression Test Complete

10/13

COMPLETE

Golden Build

10/8

COMPLETE

Performance Test Complete

10/13

COMPLETE

Mock Deployment

10/12

FAILED

Mock Deployment

10/14

COMPLETE

Deployment Plan Review

10/15

COMPLETE

Go/No-Go

10/15

COMPLETE GO

Executive Approval

10/15

COMPLETE-APPROVED

RELEASE

10/15

COMPLETE

December 6, 2004

33

4. Build
Build (What)

Visibility (Who)

Frequency (When)

Purpose (Why)

Private
System
Build

Individual
Developer

One or more times

per change task

Provide feedback to the

developer of the changes
just implemented in the
private workspace.

Integration
Build

Whole
Development
Team

After each change

task and/or one
or more times per
day (e.g. Nightly)

Integrates the latest changes

in the repository and
runs automated tests
against the release
providing feedback to the
development team.

Release
Build

Independent Test
(QA or V&V)
Team and
Customers

At the end of each

iteration/release

Packages release for

distribution to the
development teams
customer.

Appleton, Brad. Agile Build Promotion: Navigating the Ocean of Promotion Notions, CM Crossroads, September 2004. http://
www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=32900

December 6, 2004

34

5. Package
Create

the release package containing

all release deliverables
File: tar, .zip, .exe, .ear
Medium: CD-ROM, tape, file server
Bill of Materials lists whats in the
package
Final release package is golden build
December 6, 2004

35

6. Deploy
Deployment

Plan

Install

Team
Deployment Plan Review
Mock Deploy
Mock Rollback
Contact List

December 6, 2004

36

6. Deploy: Sample Steps

1

Nitra "Version 2.3.3" Deployment

WLS 8.1 on SMTP servers

Thomas

Verify email router updates in env file

Thomas

Verify UPSShip updates in Env File

Thomas

Validate receipt of Nitra, LMCC Tar Balls and

Mapping Loader script

Claus

10

Add Scion Altercast fonts (Bug 61506)

Thomas

11

Prep-Outage Tasks

1 hr

12

Hot Backup Nitra DB

55 min

9:00 PM

9:55 PM

Claus

13

Check 3rd Party Services (Chrome, AIC, Vicinity)

10 mins

9:30 PM

9:45 PM

Jeff M

14

Team Check-In

5 mins

9:45 PM

9:50 PM

Team

15

Check CDC design flag to true

5 mins

9:45 PM

9:50 PM

Thomas

16

Outage

17

NO DEALER ACCESS

5 mins

9:50 AM

9:55 AM

Thomas

18

NO CONSUMER ACCESS

5 mins

9:55 AM

10:00 PM

Thomas

22

Shutdown Servers

40 mins

23

Disable Nitra Monitoring

3 mins

10:05 PM

10:08 PM

Thomas

24

Disable Incoming Leads (no consumer access)

2 mins

10:08 PM

10:10 PM

Thomas

25

Allow Existing leads to pass (Big IP)

2 mins

10:10 PM

10:12 PM

Thomas

26

Bring down WLS (Nitra and LM)

30 mins

10:12 PM

10:42 PM

Thomas

27

Snapshot net-app (back up file system)

2 mins

10:42 PM

10:44 PM

Thomas

28

Deployment time noted (for rollback purposes)

1 mins

10:44 PM

10:45 PM

Thomas

29

Database Modifications

5 mins

31

LMCC DB Script

10 min

10:55 PM

11:05 PM

Claus

32

Run script for Mapping loader

10 min

11:05 PM

11:15 PM

Claus

33

Code Deployment (LM and Nitra Clusters)

40 mins

December 6, 2004

Steven

37

7. Test
Plan

ahead for test environments

Execute Test Plan

Functional test: does it function as required?

Regression test: did old bugs reappear?
Performance test: does it perform within

acceptable limits?
Reliability test: does it function consistently
over time?
Stress test: does it function consistently under
volume?

December 6, 2004

38

8. Control Change

Baseline scope, requirements, design

Accommodate controlled change

Avoid scope creep

Allow change, its reality

Change Control Board (CCB)

Cross-functional representation
Impact analysis
Business drivers

Change requests
Change request decision (approve, reject, defer)

December 6, 2004

39

9. Assess Readiness

Release Criteria define done

Organizational Readiness

Defined Deliverables
Features: Must Have vs Nice to Have
Quality: test results, metrics
Schedule: time-to-market criteria
Objective, quantifiable criteria preferred
Support, Training, Marketing, Sales

Acceptable Risk

December 6, 2004

40

9. Assess Readiness: Release Criteria

Quality

Testing Complete (functional, regression, performance, reliability, stress)

Quality

Successful Mock Deployment

Quality

Successful Mock Rollback

Quality

"Golden" Build in Hand

Features

All must have features complete

Features

Documentation and Help updated

Features

Release Notes Accepted by Customer Support

Schedule

Schedule Clearance (no conflict)

Schedule

Outage Notification Processed

Schedule

Release Package Delivered to IT

Schedule

Acceptable Deployment Night Coverage

Schedule

Acceptable Post-Deployment Coverage

Quality

Go/No Go Meeting yields Go decision. Punch List completed.

December 6, 2004

41

Assess Readiness: Metrics

Common metrics: fix rate, find rate, defect
density, rate of change
Example: find rate vs fix rate
Stability reached at
week 35 of testing,
when defects fixed
exceeds defects
opened.

December 6, 2004

42

9. Assess Readiness: Go/No-Go

Release Readiness Assessment

Go/No-Go

Are all release criteria equal?

Can any criteria be waived?
Formal sign-off by stakeholders
Meeting, email vote
Recommendation by Release Manager
Go with Punch List

Executive Approval
December 6, 2004

43

10. Release

Formal

release by Configuration Mgmt

Deploy to Production or Distribute to
Customer

December 6, 2004

44

10. Release Programs

Alpha

= test by in-house department

Beta = test by small group of customers

Pilot
Early Adopters
General

Availability (GA) = all customers

December 6, 2004

45

Would you ship it?

Situations
Is it ready?
Would you ship it?

December 6, 2004

46

Situation #1: Class registration

Criteria

Zero priority-one defects

Ready in time for class registration on April 1

Data

One priority-one defect affecting 100 music

majors out of 3,000 students.

Fix estimate is 3 days.
Today is March 15.

Ship

it? More data?

December 6, 2004

47

Situation #1: Other Factors

Cost: Penalty for late delivery?

Confidence: accurate estimate? Complexity?
Quality: Danger of breaking something else? How
much testing is needed?
Schedule: Holidays? Vacation? Customer
availability? Hard deadline?
Relationship: Unhappy customer? Reference
site?
Workaround: Manual? Send staff on site?

December 6, 2004

48

Situation #2: Java web search tool

Criteria

Acceptable performance: + 10% current

Backwards compatibility

Data

Performance on page load is 5% better

Performance on search is 20% worse
Fix will require customer to download new
Java Virtual Machine (JVM)

Ship

it? More data?

December 6, 2004

49

Situation #2: Other Factors

Quantify:

How much loss? < second?

Accuracy: performance test environment?
Alternatives: Optional download?
Automatic download?
Support: Ease of download? Increased
support load?
Competition: Market leader? Slipping?
Usability: lose customers if slow?
December 6, 2004

50

What did we learn?

Release Management Definition
Key Practices
Key Principle
Release Process
Features, Quality, Time to Market

December 6, 2004

51

Release Management Definition

The process by which a product is made ready
for distribution to customers. Release
management is the coordination of all the
activities leading to and including product
release to customers.
The purpose of release management is to
ensure products are ready when promised.

December 6, 2004

52

Key Practices

Release Planning: Manage and publish

release schedule
Define deliverables included in a release
Manage dependencies across components
and activities in the release
Coordinate activities across company
Control changes that would impact the
release schedule
Establish release criteria
Assess release readiness

December 6, 2004

53

Key Principle: Define Done

Deliverables
Dependencies
Avoid

scope creep but allow change

Release criteria

Objective
Quantifiable
Cross-functional
December 6, 2004

54

Release Process
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

Define Content
Assign Version
Plan Release
Build
Package
Deploy
Iterate
Test
Control Change
Assess Readiness
Release

December 6, 2004

55

The Challenge
Time to Market vs Features vs Quality
Only one top priority
Balance dont ignore the other two
Release criteria reflect priorities
Product development methodology supports priorities:

concurrent engineering, test-first development, prototyping

Time to Market

Features

December 6, 2004

Quality

56

Q&A

THANK YOU!

December 6, 2004

57