ESD.

36J System & Project Management

+

Dynamics of Project Performance

System Dynamics and Project Management

Class Three (9/30/03)

Copyright 2003
James M. Lyneis

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

What did you conclude re. the

effect of productivity, quality,
and rework discovery time on
time required to finish the
project?

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

Sensitivity of Completion Date to

+/-33% Variation in Parameters

Quality - Completion Date

Rew ork Discovery Delay Completion Date

Productivity - Completion Date

80

80

80

60

60

60

40

40

40

20

20

20

Low Q

9/30/03

Base

High Q

- ESD.36J SPM

Low Pdy

Base

High Pdy

Low Rw dd

Base

Copyright 2003
James M. Lyneis

High Rw dd

Sensitivity of Total Work Done to

+/-33% Variation in Parameters

Quality - Total Work Done

Rework Discovery Delay - Total

Work Done

Productivity - Total Work Done

200

200

200

150

150

150

100

100

100

50

50

50

Low Q

9/30/03

Base

- ESD.36J SPM

High Q

Low Pdy

Base

High Pdy

Low Rwdd

Base

Copyright 2003
James M. Lyneis

High Rwdd

Sensitivity of Completion Date

Variations in Quality have biggest

impact -

Nonlinear effect
Determines total amount of work

Rework discovery time has the least

impact (but the simple model does not
have a quality on quality feedback).

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

+
Graph for Undiscovered Rework

Quality = .5
6

Quality = .75
2

Quality = 1.0

0
0

10

15

20

25
Time (Month)

30

35

40

45

Undiscovered Rework : Class2 Step1

Undiscovered Rework : Class2 Step1 Q100
Undiscovered Rework : Class2 Step1 Q50

9/30/03

- ESD.36J SPM

50
Task
Task
Task

Copyright 2003
James M. Lyneis

+
100

Graph for Work Done

Quality = .75

Quality = 1.0

75

Quality = .5
50

25

0
0

10

15

20

25
Time (Month)

30

35

40

45

Work Done : Class2 Step1

Work Done : Class2 Step1 Q100
Work Done : Class2 Step1 Q50

9/30/03

- ESD.36J SPM

50
Task
Task
Task

Copyright 2003
James M. Lyneis

10

Rework Shows Up As Revised

Work Products
Accomplishment of Revisions
to Design Products (e.g., Drawings)

REV 0
REV 1
REV 2
REV 3...

TIME

Virtually all complex projects have revision cycles.

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

11

Managing the rework cycle

Do common project metrics handle

(undiscovered) rework?

Do management incentives encourage

the discovery of rework?

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

12

9/30/03

Additional Metrics (Used in

Model)
Fraction Perceived to be Complete =
(Work Done + Undiscovered Rework) /
Initial Work to Do
Fraction of Work Really Complete =
(Work Done) / Initial Work to Do

- ESD.36J SPM

Copyright 2003
James M. Lyneis

13

Perceived vs. Really Complete

in Simple Rework Cycle Model
Progress
1
0.75
0.5
0.25
0
0

12

18

24
30
36
Time (Months)

Fraction Perceived to be Complete : Step2

Fraction Really Complete : Step2

9/30/03

- ESD.36J SPM

42

48

54

60

Fraction
Fraction

Copyright 2003
James M. Lyneis

14

On an actual project
Perceived vs. Actual Progress: SRS Development
100.

Perceived

Actual

75.

Disguised results
from actual
aerospace project

50.

25.

0.

1/93 1 Year2
1/94
Year

Year
1/953

Year
1/964

Year
1/975

Year
1/986

1/997
Year

Year
1/008

Year
1/019

1/02
Year 10

TIME
c:\mydocu~1\jml\hughes\hisjml.dyn 8/19/97 9:29
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

15

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

16

Examples of Project Behavior Modes

Project
Staffing

Typical
Plan

Time
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

17

Examples (continued)

Productivity
(Normalised)
2

Typical
Plan

Time
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

18

Dynamics of Project Performance

The rework cycle

Quality
Undiscovered rework

Feedback effects on productivity and work

quality

9/30/03

Positive, re-enforcing, often vicious

circles
Negative, controlling

Knock-on effects between work

phases
Copyright 2003
- ESD.36J SPM

James M. Lyneis

19

A system dynamics model usually represents several

phases of work, but is more aggregate than a CPM
model

Software
Design

Software Code
and Test

System
Engineering

Integrate
and Test

Hardware
Design

9/30/03

- ESD.36J SPM

Hardware Build
and Test

Copyright 2003
James M. Lyneis

20

Peace Shield Air Defence System Model

Architecture

Software
System
Engineering

CONUS
Software Development

Top Level
& Detailed
Design

Code
& Unit
Test

HW
Installation
& ICO Test

Op. &
Main/
ILS

Software Support

SRS
Development

Mgt. &
Admin.

Integration
& Type II
Test

Type I
Test

KOSA
SW & Testing

Downstream
Progress,
Availability,
& Quality Effects

Program Management Office

Upstream Rework
Discovery Effects
Support Effects

Customer

9/30/03

Hardware

- ESD.36J SPM

Logistics

Subcontractors

Other HASI
Programs

Copyright 2003
James M. Lyneis

21

Inter-phase Effects

Downstream &/or Parallel --

Availability of work products

Quality of work products
Upstream --

Rework discovery
Above apply internally and to suppliers

General --

Sharing of staff
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

22

Inter-phase Connections:
Upstream-Downstream

Progress; Quality

Design
LABOR
APPLIED

WORK

WORK
BEING DONE

TO BE
DONE

LABOR
APPLIED

Productivity Quality

KNOWN

WORK

WORK

REALLY

TO BE
DONE

DONE

UNDISCOVERED

REWORK

REALLY
DONE

UNDISCOVERED

REWORK

REWORK
DISCOVERY

WORK

WORK
BEING DONE

KNOWN

REWORK

Productivity Quality

REWORK

REWORK
DISCOVERY

Progress

Construction
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

23

Inter-phase Connections: Parallel

Software
Design

LABOR
APPLIED

WORK

Productivity Quality

WORK
REALLY
DONE

WORK
BEING DONE

TO BE
DONE

KNOWN

UNDISCOVERED

REWORK

REWORK
REWORK
DISCOVERY

Progress:
Quality

Progress;
Quality
LABOR
APPLIED

WORK

Productivity Quality

KNOWN

UNDISCOVERED

REWORK

REWORK
REWORK
DISCOVERY

9/30/03

- ESD.36J SPM

WORK
REALLY
DONE

WORK
BEING DONE

TO BE
DONE

Hardware
Design
Copyright 2003
James M. Lyneis

24

Dynamics of Project Performance

The rework cycle

Quality
Undiscovered rework

Feedback effects on productivity and work

quality

Positive, re-enforcing, often vicious circles

Negative, controlling

Knock-on effects between work phases

9/30/03

Availability and quality of work products

Progress to discover upstream rework

- ESD.36J SPM

Copyright 2003
James M. Lyneis

25

How might we represent suppliers?

platforms? ...

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

26

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

27

9/30/03

If the world consists only of stocks and flows,

what are those other variables indicated on the
diagrams??

Auxillaries
Constants (e.g., factors which may be
stocks or flows, but which do not
change over the time span of the
simulation)
External inputs

- ESD.36J SPM

Copyright 2003
James M. Lyneis

28

Auxillaries or Converters Serve

Several Purposes ...

Break up rates into meaningful

components
Provide alternative measures for stocks
or flows
Reduce diagram clutter

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

29

Potential Work Rate

Potential Work Rate = Staff * Productivity

Tasks/Month

9/30/03

- ESD.36J SPM

People * Tasks/Month/Person

Copyright 2003
James M. Lyneis

30

Productivity & Quality

Productivity = ??

Quality = ??

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

31

9/30/03

Productivity -PRODUCTIVITY = NORMAL

PRODUCTIVITY * EFFECT OF STAFF
EXPERIENCE * EFFECT OF QUALITY OF
PRIOR WORK * ...
Dimensions:
Productivity -- Tasks/Month/Person
Normal Productivity -- Tasks/Month/Person
Effects -- Dimensionless

- ESD.36J SPM

Copyright 2003
James M. Lyneis

32

9/30/03

Quality -QUALITY = NORMAL QUALITY * EFFECT

OF STAFF EXPERIENCE * EFFECT OF
QUALITY OF PRIOR WORK * ...
Dimensions:
Quality -- Fraction
Normal Productivity -- Fraction
Effects -- Dimensionless

- ESD.36J SPM

Copyright 2003
James M. Lyneis

33

9/30/03

Effect of Staff Experience

Effect of Experience = Function ( Years
on Project)

- ESD.36J SPM

Copyright 2003
James M. Lyneis

34

Effect of Staff Experience

Effect of Experience = Function (Years on Project)

1.0
.8

.6
.4
.2
0
2
9/30/03

- ESD.36J SPM

10

Years on Project
Copyright 2003
James M. Lyneis

35

How do we determine these effects?

The effects are first estimated based on

common sense, later to be verified
during model calibration.
It helps to specify likely values at
extreme points, and draw a smooth
curve in between.

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

36

Effect of Staff Experience

Extreme points -X

1.0
.8

.6
.4
.2

0
2
9/30/03

- ESD.36J SPM

10

Years on Project
Copyright 2003
James M. Lyneis

37

Connecting the extreme points ...

1.0
.8
.6
.4
.2

0
2
9/30/03

- ESD.36J SPM

10

Years on Project
Copyright 2003
James M. Lyneis

38

Some other connections are plausible ...

1.0
.8
.6
.4
.2
0
2
9/30/03

- ESD.36J SPM

10

Years on Project
Copyright 2003
James M. Lyneis

39

Others are not ...

1.0
.8
.6
.4
.2
0
2
9/30/03

- ESD.36J SPM

10

Years on Project
Copyright 2003
James M. Lyneis

40

9/30/03

Relationships Similar to the Experience Effect Would

Be Specified For Other Effects on Productivity and
Quality
Experience
Tools
Adequacy of supervision Processes
Availability of resources Availability of supplier
information &/or materials
Morale
...
Schedule pressure
Fatigue
Quality of prior work
Skills available vs. needs

- ESD.36J SPM

Copyright 2003
James M. Lyneis

41

Note difference with statistical

estimation --

Productivity = a + b*experience+c*prior
quality

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

42

Rules of Thumb and Checks

1. Variables and constants should correspond to real system

variables
2. Remember to represent delays -- in flow channels, in
perceiving information, and in making or implementing
decisions
3. Distinguish desires from realizations
4. Make sure that equations balance dimensionally
5. Check the validity of equations at extreme points, e.g.,
zero or very high values of the inputs
6. Avoid the use of imbedded functions -- they make the
equations harder to understand and can lead to errors

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

43

Connecting Morale & Overtime ...

from a students email:
On a couple of occasions in lecture yesterday you discussed
"Overtime" as it relates to "Morale". In doing causal loops, I find
that the relationship between the 2 can be different and
dependent on time. For example, initially the morale of
employees may increase or support the increase of overtime
which provides a positive relation between the 2. However, as
the overtime continues, I believe the morale will decrease as
employees will start to desire a life outside of work which
supports a negative relation between the 2.
I guess that in the actual SD modeling the developer would create
a curve/equation for morale over time, but what about for causal
loops? How do you handle the dynamics with causal loops?
Maybe the loops are just an initial starting point for the
development?

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

44

9/30/03

Representing morale

- ESD.36J SPM

Copyright 2003
James M. Lyneis

45

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

46

Calibration (Parameter Estimation)

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

47

Parameter Estimation in System

Dynamics

A priori estimates based on

direct observation
educated guesses by managers
similar parameters from other
applications

Refine initial estimates in the process

of calibration
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

48

Calibration is part of the scientific

method
Calibration is not curve-fitting via
exogenous variables, but systematic
comparison of model output to all
data and resolution of any
discrepancies

9/30/03

changes in structure
changes in parameters
decision to ignore (or sometimes use exogenous
effect)

- ESD.36J SPM

Copyright 2003
James M. Lyneis

49

Calibration Should Be a Part of the

Standard Method

9/30/03

Data is a real source of information

Calibration often uncovers
errors/omissions in insight-based
models
Cost is small relative to the stakes in
most large organizations

- ESD.36J SPM

Copyright 2003
James M. Lyneis

50

Why do we calibrate a model?

9/30/03

Assure model contains all of the structure

necessary to create problem
Accurately determine leverage points
Accurately price cost-benefits of
alternatives
Sell results to those not on project team
Generate a benchmarking data base for
future use and learning

- ESD.36J SPM

Copyright 2003
James M. Lyneis

51

Sources of information for calibration

Hard, numerical time series (e.g., for

staffing, work accomplished, overtime
levels, etc.)
Soft, management observations of
what happened (e.g., how morale,
experience, and other factors might
have caused productivity and quality to
change)

9/30/03

such observations must be tied to what

was happening on the project

- ESD.36J SPM

Copyright 2003
James M. Lyneis

52

Project Dynamics
Time
Remaining

Skill &
Experience

Staff
Organisational
Size
Changes

Morale

Turnover

Hiring

Overtime
Scheduled
Completion
Time

Schedule
Pressure
Productivity

Quality

Out-of-Sequence
Work

Expected
Completion Availability
of Prerequisites
Time

Progress

Added
Work

Work Quality
to Date

Work To
Be Done

Work
Really Done

Known
Rework

Undiscovered
Rework

- ESD.36J SPM

Hours
Expended
to Date

Expected
Hours at
Completion

Obsoleted
Work

Rework
Discovery

Perceived
Progress

9/30/03

Equivalent
Staff on
Project

Staffing
Requested

Data likely to
be available
Copyright 2003
James M. Lyneis

53

Once calibration is completed ...

We have a model that is useful for:

Learning

what really happened on prior projects

what could have been done better

Dispute resolution

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

54

9/30/03

What happens if you cant calibrate?

Use logical parameter estimates
Simulation model still better than mental
models

- ESD.36J SPM

Copyright 2003
James M. Lyneis

55

Topics

Review Practice for 9/30

The dynamics of project performance -knock-on effects

Modeling the feedback effects

Model Calibration
Practice for 10/2
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

56

Lessons

Rework (Errors) Can Linger for Long Periods of

Time as Undiscovered Rework

Reducing Rework, and Detecting it Earlier, is the

Best Way to Cut Cycle Times, Finish Earlier, and
increase Delivered Quality

Common metrics & reward systems do not

encourage high quality and rework discovery

Interactions between phases of work, different

projects, suppliers, create opportunities for knockon effects

Calibration will improve the accuracy of our

parameter estimates, and ultimately policy decisions
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

57

Practice for 10/2

1. Download the Class2 Step2 model. This is

the rework cycle model with a variable time to
discover rework. Time to discovery rework,
as illustrated in the following figure, depends
on a maximum time and a minimum time, and
an effect of work progress. Time to discover
rework falls from the maximum to the
minimum time as fraction complete increases.

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

58

+
Productivity
Minimum Time to
Perform a Task

Potential Work
Rate

Maximum Work
Rate

Rework cycle
with variable
time to discover
rework

Staff

Feasible Work
Rate
Work Accomplishment
Project Finished
Switch

Quality

Work to Do
Rework Generation

Undiscovered
Rework

Work Done

<Rework Generation>
Rework Discovery
Maximum Time to
Discover Rework

Rate of Doing Work

Time to Discover Rework

<Work Accomplishment>

Minimum Time to
Discover Rework

<Work Done>

Cumulative
Work Done

Work Believed to
Be Done

Fraction Perceived
to be Complete

Effect of Work Progress

<Initial Work to Do>

Fraction Really
Complete

9/30/03

- ESD.36J SPM

Table for Effect of

Work Progress

Copyright 2003
James M. Lyneis

59

Practice for 10/2

2. Add the effect of prior work quality on quality

as discussed in class and illustrated in the
following figure.
3. Simulate the model and compare to the
original Class2 Step 2 results. How does the
addition of quality on quality feedback affect
project performance?
4. If we essentially eliminate undiscovered
rework by setting the maximum time to
discover rework to 0.25 months, what
happens to project performance?
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

60

Productivity
Minimum Time to
Perform a Task

Potential Work Rate

Maximum Work
Rate
Feasible Work Rate

Staff Level

Normal Quality
Quality
Work
Accomplishment
Table for Effect of
Prior Work Quality on
Quality

Effect of Prior Work

Quality on Quality

<Initial Work to
Do>

Project Finished
Switch

Work to Do
Rework Generation

Average Work
Quality

<Work Done>

Undiscovered
Rework

Work Done

<Rework Generation>
Rework Discovery

Maximum Time to
Discover Rework

Rate of Doing Work

Time to Discover Rework

Minimum Time to
Discover Rework

Cumulative
Work Done

<Work Accomplishment>

Work Believed to
Be Done
Fraction Really
Complete

Effort Expended
Fraction Perceived
to be Complete

Cumulative
Effort
Expended

Effect of Work Progress

<Staff Level>

<Project Finished Switch>

<Initial Work to Do>

Table for Effect of
Work Progress

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

61

Hints -- 1

Effect of Prior Work Quality on Quality:

kup - Table for Effect of Prior Work Qua
1

0
0

Note: The effect of prior quality on current quality is assumed to be proportional -- and
error in past work creates an error in current work. Given that in this simple model
quality represents several effects of work errors, this strong relationship may be
reasonable.
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

62

Hints -- 2

Average Work Quality:

Max(1e-006,Work Done)/Max(1e006,Work Believed to Be Done)
[Note: the max in the denominator
prevents division by 0, and in the
numerator allows the equation to return
a quality of 1.0 when no work is yet
completed.]
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

63

Hints -- 3

Normal Quality:
Set this to 0.85 (vs. 0.75 in Class2 Step2
model) to reflect that we are now
explicitly representing other factors
affecting project quality.

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

64

Graphical Functions
in Vensim

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

65

Step 1 - Set Up Model As Shown Below

Normal
Quality

Quality

Effect of Prior
Work Quality
Table for Effect of
Prior Work Quality
9/30/03

- ESD.36J SPM

Average Work
Quality
Copyright 2003
James M. Lyneis

66

Step 2 -- Equations ...

Quality = Normal Quality*Effect of Prior

Work Quality
Effect of Prior Work Quality = Table for
Effect of Prior Work Quality(Average
Work Quality)

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

67

Step 3 -- Define the Table

Click to define equation for Table for

Effect of Prior Work Quality
Select the Type as Lookup
Click on the As Graph button to see the
function
You can then type in values or create a
line on the blank graph
You can edit the values directly if desired
9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

68

Voila!

kup - Table for Effect of Prior Work Qua

1

0
0

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

69

In the short-term, overtime increases

morale and productivity
+

Productivity

Morale
+

Income

Overtime

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

70

But in the longer-term, sustained overtime

reduces morale and productivity ...

Productivity

Morale

Average
Overtime

Income

Overtime

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

71

Connecting the feedbacks through

progress ...
+ Productivity

Morale
+

Progress
+

Average
Overtime

Income

+
-

Overtime

Staff Needed

9/30/03

- ESD.36J SPM

Copyright 2003
James M. Lyneis

72