You are on page 1of 32

?    ?

  
 
 

 
   
     

Peter Baxter, DISTRIBUTIVE MANAGEMENT


Hampton Roads INCOSE Decision Analysis Conference - Nov 2009
j @ 

earn:
 How to construct and weight evaluation
criteria.
 How to make pair-wise comparisons of
alternatives.
 How to expand the example for more
complex, nested types of criteria.
 How a measurement process can support
the Saaty method.

Ê   



]       
 @

è   
  @

The Analytic Hierarchy Process (AHP)

Compute the matrix values for Õ and ͙ voila!


Ê   



-
 @     
@    

›   
›   
› ÿ 9 5 3
ÿ ÿ 3 ÿ5 ÿ
 ÿ9 ÿ3 ÿ ÿ ÿ9
 ÿ5 5 ÿ ÿ3
Ê   
 ÿ3 9 3 ÿ



·   @  @ @
 Don͛t need an absolute scale
(if you already know the size of all but one)
 Relative scale approximates difference

Ê   



       

Ê   
@  
    



Î ]       

In ÿ Weber found, for example, that people while holding in


their hand different weights, could distinguish between a
weight of 20 g and a weight of 2ÿ g, but could not if the
second weight is only 20.5.On the other hand, while they
could not distinguish between 0 g and ÿ g, they could
between 0 g and 2 g, and so on at higher levels. We need
to increase a stimulus    
 where our senses can first discriminate between  
.   (jnd). The
ratio     .

          


      . 
           
      .
Ê   



ë ]   ] @

 People are inconsistent at providing an absolute


scale to evaluate objects.
How much does Rock A weigh?
How much does Rock B weigh?

 There are better at comparing pairs of objects.


Does Rock A weigh more than Rock B?

Ê   



ÿ        

ÿ. Select criteria(s) to evaluate


2. Define comparison scale
3. Perform pair-wise comparison
. Check consistency
5. Calculate values

Ê   



      @

ÿj
   ÿ
 @ ]   @
 A controlled test where existing software sizes are known.

 Ask 30 grad students to estimate the SOC of common data


structures like stack, queue, list.

 Estimate SOC three ways:

ÿ. Guess a number.

2. Compare to one reference structure using numeric


scale.

3. Compare to one reference structure using relative scale.

 Plot results

Ê   
From ͞Establishing Software Size Using Pair-wise Comparison Method͟


 by Eduardo Miranda
ÿ
   ÿ
 @ ]   @

Ê   



ÿè
   ÿ
@ @ 
Miranda͛s Observations

 The high variability of the ͞finger in the wind approach͟,


which is almost two to three times bigger than the
corresponding paired comparisons method.

 The high correlation, r = .9 9, existing between the


relative sizes of modules independent of the estimation
method employed. This seems to corroborate the
premise that the human mind is better at establishing
differences than at guessing absolute values.

Ê   



ÿ-
   j
     
 From an SEI study ͞Requirements Prioritization Case Study
Using AHP͟ by Nancy Mead.

 Given nine security requirements, decide which one(s) have


greatest cost-benefit.

 Benefits

͞By using AHP, the requirements engineer can


also confirm the consistency of the result. AHP can
prevent subjective judgment errors and increase the
likelihood that the results are reliable.͟

Ê   



ÿ·
   j
 @
Approach
ÿ. Review requirements for completeness.
2. Apply pair-wise comparison for value.
3. Apply pair-wise comparison for implementation
cost.
. Calculate AHP matrix and diagram for value & cost.
5. Use resulting diagram for analyzing requirements.

Ê   



ÿ
   j
 @ @  

Ê   



ÿÎ
   j
 @ @
Compare one attribute of each requirement to another
requirement using a relative scale.
#ÿ is same #ÿ is greater
as #ÿ than #2

#2 is much
less than #

Ê   



ÿë  @  @  

 If: A>B>C
 Then: C>A is wrong

AHP contains a technique to calculate the extent of


pair-wise consistency, which can then be
compared to a consistency tolerance.
AHP can also indicate which pair-wise comparison
(like the one above) is inconsistent.

Ê   



j
   j
 @ @   

Ê   



jÿ
   j

 
Client feedback
 It may be beneficial to see the consistency matrix.
 Understand weight of cost and value.
 Difficult to understand the motivation of each
reviewer.

Ê   



jj @    

From Saaty:
 Evaluate the best city in China for Disney
to build a new theme park.
 Determine optimum foreign relations
policy for dealing with Iran.
 Estimate market share of ͞super͟ retail
stores.
 Selecting a school.

Ê   



   
 
jè    
 

Goal is for Measurement to support AHP estimation

Use AHP Estimate

Capitalize Plan

Monitor Measurement Process


Ê    & Control



j-
    
 @
 ›

  Õ
ÕÕÕ
  › › 
›  


Õ 

mÕ
Õ
Õ
Õ 
›
 

  
 › › 
› 
› 
   ›
›  
m ÕÕ
Õ 
j· ]  @ 

Measurement ͞Requirements͟ = information needs


 Based on your business needs.
(not a pre-defined list)
 Information needs are:
 ͞Input͟ to the measurement process.
 Provided by the management and technical process that
need information to perform their jobs.
 Become the requirements for measurement process.
 Refined into measures and then resulting information
products are provided to the ͞users͟.
Ê   



j
 @ @  @


Estimation ProjectProgram Store

 Cost

 Staffing , Staff Hours


Factors estimate
using HP
 Functional  Requirements Size

 SOC

 Defects

Usually there is opportunity for measurement to support AHP since


there is overlap between what is estimated and what is measured
Ê   



jÎ     
  

 Measurement contains project attributes:


 ifecycle model

 Principle architecture

 Application domain

 AHP estimation of projects with similar attributes.

 Reference sizes should not differ by more than an


order of magnitude.

Ê   



jë    

Estimate total staff hours in person years using data from


five completed projects sharing similar attributes.

Ê   



  @

͞Requirements Prioritization Case Study Using AHP͟


by Nancy Mead Software Engineering Institute

͞Establishing Software Size Using the Paired


Comparisons Method͟ by Eduardo Miranda

͞Relative Measurement and Its Generalization in


Decision Making Why Pairwise Comparisons are
Central in Mathematics for the Measurement of
Intangible Factors The Analytic
HierarchyNetwork Process͟ by Thomas Saaty

Ê   



ÿ  @ 

Ê   



j

Peter Baxter
Distributive Management
www.distributive.com
pbaxter@distributive.com

Ê