0 оценок0% нашли этот документ полезным (0 голосов)

26 просмотров13 страницExperimental Design

Sep 21, 2016

© © All Rights Reserved

PDF, TXT или читайте онлайн в Scribd

Experimental Design

© All Rights Reserved

0 оценок0% нашли этот документ полезным (0 голосов)

26 просмотров13 страницExperimental Design

© All Rights Reserved

Вы находитесь на странице: 1из 13

www.elsevier.com/locate/ecolmodel

many-parametered models

Diane L. Beres a,*, Douglas M. Hawkins b

b

a

Conser6ation Biology Program, Uni6ersity of Minnesota, 1987 Upper Buford Circle, St. Paul, MN 55108, USA

Department of Applied Statistics, 357 Ford Hall, Uni6ersity of Minnesota, 224 Church St. SE, Minneapolis, MN 55455, USA

Received 7 June 1999; received in revised form 15 February 2001; accepted 23 February 2001

Abstract

A method for sensitivity analysis of a simulation model is presented, together with an illustrative example. The

PlackettBurman technique allows the concurrent consideration of numerous parameters. The required number of

model scenarios necessary for completion of the analysis is approximately twice the number of parameters.

Advantages to the technique include simultaneously investigating all parameters, as well as acquiring information

about two-way parameter interactions, by means of a relatively small number of scenarios. A drawback is aliasing

among interactions, which may confound interpretation. 2001 Elsevier Science B.V. All rights reserved.

Keywords: Sensitivity analysis; Parameter interactions; Plackett Burman; Simulation model

1. Introduction

Sensitivity analysis illuminates the relative importance of a models parameters in bringing

about outcomes. In a sensitivity analysis, one

systematically and comprehensively tests to see

how changes in the parameters of the model affect

the models output (Starfield and Bleloch, 1991,

p. 58). Which of the parameters exert a significant

influence on the output variables? Which are inconsequential? Do increments in any parameters

* Corresponding author. Present address: Department of

Biology, Ripon College, PO Box 248, 300 Seward Street,

Ripon, WI 54971-0248, USA; Tel.: +1-920-7485874; fax:

+1-920-7487243.

E-mail address: beresd@mail.ripon.edu (D.L. Beres).

Without the answers to these questions, understanding of a model is incomplete. Recommendations based upon a model without explicit

sensitivity analysis lack foundation. Thus, thorough sensitivity analysis is a significant aspect of

every modelers job. Unless the hierarchy of

parameter strength is revealed, along with any

information or insights gained from the model,

the modeling task is unfinished (Beres et al.,

2001). As the term sensitivity analysis is used in

this paper, it applies to a wide range of models,

including both simulation and analytic models.

Despite the acknowledged importance of sensitivity analysis, there is a dearth of information

(Henderson-Sellers and Henderson-Sellers, 1996,

p. 291) on how it should be performed. There is

0304-3800/01/$ - see front matter 2001 Elsevier Science B.V. All rights reserved.

PII: S 0 3 0 4 - 3 8 0 0 ( 0 1 ) 0 0 2 7 1 - X

172

sensitivity analysis of stochastic models (McCarthy et al., 1995, p. 93). A common approach

to sensitivity analysis is to explore the effects of

changing parameters, one at a time, on a target

output variable (Henderson-Sellers and Henderson-Sellers, 1996; Swartzman and Kaluzny, 1987).

This procedure holds all other things are equal

while each parameter is altered in turn, to determine its effect in isolation from the possible effects of others. Indeed, the definition for the

sensitivity, S, of a parameter, P given by S=

[(x/x]/[(P/P], where x is the state variable under

consideration (Jorgensen, 1994, p. 23) implies

that the analysis will proceed one parameter at a

time. Often an increment (up or down) of 10% of

the nominal or usual value is tested (McCarthy

et al., 1995; Burgman et al., 1993). The sensitivity

investigation may consider the state variables of

greatest interest in the model (Jorgensen, 1994,

p. 57) and omit other parameters presumed

inconsequential.

Even when the sensitivity of all individual

parameters is investigated, the one-at-a-time

(OAAT) method does not uncover potentially

important interactions between two parameters

(Daniel, 1973). Possibly significant effects could

be produced by a pair of parameters acting in

concert (Burgman et al., 1993). Such interaction

effects would be o6er and abo6e the sum of the

individual effects of the two parameters in question. Furthermore, these interactions would not

necessarily be caused by pairs of the most important individual parameters. Thus, an experimental

design for sensitivity analysis, which will give

insight into these otherwise unknown interactions,

is needed (Henderson-Sellers and Henderson-Sellers, 1996; Daniel, 1973). The modeler is interested

in both the main effects of individual parameters and also the 2-way interactions of pairs of

parameters (Swartzman and Kaluzny, 1987). Potentially, there could also be notable higher order

interactions (3-way, 4-way, etc.). The sparsity of

effects principle (Montgomery, 1997) notes that

these are much less common than 2-way interactions, so higher order interactions will be disregarded in the ensuing discussion.

One way to the obtain the interactions of interest would be to implement the sensitivity analysis

by means of a complete factorial design. The

complete factorial would consist of all possible

combinations (assemblies) of selected high and

low values for the parameters. Using an upper

and lower value for each parameter, such a design

would reveal interactions of all orders in addition

to main effects. The number of distinct scenarios

required to execute the complete factorial is 2c

where c is the number of parameters and 2 is a

consequence of using two values for each. For

even as few as 10 parameters, implementing a

complete design would be inconvenient (210 =

1024), to say the least. Henderson-Sellers and

Henderson-Sellers (1996) recommended using

fractional factorial experimentation to reduce the

number of scenarios needed for the sensitivity

analysis. However, for 10 parameters, even the 1/4

fractional design would require a fairly large number (28 = 256) of distinct scenarios.

Another approach might be to devise a design

combining OAAT technique together with consideration of all pairings of parameters. Such an

approach is feasible when there are but a few

parameters (56 scenarios for the case of 10

parameters). However, if the model contains a

large number of parameters, as may be the case

with ecological models, even this combined design

might not be practicable. A better option is explained below.

A Plackett Burman (PB) design (Plackett and

Burman, 1946) with foldover (Box et al., 1978;

Montgomery, 1997) provides an alternative which

allows simultaneous examination of the entire

suite of parameters and is both convenient and

informative. The PB design specifies a selected

subset of the scenarios prescribed by a complete

factorial. Thus, in contrast to the sampling

schemes for stochastic model sensitivity analysis

described by Swartzman and Kaluzny (1987), PB

provides a simple blueprint for accomplishing the

investigation. Furthermore, PlackettBurman

sensitivity analysis (PBSA) is context-free; it is

equally useful for any sort of model having many

parameters (and a numerical response), but especially for computer-coded simulation models.

PBSA addresses the subject of the sensitivity of

stochastic models in a prescriptive way.

(foldover is explained below) enables the estimation of effects with the same accuracy as if

attention had been concentrated on varying a

single component throughout (Plackett and Burman, 1946, p. 305) and requires approximately

twice as many scenarios as parameters (components). The precise number of scenarios for the

folded-over design is 2 times that multiple of 4

which is next greater than the number of parameters. For example, with 10 parameters, 24 scenarios would be needed: 12 is the next larger multiple

of four, and 212 is 24. Thus, with 10 parameters PBSA provides a considerable savings over

the 1024 scenarios which would be required for a

complete factorial design. And PBSA is superior

to a half- (or quarter-) factorial design because, in

the case of 10 parameters, that design would still

need 512 (or 256) scenarios. Hence PBSA has the

advantage of efficiency while still being able to

detect interactions. Also, effects of a parameter

are not measured with all other things being

equal, but are averaged over variations made in

all other parameters.

Plackett and Burman devised their designs in

the context of assemblages of components in a

manufacturing process. Ergo, the terms component (manufacturing), factor (experimental design),

and

parameter

(modeling)

are

interchangeable in this discussion. Plackett and

Burman described the problem of determining

Table 1

Concise outline of method for PBSA of simulation model

(1) Determine parameters to be tested; select upper and

lower values.

(2) Find appropriate PB pattern.

(3) Create PBSA matrix.

(4) Run prescribed scenarios.

(5) Calculate (main) effect of each parameter on target

response variable.

(6) Sort effects; decide which are important.

(7) Calculate effects of two-way interactions of paired

parameters.

(8) Sort interaction effects; decide which are important.

(9) Interpret results.

173

suitable tolerances for the components of a certain assembly; more generally of ascertaining the

effect of quantitative or qualitative alterations

in the various components upon some measured

characteristic of the complete assembly (Plackett

and Burman, 1946, p. 305) This description

is a reasonable definition of sensitivity analysis if

the measured characteristic is understood

to be an output variable from the model under

scrutiny.

To summarize, the rationale for PBSA includes

these points:

PBSA is not a OAAT method

PBSA finds 2-way interactions

PBSA is not restricted to any particular type of

model

PBSA is prescriptive, using pre-determined

designs

PBSA is efficient in terms of number of scenarios needed

PBSA designs for up to 100 parameters are

readily available

PBSA rankings are easy to compute

PBSA works with categorical as well as numerical parameters

PBSA does not require parameters to be considered over identical intervals

PBSA is statistically sound

Despite these virtues, PBSA does not appear to be

well known by ecologists. None of the references

located in a science citations search on the Plackett and Burman (1946) paper were from ecological

or biological literature.

Below is a guide for performing PBSA, accompanied by a detailed example. This tutorial is

intended for an audience conversant in the vocabulary of mathematics and modeling. No particular

statistical background is required. An outline of

the procedure (Table 1) provides an overview. The

guide consists of explanation for each of the steps

in the Table 1 outline. A sample analysis, crafted

to illustrate PBSA technique, follows the guide. A

conventional OAAT sensitivity analysis for the

Example Model follows the PBSA. The drawbacks of the OAAT are delineated. Further information on PBSA may be found in many standard

texts on design and analysis of experiments.

174

analysis

Enumerate a list of the models parameters,

ordered in any way which makes sense. Any

ordering may be used, but it must remain fixed

throughout the procedure. There is no particular

advantage to paring down the number of parameters n; all should be included in order to gain the

maximum benefit of the PBSA. For each parameter, select an upper and lower value. These values

should be the endpoints of a plausible range for

the parameter. The typical 910% rationale

should be followed only if it makes sense in the

models context. The ranges do not need to be

identically sized, but their selection should be

reasoned. The analysis will indicate the effects of

varying each parameter by the amount of the

chosen range.

pattern

A paper in Biometrika (Plackett and Burman,

1946) is the original source for PB patterns. These

patterns are lists, or strings, of (ordered) pluses

and minuses. The designs come in sizes which are

multiples of four. The article contains patterns for

all designs up to 100, except for 92 (for which see

Baumert et al., 1962). Texts on design and analysis of experiments (e.g., Montgomery, 1997) may

also have PB patterns which would accommodate

up to a couple dozen parameters. The actual

length of the string, i.e., the number of signs in

the list, is one less than the design size d. Select

any d for which the number of model parameters

n Bd; the smallest such d requires running the

fewest sensitivity analysis scenarios of the model.

It is not required that the smallest possible d be

used, only that d should be larger than n.

row of all minuses. The matrix at this point has d

rows. The foldo6er consists of another set of d

rows, having the exact opposite signs of the original rows. This completes the PBSA matrix; it has

2d rows, the last consisting of all pluses.

Before the foldover, the matrix contains a Resolution III design, a statistical term which means

that the main effects of individual factors are

separate from each another, but may be impossible to distinguish from some 2-way interaction

effects, i.e., they are aliased. Adding the foldover

raises the design to Resolution IV (Box and

Draper, 1987). Increasing the resolution means

that the main effects are not aliased with one

another nor with 2-factor interactions, although

some of the 2-factor interactions may be aliased

with one another. It is not difficult to resolve

apparent aliasing, with additional runs, but the

explanation of how to do so is beyond the scope

of this paper.

Each row of the PBSA matrix specifies a distinct scenario of the model. Each column is associated with a particular parameter/factor/

component. Column 1 is assigned to the first item

in the established parameter list, Column 2 the

second, etc. If the number of parameters is less

than the number of columns (i.e., nBd 1), ignore the surplus columns. The signs in the rows of

the PBSA matrix indicate whether the upper or

lower of the extremes chosen for each parameter

should be used in a particular scenario. Where

pattern has plus, use upper parameter value;

where pattern has minus, use lower parameter

value. Run the 2d scenarios specified and record

the results. If the model is stochastic, determination of the appropriate number of replicates is the

province of the modeler.

2.3. Create a Plackett Burman sensiti6ity

analysis matrix

The matrix is an array of signs. The top row is

the PB pattern. All the cyclic permutations of this

lower values for the parameters) will produce

output (target variables or quantities). The number of output variables will depend on the model

follows the procedure explained here. To determine the (main) effect of particular component/

factor/parameter, sum the (signed) outputs as

follows. Take the signs from the PBSA matrix

column associated with that parameter, attribute

the signs to the output values, row by row, sum

these numbers and then divide the sum by the

design size (d). Thus, for each parameter an effect

on the target variable has been given a PB score.

Each column of the PBSA matrix contains d

pluses and d minuses. An equivalent way to calculate the score is to average of all of the plus

outputs and subtract from this the average of all

of the minus outputs.

The major influences are caused by the parameters having the largest scores from the preceding

procedure. The standard interpretation of the

(main) effects is a change of this amount in

component/factor f produces a corresponding

change of indicated size in response variable r.

Alternately, it can be said that the ratio of the

size of the change in factor to the size of effect has

the observed magnitude. The sign of the score

indicates the direction of the change for the target

variable. Thus, positive scores indicate an increase

in the response when the parameter is raised and

negative scores mean a decrease in the response

accompanies an increase in the parameter.

of the parameters

To determine the effects of 2-way interactions

of pairs of parameters, find the appropriate sign

for the pair by multiplying the signs for the

individual parameters, row by row: if the signs are

the same, the result is plus, if they are different,

minus. The matrix of sign products thereby created will have C(n,2) columns, each column representing one of the possible pairs. Associate these

derived signs with the output for each scenario

run (row by row, as for main effects); sum the

(signed) outputs, divide the sum by d, and sort the

results. Identical scores may occur for more than

175

paragraph).

For each target variable, the influence of pairs

can be determined, subject to aliasing. That is,

some of the effects are actually the consequence of

influence from more than one pair. Those pairs

having the same PBSA score are members of an

alias group. For tractability, assume that the pairs

within an alias group whose constituent elements

were determined individually to be most significant are the ones responsible for the main portion

of the groups effect and the other pairs may be

disregarded sparsity of effects principle (Montgomery, 1997). But, in truth, the observed effects

are the result of a linear combination of the pairs

having the same outcomes from the calculation

procedure. Thus, the assumption is that the coefficients for some of the terms in the linear combination are very small, making those terms negligible.

are important

Use the same process as that applied to the

individual parameter effects, explained above.

The standard interpretation of the 2-way interactions is changes of these amounts in components/factors/parameters f and g together produce

a corresponding change of indicated size in response variable r. This effect is over and above

the individual influences of factors f and g. At a

more sophisticated level of interpretation, a normal probability plot of the effects may be used to

draw a line between those large enough to be

real and those that could be due to randomness

in the model runs (Daniel, 1976).

Meaning and consequences of the analyses lie

within the context of the model and remain the

judgment of the investigator.

sensitivity analysis to a simple model

The example here has been drastically sim-

176

Table 2

Parameter values selected for PBSA of Example Model

Parameter

Lower value

Upper value

S0 (chick survival)

S1 (juvenile survival)

S2 (subadult survival)

S3 (adult survival)

DC (double-clutching)

RF (release fraction)

0.675

0.675

0.765

0.85

No

0.7

0.825

0.825

0.935

0.99

Yes

0.9

plified from a real population model. It is designed to focus on the PBSA procedure and allow

uncomplicated explanations. The example is

derived from a detailed model of an endangered

North American bird which has its population

augmented through a captive breeding program.

However, the point of this paper is to focus on

the PBSA features and technique, not the particular model or species. The example has been

severely pared down to present a useful illustration; it is not intended to represent reality. A full

analysis of the actual model utilized 128 combinations of its 57 parameters and reported on three

response variables (Beres, 2000). The details of the

model structure are unimportant for the understanding of PBSA methodology and are not discussed here. The Example Model predicts the

number of individuals after a designated period of

time in a small population of long-lived birds. The

population has two components, captive and wild.

Annual releases from a captive-breeding program

augment the wild population. Outputs other than

population are available from the actual model,

but one response variable suffices for purposes of

demonstrating the execution of PBSA. The size of

the wild (sub)population at the end of a 30-yr

period is the response variable used here. It

should be observed, however, that the PBSA procedure is equally applicable to all sorts of manyparametered ecological models, not only to

population models.

The example analysis examines six parameters:

annual survival probability for wild birds in each

management should include double-clutching

(double-clutching, commonly used to increase a

captive population faster than normal, refers to

removal of the first clutch of eggs from a nest in

order to promote the laying of a second clutch) in

the captive population (DC), and the portion of

the years captively-bred juveniles to release into

the wild (RF). Extreme values have been selected

for each parameter (Table 2). The ordering, S0,

S1, S2, S3, DC, RF remains fixed throughout the

procedure. Observe that the ranges selected for

the survival parameters are not uniform in size.

Note that two of the parameters (DC, RF) are

not attributes of the population itself, but are

management decisions about the population. Furthermore, one of these (DC) is essentially a

switch, i.e., either DC is in effect (on) or not

(off); there is no partial DC.

pattern

Consulting Plackett and Burman (1946) provides the string of seven signs [+ + + +

] which corresponds to a PB design of size eight.

If there were eight parameters, design size 12

(having 11 signs) would be needed. In that case,

three columns of the constructed matrix would be

ignored. It is convenient to disregard the final

columns, but any three could be inactive. For

the example, six of the seven design columns are

needed; the 7th is unused.

analysis matrix

The pattern itself forms row 1, its cyclic permutations form rows 27. In this case, the permutations were cycled to the right. It makes no

difference to the results of the analysis which

direction is used. Row 8 is all minuses, completing

the top half of the desired matrix (Table 3). Next,

each of the signs in the top half is switched to give

an additional eight rows, and these are appended

to the previous matrix, yielding the PBSA matrix

(Table 4). Note that the final row is all pluses;

there are 16 rows and seven columns.

The PBSA matrix has 16 rows hence 16 scenarios are necessary for the analysis. Each of the first

six columns is associated with a parameter and

each row specifies the parameter settings for a

scenario. The 7th column is disregarded. Hence,

combining Table 2 (transposed) and Table 4 specifies the actual parameter settings for the required scenarios of the model (Table 5).

Replication of each scenario must occur at an

appropriate level for the model being analyzed.

Each scenario of the Example Model was run

with 1000 replicates using the same settings. The

Table 3

Top half of PBSA matrixa

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

Table 4

Complete PBSA matrixa

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

a

Top half is identical to Table 3. Lower half is formed by

foldover: signs in the lower eight rows are opposite to those in

upper eight rows. Columns l6 associate with the parameters:

S0, S1, S2, S3, DC, RF. Final column is unused.

177

target response variable P, wild population after

30 years.

To determine the effect of S3 on P, take the

vector of signs from Column 4 of the PBSA

matrix (Table 4) and associate row by row with

the output numbers from the population column

of Table 5. The sum of the resulting signed means

[53.4+128.9+126.8 + 62.2 17.8 25.4+ 39.0

+ 235.6 + 33.1 12.1 11.1 25.2 + 86.9 + 61.5

40.85.9] is 582.1. Divide the sum by eight

which is the design size d. The main effect of S3

on P is thus determined to be (582.1)/8 = 72.8.

Explanation is given in the next section. The

actual output numbers and computations contained three decimal places; only one is reported

here.

In the same way, calculate the (main) effects on

population for the other parameters (Table 6). If

there are multiple output variables of interest,

their values as derived from the same 16 scenarios

may be used to calculate effects on them. Additional scenarios are not required. A spreadsheet,

such as EXCEL, makes the computations quite

simple, even for a large number of parameters and

output variables.

Sorting the scores from the previous step gives

a ranking for the main effects (Table 7).

Examination of Table 7 indicates that all of the

effects are positive. Intuitively, the effects of improved survival should be positive on the population, so this seems reasonable. The analysis

provides evidence that the two management

strategies have the potential to increase the population over the time span modeled. However, the

most significant effects are shown to be the survival of birds in the two highest age classes, S2

and S3.

The magnitude of the effects for S2 or S3 is

seen to be approximately twice that of the next

highest parameter DC. Survival for the youngest

178

Table 5

PBSA parameter values for Example Model, with population (response variable) outputa

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

Scenario

a

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

S0

S1

S2

S3

DC

RF

Population

0.825

0.675

0.675

0.825

0.675

0.825

0.825

0.675

0.675

0.825

0.825

0.675

0.825

0.675

0.675

0.825

0.825

0.825

0.675

0.675

0.825

0.675

0.825

0.675

0.675

0.675

0.825

0.825

0.675

0.825

0.675

0.825

0.935

0.935

0.935

0.765

0.765

0.935

0.765

0.765

0.765

0.765

0.765

0.935

0.935

0.765

0.935

0.935

0.85

0.99

0.99

0.99

0.85

0.85

0.99

0.85

0.99

0.85

0.85

0.85

0.99

0.99

0.85

0.99

Yes

No

Yes

Yes

Yes

No

No

No

No

Yes

No

No

No

Yes

Yes

Yes

0.7

0.9

0.7

0.9

0.9

0.9

0.7

0.7

0.9

0.7

0.9

0.7

0.7

0.7

0.9

0.9

53.4

128.9

126.8

62.2

17.8

25.4

39.0

5.9

33.1

12.1

11.1

25.2

86.9

61.5

40.8

235.6

( 914.4)

( 926.4)

( 925.1)

( 914.9)

( 96.6)

( 98.9)

( 911.5)

( 93.4)

( 99.6)

( 95.5)

( 95.5)

( 98.7)

( 920.6)

( 915.1)

( 911.9)

( 946.3)

Population numbers (birds) are means ( 9 S.D.) over 1000 replicates having the settings specified by the particular scenario.

Table 6

Effects of parameters on response variable P (population)

S0

S1

S2

S3

DC

RF

10.7

22.4

60.0

72.8

31.8

18.0

Table 7

Effects (of parameters on response variable) in descending order

S3

S2

DC

S1

RF

S0

72.8

60.0

31.8

22.4

18.0

10.7

release fraction RF has a relatively small

influence.

Specifically, for the adult age class S3, increasing the survival by 0.07 is expected to provide the

wild population with approximately 35 more birds

(not necessarily adults) over the modeled time. To

draw this conclusion, find the point halfway between the extreme values chosen for S3 (Table 2),

(0.99 0.85)/2 =0.07. This represents the amount

of change that was under consideration in the

PBSA procedure. Hence, this amount of change

(up or down) is found to have an effect on the

wild population of (72.8)/2 =36.4, or about 35.

The direction of the change in S3 and the anticipated increment in Population is the same; if S3 is

decreased by 0.07, then the result would be 35

fewer birds. For survival in the youngest age class

S0, the effect of a increment of 0.075, either up or

down, is a change of approximately five birds, in

the same direction, at the end of the time period

modeled. The size of the change for S0 was calculated by (0.8250.675)/2 = 0.075 and the score

10.7 was divided by two to get the size of the

related effect. Although the increment in S0 is

larger than the one is S3, it has a smaller effect.

Approximately 30 more birds in the wild population (over the modeled time) is the anticipated

Scenario (row)

S3S0

S3S1

S3S2

S3DC

S3RF

S2S0

S2S1

S2DC

S2RF

S1S0

S1DC

S1RF

S0DC

S0RF

DCRF

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

a

Signs derived by multiplying the appropriate columns of Table 4. Pairs, obtained from all combinations of two parameters, may be placed in any sequence, and order of the 2 elements

within any pair does not matter.

Table 8

Signs used to calculate effects of parameter pairsa

179

180

effect of including double-clutching in the husbandry for the captives. Thus, eliminating doubleclutching would reduce the size of the population

by 30 compared to what would otherwise be

predicted. Because DC is a discrete parameter,

there is no amount of change to calculate, as

can be done for the survival parameters.

of the parameters

Two-way interactions are the consequence of

parameter pairs acting together. There are C(6,2)

or 15 such pairs for the Example Model. Find the

appropriate signs for the pair effects and then

associate the signs with the output similarly to

above. The appropriate sign for the pair is found

by multiplying the signs for the individual

Table 9

Interaction scores for all parameter pairs in Example Model

S3S0

7.6

S3S1

16.6

S3S2

35.6

S3DC

17.7

S3RF

18.4

S2S0

9.2

S2S1

18.4

S2DC

15.7

S2RF

16.6

S1S0

15.7

S1DC

9.2

S1RF

35.6

S0DC

18.4

S0RF

17.7

DCRF

7.6

Table 10

Interactions by decreasing impacta

Pair

Interaction effect

S1RF

S3S2

S0DC

S2S1

S3RF

S0RF

S3DC

S2RF

S3S1

S1S0

S2DC

S1DC

S2S0

DCRF

S3S0

35.6

35.6

18.4

18.4

18.4

17.7

17.7

16.6

16.6

15.7

15.7

9.2

9.2

7.6

7.6

if opposite, minus. Thus, for the pair S3 S0 the

(Table 4) first row product is plus times minus

which gives minus. The 5th row product is minus

times minus which gives plus, as does the 7th row

plus times plus. The second column in Table 8

contains the string of signs which arise from

pairwise multiplication of the sign columns for S3

and S0: [ + + + + + +

+ + ]. Find the remaining signs in the same way.

In the same manner as for the main effects,

associate the column of signs with the output

population numbers row by row. The PB scores

for the interaction effects are then calculated as

before. Sum each of the 15 columns and divide

each sum by eight, the PB design size. Thus, for

S3 S0, add [ 53.4 128.9 126.8+ 62.2+

17.8 25.4 + 39.0 +235.6 33.1 12.1 11.1

+25.2+86.961.5+40.8+5.9] to arrive at

61.0. The division by eight results in 7.6 (Table 9).

As before, actual output numbers and computations contained three decimal places; only one is

reported here. Note that the same outputs, in this

case means over 1000 replicates from 16 distinct

scenarios, are used for the computations in both

the main effects and the 2-way interaction effects.

are important

When all 15 interaction effects have been calculated (Table 9), they may be sorted (Table 10).

The situation at this point is not as straightforward as was the interpretation of the main effects.

There are some pairs and even a trio of interactions having the same PB score, the result of

aliasing. The S1 RF pair together with the S3

S2 pair actually produce the effect with magnitude

35.6. However, since the S3 and S2 main effects

were large but the S1 and RF main effects were

modest, the sparsity of effects assumption mentioned earlier may be invoked. The simplifying

assumption is that the part played by the S1 RF

pair may be disregarded and the response attributed to S3S2. Thus, the pair S3 and S2

acting in concert produce an effect in addition to

the individual effects produced by S3 and S2.

Hence, when both S3 and S2 are augmented by

Table 11

Selected interactions in decreasing ordera

Pair

Interaction effect

S3S2

S2S1 or S3RFb

S3DC

S3S1 or S2RFb

S2DC

S2S0

S3S0

35.6

18.4

17.7

16.6

15.7

9.2

7.6

a

One pair from each alias group was chosen, by importance

of individual effects. Only pairs involving S3, or S2 were

included.

b

If it were deemed necessary, the question over which of the

pairs is chiefly responsible for the effect could be resolved.

half the distance between their plus and minus PBSA values, the population should be increased by 84.2, the sum of 36.4, 30.0 and 17.8, or

half the sum of 72.8, 60.0, and 35.6. Observe that

the interaction effect S3S2 has similar magnitude to the individual effect of DC (35.631.8).

If the interactions are pared down to those

concerning only the two parameters whose main

effects were the largest, a simpler table results

(Table 11). A reduction of this sort (based upon

the sparsity of effects principle) may be useful for

situations involving a large number of parameters.

Further consideration of the relative sizes of the

remaining pairs (Table 11) leads to the observation that only the S3S2 seems large enough to

matter much. However, if it were considered desirable to resolve the question of attributing the

major portion of the effect from a particular alias

group, this could be accomplished with some additional runs. Discussion of this refinement is

beyond the scope of the present paper.

As stated above, it is the responsibility of the

modeler to make sense of the outcomes of the

analysis within the context of the model. As each

effect is examined, the assumption is that the

effect is linear over the interval being considered.

In carrying out an actual PBSA, it might be

more productive to integrate the two ranked listings of main effects and interaction effects prior

181

were segregated here to facilitate explanations,

but in practice they should be considered

together.

4. Discussion

In the case of the example, recommendations

for the management are possible, as well as advice

regarding data collection. If the most significant

of the parameters does not have values based on

good data, then the suggestion would be to obtain

more/better data, if possible, or to be cautious in

the decision-making if actions must be taken without more data. If there is confidence in the quality

of the data, then the counsel would be to do all

possible to maximize survival for the two upper

age classes. Furthermore, double-clutching in the

captive population appears to be a useful strategy

because the effect of DC, though smaller than S3

and S2, is still appreciable. If management questions involve making choices, the PBSA may assist in ranking them. For example, it would be

more advisable to choose actions which ensure a

high value for S2 than to expend resources on

improving S1. Debate about small changes in RF

would not be productive; RF makes little difference in population. Because they are derived from

the analysis of the Example Model, all of these

conclusions are hypothetical. They are merely intended to illustrate the scope of inference.

While PB with foldover provides a clear estimate of the main effects and has a very good

chance of finding the important interactions (Box

et al., 1978), it does not guarantee uncovering the

latter. The tradeoff between efficiency and complete information must be acknowledged. Also,

apparent interaction effects may be confounded

(aliased) and thus, care must be used in interpretation of the output from the design (Montgomery, 1997).

5. Comparison

To compare with the PBSA, a conventional

OAAT sensitivity analysis was performed on the

182

For each of these except DC, a decrease of approximately 10% from its default (usual or best

estimate) value was explored. DC is categorical,

so 10% is meaningless in its case. The default and

tested (alternate) values are displayed in Table 12.

The ranking of the parameters from this analysis (Table 13) is the same as the main effects

ranking derived from the PBSA (Table 7). The

positions in the list were determined by finding

the ratios of the amount of change to the default

result. The nature of DC makes it impossible to

calculate (F(x +Dx) F(x))/Dx and, thus, percent changes for all parameters were calculated

instead. The relative importance of the effects for

DC and S1 is different. Most importantly, OAAT

fails to provide any information about

interactions.

Table 12

Parameter values used for OAAT sensitivity analysis of Example Model.

Parameter

Default

Alternate value

S0 (chick survival)

S1 (juvenile survival)

S2 (subadult survival)

S3 (adult survival)

DC (double-clutching)

RF (release fraction)

0.75

0.75

0.85

0.95

Yes

0.8

0.675

0.675

0.765

0.85

No

0.7

Table 13

OAAT sensitivity analysis outputs (mean 9 S.D.) from Example Model, ranked by percent change

Scenarioa

Population

All defaults

S3

DC

S2

S1

RF

S0

72.0

27.1

41.5

41.7

61.3

65.0

67.4

( 9 17.3)

( 9 8.8)

( 912.3)

( 911.9)

( 9 15.4)

( 915.4)

( 916.3)

%

Changeb

na

62.4

42.4

42.1

14.9

9.7

6.4

Because there is no delta for DC, percent changes were

calculated for each parameter: the change in population was

divided by the default population.

b

6. Conclusion

PBSA makes possible the consideration of a

multitude of parameters simultaneously. It requires a modest number of scenarios (approximately twice the number of parameters) in

comparison to those that would be needed for a

complete factorial design. There is no need to

restrict the analysis to the presumed most powerful parameters. All parameters may be easily investigated. Most importantly, PBSA enables the

modeler to uncover 2-way interactions which

might not otherwise be suspected to be influential.

In setting up PBSA, the parameters do not need

to be tested over identical intervals, but rather the

upper and lower values chosen for the analysis

should be selected as the reasonable extremes for

the particular parameter. Categorical parameters,

such as yesno decisions, may be included in the

analysis.

The main effects are clearly delineated in the

analysis, and are separate from the 2-way

interactions, as well. The interaction effects

may be aliased, but further analysis (not

presented here) makes it possible to tease these

apart, if desired. Within an alias group the effects

are caused by a linear combination of the pairs

involved. The sparsity of effects principle

allows the assumption that the greatest contribution is made by the pair whose individual

components were of greatest importance. However, these individual parameters may not be the

most influential of all single factors. Thus, the

PBSA may expose otherwise undetected

effects. PBSA is simple, convenient, and informative. For ecological modelers, PBSA is an essential tool.

Acknowledgements

During preparation of the manuscript,

valuable suggestions were made by Anthony M.

Starfield, Norbert J. Kuenzi, Robert G. Haight

and Karl A. Beres. The authors also thank two

anonymous reviewers for their useful recommendations.

References

Baumert, L., Golomb, S.W., Hall, M., 1962. Discovery of an

Hadamard matrix of order 92. American Mathematical

Society Bulletin 68, 237 238.

Beres, D.L., 2000. A methodological study of modeling for

California condors. PhD dissertation, University of Minnesota.

Beres, D.L., C. Clark, G. Swartzman and A.M. Starfield,

2001. Truth in modeling. Natural Resource Modeling 14

(3) in press.

Box, G.E.P., Draper, N.R., 1987. Empirical Model-building

and Response Surfaces. Wiley, New York, p. 669.

Box, G.E.P., Hunter, W.G., Hunter, J.S., 1978. Statistics

for Experimenters: an Introduction to Design, Data

Analysis, and Model Building. Wiley, New York,

p. 653.

Burgman, M.A., Ferson, S., Akcakaya, H.R., 1993. Risk

Assessment in Conservation Biology. Chapman & Hall,

London, p. 314.

183

Daniel, C., 1973. One-at-a-time plans. Journal of the American Statistical Association 68, 353 360.

Daniel, C., 1976. Applications of Statistics to Industrial Experimentation. Wiley, New York, p. 294.

Henderson-Sellers, B., Henderson-Sellers, A., 1996. Sensitivity

evaluation of environmental models using fractional factorial experimentation. Ecological Modelling 86, 291 295.

Jorgensen, S.E., 1994. Fundamentals of Ecological Modelling,

Second ed. Elsevier, Amsterdam, p. 628.

McCarthy, M.A., Burgman, M.A., Ferson, S., 1995. Sensitivity analysis for models of population viability. Biological

Conservation 73, 93 100.

Montgomery, D.C., 1997. Design and Analysis of Experiments. Wiley, New York, p. 704.

Plackett, R.L., Burman, J.P., 1946. The design of optimum

multifactorial experiments. Biometrika 33, 305 325.

Starfield, A.M., Bleloch, A.L., 1991. Building Models for

Conservation and Wildlife Management. Burgess International Group Inc., Edina, MN, p. 253.

Swartzman, G.L., Kaluzny, S.P., 1987. Ecological Simulation

Primer. Macmillan, New York, p. 370.

## Гораздо больше, чем просто документы.

Откройте для себя все, что может предложить Scribd, включая книги и аудиокниги от крупных издательств.

Отменить можно в любой момент.