Computers & Operations Research 33 (2006) 18 31

www.elsevier.com/locate/cor

A problem structuring front end for a multiple criteria decision support system
Ralph Scheubreina , , Stanley Ziontsb
a University of Hohenheim, Business Administration Institute (510A), 70593 Stuttgart, Germany b State University of New York at Buffalo, School of Management, Buffalo, NY 14260-4000, USA

Available online 6 July 2004

Abstract Structuring a problem is a key part of decision making. For multiple criteria decision problems, dening the criteria is an important element of the structuring process. To provide a decision maker with a general instrument for identifying relevant criteria, two methods representing different approaches were empirically evaluated. This evaluation showed that Kellys repertory grid technique in particular has several useful features. Accordingly, the repertory grid technique was adapted to build a problem structuring front end for the aspiration-level interactive method proposed by Lot, Stewart, and Zionts. 2004 Elsevier Ltd. All rights reserved.
Keywords: Aspiration-level interactive method (AIM); Repertory grid technique; Problem structuring; Multiple criteria decision making (MCDM)

1. Introduction Problem-solving activities pervade all aspects of human life. Solving a problem generally involves making one or more decisions. The resources invested in the problem-solving effort depend on various factors. Relevant to this article in particular is the perceived importance and the complexity of the problem under consideration. One aspect of complexity is how much problem information is available. In [1] this amount of information is used to provide a classication into the three broad classes: well-structured problems,
Corresponding author.

E-mail address: rscheubr@uni-hohenheim.de (R. Scheubrein) 0305-0548/$ - see front matter 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.cor.2004.05.014

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31 Table 1 Decision matrix with alternatives xi , criteria cj , and outcomes yi,j Criteria c1 y1,1 y2,1 . . . yi, 1 c2 y1,2 y2,2 . . . yi, 2 .. .

19

Alternatives

x1 x2 . . . xi

cj y1,j y2,j . . . yi, j

semi-structured problems, and ill-structured problems. A well-structured problem is understood and an appropriate solution procedure may be formulated. In contrast, an ill-structured problem must be dened in order to solve it. A semi-structured problem is in between these two extremes; i.e., some aspects of the problem are well-structured whereas others are not. In order to solve complex problems many processes have been proposed (see, e.g., [2] for a survey). Such processes may be interpreted as a means of gradually transforming an ill- or semi-structured problem into a well-structured one. This is achieved by gathering relevant information, organizing it according to a specic scheme, and evaluating it accordingly. In many real life problems, choosing between possible courses of action may be difcult because it requires balancing several factors. Multiple criteria decision making (MCDM) deals with situations in which the decision maker has several conicting objectives (see, e.g., [3] for a recent survey). There is generally no perfect alternative, and a good compromise must be identied. Methods for supporting MCDM can be divided into two broad classes [4], namely into approaches for multiple objective decision making (MODM) [5] and for multiple attribute decision making (MADM) [6]. In MODM the alternatives are often dened implicitly, e.g., by the restrictions of a mathematical program. For MODM problems the decision maker must come up with or design the most preferred alternative by assigning values to decision variables. For MADM problems, the decision maker must choose from a set of alternatives that is typically dened explicitly. Of course not all the alternatives may be known a priori. In other words, the decision maker selects a most preferred alternative. For MADM in a deterministic context, a central data structure is the decision matrix (see Table 1). In this matrix the alternatives xi X, the j criteria cj C , and their outcomes yi,j := cj (xi ) are recorded. i If the decision matrix is known, the dominance relation on the alternatives can be identied. An alternative is said to dominate another if the rst alternative is at least as good as the second in every criterion and strictly better in at least one criterion. Alternatives which are not dominated by any other alternative are called nondominated alternatives. A main concern of MCDM is how to structure a decision problem [7]. With respect to the decision matrix, an ill-structured problem may be characterized by the fact that some alternatives, criteria and perhaps also outcomes are unknown. To structure the problem, these elements of a decision matrix have to be determined as part of the decision making process. Under the umbrella term Soft OR, many problem structuring procedures have been developed (see, e.g., [7] for a survey). Such approaches to problem structuring are often extended procedures, emphasizing the creativity of a group of people, and are typically organized by a facilitator or a decision analyst. In that literature, the recommendation is often given to concentrate rst on the criteria and then to dene the alternatives. A typical MCDM-related example is Keeneys value-focused thinking [8]. He argues

20

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

that decision making should always start by focusing on the criteria even if some alternatives seem to be obvious ([9, p. 537] values rst principle). For decisions with a strategic impact (e.g., the case presented in [9, p. 538543]) an extensive effort seems to be justied. Many decisions in an organization are made on a much lower, operational level where fewer resources are available. Therefore it seems interesting to check whether the values rst principle is also appropriate for an operational decision situation. Accordingly, a eld study on operational decision making of students was carried out. In the following section, the rst phase of the eld study, which evaluates two methods of supporting problem structuring, is described. Next, the acceptance of a multiple criteria decision support system, which implements the aspiration-level interactive method (AIM) [10], is tested. The result of the second phase of the eld study is presented in the third section. In the fourth section a decision support system which integrates a problem structuring approach and the aspiration-level interactive method is presented. The evaluation of that system represents the third phase of the eld study. A brief summary in the fth section concludes the article.

2. Evaluation of two methods for problem structuring Decisions can be broadly classied into strategic, tactical, and operational depending on their overall impact. Strategic decisions are higher-level and longer-term, whereas operational are lower-level and short term. Tactical decisions are in between. An operational decision situation usually has the following characteristics: The decision is de facto made by one person. The time to make the decision is limited and is relatively short. No external analysts are usually involved because of time and money restrictions. For operational decisions, a decision maker might be interested in a systematic procedure to elicit relevant criteria. As a basis for creating and evaluating an appropriate procedure, a eld study was conducted. The eld study consisted of three phases: Phase 1: Evaluation of two methods which represent opposing problem structuring methodologies. Phase 2: Evaluation of a multiple criteria decision support system without a problem structuring front end. Phase 3: Evaluation of an integrative system which is based on the conclusions from the previous phases. The subjects participating in the eld study were students of a university business school. Table 2 summarizes the demographic data of these participants. In order to mimic an operational decision situation, the subjects were asked to prepare a typical decision problem which they were currently facing. The rationale is that the students should be actively and seriously involved in the decision problem. The problem most often presented was choosing a car to purchase (22% of all problems in this phase of the eld study). Other problems included choosing a graduate school, choosing a major, taking a leave

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31 Table 2 Demographic data of the study participants Subjects Age 37 (56% female, 44% male) Minimum 18 Freshman 12% US 84% Mean (Std. Dev.) 23.7 (6.07) Sophomore 9% Junior 55% Other 16%

21

Maximum 44 Senior 24%

Year

Nationality

of absence from their studies (or not), deciding on involvement in university club activities, moving to a new location, deciding where to go and how to travel during a school holiday, choosing a watch to purchase, and choosing a snowmobile to purchase. In accordance with a typical decision situation on an operational level, all the discussed problems had a single decision maker. For those students who had inappropriate problems (e.g., not enough data or incomplete information), the default problem of selecting a cell phone contract was used. The data of this case study was based on actual plans of six cell phone companies operating in the Buffalo, NewYork area. The data was condensed into a decision matrix having 22 alternatives and 17 criteria. This decision problem was used by 28% of the students during the rst phase of the study. Taking into account both the necessary investment and the long-term effect of the decision, 67% of the problems discussed during this phase can be characterized as operational decisions for a student whereas the other problems are more tactical or strategic. Each subject was supported by a facilitator who directed the subject to work according to one of the two methods under consideration. The facilitator avoided acting as a decision analyst who would have tried to help structure and solve the decision problem. This role of the facilitator reects that for an operational decision, normally no external analysts can be involved. In order to simulate a time restriction, each subject was instructed to work on the problem for one hour. Within this time limit, the subject was asked to structure the problem by articulating the available alternatives, the relevant criteria, and the respective outcomes. Once this data was provided, the facilitator generated the decision matrix. Next, the facilitator identied the nondominated alternatives. Working on a decision problem was stopped after 75 minutes, even if the problem structuring process had not been completed. The focus of the rst phase of the eld study was the comparison of two different methods of eliciting the alternatives and the criteria, namely the KepnerTregoe approach and the repertory grid technique. The KepnerTregoe [11] approach for management decisions can be characterized as criteria-oriented, following the values rst principle. This approach originally consists of two phases: problem analysis and decision making. The problem analysis phase is not considered here in more detail, because the subjects participating in the eld study had already identied their problems. The decision making phase consists of seven stages (see [2, p. 336342] for details): (1) Establish objectives. (2) Classify objectives into must and want requirements.

22

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

Table 3 Typical layout of a repertory grid Constructs ( = 1) C1 C2 . . . Cj Elements E1 E2 ... Ei Contrasts ( = 5) 1 C 2 C . . . Cj

(3) (4) (5) (6) (7)

Develop alternatives. Compare alternatives using the objectives. Make tentative selection. Evaluate adverse consequences. Choose and implement the best alternative.

In contrast, the repertory grid technique can be characterized as an alternative-oriented method. It was developed by Kelly [12] in the context of the theory of personal constructs in the eld of psychology. The theory has its origin in the need to understand and map an individuals thinking process. Giving a complete overview of the theory is beyond the scope of this article. The essence relevant to this article is that an individuals expectations may be considered as a nite set of personal constructs which the person uses to evaluate a particular phenomenon. Each construct is bipolar; i.e., a constructcontrast pair forms a dimension for assessing a phenomenon. As a means of operationalizing the personal construct theory, the repertory grid technique is intended to allow an individual to exhibit his idiosyncratic constructs with a minimum of interviewer bias. The repertory grid technique has been used in a variety of business contexts reecting the exibility of this procedure (see [13,14] for surveys). In the context of MCDM, in [15] a procedure based on the repertory grid technique is used to assess the importance of criteria. While there are various modications of the repertory grid technique, three phases are typical [16]: (1) Identication of the elements of the considered phenomenon. (2) Elicitation of the distinguishing constructs. (3) Construction of a grid using the evaluation of the elements and the constructs. To elicit the constructs, the triad method is commonly used. The interviewee is confronted with three elements and asked to consider ways in which two are alike but different or opposite from the third (see, e.g., [17] for details). Table 3 shows the typical layout of a repertory grid (adapted from [14, p. 40]). First elements under consideration are recorded in the header of the grid. Then, using the triad method, a construct is elicited. The two poles of each construct are written on the left side and the right side, respectively. If an element matches the left-side pole it is rated with 1; if it matches the pole on the right side it is rated with 5. If an element falls in between then a rating in the range from 2 to 4 is given. For each subject in the rst phase of the study, the facilitator used either the KepnerTregoe approach or the repertory grid technique. The subject then tried to structure the problem using the method introduced.

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31 Table 4 Comparison of elicitation methods KepnerTregoe 7 8 7.7 2 5 3.7 33%

23

Criteria elicited

Minimum Maximum Mean Minimum Maximum Mean

Repertory grid 3 7 6 3 7 5.5 87%

Alternatives elicited

Time restriction met

The facilitator supported the subject only in questions related to the method but not in structuring the problem. Table 4 shows the result of this experiment. On average, when using the KepnerTregoe approach, the subjects identied more criteria; whereas when using the repertory grid technique, the subjects generated more alternatives. This reects the sequence of activities of the methods; i.e., a higher output was generated by the subjects in the earlier stages of each method. One important difference between the two methods is the time required. When using the repertory grid technique it was almost always possible to complete the problem structuring process within the given time frame, whereas when using the KepnerTregoe approach, in only 33% of the experiments the subject was able to complete the problem structuring process within the given time frame. Based on the above result, it appears that the repertory grid technique is preferable if tight deadlines must be met. The decision to implement the repertory grid technique as the problem structuring method for a multiple criteria decision support system was made. The methodology of this decision support system is presented in the next section.

3. The Aspiration-level interactive method AIM [10] is an MADM approach intended to help identify the most preferred alternative in a decision matrix. The basic idea of AIM is that a decision maker explores the efcient frontier by interactively adjusting levels of aspiration and obtaining feedback. For each aspiration level, the system gives the nearest nondominated solution, as well as a ranking of alternatives, based on a metric constructed using the current levels of aspiration. The approach used in AIM can be easily adapted to various contexts (see [18, Chapter 10] for some cases studies). In [19] AIM is used to support the selection of strategic initiatives of a Balanced Scorecard. Ref. [20] compares AIM and the conjoint analysis for predicting consumer choices. While the original AIM software provides various functions for supporting a decision maker (see [10] for details), the process basically consists of four steps as depicted in Fig. 1. First, the decision maker denes active alternatives. Next the decision maker has to specify the criteria. Basically, three criteria function types are supported: maximizing, minimizing, and target (see Fig. 2). The decision maker can also choose a qualitative criterion which is a maximizing criterion with a ve-point

24

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

Fig. 1. Decision process supported by AIM (straight process of ExcelAIM).

Fig. 2. Criteria function types.

scale: i.e., an outcome is rated as awful, poor, fair, great, or superb. For each criterion, the decision maker can specify up to four parameters to express his perception of the value of an outcome. With the denition of the alternatives and criteria, the system can set up a decision matrix and the decision maker enters the outcomes during the third phase. Finally, the decision maker interactively explores the efcient frontier by adjusting levels of aspiration. The system supports this adjustment by providing various feedback information on the so-called basic display. Based on the current levels of aspiration, the system reports the proportion of alternatives that satisfy the level with respect to each criterion individually and to all criteria when considered together. In addition, the basic display shows the nondominated alternative having the minimum distance from the current aspiration level. Also, the system can compute a ranking of alternatives using a metric based on the current aspiration levels (see [10] for details). Using information provided in the basic display, the decision maker adjusts the levels of aspiration until he nds an acceptable compromise in the set of alternatives. Fig. 3 shows an example of this decision process for a problem with two minimizing criteria. In this example, the decision maker starts at the nadir point in the upper right corner. He then adjusts his levels of aspiration six times until he nds an alternative which he views as the best compromise. The numbered positions on the right side of Fig. 3 are the levels of aspiration chosen in each iteration. The decision maker cannot select a level of aspiration arbitrarily but has to choose an achievable level for each criterion. As shown on the right side of Fig. 3 the levels of aspiration can only be set at grid intersections. The idea is that the selected levels of aspiration are always related to alternatives. In addition to this interactive adjustment procedure, AIM has other procedures included (see [10] for details). AIM was implemented as a DOS-based application some years ago [10]. For the eld study it was reimplemented in Microsoft Excel 2002, which is part of Ofce XP. This new implementation is named ExcelAIM. In this system, the data is stored in standard Excel worksheets, while the functions for the interactive adjustment procedure are implemented in the programming language Visual Basic for Applications which is integrated in the Microsoft Ofce product family. The source code of ExcelAIM

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

25

Fig. 3. Example of adjusting the levels of aspiration.

has a size of 250 KB, while a complete workbook requires about 1 MB. One of the main advantages of an application based on a spreadsheet package is that today many end-users are sufciently computer literate to be able to modify the application according to their specic needs (see [21] for an empirical study). Additionally, spreadsheet applications can often be integrated easily in a working environment. In particular, recent versions of Excel provide functions for accessing data stored in XML-format and in HTML-format (see [22, Chapter 27] for implementation details). This feature is useful for a decision support system in a business context because a lot of information is today accessible in this way both in the companys intranet and the Internet. In addition to this spreadsheet implementation an Internet version is currently under development (see http://www.scheubrein.com/webaim/). ExcelAIM implements two decision processes, namely the straight process and the standard process. The straight process of ExcelAIM basically implements the AIM approach as presented above (see Fig. 1). This process can be used if all data necessary for the decision matrix is available and the decision maker can proceed to directly solve his problem. In the second phase of the eld study, all subjects were asked to use the straight process of ExcelAIM to select a cell phone contract from the prepared case study. As soon as the subjects had made the decision using ExcelAIM, they completed a questionnaire. This questionnaire was inspired by the one used in [23] to evaluate commercial multiple criteria decision support systems. The subjects were asked to answer the following questions on a ve-point scale, for which one point indicates the worst rating and ve points the best rating. (1) (2) (3) (4) (5) (6) (7) (8) (9) How demanding is it to get the program running? How easy is it to use the program? How comprehensible is the process? How reasonable is the process? Did you learn much about the decision problem? How easy is it to tell the system what is relevant in making the decision? How strong is your condence in the process? How strong is your condence in the result of the process? How user-friendly is this program?

26

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

Table 5 Evaluation of ExcelAIMs straight process Question 1 2 3 4 5 6 7 8 9 10 11 12 Overall Minimum 3 3 2 3 2 2 3 4 2 2 2 2 2 Mean (Std. Dev.) 4.2 (0.83) 4.2 (0.73) 3.9 (1.04) 4.0 (0.82) 3.4 (0.87) 3.9 (1.12) 4.3 (0.75) 4.4 (0.51) 3.8 (1.07) 4.0 (1.00) 4.2 (0.93) 3.8 (1.14) 4.0 (0.27) Maximum 5 5 5 5 4 5 5 5 5 5 5 5 5

(10) How useful is this system for supporting decision making? (11) How satised are you with this methodology for decision making? (12) Would you make the recommendation to a friend to use the system for this decision problem? These questions cover a broad range of aspects which inuence the acceptance of a decision support system. In particular, there is one group of questions related to the implementation of the software, one group about the methodological approach to supporting the decision making, and a third group concerning the realization of the approach in the software under consideration. Table 5 summarizes the results of the subjects assessments. Overall, the mean assessment of all questions is 4.0 (with a standard deviation of only 0.27), which indicates a high acceptance of both the approach and the implementation. The subjects had a very high level of condence in the result of the decision making process (see question 8 in Table 5). In addition to the straight process, ExcelAIM also supports a second process. The details of that process and its assessment by the subjects are presented in the next section.

4. Problem structuring and decision making with ExcelAIM ExcelAIMs standard process is an extension of the straight process discussed earlier. The standard process helps the decision maker identify criteria that are relevant by using the repertory grid technique. Fig. 4 shows the steps of this process.

Fig. 4. Standard process of ExcelAIM.

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31 Table 6 Example of the temporary decision matrix after the elicitation of three criteria using the triad method x1 x2 x3 x4 x5 x6 c1 1 1 1 0 0 0 c2 0 1 1 1 0 0 c3 0 1 1 1 0 0

27

1 1 3 2 0 0

Table 7 Example of the temporary decision matrix after the elicitation of an additional criterion using the paired comparison method x1 x2 x3 x4 x5 x6 c1 1 1 1 0 0 0 c2 0 1 1 1 0 0 c3 0 1 1 1 0 0 c4 0 0 0 0 1 1

1 1 3 2 1 1

First the alternatives and criteria are elicited. As a starting point the decision maker has to provide at least three alternatives. Next the triad method of the repertory grid technique is used to differentiate the alternatives. The decision maker selects three alternatives and inputs a criterion in which two are similar (i.e., both are rated either as good or as bad) and one is the opposite. This input is stored by the system in a temporary decision matrix. For each assessment of type good, an outcome of 1 is stored; for each assessment of type bad, an outcome of 1 is stored. For each alternative that is not an element of the triad that produced the criterion, an outcome of 0 is stored. The rationale for these values is that the decision maker probably selects extreme alternatives to dene a criterion and therefore the value 0 is used to approximate the outcomes of the probably non-extreme alternatives. Table 6 shows an example with six alternatives and three criteria. In this example the rst criterion c1 was elicited with the triad x1 (rated good), x2 (rated bad), and x3 (rated bad). The second and third criterion is created similarly in this example. This elicitation using the triad method continues as long as the decision maker can identify additional criteria. Next, the system computes the sums of the columns in the temporary decision matrix (last column in Table 6). If two alternatives are having a comparable overall score, the system presents this pair to the decision maker and asks if they can already be differentiated using the available criteria, or if an additional criterion is required. In the example in Table 6 the rst pair presented to the decision maker would be x5 and x6 . Table 7 shows the temporary decision table after the decision maker has provided a new criterion c4 and rated the alternative x5 as good and the alternative x6 as bad with respect to that new criterion. Again this elicitation continues as long as the decision maker can identify such pairs of alternatives. To conclude the rst step of the standard process, the system searches the temporary decision matrix for alternatives which score either extremely well or extremely poorly. Suppose an alternative is rather poor. The decision maker is asked if such an alternative is really poor in so many criteria, or if there exist

28

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

Fig. 5. ExcelAIMs basic display.

criteria for which the alternative performs well. Posing this question might lead to the elicitation of an additional criterion. In the example in Table 7 the rst alternative presented to the decision maker by the system would be x3 . The decision maker would then be asked if there exists any criteria by which x3 would be judged as good. In the second step of the standard process the denition of the criteria is put into more concrete terms (see Fig. 2 in the previous section). For each criterion the decision maker selects a type (i.e., maximizing, target, or minimizing) and provides the parameters (i.e., range of feasibility, and indifference thresholds). Next, the outcomes are recorded and ExcelAIM computes the feasible and the nondominated alternatives. Based on this analysis, the decision maker can rene the set of criteria and their parameters. This analysis can also be a reason to add a new criterion if an obviously good alternative is found to be dominated. If the nal decision is the selection of exactly one alternative, then the decision maker has the option of instructing the system to exclude all dominated alternatives from further consideration. Finally, the decision maker uses the basic display of ExcelAIM to adjust his levels of aspiration (see Fig. 5). The basic display shows the current levels of aspiration and provides a button for each criterion to raise or lower that criterion to the next better or worse level, respectively. The basic display also provides a ranking of alternatives based on the Tchebycheff distance measure constructed from the given levels of aspiration. The algorithm used assures that only a nondominated

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31 Table 8 Evaluation of ExcelAIMs standard process Question 1 2 3 4 5 6 7 8 9 10 11 12 Time needed Overall Minimum 3 4 4 4 3 3 2 4 3 4 4 2 4 2 Mean (Std. Dev.) 4.5 (0.84) 4.7 (0.52) 4.2 (0.41) 4.5 (0.55) 4.3 (0.82) 4.0 (0.89) 4.0 (1.10) 4.5 (0.55) 4.3 (0.82) 4.5 (0.55) 4.7 (0.52) 4.0 (1.10) 4.5 (0.55) 4.3 (0.25) Change +0.3 +0.4 +0.2 +0.5 +0.9 +0.1 0.3 +0.1 +0.5 +0.5 +0.4 +0.2 n.a. +0.3

29

Maximum 5 5 5 5 5 5 5 5 5 5 5 5 5 5

alternative can get the best rank. Let aj denote the level of aspiration currently set for criterion cj . Let the function vj map an outcome onto the interval [0,1] according to the function type of criterion cj (see Fig. 2). Then the distance of alternative x to the current levels of aspiration is dened by d(x) := maxj |vj (cj (x)) vj (aj )|. Based on this distance measure, the following algorithm computes the ranking of the alternatives. (1) (2) (3) (4) (5) (6) (7) Empty the ranking. Let X X be the set of all feasible alternatives of X. Let X X be the set of all nondominated alternatives of X . Let x := arg minx X (d(x)). Put the alternative x on the best rank which is still free. Let X := X \{x }. If X = go to step 3.

ExcelAIM does not prescribe the presented sequence of steps for decision making (see Fig. 4). Instead, the decision maker can jump back and forth adding, modifying, or deleting alternatives and criteria. The subjects in the study consistently used this feature of ExcelAIM often. Our interpretation is that the subjects used feedback provided by ExcelAIM to iteratively capture the important aspects of their problem. The intention of the nal phase of the eld study was to compare ExcelAIMs standard process to its straight process. Therefore the case study and the questions which were already presented in the previous section were used again. Additionally, the subjects were asked if a decision could be made in a reasonable amount of time with the system (question time needed). Table 8 summarizes the results of the subjects assessments and relates them to the data presented in Table 5.

30

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

In eleven out of twelve issues, ExcelAIMs standard process was rated better than its straight process. The only issue in which it was rated worse is the condence in the process (see question 7). In fact, in the standard process ExcelAIM actively queries the decision maker according to the repertory grid execution schema; i.e., primarily the system has control over the process. In contrast, in the straight process the decision maker controls the system by setting up the decision matrix. Letting the decision maker do the setup manually probably leads to a higher degree of condence in the process. The greatest improvement in the rating concerned the insight in the problem gathered when using the system (see question 5). It seems that the subjects regarded the repertory grid technique as a very useful instrument in this respect. 5. Conclusion In this article a eld study about structuring multiple criteria problems on an operational level is presented. Such decision problems are characterized by the fact that few resources in terms of time and money are available. The study has led to the implementation of the decision support system ExcelAIM. ExcelAIM combines the repertory grid technique with the aspiration-level interactive method (AIM). The underlying mathematics are easy to understand even for non-expertsa feature that is crucial for the acceptance of a decision support system in a business environment. References
[1] Keen PGW, Morton MSS. Decision support systemsan organizational perspective. Reading, MA: Addison-Wesley; 1978. [2] VanGundy AB. Techniques of structured problem solving. New York: Van Nostrand Reinhold; 1988. [3] Habenicht W, Scheubrein B, Scheubrein R. Multiple criteria decision making. In: Derigs U, editor. Theme 6.5 Optimization and Operations Research of the Encyclopedia of Life Support Systems (EOLSS). Developed under the auspices of the UNESCO. Oxford, UK: Eolss Publishers; 2002. http://www.eolss.net. [4] Korhonen P, Moskowitz H, Wallenius J. Multiple criteria decision supporta review. European Journal of Operational Research 1992;63:36175. [5] Hwang C-L, Masud ASM. Multiple objective decision makingmethods and applications. Berlin: Springer; 1979. [6] Hwang C-L, Yoon KP. Multiple attribute decision makingmethods and applications. Berlin: Springer; 1981. [7] Belton V, Stewart TJ. Multiple criteria decision analysisan integrated approach. Boston: Kluwer; 2002. [8] Keeney RL. Value-focused thinkinga path to creative decision making. Cambridge, MA: Harvard University Press; 1992. [9] Keeney RL. Value-focused thinkingidentifying decision opportunities and creating alternatives. European Journal of Operational Research 1996;92(3):53749. [10] Lot V, Stewart TJ, Zionts S. An aspiration-level interactive model for multiple criteria decision making. Computers & Operations Research 1992;19(7):67181. [11] Kepner CH, Tregoe BB. The rational manager. Princeton, NJ: Princeton Research Press; 1976. [12] Kelly GA. The psychology of personal constructs. New York: Norton; 1955. [13] Jankowicz A. Applications of personal construct theory in business practice. In: Neimeyer G, Neimeyer R., editors. Advances in personal construct psychology, vol. 1. New York: JAI Press; 1990. pp. 25788. [14] Stewart V, Stewart A. Business application of repertory grid. London: McGraw-Hill; 1981 http://www.enquirewithin.co.nz/ BUSAPP/business.htm. [15] Rogers M, Bruen M. A new system for weighting environmental criteria for use within ELECTRE III. European Journal of Operational Research 1998;107:55263. [16] Gammack JG, Stephens RA. Repertory grid technique in constructive interaction. In: Cassell CM, Symon G., editors. Qualitative methods in organizational research: a practical guide. London: Sage Publications; 1994. pp. 7290.

R. Scheubrein, S. Zionts / Computers & Operations Research 33 (2006) 18 31

31

[17] Easterby-Smith M, Thorpe R, Holman D. Using repertory grids in management. Journal of European Industrial Training 1996;20(3):330. [18] Olson DL. Decision aids for selection problems. New York: Springer; 1996. [19] Scheubrein R, Bossert B. An internet system to apply the balanced scorecard concept to supply chain management. In: Kksalan M, Zionts S., editors. Multiple criteria decision making in the new millenium. Berlin: Springer; 2001. pp. 34857. [20] Angur M, Lot V. A comparison of aspiration level interactive method (AIM) and conjoint analysis in multiple criteria decision making. In: Karwan MH, Spronk J, Wallenius J., editors. Essays in decision makinga volume in honour of Stanley Zionts. Berlin: Springer; 1997. pp. 6073. [21] Leon L, Przasnyski Z, Seal KC. Spreadsheets and OR/MS models: an end-user perspective. Interfaces 1996;26(2):92104. [22] Albright SC. VBA for modelersdeveloping decision support systems with microsoft excel. Pacic Grove, CA: Duxbury; 2001. [23] Zapatero EG, Smith CH, Weistroffer HR. Evaluating multiple-attribute decision support systems. Journal of MultiCriteria Decision Analysis 1997;6:20114.