STRATEGIC EVALUATION

A DECISION MODEL
FOR STRATEGIC
EVALUATION OF
ENTERPRISE
INFORMATION
TECHNOLOGIES
Joseph Sarkis and R. P. Sundarraj
Enterprise information technologies (EITs), which are strategic systems seeking to
integrate the processes and databases of the entire organization and beyond, require
a significant investment of money and human resources in return for the promise of a
global business model and its associated far-reaching benefits. Their evaluation/justi-
fication must be completed with organizational goals and requirements included in
the decision, or the organization could lose financially and competitively. Besides tra-
JOSEPH SARKIS is an
associate professor in ditional financial models, e.g., ROI (return on investment), that are primarily meant for
the Graduate School of short-term financial justification purposes, there is a paucity of methods for the eval-
Management at Clark uation of the strategic and intangible costs and benefits that EITs afford organiza-
University, Worcester, tions as a whole. This article introduces the use of a robust quantitative technique
Massachusetts. He is called the analytical hierarchy process (AHP) that can integrate a diverse range of
also a certified produc- factors (strategic and operational, and tangible and intangible) into one model. The
tion and inventory approach can be easily understood by managers and analysts and has a history of
manager (CPIM). application to other types of strategic justification decisions.
R. P. SUNDARRAJ is
an associate professor of
Information Systems at
Clark University in
NTERPRISE INFORMATION TECHNO- gaining long-term advantages. Even with this
Worcester, Massachusetts.
He has also consulted
with Fortune 100 com-
E logies (EITs)1 are software systems that
change the architecture of information
restricted application, the size of the EIT mar-
ket, which can be somewhat judged by the
panies regarding the technology (IT) and change and inte- revenues of EIT companies themselves, has
development of deci- grate the processes of the entire organization been massive. For example, the top EIT ven-
sion support and other
software systems for and beyond.2 The size, scope, and cost of an dors current revenue is $3.3 billion.3 In addi-
materials and market- EIT system are organizationally significant. As tion, EIT vendors are seeking to expand their
ing management. a result, the implementation of EIT has pri- market base to medium-sized corporations by
marily been for large corporations that can releasing targeted versions that are cheaper.
afford this strategic step, with the hope of These indicators have led the market research

62 I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
 STRATEGIC EVALUATION
firm AMR to project the EIT markets revenue
to attain $50 billion by the year 2002.4
EITs are strategic technologies whose ben-
efits are distributed throughout the organiza-
tion and cannot be felt immediately after their
implementation. Examples exist where an
A successful
EITs short-term costs and the lack of fit with
respect to the current systems of an organiza-
implementa- tion have led to the bankruptcy of the organi-
tion requires zation.5 Even in cases when one can see a net
improvement attributable to the implementa-
a clear tion of EIT, such improvements are often
business intangible. In fact, according to a recent sur-
vision with vey,6 while an EITs long-term annual return
can be nearly two-and-one-half times its oper-
quantifiable
ating and maintenance costs, it may be the
objectives case that the short-term return on an EIT
that can be investment is actually negative:
achieved and Average time to implement: 23 months
Average implementation
delivered. cost: $10.3 million
Net present value of
implementation: -$1.5 million
Hence, according to Barry Wilderman,
Vice President of the Meta Group, Traditional
measurement methods focused on cost reduc-
tion [alone] do not measure the true value of
[these] solutions. That is, one also needs to
employ techniques for the integrated evalua-
tion of both tangible and intangible factors.7
In a recent article, Holland and Light8 make
it clear that a successful implementation
requires a clear business vision with quantifi-
able objectives that can be achieved and deliv-
ered. Given quantified importance levels of
the various factors, this article describes a
robust quantitative technique to evaluate EITs
on both strategic and operational levels. The
importance levels of the factors are obtained
through managerial inputs. This process corre-
sponds well with management hierarchy,
allowing upper-level managements involve-
ment in the evaluation of strategic factors and
line managers evaluating the operational
ones. The method can be implemented on a
spreadsheet (see Exhibit 1) and is therefore
accessible to a wide-ranging audience. For
completeness, a framework to manage EITs is
presented next.
A FRAMEWORK FOR MANAGING EITS
Since EITs are strategic systems with long-term
and broad organizational implications, they
need to be evaluated and justified within the
context of organizational goals, strategies, and
policies. To provide a foundation for the deci-
I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
sion evaluation method proposed in the next
section, a framework for formulating goals,
strategies, and policies is outlined (see Exhibit
2), defining how the EIT-justification step is
incorporated within this framework. This for-
mulation process is arduous, but is the neces-
sary effort that must be anticipated for the
successful integration of complex and strategic
systems into an organization.
Strategy Formulation and Integration
In any strategic justification and development
framework, the decision for the development
and implementation of strategic technologies
and programs begins at the upper levels of
management. The technology selected should
fit well within the vision, goals, and strategic
objectives of the organization. To define its
goals and strategies, an organization usually
undergoes the SWOT-MOSP process, in which
Strengths and Weaknesses are assessed in the
light of environmental Opportunities and
Threats and then Missions, Objectives,
Strategies, and Policies are developed.9 One of
the results of this step in the process is deter-
mination of an organizations core competen-
cies.10 It is these competencies that need to be
supported by the specific technology that is
chosen.
Process Planning and Engineering
At the next level is the initiation of process
plans that support the organizational compe-
tencies identified earlier and that in turn get
supported by the chosen system (EIT or oth-
erwise). Also known as the reengineering
phase,11 three studies are usually undertaken
at this stage: (1) AS-IS, (2) SHOULD-BE, and
(3) TO-BE. The AS-IS study provides baseline
measures for later justification purposes and
provides measures for post-implementation
auditing. The SHOULD-BE study exhibits how
the current system should function after non-
automation or non-hard technology improve-
ments (simplification and rationalization
approaches, such as total quality management,
group technology, etc.) are applied to the sys-
tem. The basic caveat is that a currently disor-
dered system will lead to a disordered EIT
system as well. The TO-BE study is used to
define the system necessary to meet the objec-
tives set forth by the strategic units.
System Design and Functionality
Requirements
The next step in the process is to determine
what alternate systems configurations (EIT or
63
 STRATEGIC EVALUATION

EXHIBIT 1 Spreadsheet Implementation

Ever since its introduction, the spreadsheet has become a valuable tool for a variety of managerial analyses. To aid this
process, how this technique can be executed on a spreadsheet is shown. Revert to the example in Exhibit 2, which is
composed of four strategic performance metric factors, seven EIT factors, and three EIT alternative factors, but alterations
can easily be made by changing the size of the problem and copying the key formulas, as shown below.

EXHIBIT 1a Sample AHP Calculations on Spreadsheet

Illustration of Step 2: Here the user is asked to compare the importance of one factor over another factor, with respect
to a third controlling factor. As an example, the top part of the spreadsheet gives the PWC matrix showing comparisons
among the Strategic Factors with respect to Decision. Likewise, other PWC matrices (as given in Exhibit 3) would need
to be elicited from users.

Illustration of Step 3: Step 3 then requires calculation of the relative importance weights of the factors with respect to
their controlling factors. The method to calculate the weights of the Strategic Factors on the decision are also shown at
the top of the spreadsheet. For example, the sum of the elements of the Time strategic factor (Step 3a) is 5.833 (see cell
B7 as well as its formula in the key formulas box on the bottom left of the spreadsheet). The next step is to normalize this
column by dividing each element in that column by this sum (Step 3b), as shown in cell G3, in which the original value 1
is divided by the column sum 5.833. The final step (Step 3c), which gives the relative importance weights, is to average
the rows of the normalized matrix (see cells K3 through K3 and their copies in L10 through L13). The circled 1 corresponds
to the results of the parenthetical 1 of expression A.1. The results show that Cost is the most important strategic factor at
a value of 0.610 followed by Time, Flexibility, and Quality, respectively.

Illustration of Step 4: The calculations with the other PWC matrices would follow similarly and are hence skipped. The
resulting relative importance weights of EIT factors on each of the Strategic Factors (wJ,k for all k) are shown in cells F16
to I22. From these weights, we determine the relative importance of each of the EIT factors on the final decision by deter-
mining the cross product of the weight vectors of the EIT and the Strategic factors. For example, the value in cell L16
the relative importance weight of EIT Cost on the final decision is determined by the cross product of the relative impor-
tance row-vector of EIT Cost on the Strategic Factors (cells F16 to I16) and the relative importance column-vector of
Strategic Factors on the decision (cells L10 to L13), i.e., L16 is 0.063 0.198 + 0.200 0.610 + 0.150 0.071 + 0.150
0.121 = 0.163 (see the formula for L16 in the Key Formulas box). The circled (4b) corresponds to the results of the paren-
thetical (4b) of expression (2). For this example, Adaptability seems to have the largest importance, with a value of 0.206.
The final step (Step 4c in equation 2) of the process, which is similar to the above step, is to incorporate into the decision
analysis the relative importance weights of the EIT Alternatives on each of the EIT factors (wI,j for all j ). These values are
shown in cells D25 to J27. To obtain the relative importance of an EIT Alternative on the decision, compute the cross prod-
uct of the importance vector of that alternative on the EIT Factors and the importance vector of the EIT Factors on the
decision. For EIT Alternative 1, as shown in the formula box, this translates to the cross product of D25 through J25 and
L16 through L22, i.e., L25 = 0.400 0.163 + 0.100 0.206 + + 0.430 0.101 = 0.306). Again, the circled (4c) corre-
sponds to the results of the parenthetical (4c) of Expression 2. As can be seen, the EIT Alternative 2 must be selected in
this example, since its weight of 0.379 is the highest.

64 I N F O R M A T I O N S Y S T E M S M A N A G E M E N T
S U M M E R 2 0 0 1
 STRATEGIC EVALUATION

our focus will be on the evaluation of intangi-

EXHIBIT 2 A Strategic Framework for the Management of EITs ble factors based on managerial inputs. To
accomplish this task, details on a method
known as the analytical hierarchy process
(AHP) are described below.
Uncertainty and
External Competitive
Environment System Implementation
Systems implementation involves:
Acquisition and procurement Actual pur-
chase of software, hardware, and supporting
Corporate
Strategic Planning equipment, personnel
Operational Planning The project plan
necessary to bring up the system
Functional Implementation and installation The
Strategic Planning
and Design actual implementation and start-up step
Integration Linking the systems to each
other and other organizational systems
Process and
Systems
AHP Engineering Maintenance and Auditing
model This last feedback stage, although very
used for Reconciliation
this stage. Configuration Design and Adjustment important from a continuous-improvement
and Functionality of Factors,
Measures, perspective, is one of the more neglected
Requirements
and Strategy steps. Auditing should:
Encourage realistic preparation of invest-
Systems
Evaluation and ment proposals
Justification Help improve the evaluation of future pro-
jects as well as the performance of current
projects that are not proceeding as planned
Systems
Implementation Call attention to projects that should be dis-
continued

Post-Implementation A METHOD FOR EIT EVALUATION AND

Audit
otherwise) are required. The technical devel-
opment of various information technology
functionalities will need to be addressed at
this stage of the process development. The
configuration and data relating to alternative
systems should finally be integrated at the jus-
tification phase of the development cycle.
System Evaluation and Justification
It is this stage that primarily deals with the
analysis to determine the economic, technical,
and organizational (operational) feasibility and
justification of the system or subsystem. The
justification step will have to consider a vari-
ety of factors: tangible, intangible, financial,
quantitative, and qualitative. Since the analysis
of the tangible factors (e.g., finance) is well-
studied using methods such as internal rate of
return (IRR) and return on investment (ROI),
I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
JUSTIFICATION
In order to make a decision regarding an EIT
implementation, organizations find it neces-
sary to conduct some sort of financial analysis
(e.g., ROI) to assess the short-term viability of
an EIT investment. However, strategic invest-
ments such as EITs are expensive and risky,
and they need to fit well within organization-
al goals, strategies, policies, and systems, as
outlined earlier. To evaluate this fit, an array of
intangible factors would need to be evaluated
and integrated with the more quantitative
ones. A well-structured analysis of the intangi-
ble factors must involve people with a wide
range of experiences and responsibilities (i.e.,
job functions and management levels). This
section provides a decision methodology
(AHP) for such an evaluation and integration.
AHP, which was developed by Saaty,12 has
been applied in the past to justify the acquisi-
tion of strategic technologies and evaluation of
strategic projects.13,14 Yet its application to
information technology evaluation is lacking.
65
 STRATEGIC EVALUATION
EXHIBIT 3 A Detailed Hierarchical Decision Model
Incorporating Strategic and Operational Factors
Decision
Objective
Strategic
Performance Time
Metric Factors
EIT
Operational Costs Adaptability Openness
Factors
EIT
ALT1
Alternatives
STEPS OF AHP
There are four major steps in applying the
AHP technique:
1. Develop a hierarchy of factors impacting
the final decision. This is known as the
AHP decision model.
2. Elicit pairwise comparisons between the
factors using inputs from users/managers.
3. Evaluate relative importance weights at
each level of the hierarchy.
4. Combine relative importance weights to
obtain an overall ranking of the candidate
EIT alternatives.
Step 1: Factors to evaluate EIT An AHP
Model
The AHP model presented here consists of a
hierarchy of four levels, illustrated in Exhibit 3.
At the top level is the decision or objective
itself, and the bottom level consists of the EIT
alternatives under evaluation. The factors,
which comprise the middle levels, consist of
strategic-type factors, which are related to the
concerns of upper-level management, as well
as EIT-type factors, which are more related to
the people who would be managing/using the
system.
Strategic Factors for EIT Evaluation.
Strategic systems need to be evaluated on
strategic performance metrics that are linked
to organizational strategy. While financial per-
66 I N F O R M A T I O N S Y S T E M S
S U M M E R
Select Enterprisewide
Information Technology
Cost Quality Flexibility
Ease
Security Scope Support
of Use
ALT2 ALT3
formance metrics such as stock price and
earnings per share can be used to evaluate an
organization from a shareholders perspective,
operational strategic-performance metrics
should be used to evaluate how an organiza-
tion is doing with respect to the customer.15 It
is these strategic performance metrics that are
most important from the perspective of strate-
gic project management. These measures
include cost, quality, flexibility, and time-
based metrics. Varying names have been pro-
vided for each of these metrics, but they cover
the gamut of organizational and operational
performance measures. The relative impor-
tance of the performance metrics should be
determined relative to overall organizational
strategy. Skinners16 focused factory para-
digm states that certain business characteristics
may require a company to put its effort on a
subset of the above metrics. For example, a
customized medical equipment manufacturer
might place a high value on Quality, while in
mature industries, Cost may be the most
important factor.
EIT Operational Factors. This set of factors
relates to the performance of the EIT alterna-
tives at the operational level. The seven factors
considered here are as follows.17
Operational costs. Operational costs are
expenses that are directly associated with
the EIT system. Examples of these direct
2 0 0 1
M A N A G E M E N T
 STRATEGIC EVALUATION
costs include the actual cost of the system
software and hardware, training costs, and
maintenance costs. Operational costs differ
from the strategic costs described earlier,
where focus is on the cost to an entire orga-
nization.
Adaptability. This factor considers issues
relating to the fit of the EIT alternative with
respect to the organizational and general
business environment. These issues include
the difficulty of coding the underlying busi-
ness rules into the EIT alternative; the fit of
the EIT alternative with respect to the cur-
rent hardware, software, and the sophistica-
tion levels of would-be users; and the fit of
the EIT alternative with the external busi-
ness environment (an example of the last
issue could be E-commerce, which is
becoming prevalent).18
Openness. The architecture of EIT systems
must be platform independent. All operating
systems, communication, and database
servers should be implemented as abstrac-
tions. This allows for the creation of one set
of code that can be applied across a hetero-
geneous network without requiring users to
have knowledge of where the components
are physically resident.
Security. Since information is an asset of an
organization, the security of the databases
and of the EIT processes must be invio-
lable.19
Scope of EIT modules. EIT systems may be
composed of a number of different mod-
ules. Different vendors may offer differing
modules with differing capabilities/scope.
Some may represent extensions of tradition-
al MRP II systems, while others may offer
scope for conducting advanced planning,
decision support, and performance manage-
ment. A large company with many functions
may require better scope, whereas a smaller
one may not.
Ease of use. Employees need to learn the
system with the minimal possible training,
and materials for such training must be
available.
Customer support. The complexities of EIT
necessitate customer support to ensure that
employees use the system and also perceive
it to be of value and that future require-
ments will be met.
The relationship in the hierarchy (represented
by the connecting lines in Exhibit 3) between
the strategic and operational factors implies
that each of the operational factors will have
I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
some relative impact on the strategic perfor-
mance factors. The alternatives of Exhibit 3
and their relationships to the operational fac-
tors mean that each of the alternatives will
have some level of operational characteristic.
These relationships will now need to be eval-
uated using the next three steps.
Step 2: Elicit Managerial Comparisons
The AHP model is usually finalized after a
considerable amount of discussion at all levels
of management and operations. This discus-
sion may be even more valuable than the
actual values assigned because structuring the
decision problem helps managers see a logical
whole picture and the issues involved in
such a complex decision. As alluded to earli-
er, it is possible for two organizations to have
a different set of factors for EIT evaluation,
and, even if the factors are similar, their impor-
tance levels can differ significantly from one
organization to another. Given a set of factors
impacting the final decision, the purpose of
this step is to develop a meaningful manner of
extracting user input regarding the decision
alternatives. One way to obtain such input is
to ask managers to first assign weights to each
of the factors and then score the EIT alterna-
tives on these factors. The overall ranking of
an EIT alternative i can then be given as fol-
lows, where Scoreij is the score of alternative i
for factor j and Weightj is the importance of
factor j with respect to the decision.
Rank of Alternativei =
(1)
Scoreij *Weightj for all i
factors j
However, according to cognitive psychologi-
cal literature20, the problem with this approach
lies in the difficulty of assigning precise weights
on each of the factors, especially when the num-
ber of factors or alternatives are in the range of
seven or more. Moreover, such a method does
not facilitate the structured thinking process nec-
essary for making complex decisions. A more
practicable way (which is the AHP-based
approach) is to systematically elicit inputs by ask-
ing users to evaluate the relative importance of
one factor when compared to another factor
pairwise comparisons21 with respect to a third
controlling factor. The controlling factor is one
level up in the AHP model. For each controlling
factor, the corresponding set of pairwise compar-
isons (PWCs) can be represented in matrix form.
That is, for the relative importance of one strate-
67
 STRATEGIC EVALUATION
EXHIBIT 4 Pairwise Comparisons Characteristics
PWCs Example PWC questions
Comparisons among strate- With respect to the deci-
gic performance metric sion, how much more
factors with respect to the important is cost as com-
decision controlling factor. pared to quality?
With respect to the deci-
sion, how much more
important is quality as
compared to flexibility?
Comparisons among EIT With respect to overall qual-
factors with respect to ity how much more impor-
each strategic controlling tant is adaptability than
factor ease of use?
Comparisons among EIT With respect adaptability,
alternatives with respect to how much better is
each EIT controlling factor. Alternative 1 as compared
to Alternative 2?
With respect to EIT scope,
how much better is
Alternative 2 as compared
to Alternative 3?
gic factor with respect to other strategic factors,
the controlling factor would be the decision
itself, and there will be one PWC matrix. For the
relative comparisons among EIT Factors, there
will be four controlling factors corresponding to
the four strategic factors (four PWC matrices),
and for the relative comparisons of the EIT
Alternatives, there will be seven controlling fac-
tors corresponding to the seven EIT Factors
(seven PWC matrices). Specifically, for the set of
factors involved in Exhibit 3, the characteristics
of the PWCs to be made are given in Exhibit 4.
Saaty12 suggests that the values assigned to
the comparisons of the factors be made in the
range 1/9 to 9. A 9 indicates that one factor is
extremely more important than the other, a 1/9
indicates that one factor is extremely less impor-
tant than the other, and a 1 indicates equal
importance. Also, if the importance of one fac-
tor with respect to a second is given, then the
importance of the second factor with respect to
the first is the reciprocal; that is, if cost is twice
as important as flexibility, then flexibility is half
as important as cost. Hence, in column three of
Exhibit 4, there is a division by two in the
expressions for the number of PWC questions.
Step 3: Calculate Relative Importance
Weights
From each PWC matrix given in Exhibit 4, we
calculate the relative ranking of factors with
68 I N F O R M A T I O N S Y S T E M S
S U M M E R
Comments Number of PWCs
# of PWC matrices = 1 1 6 = 6 questions
Comparisons per matrix = 4
3/2 = 6
# of PWC matrices = 4 4 21 = 84
Comparisons per matrix = 7
6/2 = 21
# of PWC matrices = 7 7 3 = 21
Comparisons per matrix = 3
2/2 = 3
respect to the corresponding controlling factor.
Specifically, for our AHP model, we obtain:
One set of relative importance of the strate-
gic factors with respect to the decision
Four sets of relative importance weights of
the EIT factors (one set with respect to each
of the four Strategic controlling Factors)
Seven sets of relative importance weights of
EIT alternative (one set with respect to each
of the seven EIT controlling factors)
The procedure to obtain the relative impor-
tance weights for each PWC matrix is as fol-
lows (Saaty12 gives more accurate but complex
methods):
Step 3a: Obtain the column sum.
Step 3b: Divide the elements of the PWC
matrix by the column sum.
Step 3c: Finally, to obtain the relative weight
of each factor with respect to its controlling
factor, average the elements of the corre-
sponding row.
Step 4: Calculate the Overall Weight of
Alternatives With Respect to Decision
The overall importance weight of the EIT
alternatives (with respect to the decision) is a
weighted average of the weights computed in
Step 3. That is,
M A N A G E M E N T
2 0 0 1
 STRATEGIC EVALUATION

EXHIBIT 5 A Simplified Hierarchical Decision Model for the Illustrative Case Study

Decision Select Enterprisewide

Objective Information Technology

Strategic
Performance Quality Cost
Metric Factors

EIT
Ease
Operational Openness
of Use
Factors

EIT
ALT1 ALT2 ALT3
Alternatives

4c agers believe the organizational drives and

4b
4a
(2)
Weight for EIT Alternative i=
weighti,j weightj,k weightk,dec for all i
i kK
where weightk,dec is the relative weight of
strategic factor k with respect to the decision,
weightj,k is the relative weight of EIT Factor j
with respect to strategic factor k, and weighti,j
is the relative weight of EIT alternative i with
respect to EIT factor j.
AN ILLUSTRATIVE CASE STUDY
In order to understand how the calculations
work, consider an organization that is evaluat-
ing two EIT alternatives. Senior management
has decided that cost and quality are the two
strategic factors that need to be considered,
and middle managers have included the EIT
factors: openness and ease of use.22 The steps
given below correspond to those in the last
section. A spreadsheet implementation of the
method is given in Exhibit 5.
Step 1. The AHP decision hierarchy model
for these incorporating the factors of the illus-
trative case study is shown in Exhibit 1.
Step 2. The PWCs needed for this example
are shown in Exhibit 6. We can see in this
example from question 1 that the upper man-
I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
goals are more quality focused than cost
focused. For the question of how much more
important is cost than quality to our strategic
decision, we find that cost is much less impor-
tant at 1/4 or a 0.25 relative weighting. For the
reciprocal question of how much more impor-
tant is quality than cost, a 4 is assigned in the
matrix. These values appear in the second col-
umn of Exhibit 6. A similar set of PWC matri-
ces is formed for each level of relationships.
Step 3. The calculations for the illustrative
example are shown in Exhibit 7. For example,
for the first PWC matrix, the column sums
obtained from Step 3a are 5 and 1.25, respec-
tively (as given by the shaded elements in col-
umn 2). The PWC elements are then divided
by the respective column sums as needed by
Step 3b (this gives: 1/5 = 0.2; 4/5 = 0.8;
0.25/1.25 = 0.2, and 1/1.25 = 0.8). The relative
importance weights of each of the various fac-
tors with respect to their controlling factors
(Step 3c) are then the average of the elements
of the corresponding row, as shown in the
fourth column of Exhibit 7. Notice that quali-
tys relative importance is 0.8 and costs rela-
tive importance is 0.2. Rows 4 and 5 of Exhibit
7 show that managers believe Alternative 1 is
better than Alternative 2 in terms of openness
(0.8 to 0.2 relative weights, respectively) and
that Alternative 1 is worse than Alternative 2 in
terms of ease of use (0.17 to 0.83).
Step 4. To determine the aggregated values
69
 STRATEGIC EVALUATION

EXHIBIT 6 Step 2 of Illustrative Case Study

Number Matrix

1. Comparisons among strategic factors with Cost Quality

respect to decision Cost 1.00 0.25
Quality 4.00 1.00

2. Comparisons among EIT Factors with respect Open Ease

to the cost strategic factor Open 1.00 2.00
Ease 0.50 1.00

3. Comparisons among EIT factors with respect Cost Quality

to the quality strategic factor Cost 1.00 0.33
Quality 3.00 1.00

4. Comparisons among EIT alternatives with Alt 1 Alt 2

respect to the Openness EIT factor Alt 1 1.00 4.00
Alt 2 0.25 1.00

5. Comparisons among EIT alternatives with Alt 1 Alt 2

respect to the ease EIT factor Alt 1 1.00 0.20
Alt 2 5.00 1.00

EXHIBIT 7 . Step 3 of Illustrative Case Study

PWC Matrix Step 3a: Step 3b: Element/ Step 3c: Average
Column Sum Column Sum Rows Elements

1. wrt Decision
Cost Quality 5.00 1.25
Cost 1.00 0.25 1.00 0.25 0.20 0.20 0.20
Quality 4.00 1.00 4.00 1.00 0.80 0.80 0.80

2. wrt Cost
Open Ease 1.50 3.00
Open 1.00 2.00 1.00 2.00 0.67 0.67 0.67
Ease 0.50 1.00 0.50 1.00 0.33 0.33 0.33

3. wrt Quality
Cost Quality 4.00 3.00
Cost 1.00 0.33 1.00 0.33 0.25 0.25 0.25
Quality 3.00 1.00 3.00 1.00 0.75 0.75 0.75

4. wrt Open
Alt 1 Alt 2 1.25 5.00
Alt 1 1.00 4.00 1.00 4.00 0.80 0.80 0.80
Alt 2 0.25 1.00 0.25 1.00 0.20 0.20 0.20

5. wrt Ease
Alt 1 Alt 2 6.00 1.20
Alt 1 1.00 0.20 1.00 0.20 0.17 0.17 0.17
Alt 2 5.00 1.00 5.00 1.00 0.83 0.83 0.83

70 I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
 STRATEGIC EVALUATION
of the operational factors on the decisions
(rolling up the weights), first observe from
Exhibit 7 that the importance of cost and qual-
ity on the decision (Step 4a of Expression 2)
is, respectively:
T op
Weight
Weight
cost, decision
quality, decision
= 0.2
= 0.8
management
can look at Using this, the weights of Openness and
Ease-of-use on the decision (Step 4b) are
the strategic given by:
factors,
whereas Weight openness, decision =
weight openness, cost weight cost, decision+
operational weight openness, quality weight quality, decision
people can
= 0.67 0.20 + 0.25 0.80 = 0.334
evaluate the
EIT Weight ease, decision =
weight ease, cost weight cost, decision+
operational weight ease, quality weight quality, decision
factors and
= 0.33 0.20 + 0.75 0.80 = 0.666
alternatives.
The final sets of weights (Step 4c) of the
alternatives with respect to the decision are
Weight Alt1, decision =
weight Alt1, openness weight openness, decision+
weight Alt1, ease weight ease, decision
= 0.8 0.334 + 0.17 0.666 = 0.38
Weight Alt2, decision =
weight Alt2, openness weight openness, decision+
weight Alt2, ease weight ease, decision
= 0.20 0.334 + 0.83 0.666 = 0.62
This implies that Alternative 2 is the pre-
ferred choice for the organization with a 0.62
relative weight.
ADVANTAGES AND DISADVANTAGES OF
THE METHOD
As with any tool, a number of advantages and
disadvantages are associated with this tech-
nique. It will benefit the manager and analyst
to be aware of some of the major issues.
Advantages are:
AHP allows for communication among vari-
ous levels, and, because of its hierarchy, it can
be made to mirror an organization, i.e., top
management can look at the strategic factors,
whereas operational people can evaluate the
EIT operational factors and EIT alternatives.
AHP provides a systematic and comprehen-
sive evaluation of the relative importance of
I N F O R M A T I O N S Y S T E M S
S U M M E R
the factors. This approach is a way of getting
around cognitive psychological problems that
arise when an individual is asked to compare
a large number of factors simultaneously.
The method allows for the tricky integration of
both intangible and strategic factors with the
(3)
more quantitative and operational measures.
The technique has been successfully applied
for the evaluation of other strategic tech-
nologies. This supports its use for strategic
information technology problems, which
have been rarely addressed.
(4) Disadvantages are
Since the technique is heavily reliant on the
use of perceptions and subjective data, an
optimal answer may not be forthcoming. It
is hence sensitive to whom the decision
makers are and their roles within the orga-
nization.
Inconsistencies may also occur in the evalu-
ation, and, although they can be reconciled,
management may have to accept some level
of inconsistencies.
Complex AHP models may require a signifi-
cant amount of management time because of
the number of pairwise comparison questions
(5) involved. Clearly it would be wise to employ
this approach only if the cost of the technol-
ogy outweighs that of the decision process
itself, as is likely to be the case with EITs.
CONCLUSIONS AND MANAGERIAL
IMPLICATIONS
The effort required to manage strategic sys-
tems such as EITs can be large. Yet, to ensure
long-term viability and appropriate fit with
the organizational directions, such a process
should be envisioned. If organizations are
looking to spend millions of dollars for the
sake of implementing EIT systems, they will
face substantial risk. To minimize this risk, a
good strategic framework (as given in this arti-
cle, for example) is necessary.
Any supporting tool to help evaluate EITs
in the context of this framework will have to
consider a range of factors: strategic, opera-
tional, tangible, and intangible. Exemplary fac-
tors and a decision model that integrates them
is presented in this article. The modeling
approach presented was the analytical hierar-
chy process (AHP) which has been applied in
a number of strategic decision environments.
Other models can and should be utilized to
help support the fuller context of a true strate-
gic decision. The model was presented in a
way that could help managers of information
M A N A G E M E N T
2 0 0 1
71
 STRATEGIC EVALUATION
technology to address the issue with relatively
little support from a decision analysis expert.
Some advantages and disadvantages of the
approach were also detailed.
The approach presented here has several
implications for management. First, while it is
T his
clear that a number of factors must be consid-
ered in a strategic decision such as this, it is
process often unclear as to what the factors must be,
what their relationships are, and who may be
allows
involved in the decision process. By providing
managers to a compendium of factors and integrating them
understand within a decision hierarchy, we have provided
the full a managerial decision-making structure and
answer. For this decision environment, strategic
picture in one factors may be evaluated by senior manage-
model, while ment, EIT factors may be evaluated by line
making sure managers or middle-level managers, and a list
of EIT alternatives may be obtained by key
that end-users.
incongruen- Second, our approach provides a struc-
cies among tured manner of evaluating factors. When a
variety of factors are included, the decision-
levels of making process could quickly get complex
management and confusing. The approach of considering
are ironed only two factors at a time (PWC) mitigates this
issue. We can also argue that the structured
out. decision environment forcing managers to go
through these steps of evaluating the factors
and their relationships could be more valuable
than the final score or ranking of the alterna-
tives. This process allows managers to under-
stand the full picture in one model, while
making sure that incongruencies among levels
of management are ironed out.
Third, as illustrated in the case study, the
factors presented herein are a guideline, and it
is not necessary for an organization to include
all them. Keeping in mind the trade-off
between a models complexity and its useful-
ness, managers must prune the factors that
may not be relevant to their particular organi-
zational context, so that any discussion and
evaluation of the factors would be fruitful
from the decision-making perspective.
Notes
1. We use the term EIT instead of the more pop-
ular enterprise resource planning systems
(ERPs). An EIT is slightly broader in scope
than an ERP and includes technologies for
advanced planning and decision support.17
Conventional ERP products do not provide a
sophisticated level of support for these
advanced functions.
2. Bingi, P., Sharma, M. K., and Godla, J.K.
(1999). Critical Issues Affecting an ERP
Implementation, Information Systems
Management, 16(3): pp. 714.
72 I N F O R M A T I O N S Y S T E M S
S U M M E R
M A N A G E M E N T
2 0 0 1
3. Laughlin, S. P. (1999). An ERP Game Plan,
Journal of Business Strategy, Jan.-Feb., p. 32.
4. Trunick, P. A., (1999). ERP: Promises or Pipe
Dreams, Transportation and Distribution,
40(1): p. 23.
5. Bartholomew, D., Jesitus, J., and Kreitzberg, C. B.
(1997). Managing the Internetworked
Corporation: Promises Versus Reality,
Industry Week, 246(20): pp. 2636.
6. Anonymous (1999). ERP Averages,
Computerworld, April 5, p. 6.
7. IT Pro, (1999). Return on ERP Investment
Takes Time, Cyber IT Professional, IEEE
Computer Society. Available:
http://computer.org/itpro/it1999/html/f3007.htm.
8. Holland, C. and Light, B. (1999). A Critical
Success Factors Model for ERP Implementation,
IEEE Software, 16(3): pp. 3036.
9. Adler, P. S. (1989). Technology Strategy: a
Guide to the Literature, Research in
Technological Innovation, Management and
Policy, JAI Press, Greenwich, CT, pp. 25151.
10. Prahalad, C. K. and Hamel, G. (1990). The
core competence of the corporation, Harvard
Business Review, 90(3): pp. 7991 (May-June).
11. Hammer, M. and Champy, J. A. (1993).
Reengineering the Corporation: A Manifesto for
Business Revolution, New York: Harper
Business Press.
12. Saaty, T. L. (1980). The Analytic Hierarchy
Process: Planning, Priority Setting, Resource
Allocation, New York: McGraw Hill.
13. Albayrakoglu, M. (1996). Justification of New
Manufacturing Technology: A Strategic
Approach Using the Analytical Hierarchy
Process, Production and Inventory
Management Journal, 37(1): pp. 7177.
14. Sarkis, J. and Sundarraj, R.P. (1999). Hub
Location at Digital Equipment Corporation: A
Comprehensive Analysis of Qualitative and
Quantitative Factors, Working Paper, Clark
University, Worcester, MA.
15. Hayes, R. H., Wheelwright, S. C., and Clark, K. B.
(1988). Dynamic Manufacturing, New York:
The Free Press.
16. Skinner, W. (1974). The Focused Factory,
Harvard Business Review, 52(3): pp. 113122.
17. Sarkis, J. and Sundarraj, R.P. (2000). Factors
for Strategic Evaluation of Enterprise
Information Technologies, International
Journal of Purchasing, Distribution and
Logistics Management, 30(3/4): pp. 196220.
18. Bakos, Y. (1998). The Emerging Role of
Electronic Marketplaces on the Internet,
Communications of the ACM, 41(8): pp. 3542.
19. Ghosh, A. K. (1998). E-commerce Security,
New York: John Wiley & Sons.
20. Miller, G. A. (1956). The Magic Number Seven
Plus or Minus Two: Some Limits on Our
Capacity for Processing Information,
Psychology Review, 63(3): pp. 8197.
21. Our systematic, objective, yet practicable
approach is important because the simple
scoring technique given by (1) is in use in
several Fortune 100 companies to whom we
have talked.
22. Because the AHP technique entails a number
of PWCs (see next section), it is better to (up
front) prune factors which are considered to be
unimportant by the organizations managers.