Information & Management 36 (1999) 63±75

Research
Project selection criteria by strategic orientation
James J. Jiang1,a, Gary Kleinb,*
a
Department of Computer Information Systems, School of Administration and Business, Louisiana Tech University, Ruston, LA 71272, USA
b
College of Business and Administration, University of Colorado at Colorado Springs, 1420 Austin Bluffs Parkway,
Colorado Springs, CO 80933-7150, USA

Received 29 July 1997; revised 20 June 1998; accepted 21 January 1999

Abstract

One information planning decision involves project selection from among a portfolio of options. This involves multiple steps,
including selection and weighting of alternatives. Choice and weighting on criteria become crucial in the selection of the
projects to pursue. A survey conducted by the authors found that organizations with an expectation of future IS importance
rely heavily on organizational goals, management support and environmental factors. Organizations with low strategic
expectations of IS rely more heavily on management support, political considerations, and risk. The results allow managers to
position selection criteria according to their strategic use of information technology. # 1999 Elsevier Science B.V. All rights
reserved.

Keywords: Project management; Strategic grid; Information system selection

1. Introduction will assist an organization in executing its business

For years, the planning of improved information
systems (IS) has been one of the challenges facing IS
professionals [23]. Effective IS planning can help
organizations understand the impact of IS on the
overall organizational strategy, and use them to reach
business goals, before deciding to develop new IS. IS
planning puts a ®rm in a better position to evaluate
potential IS projects and facilitates effective resource
allocation.
By de®nition, IS planning is ``the process of iden-
tifying a portfolio of computer-based applications that
*Corresponding author. Tel.: +1-719-262-3157; fax: +1-719-
262-3494; e-mail: gklein@mail.uccs.edu
1
Tel.: +1-318-257-3445; fax: (+1-318-257-4253; e-mail:
jiang@cab.latech.edu
0378-7206/99/$ ± see front matter # 1999 Elsevier Science B.V. All rights reserved.
PII: S - 0 3 7 8 - 7 2 0 6 ( 9 9 ) 0 0 0 0 9 - 9
plans and realizing its business goals'' [26]. This
portfolio might include a wide variety of applications,
including accounting, human resources, marketing,
manufacturing, research and development systems,
electronic mail systems, expert systems, decision sup-
port systems, CAD/CAM, and so forth. To allocate
scarce resources among diverse IS projects, IS profes-
sionals often need to conduct a multiple-round, in-
depth evaluation of potential IS projects. The evalua-
tion criteria routinely include technological, eco-
nomic, and behavioral aspects [22].
One group of project selection literature describes
various methods. A comprehensive taxonomy of these
is provided in Ref. [14]. In general, they are divided
into four subcategories; comparative approaches [33],
scoring models [21], bene®t contribution or economic
models [9], and optimization methods [6]. However,
64 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75
none of these has achieved even limited use in industry
[4]. Even though complex computer-based models
were shown to have more of the desired features, their
use is also not well accepted [17].
Use of a practical IS project selection approach is
said to be essential for an effective IS planning [30]. As
a result, IS professionals often adopt the analytic hier-
archy process [37], which includes three major steps:
1. identify and select criteria;
2. weight the criteria and build consensus about their
relative importance; and
3. evaluate the project proposals using the weighted
criteria.
To implement such a technique requires a ®rm grasp of
the importance of each individual criterion.
Criteria can represent organizational needs that
address the impact of the IS on business objectives
[24]. Other criteria focus on project risk, including
complexity, budget, development time, and newness
of technology. The political dimension in system
prioritization is also emphasized and the importance
of considering the competitive environment is stressed
[25]. Once criteria are chosen, the decision makers
must place importance weights on each chosen ele-
ment, but little help is given in the literature. The
Fig. 1. Strategic grid (modi®ed from Ref. [31]).
weighting of criteria is a critical step, since an impro-
per selection of weights can radically alter the selected
portfolio [20].
We also suspect that the IS professionals' perceived
importance of evaluation criteria can be changed by
the strategic relevance of IS to an organization (e.g.
Refs. [28, 34]). One of the most useful IS architecture
planning methodologies is the IS strategic grid [29].
This is a contingency approach that can be used to
determine the strategic relevance of IS to the organi-
zation. The strategic grid is a useful structure for
explaining differences in IS postures for organizations
and provides a framework for categorizing organiza-
tions for the purpose of studying differences. The
scheme de®nes a continuum over a two-dimensional
plane, along which a ®rm moves as it assimilates IS
technology, as illustrated in Fig. 1. Four major zones
have been identi®ed:
Strategic organizations, where IS activities are
critical to existing operations, and planned IS appli-
cations are critical for future success.
Turnaround organizations, which are not totally
dependent on current IS to function but new IS
methods, vital for reaching organizational objec-
tives, have been developed
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75 65

Factory organizations, which are dependent on IS Table 1

for their day-to-day operations but will not gain Project evaluation criteria
signi®cant advantage from further development. 1. Financial related criteria:
Support organizations, which are neither highly f1: benefit/cost ratio
dependent on IS nor will be improved by them in f2: rate of return
the future. f3: contribution of profitability
f4: growth rate
Each of these classes has very different needs for IS f5: payback period
and technology. IS professionals must be able to make 2. Organizational needs related criteria:
more informed decisions on the IS mission and o1: contribution to organizational goals/objectives
resource allocation for competing IS projects. o2: aid the organization in competing in the market
o3: internal political decisions
o4: importance to the organization for the future success
2. Methodology o5: importance to the functioning of the organization
o6: public relation effect
o7: importance to organization's critical success factors
This study examines how IS professionals' project
selection decisions are affected by their IS environ- 3. Competing environmental related criteria:
ment. Along with other researchers [41], we contend e1: required by regulations
e2: response to competition
that the strategic relevance of IS to the organization e3: required by customers/suppliers
affect IS professionals' perceived importance on e4: new industry standards
project selection criteria. To provide evidence to sup- e5: lawsuit requires information
port this belief, we designed an instrument that allows 4. Technical related criteria:
examination of project selection criteria relative to the t1: isolated, simple, and modular project
organizations position in the strategic grid. The survey t2: high visibility of project
was conducted using IS professionals as subjects. t3: basic subsystem to system
t4: basic module for operations
t5: availability of skilled IS personnel
2.1. Instrument t6: availability of needed technology
5. Risk related criteria:
The questionnaire (see Appendix A) consisted of
r1: technical risk
multiple parts: two assessing the strategic relevance of r2: structure risk
IS in the organization; another measuring the impor- r3: risk of cost overruns
tance of an IS project selection criteria. A ®nal part r4: size risk
collected demographic data. The scales used to deter- 6. Management support related criteria:
mine the strategic relevance of IS to the organization m1: political acceptance
was adopted from [31]. The questionnaire of IS project m2: end-user understanding, cooperation, and commitment to
selection criteria was designed to be consistent by project
m3: top management support
including items found in a review of the literature [11,
m4: match with users' interest/work load
13, 36, 39]. The six subcategories of IS evaluation m5: middle management support
criteria include: ®nancial, organizational, competing
environment, technical, risk, and management (see
Table 1). new project portfolio. All questions were recorded on
To ensure understandability, the researchers ®rst a ®ve-point Likert-type scale (1, not important; 2,
had two colleagues critically review the questions. slightly important; 3, somewhat important; 4, impor-
Their suggestions were incorporated into the ques- tant; and 5, very important).
tionnaire. Second, a pilot study used the questionnaire
on ®ve MBA students who were already full-time IS 2.2. Subjects
professionals. All ambiguous items were removed or
modi®ed. A ®nal of 32 items remained. Subjects were The questionnaire was then distributed to 300 mem-
asked to rate the importance of the items as if facing a bers of the Data Processing Management Association
66 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75
(DPMA) randomly selected from the membership
roles in the region that includes Michigan, Illinois,
Pennsylvania, Ohio, Indiana, and Wisconsin. The
sample was chosen because these members of DPMA
come from a wide variety of organizational settings.
Self-addressed return envelopes for each question-
naire were enclosed. Ninety-three questionnaires
(for a response rate of 30.1%) were returned to the
researchers. This rate appears consistent with other
mail surveys. Five questionnaires were eliminated due
to missing data, leaving a ®nal sample of 88 subjects.
The participants in this study held a variety of posi-
tions within the computer ®eld, re¯ecting considerable
heterogeneity with respect to IS professions. Table 2
presents a summary of their demographic character-
istics. The pro®les match those of published studies
involving DPMA members [15, 16].
Each of the demographic variables was run in a
separate MANOVA to determine any introduction of
bias by the study population. The independent vari-
ables were the mean scores of the metric categories.
Table 2
Sample population demographics
Gender
male
female
Size of organization (annual gross revenues)
<1 million
1±50 million
51±250 million
251±1 billion
1±5 billion
>5 billion
Age
<30
31±40
41±50
>50
Position
technical support staff
system analysts
project leaders
IS department/division managers
IS executive
Involvement in IS project selection
± 76
no
Each demographic variable was treated as the depen-
dent variable in separate MANOVA models. No model
found any signi®cant relation. This indicates a lack of
confounding on the part of the sample's demo-
graphics.
2.3. Metrics
Determination or veri®cation of criteria categories
is the ®rst step to permit examination of the differ-
ences by strategic posture. A principal components
analysis was conducted. Seven components had eigen-
values greater than one and were retained for further
analysis. These factors were used in a Varimax rota-
tion with loading results shown in Table 3. Item
loadings greater than 0.50 were assigned to the seven
global metrics as shown in Table 4.
Examination of the items indicates the seven global
items generally follow the a priori metrics, with one
exception: Factor #3, which is composed of internal
political decisions, regulations, and political accep-
tance. These three items are used as a seventh metric
termed `political reality'. This factor has been
proposed as a valid concern during system selection
[3, 27].
The results from the principal components analysis
64 drive the computation of seven global metrics. Each of
24 these is computed as an average of the items loaded
onto the associated factor. The Kaiser's measure of
7 sampling adequacy [19] was conducted and the results
25 (MSA ˆ 0.08027) indicated the adequacy of the sam-
21 ple size. The homogeneity of the items within each
22 factor was established further by computing their
6
internal consistency reliability coef®cient (coef®cient
7
) applying the formula recommended by [8]. The
seven metrics wound up with s ranging from 0.64 to
4
25
0.84, which suggests they are good results for social
31 science research. In order to assure the instrument is
28 consistent and agrees with other established measures
of the same attributes, we correlated the mean
11 responses of each factor to the one question directly
16 asking about the overall importance of that factor [2].
11 The results are shown in Table 5.
37
12
2.4. Hypotheses
Recent results in the literature indicate that we
12
should expect differences in the weighting of criteria
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75 67

Table 3
Factor loadings after varimax rotation

Variables Factor #

1 2 3 4 5 6 7
f1 0.23 0.03 0.00 0.75 0.07 0.10 0.15
f2 0.22 0.22 0.03 0.81 0.05 0.06 0.10
f3 0.02 0.26 0.00 0.70 0.25 0.01 0.03
f4 0.32 0.14 0.00 0.64 0.09 0.02 0.02
f5 0.18 0.05 0.12 0.65 0.10 0.20 0.15

o1 0.59 0.05 0.00 0.15 0.14 0.23 0.14

o2 0.44 0.18 0.22 0.27 0.46 0.03 0.28
o3 0.04 0.03 0.78 0.07 0.08 0.08 0.01
o4 0.60 0.01 0.24 0.08 0.23 0.14 0.19
o5 0.76 0.13 0.06 0.14 0.01 0.10 0.07
o6 0.55 0.32 0.25 0.11 0.09 0.26 0.07
o7 0.72 0.12 0.00 0.21 0.04 0.10 0.07

e1 0.00 0.04 0.60 0.15 0.30 0.13 0.11

e2 0.24 0.03 0.01 0.14 0.80 0.15 0.15
e3 0.04 0.14 0.01 0.00 0.88 0.04 0.18
e4 0.11 0.12 0.33 0.45 0.53 0.06 0.02
e5 0.11 0.00 0.47 0.38 0.44 0.24 0.04

t1 0.01 0.61 0.14 0.14 0.00 0.16 0.27

t2 0.02 0.57 0.39 0.23 0.19 0.17 0.11
t3 0.16 0.73 0.12 0.05 0.14 0.33 0.02
t4 0.14 0.71 0.08 0.1 0.01 0.20 0.15
t5 0.30 0.75 0.02 0.16 0.00 0.07 0.00
t6 0.40 0.69 0.06 0.10 0.03 0.20 0.08

r1 0.05 0.32 0.09 0.14 0.23 0.12 0.70

r2 0.03 0.03 0.09 0.15 0.08 0.09 0.80
r3 0.00 0.37 0.10 0.45 0.06 0.19 0.35
r4 0.02 0.11 0.00 0.57 0.10 0.36 0.40

m1 0.02 0.15 0.75 0.00 0.09 0.27 0.24

m2 0.33 0.17 0.18 0.17 0.24 0.33 0.41
m3 0.22 0.07 0.01 0.02 0.01 0.74 0.07
m4 0.14 0.16 0.34 0.31 0.00 0.57 0.15
m5 0.15 0.30 0.33 0.13 0.14 0.59 0.26

Table 5
Table 4 Correlation
Composition of metric scales
Financial attainment 0.77* a
Metric Items Cronbach's 
Organizational goals 0.74* a
Organizational goals O1,O4,O5,O6,O7 0.84 Technical consideration 0.81* a
Technical considerations T1,T2,T3,T4,T5,T6 0.64 Management support 0.82* a
Political reality O3,M1,E1 0.73 External environment 0.73* a
Financial attainment F1,F2,F3,F4,F5,R4 0.77 Riskiness 0.82* a
External environment E2,E3,E4 0.72 Political reality Ð
Management support M3,M4,M5 0.73 a
Riskiness R1,R2 0.69 Asterisk indicayes signi®cance at p < 0.001 level. The correlation
value of political reality is missing.
68 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75
based on strategic orientation of the ®rm [1]. These
differences could lead to a variation in the metrics
selected for project evaluation and the weights
assigned to the metrics. On this premise, we developed
the global hypothesis:
Ho: There is no difference in metric weight across
strategic grid quadrants.
Ha: There is a difference in metric weight across
strategic grid quadrants.
We examined the following alternate hypotheses:
H1: The criteria of meeting organizational needs and
objectives will be perceived more important by IS pro-
fessionals in a strategic IS ®rm than those in other ®rms.
An organization in the strategic category depends
heavily on information technology for identifying
opportunities and avoiding risks. IS professionals in
these ®rms must understand how the business strategy
in¯uences business processes and organization and
then prompt an alignment with the IS architecture and
organization [5]. We, therefore, predict that the criteria
of meeting organizational needs and objectives will be
perceived more important by IS professionals in a
strategic IS ®rm.
H2: The criteria of the external environment will be
perceived more important by IS professionals in a
strategic IS ®rm than those in other ®rms.
Firms in the strategic category are likely to be depen-
dent now, and in the future, on information technology
for survival in the marketplace. They are most likely to
rely heavily on information technology to provide a
competitive advantage. They often adopt a leading
edge strategy and have a strong commitment to
resources, technology innovation, and strong user
commitment to new technology [10].
H3: The criteria of ®nancial consideration will be
perceived more important by IS managers in a turn-
around IS ®rm than those in other ®rms.
Firms that fall into the turnaround classi®cation are
not heavily dependent on the IS function at present.
They may, however, look to new applications in the
future to bolster their competitive position. These
®rms may be unable to muster or commit the resources
necessary to pursue a leading edge approach and,
therefore, concentrate on high return-on-investment
projects [7].
H4: The criteria of management support will be
perceived more important by IS managers in a support
IS ®rm than those in other ®rms.
Management support of an IS is critical to its success.
However, ®rms that fall into this category are neither
dependent on IS, nor will new IS applications be
likely to be critical in the future. If a new technology
has not been viewed as logical and necessary, then
major resistance can be expected [38]. On the other
hand, ®rms that have a strategic emphasis are more
likely to have fostered the willingness to be creative
and reception to change. Thus, we expect that support
®rms will place more emphasis on this criteria in a
relative sense, though we expect it to be important
for all.
H5: The criteria of project risk will be perceived as
being more important by IS managers in a support ®rm
than those in other ®rms.
Unlike organizations in the remaining quarters, an
organization that falls in the support classi®cation is
less likely to have experience in developing a leading
edge technology. The application of such technology
contains risk elements shunned by organizations with
few strategic plans for IS [35]. We, therefore, argue
that IS professionals in a support organization will
consider the project risk factor more importantly than
in other ®rms.
H6: The technical criteria will be perceived more
important by IS professionals in a strategic ®rm than
those in other ®rms.
Organizations that fall into the strategic quadrant often
adopt a more aggressive approach to new technology
[12], which may demand a different set of technical
requirements, including IS personnel skills, the latest
hardware, and integration with existing systems.
Therefore, we predict that IS professionals in strategic
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75 69

organizations will place more weights on the technical Table 7

criteria for project selection. Metric means and ranks by strategic class

Support Turnaround Strategic

Sample size 18 44 26
3. Results
Organizational goals mean 3.52 3.92 a 4.25 a,b
First, apparently no respondents fall in the Factory Technical considerations mean 3.29 3.32 3.78 a,b
quadrant of the strategic grid. There may be several Political reality mean 3.61 3.47 3.55
Financial attainment mean 3.22 3.69 a 3.67 a
reasons for this. This is consistent with other studies External environment mean 3.17 3.72 a 4.04 a,b
where the factory quadrant is sparsely populated [31]. Management support mean 3.74 3.96 4.08
This is, in effect, good news in that organizations with Riskiness mean 3.58 3.41 3.46
a strategic emphasis on current systems are not con- Organizational goals rank 4 2 1
sidering lowering the importance of systems in the Technical considerations rank 5 7 4
future. The rarity of factory organizations in studies Political reality rank 2 5 6
could also be due to a professional bias on the part of Financial attainment rank 6 4 5
the respondents. IS professionals may not admit or External environment rank 7 3 3
Management support rank 1 1 2
recognize that information technology has no strate- Riskiness rank 3 6 7
gic, future impact in their organization. Or organiza-
tions with little emphasis on future IS may not have a Spearman's rank correlation
Support 1.00 0.18 ÿ0.07
suf®cient population of IS professionals to catch in a Turnaround 0.18 1.00 0.75
random sample. Strategic ÿ0.07 0.75 1.00
In order to determine whether there is a relationship a
Statistically larger than support mean using one-tailed two-
between the strategic importance of IS in an organiza- sample t-test at 0.05.
tion and project selection criteria, a MANOVA test b
Statistically larger than turnaround mean using one-tailed two-
was conducted. The dependent variables are the seven sample t-test at 0.05.
metrics and the independent variable the classi®cation
on the strategic grid. The result is shown in Table 6.
The MANOVA result (F ˆ 2.30 (Wilks'  and p- nizational goals. Thus, the general hypothesis is
value ˆ 0.001) suggests that there is a signi®cant rejected to support evidence of differences.
relationship between the strategic importance of IS Speci®c mean and rank scores for the seven criteria
in an organization and the relative importance of IS are shown in Table 7. These scores indicate that all
project selection criteria. The independent ANOVA metrics are considered to be at least somewhat impor-
results indicate that the differences exist on the tech- tant in three quadrants. The rank scores effectively
nical considerations, external environment, and orga- illustrate how the metrics vary by quadrant, even

Table 6
MANOVA of strategic grid classification and metrics

Term Test value/mean square DF1 DF2 F-Ratio Prob Decision (0.05)

Wilks'  0.69 14 158 2.30 0.01 reject

Organizational goals 2.87 2 85 8.48 0.00 reject
Technical considerations 2.01 2 85 4.12 0.02 reject
Political reality 0.13 2 85 0.15 0.86 accept
Financial attainment 1.53 2 85 2.90 0.06 accept
External environment 4.06 2 85 5.96 0.00 reject
Management support 0.61 2 85 1.05 0.35 accept
Riskiness 0.19 2 85 0.24 0.78 accept
70 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75
though the mean scores may not seem very different. It
is important to remember that formal selection tech-
niques do have sensitivity problems, especially on the
top ranked (weighted) measure [40].
The rank correlation shows how different the three
reported quadrants can become. The rank correlation
between the support quadrant and the turnaround
quadrant is only 0.18 (a 1.0 being perfect correlation).
The correlation between support and strategic quad-
rants is ÿ0.07; i.e. almost no similarity. The two
quadrants in the high strategic expectations for future
information systems is 0.75, i.e. a fairly high correla-
tion. This is an indication that differences may be a
two-dimensional issue, but there is insuf®cient data in
our sample to test this concept.
Hypothesis 1 is rejected, as the mean of the
organizational goals is signi®cantly greater for the
Strategic quadrant than either of the other ones. The
same holds true for Hypothesis 2 over external envir-
onment criteria and the technical considerations of
Hypothesis 6. In Hypothesis 3, a rejection can be
made for the Turnaround quadrant over the Support
quadrant, but not over the Strategic quadrant, again
showing less difference over the future dimension
than anticipated.
The only surprises are the failure to reject hypoth-
esis 4 and 5, indicating that there are not any major
differences across strategic postures for management
support nor with risk. Perhaps the wisdom of gathering
managerial support has taken root. This lack of a
difference is also unique, in that this dimension ranks
the highest in both, the support and turnaround quad-
rants, and it is still second in the strategic quadrant.
Such a consideration of ranks is of importance, since
many project selection techniques use dominance or
lexicographic ordering. Lack of difference in risk
concern and the consistently low importance indicate
a universal acceptance of the risk, at least in informa-
tion investments.
The political reality metric has similar values in the
mean across quadrants. However, this metric is a high
priority scale in the support quadrant, but is a much
lower priority in the remaining quadrants. On account
of the unanticipated emergence of this criteria, no
formal hypothesis has been presented, but the results
match previous literature in that political factors
have continued signi®cance in system success [18].
Further, organizations in the support grid tend to be
reactive to technology and may not have established
decision structures to deal with technology issues
[32]. Thus, systems required by regulations and
determined by internal political decision will be
emphasized, thereby pushing the rank higher than
for other criteria.
4. Discussion
A survey of IS professionals was conducted to
determine the importance of various criteria in the
selection of IS projects. The responses were categor-
ized according to a strategic grid model in order to
verify differences in criteria weighting across differ-
ences in strategic outlook. The results indicate that
selection criteria does indeed vary according to stra-
tegic posture, although no criteria were rated as unim-
portant by any class.
Organizations with a `strategic' orientation rate
organizational goals most highly. This follows with
management support, responding to the external envir-
onment, technical considerations, ®nancial attain-
ment, political reality, and risk. In general, a ®rm
well established in strategic IS use understands the
need to match the selection of IS projects carefully to
the goals of the organization.
Organizations with a `turnaround' orientation rank
criteria similar to those with a pure strategic posture.
The major difference is the drop of technical consid-
erations to the bottom of the stack. Perhaps organiza-
tions in this class are not preparing fully to grasp the
technology that must accompany strategic advances in
information technology usage.
Lastly, some organizations do not express current
use of critical systems, nor do they expect any in the
future. These `support' organizations still rank man-
agerial support highly, but follow that with political
goals and risk considerations. It is clear that these
organizations are merely reacting to technology needs
and advances.
The results show clearly that differences exist in
strategic posture of an organization. The differences
allow organizations to compare their approach to
those of other organizations in the same category
for benchmarking or to identify dif®culties in main-
taining the desired level of competitiveness. Organi-
zations wishing to switch from turnaround status
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75 71

should not continue to choose projects on political should be modi®ed to account for the missing cate-
and risk factors. gory, which has been observed in only small numbers
If one views the grid of Fig. 1 as a map for transi- in other studies. Likewise, the sample size limits
tions, the organization has a set of asserted standards. generalizability of the absolute values attained for
An organization wishing to shift from a support to a criteria importance, putting more emphasis on the
turnaround status, they should retain the importance of rankings and changes in the level of importance of
management involvement, but elevate the considera- the criteria.
tion of meeting future ®nancial projections, satisfying
organizational goals with the system, and consider Appendix A
how the new systems will enable better positioning in
the external environment. To further move toward the Information System (IS) projects selection criteria
strategic quadrant requires that an organization retain
the importance of existing measures, but also elevate A.1 Part I. Strategic importance of information
the consideration of technical advances and further systems (IS)
elevate the match of the system to the competitive
environment and organizational goals. a. To what extent Not To a
It should be noted that transitions through the are the existing at all great
quadrants does not `swap' importance of any criteria information extent
in an absolute sense. Criteria retain importance once systems important
they enter the picture and others join them in terms of to the organization
consideration. This is a testimony to the complexity 1 2 3 4 5
levels in strategic systems. In addition, where more b. To what extent Not To a
criteria must be met, more resources must likely be are the information at all great
committed. Thus, it may be dif®cult to muster systems planned for extent
resources to go straight from the support posture to the future important
the strategic posture without passing through the turn- to the organization.
around quadrant. 1 2 3 4 5
Reversal of the move to the strategic posture is also
a possibility. An organization could intentionally, or
through lack of attention to IS, adequately maintain A.2 Part II. Information systems (IS) characteristics
existing systems of strategic importance, but fail to
plan and deploy systems of future importance. This The following pages contain groups of questions
would drop the organization into the factory quadrant. relating to the status of information systems in the
The data do not exist in our study to show how the organization and their importance. Most of the ques-
criteria would be modi®ed to achieve this particular tions comprise two parts: the ®rst relates to the present
shift. information system, that is, those presently operating
The results of this study must, however, be viewed in the organization, while the second relates to the
with caution. Even though the sample represented a information systems planned for the future or cur-
good population of IS professionals, it excluded one of rently being developed. Please read each question
the theorized classes. Even though the model has been carefully and circle one appropriate answer by using
repeatedly validated, perhaps the model or instrument the following scale: Not at all (1); To a great extent (5).

1. To what extent are information Current Future

systems applied to advance these
CSFs?
1 2 3 4 5 1 2 3 4 5
72 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75

2. To what extent do the information Current Future

systems aid the organization in
competing in the market?
1 2 3 4 5 1 2 3 4 5
3. To what extent does the information Current Future
aid the organization in increasing
its profitability?
1 2 3 4 5 1 2 3 4 5
4. To what extent does the information Current Future
enable the organization to develop
and supply new products or
services or to improve existing
products and services?
1 2 3 4 5 1 2 3 4 5
5. To what extent do the information Current Future
systems serve the middle
management echelons?
1 2 3 4 5 1 2 3 4 5
6. To what extent are the information Current Future
systems used for administrative
and operational application?
1 2 3 4 5 1 2 3 4 5
7. To what extent do the information Current Future
systems enable significant improve-
ment in the organization's financial
status such as reducing manufacturing
costs, minimizing inventory, and
improving procedures for accounts
receivable?
1 2 3 4 5 1 2 3 4 5

8. To what extent are the information Current Future

systems critical to the functioning of
the organization?
1 2 3 4 5 1 2 3 4 5

Critical success factors (CSF) are de®ned as being scale: 1 ˆ Not important 2 ˆ Slightly important
areas in which performance must be good if the 3 ˆ Somewhat important 4 ˆ Important 5 ˆ Very
organization is to continue to exist and develop. important
Economic and ®nancial criteria

4.3. Part III. Criteria used for selecting an 1. Benefit/cost ration 1 2 3 4 5

information system (IS) project 2. Rate of return 1 2 3 4 5
3. Contribution of profitability, 1 2 3 4 5
In this section, from your perspective, you will e.g. reducing manufacturing
evaluate the importance of ®ve categories of IS project cost, minimizing inventory.
selection criteria. Please use the following rating 4. Growth rate 1 2 3 4 5
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75 73

5. Payback period 1 2 3 4 5 6. Availability of needed 1 2 3 4 5

6. The overall importance of 1 2 3 4 5 technology
the above financial criteria 7. The overall importance of 1 2 3 4 5
the above criteria
Organizational/institutional criteria
Information/decision considerations
1. Contribution to organiza- 1 2 3 4 5
tional goals/objectives 1. Contribution to quality 1 2 3 4 5
2. Aid the organization in 1 2 3 4 5 of decision making for
competing in the market administrative
3. Internal political decisions 1 2 3 4 5 2. Better information 1 2 3 4 5
(e.g. personal preference of 3. Information easy to assimilate 1 2 3 4 5
decision makers) 4. Faster availability 1 2 3 4 5
4. Importance to the organiza- 1 2 3 4 5 5. The overall importance of 1 2 3 4 5
tion for the future success the above criteria
5. Importance to the 1 2 3 4 5
functioning of the organization Risk factors
6. Public relations effect 1 2 3 4 5
(e.g. improve corporate image) 1. Technical risk (e.g. degree 1 2 3 4 5
7. Importance to organization's 1 2 3 4 5 of knowledge of both, user
critical success factors and team of the proposed
8. The overall importance of 1 2 3 4 5 new application)
the above criteria 2. Structure risk (e.g. change 1 2 3 4 5
of organizational structure,
Environmental criteria procedures)
3. Risk of cost overruns 1 2 3 4 5
1. Required by regulations 1 2 3 4 5 4. Size risk (e.g. # of user 1 2 3 4 5
(e.g. Federal, State) projects involved, estimated
2. Response to competition 1 2 3 4 5 project time)
(e.g. response time must equal 5. The overall importance of 1 2 3 4 5
or better competitors') the above risk criteria
3. Required by customers/ 1 2 3 4 5
suppliers Managerial consideration
4. New industry standards 1 2 3 4 5
5. Lawsuit requires information 1 2 3 4 5 1. Political acceptance 1 2 3 4 5
6. The overall importance of 1 2 3 4 5 2. End-user understanding, 1 2 3 4 5
the above criteria cooperation, and commitment
to project
Technical criteria 3 Top management support 1 2 3 4 5
4. Match with users' interest/ 1 2 3 4 5
1. Isolated, simple, and 1 2 3 4 5 work load
modular project 5. Middle management support 1 2 3 4 5
2. High visibility of project 1 2 3 4 5 6. The overall importance of 1 2 3 4 5
3. Basic subsystem to system 1 2 3 4 5 the above criteria
4. Basic module for operations 1 2 3 4 5
(e.g. data base system) Other Criteria: (Please specify)
5. Availability of skilled 1 2 3 4 5
personnel ÐÐÐ 1 2 3 4 5
74 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75

ÐÐÐ 1 2 3 4 5 6. Approximate annual gross revenues of your

ÐÐÐ 1 2 3 4 5 organization:
ÐÐÐ 1 2 3 4 5
___ under 1 million
The overall importance of 1 2 3 4 5
___ 1±50 million
the above criteria
___ 51±250 million
___ 250 million±1 billion
A.4 Part IV. Demographic information
___ 1±5 billion
___ 6 billion
1a. Which best describe your current position? (If
you are an IS personnel.) 7. Which one best describes the size of IS related
employees in your location?
(a) IS technical support staff
(b) system analyst ___ <10 people
(c) IS project leader ___ <50 people
(d) IS department/division manager ___ <100 people
(e) IS executive ___ <250 people
___ <1000 people
1b. Which best describe your current managerial
___ 1000
position? (If you are a non-IS related personnel)
8. Does your organization have a formal steering
(a) executive as part of the top management
committee for selecting the IS projects for develop-
(b) department/division manager
ment?
(c) assistant manager
(d) business/technical professionals Yes ___
No ___
2. Have you been involved in the decision of
selecting IS projects for development in the past?
Yes ___ References
No ___
3. Who are involved in the selection of IS projects in [1] R. Agarwal, L. Roberge, M.R. Tanniru, MIS planning: a
methodology for systems prioritization, Information and
your organization? (Please check all applied.) Management 27, 1994, pp. 261±274.
top management ___ [2] J.W. Bailey, J. Pearson, Development of a tool for measuring
and analyzing user satisfaction, Management Sciences 29,
functional areas managers ___ 1983, pp. 530±545.
IS department managers ___ [3] R.P. Bostrom, J.S. Heinen, MIS problems and failures: a
IS project leaders ___ socio-technical perspective. Part I. The causes, MIS
IS technical staff ___ Quarterly 4(1), 1980, pp. 21±34.
End-users ___ [4] M.S. Brenner, Practical R&D project prioritization, Re-
search±Technology Management 37(5), 1994, pp. 38±42.
Others (please specify) ___ [5] J.D. Callon, Competitive advantage through information
4. Gender: technology, McGraw±Hill, New York, NY, 1996.
[6] R.L. Carraway, Schmidt, An improved discrete dynamic
Male ___ programming algorithm for allocating resources among
Female ___ interdependent projects, Management Science 37(9) (1991)
1195±1200.
5. Age: [7] J.I. Cash, F.W. McFarlan, J.L. McKenney, L.M. Applegate,
Corporate Information Systems Management: Text and
<30___ Cases, third edn., Irwin Inc., Homewood, IL, 1992.
30 to 40 ___ [8] L.J. Cronbach, Coef®cient alpha and the internal structure of
40 to 50 ___ tests, Psychometrika 16, 1951, pp. 297±334.
[9] B.V. Dean, M.J. Schniederjans, A multiple objective
50 ___ selection methodology for strategic industry selection
 J.J. Jiang, G. Klein / Information & Management 36 (1999) 63±75
analysis, IEEE Transactions on Engineering Management
38(1), 1991, pp. 53±62.
[10] P.F. Drucker, The information executives truly need, Harvard
Business Review 73(1), 1995, pp. 46±56.
[11] D.H. Drury, A survey of data processing steering commit-
tees, Information and Management 9, 1985, pp. 1±7.
[12] M. Earl, Management Strategies for Information Technology,
Prentice±Hall, New York, NY, 1989.
[13] B. Fazolallahi, A framework for MIS planning, Ph.D.
Dissertation, School of Management, Syracuse University,
Syracuse, NY, 1984.
[14] D.L. Hall, A. Nauda, An interactive approach for selecting
R&D projects, IEEE Transactions on Engineering Manage-
ment 37(2), 1990, pp. 126±133.
[15] M. Igbaria, J.H. Greenhaus, The work attitudes of MIS
personnel: a survey of DPMA members, Information
Executive 4(2), 1991, pp. 48±53.
[16] M. Igbaria, S.R. Siegel, The career decision of information
systems people, Information and Management 24, 1993, pp.
23±32.
[17] M.G. Iyigun, A decision support system for R&D project
selection and resource allocation under uncertainty, Project
Management Journal 24(4), 1993, pp. 5±13.
[18] M.C. Jones, K.P. Annett, Current practices in management
information systems, Information and Management 16, 1993,
pp. 61±69.
[19] H.F. Kaiser, A second generation little jiffy, Psychometrika
35, 1970, pp. 402±415.
[20] G. Klein, H. Moskowitz, A. Ravindran, Comparative
evaluation of prior versus progressive articulation of
preference in bicriterion optimization, Naval Research
Logistics Quarterly 33, 1986, pp. 309±323.
[21] F. Krawiec, Evaluating and selecting research projects by
scoring, Research Management 27(2), 1984, pp. 21±25.
[22] D.W. Kroeber, H.J. Watson, Computer-Based Information
Systems: A Management Approach, Macmillan, New York,
NY, 1984.
[23] A.L. Lederer, A.L. Mendelow, Issues in information systems
planning, Information and Management 10, 1986, pp. 245±254.
[24] A.L. Lederer, A.L. Mendelow, Information resource plan-
ning: overcoming dif®culties in identifying top manage-
ment's objectives, MIS Quarterly 11(3), 1987, pp. 389±400.
[25] A.L. Lederer, A.L. Mendelow, The impact of the environ-
ment on the management of information systems, Informa-
tion Systems Research 1(2), 1990, pp. 205±222.
75
[26] A.L. Lederer, V. Sethi, The implementation of strategic
information systems planning methodologies, MIS Quarterly
12(3), 1988, pp. 444±461.
[27] M.L. Markus, Power, politics, and MIS implementation,
Communications of the ACM 26(6), 1983, pp. 430±444.
[28] F.W. McFarlan, Portfolio approach to information systems,
Harvard Business Review 62(3), 1984, pp. 98±103.
[29] F.W. McFarlan, J.L. McKenney, R. Pyburn, The information
archipelagoÐplotting a course, Harvard Business Review
61(1), 1983, pp. 145±156.
[30] G.M. McGrath, C.N.G. Dampney, E. More, Planning for
information systems integration: some key challenges,
Journal of Information Science 20(3), 1994, pp. 149±160.
[31] S. Neumann, N. Ahituv, M. Zviran, A measure for
determining the strategic relevance of IS to the organization,
Information and Management 22, 1992, pp. 281±299.
[32] R. Nolan, Managing the crisis in data processing, Harvard
Business Review 57, 1979, pp. 115±126.
[33] L.P. Pleban, H.K. Jain, Evaluating research proposals
with group techniques, Research Management 24, 1981,
pp. 34±38.
[34] M.E. Porter, Competitive Strategy: Techniques for Analyzing
Industries and Competitors, The Free Press, New York, NY,
1980.
[35] G. Premkumar, W.R. King, Organizational characteristics
and information systems planning: an empirical study,
Information Systems Research 5(2), 1994, pp. 11±19.
[36] J.M. Ruhl, L.M. Parker, The effects of experience and the
®rm's environment on manager's project selection decisions,
Journal of Managerial Issues 6(3), 1994, pp. 331±349.
[37] T.L. Saaty, The Analytic Hierarchy Process, McGraw±Hill,
New York, NY, 1980.
[38] R.J. Serling, Legend and Legacy: The Story of Boeing and
Its People, St. Martin, New York, NY, 1992.
[39] P. Shoval, R. Giladi, Determination of an implementation
order for IS projects, Information and Management 31, 1996,
pp. 67±74.
[40] E. Triantaphyllou, S, aÂnchez, A.A sensitivity analysis
approach for some deterministic multi-criteria decision-
making methods, Decision Sciences 28(1), 1997, pp. 151±
194.
[41] S. Tukana, R. Weber, An empirical test of the strategic-grid
model of information systems planning, Decision Sciences
27(4), 1996, pp. 735±763.