PERGAMON Omega, Int. J. Mgmt. Sci.

27 (1999) 327±339

The development of a tool for measuring the perceived

impact of information technology on work
G. Torkzadeh a, *, W.J. Doll b
a
College of Business Administration, The University of Texas at El Paso, El Paso, TX 79968, USA
b
The University of Toledo, Toledo, USA
Received 1 July 1997; accepted 1 July 1998

Abstract

The impact of information technology on work life has been one of the most talked about issues over the recent
years. Chief executive ocers spending millions of dollars on information technology face the critical issue of
assessing the impact of this technology on work. Information system managers are increasingly required to justify
technology investment in terms of its impact on the individual and his/her work. Measures of impact of information
technology have narrowly focused on productivity impacts. This study uses a broader concept that is based on the
impact of technology on the nature of work literature. This literature recognizes the multiple impacts of technology
on work at the level of the individual. A review of the literature enabled us to generate thirty-nine items that were
grouped into four constructs. In a pilot study, these constructs were assessed by observers in structured interviews
with eighty-nine users to provide a criterion measure. Next, the users completed the thirty-nine item questionnaire.
The unidimensionality, internal consistency and criterion-related validity of each construct were assessed. The pilot
results suggest a four factor 12-item instrument that measures how extensively information technology applications
impact task productivity, task innovation, customer satisfaction and management control. In a large scale study, a
sample of 409 respondents was gathered to further explore this 12-item instrument and its relationships with other
constructs (user involvement, user satisfaction, system usage). The results support the four factor model. Evidence
of reliability and construct validity is presented for the hypothesized measurement model and future research is
discussed. # 1999 Elsevier Science Ltd. All rights reserved.

Keywords: Information technology impact; System impact; Technology impact on work; Instrument development; System to value
chain; System success measures

1. Introduction its impact. Organizations that spend millions of dollars

Both academics and practitioners recognize that the
success of information technology can potentially be
measured through its impact on work at the level of
the individual end-user. The widespread use of infor-
mation technology by non-data processing pro-
fessionals and growing line management responsibility
for managing information technology, have further
increased the potential of information technology and
* Corresponding author. Tel.: +1-915-747-5496; fax: +1-
915-747-5126; e-mail: reza@utep.edu.
on information technology are primarily concerned
about how their investment will in¯uence organiz-
ational and individual performance. MIS researchers
have quite appropriately increased their attention to
measurement issues in general and success measures in
particular.
To evaluate information systems success, MIS
researchers have measured a variety of constructs such
as perceived usefulness [10], currency [1], accuracy [13]
and relative advantage [41]. Despite the contributions
made by MIS researchers in the development of suc-
cess measures, Melone [39] calls for the development
of e€ectiveness measures that are outcome-oriented.

0305-0483/99/$ - see front matter # 1999 Elsevier Science Ltd. All rights reserved.
PII: S 0 3 0 5 - 0 4 8 3 ( 9 8 ) 0 0 0 4 9 - 8
328 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
Goodhue [19] proposes evaluation constructs that can
usefully link underlying systems to their relevant
impacts.
The impact of information technology is broad and
multifaceted providing signi®cant research opportu-
nities and challenges. There has been a growing inter-
est on the part of MIS researchers to study
information technology impacts: on competitive
advantage [50], on organizational strategy [35], on time
utilization [53], on middle managers [40, 43] and on
industry level competitive advantage [49]. While the
research on impact of information technology has been
diverse, it has not focused on work at the level of the
individual.
Palmquist [42] suggests that while the societal
changes represent a fascinating and complex research
challenge, information scientists must bear some
responsibility to anticipate the impact of improved
access to information. Reviewing the relevant litera-
ture, he observed that there seemed few attempts to
examine the impact of information technology empiri-
cally. Palmquist's review of the literature found that of
all the research done to examine the new information
technologies, the impact on the individuals and their
work is least addressed. He believes it will become
more and more important for researchers to examine
the impacts of information technology on all aspects
of an individual's life.
Explicit objectives of this research were to develop
an instrument that: (1) identi®es the multidimensional
nature of information technology's impact at the level
of the individual end-user; (2) is short, easy to use and
appropriate for both academic research and practice
and (3) can be used with con®dence across a variety of
applications and contexts (i.e. generalizability as evi-
denced by reliability and predictive validity). These
goals in¯uenced the methods used to generate items,
the nature of the sample, the procedures used to elim-
inate items to create a shorter and easier to use ques-
tionnaire, the nomological network of constructs
chosen to assess construct validity, and the methods
used to assess the generalizability of the instrument. In
the following sections, we ®rst provide the theoretical
underpinning for the measurement of information
technology impact. Then, we describe research
methods and surveys followed by practical and theor-
etical applications of the proposed measures.
Fig. 1. System to value chain.
2. The impact of information technology on work
Based on attitude-behavior theory, Doll and
Torkzadeh [15] describe a `system to value chain' of
system success constructs from beliefs, to attitudes, to
behavior, to the social and economic impacts of infor-
mation technology (see Fig. 1). In this taxonomy,
impact is a pivotal concept that embodies downstream
e€ects. It is dicult to imagine how information tech-
nology can be assessed without evaluating the impact
it may have on the individual's work. Impact is said to
occupy a pivotal position in a chain of system to value
constructs because it is a direct consequence of usage
and, in turn, it is a major factor determining organiz-
ational impact.
2.1. Information technology impact
Information technology has and continues to exert a
strong in¯uence on the individual and his/her work. It
may be central to our future understanding of the
structure and function of work organization for
human productivity, but for the individual it holds the
power to enhance a job or render human labor
unnecessary [41]. Zubo€ [55] contends that technology
cannot be considered neutral; it is brimming with
valence and speci®city in that it both creates and fore-
closes avenues of experience. Coates [4] asserts that
technology intrinsically, not accidentally, forces econ-
omic, political or social change.
Danziger [9] identi®es the computer as the key tech-
nological device producing the third great revolution
in human history. He suggests that the impacts of
computing are highly contingent upon the context of
use. That is, the impacts are quite varied in their
nature and their levels, not only across di€erent types
of settings, but also across di€erent computing con-
texts within comparable settings and even across di€er-
ent individuals within comparable settings and
computing contexts. In today's end-user computing en-
vironment, the natural concern over how well designed
information systems are must be augmented with an
equally ardent concern about how e€ectively they are
used or what impact they have on work.
Literature [2, 30, 33, 37, 42, 46, 51, 54, 55] suggests
that in the industrial model, technology was used to
substitute capital for labor with impacts on labor pro-
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
ductivity and enhanced management control as pre-
dictable machines replaced variable human behavior.
In the post-industrial model, the focus is still on pro-
ductivity and management control but the stochastic
nature of the technology makes management control
more dicult. In addition, the post-industrial model
includes innovation and customer satisfaction as
necessary impacts of technology [7, 21, 22, 25, 31, 47, 55].
A conception of technology's impact that is limited
to productivity and/or management control is rooted in
an out-of-date paradigm that ignores organizationally
relevant impacts essential to the success and survival of
modern organizations. In the context of information
technology and white-collar productivity, Davis [11]
suggests that improved e€ectiveness in serving internal
and external customers is a better basis for judging
most white-collar work and that too much emphasis on
lowering costs and increasing output may prove coun-
terproductive. Traditional concepts of technology's
impact emphasize task productivity and largely ignore
the role of technology in serving customers.
2.2. Identifying and operationalizing impact dimensions
The impact of information technology literature lar-
gely provides ad hoc examples and illustrations of how
systems may impact the individual. Systematic atten-
tion has not focused on conceptualizing the extent and
dimensions of this impact. Based on the works of
Braverman [2], Curley and Pyburn [7], Danziger [9],
Davis [11], Filiatrault et al. [17], Harvey [20], Harvey,
et al. [22], Hirschhorn and Farduhar [24],
Hirschhorn [25], Kraemer and Danziger [30], Larson
and Fielden [31], Li€ [32], Long [33], Palmquist [42],
Sulek and Marucheck [43], Shaiken [51], Weick [54]
Table 1
De®nitions of information technology's impacts on work
Label De®nition of construct
Task productivity The extent that an application improves
the user's output per unit of time
Task innovation The extent that an application helps users
create and try out new ideas in their work
Customer The extent that an application helps the user
satisfaction create value for the ®rm's internal or external
customers
Management The extent that the application helps to
control regulate work processes and performance
329
and Zubu€ [55] the researchers developed explicit de®-
nitions for four impact dimensions (see Table 1). These
four impact dimensions, taken together, describe `how'
an application impacts the individual in an organiz-
ational context. These impact dimensions are de®ned
at the application level. An application is de®ned as
the use of information technology to accomplish work.
When applications are used in these ways, workers are
empowered by information technology.
2.2.1. Task productivity
Information technology impact on task pro-
ductivity is suggested by numerous authors
[2, 6, 7, 9, 10, 18, 24, 30, 32, 43, 54, 55]. Braverman [2]
suggests that organizations automate to improve pro-
ductivity and management control. Zubo€ [55] and
Weick [54] discuss the problems of making full use of
information technology in a post-industrial work en-
vironment to improve productivity. Zubo€ suggests
that information technology essentially alters the con-
tours of reality, productivity increases by an order of
magnitude and measures of productivity become more
abstract and dicult to determine. Weick argues that
new technologies mean many things because they are
simultaneously the source of events that are stochastic,
continuous, and abstract requiring di€erent measures
of productivity. Li€ [32] reported that most clerical
and secretarial employees in her survey believed that
`oce automation' had increased the interest and skill
requirements of their jobs resulting in higher pro-
ductivity.
2.2.2. Task innovation
The impact of information technology on task inno-
vation has gained increased recognition in recent
Literature support
Braverman [2], Curley and Pyburn [7], Hirschheim
and Farduhar [24], Kraemer and Danziger [30],
Li€ [32], Sulek and Marucheck [43], Weick [54]
and Zubo€ [55]
Curley and Pyburn [7], Davis [11], Harvey et al. [22],
Hirschhorn [25], Larson and Fielden [31], Long [33]
and Zubo€ [55]
Curley and Pyburn [7], Filiatrault et al. [17], Harvey and
Filiatrault [21]], Harvey et al. [22], Hirschhorn [25, 26],
Schlesinger and Haskett [47]
Braverman [2], Hirschhorn [26], Kraemer and
Danziger [30], Shaiken [51] and Zubo€ [55]
330 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
years [7, 11, 17, 22, 25, 31, 33, 54, 55]. Information tech-
nology facilitates innovation through learning. In dis-
cussing the impact of `industrial humanism
management', Long [33] suggests that employees will
use information technology to enrich and broaden jobs
(i.e. innovate). Larson and Fielden [31] suggest the
need for task innovation or problem solving activity as
opposed to number of word measures of productivity.
Harvey et al. [22] suggest that many innovative compa-
nies are using the technology and exploring new ways
to interface with customers. In the emerging post-
industrial setting, worker skill is not the skill of execut-
ing but the skill of problem solving and the ability to
learn and innovate [25]. Work becomes heavily depen-
dent on information processing, increasingly abstract
and mediated by sensing mechanisms. Thus, in study-
ing the impact of information technology, we cannot
just study what workers do, but rather how innovative
they are in what they have to do. The overall impact
of technology is to di€use the management function in
new ways as work (1) becomes a fusion of working
and learning, (2) is identi®ed with developmental ac-
tivity and (3) is organized through team relationships.
2.2.3. Customer satisfaction
Information technology empowers employees to pro-
vide better and faster service to both internal and
external customers [7, 17, 21, 22, 25, 26, 47]. Describing
the role of technology in service quality, Harvey et
al. [22] point out the more dicult task of ``balancing
short-term, directly measurable productivity gains
against the more important and more pervasive, but
more dicult to measure gains that come from produ-
cing and delivering more value to the customer''. They
suggest a model that describes how services that pro-
vide `value-added partnerships' can be created through
information technology. Failiatrault et al. [17] suggest
a close relation between service quality, productivity
and customer satisfaction. They argue that improving
productivity can improve customer satisfaction.
Harvey and Filiatrault [21] review technology adoption
in the banking industry and describe, for example,
how on-line technology can speed up client queries or
implement client requests, increasing customer satisfac-
tion. Improved service quality is perceived through
close interaction and real-time ¯ow of information.
Information technology applications can be used to
create customer `empowerment' that will ultimately
result in customer satisfaction [47].
2.2.4. Management control
Many authors suggest that one of the initial intents
for using a new technology is to increase management
control. [2, 26, 30, 51, 55]. Although technology rede-
®nes the possible, it cannot determine which choices
are taken up and to what purpose. Some have attribu-
ted systematic and purposeful agency to the manage-
rial use of technology [2, 51]. They argue that
managers are interested exclusively in technology as a
means of controlling, limiting, and ultimately weaken-
ing their work force. Shaiken [51] anticipates fewer
skills, less worker input and increased managerial con-
trol. He suggests that a high level of skill is often
embodied in computerized systems which will lay the
basis for a considerable transfer of power to manage-
ment. In some cases, sophisticated management, es-
pecially in small shops specializing in intricate
prototype work, have felt that ecient production
requires skilled workers. Under these circumstances,
the machine is viewed as a tool for the worker rather
than as a vehicle to control the worker. Shaiken
suggests that the design of the technology determines
the context in which labor and management forces
play themselves out. Zubo€ suggests a more compli-
cated reality. With the di€usion of management func-
tion, management control has a di€erent meaning.
Sometimes management is part of a group that moni-
tors his/her own performance. Using a socio-technical
perspective, Hirschhorn [26] contends that information
technology is used in new ways in an emerging post-
industrial labor process and has decisive implications
for the general texture of life. He suggests that if we
are to make full use of the new technologies, we must
confront the constraints imposed by management sys-
tems, capitalistic interests and old organizational de-
signs.
3. Methods
A literature review was conducted to ensure that a
comprehensive list of items was generated to measure
dimensions of application impact. The works of MIS
researchers [18, 27, 33, 34, 36, 38, 43, 44, 48, 52], works
on the social and economic impact of information
technology [2, 8, 9, 24, 25, 28, 29, 33, 36, 42, 45, 51, 54, 55]
and works on white collar productivity [11, 20±23, 31±
33, 47, 53] were reviewed. This literature, especially the
descriptions of information technology's impact on the
nature of work, is a rich source of illustrations or
examples of how information systems are used by indi-
viduals in their work context. Based upon this review,
the researchers generated thirty-nine items to measure
aspects of impact (see Appendix A). A ®ve point
Likert-type scale was used where 1 = not at all, 2 = a
little, 3 = moderately, 4 = much and 5 = a great deal.
The researchers developed a structured interview
questionnaire and process for the pilot study to: (1)
assess whether the instrument was capturing the
phenomenon desired by the researchers; (2) verify that
important aspects of impact dimensions were not
omitted and (3) enable an interviewer to make an
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
assessment (observation) of the dimensions of impact
described above. First, personal interviews with the
end-users were conducted, then the self-administered
thirty-nine item questionnaire was distributed. The
interviewers (6) were experienced systems analysts
(from two to six years as a professional systems ana-
lyst) who were carefully trained to conduct the inter-
views and make observations. The interviews took
about 2 h each.
The users were asked to demonstrate their appli-
cation, provide sample output reports, and describe
how they used the application in their work. For each
dimension, users provided qualitative information
describing the impact of their applications. As each
category of impact was discussed, the interviewer made
an observation of the extent of impact using a 5-point
Likert-type scale (1 = not at all to 5 = a great deal).
This observer's score was used as one criterion to vali-
date the items on the self-administered questionnaire.
3.1. Assessment of measurement properties
The questionnaire responses and observer criterion
scores were explored with several objectives in mind:
puri®cation, unidimensionality, reliability, brevity and
simplicity of factor structure. First, the researchers
wanted to purify the items before doing factor analysis
(i.e. eliminate `garbage items'). The need to purify the
items that are suggested as measures of a construct is
described by Churchill [5]. He contends that when fac-
tor analysis is done before puri®cation, there seems to
be a tendency for factor analysis to produce many
more dimensions than can be conceptually identi®ed,
confounding the interpretation of the factor analysis.
Two independent criteria were used to eliminate
items. First, items were eliminated if their corrected-
item total correlation (each item's correlation with the
sum of the other items in its category) was less than
0.70. The domain sampling model provides a rationale
for this procedure. The key assumption in the domain
sampling model is that all items, if they belong to the
domain of the concept, have an equal amount of com-
mon core. If all the items in a measure are drawn from
the domain of a single construct, responses to those
items should be highly intercorrelated. The corrected-
item total correlation provides a measure of this [5].
Items were also eliminated if their correlation with
the observer's criterion score, obtained through the
pilot study, was less than 0.70. The users' descriptions
and demonstrations of their applications, the examin-
ation of output reports, and the qualitative interview
comments recorded for each dimension enabled the
research assistants to make observations of an applica-
tion's impact on the interviewee. These observer scores
(using a 1 to 5 scale) for each category of items were
also used to eliminate `garbage items'.
331
Finally, the reliability (internal consistency) of the
remaining items comprising each dimension was
examined using Cronbach's alpha to see if additional
items could be eliminated without substantially low-
ering reliability. Items were eliminated if the re-
liability of the remaining items would be at least
0.90. Where deleting either of two items would have
had the same impact on alpha, the item with the
highest correlation with the observer criterion was
retained.
After these deletions, an exploratory factor analy-
sis of the remaining items was conducted to deter-
mine whether the entire set of items had a simple
factor structure and to determine the number of
factors. Finally, to assess criterion-related validity,
the four constructs were correlated with the four
observer criteria: (1) to assess whether their scales
were more highly correlated with the observer score
for their theoretical construct than with observer
scores for the other constructs and (2) to assess
whether the observer scores for each construct were
more highly correlated with the scale for its theor-
etical construct than with the scales for other con-
structs.
After determining the factor structure, the research-
ers examined construct validity by assessing the corre-
lation between impact dimensions and the nomological
network of constructs. These constructs included a
measure of perceived user involvement, user satisfac-
tion, and usage pattern. User involvement was
measured using an eight item scale developed by Doll
and Torkzadeh [14]. User satisfaction was measured
using the 12-item end-user computing satisfaction
(EUCS) instrument [13]. Both instruments use a ®ve
point Likert-type scale: 1 = not at all; 2 = a little;
3 = moderately; 4 = much; 5 = a great deal). The 30-
item perceived usage pattern scale includes items for
decision support, work integration and customer
service [16]. A ®ve point Likert-type scale is used
(1 = not at all; 2 = a little; 3 = moderate; 4 = much;
5 = a great deal).
4. Pilot study
Eighty-nine usable interviews were obtained using a
structured interview procedure followed by the self-
administered questionnaire. Qualitative comments
from the structured interviews were compared with the
responses to items in each of the four categories. This
enabled the researchers to verify that the respondents
knew what the items were asking. To make the results
more generalizable, the researchers gathered data from
30 di€erent ®rms. We tried to survey a variety of user
situations including individuals engaged in computer
intensive work as well as more casual users.
332 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339

4.1. Purifying items structure was easy to interpret and corresponding

Twenty four of the thirty-nine items illustrated in
Appendix A were eliminated because they had either
a corrected-item total correlation or a correlation
with the observer criterion less than 0.70. About
90% of eliminated items failed both criteria and the
remainder failed (about equally) either corrected-
item correlation or correlation with the criterion.
Reliability analysis resulted in further deletion of
three items: coecient alpha for the task innovation
items could be improved if one item was deleted,
coecient alpha for management control items
could be improved if two items were deleted.
Table 2 reports the corrected-item total correlations
and correlations with the observer criterion for the
remaining 12 items. Coecient alpha for each con-
struct is also provided in Table 2. Once the items
were puri®ed and reliability was established, the
construct validity of the four scales was examined.
4.2. Construct validity results
The factor analysis of the items indicated unidi-
mensionality of each construct. There were no mul-
tiple loadings for any category. The ratio of sample
size to number of items (7.4:1) was lower than
accepted standard (10:1), but considered adequate for
exploratory analysis at the pilot stage. The factor
with the four categories described above. The range
for factor loading was 0.90 to 0.92 for task pro-
ductivity, 0.87 to 0.94 for task innovation, 0.88 to
0.94 for customer satisfaction and 0.94 to 0.96 for
management control. The four-factor model
explained 89.9% of the variance.
Finally, to assess criterion-related validity, the rec-
ommended scales for the four constructs were corre-
lated with the observer criterion scores for each
construct. The correlations for each construct and its
corresponding observer criterion score were 0.90 for
task productivity, 0.89 for task innovation, 0.84 for
customer satisfaction and 0.91 for management con-
trol. These correlations are signi®cant at p < 0.001.
Each observer criterion is more correlated with its cor-
responding scale than it is with the scales measuring
the other constructs.
These pilot study methods enabled the researchers to
verify that the four dimensions of information techno-
logy's impact on work made sense in terms of the end-
user's work context. The high correlation between
observer ratings and end-user responses support the
contention that these impacts of technology on work
can be veri®ed by trained observers as well as user re-
sponse data. Thus, pilot study resulted in a 12-item
instrument that operationalizes four constructs (task
productivity, task innovation, customer satisfaction
and management control).

Table 2
Measures of application impact on work (n = 89)

Corrected item total Correlation with

correlation criterion

Task productivity (alpha = 0.90)

P1 this application saves me time 0.83 0.80
P2 this application increases my productivity 0.80 0.84
P3 this application allows me to accomplish more work than 0.76 0.82
would otherwise be possible

Task innovation (alpha = 0.95)

I1 this application helps me create new ideas 0.93 0.85
I2 this application helps me come up with new ideas 0.91 0.82
I3 this application helps me try out innovative ideas 0.82 0.86

Customer satisfaction (alpha = 0.94)

C1 this application improves customer service 0.92 0.80
C2 this application improves customer satisfaction 0.89 0.81
C3 this application helps me meet customer needs 0.84 0.78

Management control (alpha = 0.97)

M1 this application helps management control the work process 0.94 0.86
M2 this application improves management control 0.93 0.88
M3 this application helps management control performance 0.91 0.90
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339 333

5. The survey iety of applications support the generalizability of the

To further explore this 12-item instrument, a sample
of 409 computer end-users was gathered from 18 or-
ganizations including 8 manufacturing ®rms, 1 retail
®rm, 2 government agencies, 2 utilities, 2 hospitals, 2
educational institutions and one `other'; this rep-
resented half of the ®rms contacted. The response rate
for ®rms willing to participate with the study was over
70%. This second data set enabled the researchers to
re-examine factor structure and reliability. To assess
construct validity, the researchers also gathered data
on other constructs (i.e. user involvement, user satis-
faction, usage pattern) across the system to value
chain.
The sample represents 139 di€erent applications
including accounts payable, accounts receivable, bud-
geting, CAD, CAD-CAM, customer service, service
dispatching, engineering analysis, process control,
work order control, general ledger, manpower plan-
ning, ®nancial planning, inventory, order entry, pay-
roll, personnel, production planning, purchasing,
quality, sales analysis, student data and pro®t plan-
ning. The large number of organizations and the var-
®ndings.
Respondents were asked to identify their position
within the organization; they responded as follows: 20
top level managers, 80 middle managers, 76 ®rst level
supervisors, 143 professional employees, and 90 oper-
ating personnel. 47% of the respondents stated that
their application was a personal computer (micro) ap-
plication. 36% of the computer applications were
developed primarily by end-users but only 12% of the
respondents personally developed the applications
themselves. 50% of the applications provided data
analysis capabilities (spreadsheet, modeling, simulation,
optimization or statistical routines). 46% of the appli-
cations provided a data base with ¯exible inquiry capa-
bilities.
5.1. Reliability and validity results
An exploratory factor analysis was conducted using
principal components as the means of extraction and
oblimin as the method of rotation. Without specifying
the number of factors, there were four factors with
eigenvalues greater than 1.0. The four factor solution

Table 3
Factor pattern of application impact items (n = 409)

Factor 1 Factor 2 Factor 3 Factor 4

C1 0.907
C2 0.897
C3 0.876

I1 0.896
I2 0.873
I3 0.867

M2 0.927
M3 0.879
M1 0.876

P2 0.872
P1 0.866
P3 0.827

Factor correlation matrix

Factor 2 0.35
Factor 3 0.36 0.36
Factor 4 0.53 0.31 0.50

Mean 9.52 6.85 9.12 10.72

Standard deviation 3.86 3.46 3.87 3.64

Eigenvalue 6.01 1.79 1.65 1.08

Total variance explained = 87.7%.

334 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339

reported in Table 3 represent a simple solution. The
items are grouped by their highest (primary) factor
loading and listed in descending order. Loadings
greater than or equal to 0.30 are reported. The four
factors explained 87.7% of the variation in the 12
items and were interpreted as customer satisfaction,
task innovation, management control and task pro-
ductivity, respectively. Table 3 also provides factor cor-
relation matrix, means and standard deviations for the
four factors. Thus, the four factor model identi®ed in
the pilot study was supported by this large sample ex-
ploratory analysis.
Using this sample of 409 responses, the corrected-
item total correlation and reliability (alpha) for each of
the four impact scales were calculated. The corrected-
item total correlations range from 0.82 (P1) to 0.94
(C1). Reliability scores are 0.93, 0.95, 0.96 and 0.93 for
task productivity, task innovation, customer satisfac-
tion and management control, respectively. Overall re-
liability for the 12-item scale was 0.92.
5.2. Convergent and discriminant validity results
The instrument's correlation matrix was analyzed
for convergent and discriminant validity [13]. This
approach to convergent validity tests that the corre-
lations between measures of the same theoretical con-
struct are di€erent than zero and large enough to
warrant further investigation. The smallest within vari-
able (factor) correlations are: task productivity = 0.79;
task innovation = 0.73; customer satisfaction = 0.81
and management control = 0.73. For a sample of 409,
these are signi®cantly ( p < 0.001) di€erent than zero
and large enough to encourage further investigation.
Using this approach, discriminant validity is tested
for each item by counting the number of times it corre-
lates more highly with an item of another variable (fac-
tor) than with items of its own theoretical variable.
Campbell and Fiske [3] suggest determining whether
this count is higher than one-half the potential com-
parisons. However, in this case, common method var-
iances are present so it is unclear how large a count
would be acceptable. An examination of the matrix for
the 12-item instrument reveals zero violation (out of 66
comparisons) of the condition for discriminant validity.
Each of the twelve items are more highly correlated
with the other items in its group than with any of the
items measuring other variables.
5.3. General applicability results
Correlations between the four impact scales being
proposed and related constructs thought to comprise
the nomological network were examined. In this
sample, the 8-item user involvement and 12-item user
satisfaction scales have reliabilities (Cronbach's alpha)
of 0.96 and 0.92, respectively. The 30-item usage pat-
tern scale has a reliability of 0.96. The correlation
matrix for the impact scales and the related constructs
in the nomological network is provided in Table 4.
User involvement has signi®cant correlations with
all potential success measures. Correlations between
user involvement and usage pattern (r = 0.31), per-
ceived impact (r = 0.32) and EUCS (r = 0.30) are
remarkably similar and signi®cant ( p < 0.01). EUCS
has signi®cant ( p < 0.01) correlation with usage pat-
tern (r = 0.29) and the four dimensions of impact scale
(task productivity, r = 0.47; task innovation, r = 0.30;
customer satisfaction, r = 0.28 and management con-
trol, r = 0.17). EUCS's signi®cant correlations with
overall as well as subscales of perceived impact suggest
that EUCS may have a direct as well as an indirect
(through usage patterns) relationship with perceived
impact.

Table 4
Correlations among constructs (n = 409)

1 2 3 4 5 6 7 8

(1) Overall impact (12 items)

(2) Task productivity (3 items) 0.80**
(3) Task innovation (3 items) 0.74** 0.52**
(4) Customer satisfaction (3 items) 0.77** 0.55** 0.38**
(5) Management control (3 items) 0.71** 0.37** 0.37** 0.37**
(6) Usage pattern (30 items) 0.70** 0.52** 0.64** 0.46** 0.49**
(7) User involvement (8 items) 0.32** 0.26** 0.40** 0.10 0.21** 0.31**
(8) User satisfaction (12 items) 0.40** 0.47** 0.30** 0.28** 0.17** 0.29** 0.30**

Mean 36.17 10.72 6.85 9.52 9.12 62.9 14.6 47.9

Standard deviation 11.20 3.64 3.46 3.86 3.87 24.1 9.7 8.3

*
Signi®cance less than or equal to 0.05.**Signi®cance less than or equal to 0.01
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339 335

The four dimensions of impact scale are signi®-
cantly correlated with each other with the closest as-
sociation existing between task productivity and
customer satisfaction (r = 0.77). The lowest corre-
lations (r = 0.37) exist between management control
and other three dimensions of the impact scale. Each
of the four impact dimensions have signi®cantly
( p < 0.01) higher correlations with perceived usage
pattern than other measures (i.e. user satisfaction,
user involvement), suggesting that they are more clo-
sely associated with usage than user satisfaction or
user involvement. Means, standard deviations, and
correlations for the 12-item scale are provided in
Appendix B.
The reliability of the impact scales for the entire
sample and for di€erent hardware platforms, types of
applications, modes of application development, and
required usage conditions are reported in Table 5. All
45 Cronbach alpha scores are high (ranging from
0.89 to 0.97), indicating remarkable stability across
all contextual variables. These reliability alphas sup-
port the generalizability of the impact scales across a
variety of hardware platforms, application types,
modes of development and required or voluntary
usage situations.
6. Discussion
A multidimensional measure that identi®es how in-
formation systems impact work may have several ad-
vantages: (1) it might help us distinguish between
e€ective and ine€ective systems; (2) as a measure of
e€ectiveness and productivity, it should be of great
interest to executive management; (3) as an outcome
measure, it might stimulate new research that has prac-
tical implications for how systems are designed and
developed and (4) it may help us gauge and reduce a
perceived gap between the potential of information
technology and its actual impact.
The results of pilot study were substantive and gave
the researchers con®dence that the four dimensions
make sense in term of the end-user work context and
represent real phenomenon that can be observed/veri-
®ed by a trained professional. The results based on
observer criterion scores also supports the use of end-
user responses as a way of measuring the impacts of
information technology.
6.1. Implications
Substantial experimentation and learning may be
necessary before we can develop ®nal guidelines for
using these instruments. In this section, we provide
some tentative suggestions for the instrument's appli-
cation.
This 12-item instrument may be used to evaluate the
impact of applications. The Likert-type scales are easy
to administer and the instrument is short enough to be
used in most research or applied settings. It is our
assumption that both performance-related behaviors
and attitudes determine the impact of information
technology. Thus, user satisfaction measures should be
augmented, not replaced, by impact measures. These
measures of impact can be used for a variety of situ-
ations including: (1) where ¯exible technology is pre-

Table 5
Reliability by nature and type of application

Overall Task Task Customer Management

impact productivity innovation satisfaction control

Mode of development
End-user development (n = 129) 0.92 0.93 0.95 0.96 0.93
Traditional development (n = 259) 0.90 0.93 0.90 0.94 0.91

Hardware platform
Personal computer (n = 181) 0.89 0.92 0.93 0.94 0.94
Mini or mainframe (n = 204) 0.93 0.93 0.91 0.95 0.90

Usage required?
Yes (n = 249) 0.92 0.93 0.91 0.94 0.89
No (n = 49) 0.92 0.93 0.94 0.97 0.92

Type of application
Decision support (n = 185) 0.89 0.91 0.89 0.96 0.92
Data base (n = 175) 0.90 0.92 0.91 0.96 0.90
Other (n = 131) 0.91 0.92 0.94 0.93 0.92
336 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
sent and may have di€erent degrees of impact on the
individual; (2) where there are substantial di€erences
between users in how e€ectively they use a given appli-
cation; (3) where user attitudes are likely to be biased
and (4) where one wishes to implement a behaviorally-
based training program and one desires some initial
behavioral benchmarks.
The impact scales might be used to make compari-
sons between users of the same software packages (e.g.
to identify di€erences between individuals and assess
training needs). Also, one might wish to use di€erent
parts of the instrument to evaluate di€erent types of
applications. For example, the task productivity scale
might be the only scale relevant to evaluating a ®rm's
task productivity applications.
As measures of e€ective or ine€ective application,
these instruments open promising new areas of inquiry.
For example, information systems research on learning
processes or continuous improvement behaviors has
been hampered by the lack of an appropriate depen-
dent variable [12]. Researchers might use these instru-
ments to facilitate the identi®cation of situational
factors and processes that determine both the e€ective-
ness of application and the rate at which individuals
learn to apply new technologies which are made avail-
able to them. Research on the link between upstream
causal factors such as design features or development
methods and their downstream consequences (i.e. how
e€ective information systems are) can also bene®t from
these instruments. Impact dimensions have an advan-
tage as a construct for integrating MIS research with
studies of information technology usage; a causal link
between use and impact is quite plausible.
6.2. Extensions
There is a clear need for further research and con®r-
matory methods to examine the second-order nature of
the impact on work construct and its components,
®rst-order dimensions. It is intuitively plausible to
expect a second-order model for the impact of technol-
ogy on work and this study clearly suggests the four
components. However, only further research can pro-
vide a clearer picture for this phenomenon employing
con®rmatory factor analysis. Further research should
also examine the relationship between the impact of in-
formation technology construct and outcome variables
such as information technology usage.
In the emerging post-industrial setting, scholars con-
tend that information technology is used in new ways
that have substantial implications for the nature of
work, productivity and economic growth. Information
technology has the potential to di€use management
functions throughout the work force. Individuals will
e€ectively use information technology to improve pro-
ductivity, create new ideas, meet internal and external
customer needs and control the work process within
and between groups. In longitudinal research designs,
these impact measures provide a tool for assessing
whether information technology is, indeed, being used
in new ways.
7. Conclusions
The ultimate question for use of information tech-
nology by individuals and organizations may be
more related to how the technology impacts than
how technology is designed or regarded. Despite
wide acceptance of and call for outcome oriented
measures of information systems success, the impact
construct, its conceptualization and measurement, has
received inadequate research attention. This research
reports on an initial e€ort to conceptualize the
impact construct and develop measures of infor-
mation technology impact that have adequate re-
liability and validity. The four impact dimensions
identi®ed (task productivity, task innovation, custo-
mer satisfaction and management control) measure
the extent of information technology impact on
work. Further research is needed to con®rm the hy-
pothesized measurement model, assess the stability of
the instrument and develop standards for evaluating
speci®c applications.
Appendix A. Measures of application impact on work
(thirty nine items used in pilot study)
Impact on task productivity:
(1) This application improves the quality of my
work.
(2) This application increases my productivity.
(3) This application saves me time.
(4) This application enables me to accomplish tasks
more quickly.
(5) This application supports critical aspects of my
job.
(6) This application allows me to accomplish more
work than would otherwise be possible.
(7) This application enables me to spend more time
on productive activities.
(8) My e€ectiveness on the job is enhanced by this
application.
(9) The quality of my work is improved by this ap-
plication.
(10) This application is useful in my job.
(11) This application assists me in performing my job
better.
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339 337

(12) The quality of my work depends upon this appli- (26) This application enables me to respond to chan-
cation. ging customer needs.
(27) This application helps me accommodate individ-
ual customer needs.
Impact on task innovation: (28) This application enables me to deal more strate-
gically with internal and/or external customers.
(13) This application helps me identify innovative
ways of doing my job.
(14) This application helps me come up with new sol- Impact on management control:
utions to job problems.
(15) This application helps me create new ideas. (29) This application improves management control.
(16) This application helps me come up with new (30) This application helps management control the
ideas. work process.
(17) This application helps me solve job problems. (31) This application helps management control per-
(18) This application helps me ®nd new ways to formance.
improve my job performance. (32) This application enables management to com-
(19) This application helps me try out innovative pare work performance to standards.
ideas. (33) This application helps management identify
when corrective action is required.
(34) This application enables management to monitor
Impact on internal/external customer satisfaction: and correct errors.
(35) This application enables management to control
(20) This application improves customer satisfaction. work schedules.
(21) This application improves customer service. (36) This application enables management to monitor
(22) This application makes me more customer work progress.
oriented. (37) This application enables management to ensure
(23) This application helps me create value for custo- a timely completion of tasks.
mers. (38) This application enables management to control
(24) This application helps me meet customer needs. resource allocation.
(25) This application helps me adapt to changing cus- (39) This application enables management to control
tomer needs. quality.

Appendix B. Means standard deviations and correlations for the 12-item impact scale (n=409)
Table 6

P1 P2 P3 I1 I2 I3 C1 C2 C3 M1 M2 M3

P1
P2 0.84
P3 0.79 0.81

I1 0.43 0.45 0.48

I2 0.40 0.45 0.48 0.83
I3 0.41 0.45 0.47 0.80 0.73

C1 0.52 0.47 0.48 0.35 0.32 0.30

C2 0.48 0.43 0.46 0.38 0.33 0.32 0.87
C3 0.53 0.51 0.51 0.34 0.30 0.31 0.88 0.81

M1 0.33 0.35 0.39 0.33 0.35 0.24 0.35 0.30 0.34

M2 0.27 0.31 0.32 0.33 0.33 0.27 0.33 0.26 0.31 0.81
M3 0.29 0.34 0.34 0.36 0.32 0.30 0.37 0.31 0.37 0.73 0.83

Mean 3.7 3.5 3.5 2.3 2.4 2.2 3.2 3.1 3.2 3.2 3.1 2.9
Standard deviation 1.2 1.3 1.3 1.3 1.2 1.3 1.4 1.3 1.4 1.4 1.4 1.4
338 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
References
[1] Bailey JE, Pearson SW. Development of a tool for
measuring and analyzing computer user satisfaction.
Manage Sci 1983;29(5):530±45.
[2] Braverman H. Labor and monopoly capital. New York,
NY: Monthly Review Press, 1974.
[3] Campbell DT, Fiske DW. Convergent and discriminant
validation by the multitrait±multimethod matrix. Psychol
Bull 1959;56(1):81±105.
[4] Coates JF. Science, technology and human rights.
Technol Forecast Social Change 1991;40:389±91.
[5] Churchill GA, Jr. A paradigm for developing better
measures of marketing constructs. J Market Res
1979;16(1):64±73.
[6] Cooper RB, Zmud RW. Information technology im-
plementation research: a technology di€usion approach.
Manage Sci 1990;36(2):123±39.
[7] Curley KF, Pyburn PJ. `Intellectual' technologies: the
key to improving white-collar productivity. Sloan
Manage Rev 1982;0:31±9.
[8] Danziger JN, Kraemer K. People and computers: the
impacts of computing on end-users in organizations.
New York, NY: Columbia University Press, 1986.
[9] Danziger JN. Social science and the social impacts of
computer technology. Social Sci Q 1985;66(1):3±21.
[10] Davis F. Perceived usefulness, perceived ease of use, and
user acceptance of information technology. MIS Q
1989;13(3):319±42.
[11] Davis TRV. Information technology and white-collar
productivity. Acad Manage Executive 1991;5(1):55±67.
[12] DeLone WH, McLean ER. Information systems success:
the quest for the dependent variable. Inf Syst Res
1992;3(1):60±95.
[13] Doll WJ, Torkzadeh G. The measurement of end-user
computing satisfaction. MIS Q 1988;12(2):259±74.
[14] Doll WJ, Torkzadeh G. The measurement of end-user
software involvement. OMEGA 1990;18(4):399±406.
[15] Doll WJ, Torkzadeh G. The measurement of end-user
computing satisfaction: theoretical and methodological
issues. MIS Q 1991;15(1):5±10.
[16] Doll WJ, Torkzadeh G. Developing a multidimensional
measure of system-use in an organizational context. Inf
Manage 1998;33:171±85.
[17] Filiatrault P, Harvey J, Chebat J. Service quality and ser-
vice productivity management practices. Ind Market
Manage 1996;25:243±55.
[18] Fuerst WL, Cheney PH. Factors a€ecting the perceived
utilization of computer-based decision support systems.
Decision Sci 1982;13(4):554±69.
[19] Goodhue DL. Understanding user evaluations of infor-
mation systems. Manage Sci 1995;41(12):1827±44.
[20] Harvey J. Operations management in professional service
organizations: a typology. Int J Oper Product Manage
1990;10(4):5±15.
[21] Harvey J, Filiatrault P. Service delivery processes: new
technology and design. Int J Bank Market
1991;9(1):2531.
[22] Harvey J, Lefebvre E, Lefebvre L. Technology and the
creation of value in services: a conceptual model.
Technovation 1993;13(8):481±95.
[23] Hirschheim R. Oce automation: concepts, technologies
and issues. Reading, MA: Addison-Wesley, 1978.
[24] Hirschhorn L, Farduhar K. Productivity, technology and
the decline of the autonomous professional. Oce
Technol People 1985;2:245±65.
[25] Hirschhorn L. The post-industrial labor process. N Pol
Sci 1981;0:11±32.
[26] Hirschhorn L. Beyond mechanization: work and technol-
ogy in a postindustrial age. Cambridge, MA: The MIT
Press, 1984.
[27] King WR, Rodriquez JI. Evaluating MIS. MIS Q
1978;2(3):43±52.
[28] Kling R, Scacchi W. The web of computing: computer
technology as social organization. Adv Comput
1980;21:1±90.
[29] Kling R. Social analysis of computing: theoretical per-
spectives in recent empirical research. Comput Surv
1980;12(1):61±110.
[30] Kraemer KL, Danziger JN. The impacts of computer
technology on the worklife of information workers.
Social Sci Comput Rev 1990;8(4):592±613.
[31] Larson RW, Fielden JS. The managerial clerk syndrome.
Bus Horizon 1985;0:26±34.
[32] Li€ S. Clerical workers and information technology: gen-
der relations and occupational change. N Technol Work
Employment 1990;18:4±28.
[33] Long RJ. The impact of new oce information technol-
ogy on job quality of female and male employees.
Human Relat 1993;46(8):939±61.
[34] Lucas HC, Jr. Performance and the Use of information
systems. Manage Sci 1975;21(8):908±19.
[35] Mahmood MA. A comprehensive model for measuring
the potential impact of information technology on organ-
izational strategic variables. Decision Sci 1991;22(4):869±
97.
[36] Markus ML, Robey D. Information technology and or-
ganizational change: causal structure in theory and
research. Manage Sci 1988;34(5):583±98.
[37] Markus ML. Understanding information systems use in
organizations. Ph.D. dissertation, Case Western Reserve
University, 1979.
[38] Mason RO. Measuring information output: a communi-
cation systems approach. Inf Manage 1978;1(5):219±34.
[39] Melone NP. A theoretical assessment of the user-satisfac-
tion construct in information systems research. Manage
Sci 1990;36(1):76±91.
[40] Millman Z, Hartwick J. The impact of automated oce
systems on middle managers and their work. MIS Q
1987;11(4):479±91.
[41] Moore GC, Benbasat I. An empirical examination of a
model of the factors a€ecting utilization of information
technology by end users. Working Paper. University of
British Columbia, 1992.
[42] Palmquist RA. The impact of information technology on
the individual. In: Williams ME, editor. Annual review
 G. Torkzadeh, W.J. Doll / Omega, Int. J. Mgmt. Sci. 27 (1999) 327±339
of information science and technology (ARIST), vol. 27.
Medford, NJ: Learned Information, 1992:3±42.
[43] Pinsonneault A, Kraemer KL. The impact of infor-
mation technology on middle managers. MIS Q
1993;17(3):271±92.
[44] Robey D. User attitudes and management information
system use. Acad Manage J 1979;22(3):527±38.
[45] Rule J, Attewell P. What do computers do?. Social Prob
1989;36(3):225±41.
[46] Shannon CE, Weaver W. Work transformed: automation
and labor in the computer age. Lexington, MA:
Lexington Books, 1986.
[47] Schlesinger LA, Haskett L. The service driven service
company. Harvard Bus Rev 1991;0:71±81.
[48] Schewe CD. The MIS user: an exploratory behavioral
analysis. Acad Manage J 1976;19(4):577±90.
[49] Segars A, Grover V. Strategic group analysis: a meth-
odological approach for exploring the industry level
impact of information technology. Omega 1994;22(1):13±
34.
339
[50] Sethi V, King WR. Development of measures to assess
the extent to which an information technology appli-
cation provides competitive advantage. Manage Sci
1994;40(12):1601±27.
[51] Shaiken H. Work transformed: automation and labor in
the computer age. New York: Holt, Rinehart and
Winston, 1985.
[52] Srinivasan A. Alternative measures of system e€ective-
ness: associations and implications. MIS Q
1985;9(3):243±53.
[53] Sulek JM, Marucheck AS. A study of the impact of an
integrated information technology on the time utilization
of information workers. Decision Sci 1992;23:1174±91.
[54] Weick KE. Technology as equivoque: sensemaking in
new technologies. In: Goodman PS, Sproull LS et al.,
editors. Technology and organizations. San Francisco:
Jossey-Bass, 1990:1±44.
[55] Zubo€ S. In the age of the smart machine: the future of
work and power. New York, NY: Basic Books, 1988.